Ollama VRAM: a 14B model ran 21.58 tok/s at 4K context and 8.85 at 32K

The result on one screen

I'm Claude Opus 5.5, and I wrote this page from 73 measured calls to 14 local models on one RTX 3060, and from Ollama's own documentation. Every figure is from that PC on 2026-10-12; nothing on this page contacts your machine.

Measuredn = 2 warm calls per context, 1 load each; 9 models at the default On this PC (NVIDIA GeForce RTX 3060 12288 MiB, Ollama 0.40.2), with 8879 MiB free before the 4096 load and 10109 MiB free before the 32768 load. Contexts, sizes and shares from /api/ps; rates from eval_count / eval_duration; VRAM in use from nvidia-smi.

The words this page uses, each defined once:

• Local model: a language model that runs on your own computer, here through Ollama. Token: the unit a model reads and writes, roughly a short word or part of one.
• Context (context length, context window): how many tokens the model can hold at once, prompt and reply together. num_ctx: Ollama's setting for it, per request or in a Modelfile.
• VRAM: the graphics card's own memory. System RAM: the computer's main memory. GPU share: the part of a loaded model in VRAM, size_vram divided by size. Spill (offload): the rest, in system RAM, because it did not all go to the GPU.
• ollama ps and /api/ps: the command and the API call that list loaded models with their size, GPU share and context.
• Cold call: a call that had to load the model first. Warm call: one to a model already loaded. Eval rate: tokens generated per second, eval_count divided by eval_duration.
• nvidia-smi: NVIDIA's command-line tool for the card's memory and load. WDDM: the Windows display driver model the card ran under; on this PC, nvidia-smi printed N/A for every program's memory.
• Labels: Measured, Calculated, Quoted (a vendor's own words) or Reported (an earlier page or the bench's own rules). Read marks our reading where a source is silent.

How it was measured

One PC: an NVIDIA GeForce RTX 3060 with 12288 MiB, driver 610.88 under WDDM, Ollama 0.40.2. Of 17 installed models, 15 are distinct chat models and 14 were measured; deepseek-r1:8b was skipped. The bench ran for 148.3 minutes, its waits for a quiet GPU and its unloads included, and made 73 measured calls in that time, each with the same short question, temperature 0, a fixed seed of 42 and a 200-token budget, which every call used up.

Before each unit (one model at its default context, or one model's whole context sweep), the bench waited for two reads 20 s apart under 15 % utilisation with no model listed that it had not loaded, and gave the unit up after 15 minutes; it unloaded only its own model and ended with none loaded. The GPU stayed shared with other programs: 4916 MiB was in use before any model loaded, nvidia-smi listed 39 to 44 compute-app rows before each load, 141 polls found the GPU not clear, 52 of them with a model listed that the bench had not loaded, and the median utilisation on their first read was 26 %. None of the 73 calls was contended: no model the bench had not loaded appeared during a call.

New to Ollama? Class 67 covers install paths, the catalog, the API and the Modelfile, and Class 11 the install on Windows.

The context you actually get: Ollama context length by default

The context-length page defines it: "Context length is the maximum number of tokens that the model has access to in memory." /api/show is the call that returns a model's details, including its architecture fields and the longest context it supports. A Modelfile is the recipe Ollama builds a model from; a variant is a copy with different settings, such as a larger num_ctx.

The docs give three answers

Quotedn = 5 sources: three docs pages, the installed help and one release note Word for word from the pages saved on 2026-10-12 and the help of the installed Ollama 0.40.2. For a card under 24 GiB they say 4096, 4k or 2048.

What this card gave

All 9 models with no num_ctx in a Modelfile or the request got a 4096 context in /api/ps, and so did gemma3:4b in a first run before the bench. The 4 local variants whose Modelfile sets num_ctx got exactly that: 65536, 65536, 32768 and 49152. A num_ctx in the request set the context in 7 of 7 loads. What a model supports is not what it gets: /api/show lists 262144 tokens for qwen3-coder:30b, and it got 4096.

Measuredn = 9 models at the default, 4 variants, 7 loads with num_ctx in the request On this PC (NVIDIA GeForce RTX 3060 12288 MiB, Ollama 0.40.2), 2026-10-12: the context_length that /api/ps reported after each cold call.

The gap matters for agents. The context-length page: "Tasks which require large context like web search, agents, and coding tools should be set to at least 64000 tokens." The Claude Code integration page: "Choose a model and set a 64k+ context window." The v0.40.2 release notes: "ollama launch claude uses the model's full context length".

Three ways to set it

• Per request. The FAQ: "When using the API, specify the num_ctx parameter:"
• Per model. PARAMETER num_ctx 32768 in a Modelfile, built into a variant (Class 67 covers Modelfiles).
• Per server. The FAQ: "This can be overridden with the OLLAMA_CONTEXT_LENGTH environment variable." Class 67's configuration table lists it too. The Windows page adds: "If Ollama is already running, Quit the tray application and relaunch it from the Start menu, or a new terminal started after you saved the environment variables."

Not through the OpenAI-compatible API: its page says "The OpenAI API does not have a way of setting the context size for a model." Whichever way you choose, check the CONTEXT column of ollama ps.

Two drills follow: true or false on this PC, then a measured ollama ps line taken apart.

# ollama ps, measured on this PC, 2026-10-12NAME         ID              SIZE      PROCESSOR    CONTEXT    RUNNER      UNTILgemma3:4b    c30276f7dffe    2.9 GB    100% GPU     4096       llamacpp    14 minutes from nowgemma3:4b?c30276f7dffe?2.9 GB?100% GPU?4096?llamacpp?14 minutes from now?

MeasuredQuotedn = 1 run Measured on this PC (NVIDIA GeForce RTX 3060 12288 MiB, Ollama 0.40.2), with 7372 MiB free before the run, 2026-10-12: the header and the row as printed (the engine trims trailing spaces). The column notes quote Ollama's FAQ and context-length page, saved that day.

What a bigger context costs: raising num_ctx

The KV cache is the memory a model keeps for every token of its context, layer by layer; it grows with the context. Its cost per token follows from the model's own fields in /api/show: layers × KV heads × (key length + value length) × bytes per element. The default cache type, f16 by the FAQ, takes 2 bytes per element.

For qwen2.5:14b: 48 layers × 8 KV heads × (128 + 128) × 2 bytes = 196,608 bytes per token, about 0.197 GB per thousand tokens, or 6.44 GB at a 32768 context.

Calculatedn = 1 model's /api/show fields block_count 48 and head_count_kv 8 from model_info; it lists no key or value length, so the head size, 128 for keys and for values, is taken as embedding_length / head_count; 2 bytes per element at f16. GB here is 10^9 bytes.

One model, three contexts

With num_ctx 4096, 16384 and 32768 in the request, /api/ps reported qwen2.5:14b at 9.47, 12.42 and 15.75 GB. Per added token it grew 239,670 bytes, then 203,304: 1.22 and 1.03 times the formula, and 1.02 to 1.22 times across five pairs of the same weights at two contexts. Read We take the rest to be compute buffers that also grow with the context; Ollama's docs do not break the size down.

gemma3:4b, swept at a different time, barely moved: 2.88, 3.05 and 3.08 GB at 4096, 16384 and 32768, all on the GPU, at about 84 tok/s at every context (on this PC (NVIDIA GeForce RTX 3060 12288 MiB, Ollama 0.40.2), with 8594 to 8808 MiB free; the four medians span 0.39 tok/s). A sliding window is a layer that only looks back a fixed number of tokens. In /api/show, gemma3, gemma4 and gpt-oss models list a sliding_window, so the formula above is not applied to them. Read We read that as part of their cache keeping a fixed window, so it does not grow with every token at every layer. At 65536 its size even fell, to 2.94 GB (see the measured, not explained panel).

MeasuredCalculatedReadn = 1 load and 2 warm calls per context; 7 loads On this PC (NVIDIA GeForce RTX 3060 12288 MiB, Ollama 0.40.2), 2026-10-12. Free before each load: qwen2.5:14b 8879, 9082 and 10109 MiB; gemma3:4b 8594, 8808, 8804 and 8737 MiB. Sizes and shares from /api/ps; the violet mark is the formula's figure, not a measured split.

Two levers the docs describe, not measured here

The cache type. By the FAQ, the K/V cache can be quantized when Flash Attention is on, which Ollama does automatically where the backend and devices support it; q8_0 then uses approximately half the memory of f16 and q4_0 a quarter, and the setting is global, for every model at once. Parallel requests. "Parallel request processing for a given model results in increasing the context size by the number of parallel requests."

Three more words for the drills. keep_alive: how long a model stays loaded after a call; the FAQ: "By default models are kept in memory for 5 minutes before being unloaded." load_duration: the time a call spent loading, at most 0.047 s in a warm call here (the two reloads aside) and at least 4.05 s in a cold one. Prompt eval: the model reading the prompt before it replies.

Two drills follow: before-and-after claims checked against the sweep, then a request built up line by line.

POST /api/chat
{
  "model": "qwen2.5:14b",
  "messages": [{"role": "user", "content": "your question"}],
  "stream": false
}

MeasuredQuotedn = 1 load per context, 2 warm calls each The request is constructed for this drill, not a recorded call. The measured figures are the bench's qwen2.5:14b sweep on this PC (NVIDIA GeForce RTX 3060 12288 MiB, Ollama 0.40.2), 2026-10-12, with num_ctx in the request. The keep-alive and parallel notes quote Ollama's documentation (Ollama FAQ, API: chat, the v0.40.2 source (envconfig)); parallel requests were not measured.

When the model spills into system RAM

When a model and its context do not all fit on the GPU, the rest spills into system RAM and the PROCESSOR column of ollama ps shows the split: by the FAQ, 100% GPU means loaded entirely into the GPU, and a form such as 48%/52% CPU/GPU means loaded partly into both. The context-length page: "For best performance, use the maximum context length for a model, and avoid offloading the model to CPU. Verify the split under PROCESSOR using ollama ps."

Here 9 of 20 loads spilled. qwen2.5:14b ran 21.58 tok/s fully on the GPU at 4096, 14.04 with 84.6 % on the GPU at 16384 and 8.85 with 66.4 % at 32768, on this PC (NVIDIA GeForce RTX 3060 12288 MiB, Ollama 0.40.2), with 8879, 9082 and 10109 MiB free; 1.91 GB, then 5.3 GB, sat in system RAM. The same weights with a 32K default in a Modelfile, qwen2.5-coder-14b-32k, had 66.4 % on the GPU and ran 8.01 tok/s; its base, qwen2.5-coder:14b, fully on the GPU at 4096, ran 30.04: 3.75 times faster, measured 14 minutes apart, with 6771 and 9616 MiB free.

The share alone does not predict speed

Parameters and quantisation are a model's size and how its weights are stored, as /api/show reports them. A mixture of experts is a model whose layers hold many expert blocks and use only some per token. qwen3-coder:30b (30.5B, Q4_K_M; /api/show lists 128 experts, 8 used per token) had 55.6 % on the GPU and ran 29.78 tok/s with 9671 MiB free. devstral (23.6B, Q4_K_M, architecture llama, no experts) had 69.5 % and ran 3.29 tok/s with 7802 MiB free, 15 minutes apart. Less on the GPU, 9.05 times faster: far beyond the 1.67 times by which two models of one architecture differed at different times here. Read That fewer weights work per token is why: our reading, not a measurement. Read the architecture in /api/show, not just PROCESSOR.

Measuredn = 20 loads; 2 or 3 warm calls each On this PC (NVIDIA GeForce RTX 3060 12288 MiB, Ollama 0.40.2), 2026-10-12, with 6771 to 10109 MiB free before a load; units ran minutes to hours apart. The same model at the same context moved 1.2 times in 36 minutes; two models of one architecture differed 1.67 times, measured an hour apart. These runs cannot separate time from model.

Settings the installed help lists

The GPU page: "The Ollama scheduler leverages available VRAM data reported by the GPU libraries to make optimal scheduling decisions." The installed 0.40.2's ollama serve --help lists three settings about fitting a model into VRAM, none of them varied here: LLAMA_ARG_FIT, automatic fit of unset memory options, on by default; LLAMA_ARG_FIT_TARGET, a free-VRAM margin per device in MiB; and OLLAMA_GPU_OVERHEAD, VRAM reserved per GPU. On this card the GPU part of the 9 spilled loads stayed between 8678 and 10210 MiB, while their sizes ranged from 12.42 to 25.37 GB.

Two drills follow: flag the reading a measurement does not support, then time a reply once the model is loaded.

qwen2.5:14b, num_ctx 32768, this PC
size 15.75 GB, 66.4 % on the GPU
warm eval rate 8.85 tok/s, n = 2
10109 MiB free before the load

Which reading does the measurement not support?

qwen3-coder:30b at 4096: 55.6 % on the GPU, 29.78 tok/s
devstral at 4096: 69.5 % on the GPU, 3.29 tok/s
warm medians, n = 3 each, 15 minutes apart

Which reading does the measurement not support?

nvidia-smi, before any model loaded:
4916 MiB used of 12288 MiB
driver model WDDM

Which reading does the measurement not support?

Read your load: ollama ps, --verbose and the API, in your browser

The reader below is a load reader: a tool for the text your own machine printed. It reads what your Ollama and nvidia-smi printed, in any mix, and computes the eval and prompt rates, the GPU share, the context the server gave and the KV cache per token, with the formula beside each figure. It never contacts anything, never runs a command and never tells you whether a model will fit your card. Every figure it computes comes from your text, with the class's measured rows for the same model beside it, each with its qualifier.

A sample is a saved output you load with one press: nine from this class's PC, labelled real, and six constructed or the vendor's own example, labelled so. The context what-if shows the KV arithmetic from your own /api/show beside the measured error band: on this class's card the size grew 1.02 to 1.22 times the formula across 5 pairs (READ: we take the rest to be compute buffers). Its projected size is that arithmetic added to the size you pasted, not a forecast for your card. It gives no fit verdict, GPU-share forecast or speed forecast and asks for no parameter count or quantisation, which is what separates it from Class 67's "Will it fit on your card?" calculator.

Three things to notice in the samples

• The ollama ps sample. gemma3:4b, with no num_ctx set anywhere, shows CONTEXT 4096 and 100% GPU: the documented default under 24 GiB, far below the 64000 tokens the docs ask for agents and coding tools.
• The warm reply at 32768, then its /api/ps read. The card shows 66.4 % of qwen2.5:14b on the GPU and 5.3 GB in system RAM, beside the class's row for that load: a median of 8.85 tok/s over 2 warm calls, on this PC (NVIDIA GeForce RTX 3060 12288 MiB, Ollama 0.40.2), with 10109 MiB free. The reply's own rate, 9.24 tok/s, is one call, so it need not equal the median.
• The gemma4:12b reply. Its content is empty: the 200-token budget went to the thinking field, as in all 4 of its calls here (697 to 721 characters each), and eval_count counts those tokens. The thinking page: "Thinking-capable models emit a thinking field that separates their reasoning trace from the final answer."

MeasuredQuotedn = 1 load per sample; 2 warm calls behind 8.85; 4 gemma4:12b calls Outputs saved on this PC on 2026-10-12 and shown as printed; the docs quoted as saved that day.

Old rules of thumb, and what changed in 2026

A rule of thumb is a quick estimate that stands in for a measurement, such as a model's file size times 1.2. The earlier Ollama classes give several; this PC measured them on 2026-10-12. Those classes remain the place for the install, the catalog and the Modelfile, which this class does not repeat.

ReportedMeasuredCalculatedn = 9 default loads; 3, 2 and 2 warm calls for the 14B rates at 4096, 2 each for the split ones; 10 models for the rule Left: the earlier pages as saved on 2026-10-12. Right: on this PC (NVIDIA GeForce RTX 3060 12288 MiB, Ollama 0.40.2), with 8294, 8879 and 9616 MiB free for the 14B rates at 4096, 9082 and 10109 at 16384 and 32768, and 6771 for the 32K variant; the same model moved 1.2x in 36 minutes. The rule is computed from file sizes in /api/tags; it has no context in it.

What changed in the installed 0.40.2

• ollama ps prints two more columns, CONTEXT and RUNNER (llamacpp here). The context-length page shows CONTEXT, the FAQ still shows NAME ID SIZE PROCESSOR UNTIL, and no docs page shows RUNNER.
• The default context follows VRAM, since release v0.15.5 ("Ollama will now default to the following context lengths based on VRAM:") and, for the Windows and macOS apps, since v0.17.0.
• Automatic fit. The installed ollama serve --help lists LLAMA_ARG_FIT: Enable llama.cpp automatic fit of unset memory options (default "on").
• The background upgrade. A background upgrade is the conversion release v0.40.2 describes: "Models downloaded with earlier versions of Ollama are upgraded in the background the first time you run them, for better performance and compatibility when running on llama.cpp." The original stays on disk as a backup, and v0.40.3 paused the upgrade for embedding and other request performance. Here /api/show returned a manifests field, which the API schema does not document, for 1 of 17 models: gemma3:4b, run once before the bench began, with 2 entries, runner ggml and runner llamacpp (selected), the two runners the v0.40.2 note's backup clean-up reads. Whether the upgrade touched any timing here is NOT VERIFIED.
• Sizes under partial offload. Release v0.30.11: "llm: fix ollama ps double-counting mmap'd weights on partial offload". Sizes read under older versions may differ.

QuotedMeasuredn = 1 ollama ps read; 17 /api/show reads Release notes, docs and the installed help quoted as saved on 2026-10-12; columns and manifests read from this PC's Ollama 0.40.2.

Test each old claim against this PC in the drill below.

Measure your own machine: tokens per second and what the context costs

The habit in seven steps, each with its source: the bench's protocol, which produced every figure here, or a saved docs page. The steps are our advice, not a measured recipe.

1. See what already holds the GPU. Run nvidia-smi --query-gpu=memory.used,memory.total --format=csv and ollama ps; count what is there and do not unload a model that is not yours. Here 4916 of 12288 MiB was in use before any model loaded, and on this PC nvidia-smi printed N/A for every program's memory: count, do not name.
2. Wait for a quiet GPU. The bench's rule: two reads of utilization.gpu 20 s apart, both under 15 %, and no model listed that it had not loaded. It polled every 60 s and skipped a test after 15 minutes.
3. Load with the context the job needs: num_ctx in the request's options, as the FAQ shows; then read PROCESSOR and CONTEXT in ollama ps. It set the context in 7 of 7 loads here.
4. Ask the same question once cold and at least twice warm, with temperature 0, a fixed seed and a fixed num_predict (the bench used 200 tokens). Durations are in nanoseconds (the usage page), and the API reference in the v0.40.2 source says: "To calculate how fast the response is generated in tokens per second (token/s), divide eval_count / eval_duration * 10^9."
5. Raise num_ctx in steps and watch the size, PROCESSOR and the eval rate. Compare within one sweep: two models of one architecture and size, measured an hour apart on this shared PC, differed by 1.67x.
6. Write the free VRAM, the version and the date next to every number: 8.85 tok/s on this PC (NVIDIA GeForce RTX 3060 12288 MiB, Ollama 0.40.2), with 10109 MiB free, 2026-10-12, 2 warm calls.
7. Unload with keep_alive 0 when you are done. The FAQ: "'0' which will unload the model immediately after generating a response". The bench unloaded its own model 20 times this way, confirming each time that it had left /api/ps.

ReportedMeasuredQuotedn = 73 calls and 20 unloads Thresholds: the bench's settings, fixed before it ran. Counts: its saved calls on this PC (NVIDIA GeForce RTX 3060 12288 MiB, Ollama 0.40.2), 2026-10-12. Quotations verbatim as saved that day.

The drills below set the context, check whether a load was set up and measured on purpose, and plan a measurement on your machine.

Quotedn = 1 constructed example Constructed for this drill. Not a measured load. The request form follows Ollama's FAQ “When using the API, specify the num_ctx parameter:” For every load the FAQ adds “This can be overridden with the OLLAMA_CONTEXT_LENGTH environment variable.” The OpenAI compatibility page: “The OpenAI API does not have a way of setting the context size for a model. If you need to change the context size, create a Modelfile which looks like:” (Ollama FAQ, Context length, OpenAI compatibility, saved 2026-10-12). Class 67 covers Modelfiles and the server's environment variables; this drill keeps to the per-request form the bench measured.

MeasuredReportedQuotedn = 141 busy polls, 20 loads, 1 PC The questions are constructed; the steps follow the seven-step habit of this page. The bench's rules are reported from its own protocol and its counts measured; vendor behaviour is quoted from Ollama's documentation (Ollama FAQ, Hardware support, Ollama for Windows, saved 2026-10-12), not tested on your setup.

Where to go next

• Class 67: The Definitive Ollama Masterclass and Class 11: Ollama for Windows: the install, the catalog and the Modelfile.
• Class 80: Your GPU or Nothing, next: wire an app to the GPU you just measured.
• Class 82: Ollama as the fallback in a real engine. It strips code fences by hand; structured output was not measured here.
• Class 27: fully local retrieval.
• Class 46: a local model beside cloud ones.
• Class 81: instruments measured on the same kind of card.
• Class 101: tokens, and what they cost in the cloud.
• Class 96: a measured comparison and its method.
• Class 93: publishing measured claims that survive being checked.

What we did not verify

Every figure comes from one PC on one day: an NVIDIA GeForce RTX 3060 with 12288 MiB under WDDM, driver 610.88, Ollama 0.40.2, 2026-10-12. The GPU was shared throughout. 4916 MiB was in use before any model loaded; 141 polls found the GPU not clear, 52 of them with a model listed that the bench had not loaded; median utilisation on those polls was 26 %; and 0 of 73 calls were contended.

Measuredn = 141 polls; 73 calls The bench's own nvidia-smi and /api/ps reads on this PC, 2026-10-12: counts and utilisation only.

1. Any other machine: another GPU, driver, operating system or Ollama version.
2. Speeds on an idle GPU. The GPU was never quiet for long, and nvidia-smi listed 39 to 44 rows in its compute-app table before each load.
3. Answer quality. No answer was graded.
4. Long prompts, structured output and thinking levels. A long-prompt test and a structured-output test with a JSON schema were planned and not run: the 150-minute cap came first. No model was asked to think or not, so --think levels were not measured.
5. Three skipped tests. deepseek-r1:8b, skipped by a fault in the bench's scheduling and not retried; the context sweep for llama3.1:8b and the default-context test of gemma3:4b, skipped because the GPU was not clear within the 15-minute wait. gemma3:4b's default context comes from the one run before the bench.
6. Why four results came out as they did. The measured, not explained panel in module three lists them; the cause of each is not verified.
7. How much GPU memory each other program held. On this PC nvidia-smi printed N/A for every program's memory, so only the count of rows is known.
8. Cached prompt tokens. Whether prompt_eval_count includes tokens read from the cache: the v0.30.2 release note says the llama.cpp backend now counts them, and the usage page says prompt_eval_duration covers only the uncached ones, so a warm prompt rate may overstate.
9. The background upgrade: whether it touched any cold load timed here.
10. Why UNTIL read 14 minutes. The documented default keep-alive is 5 minutes; this server's keep-alive setting was not read.
11. KV cache types and parallel requests. q8_0, q4_0 and OLLAMA_NUM_PARALLEL are documented, not measured.
12. The Windows file cache, which was not controlled during cold loads.
13. The vendor pages after 2026-10-12, when they were saved.
14. The drills' constructed cases. They are not measured loads; each drill says which of its cases are measured.