Local LLM VRAM Calculator

Estimate GPU memory for an open-weight model: FP16 or Q8–Q3 weights, KV cache for your context length and overhead, with the formula and config.json values.

Private by design: everything you enter is processed in your browser. It is not sent to our servers.

Model data checked on 2026-10-14.Limits and architecture values come from the pages below (57 sourced values; each records its own source and date). Provider pricing changes, so confirm on the provider’s page before committing budget.ai.google.dev · developers.openai.com · github.com · huggingface.co · huggingface.co · huggingface.co · huggingface.co · huggingface.co · huggingface.co · huggingface.co · huggingface.co · huggingface.co · huggingface.co · platform.claude.com

How to use

  1. Pick an open-weight model. Parameters, layers, KV heads and head size are filled from its published config; choose “Custom” to enter your own.
  2. Choose a quantisation (FP16, Q8, Q6, Q5, Q4 or Q3), the context length you will run, the KV-cache precision and the number of simultaneous sequences.
  3. Optionally pick your GPU memory to get a “fits / does not fit” answer, and adjust the runtime overhead allowance.
  4. Read the weights, KV cache and total, and the formula with your numbers substituted.

Worked example

Running Qwen2.5 7B Instruct as a Q4_K_M GGUF with an 8,192-token context on an 8 GiB GPU.

Parameters7,615,616,512
Bits per weight (Q4_K_M)4.8944
Layers / KV heads / head dim28 / 4 / 128
Context / KV precision / batch8,192 / fp16 / 1
Overhead allowance1 GiB

Result

Weights4.34 GiB
KV cache0.438 GiB (57,344 bytes per token)
Overhead1.00 GiB
Total5.78 GiB
Fits in 8 GiB?Yes, 2.22 GiB to spare

KV cache per token = 2 × 28 layers × 4 KV heads × 128 head dimension × 2 bytes = 57,344 bytes; × 8,192 tokens = 0.4375 GiB.

How it works

Weights = parameters × bits per weight ÷ 8. The bits per weight for each quantisation come from llama.cpp’s own quantisation table (measured on Llama-3.1-8B): F16 16.0005, Q8_0 8.5008, Q6_K 6.5633, Q5_K_M 5.7036, Q4_K_M 4.8944 and Q3_K_M 3.9960. They are slightly above the nominal 8, 6, 5, 4 and 3 bits because block scales are stored too.

KV cache = 2 × layers × KV heads × head dimension × bytes per element × context tokens × sequences. The factor 2 is one key and one value vector per head per layer. Models that use grouped-query attention have fewer KV heads than attention heads, which is why their KV cache is much smaller than the hidden size suggests.

Total = weights + KV cache + overhead. Overhead (GPU runtime context, activation buffers, fragmentation) is not a fixed number; the default of 1 GiB is an assumption you can change. Sizes use GiB (1,073,741,824 bytes), the unit most GPU memory figures use.

Architecture numbers come from each model’s config.json on Hugging Face (num_hidden_layers, num_key_value_heads, hidden_size, num_attention_heads). The config has no head-dimension field for these models, so head dimension is derived as hidden size ÷ attention heads. Parameter counts come from the Hugging Face API’s safetensors totals.

Common errors

bitsPerWeight must be between 0 and 32.
Enter bits per weight greater than 0 and at most 32 for a custom quantisation.
parameters, layers, kvHeads, headDim and batch must each be at least 1.
Fill every custom field with a positive whole number.
Context length N is above the model's configured maximum of M tokens.
The model was configured for a shorter context. Lower the context, or confirm the model supports extended context (for example through rope scaling).
gpuGiB must be greater than 0.
Choose a GPU size or leave it empty to skip the fit check.

Limits

  • This is an estimate, not a measurement. Different runtimes (llama.cpp, vLLM, Ollama, MLX) allocate memory differently, and some keep extra buffers.
  • Bits per weight were measured on one model; other models quantise to slightly different sizes. Mixed-precision files (for example the output layer kept at higher precision) vary.
  • The 8-bit KV cache option assumes exactly one byte per element and ignores scale overhead.
  • Multimodal encoders, LoRA adapters, speculative-decoding draft models and mixture-of-experts routing are not modelled; the listed models are dense text models.
  • Only five models are listed, each with fields verified against its published config.json. Others can be entered as Custom.

Privacy

Only numbers are processed, in your browser. Nothing is sent anywhere. No account is needed, no cookies are set and no analytics run. See thePrivacy Policy. The “Copy share link” button puts your settings after the “#” in the address; browsers do not send that part to servers, and the page’s canonical address never includes it.

FAQ

Why is the KV cache so much smaller than I expected?

Grouped-query attention stores keys and values for only a few KV heads, not all attention heads. Qwen2.5-7B uses 4 KV heads against 28 attention heads.

Does a longer context always need more VRAM?

Yes, KV cache grows linearly with context length (and with the number of simultaneous sequences).

What do GiB and GB mean here?

This tool uses GiB (1024³ bytes). GPU memory sizes such as “24 GB” are normally binary gigabytes, so you can compare directly.

Can I run a model that “does not fit”?

Often by offloading layers to system RAM or lowering the context, at the cost of speed. This tool only answers whether everything fits in the GPU.

Where do the parameter counts come from?

From the safetensors metadata that Hugging Face publishes for each repository. The source link beside each value is in the data file.

Embed this tool

Paste this snippet into your own page or docs to show the tool inline. The embedded view works the same way (everything runs in the visitor’s browser) and carries a small link back to MakeTools.

Sources

Page last updated 2026-10-14. Written by Prajyot Mulik; see the editorial policy.