
LLM VRAM & KV Cache Sizing Guide: Exact Memory Formulas for GQA, MLA, and Quantization on Linux
A VRAM estimate is a screening tool, not proof that an LLM server will start or sustain a workload. The title’s “exact” formulas apply only when their stated tensor layouts and inputs match the implementation. Model artifacts, serving engines, attention backends, GPU topology, and concurrent workloads can all change the actual allocation.
Use the estimate to identify configurations that are unlikely to fit, then validate a pinned deployment on each GPU rank. No calculator, model-specific footprint, or capacity figure in this guide has been independently executed or measured.
Start with the allocation budget
For a particular rank, use this planning relationship:
$$M_{\text{required}} \approx M_{\text{weights}} + M_{\text{KV}} + M_{\text{transient}} + M_{\text{runtime}} + M_{\text{other}}$$
Weights include resident tensors and quantization metadata. KV includes the engine’s allocated cache, not just useful token payload. Transient memory includes prefill activations and workspaces; runtime memory can include contexts, communication buffers, and graph pools. Other use includes allocations by unrelated processes, allocator reserve, fragmentation, and deployment-specific overhead.
These terms can overlap in time or be reserved differently by an engine, so do not add measurements from incompatible phases blindly. Record available memory per GPU before startup and observe the peak during representative startup, prefill, and decode. There is no universal fixed runtime allowance.
Decision path:
- If the artifact’s resident weights alone exceed a rank’s available memory, revise the deployment before estimating cache capacity.
- If weights appear to fit, estimate the requested cache payload under an explicitly verified layout.
- If the estimate leaves little or unknown space for peak allocations, treat the configuration as unvalidated and measure it on an isolated or appropriately scheduled system.
- If the engine rejects the requested sequence or workload during validation, reduce the request or change the deployment and repeat the checks. Do not interpret a fractional sequence as a successful fit.
Use bytes consistently: decimal GB and binary GiB differ. Also record whether a GPU’s reported free memory already excludes other processes’ allocations.
Weight precision is not a footprint
A parameter count multiplied by a nominal bit width is only a starting estimate. It assumes that every counted parameter uses that representation and ignores metadata, padding, alignment, unquantized tensors, tied or duplicated weights, conversion buffers, and engine-specific packing.
For example, under the hypothetical layout of 128 four-bit values with one 16-bit scale and one four-bit zero point, the stored payload is:
$$4 + \frac{16+4}{128}=4.15625\text{ bits per value}$$
That arithmetic does not establish the layout or effective footprint of an AWQ artifact. The AWQ paper, version 6, describes activation-aware, low-bit weight-only quantization and scaling based on offline activation statistics; it does not establish this example’s packing or a particular model’s resident VRAM.
Before planning from a quantized checkpoint, pin its revision and inspect its tensor inventory, quantization configuration, scale and zero-point representation, packing, and any tensors kept at higher precision. Check the serving engine’s pinned release for repacking or additional resident copies. Disk size alone is not a measured GPU allocation, and weight precision does not determine KV-cache precision.
Estimate conventional KV payload
For an assumed uniform attention layout that stores one key and one value vector per KV head for every cached token, the logical payload is:
$$\text{bytes per token}=2 \times L \times H_{\text{KV}} \times D_{\text{head}} \times b$$
Here, $L$ is the number of layers using that layout, $H_{\text{KV}}$ the number of KV heads per layer, $D_{\text{head}}$ the stored dimension per head, and $b$ the bytes per stored element. The factor of two represents keys and values. For simultaneous sequences with no assumed sharing, multiply by the sum of their cached token counts.
This is a conditional tensor-size calculation, not an engine allocation. Verify dimensions and layer types against the pinned model configuration and implementation; do not infer a missing head dimension without checking the architecture. Sliding windows, hybrid layers, prefix sharing, cache blocks, padding, and scheduler policy can change the relationship between logical tokens and allocated memory. Quantized KV formats may add scales, packing and alignment overhead, so nominal bits per element alone do not give an exact byte count.
The vLLM Paged Attention documentation explicitly labels its kernel walkthrough historical and says it no longer describes today’s code. Use it for context, then inspect the documentation and backend implementation for the release actually deployed.
Treat MLA and parallel placement separately
The DeepSeek-V3 technical report, version 2, identifies Multi-head Latent Attention as part of that model’s architecture. That fact alone does not specify the cache dimensions, byte format, or placement used by a particular serving backend. Do not substitute a conventional GQA formula or an assumed MLA formula for an engine-specific cache layout.
For tensor or other parallelism, do not pool GPU memory and divide a single cache estimate across devices. A deployment must fit on every rank; weights and cache can be replicated, sharded unevenly, padded, or placed differently by architecture and engine. Check supported parallel degrees and divisibility constraints in the pinned release, then inspect its rank-level allocation behavior. If those details are unavailable, there is no defensible multi-GPU capacity recommendation from this estimate.
The FlashMLA repository reports a breaking change in its 2026-09-30 release: support for Hopper and earlier models, including DeepSeek V3, V3.2, and V4.0, was removed, and FP8/FP4 KV-cache formats changed. For those models or the old format, the notice directs users to an earlier commit. Verify that commit, hardware, model, engine integration, and cache format together before upgrading; a moving repository page is not a substitute for a pinned compatibility record.
Validate a candidate deployment
Use this bounded checklist with the official documentation for the exact engine release; flags and supported formats vary by release.
- Pin inputs: record model and quantized artifact revisions, engine and attention-backend versions, driver/runtime versions, GPU model and topology, and parallel configuration.
- Define the workload: record maximum prompt and output lengths, concurrent sequences, prefill batching, and expected prefix sharing. Include the worst relevant combination, not merely an average request.
- Establish a baseline: record per-GPU free memory and other active users or processes. Preserve their workloads when scheduling measurements.
- Measure by phase: capture per-rank memory after model load, after cache initialization, and at representative peak prefill and decode. Record engine allocation and cache reports alongside device-level observations.
- Check the request: verify that at least one full requested sequence fits under the engine’s own admission rules, then test the intended concurrency. Record rejection, preemption, and latency as well as memory.
- Keep margin: choose operational headroom from observed variability and the service’s tolerance for failure. Revalidate after changes to the model, backend, cache format, topology, or workload.
A failed test is evidence about that pinned configuration and workload, not a universal crash threshold. A successful test establishes only the conditions exercised; it does not guarantee future capacity.
Primary sources
- AWQ paper, arXiv:2306.00978 — activation-aware weight-only quantization.
- DeepSeek-V3 technical report, arXiv:2412.19437 — identifies MLA in DeepSeek-V3.
- FlashMLA repository — release compatibility and cache-format notice.
- vLLM Paged Attention documentation — historical status of the kernel walkthrough.
Validation boundary: The supplied source excerpts support the claims attributed to them; they do not validate a runnable command, a model-specific allocation, or a capacity recommendation. No deployment or benchmark was executed for this article. Source-review notice: November 13, 2026; the supplied excerpts do not independently establish what live pages contained on that date.