How to Fix Ollama Context Window Truncation (num_ctx) for AI Coding Agents
LLM Serving(Updated )

How to Fix Ollama Context Window Truncation (num_ctx) for AI Coding Agents

Start with the loaded model

A coding agent’s context budget, a model’s configured limit, and the allocation shown for a running model are different things. If a request exceeds the available budget, check the documentation for the installed client and server versions to determine how they handle it. An agent missing a detail is not, by itself, proof of a context-window problem.

Inspect the model while it is loaded:

ollama ps

The Ollama context guide shows a CONTEXT column for the allocated context and a PROCESSOR column for the CPU/GPU split. Column availability can vary by release. If CONTEXT is absent, consult the diagnostics for your pinned release; a successful response does not establish the allocation.

Use this decision tree:

  1. Is the model absent from the list? Send a normal request through the same client, then inspect while the model remains loaded.
  2. Is the displayed context lower than intended? Examine the model definition, server default, and client or request settings. Check which controls and precedence rules apply to your installed versions.
  3. Does the allocation match, but the agent still miss information? Inspect the client’s file selection, conversation handling, and request configuration. A large allocation does not prove that a particular file was sent.
  4. Does a larger allocation create memory pressure or CPU offloading? Consider a smaller allocation, fewer concurrent workloads, less input, or more available capacity. Recheck with the actual workload.

Keep prompts and credentials out of diagnostic captures unless their disclosure is explicitly intended.

Budget the whole conversation

Context is measured in tokens, not characters or lines. Allow room for system instructions, tool definitions and results, conversation history, selected files, the current prompt, and generated output. Source code that occupies 16k tokens on its own can leave insufficient room for the rest of the request and response.

Before raising the limit, narrow the files the agent selects and remove redundant tool output. A fresh task can reduce stale history, but carry forward requirements and decisions the agent still needs. If using a summary, review it for omissions before relying on it.

The context guide describes VRAM-dependent defaults of 4k below 24 GiB, 32k from 24 to under 48 GiB, and 256k at 48 GiB or more. It recommends at least 64,000 tokens for tasks such as agents and coding tools, while warning that longer context requires more memory. Treat that recommendation as a target to evaluate against the model, workload, and available capacity.

Documentation defaults also differ: the Modelfile reference lists 2,048 for num_ctx (see Ollama Modelfile documentation), while the Ollama FAQ states 4,096. The supplied excerpts do not reconcile these descriptions or establish a universal default. Check the documentation for the pinned release and inspect the loaded allocation instead of choosing a default from one page.

Configure a named model

A Modelfile can give an agent a recognizable model target with a requested context size. The Modelfile reference documents FROM, PARAMETER num_ctx, model creation, and inspection of a model’s Modelfile.

First check that Modelfile.coding-32k does not already exist in the working directory. Create it with an editor, using a base model available to your installation:

FROM llama3.2
PARAMETER num_ctx 32768

Choose a model name that does not conflict with an existing model, then create and query it:

ollama create coding-context-32k -f ./Modelfile.coding-32k
ollama run coding-context-32k

An answer does not verify a 32,768-token allocation. While the model is loaded, inspect ollama ps and its CONTEXT column if your release exposes it. Inspect the model definition separately:

ollama show --modelfile coding-context-32k

The definition is evidence of configuration, not proof of the allocation used for every request. Check the agent’s request path separately, and verify that the chosen context is appropriate for the model and available memory.

Configure a request or server default

The FAQ places num_ctx inside the options object for /api/generate. For a client using that endpoint, the relevant request body can look like this:

{
  "model": "coding-context-32k",
  "prompt": "Explain the locking assumptions in this function.",
  "options": {
    "num_ctx": 32768
  }
}

This illustrates a payload; it does not establish that a particular coding agent sends it. Check the client documentation for the installed release and inspect its configuration. Do not assume that an agent’s own context-budget setting maps to the server’s num_ctx, or that it leaves the model setting untouched.

The context guide and FAQ also describe OLLAMA_CONTEXT_LENGTH as a server default. Before changing a running service:

  • Identify the server process and the configuration mechanism used by that installation. Avoid starting a second server merely to test a setting.
  • Consult the pinned release’s documentation for the applicable configuration and restart procedure. Coordinate any interruption with active users.
  • After the change, load the target model through the intended client and inspect the allocation where supported.
  • If it differs from the requested value, check model and request settings and the release-specific precedence rules.

Account for memory and concurrency

The context guide explicitly warns that increasing context length increases the memory needed to run a model. It also recommends checking the PROCESSOR split with ollama ps. The FAQ explains that this column can indicate a full GPU allocation, a full CPU allocation, or a split between them.

Plan with headroom rather than treating the context value as a complete memory estimate. Model choice, other workloads, and concurrent requests can affect the capacity available to a deployment. The supplied excerpts do not establish a formula for predicting its allocation.

Increase context in steps on the target workload. At each step, observe available memory and the loaded model’s processor split; measure latency if it matters to the decision. If resource pressure appears, first determine whether reducing context, input, or concurrent workload meets the use case. Offloading is a useful diagnostic signal, but it does not explain every slow response or guarantee a particular performance outcome.

Primary sources and validation boundary

This article is based on the supplied source excerpts. No command, model, API request, GPU workload, or benchmark was executed or independently validated for it. The excerpts describe documentation available for review and do not establish what was true on the original publication date. Verify behavior against your pinned client, server, and model releases.

Source-review notice: November 13, 2026.

BC

Written by B. Cho

Systems & AI Infrastructure Engineer

Specializing in GPU inference engines (vLLM, SGLang, Ollama), CUDA memory allocation, and Model Context Protocol (MCP) infrastructure on Linux workstations.