The Context Window Is a Budget

You are halfway through a refactor. Pi has read forty files, run the test suite twice, and rewritten the same module three times because each attempt turned out to conflict with the last. The session is still on disk, but the request has grown too large. If Pi compacts it, older messages are represented by a summary rather than sent in full.

Most people meet this as a mood: Pi got worse at the task. It is easier to reason about as a budget. Each request consumes a varying number of tokens within the model’s context window, across components you can inspect and, in several cases, change.

Wrong: “The context window is a container. The conversation is in it, and the model reads the container.”

Correct: “The context window is a budget. Its components compete for space; compaction may replace older conversation messages with a summary.”

The composition of one request

Per how-pi-works.md, Pi assembles a model request using the system prompt, the active branch, available tools, and model settings. Context files contribute to the prompt, and skill descriptions and any loaded skill instructions contribute to what the model can read. They are not an additional, separate layer on top of the request.

    flowchart TD
  SP["system prompt<br/>base instructions and context files"] --> CTX
  AB["active branch<br/>one path through the tree"] --> CTX
  TL["tool definitions<br/>for active tools only"] --> CTX
  SK["skill descriptions<br/>always present"] --> CTX
  SI["skill instructions<br/>only when loaded on demand"] --> CTX
  CTX["contextTokens"] --> CMP{"over the line?"}
  RSV["reserveTokens<br/>16384 by default"] --> CMP
  CMP -->|no| REQ["request sent"]
  CMP -->|yes| CP["compaction"]
  CP --> REQ
  

The full skill body enters only when a task matches its description, which is what stops a directory of skills from costing you context before you have used any of them.

Two details follow directly. usage.md adds that the footer shows current context usage, and /session reports the session file, its ID, message count, token usage and cost. Both are your instruments. Read them before you guess.

The active branch is not the session file

A session is a tree. Each entry has an id and a parentId, and the current entry identifies the active branch — the path from the root to the current leaf. Pi derives the request’s conversation history from that path, then applies compaction and context edits. Abandoned branches stay in the file but are not included directly in the request; their contents can still appear indirectly in a branch summary.

Wrong: “The session is what the model sees.”

Correct: “The model sees a projection of the active branch, after compaction and context edits. The session file also holds alternative branches.”

This is the cheapest context saving available to you and it costs nothing. When an approach stalls, open /tree, select an earlier user message, edit it, and submit. You get a fresh branch that does not carry the failed attempt.

The trigger is arithmetic, not judgment

compaction.md states the condition exactly:

contextTokens > contextWindow - reserveTokens

reserveTokens defaults to 16384. It is a margin used to decide when to compact so there is room for the model’s response; it is not a guarantee that the provider will accept every request below the threshold. settings.md calls the setting compaction.reserveTokens, configurable in ~/.pi/agent/settings.json or <project-dir>/.pi/settings.json.

Pi checks this between turns, after tool results are appended and before the next assistant response. It also checks before a new user prompt. So the threshold is enforced against the projection it will really send, not against an estimate taken earlier.

There is one asymmetry worth knowing. A provider context-overflow error, or an early stopReason: "length", can select one compact-and-retry recovery attempt. That recovery is a repair path, not a strategy. message-types.md lists "length" as a terminal stop reason on AssistantMessage, alongside "stop", "toolUse", "error", "aborted", and "deferred".

What a compaction replaces

Compaction is a summary, not a deletion. Pi chooses a cut point, summarises the messages before it, records the summary and the first entry it kept, and rebuilds the request as the summary plus the kept messages. The original entries stay in the file. This chapter needs only the consequence for the budget: after a compaction the request is smaller by whatever the summary does not carry.

How the cut point is chosen, why it never lands on a tool result, what the summary contains, and how an extension can take the summary over are chapter 23’s subject, and the reason this chapter stops here.

Tune the budget per model

Both token settings resolve independently: model override first, then the ordinary setting, then the built-in default. Keys are exact, case-sensitive provider/modelId values.

{
  "compaction": {
    "enabled": true,
    "reserveTokens": 16384,
    "keepRecentTokens": 20000,
    "modelOverrides": {
      "some-provider/big-model": { "reserveTokens": 400000 }
    }
  }
}

For a model with a 1M context window, that override triggers compaction above 600K tokens while keeping the ordinary 20000 recent tokens. enabled stays global; it is not model-specific.

Set "enabled": false to turn off automatic compaction. /compact still works — sessions.md says disabling automatic compaction does not disable the manual command.

Give the model a better cut

/compact accepts instructions. Use them when the next phase of work does not need the same detail as the last one:

/compact Preserve the failing test names and the chosen error-handling pattern. Drop the file-listing digressions.

This is the manual lever the built-in threshold cannot give you. Automatic compaction is generic; a manual compaction can be aimed.

What compaction does not do

It does not shrink the system prompt, it does not summarize tool definitions, and it does not shorten the messages after firstKeptEntryId. If your context pressure comes from a very large AGENTS.md, from fifteen enabled tools, or from a habit of cat-ing whole files through bash, compaction is treating the symptom. Fix the source.

It also does not erase the earlier entries. Navigate to a point before the compaction entry and that summary is no longer on the active path; the original messages can again contribute to a request, subject to the context budget and any other edits on that branch. A context_edit omission is likewise branch-relative, as session-format.md explains. Recoverable history is not the same thing as history that will fit into the next request.

Read your own budget

Before blaming the model, look:

/session

Compare tokensBefore on the last CompactionEntry with the threshold contextWindow - reserveTokens. If compaction happens every few turns, inspect how quickly tool results and other messages rebuild the context. Reduce oversized tool output, consider whether keepRecentTokens retains too much history, and check whether the system prompt or tool definitions impose a large fixed cost. Raising keepRecentTokens retains more history after compaction; it does not by itself make repeated compaction less likely.

What was run

Chapters 4, 5, 23 and 24 are tested on one shared history, so that they read as one subject, state over time, and not as four. It is three turns in a persisted session: question A and its answer; question B, which makes a bash tool call and then answers; question C and its answer. examples/harness/history.ts builds it, and each chapter reads a different property of the same run.

Chapter 4’s question is what is sent, and when does the budget become a problem? examples/ch04-budget/ answers it with the real session:

• the footer’s figure is real: getContextUsage() reports tokens against the window, percent is their ratio, and from the second turn on each answer makes the request larger. The first reading is higher than the second because it includes the system prompt and tool declarations, so do not compare it with what follows;
• the trigger is arithmetic. With the threshold set to 3000 tokens, compaction did not happen on the first turn, the turn before it was at or below the threshold, and the entry it wrote recorded a tokensBefore above it;
• a larger reserve compacts earlier: the same conversation under two reserves compacted on two different turns, in the expected order;
• a per-model override changed the threshold for faux/faux-1, and a key with the wrong case did nothing;
• enabled: false stopped automatic compaction and the manual command still worked;
• /compact instructions reached an extension’s session_before_compact handler unchanged;
• after navigateTree() to question B, the next request contained question A and not question C, while the file still held C: the model sees the active branch and not the file;
• only active tools are declared to the model: with tools: ["read"] the request declared exactly read.

The budget in one view

The arithmetic above, gathered. Each row is a thing that costs on every request and a way of reducing it — and the last column is what it costs you to do so, which is the part a percentage does not tell you.

Two rows deserve emphasis because they are the ones people forget. Persistent context files are a standing tax: compaction does not shorten them, so a large AGENTS.md continues to consume space whenever it is included. Tool results can also grow the active history quickly — a result containing a thousand tokens can consume roughly that much space on subsequent requests until compaction, a context edit, or a branch change removes it from the request projection. That is a strong reason to truncate large results and provide a file path for details instead.

Next

A budget only means something against something you can go back and inspect, and so far everything in this chapter has been arithmetic about content you cannot open. The next chapter opens the file itself: what is on disk, what shape it takes, and which of its lines the model ever saw.

Chapter 5: sessions and where they live.