Compaction

Two hours into a refactor, the footer says you are at 87% context. Nothing is wrong. Then it says the agent compacted, and the model asks you to re-explain the schema it was told about an hour ago.

That re-asking is one possible consequence of compaction, not necessarily a defect in your prompt. The original entries remain in the session file, but the older portion selected for compaction no longer reaches the next model request as individual messages. Pi sends a generated summary in its place and keeps the more recent messages. The summary may be produced by the current model, and it can omit details that the earlier conversation contained.

This chapter is about that mechanism: when it fires, what it replaces, and where you can intervene.

Two mechanisms, one idea

“Both use closely related structured formats and track file operations cumulatively. Summarization requests disable prompt-cache writes because these one-off prompts are unlikely to be reused.”

That last clause concerns the summarization request: Pi disables prompt-cache writes for that one-off request. The next normal agent request is rebuilt with a summary replacing part of its previous history, so its cache reuse may also change. Do not infer that all caching stops during compaction or that cache hits necessarily resume immediately afterward.

When it fires

The trigger is arithmetic, and chapter 4 owns it: contextTokens > contextWindow - reserveTokens, checked after tool results are appended and before the next request, before a new user prompt, and once more as a recovery when a provider rejects a request as too long. reserveTokens (default 16384) exists “to leave room for the LLM’s response”, which is why the trigger is a subtraction and not a comparison against the window. What this chapter adds is the part chapter 4 does not need.

The between-turn check is skipped in exactly one case: “when the completed tool batch terminates the run and no queued message requires another response.” No next request, no need to free space for it. And the recovery is a repair path with a hard limit: “A provider context-overflow error or an early final stopReason: "length" can select one compact-and-retry recovery attempt.” One attempt, not a loop. Chapter 29 is where recovery is told in full.

What compaction actually does

Five steps, from compaction.md:

1. Find cut point — walk backwards through the finalized session projection, accumulating token estimates until keepRecentTokens (default 20000) is reached.
2. Extract messages — collect projected messages from the previous kept boundary, or session start, up to the cut point.
3. Generate summary — call the LLM with a structured format, passing the previous summary as iterative context when one exists.
4. Append entry — save a CompactionEntry with the summary and firstKeptEntryId.
5. Rebuild context — the session rebuilds context for the next request using summary plus messages from firstKeptEntryId onwards.

Step 3 is the one that surprises people. When there is an earlier compaction summary, Pi passes it into the next summarization request along with the newly selected history. This preserves continuity without recovering the original detail that the earlier summary omitted. Repeated compaction can therefore accumulate omissions, but the effect is not necessarily gradual or predictable.

Drawn out, using the entry numbering from the documentation’s own example. Entries 0 to 9 exist before compaction; entry 10 is the compaction entry appended after it:

    flowchart TD
  subgraph FILE["Session file — every entry stays"]
    direction LR
    e0["0 hdr"] --> e1["1 usr"] --> e2["2 ass"] --> e3["3 tool"] --> e4["4 usr"] --> e5["5 ass"] --> e6["6 tool"] --> e7["7 tool"] --> e8["8 ass"] --> e9["9 tool"]
  end
  e9 --> c10["10 compaction"]
  subgraph NEXT["Next request — rebuilt context"]
    direction LR
        sys["system"] --> sm["summary of 0 to 3"] --> k4["4 usr"] --> k5["5 ass"] --> k6["6 tool"] --> k7["7 tool"] --> k8["8 ass"] --> k9["9 tool"]
  end
  c10 --> sm
  e4 --> k4
  e9 --> k9
  

The first box is the session file, and compaction only ever appends to it. The second box is what the model is given, and it is a different thing: system prompt, the summary, then the entries from firstKeptEntryId — entry 4 in this example — onwards.

Wrong: “compaction deletes the old messages.” Correct: it changes what the next request is built from. session-format.md: Pi “replaces older summarized entries with the compaction summary and keeps the range beginning at this entry.” compaction.md adds that “Omitted raw entries remain stored but do not affect cut selection, summaries, checkpoints, or token estimates.” Nothing is deleted from the file, and what the model lost is real anyway.

Cut point rules

Valid cut points are user messages, assistant messages, BashExecution messages, and custom messages. Then the rule that prevents half a tool call from existing:

“Never cut at tool results (they must stay with their tool call).”

A cut between a tool call and its result would leave the model asking about a tool invocation that never returns. The retained portion can therefore exceed keepRecentTokens in order to respect a legal cut point; the amount of overshoot depends on the messages and tool-call grouping, not on a guaranteed small tolerance.

There is a second case called a split user-message span. “A user-message span starts with a user message and includes all turns until the next user message.” Normally compaction cuts at one of those boundaries. But when a single span exceeds keepRecentTokens on its own — one long turn with many tool calls — the cut lands inside it, at an assistant message. The doc marks this isSplitTurn = true, with messagesToSummarize empty and the early part kept as turnPrefixMessages, and Pi “generates two summaries and merges them”: one for previous context, one for the user-message-span prefix.

The entry it writes

The shape, reproduced from session-format.md. This is a type declaration for reference, not an executable example.

interface CompactionEntry<T = unknown> {
  type: "compaction";
  id: string;
  parentId: string | null;
  timestamp: string;
  summary: string;
  firstKeptEntryId: string;
  tokensBefore: number;
  usage?: Usage;       // LLM usage that generated the summary
  fromHook?: boolean;  // true if provided by extension (legacy field name)
  details?: T;         // implementation-specific data
  systemMessage?: SystemMessage; // complete prompt and tool state at this boundary
}

// Default compaction uses this for details:
interface CompactionDetails {
  readFiles: string[];
  modifiedFiles: string[];
}

firstKeptEntryId is the whole mechanism. “Pi replaces older summarized entries with the compaction summary and keeps the range beginning at this entry.” Change that field and you change which history the model sees.

tokensBefore is recalculated “from the rebuilt, context-edited session projection before writing the new CompactionEntry, so the token count reflects the actual pre-compaction context being replaced.” Note what it does not do: “Omitted raw entries remain stored but do not affect cut selection, summaries, checkpoints, or token estimates.”

The summary format

Both compaction and branch summaries use structured output. Compaction summaries have sections Goal, Constraints & Preferences, Progress, Key Decisions, Next Steps, and Critical Context. Branch summaries stop after Next Steps. “Pi appends file lists to either format when relevant.”

## Goal
[What the user is trying to accomplish]

## Constraints & Preferences
- [Requirements mentioned by user]

## Progress
### Done
- [x] [Completed tasks]

### In Progress
- [ ] [Current work]

### Blocked
- [Issues, if any]

## Key Decisions
- **[Decision]**: [Rationale]

## Next Steps
1. [What should happen next]

## Critical Context
- [Data needed to continue]

Two deliberate choices here. Critical Context exists because the summary’s job is not narrative completeness — it is carrying the facts the next request needs. The Markdown block above shows the main sections, not every possible addition: Pi can also append <read-files> and <modified-files> lists when relevant. Those lists help the next turn recover which files were inspected or changed; they are not a guarantee that every consequential detail survives summarization.

Before summarizing, messages are serialized to labeled plain text rather than sent as a conversation: [User]:, [Assistant]:, [Assistant tool calls]:, and [Tool result]:. Thinking blocks are omitted from this serialization. The format is intended to steer the summarizing model to describe the conversation instead of continuing it; it does not enforce that behaviour. Tool results are truncated to 2000 characters, so information beyond that limit may not reach the summarizer at all.

Configuring it

The settings, enabled, reserveTokens, keepRecentTokens and the per-model overrides, are chapter 4’s. One subtlety belongs here because it concerns the summary itself: “reserveTokens also influences summarization output limits, capped by the model’s maximum output tokens; it is not solely a trigger threshold.” A 400000 reserve on a model with a 64K output cap does not make summaries longer, because the output cap binds first.

Taking it over with an extension

session_before_compact fires before auto-compaction or /compact, and has two documented outcomes. It can supply the summary while retaining the cut point prepared by Pi. The excerpt below demonstrates the return shape, not a production summarizer: it keeps only the first 500 characters of serialized messages and ignores both previousSummary and turnPrefixMessages. Copying it unchanged into a real session can permanently omit essential context from subsequent model requests:

pi.on("session_before_compact", async (event) => {
	const { preparation } = event;

	// preparation.messagesToSummarize : the messages that are about to be replaced
	// preparation.previousSummary     : the last compaction's summary, if any
	// preparation.firstKeptEntryId    : where the kept messages start
	const serialized = serializeConversation(convertToLlm(preparation.messagesToSummarize));

	return {
		compaction: {
			summary: `Custom summary of ${preparation.messagesToSummarize.length} message(s):\n${serialized.slice(0, 500)}`,
			firstKeptEntryId: preparation.firstKeptEntryId,
			tokensBefore: preparation.tokensBefore,
		},
	};
});

Or it can cancel:

pi.on("session_before_compact", async (event) => {
	return event.reason === "manual" ? { cancel: true } : undefined;
});

The event also carries branchEntries, customInstructions, reason ("manual", "threshold" or "overflow"), willRetry and signal. The first draft of this chapter showed both outcomes as two return statements in one handler, which cannot run, and a summaryResponse that was never defined. They are two handlers because they are two decisions.

Both were run, in examples/ch23-compaction/, through a real session whose keepRecentTokens was lowered so that a short scripted conversation had something to summarise:

• the summary came from the extension and the cut point stayed Pi’s: the entry has its firstKeptEntryId, and is marked fromHook: true;
• the next request was built from the summary plus the kept messages: the oldest question was no longer sent as a message, the most recent was, and the summary itself arrived as a user-role message (chapter 26’s compactionSummary conversion, observed);
• every message entry was still in the file afterwards, which is “compaction does not delete” as a count;
• a handler that cancels a manual /compact stops it: no compaction entry appears.

This chapter is also tested on one shared history, so that they read as one subject, state over time, and not as four. It is three turns in a persisted session: question A and its answer; question B, which makes a bash tool call and then answers; question C and its answer. examples/harness/history.ts builds it, and each chapter reads a different property of the same run.

On that shared history, this chapter tests what compaction replaces, and what survives it, in examples/ch23-compaction/state.test.ts:

• the cut point did not fall at a tool result in the seven tested keepRecentTokens settings from 50 to 2500; the following requests contained equal counts of tool calls and tool results, consistent with the rule that a cut must not separate them;
• compaction is iterative: the first compaction had no previousSummary, and the second was given the first’s summary;
• fileOps is three Sets: a file the model read was in read, a file it wrote was in written, and edited was empty;
• a single long span splits. One user message followed by six tool-calling turns produced isSplitTurn: true, messagesToSummarize empty, and eleven turnPrefixMessages, exactly the case the paragraph above warns a handler about;
• compaction does not shrink the system prompt or the tool declarations: the request after it still carried a system message with bash declared;
• it replaces what the model reads and nothing in the file: navigating to the entry before the compaction sent the original messages again and not the summary;
• an extension that cancels leaves the entry count unchanged and writes no compaction.

One important limitation of the earlier example is now visible. A split user-message span can put the early part of the span in preparation.turnPrefixMessages while leaving messagesToSummarize empty; that example then summarizes none of the prefix. It also omits previousSummary, even on ordinary repeated compactions. For a real extension, account for both the prior summary and the split-turn prefix, preserve needed file-operation metadata, and test what the provider receives afterward. The shipped custom-compaction.ts is a more complete starting point; do not copy the truncated illustration as a general-purpose compactor.

If you want to generate the summary with a different model, serializeConversation is the documented path: convert AgentMessage[] with convertToLlm, serialize to text, send it wherever you want. session_compact_failed is the terminal counterpart — “useful for telemetry extensions that need to pair session_before_compact attempts with terminal outcomes” — carrying reason, errorMessage, aborted, willRetry, and fromExtension.

Failure is not a dead end

“Compaction can fail if the provider is unavailable or cannot accept the summarization request. Correct the provider problem and run /compact again.” The failure mode is visible in the event stream as compaction_end with aborted: false and an errorMessage, and separately as summarization_retry_scheduled when a retry is scheduled.

One asymmetry worth knowing: “Pi does not automatically carry file lists from extension-generated summaries whose fromHook field is true; extensions manage their own details format.” If an extension-produced compaction drops those lists, the following default compaction cannot automatically inherit them from that entry. Newer tool calls may contribute new file operations, but they do not reconstruct the omitted history. Preserve the relevant metadata explicitly when implementing custom compaction.

If the model forgets a schema after compaction, investigate what the summary retained before assuming the original instructions were ignored. Put stable project conventions in a context file (see chapter 28) so they can be loaded independently of conversation history. For large or precise schemas, also keep the authoritative schema in a repository file the agent can reread rather than relying on any summary to reproduce it exactly.

The other exit from a full context is not compression at all. When a session takes a wrong turn, /tree, /fork, and /clone are three different answers to the same question.