Failures, Retries, and Recovery

The provider returns 529 overloaded. Pi waits two seconds and tries again. Twice more. Then it stops and tells you the model is unavailable.

That is correct — and it is the least interesting failure Pi handles. There are three responses to a failed request, they are not interchangeable, and picking the wrong one is how an agent ends up retrying a request that could never succeed.

Three responses, three meanings

Retry. The failure was transient. The request was fine; the server was not. Repeat it.

Recover. Pi recognizes a context-overflow failure, or a qualifying early final stopReason: "length", and may compact the conversation before making one fresh attempt. This repairs the context supplied to the model; it is not a general remedy for every deterministic error.

Stop. The request was wrong, or the condition is permanent. Retrying produces the same failure.

The mechanisms Pi keeps apart, and what each one does:

    flowchart TD
    F["a model request fails"] --> T{"recognized transient provider error?"}
    T -->|yes| R["agent-level retry with backoff"]
    R -->|succeeds| OK["run continues"]
    R -->|retries exhausted| S["auto_retry_end: success false"]
    T -->|no| O{"recognized overflow or qualifying early length stop?"}
    O -->|yes| C["omit the failed attempt from projected context; compact"]
    C -->|compaction succeeds| N["one fresh recovery run"]
    C -->|fails or is cancelled| Z["no internal recovery retry"]
    O -->|no| X["stop and report the provider failure"]
    G["a tool call occurs"] --> H{"tool_call handler blocks or fails?"}
    H -->|yes| B["tool not executed; error result for the model"]
    H -->|no| I{"tool execute succeeds?"}
    I -->|no| E["error result for the model"]
    I -->|yes| OK
  

These are different paths through Pi’s runtime. Pi classifies recognized transient provider failures for agent-level retry and recognized overflow or qualifying length failures for compaction recovery. Extension authors still have a classification responsibility when integrating a provider whose error wording Pi does not recognize. Tool-call failures follow their own path and should not be mistaken for provider-request retries.

Retry

rpc-commands.md defines the scope: “Enable or disable automatic retry on transient errors (overloaded, rate limit, 5xx).” Three categories, all of which mean try again in a moment.

{"type": "set_auto_retry", "enabled": true}

{"type": "abort_retry"}

The second command — “Abort an in-progress retry (cancel the delay and stop retrying)” — exists because the delay between attempts is exponential and can reach a minute. retry.maxAgentDelayMs defaults to 60000.

The agent-level settings:

And a separate provider-level layer:

Note the default of retry.provider.maxRetries is 0, and the documentation explains why with a warning rather than a rationale: “Keep retry.provider.maxRetries at 0 unless provider-level retries are required. Provider retries can delay Pi from handling quota and usage-limit errors itself.”

The important separation is between the provider library’s internal attempts and Pi’s agent-level retry decisions. If the provider library retries internally, Pi does not receive each intermediate failure as a separate agent-level retry event, and handling of quota or usage-limit errors can be delayed. Stacking the two layers can therefore increase latency and make diagnosis harder; keep provider-level retries disabled unless the integration needs them.

Timeouts sit next to retries because a hang is a failure with no error message. httpIdleTimeoutMs defaults to 300000 and “Set to 0 to disable.” websocketConnectTimeoutMs defaults to 15000. Turning a timeout off does not disable retries; it means a dead connection now looks like a slow one.

Recovery

Recovery is a different mechanism with a different trigger and a hard limit.

compaction.md: “A provider context-overflow error or an early final stopReason: "length" can select one compact-and-retry recovery attempt.”

One attempt. The context-overflow error means the request exceeded the window — sending it again, unchanged, will overflow again. The only way forward is a smaller request.

The ordering matters and is specified step by step:

persist final assistant response
→ extension/public turn_end
→ extension/public agent_end
→ append context_edit omissions for the selected attempt
→ for overflow/length: run session_before_compact and append compaction on success
→ start the retry as a fresh run

Three things to notice. The attempt that failed is still visible — turn_end and agent_end fire for it normally, and “Raw transcript history, exports, billing totals, and history-search extensions can still inspect the omitted attempt.” Recovery repairs the model context, not the record. Second, the omissions are context_edit entries, so they are branch-relative and append-only: the raw entries stay, they just stop reaching the model. Third, the retry is “a fresh run.” Not a resumption.

When recovery does not work: “If recovery compaction fails or is cancelled, Pi keeps the omission edits, appends no compaction, and schedules no internal retry.” Omissions kept, no compaction, no retry — the session ends with a repaired-but-summariless projection and the failure visible to you.

agent_before_settle “sees the repaired projection after recovery processing.” An extension inspecting that boundary can therefore see the context after omission edits and any completed recovery work, rather than assuming that the original failed attempt is still in the active projection. It is not, by itself, a record of the exact request a future run will send: later request-local transformations may still apply, and a failed or cancelled recovery schedules no internal retry.

Events that tell them apart

json.md gives a distinct event family to each. This is how you tell what Pi is doing without watching the transcript.

Retry:

{"type":"auto_retry_start","attempt":1,"maxAttempts":3,"delayMs":2000,"errorMessage":"529 overloaded"}
{"type":"auto_retry_end","success":true,"attempt":2}

“On final failure, auto_retry_end has success: false and a finalError string.”

Summarization retry is a separate family, for the compaction call itself:

{"type":"summarization_retry_scheduled","attempt":1,"maxAttempts":3,"delayMs":2000,"errorMessage":"terminated"}
{"type":"summarization_retry_attempt_start","source":"compaction","reason":"threshold"}
{"type":"summarization_retry_finished"}

“For a branch summary, source is "branchSummary" and reason is absent. The reason on a compaction retry is "manual", "threshold", or "overflow".”

Compaction and recovery, where reason is "manual", "threshold" or "overflow" — and overflow is the one that tells you a recovery is in progress:

{"type":"compaction_start","reason":"overflow"}

{
  "type": "compaction_end",
  "reason": "overflow",
  "result": {
    "summary": "...",
    "firstKeptEntryId": "abc123",
    "tokensBefore": 150000,
    "estimatedTokensAfter": 32000
  },
  "aborted": false,
  "willRetry": true
}

“Successful overflow recovery sets willRetry to true before Pi retries the prompt.” If compaction was aborted, result is absent and aborted is true. If it failed, result is absent, aborted is false, and errorMessage describes the failure.

So: auto_retry_start announces an agent-level retry attempt; a compaction_start with reason: "overflow" identifies context-recovery compaction; and a successful compaction_end with willRetry: true indicates that Pi intends to make a fresh recovery attempt. auto_retry_end with success: false means the automatic transient-error retry path was exhausted; other failure paths can stop without emitting that event.

Where the classification comes from

For a custom provider, custom-provider.md is explicit about the boundary you must not cross: “Pi can compact and retry after recognized context-overflow errors. If the service uses an unknown message, normalize only that provider’s overflow response to context_length_exceeded in a guarded message_end handler.” And then, just as firmly: “Do not rewrite rate limits or transient provider failures as context overflow. Those failures use Pi’s normal retry behavior instead.”

import type { ExtensionAPI } from "@earendil-works/pi-coding-agent";

// Replace this identifier with the provider whose nonstandard overflow
// response you have verified. Do not normalize errors from other providers.
const affectedProvider = "your-provider-id";

export default function (pi: ExtensionAPI) {
	pi.on("message_end", async (event) => {
		const msg = event.message;
		if (msg.role !== "assistant") return;
		if (msg.provider !== affectedProvider) return;
		if (msg.stopReason !== "error") return;
		if (!msg.errorMessage?.includes("model is too long for this context")) return;

		return {
			message: { ...msg, errorMessage: "context_length_exceeded" },
		};
	});
}

The handler returns { message }, not the message itself: the result type is MessageEndEventResult, whose message field replaces the finalised message and must keep its role. The first draft of this chapter returned { ...msg, errorMessage } directly, which does not satisfy that result type. The revised example also requires a real affectedProvider identifier: checking only stopReason and a substring would contradict the provider-specific classification rule above. Test both a genuine overflow from that provider and a similar error from another provider before relying on the handler.

The original error-text normalization was run in examples/ch29-failures/ against a scripted provider that fails with model is too long for this context, in a session whose compaction was enabled. The results below establish that recovery path. The provider-identity check added to the example above is a separate hardening change: set the identifier to match the test provider and rerun the fixture, including a negative test for a different provider, before treating that guard as tested.

• with the handler, the events were compaction_start with reason: "overflow", then compaction_end, then a second run that produced the answer. The failed attempt was omitted from context with a context_edit entry rather than deleted, and a compaction entry was appended: the sequence the recovery section above describes, observed;
• without the handler, the same message was just a failure. No compaction was attempted, because Pi did not recognise it;
• with the handler present, a rate-limit message that merely contained the word length was left alone.

The classification is consequential in both directions. An unrecognized overflow can be reported as an ordinary failure rather than triggering compaction recovery; a transient error incorrectly rewritten to context_length_exceeded can trigger unnecessary compaction and context loss. The example’s narrower phrase match does not mistake an error merely for containing the word “length” — that is why the negative rate-limit test matters, alongside a provider-identity check.

Failures inside extensions

The retry machinery is only about model requests. Tool and handler failures follow different rules, and extensions.md draws the line:

“Pi reports handler errors and continues where possible. A tool_call handler failure blocks the tool as a fail-safe; a tool execution failure becomes an error result for the model.”

Two different destinations, and the difference is about execution, not about the model seeing something. A tool_call handler that returns { block: true } prevents the tool from running — execute() never runs, so there is no real effect to report. But the loop still emits a tool result for the call, carrying isError: true and your reason, and that result does reach the model as data. Chapter 16 and chapter 39 both describe this: a blocked call comes back as an error result the model can read and respond to. What is prevented is the tool; what is communicated is the refusal. A tool execution failure behaves the same way from the model’s side, and “Throw from execute() to produce a failed tool result” is the documented way to fail deliberately. For state you want to carry even when a result carries data: “return the result with isError: true instead of throwing: the model sees an error, and scripts still receive structuredContent.”

Shell commands are the third case. rpc-commands.md documents excludeFromContext on bash: “Set excludeFromContext to true when the command output should be stored in the session but omitted from the model context on the next prompt.” A build log nobody needs to see again is not a failure, but it is the same shape of decision — keep it out of what the model reads.

Cleanup after a failure

extensions.md: “Release resources in session_shutdown even when normal operation attempted cleanup. Keep cleanup idempotent because cancellation, reload, session replacement, and process exit can converge on the same path.”

Four different ways to end, one cleanup function. Idempotent is the requirement, not tidiness.

And “Use ctx.shutdown() to request an orderly process shutdown.” This gives Pi the opportunity to dispatch its normal shutdown handling and release session resources. It is not a durability guarantee: an abrupt process termination, runtime crash, or forced kill may bypass extension cleanup. Persist anything important before shutdown rather than relying on the shutdown hook as the only write point.

Configuring it

A reasonable starting point, with the reasoning visible:

{
  "retry": {
    "enabled": true,
    "maxRetries": 3,
    "baseDelayMs": 2000,
    "maxAgentDelayMs": 60000,
    "provider": {
      "timeoutMs": 300000,
      "maxRetries": 0,
      "maxRetryDelayMs": 60000
    }
  },
  "httpIdleTimeoutMs": 300000,
  "transport": "auto"
}

The configuration enables agent-level retries for recognized transient provider errors, leaves provider-level retries off, and keeps request timeouts finite. It also shows transport: "auto", which is already the documented default: writing it explicitly does not improve reconnection behaviour. Choose a different supported transport only when the provider and the observed connection behaviour justify that choice; transport selection and retry classification are separate concerns.

When a retry is the wrong answer

A bad API key, an unknown model, or a malformed endpoint requires correcting configuration rather than retrying unchanged. Pi’s documented automatic retry scope is recognized transient provider failures; do not assume it will repeat every deterministic error three times. If a service misclassifies a permanent error as transient, disable automatic retries while diagnosing it or correct the integration’s classification. models.md points to the relevant checks: usable authentication, the configured API type, and upstream compatibility with the expected request fields.

The shape of it

Classify the failing operation first. For a model request, a recognized transient provider error can use backoff and retry; a recognized context overflow or qualifying early length stop can use Pi’s bounded compaction recovery; other provider errors generally need reporting or an external fix. A tool execution failure becomes an error result for the model, while a tool_call handler failure blocks the tool as a fail-safe. Compaction is a specific context repair, not a generic retry for every fixable error.

That is the end of Part III. You have the run loop, the queue that redirects it, the tree that rewinds it, the messages it carries, the file that stores them, the instructions that outlive them, and the failure paths that end it. None of that tells you which one just misbehaved in your extension — which is a different question, and a mechanical one.