Models and Thinking Level

Something is not working. The agent is not using your tool. It is ignoring an instruction. It stops halfway. Before you go looking in your extension, check which model is running and what thinking level it is set to.

This is not a joke about blaming the model. It is a description of how the system is layered: model choice and thinking level sit underneath every mechanism in this book, and they change behaviour in ways that look exactly like bugs in the layer above.

Model, provider, credential

Three words that the documentation keeps distinct and that people routinely collapse.

A model generates Pi’s responses. A provider is the service or account Pi uses to reach that model. A credential is what makes the connection possible — a subscription, an API key, or something ambient.

The consequence is a specific and very common failure. Your custom model is configured, the entry parses, the file reloads, and nothing appears in /model. The reason is usually authentication: the picker shows models whose providers have usable authentication. A model can load from models.json and still be unavailable until Pi can resolve credentials for it.

Authenticating

Run /login, choose a provider, and follow the prompts. Credentials saved through this login flow go in auth.json in the agent directory; credentials supplied through environment variables need not be written there. /logout removes stored authentication for the selected provider.

Or supply an API key through the provider’s environment variable. This is the right choice in CI and anywhere Pi should not write credentials to disk.

When several sources are configured, Pi resolves them in this order:

    flowchart TD
  A["1  a runtime --api-key"] --> B["2  a stored auth.json credential"]
  B --> C["3  an apiKey from models.json"]
  C --> D["4  provider environment variables,<br/>or ambient cloud credentials"]
  

Provider extensions may define their own behaviour.

The first source that yields a credential wins, and the rest are never consulted. That single rule explains a family of confusing symptoms. A key you pass at runtime beats a stale auth.json, which in turn beats an apiKey you wrote into models.json, which beats the environment variable you just exported. If you changed an environment variable and the agent still cannot authenticate, first check which credential source won. Then check that the variable is present in the process that starts Pi. A variable exported in a different shell may never reach that process.

Note the ordering, because it is the opposite of the intuition and it is what the diagram above says. A credential in models.json sits below stored authentication, not above it: it is the way to give a custom or self-hosted model its key, not a way to override a key you have already logged in with.

Wrong: “My key is in the environment, so Pi must be using it.”

Correct: “My key is in the environment, but Pi uses it only if the process received it and no higher-priority source resolved first.”

One warning belongs here. Keep auth.json, API keys and any commands that produce credentials private. A trusted project can load extensions that run in the Pi process with the user’s operating-system permissions; project trust is not a sandbox. Chapter 8 covers trust; chapter 42 covers authority and isolation.

Selecting a model

/model

opens a picker of available models with usable authentication. Ctrl+S on a model saves it as the default for new sessions.

Ctrl+P cycles through available models. /scoped-models controls that cycle and saves the selection, and you can configure model patterns through settings.

Two behaviours worth internalising:

A session records its model changes. Resuming the session restores the model and thinking level it was using, without changing your defaults for new sessions. So “it behaves differently now” may mean the session switched models two hours ago and never switched back.

The catalogue is not static. Pi starts with a bundled catalogue and can overlay newer data from pi.dev; cached data stays available offline. Force a refresh with pi update --models.

Thinking level

/thinking

selects the thinking level for the current model, and Pi limits the choices to the levels that model supports. Ctrl+S saves the startup level.

The full set of settings values is "off", "minimal", "low", "medium", "high", "xhigh", "max". The startup default is "medium".

Token budgets for minimal, low, medium and high come from thinkingBudgets. modelThinkingLevels sets per-model startup levels keyed by exact provider/modelId.

{
  "defaultThinkingLevel": "medium",
  "modelThinkingLevels": {
    "anthropic/claude-sonnet-5": "high"
  },
  "thinkingBudgets": {
    "high": 32000
  }
}

Thinking blocks appear in the transcript. hideThinkingBlock hides them. This is a display setting, and it matters for extension authors: what you see in the terminal is not necessarily what was stored.

Thinking level is another variable to control when debugging. The same model at off and at high may approach a task differently. If your extension’s tool is being called in a way you did not expect, confirm the model and thinking level before changing the tool.

Sampling per thinking level (1.0.2)

Since 1.0.2, a model entry can carry samplingParams as free-form defaults and samplingParamsByThinkingLevel as per-level overrides, on OpenAI-compatible APIs.

{
  "id": "qwen-thinking-model",
  "reasoning": true,
  "samplingParams": { "temperature": 1.0, "top_p": 0.95 },
  "samplingParamsByThinkingLevel": {
    "off": { "temperature": 0.7, "top_p": 0.8 },
    "high": { "top_k": 20 }
  }
}

Three details are worth stating because they are the ones you would otherwise get wrong. The keys are Pi’s thinking-level names — off, minimal, low, medium, high, xhigh, max — not the provider’s own thinkingLevelMap values. The merge order is: clamp unsupported levels, then apply model samplingParams, then the effective level’s override, then any request-level samplingParams, with later values winning per key. And a level you do not list simply inherits the model defaults rather than failing.

The field is restricted to openai-completions, openai-responses and azure-openai-responses; other APIs ignore it silently, which is the sort of thing worth knowing before you conclude a knob does nothing.

Thinking level and sampling temperature are distinct controls: one selects a model-supported reasoning setting, while the other, where supported, affects sampling. Configuring sampling parameters by thinking level deliberately couples them. For example, a low temperature at high reasoning and a higher one at off expresses a choice you made, not a Pi requirement. Chapter 42 argues that application-specific policy should be owned by the application, even when the configuration field carrying it belongs to the model layer.

Model startup and defaults

Three settings govern initial model selection and cycling:

enabledModels defaults to all available models and uses the same format as the --models flag.

Prompt caching, and why it appears here

Caching is a model-level concern, and it shows up in agent behaviour more often than you would expect.

promptCache declares the provider’s best-effort cache lifetime, in seconds, for the short or long retention tier:

{ "id": "claude-sonnet-5", "promptCache": { "short": 300, "long": 3600 } }

Choose the conservative end of any published range. A model without a lifetime for the active tier is not eligible for cache warming.

cacheWarming is "off", "streaming" or "idle", defaulting to "streaming". Warming runs only when the model declares a cache lifetime and Pi estimates at least $0.05 in avoided cache-miss cost. Refresh usage counts toward session totals but does not enter model context — which is a detail to remember when your context accounting and your billing disagree.

showCacheMissNotices surfaces significant cache misses, successful warming, compaction usage and provider recovery. If you are seeing unexplained latency or unexplained token counts, turn it on before you instrument anything.

Models that are not chat models

Two categories exist that do not appear in /model, and knowing they exist prevents a class of confusion.

Classifier models do not chat. They answer typed questions about JSON state: choose one of several options, answer yes or no, or produce a score, each with probabilities. As of 1.0.1 that list includes Cloudflare’s Clef and Clef Flash from Workers AI, alongside the TypeSafe Jev models; the new entries are @cf/cloudflare/clef and @cf/cloudflare/clef-flash under the cloudflare-workers-ai provider, and they are usable from codemode scripts and from extensions. The classification-vs-chat distinction did not change with them, which is the point: a new classifier appears in a catalogue row and never appears in /model, and readers who have not internalised that spend time looking for it.

Image models generate images from a prompt and optional input images.

Neither appears in the /model picker. Both are reached through the codemode tool, which a built-in extension registers inactive. Enable it yourself with:

{ "defaultTools": ["+codemode"] }

Extensions reach them directly instead, without codemode, through ctx.modelRegistry.classify() and ctx.modelRegistry.generateImages().

If you have been looking for an image model in /model and it is not there, this is why, and it is not a bug.

Endpoints you can talk to

When Pi does not already support the provider or endpoint you need, consider three different integration approaches. They are alternatives for different requirements, not necessarily consecutive steps:

1. A compatible endpoint. If the endpoint speaks an API Pi already supports — most Ollama, LM Studio, vLLM, SGLang and proxy deployments — add it to models.json in the agent directory.

   {
     "providers": {
       "ollama": {
         "baseUrl": "http://localhost:11434/v1",
         "api": "openai-completions",
         "apiKey": "ollama",
         "models": [
           { "id": "qwen2.5-coder:7b" }
         ]
       }
     }
   }
   

   The dummy key makes the model available; Ollama ignores it. For an authenticated endpoint, apiKey and header values accept $NAME or ${NAME} environment interpolation, a literal, or a leading !command. Commands run at request time and are not cached by Pi.

   Opening /model reloads the file.

2. A custom provider extension, when the provider needs custom streaming, model discovery or authentication behaviour.

3. A virtual model, which registers under your own name and routes each request to whatever you decide — including a classifier. Chapter 42 returns to this, because “route it somewhere else” and “it may do that” are different decisions.

One warning from the documentation that is worth repeating because it is a common and expensive mistake: describe only verified differences in an endpoint’s request or response behaviour in your compatibility settings. Do not enable them because the endpoint advertises OpenAI or Anthropic compatibility.

What this chapter is for

The practical content is the tables and the precedence order. The idea is smaller and more useful:

model choice      affects what is possible
thinking level    affects how it approaches the task
credentials      affect whether anything happens at all
cache behaviour   affects latency and reported usage

None of these are your extension. All of them can produce a symptom you will otherwise debug for an hour in the wrong layer. When something is behaving impossibly, check this chapter’s list before you check your code.

What was run

examples/ch03-credentials/ runs credential resolution through Pi’s real ModelRuntime, over real auth.json and models.json files:

• the environment variable is used when nothing else is configured, and with nothing at all there is no credential;
• a stored auth.json credential beats the environment variable, and so does an apiKey in models.json;
• the order is exactly the one given above. With all four sources present the runtime key won. Removing it, auth.json won. Removing that, models.json won. Removing that, the environment variable was used. The first source that yields a credential wins and the rest are not consulted;
• a model can load from models.json and still be unavailable: the entry parsed and the model existed, but it was not in the available list until the provider had authentication, and a dummy key was enough to make it appear.

A test checks whether changing the source of a credential affects this particular agent run. The same extension (the force-push guard) and the same script ran with the provider’s key written three ways: a literal, $PI_TEST_KEY, and !echo from-a-command. The three resolved to three different credentials, while the scripted run produced the same roles, two tool results and request count, with the guard blocking the force push each time. In this test, credential sourcing did not change agent behaviour; that is not a guarantee about all providers or all runs.

Next

You now know what the loop runs on. What it runs with is the next constraint, and it is finite: every request competes for a fixed window, and your instructions are one line item among several rather than the whole container. Knowing that turns “the agent got worse” into arithmetic you can read.

Chapter 4: the context window as a budget.