The Smallest Useful Model Call

Part 2 — Get the Model Out of the Chat Box

The smallest working call

CodeAI’s OpenCode cognition adapter, invoked directly without the recorded-call runtime, does the minimum well. Chapter 1 reached it through prompt(), which builds the CallSpec for you. Underneath, one CallSpec becomes one user message, the message goes out as an ordinary HTTP request, and a CallResult comes back:

adapter = OpenCodeCognitionAdapter(model="mimo-v2.5", protocol="chat_completions")
result = adapter.invoke(spec)  # one CallSpec in, one CallResult back
if result.status != "succeeded":
    raise RuntimeError(result.error)
review = result.raw_output

It keeps failure in its own channel instead of passing a diagnostic off as a review. It returns the usage with its source label alongside the convenient string. Those two habits already put it ahead of a great deal of production code.

Now ask it the questions Chapter 10 said every call must answer. Which gateway served this? Which model, at which revision? What did it cost, under which price table? How many attempts did it take? What exactly came back before anything parsed it — and where is that stored now that the process has exited?

The string answers none of them. The in-memory CallResult answers some of them, until the process ends.

What is the smallest model call that is still useful for everything Part 1 asked of it?

The answer is larger than a function that returns text, and the size of that gap is the subject of this chapter. A string is the smallest working call. The smallest useful call is a recorded process event.

Why “useful” costs more than “working”

Every requirement below was argued earlier and is now due.

None of that is intelligence. All of it is bookkeeping around one stochastic step, and all of it is deterministic: Chapter 3’s rule, applied to the call itself.

Three identities that must never collapse

task_id  ≠  call_id  ≠  attempt_id

Each answers a different question.

• Task — what are we trying to get done? Review paragraph P for unsupported claims.
• Call — what did we decide to ask, of which chamber? Ask deep-review to review P.
• Attempt — what happened on the wire, this time? Sent at 07:49:33; answered 21.8 seconds later.

Collapse them and specific things become impossible to say. Here is the defect that shows why. It comes from a single-result client inspected while writing this book, not CodeAI’s adapter, and one that is otherwise good. When an explicit output limit is set and the first response comes back empty, the client retries once, at twice that limit. The caller calls it once and receives one result. Two requests reached the provider; one usage figure came back. A cost computed from that result understates the work, and nothing in the returned value says so.

A retry is not a new task and not a new call. A retry is another attempt beneath the same call. Once attempts exist as records, “how many times did we actually ask?” has an answer that does not depend on reading adapter source.

The three identities as a recorded flow, in write order:

    flowchart TD
    T["task<br/><i>what are we trying to get done?</i>"] --> C["call<br/><i>what did we decide to ask, of which chamber?</i>"]
    C --> M["manifest<br/><i>intent, frozen before any attempt</i>"]
    M --> A1["attempt 1<br/><i>what happened on the wire, this time</i>"]
    A1 --> O1["raw observation 1<br/><i>preserved before parsing</i>"]
    A1 -.->|"retry: another attempt<br/>beneath the same call"| A2["attempt 2"]
    A2 --> O2["raw observation 2"]
    O1 --> CC["call.completed"]
    O2 --> CC
  

Intent before effect

The central move in CodeAI’s recorded call is to write down what it intends before anything happens, and to record what it observed separately.

Immediately before a request leaves, CodeAI freezes a manifest: the chamber, the logical model requested, the occupant it resolved to, the pricing version in force, the prompt and context identity, the parameters the caller asked for, and the adapter’s sanitized view of the effective parameters. At this stage, that last field is not proof of the exact wire request; the sanitizer can omit fields, and Chapter 12 will add an exact prepared-request identity. Each attempt then records execution evidence: when it started and finished, the provider’s request id, the usage reported or inferred with its source, and a pointer to the preserved raw observation. The attempt’s status and error_kind are conclusions about that evidence, not transport facts returned by the provider — a distinction the live runs below expose.

A simplified extraction of the two records:

@dataclass(frozen=True, slots=True)
class CallManifest:            # intent: written before any attempt
    call_id: str
    task_id: str
    chamber: str | None
    requested_model: str | None
    provider: str | None             # the gateway, e.g. "opencode"
    resolved_model_id: str | None
    provider_revision: str | None    # None means unknown, never a timestamp
    pricing_version: str | None
    prompt_hash: str | None
    requested_parameters: Mapping[str, Any]
    effective_parameters: Mapping[str, Any]

@dataclass(frozen=True, slots=True)
class AttemptRecord:           # observation: one per try
    attempt_id: str
    call_id: str
    task_id: str
    attempt_index: int
    protocol: str | None             # wire dialect, kept apart from the gateway
    provider_request_id: str | None
    status: str
    error_kind: str | None
    usage: Usage                     # counts may be None, with a source label
    cost_usd: float | None
    raw_artifact: ArtifactRef | None # content-addressed, stored before parsing
    normalizer_version: str | None

This is not a new idea dressed up for AI. The W3C provenance model separates the entities that exist, the activities that generate and use them, and the agents responsible, and relates them with terms such as used and wasGeneratedBy (Moreau & Missier, 2013). Read the recorded call in those terms. An attempt is an activity; it used the manifest; the raw observation is an entity that wasGeneratedBy that attempt; the interpreted result is a further entity derived from the observation. Keeping those links is what lets a later process ask which observation produced which conclusion.

The write order is the design

CodeAI appends events in a fixed order: call.requested, call.manifest, then for each attempt attempt.started, the provider request, the raw artifact, attempt.completed, and finally call.completed.

The order matters because of what happens when a process dies in the middle. Birrell and Nelson, describing one of the first practical remote procedure call systems, stated the guarantee plainly. If a remote call returns, the procedure ran precisely once. If an exception is reported instead, it ran “either once or not at all—the user is not told which” (Birrell & Nelson, 1984). A model call over HTTP inherits exactly that uncertainty. The gateway may have served the request and counted it; your process may never know.

CodeAI cannot remove that ambiguity, and the runtime’s own contract says so: it promises clean-restart durability, not exactly-once execution. What writing intent first buys is narrower and valuable. If the process dies after attempt.started, the ledger holds an attempt with no completion — a visible ambiguity instead of a silent loss. An interrupted call is always inspectable as unresolved, because its manifest was written before anything was attempted.

A constructed crash window makes the states concrete — fake adapter, no network, same identities as the trace above:

crash after call.manifest, before attempt.started
  ledger: call.requested, call.manifest — no attempt
  safe reading: no provider effect was possible; starting is safe

crash after attempt.started, before any completion
  ledger: call.requested, call.manifest, attempt.started — no attempt.completed
  safe reading: the provider may or may not have served it; reconcile from
  the gateway (or a later reading of its books) before deciding, never repeat blindly

The second row is the one to internalize. It is not a failure with a retry button; it is a question the ledger leaves open on purpose, with the intent still readable beside it. A later part of the book gives that question a name and a procedure; until then, “unknown outcome, do not repeat automatically” is the complete discipline.

Four states that are not the same

The rule that runs through the whole implementation:

unknown  ≠  absent  ≠  zero  ≠  inferred

Concretely:

Zero is a measurement. Treating “nothing was reported” as zero is not a harmless default. Rubin showed that conclusions from missing data depend on assumptions about the process that made the value missing (Rubin, 1976).

A failed model call is one place where that process matters. A timeout, dropped connection or rejected response can leave usage unavailable even when the provider may have performed some work. This chapter does not measure how often that happens, so it should not invent a frequency. The narrower consequence is enough: if unknown usage is filled with zero in those cases, cost can be understated. That application to token accounting is this book’s argument, not Rubin’s result.

Two consequences in the implementation are worth noticing.

Call totals go unknown if any attempt is unknown. If attempt 1 reported no usage and attempt 2 reported 264 input tokens, the call’s total input is recorded as unknown rather than 264. That is honest, and also lossy: “at least 264, completeness unknown” would preserve more without lying. The current choice is the conservative one, and Chapter 13 should revisit it when it designs usage interpretation.

A number the gateway states is not automatically a price. The live OpenCode response below includes a field "cost": "0". CodeAI does not import it, and the book’s own preserved calls show why. The same string appears in every OpenCode response this book kept, whatever the model: on subscription models such as mimo-v2.5, kimi-k2.7-code and gpt-5.6-luna, and on muse-spark-1.3-contributor-free, a free model.

Those are different billing contexts. OpenCode Zen includes routes with a zero monetary token price, some offered temporarily or with data-use conditions (OpenCode Zen). OpenCode Go is a subscription with usage allowances rather than a simple per-call cash charge (OpenCode Go). A response field containing zero does not by itself tell you whether a call was on a free route, covered by a subscription, consumed scarce allowance, or merely used a gateway-specific accounting field. The call below went to a Go subscription model, not a free one.

A field that reads the same across those situations cannot establish price semantics by itself, so until its meaning is established the honest record is unknown. The larger lesson for Chapter 6 is that pricing has a shape, not just a rate: free with conditions, subscription or quota-shaped access, or a price per token. A simple per-token table cannot represent all of them.

An offline trace, run end to end

What follows was executed for this chapter against CodeAI as it stood at this stage, with the fake adapter: no network, no credentials. The fixture’s “review” is a canned string. It is evidence that the runtime records execution, and no evidence of model ability whatsoever.

Three separate processes: create a task, record one call through the deep-review chamber, then reopen the ledger and inspect the call.

$ codeai run create "Review paragraph P for unsupported factual claims" \
    --success "every flagged claim names a checkable source"
run_id: b464dd30-ccee-4d67-855b-b030ea41537a
task_id: 64883b78-fc50-4840-aa87-06b86b1d72f3

$ codeai call --task 64883b78-… --chamber deep-review --logical-model review \
    --prompt "Review this paragraph …" \
    --response "The one-second latency claim requires measurement."
call_id: b2a5c12e-120d-479e-a51d-91e1b07174e3
chamber: deep-review
requested_model: review
provider_revision: UNKNOWN
pricing_version: 2026-09-01
attempt 1: id=7ad5f5f2-bc6b-4497-a001-7d00663e5287 status=succeeded
  raw_artifact: e75d23fa88618ac369653efc87343fe4c39bdf5acec7c43d775971f7e2ece453
  usage_source: measured cost=None
call_status: succeeded
task_status: not automatically completed

$ codeai calls show b2a5c12e-120d-479e-a51d-91e1b07174e3      # a new process
call_id: b2a5c12e-120d-479e-a51d-91e1b07174e3
attempt 1: id=7ad5f5f2-bc6b-4497-a001-7d00663e5287 status=succeeded
  raw_artifact: e75d23fa88618ac369653efc87343fe4c39bdf5acec7c43d775971f7e2ece453
call_status: succeeded

The identities survive a process boundary. The revision is unknown and says so. The cost is None because the fake model has no price, not because it was free. The first command also reports task_status: not automatically completed; the second process reopens the same call and still sees call_status: succeeded. Together they establish the boundary Chapter 14 is built on: the cognition step succeeded, and the task is not complete.

A second constructed run exercises two attempts under one call. Attempt 1 returns an empty body and reports no usage; attempt 2 returns text with 264 input and 41 output tokens. The caller also asks for a reasoning_effort setting the fake provider does not support.

call_id: call-001      task_id: task-001      chamber: deep-review
requested_parameters:  {temperature: 0.2, reasoning_effort: low, max_tokens: 256}
effective_parameters:  {temperature: 0.2, max_tokens: 256, model: mimo-v2.5}

attempt 1  transient_failure  error_kind=empty_output  usage=unavailable    raw=e4a89f4a…
attempt 2  succeeded                                   usage=264/41 measured raw=52bdc0c1…

call_status: succeeded      total_input_tokens: unknown      total_cost: unknown

Same task, same call, two attempt identities, two preserved observations. The unsupported setting is visibly absent from the adapter’s recorded effective request, which is Chapter 12’s whole subject arriving early. The fake adapter itself still received the logical CallSpec; this record is not yet proof of an exact wire body. One detail deserves a flag. The fake adapter reported attempt 1 as succeeded with no text; the runtime reclassified it as a transient failure and retried. That reclassification is a policy decision, and we will return to it.

Four live calls, and what the record caught

Then the same path ran against the real gateway: OpenCode, model mimo-v2.5, through CodeAI’s recorded runtime rather than an ad-hoc request. The ledgers from those runs were written to temporary directories. They have since been exported, scanned for credentials, and preserved with this book under experiments/applied-ai/evidence/ch11-live-opencode/.

It is tempting to tell this as “four attempts: three failures, then success.” That would break the definition this chapter just gave. These were four separate recorded calls in four separate workspaces, each with exactly one attempt, with CodeAI’s code changed between them. They are experiments, not retries.

Run 4 recorded everything this chapter set out to capture: gateway opencode, protocol chat_completions, model mimo-v2.5, revision unknown, provider request id gen-1789285773-…, 264 input and 256 output tokens measured, latency 21.8 seconds, cost unknown, and the raw observation stored under sha256:1a459bea….

The flattering reading — that recording execution can turn a failure into inspectable evidence — holds for these runs, because each preserved response pointed directly at the next fix. Do not generalize it into “every failure becomes diagnosable”: the crash window above deliberately preserves cases whose provider outcome remains unknown.

The more useful reading is less comfortable. The record’s observations were accurate. Several of its interpretations were not.

• Run 1 was labeled an authentication failure. The response body said it was an edge rejection; the credential was never checked. The classifier maps 401 and 403 to “authentication” from the status code alone.
• Run 3 was labeled a provider error. A 400 says the request was at fault — ours.
• Run 4 was labeled a success. Open its preserved observation and it reports "finish_reason": "length". The model used all 256 output tokens it was allowed, and the review it returned ends mid-sentence:

This statement contains an **unsupported factual claim** because it makes an
absolute guarantee ("every request within one second") without providing
evidence, context, or qualifications.

**Why it's unsupported:**
- **Absolute language** ("every

At that stage, CodeAI read no finish or stop reason anywhere, so a truncated answer and a complete one were recorded identically.

Every one of those errors was found in the same place: the preserved raw observation, read after the fact. Had CodeAI stored only its interpretation — authentication_error, provider_error, succeeded — the misreadings would be permanent and invisible. Keeping the observation separate made those conclusions challengeable without rewriting the evidence. At this stage, however, normalizer_version is provenance for normalization; it does not yet make the interpretation itself a first-class, replaceable, versioned record. That is the problem Chapter 13 exists to solve.

The same observation holds one more fact the current interpretation ignores: 192 of the 264 input tokens were served from cache. Whether that changes the cost depends on the gateway’s semantics — again, Chapter 13.

What this chapter deliberately leaves out

The recorded call is the substrate. The chapters that follow each take one part of it seriously:

• Chapter 12 — why requested and effective parameters differ, and how one operation survives several wire dialects without leaking them into the runtime. The live runs already showed that the gateway and model can be right while the protocol is wrong.
• Chapter 13 — who decides what usage, cost and status mean, and how an interpretation is versioned and replaced without touching the observation.
• Chapter 14 — who owns the work once a call succeeds, since success here means only that a cognition step produced output.
• Chapter 22 — retries as policy: when repeating a request is safe, and what happens to side effects.

Where it is still weak

The implementation earns the claims made above and no more. These are the gaps an honest reader of the code finds, in rough order of consequence:

1. Completion is not recorded. No finish or stop reason is read, so truncated output is indistinguishable from complete output.
2. Classification is unversioned interpretation. Status codes and message fragments become error kinds with no version attached, and two of the four live runs were misclassified. Treating an empty “success” as a retryable failure is also a policy, and some chambers legitimately return nothing.
3. Not every call path is recorded. The sealed-fanout path, which carries experiment and arm identities, still uses the older call method and produces no attempt records — so Chapter 10’s per-chamber release protocol cannot yet run on it.
4. Effective-parameter sanitizing is a denylist. It drops any key with a credential-like word as one of its parts. Tested directly, it removes max_token and token_limit, and it would remove a real control such as prompt_cache_key — so the record of what was sent can omit something that was sent. An allowlist of known controls, plus a recorded list of what was redacted, would fail safe in both directions.
5. Redaction happens before preservation. “Raw” means sanitized decoded JSON, not bytes and headers. A sanitizer bug destroys evidence permanently, and header-only fields are never kept.
6. Identity is only unique within a ledger. All four live runs used the hand-assigned call id call-live-1. Merge their ledgers and the identities collide.
7. Ambiguity is recorded but not reported. An orphaned attempt.started is inspectable, yet nothing lists orphans for a person to resolve.
8. Cost cannot represent a subscription. The price table models per-token rates; the gateway the book uses is priced by quota, so every real call so far records cost as unknown.

These gaps are stated as they stood at this stage, and several close later in the book, each with its own evidence. Chapter 12 reads finish and stop reasons and preserves the exact response bytes of gateway calls rather than sanitized JSON. Its versioned interpretation, extended in Chapter 17, can recognize the Cloudflare block that Run 1 misread. Chapter 16 projects an orphaned attempt.started as an effect of unknown outcome instead of leaving it for someone to stumble on. By Chapter 23 the fan-out path runs through the recorded call. The credential denylist and subscription-shaped cost remain as described.

These gaps do not erase the response evidence that was actually preserved, but they narrow what observed means at this stage. The effective-parameter record can omit controls that were sent, the “raw” artifact is sanitized decoded JSON rather than exact response bytes and headers, and some call paths produce no attempt record at all. Conclusions should therefore be held to the evidence that is present, not to transport details the record never captured.

Do this now

Forty-five minutes, with a coding assistant. Make one call a recorded event.

Give your assistant this prompt. The full version, with acceptance evidence, is prompt 2 in docs/applied-ai/reader-prompts.md. Two lines ask for things CodeAI itself does not yet do — recording truncation and an allowlist — because you should build the corrected version.

In this repository, make one model call a recorded process event.
Inspect first: trace the current call path and report what is recorded today.
Then implement, reusing existing types where possible:
- separate task_id, call_id and attempt_id; a retry is a new attempt, never a new task;
- write a manifest (chamber, requested and resolved model, gateway, protocol,
  pricing version, prompt hash, requested vs effective parameters) BEFORE the request;
- record each attempt (timing, provider request id, status, usage with a source
  label, raw observation stored content-addressed BEFORE parsing);
- keep unknown as unknown: no zero-filled usage, no invented revisions, no zero cost;
- read the provider's finish or stop reason and record truncation;
- never persist credentials; capture effective parameters with an allowlist.
Prove it offline with a fake adapter: one call with two attempts, reopened in a
second process. Do not add routing, evaluation or retry policy.

Then answer these from the running system, not from the assistant’s report:

1. Kill the process between the request and its completion. Can you find the orphaned attempt?
2. Pick one recorded status. Can you open the raw observation behind it and show whether the status is right?
3. Is there any path in your code that still calls a model without producing an attempt record?

Failure modes

• Returning a string and calling it a call. The smallest working call answers none of the questions measurement needs.
• Collapsing task, call and attempt. Retries vanish into single results and costs are understated silently.
• Recording only after success. A crash mid-call leaves no trace instead of a visible ambiguity.
• Zero-filling unknowns. Usage can be unavailable on a failed attempt even when the provider may have done work, so zero can understate cost.
• Inventing a revision. A timestamp is not a model version.
• Importing a gateway’s number without its meaning. The same "cost": "0" came back for free models and subscription models alike; the field alone does not say which.
• Storing only the interpretation. Misclassifications and truncations become permanent and undiscoverable.
• Trusting the classifier. “succeeded” meant “produced text”, including text cut off mid-sentence.
• Calling separate experiments attempts. It flatters the story and breaks the definition the accounting depends on.
• Leaving one call path unrecorded. The unrecorded path is the one your experiments will quietly use.

What this chapter established

• A string is the smallest working model call; the smallest useful one is a recorded process event, because Part 1’s demands — reproducible provenance, preserved evidence, an honest invoice, paired measurement, comparable occupants — are bookkeeping around the call.
• task ≠ call ≠ attempt. A retry is another attempt beneath the same call. A single-result client shows the defect this prevents: two provider requests, one result, one usage figure.
• Intent before effect. A manifest records what was resolved and intended before the request; attempts preserve execution evidence. Status and error classifications are interpretations of that evidence, not the evidence itself.
• A remote call that fails may have run once or not at all. Writing the manifest and attempt.started first turns that into a visible ambiguity rather than a silent loss — clean-restart durability, not exactly-once execution.
• unknown ≠ absent ≠ zero ≠ inferred. Usage can be unavailable on failure paths where the provider may still have done work, so zero-filling can understate cost. A gateway’s "cost": "0", returned across free and subscription contexts, is not a price until its meaning is known.
• Executed offline: one call recorded and reopened across three processes with identities intact, call_status: succeeded kept separate from an incomplete task, and a two-attempt call in which unknown usage propagates to an unknown total.
• Recorded live, and preserved: four separate OpenCode calls, three recorded failures followed by one recorded success whose preserved payload showed truncation. The observations exposed a Cloudflare block labeled authentication, a client-side 400 blamed on the provider, and a length-truncated answer labeled success.
• That substrate is real and still weak in named places: completion state, versioned classification, the unrecorded fanout path, a denylist sanitizer, redaction before preservation, ledger-local identity, unreported orphans, and cost for subscription gateways.

Next

Look again at the second and third live runs. The gateway, the model, and the credential were all right. What was wrong was the dialect — the shape of the request and response for that particular model — and then a header that dialect required.

The recorded call made that visible. It did not make it go away. The same logical operation, “ask deep-review to review paragraph P”, has to be expressed differently depending on which route the chamber’s occupant is served on, and the next occupant may be served on another. The runtime should not have to know.

Continue with One Operation, Several Model APIs.

References

• Andrew D. Birrell and Bruce Jay Nelson. Implementing Remote Procedure Calls. ACM Transactions on Computer Systems, vol. 2, no. 1 (1984), pp. 39–59. https://doi.org/10.1145/2080.357392
• Donald B. Rubin. Inference and Missing Data. Biometrika, vol. 63, no. 3 (1976), pp. 581–592. https://doi.org/10.1093/biomet/63.3.581
• Luc Moreau and Paolo Missier (eds.). PROV-DM: The PROV Data Model. W3C Recommendation, 30 April 2013. https://www.w3.org/TR/2013/REC-prov-dm-20130430/
• OpenCode. OpenCode Go — model endpoints and pricing. Accessed 2026-09-13. https://opencode.ai/v2/docs/console/go

Implementation and evidence sources: the stage described in this chapter is preserved in experiments/applied-ai/evidence/ch11-live-opencode/ and reviewed in docs/applied-ai/reviews/ch11-technical-direction-review.md. Current CodeAI successor symbols include src/codeai/domain.py (CallManifest, AttemptRecord, AttemptInterpretation, RecordedCall, Usage, UsageSource, CostSource), src/codeai/runtime.py (invoke_recorded_call, _store_transport_observation, _aggregate_totals, interpret_attempt_as), src/codeai/interpretation.py (interpret_attempt, INTERPRETER_V1, INTERPRETER_V2), src/codeai/adapters.py (TransportObservation, sanitize_effective_params, CognitionAdapter, CallResult), and src/codeai/providers.py (OpenCodeCognitionAdapter, OPENCODE_ENDPOINTS, _classify_opencode_error). The later implementation deliberately closes several gaps reported above; the preserved evidence bundle is the source for the historical live behavior described here.