DSPy From First Principles on AIBussin — AI applications, systems and books

Why Are We Still Hand-Writing Prompts?

Fri, 28 Aug 2026 10:00:00 +0100

Almost every language-model application starts as one string.

That is not a mistake. A prompt is the fastest way to find out whether a model can do the job at all, and it lets you work directly on the behavior instead of building a framework around a problem you do not yet understand. Most good LM systems begin this way and should.

The trouble is that the prompt usually survives longer than its usefulness. It stops being a probe and becomes the specification, and by the time anyone notices, the string is four hundred words long, nobody remembers why the third paragraph is there, and changing it feels dangerous.

A Prompt Is Not Yet a Program

Fri, 28 Aug 2026 10:05:00 +0100

Chapter 1 ended with a signature and an uncomfortable amount of unfinished business. We had established that one prompt string was carrying the task semantics, the behavioral framing, the output schema, and the provider assumptions simultaneously, and that this is why three unrelated failures arrived looking identical.

The obvious response is to write a better string. That is the wrong move, and it is worth being precise about why.

A prompt is text:

Define the Contract

Fri, 28 Aug 2026 10:10:00 +0100

Chapter 2 built a boundary and then exposed a weakness in it on purpose. We designed the most natural output field available — rewritten_text, a replacement sentence — but failed to represent “no edit needed” as an explicit decision.

The model can technically return the original sentence unchanged. What the contract cannot tell us is whether that identity output means deliberate restraint or an attempted rewrite that happened to reproduce the input. That distinction matters on six of the forty-four later fixture cases — about 13.6% of the corpus. Having a signature did not prevent the mistake.

Separate What From How

Fri, 28 Aug 2026 10:15:00 +0100

Chapters 2 and 3 built a contract and said nothing about how a model should satisfy it. That was deliberate, and this chapter collects the payment.

Three objects must remain distinct throughout the comparison:

task contract what valid behavior must preserve
execution strategy how the model attempts the task
evaluation how observed behavior is judged

The signature holds the first fixed. Predict and ChainOfThought vary the second. The frozen fixture and metrics supply the third. If two of those change together, the result no longer tells us what the reasoning policy caused.

Build Programs From Programs

Fri, 28 Aug 2026 10:20:00 +0100

Chapter 4 held the contract fixed and changed the execution policy. This chapter changes the architecture.

Real applications rarely consist of one call. An editorial system might want to work out what is wrong with a sentence, produce a candidate, assess whether that candidate is risky, and hand the whole package to something that stores it.

 flowchart LR
 I[input] --> A[analyze]
 A --> W[rewrite]
 W --> S[assess]
 S --> R[return]

That is the pipeline as pitched: four stages in a line, each feeding the next. Decomposition is one of the few architectural moves that is widely treated as obviously correct. Smaller pieces, clearer boundaries, better debugging — the diagram sells itself.

Your Model Is a Dependency

Fri, 28 Aug 2026 10:25:00 +0100

Chapter 5 built a composed program and measured which of its stages earned their cost. Every number in that chapter, and in chapter 4 before it, has an unstated qualifier:

Which model executed the program?

That is not operational trivia. A language-model program is not fully described by its signatures and modules. Its behavior depends on what sits behind the LM boundary and how that thing is configured.

The model is therefore both a dependency and an experimental variable. Holding it fixed is often as important as swapping it:

Examples Are Experimental Data

Fri, 28 Aug 2026 10:30:00 +0100

Chapter 6 gave the program a dependency boundary and, more importantly, evidence that identical recorded program state can still produce different outputs. We can now say what the program is, what ran it, and why a small score delta cannot be interpreted without repeated measurement.

What we still cannot do is say whether the program is any good, because we have nothing to measure it against.

Examples in DSPy are not decoration around the program. They are experimental data with several possible roles. Optimization consumes that evidence: training data may shape candidate state, development data may select among candidates, and holdout data must remain outside both processes if it is to support an independent claim.

You Cannot Optimize What You Cannot Measure

Fri, 28 Aug 2026 10:35:00 +0100

Chapter 7 produced material. This chapter produces evidence.

That makes optimization possible. It does not yet make optimization safe.

no metric
 → no systematic optimization

metric
 → behavior can be optimized

wrong metric
 → the wrong behavior can be optimized systematically

This chapter establishes the surface on which search can operate. Chapter 9 asks what that surface rewards when the metric becomes the target.

 flowchart TD
 P[frozen program] --> E[evaluate]
 C[frozen cases] --> E
 M[metric] --> E
 E --> PC[per-case results]
 PC --> AG[aggregate result]
 PC --> FI[failure inspection]

No optimization happens here. Note that the per-case results feed two readings — the aggregate and the failure inspection — and this chapter’s finding comes from the second. The question is narrow:

When the Metric Becomes the Target

Fri, 28 Aug 2026 10:40:00 +0100

Chapter 8 built a metric and found a defect in it by reading a table. Six cases where the correct answer is capped at 0.65, discovered by grouping per-case scores and noticing that a whole category sat on an exact number.

That was luck dressed as diligence. This chapter looks on purpose.

The urgency comes from what happens next. A DSPy optimizer does not want your program to be good. It wants your number to go up, and it will find every route to that outcome, including the ones you did not intend to leave open.

Compile the Program

Fri, 28 Aug 2026 10:45:00 +0100

Chapter 9 gave the optimizer an explicit objective and showed a concrete limitation: v1 can assign very high scores to candidates that reverse important parts of the source meaning.

We are going to compile against that frozen metric anyway. Not because the limitation is acceptable, but because changing the objective now would change the experiment. v2 remains outside the main search loop as an independent semantic check.

Compilation in DSPy takes a program, training evidence, a metric, and an optimizer, and returns a candidate program state.

Few-Shot Optimization

Fri, 28 Aug 2026 10:50:00 +0100

Chapter 10 produced a candidate and refused to say whether it was any good. This chapter says.

The mechanism is few-shot optimization, and the question is practical:

Can we improve the program by selecting better demonstrations, rather than rewriting the program ourselves?

The answer, on our fixture, is the most interesting result in this book. In one run, the candidate appears to improve the metric we optimized against while damaging the metric we did not. Under seven paired fresh-process sessions, the apparent v1 gain collapses to essentially zero while the v2 loss remains large and systematic.

Optimize the Instructions

Fri, 28 Aug 2026 10:55:00 +0100

Chapter 11’s optimizer could only select demonstrations. It never touched an instruction, and it never looked at the development set — which is why upgrading its metric changed nothing.

MIPROv2 does both.

instruction candidates
+
demonstration candidates
+
metric
+
development data
 ↓
candidate program

This is a strictly larger search over a strictly larger space, evaluated against evidence the previous optimizer could not see. If chapter 11’s result was an artifact of a weak mechanism, this is where it gets corrected.

Let the Program Reflect

Fri, 28 Aug 2026 11:00:00 +0100

Chapter 12 ended on a specific limitation. MIPROv2 selected a candidate that deleted regularly from a regulated claim, and nothing in the pipeline could have told it otherwise, because the only signal it received was 0.967.

candidate
 ↓
metric
 ↓
0.967

A number can rank candidates. It cannot say which word was the problem.

The next mechanism replaces the scalar with language:

 flowchart LR
 P[prediction] --> M[metric]
 M --> SF[score + diagnostic feedback]
 SF --> RF[reflection]
 RF --> IM[instruction mutation]
 IM --> CA[candidate]
 CA -->|re-run| P

GEPA is DSPy’s evolutionary, reflection-driven optimizer. It reads per-predictor feedback, uses a separate reflection model to propose instruction edits, maintains a population of candidate programs, and returns the best one it found. The engineering idea underneath is smaller than the API:

Change One Thing

Fri, 28 Aug 2026 11:05:00 +0100

The previous three chapters ran three optimizers against a metric we had already proved was blind, and got what chapter 9 predicted: a search that pursued the objective faithfully and damaged the thing the objective could not see.

That result has an obvious interpretation and a correct one, and they are not the same.

The obvious interpretation is that the optimizers underperformed. The correct one requires an experiment, because “the objective was the problem” is a hypothesis and we had not tested it. Everything we knew was consistent with a second explanation — that DSPy’s optimizers do not do much on a small local model regardless of what you point them at.

Agents Are Programs Too

Fri, 28 Aug 2026 11:05:00 +0100

Chapter 14 held the corpus, program, model, optimizer configurations and paired harness fixed, changed only the objective, and closed the editorial task. The prediction it left behind was that a reflective optimizer improves in proportion to how much its feedback can say — and that a task whose failures carry a test name, an expected value and a traceback should therefore be the book’s best case.

To measure that we need a task with those properties. Repository repair has them. But it introduces a limitation the editorial task never had.

Search, Memory and Long Context

Fri, 28 Aug 2026 11:10:00 +0100

Chapter 15 measured an agent that had every advantage. Two files, one defect, a bounded tool surface, a budget it used well. It found both implicated files, held both halves of the bug in context, and still stopped one step short. That was a reasoning failure, and Chapter 17 exists to attack it.

This chapter is about the problem that sits before the reasoning problem. The fixture had two files. A real repository has thousands, and a bounded tool surface does not tell the program which of them to read. Something has to decide what evidence the program even looks at, and that decision is not one mechanism. It is at least five, and they are routinely collapsed into a single word.

Search the Reasoning Space

Fri, 28 Aug 2026 11:25:00 +0100

One Path Is a Decision

In Chapter 15, we built an agent. It acquired repository evidence safely. It inspected the files it was permitted to inspect. It produced a diagnosis.

It was wrong.

Not completely wrong—the agent scored 0.60. It found the relevant code. It identified that service.py references item.cost. It located the model definition. And then it concluded that Item has no cost attribute, missing the final connection: service.py should use item.price.

Don't Let the Optimizer Cheat

Fri, 28 Aug 2026 11:15:00 +0100

Opening — The Optimizer Doesn’t Have to Cheat Intentionally

In Chapters 15 and 16 we expanded the program’s access. The agent could search repositories, retrieve memories, and adaptively select evidence. Chapter 17 then added search over reasoning states, showing that tree search can allocate inference-time compute toward promising paths.

Now the question becomes urgent: once a program can search, remember, and adaptively select both evidence and reasoning, how do we prove that it did not obtain information it was never supposed to see?

From Experiment to Production

Fri, 28 Aug 2026 11:20:00 +0100

Chapter 18 ended with an information boundary we could finally audit: the firewall blocked all 288 constructed attacks in its deterministic train/dev suite. That does not make the downstream metric correct; it means the tested result was not produced by the forbidden evidence paths we attacked. Chapter 18 also said the obvious next thing — even an admissible offline result is not a production version.

This chapter closes two gaps between those two facts.

Build a Self-Improving Engineering Program

Fri, 28 Aug 2026 11:25:00 +0100

We can now assemble the book.

Every prior chapter added one mechanism because the system before it had exposed a specific limitation: a contract (Chapter 3), a measured baseline (Chapter 8), an attacked metric (Chapter 9), three optimizers and an objective swap (Chapters 11–14), tools and bounded environment access (Chapter 15), evidence selection (Chapter 16), reasoning-path search (Chapter 17), an experimental firewall (Chapter 18), and a promotion boundary with rollback (Chapter 19). This chapter does not introduce another mechanism. It asks whether the ones we have compose into a single loop without any stage quietly promoting its own output.

Beyond the Book: Production DSPy Systems (Appendix)

Thu, 03 Sep 2026 11:00:00 +0100

Chapter 20 assembled the book’s mechanisms into one loop. This appendix does something smaller and more specific: it shows three of those mechanisms already running in a production codebase that was not written for this book, and it holds itself to a stricter evidence rule than the chapters do.

Evidence rule for this appendix. There are two tiers here. The mechanics around each language-model call — UCT selection, the budget gate, the trace cache, the champion-update rule, the output parser — are deterministic software, and a committed fixture (experiments/dspy-from-first-principles/ch21_beyond_book/) exercises them with no LM and no DSPy import. Those results are measured. The kernels themselves — the production modules the excerpts are drawn from — are selection evidence from one codebase, not a generalisation. Their full source lives in the Stephanie repository, linked below and pinned to main at revision 32d02ee (2026-01-06).