Loops, Thrashing, and Retry Storms

Part VII — Debugging Agents

The agent worked for 90 steps — and the state never moved

Chapter 38 classifies single divergences. Now the practitioner watches a research agent burn 90 steps and a full API budget: search_papers → fetch_pdf → parse_failed → search_papers (same query) → fetch_pdf (same URL) → parse_failed … The contracted trace is complete, every step classifies somewhere, and the run still fails — because the defect is not one wrong step but zero net movement across thirty of them.

OBSERVATION: steps 12–41 contain 10 repetitions of the identical arg hash for search_papers(query="refund policy 2024") with identical observation hashes; working-state hash unchanged across all 10 cycles; step count 90, state-novelty count 3. HYPOTHESIS H1 (exact loop): equivalent states revisited with no new information — same args, same observations, no delta. H2 (thrashing): actions differ cosmetically (reworded queries, alternate tools) while state-equivalence holds — motion without movement. H3 (retry storm): error-triggered re-issue with backoff absent — failure count climbs, information count flat. INFERENCE: none yet — H1/H2/H3 predict different repetition signatures and separate only by progress metrics, never by step counts.

This chapter’s question: how do you detect non-progress mechanically — and which guard stops each loop shape without halting legitimate long runs?

Why “let it run longer” fails first

The obvious move — raising the step budget — fails because budgets bound cost, not progress. Four defects hide behind effort-tolerance:

1. Step-count worship. “90 steps, it tried hard.” Effort is not evidence; state novelty is. Ten equivalent cycles are one finding repeated ten times.
2. Cosmetic novelty. Reworded queries (“refund policy 2024” → “2024 refund policy rules”) look like adaptation in prose and hash to state-equivalence in the trace. Paraphrase is motion; hashes measure movement.
3. Error-concealing retries. Each parse_failed triggers a fresh search instead of branching on the parse error — the error observation is consumed as a trigger, never as information.
4. Single-loop anecdote. One halted loop “proving” the guard works. Nondeterministic agents need repeated loop-repro trials with the guard on and off.

OPINION: an agent without a progress metric is a treadmill with an odometer — impressive numbers, zero distance. Measure novelty or admit motion-blindness.

The mental model: progress is state novelty per step — new hashes, new consumed information, shrinking hypothesis sets — and loops are its absence sustained across a window. Exact loops repeat arg/observation hashes; thrashing varies surface form under state-equivalence; retry storms repeat error-triggered actions without backoff or branch. Three shapes, three detectors, three guards.

The method: progress metrics, repetition detection, budget guards

Compute all three metrics over a sliding window (window size pre-registered, e.g., 10 steps) on the contracted trace, then arm the matching guard:

1. Progress metric. Per window: distinct state hashes / steps; distinct consumed-observation IDs / steps; error-repeat rate. A progressing window adds states or consumes new information; a flat window (novelty ≈ 0 across ≥2 windows) is non-progress regardless of prose variety.
2. Repetition detection. H1: identical arg-hash + observation-hash cycles (≥2 equivalent cycles → loop). H2: distinct arg hashes mapping to equivalent state hashes and equivalent outcome classes (surface variation, outcome equivalence → thrash). H3: same error code followed by same action class with no backoff growth and no branch (→ storm).
3. Budget guards. Exact-loop guard: tabu on equivalent arg hashes within N steps (force branch or halt) — this is a tabu list / closed set, the classic cycle-detection device from state-space search (Glover, 1990). Thrash guard: cap on state-equivalent windows (halt with UNKNOWN, never with a guessed answer). Storm guard: exponential backoff with jitter, a retry budget, and an error-branch requirement (third identical error without a new branch → halt) — backoff-and-jitter from Brooker, 2015, and the circuit breaker that stops retrying a dependency that is clearly down from Nygard’s Release It!, both transplanted to a step budget.

    flowchart TD
    W["slide a pre-registered window over the contracted trace; compute novelty from hashes"] --> N{"state novelty ~ 0 across >=2 windows?"}
    N -->|no| PR["progressing — no guard needed"]
    N -->|yes| SH{"repetition signature?"}
    SH -->|"identical arg + observation hashes"| H1["H1 exact loop -> tabu equivalent args, force branch or halt"]
    SH -->|"distinct args, fixed state hash, identical outcomes"| H2["H2 thrash -> cap state-equivalent windows, halt UNKNOWN"]
    SH -->|"same error code, same action class, flat backoff"| H3["H3 retry storm -> exponential backoff + jitter + error-branch requirement"]
    H1 --> G["arm ONE guard; compare loop span off vs on, >=3 trials each; halt verdict = UNKNOWN"]
    H2 --> G
    H3 --> G
    G --> R{"span shortens across trials?"}
    R -->|yes| EN["route to enforcement"]
    R -->|no| CR["guard ineffective — route to Ch41 causal replay, never to a bigger budget"]
  

PROGRESS WINDOWS (constructed; window=10, hashes verbatim):
steps 12-21 | states: h7a1 ×10 (novelty 0.0) | args: q-hash identical ×10
            -> H1 EXACT LOOP (2+ equivalent cycles, no new information)
steps 42-51 | states: h7a1 ×10 (novelty 0.0) | args: 6 distinct q-hashes
            -> H2 THRASH (surface varies, state fixed, outcomes identical)
steps 62-71 | errors: parse_failed ×8 | retry same action, backoff 0s ×8
            -> H3 RETRY STORM (error count climbs, information flat)
RULE: hashes and error codes detect; prose variety is inadmissible.

OBSERVATION (constructed illustration, not a measured run): windows 12–21, 42–51, and 62–71 each show zero state novelty with the three distinct repetition signatures above; step counts (30/30/30) are identical and therefore useless. UPDATED BELIEF: H1 supported for window 1, H2 for window 2, H3 for window 3, for this instance; guard efficacy unproven pending the Lab’s on/off trials.

Example: halting the search loop with a detector sketch

The practitioner does not lecture the prompt (“vary your approach!”). She computes novelty and arms one guard per shape:

# non-progress detection over contracted trace (no repair yet)
def window_novelty(steps):  # OBSERVATION: hashes, not impressions
    return len({s.state_hash for s in steps}) / len(steps)
def repetition_shape(window):
    if equivalent_cycles(window) >= 2: return "H1-exact-loop"
    if surface_varies(window) and state_fixed(window): return "H2-thrash"
    if error_triggered_repeats(window) and not backoff_grows(window):
        return "H3-retry-storm"
    return "progressing-or-UNKNOWN"
# Guards (harness-level, pre-registered): tabu equivalent args (H1),
# cap state-equivalent windows then halt-UNKNOWN (H2), backoff+branch
# requirement then halt (H3). Predictions pre-written per guard.

In the constructed case the H1 tabu fires at the third equivalent search_papers call: the harness blocks the repeat and forces a branch (new source) or halt. Across ≥3 fixed trials the loop span shortens from 30 steps to ≤3 in every trial with no new answer fabricated at halt (halt verdict: UNKNOWN, never a guessed synthesis). Guard effect measured on spans, not on vibes.

No confidence in any “I will try a new approach” narration, no judge score rating effort, no agreement across trials counted as progress, and no downstream symptom (“it eventually found something”) substitutes for novelty ratios. Windows decide; adjectives do not.

Research lineage: retry storms are old, and the agent-level fix needs a signal

H3 is a distributed-systems failure with a distributed-systems fix. Brooker’s canonical treatment shows how independent retries at each layer of a stack multiply — three retries at each of three layers is a 243x load amplification — and why naive backoff is not enough without jitter, a retry budget, and a circuit breaker that stops retrying a dependency that is clearly down (Brooker, 2015). An agent that calls a sub-agent that calls a tool is exactly this multi-layer stack; the storm guard is a circuit breaker with a step budget instead of a time budget.

The agent-level alternative to a harness guard is episodic memory of failures. Shinn and colleagues’ Reflexion has the agent write a natural-language self-critique after each failed attempt and prepend it to the next attempt’s context, which measurably reduces repeated mistakes — 91% versus 80% pass@1 on HumanEval (Shinn et al., 2023). But Reflexion needs a feedback signal to reflect on, and the loops in this chapter are often feedback-starved (Chapter 29): the agent re-runs the same failing parse because nothing tells it the parse failed in a way it can act on. When the signal exists, Reflexion-style memory is the right layer; when it does not, the harness guard is the only thing standing between the agent and its budget.

The guards are runtime; the wiring can be audited at design time. Everything above stops a loop already underway. Hou and colleagues formalize the Infinite Agentic Loop — repeated model calls, tools, or handoffs “when the feedback path is not effectively bounded” — and locate its root cause in the interaction of agent logic, framework semantics, runtime observations, and termination mechanisms rather than in a coding bug; their IAL-Scan tool statically maps an agent’s feedback paths and flags the ones that can loop without a bound to a costly or state-growing operation (Hou et al., 2026). A pre-deployment feedback-path audit and the runtime guards catch different instances of the same defect; a serious agent wants both.

Lab 39: guard on/off with pre-written span predictions (proposed)

PROPOSED, not executed: no author-measured results are reported. The evidence this chapter requires is the reader’s own windowed trace.

Setup. Take one looping run (or fault-inject a parse failure that invites repetition) under the Chapter 37 contract. Freeze task, tool revisions, model/seed. The guard state (off vs. one guard armed) is the independent variable; everything else is controlled. Pre-register window size and halt thresholds before any trial.

Task.

1. Before arming, write H1/H2/H3 with distinct predicted window signatures: H1: “identical arg+observation hashes, novelty ≈ 0”; H2: “distinct args, fixed state hash, identical outcomes”; H3: “same error code, same action class, flat backoff.”
2. Measure baseline loop span (steps from first equivalent cycle to budget death or halt) with guards off, ≥3 trials.
3. Arm exactly one guard; measure span again, ≥3 trials; record halt verdicts (UNKNOWN, never fabricated answers).

Success criterion. Windowed novelty table plus on/off span comparison with per-trial halt verdicts. A raised budget or a single halted anecdote is explicitly not completion.

Companion tool: Invariant Breakpoints

What it accepts: the contracted trace, pre-registered window size and thresholds (novelty floor, equivalent-cycle cap, backoff schedule), and the guard configuration. What it performs: it computes per-window novelty, flags H1/H2/H3 spans with the deciding hash/error evidence, enforces tabu/backoff/halt guards at the harness level, and records halt verdicts as UNKNOWN with the span evidence attached. What it can establish: whether a span is non-progress, which loop shape it matches, and whether the armed guard shortens the span across trials — for the examined configuration only. What it cannot establish: why the agent loops (prompt vs. tool vs. model needs Chapters 38/41 probes), optimal thresholds in general, or that halting equals fixing. It never treats step counts, confidence, agreement, single-trial halts, or downstream symptoms as progress evidence. How its output changes your next action: H1 routes to tabu/branch enforcement; H2 to window-cap halting plus task-reformulation review; H3 to backoff + error-branch policy; guard-ineffective (span unchanged across trials) routes to Chapter 41 causal replay, never to budget increases.

Paper form, sufficient for this chapter:

Windows (w=___): novelty ___ ___ ___ | shapes flagged: ___ (steps ___)
Guard: H1-tabu / H2-cap / H3-backoff (thresholds ___)  SPAN off→on: ___→___
TRIALS ×3/×3: ___ / ___  HALT VERDICTS: ___  NEXT: enforce / reformulate / replay

Where a software implementation does not yet exist in the reader’s stack, this record is the tool. Novelty before halting.

Reusable procedure: break every loop mechanically

1. Window — pre-register size and thresholds; compute novelty from hashes.
2. Shape — H1/H2/H3 by hash/error signatures, never by prose impression.
3. Arm one — single guard per trial series, predictions pre-written.
4. Compare spans — off vs. on, ≥3 trials each, halt verdicts UNKNOWN.
5. Route — effective guard to enforcement; ineffective to causal replay, never to bigger budgets.

Failure modes

• Budget therapy. Raising step limits for a novelty-flat run. Budgets bound cost; only guards bound loops.
• Paraphrase blindness. Reading reworded queries as adaptation. Surface varies, state fixed → thrash, not effort.
• Fatal halts. A guard fabricating a final answer at halt. Halt verdict is UNKNOWN with span evidence, never a guessed synthesis.
• Triple-guard confounding. Arming all guards at once. One guard per series or the effect is unattributable.
• Single-halt triumphalism. One stopped loop declaring victory. Three off / three on minimum.
• Effort scoring. Citing steps, confidence, or eventual downstream success as loop evidence. Novelty ratios decide.

Limits, per contract: one loop analysis covers one task/configuration revision; thresholds do not transfer across tasks without re-registration; halting is cost control, not diagnosis. UNKNOWN wherever state hashes or error codes are missing.

References

• Marc Brooker. Exponential Backoff and Jitter. AWS Architecture Blog, 2015. https://aws.amazon.com/blogs/architecture/exponential-backoff-and-jitter/
• Fred Glover. Tabu Search: A Tutorial. Interfaces 20(4), 1990, pp. 74–94. https://doi.org/10.1287/inte.20.4.74
• Noah Shinn, Federico Cassano, Edward Berman, Ashwin Gopinath, Karthik Narasimhan, and Shunyu Yao. Reflexion: Language Agents with Verbal Reinforcement Learning. Advances in Neural Information Processing Systems 36 (NeurIPS), 2023. https://arxiv.org/abs/2303.11366
• Michael T. Nygard. Release It! Design and Deploy Production-Ready Software (2nd ed., circuit-breaker and bulkhead patterns). Pragmatic Bookshelf, 2018.
• Xinyi Hou, Shenao Wang, Yanjie Zhao, and Haoyu Wang. When Agents Do Not Stop: Uncovering Infinite Agentic Loops in LLM Agents. arXiv:2607.01641, 2026. https://arxiv.org/abs/2607.01641

Debugging Checklist

• Contracted trace with state hashes and error codes per step?
• Window size and halt thresholds pre-registered before trials?
• Novelty computed from hashes (not prose variety)?
• H1/H2/H3 shapes assigned by signature with deciding evidence cited?
• Exactly one guard armed per trial series with pre-written span forecast?
• Off-vs-on span comparison with ≥3 trials each side?
• Halt verdicts recorded as UNKNOWN (no fabricated answers)?
• No step counts, confidence, agreement, single halts, or symptoms cited as verdict?

What This Chapter Established

• Loop/thrash/storm mechanics: novelty-based progress metrics, three repetition signatures, and three matching harness guards — demonstrated on the constructed 90-step search case, no measured runs claimed.
• The halt-as-UNKNOWN rule separating cost control from diagnosis.
• Lab 39 as a proposed guard on/off record the reader executes; the Invariant Breakpoints contract (accepts/performs/can-establish/cannot-establish/next-action).
• What was NOT proved: any cause of looping, any threshold generality, or any fix claim. Spans shortened (conditionally); nothing universal.
• Research grounding: the H1 guard is a tabu list / closed set (Glover; classic cycle detection); the H3 guard is a circuit breaker (Nygard, Release It!) with a step budget, and retry storms are a known multi-layer amplification failure with a backoff-and-jitter fix (Brooker — 243x with 3×3 retries); the agent-level alternative is Reflexion-style episodic memory of failures (Shinn et al.), which works only when a feedback signal exists — feedback-starved loops (Ch 29) still need the harness guard; and these runtime guards have a design-time complement — statically auditing the agent’s feedback paths for unbounded loops (“Infinite Agentic Loop” / IAL-Scan, Hou et al.).

Next

Loops are now haltable — but halting is not understanding. The practitioner still cannot answer the counterfactual: had the agent branched at step 12 instead of repeating, would the run have succeeded? Guards control cost; they do not test causes. Chapter 40, “Time Travel, Replay, and Forking,” builds the machinery that does: deterministic replay from any step with disciplined forks; what replay can and cannot prove is its chapter’s to establish, not this one’s.