Skip to content
AyoKoding

Artifact: A Correction Loop Halted at Its Token Budget

A multi-round correction loop bounded by a token budget, halting and flagging the unresolved item to a human once the ceiling is reached -- exercises co-18, co-23, co-24.

Task: fix a flaky test_retry_backoff test. Budget ceiling: 20,000 tokens.

RoundTokens this roundCumulativeFeedback / result
16,2006,200Logic looks right; test still flakes intermittently.
28,50014,700Flake rate reduced from ~30% to ~5% across 20 runs, not eliminated.
39,10023,800(budget ceiling of 20,000 crossed mid-round)
[BUDGET EXCEEDED] cumulative 23,800 > ceiling 20,000 -- session halts before
generating a fourth correction attempt.
 
[Escalation, as written down]: "Session halted at token budget.
test_retry_backoff still flakes at roughly 5% over 20 runs after two
correction rounds -- improved, not resolved. Flagging for human review
rather than continuing to iterate past budget; a human should decide
whether to raise the budget, accept the residual flake rate with a
documented waiver, or take a different approach (e.g. mocking the clock
instead of tuning backoff timing)."
 
[Human]: item marked "unresolved -- escalated." Not merged. Not silently
closed.

The loop did not quietly keep spending tokens once it crossed the ceiling, and it did not merge a still-flaky test just to close the loop out -- crossing the budget triggered an explicit halt and handed the unresolved gap to a human, with the current state (5% flake rate, two rounds spent) stated plainly rather than glossed over.

Verify: the cumulative total (23,800) is shown crossing the stated ceiling (20,000), the session halts at that point rather than attempting a fourth round, and the item is explicitly flagged as unresolved -- escalated rather than merged -- satisfying ex-53's rule that the session halts at the budget and flags the unresolved item for review.

Last updated July 17, 2026

Command Palette

Search for a command to run...