The bounded agent contract
A coding agent needs an assignment with edges, evidence, and a stopping point.
2 min read · Review draft
Journal / Reference edition 01
Less folklore.
More fieldwork.
32 guides and evidence notes on the work around AI-generated code: contracts, state, tests, permissions, and the cost of being wrong.
Edition assembled 19 September 2026; individual preparation dates are shown on each article. Historical source dates are not this site’s publication dates. These are review drafts, not reports of tests we ran.
Start here / Evidence with boundaries
What METR's 2025 developer study can tell us
A measured slowdown in one setting is a warning about assumptions, not a universal speed estimate.
Read the evidence note ↗32 articles
Where a vibe prototype stops
A visible demo answers one question; a maintainable service answers many more.
2 min read · Review draft
Autonomy is a permission schedule
Grant the authority a task needs, then make escalation visible.
2 min read · Review draft
A graph is only as reliable as its state
Checkpointing helps a workflow resume; replay still needs explicit side-effect rules.
2 min read · Review draft
Approval does not make a retry safe
Human consent and duplicate prevention solve different parts of an agent workflow.
2 min read · Review draft
What METR's 2025 developer study can tell us
A measured slowdown in one setting is a warning about assumptions, not a universal speed estimate.
2 min read · Review draft
Read a SWE-bench score as a test result
A leaderboard number needs its task set, harness, and failure audit beside it.
2 min read · Review draft
Repository text is data until trusted otherwise
Prompt injection exploits the moment an agent treats encountered content as authority.
2 min read · Review draft
An MCP connection expands the job surface
Tool discovery, context retrieval, and account authority need separate review.
2 min read · Review draft
Local inference moves the control surface
Running a model nearby changes data flow and operations, but does not settle quality or security.
2 min read · Review draft
Count the review work
Generated output is only productive when the team can accept and maintain it.
2 min read · Review draft
How to evaluate an AGI coding claim
Replace the label with a definition, a task distribution, and reproducible evidence.
2 min read · Review draft
A regression test must prove the bug can return
The useful test is not the one that turns green; it is the one that distinguishes the broken behavior from the repaired behavior.
3 min read · Review draft
Freeze the behavior before moving the machinery
Characterization tests give a legacy refactor a visible boundary without pretending every old behavior is desirable.
3 min read · Review draft
Prove the dependency tree you actually changed
An upgrade is a supply-chain change, not a version-number edit.
3 min read · Review draft
Design the API failure before the retry
Clients recover reliably only when errors, side effects, and retry conditions are part of the contract.
3 min read · Review draft
A migration plan is incomplete until restore works
Dry runs reveal lock time and data assumptions; restore rehearsals reveal whether the safety net is real.
3 min read · Review draft
Accessibility evidence starts where the screenshot stops
A page can look correct and still be unusable by keyboard, screen reader, zoom, or error recovery.
3 min read · Review draft
Make the agent measure the same workload twice
A performance patch earns confidence through a stable workload, repeated observations, and an unchanged correctness contract.
3 min read · Review draft
Review generated tests as claims about the product
A large test diff is valuable only when its assertions can reject plausible wrong behavior.
3 min read · Review draft
Ship an artifact you can identify and undo
A release is safer when the reviewed source, built bytes, deployed revision, and rollback target form one traceable chain.
3 min read · Review draft
Repository context needs an owner and an expiry path
Agent instructions remain useful when they point to executable truth, fit the directory they govern, and change with the code.
3 min read · Review draft
Choose the agent surface by feedback loop
Editor, terminal, and cloud agents expose different evidence, latency, and authority—not just different interfaces.
3 min read · Review draft
Choose a visual workflow or a code runtime
The deciding issue is where complexity must remain inspectable when the happy path breaks.
3 min read · Review draft
Evaluate an agent on your repository
A useful trial measures accepted work, failure recognition, and reviewer burden on tasks your team actually owns.
3 min read · Review draft
Budget context as evidence
More repository text is not automatically more understanding; retrieval needs selection, provenance, and tests.
3 min read · Review draft
A sandbox needs network and secret boundaries
Filesystem isolation is only one edge of an agent execution environment.
3 min read · Review draft
Build traces without secret leaks
An agent trace should explain decisions and effects without becoming a second copy of every sensitive input.
3 min read · Review draft
Multi-agent systems need integration contracts
Role names do not integrate agents; typed artifacts, ownership, versions, and failure semantics do.
3 min read · Review draft
Route models by task risk
Model routing should spend capability where errors are costly and require evidence before escalating or downgrading.
3 min read · Review draft
Write the builder exit plan before launch
Source export is necessary, but a real exit also covers data, identity, configuration, domains, and operational knowledge.
3 min read · Review draft
Build a graph failure test matrix
Test every important node at the moments before, during, and after failure—not only the graph's happy path.
3 min read · Review draft
No articles match. Try another subject or reset the filters.