Reading path / Evidence & economics

Measure the work that matters

A leaderboard is not your delivery pipeline. Measure accepted changes, review time and failures under a stated task distribution.

A suggested sequence

01
What METR's 2025 developer study can tell us ↗
A measured slowdown in one setting is a warning about assumptions, not a universal speed estimate.
02
Read a SWE-bench score as a test result ↗
A leaderboard number needs its task set, harness, and failure audit beside it.
03
Evaluate an agent on your repository ↗
A useful trial measures accepted work, failure recognition, and reviewer burden on tasks your team actually owns.
04
Count the review work ↗
Generated output is only productive when the team can accept and maintain it.

A measured slowdown in one setting is a warning about assumptions, not a universal speed estimate.

A leaderboard number needs its task set, harness, and failure audit beside it.

A useful trial measures accepted work, failure recognition, and reviewer burden on tasks your team actually owns.

Generated output is only productive when the team can accept and maintain it.

Replace the label with a definition, a task distribution, and reproducible evidence.