Make the agent measure the same workload twice

Write the workload before tuning

Name the user action or system operation, input size and distribution, concurrency, cache state, machine or runtime, and metric that matters. A vague request to improve performance invites changes optimized for whatever microbenchmark is easiest to run. Choose a budget connected to user or service needs: tail response time, throughput under a stated load, memory at steady state, startup time, or an interaction metric.

Lock correctness beside speed. The same outputs, authorization checks, validation, durability, and logging requirements must hold unless an explicit tradeoff is approved. Removing work can make a benchmark faster while making the product wrong.

Instrument the relevant interval

Use monotonic, high-resolution timing and mark a boundary that corresponds to the operation under study. The W3C User Timing specification defines marks and measures for application timing; comparable server runtimes expose related facilities. Avoid including one-time setup in one sample but not the other. Decide whether cold starts, compilation, network setup, or cache misses are part of the real workload and measure them consistently.

Have the agent preserve raw observations, not only an average. Run enough repetitions to reveal spread, alternate baseline and candidate runs when environmental drift is plausible, and inspect outliers instead of deleting them automatically. This article does not prescribe a universal sample count because noise and cost vary by system.

W3C User Timing ↗

Separate lab diagnosis from field outcome

For web performance, controlled lab data and real-user field data answer different questions. Google's Web Vitals guidance recommends both: lab tools support pre-release diagnosis under fixed conditions, while field data captures actual devices, networks, content, and behavior. A laboratory improvement may not move the field distribution, and a field regression may be invisible on a fast developer machine.

For a hypothetical search page, the agent might profile server query time with a fixed dataset, run a browser scenario under specified throttling, and propose a release marker for field monitoring. The claim after local work would be limited to the controlled workload. Only later field data could support a user-population result.

Measuring Web Vitals ↗

Change one cause and report uncertainty

Start from a profile or trace, state a hypothesis, and change one meaningful variable. Re-run the same correctness and measurement harness. Report baseline and candidate distributions, environment, commit identifiers, tool versions, warm-up policy, failures, and any tradeoffs in memory, cost, or maintainability. A single best run is not evidence of a stable improvement.

Agents are useful for repeatable orchestration and diffing measurements, but they can also overfit the harness or narrate noise as causality. A reviewer should inspect instrumentation placement and confirm the changed code explains the observed effect. Keep the benchmark with the repository when affordable, and state why it may not predict production. Track memory, errors, and resource saturation beside latency when the optimization may shift cost elsewhere. If the gain disappears across repeated alternating runs, report an inconclusive result instead of selecting a favorable sample.