DiligenceBench · Economics
Price vs
performance
How agent quality changes with harness capability and estimated inference cost.
01
Complete traces
Cost per task
Estimated mean inference cost per task at published serving rates. Paths follow models from H1 to H3; Grok 4.6 and Claude Opus 5 are H3-only runs.
H1 · LoopH2 · SandboxH3 · Finance
GLM 5.2GLM 5.1MiniMax M3Kimi K2.6InklingGrok 4.6 · H3Claude Opus 5 · H3
Grok 4.6 is estimated at $0.84/task and Claude Opus 5 at $1.02/task. For each of 150 trajectories, the estimate reconstructs every model turn, cumulative context, newly added context, output, and prefix-cache reuse; token counts are approximated from serialized trace text. Published rates are then applied: Grok $2/$0.50/$6 and Opus $5/$0.50/$25 per million uncached input/cached input/output tokens. Treat these as directional rather than billing-grade figures.