Same Weights, Different Scores: Agent Benchmarks Measure Configurations, Not Model Names
An agent benchmark score reflects the model together with its tool access, reasoning budget, output cap, and serving endpoint, and those settings can move results and cost more than switching models, so the full configuration is what you need to pin and verify.
By Daniel Huber · 6 min read
An agent benchmark score measures a configuration. That configuration is a model plus the tools it was allowed to call, the reasoning and output budget it was given, the code that drove it, the endpoint that served it, and the verifier that graded it. Results published between November 3 and November 14, 2026 show those surrounding settings moving scores and costs by as much as the model choice itself, and in some cases by more. A model name is therefore too underspecified to select, evaluate, or deploy against. The thing to record and check is the full configuration.
Artificial Analysis re-ran the VIALS-AA scientific image benchmark in two formats: single-shot with no tools, and agentic with code execution. Tool use nearly doubled average scores, from about 25% to about 47%. The comparison across tiers is the striking part. Claude Haiku 5.5 with tools scored higher than Claude Fable 5.2 without tools, at roughly one-tenth of the cost.