Agent Benchmarks
Original benchmarks built and run on this site — plus a curated set of external agent benchmarks, linked to their sources.
From this site
LLM Trading Decisions →
Which language models make the best crypto-futures trading calls, and under what market conditions? A deterministic backtest of agentic vs. single-call LLM traders across 20 historical scenarios, measured against buy-and-hold, technical-analysis, and random baselines. Every run is computed and aggregated here — leaderboard, per-regime performance, and per-model breakdowns.
External benchmarks
Third-party evaluations we follow. Scores and leaderboards live at each source — follow the links for current numbers.
Software Engineering
- SWE-bench Pro ↗
Uncontaminated bug-fix and feature tasks across Python, Go, TypeScript, and JavaScript in actively maintained repositories.
- SWE-bench Verified ↗
Human-verified subset of GitHub issues; agents generate patches that must pass the repository test suite. Widely cited, though facing data-contamination concerns.
- Terminal-Bench Hard ↗
Terminal-environment tasks — compiling code, training models, configuring servers, debugging — executed in Docker containers (Stanford / Laude Institute).
- LiveCodeBench ↗
Contamination-free coding benchmark that continuously harvests fresh competitive-programming problems from LeetCode, AtCoder, and CodeForces.
Web Navigation
- WebArena ↗
Realistic multi-step web tasks across five self-hosted sites (shopping, Reddit, GitLab, OpenStreetMap, Wikipedia).
General Assistant
- GAIA ↗
General-assistant questions requiring web search, file processing, and multi-step reasoning across three difficulty levels.
- GDPval-AA ↗
Real-world agentic work across dozens of occupations; models produce documents, slides, and spreadsheets, scored by blind pairwise comparison.
Tool + Conversation
- τ²-bench (Tau²-bench) ↗
Dual-control conversational benchmark (Sierra Research) where both agent and user modify shared state in a telecom support domain.
Workplace Simulation
- TheAgentCompany ↗
Tasks inside a simulated software company — browsing, coding, running programs, and messaging simulated coworkers (CMU).