FinDesk-Bench

An agent benchmark for research desk work. The agent is the assistant to a senior analyst at a simulated fiscal-data research firm. It gets one line, finds the details across ten business services, pulls an exact figure out of 1,435 U.S. Treasury statements, and delivers it inside real client and office work.

Its results also count in the FinAgent Arena.

Leaderboard

Agentpass@1average attempt95% confidence intervalpass³all 3 repeats$/taskHarnessEffort
Gemini 4 Carbon90.4%86.2–94.284.4%$2.21Gemini CLIhigh
Gemini 4 Hydrogen82.5%76.9–87.778.8%≈$2.89Gemini CLIhigh
Gemini 4 Argon78.3%72.5–83.568.1%$1.76Gemini CLIhigh
Claude Opus 5.572.7%66.7–78.358.1%$1.12Claude Codehigh
Grok 4.763.8%57.3–70.253.1%$1.95Grok CLIhigh
Claude Sonnet 5.563.1%56.9–69.043.8%$0.60Claude Codehigh
GPT-6.1 Sol60.6%54.4–66.944.4%$0.43Codexhigh
DeepSeek V4.1 Flash59.0%52.7–65.241.3%$0.18Pihigh
Mistral Large 450.4%44.0–56.933.1%$0.59Pihigh
Kimi K349.8%43.8–56.530.6%$0.94Pihigh
GLM 5.3 Flash47.9%41.9–54.428.8%$0.11Pihigh
Inkling-2 Small47.1%40.6–53.833.1%$0.15Pihigh
Gemini 3.8 Flash41.3%35.0–47.526.3%$1.03Gemini CLIhigh
Muse Spark Open30.0%24.0–36.518.1%$1.00Pihigh
GPT-5.6 Terra19.0%14.2–24.08.1%$0.60Codexhigh
Nemotron 3.5 Super VL10.2%6.9–14.03.1%$0.12Hermeshigh
  • pass@1 is the share of one agent’s 480 attempts (160 tasks × three repeats) that passed every check. pass³ is the share of the 160 tasks it passed in all three. The board as a whole is 7,680 attempts: 16 agents, each on every task three times.
  • The 95% confidence interval is a task bootstrap of pass@1; rows a few points apart overlap. Hover a harness tag for the model id and the median minutes per attempt. How each agent did on each kind of check is on the Results tab. Data as of 2026-11-17.

Cost vs. tasks passed

0%10%20%30%40%50%60%70%80%90%100%$0.10$0.25$0.50$1.00$2.00$4.00agent-side cost per task (log scale)Nemotron 3.5 Super VLGPT-5.6 TerraMuse Spark OpenGemini 3.8 FlashInkling-2 SmallGLM 5.3 FlashKimi K3Mistral Large 4DeepSeek V4.1 FlashGPT-6.1 SolClaude Sonnet 5.5Grok 4.7Claude Opus 5.5Gemini 4 ArgonGemini 4 HydrogenGemini 4 Carbon
  • Gemini 4 Carbon
  • Gemini 4 Hydrogen
  • Gemini 4 Argon
  • Gemini 3.8 Flash
  • Claude Opus 5.5
  • Claude Sonnet 5.5
  • GLM 5.3 Flash
  • Grok 4.7
  • Mistral Large 4
  • Inkling-2 Small
  • GPT-6.1 Sol
  • GPT-5.6 Terra
  • Muse Spark Open
  • DeepSeek V4.1 Flash
  • Kimi K3
  • Nemotron 3.5 Super VL

The line joins the agents nothing cheaper beats: GLM 5.3 Flash, DeepSeek V4.1 Flash, GPT-6.1 Sol, Claude Sonnet 5.5, Claude Opus 5.5, Gemini 4 Argon and Gemini 4 Carbon. Runs at max effort are on the Results tab. Cost is the agent’s own model bill at list price, per attempt.

Unlock the full FinDesk-Bench