UK's AI Security Institute finds standard benchmarks systematically underestimate what AI agents can actually do

Summary

In a study covering seven benchmarks, the UK's AI Security Institute shows that standard AI evaluations systematically underestimate agent capabilities by capping the compute budget. On software engineering tasks, success rates jumped about 25 percent when the token budget was increased tenfold. Newer models benefit the most. Depending on the token budget, actual progress at the frontier is about 60 percent steeper than previous measurements suggested, according to AISI. The article UK's AI Secu

SpinGraph analysis pending — check back after processing.

• Microsoft follows Anthropic and OpenAI into the AI super app race with overhauled Copilot and AutoPilot agents
• Claude Code's complicated China problem involves bans on both sides of the Pacific
• Security vulnerability reports have exploded since AI models started hunting for bugs
• Meta's AI agent push is moving slower than Zuckerberg planned
• GPT and Claude failed Bridgewater's finance tests because the right answers were never public
• Tesla caps employee AI spending at $200 per week