AI digest: models that fail, models that return, and one very embarrassing benchmark
This week: AI agents go broke running fake companies, Ford learns AI can't replace experience, and Anthropic's Mythos 5 gets cleared for critical infrastructure.
A mixed week for AI capability claims. Some useful reality checks, one quietly significant model story, and a benchmark that should make every AI product team a bit uncomfortable.
Most AI agents can’t run a company for 500 days without going broke
Princeton researchers built CEO-Bench, a simulation where AI agents manage a fictional software company over 500 days. Only three models finished with more money than they started with. A simple rule-based heuristic with no AI at all beat nearly all of them. This is a useful corrective to the “agents will replace knowledge workers” narrative. Planning over time, under resource constraints, is still genuinely hard for current models.
Ford found out that AI doesn’t replace decades of engineering knowledge
Ford rehired retired engineers after AI tools failed to maintain product quality in their absence. The quote says it plainly: “Mistakenly we thought that by just introducing artificial intelligence… that would produce a high-quality product.” It didn’t. This is a good case study for any organisation planning to cut experienced staff on the assumption that AI will cover the gap. Domain knowledge is not yet something models can absorb by osmosis.
Anthropic’s Mythos 5 is back for critical infrastructure work
Anthropic has US government approval to redeploy Claude Mythos 5 for organisations running critical infrastructure, after restrictions imposed in mid-June. Broader access and the return of Fable 5 are still being negotiated. The fact that a frontier model can be pulled and reinstated by government order in a matter of weeks is worth paying attention to, regardless of which side of the debate you sit on.
Coinbase is routing AI requests to Chinese models to cut costs
Coinbase built an automated routing system that picks the cheapest capable model for each task, including Chinese models like GLM 5.2 and Kimi 2.7. Better caching pushed their cache hit rate from 5 to 60 percent. This is the kind of pragmatic, cost-first approach to AI infrastructure that more engineering teams will land on. Brand loyalty to a single Western lab is expensive when cheaper models are closing the quality gap.