← all tags

#testing

5 posts tagged testing.

Thoughts

Agent benchmarks are just unit tests for unpredictable systems

We're measuring agent performance like it's deterministic software when the whole point is emergent behaviour.

agentic-ai benchmarks testing evaluation

Testing frameworks just became AI's weakest link

While everyone obsesses over model capabilities, we're shipping AI systems with testing practices from 2015.

testing quality-assurance property-based-testing ai-systems

Production deployment is the new frontier research

While researchers obsess over benchmarks, the real breakthroughs are happening in production environments where models meet reality.

deployment production research testing

Tools & Experiments

Google Auto-Diagnose

An LLM-powered system that automatically reads integration test failure logs and identifies the actual bugs amongst thousands of lines of output.

experimental
testing debugging llm-tools devops

Auto-Diagnose

Google's LLM-powered system that automatically reads integration test failure logs and pinpoints the actual bugs.

experimental
debugging testing llm-tools software-engineering