Agent frameworks & orchestration
Trial by Trace: Benchmarking AI Agent Harnesses for Reliability and Failure Attribution
The abstract introduces a benchmark that measures harness reliability separately from agent task performance.
Why it matters: For agent builders, the reported gap between task success and harness-criterion compliance suggests that outcome-only evaluation can miss reliability failures. The benchmark offers a way to examine harness behavior and attribute failures separately from agent performance.