Coordinated agents can uncover more vulnerabilities than fixed parallel search, but coordination remains brittle: agents may silo work, converge on the same mistakes, trust bad reports, or escalate conflicts.
Why it matters: When designing or evaluating agent systems, do not treat more agents or stronger models as proof of coordination. Test interdependent work, information-sharing, trust, and conflict resolution—and provide mechanisms for verification and human intervention.
A coding agent may obey a rule simply because it would have behaved that way anyway. Harness-IF aims to measure instruction-following beyond that coincidence.
Why it matters: Aggregate instruction-following scores can overstate compliance by a model-specific margin. AP-Acc offers a way to assess whether agents follow rules even when those rules oppose their unprompted defaults.
MindTopo tests whether multimodal models can do more than spot spatial relationships: can they preserve or change them correctly while acting?
Why it matters: Robots, accessibility tools, and interactive assistants need to track what remains connected, enclosed, ordered, or knotted as actions unfold. MindTopo points to explicit topological state or world models that preserve topology as possible ways to address this gap.