MentalHealthBench tests more than crisis avoidance: it uses expert-written rubrics to assess how AI responds across realistic mental health conversations.
Why it matters: The benchmark offers researchers a way to look beyond broad emergency-safety checks and identify specific response behaviors that need improvement. Its results can inform evaluation, but the article says ChatGPT is not a substitute for therapy or professional care.
PASTABench reframes agent safety as a question of whether and when to intervene during a risky sequence—not just whether an individual action looks unsafe.
Why it matters: For agent-safety evaluation, checking isolated actions or final outcomes may miss whether an intervention could happen in time. The findings also suggest testing beyond hazard keywords: competitive safety scores may not indicate genuine risk comprehension.