Applications & products
Auditing black-box clinical prediction models with a large language model through queries alone: a blinded test with planted flaws
An abstract reports that GPT-6 Astra detected nearly all planted flaws in clinical prediction models using prediction queries alone, but many of its other reports failed checks.
Why it matters: For agent builders, the results show that prediction-query access can support faithful surrogates and detection of planted flaws, but high flaw-detection performance does not make every generated diagnostic report reliable. The abstract’s authors stress validating reports against the model and proposed discoveries against data.