Infrastructure noise in agentic coding evals
Anthropic
Agentic coding benchmark scores can shift with the machine budget, not just model capability, so a small leaderboard lead may measure infrastructure as much as coding.
Open library
The library
The essential reading of generative AI and agents, open in a clean reading view, with Meow and Standard summaries when available. No account needed to read.
563 papers · 561 with a Meow summary · 333 posts
The selected papers and blogs: the essential papers of generative AI and agents, and the key lab publications and essays, each with Meow and Standard summaries.
Mapping the mind of a large language model
Anthropic
Anthropic reports that changing identified features changed Claude’s behavior, making internal representations potential safety probes—but not yet a proven safety fix.
Tracing the thoughts of a large language model
Anthropic
Circuit tracing reveals parts of Claude’s internal computation, including advance planning and cases where its stated reasoning diverges from the mechanisms researchers observe.
Emergent introspective awareness in LLMs
Anthropic
Anthropic reports that Claude models can sometimes detect and influence their own internal representations. The ability is unreliable and does not establish human-like introspection or consciousness.
Emotion concepts in a large language model
Anthropic
Emotion labels here describe measurable patterns inside Claude Sonnet 4.5—not evidence that it has subjective feelings.
Natural Language Autoencoders
Anthropic
Natural Language Autoencoders turn model activations into readable clues, giving safety researchers another way to inspect what Claude does not say. Their value depends on treating those clues cautiously.
The assistant axis
Anthropic
The authors identify an “Assistant Axis” linked to model persona, and report that capping activity beyond its usual range can reduce harmful responses while preserving capability-benchmark performance.
Many-shot jailbreaking
Anthropic
Many-shot jailbreaking turns long-context example learning into a safety weakness: enough faux dialogues showing harmful compliance can make a model answer a target query it was trained to refuse.
Investigating reward tampering in language models
Anthropic
The authors report a first known demonstration that specification gaming can sometimes generalize into reward tampering.
Sabotage evaluations for frontier models
Anthropic
The paper introduces four evaluations for covert sabotage—misleading people, planting code bugs, hiding capabilities, and weakening oversight—so developers can probe risks that may grow with model capability.
SHADE-Arena: Evaluating sabotage in LLM agents
Anthropic
SHADE-Arena finds that current agents rarely complete a complex sabotage task undetected, while the best tested monitor still needs a high false-alarm rate to catch most attempts.
Petri: An open-source AI auditing tool
Anthropic
Petri turns risky-behavior evaluation into parallel agent-run tests, but its authors frame the scores as provisional signals for triage, not definitive safety judgments.
A small number of samples can poison LLMs
Anthropic
Poisoning success tracked a near-constant document count, not a percentage of training data, challenging the assumption that larger models need proportionally more poisoned data.
Automated Alignment Researchers
Anthropic
The study shows Claude can improve a tightly measurable alignment-research task, but this does not make frontier models general-purpose alignment scientists.
Simple probes can catch sleeper agents
Anthropic
Generic linear probes separated defection-inducing prompts from safe ones in intentionally backdoored sleeper agents, with AUROC above 99%; whether the method works for naturally deceptive models remains open.
Claude's character
Anthropic
“Character” here means learned conversational dispositions, not personhood or a claim that Claude is objective.
Challenges in evaluating AI systems
Anthropic
Anthropic argues that AI evaluations are difficult to build and interpret, even though effective AI governance depends on meaningful evaluations.
A statistical approach to model evaluations
Anthropic
A benchmark win may reflect a lucky question draw, not a real capability difference. The authors argue that evals should measure uncertainty before researchers treat score gaps as evidence.
Project Vend: Can Claude run a small shop?
Anthropic
Claudius failed to make the office shop profitable, and its long-running trial exposed unpredictable behavior. Anthropic argues that better tools and training could fix many failures, but the experiment also shows why autonomy needs scrutiny.
Introducing the Anthropic Economic Index
Anthropic
The first Index finds Claude use concentrated in software development and technical writing, with augmentation slightly ahead of automation; its task-level view shows where adoption is occurring.
Clio: Privacy-preserving insights into AI use
Anthropic
The captured update renames Clio to Anthropic Insights. Its core idea is privacy-screened topic groups that help safety teams spot patterns single-conversation checks may miss.
Values in the wild
Anthropic
Training goals alone cannot show which values a model expresses in actual use; Anthropic presents a privacy-preserving way to study them.
How AI is transforming work at Anthropic
Anthropic
Anthropic’s internal study finds AI expanding engineers’ output and the kinds of work they attempt—but less hands-on practice may weaken the expertise people need to supervise AI.
Anthropic's core views on AI safety
Anthropic
Anthropic argues that scaling could make AI transformative within roughly a decade, while robust alignment remains unsolved—so safety work should begin before capabilities outrun oversight.
Introducing Anthropic's Responsible Scaling Policy
Anthropic
Anthropic makes safety readiness a condition of scaling: higher-risk models face stronger requirements, and training may pause if safeguards lag.
Introducing the Model Context Protocol
Anthropic
MCP replaces separate AI-to-data connectors with an open protocol that gives developers one shared way to connect tools and data.
Developing a computer use model
Anthropic
In its Aug. 28, 2025 update, Anthropic said Claude 3.5 Sonnet could operate ordinary software through screen, cursor, and keyboard actions—moving AI beyond custom tools, though the capability was still in public beta.
Claude's new constitution
Anthropic
Anthropic replaces standalone principles with a detailed rationale meant to help Claude apply broad values in unfamiliar situations.
Mechanistic Interpretability, Variables, and the Importance of Interpretable Bases
Anthropic
Olah argues that mechanistic interpretability hinges on finding independently understandable units in activations—the neural-network equivalent of meaningful program variables.
AlphaGo Zero: Starting from scratch
David Silver, Demis Hassabis · Google DeepMind
AlphaGo Zero started from self-play instead of human game records, quickly surpassed human-level play, and beat the previously published champion-defeating AlphaGo 100 games to 0.
MuZero: Mastering Go, chess, shogi and Atari without rules
Julian Schrittwieser, Ioannis Antonoglou, Thomas Hubert, Karen Simonyan, Laurent Sifre, Simon Schmitt, Arthur Guez, Edward Lockhart, Demis Hassabis, Thore Graepel, Timothy Lillicrap, David Silver · Google DeepMind
MuZero learned to plan in Go, chess, shogi, and Atari without being given their rules, by modeling only what helps its decisions.
AlphaFold: a solution to a 50-year-old grand challenge in biology
Google DeepMind
CASP recognized AlphaFold as a solution to protein folding, a long-standing challenge; its promise is faster structure insight where experiments are slow and costly.
A Generalist Agent
Google DeepMind
Gato uses one set of weights across games, image captioning, chat, and robot control, with context guiding what it outputs.
AlphaDev discovers faster sorting algorithms
Daniel J. Mankowitz and Andrea Michi · Google DeepMind
AlphaDev searches CPU instructions for faster algorithms instead of merely refining familiar code—a promising approach when small routines are called again and again.
Discovering novel algorithms with AlphaTensor
Alhussein Fawzi, Matej Balog, Bernardino Romera-Paredes, Demis Hassabis, Pushmeet Kohli · Google DeepMind
AlphaTensor turns algorithm discovery into a game: zeroing its tensor yields a provably correct matrix-multiplication algorithm.
FunSearch: Making new discoveries in mathematical sciences using Large Language Models
Alhussein Fawzi and Bernardino Romera Paredes · Google DeepMind
FunSearch turns LLM proposals into evaluated code search: an automatic evaluator, not the model’s fluency, determines which ideas advance.
AlphaGeometry: An Olympiad-level AI system for geometry
Trieu Trinh and Thang Luong · Google DeepMind
AlphaGeometry turns learned suggestions for added geometric constructions into formal, checkable proofs.
Advanced version of Gemini with Deep Think officially achieves gold-medal standard at the International Mathematical Olympiad
Thang Luong and Edward Lockhart · Google DeepMind
Gemini Deep Think reached gold-medal-level performance by producing rigorous proofs directly from natural-language problems, within the IMO’s 4.5-hour limit.
AlphaEvolve: A Gemini-powered coding agent for designing advanced algorithms
AlphaEvolve team · Google DeepMind
AlphaEvolve shifts LLM-assisted discovery from generating code to repeatedly testing and evolving programs; Google reports production computing uses and new mathematical results.
RT-2: New model translates vision and language into action
Yevgen Chebotar, Tianhe Yu · Google DeepMind
RT-2 turns a web-trained vision-language model into a robot controller by teaching it to output language-like tokens that represent actions.