Open library
The library
The essential reading of generative AI and agents, open in a clean reading view, with Meow and Standard summaries when available. No account needed to read.
560 papers · 558 with a Meow summary · 333 posts
The selected papers and blogs: the essential papers of generative AI and agents, and the key lab publications and essays, each with Meow and Standard summaries.
Locating and Editing Factual Associations in GPT
Kevin Meng, David Bau, Alex Andonian et al.
We analyze the storage and recall of factual associations in autoregressive transformer language models, finding evidence that these associations correspond to localized, directly-editable computations. We first develop a causal intervention for identifying neuron activations that are decisive in a model’s factual pre…
Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 small
Kevin Wang, Alexandre Variengien, Arthur Conmy, Buck Shlegeris & Jacob Steinhardt
Research in mechanistic interpretability seeks to explain behaviors of machine learning (ML) models in terms of their internal components. However, most previous work either focuses on simple behaviors in small models or describes complicated behaviors in larger models with broad strokes. In this work, we bridge this…
Towards Automated Circuit Discovery for Mechanistic Interpretability
Arthur Conmy, Augustine N. Mavor-Parker, Aengus Lynch et al.
Through considerable effort and intuition, several recent works have reverse-engineered nontrivial behaviors of transformer models. This paper systematizes the mechanistic interpretability process they followed. First, researchers choose a metric and dataset that elicit the desired model behavior. Then, they apply act…
Representation Engineering: A Top-Down Approach to AI Transparency
Andy Zou, Long Phan, Sarah Chen et al.
We identify and characterize the emerging area of representation engineering (RepE), an approach to enhancing the transparency of AI systems that draws on insights from cognitive neuroscience. RepE places representations, rather than neurons or circuits, at the center of analysis, equipping us with novel methods for m…
Discovering Latent Knowledge in Language Models Without Supervision
Collin Burns, Haotian Ye, Dan Klein et al.
Existing techniques for training language models can be misaligned with the truth: if we train models with imitation learning, they may reproduce errors that humans make; if we train them to generate text that humans rate highly, they may output errors that human evaluators can’t detect. We propose circumventing this…
Scaling and evaluating sparse autoencoders
Leo Gao, Tom Dupré la Tour, Henk Tillman et al.
Sparse autoencoders provide a promising unsupervised approach for extracting interpretable features from a language model by reconstructing activations from a sparse bottleneck layer. Since language models learn many concepts, autoencoders need to be very large to recover all relevant features. However, studying the p…
The Geometry of Truth: Emergent Linear Structure in LLM Representations of True/False Datasets
Samuel Marks, Max Tegmark
Large Language Models (LLMs) have impressive capabilities, but are prone to outputting falsehoods. Recent work has developed techniques for inferring whether a LLM is telling the truth by training probes on the LLM’s internal activations. However, this line of work is controversial, with some authors pointing out fail…
Refusal in Language Models Is Mediated by a Single Direction
Andy Arditi, Oscar Obeso, Aaquib Syed et al.
Conversational large language models are fine-tuned for both instruction-following and safety, resulting in models that obey benign requests but refuse harmful ones. While this refusal behavior is widespread across chat models, its underlying mechanisms remain poorly understood. In this work, we show that refusal is m…
persona vectors: monitoring and controlling character traits in language models
Runjin Chen, Andy Arditi, Henry Sleight et al.
Large language models interact with users through a simulated “Assistant” persona. While the Assistant is typically trained to be helpful, harmless, and honest, it sometimes deviates from these ideals. In this paper, we identify directions in the model’s activation space—persona vectors—underlying several traits, such…
Mapping the mind of a large language model
Anthropic
Anthropic reports that changing identified features changed Claude’s behavior, making internal representations potential safety probes—but not yet a proven safety fix.
Tracing the thoughts of a large language model
Anthropic
Circuit tracing reveals parts of Claude’s internal computation, including advance planning and cases where its stated reasoning diverges from the mechanisms researchers observe.
Emergent introspective awareness in LLMs
Anthropic
Anthropic reports that Claude models can sometimes detect and influence their own internal representations. The ability is unreliable and does not establish human-like introspection or consciousness.
Emotion concepts in a large language model
Anthropic
Emotion labels here describe measurable patterns inside Claude Sonnet 4.5—not evidence that it has subjective feelings.
Natural Language Autoencoders
Anthropic
Natural Language Autoencoders turn model activations into readable clues, giving safety researchers another way to inspect what Claude does not say. Their value depends on treating those clues cautiously.
The assistant axis
Anthropic
The authors identify an “Assistant Axis” linked to model persona, and report that capping activity beyond its usual range can reduce harmful responses while preserving capability-benchmark performance.
Mechanistic Interpretability, Variables, and the Importance of Interpretable Bases
Anthropic
Olah argues that mechanistic interpretability hinges on finding independently understandable units in activations—the neural-network equivalent of meaningful program variables.
Gemma Scope: helping the safety community shed light on the inner workings of language models
Language Model Interpretability team · Google DeepMind
Gemma Scope extends interpretability tools to Gemma 2 2B and 9B, with sparse autoencoders trained across every layer and sublayer—not just tiny models or one layer.
Feature Visualization
Chris Olah · Distill
Feature visualization creates inputs that show what a network responds to, but raw optimization can exploit high-frequency artifacts; useful interpretations need diversity and constraints.
The Building Blocks of Interpretability
Chris Olah · Distill
What could an explanation show if it followed a network’s learned concepts, not just highlighted pixels?
Dario Amodei — The Urgency of Interpretability
Dario Amodei
Amodei argues that interpretability could help steer AI’s deployment, but its diagnostic tools must advance before models become more capable.
A global workspace in language models
Anthropic
A silent, word-linked J-space appears to carry some of Claude’s deliberate reasoning—and interventions show that changing it can change answers.
DeepSeek-V4-Flash means LLM steering is interesting again
Sean Goedecke
Steering changes a model’s internal activations, and DeepSeek V4 Flash makes it easier to try locally. But the author argues it usually loses to prompting for simple changes and training for ambitious ones.