Applies sparse autoencoders to a small transformer and recovers many features that are far more interpretable than individual neurons, offering a practical response to superposition.
This is an external publication hosted at transformer-circuits.pub. AI Governance LK did not write it; the summary and notes on this page are our own.
Why it matters
Sparse dictionary learning became one of the main tools in interpretability after this work. The interactive feature browser that accompanies the paper is worth exploring in its own right.
What you will learn
- Explain how a sparse autoencoder decomposes activations
- Evaluate claims of feature interpretability
Before you start
Suggested background. This is editorial guidance, not a requirement.
- Toy Models of Superposition
How to cite
Bricken, T., Templeton, A., Batson, J., et al. (2023). Towards Monosemanticity: Decomposing Language Models With Dictionary Learning. Transformer Circuits Thread.