Applies sparse autoencoders to a small transformer and recovers many features that are far more interpretable than individual neurons, offering a practical response to superposition.

This is an external publication hosted at transformer-circuits.pub. AI Governance LK did not write it; the summary and notes on this page are our own.

Why it matters

Sparse dictionary learning became one of the main tools in interpretability after this work. The interactive feature browser that accompanies the paper is worth exploring in its own right.

What you will learn

  • Explain how a sparse autoencoder decomposes activations
  • Evaluate claims of feature interpretability

Before you start

Suggested background. This is editorial guidance, not a requirement.

  • Toy Models of Superposition

How to cite

Bricken, T., Templeton, A., Batson, J., et al. (2023). Towards Monosemanticity: Decomposing Language Models With Dictionary Learning. Transformer Circuits Thread.