Mechanistic interpretability tries to reverse-engineer what neural networks compute. It is one of the most hands-on areas of AI Safety, and one of the easiest to start contributing to.

A reading order

  1. Zoom In: An Introduction to Circuits — the core ideas, with pictures.
  2. A Mathematical Framework for Transformer Circuits — how to read a transformer.
  3. Toy Models of Superposition — why individual neurons are often confusing.
  4. Towards Monosemanticity — a practical response.

A first experiment

Install TransformerLens, load a small model and cache its activations on a sentence. Then try zeroing out one attention head and see how the model’s prediction changes.

logits, cache = model.run_with_cache("The capital of USA is")

The point is not the result but the habit: form a hypothesis about a component, intervene, and check.