Argues that systems trained with today’s methods could learn to exploit flaws in their reward signal, internalise goals that differ from those intended, and pursue power-seeking strategies — and explains why each is hard to detect.

This is an external publication hosted at arxiv.org. AI Governance LK did not write it; the summary and notes on this page are our own.

Why it matters

The paper connects classic alignment concerns to the concrete training pipelines used for modern models. It is a good bridge between introductory material and the more specialised literature.

How to read it

Work through it section by section and, for each claim, ask what evidence would change your mind. That habit is more valuable than agreeing or disagreeing with the conclusions.

What you will learn

  • Explain situationally-aware reward hacking
  • Describe how misaligned goals could generalise beyond training
  • Evaluate the paper’s arguments critically

Before you start

Suggested background. This is editorial guidance, not a requirement.

  • Familiarity with how large language models are trained

How to cite

Ngo, R., Chan, L., & Mindermann, S. (2024). The Alignment Problem from a Deep Learning Perspective. International Conference on Learning Representations. arXiv:2209.00626.