Shows that an agent can learn complex behaviour from a reward model trained on human comparisons of short clips, using feedback on a small fraction of its interactions. The method underlies later RLHF work.

This is an external publication hosted at arxiv.org. AI Governance LK did not write it; the summary and notes on this page are our own.

Why it matters

Most widely deployed assistants are trained with some form of reinforcement learning from human feedback. This paper is where the modern version of the idea starts, and its experiments make the strengths and weaknesses of human preference data easy to see.

What you will learn

  • Explain how a reward model is learned from pairwise preferences
  • Describe the limits of preference-based feedback

Before you start

Suggested background. This is editorial guidance, not a requirement.

  • Basic reinforcement learning

How to cite

Christiano, P., Leike, J., Brown, T. B., Martic, M., Legg, S., & Amodei, D. (2017). Deep Reinforcement Learning from Human Preferences. Advances in Neural Information Processing Systems 30.