Shows that an agent can learn complex behaviour from a reward model trained on human comparisons of short clips, using feedback on a small fraction of its interactions. The method underlies later RLHF work.
This is an external publication hosted at arxiv.org. AI Governance LK did not write it; the summary and notes on this page are our own.
Why it matters
Most widely deployed assistants are trained with some form of reinforcement learning from human feedback. This paper is where the modern version of the idea starts, and its experiments make the strengths and weaknesses of human preference data easy to see.
What you will learn
- Explain how a reward model is learned from pairwise preferences
- Describe the limits of preference-based feedback
Before you start
Suggested background. This is editorial guidance, not a requirement.
- Basic reinforcement learning
How to cite
Christiano, P., Leike, J., Brown, T. B., Martic, M., Legg, S., & Amodei, D. (2017). Deep Reinforcement Learning from Human Preferences. Advances in Neural Information Processing Systems 30.