Proposes training agents through a debate game judged by a human, on the idea that it is easier to judge an argument between two capable systems than to evaluate a single system’s answer directly.
This is an external publication hosted at arxiv.org. AI Governance LK did not write it; the summary and notes on this page are our own.
Why it matters
Debate is one of the canonical proposals for scalable oversight. Its central question — can a less capable judge reliably supervise more capable systems? — is still open and is directly relevant to professional settings such as legal review.
What you will learn
- Explain the debate protocol and its intended advantage
- Identify assumptions the proposal relies on
Before you start
Suggested background. This is editorial guidance, not a requirement.
- Familiarity with RLHF
- Some comfort with formal arguments
How to cite
Irving, G., Christiano, P., & Amodei, D. (2018). AI Safety via Debate. arXiv:1805.00899.