Proposes training agents through a debate game judged by a human, on the idea that it is easier to judge an argument between two capable systems than to evaluate a single system’s answer directly.

This is an external publication hosted at arxiv.org. AI Governance LK did not write it; the summary and notes on this page are our own.

Why it matters

Debate is one of the canonical proposals for scalable oversight. Its central question — can a less capable judge reliably supervise more capable systems? — is still open and is directly relevant to professional settings such as legal review.

What you will learn

  • Explain the debate protocol and its intended advantage
  • Identify assumptions the proposal relies on

Before you start

Suggested background. This is editorial guidance, not a requirement.

  • Familiarity with RLHF
  • Some comfort with formal arguments

How to cite

Irving, G., Christiano, P., & Amodei, D. (2018). AI Safety via Debate. arXiv:1805.00899.