
Status: Proposed. Nothing on this page has been peer reviewed. Treat any outputs as preliminary.
A proposal to test, with a small controlled study, whether reviewers with limited legal training catch more errors in AI-generated contract summaries when given structured oversight aids such as claim-by-claim source links.
Research question
Do structured oversight aids improve the rate at which reviewers detect material errors in AI-generated legal summaries, without increasing false alarms?
Motivation
Scalable oversight asks how less capable judges can supervise more capable systems. Legal review is a realistic, high-stakes setting in which to study the question.
Methodology
Proposed: a between-subjects design comparing three conditions — no aid, source links, and source links plus a checklist — on summaries seeded with known errors. Ethics review would be obtained before any data collection.
Findings
No findings have been published for this project.
Limitations
- Not started: design may change substantially.
- Seeded errors may not resemble real errors.
Status
This is a proposal. No participants have been recruited and no data exist. We are looking for collaborators with experience in human-subjects research.