Status: Proposed. Nothing on this page has been peer reviewed. Treat any outputs as preliminary.

A proposal to test, with a small controlled study, whether reviewers with limited legal training catch more errors in AI-generated contract summaries when given structured oversight aids such as claim-by-claim source links.

Research question

Do structured oversight aids improve the rate at which reviewers detect material errors in AI-generated legal summaries, without increasing false alarms?

Motivation

Scalable oversight asks how less capable judges can supervise more capable systems. Legal review is a realistic, high-stakes setting in which to study the question.

Methodology

Proposed: a between-subjects design comparing three conditions — no aid, source links, and source links plus a checklist — on summaries seeded with known errors. Ethics review would be obtained before any data collection.

Findings

No findings have been published for this project.

Limitations

  • Not started: design may change substantially.
  • Seeded errors may not resemble real errors.

Status

This is a proposal. No participants have been recruited and no data exist. We are looking for collaborators with experience in human-subjects research.