Capture-recapture method estimates AI agent recall from missed items

The author argues that evaluating AI extraction agents using only LLM judges or volunteers creates a blind spot because it fails to measure missed items that no checker detected. By applying a capture-recapture method traditionally used in ecology, the author demonstrates that standard AI evaluation methods significantly underestimate missed data and overstate recall. The author suggests using independent multi-source extractors, jackknife estimators, and small manual audits to achieve more accurate recall assessments. Open article →