All Models Are Wrong, But Are Risk Assessments Useful?

Abstract

This chapter examines the usefulness of a recent development in American pretrial law and policy: actuarial risk assessment tools. To address the United States’ crisis of mass pretrial incarceration, actuarial risk assessments have been proposed as a technical, bipartisan reform that may lower jail populations and reduce racial disparities in pretrial incarceration, all without compromising public safety. These algorithmic tools assign defendants risk scores based on the rate at which people with similar characteristics missed court dates, were arrested, or were arrested for a violent crime while on pretrial release. Rather than study whether risk assessments can produce a fair distribution of outcomes, this chapter conducts a functional analysis of pretrial risk assessments, examining their job performance in the tasks that they have been assigned. As it turns out, risk assessments cannot predict what matters, and what they can predict does not matter. Actuarial risk assessments’ predictions of serious crime are wrong almost all of the time.

Publication
Handbook on Pretrial Justice (Christine S. Scott-Hayward, Jennifer E. Copp & Stephen Demuth eds., Routledge 2021)
Colin Doyle
Colin Doyle
Associate Professor of Law