Computational Clinical Judgment: Predicting Risk with Large Language Models

For seventy years, research has shown actuarial methods outperform clinical judgment. Yet actuarial approaches have limitations. They generally rely on structured data, cannot exploit rare case-specific details, have limited accuracy where outcome data are scarce or incomplete, and cannot offer case-level justifications. Large language models (LLMs) offer a new approach to prediction. Like actuarial methods, they aggregate information algorithmically, but like clinicians, they bring general knowledge and can provide case-level justifications. We prompted six LLMs to assess rearrest risk from 141 parole hearing transcripts and compared their predictions to a machine learning model trained on 4,000 cases with 91 administrative variables. GPT-5 matched this actuarial baseline on standard accuracy metrics and contributed independent predictive signal the actuarial model did not capture (p = 0.007), despite no task-specific training on rearrest outcomes. Its added signal came primarily from more accurately identifying low-risk individuals. However, its stated justifications did not reliably explain its predictions, echoing a longstanding limitation of clinical judgment.

Ryan Copus joined the University of Missouri-Kansas City School of Law in 2020. His research focuses on technically and ethically responsible ways to apply the power of machine learning/AI to the study and regulation of legal decision-making, with a particular focus on civil and appellate procedure and judicial administration. He teaches Civil Procedure as well as Law, Technology, and Public Policy. Prior to joining the faculty, Professor Copus taught Legal Research and Writing as a Climenko Fellow at Harvard Law School.

Professor Copus graduated from Harvard Law School in 2007. After clerking for the Honorable David C. Godbey of the United States District Court for the Northern District of Texas and practicing as litigation associate for Goodwin Procter LLP., he completed his Ph.D. at Berkeley Law School’s Jurisprudence and Social Policy Program in 2017, with specialization in law and economics. His dissertation, “Machine Learning and the Reliability of Adjudication,” developed predictive models of decision-making in the United States Court of Appeals for the Ninth Circuit Court and the California Board of Parole Hearings, using them to investigate the promises and limitations of algorithm-assisted justice.

Moderator: Benjamin Chen, Associate Professor & Director of the Law and Technology Centre, The University of Hong Kong Faculty of Law

To register, please go to https://hkuems1.hku.hk/hkuems/ec_regform.aspx?guest=Y&UEID=106717.

For inquiries, please contact Ms. Grace Chan at mcgrace@hku.hk / 3917 4727.