Should artificial intelligence be used to improve decision-making in courts? A new working paper finds that one example of an AI algorithm not only fails to improve the accuracy of judicial decisions, but the technology itself performs worse than humans.
“Many researchers have focused on whether algorithms are biased or whether AI is biased,” said co-author Kosuke Imai, a professor of government statistics. “What they haven’t really looked at is how the use of AI might affect human decisions.”
While AI recommendations are used in several fields, including criminal justice, healthcare, and even business, the final decision maker is usually a human. With this in mind, the researchers compared criminal bail decisions made by a single judge with recommendations generated by an AI system. Specifically, they analyzed the impact of AI on whether cash bail should be imposed.
30% — Percentage of cases in which judges rejected AI recommendations
The randomized controlled trial was conducted in Dane County, Wisconsin, and focused on whether arrestees were released on bail or forced to pay bail. Led by Imai and Jim Greiner, the S. William Green Professor of Public Law Emeritus at Harvard Law School, the researchers focused on hearings conducted by a single judge over a 30-month period, from mid-2017 to the end of 2019. They also analyzed arrest data for defendants up to 24 months later.
The results showed that the AI alone performed worse than judges in predicting recidivism, in this case by imposing strict limits on cash bail. At the same time, there was little difference in the accuracy of human-only and AI-assisted decision-making. Judges ruled against the AI’s recommendations in just over 30 percent of cases.
“This was surprising,” Greiner says, “and given the evidence that we’ve presented that algorithms can outperform human judgement, it appears that the algorithm was set too harshly. It was over-predicting that people who were arrested would cheat, and it was predicting that they would cheat frequently, and as a result it was recommending measures that were too harsh.”
The professors argued that this problem could be solved by re-adjusting the algorithm.
“It’s much easier to understand and modify algorithms or AI than humans,” Imai says. “It’s much harder to change humans or understand why they make the decisions they do.”
“The beauty of AI and algorithms is that they enable transparency.”
Kosuke Imai
The AI studied here didn’t specifically consider race, but instead focused on nine factors related to age and past criminal history. Imai, an expert in deploying statistical models to expose racial injustice, attributes inequities regarding bail to a variety of social factors, particularly those related to criminal history.
He acknowledged that the findings could be cause for concern, but noted that humans are biased too. “The beauty of AI and algorithms is that they allow for transparency,” he said. The key is to have open-source AI that is readily available for empirical evaluation and analysis.
Greiner added that the way the criminal justice system currently uses AI and unguided human judgment needs to be studied for improvements. “I don’t know if this is any comfort, but my response to people who are fearful or skeptical of AI is, be fearful or skeptical of AI, but be even more fearful or skeptical of unguided human judgment,” Greiner said. The way the criminal justice system currently uses AI and unguided human judgment needs to be studied for improvements.
Other co-authors on the paper are Eli Benmichael, assistant professor of statistics and data science at Carnegie Mellon University, Zhichao Zhang, professor of mathematics at Sun Yat-Sen University in China, Melody Huang, a postdoctoral researcher at the Wojcicki-Tropper Harvard Institute for Data Science, and Suan Xing, a doctoral candidate in political science in the Kenneth C. Griffin School of Arts and Sciences.
