Predicting crime before it occurs

But against the backdrop of statewide calls for reform, Berk was already quietly working on a fix: an algorithm that could spit out a prediction of how likely it is that a would-be parolee will re-offend. Berk had begun building a similar algorithm for Philadelphia’s criminal-justice system in 2006, the year Philadelphia logged the highest murder rate among major cities. At the time, Philadelphia’s Adult Probation and Parole Department had 295 officers supervising nearly 50,000 individuals. The department asked Berk to predict which of the 50,000 would commit a serious crime within two years. “Our vision was that every single person, when they walked through the door, would be scored by a computer,” says Ellen Kurtz, the department’s director of research. The department would then use the score—low-, medium-, or high-risk—to decide how intensively to supervise released offenders. Officers assigned to low-risk individuals would handle up to 400 cases, and those monitoring high-risk offenders would have about 50.

Drawing from criminal databases dating to the 1960s, Berk initially modeled the Philadelphia algorithm on more than 100,000 old cases, relying on three dozen predictors, including the perpetrator’s age, gender, neighborhood, and number of prior crimes. To develop an algorithm that forecasts a particular outcome—someone committing murder, for example—Berk applied a subset of the data to “train” the computer on which qualities are associated with that outcome. “If I could use sun spots or shoe size or the size of the wristband on their wrist, I would,” Berk said. “If I give the algorithm enough predictors to get it started, it finds things that you wouldn’t anticipate.” Philadelphia’s parole officers were surprised to learn, for example, that the crime for which an offender was sentenced—whether it was murder or simple drug possession—does not predict whether he or she will commit a violent crime in the future. Far more predictive is the age at which he (yes, gender matters) committed his first crime, and the amount of time between other offenses and the latest one—the earlier the first crime and the more recent the last, the greater the chance for another offense.