Sotomayor’s Damned Statistics

posted at 12:53 pm on May 30, 2009 by Legal Insurrection

“There are lies, damned lies, and statistics.” So it is said. We now have reached stage three of the Sotomayor debate, because we are into statistical analysis of her opinions and dissents.

First, some backdrop on statistics as relates to Supreme Court nominees. During the confirmation process for Samuel Alito, then law professor Cass Sunstein did a statistical analysis of Alito’s dissents at the request of Ted Kennedy. The analysis was filled with qualifications and caveats cautioning about reading too much into statistical analysis of judicial opinions. Here is Sunstein’s punch line:

But to make that story short and simple: When there is a conflict between institutions and individual rights, Judge Alito’s dissenting opinions argue against individual rights 84% of the time. In almost all of the cases in which Judge Alito dissented in order to reject individual rights claim, he was sitting on a court with a majority of Republican appointees. A summary statistic of this kind must be taken with many grains of salt and with appropriate qualifications; let me now add a sense of context and a few complexities.

Among the difficulties of assessing voting records even for dissenting opinions, Sunstein noted, is that the opinions need to be “coded” to assess whether the vote is for or against “individual rights” and the one thing which does not enter into the analysis is whether Alito was correct as to the law:

An additional way of evaluating these dissenting opinions is less statistical; it involves an assessment of their merits and their relationship to preexisting law. Any such assessment will, of course, involve a high degree of discretion. But a preliminary analysis suggests two points. First, Judge Alito’s opinions are carefully reasoned, well-done, attentive to law, lawyerly, and unfailingly respectful to his colleagues. Second, it is fair to say that he has often dissented, in a way that rejects individual rights claims, even though the law, fairly interpreted, could well be taken to support those claims. Hence he has exercised his own discretion, not lawlessly but in a way that helps to illuminate his general approach to the law.

Sunstein went on in the letter to explain that while analysis of dissents may have some value, the analysis of majority opinions has little value as a predictor of how a Supreme Court Justice would vote once elevated to the Supreme Court:

Judge Robert Bork, for example, was in agreement on the D.C. Circuit with then-Judge Ruth Bader Ginsburg about 91% of the time – which did not suggest that they would vote in the same way on the Supreme Court! An important question for inquiry would have been the other 9%.

In plain English, Sunstein’s attempt to statistically categorize dissenting opinions was full of caveats, assumptions, and value judgments rendering any such assessment political garbarge. Not surprisingly, Ted Kennedy (who invented Borking) took Sunstein’s analysis, without mentioning the limitations and caveats, to argue that Alito would be hostile to individuals:

“In an era when too many Americans are losing their jobs, or working for less and trying to make ends meet, in close cases Judge Alito has ruled the vast majority of the time against the claims of individual citizens,” Kennedy said. “He has acted instead in favor of the government, large corporations, and other powerful interests. In a study by a well-respected expert, Professor Cass Sunstein of the University of Chicago Law School, Judge Alito was found to rule against the individual in 84 percent of his dissents. To put it plainly, average Americans have had a hard time getting a fair shake in his courtroom.”

Statistical noise now has entered the Sotomayor debate in her defense. At SCOTUS Blog, Tom Goldstein has read each of the race-related decisions in which Sotomayor participated, and finds as follows (italics mine):

Other than Ricci, Judge Sotomayor has decided 96 race-related cases while on the court of appeals.

Of the 96 cases, Judge Sotomayor and the panel rejected the claim of discrimination roughly 78 times and agreed with the claim of discrimination 10 times; the remaining 8 involved other kinds of claims or dispositions. Of the 10 cases favoring claims of discrimination, 9 were unanimous. (Many, by the way, were procedural victories rather than judgments that discrimination had occurred.) Of those 9, in 7, the unanimous panel included at least one Republican-appointed judge. In the one divided panel opinion, the dissent’s point dealt only with the technical question of whether the criminal defendant in that case had forfeited his challenge to the jury selection in his case. So Judge Sotomayor rejected discrimination-related claims by a margin of roughly 8 to 1.

Goldstein’s analysis clearly is well-intentioned, but also subject to differing interpretations. One interpretation, advanced by Goldstein, is that the statistics show that “it seems absurd to say that Judge Sotomayor allows race to infect her decisionmaking.”

But to use Ted Kennedy’s interpretation of statistics for Samuel Alito, one also could conclude that Sotomayor is hostile to minorities and racial discrimination claims. Hogwash. There is nothing in Sotomayor’s background to suggest such hostility, which simply proves the point that these sort of statistics are meaningless.

The other interesting aspect of Goldstein’s analysis is how he treats dissents. By focusing on majority opinions, Goldstein is going against Sunstein’s admonition as to Alito that majority opinions are not particularly meaningful for statistical purposes. Whereas Sunstein focused on Alito’s dissents, Goldstein focuses on Sotomayor’s concurrences.

Perhaps Goldstein focused on Sotomayor’s majority opinions because Sotomayor had so few dissents in the subject-area, meaning that she voted with her co-panelists almost all the time:

In sum, in an eleven-year career on the Second Circuit, Judge Sotomayor has participated in roughly 100 panel decisions involving questions of race and has disagreed with her colleagues in those cases (a fair measure of whether she is an outlier) a total of 4 times.

Goldstein treats the existence of a dissent as a “fair measure” of whether a judge is an “outlier,” but that is not necessarily so. Does the lack of dissenting opinions mean that Sotomayor is in the mainstream? Maybe. Does it mean that she does not show judicial independence of thought? Maybe. Does it mean nothing? Maybe.

Needless to say, Goldstein’s analysis of Sotomayor’s opinions is being spun as “plainly provid[ing] the best evidence of the kind of judge she will be” or ” fairly convincing evidence.” Wrong. Statistical aggregation of judicial opinions certainly is part of the evidence, but not conclusive, for all of the reasons and caveats set forth in the Sunstein analysis of Samuel Alito. The actual opinions, not the statistics, tell us much more about the candidate.

The emptiness of statistical analysis of judicial opinions also highlights the need to really understand who a nominee is, what philosophy she brings to the table, and what nods and winks (if any) have been conveyed to the nominating President. I’d much rather see the internal White House memos and e-mails about Sotomayor than spend my days with illusory damned statistics.

If we knew as much about Sotomayor as the President nominating her, we could make a better-informed decision, and would not be reduced to reading judicial tea leaves.

Cross-posted with updates at Legal Insurrection Blog

Blowback