How does one measure the overall performance of the Supreme Court? This problem is at the heart of Erwin Chemerinsky’s new book, The Case Against the Supreme Court. Chemerinsky’s basic argument is that, when measured over the course of American history, the Supreme Court has done more harm than good. When faced with “its most important tasks, at its most important moments,” the Court has, more often than not, failed to serve the American people. This is a provocative claim, and it deserves serious consideration. Whether one accepts the claim, however, will likely depend on whether one accepts a series of assumptions on which it is based. They include: that it is even possible to measure Court rulings in this sort of net-benefit-to-society way; that Chemerinsky chooses the correct metrics in determining whether a particular Court ruling belongs in the failure or success column; and that Chemerinsky’s overall tally of Court failures and successes supports his larger claim that the Court has been, on balance, a failure.

These are big questions, each deserving lengthy exploration. I will only be able to touch on them briefly in this post. First, how does one even begin to measure whether a given Court opinion was a good thing or not? It is hard enough to establish any sort of objective measure of the written opinion itself. (More on this in a moment.) But add to this the difficulty of assessing the way Court rulings operate in society and the problem becomes frightfully complicated. Bad rulings can have the effect of mobilizing opposition movements that have benefits for society. (E.g., Bowers v. Hardwick, which helped mobilize gay rights activists.) Good rulings can incite backlash movements that have costs for society. (Brown is the obvious example here.) And then there are the unexpected consequences of practically all major Court rulings—the good, the bad, and the mixed. (Read the late William Stuntz’s brilliant writing on the effects of the Warren Court’s criminal justice revolution and you’ll get a sense of the dilemma here.) All of this is to say that measuring something as complicated as a Supreme Court ruling is an astoundingly difficult task, one that demands not only an assessment of the written opinion but also of the complex, often unanticipated effects of a given Court ruling in society.

What about the particular metrics Chemerinsky uses to assess major Supreme Court rulings? Two points are worth noting here. One is that Chemerinsky focuses his analysis mostly on written opinions. He generally assumes that social effects can be extrapolated from the opinions themselves. The book does not give much space to the complexities unearthed by generations of judicial impact scholarship. This is, I believe, a significant shortcoming of the book. But it is also an understandable choice in a book written by a scholar whose expertise lies predominantly in the area of constitutional doctrine, not socio-legal scholarship, and whose primary goal is to provoke debate. (The basic conclusion found in most judicial impact scholarship is that the effects of bad cases are never quite as bad as feared, while the effects of good cases are never quite as good as hoped. Thus, this scholarship would largely work to undermine or at least weaken the simplicity and boldness of Chemerinsky’s case against the Court.)

The other complication—one that Chemerinsky recognizes but does not resolve—is the difficulty, and perhaps the impossibility, of locating criteria by which to measure whether a Court decision was a success or failure that does not just reduce to the ideological commitments of the author. Chemerinsky tries to negotiate this problem by emphasizing those decisions that have been universally praised or condemned. These are the decisions that reside in either the canon or “anticanon” of Supreme Court jurisprudence: Brown, West Virginia v. Barnette, Gideon v. Wainwright, Reynolds v. Sims, on the one side; Dred Scott, Plessy, Lochner, Buck v. Bell, Korematsu, on the other. But even if these categorizations are accepted (certainly not all would agree on this point), Chemerinsky does not confine his analysis to these canonical and anticanonical cases. After all, these cases constitute only a small minority of the Court’s decisions. To make his case against the Court, he compiles a list of successes and failures that includes much more controversial categorizations, placing rulings whose merits are still quite contested—such as recent Court decisions involving affirmative action or the scope of congressional power—into either the failure or success columns. And despite his protestations to the contrary, Chemerinsky’s assessments follow predictable lines: decisions favored by liberals are successes, those favored by conservatives, failures.

Finally, once one settles on a way to define success and failure for the Court, how does one compare them? Does Dred Scott plus Brown equal success, failure, or a wash? Bad decisions that upheld bad policy (such as Plessy or Korematsu) do not provide the same evidence in the case against the Court as bad decisions that overturned good policy (e.g., Dred Scott, Lochner). Similarly, good decisions that upheld good policy (e.g., West Coast Hotel, Heart of Atlanta) do not add the same value for Chemerinsky’s purposes as good decisions that strike down bad policy (e.g., Barnette, Brown, Gideon).

All of this is to say that Chemerinsky’s book is a great conversation starter. In the end, I found the questions underlying the eye-catching thesis more interesting than the thesis itself. Regardless of whether one agrees with Chemerinsky that the Court has failed more often than it has succeeded, it is fascinating just to think through the challenges of assessing the Court’s performance over its more than two centuries of history.