Today marks the milestone of my 1,000th blog post since Appellate Strategist began publishing on February 23, 2010.
I thought we’d do a first today: comparing the two Supreme Courts we study in the same post. Specifically, since I’ve had the honor of appearing at both the California and Illinois Supreme Courts, I thought we’d compare the data for the past year in each court.
First, a bit of background. It’s not one of the more well-known areas of data analytics on appellate decision making, but data analysis of oral argument appears to have begun in 2004, with Sarah Levien Shullman’s article for the Journal of Appellate Practice and Process. Shullman analyzed oral arguments in ten cases at the United States Supreme Court, noting each question asked by the Justices and assigning a score from one to five to each depending on how helpful or hostile she considered the question to be. Once seven of the ten cases had been decided, she made predictions in the remaining three, based upon the correlations in the first seven. Shullman concluded that it was possible to predict the result in most cases by a simple measure – the party asked the most questions generally lost.
The year after Shullman’s study appeared, then-Judge John Roberts addressed the same issue in a study of Supreme Court arguments between 1980 and 2003. Like Shullman, Roberts concluded that the losing side was almost always asked more questions.
Professor Lawrence S. Wrightsman took a detailed look at Supreme Court arguments in a 2008 book. He chose twenty-four cases from the 2004 term, analyzing the number and tone of the Justices’ questions to each side. He concluded that although simple questions counts were not a highly accurate predictor of ultimate case results, question counts plus the content score were.
Timothy Johnson and three other professors published their analysis in 2009. The professors examined every Supreme Court case from 1979 through 1995, isolating both the number of questions per Justice and the number of words used in each question. Once again, the study concluded that all other factors being equal, the party asked more questions generally lost.
Professors Lee Epstein and William M. Landes and Judge Richard A. Posner published their study in 2010. Epstein, Landes and Posner used Professor Johnson’s database, tracking the number of questions and average words used by each Justice. Like nearly every other researcher, they concluded that all else being equal, the more questions a Justice asked a side, the more likely he or she was to vote against the party. Our own study, which included every civil and criminal oral argument at the Illinois Supreme Court since January 1, 2008, reached the same conclusion – by and large, the side receiving the most questions lost.
We report the average questions per side in civil cases in California and Illinois in Table 250 below, based on all arguments from May 2016 to May 2017. What we see is that the California Supreme Court is a considerably “hotter” bench than the Illinois Supreme Court is. In California, appellants received an average of 32.48 questions. Appellants in Illinois averaged only 11.53. Respondents in California averaged 30.18 questions in civil cases. Appellees in Illinois averaged only 9.26.
We report the total number of questions asked by each California Justice, segment by segment, in Table 251 below. Justice Cuellar led in the opening segment of civil cases, totaling 256 questions. Justice Liu was next at 225, followed by Justices Kruger (187), Werdegar (184) and Chief Justice Cantil-Sakauye (172). Justices Chin and Corrigan asked the fewest questions of civil appellants with 151 and 129, respectively.
The order was juggled somewhat for respondents. Justice Liu led by a substantial margin, asking 381 questions of respondents. Justice Cuellar was next with 262 questions. Justice Corrigan was third at 224, followed by Justice Werdegar (215), Chief Justice Cantil-Sakauye (170), Justice Kruger (130) and Justice Chin (116).
Justice Liu also led during rebuttals, asking 67 questions. Justice Werdegar was next at 63, followed by Justice Corrigan (51), Justice Chin (45), Chief Justice Cantil-Sakauye (36), Justice Kruger (23) and Justice Cuellar (21).
In Table 252, we report the average number of questions per segment in civil cases for each California Justice. Justice Cuellar averaged 5.12 questions during appellants’ opening argument. Justice Liu was next at 4.5. Justices Kruger, Werdegar and Chief Justice Cantil-Sakauye were bunched at 3.74, 3.68 and 3.44, respectively. Justice Chin averaged 3.02 questions during appellants’ opening argument, and Justice Corrigan averaged 2.87.
Justice Liu led during respondents’ arguments, averaging 7.62 questions. Justice Cuellar averaged 5.24 questions. Justices Corrigan and Werdegar averaged 4.98 and 4.3, respectively. Chief Justice Cantil-Sakauye was next, averaging 3.4 questions to respondents. Justices Kruger and Chin averaged the fewest questions to respondents in civil cases at 2.6 and 2.32, respectively.
Justice Liu also led during rebuttals, averaging 1.34 questions. Justice Werdegar was next at 1.26, followed by Justice Corrigan at 1.13. The other Justices each averaged less than one question during rebuttals: Justice Chin (0.9), Chief Justice Cantil-Sakauye (0.72), Justice Kruger (0.46) and Justice Cuellar (0.42).
Below, we report the average questions per segment during the same period for the Justices of the Illinois Supreme Court. For appellants’ opening arguments, Justice Thomas averaged 2.91 questions, which would have ranked sixth on the California Supreme Court. Justice Theis was next, averaging 2.69 questions. Justice Garman averaged 1.83, Justice Burke 1.17, and Chief Justice Karmeier one question during appellants’ opening. Justice Kilbride averaged 0.18 questions per opening.
The Court asked even fewer questions during appellees’ arguments. Justice Thomas led again, averaging 1.55 questions per argument – which would have been seventh in California. Justice Theis averaged 1.46 questions, Justice Garman 0.58, Justice Kilbride 0.36 and Justice Freeman 0.33. Neither Chief Justice Karmeier nor Justice Burke asked any questions of appellees in civil cases in the past year.
For rebuttals, Justice Garman led, averaging 1.08 questions. Justice Theis was next, averaging 0.69 questions. Justice Thomas averaged 0.45, Chief Justice Karmeier averaged 0.2 questions, Justice Kilbride averaged 0.09, and Justices Burke and Freeman asked no questions during rebuttals.
Join us back here tomorrow as we begin our next 1,000 posts by addressing the question we started out with: will the California Supreme Court be like every other appellate court researchers have studied, on average asking the most questions of the losing side?