Judges and juries in criminal trials are faced with deciding whether to convict or acquit a defendant. That decision should be based on the characteristics and quality of evidence in his or her case – but not on factors extraneous to the case itself. Yet, a growing body of research highlights that judge and jury decisions are affected by a number of such factors, including media exposure (Ouss and Philippe forthcoming, Lim et al. 2015); jury demographics, such as race, gender, or political affiliation (Anwar et al. 2012, forthcoming (a) and forthcoming (b)); and emotional shocks following college American football games (Eren and Mocan 2018) or NFL team wins (Chen 2016). Still, one unstudied factor in the context of criminal court outcomes is the verdict and characteristics of the previous case. That is, are there sequential decision-making biases in the judicial system, and in jury verdicts in particular? While not all juries today actually make sequential decisions, there are many instances where this occurs, including US grand juries, French juries, and Swedish lay judges.
Individuals outside of the criminal justice system make numerous decisions every day, many of which are sequential in nature. Existing research has studied the potential associated biases in such sequential decision making in many contexts, including loan officers, asylum court judges, baseball umpires (Chen et al. 2016), investors (Hartzmark and Shue forthcoming), speed dating participants (Bhargava and Fisman 2014), commuters (Simonsohn 2006), and homebuyers (Simonsohn and Loewenstein 2006). There are no such studies, however, on sequential decision-making biases in jury decisions.
In recent work (Bindler and Hjalmarsson forthcoming), we fill the gap in these two bodies of literature by testing for sequential decision-making biases in jury verdicts at London’s Old Bailey Central Criminal Court during the 18th century. Three features distinguish our work from the existing literature. First, path-dependency in criminal court decisions has not been studied. Second, the context is immensely high stakes – almost all convicts during this period (known as the ‘Bloody Code’) were either executed or transported to a penal colony. Third, this is the first test of path-dependency in group (i.e. the jury) decisions.
Would one expect biases in sequential jury decisions? A number of behavioural economics and psychological theories speak to that. Tversky and Kahneman’s (1971, 1974) gambler’s fallacy, for instance, could lead to negatively auto-correlated sequential decisions as decision makers underestimate the chance of a randomly occurring streak. In this framework, a jury – having just convicted five defendants in a row – may a priori expect the next defendant to be innocent. Alternatively, juries may compare case characteristics to ‘benchmark’ their decisions – focusing on differences between cases can lead to negative autocorrelations (or sequential contrast effects), whereas focusing on similarities can lead to a positive autocorrelation (or sequential assimilation effects). Further, jury decisions (especially in a high-stakes context such as ours) differ from many other situations in that the gory details of the case and/or the consequences of the jury’s actions (e.g. public executions) can have an emotional impact on the jury. These emotional effects may lead to autocorrelations in sequential verdicts that are either negative (due to, for instance, moral cleansing; see West and Zhong 2015) or positive (due to, for instance, updated beliefs about criminals or by changing a jury’s mood; see DellaVigna 2009).
Regardless of the channel, the existence of sequential biases in jury decisions has important implications for the judicial system – defendants with comparable cases may arbitrarily receive different outcomes depending on the order of trials.
Sequential jury trials in 18th century London
To study this question, we use a data set of more than 27,000 jury verdicts reached by over 900 juries from 1751 to 1808 and takes advantage of a unique feature of the English judicial system at that time: each jury (unanimously) decided the verdicts to multiple trials, which were presented to them sequentially. The data were extracted from a digitised version of The Proceedings of the Old Bailey. This document, dating back to 1674, was published after each (monthly) session at the Old Bailey and included an account of all criminal cases from London and the surrounding county of Middlesex trialled at the Old Bailey. Information about the jury and jurors was reported in the proceedings for these years, but not tagged in the digital files and has instead been manually coded by us. On average, each jury in the analysis sample saw 42 trials. During this period, it was not uncommon for jurors to be called for jury duty multiple times, such that there is typically at least one ‘experienced’ juror on the jury.
In the final analysis sample, 29% of defendants were females and approximately 43% of trials were for capital offenses. The prevalence of capital punishment, even for offences that would be considered minor today, is apparent in the fact that 84% of the (felony) trials were for property offense and 8% for violent offense. 63% of trials resulted in a guilty verdict.
Main analysis and findings
Simple tabulations indicate that a defendant had a 10 percentage point higher chance of being convicted if their case followed a defendant who was convicted (69% conviction rate) versus one who was acquitted (59% conviction rate). Similar raw gaps are seen regardless of whether the case was capital or non-capital, decided by a London or Middlesex jury, or for a male or female defendant.
Can these numbers be given a causal interpretation? Did the outcome of the previous case actually impact the outcome of the current case? Or does this positive autocorrelation arise because of common case characteristics that predict conviction? More precisely, to disentangle biases in sequential jury decisions from autocorrelation that arises due to a systematic ordering of cases, it must be assumed that cases are presented to the jury in an order that is not systematically sorted on the chance of conviction (or a case characteristic that predicts conviction). More specifically, it must not be sorted on any such unobservable characteristic. We use non-parametric runs tests of whether there are more or less streaks in the data than would be observed in randomly ordered cases to test for random ordering on observable case characteristics: offense type, defendant gender, and capital eligibility. The results show that the vast majority of juries face a set of cases that are randomly ordered in terms of these characteristics.
Based on that, a more formal regression analysis finds that the raw positive autocorrelation in the data is, in fact, causal in nature. This analysis studies within-jury decisions, and controls for all observable case characteristics (including more than 30 offence categories). A previous guilty verdict significantly increases the chance of a subsequent guilty verdict by between 6.7% and 14.1%. This positive autocorrelation is robust to multiple estimation strategies, independent of the extent of juror experience, and driven by the most recent cases as well as pairs of similar cases.
What explains this positive autocorrelation? The results are consistent with a sequential assimilation bias – juries may attempt to be internally consistent in their decision making, especially when two similar cases immediately follow each other. Other explanations, such as a ‘common shock’ (for instance, a hot courtroom) affecting the mood of the jury deciding cases are possible but (potentially) less likely to explain the positive autocorrelation, as the results are robust to controlling for case order decile and are only seen for the most recent lag. However, this does not rule out the possibility that the characteristics of a specific case affect the mood of the jury in the short term, and hence affect the outcome of a subsequent case. This potential ‘emotional’ bias is consistent with our finding that a lagged capital case matters over and above the lagged verdict and that there is a positive autocorrelation when a non-capital follows a capital case (even if dissimilar).
Despite the historical context, our findings are relevant to many situations today involving sequential decisions. At the individual level, one can think of teachers marking exams or caseworkers deciding on benefit applications. Similarly, at the group level, examples include hiring and grading committees or groups of judges and (most closely related to our context) grand juries. These examples differ in their nature and the outcome at stake. Clearly, there are differences in groups today with respect to social, demographic and economic diversity and group composition, and it is thus hard to precisely extrapolate our results. Yet, the fact that decision biases are seen for arguably the highest-stakes setting imaginable raises the possibility that they might be found in lower stakes situations as well. Further, as raised by Sunstein et al. (2002), criminal justice systems in which juries decide on single cases only may lead to incoherent decisions. The findings from our study suggest that the opposite case of sequential jury decisions, while possibly overcoming that issue, may lead to (potentially undesirable) path dependency in convictions.
Anwar, S, P Bayer, and R Hjalmarsson (2012) “Jury Discrimination in Criminal Trials,” Quarterly Journal of Economics 127(2):1017-1055.
Anwar, S, P Bayer, and R Hjalmarsson (forthcoming a) “Politics in the Courtroom: Political Ideology and Jury Decision Making,” Journal of the European Economic Association.
Anwar, S, P Bayer, and R Hjalmarsson (forthcoming b) “A Jury of Her Peers: The Impact of the First Female Jurors on Criminal Verdicts,” Economic Journal.
Bhargava, S and R Fisman (2014) “Contrast Effects in Sequential Decisions: Evidence from Speed Dating,” Review of Economics and Statistics 96(3): 444-457.
Bindler, A and R Hjalmarsson (forthcoming) “Path Dependency in Jury Decision-Making,” Journal of the European Economic Association.
Chen, D L, T JMoskowitz, and KShue(2016) “Decision-Making under the Gambler's Fallacy: Evidence from Asylum Judges, Loan Officers, and Baseball Umpires,” The Quarterly Journal of Economics 131(3): 1181–1241.
Chen, D (2016) “This Morning’s Breakfast, Last Night’s Game: Detecting Extraneous Influences on Judging,” IAST Working Papers16-49.
DellaVigna, S (2009) “Psychology and Economics: Evidence from the Field,” Journal of Economic Literature 47(2): 315-372.
Eren, O and N Mocan (2018) “Emotional Judges and Unlucky Juveniles,” American Economic Journal: Applied Economics 10(3): 171-205.
Hartzmark, S M, and K Shue (forthcoming) “A Tough Act to Follow: Contrast Effects in Financial Markets,” The Journal of Finance.
Lim, C S, J M Snyder Jr, and D Strömberg (2015) “The Judge, the politician, and the press: Newspaper coverage and criminal sentencing across electoral systems,” American Economic Journal: Applied Economics 7(4):103-135.
Ouss, A and A Philippe (forthcoming) “No Hatred or Malice, fear or affection: Media and Sentencing,” Journal of Political Economy.
Simonsohn, U and G Loewenstein (2006) “Mistake #37: The Effect of Previously Encountered Prices on Current Housing Demand,” Economic Journal, 116: 175-199.
Simonsohn, U (2006) “New Yorkers Commute More Everywhere: Contrast Effects in the Field,” Review of Economics and Statistics, 88: 1-9.
Sunstein, C R, D Kahnemann, I Ritov, and D Schkade (2002) “Predictably Incoherent Judgments,” Stanford Law Review, 54: 1153-1216.
Tversky, A and D Kahneman (1971) “Belief in the Law of Small Numbers,” Psychological Bulletin, 76: 105-110.
Tversky, A and D Kahneman (1974) “Judgment under Uncertainty: Heuristics and Biases,” Science, 185: 1124-1131.
West, C and C-B Zhong (2015) “Moral Cleansing,” Current Opinion in Psychology 6: 221-225.