Bias in Criminal Justice

The Judicial Pipeline

The judicial "pipeline" looks something like this:

1. Some people commit crimes.
2. Some of those people are caught by police and arrested.
3. Some of those people are prosecuted.
4. Some of those people are sent to prison.

There are a variety of alleged biases, including

1. Police are biased towards arresting some groups over others.
2. Police are biased towards shooting some groups over others.
3. Prosecutors are biased towards prosecuting some groups over others.
4. Judges are biased towards giving some groups longer sentences.

We will evaluate each of these claims in order.

Base Rates

Any discussion of bias in the criminal justice system needs to start with base rates. The best source for these that I've found is the National Crime Victimization Survey (NCVS). This is compiled by contacting random people and asking them about their experience with crime. While not perfect, the only other method I know of is looking at arrest rates, which we can't use since one of the allegations is that police arrest some groups more than others.

The results for violent crimes are Criminal victimization, 2018:

Arrests

To count arrests for violent crime, I refer to the FBI's uniform crime reporting data Table 42 Table 43 Table 41:

Unfortunately, the NCVS and the FBI treat race differently, with the NCVS counting hispanics as a separate racial category and the FBI counting it a non-racial, ethnicity category. For this reason, the non-hispanic stats aren't comparable.

I tried finding alternative write-ups of these datasets but couldn't find anyone who had reconciled this difference. Here's my attempt: 89% of hispanics identify as white QuickFacts, 2.5% identify as black Black Hispanic and Latino Americans, and 0.5% identify as asian Asian Hispanic and Latino Americans.

As far as I can tell, we have two ways to progress: (1) assume hispanic whites (blacks) have the same arrest rates as whites (blacks) or (2) assume hispanic whites (blacks) have the same arrest rates as hispanics.

The first approach yields arrest rates of 0.084, 0.127, and 0.043 for for non-hispanic whites, blacks, and asians respectively. The second approach yields estimates of 0.058, 0.125, and 0.043 respectively. The true arrest rates are, presumably, in between these ranges and I'll be using the geometric mean going forward: 0.070, 0.126, and 0.043.

We see some discrepancies here: for violent crimes, females are 12% more likely to be arrested compared to males; blacks, hispanics, and asians are arrested 80% more, 56% more, and 37% less than whites; and adults are 38% more likely to be arrested than minors.

I'm going to ignore the age demographics going forward since I think its likely uncontroversial that police give minors more leeway than adults.

Arrest Rate Explanations

One explanation is that police are biased and arrest females, blacks, hispanics, and adults more than males, whites, asians, and minors. By its nature this hypothesis is difficult to test, so let's consider some other ones first.

Here are some I could think of that I, unfortunately, couldn't find good data for.

1. Some crimes have higher clearance rates than others, so, to the extent that demographics' violent crimes differ, we could expect their arrest rates to differ. For instance, murders are solved at twice the rate of robberies, so if one one group commits more murders relative to robberies, we'd expect them to have more arrests, all else being equal. Unfortunately, I couldn't find types of violent crime broken down in this way, so I can't control for it.
2. It's possible one group commits crime in a stealthier way on average and is, therefore, arrested less often.
3. It's possible the victims of some groups are more likely to contact the police, resulting in more arrests.

For race specifically, another possibility is that police patrol high-crime areas more to effectively reduce crime. A nationally representative survey found that blacks were actually 8% less likely to be approached by the police and 21% less likely to approach the police themselves Fryer. Just as a sanity check on the reliability of self-reported data, the survey finds blacks report being arrested 2.2x more often than whites, which matches the ratio of 2.3 implied by the FBI data Table 43.

Moving on to evaluating bias towards arresting people, we find some in data from the Stop and Frisk program in NYC, which says blacks were stopped 2.3x more than whites Fryer.

All in all, the lack of good data here is disappointing but not really surprising. Given the alternative explanations, I think its likely we can bound causation here and say that at worse the bias towards arresting females, blacks, hispanics, and asians to be arrested 12%, 80%, 56%, and -37% more than men/whites.

Prosecution & Conviction

There are two stages between arrest and incarceration: prosecution and conviction. I haven't really looked into this, because it would be so hard to interpret the findings.

For instance, suppose a higher % of black arrests lead to prosecution attempts than white arrests. Does this indicate that prosecutors are biased against blacks? Or does it indicate that blacks tend to leave more evidence than whites? Or that police are less likely to arrest blacks?

Likewise, suppose a higher % of blacks are convicted. Would that indicate jury bias against blacks or prosecutor bias in favor of blacks?

I haven't figured out a way to thread these needles.

Federal Sentencing

Federal sentencing guidelines are put together by the United States Sentencing Commission. They include a table Sentencing Table that converts an "offense-level" and a "criminal history category" into a range of recommended months.

These ranges are confined by the "25 percent rule", which stipulates Federal Sentencing: The Basics

[ This rest of this section is mostly a summary of Mustard. TODO: Stacey Starr ].

All that being said, the ranges are not absolutes and judges can depart from guidelines if "there exists an aggravating or mitigating circumstance of a kind, or to a degree, not adequately taken into consideration by the Sentencing Commission in formulating the guidelines". While judges are given broad discretion, they aren't allowed to use race, sex, or a variety of other protected traits. Judges use this discretion 27.1% of the time, with downward adjustments outnumbering upward adjustments by more than 20-to-1.

I don't know enough to evaluate how objective these "offense-levels" and "criminal history categories" are, but we can look at how judges treat various demographics in their discretion controlling for these two, presumably more objective, variables. The results (Table 6) of such linear regression find statistically significant positive coefficients for blacks, males, and people making less than $5,000 per year. If our controls are sufficient to infer causation, this implies judges are biased against blacks, males, people in deep poverty, and non-citizens. The differences are 4.8, 5.5, 6.2, and 1.7 months, respectively compared to a 46 month average sentence - that is, we're seeing differences that range from 4% to 13%.

They also broke down their analysis by crime type (Table 8), finding generally much larger effects for bank robbery and drug trafficking along the race, sex, income, age, and citizenship dimensions. On the other hand, the sentence lengths tend be much higher for these crimes than average, and the biases are about the same size in percentage terms.

Like I touched on earlier, it's possible judge bias is larger since some of it may be bias in the categorizing of criminals and their crimes. On the other hand, it possible that other non-forbidden variables are causing these differences rather than raw racism, sexism, classism. Nevertheless, this analysis seems like the best we can do.

Finally, I think it's interesting to note that some scholars believe sentences harm the rich more than the poor due to greater negative effects on their reputations and future incomes. For instance, one study found people with higher legal incomes lost a greater percent of those incomes after being released from prison The effect of conviction on the legitimate income of criminals. If true, this suggests (from optimal sentencing theory) that, assuming all else is equal, the rich should receive shorter sentences than the poor. Of course, all else is probably not equal, so the point is probably moot.

(Besides desires for fairness, the obvious counter-argument is that you'd intuitively suspect high-income people are more rational in their long-term decision making, which would make them more responsive to punishment, which suggests they should receive longer sentences.)

Recidivism

While it is illegal for judges to take race/sex explicitly into account, there are lots of variables that correlate with these protected groups that can cause judges to effectively give different sentences to different groups. Some of these variables can probably be justified as relevant and important even if there is an impact disparity in practice.

To tease apart the effect of the group's protected status (as opposed to correlates), I think a good place to start for this is recidivism. If a group's recidivism rate is higher, there are likely variables besides their protected status that correlate with both their protected status and recidivism. Therefore, even if we had a hypothetically discrimination-free judge, we'd likely expect protected groups with higher recidivism rates to receive longer sentences.

Fortunately, such recidivism data is available. In the first year after release, males are more likely to reoffend than females (45% vs 35%), blacks are more likely to reoffend than whites (46% vs 40%). Results are generally similar after 9 years (84% vs 77% and 87% vs 81%) 2018 Update on Prisoner Recidivism. I couldn't find data based on income, but I'd be surprised if recidivism wasn't higher for poor people than rich people.

In other words, for sex and race, the recidivism data predict males and blacks will get longer sentences even from discrimination-free judges. This suggests the criticism implicitly leveled in the last section may be unwarranted.

Converting these recidivism rate differences into expected sentence differences is challenging, fraught with philosophical, empirical, and mathematical difficulties.

For this reason, I don't think I can do much better at evaluating sentencing other than to say that blacks, males, the very poor, and older people receive longer sentences for the same crime severities and criminal histories. At last some of the differences based on sex, race, and income may be due to legal, non-discriminatory factors.

Police Violence

Finally, we come to the most politically heated topic: police killing people.

I found two reasonably good scholarly attempt to investigate this. The first looked at data from Houston, Texas and found that blacks are 24% less likely to be shot than whites after you control for a variety of other variables in police interactions Fryer. This was confirmed by a second more nationally representative sample Geller.

One factor they didn't account for is that blacks typically live farther from trauma centers than other races, so we should expect equally severe shootings to lead to more black deaths by default Hsia.

A second avenue of looking into this is to examine whether blacks police are less likely than white police to shoot black suspects. The "police bias" hypothesis suggests this is likely to be true. While I've found studies that examine this question, none control for the number of police interactions, so, unfortunately, I don't think there's good data on this at the moment, but what little we do have doesn't support this Fyfe.

As a final aside, a study of stop-and-frisks stops in NYC did find that police are 18% more likely to use nonlethal force on blacks than whites (and 12% more often on hispanics) after accounting for a variety of other variables Fryer. Another analysis of violence during arrests found a 28% increase in nonlethal force Geller.

In short, I could find no evidence that police are biased towards shooting blacks over whites and some evidence in the opposite direction. However, there is some evidence that police are biased against blacks on nonlethal levels of violence.

"Solutions"

Probably the most popular policy proposal is to require officers to wear body cameras. Unfortunately, a meta-analysis concluded that body cameras increase violence used against police without reducing the violence employed by police Ariel. While this conclusion has been contradicted by a later trial Braga, I think its pretty clear were the balance of the evidence lies.

Another proposal is to include more women and blacks in police forces via affirmative action. Unfortunately, female officers are actually 79% more likely to kill unarmed suspects than male officers and blacks are equally likely Do White Police Officers Unfairly Target Black Suspects - the hypothesis put forward by the researchers is that since women typically have less strength than men, they the need to resort to their guns more frequently in potentially violent situations.

A third proposal is simply reducing police funding. Assuming this proportionally reduces the number of police, this is also a bad idea. Every 1% increase in police reduces the number of murders by ~0.67% Chalfin. Each year, there are 1,112 police killings Mapping Police Violence and 19,510 total homicides Assaults or Homicide. A naive model suggests that decreasing the number of officers by 1% will reduce police killings by 1% (11 deaths) while increasing non-police overall homicides by 0.67% (123 deaths). Though, thinking more about it, we also need to account for the reduction in the ~168 deaths of police each year - that increases lives saved by ~2 but doesn't change the overall conclusion: policing saves lives on net.

States spend ~$100 billion on policing efforts, which means an additional $1 billion in funding to increase police by 1% would save ~110 lives, or about $10 million per life saved. That's roughly in line with how much the government claims to value a human life Rogoff and ignores the other benefit police give: reducing non-murder crime. All of this suggests we should actually hire more police.

I suppose spending cuts might be achieved without cutting officers, either by cutting officer pay or other non-compensation costs. I see no reason to think cutting officer compensation will save lives. I've heard people claim reducing police access to expensive weapons/vehicles/equipment would save lives, but I haven't seen any evidence for this assertion.

Finally, more nuanced advocates want to have particular duties done by civilians rather than officers. Given the vast number of such duties, I haven't done much digging for evidence in this direction.

Ignoring these 3 proposals, there are others I couldn't find any good evidence in either direction: breaking up police unions or publicizing police's records.

Regarding reforms that actually have some empirical evidence behind them, there's some promising evidence from reforms in Las Vegas, but, for better or worse, they made a large number of changes in a short span of time, making identifying the efficacy of each change essentially impossible Fachner.

There's also correlational evidence that higher gun prevalence causes more fatal police shootings such that (approximately) the top 5 states by gun ownership have twice as many fatal police shootings as the bottom 5 states after accounting for some demographic factors. Correlation isn't causation and this reform is politically infeasible, but, unlike most of the interventions I've heard, this is at least some evidence supporting it. Moreover, even if it turns out to be ineffective at reducing killing by police, we should reduce the number of guns anyways.

Summary of Bias Against Blacks

Overall, we have no good evidence that police are biased towards shooting blacks. We have evidence suggesting police use ~23% more nonlethal force on blacks and that blacks receive ~10% longer sentences for similar crimes compared to whites. The strongest evidence of bias is that, for violent crime, police arrest blacks ~80% more than whites after controlling for estimates of the total amount of violent crime committed - this is (imo) the biggest question for future work. All these estimates are, of course, correlational. Determining causation on this topic is notoriously difficult.

Overall, with the exception of bias-towards-arrest, these estimated effect sizes are fairly small - not in the sense of being unimportant - but in the sense that you wouldn't expect people in their everyday life to notice them.

For instance, suppose you live in an area with blacks and whites and you see 100 white arrests and 100 black arrests. Statistically speaking, about 4% of arrests are violent Geller, but blacks are 23% more likely to see nonlethal use of force. So, you see 4 violent white arrests and 5 violent black arrests. Concluding that there is bias against blacks is silly (p=0.76). Most people don't see anywhere near to 200 arrests and even those that do are unlikely to have perfect memories or unbiased thinking, so there's absolutely no way for to detect this bias without statistical analysis.

The same general argument holds true for police homicides and sentencing: unless you see literally hundreds of these events, you won't be able to detect any bias.

None of that is to say that these biases aren't important - but the idea that people's "lived experiences" offer significant insight here seems like its overreaching. Their "lived experience" can't even demonstrate to an unbiased reasoner that any bias even exists, never mind how, why, or what solution to pursue.