ProPublica’s analysis of justified homicides by police officers, published October 10th, has been widely read and often cited in the national conversation on the use of force by law enforcement. Most often, press reports and people on social media have highlighted our finding that from 2010 to 2012, African-American teenage men age 15–19 were at 21 times as great a risk of being killed by police officers as white teenage men.

That finding has received criticism, as is often the case when a number draws a spotlight. As with any analysis, our analysis arose from a series of judgments.

It was clear from the outset, for example, that the data from the FBI’s Supplementary Homicide Report, which underlies our assessment, is deeply flawed. This has been well-documented by us and others. In fact, we characterized the data as “terribly incomplete” and said its shortcomings were “inarguable.” But the FBI database stands as the most complete national record of deaths at the hands of police.

There were, generally speaking, three arguments contesting our analysis: That we should have included men of all ages, that we should not have excluded Hispanics from our analysis, and that we should have calculated rates for a longer period of time.

Our reporting came in the midst of debate and protest over the shooting death of Michael Brown, who was 18-years old and black, so teenage black men were the natural focus of our analysis. Of particular interest was whether the perception that black young men were at greater risk was supported by the FBI’s data. What happens to older men was not the subject of our story, and including them in the analysis was therefore not germane.

Peter Moskos, an associate professor at John Jay College of Criminal Justice and former Baltimore police officer, questioned our decision to exclude Hispanics from the white population. Race and ethnicity pose tough choices for any analysis involving demographic data. Our analysis was limited to two populations, and given what we were reporting on, we chose white, non-Hispanic males age 15–19 as our “control” group. Adding white Hispanics to our calculations posed two problems we found insurmountable.

First, it is our strong suspicion that Hispanics are undercounted. The FBI data says fewer than 20 black Hispanics were killed by police officers in 33 years of records, a claim that appears inaccurate on its face. Second, we’ve seen little evidence from our reporting that law enforcement officials treat the large Hispanic populations in say, Los Angeles, in the same manner as white citizens.

Several critics asked why we focused on the three most recent years in the data, rather than a longer time horizon. Moskos argued that three years of data is too small to offer a true picture of homicides by police and that we should have looked at 15 years of FBI statistics. There is a significant problem with this approach, one that is likely to distort the results. The core of our analysis is the rate of officer-involved homicides, which is calculated by comparing the number of shootings with the size of each population.

We focused on the three-year period of 2010 to 2012 because we could rely on an accurate source for the baseline numbers the Census Bureau’s American Community Survey. Using Census 2000 and Census 2010 data for baselines assumes that the ratio of populations remain static, and that a snapshot of population rates for a subset of time can be assumed to be accurate for an entire period. We know that’s not true.

Finally, we chose the years 2010 to 2012 because they are the closest large slice of time to the present day.

To test the critics’ argument, we calculated risk ratios for as far back as the American Community Survey data goes (2008). From 2006 to 2008, the risk ratio was 9.1 to 1 (with a 95 percent confidence interval 6.19, 13.39); from 2008 to 2010 the risk ratio was 17.93 to 1 (95 percent confidence interval 9.38, 34.30). And whether 9 times as great, 17 times or 21 times, the racial disparity remains vast, and demands deeper investigation.

In the course of reporting and analyzing, three ProPublica reporters separately conferred with David Klinger, a University of Missouri-St. Louis criminology professor, by phone and email. Klinger is a recognized authority on police use of force and a retired police officer, and is frequently interviewed in the press. In our published piece, we attributed to Klinger a note of caution regarding how measurement error might alter our findings, but also quoted him expressing doubt that such error would erase the wide disparity we found. Days after we published, Klinger wrote to one of the reporters to complain that he had asked not to be quoted in the story. None of the reporters who spoke with the professor recall him making such a request in any of the interactions. But in any event, his feelings on the issue are clear, so we’re updating the original post to reflect that.

As a final note, many have pointed to our reporting as proof of police bias. That overstates our case; ProPublica found evidence of a disparity in the risks faced by young black and white men. This does not prove that police officers target any age or racial group – the data is far too limited to point to a cause for the disparity. We hoped that our analysis would spur further inquiry into why this disparity exists, which it has done, and we stand by it.