The ADL's extremism statistics make it seem like ultraright-wing violence in the US is more common than it actually is

• The Anti-Defamation League's statistics about extremism have been cited as authoritative by much of the press, including its most publicized claim that nearly three-quarters of extremist killings in the US over the past decade have been committed by "right-wing extremists," a category that includes white nationalists.
• But if you define "extremist incidents" as incidents involving violence constituting a hate crime or terrorism, or incidents where a report of the incident refers to motivation by extremist views, just 58% of the incidents cited by the ADL over a 10-year-period fit that definition.
• A major factor is that the ADL counts "non-ideological" violence, such as extremists killing one another in botched drug deals and robberies, in its overall "extremist violence" stats. That significantly increases the overall number of incidents in its analysis.
• This is not to diminish violence motivated by bigotry or politics; on the contrary, the ADL's reporting of statistics might be better served with more clearly defined and exacting standards of what constitutes "extremist violence."
• Inflating the overall threat emboldens political extremists who want the public to live in fear. It also threatens confidence in the accuracy of trusted institutions' analysis to public crises.
• This is an opinion column. The thoughts expressed are those of the author.
• Visit Business Insider's homepage for more stories.

In 2015, Michael Augustine Bournes, 59, murdered his wife and three young children, then set his house on fire, before finally fatally shooting himself. An acquaintance told police that Bournes had called him after the murders and said his wife had been "mocking him and riding him all day." The family was living "off the grid" in a house in Montana not hooked up to public utilities. Police said Bournes was "a constitutionalist who didn't believe in government" and had anti-government literature in his pickup.

In 2008, Cynthia Lynch, 43, filed an online application to join an Oklahoma branch of the Ku Klux Klan. During her initiation ceremony, police say Lynch either tried to back out or got into an argument with the Klansmen; she was shot to death and mutilated, her remains dumped nearby.

In 2006, four members of the white supremacist Aryan Brotherhood took 46-year-old Jack Christen to a remote location and demanded to know what he told police about their methamphetamine operation. One of them shot and killed Christen.

Each of these crimes is counted as an "extremist incident" by the Anti-Defamation League, whose statistics have been presented by media outlets as the gold standard for reporting on extremists of the white nationalist, anti-government, left-wing anarchist, and radical Islamist varieties in the US.

These are each deeply upsetting stories of horrific acts of violence, but when you read them, do you think "terrorism"?

The 'extremist incidents' statistic you've seen everywhere

You've probably seen the statistic based on a study by the Anti-Defamation League that about 70% of "extremist killings" over the past 10 years have been committed by "right-wing extremists," a category that includes white nationalists, Aryan prison-gang members, and anti-government extremists.

That number, with similar phrasing, has been cited in The New York Times, The Washington Post, Politico, the Atlantic, and Insider, to name just a few outlets.

The specific wording of the stat drives home the idea that ultrarightists represent the greatest threat of domestic political- or identity-based violence in the US, dwarfing the violence committed by left-wing, anarchist, and Islamic extremists combined.

The ADL was founded in 1913 with a mission "to protect the Jewish people and to secure justice and fair treatment for all" and states its "ultimate goal is a world in which no group or individual suffers from bias, discrimination or hate," according to its website.

The group acknowledges that it "tracks both ideological and non-ideological killings by extremists," because it argues to ignore non-ideological killings would be to discount "an inherent and integral part of the dangers they pose to society."

But it's the topline catchall numbers of extremist incidents and extremist murders that the group promotes, and which most often end up being repeated in the press without the important distinction regarding possible motives.

If you are literally measuring the number of violent incidents committed by people with ties to extremist groups or ideas, the ADL's topline numbers hold up.

But if you limit the definition of "extremist incidents" to incidents motivated by extremist views where violence actually occurs, is attempted, or is substantially plotted, then the numbers drop significantly.

And if you consider only incidents that either are classified as hate crimes or terrorism by law, or were described as having been motivated by extremist views by the press — as most people likely do when they think of "extremist violence" — just 58% of the incidents cited by the ADL fit that definition, my investigation found.

To be sure, the ADL's most promoted and reported statistic — ultrarightist groups accounting for a disproportionate number of hate crimes and acts of terror — still holds up.

What's of concern is the seeming inflation of the overall numbers under the vaguely defined umbrella of "extremist incidents" — by including crimes that are not targeted at minorities — which make extremists appear to be more pervasive than they would otherwise be, with greater context.

It's ultimately an issue of reporting. Complicated issues aren't easy to distill in perfectly formed nuggets, whether they come from a well-respected advocacy organization's press releases or a news organization's headlines.

As I found, it's not an easy job to perfectly analyze hundreds of incidents. This is not to imply the ADL has engaged in bad-faith efforts. But in dealing with such sensitive issues, adding as much context as possible is vital to a greater public understanding of the issue.

An investigation of the ADL's 'extremist incidents' data

With the assistance of Insider reporters and a data analyst, I investigated the data the ADL used to make its H.E.A.T. map, an interactive tool tracking the locations of "all incidents of extremism or anti-Semitism in the United States," covering everything from anti-Semitic graffiti to racist violence to domestic terrorism.

According to the ADL's site, the map was created using "data points extracted from information sources including news and media reports, government documents (including police reports), victim reports, extremist-related sources, Center on Extremism investigations."

On several occasions, I reached out to the ADL for clarification on its criteria. In an email, the ADL directed me to the H.E.A.T. map's frequently-asked-questions page, which does not fully address my questions. But in its 2018 report on extremist violence, the ADL explained: "Extreme causes often attract adherents with violent tendencies—tendencies that are reflected not only in the violence that adherents commit for their cause, but also the violence they commit against others—including rivals, spouses, children and acquaintances."

Reviewing the citations of each incident defined as "Extremist Murders," "Terrorist Plot & Attacks," and "Extremist/Police Shootouts" from 2009-2018 on the ADL's H.E.A.T. map as a starting point, my colleagues and I explored over 500 cases through news reports, law-enforcement announcements, and court documents. The investigation is solely focused on "extremist violence," which we defined as incidents where police reports, court documents, or news articles presented evidence that the incidents were motivated by extremist viewpoints.

What I found is that the actual number of violent extremist incidents — if defining such crimes as being motivated by bigotry or politics — is much smaller than the ADL's oft-reported overall number.

Many of the ADL's 'extremist incidents' are not motivated by bigotry or politics. They're often extremists killing other extremists.

In several cases I examined, it was clear that while the person or people involved might have ties to extremist groups or hold extremist views, the crimes committed by these people that were classified by the ADL as extremist incidents were not targeting protected identity groups. These would likely be the "non-ideological" crimes the ADL says it includes in its tracking of extremist incidents.

For example, if a methamphetamine dealer who happens to be a member of a racist extremist group kills a rival dealer, the ADL may consider that an "extremist killing." The same seems to be true where white supremacists kill rival white supremacists, or even their own allies they fear are police informants.

There are also cases that defy classification. For example, how does one appropriately classify the case of the former neo-Nazi who killed his roommates for making fun of his newfound Muslim faith? Can this act be properly classified as a hate crime or terrorism?

What appears true — regardless of the definition used to cover extremist incidents — is the theme that emerges from the ADL data: Extremists of all political and prejudiced varieties are typically young men with a history of violence and criminality, often with pronounced mental and emotional issues.

But by painting its findings with such a broad stroke, the ADL data might lead some to conclude that there are significantly more hate crimes and terrorism in the US than actually transpire.

Based on the incidents cited by the ADL, in most years, extremists are just as likely to kill each other, their criminal associates, or their family members as they are to kill people in protected identity groups.

Incidents of extremists harming members of protected identity groups make up a fraction of the ADL's data

Along with my colleagues, I looked into each of the 414 incidents listed in the ADL's data that resulted in 426 killings over the course of 2009-2018 — which is the time frame of the ADL's oft-cited statistic. We vetted the available data to make the most accurate accounting possible, and with ambiguous cases where we couldn't make a decisive judgment call, we erred on the side of accepting the ADL's definition of the crime.

Our methodology consisted of analyzing each case using three criteria:

1. Was the incident linked to the race or protected status of the target or victim, or some other political motivation, in the news reports or legal materials we reviewed?

2. Was the perpetrator actually linked to extremist groups or ideology?

3. Did the incident include attempted violence, perpetrated violence, or substantial steps toward violence?

After weighing these criteria, we saw a considerable difference between the ADL count and our count.

Measuring data available on the ADL's H.E.A.T. map from 2009 to 2018:

• The ADL identified 414 extremist incidents during that 10-year period. Of those, I found just 240 met the criteria that it constituted actually attempted violence and involved a perpetrator with verifiable or self-identified extremist beliefs against a member of a marginalized or targeted group or their property. That's 58% of the ADL's count.
• The ADL identified 426 extremist killings during that time frame. I found 241 met our criteria.
• Of these killings, the ADL categorized 305 to be perpetrated by right-wing and white supremacist extremists. Of those, 124 met our criteria.

A breakdown of extremist violence by ideology, 2009-2018:

Islamist extremism

• The ADL identified 88 incidents (which included a substantial number of thwarted plots) as Islamist extremist violence. I identified 85.

Left-wing extremism

• The ADL identified 26 incidents as left-wing extremist violence. I identified 24.

Right-wing extremism (including white supremacist extremism)

• The ADL identified 300 incidents as right-wing extremist violence. I identified 131.

White supremacist extremism

• The ADL identified 199 incidents as white supremacist extremist violence. I identified 57.

The dangers of overbroad labeling of 'extremism' and one-size-fits-all statistics

There are issues inherent with trying to come up with a single catchall statistic on extremist violence.

For one, the federal government's definition of extremism fluctuates wildly.

This is partly evidenced by Black Lives Matter, an unofficial group of activists who demonstrate against police brutality, being designated in an FBI memo as part of a "black identity extremist" movement. Another recent example is the Department of Homeland Security labeling a group of nonviolent climate activists as "extremists" and grouping them among white supremacists and mass murderers.

The internet's capacity to make information about extremist murders easier to collect and collate — but also its ability to flatten discussions — can turn debatable discourse into supposedly unimpeachable facts.

It matters because failing to ensure accurate, transparent data undermines a genuinely compelling assembly of data about extremism and its victims in the US.

The ADL's database of incidents contains any number of altercations and murders that illustrate the sadistic cruelty — and the political ambitions — embodied by many extremists. But it's the part where those incidents are collected, tagged, aggregated, and boiled into a single statistic that could stand for more thorough discussion.

Colin Clarke, a senior Research Fellow at The Soufan Center, a nonprofit strategy firm focused on global security issues, told me in December, "This is a really challenging space, because we're asked to determine motive, which may not always be clear."

"It is incredibly difficult to devise a rigorous and replicable methodology that will be accepted by everyone," Clarke said. As a result, he believes researchers "need to do our best to be judicious and parsimonious in data collection, lest we risk threat-inflation of over-hyping the threat, which can dilute the overall message."

Clarke doesn't want to single out the ADL or any organization, because he says any data- or statistical-based study on terrorism could be subject to methodological criticism. "What this goes to show you is that data collection is critical and that more organizations should be involved in this debate," Clarke added.

Arthur Jipson, an associate professor at the University of Dayton who specializes in extremism, told me in February that figuring out the correct conclusions is "a tricky question" because there wasn't a concerted government effort to collect data on hate crimes until the early 1990s, during the George H.W. Bush administration. Jipson also said victims of bias crimes may not report them as such to law enforcement, and state definitions of hate crimes vary from state to state. Both, he argues, potentially undercut the overall number of extremist incidents.

Jipson added that he believes advocacy organizations like the ADL are working hard and acting in good faith, but he concedes advocates "could be more thoughtful about the process of data collection from the very beginning and try to be more careful when we make grand or powerful statements."

I presented Jipson with a hypothetical scenario: He has been endowed with a huge cash grant and a team of researchers and statisticians at his disposal. With those resources, would he include crimes committed by extremists that don't have any apparent political or bigoted motive among a catchall statistic for extremist incidents?

"Probably not," Jipson said, adding: "What I would do is say, here's the raw data, here's the raw total. Now let's break it up into categories where we've had really clear evidence. I don't think [the ADL] is wrong for [using] the raw data, but I would also then say, let's compare that against data where we have this absolutely clear empirical manifestation of that criminal intent."

The data does suggest that ultrarightists are the most violent extremists. But inflating the scope of the threat is a disservice.

You may wonder why it's worth reporting lower numbers of extremist violence than have been widely reported elsewhere. You may be worried that it risks downplaying the threat, or somehow encouraging the public to not be concerned.

On the contrary. Violent extremists are inspired by each other, and the scope of their violence is often used as a recruiting tool. Inflating the overall statistics could embolden extremists who want the public to be terrorized by the seeming scope and force of their violence.

Facts matter and context matters, and how those facts are disseminated matters.

Most people probably don't know that this big scary number is inflated by the fact that white supremacists are typically violent criminals and most of their violence is based on criminality, not ideology.

By using an overbroad definition of "extremist violence" and counting incidents that were not directly motivated by politics or bigotry, the ADL does a disservice to anyone trying to accurately report on these issues. It's not dissimilar to ultranationalists cherry-picking statistics regarding violence perpetrated by undocumented immigrants. The bigger the number, the more likely the public is to be cowed by a sense of dread.

The ADL's statistic that ultrarightists commit nearly three-quarters of all "extremist violence" has been so widely cited it's now ubiquitous. And as I found, they do commit the most violence motivated by their extremism. The ADL argues that to not include non-ideological violence in its overall numbers would downplay the threat posed by extremists. But by not promoting statistics that make a clear distinction between ideological and non-ideological violence, it undermines trust in the data itself.

When potentially overbroad or misleading data cements itself into the public consciousness, bad things can happen.

Overblown statistics about inner-city crime, drugs, and the threat of Islamist terrorism have all led to policies that have ultimately hurt the most vulnerable populations through over-incarceration, zero tolerance, and the blanket suspicion of mass surveillance.

Those who relied on such exaggerated numbers surely had their hearts in the right place and felt compelled to take action. There's also the psychic effect to consider on a public bombarded with terrifyingly outsized statistics and broadly drawn narratives lacking the appropriate context and nuance.

That again is why it's crucial not to over-amplify the threats by using overbroad figures for the overall number of extremist violence. The scrutiny here is not to downplay the threat of extremists or their inherent violence. They do pose a legitimate threat. But given the vast disparity between the numbers the ADL has disseminated and the numbers which we've provided with greater context, it is a conversation worth having.

The ADL doesn't need to make hate-related violence seem more prevalent than it actually is, when the truth is already disturbing enough.

John Haltiwanger, Grace Panetta, and J.K. Trotter contributed reporting. Walter Hickey contributed data analysis.