The school shootings that weren't

How many times per year does a gun go off in an American school?

We should know. But we don't.

This spring the U.S. Education Department reported that in the 2015-2016 school year, "nearly 240 schools ... reported at least 1 incident involving a school-related shooting." The number is far higher than most other estimates.

But NPR reached out to every one of those schools repeatedly over the course of three months and found that more than two-thirds of these reported incidents never happened. Child Trends, a nonpartisan nonprofit research organization, assisted NPR in analyzing data from the government's Civil Rights Data Collection.

We were able to confirm just 11 reported incidents, either directly with schools or through media reports.

In 161 cases, schools or districts attested that no incident took place or couldn't confirm one. In at least four cases, we found, something did happen, but it didn't meet the government's parameters for a shooting. About a quarter of schools didn't respond to our inquiries.

"When we're talking about such an important and rare event, [this] amount of data error could be very meaningful," says Deborah Temkin, a researcher and program director at Child Trends.

The Education Department, asked for comment on our reporting, noted that it relies on school districts to provide accurate information in the survey responses and says it will update some of these data later this fall. But, officials added, the department has no plans to republish the existing publication.

This confusion comes at a time when the need for clear data on school violence has never been more pressing. Students around the country are heading back to school this month under a cloud of fear stemming from the most recent mass shootings in Parkland, Fla., and Santa Fe, Texas.

At least 53 new school safety laws were passed in states in 2018. Districts are spending millions of dollars to "harden" schools with new security measures and equipment. A blue-ribbon federal school safety commission led by Education Secretary Betsy DeVos is holding public events around the country, including one in Alabama Tuesday. Children are spending class time on active-shooter drills and their parents are buying bulletproof backpacks.

Our reporting highlights just how difficult it can be to track school-related shootings and how researchers, educators and policymakers are hindered by a lack of data on gun violence.

"I think someone pushed the wrong button"

The Civil Rights Data Collection for 2018 required every public school — more than 96,000 — to answer questions on a wide range of issues.

It asked what sounded like a simple question:

In the 2015-2016 school year, "Has there been at least one incident at your school that involved a shooting (regardless of whether anyone was hurt)?"

The answer — "nearly 240 schools (0.2 percent of all schools)" — was published this spring.

The government's definition included any discharge of a weapon at school-sponsored events or on school buses. Even so, that would be a rate of shootings, and a level of violence, much higher than anyone else had ever found.

For comparison, the Everytown for Gun Safety database, citing media reports, listed just 29 shootings at K-12 schools between mid-August 2015 and June 2016. There is little overlap between this list and the government's, with only seven schools appearing on both.

A separate investigation by the ACLU of Southern California also was able to confirm fewer than a dozen of the incidents in the government's report, while 59 percent were confirmed errors.

The Civil Rights Data Collection dates to 1968. The Education Department's Office for Civil Rights administers the survey every two years. Every public school is required by law to complete it. These findings often drive public conversations.

For example, the CRDC was the source of recent reports that black students were suspended from school at rates much higher than whites — information that inspired changes in discipline policy across the country.

The survey has dozens of items, ranging from how many middle schoolers passed algebra I to how many students with disabilities were restrained or secluded. It can be completed by filling out an online form or uploading data.

One item, about "Firearm Use," was required for the first time for all schools in the most recent data collection.

Most of the school leaders NPR reached had little idea of how shootings got recorded for their schools.

For example, the CRDC reports 26 shootings within the Ventura Unified School District in Southern California.

"I think someone pushed the wrong button," said Jeff Davis, an assistant superintendent there. The outgoing superintendent, Joe Richards, "has been here for almost 30 years and he doesn't remember any shooting," Davis added. "We are in this weird vortex of what's on this screen and what reality is."

"We got wind of it and nipped it in the bud"

In other cases, something may have happened, but not the firearm discharge the survey asked about.

The biggest discrepancy in sheer numbers was the 37 incidents listed in the CRDC for the Cleveland Metropolitan School District. Roseann Canfora, the district's chief communications officer, told us that, in fact, 37 schools reported "possession of a knife or a firearm," which is the previous question on the form.

The number 37, then, was apparently entered on the wrong line.

Similarly, the CRDC lists four shootings among the 16 schools of the Santa Monica-Malibu Unified School District in California. Gail Pinsker, spokeswoman for the district, says that "going back 20-plus years," no one can remember any incident involving a firearm. Their best guess, she says, is that there was some kind of mistake in coding, where an incident involving something like a pair of scissors (California Education Code 48915[c][2]), for example, got inflated into one involving a firearm (48915[c][1]).

Ray Poole, the chief of legal services for the Nassau County School District in Florida, told us that at one school where a shooting was reported, Callahan Middle School, on Nov. 21, 2015, a Saturday, a student took a picture of himself at home holding a gun and posted it to social media. "We got wind of it and nipped it in the bud." No shooting.

The CRDC shows seven shootings in DeKalb County, Ga. Police reports provided to us by that district give a sense of more of the many, many ways the data collection may have gone wrong.

At Redan Middle School, there is a report of a toy cap gun fired on a school bus — not a shooting.

The CRDC shows a shooting at Stone Mountain Middle School, but a police report shows an incident at Stone Mountain High School instead.

And district officials provided a police report showing that there was a shooting after a McNair High School football game — in August 2016, after the time period covered in the survey.

Unacceptable burden

The Education Department's Office for Civil Rights received complaints about the wording and administration of this survey even before it went out.

A June 2014 research report commissioned to improve the CRDC as a whole noted that in previous data collections, districts had experienced "unacceptable levels of reporting burden." They complained that the CRDC asks them to report information that is similar to what states already collect, but in a different format, or at a level of specificity that they don't currently track.

Also at issue, the internal report says, was a "lack of clarity in the definitions of key terms." When it came to "Offenses," the group of questions including firearm use, districts "indicated dissatisfaction with the categories provided, specifically that the CRDC categories did not align with the categories used in state reporting, other federal reporting, and/or their own district databases."

As an example of this lack of alignment, the federal Gun-Free Schools Act requires schools in states that receive federal funds to expel students who bring a gun to school and requires districts in those states to report the circumstances of such expulsions to the state — regardless of whether a gun goes off.

The state of Florida asks schools to report "weapons possession," excluding pocketknives. California asks schools to report suspensions and expulsions resulting from "possession, sale, furnishing of a firearm" or "imitation firearm."

And so on.

There's also potential for confusion within the CRDC itself. While this particular item refers clearly to "a shooting," the previous item asks about a long list of incidents, some involving "a firearm or explosive device" and others involving "a weapon."

Temkin at Child Trends, who has long studied bullying and school climate, says this wording "could cause confusion."

"Best practices in data collection are not to include double-barreled items," she says, such as asking about a "firearm or explosive device" in the same question. An explosive device could be something like a pipe bomb or even a firecracker.

NPR submitted a Freedom of Information Act request to learn more about problems with the data collection, and we received emails that schools and districts sent as they grappled with this kind of confusion. For example, the Omro school district in Wisconsin wanted to know whether a consensual paintball-gun fight involving several students should be considered an "attack with a weapon" or a "possession of a firearm."

Another reason the shooting data may show these kinds of problems, Temkin adds, is that the item is so new. "Because this was the first year this was asked of all schools, they may not have been as prepared to respond to this item."

And there's another factor at work as well: the law of really, really big numbers. Temkin notes that "240 schools is less than half of 1 percent," of the schools in the survey. "It's in the margin of error."

Liz Hill, an Education Department spokeswoman, told NPR that "at least five districts have submitted requests to OCR to amend the school-related shootings data that they submitted for the 2015-16 CRDC." The plan is to issue what is called "errata" to update the data, but the original document will not be republished, Hill said.

Hill made the point that any "misreporting" is the schools' responsibility, not the department's: "As always, data reported by recipients is self-reported and self-certified."

After we contacted the Santa Monica-Malibu Unified district about the four reported shootings, the district emailed the Office for Civil Rights to try to correct the information. No shootings happened, officials said.

The Office for Civil Rights responded on July 25:

"The CRDC accepts correction requests for up to one year from the moment the submission period opens. For the 2015-16 collection, the corrections period closed on June 30, 2018, and for this reason your data correction request cannot be accepted. However, a data note will be included on the data file to ensure users are aware of the errors you are reporting."

NPR's Clare Lombardo contributed research for this piece.