Newspaper finds suspicious test scores nationwide

ATLANTA (AP) - Hundreds of school systems nationwide exhibit suspicious test scores that point to the possibility of cheating, according to an investigation by The Atlanta Journal-Constitution.

The newspaper examined test results for 70,000 public schools and found high concentrations of scores in school systems from coast to coast.

The analysis doesn't prove cheating. It reveals that scores in hundreds of cities followed a pattern that, in Atlanta, indicated cheating in multiple schools.

The AJC reported in 2008 and 2009 about statistically improbable jumps in test scores within the 109-school Atlanta Public Schools system. Those reports led to an investigation by Georgia officials, which found that at least 180 principals, teachers and other staff took part in widespread test-tampering in the 50,000-student district.

In Sunday's editions, the AJC reports that 196 of the nation's 3,125 largest school districts had enough suspect test results that the odds of the results occurring naturally were less than one in 1,000.

For 33 districts nationwide, the odds of their test scores occurring naturally were worse than one in a million.

Standardized test scores have been at the forefront of national and local efforts to improve schools. Test performance was the centerpiece of the federal No Child Left Behind Act of 2001, which demanded higher classroom accountability. Tougher teacher evaluations that many states are rolling out place more weight than ever on the tests.

But the AJC report found that the sweeping policy shifts rely on test results that may be unreliable.

While the federal government requires states to use standardized testing, it does not require educators to screen scores for anomalies or investigate those that turn up.

"If we are going to make important decisions based on test results -- and we ought to be doing that -- we have to make important decisions about how we are going to ensure their trustworthiness," said Daria Hall, director of K-12 policy with the nonprofit Education Trust. "That means districts and states taking ownership of the test security issue in a way that they haven't to date."

In nine districts -- Atlanta, Baltimore, Dallas, Detroit, East St. Louis, Ill., Gary, Ind., Houston, Los Angeles and Mobile County, Ala. -- scores careened so unpredictably that the odds of such dramatic shifts occurring without an intervention such as tampering were virtually zero, the newspaper found.

In Houston, test results for entire grades of students jumped two, three or more times the amount expected in one year, the analysis showed. When children moved to a new grade the next year, their scores plummeted -- a finding that suggests the gains were not due to learning.

"These findings are concerning," U.S. Secretary of Education Arne Duncan said in a statement after being briefed on the AJC's analysis. He added that "states, districts, schools and testing companies should have sensible safeguards in place to ensure tests accurately reflect student learning."

Many school district officials contacted by the AJC disputed any conclusion that cheating was to blame for the swings.

Some school leaders attributed steep gains to exemplary teaching. But experts said instruction isn't likely to move scores to the degree seen in the AJC's analysis.

Cheating is one of only a few plausible explanations for such dramatic changes in scores for so many students within a district, said James Wollack, a University of Wisconsin-Madison expert in testing and cheating who reviewed the newspaper's analysis.

"I can say with some confidence," he said, "cheating is something you should be looking at."

In each state, the newspaper used statistics to identify unusual score jumps and drops on state math and reading tests by grade and school. Declines can signal cheating the previous year. The calculations also took into account other factors that can lead to big score shifts, such as small classes and dramatic changes in class size.

The newspaper also developed a statistical method to identify school systems with far more unusual tests than expected, which could signal endemic cheating similar to what occurred in Atlanta. In its approach, the score analysis used conservative measures that highlighted extremes. The methodology is more likely to overlook possible indications of cheating than to suggest problems where none exist.

The newspaper's methodology was reviewed by outside experts.

The AJC's analysis suggests that tens of thousands of children may have been harmed by inflated scores that could have kept them from getting the academic help they needed.

In 2010 alone, the grade-wide reading scores of 24,618 children nationwide -- enough to populate a mid-sized school district -- swung so improbably that the odds of it happening by chance were less than 1 in 10,000.

Experts said the findings warrant deeper investigation at the local level.

Statistical checks for highly improbable scores are like medical tests, said Gary Phillips, a vice president and chief scientist for the large nonprofit the American Institutes for Research, who advised the AJC on its methodology.

"This is a broad screening," he said. "If you find something, you're supposed to go to the doctor and follow up with a more detailed diagnostic process."

___

Information from: The Atlanta Journal-Constitution

(Copyright 2012 by The Associated Press. All Rights Reserved.)

Gallery

Fullscreen SlideshowPrevious Slide

1 of 1

Next Slide