Studies of online child prostitution criticized for methodology

A series of studies claiming to show a sharp increase in online child prostitution in Minnesota has come under criticism for using unreliable methods.

The studies by the Schapiro Group, a private Atlanta-based research and consulting firm, claimed that the number of Minnesota children being sold for sex on Internet classified sites like Craigslist and Backpage increased by nearly 65 percent from February to August last year. The reports also included data from Michigan, Georgia and New York.

The Women's Funding Network, an advocacy group that paid for the research, characterized the increase as "staggering" in staff testimony before the U.S. House Judiciary Committee. The Ramsey County Attorney's Office also used the numbers in a press release this February announcing new policies for helping children being sold for sex.

Released last spring and fall, the studies came amid a national debate about whether websites like Craigslist and Backpage should be allowed to publish classified ads for adult services. News organizations nationwide - including MPR News - reported the studies, which were used by advocacy groups to pressure states to take legal action against the websites.

View the May 2010 and August 2010 Schapiro reports here.

But a closer look at the data shows that researchers made several critical and controversial assumptions. Local police and FBI officials, statisticians and leading sociologists said that while there's no doubt that children are being sold for sex online, the Schapiro studies were too poorly conducted to provide reliable information.

Questions about the research were first reported March 23 by the Village Voice in a story that called the studies "junk science." But the weekly newspaper drew claims of conflict of interest because its parent company - Village Voice Media - owns Backpage, a classified ad website that includes an adult section.

Village Voice Media has declined to take down the adult section despite a written request sent by attorneys general from 21 states last September. Instead, the website increased its monitoring of adult ads and refers those that appear to include children to the National Center for Missing and Exploited Children. Craigslist closed its adult services section last September in response to similar letters from attorneys general in 18 states.

METHODOLOGY QUESTIONED

In the past two weeks, the studies have attracted additional criticism from experts nationwide.

"The methodology is so incomplete, and the data presentation so poor, that determining exactly what they did is impossible," University of Minnesota statistician Sanford Weisberg wrote in an email to MPR News. "This appears to be advocacy, not science."

Eric Grodsky, a sociologist at the University of Minnesota, had similarly harsh criticism. "The numbers they present are marginally better than the number we might make up," he said.

The Schapiro Group's report released last May estimated that 90 underage girls in Minnesota were being advertised in online sex ads on popular websites that month. The report said that marked an increase over an estimated 68 girls advertised in similar ads in February. The group's subsequent report in August estimated that number had increased to 112 girls.

The Schapiro Group said it arrived at those numbers by looking at photos of young women posted online, and proceeding with what experts say is a highly unscientific process.

The group's researchers began by asking 100 random adults to guess the ages of 30 to 45 random women and girls. They found they guessed the correct age 38 percent of the time.

The Schapiro Group then showed its researchers photos of girls from online prostitution ads. They multiplied the figure by 0.38 percent to arrive at the final figures for the number of underage girls engaged in prostitution in Minnesota.

Researchers then combined the daily count of presumed underage girls in sex ads with the percentage of presumed underage girls, who were advertised for an average length of two weeks and whose ads appeared for at least four days out of seven during those weeks. In May, that led to the figure of 90 girls - the number used by advocacy groups nationwide.

"This doesn't make sense," Matthew Salganik, an award-winning sociologist at Princeton University who has developed statistical methods to study those most at risk for contracting HIV. "It seems pretty sloppy."

He also criticized the Schapiro study for assuming online photos match the girls or women engaging in prostitution, a criticism shared by local law enforcement.

A VIEW FROM THE STREET

St. Paul Police Sgt. John Bandemer, who heads the Gerald D. Vick Human Trafficking Task Force, regularly monitors websites claiming to sell sex. The task force focuses much of its efforts on stopping child prostitution.

"We see a lot of photos that aren't of the real person," he said. "If we've set up a date with a girl off of an online ad, it's nowhere near the photo that they've posted on the ad. That happens all the time."

Bandemer also said that adults who sell children for sex online almost never include a photo of the child's face. Schapiro Group's studies only used photos that included the person's face. As a result, he said, it's likely that the group misidentified adults as children and overlooked actual cases of child prostitution.

Officers who specialize in tracking prostitution have gotten better at spotting underage children in online sex ads, Bandemer said, but most of the time they're still wrong.

"A decent percentage for us would be if we call 10 girls, we get five to show up, and one of them's a juvenile."

Bandemer said it's difficult to quantify how many children are being sold for sex, but he said there's been a noticeable increase since he joined the task force about five years ago. He said the problem isn't as simple as shutting down the adult section of popular websites.

When Craigslist closed its adult section, the number of sex ads didn't appear to decrease, Bandemer said. Instead, the number of ads on other websites increased. Recently, he's even noticed an increase in ads printed in yellow page directories.

"These things seem to be all cyclical," he said. "What always seems to happen are the bad guys will find a way to conduct business, and law enforcement has to catch up. And once they find that, bad guys will change, and that's just the way it's always been. It's a kind of cat and mouse game."

FBI spokesman Steve Warfield said the agency has encountered similar problems evaluating online sex ads. As a result, he said, the FBI does not track the number of ads because the count would not accurately reflect any trends in the sexual exploitation of children.

Schapiro Group initially declined MPR News' request for an interview and, via a public relations firm, offered to set up an interview with a researcher at a different agency who it said could defend the study.

But even that researcher, Michael Shively of Abt Associates, said he would have conducted the study differently and would have been more cautious about how to characterize the findings.

"If you look at the Schapiro Group's work, no one, including them, is going to tell you it is the top shelf, best science that can be done," he said. "I mean I don't think they would tell you that, but I also think it's not fair to just throw it away as complete junk."

Shively said despite the studies' flaws, the Schapiro Group should be given credit for trying to fill a research gap. He said the studies' findings were, if anything, a conservative estimate of the number of children being prostituted online.

The real problem, he said, is that few researchers study the issue, and there's no consensus about how to track the number of children being sold for sex.

DEFENDING THE DATA

One effort, funded by the U.S. Department of Justice in 2008, used a method known as respondent-driven sampling to help identify youth engaging in prostitution. Researchers connected with social service agencies to identify young people being sold for sex. They paid teenagers $20 to participate in an interview about their experiences. The teenagers were then given cards to give to their friends to be interviewed. The teenager received $10 for each friend who completed an interview. The study estimated that 3,769 children were engaging in prostitution in New York City.

"You can criticize that method, too," Shively said. "Do you honestly believe that the entire universe of the people being exploited are going to be linked to the people you get your hands on, and that there aren't any biases in that method? The answer is no."

Beth Schapiro later agreed to speak with MPR News. She defended the decision to use photos as a way to quantify child prostitution, despite the criticism from law enforcement officials and researchers.

"We approached a very complex problem by developing a scientific methodology that we've employed for counting this hidden and hard-to-find population," said Schapiro, who wouldn't say how much her group was paid for the work. "It's a very carefully crafted approach."

Shapiro defended the assumption that photos from online sex ads usually match the actual person being prostituted.

"We know that in the open market, that a product advertised should be the product delivered," she said. "Otherwise, the buyer's not going to buy. So if buyers didn't have trust in the online marketplace, then it probably wouldn't be such a popular place for these kinds of transactions."

CONFLICT OF INTEREST CLAIM

Shapiro expressed frustration with the media for criticizing the study. In particular, she cited the Village Voice article that called the studies "junk science." The reporter, Nick Pinto, works for City Pages, a Village Voice-owned publication in the Twin Cities. Village Voice Media is based in Phoenix.

Schapiro questioned the publication's motives. Village Voice Media also owns the classified site Backpage.com, a fact disclosed in a prominent editor's note alongside the story. The note said the Schapiro studies deserved the paper's scrutiny despite the risk of being criticized because the company has a financial stake in the debate.

"The fact is they make a lot of money off this industry, and they have a lot to protect," Schapiro said.

The president of Women's Funding Network, the advocacy group that commissioned the study, raised similar concerns.

"I think there's certainly a vested interest, if not a conflict of interest, in writing that this research is not credible, because it impacts what happens to their bottom line in terms of Backpage," said Christine Grumm, the group's president.

Grumm said she's not familiar with the details of how the studies were conducted, but she said, "We 100 percent back the research."

When asked if she thinks the data show a significant increase in the number of underage Minnesotans being prostituted, Grumm said, "I feel comfortable saying that young girls are sold online through these vehicles such as Backpage. Whether it's one or whether it's 10 or 15 or 20, you want to make sure that not one girl gets sold online."

Despite the criticism, some advocates said they plan to continue using the studies to highlight the issue.

Linda Miller, the executive director of Civil Society, a group that helps victims of human trafficking, regularly uses the studies' findings in presentations about sexual exploitation of children. In her own work, she said she's encountered fewer than 10 underage girls sold for sex on websites like Craigslist and Backpage.

"I'm not going to spend the time to try to understand the study," Miller said. "I'm just going to say that given all of the things I'm experiencing and I know, that study is the only way to get that information."

The Women's Funding Network plans to continue working with the Schapiro Group on future studies, but the group's president said she is open to exploring other approaches to tracking the problem.