Child sex trafficking and the 5 Ws and How: Another statistic gone astray

When a reporter covers a problem like child sex trafficking, how hard should she look at the numbers?

The report from KPTM, Fox42 news in Omaha, starts out strong:

You might not think of Omaha as being a hotbed for the slave trade, but if you don’t, think again.

There is evidence that hundreds of girls, some in their early teens are being sold as sex slaves in truck stops in our area.

Shocking. And it gets even more so:

300,000.  That’s a big number and it’s the number of children who are kidnapped and forced into sex trafficking each year according to the U.S. Department of Justice.

There are three errors in that statement, not even counting the questionable status of the number itself. This is another lesson in the risk of reporting statistics. And as I wrote last week, applying the 5 W’s and How could have prevented this mistake.

You may remember the debate about data on sex trafficking from the Village Voice’s 2011 report, which used actor Ashton Kutcher as a punching bag. Kutcher’s response was a classic ad hominem argument: the Voice was tied to a site that features ads for prostitution, ergo its reporting should be rejected. And then others picked sides, seemingly basing their choices more on personalities than facts. Jezebel was upset that the Voice article was snide, for example. But the Voice’s analysis of the problems with the most commonly cited figure for underage sex trafficking was correct. And if you would prefer a source with clean hands, the Christian Science Monitor examined the issue last year and wrote:

One of the most troubling statistics [Kutcher and Demi Moore] shared was that there are 100,000 to 300,000 sex slaves in the US … The problem: The statistics are wrong.

Those figures came from a 2001 University of Pennsylvania study (“The Commercial Sexual Exploitation of Children in the U.S., Canada and Mexico”) that estimated that there might be 100,000 to 300,000 children at risk of becoming trafficked prostitutes because of an array of negative circumstances, from homelessness to drug addiction. The number of actual sex-trafficking victims has been estimated by the US government to be in the tens of thousands, but even those numbers have been criticized as unfounded and far too high; between 2008 and 2010, federally funded human-trafficking task forces opened 2,515 suspected incidents of human trafficking for investigation. Among those cases, only 248 suspected sex-trafficking victims under the age of 18 were identified.

So let’s apply the 5 W’s and How to see what’s going on.

Who: The original source isn’t the Department of Justice or, as noted in some stories, the National Center for Missing & Exploited Children. In both cases, they’re relying on a study by Richard Estes and Neil Weiner of the University of Pennsylvania (read a PDF of the executive summary). Saying that the source is the DOJ makes this sound like an official government finding, possibly based on hard data.

What: According to the Voice story, Estes and Weiner originally made a strong claim for their statistics, but were argued into softening their description. I can find a version of the full report online that lists “estimated number of child sexual exploitation cases” for American-born children as 293,746. Even there, the report notes that the figure is based on adding up estimates of prevalence among various categories of youths. Since children may fall under more than one category — say, both a female gang member and a runaway — it lists a “medium scenario” of 258,497, based on a guess of the overlaps.

But in the summary linked above, that same chart is now labeled “children at risk of commercial sexual exploitation [I added the emphasis].” So the figure — more like 250,000 than 300,000 — is not of children actually sexually exploited, but those who the study’s authors consider to be at risk. Indeed, the report summary notes explicitly that it cannot estimate the actual number of children affected.

When: The report was published in 2001. The study itself made note of some facets of its calculations that showed dramatic changes in the few years before it was conducted. One might presume that — with another decade of state and federal efforts targeting this issue — the numbers might be different today.

Where: The study does claim national scope. However, it targeted only 17 large U.S. cities, looking specifically at those that had “known problems” with commercial sexual exploitation of children. Extrapolating from that to national figures raises some questions, at least.

Why: The original study was begun in 1999, five years after the federal government began keeping statistics on sex trafficking worldwide. The Department of Justice helped pay for the study. Perhaps more importantly, some version of its numbers have been used often by those trying to stop underage sex trafficking or sex trafficking in general.

Note, for example, how Kutcher cites “100,000 to 300,000” as the figure for child sex trafficking in the U.S. The National Center for Missing & Exploited Children took the “medium scenario” from the report — saying that it represents an estimate of actual children victimized — and decided to be “conservative” and set a base level of 100,000:

… [T]here are caveats. First, the researchers reported that 293,000 US children are”at risk” of commercial child exploitation each year. Being at risk is different from actually becoming a victim. However, they also reported that the number of 10 — 17 year olds involved in commercial sexual exploitation each year likely exceeds 250,000, with 60% being runaway, throwaway or homeless youth. The second caveat is that commercial sexual exploitation is broader than just child prostitution, but there is little doubt that the commercial sexual exploitation of runaway, throwaway and homeless youth is overwhelmingly prostitution — 60% of 250,000 is 150,000.

Further, the Justice Department’s National Incidence Study on Missing, Abducted, Runaway and Thrownaway Youth (NISMART II) estimated nearly 1.7 million runaways and throwaways each year, of which just 357,600 are reported to police. The report found that 1.6 million are 12 – 17 years old, and 1.3 million are gone from 24 hours to 6 months. These are the kids most at risk of becoming victims of child prostitution and child trafficking. Thus, while 100,000 is just an estimate, we believe it is reasonable, conservative and based on sound empirical research.

So, the NCMEC accepted the total figure from the study as a measure of those “at risk,” but decided that the “medium scenario” — which is just an accounting for overlap in the categories of at-risk situations — was actual victims. Then it waved its hands and decided to settle on 100,000. And that got turned into the bottom of a range going all the way up to the study’s highest figure.

One would expect those who are fighting a problem to depict that problem in the harshest possible light. Given a choice between citing the number of underage prostitutes actually arrested in the country — a tiny amount — and the numbers in the Penn study, the choice for Kutcher, the NCMEC and others is clear. But it’s the job of journalists to do more than just picking from Column A or Column B.

How: The Voice talked to a professor whose work was cited in the Penn report, David Finkelhor. His comment on the researchers’ entire approach:

“As far as I’m concerned, [the University of Pennsylvania study] has no scientific credibility to it,” he says. “That figure was in a report that was never really subjected to any kind of peer review. It wasn’t published in any scientific journal.”

The Voice also asked Steve Doig, a journalism professor whose specialty is examining the numbers in studies like this, what he thought:

Many of the numbers and assumptions in these charts are based on earlier, smaller-scale studies done by other researchers, studies which have their own methodological limitations. I won’t call it ‘garbage in, garbage out.’ But combining various approximations and guesstimates done under a variety of conditions doesn’t magically produce a solid number. The resulting number is no better than the fuzziest part of the equation.

It would be obvious from just a reading of the methodology section of the study that Doig’s correct. This is not to say that the report is useless, but its underpinnings are far too weak to have sustained the burden of being the sole source for more than a decade of reporting on the issue.

I emailed the reporter at Fox42 to ask her how she came up with her cited figure of 300,000 children “kidnapped and forced into sex trafficking.” Her response: she talked to some FBI agents, a victim supplied by an advocacy group, and truck stop owners who were part of another advocacy group.

Look at what she got wrong: First, the DOJ was only citing the Penn study; it wasn’t the source itself. Second, a current DOJ citation uses the “at risk” qualifier, which the story didn’t. Third, the story takes the claim of being sexually exploited and converts it into being kidnapped. A short search of Internet reporting on the issue would certainly have turned up the Village Voice article, which includes this passage:

How many kids are involved in sex slavery—forcibly taken into the trade and abused?

“That number would be small,” Estes acknowledges. “Kids who are kidnapped and sold into slavery—that number would be very small.”

When we talk about very small, what sort of number are we talking about?

“We’re talking about a few hundred people.”

It’s been 12 years since the original study. It’s been two years since the Voice’s thorough debunking. It’s been a year since the CSM weighed in. And still, a sloppily manipulated, misconstrued version of a number that was questionable in the first place is being cited. It’s not just Fox42; I found other recent examples. And the number lives on in arguments made by advocacy groups, in uncorrected web pages, and who knows how many other places.

For all that we talk about journalists being objective, I don’t know of many who would apply that strictly to issues such as the sexual exploitation of children. Journalists want their reporting to make a difference. The Fox story I’ve singled out ends with this: “If you or somebody you know is trapped in a sex trafficking situation, call the national human trafficking resource center hotline at 888-373-7888.” When the reporter replied to my email, she wrote “It’s important to spread awareness about this issue.”

Advocacy journalism has its place. But the “journalism” part has to get equal footing.

Last note: I sent a follow-up email to the reporter noting my objections and asking if she had anything further to add. I’ll update this post if I get a response.