The Five W’s & How: Applying them to an individual fact

St. Catherine is beheaded (Image taken at Unterlinden Museum, Colmar, France by John Kroll)

The Five W’s (who, what, when, where, why) and How are journalism’s double trinity. They’re generally applied to whole stories, as Jeremy Porter notes on the Journalistics blog. But they’re also a key to fact-checking, especially when you’re reporting on statistics dropped into speeches or such. An excellent example of that is a BBC News Magazine report on the claim that 100,000 Christians a year are killed because of their religion. That figure’s even been cited by the Vatican.

It’s the kind of number that should immediately raise suspicions in a journalist: How could anyone count?

The BBC’s Ruth Alexander took a closer look. The way she did it is instructive for all reporters.

Who: She traced the number back to its original source, an annual report by the Center for the Study of Global Christianity. Numbers like this are often dropped into other reports without attribution. The Vatican, for one, said only that the 100,000 figure came from “credible research.” Often, as a number gets passed from one telling to another, the original source is replaced with whichever repeater sounds most believable. Thanks to the Internet, it’s relatively easy today to track back. Only by getting to the original source can you reliably find answers to the other questions.

What: Once you’ve found the original source, you want to know what was actually said. As Alexander’s story notes, some variations on the martyrdom number include claims that the 100,000 counts only Christians killed by Muslims. That’s not what the original report said.

When: Numbers are often stripped of their context. In particular, they can be years old. A reporter has to ask: Is this still true? The CSGC’s number is an average of deaths from 2001 to 2010. The group’s own director concedes that a unique factor concerning that period means the number is already well out of date.

So it’s probably decreasing year by year right now, but the method is not exact enough to [make those adjustments], so I’ve just kept it at 100,000 the last couple of years but I’m likely going to have to lower it unless something comes to our attention.

Where: Location can be another contextual issue. Is a number meant to refer only to a limited area being applied globally? In this case, that’s not the case; the original source did indeed report the 100,000 deaths as a worldwide total. But there is “where” issue: The number is reported as a worldwide total, which implies it’s a widespread issue. But about nine out of every 10 of those deaths occurred in one country, the Democratic Republic of the Congo.

In a broader sense of “where,” this is a common problem in health reporting. Studies with tiny sample sizes get reported without proper caveats. Psychology experiments are touted as global truths when it turns out the people studied were limited to students at an elite college.

Why: Does the source possibly have motives that go beyond a simple quest for the truth? In the case of the martyrdom number, the center is affiliated with a seminary. The BBC doesn’t indicate any reason to suspect bias. However, the group’s data analysis is limited to studying Christianity. Citing a number for Christian martyrs raises questions about similar numbers for other faiths (and about the ratio of those numbers to total believers). Since the CSGC doesn’t tally figures for other faiths, however, trying to make comparisons could be difficult or impossible.

How: This is the broadest question, and also the most important. How was that number calculated? In this case, asking “how?” revealed that the number was completely misleading. First, it’s an average based on a rough guesstimate of 1 million deaths over the decade. Second, 90% of those deaths come from one country. Third, the number of deaths from that country is calculated by taking a reported total of all deaths in the DRC’s civil war — more than 4 million — and arbitrarily assuming that 20% of those who died were Christian (an assumption based on a 30-year-old estimate of the percentage of Christians in an average African country). Is the DRC average in this respect? Probably not — the DRC is a Christian country, the BBC reports. Does that 20% figure still apply? Was it ever correct, or just a figure spitballed by someone at its source, the World Christian Encyclopedia?

But that’s not all. Even if all of the guesses above were correct, that only says about 900,000 Christians lost their lives in the DRC’s violence. Were they martyrs? The BBC notes that experts say the DRC’s infighting was based on ethnicity, not religious affiliation. And it was Christians killing other Christians. That can still constitute martyrdom, of course; the Catholic Church has a long list of those it claims as martyrs at the hands of English Protestants, while the Protestants have a list of similar length of those killed by Catholic monarchs. But when you cite a figure of Christian martyrs, one naturally expects their killers were objecting to Christianity as a whole.

Bottom line, according to the BBC’s analysis and some outside experts: The actual number of Christians martyred for their faith each year is probably somewhere under 10,000, out of a worldwide total of believers that Pew Research estimates to be 2.18 billion.

The Five W’s and How are taught to every journalist. But too often, they’re asked only superficially — making sure, for instance, that a preview of a concert includes the date, time, location and price. If reporters really understood and applied the Five W’s and How rigorously, Steve Buttry wouldn’t need an entire workshop explaining them. (Correction: Steve points out that the workshop in question was aimed at nonjournalists. So, bad example for my argument — but his post is good explanation of the topic, worth reading.) If editors really insisted on applying them to stories, you’d never see stories treating polls on behalf of political candidates treated as seriously as those from nonpartisan groups, and you wouldn’t see polls reported solely based on what was included in a news release, without reference to the specific questions asked and the method by which participants were selected.

The sad fact is that even great journalists fall prey into the error of not letting the facts get in the way of a good story. The most disappointing element of the BBC story is its ending. Ruth Alexander talked to John Allen, a Vatican reporter whose work I much admire. He’s written a book which cites the misleading martyrdom number. When presented with the BBC’s findings, he said:

I think it would be good to have reliable figures on this issue, but I don’t think it ultimately matters in terms of the point of my book, which is to break through the narrative that tends to dominate discussion in the West – that Christians can’t be persecuted because they belong to the world’s most powerful church.

The truth is two thirds of the 2.3 billion Christians in the world today live… in dangerous neighbourhoods. They are often poor. They often belong to ethnic, linguistic and cultural minorities. And they are often at risk.

And ultimately I think making that point is more important than being precise about the death toll.

When a good journalist says that using a number that inflates a problem about 900% is acceptable imprecision, we know that the Five W’s and How are suffering their own martyrdom.