How to turn website statistics into newsroom change

Site statstics can lead you to a problem, but analysis will direct you to the solution.

Once Stijn Debrouwere got through ridiculing the way the news industry uses website data, he did get to some positive approaches.

Here’s one weird trick I learned from Eric Ries. No, it’s actually more like a four-step program.

1. figure out what is important to your organization, what your goals are
2. think of a couple of ways in which you could move the needle on one of those goals, pick a project
3. assemble a team that will actually execute said project
4. then, and only then, think about a good metric the team can use to see whether they’re making progress.

And he translated that into newsroom terms:

1. Overarching goal: more loyal readers, reading more
2. Project: encourage readers to spend time on less prominent articles
3. Team: multiple designers, coders and testers working together
4. Metrics: pages per visit, pages per day, # of underperforming articles in a story package

Debrouwere adds what could be a fifth point: “Not only do we keep track of those metrics on an ongoing basis, many of the changes the next gen web team make are A/B tested. Those A/B tests compare a new or improved feature against the website as it currently is, by giving some people the new version and some people the old version.” That’s an excellent suggestion, although the pure A/B test he describes — manipulating your site so that equal numbers of users get different versions — may not be feasible, or even possible, for you. What is possible is applying different approaches to different posts or projects. While the comparison then would be subject to more variables, if you’re doing online journalism right, you’re posting a lot and will have a chance to average several experiments.

This from Debrouwere is in the context of criticizing the use of stats dashboards that give an overview of site activity. But those dashboards — and just noodling around with deeper site stats, without a goal in mind at first — are useful. First, because they get stats-phobic editors and reporters used to seeing numbers about traffic — a paradigm shift for newsrooms. Second, digging around in the data can lead to the kind of what-if questions that Debrouwere encourages us to pursue.

I’m in favor of anything that will get editors and reporters paying attention to stats, including putting those live dashboards up on screens around the newsroom. That’s no different from posting databases online and inviting readers to help us spot intriguing entries.

Debrouwere is correct that just noodling around and asking questions isn’t enough. That’s where it becomes important to have people in the newsroom who will do the deeper digging. (One of the useful innovations of Advance Digital’s approach is the creation of jobs called “content data analysts” in its newsrooms. Guess what they do?)

Finally, what he highlights as a goal — encouraging readers to spend more time on less prominent articles — is something all newsrooms should be keying on. Like all newspapers, The Plain Dealer took pride in the large projects it undertook. The kind that show up in your Sunday paper bulging with sidebars, infographics, lists, profiles, and more sidebars. Very impressive. But when we looked at traffic numbers, we could see real problems:

1. Sunday traffic was weak overall; dumping projects then produced meager numbers.

2. Dumping all the elements of a project at once meant very weak readership for anything beyond the main story and/or an index page.

3. Using a preview story or video on a weekday before a big project would do well — for the preview; it didn’t pay off in much improvement for the project itself.

Looking at those facts, and noodling around with site stats in general, I became convinced that the print concept of a project needed to be blown up and rebuilt for online. That’s why I’m hoping Advance pushes ahead and refines what it’s pitching to newsrooms as iterative journalism.

The basic idea: Time on the homepage equals prominence, but readers expect the site to change over the course of a day. So:

1. Create multiple posts. This goes beyond what you’d do in print to include things such as an aggregation post of other coverage, a post of reaction to the project so far, polls, and so on.

2. Spread the posts over time to keep the topic on the homepage. Each new headline is not only an opportunity to attract readers who hadn’t noticed the project before, but also to get readers familiar with it to return and read more than they would if everything appeared at once. (Note that this doesn’t mean just dumping an installment every day at 6 a.m.; it includes spreading posts out so that they’re freshest at different high points of overall site traffic.)

3. Let a story develop across multiple posts. This was the hardest sell in our newsroom, and I left with that job still undone. It is OK to wait until you’ve got everything nailed down, just as you would with a print project. But it’s also OK — and possibly better — to post what you know when you know it, even to encourage users to help you think of questions to pursue or suggest angles you’ve missed.

4. Don’t let a project end. In print, most projects are finite: three days and we’re done, on to the next subject. Online, though, that content is still available, and squeezing all the readership juice out of older stories is an important part of building traffic and loyalty. Merely reposting the same content in a highlights promo won’t be as effective (I’ve seen it done, and seen it fail) as posting new stories or other elements and then linking back to the originals. Fresh is what works best online.

What this all means: The use of site data should be a continuum. Get the broad numbers out there. Use them to get your newsroom asking questions. Then pursue those questions, with the kind of targeted, scientific approaches that Debrouwere advocates.

This series of posts began with a defense of stats dashboards.

Part 2: Why journalists should love live stats.