CJR flings another handful of poo at nola.com and Advance Digital

Bullseye shown before application of flung dung.

I’m assuming there’s a dartboard in the offices of the Columbia Journalism Review that has Advance Digital or one of its top bosses in the bullseye, because at least two of its staff — Ryan Chittum and Dean Starkman — seem to be determined to poke as many holes in the Newhouse company’s plans as possible (and then some).

In the latest example, Starkman consults on the set-up of a study of content at Advance’s New Orleans operations, then writes about the study and (surprise!) defends its findings and methodology. Starkman’s defense falls flat, and here’s why:

Homepage does not equal front page

The study (by students at Tulane University) sampled content from nola.com by counting only stories in “the large banner for the top story, and the following boxes below with anywhere from 3-10 stories per collection day.” Advance websites put a lot of focus on a steadily updated stream of headlines as well, which it calls the “river of news,” but the students ignored any headlines in the river “as they changed irregularly and lacked curation.”

For many years, under several website designs, I had a hand in selecting the stories that were featured on another Advance site, cleveland.com. The assumption that the river wholly lacks “curation” is questionable. While in theory each reporter posts his or her news independently and as soon as it’s ready, in practice it’s not that simple. Certainly, I worked with reporters and their bosses to manipulate posting times for various reasons, such as to avoid a stack-up of stories that would leave us without fresh timestamps later on. And I was always aware of what was in the river, and took that into account in deciding what stories would appear in the display boxes. Among other reasons, I knew from stats that stories in the river were about as likely as those in display boxes to draw the reader’s eye.

This is not your father’s newspaper

Jim Amoss of nola.com raised a similar objection, which Starkman notes. Amoss also pointed to content which didn’t appear on the home page. Starkman dismissed those concerns with this:

All content analyses have to restrict the scope of their research, and there is no better place to look for a news organization’s priorities than at its front page and its home page.

Of course, the study didn’t look at the whole home page; it looked at only a small subset of the headlines on it. On the other hand, I would tend to be leery of anyone trying to equate the exposure of stories on the home page with those that only appear on interior pages. Not that Amoss is wrong — I don’t know nola.com’s stats on views for interior pages — but in general, on a news site, traffic that doesn’t go directly to a particular story (via Google, social media links or whatever) is likely to come mostly via the home page and, perhaps, a few key sports pages.

However, there is an undercurrent to Starkman’s defense that needs to be addressed. That’s the assumption that there are close parallels between a print newspaper and its associated website in the way stories are chosen or readers engage, and the further assumption that what may be true of websites for more traditional companies is also true of Advance Digital.

Advance is very much trying, not to rebuild the newspaper online, but to create sites and news-gathering operations that are based on the digital environment. It’s convenient to refer to the homepage as the equivalent of the print front page. But at least in Cleveland, as we became more comfortable with the new tools and more aware of reader patterns, that’s not how we operated.

So Starkman’s pronouncement that it’s OK to compare the web homepage to the front page because you always have to make choices is like the guy the cop found looking for his keys under a lamppost. When the cop asks, the guy says he lost the keys in the next block. “Then why,” the cop inquires, “are you looking here?”

“Because the light’s better.”

Advance websites don’t have editions

Stories were collected online between midnight and 10 a.m. Starkman allows as how it might be useful to follow up with a study that is broader.

I say the time restriction is another gaping hole. Did I care what stories were featured on cleveland.com’s homepage between midnight and, say, 6 a.m.? Not one whit. Stats told us traffic was dead (except perhaps for West Coast Browns fans), and nothing we did would lure them out of bed. From 6 to 10 a.m., we’d have the morning rush — but if we’d done our job properly and posted the previous day’s news as it happened, what we had to post fresh in the early morning would include a lot of soft stuff — the weather, celebrity gossip, overnight sports. There’s a reason the “Today” show is not the same as the evening news.

I don’t know the flow of traffic at nola.com, or the flow of news in that city. But that’s the point: Neither, evidently, do Starkman or the Tulane students. A study designed without adequate understanding of the object to be studied is doomed from the start.

For the period covered, the study counted the “quantity of original news stories.” This actually found a positive for nola.com — a big increase compared with 2011. But as a veteran of Advance, I know that the definition of “news story” changed significantly over that time. What once was a single story will today likely be a set of posts. For example, a breaking news story that once might have generated only a final tell-it-all article — and still may look like that in print — will likely be a string of short posts online, each one hung on the latest developments. Features, investigations and all other content are likely to be broken down into more pieces and parts, too.

Features do not equal low quality

The study’s measure of the “quality” of the journalism is a count of “hard and soft news.”

I would not be surprised if there’s more of what the study’s parameters would label “soft news” on nola.com nowadays. And it may be true that the quality has declined. But the two statements don’t have anything to do with each other.

Again, Advance is not just tweaking the old model. It’s creating a new one, almost from scratch. And as Advance leaders have said publicly, that model has a lot to do with what people actually want to read online. Imagine that: It’s operating like a business that wants to attract more customers. I know this is a bizarre concept in an industry that has proved to be so adept at paring its customer base year by year. But gauging the “quality” of what Advance does by whether it fits an old definition of what a newspaper should be is pointless.

The study does include one gauge that starts to get at real issues: It counts the number of sources per story. I’d have suggested also:

• Assembling a short list of common grammar errors and counting those.
• Fact-checking stories and counting the mistakes (and also checking to see how many were publicly corrected).
• Presenting a selection of stories, with anything identifying their timeliness removed, to focus groups or journalism profs and having them rated on the basis of completeness and clarity. (This would still be tricky, given the fragmented nature of the online reporting now, but perhaps some adjustments and controls could be worked out.)

But none of these measures — including the source count — can be considered of much use unless the study’s underpinnings are far more solidly based on an understanding of how nola.com works and what it’s trying to accomplish. And any study that starts with the preconceptions of an old (and not entirely valid) newspaper model will fail when gauging what’s going on at Advance’s sites.

My gripe is not with Tulane. For a class project, this is a good effort, and I hope the students will learn a lot from its shortcomings. But CJR’s determination to dance on the grave of a business model that seems to still be learning to walk is, at best, curious.

Unlike Starkman and Chittum, I am not sure what the ultimate fate of the Advance model will be. No doubt that will depend at least as much on the people in charge throughout the company as it will on the model itself. I’m no Advance apologist; I left by my own choice last year. But having been on the inside for a long time, I understand what Advance is trying to do, and why. I hate the way some people have not only written this experiment off, but seem quite gleeful about anything that can be made to fit their beliefs.