The Columbian uses A/B testing to measure the effectiveness of color, layout and other visual elements, among other things, on columbian.com. (The Columbian)
What do newspaper subscribers actually want to read?
Fifty years ago, the answer was simple: Who cares? They would read what the editor wanted to read. If the editor was interested in city government news, that’s what readers got. If the editor liked politics, readers would get their daily dose.
That doesn’t hold true nowadays, which explains why The Columbian has a full page of comics every day instead of a full page of cruise ship news.
In fact, we are going about as far away from the traditional editor’s instinct as possible, with sophisticated new online tools that measure what readers look at, and, perhaps more importantly, what stories lead readers down the path to buy a subscription. Which story finally convinced them to give us their credit card number?
We also have some ways to measure how readers react to the design of our web page, or the user experience. Here’s an example.
Our home page, www.columbian.com, has a blue “subscribe” button at the top that is displayed to readers who are not logged in to an account. If they click, they are redirected to the subscriptions page. But would more people subscribe if the button were green? Not long ago we tried what is called an A/B test, where a random group of customers saw a blue button and another group saw green.
Alas, there was no distinct preference. We tried a bunch of other colors — red, orange, etc. After more than 2 million tests, the results still showed no preference. So we left it blue.
OK, maybe that wasn’t a great example.
We haven’t tried any A/B tests in the newsroom. But our new analytic tools have been watching online readers for several months now, recording their behavior.
The builders of these tools say it is important to let the database grow before we start using it to make decisions, so we’ll be taking very slow steps as we learn to analyze and use this data to craft better stories.
So far I haven’t seen any real surprises. Columbian subscribers love their local news, local sports and information about local events. They are less keen on health news, which I think is probably related to the weariness over the COVID-19 pandemic and all of the politics surrounding vaccines and wearing of masks.
Innovation Editor Will Campbell and Web Editor Amy Libby recently had a chance to drill a little bit deeper into the data. The stories that seemed to be the biggest hits with readers were those that had an unusual element. To make up an example, if a candle causes a house fire, it’s not likely to motivate readers to subscribe. But if we wrote a story about how pets knocked over candles and started 20 house fires last year, it might prompt readers to subscribe.
It’s going to be a challenge to understand all of this data. But the bigger challenge, I think, will be to use it wisely.
A few years ago, we were solicited to buy software used by TV stations and some newspapers that measures clicks on stories. An editor could use the information to tell which reporters wrote the most popular stories. By extension, those reporters could be rewarded, and the “losers” could be punished. Furthermore, the “best” stories might be what we call “clickbait,” sensational articles that fail to provide depth or context.
We didn’t buy that software, and I would never be part of an organization that operates that way. We want to use data to assist our employees and serve our subscribers.
I am fine with this approach to using data. Giving customers what they want and need is a tenet of a free market economy. But journalists need to serve them within an ethical framework, making sure the stories are accurate and worth telling. From what I can tell so far, this is exactly what our paying customers want, too. But I still wish we had a daily page of cruise ship news.
Morning Briefing Newsletter
Get a rundown of the latest local and regional news every Mon-Fri morning.
Support local journalism
Your tax-deductible donation to The Columbian’s Community Funded Journalism program will contribute to better local reporting on key issues, including homelessness, housing, transportation and the environment. Reporters will focus on narrative, investigative and data-driven storytelling.
Local journalism needs your help. It’s an essential part of a healthy community and a healthy democracy.
