Stop Searching for ROI: Cost-Effectiveness Analysis Is What You Need

From the Desk of Katie Delahaye Paine

I love search-and-replace. What a simple, brilliant tool for quickly revising everything from new proposals to custom methodologies. I especially love the search-and-replace font feature in PowerPoint. With just a couple of clicks you can correct every wrong font in a 60-page PowerPoint presentation.

I really, really, really wish we could do search-and-replace for the entire Internet. Or at least for the minds of all the writers and self-styled social media and PR experts out there who continuously confuse ROI with cost-effectiveness analysis. ROI is an accounting equation, but because it has so much Google juice, everybody and his brother use it in their headlines.

I know this because, just to ensure that I start my day off with a bit of agita, I have a Google Alert set up for “PR ROI” and “social media ROI.” Invariably, 50 percent of what shows up has little (or nothing) to do with actual ROI. Rather, it talks about ways to justify one’s existence. These posts are meant to be read by PR and social media people who who are allergic to math, because anyone who can do math or understands rudimentary accounting would realize that the advice is pure BS.

Don’t get me wrong, I like and respect many of the people who write these posts. They just don’t understand what ROI is.

Take Scott Severson, CEO of Brandpoint, a content marketing agency (who, in full disclosure, I used to party with in my Silicon Valley days). His piece earlier this week on how to measure content marketing got my blood boiling before I’d finished my first cup of coffee.

In fairness, Severson made many excellent points about how to evaluate the effectiveness of your content marketing by making comparisons to other tactics. He just wasn’t using the right language.

His headline promised “Content Marketing ROI,” but what he was calling ROI was in fact a cost-effectiveness comparison. What would it have cost you to buy clicks rather than earning them with your content marketing? Certainly a useful way to justify one’s existence, but not ROI.

Severson based his premise on an idea of one of my social media idols, David Meerman Scott (his book The New Rules of Marketing & PR was my bible when I first got into social media). In this piece Scott postulates that one should not consider inbound marketing efforts (e.g., content marketing) as an expense, but rather as an investment, the way you would consider purchasing Google AdWords. Nothing in his theory actually measures return vs. investment, but in principle Scott wasn’t entirely off base with his suggestion. He advised comparing the cost of  purchasing Google AdWords to the cost of creating the content that earns you an equivalent search ranking. That’s cost-effectiveness analysis, and can be a very handy technique. It is not, however, ROI.

I’ve even done it myself: Back in 2009 I was spending some $2,000 a month using Google AdWords to generate leads and traffic to our website. I stopped the campaign and upped the amount of organic content I was posting to Twitter and our blog. After 3 months I calculated the difference in leads and traffic. There was no difference whatsoever, so I suppose that the “value” of the content I was generating was about $2,000 a month (if I didn’t factor in the cost of my time to create the content).

Scott goes on to suggest that people calculate “…AdWords Value Equivalency, which is sort of similar to the traditional PR metric of Advertising Value Equivalency (AVE).” Which is a bit confusing. Cost-effectiveness comparison and AVE are definitely not the same thing. Comparing the cost of doing content marketing against the cost of generating the same results with AdWords is a cost-effectiveness comparison. AVE, on the other hand, uses the cost of advertising space, without regard to whatever content was produced. Which exposes the entire fallacy of advertising equivalency: No one would actually pay for most of the articles that appeared in the space that goes into an AVE calculation. As every regular reader of this space will know, AVE is not a metric any PR professional should recommend.

So, let’s review:

Return on Investment (ROI) is an accounting calculation. ROI is the difference between the return (R) and the investment (I) required to achieve that return, divided by the investment. Here it is as a formula: ROI = (R- I) / (I), where (R) and (I) are amounts of money, like dollars or euros, and the result, ROI, is expressed as a percentage. ROI can only rarely be applied to either social media or PR, because it’s very difficult to accurately attribute dollar values to the revenue and the investment. Everyone in the C-suite loves to hear ROI, but they hate to hear the term ROI misused.

Cost-effectiveness analysis (CEA) compares the relative costs and outcomes (effects) of two or more courses of action. CEA (not to be confused with cost-benefit analysis, which only works if you can actually calculate a bottom line benefit), is what most people are referring to when they think they’re doing or talking about ROI.

And in fact, most of the calculations purported to be ROI aren’t even comparing outcomes, they’re comparing outputs or activities. A click on a link is not an outcome unless you’re an online retailer that can actually calculate an outcome, e.g., a sale or donation from that click. An article, post, or clip isn’t an outcome either, unless you can tie it to some tangible business value. Consumer companies like Procter & Gamble, Disney, and a few non-profits know the value of a post because they have decades of data that can accurately correlate clicks or clips to actual sales or donations. AT&T found that PR was 5-7 times more cost-effective than paid media.

If you want to explore all of this in greater depth, Fraser Likely has written a great paper on this topic. ∞

(Thanks to Dave Ramsey for the image.)

Paine Publishing provides communications, PR, and social media measurement training. Katie Delahaye Paine, founder of Paine Publishing, provides social media measurement consulting and customized measurement services.