(Please note: This piece originally appeared as a free article in the early June 2016 edition of The Measurement Advisor newsletter.)
This month’s Measurement Menace Award goes to all those companies who try to oversimplify communications measurement by cooking up undisclosed metrics and then hiding them in black boxes and mysterious algorithms. Although they claim to be making measurement simple, they actually complicate our world by touting a product that makes measurement more difficult.
Here are four of the worst offenders:
#1. 70 Secrets in a Black Box
Two years ago, Ogilvy Australia developed a proprietary measurement system that included “more than 70 different tools and techniques to measure PR campaigns.” As I said at the time:
“Imagine that: 70 tools in one giant black box. I wonder how many of them are in compliance with industry standards? I wonder how many offer results that can be verified and replicated? How many produce any sort of insight?
The world does not need 70 different measurement tools. Nor does it need another proprietary measurement system to bamboozle clients into spending more money with their PR agency.”
#2. Two Black Boxes Are Worst Than One
Another egregious example of this type of villain is the unholy union of Starcom MediaVest Group and ShareThis. Back in 2013 they used the dark magic of algorithms to blend two proprietary indices into a third “universal measure of social activity on the web.” …What could go wrong?
#3. Totality My Ass
Porter Novelli made the same mistake with its PR Sonar, claiming that it tracks “the totality of the conversation.” It may very well be a wonderful algorithm, but no, it does not track the conversations that go on in my living room, or in business class on a flight to London, or around a soccer field. It doesn’t track text messaging or private conversations that take place on Facebook. Nor does it track emails, direct messages on Twitter, or private messages on LinkedIn. What it does is track about 1% of the “totality” of conversations.
#4. The Dumbing Down of Social Media ROI
Those who try to dumb down social media ROI do themselves and the industry no favors. A particularly grievous example is from 2014, when Mia Iverson announced her formula for social media ROI using tweets and likes for the “R” in ROI. It wasn’t even remotely related to real ROI. She suggested that you take the “desired action” (likes or click-throughs) and divide by the total impressions. To quote: “…you can take your number of followers [impressions] and the number of click-throughs [desired actions] and calculate your ROI accordingly. Problem solved.” (Fun fact: She’s actually closer to calculating some sort of metric of engagement.)
The good news is that the industry is evolving and clients increasingly realize that anything that purports to measure “ROI” has to be customized to be accurate. ∞
