by Katie Delahaye Paine—In my first 25 years of measuring communications, I’ve learned a thing or two ’bout birthin’ measurement programs (to fracture a quote from Thelma McQueen in “Gone with the Wind”). Especially today, having now helped a dozen companies move away from AVEs to Barcelona-Compliant measurement programs that deliver results that reflect business goals. It’s an exhilarating process but not without its challenges. One of the biggest is surviving the first few months of a new program. The problem is that once you’ve weathered the RFP and vendor selection process, you’ll probably have high expectations to see your data roll in and the insights roll out. Well, I hate to burst your bubble, but it’s not as easy as that. If you are going to survive and even prosper during the several-month process of getting your program running smoothly, there are certain common pitfalls and unrealistic expectations you need to look out for.
Here are my Top 5 Unrealistic Expectations For Starting A Measurement Program:
Expectation #1:
“I’ll get my magic number!”
You expect to be presented a “measurement number” that will justify your budget and answer all questions the moment you implement a new tool. It’s not going to happen.
- Reality: Measurement isn’t a single number, so stop searching for unicorns.
Your CFO doesn’t go into the Board of Directors meeting armed only with your stock price or your sales for the quarter. He or she probably presents metrics showing profit, customer satisfaction, reputation, as well as sales growth and possibly even employee retention—all things that will impact the organization’s future success.
So why would Communications, made up of such a myriad of different components, think it can demonstrate results with one number? The best thing you can do is to agree upon three to five metrics that reflect the totality of your department’s efforts on the organization’s goals. An Optimal Content Score or Social Engagement Index are good places to start because they force you and your leadership team to agree on the key factors of both media relations and social media that impact the business. - Reality: Tools do not measure success.
Tools provide data, and if you buy the right one, you might actually get the right data to demonstrate your success. Do not expect a tool to provide you with instant measurement—no matter what the smooth salesperson said. - Reality: Measurement is a long-term process.
It will take you time to identify changes and trends. Your manufacturing VP doesn’t report on the productivity for today, he or she reports on changes in productivity month-to-month, quarter-to-quarter, and year-to-year. That’s why you need to set expectations correctly. It will take 6 to 9 months after your program is fully up and running before you can demonstrate trends.
Expectation #2:
“My results are on the way!”
- Reality: It will take time to get your ducks in a row.
Unless you know how to do a Vulcan mind meld, you will need to spend time with your measurement vendor’s staff to very clearly define the parameters of your program. You will need to define subjects, messages, campaigns, and any other details you will be tracking. If you somehow give your vendor the wrong messages, incorrect competitors, or out-of-date product names, then it’s up to you to discover it and tell them. If it’s on your website, they will assume it to be current. And if it’s not on your website and you haven’t told them about it, how will they know it exists? - Reality: A single month does not results make.
We know that everyone just can’t wait to see those pretty charts and graphs the vendor promised, but hold your horses. There’s a logical timeline, and it does not begin with yesterday and end with day after tomorrow. You will need to check and double check the data (see how to do a validity test) before showing it to anyone. On top of that, a month’s worth of data looks pretty silly on a chart because you have nothing to compare it to. Even if you’re just measuring one event, it’s always more useful if you can compare results with other events or prior years. Bottom line, one month’s data does not a terrific report make. Good reporting takes time.
Expectation #3:
“Now that I’ve sorted out my vendors, everything will be easy!”
- Reality: Not.
You’ve just gone through an incredibly complicated screening process to find the right vendor, and now you expect everything will be simple and easy? Don’t expect to sign the purchase order and have your metrics magically appear. It doesn’t work that way. There is set-up time, verification, and testing for any tool. No matter how simple or sophisticated a tool is, it will always take you at least eight weeks, and likely more, to get consistent and useful data. Between set up, training, and validation, you’ll be lucky to get your results by the end of the quarter.
Expectation #4:
“I’m spending a ton of money, so my data will be 100% perfect from day one!”
- Reality: Chances are your data is dirty, dirty, dirty.
The more data there is, the greater the possibility that some of it is wrong. Expect glitches in your data, especially to start. Computers, like children, take a while to learn the rules, and without rules they’ll spend a lot of time getting dirty. For the first few months you are going to be sorting out search terms and media outlet specifics, and you and your computer(s) are going to make some mistakes:- At first you will probably define search terms that are far too broad, and you will be inundated with mentions that have nothing to do with your organization(s). We did a recent study of Google’s Calico project. While I expected a large number of calico cat videos, I had no idea there were that many varieties of calico fabric for quilts. Or ask the folks at SAS—the analytics software company, not the airline or the Semester at Sea folks—how many “exclude” terms they have. Last I checked it was around 2000!
- Beware stock ticker symbols. Twitter turns up some bizarre abbreviations. Turns out that the ticker symbol for Astra Zeneca (AZN) is an abbreviation for Asian on Twitter and turns up some really nasty stuff. So unless you want to spend your days reading bizarre tweets, you need to be very careful about your search strings.
- Assume it will take at least three months to get it right. You can shorten this cycle if you chose to work with a very clearly defined list of media outlets, defined by specific URLs. Which means not, for instance, asking for “Forbes.” Which of the 100+ URLs did you have in mind?
Expectation #5:
“My first report will explain everything and make me a hero!”
- Reality: Your first report will be confusing.
No matter how clear the program seems to be, your first report is going to throw you for a loop. There will be charts that look weird and use language you may not understand. Until you see it visually up there on the screen, you probably won’t understand what you’ve ordered. That is the nature of data. And remember, some of your early data will probably not be what you expected. Your job is to translate that confusion into something easily digested by senior leadership. - Reality: You will be pecked to death by questions.
When seeing something new, people will naturally pepper you with questions. They will question your data and the methodology, so be prepared. Imagine every question you can possibly be asked and make sure you have an answer. If you do, you’ll come out a hero and no one will ever question you again. ∞
