This is the first in a Measurement Advisor series of true stories about organizations struggling to implement standards-compliant measurement systems.
A prospective client came to me one day and laid out 30 separate measurement reports that her organization had been paying for. She had just taken over responsibility for public relations and social media measurement, had heard about The Barcelona Principles and the new PR Measurement Standards, and was looking for a way to bring some sanity to the company’s metrics. The need was particularly great since none of the reports used the same metrics, definitions, or criteria, and none were being used to make decisions. She knew she had a golden opportunity to improve measurement efficiency and become standards-compliant at the same time.
This client is not alone. Many organizations are faced with a confusing and inefficient hodgepodge of multiple measurement vendors and systems. That situation was the motivation behind The Institute for Public Relations and The Conclave to create standard terms and methodologies for measurement. The challenge is that while clients envision a simple, easy-to-use platform that provides consistent metrics at the click of a mouse, such systems, like unicorns, only occur in fairy tales.
The ugly reality is that the journey to a dashboard upgrade is full of dragons and ogres in the form of vendors, entrenched structures, and old habits and beliefs. And it takes a lot of white knights on horseback—or women with magic wands wielding research and standards documents—to banish those beasts.
Which is why, in the next few editions of The Measurement Advisor, we’ll be bringing you true tales from the trenches of standardization. Stories of real people, working for organizations you’d recognize, and their struggles to implement standards-compliant, effective measurement programs. WARNING: Not all of these stories have a happy ending, so we’ve disguised the identity of the players.
In this first chapter, we’ll talk about the hardest part — the process of defining success in a way that relates to business objectives.
At the very core of the new standards and The Barcelona Principles is the principle of tying efforts to outcomes. Few organizations find it hard to argue with the idea of tying their communications efforts to corporate goals. But it’s often difficult to convince them to give up AVEs and counting “likes.” And it’s even harder to get sufficient face time with senior management to hammer out a consensus about what success looks like in terms of business results. The larger the organization the bigger the challenge it becomes. Read on.
#1: PR is the Spark
At one very large destination marketing organization we met with the CMO, a very research-minded manager. This manager had ROI statistics on everything in his organization except PR and social media. After he vented his frustrations about the bogus numbers PR had been providing, I asked him what he thought PR’s role was in the visitor’s path to purchase a ticket to visit. He thought for a minute, then answered: “It’s the spark,” he said, “The thing that gets them to consider coming.”
That insight was just what we needed. In short order we used existing research to identify what specific elements had to be included in a story to light that spark. Those elements had to:
- Leave a reader more likely to consider visiting
- Contain a desirable visual
- Offer a recommendation
- Dispel a myth
So, the first definition of success would be the percentage of coverage in earned media (social as well as traditional) containing those elements. We then developed an “optimal content score” the organization could use in its marketing mix model to determine earned media’s contribution to the departmental ROI. Unfortunately, the CMO left soon after those decisions were made, and, in the subsequent power vacuum that followed, the organization fell back under the sway of the wicked witch who inserted AVEs as a requirement on their next RFP.
#2: A Travel Destination Defines Proxies for Intent to Visit
For another travel destination, the story had a much happier outcome. The initial process of defining success was simple, partly because they had a communications staff of less than 10 people. We were able to get the executive director in the same room with the staff and a few of the agencies. Our discussion was productive, and we agreed that getting traffic to the “Visitor Guide” page on the website was an acceptable proxy for intent to visit.
We arrived at the second proxy through survey research. They already used quarterly surveys to measure actual intent, awareness, and preference. So we inserted a question to identify people who had heard news via earned media. We were then able to track the extent to which people who had seen destination-related news in key markets were influenced to perceive the city as a fun place to go with friends – a key driver of actual visits.
Two years later, the metrics proved the value of PR in the face of huge budget cuts. A comparison of cost effectiveness between an expensive advertising campaign and PR showed PR as the clear winner. As a result, paid ads were cut while the PR budget remained intact.
#3: A Nonprofit Defines Proof of Donations
For a nonprofit organization, we agreed that key metrics had to involve expanding the membership base and increasing revenue. Unfortunately, we couldn’t get the actual membership and revenue data. However, after much discussion we agreed that unique sessions for the “Join” and “Volunteer” pages, combined with sessions of the post-donation “Thank You” page, was the metric we needed. Using those numbers the organization could easily show the relative effectiveness of social outreach vs. events vs. media relations.
#4: A Large Government Organization Redefined “Positive”
For large government institutions these days the possibility of getting positive coverage is about as likely as Red Riding Hood eloping with the wolf. For one controversial international institution, there was significant debate over what constituted “positive” coverage. They ended up dropping the term altogether and defined “desirable” coverage as content which left the reader less likely to oppose the institution.
#5: A Major Health Care Company Chooses 4 Communications Goals (and Metrics to Go With Them)
At a major health care company, the metrics started with a strategic plan that outlined 10 imperatives for the year. (I know, ten is way too many, but we arrived too late to debate that decision.) In a brilliant and highly pragmatic move, the new head of communications operations decided that his department could measurably contribute to 4 of those imperatives:
- Increase brand awareness & familiarity. This would be measured by increase in desirable share of voice in both traditional & social media, and increase in new visits to the company website.
- Increase awareness & brand reputation index among key stakeholders (from a global survey of key stakeholders).
- Protect the brand. This would be measured by decrease in undesirable share of voice in traditional & social media, and expansion of the marketable universe via an increase in desirable social media engagement.
- Promote the culture. This would be measured by the % increase in employee knowledge of corporate strategy & culture, as demonstrated by employee quizzes.
The final metrics used on the dashboard were:
- % decrease in those who are unaware of the brand among key stakeholders
- % increase in awareness of brand positioning among key stakeholders
- % increase in non-employee engagement with online brand properties and networks
- % reduction in share of undesirable voice
- % increase in share of desirable voice
#6 High-end Retailer Defines the Perfect Media Placement
For a major high-end retailer, the meeting to define metrics was crowded, contentious, and took an entire day. In the end, the only outcome metrics agreed on were media–related. “Success” was defined as the percentage of items that contained a desirable visual, a key message, and a call to action. Two years later they remain AVE free, and are now expanding their metrics to include their social outreach program.
There are three clear lessons that cut across all types of organizations:
- Get consensus up front between external relations and the CMO, CEO, or highest ranking person who will see your reports.
- Don’t let a desire for perfection come between you and a great measurement dashboard…You may not always be able to measure exactly what you want, but many times you can find an acceptable proxy.
- Keep it simple. No one can keep track of more than 3-5 numbers anyway.
Coming next month:
The next big challenge…Finding the data and making it consistent & accurate. 
