Five years on from the Barcelona Principles, how does the PR industry measure up when it comes to compliance?
By Katie Delahaye Paine—There’s a saying that most veteran gardeners know well: “First year they sleep, next year they creep, third year they leap.”
For some species, that’s a bit optimistic. Sure run-of-the-mill perennials follow those rules, but it takes at least five years for double pink lilacs or a Harry Lauder’s Walking Stick to go from leaf cutting to blooming bush, and there are some Smoke Bushes I propagated that have never smoked.
Barcelona Compliance: Sleep, Creep, or Leap?
As I look out on my garden today, exploding with color in some spots and still creeping in others, I’m reminded of the progress that the PR industry has made in compliance with the Barcelona Principles, along with adoption of the standards and other measurement tools that have been developed from them:
- Leaping—Since they were officially adopted five years ago, one of the Principles has definitely leapt. Principle #6, which began as the nebulous statement, “Social media can and should be measured,” by 2013 had blossomed into published standards for measuring social media, thanks to a cross-industry effort from The Conclave. Lots of people are now measuring their social media, and while not everyone is doing it perfectly, they’re doing it.
- Creeping—We have made creeping progress towards “Increasing the importance of goal setting” (Principle #1), and “Measuring quality not quantity” (Principle #4). The Valid Metrics Framework is helping with these. There is no doubt that more organizations (even granting that mine is a self-selecting audience) that are starting their measurement programs with SMART objectives and a focus on measuring outcomes, not just activities (Principle #2). The advent of social media analytics has also pushed the industry closer to measuring communications’ impact on business results (Principle #3). But we still have a long way to go before PR people routinely incorporate business results into their measurement programs. (If they were doing so, I wouldn’t have had to write this eBook: 50 Shades of PR ROI.)
- Sleeping—Sadly, much of the profession is still asleep in its AVE coma (Principle #5), and they don’t even know what replicability and transparency are (Principle #7).
There are many who thought that just getting the Principles adopted constituted success. And it certainly was, if only a successful first step. (Read the little-known story behind the Principles’ development here: “The Secret Backstory of the Barcelona Principles”) Their adoption received wide and congratulatory coverage both in the U.S. and in Europe, helped by the broad coalition of organizations that endorsed them. Gosh, it probably got millions of impressions, so for those troglodytes that are still using AVEs, they’d say it was worth a gazillion dollars.
But the truth is that The Principles, and the standards and other tools that have grown from them, are still fighting for recognition and acceptance. As an example, notice that (to date) only 110 companies and organizations have pledged to support the PR research standards published by The Coalition.
Measuring PR’s progress on Barcelona Compliance
Let’s evaluate the progress of the PR measurement industry in a way that is compliant with those very principles. Ironically, there aren’t any consistent surveys in place to actually get all the required metrics (IPR, USC Annenberg, are you listening?). But digging around in various studies, I did find the following (using a modified version of AMEC’s Valid Metrics Framework).
| The question you want to answer: | The metric to answer it: | How to calculate the metric: |
|---|---|---|
| 1. How engaging is your content? | Engagement rate | For a given time period, take the total number of engagements (shares + likes + comments) generated and divide by the total number of posts that generated those engagements. Engagement Rate = Total Engagements / Total Posts |
| 2. To what extent is your content becoming more engaging over time? | Change in engagement rate (see above) | Calculate engagement rate (see above), weekly, or monthly, and then compare. |
| 3. How engaged is your audience? | Percentage audience engagement: The percentage of the people who follow you on Twitter or like you on Facebook or are part of your group on LinkedIn that are actually engaging with your content. | Divide the number of followers who engage with your content by the total number of followers. Percentage Engagement = # Of Followers Who Engage / Total Number Of Followers |
| 4. To what extent is your audience growing more engaged with your content over time? | Change in percentage audience engagement (see above) over time | Calculate percentage audience engagement (see above), monthly, or weekly, and then compare. |
| 5. As measured by engagement, what was the most effective piece of content we published last month? | Engagement rate per piece of content | Track the total number of engagements for each hashtag or piece of content. Rank them from highest to lowest. If appropriate you can give extra weight or a multiplier for desirable comments that leave the reader more likely to do business with or support your organization. If comments are strongly undesirable or there is a large number of both desirable and undesirable comments, you may want to calculate the net ratio between positive and negative to compare change over time. |
| 6. How effective is social at driving traffic and/or revenue? | Percent conversions from social | Make sure you’ve set up conversions in Google Analytics, Omniture, or whatever you are using. Then track the percentage of all conversions that come from social (in Google Analytics you can find it under Conversions –Multi Channel Funnels). |
| 7. What was the most cost-effective campaign in terms of engagement? | Cost per engagement (CPE) by campaign | For each campaign, take the total all-in costs of the campaign and divide it by the total number of engagements per campaign. This is the CPE for that campaign. Now compare CPE between campaigns. |
| 8. What was the most cost effective campaign in terms of driving conversions? | Cost per conversion (CPC) by campaign | For each campaign, take the total all-in costs of the campaign and divide it by the total number of conversions per campaign. This is the CPC for that campaign. Now compare CPC between campaigns. |
| 9. Which was better: paid social ads or organic social outreach? | Cost per conversion (see above) | Calculate cost per conversion (see above) for both and compare. |
As a profession we’re not yet following our own rules. We feel strongly that this should change. Read “The Barcelona Principles Compliance Manifesto” to find out just what our industry needs to do.
Overall, I’d sum up by saying that most of the industry is creeping towards Barcelona Principles compliance. Most organizations are still blocked by old school thinkers that refuse to adapt their ways. But they are slowly changing as the old guard is forced out by the digerati who take their budgets and slap metrics on everything. Others fight their way through the jungle of bad metrics and bad data that still plagues the industry, obstructed by vendors who can’t sever their financial addiction to AVEs and other dubious calculations. Even those that are in an AVE coma are slowly but surely being dragged into modern metrics, thanks to social media and the implausible nature of putting values on column inches in a digital and social environment. Most vendors are still measuring the way they used to. Sadly, while the Barcelona Principles raised the visibility of the need for these elements in measurement, few PR agencies and fewer vendor have done anything to make measurement much better. Even the sponsors of this year’s AMEC Measurement Summit have not, in general, pledged to support the new measurement standards (read “Companies Bring Shame to Measurement”).
But the very good news is that there are plenty that have taken the leap into goal-driven, quality measurement of what really matters to their business. Southwest Airlines, NATO, General Motors, McDonald’s, Atlantic City Alliance, AbbVie, and the National Wildlife Federation are just a few that have done the work to make their measurement far more meaningful. (To read about more successful transitions to Barcelona compliant measurement, read our posts “Crawling to Barcelona Compliance, Parts 1, 2 and 3.”) We know there are dozens of other measurement heroes out there, and we’d love to add you to this list. Let me know.)
In the end, I give the industry a solid C+. We still focus much too much on outputs and silly things like likes and impressions, without stopping to really question whether our work is impacting the business or the mission of the organization. But we have made progress since that June day in Barcelona. I’d say the garden is coming along nicely. ∞
