“Impact” is the new “engagement.” It’s a catchphrase that represents something that everyone seems to want but few are clear on the definition. What is clear is that some kind of new measure of impact is needed.
The old measures no longer work
The current interest in measuring impact stems from the growing dissatisfaction with impressions as a way to gauge anything valuable. It used to be that we equated impressions with the actual circulation of a publication; it was just easy to assume that everyone who subscribed to a publication read all of it, all the time. This assumption was of course ridiculous, but it didn’t seem to matter.
Then, when digital and social media came along, clicks became the new impressions. It was easy to assume that everyone who clicked read all of a page, all of the time. Numbers got a lot bigger. Management got happier because big numbers got them bigger pay checks. The notion that those big numbers actually reflected the actions of billions of consumers was, of course, also ridiculous, but until recently that didn’t seem to matter either.
Today, with the revelations of Russian troll factories, bots that auto-respond to everything, and the inflation of response numbers by Facebook and YouTube, the old impressions assumptions have lost all credibility.
Chip Giller, founder and CEO of Grist and Katharine Wroth, manager of special projects at Grist, capture the problem nicely:
“Yes, it’s vital to know how many people a news outlet is reaching, and which links those people are clicking. But other questions are equally important, if not more so: Are people actually reading or watching the content that they access? Are they sharing or commenting on it? Does their engagement with the content spur offline conversation and action? Does that engagement ultimately lead to shifts in public opinion or policy? The answers to those questions are much harder to determine, but they are essential to understanding the impact of a media organization.”
A research report by the Media Impact Project reveals just how flawed the systems we use every day are. While we may define an “article” as one thing, media outlets use very different definitions. Some contain so many irregularities that they return inaccurate results or occasionally break automated systems. (Anyone who has had to clean up a media database in the last five years is sighing in sympathy.) For example, they found that a story that was updated multiple times or is in a different language may have a different page title but the same URL. And if your system is counting on unique URLs you will end up either under-counting or over-counting.
In search of the impact of a post
Which is why we need an improved way to calculate the impact of any given article or post. A lot of people are on the hunt:
- Regina Conway wrote recently in Forbes of her organization’s successful efforts to develop a placement quality score. Many aspects of which are similar to the 5-step process presented below.
- AirPR was one of the first firms in the industry to try to get to a better number than impressions. They use what they call their Relevance, Resonance and Reputation model that looks at the reputation of the publication, the relevancy of the outlet to your business, and the degree to which the content “resonates”—i.e., is shared with others. If you can afford to hire a media measurement company, I strongly suggest you look into their model.
- Cision’s Impact score leverages the company’s relationship with publishers and websites to provide clients with detailed data from the media outlets themselves to create what they call “total true reach.” It is calculated by multiplying the average reads per article by the number of articles. Like many monitoring companies today, they also provide the number of shares (aka “amplification”) which presumably could be added to the true reach calculation.
- Evaluating media impact is a hot topic in academic circles these days. The above-mentioned Media Impact Project, created by Knight Foundation and the Bill & Melinda Gates Foundation in 2013, is part of the USC Annenberg Norman Lear Center. It exists to measure the impact media has on audiences. It has funded a number of interesting efforts you can find here.
- Similarly, Stanford’s Social Innovation Review delved into the issue of measuring media impact, pointing out that traditional media metrics like unique visits and time on site analytics are seriously flawed. “Time on site is especially problematic; a better term for it might be ‘time on site while intending to read an article but wandering away to put the kettle on, then taking a phone call from Aunt Midge, then—wait, what was I doing?’ ”
But even with all that brain power, there is very little clarity on what “impact” really means.
5 steps to developing your customized impact index
The goal for your index score is to better understand which activities/campaigns/programs and content you’ve created—be it blog posts, press releases, media placements or media tours—had the most impact on your audience. Thus, by necessity, there are a lot of moving parts in that answer. But if your goal is to get earned media it all boils down to four factors:
- The quality and credibility of the media outlet or author that is covering your organization
- The level of influence that a media outlet or author has on your stakeholders
- The quality of the story itself: In other words does it include the requisite elements that will make people pay attention and remember your message?
- The relevance of the subject to your audience
Which is why there isn’t one single definition of impact, just as their isn’t a single definition of what is “good press.” Each organization needs to define “impact” relative to their own stakeholders, audiences, and mission.
And then there’s the expectation issue. If you’ve been showing your management very large numbers—billions of impressions or millions of engagements—implementing an impact score will require some resetting of expectations.
What we recommend is an index, a number that rates each article or post based on its quality, credibility, influence, and relevance. Indexes are more accurate and consistent than absolute numbers. If you just want an absolute number, then throw out impressions and just use clicks and shares. At least they reflect the possibility that you’ve reached a human.
But if you want a single number to reflect the impact your media has each month, here’s how you do it using numbers you probably already have or that your media monitoring company should provide. Here are a few data points you will need.
- The author of the article
- A list of your most influential authors
- A list of your top tier media sorted by relevance
- The tone or sentiment of each item
- A list of your key messages
- The number of articles containing one or more key messages.
- The number of times a photo or video of your brand was included in coverage
- The number of times the item was shared or commented on
Step 1: Figure out who or what influences your target audience and how much they are paying attention to you
How big is your target market? You do not need to reach everyone with a pulse, you only need to reach the people that will make a difference to your bottom line. So prioritize your stakeholders and concentrate on the people that really matter.
The point isn’t to count the total number of eyeballs you’ve reached, but to calculate what percentage of the really important authors have covered you each month. So make a list of your top media influencers (including key authors, YouTubers, bloggers, Twitter handles, or whatever form the media influences take) that have impact on your audience. This is where an influence mapping tool comes in handy. If you know a few of the top people in a field, you can use that tool to find others like them that also write about your topics.
Obviously not every influential author will cover you constantly, so each month, look at that list and count how many of those people on that list have covered you. So for example if your Influential Author list has 50 authors on it and 25 covered you in March, you’ve reached 50% of them, and so your score for the month would be 0.5. This is an indicator of how much of your messages are getting out to influencers.
Step 2. The publication relevance tiers
But, you’re probably wondering, what about all those other journalists and publications that cover us, don’t they count? They absolutely do count, just not as much. You’re not trying to change the minds of 6 billion people. So you don’t need to measure impact based on every media outlet on the planet, or even every media outlet in America. You are trying to impact your specific stakeholders and/or audiences, so you need to know what they pay attention to. What are they reading? Where or whom do they go to for credible information? What influences them? Your ad agency may have easy access to this information, or you may have to do your own survey in order to get accurate data.
So, if you don’t already have a Top Tier media list you need to create a list of all those media outlets that specifically influence your target audiences or stakeholders. We advise prioritizing them into 2 or 3 tiers based on a combination of circulation, engagement, and influence on your audience. When you are setting your priorities, pay attention not just to a journalist’s number of followers, or the outlet’s alleged circulation, but also the degree to which there is engagement with posts. You want to know that there are people engaging in the journalist’s ideas.
In terms of an impact score, the most important publications, call them Tier 1, get an index rating of 1.0. Tier 2 publications, because they are less important, get an index rating of .50. Tier 3 publications, the least important, get an index rating of .25 points. Go through your list and assign a relevance score to each publication. At the end of the month or quarter—whatever your reporting cycle happens to be—calculate the average impact rating for all the media outlets that have covered you. Count each outlet once, no matter how many articles appeared in it over the month.
Step 3: The article quality score: What elements in an article make an impact?
Now you need to know what elements in a story might make it memorable and penetrate through the clutter. Which is why you want to create an article quality score. You might want to read this article first for more details on how to identify and prioritize article elements to create an article quality score.
To figure out which elements of articles are important, look at your articles from the past that have resonated with clients or prospects. The ones your sales force takes with them on every sales call, as well as the ones that generated the most response around the last trade show, and online. What made them so effective? Was it the message? The visual? Was it who was quoted? Those are indicators of impactful articles. Determine what was your perfect 10 article and what elements went into it.
Now do the same for your worst nightmare articles. The ones that include undesirable elements, like negative messages, or no key message, or those that recommend a competitor.
Make a list of the elements, both positive and negative, and rank them based on their importance. Now assign a weight to each element that represents its importance. For instance, for positive elements, a call to action might be worth 3 points, the presence of a key message might be worth 2 points, and headline mentions might be 1 point. For negative elements, a negative message might be worth -3 points, and a competitor recommendation might be -2 points. A negative headline is probably twice as bad as a negative mention buried deep in a post. So the former might get a -2 and the latter a -1. Make sure the sum of all the possible weights for positive is 10, and for negative is -10. Like in this example:
| Perfect 10 | Score | Worst nightmare | Score |
|---|---|---|---|
| Conveys a key message | 2 | Doesn’t convey a key message | -1 |
| Positive, i.e., Leaves a reader more likely to do business with, invest in, or work with your organization | 1 | Negative, I.e., Leaves a reader less likely to do business with, invest in, or work your organization | -2 |
| Desirable positioning | 2 | Undesirable positioning | -2 |
| Desirable visual | 2 | Contains an undesirable visual | -2 |
| Appears in Tier 1 media | 1 | Not in Tier 1 media | 0 |
| Desirable headline | 1 | Undesirable headline | -3 |
| Contains a quote from an ambassador or spokesperson | 1 | No quote | 0 |
| Total | 10 | -10 | |
Analyze each item of coverage in your database for the inclusion of these elements. The sum of the individual elements’ scores is the article quality score for that article. At the end of the month, average all your individual article quality scores to get an article quality index for the month’s coverage.
Step 4: Rate your articles by their amplification
Understanding the amplification that you’re getting from earned media is another key element of measuring impact. Most media measurement companies these days provide some data on the extent to which any given article is shared in social media. It typically looks something like this…
…listing the number of shares for any given article. If your measurement company doesn’t provide this, then find one that will.
Bear in mind that a lot of retweets and comments and likes these days are generated by robots, so you may not have a perfect count, but what you’re aiming for is to determine the relevance and resonance your placed content has with your audience.
Each article gets a number based on the number of shares and comments (or whatever data your vendor provides). Now go back into your data and calculate the average amplification of your articles over the past 6 months (or 3 months, if that’s all the data you have). That’s your baseline. Now average the amplification of this month’s articles, and compare it to your baseline. The percentage improvement is your amplification score for the month.
Step 5: Create your weighted impact index
Now you have metrics for each criteria that defines “impact.” You can track those as separate individual metrics, or combine them into a single score. That score is calculated as the relative quality times the sum of influence, relevance, and amplification or:
Impact Index = Q * (I + R + A) where:
Q = Article quality score from Step 3
I = The percentage of influential authors that cover you from Step 1
R = The average relevance score of all the media you are tracking from Step 2
A = Amplification from Step 4
So let’s say you have a nightmare month and all your stories focus on your poor financial performance, even though you got most of your positive messages across. So your article quality index for the month is a 2. But all those stories showed up in the most important publications, so 80% of your most influential authors wrote about you and the average relevance score was a 0.5.
And of course the stories were widely shared so your amplification rose 20%. Your impact score for the month would be calculated as 2 * (.8+.5+.2) = 3.
Or perhaps you had a great month and your article quality index for the month is 6, 50% of your influential authors covered you, and people shared your content like crazy so your amplification rate doubled compared to the baseline. But, all the key publications were distracted so your relevance score is only a .5. So your score for the the month would be 6 * (.5+.25+2) your score would be 16.5
As you can see, the weighted media impact index captures the meaningful aspects of earned media’s contribution to the communications effort in a single number, but it can also be broken down into all of its component parts to discover exactly what aspects of your program contributed to that score.
∞
(Image by Steve Buissinne on Pixabay)
