How to Break Up with Impressions: 4 Alternative Metrics That Really Matter

woman-grabs-mans-leg

Impressions are most definitely not part of the future of measurement. That’s why we’ve updated this popular step-by-step guide on how to give them the boot. This article is part of our special Future of Measurement issue.

Here’s a typical PR person’s dream sequence:

You: Impressions, we need to talk.

Impressions: The four worst words in any relationship.

You: That’s just it, this isn’t about relationships. I know we’ve been together for years, but I’ve come to realize that we are just not meant to be together. It’s not that I don’t want a relationship with metrics, I just don’t want a relationship with your metrics.

Impressions: I don’t understand why.

You: Let’s start with the fact that you’ve been misleading me for years, lying to me about how big you are, your power, and how far you can reach. You made me believe in things that weren’t true, and pursue strategies that were misguided. You let me down, and even made me look like an idiot to the senior leadership team. Remember that time I reported that we earned 3.5 billion impressions in a quarter? That’s roughly the population of the entire planet. I haven’t been invited back to the Senior Leaders’ Team meetings since. Now even the media keeps making fun of me.

Impressions: Fine, if that’s what you want, good luck finding anyone better. You’ll be back… you’ll be back like all the rest who think they can live without me. They try, but then they lose clients and don’t get budgets approved and end up on the streets, begging for me.

You: Ha! Fat chance. I’ve already found a better alternative… (Read on to find out what the better alternatives are.)

Impressions and AVEs have been the bad boy (or girl) friends of the PR industry for years. We all know they’re not good for us, and that they abuse the integrity of our profession. But for years we’ve returned to them over and over again, no matter how many times they lie to us and make us look stupid.

How to Break the Impressions Habit in 4 Easy Steps

But just as there are a myriad of books to help us get over our bad relationships, there are a ton of tools and techniques to help us get beyond counting impressions and column inches. Here are four steps to help you move on to better metrics. The ones you’ll want to spend the rest of your life with.

Step 1. Understand the connection between what you do and your business.

You can probably articulate your organization’s key messages by heart. Behind those messages is (or should be) a business plan that all the communications efforts help achieve. Within that plan there are probably sales targets, market share objectives, growth strategies, and a lot of hard financial goals to which communications contributes.

It is no longer enough to simply assume that everyone recognizes the contribution that communications makes. You need to be able to draw a very clear line between what you spend your days doing and how you help to achieve the organizational business plan.

Your organization’s business goal might be to grow sales by 5%. Or to increase market share among Hispanic women by 10%. Or to grow your donor base by 20%. Whatever, you and your leadership need to come to a mutual understanding about how communications supports that goal.

In the for-profit world, communications is probably expected to do one of the following:

  • Grow the marketable universe (the number of people interested in your products, typically a database of customers and prospects)
  • Bring in qualified leads
  • Position or re-position the organization or a product
  • Manage the organization’s reputation
  • Communicate specific messages
  • Raise awareness
  • Dispel negative myths

In the not-for-profit world things aren’t all that different. Typically, communications is expected to:

  • Bring in new volunteers
  • Attract new donors
  • Raise awareness for the cause or the mission
  • Assist in the sale of the organization’s products
  • Sway public or legislative opinion

In public affairs or government organizations, communications’ function is typically to:

  • Increase knowledge of an issue or a problem
  • Increase support for a policy
  • Increase awareness of a policy or risk

(This are obviously not comprehensive lists, but will give us some objectives to work with and a clearer definition of what matters.)

You need to agree with your leadership on:

(a) just which of these goals are expected of you, and

(b) what aspect(s) of your efforts demonstrate measurable progress toward those goals.

The following steps will help you demonstrate that progress.

Step 2. Know what data is available and understand the parameters within which you are measuring.

Conduct an audit of your organization’s existing data. And we’re not talking financial audit. Before you plan to start collecting data yourself, you need to take into account the data that is already available. That way you won’t duplicate efforts or waste money.

  • Do you have attendance figures for prior events?
  • Are there market studies of customers, prospects, or employees?
  • Is there a database of prospects, or accurate data on the number of new donors acquired, or volunteers added to the rosters?

Check with marketing, accounting, your library, the competitive intelligence folks, and anyone who may have ever commissioned a study.

No matter how much you may want to be a hero, measurement is not something that can be done alone. You need to get buy-in from the entire organization. Don’t groan about it: pull up your britches, bake some cookies to pass around, and visit a few offices. You might actually find that the annoying accountant down the hall has some very important data that can help your cause.

The point is that, like for any good organization, the results are greater than the sum of the parts. So just because you think you know the ROI for your direct mail, don’t be so sure that there aren’t other factors you need to take into account. The best scenario is to put everyone in a room and get them to answer the questions listed in the next step…

Step 3. Set priorities.

You can’t measure everything perfectly, so start by setting some priorities. What are the most crucial (highest investment) priorities, and who are the most important audiences? Define them and figure out how to measure them.

Get answers to the following key questions:

  • Where do your audiences go for trusted information about you and your peer organizations?
  • What is most important to them about your specialty area? Why do they care?
  • What do they currently know, perceive, or believe about your organization?

Prioritize your audiences based on how they benefit the organization. Force rank them, no ties allowed. Here’s a fun way to do this, if you’ve got all the key players in a room: Put a list of your audiences up on the wall. Now give everybody ten colored sticky dots (the kind you use for yard sales) and tell them that each represents $10,000 worth of communications budget. Instruct everyone to stick their dots up on the wall to “vote” on which audiences are most important. They can spend all the dots on one audience, or spread them around as they see fit. At the end, you should have a clear idea of what audiences are most important to measure and track. From there you can define the criteria you’ll be using to determine your success.

What will you compare to? Measurement is, above all, a comparison tool. You can’t claim improvement unless you can answer “Improvement over what?”

So, as part of this priority-setting process, make sure you all agree on a reasonable benchmark(s). Is it a chronological one, e.g., this time last year? Or event driven, e.g., before or after the new CEO or CMO came on board?

Step 4. Pick your alternative to impressions: Here are 5 options.

Almost anything (except AVEs) is preferable to counting impressions. There are various alternatives to choose from, but just which is the best one depends on:

  • The goals you’ve defined,
  • The parameters you’ve identified, and
  • The budget.

Here are five recommended alternatives:

1. Optimal Content Score

The Optimal Content Score (OCS) is very useful because it can be completely customized to your own situation. It’s a technique that assigns a specific numerical score for the quality of each story, post or item of communication. These scores can then be averaged and compared.

Think of all the components of a good item of communication. All the ingredients—like a great image or brilliant headline—that would motivate your target audience to purchase, support, donate, or change a habit or opinion. A story or post that includes all of these would be scored a perfect 10, correct?

Now, what components might that perfect 10 item contain? It probably mentions your brand in the headline, contains at least one key message, and perhaps a quote from one of your spokespeople or influencers. Other possible characteristics might include sentiment (positive, negative, balanced, or neutral), dominance (the extent to which your organization dominates the story), and visibility (is there a photo, a caption, or a call-out that includes your brand?)

On the opposite site, what’s a -10? It’s your worst nightmare story, perhaps one that perpetuates a myth, miss-positions the product, mentions the competition but not you, or contains an incorrect message or misleading information. Whatever those elements are, they are included on the negative side.

Put all these positive and negative elements into a chart that looks like the one below. Make sure that all the various weightings add up to +10 and -10 (so that those will be the highest and lowest scores possible for an item):

OptimalContentScoreSample

Now select all the stories that have appeared in key outlets that you know influence your target audience. For each one calculate its score—which will be somewhere between -10 and +10—based on the OCS framework you’ve developed.

Now take the average OCS for all stories that appear in a given month. That is your OCS for the month. Track this month to month. Ideally, now that you’re focusing on conversations that contain these important elements, your average OCS should improve month after month.

You can now use your OCS averages to compare campaigns or time periods, and also to easily correlate your results to web traffic and other outcome metrics. Remember that your OCS measures your ability to get a good story into the media. It is not adjusted for the number of people who actually saw a story, so it does not directly measure the extent to which you reached your audience.

2. A/B Testing

As the name implies, A/B testing compares the results of two versions (A and B) which are identical except for one variation — e.g., a different headline, visual, or call to action — that might impact a user’s behavior. It can easily be used to test two different campaigns or tactics against each other. A/B testing is also used for websites, where you can easily vary one element. Some email services (MailChimp, for one) can automate the process for email campaigns, a very useful feature.

A/B testing can also be used to compare different PR tactics. For example, let’s say you’ve always issued invitations to your organization’s events via email. To test how effective reaching out via social media is, you would target half of your audience with email and the other half with social media. Which got more responses?

3. Cost Effectiveness Metrics

Cost per thousand (CPM) is a commonly used way of judging the efficiency of one medium over another, and has been used in advertising for years. PR can use it, too.

Let’s assume that getting exposure for your key messages is the goal. To measure the effectiveness of one launch tactic over another, you divide the budget for each event by its success, and compare the two results. So, if tactic A garnered 1000 mentions and cost $100, then its CPM is 100/1000 = $.1. If tactic B resulted in 2000 mentions and cost $400, then its CPM is 400/2000 = $.2. So you got twice as much bang for your buck with tactic B.

Here’s a real life example: Back in my corporate days, a product manager came into my office and told me that he wanted to do a major party to launch his new product. His rationale was that the last launch included a very fun party that was well attended.

I explained to him that the agreed-upon goals for the last four launches were to  communicate to the media and to potential customer key messages and positioning about the new product.

I showed him my analysis of the data from the past four product launches.  The data showed that the last major event—which cost $300,000—was singularly ineffective at getting key product benefits across to the media and influencers. The data also showed that a $15,000 press tour was very successful at getting key messages into our top tier media. I showed him the chart below in which I calculated the cost per message communicated (the cost of each event divided by the total reach for the key messages). He agreed and I saved my budget from a $300,000 mistake.

Lotus chart

Cost per message communicated for each of four product launch events.

A nonprofit example: Suppose your goal is to increase the number of new donors. Count the number of new donors each month. Divide that number into the total amount (including salaries) that you spent trying to communicate with those donors. Compare the cost per donor acquired to last year, or to a different tactic.

4. Awareness Testing

The goal of many PR programs is to raise awareness for a particular issue or message. Many people think that by counting impressions they are measuring awareness, but that is incorrect. The only way to measure an increase in awareness is to survey your target audience both before and after your event or campaign, then compare the results to see if and how their knowledge or opinion has changed.

In the past, such pre/post awareness surveys were expensive and cumbersome. But today, with tools like SurveyMonkeySurvata, and Qualtrics, it is relatively simple to do such a study.

If your raison d’être is educating the public or getting your messages across, you not only need to know if your idea was seen by your audience (via message analysis), but you also need to determine whether they heard it or not, or if anyone changed their mind as a result of seeing it.

Doing pre/post awareness surveys is too expensive, right? Wrong. Think creatively. Can you build a “test” into your website or next event to gauge people’s awareness? Give them a prize for getting three right answers. Or create a “Passport” system, for which they answer questions and then have their passport stamped to signify they know one of your messages.

5. Proxies

Many times the data you’d like to have is simply not available. This is where your creativity comes in. An acceptable proxy is a metric that everyone, especially senior leadership, agrees is representative enough of the desired action to be used as a substitute.

For example: There was a person in charge of Distance Learning for a certain state who was struggling to find meaningful measures of success. He could readily count the number of student enrolled in distance learning programs in the state, but those numbers wouldn’t convince the legislature or even his managers that his programs were really paying off.

However, the department did have data on what kind of jobs students got after they graduated, and that became an acceptable proxy. As it turned out, 75% of those enrolled in Distance Learning went into high paying high-tech jobs. Clearly, the program was turning out better educated and better trained potential employees.

Even better, these well-trained employees would have great appeal to any company considering relocating to the state. In fact, he discovered that the State Office of Economic Development knew that the presence of an educated workforce was the number one driving factor for companies considering relocating to the state. He then determined that typical states allocated up to $10,000 per job in tax incentives to entice companies to move to their state. So he compared cost of the distance learning program to the cost of tax incentives to demonstrate that the learning program not only paid off, but it was a far better use of state funds.

Here’s another example: For an international non-profit, institutional barriers prevented the comms team from having access to the donor database. This meant they couldn’t measure their efforts against their goal of increasing revenue. So instead, they used Google Analytics to examine their website traffic, and realized that the only way anyone sees the Thank You page is if they made a donation! So the number of times the Thank You page was served up became an acceptable proxy for revenue. ∞

Thanks to Rebel Desk for the image.

About Author

Katie Paine

I've been called The Queen Of Measurement, but I prefer Seshat, the Goddess.

Leave a Reply