An index is essentially measurement shorthand—a way to boil an ocean of data into a simple, easily understood number. Think of a CPI (Consumer Price Index), BMI (Body Mass Index), or NASDAQ composite index for communications measurement.
An index serves as a statistical way to take different sets of data points and aggregate them into a single number. For example, BMI takes into account your height, weight, age, and other factors. Because those numbers range from small (5 feet) to large (350 pounds), scientists and statisticians have developed ways to normalize data in a way that makes it comparable and allows you to know how you rank relative to the rest of the population.
When it comes to communications measurement, organizations (and the individuals who are a part of them) need similar formulas. For example, if you’re trying to show connections between social engagement and goal conversions on your website, then you might be comparing thousands of different types of engagement with varying degrees of weight and importance from four different platforms to a relatively small number of conversions. The solution is an engagement index, and we’ve explained how to create one here.
What makes customization so important is that the various elements in an index are weighted according your organization’s priorities and goals. Obviously engagement for a B2B startup looks very different than engagement for Coca-Cola or the World Wildlife Fund.
Regardless of the elements or the weightings, what is most important is that your index is an accurate reflection of the dimension that is being measured. To make sure it is, you need to validate the index by testing whether it can predict indicators related to the measured variables.
Follow these 5 steps to correctly set up your index:
Step 1: Get consensus about what moves the needle
As with every measurement project, you need to start with consensus around your definition of success, and how your activities impact the organization’s goals. This will require a meeting of the minds between you and senior leadership (or whoever will be seeing the index in a report). If management doesn’t buy into the criteria included in the index, all the statistical calculations in the world won’t make it useful.
Step 2: Assign weightings
Every organization has different priorities and different ideas about what constitutes successful communications. Understanding these priorities is key when you’re developing an index. For example, when the Atlantic City Alliance (ACA) was launching a promotional campaign to assure visitors that its beach and boardwalk were open for business and not damaged by Hurricane Sandy, visual proof was vital. So the presence of undesirable visuals in its media coverage was scored as twice as bad as a negative story. At the same time, research had shown that their key messages were an important factor in getting visitors to consider a visit to Atlantic City, therefore the presence of a key message was scored as three times more important than a positive story. Using existing consumer research, we helped ACA get consensus on the relevant weightings of all the criteria that they felt were important. Their final Media Quality Index (MQI) looked like this:
[table “18” not found /]Step 3: Normalize your data
One of the most frequent uses of indexes is when you are trying to use numbers of varying sizes. For example, if you’re trying to develop a social media engagement index, you are probably dealing with hundreds of likes, but probably only a few dozen shares or comments. If you try to graph those numbers on a chart to show change over time, chances are the smallest number won’t even show up. The solution is a weighted index that might look like this:
[table “19” not found /]In this case, each action is given an appropriate weighting relative to its importance in the marketing mix. What is tracked is the increase or decrease in the total score for the period. If you want to add in data from Google Analytics, that would also be tracked as a percent increase or decrease from the prior period.
One of the easier ways to normalize data across different functions or criteria is to simply track the percent change in that data month-on-month or week-to-week. Since you are using only percentages, you don’t have to worry about giant differences in the total numbers.
Step 4: Gather data
As soon as you have consensus on your metrics and weightings, you will need to apply them to real data. Until you see your index fully populated with its relevant data and its total score calculated, you won’t know whether it makes sense. So gather at least three months’ worth of data. If you plan to correlate the data with other data streams (for instance, correlate web traffic with your Media Quality Index), then you’ll need to have daily data for both media and for web traffic. The more granular the data, the better your ability to test and validate the data will be. So, where possible, make sure you have data for every day in whatever period you are using for the test.
Step 5: Test and validate your index
There are two critical elements when testing and validating your index. Since you will be using data from the past (i.e., what actually occurred), you’ll be able to examine it to see if it makes sense. For example, if you sense that last October was a kick-ass month, but your new index scores it as the worst month of the year, then there might well be something wrong with your index, your formulas, or your calculations. So the first step is to make sure it passes the sniff test.
To validate the data requires a little more work. A valid index accurately reflects the outcome that you and leadership agreed upon in Step 1. For example, if your efforts are intended to increase conversions in Google Analytics, and your index is designed to measure your efforts, then an increase in the index should correlate with an increase in conversions. (For a primer on how to do correlations in Excel, watch our video.) If your efforts are designed to increase registrations or votes on a petition, then your index scores should be positively correlated with the number of petition signers. If they aren’t, then you will need to go back and correct the weightings or your calculations.
∞
