Measuring the effectiveness of academic communications? The following seven steps will guide you through the process.
Step 1: Define your objectives
The most important thing to remember about any measurement program is that you become what you measure. The metrics that you define as important will be the ones that everyone in communications will attempt to achieve, so getting them right is crucial. One exercise we put our clients through is to get them to close their eyes and imagine the president of their institution delivering a case of Dom Perignon to their desk in gratitude for the best year ever.
Then I ask them: “What has changed? What is different from a year ago?”
For example, if the university’s goal is to build the endowment, then the communications objective may be to increase the percentage of alumni who donate, or an increase in average alumni donor amount. If you rely on the web to attract applicants, you might want to make web traffic and application downloads a key metric.
The best way to determine this is to sit down with your president and his or her leadership team to get consensus about how they perceive the role that communications plays in achieving the institution’s long- and short-term goals.
Step 2: Identify and prioritize your audiences
The next step is to understand the demographics and psychographics of your various audiences. Remember that your multiple constituencies are not homogeneous. They each have different needs, wants, expectations, and desires. There are new students, returning students, and former students. You have tenured faculty, part-time faculty, instructors, etc., all with different needs, wants, and expectations. So figure out just who it is you need to talk to and maintain relationships with.
Typical audiences within the academic environment include:
- Faculty and staff
- Administration
- Alumni
- Students
- Parents
- Prospective students
- Parents and advisors of prospective students
- Donors
- Foundations
- Federal government
- State government
- Local elected officials and the local community
- The local media
Next, prioritize which relationships are most important. To do that, you need to understand how a good relationship with each constituency benefits your institution, and how a bad relationship can hurt it. Good relationships with students might keep peace on campus, but in reality, given their predilection for social media, the ultimate outcome of a good relationship with students might be to have their social posts reflect that good relationship to the world.
So make a list of all your constituencies and next to each one list the benefits that a great relationship with that group would bring to your institution. Then gather the senior leadership team and key communications people within the institution and get them to prioritize the audiences. The way we do it, everyone gets 10 colored sticky dots. Each one is worth $100,000 of communications budget—just think of it as Monopoly money. I then list the audiences on a whiteboard, and ask all participants to step up and “spend” their budgets by sticking their dots next to the audience(s) they think are most appropriate. The audience with the most dots is clearly the most important.
Step 3: Understand where they’re coming from
When you’re thinking about metrics, remember that audiences do not live in a vacuum, nor do they only hear your messages. In today’s society any one of your students/faculty/alumni, etc. is hit with some 5,000 messages a day from a wide variety of sources. Some, like advertising and direct mail, you can control. Others, like the media, you can influence but can’t control.
A good measurement program looks at as many different influences as the budget allows. At the very least you should take into account what your students/faculty/alumni, etc. are seeing in the media, what they’re exposed to online, what they’re hearing from you, what they’re taking away in terms of impressions, awareness, or understanding, and ultimately what actions do they take, e.g., download applications or brochures, donate, etc.
All these various influences affect your relationship. So it’s necessary to understand and quantify those influences if you want to understand why the relationship is improving or declining.
Step 4: Establish a benchmark
If I tell you that 35% of alumni donate to your university, what does that tell you? Essentially it says that 65% aren’t donating, but it doesn’t tell you whether that is a good number or a bad number. Do your peer institutions have higher or lower rates of alumni giving? Is your rate of alumni giving going up or down?
Since measurement is a comparative tool, you need to determine what you will benchmark your results against. Ideally you’d compare yourself to peer or a competitive institutions. Limit the number of entities in any given study to no more than five. Three is ideal, anything more than five becomes unwieldy.
Step 5: Decide what data you need and how you’re going to get it
Once you have your objectives clear and your benchmarks established, it’s time to figure out what data you need in order to create the metrics you’ve selected. You will want to consider:
Measuring media
No matter what type of institution you work for or the nature of your goals, you only need to reach the target audiences that will help you get to those goals. So don’t worry about reaching 3 billion eyeballs or trying to analyze the world. Make a list of the media sources that are most important to your stakeholders. If faculty is your number one priority audience, you need to make sure you’re measuring academic journals. If students are the number one audience, you need to remember that this group watches very little mainstream media and gets most of its information from online sources. So you should be monitoring social media sites like Instagram, YouTube, or—as the landscape changes quickly—whatever a quick survey tells you they pay attention to.
One thing to remember when doing media content analysis to to make sure your readers match your target audience. Communications professionals are not the best readers, as we do not read the media like normal human beings. We are far too quick to spot a key message (particularly our own) and are much more sensitive to reporter’s opinions than are normal human beings. So find a student or someone else to read it for you.
Measuring perceptions, reputation, and relationships
To measure perceptions, consideration, preference, and relationships, you will need to do a survey. Drs. James and Laurie Grunig have developed a standard survey instrument that can be used with a high degree of accuracy to judge the health of your relationships. Another option would be to ask NPS (Net Promoter Score) questions. And to measure faculty, you might want to consider a Leader Say-Do Survey.
Measuring impact
Audience member behavior can take many forms, from subscribing to an email list to downloading applications, attending an event, or making a donation. All are relatively easy to measure, once you agree on how communications contributes to that action. You may need to get together with senior leadership and agree on acceptable proxies. For example if your goal is to increase alumni giving, you may need to establish a unique URL for communications activity and track traffic from that URL to the giving page.
The key is to keep track of your activities on a daily basis, and then compare your efforts to your media activity and, ideally, to the attitude research. With sufficient data you will be able to see correlations between activities in the media, communications with your students/faculty/alumni, etc., and behavioral outcomes.
Step 6: Analyze the data and glean insight
All the data in the world is simply trivia if you can’t draw some conclusions from it. So when the data is in, look for trends, differences between new students/faculty/alumni, etc. and old students/faculty/alumni, males and females, etc.
By all means look most carefully at the bad news and the failures, because that is where you will learn the most. Rank order results from worst to best. That way you can quickly see what activities didn’t work and which ones did.
Step 7: Make changes and measure again
Once you have your measurement program in place, you need to make it a regular, ongoing part of your communications process. When the data comes in, you learn from your mistakes, make changes, and then see the results in the next reporting process.
One caveat: Make sure your data is available and timely when you need it to make decisions. If you do all your budgeting and planning in August, having end-of year data does you no good whatsoever. The data should be fresh just as you are beginning your plan. ∞
