For this month’s Measurement Maven award, we decided to shine the spotlight on three particular case studies. They illustrate a range of objectives and measurement techniques that we commend, so read on:
1. A government institution rates events on trust, reach, and efficiency
Many organizational events are designed not to sell or collect leads, but to educate and improve relationships with key stakeholders. Think of industry analyst meetings or developer conferences.
One large federal institution organizes dozens of events each year with the specific goal of increasing trust in their organization. To evaluate the performance of each event it administers a four-question survey via Survey Monkey, supplemented with onsite paper surveys. Respondent are asked, on a scale of one to seven, the extent to which they agree or disagree with the following statements:
- “Whenever this organization makes an important decision, I know it will be concerned about people like me.”
- “This organization has the ability to accomplish what it says it will do.”
- “Sound principles seem to guide this organization’s behavior.”
- “I can see that this organization wants to maintain a relationship with people like me.”
- “Generally speaking, I am pleased with the relationship this organization has established with people like me.”
The results are compiled into a trust score that averages the percent of agreement with each statement. The trust score for each event can then be compared each quarter to evaluate which events were most and least successful at building trust.
Perhaps even more valuable, the institution also factors in the number of people reached, both in person at the event as well as through social and traditional media before, during, and after the event. They also factor in the budget for each event.
They then produce a chart like the one below that quickly shows senior leadership which events were most and least effective and efficient at both building trust as well as reaching their audience.
The size of the circles reflects the reach. The farther to the right, the more trust was generated. The vertical scale is the relative budget, so events in the lower right hand quadrant were the most efficient.
2. A pharmaceutical company compares event presentations to sales visits
Several years ago my company helped Glaxo pharmaceuticals determine the relative value of two different sales techniques. At the time they were promoting their products both with direct-to-doctor sales visits and with presentations at medical meetings. We used a cost-per-minute calculation to compare the effectiveness of the two sales approaches.
The company knew that it cost them $300 to get a sales person into a doctor’s office for a sales visit. Fortunately, their sales team kept records of how many leads and prospects they received, and the time they spent on sales visits. The sales people reported that, on average, each discussion with a doctor lasted about 5 minutes. So, that’s $60 per minute spent in front of a prospect.
A presentation at a certain medical meeting cost $25,000, and the presentation reached 125 people during the course of the show. That ends up being 750 doctor-minutes per show, for an average cost of $33 per minute. So that particularly event was nearly twice as cost-effective as their traditional sales tactics.
3. A corporate employee meeting saves $8.5 million
Like many organizations today, a certain major bio-tech company was having difficulty finding enough talent to fill their many job openings. And, again, like many other organizations, the company realized that employee referrals were key, and that women played a huge role in the recruitment process.
Seeing an opportunity to address two problems at once, the company came up with a concept that was truly groundbreaking. They invited (and paid for) all their female employees to each invite one teen-aged daughter or female friend to attend a two-day event featuring high profile female celebrities from the media, entertainment, NGOs, and business. The attendees heard stories of women’s empowerment from the company’s leadership team as well as from external inspirational speakers.
The company had historically used follow-up email surveys to measure attendee satisfaction. Given the expense and ground-breaking nature of the event, they wanted more robust research, so they asked my company to help them measure the success of the event.
Goals
The first challenge was getting consensus on the goals. Lacking any formal written objectives, we proposed that we measure changes in perceptions and attitudes on the part of the attendees. That meant doing pre and post surveys.
Surveys
The company had recently done an employee engagement study. So we adapted some questions from that survey and added in some that were designed to measure the extent to which attendees:
- Were inspired by the event,
- Were more likely to stay with the company and aspire to positions of greater authority, and
- Believed that the company worked hard to empower women across countries, departments, and titles.
We administered the baseline study (the pre survey) six weeks before the event took place — before the real communications push began, but after the event had been announced.
Some of the questions asked included:
- “Do you believe that women have more, fewer, or the same opportunities to advance at this company?”
- “Do you believe your gender is helping, inhibiting, or has no impact on your success?”
- “In thinking about women in the workplace, please tell us the extent to which you believe that by working together women’s impact is greater?”
- “How often do you think about finding another job?”
- “To what extent do you agree with the following statements:”
- “The company makes it easy for people from diverse backgrounds (age, race, gender, culture, physical capabilities, sexual orientation, etc.) to fit in and succeed.”
- “I believe there are good career opportunities for me here.”
Two weeks after the event, a second survey was distributed to the same employee list, with a qualifying question: “Did you attend the event?”
Results
When the results came back it was clear from the responses that the people who had attended the event were significantly more likely:
- To remain at the company, and
- To believe that there were more opportunities to advance.
Cost savings attributed to the event
The company calculated the bottom line value as follows: (numbers have been slightly altered for simplicity’s sake)
- Cost of the event: $1 million
- Percent point increase in employees who said they were unlikely to look for a job in the next 5 years: 10%
- Number of female employees attending the event: 8,500
- Average salary: $50,000
- Replacement cost per employee (average replacement cost is about 20% of annual salary): $10,000
So, if 10% of attendees (850) were less likely to change jobs., then the recruitment savings as a result of the event was $8.5 million.
ROI for the event
The ROI calculation based on these figures would be:
Cost savings ($8.5 million) – Cost of the event ($1 million) = $7.5 net, divided by cost ($1 million) = 750%
—>For the full list of Paine Publishing’s Measurement Mavens, click here.
∞
