If it hadn’t happened in my home state, I might have not picked up on the recent quintessential measurement mano-a-mano between Jake Leon, Communications Director for New Hampshire’s Department of Health and Human Services (DHHS), and 2nd District Executive Councilor Cindy Warmington.
Leon was attempting to get budget approval for another no-bid state contract with agency GYK Antler, to fund a second advertising campaign meant to increase vaccination rates in New Hampshire. Warmington was trying to get decent data with which to make an informed decision on whether she should approve the request. Their exchange is a good lesson for agencies trying to get budget approval: Make sure your advocate is armed with good data.
Their back-and-forth was so instructive that it enabled us here at Paine Publishing to choose both our Measurement Menace of the Month and our Measurement Maven of the Month at the same time.
Here is the short-and-quick version:
- The Measurement Menaces of the Month are Jake Leon, Communications Director, New Hampshire Department of Health and Human Services, and GYK Antler, the ad agency for the DHHS.
- The Measurement Maven of the Month is NH 2nd District Executive Councilor Cinde Warmington.
Below is the longer version. Which is, indeed, a bit long. Because I love both measurement and my great State of New Hampshire, and I hate COVID-19.
The background: vaccine ads and state politics
Going into this brouhaha, New Hampshire’s DHHS had already spent nearly half a million dollars on a campaign of GYK Antler pro vax ads. The result was that New Hampshire still ranked last in vaccination among the New England states. (In fact, some 600,000 of us remain unvaccinated.) Now Leon and GYK Anther wanted a shot at a second campaign.
Warmington opens with a reasonable question…
Cinde Warmington
Warmington read my mind, asking exactly what I would have: “To what extent did the $435K ad campaign contribute to increasing the number of vaccinated people in the State of New Hampshire?”
…and Leon responds with nothing but useless big numbers.
Jake Leon
Grilling Leon for campaign metrics was an exercise in futility. His response was complete and utter measurement malarkey. Lots of numbers, but none that linked the campaign ads with the goal of getting more people vaccinated.
— He reported that they had run 12,789 television and radio spots. Using taxpayer dollars to pay an agency to run ads is not anyone’s definition of success in preventing the spread of COVID. Warmington replied with an accusation: since so many of the ads featured flattering images of the Governor, then perhaps his campaign should have been paying for them!
— Leon then went on to cite the campaign’s 25.5 million impressions. Yet Leon and GYK Antler offered no evidence that any of those impressions actually reached, never mind persuaded, any Granite Staters to go out and get their shots. Anyone with a bank account could easily purchase that many impressions. In fact, it’s not hard to get that many impressions for free. Just ask any brand that is suddenly trending on Twitter for doing something stupid. (As we constantly remind you, impressions are like sperm: there are lots of them but few do what they are intended to do.)
And why did we need to reach (and spend the money to reach) 25.5 million people? I know New Hampshire folks are stubborn and stuck in their ways. But did we really need to hit our unvaccinated population 42.5 times each to get through to them?
More importantly, where were they focusing their efforts? Unless they have truly unlimited budgets most agencies would focus on the target audience most in need of persuasion. In New Hampshire there are several counties with vaccination rates of less than 50%. The total number of people in those counties is 116,405. So, again, why did taxpayers need to fund a campaign to reach 25.5 million?
— So, by the way, Mr. Leon, could you please let us know how many people actually got vaccinated after seeing those ads?
No, as it turned out, he couldn’t. Leon claimed credit for 74,584 clicks to the Vaccines.NH.Gov website, which is New Hampshire’s central resource for vaccination information. But, unless GYK Antler or DHHS put tracking tags on their ads, we have no way of knowing just how many of those 74,584 clicks were inspired by something that GYK Antler actually did. Those clickers could have been prompted by news about the rising cases, something their doctor said, or a friend or relative who became sick.
And, unfortunately, when you go to Vaccines.NH.Gov and click on “Find Covid-19 Vaccines,” you are taken to the national site vaccines.gov. So, unless the feds are sharing data with New Hampshire’s DHHS, we may never know how many of the vaccination-curious went any farther than those clicks.
— To top it off, Leon couldn’t even document whether or not the ads were persuasive. The least they could have done is a basic pre/post survey (available free to Google advertisers) to see if people’s intent to get vaccinated increased. No such data was provided.
Alas, the good guys don’t always win
In a triumph for bureaucracy and bad measurement, the new campaign contract passed 4-1, with Warmington opposed. “We need to treat this like the public health crisis that it is,” she said, following the meeting. Given that the first contract was sole-sourced and signed retroactively by the governor, no one was really surprised when it was renewed. The worst part of the story is that since the program was approved, NH’s vaccination rates have plateaued.
If I were Councilor Warmington, I’d make sure that DHHS handed over the complete data set from Google Analytics, or whatever system they are using, to a data journalist for a thorough analysis.
At this point, I’m left wondering if a more effective way to get people vaccinated would be for the State of New Hampshire to just send an Uber and an engraved invitation from the Governor to every unvaccinated person in the high-risk counties. Okay, Ubers aren’t readily available in many of those counties. But still, spending $10 on ads for every unvaccinated person in high-risk counties—many of which won’t even be able to view those digital ads because they don’t have wifi—seems like an odd strategy. Granted, the cost to care for those folks if and when they get sick will be roughly 100 times greater, but the better question is: “Why aren’t we willing to spend more to get better data and better results?”
Still, the whole duke out was a good vehicle by which to crown this month’s Measurement Maven and Menace. Congrats to Cinde Warmington, our Measurement Maven of the Month, and condolences to Jake Leon and GYK Antler, our Measurement Menaces of the Month! ∞
Photo above by Arun Clarke on Unsplash.
