Depending on what part of the country you’re from, your degree of football fandom and your profession, after this year’s Superbowl you are either a) ecstatic b) morose c) bored or engaged in endless debates about which ads were “the best” and which “totally sucked.” The latter is what all my peers seem to be doing.
My question to all my pontificating peers is: “How are you defining ‘best’?”
Presumably, the people who wrote those $5 million + checks will define best as either “brought in new revenue” or “saved money” which in marketing speak probably means “sold more stuff than any of our marketing programs” or “got brand lift cheaper than anything else we’ve done.” As Christopher Penn would remind you – “If you’re not talking money, you will get fired.”
Spoiler alert – no one that I could find is measuring Superbowl ads on those criteria. Rather, we have a plethora of choices on how “best” is defined. The USA Today Ad Meter[1] tests for “likeability” i.e. how much did you like it. They voted for the NFL ad featuring a food fight. Then there’s Adweek that uses internal experts to declare winners and losers, seemingly based on creativity. They preferred the creativity of the Bud Light/Game of Thrones ad. Interestingly, the one thing everyone seemed to agree on was that the Burger King ad featuring Andy Warhol was the worst, mostly because much of the audience didn’t have a clue who Andy Warhol was and the rest didn’t care.
The closest thing to measuring outcomes is Unruly’ s EQ Index [2] which asks its panel whether they “want to learn more” or “intend to purchase.”
As always, you learn the most from the losers. When I reordered the Unruly data from worst to best, I got much more interesting results
- Most disgusting: Devour Frozen Food Porn – because it really is disgusting.
- Generating the most contempt: Michelob Ultra – Obviously a woman on a mountain top clicking her fingernails on a bottle of beer is the Superbowl equivalent of fingernails on a blackboard.
- Most confusing: Olay Killer Skin – I’ve watched it three times and I’m still confused.
- Least favorable: Olay Killer Skin – Creepy doesn’t equal favorable
- Least Authentic Devour Frozen Food Porn (presumably because real frozen food addicts don’t look like the ones in the commercial.)
- Lowest Purchase Intent: Bumble The Ball is in her Court – (Great message, but if you’re advertising a new app, it would be a good idea to tell people what it does.)
- Finally, almost no brand did much to inspire people to “Find out More” Olay, Michelob, Stella and Pepsi” all tied with just 1/3 of the panel saying the ad inspired them to find out more. Presumably because they already knew all that they wanted to know about those brands.
I gained a lot of respect for Unruly’ s methodology when I saw that they ranked two of my favorites at the top of their overall best of list: Microsoft’s “We All Win” ad and Google’s 100 Billion words. Both scored exceptionally high on emotional appeal – a more scientific version of my Lachrymotic Scale” weighted according to how many tissues I’ve used by the end.
With the exception of the Andy Warhol Burger King ad, I vehemently disagreed with most of the rankings. Particularly irksome was that no one but me and all my Facebook friends and fellow journalists put the Washington Post’s ad at the top of their lists. (So far I’ve gone through at least three tissues every time I’ve watched it.) Actually, now that the ad has been around for a week, it’s gone up in Adweek’s rating for the conversations it engendered.
And by now I’m sure many of you will argue that these ads weren’t intended to spur sales directly. Instead you will argue that they were intended to burnish an image or enhance perceptions of brand values. But your brand image is directly connected to stock price, customer preference and/or your ability to hire talent, all of which ultimately relate to money.
That’s why I decided to do some analysis of my own, to understand the commentary around each ad and the brand sentiment it conveyed. So, thanks to some help from Talkwalker’ s AI-driven system, I analyzed the conversations during and immediately after the game to determine the volume, prominent themes and overall sentiment of that discussion complete with breakdowns by gender.
In terms of earned media share of voice, not surprisingly, the NFL dominated, but when you dig into it, most of the conversation wasn’t about their ad, but rather about the presence/absence of players and celebrities in the wake of the Colin Kaepernick controversy, so I didn’t include it in my analysis.
In terms of specific ad conversations, while it may not have won on favorability, the Washington Post ad generated by far the most conversations, no doubt helped by the President tweeting about it and the controversy that generated. This become clear when you look at the issues discussed.
And if you just have to compare the volume, here’s who earmed the most “buzz.”
Brand engagement on social media aligned closely with overall visibility, with Pepsi, The Washington Post and Amazon/Alex, Bud Light and Microsoft again topping the list. What’s most interesting is how little engagement the rest of the brands generated.
But any definition of “Best” and “Worst” clearly has to include quality, so I compared the percent of positive, negative and neutral sentiment in the conversation to the findings of the panels and pundits.
Bud Light tops the list of positive conversations mostly due to the coming together of a favorite beer with Game of Thrones. Bumble, which didn’t fare well with the Unruly panel, ended up int the top five in terms of positive conversation thanks to the well-received female empowerment message. The Washington Post ad quickly became politicized thanks to the President’s tweet, and thus it topped the list of negative conversation. The conversationalists in earned media obviously agreed with the pundits since the Burger King Andy Warhol ad was just as disliked by the general public as it was by the pundits.
But the biggest a-ha moment was the differences in the sentiment expressed by the different genders. When I downloaded the data and started playing with Pivot Tables, I found that the I wasn’t alone in my favorites. Women were more likely to comment favorably about the Microsoft
and Washington Post ads. It’s impossible to know the extent to which the favorability towards the Post ad was influenced by feelings towards the president, but the difference in sentiment is worth noting for all three ads.
while men preferred the Bud Light ad.
All of which reinforces the importance of knowing who you’re really targeting when you’re running $5 million + ads. [3] Ultimately the success or failure of a $5 million + investment in a Superbowl Ad may or may not be justified. But you will never know unless you know whether you’ve reached your target audience, whether you’ve motivated that audience to eat your burgers or whatever it is that you’re trying to get them to do. And at the end of the day, there aren’t a lot of people who are going to run out at night and spend a lot of money on a car, or home security system or anything else because they saw a 30 second spot. They will see it, check out what peopel are saying on Facebook or Consumer Reports or Amazon Reviews or Next Door. They’ll talk to their friends, their followers on Instagram, and whatever else motivates them. So closing the deal is never going to be attributed to a single ad, story or Instagram post. In the end, it’s not whether someone liked your ad, it’s whether your entire communications effort ultimately moved the audience to act.
[1] USA Today Super Bowl Ad Meter is an annual survey taken of television commercials by USA Today in a live poll during the telecast. The survey uses a live response on a zero-to-ten scale (zero being the worst, ten the best) of focus groups based in McLean, Virginia, the newspaper headquarters and one (or more) site(s) around the country.
[2] Unruly.co produces an EQ index that measures effectiveness based on “likely emotional social and business impact” using a survey panel of 1500 US respondents who record the intensity of their emotions as well as brand favorability, authenticity and purchase intent on scale of 1 to 10. I liked their system best because I agreed with it, naturally
[3] Not all conversations are included in the male/female breakdown. Tweets by organizations or ones that for whom the gender is unidentifiable have been omitted.
