8 Hard Truths about Attribution that We Learned at the Summit

An image to illustrate the panel discussion of attribution for PR at Paine Publishing's 2022 Summit on the Future of Measurement.
Panel discussion at the 2022 Paine Publishing Summit on the Future of Communications Measurement: Katie Paine, Rob Key, Jamin Spitzer, Mark Stouse, and Michael Ziviani.

Attribution is the most frequent subject of questions I get from conference attendees, academics, and PR people. So we made attribution the major theme of this year’s Summit on the Future of Communications Measurement. We gathered a panel of data-driven heavyweights to discuss the topic, including:

Our panel members are grounded in the kind of serious statistical sophistication that gives most PR people nightmares. They are in the business of unearthing difficult-to-accept analytical truths that comms really, really needs to understand. Here’s a taste of their talk. 

Hard Truth #1: Attribution isn’t what you think it is

When it comes to measurement, the term “attribution” has at least a couple shades of meaning. Most people doing day-to-day measurement would like to be able to demonstrate that their activities did something specific. Like sell widgets or put butts in seats. And what they mean by “attribution” is, “This thing I did caused that behavior.” That sort of attribution is what most people ask me about.

But calculating that requires doing multivariate regression with a lot of data, and very few PR or comms people have the data or the skills. As Stouse pointed out, “If you Google the definition of “attribution” you’re talking about math, and it means causality. It is not saying that ‘We did that and this person clicked on this and we can attribute that action to the thing we did.’ The whole multi-touch attribution is a misnomer, it’s never been attribution at all. Is it valuable information around the customer journey? Absolutely. Can you run pattern recognition around it, absolutely? Can we say it is causal? No you can’t. That’s never been a reality.”

Most communications professionals confuse attribution with contribution. For years we’ve been showing a relationship between specific activities and desired outcomes, but that only shows contribution, not cause. There are a lot of other things going on too, so it’s not possible to draw a straight line between one cause and one effect.

As Stouse says, “It really matters how attribution is defined. The correct definition is not which story or ad or other singular artifact led to a specific outcome like a specific sale. The correct definition is, ‘Attribution is the analytical practice of determining which of potentially many independent variables (causes) contribute the most to different dependent variables (effects or outcomes).’

“It’s true that it’s not possible to attribute a specific sale to ‘one thing’ — an article, advert, or whatever. But it is absolutely real to compute PR’s or comms’ contribution to sales performance across any number of dependent variables.”

Spitzer elaborates: “I can attribute a bunch of website visits to a unique launch or news cycle. But I can’t necessarily attribute them to a single story. Nor can I point to a specific sales outcome.”

Key agrees. In analyzing social conversations over the years, he has found that you can see which attributes are bubbling up, and maybe see which ones best fit with the client’s business. But there is no way to discover all the various influences that go into a decision. “You can identify all the needs and trusts and beliefs you want, but there are too many other things out there. From COVID to politics there’s just too much change to accurately predict what might happen. We can show that certain attributes are predictive of revenue, but that’s not causality.”

“We can show that certain attributes are predictive of revenue, but that’s not causality.”
—Rob Key, founder and CEO of Converseon

That said, Key adds, “We can clearly demonstrate topics and perceptions that predict business outcomes. We know, for example, for a company like McDonald’s if they were to increase positive perception on key environmental attributes by 25% that it would generate $140 million of quarterly incremental revenue. That is modeled and defensible. What I’m saying is you can measure attributes and topics and their impact on revenue and then you can reverse engineer what contributed to that revenue.”

The panel agreed that tools like Converseon and Proof use accepted analytical processes and can show us how to make better decisions every day. They are valuable and meaningful and insightful. But they don’t show causality.

Hard Truth #2: PR frequently gets lost in the process

The data on the actual impact of PR is mixed. While the impact of negative PR in a crisis can occasionally be shown to impact the business, both Key and Spitzer cited data as to how little impact an actual press release or campaign actually has. Key has found endless factors impacting a purchase. His research reveals that only about 13% of the impact came from word of mouth (both online and offline), but you can’t identify what actually would drive that.

Spitzer reminded us that the alleged “impression” is just an opportunity to be seen. When he analyzed actual site traffic, comparing reported UVMs (Unique Visitors per Month) to the the actual number of visitors, the real number was .003% of the purported unique visitors per month claimed by any given site.

Hard Truth #3: It is impossible to overstate the importance of the things you don’t control

The panel pointed out that too often PR and comms people only focus on what they are doing, missing the huge volatility and velocity of change beyond their control. They have no idea how that change impacts either positively or negatively what they are doing, or the outcomes of what they do. They get caught up in what they are saying and miss the perceptions of the audience. Everyone concurred that it is impossible to overstate the importance of all the things you don’t control. Which Key thinks is probably two-thirds of the model, when you look at the entirety of an organization.

Stouse argues that some things just aren’t knowable. “If you’re riding a wave and the goal is to land safely on the beach, you can have all the skill and knowledge in the world, but you really don’t know what else is in the water. You just don’t know all the things over which you have no control.”

Hard Truth #4: 90% of decisions are unexplained (but proxies are better than nothing)

In the end, all agreed that you can make inferences based on good data and models, but they are still inferences, they aren’t actual proxies. Ziviani pointed out that the signal to noise ratio is just too much for most people to get their heads around. As much as 90% of decisions are unexplained.

“The signal to noise ratio is just too much for most people to get their heads around. As much as 90% of decisions are unexplained.”
—Michael Ziviani, founder and CEO of Precise Value.

That said, if everyone agrees that something can be used as a proxy, then it’s a proxy. The problem arises from all the elements that the typical PR person is too buried in their to-do list to think about. Like the impact of employees, social media posts, what industry analysts or legal are saying, or whatever their lobbying arm is donating to.

Hard Truth #5: PR can’t handle the (data-driven) truth

Another theme of the discussion was the gap between what current technology produces and the analytical sophistication of actual PR practitioners. Spitzer observed that many comms pros just need to show that they did something, and that something then happened. “Most PR people want something that is as easy to grasp as AVEs, but more accurate than UVMs, and demonstrates that you have a meaningful impact on the business. Because at the end of the day, if you show that something works and helps the business, it will get funded, and the programs that aren’t showing that will not.”

They agreed that the problem is that it’s hard to be data-driven when you are so used to responding to whatever issue has panicked the folks in charge. As Stouse put it, PR and comms are seen as tactical rather than strategic advisors. “They went into PR because they hated math. It kills me. When we do market mix modeling and include PR, the value of PR is far higher than anyone imagines. So PR would be a winner if they could only get into the analytics. But they would rather be drawn and quartered than dive into the data.”

“When we do market mix modeling and include PR, the value of PR is far higher than anyone imagines. So PR would be a winner if they could only get into the analytics. But they would rather be drawn and quartered than dive into the data.”
—Mark Stouse, founder and CEO of Proof Analytics;

Spitzer knows the advantage of good data firsthand. “The first time I presented to the CMO he had actually read the deck before the meeting. He said, “What I really like is that this is the first time anyone didn’t start a conversation by telling me what a great job they’d done. It was the first time we had real data integrity from communications.”

Hard Truth #6: Basic listening is still essential

Despite all these barriers, the panel did agree that you can at least use all this data to stop being blindsided. Both Key and Stouse use their data to better understand what actually moves the proverbial needle for an organization. So, their clients can say, ”Yes, we need to do something,” or “No, we can wait for it to blow over.”

Which is why, despite all the advances, basic listening is still essential. Your customers, employees, or other stakeholders may well be talking about things that can or will actually move the business. And as Key points out, there are just as many other things that PR people worry about that don’t help the business.

Hard Truth #7: The only time that “one story” can drive causality is in a crisis

Crisis plays a unique role in attribution because it’s one of the few times you can actually show cause and effect. During his time at Microsoft, Spitzer studied the impact of negative news cycles on overall trust. He found that they never had a cycle that lasted long enough to impact revenue. Which is as much a tribute to good crisis communications as anything else.

But when they took a longer view, they did find an impact. “We were tracking the impact on expressed trust, and when we looked at the impact of the Cambridge Analytica privacy scandal on Facebook (that went on for months), it did have a correlative difference.”

Stouse reports that his data almost always show that PR produces more value for less money, especially when it comes to managing a crisis. In his experience, you can never point to one story or one ad that makes a huge financial difference, unless it’s a crisis. “Crisis is the one area that we can show that PR stands out. Stories that are really, really bad do have a financial impact. It’s proof that reputation can be destroyed in a day. The feedback loops are much faster in a crisis. Generally, the time lag obfuscates ROI for PR. But in a crisis you will see the results much faster, even though there may be a time lag before it actually impacts the business.”

Hard Truth #8: Lack of data is a sign of your incompetence

There was agreement that having good data at your fingertips is essential. According to Stouse, “I hear from CEOs all the time that the comms data situation is so bad that they see it as a sign of incompetence. If you don’t even have data to do the analytics that represents your performance, what does that say? Then comms wonders why they are seen as tactical and not strategic. That’s why they are not included in the planning.“

The panel agreed that comms people who shun metrics are doing the industry a disservice, because everyone else in the organization is moving towards metrics. As Spitzer tells his clients, ”You better be on the metrics journey, because if you keep dragging your feet someone else who doesn’t understand comms at all is going to give you metrics and you won’t like them. People don’t need to know how good everything is. They need to know what they can fix, and how.”

“People don’t need to know how good everything is. They need to know what they can fix, and how.”
—Jamin Spitzer, founder and lead partner at Abacus Insight Partners

Hard Truth #9: Big numbers are bad

Everyone had a horror story about data inflated by “pass along” values or overblown impression counts. Decades later, Stouse still remembers when HP’s CEO Mark Hurd shredded the head of comms. “It was some big launch we were reporting on, and the guy that was running comms was a big believer in the pass-along factor. So, he strapped a multiplier on his impression count, and it came out around 6 or 7 billion.”

In front of the entire all-hands meeting, Hurd berated the hapless comms guy, “You are telling me that every last person in this world, including illiterate peasants in the backwaters of China, have heard about this announcement at least once?” He then went off on a rant about bad numbers. It was brutal, remembers Stouse. ∞

About Author

Katie Paine

I've been called The Queen Of Measurement, but I prefer Seshat, the Goddess.

Leave a Reply