From the Desk of Katie Paine
Item 1: I am three weeks late with a measurement analysis for a certain client because it took seven attempts to get a clean database. I’ve had to rewrite parts of this report half a dozen times. The problem? Bad search strings, lack of good de-duping protocols, and poor filtering.
Item 2: For another project I am doing what all organizations should do when they select a vendor: Compare results. What a mess — I am constantly comparing apples to fish. It’s not impossible to reconcile one vendor’s 4-point scale with another’s 7-point scale, but it adds several extra steps to every chart and report you write.
Those two examples are just a taste of a very frustrating month I’ve had dealing with bad data and uncooperative vendors. That’s why this month’s issue focuses on dirty data (read more: “Dirty Data Dooms Measurement: Here Are 5 Tidy-Up Techniques”), and how to make sure you find good vendors (read more: “The 10 Biggest Mistakes When Hiring a Media Monitoring Vendor”). After a 25-year career on the vendor side of measurement, I am now seeing the world through the client’s eyes. I get calls and emails several times a week from clients who are similarly frustrated with the state of measurement services (read more: “5 Real Life Horror Stories of Bad Data”). It’s a very depressing situation.
The good news is that this is exactly the sort of situation that standardization of terms and collection practices can fix. These standards have recently been developed and are now in place and ready to use. In particular, a great deal of data and vendor problems would be avoided if:
(1) More vendors completed the Sources and Methods Transparency Table developed by the #smmstandards. Clients would have more visibility into the process and a better understanding of what they’re getting.
(2) More suppliers followed the Sample Codebook for Media Analysis developed by Eisenmann, O’Neil, and Geddes. Consistency in coding definitions would avoid a huge amount of hassle, confusion, and wasted time.
I’ve always been an proponent of standards from a theoretical standpoint, but now that I’m actually in the trenches dealing with the confounding cacophony that is the measurement marketplace, I realize that standards aren’t just a “nice to have,” they are a must have. Why do we make measurement so difficult?
I love the changes and discussions that the Barcelona Principles have provoked, but I think it’s time for a more radical statement. I offer The Queen’s Decree:
- Anyone sending out an RFP for measurement services is hereby ordered to use the Standards Compliance Statement for Inclusion in Research RFPs, to insure that vendors understand that they are expected to adhere to standards.
- Anyone buying measurement services is hereby ordered to use the Sources and Methods Transparency Table and to instruct vendors to use the Sample Codebook for Media Analysis. They should refuse to hire any consultants, agencies, or vendors that have not pledged to support the standards.
- Anyone selling measurement services should pledge to adhere to the standards and make every effort to be consistent with the Standard Codebook.
- Anyone advising clients or agencies on measurement selection should refuse to recommend any agency or vendor that hasn’t pledged to support the standards or refuses to comply with them.
Or… “Off with their heads!” ∞
