All communications measurement vendors do three basic things with relative competence:
- They collect data,
- They tag data, and
- They report on that data.
The extent to which any particular measurement vendor is the right one for you depends a great deal on your particular internal needs. Which is why we created our handy Vendor Selection Chart around the most typical of those needs. Here are a few tips to picking the right one:
First let’s talk about data collection.
Most use third-party sources like Boardreader, Moreover, Gnip, or Datasift to collect your social and traditional media mentions. They may also use APIs provided by various social platforms. No matter how they get it—even if they’ve developed their own crawlers—they still need licenses from the publishers to collect the data. To a great extent those licenses determine what shows up in your database.
Tip #1: Don’t try to collect everything.
You’ll never be guaranteed every single mention. And if a measurement vendor promises to get you every single mention, then chances are you’ll end up with a database full of junk articles. The key to good data collection is good filtering. You need tight enough filters to screen out the fake news, the irrelevant mentions, the press releases, job offerings, and other dregs of the publishing world that are bound to show up if you search too broadly.
Tip #2: Don’t hire a vendor that can’t collect all the media outlets and platforms that are on your top tier media lists.
Don’t worry about every media outlet you’ve ever heard of. Focus on the ones that are important to your target stakeholders.
Tip #3: Make sure they can get you the whole story.
Provide your target media list to any vendor you are considering and then make sure that they can in fact collect the full text of all the stories in each outlet. They won’t be able to store the full text for very long, but you at least need access to the whole story if you’re going to know what they’re saying or writing about you.
It’s all about the tags.
Data collection produces a giant bucket of stories that presumably mention your organization or something you want to track. Tags are how vendors pull the concepts, themes, messages, entities, companies, products, or anything else you might possible care about out of the bucket of content. The quality of tagging will greatly determine the quality of the data you receive. In those tags should also be “not terms”—words and phrases that you don’t want to see or are irrelevant to your research. For example, if your company is SAS (the software technology company) you don’t want content about SAS the airline or Surfers Against Sludge. A good system may have thousands of not terms depending on how many companies and products you are tracking.
Tip #4: Test your tagging.
It would be lovely if once your tags were created you could sit back and watch the beautiful, bountiful, and clean data arrive in your dashboard. But that never happens. Any search string or set of tags is bound to be wrong in some way or other. So make sure whatever measurement vendor you choose or are considering provides a test set of data for you to review to make fixes to their searches. There may be numerous iterations to this process, but—trust me—it’s worth it.
Tip #5: Make sure you can customize the analysis to meet your needs.
Every company has specific messages, competitor lists, product definitions, initiatives, campaigns, spokespeople, influencers, and even bespoke definitions of engagement and positive, negative, and neutral sentiment. You must ensure that these specific priorities and definitions of your organization are reflected in whatever data your vendor provides—and that you in turn will provide to leadership and use to make decisions. If a measurement vendor can’t customize its platform sufficiently for your needs, then move on to find one that can.
Now, about those reports…
Depending on the sophistication (i.e., cost) of the platforms you are looking at, you will have various choices in how your data is reported. Chances are good that your data will show up on your dashboard in chart form, and any decent vendor will allow you to customize those charts to some extent.
Tip #6: Make sure the reporting capability of the vendor matches the needs of your leadership team.
While the standard reporting colors for sentiment are red for negative and green for positive, I once had a color-blind client for whom that wasn’t acceptable. We had to modify our platform to make change the color of every chart. Not every vendor can do that. Similarly, if your boss hates pie charts and that’s all a vendor can give you, find a different vendor.
Tip #7: Check the timing and when reports are updated and available.
If you are using human-coded or human-curated copy, then it will take at least a day or two at the end of a quarter to get all the data coded and into the system. Make sure your required delivery date matches the capabilities of the vendor. Two weeks is marginally acceptable, longer than that and they’ve got a lousy QA/QC process.
Tip #8: Make sure you can export all your data into Excel.
If the system you are considering doesn’t allow you to export all your data into an Excel or CSV file, run away. Sooner or later, you will need to do a deep dive into your data and the best way to do that is with an Excel Pivot table. (See our YouTube video on how to create pivot tables.) Also, you may find something in the data that doesn’t look right, and then the only way to figure out what’s wrong is to go into the data itself. So, easily exported data is a necessity. PDFs don’t cut it. Too often vendors only allow you to PDF charts or save them as an image, but that’s pretty much useless for analysis purposes. Exporting data into Excel and generating charts from there is a much better system. ∞
(Photo by D A V I D S O N L U N A on Unsplash)
