Whether they bill themselves as social listening tools, measurement services, or media monitoring firms, there are now more than 350 companies out there pitching their services to hapless communications professionals. And yes, almost all of them offer sentiment analysis which is a fancy name for the art of gleaning how people feel about your brand by reading what they have to say.
But there are some important caveats to what humans and computers can and cannot do. So before you rush out and commit to a social media measurement system, you need to reflect on some specifics.
Here are six questions to help you decide which method of sentiment analysis is best for you:
Question 1: Do you receive more than 2,500 qualified mentions a month in social media?
That’s not including spam, content-farm generated copy, or mentions of a similar sounding brand (e.g., “CarMax” the car superstore vs. “Carmex” the lip balm). As you can see by this chart, if your volume falls below this mark, then it may well be more expensive to program a computer to accurately analyze your coverage that it is to hire a trained professional.
Question 2: Do people express any sentiment at all in discussing your brand?
Sentiment analysis requires quantities of data to be accurate. If you market a B2B product, then it may well be that all the conversations out there are factual discussions of features. It may not be possible for a computer to glean any sentiment from the conversations.
Question 3: Do you have any direct interaction with customers?
If you have no direct customer interaction, then it will be very difficult to determine whether the expression of sentiment has any real impact on your business. You can tie sentiment to customer behavior only if you are an online retailer, or are in a field where people make reservations, or register online.
Question 4: What level of accuracy is acceptable to your executive leadership?
Most automated sentiment analysis tools get sentiment right about 60% – 70% of the time. If that is good enough, then you can use an automated system. If not, then you need to use humans coders to conduct a random sample check of the data to ensure a higher degree of accuracy.
Question 5: What level of detail do you need from your sentiment analysis system?
If you need to track complex messaging, quotes, issues, positioning, or other esoteric details, then chances are you will need a human to do so.
Question 6: Do you run numerous campaigns which will require different search terms and different message tracking and different definitions of positive or neutral?
Computers take weeks to reprogram, test, and fix. If you need fast turnaround on changes to your system, then use a human.
Happy picking!
∞
Good review. I’d add that some new machine learning technologies (disclosure: as one provided by Converseon) are now attaining near human level precision for sentiment and other annotations, like emotion.. Other items to ask about is at what level is the analysis done (record, sentence or the most meaningful, entity/facet)? And look for tools that don’t just use boolean queries anymore, but instead use machine learning approaches. These often can improve average relevancy of the dataset from 10-30% to 80%+ quite effectively.