by Heather Fysh
Evaluating budgets for social media measurement can be overwhelming. There are so many options that it is nearly impossible to determine which provides the most value.
When facing this daunting task, keep in mind that the most important aspect to budgeting for measurement is to ensure that the information you purchase is both of good quality and answers your most important questions. If a more general assessment of the universe is desired, then a cheaper, high volume, computer-coded system will work. But if you want a deep dive into what is actually happening in social media in regards to sentient, engagement, messaging, and relationships, then expect a hefty price tag.
So, a best practice when budgeting for measurement is to first determine:
1. What questions need to be answered? and
2. How important is the quality of the data?
Pricing in social media measurement is really dependent on just what you want to extract from all the chatter. As your need increases for either more complex information or higher quality data, the time and expense of collecting the data increases. The deeper the dive and the better the quality, the more it’s going to cost!
Three levels of cost, information, and data quality
1. High volume collectors, like Radian6 and Meltwater, are inexpensive. They provide low quality data and very basic information, like share of voice and a count of product mentions. If you are searching on minimal keywords, then the reliability of your data will be fair at best. If you are looking for low cost monitoring of your media coverage this would be the way to go.
2. A computer-automated coding system, used by vendors like Vocus and BurrellesLuce, is more costly, but can be programed with 95% accuracy. Data quality will be improved to a medium level but, still, you are only going to get standard information. This model can’t deliver the deep dive of sentiment, messaging or a realistic look at engagement and relationship information.
3. Human coding, from companies like Visible and Prime, provides the highest level of quality. It also can provide information that no computer can match, including sentiment, messaging, and/or a realistic look at engagement and relationship information. And that’s the type of information that can really provide a glimpse into what’s working and what’s not in social media programs.
Quality information is costly. It takes a lot of human interaction and vast attention to detail. Humans are paid by the hour or by the piece, and that cost is included in the standard charges. The benefit of an investment in human coding is that it provides information that gives a realistic perspective on social media efforts. Basic information can still be provided, but human coding opens the door to information that can ultimately prove to be invaluable to understanding and improving social media efforts.
Q: I’ve heard that manual reading is really important for quality purposes but it is very expensive. I don’t understand why it should be so expensive. Why can’t you just pay some unemployed mom to read clips for $10 an hour? How much does it really cost to read a clip?
HF: The cost of reading a clip varies depending on several different things:
- What type of content is being coded? For example, a broadcast clip takes far more time to review and code than a Facebook post.
- What metrics are being coded for? Product mentions, company names, and people are easy to spot. However, if you need to know things like sentiment and messaging, content can be more difficult to code.
- What level of quality is required? Humans not only code clips, they also act as the most accurate filter around. They can eliminate content that isn’t qualified and recognize bad content at a much better level than the best programed computer model.
- What is the volume? If the entire universe is being tracked, it will take a lot of person-hours to code and filter. It is better to pick a small set of data for human coding, whether it be a top tier list or a list of key influencers.
Bottom line, human coding does cost more, but the level of coding and quality cannot be beat.
Q: I’ve been using Google Alerts, which is free. Now I’m overwhelmed and want to get a vendor in but I’m confused by the costs. Vocus and Meltwater say they can do what I need for $15,000 or so, and then other firms like Visible and Prime Research say it will be upwards of $50,000. What’s the difference? Doesn’t all the content come from the same places?
HF: Content comes from many different places. Vocus and Meltwater are content providers and can offer monitoring with some basic analytics. Options include a pretty chart that shows what sources the information came from, what products are mentioned, and even a computer generated sentiment score. However, the data driving those bars on the charts is not filtered. That means the data contains job postings, sidebar mentions, related stories, and content that may not be relevant to the study.
Firms like Visible and Prime don’t necessarily provide content, they have to buy it. Hence, the cost is passed onto you. They also provide a more robust study with better filtering and more in-depth information to evaluate. They cost more because they are giving you a one-stop place to get monitoring and measurement.
Q: We don’t get a lot of activity in social media. What should I put in my budget for social media measurement?
HF: Budgeting for social media can be tricky. If the norm is a low volume, it is recommended to pick the sources that are relevant to you rather than looking at the whole social media universe. Even the lowest of volumes can balloon if something goes viral. When budgeting for social media measurement it is best to decide what sources, how much content, and what information you want to know about it before putting on a price tag. Once those questions are answered, it is easier to know how much the cost will be. A 12 month social media study that monitors roughly 300 items per month and looks at Facebook, Twitter, and blogs and measures sentiment and messaging will cost on average $25,000 to $40,000 per year.
Q: What’s the difference in cost between monitoring and measurement?
HF: There is a substantial difference between monitoring content and measuring it. Monitoring is vastly cheaper as all you are doing is collecting data. You don’t get much useful information by just capturing 100 clips a month. Measurement is more costly as it involves diving deeper into the data. Measurement provides insights as to how effective your communication efforts are as well as where people are engaging.
Q: I’m really disappointed with my measurement vendor. I pay them $20,000 a year but I don’t get any insight from the data, just a lot of numbers that don’t tell me much. What am I doing wrong?
HF: If the numbers aren’t telling you much, then it’s time to have a conversation with your vendor. Most organizations will work with you to help get you the information that is most useful. It is important that you have a full understanding of what you are getting and that it is useful. If the information provided isn’t useful, then what’s the point?
Q: We have a company doing measurement for us in the U.S., but want to expand our services to our 8 key markets: Japan, China, Australia, UK, France, India, Germany, and Latin America. How much should I budget?
HF: When looking to expand your measurement program into foreign media it is important to know each market you want to track. Retrieving items for foreign countries can be difficult and expensive. The best way to budget for this type of media is to research what the average cost is per country. That will help give you a better perspective on what a realistic budget will be.
Q: I just inherited a very expensive $200,000 measurement program that measures all our media coverage around the world. Should I put out an RFP to see if I can get it cheaper?
HF: Putting out an RFP can’t hurt. Evaluating what is available is key as the measurement market is ever evolving and new vendors come about all the time. Keep in mind that a measurement program that measures all media coverage around the world is going to be quite expensive. You may want to consider narrowing down the markets to what is most important to save on cost.
∞ image: Tv Tropes
Heather – thanks for the article. First of all, i don’t intend to write in all caps but for some reason this comment platform is only allowing me this option.
Your article provides good advice, but i’d like to make a few important comments. Human analysis is much more complex than as above. Effective human coding requires intercoder reliability and trained analysts. At converseon, a strong performer in the new Forrester Wave of Social Listening platforms q1 2014, (where we also were awarded highest overall grade for consulting/research, data processing and sentiment analysis), we use three independent humans and have an adjudication process. This is essential given the inherent bias in single human analysis. Secondly, with the growth of advanced machine learning/natural language processing, the newest generation of technologies (such as our ConveyAPI technology) is able to attain near human level precision as vast scale. Humans are not excluded from this loop however technologies are able to analyze much larger data sets and more complex classifiers (like emotion, intent, etc.) without sacrificing data quality. These are important elements. Thanks for writing.