The 9 dumbest ways to measure social media

by Kent Lewis -

In the past, I’ve written extensively on the topic of social media, from platform evaluation and optimization to management and measurement. Today I want to address a troubling trend in social media: meaningless metrics. There are a host of issues created by meaningless metrics, which include but are not limited to: wasting time, wasting money, and making bad decisions based on inaccurate, incomplete, or incomprehensible data.

The 9 dumbest ways to measure social media

The primary challenge is having the knowledge and discipline to provide context around data sources, collection, reporting, and analysis methodology. In this article, I will outline nine relatively meaningless metrics, from general to specific, and how to make them more meaningful.

Size doesn’t matter

What is a “like,” follower, fan, or +1 worth? By many accounts, not much. So why do marketers measure the success of social media by these relatively arbitrary numbers?

The primary reason is that the data is easily available. It’s also easy to comprehend as well, but that doesn’t make it valuable. Marketers might also fall into the trap of comparing metrics across each platform. A Facebook “like” doesn’t carry the same weight as a Twitter or LinkedIn follower, unless you agree they are all fairly useless as indicators. One excellent example is the Wildfire Facebook Leaderboard, which normalizes “like” values — effectively comparing Facebooks to Shakiras.

The fix: If you’re insistent on trending the growth of your fan base, at least add context. Track the growth of your fan base in relation to competitors. As I’ve outlined in a previous social media metrics article, there is more value in measuring engagement and other metrics relative to fan base growth.

Speed kills

There’s no need to reinvent the wheel here, even in the context of a speed metaphor. The insanity of measuring the velocity of tweets is best summed up by Stephen Colbert and his Numbotron 3000.

According to major media outlets, Michelle Obama garnered a record 28,000 tweets per minute at the 2012 People’s Party Congress of Charlotte. Colbert makes fun of the media for its inability to put context around the metric. There are many reasons a velocity metric is nearly meaningless, the primary being that it does not take sentiment into consideration.

Similarly, businesses that believe measuring the volume of content has any value in relation to the quality of the content are shortsighted. There is no prize for generating more tweets than your competitors, other than a greater probability of fans unfollowing you.

The fix: Add context to velocity-based metrics. Start with sentiment, then consider engagement and resulting actions taken by fans.

Getting overly sentimental

According to researchers, even the best social monitoring tools achieve 70 percent accuracy in terms of assessing sentiment. As such, large brands might risk making strategic decisions based on inaccurate, incomplete, or incorrectly interpreted data.

Smaller brands have a different but similar challenge: being able to pull insights from a relatively small data set. Smaller companies lack conversation volume that creates statistical significance, which can lead to false positives — making decisions based on unrepresentative, inaccurate, and therefore misleading data.

The fix: Marketers should manually review social conversations to verify the accuracy of sentiment provided by automated social platforms. Take the sentiment results with a grain of salt and focus more on the trends than the hard data itself. Smaller businesses should be careful with sentiment data as well. If data is limited, increase the time horizon or sources for collection.

Boiling it down

Klout has been criticized by many for its simplified single scoring methodology. While I appreciate the concept (as does Google, which filed a patent for a similar ranking algorithm a few years back), I see the shortcomings.

The primary concern with Klout is what it does measure: a limited slice of the social universe. The secondary concern is what Klout doesn’t measure: a larger slice of the social universe. For example, I have two Twitter profiles, and both rank relatively well, despite sharing similar content. (I had a reason for two profiles originally, but that’s another story for another time.) Also worth noting: You can “game” your Klout Score by doing somewhat arbitrary activities like posting tips on Foursquare, yet an active presence on Pinterest has no value (for the time being at least) despite being the third most popular social platform by time spent.

More depressing than being easily gamed by humans is the fact that Klout scores have been vulnerable to manipulation by robots and related tools. As such, it’s dangerous when companies use it to make decisions, from hiring to assessing the “influence” of fans and rewarding (or ignoring) them accordingly.

The fix: Take Klout with grain of salt and a shot of tequila. To get the most value from Klout, focus on trending your score and benchmarking against competitors.

Digging a hole

A few years ago, we noticed a huge spike in traffic on a client’s website. We quickly determined the source: StumbleUpon. Digging deeper into the data, we noticed a vast majority of the traffic left the site immediately, which shot the bounce rate through the roof and left little in terms of valuable visitors.

Similar stories can be told about Digg and Delicious. Marketers that put too much (or any) weight into bookmark websites may be Quantity of traffic rarely equates to quality of traffic, and long-gone are the days of easy website monetization via advertising or affiliate links, thanks to Google’s Penguin and Panda updates.

The fix: It is always important to assess traffic sources and whether or not the traffic is qualified. Beyond qualified leads, sales, or other forms of traffic monetization, there are other benefits to referring traffic from bookmark sites: the potential for new (quality) inbound links and the associated increase in search engine rankings, however miniscule. By tracking and trending link and ranking metrics, you might be able to measure the impact of these sites.

Miscellaneous metrics

Rather than bore you to death by picking apart specific platform metrics, I figured it more prudent to highlight a few of the most misleading and meaningless metrics across the spectrum. While just a sampling, the metrics outlined below best represent shortcomings not only with platforms but with measurement in general.

Comments: Considering a good deal of blog, article, or page comments might consist of spam or lack relevance, counting comments can be a waste of time.

Fix: Assess the quality and sentiment and take the opportunity to respond and engage as appropriate. Break down comments into ratios like comments-per-post.

Google+ +1s: These are as useless as Facebook “likes,” which isn’t surprising, considering Google+ was designed to be a Facebook killer. The +1 button is now ubiquitous and easy to use, but that can diminish the value of receiving a +1. Fix: Similar to previous advice, focus on trending and benchmarking competitors. If you’re truly interested in assessing the ROI on Google+, consider alternative metrics like the number of friends in a Circle or analyze comment quality and sentiment.

Impressions: Most commonly associated with YouTube, this metric is very one-dimensional and therefore misleading. What can you learn from a YouTube video being viewed 1,000 times? What if 990 of the views were from your mother, a university student in China, or even a robot? Impressions can be associated with anything, including page or image views, and the problem persists. Fix: For YouTube, be sure to take a deep dive into analytics to evaluate retention rates as well as viewer origin and demographics. For other platforms and media types, try to tie the impression metrics to higher-value key performance indicators like conversion rates, qualified leads, or sales.

Downloads: For this article, I’m going to associate mobile with social in order to make a point. Similar to collecting email addresses, counting downloads is not a direct correlation to leads or sales. The challenge is in the ability to assess the quality of the download in terms of impact on your business, especially considering low usage rates across most apps. Fix: Identify and measure power-users of your apps by looking at repeat usage. Also take a closer look at user data to determine if you are truly reaching your target end users.

Peter Shankman, founder of Help a Reporter Out, best summed up meaningless metrics with this statement: “Numbers don’t matter. What’s important is that you understand your audience, know what they want, and give it to them.”

For additional thoughts on improving your social media metrics, read my article, “The Social Media Metrics That Truly Matter.”

Other resources:

Kent Lewis is president and founder of Anvil Media, a search engine marketing agency based in Portland, Ore.

On Twitter? Follow Lewis at @kentjlewis. Follow iMedia Connection at @iMediaTweet.

Smiling businessman showing a calculator” image via Shutterstock.

Fields in red are required.
Company Revenue

Less than $ 1 Million

$ 1 - 10 Million

$ 11 - 50 Million

$ 51 - 100 Million

More than $ 100 Million

[ close ]