Does Social Work? Measuring Community Effectiveness

Page 2 of 3


Measuring Success
On the surface, measuring the success of a community seems easy enough. Detailed metrics on the number of page views, unique visitors, time spent on site, and a host of others are readily available to any site administrator or community manager through most free analytics tools, such as Google Analytics, Clicky, and StatCounter, with more in-depth data available through paid upgrades.

While it may be inherent to not only focus but place value on every single measurement of community activity made available, the relevancy of each metric will vary based on company goals and objectives.

Another noteworthy finding from Deloitte's "2009 Tribalization of Business Study" was significant gaps between community goals and how success is being measured, despite the fact that most companies agree that goals should be at the core for measuring success. In some cases the metrics being discussed and valued may not provide the insight needed to adequately measure success. This is a sentiment shared by social media experts and researchers alike.

Chris Brogan, a highly influential blogger, public speaker, and president of the New Marketing Labs agency, takes a more holistic approach when defining metrics: "Metrics could be anything from conversions from the community platform to customers to retention to any number of others like ‘sales from community channel' or upsells, etc. There are many ways to convert-reduction in call volume to service if the community is a peer support community. There are lots of ways to skin the cat that go beyond typical marketing metrics."

"It all starts with the objectives of the online community," according to John Cullen, director of marketing at Zoombak Personal GPS Locators. "Is the objective to increase sales, product/service support, or feedback? Until you know the objectives it will be impossible to measure effectiveness."

Scott Dodds, client services engagement manager at Lithium Technologies, says that a true measurement of success is twofold. He believes that it is as equally important (necessary) to measure progress toward business objectives as it is to measure community health and activity. "Inability to track the former means the community could be thriving, but the business pulls out because it cannot see the value. Inability to track the latter will lead to an unhealthy community that fails to retain the most productive super users or [that] simply never gains enough traction with members to reach critical mass." If the success of a community requires both business success and member success, effective measurement must include an array of metrics. However, Dodds warns that understanding what they tell you can be a difficult process. "Many are lagging indicators that tell you what happened but don't necessarily show you where you are now or give you an idea of where you are headed."

At a recent panel discussion called Meet the New Media held at North Carolina State University, Mark Schurtman, senior data analyst at Red Hat, told an audience of business professionals that a key metric beyond what he called the "standard engagement metrics" is who is reading and sharing your information. His recommendation was to keep a close eye on content consumption, contribution, and social bookmarking.
Foster from Fast Wonder suggests identifying what she calls success metrics. "Your success metrics are a small number of items that you use to determine success or failure over a period of time," she says. "When you have your goals defined and you know what you want to accomplish, then you define the metrics that will tell you whether or not you are achieving your goals."

New Math
The administrators of an online community for alumni of Clayton College of Natural Health, an online college based in Birmingham, Ala., gauge community effectiveness on user feedback, testimonials, and the overall number of users, which has grown to more than 2,000 in a little over a year. "Those numbers seem prety good to me considering it's a ‘closed' community available only to students and graduates," says communications director Tara Brown. "We have new registrants daily."

While new registrants may be a very important metric for an alumni organization or community geared toward attracting new business, it may not mean as much if the goal is to build a loyal subscriber base or a tightly knit group of brand enthusiasts. In that case, the number of returning visitors would be a much more effective metric to monitor since new membership does not necessarily translate to engagement or guarantee return visits. "To measure the effectiveness of a community, more than quantitative data, qualitative analysis of the data is required," says Reema Nagpal, an M.B.A. student researching analytics in a Web 2.0 environment at Great Lakes Institute of Management in Chennai, India. "For measuring the effectiveness, you may ... also look out for the customer's profile, the diversity of the customer segment visiting your community, how many times the same customer visits, and what they actually talk about."

Much like Nagpal, Cassie Boorn of the Solamar Marketing Agency pays close attention to budding relationships among members: "I watch the way in which the community works together. Seeing the community interact and support each other is what makes a community website effective."

Greg Limperis, an educator and founder of the online community Technology Integration in Education, relies on a page-view map to monitor the global reach of his community, a metric he deems invaluable. Communications professional Jayne Cravens, on the other hand, dismisses the importance of page views altogether: "Page views are meaningless in determining online community success," she says. Cravens believes that the number of subscribers is a good measure but that the number of posts and specific number of members who post actively are much better metrics for determining success.

Page 2 of 3