Market research through social networking has limits

The 2011 cycling season was around the corner, and Mike Brcic was designing a new poster campaign for his Toronto-based bicycle-tour operator. His plan was to post the ads in bike shops across the city, calling out to potential customers. But which image would pack the hardest punch?

Brcic, “lead guide” and owner of Sacred Rides Mountain Bike Adventures, browsed through photos he’d collected over 15 years of touring and came across one of his personal favourites: “It’s a guy standing on the edge of a cliff in Chile with his bike and the Andes in the background at sunset. It’s a spectacular photo.”

But he wasn’t sure what his customers would think. So, what did he do? Set up a focus group? Call a market-research firm? Not even close. Without thinking twice, Brcic posted a mock-up of the poster on Facebook and waited for his firm’s fans to tell him exactly what they thought. In other words, he did his own market research.

Brcic is far from alone in embracing doityourself research. Free or cheap online tools—including social-media sites such as Facebook and LinkedIn, as well as survey generators such as SurveyMonkey and Zoomerang—offer an array of new research options that businesses of all sizes are clamouring to exploit. Business-data provider Thomson Reuters forecast last October that the US$500 million now spent using DIY methods—less than 2% of the US$32 billion spent worldwide on market research— could reach US$3 billion by 2016.

Despite this rapid rise, however, DIY research is no panacea. At best, these tools offer a quick and cheap way to gauge customer sentiment. At worst, they lead amateur researchers to fl awed results and faulty business decisions. The trick is to know when to rely on DIY tools—and when not to. Brcic, for one, is a veteran in the field. He has been using online tools to do research since 2007, when Facebook launched its fan pages. Since then, Sacred Rides has built a community of 8,200 mountain-biking enthusiasts. Most of them aren’t customers. They’re just people who enjoy videos and articles on mountain biking.

But that makes them potential customers, and Brcic doesn’t miss an opportunity to reach out to them. He writes a weekly email newsletter sent to 10,000 subscribers, as well as a blog that incorporates a Facebook widget so the company’s legions of fans can join the discussion. And Brcic has everything he posts on Facebook set to publish automatically on Twitter.

For more specific answers, Brcic conducts surveys. The free-to-use Facebook Questions app comes in handy for quickly testing a hypothesis. And for about $300 per year, SurveyMonkey provides deeper analysis, with charts to organize data and graphs to visualize results.

Last March, though, when Brcic posted his “rider on a cliff” photo, he was just looking for feedback. Within minutes, surprising insights started to come in. One was that about a quarter of the respondents expressed terror at the thought of biking near what appeared to be a 10,000-foot drop—a perception that Sacred Rides has struggled with. “People have this idea that we only cater to super-hardcore mountain bikers,” says Brcic. “That’s absolutely not the case.”

Later that first day, he posted a few more images (less frightening ones) and noticed that fans were voting for their favourites by clicking the “Like” button. This inspired Brcic to conduct an experiment: he announced to his Facebook fans that Sacred Rides would be “crowdsourcing” the design for its poster and invited them to participate by voting and commenting.

From there, the process was iterative. Brcic posted a series of images of bike riders; his fans questioned why all of them were men. Brcic posted potential taglines to go with the posters; the company’s fans voted on the ones they liked best, even chiming in with their own.

In the end—after nine rounds of feedback and hundreds of comments over a week of research—Brcic and Sacred Rides had their poster. In fact, Brcic decided to go with two posters: one with a man and one with a woman. He also chose two taglines: one written in-house and another written by a fan. Overall, says Brcic, he was thrilled by how the process turned out.

Brcic makes it look easy, but there are pitfalls to consider when it comes to DIY research. Luke Zukowski, co-owner and director of Reveal Research Inc., a Vancouver-based market-research fi rm that caters to small businesses, says tools such as SurveyMonkey and Facebook can lead amateurs to make errors. “The problem is that these are just tools,” he says, “and many small-business owners lack the expertise to design a research project properly.”

As an example, Zukowski points to the common problem of improper sample selection. Facebook groups are often too small to yield statistically valid results. Even a high response rate from a large group such as Brcic’s can’t represent all your potential customers, because a social media sample, by definition, excludes anyone who doesn’t use social media. It also excludes anyone who hasn’t heard of your brand or company—precisely the people whom businesses want to reach.

Then there’s “confirmation bias,” which occurs when untrained researchers ask leading questions that can bias the responses. Asking whether a politician is “competent,” for instance, will prompt a sharply different response than asking whether the same politician is “incompetent.”

Zukowski stresses that market research is a professional service, and amateurs don’t know what they don’t know. “If you’re serious about finding the correct answers and improving your business-decision process, you should contact a professional market researcher,” he says.

That’s not entirely fair, says Dilip Soman, a marketing professor at the Rotman School of Management in Toronto. Soman says that while DIY research certainly has limitations, it remains an important part of good market research. The thing to keep in mind is which kind of information you’re after: quantitative or qualitative.

With quantitative research, you direct precisely worded, closed-ended questions at a specific group of people. This is useful if the sample size is small (and thus the margin for error is greater). For example, if your target group is 100,000 people and you survey 700 of these people, you can limit the margin of error to 5%—but only if the researcher knows enough to correct for biases in the sample.

Social-media research lives somewhere else entirely—in the realm of qualitative research, in which you ask broad, open-ended questions to spark discussion. Here, numerical accuracy is less important because you’re still in the exploratory phase of the research, seeking insights wherever you can find them. This is the kind of research that has long been done through focus groups, says Soman. Users of social-media tools are now applying this method on a much grander scale.

Even Zukowski will admit that there’s a place for DIY research. With SurveyMonkey, for instance, you can run an automated customer-satisfaction survey right after a client completes a transaction with your fi rm. “It’s very useful to have some kind of feedback mechanism for your current client base,” says Zukowski. “The difficulty is in trying to reach the general population.”

Beyond the big social-media and survey names, Soman points to startups such as CrowdTap.com and GutCheckIt.com, which put researchers in instant contact with people from a given target group. Want to know what a woman earning US$100,000 per year and living in the U.S. Northeast thinks about your product? For US$40, GutCheckit.com can, within seconds, put you in touch with someone who meets these criteria. This service won’t correct for a poorly worded question, and it won’t put you in touch with Canadians. But it can go a long way toward homing in on the right sample.

Facebook doesn’t provide such a targeted demographic sample, says Brcic, so he takes his fans’ suggestions with a grain of salt: “If you rely on your fans too much and not on your own intuition, that can stunt your capacity to innovate.”

Social-media tools also present a real risk of distraction, as fans can take the conversation wherever they choose. Brcic recalls how a single comment about an article on his firm’s Facebook page about cycling through Nepal derailed the discussion. “Somebody posted something about how obnoxious it was to be riding this expensive bike through a poor country,” he says, “and the whole comment thread got hijacked.”

But that’s a trade-off you have to live with in a wide-open discussion. Sometimes, a distraction leads to a serendipitous insight; sometimes, not. Still, by listening carefully, Brcic has been able to make profitable changes to his business. For one thing, he has confirmed that not all his customers are hardcore biking daredevils, so Sacred Rides is launching trips that cater to beginners—a new niche for the firm.

Brcic is also doing his best to steer clear of promotional images that might terrify his clientele.