MROC Talk Blog

You’ve heard of them. You’ve avoided them like the plague. You’ve probably seen them creep their way into your market research studies… Of course, I’m talking about “professional” respondents. You know them because they’re everywhere – in every panel, taking every survey and now you’re thinking… Will they find a way into my research community?

The short answer: they could, but it’s going to be more difficult for them to hide. Whereas panels, survey research and even short-term bulletin boards and other qualitative methodologies may allow them to be relatively anonymous, a research community is a different ballgame.

Depending on the size of the community, a “professional” respondent can be much easier to spot (and remove if necessary). With members being required to create a profile page, interact constantly with other members, blog (from time-to-time) and generally participate in more than simple surveys and polls, it’s much harder for the “pros” to hide. And we tend to think that’s a good thing Do you agree?

We get the question a lot… What’s the difference between a community and a panel for market research? Unfortunately, the names are frequently used interchangeably, causing quite a bit of confusion. I’ve even heard an online research community called a “community panel” (more than once). If you’ve ever wondered this yourself, here are a few of the differences:

• Online research communities are primarily a qualitative methodology, whereas panels are primarily quantitative.
• Online research communities provide members with a place to connect with one another, whereas (most) panels don’t provide a venue for member-to-member interactions.
• Research panels seek to discuss the “what,” whereas online research communities are designed to address the “why.”
• Research panels primarily use surveys (with the occasional invite to a qualitative discussion), whereas online research communities primarily use discussions, blogs, photo sharing, etc.

As time goes by, I have a feeling the confusion will subside. For now, hopefully this helps clarify!

Every day it seems like we come across another new “community building” start-up that promises to change the world through their whiz-bang software-as-a-service (SaaS) offering. In fact, just today we had a friend of ours in the industry forward a marketing piece from a platform provider asking if we were aware of their latest offering.

While some of these offerings are admittedly really cool, and I am always a big fan of innovation in the community platform space, I constantly have to take a step back and remind myself of something we discuss here at PluggedIN – the concept that the people running the community and the research activities you run are more important than the features.

I’m not dismissing the role of a great platform with tons of valuable features and a great user experience. In fact, this is something we’re constantly working on with the PluggedIN Community Research Platform . I’m merely suggesting that the researchers and community managers/moderators are what make the experience the most rewarding for the clients and community members. Without careful researcher guidance and facilitation, a research community won’t be nearly as valuable as it could be.

It’s really hard to keep this in mind when you see the latest features or dream up some great new research technology that could be integrated into the platform. We’re constantly taking a step back and forcing ourselves to ask… What does this do to contribute to the research objectives? Is this going to allow us to better understand the audience? Test concepts? Collaborate?

Just my two cents though What have your experiences been?

I recently came across a comment from another research community vendor stating that they were against the idea of paying incentives in their communities. Their contention was that incentives take away the authenticity of the conversation, and that their members participate out of affinity for the topic or brand.

To some extent, they have a point. People will participate out of genuine intrinsic motivations and a desire to make a difference. However, our experience is that this no-incentive policy does not lead to sustainably high levels of participation (and therefore unhappy clients), even with the best community moderators leading the charge.

There are a couple of reasons why it’s appropriate to pay incentives to members of a market research online community, including:

• Appreciation – When you pay an incentive, it shows that you value their contributions beyond a simple “pat on the back.” In a long-term community, that’s critical.
  
• Time – On average, people spend much more time participating in the activities of a research community than in other methodologies.
• Research Precedent – The expectation with many participants is that market researchers pay honorariums for participation. While not the best reason to consider continuing paying an incentive, it nonetheless factors into the decision of whether or not to pay an incentive.

The honorarium need not be cash. It can be anything from product incentives, discounts or giveaways. The point is that it has to be something that fairly values members’ contributions to the research community (and ultimately to the company sponsoring the community).

Unfortunately, I worry this community vendor might be recommending a no-incentive policy to their clients because they also recommend large-scale communities (in the thousands). If they tried to pay an appropriate incentive for thousands of members, the cost of the honorariums would likely exceed the cost of consulting each month (translation: fewer clients).

Taking it back a step from research-specific communities, the issue of monetizing community contributions is something that community managers across the web have been trying to figure out for years. A big initiative in social media is determining how to monetize user generated content. The only difference here is that market researchers have been providing honorariums to participants since the industry began.

There are many cases when a market research online community is appropriate to use… Say, for example, you want to collaborate with a target audience on the design of a new product. That would be a great application for a MROC. Or perhaps you want a venue to quickly get feedback from customers and treat them as trusted advisors in your company’s decisions… That would be another great application for a MROC.

However, for every great application for a MROC, there are an equal number of poor applications…

At PluggedIN we sometimes get requests from companies who want to use a research community for marketing objectives (e.g., buzz building, product launches, customer loyalty/retention, etc…). An example that comes to mind is a request to create a research community that is public and grows through viral campaigns to be a venue for spreading the word about a product. That isn’t a great use of a market research online community, and here’s why…

To understand why, look no further than the most “conventional” qualitative market research methodology – focus groups. Many times I’ve moderated groups where clients will see that the people in the room are clearly confused/excited/nervous/etc… about their product. That’s when you get the inevitable request to tell them about X new product, or Y benefit of another product. If I marched back into the room and tried to “sell” them on the product, I would being doing a huge disservice to the client and their research… The respondents would see me as another vehicle for marketing the client’s products, and their instant “avoid & ignore” mechanism would go up….

For the same reason, you can’t talk of research in one breath and then marketing the product in another. It just doesn’t work, particularly in a research community. Once you’ve crossed the threshold to marketing, you’ve discredited the “independent” analysis of a researcher, which really helps community members open up and be candid in their feedback.

Consider this as you launch your next social media initiative and plan to “tack on” research as an objective (which is frequently what happens). It’s never wise to mix the two, if you can at all avoid it…

Just my two cents, of course We’re always open to hearing the other side of the coin (where marketing and research make sense to combine in the same venue), so feel free to post your comments here!

One of the biggest initial decisions in setting up a market research online community is the appropriate size of the community. Unfortunately, there is no clear consensus among MROC vendors as to the “ideal” size of a research community. It’s likely the debate will never really be resolved, but I’ll do my best here to outline my position on why smaller communities are generally the best way to go…

Some MROC providers tend to recommend much smaller community sizes (on the order of 50-500 members). These companies contend that communities are a primarily qualitative methodology and therefore a smaller sample size is sufficient to get an idea of the range of potential answers/issues. Other MROC vendors tend to recommend larger communities, suggesting that they can be used for a variety of qual/quant applications.

At PluggedIN, we feel that smaller is generally better, with a few caveats… We tend to see communities as a “game changing” qualitative market research tool. Quant researchers have had their heyday with the innovative panel management tools that have been developed over the past few years. They already have the toolset to tap into an audience for quantitative research needs. In this sense, I completely agree with Brad Bortner’s (of Forrester Research) report on the segment (which can be purchased here ).

As I see it, the advantages of a smaller (defined here as 50-500 member) community are as follows:

• The ability to really get to know an audience – With a smaller community (especially on the order of 100-200 people), it is much easier to get to know the audience and learn about members. As a community moderator, I can say from experience it’s much better to really study a select group of people to get to know the segment rather than focus on how many responses you are receiving. I like to really get to know the audience when I’m digging for customer insights, and it helps when it’s a smaller group. This also makes it easier to understand the context for a given response to a research activity.
• Less tendency for social loafing – When community members know they are one of thousands, the tendency to sit back and let others do the real work can be strong. In a smaller community, people realize they need to pull their weight and that their individual contributions are more meaningful to the overall success of the community. That is important to have when your goal is to receive in-depth, insightful comments from the community.
• Ability to focus on the feedback – Imagine launching a discussion in a community with 2,000 members. I would guesstimate you would receive maybe 400 or so responses in a fairly active community. No one (including the moderator) is going to take the time to read through all 400 responses. Unfortunately, this means that the valuable “across the table” conversation tends to suffer. A better goal is to get around 40-50 individual responses to a discussion (and maybe even less!), which is far less overwhelming for both members and moderators, and leads to a higher quality discussion.
• The “exclusivity” element – Feeling like you are part of a select advisory community of 150 is “qualitatively” different than feeling like you are one of thousands selected to provide their opinion. That leads to members that are more committed to the overall goals of the research community.
• Quality over quantity – The tendency with a larger community is to focus on quantitative metrics for success of the community. When you focus on how many responses you got to a given activity, you’re missing the point entirely. The focus should be on the depth and quality of the insights emerging in the community.
• Dunbar’s number – I won’t go into too much depth here (I’ll leave that for a future post), but Dunbar’s number is based around the idea that people can’t make more than approximately 150 connections/relationships with other individuals before group cohesion starts to break down. More on this (rather complex) concept is outlined on Wikipedia here ( http://en.wikipedia.org/wiki/Dunbar%27s_number ). The communities I’ve moderated around that size tend to be better than larger communities. Maybe I’m a bit biased though, knowing this principle beforehand
• Lower cost – Last, but not least for many research community buyers, is the cost of setting up a research community. At thousands of members who have been recruited specifically according to a pre-defined set of criteria, that can get expensive. Add to that the cost of incentivizing members (beyond a simple sweepstakes strategy), and you are talking “hard costs” for the community in the tens of thousands each month/year. That can get to be cumbersome when what you are really looking for is the output and qualitative research insights. As with any type of research, cost should ideally not be the final determining factor. However, it usually weighs fairly heavily into the upfront community design decisions.
  

To be fair, there are times when going with a larger community can make sense. One situation is when you have multiple segments to study and you don’t think it justifies setting up multiple communities. In these situations, it might make sense to combine segments in a single community and split them up once they have arrived. Another situation is when you want a hybrid of qualitative and quantitative feedback. Having the larger set of members to recruit from will likely result in the ability to run surveys more effectively.

Where I’ve seen companies go wrong is when they try to get the “best of both worlds,” by creating a large community for research and hoping that by virtue of size other marketing objectives will be solved. In my experience, that is not the most appropriate use of a market research online community.

PluggedIN has taken something of an “in between” position. We strongly recommend smaller communities for most applications, given many of the advantages I outlined above. However, there are times when a larger research community make sense. To that end, we created a “Neighborhood” community product that revolves around qualitative feedback, and a “Nation” product that revolves around a hybrid of qualitative and quantitative feedback (potentially with multiple segments).

If you’re looking into setting up your own research community, just be sure to carefully consider your objectives and how you intend to use the community before determining the appropriate size. The vendor you choose will likely help you make the appropriate determination.

As you probably know, there is a ton of talk about “community” and “social media” out there. It seems like every day there is a new blog dedicated to social media, with some being better resources than others (Chris Brogan’s blog at www.chrisbrogan.com is the first that comes to mind as a good example).

However, the nuances of a market research online community (MROC) really are different, and the confusion that is caused between the two has led to quite a bit of confusion among researchers on the client and vendor side. Below are just a few of the differences between generic “communities” and other social media initiatives and a market research online community. By no means is this list comprehensive – it’s more of a general guide to outline some of the high-level differences…

1. Public vs. Private – Many of the social media initiatives being launched today are very public initiatives, whereas market research online communities are (generally) private. There are quite a few reasons for a private community, which I’ll be sure to cover in future blog posts (maybe in the form of a “Top 10″ list?).
2. Purpose & Objectives – Many of the communities being launched nowadays are for marketing purposes, with consumer insights being an ancillary objective/benefit of setting up the community. A market research online community has a single dedicated purpose – to develop a continuous source of research insights that can be used across an organization.
3. Size – With marketing running the show, you can imagine that the size of a general community tends to grow quickly. After all, marketers need impressions, clicks, conversion, etc… to justify their existence. Researchers, however, are interested in depth and getting a much closer look at an audience. For that reason, most market research online communities tend to be smaller affairs. However, there is quite a bit of debate in the industry about the “ideal” size of a research community, which I’ll be sure to cover in future posts…

There are others, but these are some of the “biggies.” I hope this helps clarify, as I think there is some much needed clarification in the space. It’s frustrating to see that research objectives are just lumped into general community initiatives and not given their due respect as separate initiatives. I’m welcome to thoughts contrary to my opinions, so feel free to post a comment if you think otherwise!

A MROC is a fancy-sounding acronym for a “Market Research Online Community.” I wish I could say that PluggedIN coined the term, but that credit goes in part to Forrester Research in their report on the space in April 2008 titled “Will Web 2.0 Transform Market Research?” If you haven’t already, I’d recommend buying that report for a great overview of what they are (find it here), how firms are using them and where things are heading, but I digress…

The report defines a MROC as “captive interactive groups of people online joined together by a common interest, which are systematically harvested for qualitative market research purposes.” I think that definition fits well at a very high-level, but in reality there are a ton of additional details that go into the definition.

For example, the “groups of people” vary in size from small-ish groups of 50-1,000 people to very large groups of 1,000 – 10,000 people. The “common interest” ranges from a shared enthusiasm for a brand and its’ products/services to a shared hobby or demographic background (i.e., Millennials, Gen X, etc…).

The telling part of the definition is the inclusion of qualitative market research and the purposeful exclusion of any reference to quantitative research. There are debates as to whether a community can really serve a “hybrid” role of gather qualitative and quantitative feedback (which I intend to cover in future posts). The key is to realize that these communities are ideally suited for qualitative feedback, but with the right design they can be used successfully for hybrid qualitative and quantitative feedback.

I think that about sums up the general definition of a MROC, and I hope I provided enough to give you a general idea of what one is all about… Future posts should help clarify a lot more with some thought provoking ideas about how they can be used.