Last week’s CASRO Online Research event in Las Vegas included what is sure to be one of the most memorable industry conference sessions for quite some time. Toluna’s John Bremer moderated a panel discussion with actual MR panelists who shared their motivations, frustrations and expectations about participating in online research panels. The panelists were exceedingly candid; they revealed what keeps them engaged (naturally, financial incentives work best) and what causes significant frustration (a big complaint is regularly being invited to surveys for which they don’t actually qualify). Some admitted to belonging to the same panel under multiple accounts, to taking the same survey more than once – and even to generating enough incentives to pay the rent. While few of the panelists’ revelations were surprising, it was novel to hear this information straight from the mouths of respondents.

Market researchers are already aware that competition for panelists is at an all-time high and panel attrition is a serious issue.  We all know that the MR industry needs to ensure we are providing a satisfactory experience to respondents; not only because it affects engagement, but also because it influences data quality. Yet poor questionnaire design and usability errors continue to be serious problems.  Inadequate querying practices too often result in unrepresentative samples and the over-sending of survey invitations. Ineffective de-duplication processes permit individuals to complete the same survey multiple times. Our industry is well aware of these issues, and the panel of panelists at CASRO Online readily reinforced the message.

So what’s the answer? Utilization of effective survey software and panel management software goes a long way toward solving these problems. Today’s sophisticated survey and panel solutions provide tools – both automated and customizable – that enhance panel performance and keep respondents happy and engaged. In situations where outside sample and/or multiple sample sources are utilized, respondent authentication solutions such as TrueSample® are also beneficial to verify, de-dup and monitor survey participants. Kinesis proudly integrates to TrueSample, and readily provides many additional next-generation features (multimode community websites, advanced email delivery tools, panel health assessments, and more) to empower our clients and facilitate successful market research panels.

 
There are many resources available, including several from Kinesis, that detail best practices for mobile surveys. Harder to find are best practice recommendations specific to mobile survey invitations; while there are actually numerous invitation options to consider, little information is available to the MR industry as to which are most effective for any given mobile research project. This is primarily because a majority of today’s mobile survey traffic comes from respondent intercept on mobile websites and ads, rather than from mobile panels. Things are starting to change however – Kinesis is seeing significant growth of mobile and multimode panels among our clients and throughout the industry – and therefore effective mobile invitation delivery is becoming a critical issue. The challenge is that there are many mobile invitation modes to consider, and determining which mode(s) to utilize can be tricky.  Below Kinesis has briefly identified both the well-known and emerging mobile survey invitation options available to researchers, with some recommendations for use.

Email. Just as is the case with desktop surveys, email is a predominant survey invitation method used in mobile research, and rightfully so. Email is a very inexpensive invitation option, recipients are highly familiar and use it regularly, and delivery times can be precisely scheduled. Email remains a strong and perhaps the best option for multi-mode (web/mobile) studies in that it provides respondents with the ultimate flexibility to decide when, where and via what device they will participate.

There is also an email-to-text option that uses the built-in email address specific to each mobile phone rather than the recipient’s third-party email application, but since many mobile subscribers do not know and do not use their mobile email address, Kinesis does not generally recommend this option. While it can be useful in camera phone ethnographies and mobile diaries, today’s mobile devices are sophisticated enough to utilize the third-party email applications for these projects, and therefore this invitation mode is becoming obsolete.

SMS.  SMS is a widely used mobile communication method, particularly among younger mobile subscribers, and enables the survey invitation to be sent as a text message. SMS short codes are well-suited when targeted recipients’ mobile phone numbers and email addresses are not known, and are currently the most popular option for recruiting feedback at public venues because they are fast/easy to type and automatically return the survey URL (or other content). SMS is a highly immediate option, yet it can be expensive. Per-usage fees are often required which can make implementation undesirable if the sample size is large, and may involve usage fees for the respondents themselves (although there are some new free-to-recipient SMS services emerging – the MR industry should keep watch here). Additionally, in Europe and other places where GSM network standards are used, WAP Push is available. WAP Push utilizes an encoded message which can be used to send links that open up directly in the mobile browser, and has proven to have higher completion rates than basic SMS.

MMS (Multimedia Messaging Service). MMS is like SMS but it can also send sound, images and video files in addition to basic text. Currently MMS usage is very expensive and there is not a single standard in use. Additionally, email provides much higher resolution for images, and mobile email supports other forms of media more readily, therefore Kinesis does not recommend MMS.

QR Codes. QR codes are a further option for pubic venue research where targeted recipients’ mobile phone numbers and email addresses are not available. QR code invitations are inexpensive to implement (the only required cost is signage/display printing) and ensure that surveys are accessed from the mobile device like SMS invitations. Respondents for QR code surveys are limited to those who have an installed QR code reader/scanner app on their mobile device, but QR code knowledge and usage is growing very quickly worldwide, and it will likely be a very viable invitation option moving forward.  QR codes may also be used in conjunction with short codes to appeal to the broadest possible audience and increase response rates.

Location-based.  Utilizing the location-tracking technology that comes standard with most smartphones, survey invitations can be triggered based on the targeted recipients’ physical location. Technologies such as geo-location, geo-fencing, Bluetooth, etc., ensure real-time invitation delivery, and can also confirmation where a survey was completed. There are limitations with this mode however; mobile users have the ability to turn off this feature, location tracking is still sometimes imprecise, and its usage may require compliance with regional privacy laws. Researchers need to be careful not to abuse or alienate respondents with location tracking until the technology becomes more mature, so therefore today it should be used with caution.

In-App Messaging.  Use of smartphone in-app messaging obviously first requires that a custom mobile research app has already been developed and downloaded to the phones of the participants. For research panels where regular communication is necessary and expected, apps are a good option. Beyond the development and maintenance costs, app messaging is typically free and invitations pop up similarly to text messages.

Social Media. Several survey software tools now offer survey invitation integration with leading social media sites like Facebook and Twitter, and this is a good option for multi-mode surveys since these sites are readily accessed by both web and mobile browsers. Social media sites are a good recruitment source for brands and companies with an active and successful social media presence, however survey invitations posted to a Facebook Wall or Twitter Feed may be easily ignored, and therefore response rates may be lower than expected.

Naturally, determining which mobile invitation option(s) to use is dependent upon the type of survey being conducted, the targeted demographic groups, and also the regions in which the project will take place.  Significant research-on-research will be necessary to determine industry-wide mobile survey invitation best practices, but in the meantime, hopefully this blog can shed some light on both the pros and cons of the options available for today’s mobile projects.

With tight budgets and shrinking revenues in the face of a weak economy, more market research firms and brands are electing to bring management of their panels in-house. Self-management can offer greater control over the panel’s processes, features and usage, and provide a means to monetize other assets, such as website visitors. The problem is that successful panel management is tougher than it looks. Smaller research companies and brands often lack some of the skill sets that are crucial for maintaining a thriving panel. Larger organizations may very well have all of the necessary skill sets to manage their panels internally, but those skill sets reside in many different employees across several departments, and thus significant collaboration is required to achieve success.

For any research company or brand that is attempting in-house panel management, a full assessment of the required skill sets and communication channels should occur. Does your business have the resources and defined processes in place to ensure value is realized? Here is a high-level summary to consider.

Like all customer-facing business initiatives, research panels require a project manager to provide the general direction, define recruitment and engagement practices, oversee all components of the panel, and drive its success. A technical person with a keen sense of aesthetics/design skills is needed to develop the panelist website, integrate all required functionality, and thoroughly test for usability. Like all websites, a panel website requires continual attention to ensure dynamic content and remain memorable to panelists.  A compliance manager is necessary to ensure that personal and financial data is adequately secured, and that privacy policies and terms of use statements are available and up-to-date. Additional resources may be needed to manage the panelist authentication process, and perform ongoing list management/conversion tasks such as email-to-address matching and de-duplication. Customer support personnel are required to address any panelist support issues in a timely fashion. Recruitment expertise may be necessary if river or social media sample is to be introduced, if customer lists are to be augmented with external sources, or if multiple sources and modes are being managed. If the panel website allows for any discussion forums and/or content upload, a moderator should also be in place to regularly review activity and keep discussions flowing. And naturally, employees with market research know-how are essential to develop the surveys, manage invitation design and delivery (effective invitation design and delivery is an art unto itself, requiring knowledge of spam scoring, email deliverability, and subject line writing), provide incentive fulfillment, and interpret the data to deliver meaningful insights.

Often research companies and brands believe they can save money and maintain better control of the panel if it is managed in-house, and in many cases they are right.  But sometimes companies utilizing in-house management ignore ongoing development and personnel costs, and/or the cost of panel atrophy, which can occur from lack of adequate panel attention.  A panel should be thought of as a core business asset and should be monitored for performance on a regular basis.  While some companies find success, many others fail because they press on with in-house management when they lack the necessary skill sets and resources, and/or because critical cross-department collaboration does not occur.

Research panels provide immeasurable valuable when implemented and fostered correctly. Each organization must evaluate its capabilities with an honest assessment, and for those that lack any of the crucial panel management components, outsourcing can save not only time and money, but ensure panel health as well. An additional route to consider is a hybrid solution in which the research firm or brand maintains responsibility for certain aspects of panel management, while outsourcing others that require more specialized skills and knowledge. Many vendors, including Kinesis, allow for panel management services to be customized, and can even assist in the evaluation process of your internal resource capabilities.

Launching and sustaining a thriving market research online community (MROC) website is a critical component of panel-based research, yet doing so can be a significant challenge even in the best of circumstances. With the vast number of websites and social media tools competing for panelists’ online time, researchers need to ensure that our MROCs offer compelling content, desirable incentives, and ease of use so that panelists remain engaged.

Another component of MROC success – and one often overlooked – is device compatibility. So often we as researchers look only to fulfill our research objectives, but communities, in order to be successful, must also fulfill the objectives of its members. Today one of the quickest ways to frustrate (and potentially lose) panelists is by not supporting a mobile version of your MROC website. With the extremely faced-paced growth of smartphone and tablet sales, all applications should now be designed to support users regardless of the device that they choose to utilize, and research panels and communities are no different. A Compuware study published in July indicated that consumers are increasingly growing impatient with websites that do not provide a satisfactory mobile experience. Some of the study’s key findings included:

• Mobile users’ expectations for mobile website speed continue to increase. 71% of global mobile web users expect websites to load as quickly, almost as quickly or faster on their mobile phone compared to the computer they use at home — up from 58% in 2009.
• More than 80% of mobile web users would access websites more often from their phone if the experience was as fast and reliable.
• A bad experience on a mobile website leaves mobile web users much less likely to return to, or recommend, a particular website. Nearly half of mobile web users are unlikely to return to a website that they had trouble accessing from their phone, and 57% are unlikely to recommend the site.

Given these consumer expectations, and knowing that researchers are increasingly competing for panelists, does it make any sense to have MROCs that support only certain types of browsers? Unequivocally, no. Furthermore, mobile users seem to be a group of potential panelists that are especially ideal for MROC recruitment. Statistics from some of the largest online communities indicate that a significant percentage of their members already engage with these communities via mobile devices. Consider that one third of Facebook’s 750 million users regularly access the application from their mobile phone, and 43% of Twitter users do. This means that a large segment of individuals who participate in online communities desire to do so using a mobile device.

Researchers who plan to launch a new MROC, and also those whose existing MROC is currently limited to computer browser interaction, need to immediately take steps to ensure mobile device compatibility. Failure to do so seriously undermines the potential of the MROC, and ultimately the quality of your research.

Good news for market research fans in the US! ESOMAR decided to condense the best presentations from its previous events with an ESOMAR Best Of road tour – coming to a location near you. The kick-off was this week in New York, with more US cities to be added, and possibly other international destinations as well. Here is a brief recap of the most thought provoking presentations…

Frederic John of MasterCard focused on the fact that market research has never been “at the table” so to speak for major business decision-making events. Moving from the “order taker” of the 70s to providing “insights” today, he emphasized that the market research industry must move to become “the guardian of the corporation.” How to get there? Better panel quality was the #1 point he hit home – the only means to be able to rely on research findings.

John’s ideas couldn’t have meshed better with the presentation by Steve Gittelman of Mktg. Inc. Using a model that combines behavioral data with demographic profiling, Gittelman showed inconsistent data findings depending upon the panel source – and even within the same panel when tracking through time. The findings replicated across international regions, and unfortunately, the stronger/more popular the use of online panels in a particular region, the greater the variance in findings over time.

Mktg. Inc. has developed an algorithm for model-based sampling that uses a combination of behavioral and demographic data, non-probabilistic in nature but tied to a probabilistic framework. Sound complicated? It is … But right now it is best opportunity we have heard for the market research industry to become the “guardian of corporate interests” – by offering a means (regardless of region of the world) to ensure that the metrics provided to our clients are indeed representative of what we are trying to measure, and can be projected to the desired based population. We say: keep listening to anything you hear about the Grand Mean Project.