Question: What are some of the steps KS&R takes to ensure the accuracy of results from surveys?
- Posted by a Business Unit Manager at a Healthcare organization

Answered by Elaine Hynes, Director of Strategic Analysis
Friday, August 20, 2010

Great question! Here at KS&R, ensuring "clean" data results is a front-and-center priority. We employ several techniques and "tricks of the trade" to find and remove results that are not squeaky clean, throughout the various stages of a market research project.

  1. Prior to survey launch
    • Validation: every attempt is made to validate that the prospective respondent is actually who they say they are. For example, we will take self-reported identification variables collected during the screening process (name, address, gender, etc.) and match these up against other known sources of data such as client provided database records, third party sources (i.e, Dunn & Bradstreet), etc.
    • Uniqueness: often, sample records that are obtained from third parties or client lists will contain multiple records that relate to the same prospective respondent. We engage in various techniques to remove duplicate records, so that a given respondent is not given an opportunity to take a given survey more than once.
  2. During survey execution
    • Respondent engagement: several techniques are used to identify the engagement level of each respondent, and disqualify respondents based on lack of real engagement
      • Time stamps: at various points throughout the survey, we place time stamps, indicating how long the respondent is taking to answer a specific question or group of questions. This helps ensure survey quality in several ways: it helps identify lengthy and/or confusing questions during a pre-test, that can be revised prior to full survey launch; it can identify unengaged respondents who are speeding through a survey without giving an appropriate amount of consideration to the questions at hand
      • Red herring/quality assurance questions: at various points throughout the survey, we will add close-ended questions that have response choices that are 'off topic' (red herrings) or specifically ask them to choose a specific answer "for quality assurance purposes". If respondents choose the red herring and/or do not respond as requested, we have identified unengaged respondents
    • After survey completion
      • Data cleanup: after the results are collected, several techniques are used to remove biased data
        • Pattern response analysis: we conduct an analysis on the pattern of respondents' answers when a survey consists of a number of scaled questions (e.g., Please rate your agreement with the following statements on a 5-point scale, where '1' means 'little or no agreement' and '5' means 'strong agreement'). Doing this identifies respondents who pick the same scale for an entire series of questions. Those respondents' results are removed
        • Unengaged removal: we identify and remove respondent results who have not answered the quality control questions as directed, as well as those who have chosen 'red herring' choices
        • Outlier check: at times, there can be a respondent whose answer to a specific question , while accurate, is a random oddity that does not statistically 'fit' within the normal range of responses, compared to the population of those being asked to take the survey. We call these responses 'outliers', and typically remove them from that respondent's results. This ensures that statistics that are provided as part of the research findings are not skewed by the outlier response (e.g., mean scores)
The net outcome of these efforts is a removal of biases that would affect the accuracy of the final research results. If you would like to know more about the topic, feel free to contact Elaine directly at ehynes@ksrinc.com.
Other "Ask the Expert" Topics
Understanding market share is critical in our line of business [mobile equipment] and we have in the past engaged in different types of market sizing studies that have been very involved from a sample and analytical standpoint. With both budget and time becoming increasingly scarce, how can we get to the nuggets with a less involved approach?
Posted Friday, April 09, 2010
What tactics do you recommend for maintaining the respondent’s interest to complete a survey online?
Posted Monday, March 15, 2010
I need to understand what product attributes are important to consumers. When I’ve asked consumers to rate how important attributes are to them, all of the attributes tend to rate highly. How can I understand what is really important to them?
Posted Thursday, February 25, 2010
Given the economic downturn, we are finding it is more important now than ever before to build enduring brand relationships with end clients. Right now, we need to comprehensively assess our brand’s performance, including image and communications. What’s the best way to go about testing these in a holistic way?
Posted Friday, December 04, 2009
My target customer is a hard-to-reach C-level professional and I’m interested in doing a service need-gap study. What is the most effective way to reach them?
Posted Friday, December 04, 2009
Ask a Question

Have a question for our experts? Post it here and we'll answer it within 24 hours.

Name
Email Address
Question
Verify Image Text

Meet the Experts
Lucinda Burch
Project Manager
Read Bio
Chris Reimann
Principal
Read Bio
Russ Wojcik
Data Associate
Read Bio
KS&R, Inc.
1-888-8KSRINC
315-470-1350
Fax: 315-471-0115
http://www.ksrinc.com
hq@ksrinc.com
Copyright © 2012, KS&R, Inc. All Rights Reserved.
Privacy Notice
Steven Engelbrecht close window

Steve is a market research expert. Lorem ipsum dolor sit amet, consectetur adipiscing elit. Duis cursus lorem tellus, in faucibus urna. Et tempus placerat, magna eros mattis neque, nec consectetur quam neque sed est. Praesent pretium eros quis libero scelerisque in pretium magna vehicula. Nulla facilisi. Morbi in pharetra magna. Praesent egestas ante in ipsum ornare suscipit.

Phasellus aliquam, erat eu ultricies ullamcorper, urna ligula luctus enim, a commodo ligula nisl ac neque. Fusce faucibus, purus sit amet vehicula consequat, nunc augue bibendum lacus, sit amet luctus orci erat ut nunc. Praesent tincidunt ipsum commodo lectus scelerisque a accumsan enim rutrum. Proin suscipit est in velit hendrerit rutrum. Proin sagittis lacus eget mauris hendrerit consequat. Vestibulum luctus dictum dui, vitae rhoncus lectus pulvinar id. Duis ut tortor tellus. Nam ac molestie urna. Morbi interdum faucibus ipsum. Vestibulum at arcu lectus, in mattis quam.
Melissa Jusianiec close window

Melissa is the Manager of Marketing Services at KS&R. She has extensive experience in quantitative research design and leads the group responsible for conducting advanced analysis, including conjoint, factor analysis, correlation and structural equation modeling.
Lynne Van Dyke close window

Lynne Van Dyke, Vice President and Founding Partner, has 25 years of marketing and research experience, and heads KS&R’s Qualitative Center of Excellence. She is a Master Moderator who has successfully conducted thousands of focus groups, and is a recognized expert in the development and use of projective techniques.
Chris Reimann close window

Chris Reimann, Principal, has over a decade of experience at KS&R in customer satisfaction/value assessment, market segmentation, brand tracking, marketing communications message testing and product development.
Kris Fischer close window

Kris brings 10 years of client management and research experience to KS&R. She specializes in brand tracking, loyalty and retention, product and service development, and concept testing.
Yvette Wikstrom close window

Yvette has spent 16 years in international marketing consulting and research. She is an expert in relationship and brand management, loyalty, satisfaction, segmentation and linkage protocols.
Lucinda Burch close window

Lucinda holds an MMR degree from the University of Georgia, and has more than 10 years of marketing research and analytic experience.
Kathleen McKnight close window

Kathleen has nearly 20 years of market research experience with a strong emphasis on the Technology sector. She has managed and executed numerous multi-country quantitative and qualitative studies.
Russ Wojcik close window

Russ has ten years of market research experience with KS&R, including seven years experience programming surveys using phone and online methodologies.
Robert Mortenson, Ph.D. close window

Bob has over 25 years of experience in the telecommunications and information technology industries. He is a Vice President and Principal at KS&R. His Ph.D. in Mathematics provides him the skills for rigorous analysis. Bob leads a wide range of engagements, including competitive benchmarking, channel strategy creation, customer/market segmentation, and opportunity analysis.
Scott Woodward close window

Scott has nearly 10 years of market research experience with a strong background in consumer electronics and technology sectors. At KS&R, he has managed and executed numerous multi-country qualitative studies that have utilized both online and in-person approaches.
Elaine Hynes close window

Elaine has 15 years of market research experience with a strong telecommunications/technology background. At KS&R, she is Director of Strategic Analysis, managing qualitative and quantitative projects related to the Business-2-Business sector, including customer satisfaction, distribution channels, customer churn/loyalty/winback and product/service development.
Joseph Snyder close window

Joe brings far-reaching experience in information systems, telecommunications technology and market-research data collection to KS&R. He guides daily operations of the KS&R INSITE resource center for data collection and information management, and has successfully managed studies across all industry sectors.
Jennifer Ho Manion close window

Jennifer has over 12 years of marketing and research experience and has managed many research studies for Fortune 500 companies, especially in the communications and technology segments.
David Ouellette close window

David has an MBA from Clarkson University and a BA in Economics and Psychology from St. Lawrence University. He has more than 5 years experience managing quantitative research in both Consumer and B2B space with a heavy focus on market sizing, win/loss research, product development and customer satisfaction.

Effective December 1, 2009

Our Commitment To Privacy

Your privacy is important to us. To better protect your privacy we provide this notice explaining our online information practices.

The Information We Collect:

This notice applies to all information collected or submitted on the r2r website. On some pages, you can order products, make requests, and register to receive materials. The types of personal information collected at these pages are:

Name, Company, Email, Title

Our Commitment To Data Security

To prevent unauthorized access, maintain data accuracy, and ensure the correct use of information, we have put in place appropriate physical, electronic, and managerial procedures to safeguard and secure the information we collect online.

Additional Information

In addition to the data listed above, we collect anonymous usage information through server and offsite technologies to better serve our website visitors.

Click here to close this window