Guide 10
New
Guide 10
Cognitive Testing Your Survey
A method to capture issues or improvements in a draft survey
Trigger
Use after drafting your survey questions; consider when planning a study timeline that involves a survey
Step 1

Schedule 4-5 participants for an interview

Using your Most Informative Participant (MIP) definition, recruit five participants to test out your survey. Recruiting early can give you enough time to create the survey, test it and launch it.

TIP: Don’t start recruitment until you’ve aligned on your research plan.

Step 2

Have each participant read each survey item out loud

Share the survey:

  • (in-person) print draft survey out and give each participant a copy to review
  • (remote) share a link to the draft survey

Step 3

Test how each survey item is being interpreted

Ask each participant these three questions for every survey item you’re considering:

  • “Do you feel you understand what this question is asking?”
  • “How would you answer this question?”
  • “What, if anything, can be done to make this question easier to answer?”

TIP: For open-ended questions, have the participant vocalize their answer instead of asking Question 2 above.

Step 4

Capture the interpretation data and make changes

Consider making a quick chart (using Microsoft Excel or Google Sheets) to make capturing and recognizing patterns easier. There’s an example chart below you can copy or modify.

Make sure to ask probing or clarifying questions if you don’t fully understand why a question or response choice is interpreted in a certain way.

Step 5

Repeat steps 3 & 4 for all of your draft survey items

Have each participant complete the draft survey as you discuss them. You can use all of the collected data to see if they’re (a) relevant to answer your research questions or test your research hypotheses or (b) easy to analyze and interpret.

Step 6

Make meaningful changes to your survey

Part 1
Part 2
Part 3