Guide 13

New

Guide 13

Member Checking Qualitative Findings

A validation technique to improve the quality and credibility of your findings

Trigger

Review when finalizing a study timeline; use after qualitative data collection and analysis; budget at least three days to member check findings

Step 1

Recruit or ask 4-5 informative participants to take a survey

Inform participants that you’d like their feedback and approval on the findings from your research study. To do this, you’ve designed a survey to get their feedback.

TIP: At the end of qualitative research sessions, ask consenting participants if they’d like to review your findings before sharing them with your stakeholders. It’ll shorten the time between analyzing data and reporting findings.

Ensure your participants can easily understand and complete the survey by giving clear instructions. This includes the purpose of the survey, what to do if a participant disagrees with a theme, and your contact info if there are any questions. You can use the example message below.

‍

Step 2

Add all of your major qualitative themes and findings into a survey

Write all your qualitative themes and findings in short and easy-to-read sentences. Avoid the use of jargon. Follow all the survey best practices by making the survey short, conversational, and not mentally taxing to complete.

‍

Step 3

Underneath or for each theme or finding, ask these two survey questions

“How well does this finding accurately reflect your experiences?”
“What, if anything, has to be changed to make this finding more accurate to you?”

The first question should be required, while the second one is optional but highly encouraged.

TIP: You can list relevant or representative quotes with the themes to help your participants understand each theme better.

‍

Step 4

Send the member check survey out

If you have a low response rate, send out reminders to complete the survey and that findings are not yet 100% finalized.

‍

Step 5

Make a plan for disagreeing or contradictory feedback

Be prepared to see feedback that contradicts your qualitative themes and findings. This is a good thing, because you want to catch issues before you start making/finalizing your report.

Below are possible strategies for dealing with contradictory or negative feedback:

Triangulate or compare qualitative findings with another method or data source
Showcase discrepancies as a case study / specific callout in your report
Make small but intentional changes to your findings & themes
Have 1-on-1 conversations participants to better understand their disagreements
Ignore discrepancies completely (not recommended)

‍

Step 6

Collect and review feedback, making changes you feel are appropriate

Remember that you, the researcher, will be held accountable for findings from your qualitative study. Review what each participant says and use your best judgement on whether or not you need to update your findings. However, not all disagreeing feedback will be relevant or meaningful.

GUIDE PARTS

Part 1

Part 2

Part 3

Part 1

Part 2

Part 3

Tips & Advice

If you’ve studied something traumatic, be conscious about resurfacing that trauma by asking participants to check themes.
Create a blank copy of the member check survey before you start your qualitative data analysis. Then regularly add or update your qualitative findings as you analyze your data.
If you can handle the logistics, consider letting participants send a voice memo or recording with their feedback.
If you can, consider compensating member check participants. Use the same compensation rates or structures you use for any survey research.
Having more findings will make the member check survey longer, so only check 4-7 themes or findings to check. Check only the findings or themes you’re confident about.
Let participants know they can keep or be sent a copy of their response if they want.
You can check out this resource for more help.