“Good” Implies Standards

Your context always changes in research because you never really study the same thing twice. But across studies, there are standards or “rules” that govern how “good” your qualitative research is. Once you know these “rules,” you can figure out how to best research wherever you work, no matter your constraints. The standards covered here are credibility, dependability, confirmability, and transferability.

Please note that “good” qualitative research isn’t a simple black-and-white answer; it requires thoughtful reflection and planning long before you collect a single shred of qualitative data. It’s not as simple as “it’s good” or “it’s bad” but being able to recognize how far off your qualitative research is from what’s accepted.

For each standard, you’ll see several questions you can ask yourself. If you find yourself answering “yes” to many of the provided questions, then it suggests that your qualitative research is being done in a thoughtful, rigorous way.

You, the researcher, practice reflexive thinking when you consider how “good” your qualitative research is.

If you find you answer “no” a lot to these questions, don’t get discouraged. It’s a strong signal that how you conduct qualitative research needs to be improved.  

Ultimately, you, the researcher, practice reflexive thinking when determining how “good” your qualitative research is. You must maintain integrity when reviewing these questions. Without it, your qualitative research is no better than talking to people and writing down some quotes.

Let’s look at the first qualitative standard: credibility.

‍

Credibility

Credibility is about asking yourself, “Are your findings believable and plausible, given the data you collected?” If you showed your final findings to your participants, would they view those findings as accurate or reflective of their experiences? When you shared your qualitative findings, did they resonate with your intended audience?

“Are your qualitative findings believable and plausible, given the collected data?”

Use the questions below to see if you're conducting credible qualitative research or if you need to take steps to do so.

Assessing Qualitative Credibility

• Give your collected data, relationship to the topic, context, and participants, how reasonable are your findings?
• Are findings reasonably accurate from the participant's perspective?
• Did you spend enough time in the context-of-interest to understand beyond the surface level?
• Did you seek out and study any negative or contradictory evidence?
• Did you mark down and communicate any areas of uncertainty to your team?
• Did you member-check your findings?

One helpful action you can take is known as member checks. You take your early qualitative findings and share them back with your participants to see how well those findings match or reflect their experiences. Check out the guide for more on member checking.

Guide 13: Member-checking Qualitative Findings

‍

Dependability

The next standard is dependability. The question to ask yourself is, “Were you consistent when collecting, analyzing, and interpreting the data?” Given that you might have to run qualitative studies without the help of another researcher or stakeholder, it’s up to you to be consistent across every phase of a study.

“Were you consistent when collecting, analyzing, and interpreting your qualitative data?”

Without dependability, your qualitative study and its  findings) might be very different or distorted from the data you collected and how you analyzed that data. Without dependability, it’ll be hard to trust or verify both your qualitative study design and your analysis.

Assessing Qualitative Dependability

• Are your findings consistent with the data you’ve analyzed?
• Did you code and categorize your raw data in a consistent, repeatable way?
• Did you name and describe codes in an understandable and reviewable codebook?
• Are your codes, categories, and themes on the same level or congruent? (Jump to this Handbook for more)
• Could another researcher or stakeholder follow & understand your qualitative data analysis process?
• When possible, have you had other researchers review your analysis steps and findings?

‍

Confirmability

Confirmability and transferability are the next two standards. What’s unique about these two standards is that it’s not from your lens but from the lens of another researcher or stakeholder. Let’s start with confirmability.

Confirmability is about asking yourself, “could someone review all of your work (e.g., raw data, codebooks, coding, thematic diagrams, etc.) to see step-by-step how you got to your findings?” This standard is somewhat related to dependability because leaving artifacts (e.g., code books, interview transcripts, thematic diagrams, etc.), showcases how you went from raw qualitative data to findings.

"Could someone review all of your work to see step-by-step how you got to your qualitative findings?”

Another researcher (or even a stakeholder with domain knowledge) should be able to review your artifacts and follow your process. It can be helpful to leave all of your raw qualitative data and findings in one place (like a research repository or shared file system) for new and non-researchers to review.

Confirmability is a gap in experience research for several, overlapping reasons. It can be challenging to assess without other researchers working where you work. Currently, there aren’t any useful, flexible, or powerful tools to help manage and track qualitative data and its findings over time. Stakeholders also might not care about confirming or reviewing your qualitative studies, with many taking qualitative findings at face value.

Sadly, poor or even unethical qualitative research does happen. It’s up to each individual researcher to think critically and leave evidence of their qualitative studies to help catch and prevent these issues. Involving your stakeholders throughout a qualitative study can be one way to see if you’re roughly meeting this qualitative standard.

Assessing Qualitative Confirmability

• Did you explain (in your final reporting deliverable) any of your personal biases and/or assumptions that affected the data you collected, how you analyzed the data, and how you came to your conclusions?
• Did you try to minimize the unnecessary or biased impact of your personal lens on the research?
• Are your findings based on the data you collected and the structured qualitative data analysis you did, or your own subjective view on what participants said?
• Did you leave analysis journals and analytical memos for another researcher to review?

Guide 11: Using an Analysis Journal

‍

Transferability

The last standard we cover is transferability. Transferability is possibly the most challenging but the most sought-after standard in qualitative research. It’s about asking yourself, “could findings be applied to other, similar samples or contexts?” Transferability is similar to external validity. What sounds simple is currently a heavily debated idea.

“Could your qualitative findings be applied to other, similar populations or contexts?”

Qualitative research tends to use non-random sampling, making it hard to know if findings from your sample will generalize or transfer to other similar samples or contexts. Some researchers also believe that qualitative generalizations don’t apply simply because of the constructivist research philosophy.

They claim that interpretations of people and their experiences aren’t transferable because each context is different. Along with changing contexts, every researcher and their participants have unique, un-generalizable lenses on the world, making it more unlikely that qualitative findings will transfer.

Others believe that the burden isn’t on you but on the researcher who wants to generalize your findings. You leave enough detail and evidence from your qualitative research for another researcher to come along, read through it all, and make the decision of “yes, these qualitative findings should be applicable for my current research question.” If this happens, you should be able to consider your qualitative research transferable.

This can happen with something known as an audit trail. An audit trail is a process of leaving your session notes, analytic memos, codebooks, raw data, and more for the next researcher (or interested stakeholder) to review or assess your research (similar to what you might do for dependability). The more detail and thinking you can include, the greater chance another researcher can transfer your findings appropriately. Read more here and here.

Assessing Qualitative Transferability

• Have you provided enough detail (such as rich descriptions) about the study, the sampling technique, the participants, the context, and analysis for another researcher to see if they can generalize your findings?
• Have you studied as many different or wide-ranging participants, situations, or events as possible (aka maximum variation)?

‍

"Good" Depends on You

Review these standards before and during your qualitative studies. If you can recognize what standards you aren’t meeting, make plans to change them. Remember, make impactful changes to your qualitative studies before you collect any data, not after, for the most success.

Conducting “good” qualitative research depends on being honest, rigorous, and ethical. You can only train and standardize yourself and other researchers so much, but it’s up to each researcher to make ethical decisions. The questions in this section are ways to keep yourself honest during the messiness of research.

You might be forced to cut corners in your qualitative research with short timelines and volatile stakeholders. In the short term, that’s okay. You can’t do great research if you don’t do any research. But over time, you’ll have to work to ask yourself these questions regularly, making it more likely you conduct “good” qualitative research.

Asking yourself these questions is great, but how do you translate the answers and ideas into action?