Introduction

The language around mixed methods study designs can be confusing or even overwhelming. The same design is given different names across different practitioners (you can see a brief list of the many variations in this table, on pages 60-62).

In an effort to make the following mixed methods designs easier to understand/apply, in the Fruitful research library, the naming of popular mixed designs are built around how the methods are integrated. Namely, these are building, merging, and enriching types of integration (covered more in Topic 2).

To give you as much structure to build your mixed methods research knowledge on, here’s  a table that overviews all three designs.

The development mixed methods design is the formal survey design process. You can read more about survey design in this Handbook.

‍

Overview of Common Mixed Designs

The uppercase, lowercase, and arrows used in this table are common ways of visualizing your mixed methods research study design. This visual language or notation was popularized by mixed methods researchers John W. Creswell (2003), Abbas Tashakkori (1998), Charles Teddlie (1998), and Janice M. Morse (1991). A lot of the mixed methods research content online use these shapes and arrows for illustration purposes. While helpful for you, the researcher, please note that it’s not readily apparent nor interesting for your stakeholders. Use the notation to support your mixed methods research planning, but don’t expect it to help sway unconvinced or research-resistant stakeholders. You can learn more about mixed methods design notation here.*

Let’s review each design. For each, you’ll learn about triggers or reasons to use a particular design, strengths to benefit from and weaknesses to overcome or manage, and other additional considerations.

‍

Building-Generalization Design

The generalization design is a two strand (sequential) mixed design. It places the data priority on the qualitative strand. The generalization design is great for exploring and deepening your understanding first with a qualitative strand before looking at prevalence and causation with the quantitative strand after. Consider this design when you want to understand if your qualitative findings reflect and exist within the larger population-of-interest.

Many new researchers take their first steps into the world of mixed methods research by implementing a generalization design. Due to its two strand (sequential) nature, you can be more thoughtful about what to generalize and quantify, while still allowing time to deeply understand your participants and qualitative findings.

In online mixed methods literature, the generalization design is commonly referred to as an exploratory-sequential design. You arrive at qualitative themes and findings and look to explore the prevalence of such findings with a larger, representative sample.

The table below outlines the mixed methods properties for this design (go back to Topic 2 in this current Handbook to familiarize yourself with these properties).

The most common example of the generalization design in practice is by conducting interviews first and then running a survey to quantify those interview findings at scale. Or you could also conduct observations and then turn to quantitative log data (such as time-spent, number of complaints, bounce rates, etc.) to quantify the qualitative themes.

You can review the strengths and challenges of the generalization design below. You can also use the table below as a way to recognize if this design is appropriate — or feasible! — given your research questions and goals.

Below is a visual walkthrough of the many decisions and actions you might take to complete a generalization mixed design. It connects the various properties and strengths and challenges of this design and showcases how it comes to life in practice.

And below are more granular behaviors and outputs for this design. You can also use them as a handy checklist or reference.

Generalization Design Behaviors and Outputs

• Collect qualitative data
• Structure and analyze the qualitative data
• Arrive at qualitative themes and findings that you need to quantify
• Write a survey to help quantify those qualitative themes (ideally by grouping together items that address relevant constructs, variables, scales, and/or topic)
• Collect quantitative data and analyze it to see which qualitative themes resonate with the target population and the relevant proportions of those responses
• Interpret your qualitative and quantitative findings and blend them together to write key findings to your research questions

You can learn more about the generalization design in this article or in this resource.

‍

Building-Development Design

The development design is a two strand (sequential) mixed design. It’s used to help you design a survey instrument. This type of design is used to generate hypotheses from the qualitative strand, which the main purpose of being tested in the quantitative survey.

You might notice the diagram above is similar to the generalization design above. That’s not an error! There’s a subtle difference between the development and generalization design. The generalization design places the data priority on the first qualitative strand, while the development design prioritizes the second, qualitative strand.

The generalization and development design both start with a qualitative strand but the difference is where the data priority is placed.

The qualitative strand, which also happens first, is used only to help develop the best survey instrument possible. The development design often works with smaller but multiple qualitative samples to identify, create, and validate the best possible survey tool.

The development design starts with a qualitative strand, commonly semi-structured interviews around the intended survey topics. After the findings help draft the survey, another round (or multiple if needed) of qualitative research to cognitive test the survey is conducted. Based on changes, the final survey is launched and data is collected.

Based on the qualitative strand, there’s even a chance that you don’t need the second quantitative strand. You might end up addressing the survey’s intended purpose with the focus and iteration of the qualitative findings. You could still develop and test the survey but it’s all based on the needs and goals of your study. This is one reason that this type of design can be planned or opportunistic.

You can use the table below to help determine if a development design is appropriate for your needs.

Let’s bring this design, its properties, strengths and weaknesses to life in the diagram below. And beneath it are more concrete behaviors and outputs for this design. Once again, you can use them as a reference.

Development Design Behaviors and Outputs

• Make a plan to create a survey instrument (that studies a construct or latent concepts; see this Topic  for more on constructs)
• Align on the survey’s focus, topics and goals
• Based on that alignment, conduct qualitative research to (a) validate if such topics are relevant to the target population and (b) capture missing or unexpected topics to be studied in the survey
• Structure and analyze the qualitative data into qualitative themes and findings
• Create a draft survey (based on the qualitative findings and the overall survey goals) that’s as succinct as possible
• Cognitive test and pilot the draft survey with another qualitative sample and make changes (see more about cognitive testing in this Topic or in Guide 10: Cognitive Testing your Survey)
• Based on the testing results, iterate and conduct your quantitative analysis to ensure the validity and reliability of the survey items (you can read more about this mathematical process here)
• Launch the survey and analyze your data (you can read more about quantitative data analysis in this Handbook
• Interpret your quantitative survey results and qualitative findings and blend them together to write key findings to your research questions

You can read more about the development design in this long but well-structured article.

For the final connecting mixed methods design covered in this Topic, let’s flip the priority and have the quantitative strand go first.

‍

Building-Explanation Design

The explanation design is a two strand (sequential) mixed design. It starts with a quantitative strand and then uses the qualitative strand to explain the quantitative results. If you see unexpected or confusing quantitative results, you can tailor your qualitative data collection to dig deeper and understand those results.

In online mixed methods literature, the explanation design is referred to as an explanatory-sequential design.

Below are the explanation mixed design properties.

There are different ways to apply this design in practice. You could start with a quantitative content analysis of your product’s ratings and comments to find patterns and frequencies. You could then use those patterns to inform an discussion guide to better understand the quantitative data. Or you could start with a regularly recurring survey (like one that measures satisfaction) and then conduct phone interviews to better understand high or low satisfaction scores.

Above and below are the strengths and weaknesses for this design, a visual walkthrough, and set of behaviors and outputs for the explanation mixed design. Use all three sections to either confidently choose this design or to consider another mixed design altogether.

Explanation Design Behaviors and Outputs

• Collect quantitative data to address quantitative research questions
• Reduce your quantitative data into descriptive and inferential statistics
• Based on quantitative results, align on what results are are confusing, ambiguous, unexpected, outlier, or alarming with your stakeholders
• Based on the results, determine who’ll be contacted or recruited for the qualitative data collection
• Select a qualitative research method to further understand those quantitative results
• Collect the focused and specific qualitative data
• Structure and analyze the qualitative data based on low understanding in the quantitative results and arrive at qualitative themes and findings
• Interpret the quantitative findings using the qualitative finding to explain those results and address your research questions

You can read more about the explanation design in this article or this article.

‍

Closing Thoughts

Building mixed methods integration is a very common and popular approach to take. Generalizing or explaining findings and developing a proper survey instrument are incredibly helpful mixed designs to use, no matter where you work. The two strand (sequential) design allows you the chance to think carefully about both strands.

In the next Topic, let’s review mixed designs with only one strand (a.k.a. concurrent).