Introduction

When new researchers first come across mixed methods approaches, they can find it both exciting and confusing. There are lots of benefits to taking a mixed methods approach, with just as many pitfalls, weaknesses, and challenges. In this Topic, let’s take a foundational look at what mixed methods research is, why its powerful but complex, and triggers or clues you can look for to recognize if its the best approach for you.

Let’s start with a simple, but tricky question: what’s the difference between a research study that uses two research methods and a study that takes a mixed methods approach?

‍

Mono, Multi, & Mixed Methods Studies

In most of the studies you’ll design and run, you’ll be using one research method at a time. Based on your needs, you might choose to two or more methods in one study. And if you have complex qualitative and quantitative research questions, then you might take a mixed methods approach.

A mixed methods study aims to collect both qualitative and quantitative data in a single study to answer a common research question or topic.

The table below describes the characteristics of different study designs based on research methods. Being able to distinguish between these types of studies can be one place to establish credibility among your stakeholders.

Read “Property: Integration” in Topic 2 for a closer look at mixing. For more on meta-inferences, scroll down to the section labelled “The Pragmatic 3D Glasses of Truth”. And jump to Topic 3 or Topic 4 in this Handbook for specific mixed methods study designs.

The goal of a mixed methods study design is to collect both qualitative and quantitative data in a single study to answer a common research question or topic. To collect both kinds of data, researchers commonly use at least one qualitative and one quantitative method in a single study. If you need, you can also use more than one qualitative or quantitative method to collect specific kinds of data in a mixed methods study.

So, a mixed methods study is less about the specific methods you use, but more on how those methods are mixed. But what does mixing even mean?

‍

What does It Mean to “Mix” Methods

The “mixed” term in mixed methods can make this type of research seem simpler than it actually is. It’s not just about one qualitative or one quantitative method, but how those different research philosophies, sampling techniques, types of data, analytic approaches, and interpretations work together. Similar to the old adage, the sum of a mixed methods study is more than the individual qualitative and quantitative approaches alone.

In a mixed methods study, you’ll likely collect some data that won’t be mixed. For example, you might collect and mix qualitative and quantitative data during a usability test. How long a participant took to complete a task, however, is purely quantitative and won’t be mixed with collected qualitative data.

In practice, however, mixed methods are anything but simple. For starts, here’s an important fact: you can mix any and every element within a single study (covered more in Topic 2 in this Handbook) and described in the table below.

Mixing means the qualitative and quantitative data/findings influence each other, leading to an understanding that couldn’t be reached by one set of data/findings alone.

For example, you can mix research questions by having multiple qualitative and quantitative questions centered around a common topic. You could understand the qualitative and quantitative characteristics of a single research question or use two methods to address different but related research questions. You can mix sampling techniques together (aka random sampling and non-random sampling). You can even mix the data you collect within a single method (like collecting qualitative open-ended data and some quantitative closed-ended data in a single interview).

You can also mix your data analysis process together (such as using qualitative data analysis principles when analyzing quantitative data, an idea covered more in Topic 4). Finally, you can even mix how you interpret your qualitative and quantitative findings using something known as a joint display (covered in the guide below and in this article.)

Guide 20: A Joint Mixed Display Checklist

The guide below can help you understand why and how to mix methods together. But, ultimately, it’s up to you, the researcher, to figure out what, when, and most importantly, why you want to mix methods together. You might need to blend or combine some elements together given your constraints and timelines and that’s okay. Be very clear and intentional about what elements need and should be mixed.

Guide 19: Choosing a Mixed Design

Mixed methods research is flexible. You, the researcher, are allowed to shape and mold your mixed methods study how you need.

Moving beyond definitions, if your stakeholder asked you why you feel a mixed methods approach is best for a problem, topic, or hypothesis they have, how would you explain its strengths and risks?

‍

Why Mix Methods?

There are many beneficial reasons for designing and executing a mixed methods study. The strongest reason is that the qualitative and quantitative data & findings work together, allowing you to understand your research questions stronger than one method alone, leading to meta-inferences. It’s blend of understanding using both the constructivist and realist research philosophies.

Mixing methods gives you a clearer, deeper understanding of your research questions that couldn’t be achieved by one method alone.

Other reasons to mix methods are listed below. It’s also possible to mix methods for multiple reasons based on your study goals and research questions. All of these reasons are covered throughout this mixed methods Handbook. And as with all things research, mixed methods research isn’t without its weaknesses. The list below briefly describes these downsides (they’re also covered throughout this current Handbook).

If the positive reasons resonate with the needs of your stakeholders, and you feel confident you can manage the negative reasons, then a mixed methods study might be the best thing. Keep reading this Handbook to get a closer look at mixed methods or use the guide below to start prepping for your study.

Guide 19: Choosing a Mixed Design

The focus of this Handbook is defining, designing, and understanding how to use mixed methods research as a UX researcher. However, the section above deceivingly hides an important and evolving debate: can you actually mix a qualitative and quantitative method together?

‍

The Pragmatic 3D Glasses of Truth

As mentioned in their respective handbooks, qualitative and quantitative research make different assumptions about the truth and your ability to understand that truth. Qualitative researchers assume a constructed version of the truth, where you must interpret and understand the world around you. On the other hand, quantitative researchers take a realist or objectivist stance which suggests that the truth is out there, independent of people, and is waiting to be discovered.

Researchers debate if you can actually mix the qualitative and quantitative philosophies together.

While both sets of researchers respect each other, discussion around mixed methods is where things get interesting. Taking a mixed approach means you’re trying to bridge together the two research philosophies. You’re trying to blend experiences & stories with numbers & variables. The debate at the heart of mixed methods research asks: Can you truly mix the qualitative and quantitative philosophies to come to a more complete understanding of the truth?

To make sense of this, let's introduce the third type type of scientific reasoning: abductive reasoning. This type of reasoning exists somewhat between inductive and deductive reasoning. Abductive reasoning works to offer the most logical and pragmatic explanation given limited and incomplete data. It’s about making the best inference with what little you have. For mixed methods research, abductive reasoning fuels the pragmatic research philosophy. Below are some quick facts about the pragmatic research philosophy, and you read more about the pragmatic research philosophy here and here.

Quick Facts about the Pragmatic Research Philosophy

• It assumes that one method alone is not enough to understand the complex world of people and their experiences
• Its focused more on applying, doing, solving than purely understanding
• Its focused on a specific context, environment, or problem
• Its goal is something tangible to benefit others and to use research in action
• Its not as concerned with the theoretical or abstract side of human nature/reality
• It’s focused on how to study something, rather than the why to study it
• It places greater emphasis on the outcomes and actions taken after the study than the causes of what was learned
• It requires a mixed methods approach to make the most of qualitative and quantitative data together

You can think of the pragmatic research philosophy as a pair of 3D glasses. The colored lens represents either qualitative and quantitative research. Using one lens alone means your brain doesn’t see a fully three-dimensional image. But if you use both lenses together, you’re able to see an almost tangible, leaping-towards-you image. You know the image isn’t actually three-dimensional but for you, it doesn’t mean it’s any less real.

To extend the 3D glasses metaphor further, if the quality of your glasses or the content you’re looking at is of poor quality, you’ll only ever get a distorted, skewed image (or understanding). But if you know why you want to mix methods and take care to design an appropriate mixed methods study, your pragmatic understanding of your research question/topic will be incredibly rich and meaningful, more-so than a set of individual or separate qualitative and quantitative findings.

“A meta-inference is an overall conclusion, explanation, or understanding developed through an integration of the inferences obtained from the qualitative and quantitative strands of a mixed methods study.” - Manfred Max Bergman

The 3D image that you see is known as a meta-inference in mixed methods literature. Above is a direct quote defining meta-inferences from Manfred Max Bergman, a previous editor of the Journal of Mixed Methods. And you can read more about meta-inferences in this limited preview of “Advances in Mixed Methods Research: Theories and Applications”, edited by Manfred Max Bergman, or in this article.

A meta-inference is a finding, insight, pattern, or understanding that can only be obtained by mixing qualitative and quantitative data together.

The meta-inferences you arrive at could be one part of the findings coming out of a mixed methods study. You’ll also have qualitative and quantitative findings for both sets of research questions. But your meta-inferences are answers to mixed methods research questions.

‍

Mixed Methods Research Questions

Every mixed methods study you run will involve collecting some qualitative and quantitative data. These data can turn into findings that answer separate research questions (qualitative data to generate qualitative findings to answer qualitative research questions and a similar pattern for the quantitative element).

But the data and findings could also be used to answer the same, mixed methods research question. The answer to a mixed methods research question is a meta-inference or an answer that’s only possible or complete because there are qualitative and quantitative findings (you can read more here).

Based on the type and points of integration you need in your mixed methods study, you might have an overall mixed research questions to address (outside of your contextual qualitative and quantitative research questions). The table below outlines some of these mixed research questions, based on the type of integration a mixed methods study uses.

Check out “Property: Integration” in Topic 2, Topic 3 , or Topic 4 in this mixed methods Handbook for more on integration types.

You could also have several kinds of mixed research questions in one mixed methods study. And these mixed research questions could sit next to un-mixed research questions (like when you collect some un-mixed qualitative data to answer a tangential qualitative research question within a larger, mixed methods study). It's up to you, the researcher, to write qualitative, quantitative, and mixed research questions.

‍

Closing Thoughts

As discussed in earlier Handbooks, qualitative and quantitative studies have different strengths and weaknesses. But it’s not as simple as just picking two or more methods when taking a mixed approach; it’s about how your qualitative and quantitative are combined, integrated, or mixed. Let’s understand the many intersecting properties before taking a mixed approach. Then, using those properties, let’s review the most common and powerful mixed designs that you can apply directly to your work.