Guide 01

New

Guide 01

Research FAQ for Stakeholders

A handy resource that you can use to educate stakeholders about experience research

Trigger

Use when joining a new, immature research culture; use or review during stakeholder interviews.

What is experience research?

Experience research is an applied science to understand how people perceive, use and build relationships with products and services. You can read more in this Handbook.

‍

What can experience research do?

Below are some main ways experience research can influence or impact products and services. You can read more in this Topic.

Experience Research Strengths

Turn good product ideas into great or delightful ideas
Catch and address bad product ideas early
Identify and fix usability issues
Uncover hidden or unexpected product opportunities, solutions, or ideas
Reduce or help prioritize options or planned work
Fill gaps or explain patterns in product analytics and log data
Track large-scale emotion and behavior

‍

What can’t experience research do?

While experience research can be a powerful way to build profitable, usable, and scalable products and services, it’s not perfect. Below are some limitations or weaknesses of experience research. You can read more in this Topic.

Experience Research Weaknesses

Can’t guarantee that products or services will be profitable or successful
Can’t guarantee product retention or engagement
Can’t accurately or reliably predict the future
Can’t establish causation easily (as “x caused y to happen”)
Can’t make moral judgments based on the data (the data are what they are)

‍

When should you conduct experience research?

As early and as often as you. To get the most value for the cost/effort of research, conducting research at least 1-2 months before something should be released is best.

‍

How often should you conduct experience research?

As often as you can. Try to budget at least 1 study per month. Keep in mind, there are also reasons to avoid or skip research.

‍

What is a research study?

A research study is a structured process for answering specific research questions or testing research hypotheses. Every study has the following phases: aligning and agreeing on what to study, recruiting participants, designing a study using research methods, collecting qualitative or quantitative data, analyzing the collected data, and delivering findings, insights, and recommendations in a report.

The table below contains the minimum parts needed to conduct a research study. You can read more in this Handbook.

‍

What are research methods?

A research method is a procedure, technique, or series of explicit actions to collect specific data. Most methods are either qualitative or quantitative.

Research methods also have deeper, philosophical assumptions and beliefs that affect how to collect, analyze, interpret, and report findings. You can read more here.

‍

What is qualitative research?

Qualitative research is focused on understanding a concept, idea, or experience from the perspective of the people you’re building for. Commonly, researchers use purposeful, non-random sampling to recruit and learn from participants. Common qualitative methods include interviews, observations, focus groups, and field research.

Other important ideas to consider when designing a qualitative study are listed below. Jump to this Handbook for more.

‍

What is quantitative research?

Quantitative research uses numbers to describe people's expectations, attitudes, and behaviors towards a topic, product, or experience. Commonly, quantitative research uses some form of random-sampling to recruit and learn from participants. Common quantitative methods include surveys, experiments, unmoderated usability testing, tree testing, or heat-mapping.

Other important ideas to consider when designing a quantitative study are listed below. Jump to this Handbook for more.

‍

How long does it take to conduct a single experience research study?

A tactical research study can take a few days to a few weeks, while a strategic research study can take a few weeks to a few months, based on how many research questions there are, the number of research methods used, and the timeline for using or applying what’s learned from research. Many businesses conduct tactical research regularly while conducting strategic research to help plan or ideate ideas for the next business quarter.

‍

What is a fruitful research question?

A fruitful research question is valuable to your stakeholders, meaningful to your participants, and sustainable for you to study. In addition, a fruitful research question requires that you understand and align how the data and findings will be used or applied after the study ends. Below are additional qualities. You can read more in this Handbook.

Qualities of Fruitful Research Questions

Valuable to your stakeholders
Aligned and agreed upon with your stakeholders
Meaningful to your participants
Clear description or understanding of who the ideal participant is
Sustainable for you to study (over time)
Specific but not answered with a yes/no, a single statistic, or a short response
Actionable because the post-study learning is designed to be used
Addressable within your constraints
Gives guidance and boundaries when collecting, analyzing, and reporting data
Focused on a product-related problem or a prediction about the product

‍

What is participant recruitment?

To collect data to answer your research questions or topics, you'll need to recruit and learn from participants. The act of recruiting, contacting, or scheduling participants is known as recruitment.

Recruitment uses ideas from sampling theory, where you define a population or segment of interest, and select or sample participants from an accessible sampling frame. From the frame, you use a sampling technique to select a desired or preferred number of participants to contact, known as your sampling pool.

From the pool, you learn from a handful of participants which is your sample. You can read more about sampling and the diagram below here.

‍

How many participants should you learn from?

Below are some practical sample sizes to consider using in your research. Each table corresponds to a different type of research or sampling technique. You can learn more in this Handbook in Collection 2.

For qualitative Research

For quantitative research, when using Random sampling

For quantitative research, when using non-random sampling

‍

How can findings be reported?

Select and align on an appropriate reporting format based on your available study timeline, tools available, type of research, and urgency of findings. You can read more about reporting in this Topic.

‍

What does it mean to analyze your data?

Data analysis is a process of breaking down the patterns, trends, and relationships in your collected data to answer specific research questions or test research hypothesis.

First, your collected data must be cleaned or structured to make analysis easier and more accurate. Then, based on the research questions and study goals, specific bits of data are reviewed, compared, labelled, summarized, or visualized to better understand the meaning or pattern within it. Finally, the patterns are synthesized into a coherent and data-backed conclusion or viewpoint that should answer the research questions or help reach the study goals.

Qualitative data analysis (QDA) uses qualitative coding or the process of labelling and describing qualitative, text-based data.

Quantitative data analysis requires a clear plan for analysis before collecting data, as you can't add data after data collection ends. You might describe or graph the data in your exploratory data analysis (EDA) phase or use significance tests to see if your sample data is a good estimate for your population or segment.

Jump to this Collection for more data analysis.

‍

What are research recommendations?

Recommendations are subjective but data-based considerations or suggestions to apply what was learned to the product or decision-making process. The research question(s) and post-study goal(s) help define what should and shouldn’t be recommended after a study ends. Research recommendations are optional and need to be requested from stakeholders or those paying for the research.

‍

What should stakeholders or non-researchers know before conducting experience research?

You can read more about reframing research here and building your research culture here.

Starting Research Expectations to Discuss and Align On

Research is best done as early as possible to get the most value and impact
Research is almost impossible to do well the day or week before a release
Research has several phases (such as align, recruit, design, etc.), and stakeholders can get involved in every phase
Fruitful research requires a clear, aligned understanding of how the findings and data will be used after its collected
Research recommendations are judgments based on data and its interpretation, not something definitive
Making and presenting research reports aren’t as impactful, valuable, or quick as stakeholders being directly involved in a research study
Secondary data can be as valuable and often faster than collecting new, primary data for every research question

‍