Four Goals to Reframe Research

If you struggle to get any research done, you’ll need to directly address your stakeholders' opinions, concerns, and expectations about research. Things won’t get better or easier if you don’t make the effort to reframe what research can be for your stakeholders or the business.

What can seem daunting can be broken down into four goals (above). It’s important to know that, in practice, the goals won’t be reached linearly. You might have to jump around, but that’s okay. Your aim is to reach all four goals through discussion, collaboration, and education, not to follow the goals perfectly.

So, how exactly do you help your stakeholders view experience research as a powerful accelerator for building great products and a critical safeguard for avoiding costly mistakes? Goal 4 is the topic for the entire next Handbook, so let’s focus on the first three goals here.

‍

Goal 1: Understand Experience & Expectations for Research

The first goal for reframing research is to understand where your stakeholders are today. Not physically, but their expectations, assumptions, emotions, and experience with research. Are you the first experience researcher they’ve worked with or the tenth? What does “research” mean to them? What expectations or hesitations do they have? You can’t effectively reframe research without first recognizing how your stakeholders think about research.

You might have to educate your stakeholders out of any negative ideas they have about research first.

In businesses where research is new or uncomfortable, you might have to first educate your stakeholders out of any bad ideas they have about research. One common but incorrect idea is that research is luxurious, discretionary, or extraneous. But this approach is similar to building something in the dark.‍

In a business context, building in the dark means limiting or ignoring outside feedback, perspectives, and opinions when building a product or service. Stakeholders rely more on their limited data or only their available skill sets to make product decisions. In some ways, building in the dark is like groupthink, a social phenomenon where everyone agrees to continue the status quo. In practice, you might hear stakeholders say “this is how we’ve always done things.”

In the dark and with groupthink, you’ll find product ideas or decisions aren’t discussed but directly implemented. In many businesses, many people benefit from the status quo, so there’s very little incentive to try new ideas. There isn’t room for growth, innovation, or discovery.

Interestingly, the businesses that choose to consciously build in the dark also feel the direct negative consequences when products don’t work as expected. Their products have low sales or retention. And products are built in the dark, stakeholders and businesses are implicitly making a promise to themselves: to borrow future work time to fix past mistakes.

If your stakeholders build in the dark, they’re promising to borrow future work time to fix past mistakes.

Below are some indicators to recognize if your stakeholders build in the dark.

Building in the Dark Stakeholder/Business Indicators

• Stakeholders rarely see or interact with people that use the product
• Stakeholders make decisions primarily or only with internal data sources
• The business doesn't test and refine products before released
• The business doesn't dismiss or eliminate negative or unnecessary features, ideas, or products
• Product decisions are made without collaboration or input from other teams or departments
• The product roadmap isn't influenceable by the people that use the product
• Goals and milestones on the product or business roadmap mostly or only benefit the business

If many of the indicators hold true where you work, it’s a clear sign you’ll need to directly discuss and learn about the reasons for the lack of outside feedback. You'll have to deal with research objections.

Below are some of the most common objections to research. But instead of taking these objections at face value, you’ll need to learn more about what your stakeholders are actually objecting to. Instead of confrontation, the right column contains questions to get your stakeholders talking.

Treat conversations with stakeholders the same as any qualitative interview: be patient, take notes, and probe into responses. The goal isn’t to prove your stakeholder’s objection wrong but to break down their objection to find what’s important to them.

Once you understand where stakeholders or the business are now, your next goal is to replace/update negative ideas about research or strengthen/expand the positive ones.

‍

Goal 2: Discuss & Educate about what Research Is

While you might not be able to start from scratch, you’ll need to provide good and helpful ideas about research for your stakeholders to absorb. What is research? What can and can’t it do? What does good research look like? While there are a ton of ideas to introduce, use the guide below as a cheat sheet for core topics to educate on. Copy/paste the ideas into a slide deck or explain them during a meeting.

Guide 1: Experience Research FAQ for Stakeholders

You’ll notice that the guide doesn’t focus heavily on research methods. After all, if you can’t get your stakeholders to even agree that research is important, what good would it do to list out research methods you could use?

If you want a great reference all about research methods, you can purchase the "Universal Methods of Design" by Bella Martin and Bruce Hanington.

Reframe research as being a modular, lightweight,  customizable, and scalable process.

Instead, look to reframe research as being a modular, lightweight, customizable, and scalable process. You need to explain how value can be generated with small samples, how methods can be mixed around and together, and how studies can range from a few hours to a few days.

Every part of the research process should be seen and educated as a discrete block or unit, from alignment to delivery. It doesn’t mean analyzing data before collecting it. But it does mean how you analyze, with who, in what ways, and other considerations, are all flexible and modifiable. In one study, you might be the only person analyzing your survey data; in the next, you might have four stakeholders help code the open-ended survey responses.

Other strategies to make research feel and become more modular and flexible are listed below. You can learn more about each by jumping to their respective handbooks or guides.

Strategies to Showcase Research as being Modular and Flexible

• The value generated from “small” samples and the drawback of “large” samples (for more, jump to Collection 2, Handbook 2)
• Implement study breakpoints for qualitative research (for more, jump to Collection 2, Handbook 2, Topic 3)
• Use unmoderated and remote methods or data collection techniques (read more here)
• Create different study compensation rates (less for tactical, more for strategic research), with more coming later in a Handbook on recruitment
• Diversify how you report findings (from email highlights to formal presentations) (for more, jump to Collection 6, Handbook 1 for more)
• Explain how research can be used for different reasons: to understand the problem better, to validate solutions, to discover hidden opportunities

Research education is a good place to start, but you need to go further and contextualize and figure out how research can work in your immediate context.

‍

Goal 3: Contextualize and build Research Practice Together

Contextualizing research means aligning on answers to three questions with your stakeholders or business:

• Who’s asking for and benefiting from the research?
• What’s the explicit process for conducting research?
• What can your stakeholders expect or request from research?

Let’s start with the first question and work our way down.

Based on how your business is structured, you might be conducting research for one specific team or product or across several teams and products. The former is an embedded or product research model, while the latter is a centralized research model.

‍

If you work in an embedded or product model, you're sticking with one core set of stakeholders and a specific product focus. You might run a survey in one study or fly to a site for another, but your focus is always on this one product or service. Embedded research models tend to mean better stakeholder-researcher collaboration, a more intentional research strategy, and a consistently improving product or service.

On the other side, you might need to work in a centralized research model.  In this model, stakeholders – or even entire teams, departments, or businesses – come to you with questions to study. Think of this model as being an agency or consultancy model inside a business. From study to study, you’ll focus on different parts of the business.

For example, you might study how to improve the onboarding experience one month and then run a card sort for the marketing team the next. Centralized models can be effective when there are limited researchers and resources (as is common with immature research cultures, read more here).

The next question has to do with your research operations or how the processes, tools, and people who do research all fit together. How do you actually recruit participants? What are the steps for legally and quickly compensating someone? What are the tools you have access to and how effective are they? Understanding answers to these questions help you understand what’s possible and what’s not.

Other topics to align on and understand your research operations are listed below.

Starting Research Operations Topics to Discuss

• Available sampling frames or participants lists for study recruitment
• Relevant sampling or recruitment rules or constraints (such as you can only contact people who've opted into research or email only 250 customers at a time for a survey)
• Possible study compensation rates or strategies
• Legally and financially compliant methods for paying participants post-study
• Available tools (for research methods, note taking, file sharing, storage, etc.)
• Available research budget and how often it can or will be refreshed
• Available data sources (such as product analytics dashboards, quarterly reports, external research vendors, or articles)

In immature research cultures (covered more in Topic 4), you’ll find that you have a few resources and tools. Don’t let this discourage you. Pick the weakest or most unreliable part of your research operations and discuss its negative impact with your stakeholders. What’s harder or impossible for you to do? What makes research take longer than preferred?

Similar to discussing any research objections, you’ll have to discuss issues and challenges in your research operations with your stakeholders. Check out this great resource from the global ReOps (Research Operations) community and this upcoming book by research operations leader Kate Towsey for more.

The final question is, “What can your stakeholders expect or request from research?” You'll run into issues if you don’t properly set expectations. Your stakeholders might demand a specific type of data or research method, not knowing you might not have the ability to meet their demands. You might be asked to come “UX” a product the day or week before release. Worse, your stakeholders might view research as definitive or unchanging. Without setting expectations, you’ll have to re-educate your stakeholders about what’s possible and practical every time they ask for research.

Below are some starting research expectations you should align with your stakeholders or the business. It’s a starting list, so customize and add new ideas that are relevant to where and how your research.

Starting Research Expectations to Discuss and Align On

• Research is best done as early as possible to get the most value and impact
• Research is almost impossible to do well the day or week before a release
• Research has several phases (such as align, recruit, design, etc.), and stakeholders can get involved in every phase
• Fruitful research requires a clear, aligned understanding of how the findings and data will be used after its collected
• Research recommendations are judgments based on data and its interpretation, not something definitive or objective
• Making and presenting research reports aren’t as impactful, valuable, or quick as stakeholders being directly involved in a research study
• Secondary data can be as valuable and often faster than collecting new, primary data for every research question

It’s also important to explain and discuss the very limits of what experience research can do. For example, casual statements like “a change in X caused a change in Y” are hard to prove or support. People are complex, changing creatures. Everyone is unique in their own way. And the world is built up of changing, multidimensional, and interconnected systems.

Lurking variables or the things you didn’t measure or record can influence the patterns and relationships you find in your data. As an experience researcher, you can’t always establish causality, but you can demonstrate correlation. Things are related or linked to each other, not that "this" caused "that". Check out this approachable PDF for more on how correlation isn’t causation.

It’s your interpretation of data that’s valuable, not the data by itself.

Because experience research is a form of applied science, you need to also explain how you, the researcher, must interpret the data and come to conclusions. Data without interpretation or context is worthless. It’s your interpretation of data that’s valuable, not the data by itself. With all of its rigor and standardization, this also means that science can’t make moral judgments.

For example, you might see data of an addictive, harmful product or feature being very profitable. But you’re the one interpreting data as being harmful or negative. The data are what they are, regardless of your morality or empathy. You have to balance between being objective and human-centered when collecting, analyzing, and interpreting data.

One final expectation you should set with your stakeholders has to do with the future. Researchers can objectively help stakeholders or businesses foresee the negative consequences of their decisions and take action to correct them. But this doesn’t mean that you can see into the future or make absolute guarantees. No amount of research can guarantee people will buy or use this product or service, no matter how usable it is.

No experience researcher can accurately or reliably predict the future.

If research is done poorly or with bias, an experience researcher is no better than a fortune-teller for predicting the future. But if research is conducted with collaboration and intention, then that same researcher can recognize patterns, probabilities, and pathways for product success.

The takeaway is to position yourself as a partner when building products, not as an expert. Becoming a researcher that’s collaborative and structured is the goal, not to be seen as someone with a crystal ball or omnipotent.

Reframing research as something dynamic and flexible is the first clear step to building a sustainable research culture. But what exactly is research culture? And how can you grow or shape yours?