Why stakeholders don’t like paying for research

Let’s begin from a place of honesty: stakeholders don’t often like paying for UX research. While much of the free and paid content online might inform you of this sentiment, let’s dive deeper into why they don’t want to pay. Our research has shown that there are three clear reasons:

• The emotional impact of research
• The lump sum or recurring cost of research
• The delayed or non-obvious impact or value generated from research

First, emotions. For many stakeholders, paying participants to tell them what they hate about the product can feel counterintuitive. When a senior product director has spent years trying to create a human and thoughtful product or service, harsh criticisms about that product can cut deep.

When “negative” feedback comes out of research, some stakeholders have a knee-jerk reaction to avoid or limit research. From their lens, it’s much simpler to focus on the data they do have and their “product intuition”.

Hearing lots of feedback that a product stinks can make stakeholders avoid or limit UX research.

What’s also confusing and contradictory is that the stakeholders who only trust UX research when the sample sizes are “large” (such as after 100 interviews or 10,000 survey responses), will still act against research findings that challenge or question their product roadmaps and strategies. (This is one reason that the most impactful UX researchers balance the fine line between being objective and respectful when reporting. You can read more in this Handbook.)

The next reason is that UX research has a cost. When you compare the cost of research against everything else a business spends its limited resources on, you’ll notice that it’s one of the smallest expenditures for a business. One year of  of iterative, high-quality UX research might cost $10,000 but hiring a senior engineer can easily cost over $100,000. While the engineer does has a tangible impact on the product, regular well-structured research can help validate and define the product roadmap for months to come, helping ensure that the engineer is working on the most important things.

The cost of compensating participants is on top of any additional costs of research tools. Paying for a yearly or monthly subscription for usability testing tools, a survey design platform, a research repository, and a remote video conferencing tool can all add up. What’s even more counterintuitive is when your stakeholders demand or expect a large sample size. As discussed throughout Collection 2: Recruit, larger samples will take longer to recruit for and be more costly. After all, all of those participants will want some amount of compensation for their time.

One final reason that most stakeholders don’t like to pay for research is how slow their return-on-investment (ROI) can. Covered briefly in the last Topic, it can be hard to establish clear, financial impact from research. Did offering $50 for 10 interview participants result in more business impact than the cost of that $500? It’s hard to tell.

Unfortunately, most UX research results in positive financial impact in the long-term. Months and years of UX research will always shape the product roadmap and help stakeholders make smarter decisions. More time will be spent building the right thing, instead of fixing usability and preference issue. However, most stakeholders measure impact on the quarterly basis. Can you really showcase that your usability test and interviews resulted in more money in just 90 days?

UX researchers are stuck between asking for compensation funds and tangibly showcasing the impact of research in short timelines.

As it stands right now, there isn’t a one-size-fits-all answer to help your stakeholders measure the ROI of research. There are several resources linked below but know that the cost of research and the return of that research will likely be an ongoing topic of conversation with your stakeholders. You can use the last Topic and this Topic (”How to Reframe Research for Stakeholders”) to engage in conversation about UX research and its value.

Research has also shown that many research-resistant stakeholders don’t fully understand what study compensation even is and why it matters.

‍

Study Compensation Defined

Terms like study payments, reimbursement, remuneration, or incentives can make a simple idea more opaque. But what exactly is study compensation?

Study compensation is an exchange between someone’s time and the value of the data provided in that time.

In simple terms, all study compensation is an exchange. The exchange is between the time required for someone to provide data in a research study and the value of that collected data.  being offered (typically money). Instead of doing something else, someone took the time to tell you about their experiences.

Note that this exchange is always based around time. Time is the one true constraint everyone deals with — participants, researchers, and stakeholders alike. Instead of spending their time doing whatever they were going to do, someone took the time to tell you about their stories and experiences. If your study takes a very long time (think multiple hours or even days), more study compensation should be offered because you’re ultimately collecting more data.

The exact process for paying and tracking participant compensation is outside of the scope of this book. You’ll need to sit down with your finance/budgeting partners to see how you should correctly and legally pay participants.

But is even ethical to pay participants when they take part in your research?

‍

Data into Profit

On the surface, UX research can seem kind-hearted and altruistic. In practice, however, research helps a business make smarter decisions in the pursuit of financial goals. That’s neither good nor bad but it’s an important reason of why UX researchers are paid well and why seasoned product leaders turn to researchers for their insights.

You need to pay participants because their data will be used to make financial decisions.

While it’s hard to establish a return-of-investment (ROI) on UX research, it’s clear that research can save time and money. UX research can help a business move up timelines if a product shows promise, which can lead to greater revenue and brand awareness. UX research can also save the reputation of a business when findings help refine bad products into great ones. If done regularly, UX research can also help expose hidden opportunities and unexpected friction that can be transformed into attractive and usable features, products and services.

New UX researchers especially need to realize that their work supports a business. If you can pay, then you must pay participants. Every UX researcher who works in a for-profit business has a professional obligation to stand up and fight for proper study compensation. Without it, UX research can feel like taking someone’s data instead of a consenting, positive interaction. (If you work with populations that can’t be paid, you can ignore this sentiment.)

Beyond the ethical obligation to pay participants, offering study compensation can be beneficial for you when you recruit participants.

‍

Compensation Leads to Better Recruitment

A clear win that comes with study compensation is your ability to more effectively select or filter who participates in your study. Imagine two different researchers running the same study – a 60-minute remote video call. What’s different is one researcher offers $50 as study compensation while the second offers nothing.

You don’t have to be a behavioral economist to know that the $50 amount will not only draw more attention (or study awareness, the first phase in the participant experience), but more potential participants will reach out to get involved. If a lot of people are interested in your study, you have the luxury of reviewing and selecting only the most informative participants to learn from.

Offering study compensation can also sway those hesitant to participate to show up and provide their feedback. If you struggle to hear feedback from those “average users” (aka those not super passionate or angry about your product), compensation can help you attract these people. In these cases, study compensation helps address the participant nonresponse issue (covered more in this Topic).

Compensation can also lead to fewer no-shows or drop-outs. For many participants, knowing that completing a longer survey or remote semi-structured interview will lead to some amount of money means they’ll be focused and engaged for longer. Consider offering higher compensation when you know your study will be longer than 60 minutes or take place over several days. Yes, you’ll always have some participants abandon your study but it can help keep this number low.

Study compensation can result in fewer no-shows and drop-outs.

If the entire participant experience was positive and compensation was accurately received in a timely manner, you can also increase the odds you can re-learn from past participants. If someone had a good experience (and made some money!), they’re much more likely to want to participate again. If you need to pilot test an interview guide or cognitive test a survey, past participants can help, especially when you’re running low on time.

While compensation rates can positively affect your recruitment, they tend to be static. Every participant — no matter how diverse or unique — gets the same $50 gift card. But even new UX researchers quickly realize how diverse every participant is. To move beyond offering one, financial compensation option, you should look to offer multiple ways. Your compensation should be seen as an incentive, tailored to each participant.

To do this, you need to first understand that many participants take part in research beyond the financial compensation offered.

‍

Participants are more than money-oriented

Take a moment and ask yourself this question: Why did your last participant show up to your study? Why did they do your interesting research activities instead of something else? Was it really about the money or was it really about something else?

Think about participant motivations as being altruistic plus something else.

The vast majority of your participants have a baseline motivation of altruism: they want to help. What gets more complex is why else they want to participate in your research. After all, your participants don’t have endless time, resources, and attention to help everyone they want. The table below shows some common motivations for people showing up to do research.

You can use this list as a place to start about why people show up to participate. Once you know what their motivation is, you can tailor your session, your activities, your energy, and how you build rapport to use that motivation rather than dismissing or ignoring it. Keep in mind that the motivations to participate might differ if working with public/government or an internal employee experiences. There might be legal or financial rules on how you can compensate participants so work with your legal, finance, or human resources stakeholders to avoid issues.

You should also track participant motivations over time to find patterns. What’s the most common reason people want to participate? Can you identify any similar motivations within a specific segment? How can use these motivations to improve your recruitment efforts? If you can, try to use the reasons or motivations for participation in future research recruitment efforts.

To go one step further, you need to connect those diverse motivations to different types of study compensations. If communicated correctly, your study compensation becomes an incentive, attracting different kinds of participants.

‍

Different Compensation Incentivizes Different People

Imagine you’ve volunteered to be a participant in a study. Before the study starts, you’re able to review and pick from one of three study compensation options:

• $75 for a 60-minute study
• $50 for a 60-minute study and a 30-minute chat with the product owner
• $40 for a 60-minute study and a 20% off at the company’s online store

Which one would you pick? Which option do you find irrelevant or uninteresting? Notice how your interest changes when you look at the options. Is talking to the product owner exciting or scary? Does the online store have cool products? Will the money be sent as cash, as a gift card, or via digital payment? While you might record these as being study expenditures, your participants will should view one of the options as an incentive.

Your participants will view your study compensation as something binary: it will or won’t be seen as a meaningful incentive.

An incentive is something that triggers someone to behave or respond in a particular way. For your participants, your study compensation is something binary: it will or won’t be seen as a meaningful incentive. If it’s seen as an incentive, your compensation will nudge someone to participate. If it isn’t, you’re stuck with no participants and money left unused.

Make sure to ask any current participants their thoughts on the current compensation options. Do they feel appropriate? What options stand out and why? In research mature cultures, offering different compensation options can help increase recruitment efficiency while stretching budgets.

Going back to the three example compensation options, your potential participants will also react differently to each option. In their heads, they’re also asking themselves questions about each option. If you only offer financial study compensation, you’re limiting the range of people you could be learning from because not everyone is incentivized by money.

‍

Possible Compensation Options to Try

Other types of study compensation are listed below. If you can, try to pair some amount of financial compensation with a sustainable, valuable non-financial study compensation. Using both maximizes your chances of recruiting informative participants quickly. You can learn more about different compensation options here.

Financial Study Compensation Options to Try

• One-time cash payment (digital or physical)
• One-time gift card (from your company or another)
• One-time prepaid debit card (can be used anywhere)
• One-time “coupon” (ex. “no subscription free for one month” or “next two deliveries are free”)

Non-Financial Study Compensation Options to Try

• Discounts (to use at the business)
• Sneak peeks (limited access to see, use, or influence early versions of a product)
• Short conversations/meeting with the core team (like a product manager or UX designer)
• Company/headquarter/site visits
• Recognition of participation across social media channels
• Swag (such as stickers, coffee mugs, water bottles, sweatshirts, etc.)
• Being invited into a research panel

The two lists above are nowhere near complete. In most situations, you should consider adding more contextual compensation options.

‍

Ideating or Adding compensation options

Your resources and creativity constrain the study compensation you provide to participants. If you are going to compensate in a unique way, you can use the list of qualities below to ensure that you’re on the right track:

Qualities of Appropriate Study Compensation

• Matches the brand of the company (bigger/well-known company means larger incentive)
• Matches the study’s time demands (longer studies means larger incentive)
• Matches the study’s effort (more intensive studies means larger incentive)
• Matches the comfort needed to participate or how intrusive the study is (less comfort means larger incentive)
• Easy for participants to receive (quickly, reliably, and doesn’t create a logistical nightmare for you to disperse or manage)
• Easy for you/your team to track (to see what works and what doesn’t and record for financial accounting purposes)

One of the best places to ideate study compensation is with current or in-progress participants. If you’ve been able to recruit even a handful of participants, you want to understand why they wanted to participate. Below are four questions to ask in your qualitative studies to ideate and improve your study compensation structure (and recruitment overall).

Four Questions to Ideate Study Incentives (and Improve Recruitment)

• (Start of session) How did you learn about the research study today?
• (Start of session) Why did you choose to participate today?
• (End of session) How could we make it easier for you to participate in future research?
• (End of session) What could make you more excited to participate in future research?

You want to ask these questions and start tracking the responses. Over time, you should start noticing issues in your recruitment engine, recruitment brand, compensation/incentives, or the overall participant experience. Additional questions can be found in guide below.

Guide 16: Mapping the Participant Experience

One final note: If you work with a vulnerable or marginalized populations, be aware that you might need to offer different kinds of compensation. For example, if you’re trying to understand known drug-users or those with addictive gambling issues, consider the use of gift cards or a preloaded public transit card instead of cash (which can exacerbate their issues).

If you work with a vulnerable research population, your study compensation needs to be appropriate and meaningful.

Other vulnerable or unique populations are listed below and more reading can be found here.  Please note that the list above is not exhaustive and your research context might require you to consider different compensation methods based on how vulnerable your research population is.

General List of Vulnerable Research Populations

• Children (between the ages of 0 - 13)
• Young adults (between the ages of 13 - 17)
• Older adults (typically older than the age of 70)
• People with unique mental/physical needs or conditions (such as low visual acuity, autism, etc.)
• Racial, gender, or sexual identity minority groups
• Indigenous populations
• Low-income populations
• Undocumented populations
• Incarcerated persons or prisoners

Based on our research, many research-resistant stakeholders propose a chance to win a gift card as meaningful study compensation. Recruitment language like “Complete this survey for a chance to win a $25 Amazon gift card.” However, lottery-based compensation strategies are one of the worst, most ineffective ways to improve recruitment.

‍

Avoid/limit lottery-based compensation strategies

The easiest way to show disrespect to potential participants (besides not offering any compensation) is by instituting a lottery-based compensation structure. You might’ve seen such structures online but the most common is informing potential that they’ll be entered to win a gift card after they complete a study (like a $25 Amazon gift certificate).

You don’t want to take a boring survey for the chance of winning a $25 Amazon gift card, so why would your participants?

Avoid lottery-based compensation for two reasons: First, it sends the wrong signal to potential participants. Yes, lottery-based compensations can stretch a limited recruitment budget but it can appear disrespectful and insincere, especially when the business posting the survey generates revenue. And secondly, the implicit chance of winning doesn’t match the explicit time someone gave up to participate.

When a participant completes a survey for a chance to win a gift card, they’re not sure how many other people have already completed that survey. They’re not sure if their chances of winning are 1-in-100 or 1-in-100,000. From their lens, lottery-based compensation is a poor way to nudge them to participate.

Does this mean you should offer the highest study compensation rates possible? Not exactly. In the next Topic, let’s discuss a scalable and flexible approach to compensating participants.