Compensation works but It’s a Trade-Off

The last Topic discussed how and why offering study compensation positively benefits your recruitment processes. But you, the UX researcher, are stuck. Higher compensation rates will generate more study awareness but it’ll make research-resistant stakeholders uncomfortable and it’ll quickly wipe out your limited recruitment budget.

You can’t keep raising the study compensation rate without also losing your entire recruitment budget.

For example, if you offered $100 for a 45-minute interview, you’ll definitely get interest and be able to select highly informative participants to learn from. But if your monthly budget is only $500, offering $100 means learning from only five participants. While smaller samples have real value (see this Topic for more), five participants might not be enough for your stakeholders to make smarter decisions.

On the other hand, if you offer $0 as study compensation, you’ll never touch your recruitment budget. But you’ll also really struggle to generate study awareness or select more informative participants to learn from.

So, much should you pay participants? To offer a simple but contextual strategy, let’s start with the median hourly wage (MWH) for where your participants live.

‍

The Median Hourly Wage (MHW) Strategy

Before explaining the median hourly wage (MHW) strategy, there are two things to discuss that’ll help make this Topic easier to understand and apply: First, note that this section uses the United States Dollars ($USD) when describing and defending specific study compensation amounts.

There’s nothing overtly “American” about recruitment. However, the dollar is used because it’s a currency commonly used for currency conversions/comparisons. The focus shouldn’t be on the dollar amount but on why the amounts are useful and practical. The rationale behind the explicit amounts is generalizable while the amounts themselves aren’t.

Secondly, the focus is on the median wage, not the average. In most countries (or overall user populations), there are few high-income earners and many low-to-medium income earners. Those with high incomes will skew the average, leading to a wage that’s unrepresentative of the country or population you’re trying to understand. With skewed incomes, the median is more representative and it can change based on the country or population, affording you flexibility when deciding how much to compensate participants.

With that out of the way, let’s take a look at the median hourly wage (MWH) strategy. It all starts with where your users and potential participants live. Let’s pretend that your users live in the United States of America.

Next, let’s figure out the median hourly wage for people living in America. A Google search shows that in 2022, this wage is about $23.50 per hour. Let’s round this up to $25 to make the following calculations easier to understand. This $25/hour should serve as the baseline or starting point for all future compensation discussions.

Across several sources, the median hourly range in the US for 2022 sits between $16 and $31 dollars. For the sake of an example, $25 per hour is used. However, the point isn’t to focus on the explicit numbers but how this starting number can be used to establish compensation ranges.

The median hourly wage (MHW) should serve as the benchmark for all compensation discussions.

Another quick note: make sure to round to the nearest “0” or “5” digit when establishing the median hourly wage. Why does this matter? It’ll help you calculate compensation totals easier and help potential participants view the rate more appropriately.

For example, which of the following compensation options is easier for you to understand and assess: $23.50 for a 60-minute interview or $25 for the same 60-minute interview? It’s recommended to down up to offer more value and round down to stretch your limited budget.

In some situations, you don’t want the median hourly wage of where your participants live, but how much your target user population or segment earns. For example, if you work on a product that makes booking business travel for CEOs easier (such as flights, hotels, cars, etc.), you should look at the median hourly wage of your user population. CEOs tend to make a lot of money. This means the median hourly wage of those that use the product will almost always be higher than the wage of the country they live. As a UX researcher, you’ll have to recognize the context your research will happen in and compensate appropriately.

If your users are high-income earners, you might need to use the median hourly wage of your user population, instead of the countries they live in.

It might take some time to figure out what the most representative median hourly wage is. The resources at the bottom can help you get started but you might have to work with your local, regional, or national government entities for the most accurate wages. You can even start with the average hourly wage to test out what compensation ranges work best. If you use the annual salary for a country, make sure to divide it by the rough number of days worked and hours per day to arrive at an hourly wage rate. Also make sure to use any annual salaries before taxes so that participants can receive a higher compensation rate.

In some countries, if the study compensation is so high, there might be a need for participants to disclose the amounts as “taxable income”. Make sure to check for any relevant tax codes or laws before sending compensation, especially when conducting international research.

While the median hourly wage (MWH) is a good place to start compensation discussions, if you ask any UX researcher, they’ll tell you that compensation rates depend on several factors. Using the MWH, let’s now create a scale to make it easier to compensate across a range of study and recruitment factors.

‍

The Median Hourly Wage (MWH) Scale

Using the diagram below, multiply the MHW by the following values: 0.25, 0.50, 1.5, and 2. These values (a fourth, a half, one-and-a-half, and two times the MHW) will serve as anchor points to help you narrow down when and how much to compensate participants.

Let’s now fill in this scale with some real numbers. Let’s assume the median hourly wage in the United States of America is $25 per hour. Let this number be represented as “x”. Now, let’s multiply “x” by the aforementioned values to generate the compensation scale.

From here, let’s apply the rounding rule to make every point on the scale easy to scan and assess. Remember that you can round up or down the rates.

In this USA example, the highest compensation rate considered is $50 per participant, with the lower end being $6.25 per participant. The lowest amount is also $0, where you might offer non-financial compensation (such as sweatshirts or company visits; you can read more in the last Topic). Other advice about the MWH scale are listed below for easy reference.

Note that the scale is unbalanced, meaning that there are a greater range of values below the MHW. In immature or research-resistant cultures, discussion around lower rates can help improve recruitment without requiring a large amount of money.

Also, the values above should serve as anchor points to experiment and discuss a scalable compensation strategy. Don’t use it as explicit, unchanging rules. Propose the strategy and engage in your discussion with stakeholders to find what works best for you. And know that you’ll always find reasons where you need to pay more or less. Be flexible with each study and the population you’re trying to learn from.

Given that the scale is wide, you can segment it further into two sections: rates above the MWH and rates below the MHW.

‍

MHW Scale: Moderated and Unmoderated Research

Using the USA example from before, how much should a qualitative interview participant get for a 60-minute session? What about any diary studies with multiple contacts over multiple days? How much should you compensate every survey respondent?

You can split the compensation scale into two sections: the lower end for unmoderated research and the higher-end for moderated research. Let’s take a closer look at using the scale, starting with moderated research.

Values at or above the median hourly wage (MWH) should be reserved for moderated research. You’ll spend more time and attention in this type of research. This means the cost of uninformative participants (or a no-show or drop-out) is higher than with unmoderated research. Higher compensation rates allow you to attract more participants and be more selective with recruitment, while also helping mitigate no-show issues. Other reasons to use the higher end of the MWH scale are shown in the diagram above.

On the other end, lower compensation ranges are more appropriate for unmoderated research. You’ll be able to learn from larger samples. Lower rates will also help you extend your recruitment budget. One good use of the lower-end of the scale is for past participants. If you need to test an interview guide or prototype, offering some compensation to past participants can balance recruitment speed with learning.

The table below offers a more granular look at when to consider each section of the MWH scale. Once again, it’s a place to get started. If you can, discuss specific situations on how much and when to compensate participants between the anchor points. It’s in your best interest to make study compensation as contextualized and relevant to your population as much as possible.

Let’s take this strategy and apply it to the three most commonly used UX research methods: interviews, surveys, and usability tests.

‍

Interview Suggested MWH Compensation Rates

If you’re conducting interviews, the largest factor that affects compensation rates is the length of the interview. Longer interviews mean higher rates. Higher rates, however, decrease the total number of participants you can interview in one month. You need to balance interview length (or number of interview topics) with your stakeholder’s expectations around sample size to ultimately narrow down a sample size to shoot for.

Notice how all of the rates that don’t end in a “0” or “5” digit are rounded up or down accordingly.

‍

Additional Advice on Compensating Interview  Participants

• If your interview location is in someone’s home, office, or in a lab, offer a higher compensation rate to offset the intrusion (in-lab interviews can be lower if the travel to the lab isn’t too far)
• If you have more than a handful of interview topics, exercises, or nested methods (such as a card sort, usability test, or survey), offer a higher compensation rate
• If you have more than one passive observers (such as stakeholders taking notes), offer a higher compensation rate to offset the social pressure

‍

Survey Suggested MWH Compensation Rates

If you want to use a survey, note that most surveys are uncompensated. While this might sound good, this is a bad strategy for two reasons: first, it’s disrespectful because it doesn’t compensate someone for their time and data; and, second, uncompensated surveys almost always result in low response rates. And those minimal responses tend to be skewed — those most passionate or angry will respond. Offering compensation can increase study awareness and help you reach a desired sample size.

Additional Advice on Compensating Survey Respondents

• If your survey has a lot of required questions, offer a higher compensation rate
• If your survey has a lot of required, open-ended questions, you’ll need to offer the highest compensation rate possible
• If you can, offer partial compensation, such as 80% of the compensation rate if 80% of the survey is completed (make sure to put the most critical survey items in the front)

If you’re writing a survey with a lot of required, open-ended questions, consider abandoning the survey altogether and using the survey items as the foundation for a qualitative study. Your sample size will be smaller but you’ll get more richness of data, instead of forcing respondents to write a ton in the survey.

If you can’t fairly compensate survey respondents, consider turning your survey topics into a qualitative study instead.

To maximize every dollar spent on survey compensation, make them as short as possible or look to use non-financial study compensation. Or better yet, limit or avoid using surveys altogether (especially if you can’t design a survey properly. Read more about surveys in this Handbook).

‍

Usability Testing Suggested MWH Compensation Rates

For usability testing, unmoderated tests are much cheaper as participants can do them on their own time and pace. For moderated tests, compensate them like an interview as you’ll likely be digging into specific usability issues or expectations.

‍

Additional Advice on Compensating Usability Testing Participants

• Higher fidelity requires lower compensation rate. Lower-fidelity requires each participant to be more engaged and think about each screen/feature/function/prototype being tested.
• If your unmoderated test will take longer than 30 minutes, remove or reduce the number of tasks or prompts.

Use this calculator to help define higher, more attractive compensation rates as the business and the research recruitment brand grows. You could continue compensation conversations starting at 2x the the Median Hourly Wage (MWH) when you’re able to secure more funds.

In practice, you’ll likely switch between methods, meaning you’ll see more value than using only method (running only unmoderated usability tests for 3 months isn’t recommended). Focus on the bigger picture of using your limited budget wisely, rather than stressing over seemingly small number of participants for one study (see this Topic for more on practical sample sizes).

To end this Topic, let’s see the median hourly wage strategy in action.

‍

The Median Hourly Wage (MHW) Strategy in Action

To see the MHW strategy in action, let’s walk through an example. In this example, a UX researcher was able to run four studies within one month. Each study took only one week, which is incredibly challenging to do and uncommon for most businesses. However, even with this intense research roadmap, the overall cost for the month of learning is fairly reasonable.

In this example, the lower end of the specific method-compensation rates where used (scroll back up to see the tables for interviews, surveys, and usability tests). In the end, 252 participants helped the business make smarter decisions, while costing ~$9 per participant. For roughly a fancy cup of coffee and a breakfast sandwich, this UX researcher was able to deliver qualitative and quantitative methods without breaking the bank.

This example is here to help move from abstract numbers towards a concrete situation. In practice, you’ll always find reasons to pay more, pay less, or not compensate participants at all. However, the median hourly wage (MHW) strategy offers you a strong starting point of discussion with your product stakeholders.

‍

Final Advice on Compensating Participants

While the numbers above are a good place to start, no two studies are ever alike. You’ll have to make trade-offs. Some months, you might be rigid with study compensation so you can run multiple, short studies; other times, you might need to double all of the rates to get the best data possible. Always ask yourself “If I use X dollars on this study, how will that affect later studies this month and next month?”

Ask yourself “If I use X dollars on this study, how will that affect later studies this month and next month?”

If you don’t have a plan, you’ll find yourself having to ask for additional money. Some additional factors are listed below to help you recognize when you might need to adjust the above compensation rates.

Here are some other quick tips or advice when figuring out how to best use your recruitment budget:

Additional Tips for Compensating Participants

• To start, only pay for qualitative research (qualitative research works with smaller sample sizes, meaning less total compensation for a study)
• Split costs over different months if you can (paying for a recruitment vendor this month means less budget/participants this month, but can mean running a large-scale quantitative study next month)
• Use and add more non-financial incentives (see the last Topic)
• Avoid spending more than 33% of your compensation budget on any single study
• Save money for marketing, study awareness, and creating a research panel efforts

Paying participants fairly is an important part of their participant experience. But you can’t pay your way into reliable, sustainable, and scalable recruitment. You need to go further than thinking about the next study and start thinking about the next year of research.

One effective strategy is rework your current recruitment operations into a recruitment engine. In the next Topic, let’s discuss what a recruitment engine is, how it works, and accessible ways to start building one.