The Participant Experience

In the last Topic, the idea of the participant experience was introduced. While the user experience is what someone experiences using your product or service, the participant experience is what it’s like for someone to participate in your research. How much friction is there for someone to participate? Without understanding the participant experience, you might be inadvertently slowing down your best interests.

There are four main phases in the participant experience, as shown above. But before someone even becomes aware of your study request, you, the UX researcher, have to narrow down who to recruit.

‍

Your Recruitment Criteria

When you set out to recruit participants, you have a general idea of who you trying to learn from. It’s unlikely that everyone in your accessible sampling frame(s) will be informative for your specific research questions and study goals. You need to define your recruitment criteria. These are the characteristics or requirements you need to meet to generate a meaningful and informative sample within your constraints.

Recruitment criteria help you narrow down who to learn from, given your study goals and resources.

One recruitment criterion is your Most Informative Participant (MIP) definition, discussed more in this Topic. Other recruitment criterion are listed below. Note that some of the criterion are can be fixed between studies (such as compensation for an hour-long interview) while others are dynamic (such as your desired sample size or your study timelines).

Your recruitment criterion helps give you focus. But never forget that all participation is voluntary. Your participants are willingly choosing to participate in your research. They have the right to not participate or leave a study after starting. You have to factor their voluntary attitude when recruiting. If being a participant in one of your studies is uninteresting, irrelevant, cumbersome, or unpaid, you’ll always struggle with participation.

Unless you understand their expectations or hesitations, your current participant experience might be removing or ignoring informative participants altogether. Worse, your participant experience might have so much friction that participants might start and leave a study altogether. In practice, that “linear” flow of the participant experience resembles a funnel: those that start the study don’t always finish it.

‍

Funnels in the Participant Experience

The diagram shown at the top is an oversimplification of the true participant experience. In practice, there are two distinct funnels where participants can leave or ignore your study: before and after someone starts your study. Let’s add in more details to the above diagram.

Before taking a closer look at each phase of the participant experience, let’s discuss this diagram further. From a researcher’s perspective, the participant experience is two different funnels strung together. The funnel is similar to a marketing or sales funnel, where there’s a large number of possible or potential customers but only a small fraction actually purchase the product. (You can read more about sales funnels here).

The first funnel in the participant experience happens before a participant provides a single shred of data. Someone has to be first become aware of the study, and then, be allowed to start the study if they meet your necessary recruitment criterion (discussed above).

The second funnel starts after a participant actually begins your study. Remember that not everyone who starts a study will successfully complete it, for a range of reasons covered more below in Phase 3: Study Engagement. Please note that Phase 4: Study Compensation is not a funnel; anyone who meaningfully and successfully completes your study is guaranteed fair and timely compensation.

Within both sets of funnels, there are two main ways a potential participant can leave: by nonresponse or by opt-out. Nonresponse (read more about nonresponse in the last Topic) is passive as someone didn’t move to the next phase because they couldn’t or didn’t want to continue. Opt-outs, however, are active. Someone is consciously removing themselves from being considered for this or future studies. This could mean formally leaving a research panel, turning on research notifications, or abandoning a product or service altogether.

When the study starts, this opt-out turns into a drop-out, where someone starts a study but leaves before it ends. Nonresponse, opt-outs, and drop-outs all have to be recognized and managed when you’re improving your participant experience.

With the general structure and shape of the participant experience covered, let’s dig into each phase to see what’s like to be a participant. Let’s start with arguably the most important phase, study awareness.

‍

Phase 1: Study Awareness

Study awareness is a crucial but often overlooked part of the participant experience. How does someone become explicitly aware that you’re looking to learn about their experiences, stories, and emotions? In some situations, you might post a message on social media asking for participation; in others, you might look to attract participants within the product or service itself. It all depends on your study needs but it’s always focused on the same problem: how do you attract informative participants as quickly as possible?

The list below covers issues in Phase 1. They’re relatively ranked by the most urgent or critical issues at the top.

Participant Experience Issues in Phase 1: Study Awareness

• An informative participant didn’t notice or recognize your research request
• An informative participant noticed your research request but didn’t want to participate
• An informative participant noticed your research request but couldn’t participate
• An uninformative participant noticed your research request and wants to collect the study compensation
• An informative participant noticed your research request and opts-out from future contacts or alerts about research

Study awareness is the most important phase because it’s at the start/top of the participant experience funnel. And unfortunately, that makes awareness a simple numbers game: the more people that become aware of your study, the more people that move to later phases and end up completing your study.

The more awareness you can get for your study, the greater the chance of reaching a desired, informative sample.

From a research perspective, having high study awareness also affords you the luxury of screening and being more thoughtful when scheduling participants. For every study you run, be conscious about how, when, and where people learn about participating.

The list below contains some of the main points you need to communicate to garner the best study awareness possible.

Study Recruitment Main Points to Cover

• The purpose of the study or how any collected data will be used
• The primary research questions you’ll be studying
• A brief description of how participants will provide data (e.g., via an interview, survey, etc.)
• A brief description of the Most Informative Participant (MIP)
• A study timeline (including when it starts, ends, and when compensation can be expected)
• Study compensation (and how such funds will be sent)
• Your contact information (such as name, email, and/or phone number, alongside the name of the business)
• How to sign-up or get enrolled to participate
• (If relevant) How provided data will be anonymized or if its confidential

If and when an informative potential participant becomes aware of your research request, then move to Phase 2: Qualification.

‍

Phase 2: Study Qualification / Scheduling

Imagine that your target population are city bus drivers who use a transit software to manage their daily bus routes. While any research might mean getting the attention of as many bus drivers who use the software as possible, they won’t all fit your recruitment criterion. Awareness is good to start the funnel but then you need to qualify, screen, and schedule the most informative participants.

This qualification is essentially making sure potential participants fit your recruitment criterion. This can be done in a high-effort manner such as having a phone screen with every potential participant. It can also be done asynchronously, like with a recruitment screener, which can help you lower your recruitment effort. With survey research, this qualification can happen at the start, like when you’re first couple of questions help you screen out uninformative participants.

But as you look to qualify who to learn from, your potential participants are also qualifying themselves. No matter your product, brand, or study compensation, every participant tries to assess if trying to participate is even worth it. Some of the questions they ask themselves before starting your study (framed as issues you must overcome or manage) are listed below for easy reference.

Participant Experience Issues in Phase 2: Study Qualification

• A potential participant doesn’t believe their feedback and data will lead to meaningful or noticeable product or service changes
• A potential participant doesn’t believe the study demands match the study compensation (such as having to drive far to a study site, having to download extra software, having a study last multiple hours or days, etc.)
• A potential participant can’t provide data during the study timeline (or can only provide data during late hours or over the weekend)
• A potential participant doesn’t know or trust the brand or reputation of the business asking for their feedback

If there are a lot of requirements or prerequisites to start your study, expect informative participants to self-select themselves out of participation. For example, if someone needs to download a Google Chrome extension to complete your unmoderated testing session, that friction will ultimately result in a smaller sample.

Whenever you can, streamline or cut the number of steps or tools needed to get started with a study. And ask yourself: “are your study demands necessary to get the best data possible?” If the answer is “no”, then simplify the demands to maximize the participant experience. Make sure to balance that simplification with what’s relevant for your research questions and study goals.

One critical issue with study qualification has to do with the brand of the company itself. If you work at a brand-new startup or a hyper-niche business, you’ll have zero name recognition when looking for participants. Many potential participants can be hesitant to take the time to give feedback when (a) they’ve never heard of the company or (b) are skeptical if receiving incentive or compensation feels unreliable or like a scam. It’s much safer to participate in a Google research study because people know and trust Google.

It’s much safer to participate in a Google research study because people know and trust Google.

To overcome this, think of your recruitment as having a brand. You want your brand to have a clear, professional, and trustworthy feel. Use high-fidelity images, video, and audio in your recruiting materials whenever you can. Your tone and language should match the business. If possible, try to recruit through the business’ professional social media or public-facing accounts for more legitimacy and reach.

Once an informative participant becomes aware and qualified, they start engaging and providing data.

‍

Phase 3: Study Engagement

The next phase of the participant experience is continuing or actively engaging with your study’s topics, questions, and exercises. Recall that Phase 3 is the start of the second funnel in the participant experience. Someone who starts your study might not finish it. Longer studies, studies that are irrelevant or mundane, or studies that complex all force participants to start but abandon a study.

Below are some quick tips to help you ensure the participants who start a study, complete it.

Tips to Reduce Participant Drop-off

• Limit or reduce the demands on a participant’s memory (i.e., don’t require people to recall info from months ago)
• Reduce or remove as many steps to get started (e.g., don’t force participants to download additional software to participate, driving to an on-site location, etc.)
• For longer studies, offer a clear sense of progress (e.g., ”Only 5 more survey questions!”, “3 more usability tasks left!”, etc.)
• Keep any study prompts or questions understandable and relevant to your MIP
• Repeat directions or instructions (assuming people won’t read)
• Provide an easy way to get in contact with you if any issues arise
• Keep qualitative sessions to 45-75 minutes and avoid/limit sessions longer than 90 minutes
• Reduce or limit your diary studies to a week and only 2-3 daily contacts/entries per day
• Limit surveys to only 10-12 questions
• Limit usability testing sessions to 20-30 minutes

You need to consciously design your study around the limited attention, memory, and energy of your participants. It’s in your best interest to run studies that are as short, structured and as informative as possible.

For an incredible book that covers the psychology and how it can affect product design — or your research studies — check out David C. Evan’s "Bottlenecks: Aligning UX Design with User Psychology".

The final phase of the participant experience is what someone gets for taking the time, energy, and attention to provide their data: being compensated.

‍

Phase 4: Study Compensation

The final phase is about how will someone receive their study compensation. This could be a direct deposit, a gift card, or digital credit within the product. For most participants, your study isn’t complete until they get their compensation.

Note that what’s easy for you to record as study compensation might not be what’s easy or desirable for your participants. You need to make it easy for all participants to receive their compensation in a timely and reliable manner. If done poorly, your participants will leave with a bad taste in their mouth and a horrible participant experience.

If there’s friction or frustration there, you’ll have to manually manage the negative emotions. And if receiving any compensation is seen as unreliable or challenging, you’ll also struggle to re-use participants in later studies (an effective to decrease recruitment time and identify more subtle issue and patterns).

Offer multiple options when finalizing study compensation: cash, gift cards, direct deposit, swag, etc.

If you can, consider offering multiple ways to compensate participants. Gift cards, cash, company swag, and other goodies can help you thank your participants while stretching your limited budget. For more tips on compensation, jump to the next Topic.

Below are some issues with the Phase 4: Study Compensation. If you can, try to include information about these issues as early as you can in the participant experience (such as Phase 1 or Phase 2). It can help you set expectations with participants, instead of having to explain when, how, and where any compensation can be used.

Participant Issues in Phase 4: Study Compensation

• A completed participant isn’t sure when they’ll receive their compensation
• A completed participant has to download a new app or create a new account online to collect or use their compensation
• A completed participant is unhappy when any compensation can only be used in select or restricted places (such as only within the product, only online but not in a store, or being given a pre-paid debit card instead of cash)
• A completed participant doesn’t feel their allotted compensation matches or reflects the time, energy, or attention they spent in your study

Note that when dealing with financial issues, you’ll have to be aligned with financial and legal stakeholders or departments before any compensation is dispersed. You don’t want you (or your UX team) to be in violation of any legal or tax codes. One way around this is to offer non-financial compensation, an idea covered more in the next Topic.

While compensation is important to participants, is it unethical to pay people for their feedback? In the next Topic, let’s take a closer look at how study compensation affects your recruitment.