The Power of Categorization

What if you could learn a ton about tackling a research question, how to roughly design a study, and even when to study something without exerting much effort? It's true. Welcome to your research superpower: categorization.

Whether you're a new researcher or an experienced researcher in a new environment, it's easy to feel overwhelmed by how much you don't know. You're not sure what everyone else is talking about in meetings. It can be terrifying to realize how much you have to learn to become effective in your role. The fastest way out of this nightmare is by categorizing the signals and stimuli around you.

While you can categorize pretty much anything new to you, let's focus on something immediate: the new, seemingly urgent, and incoming research questions from your stakeholders. If you can quickly categorize each question, then you can essentially slow time and make smarter decisions at the moment. And for the vast majority of questions, if you can categorize incoming questions into the following three buckets, you know how, when, and why to study something.

The diagram depicts the three main buckets for categorizing any research question. What's important to know is that categorization is an active process. You're constantly looking for clues about what's important to your stakeholders. You're refining unethical or bad research questions into something urgent or meaningful to pursue. You're proposing or recommending better things to study.

Let's review each category, starting with a question’s boundary.

‍

Boundaries (A Question's Scope)

The first way to categorize a research question is by its scope. What exactly are your stakeholders interested in? On the flip side, what are they not interested in? What’s irrelevant or unnecessary to study? The scope of a question tells you what to study.

The scope of fruitful questions always contains information about the Most Informative Participant and the product or service you're trying to improve. Are your stakeholders focused on the end-to-end experience or a specific interaction with the product? Or are they focused on how the mobile app syncs with the desktop app?

The scope should help you understand how broad or narrow your focus should be when you plan a study. If your stakeholders have broad/big questions ("we want to know if this product will be profitable") but only a tiny timeline to do the research, this should send a red flag to your brain. Work to split up questions into multiple, smaller studies or align something more practical. And if your stakeholders want an answer to one isolated question, encourage them to add more or propose topics to study.

‍

Nature (A Question's Identity)

The next bucket has to do with the nature or the identity of a question. In this bucket, you can categorize questions in more recognizable ways: qualitative, quantitative, a combination of qualitative & quantitative, fruitless, or even unethical. With this bucket, you can quickly recognize how best to study something.

For example, qualitative questions start with words like "why" or "how" and focus on someone's motivation or context. Suppose you recognize this characteristic in a possible research question. In that case, you quickly know that you might want to take a qualitative approach, using qualitative methods and some form of qualitative coding to analyze it.

Examples of Qualitative Research Questions

• Why do subscribers in the European market share memes significantly higher than in the Asian market?
• What role does the ranking aspect of online communities affect satisfaction for non-gamers?
• What is the role of store managers in employee satisfaction of hourly retail workers?
• What is the process that school administrators take when disciplining cyberbullying behavior?
• How do new customers engage with the new e-coupons inside the mobile grocery app?

For quantitative questions, questions tend to start with words like "What," "how often," and "how many" are focused on making causal claims or describing behaviors and patterns using mathematical relationships (jump to this Topic or this Topic for more on quantitative research).

Examples of Quantitative Research Questions

• How does decreasing the scrolling speed affect product usage?
• What’s the correlation between app usage and app reviews on a caregiving platform?
• How have smartwatch comfort ratings changed over the last three months in Canada?
• Is there a relationship between the color of the “Sign-up” button and the number of successful account creations?

You can read more about the differences between qualitative and quantitative research in Collection 3: Study Design.

But to conduct fruitful research, you also need to recognize whether a research question is unethical. While it's hard to do justice to the history of unethical research on people in this practical digital library, the main ideas & lessons learned cannot be ignored. In the past and even today, unethical businesses -- and some researchers -- take advantage of people in order to fulfill their own interests and pockets. Research that has no benefit or respect for the participant or the larger society is unethical. In simpler terms, unethical research is when people stop being seen as people and are only seen as data points.

Unethical research happens when people get treated like data points.

In research studies like these, participants are inadvertently allowing their behaviors, attitudes, and thoughts to be harvested for the company's gain. While it does happen at big companies, it can be harder to notice at smaller ones.

As a researcher, you have a moral, ethical, and professional obligation to challenge unethical research. If you're not actively trying to change the nature of your team's questions, then your actions are corroding one of the foundational values of being a researcher.  

Sadly, your stakeholders might not always listen. But when you hear or notice these unethical questions, your goal is to refine them into something ethical and valuable. You can read more about ethical research here, alongside six principles to adhere to when conducting research, as well as with this resource or this video.

‍

Value (A Question's Shelf-Life)

The last bucket to categorize research questions is by looking at their value. You can think of this as how long a particular piece of learning, knowledge or finding is relevant, accurate, and helpful in making decisions. The value of a research question asks you to consider "How fast does the goal or purpose of the research change?" Recall from Handbook 1 that goals can be either tactical (product) or strategic (unmet needs) focused.

Tactical goals change regularly. Once you identify a problem, conduct some research to understand how to address it, and make those changes, the knowledge from your tactical research has been exhausted. Or if you want to consider two different homepages, an A/B test can help figure out which one to use. The winning homepage gets implemented while the homepage that didn't perform well is essentially forgotten.

Or consider usability tests. They're focused on the immediate product and products can change often (think new features). Once a usability test's recommendations have been implemented, the test and its data don't help for future decisions. After all, the business rarely reviews usability test findings from three years ago because so much might've changed since then.

On the other side are strategic goals. They don't change often. A goal like "improve reliable and affordable access to public internet for three low-income neighborhoods in San Francisco" will take time and conscious effort to reach. The knowledge and data generated from an extensive journey map or multiple rounds of interviews can ideally be used for years to help make smarter decisions to reach that goal.

To address either a tactical or strategic goal, you'd conduct either tactical or strategic research, respectively.

Tactical research tends to be shorter (hours and days) and focused on a narrow scope of topics. It's entirely possible to start and end an entire tactical study in few days.

By their nature, strategic research, however, requires lots of planning, collaboration, and alignment before starting. However, the value generated from strategic research is worth the cost because it can lead to innovation or expose hidden opportunities.

If you work in an immature or resistant research culture, you might never get a chance to run strategic research. And that's okay. View strategic research as a goal to reach by first running quick, tactical research to demonstrate the value of experience research.

In some research organizations, senior or staff researchers conduct mostly strategic research while junior or newer researchers focus on executing tactical research. However, that doesn't mean to become a senior researcher you must conduct strategic research. You can help build products and be a senior, staff, or lead researcher by conducting rigorous and impactful tactical research. It mostly depends on where you work and what your goals are.

What’s not immediately obvious is that because tactical goals change often and strategic goals change slowly, tactical and strategic research result in knowledge and data that decays at different rates. Decaying in this context means that, over time, the knowledge or data from research is less and less relevant or accurate for making better product decisions today.

Once you know the goals of a research study, you also know the value or "shelf-life" for the knowledge or data coming out of that study. If it's towards a tactical goal, try to get started quickly. If it's towards a strategic goal, you must understand more about question, the unmet needs being studied, and the larger context first before planning your research.

Study a mix of tactical and strategic research questions for the most value every quarter.

In a fruitful research culture, you're conducting a mix of tactical and strategic research, helping your stakeholders reach their immediate goals while also being aware of the larger possibilities all around them. If you can, try to study at least 1-2 strategic research questions per quarter if you can.

Knowing what, how, and when to study something, you make it easier to stay aligned with your stakeholders so your research produces as much impact as possible. In the final Topic for this Handbook, let's discuss an important part of any research study: getting stakeholder and study alignment.