A 3-Legged Dog with Wings

Sometimes you want to measure something concrete, like how many minutes a particular segment uses a feature in a month. But other times, you want to measure something more abstract or “fuzzy” idea such as delightfulness, intelligence, or trustworthiness. How do you measure trustworthiness? How do you measure something intangible?

These “fuzzy” ideas are examples of concepts. Concepts are general and abstract ideas that your brain creates and refines based on your experiences. Pretty much everything in your brain is either a concept or related to one, and you use concepts in your everyday life. Examples include things like “thinking,” “chair,” or “handsome” because everyone has a general idea of what it is.

There isn’t one specific example that pops into everyone’s head. Concepts are peculiar in that it’s easy to use a concept in your everyday life but a challenge to define them. Let’s take a simple concept: an apple.

What exactly is an apple? Is it something red? What if an apple were purple?  What if it had wings and sunglasses?  At what point, does an apple stop being an apple? 

Why mention something so abstract and random? Concepts are based on your personal experiences. You and your stakeholders might have very different ideas about a concept (such as someone who’s never seen an apple). If you’re studying concepts in your research, you want to ensure you and your stakeholders are aligned on defining and measuring that concept. This process is known as conceptualization.

‍

Transform the Abstract

Conceptualization is when you take a concept and define it in objective, specific terms. It’s taking the concept of, say, a “dog” and explaining what you mean (ex: “A dog is an animal with four legs, a tail, that barks and tends to live with humans”). You might have a different conceptualization for “dog,” but it’s probably reasonably close to the one above.

But let's look at a more relevant example: app speed. If your stakeholders want to measure the concept of “speed,” you’d need to conceptualize what they mean before designing your study. Is “speed” how fast the native application opens up from a technical perspective, or is “speed” more about how fast the application feels to someone using the app?

Haphazardly designing a study “speed” without alignment might lead to mismatched or unmet expectations between you and your stakeholders. To avoid this, you need to write an operational definition.

An operational definition describes how exactly you’ll define and measure something in your research study. This process is known as operationalization (op-er-uh-lie-za-shun). It’s a big word, but the idea is straightforward. Let’s see revisit the speed example.

“Speed” might be conceptualized as “how fast the application feels to someone.” You might operationalize this further as “the rating someone gives when asked to describe the app’s speed when using the application” (like in a pop-up survey). This is a simple example but check out this helpful article below for more help on conceptualization and operational definitions.

Make your operational definitions as specific and objective as possible. Operational definitions are based on your context, your research question, and the other factors that you consider important when running a study. They’ll change from study to study (“how fast does it feel” vs. “are customers getting fast customer support?”).

But what happens when you struggle even to conceptualize an abstract concept? There’s a chance that it’s more complex than you initially thought. There’s a chance you’re trying to measure a construct.

‍

Construct, Dimensions & Indicators

Constructs are possibly one the most important but strange ideas in all research. These represent abstract ideas or phenomena that you can’t directly measure. Constructs are special examples of concepts. They are the most abstract things you’ll ever study because they don’t exist.

Constructs can only be indirectly measured.

Constructs like “social class,” “equality,” and “masculinity” can’t be observed in the real world. People created them to help group complex ideas and make sense of the messy world around them. You can only indirectly measure constructs by measuring things related to each construct. The things you can measure are known as indicators.

For example, suppose you were measuring the construct of “trustworthiness” for profiles on property rental platforms. You might have the number of positive comments, the number of 5-star ratings in the last 60 days, and the quality of the host’s About Me section as indicators. When you measure all these indicators and aggregate them together, you should be able to get a good understanding of the “trustworthiness” of the hosts.  

But this seems like it’s too simple, right? You’re probably thinking that measuring "trustworthiness” is more complex than what’s described above. You’re right. To measure some constructs, you might need first to define its dimensions.

A dimension is a distinct but related aspect of the larger construct. Discussing dimensions means recognizing there are two groups of constructs: those with and those without dimensions.

Unidimensional constructs don’t have dimensions. The indicator by itself does a good job of measuring the construct. For example, take body weight. While you can physically weigh someone or something, those are specific examples of the general construct of weight. Your weight is not the same as the person next to you, but both are examples or instances of the weight construct. But if you weigh something once, you’ve effectively measured the weight construct.

Multidimensional constructs have multiple dimensions and are more complex. Let’s take possibly the most common, popular, and studied constructs in experience research: usability.

You might have an idea of what usability is. But go online and look around for different definitions for usability. You’ll find that while the definitions are mostly similar, there are subtle differences across them all. Everyone has a slightly different definition of usability, even if it’s a concept that’s used regularly when improving experiences. This suggests that usability is more complex and likely a multidimensional construct.

You might see usability broken down into dimensions like “learnability,” “efficiency,” or “memorability.” These dimensions represent unique and important characteristics of the construct itself. You still can’t easily measure “learnability,” so you need to specify indicators for each dimension.

You might measure how many errors a participant makes across three different testing trials to see how “learnable” a prototype is. Combining indicator measurements for the various dimensions gives you a pretty good idea of usability.

Without support or advice, setting out to measure a multidimensional construct can be challenging. It can be hard to know if your indicators and dimensions are actually measuring the construct you want (read this for more on construct validity here).

This is easier said than done, so be cautious about assuming that dimensions and indicators for a particular construct are automatically valid. Try to find validated measurement instruments (like a survey) online when you need to measure a construct.

Concepts, constructs, and operational definitions. These ideas are incredibly important because they can make the abstract world concrete, measurable, and observable. Be skeptical and always ask yourself if how you’re measuring something is working as you intended. Work with your stakeholders to properly define concepts and constructs before you design a study to save you a lot of headaches.

Always ask yourself: "Are you measuring what you think you're measuring?"

If you know how you want to study something and have solid operational definitions, then you should start choosing methods, right? Even still, you need to get aligned with your stakeholders before starting your study prep work.