Uncovering Your Assumptions to Increase Impact
Quick note: This post is heavily based on a Data 101 workshop at Riot Games designed by absolute rockstars, Torrie Summers and Betsy Anderson - credit where credit is due! I love this framework and still use it regularly.
What are “assumptions”? And why should I uncover them?
When we begin working on a new product or feature, we often start with some sort of objective. Sometimes these objectives are lofty and vague and other times they are very specific or even a specific targeted metric. Regardless, we define a goal or an objective as our aim or desired results for a given product or feature.
After we identify this goal, we ask ourselves: what actions will we take that we think will help us achieve this goal? This is the stage when the shape of the product emerges, features get proposed and designed, tech gets selected, and components and systems are built. These actions are what we might refer to as strategies or tactics.
However, we don’t always challenge ourselves to articulate how our chosen tactics (those things we’re building in hopes it’ll help us accomplish our goal) are actually going to get us to our goal. While we may choose tactics carefully and with some amount of confidence (based on previous experience, strong design thinking, and other times intuition) we don’t always articulate or test the assumptions underlying these choices before we develop, finalize and launch a product or feature. These assumptions we build upon, that we accept as true or likely certain, aren’t always accompanied by direct evidence of proof. Here are some examples of these kinds of assumptions from a few different contexts:
Players enjoy watching combat scenes in video games
Removing formal scripted onboarding videos will inspire users to seek more social learning approaches
Daily meetings among remote team members increase employee engagement
Minimalist UI design is seen as more sophisticated by users
In situations where getting it right matters and the road isn’t well-paved, defining and testing assumptions can be the difference between a successful product and failure. Why?
Clearer Communication: Articulating assumptions is a great technique for breaking down your thought process and exposing it to others. By detailing the reasoning behind your decision-making, you allow your team members or those outside of your team an avenue to follow your logic and ask targeted questions about the pieces of your decision-making that might not make sense to them. This particular approach is uniquely suited for cross-functional teams, with a variety of different crafts represented, to poke at and challenge each others’ assumptions despite different areas of expertise.
More Effective UXR Integration: Defining the team’s assumptions allows UXR to design studies to address individual assumptions throughout development, resulting in more actionable insights. Additionally, surfacing these assumptions early allows UXR to conduct literature reviews of previous work or alert teams of existing work outside of their team that might help them make decisions without new primary research.
Post-Launch Diagnostics: When products aren’t achieving what we’d hoped, revisiting assumptions can be a great way to uncover the mechanisms impacting performance. If one core assumption is unsupported, it may be the reason your product isn’t succeeding. Outlining the assumptions ahead of time allows teams to quickly discuss and respond when things don’t go as planned.
Okay, so how do I uncover my assumptions?
The Assumption Exercise outlined in this note is designed to help teams clearly articulate the hidden assumptions lurking within the development process and then identify assumptions they’d like to test, usually with the help of UXR and Data Science. More about the exercise:
Who? This exercise is something you can do on your own, with a small group or with a large team. I’m happy to facilitate this exercise, although I’ve seen it facilitated by folks from a variety of functions (e.g., Production, UXR, Data Science).
Where does this apply? This exercise has pretty flexible applications. While I’ve primarily used this in a product development context, it can also be useful when making personal choices or even managing team health, or tackling organizational challenges. I’ll use a non-work related example to demonstrate how this exercise works below, but know that this exercise can be useful with almost any problem!
When? This exercise can be done at any point in the development process, but it is most useful when done early before we even start incubation or concepting work. I’d recommend doing this exercise at the point when your team is choosing a particular tactic to achieve your objective. This way, UXR can help test assumptions early, before you’ve built an entire experience on a potentially unproven theory.
Step-by-Step Guide
You can begin the exercise by stating a goal that you and your team would like to accomplish. In our example, let’s pretend that Jo, your friend, wants to lose 5 lbs. Start by writing this on a whiteboard so everyone can see it clearly. Sometimes, just articulating the goal can be challenging (which is a totally fine outcome, clarifying goals is important).
Once you have your goal, you define a strategy, tactic or method you’re planning to deploy in order to accomplish that goal. Often, you’ll have more than one tactic for a given goal. For the sake of the exercise, pick one tactic to focus on at a time. In our example, let’s pretend that one tactic Jo wants to explore is using a FitBit to track their steps. So your whiteboard now looks like this:
We’ll call this our goal-tactic pairing.
3. Okay, now onto the meat: the assumptions. It’s time for us to brainstorm assumptions associated with this particular pairing of a goal and tactic. An assumption is just a theory or belief that we might be accepting as true or certain. So in our example, what would have to be true about the world in order for “Using a FitBit to track their steps” to help Jo accomplish their goal to “Lose 5 lbs”? At this point, you want to encourage devious thinking. Have people articulate all the assumptions they can come up with, even ones that might seem obvious.
The first one you might come up with is “Tracking steps will lead to weight loss”. This is good, but can be broken down even further into more specific assumptions.
Here’s an example list of detailed assumptions you might brainstorm, were you Jo:
Tracking my steps will make me want to take more steps
Taking more steps will lead to weight loss
I will wear the FitBit regularly
I’ll look at the FitBit often
I can afford to buy a FitBit
A FitBit can accurately track my steps
It’s pretty common to come up with some ideas that aren’t quite assumptions, so check yourself or others when you have an inkling it might not fit on your list.
Something Jo might come up with is “I saw a research study finds that walking more leads to weight loss”, which is a data point (love it!) but not quite an assumption. These types of ideas can be reworded rather easily into assumptions (see the second bullet point).
One way to check if a given item on your list is an assumption or not is the negation test. This is where you try negating it (i.e., pretending the opposite is true) -- e.g., “Walking more does not lead to weight loss.” If it really is an assumption, this negation should pretty significantly weaken your line of reasoning that the chosen tactic will help you accomplish your goal.
4. Now that we have our list of assumptions, it’s time for the hardest part: sorting them. Generally, your assumptions will fall into two major categories:
Need to More info: For the majority of your assumptions, you’ll want to gather more evidence before accepting them. This usually happens when we don’t feel confident enough to accept them with what we already know or when they are so critical that we need a high level of confidence in order to accept them. To test any assumption, we usually need some type of data. It’s important to keep in mind that the scope of data collection should match the scope of the decision itself. For example, if Jo is collecting data to see if tracking steps leads to people taking more steps, does a large-scale primary research study make sense? This is where UXR and Data Science can be especially helpful! There are two main ways we can find the data we need to examine a given hypothesis:
Look at Existing Insights: Often, an assumption is rooted in previous studies or articles we’ve read on a topic. Even if you feel you know these exist, it can be helpful to go find these sources, review them again and confirm that your assumption is indeed supported by what they found. UXR can help here with Literature Reviews on past research and Data Science analyses. In our example with Jo, we might want to look for articles and studies that may support that walking leads to weight loss or that wearing a FitBit or tracker actually leads to people walking more.
Gather New Insights: If we don’t have existing data or research on a given topic, we may need to conduct some primary research in order to gather the data we need to assess a given assumption. This is when UXR comes in - we’re skilled at crafting research to gather data to help us test our assumptions. In our example with Jo, who doesn’t have access to all the Research resources, they might ask a friend who owns a FitBit to see if they actually wear it regularly, read reviews about the accuracy, etc.
Thoughtfully Accept: Sometimes, we decide that an assumption doesn’t need to be tested. This is usually because we have very high confidence in it even without data. Other times, it might just be an assumption that is really difficult or impossible to test. You’ll want to be super careful when sorting an assumption into this pile - if an untested assumption is core to your chain or reasoning between tactic and goal, you may want to consider exploring testing options. The important thing here is to call the assumption out as a theory that we are choosing to accept without testing, consistently include it in presentations or notes going forward and keep it in mind as we go to ship our product or test other assumptions. In our Jo example, they might be comfortable accepting they’ll wear it regularly because they already wear a watch daily or they might be comfortable accepting that the FitBit is reliable since they seem like a legit company.
Assumptions can and should be resorted throughout development, as we gather more information about our product or the market and as our confidence in our assumptions shifts. I’d recommend revisiting this at each key development stage to ensure you’re still on the right track.
Things to Keep in Mind
Do we have the right starting objective? Sometimes, this exercise prompts discussion around the goal itself and you may question if you even have the right one. With Jo, losing 5 lbs might be a tactic for a higher level goal and might be a good tactic (or a bad one) depending on what that higher level goal is. For example, if the higher level goal is “get healthier” there might be other tactics besides losing 5 lbs that might be better to pursue. On the other hand, if the higher-level goal is “ride a rollercoaster that they’re currently 5 lbs too heavy to ride”, then it might be the right objective to target. If you think your goal is linked to a higher-level goal, it could be worth mapping that out before the session or taking time in the session to discuss it. If you don’t have a good starting goal, it makes the rest of the exercise less valuable.
Patterns to look for in assumptions Once you do this exercise a few times, you’ll find there are some pretty common patterns or types of assumptions that come up. If you’re struggling to build an assumption list or want to ensure you’re covering the big bases, consider the following as it relates to your goal-tactic pairing:
Efficacy: How will your tactic produce the desired result? For example, if we want to lose weight and our tactic is to wear a FitBit, what is the chain of thought that gets us from our goal to our tactic? (wear FitBit → walk more, walk more→ lose weight)
Resources: Can I actually afford to do this thing? What will I need to have or use in order to execute this tactic?
Opportunity cost: What alternative are we giving up by doing this? What are the strengths of other solutions?
Value-add: Is there significant value added over the status quo if we do this thing? Does the problem we’re trying to solve, and the audience we’re trying to address actually exist?
Negation: What might be true about the world that would make this tactic not accomplish our goal? How can I reframe that as an assumption?
Don’t be afraid to pair this with other frameworks After identifying an objective and a tactic, you can sometimes spark discussion that warrants use of another framework to break down the problem a bit more. For example, let’s say that we have defined our goal but the room is pretty divided on the specific tactic we should use for the exercise. This might indicate two tactics that may both effectively accomplish our goal. In that case, it might be good to use risk/benefit analysis or similar framework to compare the two tactics.
