What color should this button be?
We can only do x and y, but not z, before the deadline. Can we live with that?
Sales requested a, b, and c. Which should we do?
The CEO had a new idea and asked if it’s worth pursuing.
As product builders, we make decisions all day every day.
How do we know if they are any good?
The Heath brothers, in Decisive, outline the 4 villains of decision making:
- we look too narrowly at a problem
- we look for evidence that supports our beliefs
- we let short-term emotions affect our decisions
- we are overconfident
Kahneman and Tversky spent the better part of a decade exploring the common mistakes we make, including the confirmation bias, the hindsight bias, the status quo bias, the availability heuristic, anchoring, and so much more.
The research seems to indicate that we aren’t very good at making decisions. So what are we to do?
Fortunately, the research also suggests ways we can design thoughtful decision making processes to overcome some of these shortcomings.
Before we dive into how, let’s first identify the eight critical decisions required when creating great products.
It all starts with goal-setting. A good goal allows you to trust that you are on the right path and charge down it as fast as you can. It allows you to put blinders on to everything else and relentlessly focus on the one thing that matters. Your goal.
However, if you do this with a bad goal, it can be catastrophic.
How you identify and select goals is one of the most important decision making processes in building great products.
If you are making ad-hoc decisions about goal setting, you are probably charging down too many different paths. – Tweet This
Most companies overlook this process all together. They leave idea generation to the founders or to the product team or to the sales team. They just hope it gets done.
Thanks to creativity research, we know that more ideas lead to better ideas. We also know that more ideas that are all similar to each other don’t help. They need to be categorically different.
And unfortunately, brainstorming as we generally know it, simply doesn’t work.
People generate more, better ideas individually than they do in groups. – Tweet This
And yet, most companies still rely on brainstorming to generate good ideas.
It’s hard to build great products without generating a lot of diverse ideas. And not just during the early stages of the product. It’s important throughout the entire lifecycle of a product.
So now that we’ve got a pile of ideas, how do we know which ones are good? Which ones should we pursue? Where do we invest our time and resources?
Outlining evaluation criteria and sticking to it is one of the most important jobs of the product team.
Doing so allows everyone to participate in the idea generation process. It creates a sense of procedural justice, which is critical for keeping people motivated and engaged.
More people generating ideas leads to more ideas which leads to better ideas. Having clear evaluation criteria makes this happen.
Not to mention, if you can’t identify the ideas worth pursuing, you won’t be successful.
And yet, too often product teams make intuitive judgements in the moment. This simply doesn’t work.
I can already hear some of you argue, we’ll run tests to decide which ideas to pursue.
This is far and away the worst side-effect of The Lean Startup (and it’s not Eric’s fault). It’s yours for not actually reading the book.
Stop throwing spaghetti at the wall. You can’t test everything. There aren’t enough hours in the day. You have to decide which are the most important things to test, when, and how.
This includes what to A/B test, what to usability test, when to run a survey, when traffic analysis is good enough, what data to look at and when, and so much more.
Laura Klein in Lean UX covers many experimentation methods and explains when to use each. Her book is also full of humor. You should read it.
But on top of knowing what tests are good for which situations, teams need to identify their own standards.
Experimentation serves two purposes. First, it helps us learn what works and what doesn’t work. Second, it helps us get buy-in from key stakeholders and build momentum.
Startups don’t like to focus on that second reason. But it matters.
You might believe Jakob Nielsen that you only need 5 usability participants to get good data, but if your engineers think 8 is the minimum, you might want to just run 3 more participants through the study, rather than arguing with them about the validity of the results.
Identifying your own experimentation processes and standards can go along way toward establishing norms for how your team will learn and for building confidence in what you learn.
So we’ve whittled our idea pile down to a handful of well-tested concepts that meet our evaluation criteria. How do we decide what to build first?
Do you build what’s easiest first?
Many teams misinterpret Agile to mean, yes, you build what’s easiest first.
The problem with this is you end up with a product full of the easiest things and not the best things.
Again, this isn’t a problem with Agile. It’s a problem with people’s interpretation of Agile.
Agile simply encourages you to build iteratively.
The concept of an MVP is to build the smallest product that validates your biggest assumption, not to build the easiest / quickest product. – Tweet This
Teams need to identify good ways of uncovering these assumptions and prioritizing experiments and development that validate them as quickly as possible.
Rarely is an idea ready for implementation. It almost always requires some design before it’s ready to be implemented. I don’t just mean UI or UX or visual design. I mean product design. But that’s not a decision-making process, that’s an art. We’ll tackle that in the third pillar when we discuss team and individual skills.
In the meantime, let’s assume we know what we want to build next.
Now, it’s up to the engineers to build. We are done, right?
Anyone who has ever built a product knows that’s simply not true.
Engineers can rarely build exactly what we ask for. Nor should they.
We overlook things all the time. Sometimes what we ask for isn’t very good. Often times, the way we ask them to build is downright shoddy.
You should be partnering with engineering every step of the way. And that often means being involved in deciding how the product gets built.
The requirements you send to engineering should be the beginning of a conversation. A conversation that unravels as they build. – Tweet This
The more engrossed in the code the engineers get, the more questions they are going to have, the better they are going to understand what’s possible and what’s not.
Does your team have a good process for managing these conversions? Or are you like most teams, simply writing user stories, calling it Agile, but really throwing them over the wall in a mini-waterfall approach?
If you aren’t talking to engineers every day about what they are building, you probably need to rethink how you are building product.
I used to tell our engineering team, if you don’t know how we are going to measure success, don’t build it. I wanted them to hold me accountable to knowing why we were writing every single line of code.
Measuring outcomes is absolutely critical. It tells us when our assumptions are wrong. I love it when my assumptions are wrong. It tells me that I think about my product differently than my users. And it’s critical that I uncover why.
But it’s hard to know what to measure, when, and why. Getting this process right is one of the hardest things a product team does.
The goal isn’t to measure anything and everything. It’s not to look at vanity metrics to make us feel good about ourselves. It’s to truly understand what impact we are having. This process requires a lot of thoughtful design.
The first seven processes, if designed well, will help you build great products. But they aren’t enough. You also have to bring people along with you.
It’s absolutely critical that you know what to communicate to who every step of the way. You have to set expectations appropriately. You have to do this internally with your sales team, your engineering team, your executive team; and you have to do it externally with your customers and your end-users.
Even if you get the first seven processes right, if you overlook communication, you’ll have a hard time succeeding.
It’s simple, right?
Get these eight processes right and you’ll be on your way.
Nobody said building great products was going to be easy.
But don’t worry, starting on Monday, we’ll dive deep on each of these processes.
We’ll draw on decision research, behavioral economics, learning theory, and so much more to design thoughtful processes in each of these areas.
Don’t miss out, subscribe to my mailing list to be notified when there are new posts.
David Fradin says
I disagree with your first few steps. As Tony Ulrick points out in his “Outcome Based Innovation” papers and book, one has to find 15 unmet needs. To do so he says ““Given the number of possible ways that just 15 unmet needs could be satisfied by products and services in any given market, millions of ideas would have to be generated before an exhaustive set of ideas could be created. If you assume three competing ideas for each of 15 unmet needs in various combination, then you are generating ideas on the order of three to the power of 15, which is 14 million ideas. The chances of any one idea effectively addressing 15 unmet needs are one in 14 million. Furthermore, in most markets, we find there are more than 15 unmet needs. So the number is even higher”.
So even if you do generate 14 million ideas, how long will it take to find the real 15 unmet needs?
I say a far better way is to observe.
Teresa Torres says
Thanks for your reply. I am a big fan of observation. In fact, in other posts I argue that goals should be set based on the customer journey and when the customer has success. Both are hard to figure out without immersing yourself in the customer experience. What I was trying to communicate with the first couple of steps in this article is that you should bound ideation by a goal rather than considering any and all ideas at once because as you mention you could wander through far too many ideas.
I have a more current iteration in this more recent post: http://www.producttalk.org/2015/09/outcome-based-product-management/
And I would argue observations should be an input to mapping the challenge – the first step in the new process.
David Fradin says
I agree. I just read your post How To Talk To Your Customers (Despite Henry Ford and Steve Jobs) and you clearly start out with “Observe”. And the observations, as you point out, should be at each step of the customer journey. Observations can then be followed by interviews and then surveys. Each step helps refine the next step. Steve was a keen observer of human behavior as should be every product manager.
Love your stuff. Do you have any sage advice on the ordering or lack thereof? If you have a goal to achieve, and concrete outcomes that could impact the goal, and ideas you’ve generated for achieving the outcomes, then really prioritization is happening throughout, right? For instance you must prioritize which assumptions to test first, or which experiments to run first, and not just what to build first. It’s not at all an issue with your post–more an issue with me trying to visualize this stuff. Thx.