Why Engineers Should Participate in Discovery

Have you heard? My new book Continuous Discovery Habits is now available. Get the product trio's guide to a structured and sustainable approach to continuous discovery.

Are you inviting your engineers to participate in discovery with you? If yes, do you get pushback from them? Does your organization prefer that your engineers only write code?

Marty Cagan, in Inspired, argues, “If you’re just using your engineers to code, you’re only getting about half their value.” I couldn’t agree more.

But too often we equate the value of an engineer with the code that she writes. This is a huge mistake. And, sadly, many engineers make this mistake themselves. They just want to put their heads down and write code, leaving everything else to the business.

While I don’t believe in forcing engineers to participate in discovery, I do think product teams should do everything in their power to encourage their engineers to do so. Engineers bring skills to the discovery process that other members of the product team may not have.

Engineers bring skills to the discovery process that other members of the product team may not have. This is why it’s so valuable to have them participate—even if they’d rather be writing code. – Tweet This

Engineers Will Hear Different Insights from Your Customer Interviews

Good discovery is fueled by valuable insights gleaned during customer interviews. What we hear and select as salient from our interviews is influenced by our prior knowledge and expertise.

You’ve probably had the experience of talking to a coworker after an important meeting, only to realize that you each heard different takeaways. As you talk it through, you realize you interpreted a key statement differently or you each focused on different key statements. This selection of different data and/or the different interpretation of the same data is an example of Chris Argyris’s Ladder of Inference at play.

Photo credit: This image comes from the Fifth Discipline Fieldbook by Peter Senge

Long-time readers might remember the Ladder of Inference is a model that describes how we interpret the world. It explains that as we engage with the world, we select a subset of observable data, we add meaning to that data, we make assumptions based on that meaning, and then we draw conclusions and adopt beliefs based on those assumptions.

Because we all engage with the world from a unique perspective—we each have different knowledge, expertise, and past experiences—we each select different data, attach different meanings to that data, make different assumptions, and often draw different conclusions.

This is true even for two product managers listening to the same customer interview, but it’s especially true for a product manager and an engineer, because they each bring unique experiences to the interview. The product manager and the engineer each select data, attribute meaning, make assumptions, and draw conclusions from their own knowledge and expertise. Because product managers and engineers each have their own domains, they aren’t likely to draw the same conclusions. This is a good thing.

This is exactly why we talk about the trio (i.e. the product manager, the design lead, and the tech lead) leading discovery. We want to leverage all three types of knowledge and expertise throughout the discovery process. We want to incorporate the meanings, assumptions, and conclusions that an engineer might draw from an interview that are different from what the product manager or designer might draw.

Having the trio of product manager, design lead, and tech lead involved in discovery helps you leverage all three types of knowledge and expertise. – Tweet This

What does this look like in practice? Imagine you are interviewing a Netflix customer and they tell you the following story:

The product manager might key in on, “‘Most popular’ was all TV shows” because she’s been concerned that we don’t have the right content.

The designer might hear, “we couldn’t figure out how to search for a specific movie title” because she is primarily interested in improving the user experience.

The engineer might hear, “all of the recommendations are for kids’ movies” because she might be interested in finding a technical way to detect different types of users.

All three selected different data as salient because of the perspectives they brought to the interview. Each perspective is valid and can lead to an important product improvement.

If you want to get more from your interviews, conduct them as a team. I encourage teams to have the whole trio attend as many interviews as possible.

Engineers Often Generate the Best Solutions

When it comes to generating solutions, engineers often come up with the best solutions. This surprises a lot of teams.

It’s not because engineers are the most creative or the smartest people on your team (even if some of them think so). It’s because they have a depth of knowledge about what’s possible with technology that often surpasses everyone else on the team. But there’s a caveat that we need to consider here.

When brainstorming solutions, engineers often come up with the best solutions. But it’s not because they’re the smartest people on your team—it’s because they have a depth of knowledge about what’s possible with technology. – Tweet This

The best solutions are the result of understanding what’s possible with technology and understanding what our customers need. So if an engineer is going to leverage that depth of knowledge about what’s possible with technology, they also need to build up a depth of knowledge about the customers’ context. The only way to do that is to participate in discovery.

Now some of you might argue that the product manager and the designer bring customer context. And they do, and they should. But if you think this is a sufficient solution, you’ve never been in a debate with an engineer about what you should build. We all (not just engineers) need to see, hear, and feel firsthand our customers’ experience to believe it.

Finally, engineers are in the best position to help us translate our pie-in-the-sky ideas into something feasible that could exist today. We want them as active participants in the ideation process, not just recipients or judges of the outputs of that process.

We want engineers as active participants in the ideation process, not just recipients or judges of the outputs of that process. – Tweet This

Invite your engineers to generate solutions with you. You’ll be pleasantly surprised by the result.

Engineers Save Us From Discovery Boondoggles

Imagine this: You are a product manager paired with a designer responsible for designing the car detail page of a used car website. You want to communicate the condition of a vehicle to a potential buyer and you have several photos of the car, but you are worried that the buyer won’t understand how to interpret the photos. So you generate some ways to explain to the buyer what they are viewing in each photo. You need to test these new designs. What do you do?

You could start with mockups of the new designs, but when you go to get feedback, the majority of the feedback is about why the potential buyer isn’t interested in that particular vehicle for reasons other than the condition of the car—it’s the wrong make or model, they don’t like the color, etc.

You iterate by asking participants to share with you ahead of the session a car they were recently considering. You create a tailored mockup for each session based on the user’s own data. But you continue to get irrelevant feedback. The user has already found another car they are interested in more or they have since been convinced this is not a good option.

You iterate on your mockups, trying to abstract away all of the non-pertinent details so that you can get feedback on the design elements that you do care about. But when you test these new designs, they cause mass confusion—users want to know where the missing elements are.

When we are testing designs that are dependent upon real end-user data or interests, it can be hard to test our designs with in-person rapid prototyping sessions. Instead, we need to build prototypes using real customer data. This can be hard to do unless we have an engineer helping us. But since engineers are rarely involved in the discovery process, it’s hard to prioritize this type of work against the “real” work that needs to go into the next sprint. So we skip it. Or we waste hours or days or even weeks trying to approximate the experience with crude mockups.

These are exactly the situations when we need an engineer working with us in the discovery process. We don’t have to build the whole solution (nor do we want to). But we do need to prototype with real data and that means with real code.

In the above scenario, you could identify one or a few vehicles to manually add your design changes to and use the population that is already considering those vehicles to test the design. Is this a hack? Absolutely. Will you throw away all of the code you write to test this design? I hope so. But if it’s the fastest way to learn what will work, you should do it anyway.

When engineers participate in the discovery process, we remember to use code when it’s the best way to test an idea. When engineers are missing from the discovery process, it forces the product manager to prioritize throwaway code designed to help us learn against production code to ship the next feature. This isn’t a fair battle.

We want an engineer working with us throughout discovery to help us identify these situations and to help us build prototypes that require real code.

When engineers participate in the discovery process, we remember to use code when it’s the best way to test an idea. – Tweet This

Engineers Often Ask Disconfirming Questions

As we work with our product ideas, we tend to fall in love with them. It’s human nature.

As a result, we tend to ask confirming questions. We look at all the reasons why the idea will be a wild success. But to find the flaws in our ideas, we need to ask disconfirming questions. What might go wrong?

We’ve all worked with that engineer who picks everything apart. She has no shame in highlighting why something won’t work or identifying the flaw in our favorite idea. This can be frustrating, but it’s a valuable skill to have on your product team.

A big part of discovery is identifying and testing assumptions. This devil’s advocate or “black hat” attitude is invaluable when surfacing assumptions. We need to be open to our ideas’ ugly warts so that we can design around them.

The Lean Startup encourages us to test the underlying assumptions that need to be true in order for our idea to work. But too often as product teams, we fall into the trap of testing the whole idea. This is slow and expensive. We do this because it can be hard to identify our own assumptions.

Use your engineers who are good at asking disconfirming questions to help you identify your underlying assumptions. Invite them to participate in a pre-mortem for each of your ideas. Ask them, “Imagine we built and launched this idea and it was a colossal failure. What went wrong?” Your engineers who are great at seeing the flaws in your ideas will excel at this.

Engineers tend to be strong at asking disconfirming questions because it helps them catch error cases or edge cases in their code. It’s a skill they practice regularly. Take advantage of this and invite them to participate in discovery with you.

Engineers tend to ask disconfirming questions. This is invaluable when trying to identify the assumptions you need to test. – Tweet This

Engineers Bring an Analytical Mindset to Your Product Experiments

Often times the best way to test our underlying assumptions is through quantitative experiments. Smoke screen tests or fake door tests, A/B tests, and multivariate tests are useful tools in our toolbox, but they can be hard to get value from if we don’t have the necessary analytical skills to use them well.

We need to understand statistical significance, the risk of false positives and how to mitigate them, and know how to adequately interpret our results. Our testing tools often make these steps seem simple, but they can be misleading. Our tools can dump out data, but they can’t tell us what that data means. This is where an analytical mindset is invaluable.

Many product managers and designers are strong analytical thinkers and can be well-versed in these areas. But software engineers, because of their engineering backgrounds, are more likely to have a stronger background in statistics and scientific thinking.

It never hurts to have more analytical minds evaluating your experiment design and helping to interpret results. So even if you think you have this one covered, asking your engineers to weigh in will help you catch mistakes you might otherwise miss.

It never hurts to have more analytical minds evaluating your experiment design and helping to interpret results. Ask your engineers for help. – Tweet This

It can be devastating when you communicate successful experiment results only to learn that you made a mistake in your methods or you misinterpreted your results. Avoid this fate more often by inviting your engineers to participate in your discovery.

Delivery Goes Smoother When Engineers Participate in Discovery

Product managers often complain that they don’t have time to do discovery. When I ask them how they are spending their time, inevitably the answer is always with engineers in delivery.

Delivery is urgent. We are continuously marching toward the next release. We are measured by what we ship. This is a good thing. We can’t create value for our customers or our business if we aren’t shipping code. But if we aren’t careful, it can mean that discovery plays second fiddle.

While delivery is important, product managers should spend the majority of their time in discovery. When I tell product managers this, I always get pushback. Who will help the engineers when they have questions? Who will make sure the current sprint is on track? Who will ensure that we make our release date?

The answer to all of these questions should be your engineering lead. Let’s look at why this isn’t happening today.

When a product manager does all the discovery, the engineers have no one to turn to when they have questions about what they are building except for the product manager. This is why the product manager gets pulled into delivery so often.

It’s impossible to define perfect requirements. There will always be questions about what something means. There will always be a need to revise requirements based on feasibility limitations that come up as we code. Today, on most teams, the product manager is the go-to person to answer these questions. But if the tech lead also participates in discovery, they will be equally good at answering these questions.

This not only frees up the product manager to spend more time in discovery and more time with the rest of the business, it also gives the engineers more autonomy, allowing them to move faster.

So why don’t we do this today? There’s an uncomfortable truth that most product managers don’t want to hear. It feels good to be the gatekeeper, to have all the answers. We like swooping in to be the hero when a problem arises.

But this comes at a cost. If we continue to dictate every detail to our engineers, we’ll always be bogged down in the delivery details. Invite your engineers to participate in discovery with you and empower them to answer their own questions.

If as product managers, we continue to dictate every detail to our engineers, we’ll always be bogged down in the delivery details. By inviting your engineers to participate in discovery, you empower them to answer their own questions. – Tweet This

Now this doesn’t mean you don’t spend any time in delivery. You still need to make sure the sprint is on track. There will still be questions that only you can answer. But take a hard look at what fires you can delegate to others and do so.

What to Do If Your Engineers Don’t Want To Participate in Discovery

I’ve heard from many product managers and designers that their engineers don’t want to participate in discovery. They want to spend as much of their day coding as possible. They hate meetings and they see discovery as just more meetings.

Don’t force your engineers to do discovery. This won’t get you anywhere and it will cause more problems than it solves. Instead, require customer knowledge as a prerequisite for having an opinion about what you should build.

Require customer knowledge as a prerequisite for having an opinion about what you should build. – Tweet This

What does that mean? In your next sprint planning meeting, when an engineer pushes back on a requirement or otherwise has an opinion about what you should build (remember they all do), ask them what customer knowledge they have to support their opinion. If they say they don’t have any, invite them to join you in your next discovery activity to collect some.

If you require that your product decisions be supported by customer context, everyone on the team will want to participate in discovery.

What are you waiting for? Share this article with your engineering team today.