Have you heard? My new book Continuous Discovery Habits is now available. Get the product trio's guide to a structured and sustainable approach to continuous discovery. As a discovery coach, I work with all kinds of companies and teams. Some are brand new to discovery, others are already far along in their journey.
To start, I always work with the leaders to identify one or two teams who we think will become champions for continuous discovery within the organization. Our goal is to create a bright spot to highlight what good looks like within that organization. We want to take away the “that will never work here” excuse for the rest of the teams.
Today, I want to share a case study with you about one of those bright spot teams. Before I coached this product team at SEEK, their product manager had already taken my Continuous Interviewing and Rapid Prototyping courses. The team already valued user research and user experience design. But like many teams, they were doing discovery on an as-needed basis and were scrambling from activity to activity and from decision to decision.
You’ll see as you read their story that coaching helped them add structure to their process. That structure helped them settle into a routine, which simplified their discovery activities and helped them find a more regular cadence. They transitioned from a strong discovery team to a strong continuous discovery team. That’s what I like to see.
Most of all, I’m confident that this team built habits and developed mindsets that will serve them over their lifetime as a team and over the lifetime of their individual careers, and that feels great.
My blog editor, Melissa Suzuno, conducted this interview with Chris Mercuri, Senior Product Manager, Natalia Henriquez, Iteration Manager/Business Analyst, Scott Lacey, Senior Visual Designer, and Jamieson Hickingbotham, UX Designer.
Tell me a little about your team. What are you working on? What are you trying to accomplish?
Chris Mercuri: SEEK is Australia’s biggest job marketplace and our team works on the company/partner side of the marketplace. The recruitment process can be difficult and our goal is to make it easier for small and medium-sized companies to find a suitable candidate for their job.
Our team has five developers, a UXer (Jamieson), Visual Designer (Scott), Iteration Manager/Business Analyst (Natalia), and me as Product Manager. The team has always had a “pioneering” spirit, wanting to do things differently and improve as individuals and as a team. And we have always had a strong culture of collaboration, wanting to solve problems together!
A few members of the product team—UX Designer Jamieson Hickingbotham, Iteration Manager Natalia Henriquez, Product Manager Chris Mercuri, Senior Developer Damien Ung, and Developer Annie Huang—feeling pretty happy with themselves after building their first shared customer experience map.
What was your life like before coaching with Teresa?
Chris: SEEK has been very active in trying to improve its discovery processes and our team had been trying a lot of new approaches before we started the coaching with Teresa. We already had OKRs and that was working well. We had built confidence interviewing customers, and were strong on framing the problem—separating the problem space from the solution space. However, we weren’t talking to customers enough (only every few weeks) and we were struggling to turn interview materials into business opportunities (we had a thousand stickies but didn’t know what to focus on). There were other gaps in our discovery process also.
How did you decide what to build?
Jamieson Hickingbotham: Identifying and prioritizing opportunities was informal and ad hoc. Sometimes it would be in response to big call drivers for our Customer Service team, sometimes we would be trying to solve problems in the core value delivery for large segments of our users. At this time, we lacked a clear remit. It was often up to us to decide our destiny, but we lacked a framework to approach this in a structured manner.
Chris: Previously, what we decided to build derived from customer interviews, analytics data, and other sources (customer service/live chat questions, observing users, etc.). We also had some Jobs to Be Done research at our disposal. We had gone beyond a list of solutions to a list of problems, however our level of understanding of the customer’s problems/opportunities was much less developed and we didn’t have our problem space so clearly mapped out and sized. As Jamieson said, we didn’t have a consistent, structured process.
How often did you talk to customers?
Jamieson: During the 12 months prior to our coaching with Teresa, we conducted around 30 customer interviews in different formats, including over the phone and in our users’ context.
Chris: Prior to the coaching, we had committed to going and interviewing customers in their work spaces (where they were using our application). This was taking time to arrange and then consolidate, so we were probably doing only five or six per quarter.
What did you think about conducting customer interviews?
Jamieson: Customer interviews were planned and conducted to meet a specific need, at a specific time. We would run 10–15 interviews, after which we would analyze, extract meaning, and report back to the team. Most team members came and participated in at least one interview to gain some direct exposure. As we conducted these sequentially, we found at times that the interviews didn’t give us what we were originally after. In one case we planned and ran around 15 contextual visits—a significant time investment—to find that only 2 of the participants met our target criteria. We had made assumptions about who our users were, which meant we didn’t properly screen for our needs.
Chris: As Jamieson said, we were undertaking interviews that were heavily question based to meet specific short-term needs. We were missing the important broader context of what the user was trying to get done and the effort wrapped around this process was too high, which meant we couldn’t get it done as often as we wanted to.
How often did you run product experiments?
Jamieson: Our experiment rate had improved from around three months to four to six weeks—valuable progress, but still slower than we would have liked it to be. We had begun to stagger our experiments to allow time to adjust our prototypes to initial feedback, but the process was laborious and unstructured.
Chris: We were fully onboard with the idea of rapid experimentation but were probably overcooking experiments (prototyping to high fidelity) and therefore not running enough. We were not as clear on the risky assumptions that we were testing as we wanted to be, either.
What was the process of working with Teresa like? What were some moments that stood out to you?
Scott Lacey: Our team was quite focused on the value of our user research before we began working with Teresa. What Teresa helped us to understand was the value of regular weekly discovery. Testing more regularly and less formally. Her approach challenged the way we recruited for research. We had used a combination of recruitment methods previously (including using a recruiter) but Teresa encouraged us to automate the recruiting for feedback sessions. We haven’t as yet really nailed this system of booking sessions automatically, but I feel confident that we will! Regular sessions focusing on this week’s assumptions help us to understand our customers’ needs and opportunities more clearly.
Regular sessions focusing on this week’s assumptions help us to understand our customers’ needs and opportunities more clearly.
– Tweet This
Natalia Henriquez: I valued those moments when Teresa would challenge our thinking. She encouraged us to look further, harder, and not to fall into group thinking.
Chris: It was great fun working with Teresa, who is a great person and a very experienced product leader and coach. The way the coaching is designed is that you apply the tools/principles to your work, therefore it was super practical and being able to discuss what we did weekly was hugely valuable.
There were a lot of moments that stood out to me. Recruiting customers was something that often took us more time than we wanted it to, but being encouraged to automate this activity immediately took the hassle out of customer recruitment and increased our time with customers.
The story format of interviewing that Teresa advocated proved to be an effective way to extract more context from interviews, but also led to a better conversation with customers and less preparation, which again helped us interview more often.
The story format of interviewing that Teresa advocated led to a better conversation with customers with less preparation, which again helped us interview more often. – Tweet This
The team was already very big on visual sense making, but the opportunity solution tree really helped us to map out and connect the opportunities and solutions in our domain. It also gave us a nifty way to share this with others at SEEK in a way that they could quickly make sense of, so the quality of feedback we received improved.
Being specific about the risky assumptions associated with the solutions that we were developing was something that we had explored before, but Teresa helped us to take this to a whole new level. It was really helpful for us to be clear on what results we needed from experiments to mitigate our own confirmation bias.
Were there any particular aha moments or was it more of a gradual process?
Scott: Of course, as product people we are always considering the impacts that our design solutions have on our users, but an aha moment for me was how we learned to structure and classify our assumptions. This activity signified the moment when our opportunity thinking was stress-tested. Learning how to assess our assumptions helped us better structure and classify options that we could or should test with our customers. It forced us to be more critical of our thinking so that we focused our efforts effectively and efficiently.
Senior Visual Designer Scott Lacey found that learning to assess assumptions was a critical step in deciding what to test with customers.
Natalia: We had a number of aha moments. To name a few, the moment we understood how to align the business needs with the customer needs, how to integrate OKRs with our discovery process, and the power of visualizing the journey map.
Jamieson: An aha moment for me was going through the process of storyboarding and being specific about our assumptions and what evidence we’d need to see to de-risk them. This took a process that for me was happening in a more intuitive, generalized way, and made it explicit. This highlighted gaps and inconsistencies early, gave us a tool to aid in discussion, and helped us explore many more options for experiments. The aha moments are still unfolding today as we’re seeing how our time with customers, ideation, and experiment design all feed into each other.
Being specific about our assumptions and what evidence we’d need to see to de-risk them was an aha moment. This took a process that was happening in a more intuitive, generalized way, and made it explicit. – Tweet This
Paint a detailed picture of what things are like for you now. What are you doing differently that you weren’t doing before? How has this impacted your work?
Jamieson: As a team we now have processes in place for managing our discovery activities on a weekly basis. We have checkpoints to review what we’ve learned, plan what we need to discover on for the week, and multiple lean options for finding answers to the questions we have. We no longer think first in terms of functional prototypes, and we don’t gold plate our discovery at every stage.
We start with multiple ideas, clearly articulate how we believe they will deliver value and what risky assumptions underlie that belief. This provides us a framework to prove or disprove those assumptions and identify means of measuring the relative value these ideas deliver. Our value delivery is no longer tied to the success (or otherwise) of any particular idea.
We start with multiple ideas and clearly articulate what risky assumptions underlie them. Our value delivery is no longer tied to the success (or otherwise) of any particular idea. – Tweet This
Scott: Yes, I agree with Jamieson here. The focusing on de-risking our assumptions has been really useful. Categorizing them and then prioritizing each one has helped us to be less wedded to any particular idea.
Natalia: As an Agile Delivery Manager, I now feel much more confident that our teams are living and breathing those critical Agile values. And this is not only through the early and continuous delivery of software, but also the continuous discovery and the close collaboration and engagement of our customers and the delivery teams.
Chris: There are differences in our product discovery approach. We’re interviewing customers almost every week (sometimes it doesn’t come together!), whether that be exploratory interviewing or running more evaluative experiments with specific solutions. Our knowledge of our customers’ wants, needs, pain points, and jobs has increased. You hear customer anecdotes thrown around casually from different members of the team now, which is great.
We’re pushing our ideation process further, making sure that we generate more solution options. When before we might have considered two or three options, we now try to come up with ten or more. And rather than experimenting with one preferred solution, we experiment with several (two or three) to get a better read on which idea seems more promising.
We’re much clearer on what we want from experiments—clearer about our assumptions and clearer about the behavior that we need to see to know if the solution is promising.
There has been a major difference in our attitude also. I would say that we are more confident since working with Teresa. Knowing that we’re going through a good process, and knowing our customer better is helping us to explore ideas that are more bold and ambitious and hopefully the team will increase its impact over time.
We are more confident since working with Teresa. Knowing that we’re going through a good process, and knowing our customer better is helping us to explore ideas that are more bold and ambitious. – Tweet This
How often do you engage with customers? How often do you run experiments? Are you using the opportunity solution tree?
Jamieson: We now conduct interviews weekly and are setting up regular fortnightly sessions to allow for concept testing by multiple teams. Having clearly articulated assumptions and prioritizing these means we start getting feedback much earlier and are ready to pivot where needed.
The opportunity solution tree continues to be a valuable tool in modeling what we’re hearing in interviews and how this contrasts with our product offering.
UX Designer Jamieson Hickingbotham now finds multiple ways to address a single customer need.
What have the results of this shift been?
Jamieson: The biggest shift for me has been the focus on considering multiple ways of addressing a single customer need, with clearly articulated assumed value and risk, and a structure for starting early and progressively discovering the truth of those assumptions.
Natalia: Having visibility and also being able to have a common language with our PM and UX team has been very valuable. It has increased my understanding and appreciation for the challenging decisions they have to make.
Chris: One of the most pleasing elements of the shift has been observing what the team didn’t build as a result of new discovery habits. We have modified our solutions based on what we learned in our lean experiments, and that for me is a great result!
One of the most pleasing elements of the shift has been observing what the team didn’t build as a result of new discovery habits. – Tweet This
What would you say to another team who was considering working with Teresa?
Natalia: I fully recommend this program to product delivery teams who have been struggling to understand how to engage the team in the discovery process. I would not say that it will be a smooth journey—you might face a number of emotions ranging from excitement and joy to fear, confusion, and frustration at times. But knowing that you are in touch with your customers and their needs is very rewarding. It builds your confidence and passion for your product and its customers.
Knowing that you are in touch with your customers and their needs is very rewarding. It builds your confidence and passion for your product and its customers. – Tweet This
Scott: Teresa’s clear thinking coupled with her vast product knowledge and experience really came to the fore during the course. She was great at straightening us up amidst the growing pains of learning a new way to think. We appreciated her coaching, her knowledge about how to build great products, and her wonderful nature!
Chris: I would encourage any team that is in a position to work with Teresa not to hesitate! If you have teams that are less experienced with discovery then you can give them a set of frameworks that will really help them move forward. Even if you think you are doing discovery well, Teresa will challenge your thinking, get you moving faster, and support you to go to the next level!
Even if you think you are doing discovery well, Teresa will challenge your thinking, get you moving faster, and support you to go to the next level! – Tweet This
A final note from Teresa: It makes me so happy to support a team as they level up their continuous discovery practices. This team has not only done that, but they’ve become champions for continuous discovery within SEEK. If you are interested in similar results, you can learn more about discovery coaching here.
