Have you heard? My new book Continuous Discovery Habits is now available. Get the product trio's guide to a structured and sustainable approach to continuous discovery. Is your company shifting to the product operating model? Are you trying to figure out where to start?
If so, this series on organizational change is for you. Hope Gurion and I are reflecting on our experiences as coaches who have worked with hundreds of teams, sharing what we’ve seen work—and not work.
If you missed the previous editions, you might want to start here:
- Is Your Organization Ready to Adopt the Product Operating Model?
- Avoid This Common Organizational Transformation Mistake (And What to Do Instead)
In the most recent video, we described the common mistake of trying to train everyone all at once. This creates a lot of chaos and rarely has the intended effect.
There are many reasons why this can happen, but often it’s because not everyone is ready for change. And when you try to require change before people are ready, you’re much more likely to encounter resistance.
So what should you do instead? The short answer is: Start with a smaller group of pilot teams.
In today’s conversation, we’re taking a closer look at what this actually looks like in practice. How many pilot teams should you select? How do you choose the participants?
You can watch the video of our discussion or read a lightly edited transcript below.
Full Transcript
Teresa Torres: Hi everyone, welcome back. I’m here again with Hope Gurion in our next installment in our org transformation series.
If you’ve been following along, in our most recent video, we talked about one of the most common mistakes teams make when kicking off an org transformation.
And that was this idea of, should we train all of our teams at once and how that often leads to some chaos.
And what we recommended in that video is that you actually start with some pilot teams. Pilot teams allow you to surface where there’s going to be challenges in the organization, keep it a little bit more constrained, so it’s less chaotic and you’re able to learn as you go.
So Hope, to kick us off, do you want to start with: If I’m a product leader thinking about this, how many pilot teams should I be choosing?
How Many Pilot Teams Should You Choose?
Hope Gurion: Yeah, I would say you don’t want to choose a single product team and you don’t want to choose too many pilot teams. You want to choose somewhere like two, three, four-ish pilot teams.
Depending on the number of total product teams in your org, you might do one or two, or if you have a very large org, you might want to do a little bit more than that, but you want it to be a manageable number.
Teresa Torres: Okay. So if I’m a tiny company, I have three teams. I might start with one.
Hope Gurion: You might.
Teresa Torres: If I’m a little bigger, I might pick two or three. And then if I’m giant, I might pick five.
Hope Gurion: Yes.
Teresa Torres: What’s the constraint? Why am I picking only a few teams?
Hope Gurion: Each team is going to run into different challenges and is going to have a different composition of people, skills, product types. And so if you just choose one team at one end of the extreme, an awful lot has to be right with that one team to learn whether it will work for that team, whether it will work for the org.
When you increase the number of teams, you tend to learn more. You can pattern spot a little bit better. But if you have too many pilot teams at once, it’s going to be hard for you to manage and be able to provide the support and coaching to enable those teams to be successful and to troubleshoot when they run into challenges. So that’s why it’s sort of this sweet spot of two to four-ish types of teams, depending on the size of your org.
Teresa Torres: You know what I’m hearing is we’re setting up a compare and contrast decision—
Hope Gurion: Right? Yes, we are.
Teresa Torres: Across all of our teams, what are the different obstacles they’re coming up with? What types of teams are going to get this the best? Who needs the most help, which is going to enable when we roll it out company-wide, what do we need to be prepared for?
Hope Gurion: Absolutely.
5 Factors to Consider When Choosing Your Pilot Teams
Teresa Torres: Okay. I know you have five factors that we should consider when choosing teams. Do you want to give us an overview of the five?
Hope Gurion: Here are the five things that I recommend product leaders look at when they’re selecting their pilot teams:
- They want to consider the relationships amongst the team members
- They want to think about the mindset of those individuals on the teams. How willing are they to learn?
- The skills that are resident within the people on those teams
- Their access to their target customers
- And the life stage of their product
1. Relationships among the team members
Teresa Torres: Okay, perfect. Let’s dig into each of these. So tell me about what we’re looking for with relationships on the team.
Hope Gurion: For relationships, it’s hard for a team to really work together effectively when they’re just meeting each other for the very first time. So you’re looking for people who have some depth of relationship with one another. Maybe they’ve worked together on the same team for some period of time, or maybe they’ve worked together in the past on different teams, but they do have a relationship with each other.
We don’t want to put strangers together and expect them to just nail it—work together effectively on day one. So we’re looking for mutual respect, and we’re looking for familiarity with each other.
Teresa Torres: I think there’s also something around working styles. I coached one team where the engineer and the designer just were at each other’s throats over everything. Two totally lovely people could not agree on a single thing.
And then later I actually coached them again on different teams and they totally got it and it was great. But it was just their working styles didn’t work with each other.
Hope Gurion: If you have team members that have familiarity, but they don’t know and trust each other—if they don’t like working together—I would not recommend putting them together in a pilot team.
2. Team members’ willingness to learn
Teresa Torres: For sure. Okay, let’s talk a little bit about a willingness mindset.
Hope Gurion: For most people moving to an outcome-oriented way of working, they’re going to be doing a lot of things for the first time. They’re going to run into challenges. I’ve never done this before. It’s uncomfortable.
If you have a fixed mindset as opposed to a growth mindset—if you are very attached to the status quo—you’re not going to be approaching this experiment with a willingness to learn.
And so we want to make sure that we’ve got people who are open-minded enough to embrace the inevitable challenges that come with learning new things, new ways of working, and bringing that willingness to the table so that they can persevere when they hit those inevitable challenges and moments of discomfort.
Teresa Torres: So this might be like we’re looking for our early adopters. So maybe they’ve read Marty Cagan’s books, or they’ve read Continuous Discovery Habits, or they’ve been to a conference and come back excited about some of these ideas, showing that they’re always looking for new things and have that early adopter mindset.
Hope Gurion: That’s a great signal to help you think about who might be a candidate for this. They’re always looking at new ways of doing things. It’s a great signal that you could get as a product leader.
Teresa Torres: What if this is all just brand new to my organization and nobody’s heard of it? Can I just ask for volunteers? What are other ways to evaluate willingness?
Hope Gurion: That’s a great example. Who’s raising their hand for this? If you’ve got to hunt them down and find them and you’re dragging them into being part of a pilot team, if they’re not showing up—I was working with a client recently that had a whole information session for the pilot teams. Many members of those pilot teams did not show up to the information session. That’s a signal that maybe they’re not that into you. And so you might need to rethink who’s on your pilot teams.
3. The team members’ existing skills
Teresa Torres: I like that framing a lot. Let’s talk about skills. What skills am I looking for?
Hope Gurion: We want to have a skills-complete team, right? Now, even if they’ve never worked this way, we know inevitably they’re going to be learning new skills. But if they’ve never been in an engineering role and they’re the tech lead on your pilot team, they’re going to have too many things that they need to learn.
We’re looking to develop incremental skills. So we want to have a capable product manager, a capable product designer, and a capable engineering lead.
Again, we’re not expecting them to know exactly what to do. But if they’ve never operated in that role before, they’re probably going to have too many things to learn to be an effective pilot team.
Teresa Torres: I think we’ve seen this a lot where we get a team that’s all junior right out of college, they’re still learning how to have jobs. That’s too much to add this on top of, or the person who just moved into a product management role, and they’re still trying to figure out what product management is.
I think what you’re saying is, when we’re looking at skill, we don’t want our beginners. They may eventually have to learn this way of working—we want everybody to. But for our pilot teams, we’re trying to pick the teams that are most likely to have success. And so we want people that are mature in their roles, early adopters, where really what they’re focusing on is what’s new with this way of working and not a bunch of other stuff.
Hope Gurion: Exactly right. And this is why that previous one, that willingness mindset, is so critical, because you could have somebody very experienced as a product owner, or as a graphic designer, and think that they have the skills to work this way.
And yes, they’re experienced in a certain way of doing things. But we want to make sure that they have the skills to know how to do, say, interaction design, or product management that includes strategy, not just backlog management. They’re probably going to be learning some new things, so we want them to embrace that there might be more to the role than what they’ve always done.
Teresa Torres: There’s something I want to highlight here because I think sometimes we see skill negatively interacts with willingness. And so what made me think about this was you talked about a graphic designer moving into interaction design. We actually sometimes see UXers, they come into this thinking, “I already know how to do it.” And then they think they have the skill to work this way. And it interferes a little bit with their willingness to learn maybe the difference between a project mindset and a continuous mindset.
Hope Gurion: Yeah, I think that’s a great example. But the beauty is all of that gets exposed in the pilot team model.
4. The team members’ access to target customers
Teresa Torres: For sure. Okay, tell me about customer access.
Hope Gurion: One of the foundations—it’s part of the definition of continuous discovery— weekly touch points with customers. So you need to make sure that your team is going to be able to interact with customers. Maybe not every week, but getting closer to every week during this pilot phase.
We want to make sure that if there’s any obstacles, barriers—real or imagined—that those teams are set up for success and do have access to customers that they can learn from. Because without that, they’re not going to make progress on their outcomes.
Teresa Torres: For our programs, coming in, they don’t have to know how to recruit. That’s not what we mean by access to customers. We can teach them how to find someone to talk to.
I think the barriers we see that we can’t help them overcome are things like your sales team doesn’t let the product team talk to customers.
There has to be some work at the executive level to remove some of those barriers. Or our legal or compliance team requires they go through a six-week training before they’re allowed to talk to customers. And you’re starting your pilot team before they’ve gone through that six-week training. It’s more of these institutional barriers to accessing customers, not know-how barriers.
Hope Gurion: Yes. Beautiful clarification.
5. The product’s lifecycle stage
Teresa Torres: Awesome. I think we have one more. Tell me about the product lifecycle stage.
Hope Gurion: And not every team uses this framing, or sometimes people refer to it differently. So there’s the “explore, invest, sustain, sunset,” or there’s “explore, exploit.” There’s a couple of different framings of this, but here’s what I recommend. We want to focus on pilot teams that are either at an “explore” or “invest” stage.
When you’re working on a product that’s in “sustain,” it’s usually about just keeping that product available, performant, bug free. We’re not trying to increment value creation on that product. We want it to keep delivering value.
And so there’s usually not as much discovery required and the outcomes are pretty much: keep it working as it’s always worked. And then “sunset,” there is some discovery. Or if you’re trying to replatform a product, that’s another one that you could do some discovery, but there’s going to be probably a lot of challenges for that team to create an outcome.
And so “explore” or “invest” typically provide the most learning room for a team. And that’s usually what we recommend. It’s not impossible with the others, but you get more learning velocity with an “explore” or “invest” stage product.
Teresa Torres: I recently recorded a video that said your re-platforming teams absolutely should do discovery, but they’re not your best pilot teams. Yes, eventually you want your platform teams to learn how to work this way or your re-platforming teams to learn how to work this way because they will benefit.
But the point of a pilot team is to help us surface the organizational challenges we have to tackle to be able to spread this across the organization, so we want the teams that can move fast. And that’s typically not those re-platforming teams.
Hope Gurion: Exactly.
Teresa Torres: I think also there’s a lot of cognitive dissonance with re-platforming teams on how this works because there’s just this acknowledgement that we’re just building feature parity and it’s hard to chip away at that.
Okay, but you said “explore,” which could be a zero to one product. It could be looking for a brand-new product, or it could be “invest,” in which case we have a product, but the phrase I really love that you said was: We’re looking for how to add value to the product. So we’re not just in “sustain” mode.
Okay, this is great. I feel like I can even imagine a little visual we can create of a checklist for people to think about when they’re picking their pilot teams.
Any last thoughts?
Navigating the Challenges of Being on a Pilot Team
Hope Gurion: The only other thought would be: People are people and people might like the idea of being on a pilot team, and then they may encounter resistance, and they may no longer like being on a pilot team. And willingness to learn the relationships help keep a team persevering when things get challenging or when this isn’t as easy as the old way of doing things—that’s going to happen. And so again, that willingness and the relationships, it’s going to help a team persevere.
Teresa Torres: In our upcoming videos, we’re going to get into how we support these teams as they’re going through the pilot. The thing that I’ll end with is just to remember when you’re choosing your pilot teams, you want to choose the teams that are most likely to have success working this way.
Our goal is to create a bright spot in your organization to show it can work. So you don’t want to pick the team that’s on a PIP (performance improvement plan). You don’t want to pick the team that needs the most coaching. We’re looking at mature teams that function well, that have a willingness to learn, which is why I really love this set of criteria. All right. Thank you, Hope.
Hope Gurion: Thanks.
Don’t miss the next video in this series. Be sure to subscribe below.
