Ch. 11: Measuring Impact
Continuous Discovery Habits turns five this year. And to celebrate we are reading the book together.
Each month, I am releasing an in-depth reading guide that includes:
- The chapters we will be reading
- A preview of the most important concepts we'll be learning about
- Short videos you can share with friends and colleagues to help spread the ideas
- Individual and team discussion questions to help you absorb and engage with the reading
- Team exercises to help you put the ideas into practice
- Additional reading to help you go deeper on the core ideas
We'll be discussing each month's reading in the comment section and we'll gather quarterly to discuss on a live call.
Joining late? No problem. I monitor the comments on each reading guide throughout the year. Start with the current month or go back to January—whatever works for you. You can ask for help, share what's working, and connect with other readers at any point.
If you want to participate, grab a copy of the book (or dig up your old copy), share the "Spread the Love" videos, reserve some time to do the team exercises, and register for the community sessions. Let's do this!
This Month's Reading

Chapters:
- Chapter 11: Measuring Impact
Estimated reading time: ~18 minutes
This chapter will introduce you to:
- Why you should start small with instrumentation—don't try to measure everything from the start
- How to instrument what you need to evaluate your assumption tests rather than everything you could measure
- The critical importance of measuring impact on your desired outcome, not just your assumption tests
- Why you shouldn't be afraid to measure hard things, even if they happen outside your product
- How discovery and delivery feed each other—they're not two distinct phases
Need a copy? Grab the book
Share the Love with Friends and Colleagues
We learn best in community. Use the following short videos to share the key concepts from this chapter with friends and colleagues. Invite them to participate in the book club with you.
- Business Outcomes vs. Product Outcomes - And what both represent.
Reflect & Discuss What You Read
When we reflect and discuss what we read, we absorb more of the material. It helps us put what we learn into practice. Don't skip this step.
This chapter brings us full circle—back to outcomes. But now we're not just setting them, we're measuring whether we're actually moving them. This is where the rubber meets the road. It's easy to get caught up in the excitement of successful assumption tests, but if we're not measuring impact on our outcome, we risk building solutions that don't create business value.
Individual Reflection
- Think about the last product change you shipped. Did you measure its impact on your outcome? If not, what would you need to do differently next time to make that possible?
- What's the hardest thing about your product to measure? Are you avoiding measuring it because it's hard, or have you found creative ways to chip away at it?
- Review the assumption tests your team has run recently. Are you instrumenting just what you need to evaluate those tests, or are you trying to measure everything?
Team Discussion
- Walk the lines of your opportunity solution tree together. For your current solution, can you measure not just whether it addresses the opportunity, but whether addressing that opportunity drives your desired outcome?
- Is there a connection between your product outcome and your business outcome that you're assuming is true but haven't tested? How might you test that connection?
- Which anti-pattern does your team fall into most often: getting stuck trying to measure everything, hyperfocusing on assumption tests while forgetting the outcome, or forgetting to test the connection between product and business outcomes?
Put It Into Practice
The best way to learn about measuring impact is to actually do it. These exercises will help you think through what to instrument and how to connect your discovery work to your delivery work.
Exercise: Design Your Instrumentation Plan
Time: 45 minutes
Do this: With your product trio
Start with your current solution (or the one you're about to test). Write down:
- The assumptions you need to test about this solution
- The evaluation criteria for each assumption
- Exactly what you need to measure to evaluate each assumption (be specific—are you counting people or actions? What triggers the measurement?)
- How you'll measure impact on your product outcome
- What's the smallest version of this solution you could test in production to get real data?
The goal isn't to design the perfect instrumentation. The goal is to identify what you need now to evaluate this set of assumptions and measure impact on this outcome.
Exercise: Find Your Hard-to-Measure Thing
Time: 60 minutes
Do this: With your product trio
Identify the thing about your desired outcome that's hardest to measure (like AfterCollege's "students getting jobs" challenge). Then brainstorm:
- Why is it hard to measure? (Happens off-platform? Long time delay? Requires user action?)
- What proxy could you use as a starting point, even if it's imperfect?
- How could you incentivize the behavior that would help you measure it?
- What's a small experiment you could run to start collecting this data?
Remember the AfterCollege story: they started at a 5% response rate and grew it to 37% over time. Don't be afraid to start small and iterate. The key is to start.
Go Deeper: Additional Reading
If you prefer an audio summary of this month's reading, including the book chapters and the following resources, I've included an audio version for paid subscribers at the bottom of this post.
Related In-Depth Guides
- Shifting from Outputs to Outcomes: Why It Matters and How to Get Started
- Assumption Testing: Everything You Need to Know to Get Started
Supplementary Reading
- Measure What Matters
- It's Not Enough To Know What To Measure, You Also Need to Get The Time Increment Right
- Moving Beyond Vanity Metrics: An In-Depth Case Study
- Insights from the CDH Benchmark Survey: Measuring Product and Team Performance
- Product in Practice: Iterating to an Actionable Outcome at tails.com
Other Voices
- TBM 25/53: Persistent Models vs. Point-In-Time Goals by John Cutler
- The Looking Glass: The Paradoxes of Data by Julie Zhuo
Audio Summary
This summary was produced by NotebookLM. The sources supplied were the book chapters as well as all of the additional reading.
This article is part of the CDH Book Club celebrating the 5-year anniversary of Continuous Discovery Habits. See all book club posts
Comments ()