Imagine a product team discussing new commenting functionality on their company blog. “We should let people comment anonymously.” “That will only attract trolls. We should require real names.” “We should […]
Why You Can (And Should) Experiment When Building Enterprise Products
Every time I speak about hypothesis testing, I get a series of questions about how to run experiments from people working on enterprise products. The assumption behind these questions is […]
Run Experiments Before You Write Code
The Lean Startup has a flaw. It’s a simple one. It advocates the feedback loop: Build -> Measure -> Learn. I agree wholeheartedly with this loop. The flaw is in […]
Don’t Rely On Confidence Alone
You find yourself in a conference room with your product team. The white board is covered with sketches. You have an idea that everyone is excited about. You mapped it […]
How to Estimate the Expected Impact of a Product Change
Your design team is excited about a new signup process. They are convinced it will increase registrations. Your sales team keeps asking for the same feature over and over again. […]
What to Do When You Don’t Have Enough Traffic To A/B Test
You’ve read the Lean Startup. You’ve drank the Kool-Aid. You’re ready to start experimenting. There’s only one problem. You don’t have any web traffic. If the only tool in your […]
The 5 Components of a Good Hypothesis
Update: I’ve since revised this hypothesis format. You can find the most current version in this article: How to Improve Your Experiment Design (And Build Trust in Your Product Experiments) […]
Putting the 4 Levels of Product Analysis Into Practice: A Halloween-Themed Example
It’s Halloween week! The pumpkins are carved, the candy bowl is full, and the neighborhood is about to be crawling with princesses, ninja turtles, and pirates. You’ve only got one […]