I love that A/B testing has become so popular. It’s an important tool in our toolbox. But it frustrates me that we use it so poorly. Most teams make one of the following two mistakes: They either gate releases with an A/B test trying to understand whether or not the feature had the intended impact […]
How to Choose Your Next Experiments
“I think we should use folders.” “I think we should use tags.” “I know. Let’s test them both.” Does this sound familiar? I bet it does. By far the most common mistake teams make when they are new to Lean Startup methodologies is they fall into the trap of substituting judgment with experimentation. Instead of […]
The Dos and Don’ts of Hypothesis Testing
I see many product managers struggling with how to design and run good product experiments. It’s hard to know what to test and for how long. Few of us are trained to think like scientists. The tools are hard to use. We often overgeneralize our conclusions. It’s great that experimentation is becoming more common practice, […]
Why You Can (And Should) Experiment When Building Enterprise Products
Every time I speak about hypothesis testing, I get a series of questions about how to run experiments from people working on enterprise products. The assumption behind these questions is that hypothesis testing only works with consumer Internet products. And the assumption behind that belief is that the only way to experiment is to run […]
