Failure Mode
What is a failure mode?
A failure mode is a specific category or type of error that an AI product makes when generating outputs. These failure modes are identified through error analysis, where developers review production traces to categorize the most common mistakes their AI system is making.
Identifying and cataloging failure modes is a critical step in building evals, as each failure mode can be measured with specific evaluations to track how often the error occurs and whether product improvements are reducing it.
How do teams identify failure modes?
You identify failure modes by analyzing your traces—reviewing the inputs your AI received and the outputs it generated. As you annotate these traces, you write notes about where the AI went wrong. Then you look across multiple traces to spot patterns in the errors.
These patterns become your failure mode categories. For example, you might notice that your interview coaching AI frequently suggests leading questions or general questions in its feedback. These become distinct failure modes because they represent recurring error types that need systematic measurement and improvement. One trace can have multiple failure modes.
Why do failure modes vary in severity?
Not all failure modes carry the same level of risk. Some errors are minor—slightly awkward phrasing or redundant information. Others are catastrophic failures that undermine your product's core value proposition.
For instance, if your interview coaching AI suggests leading questions—teaching students the exact opposite of what they should do—that's a critical failure mode. It's not just getting something wrong; it's actively teaching bad habits. Teams need to assess each failure mode's impact to prioritize which evals to build first and which errors demand immediate fixes.
Learn more:
- Behind the Scenes: Building the Product Talk Interview Coach
- How I Designed & Implemented Evals for Product Talk's Interview Coach
Related terms: