AI Interaction
What is an AI interaction?
An AI interaction is an exchange between a user and an AI system, captured in a trace that includes the user input, system prompts, tool calls, intermediate steps, and final LLM responses. In multi-turn conversations, it encompasses all of the back-and-forth between the user and the LLM. These interactions form the basis for evaluating and improving AI system performance.
What components make up an AI interaction?
An AI interaction consists of several key elements that work together to produce the final response:
- User input — The initial query or request from the user
- System prompts — Instructions that guide how the LLM should respond
- Tool calls — Any external tools or functions the AI system invokes during processing
- Intermediate steps — The reasoning or processing steps the system takes
- Final LLM responses — The output delivered back to the user
For example, in an Interview Coach system, a single interaction includes the system prompts that tell the LLM how to evaluate interviews, the interview transcript itself, and the coaching feedback generated in response.
Why are AI interactions recorded?
AI interactions are captured as traces primarily for evaluation and improvement purposes. By recording the complete interaction—not just the final output—teams can diagnose issues, understand where the system succeeded or failed, and make targeted improvements to prompts, tools, or overall system design. This detailed record becomes essential for running evals and maintaining AI feature quality over time.
Learn more:
- Building My First AI Product: 6 Lessons from My 90-Day Deep Dive
- How I Designed & Implemented Evals for Product Talk's Interview Coach
Related terms: