Orchestration
What is orchestration?
Orchestration is designing a workflow of multiple LLM calls that work together. Instead of asking one prompt to do everything, you break a complex task into simpler steps that LLMs handle better and sequence those steps so each one gets only the context it needs.
LLMs are better at simpler tasks. If one prompt has to identify action items, summarize the meeting, categorize by urgency, and match to owners, quality goes down. Break it into steps, each focused on one thing, and quality goes up. That's an AI workflow: a series of AI tasks orchestrated to accomplish a larger task.
How do you orchestrate an AI workflow?
Map out each step and what it needs. For meeting notes, a first call identifies action items from the transcript. A second categorizes them by urgency and needs project context. A third matches them to owners and needs the team directory. With orchestration, you can add RAG steps to retrieve the right context and send each task only what it needs. For product managers, this is process design.
Teresa Torres's Interview Coach runs as an AWS Step Function that orchestrates four AI tasks, one per rubric dimension, a custom code task that aggregates the results, and an email step.
How does orchestration fit into evals and experiments?
Orchestration is one of the three kinds of change you test against a baseline. Every time you change a prompt, a model, or an orchestration pattern, you rerun your inputs and use your evals to score the result. Orchestration changes range from breaking a complex LLM call into a series of simpler calls to more advanced changes like adding agentic tool calling or audit loops. Your error rates tell you whether the new pattern helped.
Learn more:
- AI Changes Everything (And Nothing At All)
- Building My First AI Product: 6 Lessons from My 90-Day Deep Dive
Related terms:
Last Updated: September 1, 2026