Three orchestration patterns that aren't 'just call the LLM in a loop'

There's a moment in every agent build where "I'll just call the model in a loop" stops working. The history bloats, one prompt is trying to be a researcher and a code reviewer and an email drafter at once, and quality slides. Most teams respond by reaching for a swarm of agents, and usually that's premature.

Anthropic's rule is the right one: start with a single agent and good tools, and add structure only when a simpler version demonstrably fails. A multi-agent design pays for itself in three situations: the subtasks are independent and can run in parallel, the work spans more context than one window holds, or you can't predict the shape of the subtasks until you see the input. If none of those is true, adding agents adds latency and makes failures harder to debug.

When one of them is true, I use one of three patterns. They're listed here from least to most rope to hang yourself with.

1. Routing to specialists

Routing is the pattern I use most. A main thread classifies the task and hands it to a specialist with a narrow job and its own model tier. The specialists don't talk to each other, and everything goes through the router.

I run about a dozen subagents, each one a single capability with a sharp description: a researcher that fans out across sources, a pr-preparer that reads a branch diff and writes the PR, a bug-investigator that traces execution paths and returns ranked fixes, a content-reviewer that checks a draft against brand rules and returns pass/fail. The router picks one by matching the task against each specialist's description, so the description has to be as specific as a skill's.

Routing also cuts cost. Short-output jobs go to a cheap model, and reasoning goes to a capable one. My asana-manager, dependency-auditor, and slack-drafter run on Haiku because they're mechanical. Anything that has to synthesize or touch code runs on Sonnet. Matching the model to the task is the cheapest improvement in agent design.

Reach for it when you have separable task types that a focused system handles better. What bites: misclassification sends work to the wrong specialist, and routes nobody uses go stale without anyone noticing. Keep the categories distinct and prune the ones nobody hits.

2. Parallel fan-out

When subtasks are independent, run them at once and merge the results. You can split a task into sections that run simultaneously, or run the same task several ways and reconcile the outputs (by voting, or with a guardrail model running alongside the main one).

My meeting-prep and researcher agents are fan-out underneath. meeting-prep hits attendee web profiles, past email and Slack threads, meeting notes, and internal docs in parallel, cross-references them for conflicts, and returns one brief. Run sequentially, it would take long enough that nobody would bother before a call.

Reach for it when latency is the bottleneck and the pieces don't depend on each other, or when independent perspectives raise confidence (review from several angles, moderation at several thresholds). What bites: the subtasks weren't actually independent and you get contradictions, the merge step can't reconcile them, and your token bill multiplies by the fan-out. Check that the subtasks share no state before you parallelize them.

3. Orchestrator-workers

This is the most capable pattern and the easiest to get wrong. An orchestrator decides the subtasks at runtime from the input, dispatches workers, and synthesizes what comes back. Unlike fan-out, nothing is defined in advance: the orchestrator decomposes the task as it goes. Anthropic's own multi-agent research system works this way, and Anthropic found it does well on breadth-first questions with independent branches and struggles when the subtasks need to share context.

Reach for it when you can't predict the number or nature of the subtasks, as with open-ended research or a code change whose scope you learn only by looking. What bites, hard:

• Telephone: detail degrades as everything routes through the orchestrator. Have workers write their full output to external storage and pass back references and compressed summaries. A worker can burn tens of thousands of tokens internally and still hand the orchestrator a tight 1–2k-token result.
• Effort calibration: without explicit guidance the orchestrator over-decomposes a simple query into ten workers or under-decomposes a hard one into none. Tell it how to scale.
• Cost and debuggability: this is the most expensive pattern and the hardest to trace when it goes wrong, so use it only after routing or fan-out has failed.

Rules that apply to all three

Keep state outside the context window by passing paths and IDs and pulling data in only when a step needs it. Have tools and workers return short summaries of what matters instead of everything they saw. Move to a more complex pattern only when the simpler one has demonstrably failed. A well-built single agent with the right tools beats a badly coordinated swarm on cost, on latency, and on your ability to figure out what it did.