Three orchestration patterns that aren't 'just call the LLM in a loop'

There’s a moment in every agent build where “I’ll just call the model in a loop” stops working. The history bloats, one prompt is trying to be a researcher and a code reviewer and an email drafter at once, and quality slides. The instinct is to reach for a swarm of agents. Usually that’s premature.

Anthropic’s rule is the right one: start with a single agent and good tools, and add structure only when a simpler version demonstrably fails. Multi-agent earns its cost in three situations — the subtasks are independent and can run in parallel, the work spans more context than one window holds, or you can’t predict the shape of the subtasks until you see the input. If none of those is true, more agents just buys you latency and a harder thing to debug.

When one of them is true, here are the three patterns I actually reach for, in increasing order of how much rope they give you to hang yourself.

1. Routing to specialists

The workhorse. A main thread classifies the task and hands it to a specialist with a tight job and its own model tier. The specialists don’t talk to each other; everything goes through the router.

This is most of my setup. I run about a dozen subagents, each one a single capability with a sharp description: a researcher that fans out across sources, a pr-preparer that reads a branch diff and writes the PR, a bug-investigator that traces execution paths and returns ranked fixes, a content-reviewer that checks a draft against brand rules and returns pass/fail. The router picks one by matching the task to the specialist’s description — which is exactly why the description has to be specific, the same way a skill’s does.

The other lever routing buys you is cost. Short-output jobs go to a cheap model; reasoning goes to a capable one. My asana-manager, dependency-auditor, and slack-drafter run on Haiku because they’re mechanical. Anything that has to synthesize or touch code runs on Sonnet. Matching model to task is the cheapest win in agent design.

Reach for it when you have separable task types that are each handled better by a focused system. What bites: misclassification sends work to the wrong specialist, and rarely-used routes quietly rot. Keep the categories crisp and prune the ones nobody hits.

2. Parallel fan-out

When subtasks are genuinely independent, run them at once and merge the results. Two flavors: split a task into sections that run simultaneously, or run the same task several ways and reconcile (voting, or a guardrail model running alongside the main one).

My meeting-prep and researcher agents are fan-out underneath. meeting-prep hits attendee web profiles, past email and Slack threads, meeting notes, and internal docs in parallel, cross-references them for conflicts, and returns one brief. Done sequentially it would be slow enough that nobody would run it before a call. The parallelism is the reason it’s useful.

Reach for it when latency is the bottleneck and the pieces don’t depend on each other, or when independent perspectives raise confidence (review from several angles, moderation at several thresholds). What bites: the subtasks weren’t actually independent and you get contradictions, the merge step can’t reconcile them, and your token bill multiplies by the fan-out. Only parallelize what’s truly parallel.

3. Orchestrator-workers

The powerful, dangerous one. An orchestrator decides the subtasks at runtime from the input, dispatches workers, and synthesizes what comes back. The difference from fan-out is that nothing is pre-defined — the orchestrator decomposes on the fly. Anthropic’s own multi-agent research system works this way, and they found it shines on breadth-first questions with independent branches and struggles when the subtasks need to share context.

Reach for it when you genuinely can’t predict the number or nature of the subtasks — open-ended research, a code change whose scope you only learn by looking. What bites, hard:

• Telephone. Detail degrades as everything routes through the orchestrator. The fix is to have workers persist their full output to external storage and pass back references and compressed summaries, not dumps. A worker can burn tens of thousands of tokens internally and still hand the orchestrator a tight 1–2k-token result.
• Effort calibration. Without explicit guidance the orchestrator over-decomposes a simple query into ten workers or under-decomposes a hard one into none. Tell it how to scale.
• Cost and debuggability. This is the most expensive pattern and the hardest to trace when it goes wrong. Earn it.

The through-line

State lives outside the context window, not inside it — pass paths and IDs, pull data in only when needed. Tools and workers return high-signal summaries, not everything they saw. And you don’t climb this ladder for status; you climb it one rung at a time, only when the rung below it stops holding. A well-built single agent with the right tools beats a badly-coordinated swarm on almost everything that matters: cost, latency, and your ability to figure out what it did.