Multi-Agent Orchestration in Production: Routing, Guardrails, and Escalation

It is easy to build a demo where one large model, handed a big prompt and a pile of tools, does something impressive. It is much harder to build a system that does the right thing on the ten-thousandth request, when the input is adversarial, the tools have side effects, and a wrong action costs money or trust. The gap between those two is architecture, specifically the decision to stop treating "the agent" as a single monolithic model and start treating it as an orchestrated system of specialized agents with guardrails between them.

This paper collects patterns we've used across two production multi-agent platforms, one automating guest and staff workflows for hospitality, one running e-commerce operations across marketplaces. Both converged on the same structure: a cheap classifier that routes to specialized agents, sticky sessions that keep conversations coherent, bounded tool loops, tool policy scoped by agent and by caller trust, confidence-driven escalation to humans, and hard budgets. The specifics differ; the shape does not.

One model is not an architecture

A single agent with every tool attached has two problems that only appear at scale. The first is prompt dilution: the more responsibilities and tools you cram into one system prompt, the worse the model gets at each of them. An agent that must be a concierge, an accountant, and a maintenance dispatcher simultaneously does all three poorly. The second is blast radius: if every tool is reachable from every turn, then every turn can do anything, and your worst-case action is the union of all tools, a bad place to be when some of those tools move money or delete data.

Specialization fixes both. In the hospitality platform, eight domain agents (concierge, operations, spa, food and beverage, housekeeping, revenue, maintenance, events) each carry a focused prompt and a bounded set of tools. The e-commerce platform splits similarly: an orchestrator plus specialists for data analysis, product creation, supplier research, order placing, and platform sync. Each agent is good at one job because it is only asked to do one job, and each agent can only touch the tools its job requires.

Classify, then route

If you have specialized agents, you need something to decide which one handles a given request, and that something should be cheap. Running your most expensive model just to categorize an incoming message is a waste on every single request. Both platforms front the system with a lightweight classifier: a small, fast model whose only job is to read the incoming message and name the intent, so the request can be handed to the right specialist.

This two-stage shape (cheap classification, then expensive execution only where it's warranted) is one of the highest-leverage decisions in a production agent system. It keeps the per-request cost proportional to the work actually required, and it makes the routing logic inspectable: you can log what the classifier decided and why, measure its accuracy, and correct it, in a way you cannot when routing is an emergent property of one giant prompt.

Sticky routing keeps conversations coherent

Naive per-message routing has a failure mode that users notice immediately: mid-conversation, a follow-up message gets classified into a different agent, and the thread loses its thread. The user asked the spa agent about availability, then said "yes, book the 3pm", and that terse confirmation, routed fresh, lands nowhere useful.

The fix is sticky routing: once a session is bound to an agent, subsequent messages stay with that agent unless there's a clear reason to switch. Continuity becomes the default, and re-classification the exception. This mirrors how a human handoff works (you don't get transferred to a new department because you said "yes"), and it is the difference between a system that feels like one coherent assistant and one that feels like a switchboard dropping your calls.

Bounded tool loops

Give an agent tools and a goal and it will loop: call a tool, read the result, decide the next call. That loop is where the real work happens, and it is also where runaway behavior lives. An agent that misreads a result can call the same tool forever; an agent chasing a goal it can't reach can burn through tokens and API calls without ever stopping.

The guardrail is a hard cap on iterations per run. The agent gets a bounded number of tool-use cycles to accomplish the task; if it hasn't converged by then, the loop ends deterministically rather than spinning. This is unglamorous and essential. It bounds cost, it bounds latency, and it converts "the agent hung" (an incident) into "the agent hit its limit and returned" (a handled case you can log, measure, and tune). A production tool loop without a ceiling is an outage waiting for the wrong input.

Tool policy per agent and per trust level

Which tools an agent may call is not one decision but two. The first is per agent: the concierge agent has the concierge's tools, not the revenue agent's. The second, easy to miss, is per caller trust level. The same platform serves a guest sending a WhatsApp message and a staff member in an operations console, and those two callers must not have the same capabilities even when they reach the same agent.

So trust becomes a filter on the toolset. In the hospitality platform, guest-versus-staff trust levels determine which tools are even visible before an agent runs; a per-tenant write gate governs whether write actions are permitted at all; and high-risk tools require a step-up confirmation. The result is a matrix, not a switch: capabilities are the intersection of what this agent does and what this caller is trusted to do, evaluated deterministically around every call. Guest personal data is scrubbed as it flows through, because the safest way to handle sensitive input is to not carry it further than necessary.

Confidence and human escalation

Not every request should be handled autonomously, and a mature system knows the difference. The pattern that works is to have agents report a confidence signal and to route low-confidence outcomes to a human rather than guessing. When an agent isn't sure (ambiguous request, missing data, an action beyond its remit), it escalates instead of improvising.

This is what makes autonomy acceptable in domains where mistakes are expensive. Full automation on the confident majority, human review on the uncertain tail, and a clear seam between them. It also produces a virtuous loop: every escalation is a labeled example of where the system's confidence and its competence diverge, which is exactly the data you need to improve routing, prompts, and tool coverage. A system that escalates well gets better; a system that never escalates just fails silently.

Budgets and background agents

Everything above concerns agents that respond to a request. Production systems also run agents that fire on their own: monitoring freshness, checking data quality, watching costs, raising alerts on cadences. These background agents are where cost quietly compounds, because nobody is watching a screen when they run.

The discipline is an explicit budget. Bound the tokens and tool calls any single run may consume, and enforce that ceiling in the runner so an agent that would otherwise spiral is stopped and its partial work handled. In the data platform described in our companion papers, a run-budget enforcer wraps every background agent for exactly this reason. Budgets do for cost what bounded loops do for latency: they turn an open-ended risk into a known, capped quantity you can plan around and alert on.

What production actually demands

Step back and the through-line is clear. Every pattern here is a way of making a non-deterministic system behave predictably at its edges: routing that is cheap and inspectable, sessions that stay coherent, loops that end, tools that are scoped by both role and trust, uncertainty that escalates to people, and cost that is bounded by design.

None of it is exotic. It is the same discipline any reliable distributed system demands (least privilege, bounded resources, graceful degradation, observability) applied to a new kind of component whose defining trait is that it is steerable by its input. The models will keep getting better. The orchestration around them is what determines whether "better" shows up as a more capable product or a larger blast radius. Build the guardrails first, and the intelligence becomes an asset you can actually deploy.