Chapter 5: Agent Frameworks & Multi-Agent Orchestration

In Chapter 3, you built an agent by writing the reasoning-action loop yourself. You manually managed the conversation history, parsed tool-call responses, dispatched to handler functions, appended results back to the message list, and called the model again. That manual approach teaches you how agents work at a fundamental level, but it also means you are responsible for every detail: retry logic, error handling, message formatting, token management, and execution-flow control.

The Microsoft Agent Framework provides higher-level abstractions that handle these operational details for you. Instead of writing the execution loop, you declare your agent’s instructions and tools, and the framework manages the rest. Think of it as the difference between writing raw HTTP requests by hand and using a web framework — the underlying mechanics are the same, but the framework absorbs the boilerplate so you can focus on behavior.

The Agent Framework organizes development around a small set of core ideas.

A single-agent abstraction. The framework gives you one consistent way to define an agent — a model, instructions (the system prompt), and a set of tools — regardless of whether that agent runs alone or as one member of a larger team. You construct the agent declaratively, and the framework handles instantiation, the reasoning-action loop, and the lifecycle.

Middleware. Just as a web framework lets you insert middleware into the request pipeline, the Agent Framework lets you insert logic around agent runs and tool calls — for logging, input or output validation, rate limiting, redaction, or policy enforcement. This is where cross-cutting concerns live, instead of being scattered through a hand-written loop.

Memory and context providers. The framework manages conversation history (often called a thread) so multi-turn conversations persist without you tracking the message list by hand. Pluggable context providers let an agent pull in additional grounding — for example, a Foundry IQ knowledge base from Chapter 4 — at the right point in the run.

Telemetry. The framework emits structured telemetry (traces, token usage, run status) so you can observe what an agent actually did, which is far harder when you instrument a hand-coded loop yourself.

Graph-based workflows. Beyond a single agent, the framework models multi-step processes as graphs of steps — the foundation for the orchestration patterns in Section 5.3.

A defining feature of the Agent Framework is that it supports two distinct styles of coordinating work, and choosing between them is one of the most important design decisions you will make.

Agent orchestration is LLM-driven. The model reasons about what to do next, which specialist to involve, and when the task is complete. Control flow emerges from the model’s reasoning rather than from code you wrote in advance. This style is flexible and adaptive — it handles open-ended requests and unforeseen situations well — but it is also less predictable, harder to test exhaustively, and can consume more tokens because the model is doing the routing.

Workflow orchestration is deterministic. You define the steps and the transitions between them as an explicit graph driven by business logic, and the framework executes that graph the same way every time. Agents can still be steps within the workflow, but the control flow is fixed by your design rather than decided by a model at runtime. This style is predictable, testable, and auditable — the right choice when a process must follow a known sequence (a regulated approval chain, a fixed data-processing pipeline) — at the cost of flexibility.

In practice you mix the two. A deterministic workflow might invoke an agent-orchestrated sub-step for the genuinely open-ended part of a task, while keeping the regulated stages fixed. The skill is matching the style to the requirement: reach for agent orchestration where adaptability matters, and workflow orchestration where predictability and auditability matter.

Building an agent with the framework follows a declarative pattern: define the agent’s model, instructions, and tools; start a conversation; send a message; and read the response. The framework runs the reasoning-action loop internally.

The sample below illustrates the shape of that pattern — defining tools, constructing an agent, and running it — using the financial-research agent from Chapter 3.

Compare this with the Chapter 3 implementation. There, you wrote a while loop that checked finish_reason, parsed tool_calls, dispatched to handler functions, appended tool results to the message list, and called the model again. Here, a single run call does all of that internally. You define the tools, send a message, and get a response.

The framework is the right choice when you want to move quickly and your requirements align with its built-in capabilities — an agent that calls tools, manages conversation threads, and emits telemetry, especially one that will grow into a multi-agent solution. The orchestration patterns in Section 5.3 are part of the framework, so starting there positions you to scale from one agent to a team without rewriting your foundation.

Direct API access (the hand-coded approach from Chapter 3) is preferable when you need fine-grained control the framework abstracts away, when you must minimize dependencies for a lightweight service, or when you are learning — you cannot truly appreciate what the framework does for you until you have built the loop yourself. In practice, many production systems mix both: the framework for the primary agents and orchestration, direct API calls for a specialized component that needs custom behavior.

When choosing for production, weigh observability (the framework’s built-in telemetry vs. logging you control), versioning (the framework evolves on its own release cycle, separate from the underlying model API), and testing (deterministic workflow orchestration is far easier to test than open-ended agent orchestration).

The practical trigger for going multi-agent is not a fixed number of tools. It is the emergence of distinct domains of expertise that need different instructions, different tool sets, and different behavioral constraints. When you find yourself writing a system prompt that says "If the user asks about X, do this; if they ask about Y, do that; if about Z, do something else entirely," you are encoding routing logic into prose. A multi-agent architecture makes that routing explicit.

A single agent with fifteen tools faces a combinatorial problem: it must pick the right one from fifteen options and understand the nuances of when each applies. Tool-selection accuracy degrades as the number of available tools grows. Multi-agent systems solve this by giving each agent a small, focused toolkit where selection is straightforward.

Agent specialization means breaking a complex workflow into focused roles, each handled by a dedicated agent with its own instructions and tool set. For the financial research assistant from Section 5.1, this might look like:

Each agent is an expert in its domain. Its instructions are concise, its tool set small and relevant, its constraints specific to its role. This specialization leads to better tool selection, more coherent reasoning, and more predictable behavior. The Agent Framework’s single-agent abstraction (Section 5.2.2) is exactly what you use to define each specialist; an orchestration pattern then composes them.

The Agent Framework ships a set of named orchestration patterns for composing specialized agents. Rather than hand-coding coordination as you would with the raw API, you choose a pattern that matches the shape of your task. The figure below shows a central orchestrator routing work to specialists — the mental model that underlies several of these patterns.

Sequential orchestration. Agents run one after another, each agent’s output becoming the next agent’s input. This fits tasks with natural ordered stages — for example, a Data Gathering Agent retrieves raw market data and filings, an Analysis Agent turns that into insights and comparisons, and a Report Agent formats the result into a document. Sequential pipelines are simple to implement and easy to debug because you can inspect each hand-off, but they are inherently serial: every stage must finish before the next begins, which increases end-to-end latency.

Concurrent orchestration. When subtasks are independent, multiple agents run simultaneously and their results are aggregated. Retrieving stock-performance data and searching for 8-K filings are independent, so running them concurrently reduces total latency to the duration of the slowest branch rather than the sum of all branches. The challenge is result aggregation: agents finish at different times and produce different formats, so the orchestration must reconcile their outputs into a coherent response — and decide what to do when one branch fails while others succeed (return partial results, retry, or report failure).

Handoff orchestration. One agent transfers control to another agent better suited to the current request, passing along the relevant context. Handoff is dynamic and conversational: a triage agent might handle a general inquiry and hand off to a specialist the moment the conversation turns to its domain, and the specialist can hand back or onward as needed. This pattern shines for customer-facing flows where the right expert is not known up front and changes as the dialogue unfolds. Designing handoffs means deciding what context travels with the handoff so the receiving agent is not starting blind (see Section 5.3.5).

Group chat orchestration. Several agents participate in a shared conversation, coordinated by a manager that decides who speaks next. Group chat fits tasks that benefit from multiple perspectives interacting — a writer agent and a reviewer agent iterating on a draft, or several domain experts debating a recommendation before converging. The manager enforces turn-taking and decides when the conversation has reached a conclusion. Group chat is powerful but the most token-intensive pattern, since multiple agents repeatedly read and contribute to a growing shared transcript.

Magentic-One orchestration. Magentic-One is a flexible, general-purpose pattern (originating in AutoGen research) in which a manager agent plans, delegates subtasks to specialist agents, tracks progress against a task ledger, and re-plans when steps fail or new information arrives. It is designed for open-ended, multi-step problems where the full plan cannot be specified in advance — the manager reasons about the goal, dispatches work, observes results, and adapts. Magentic-One is the most adaptive pattern and squarely an agent orchestration (LLM-driven) approach, trading predictability for the ability to tackle genuinely novel tasks.

The figure below contrasts how work flows through several of these patterns.

Most real-world systems combine patterns. The financial research assistant might run the Market Data Agent and Document Research Agent concurrently to gather data, then sequentially pass everything to the Quantitative Analysis Agent and finally the Report Generation Agent. Tying back to Section 5.2.3: the open-ended, "figure out the plan" pieces lean toward agent-orchestrated patterns (handoff, group chat, Magentic-One), while fixed, must-run-in-order pieces lean toward deterministic workflow orchestration with sequential steps.

A critical design decision in any multi-agent system is how context flows between agents. Each agent operates with its own conversation history, so you must explicitly decide what to pass from one to the next.

Full context passing forwards the entire conversation history. This gives each agent maximum information but consumes context-window space rapidly and can confuse an agent with irrelevant details from another specialist’s domain.

Summary context passing has the orchestration summarize one agent’s results before passing them on. This is more token-efficient and keeps each agent focused, but the summarization step can lose important details.

Structured context passing defines a fixed data format (such as a JSON object) that agents use to communicate. Each agent reads specific fields and writes its results to designated output fields. This is the most predictable and easiest to debug, though it requires upfront design to define the shared schema. For the financial research assistant, structured passing works well because financial data has natural structure — ticker symbols, price arrays, filing metadata — that maps cleanly to JSON fields.

Multi-agent systems add failure modes beyond those of a single agent. Knowing them helps you design orchestration that degrades gracefully.

Routing and handoff errors. The orchestration sends a subtask to the wrong specialist — a regulatory-filing question reaches the Market Data Agent instead of the Document Research Agent — or a handoff transfers control without the context the receiving agent needs.

Lost context. Information from one agent never reaches another that needs it. The Market Data Agent retrieves prices, but a context-passing bug means the Quantitative Analysis Agent never receives them and either hallucinates values or errors out.

Partial failures. In concurrent orchestration, some branches succeed and others fail. The orchestration must decide whether to return partial results, retry the failed branch, or report the whole request as failed.

Inconsistent results. Two agents produce contradictory information — one reports a stock price while another finds a filing indicating a stock split that should have adjusted it. Without a reconciliation step, contradictory data reaches the user.

The mitigations mirror single-agent error handling, extended to coordination: retry transient failures with exponential backoff; provide fallback paths so the system still delivers partial value when one agent is down (graceful degradation); validate each agent’s outputs before combining them; and define explicit escalation conditions where the system stops and hands off to a human rather than fabricating an answer. The decision tree below summarizes how to route different failure types.

The course capstone is an agentic multimodal build: an agent that combines the capabilities you have assembled across the agent arc — knowledge from a Foundry IQ knowledge base (Chapter 4), tools called directly or through MCP (Chapters 3 and 4), and multimodal inputs processed with Azure AI Content Understanding (which you will study later in the course). The orchestration and framework skills from this chapter are what let you assemble those pieces into a coherent, maintainable system — as a single capable agent or, where the work genuinely splits across domains, a small team of specialists.

You scope the proposal here, in this chapter. The full design document and a recorded walkthrough are produced later, in Chapter 8, once you have studied the remaining services your capstone may draw on. Scoping early means you can let your capstone idea mature alongside the rest of the course.

The first step is determining whether an agentic solution is the right approach. AI agents excel at workflows that interleave reasoning with action and knowledge: pulling information from documents and systems, deciding what to do next, calling tools, and synthesizing a grounded answer. They struggle with tasks that demand perfect accuracy with zero tolerance for error, tasks with insufficient or poor-quality data, and tasks where the cost of the AI solution exceeds the cost of the manual process it replaces.

When scoping, ask three questions. First, what is the manual process today, and where are the bottlenecks? Second, which bottlenecks involve work an agent can do well — reasoning over knowledge, calling tools, interpreting multimodal inputs? Third, what is the cost of the agent making a mistake, and is that cost acceptable?

Every solution exists to serve users. Before choosing services or architecture, define who your users are, what they need, and how they will interact with the agent.

A solution for expert users (claims adjusters processing hundreds of claims a week) looks different from one for occasional users (a customer checking a claim once a month). Experts want speed, shortcuts, and detail; occasional users want simplicity, guidance, and reassurance. Both need accuracy, but they define it differently — the adjuster wants precise policy-language citations; the customer wants a clear, jargon-free status update. Define your target users by role, frequency of use, technical proficiency, and primary pain points, then design the agent’s interaction around their workflow rather than asking them to adapt to the agent.

A strong agentic capstone draws deliberately on the three capability areas of the agent arc. For each one, name a specific need rather than reaching for a service because it is available.

Whether the result is one agent or a small multi-agent team is a design choice you justify using Section 5.3: stay with a single agent unless distinct domains of expertise warrant specialists coordinated by an orchestration pattern.

Before committing, assess feasibility across four dimensions. Technical feasibility: can the chosen capabilities actually perform the tasks at the required accuracy? Run a small proof of concept. Data feasibility: do you have access to the knowledge and inputs the agent needs, in a usable format, within privacy and compliance constraints? Operational feasibility: can you deploy, monitor, and maintain the solution with the subscriptions, quotas, and expertise available? Economic feasibility: does the per-transaction cost justify the value relative to the manual process?

In Lab 10, you build a chat agent using the Microsoft Agent Framework SDK, replacing the hand-coded loop from your earlier agent work. As you work through the lab, focus on:

In Lab 11, you compose several specialized agents into a multi-agent solution using the framework’s orchestration patterns. Focus on:

The capstone proposal is a one-page document (approximately 400-600 words) in which you propose an agentic multimodal solution for a real-world problem. The proposal has four required components.

Problem domain. Define the problem your agent addresses. What is the current pain point? Who experiences it? Why does the current process need improvement? Be specific — "customer service is slow" is a symptom, not a problem definition. Dig deeper: what makes it slow, where do delays occur, and what information is hard to find?

Target users. Identify who will use your solution — their role, technical proficiency, and primary workflow. Will the agent replace a step, augment a step, or add a new capability? Revisit Section 5.4.3 on user-centered design.

Capabilities: knowledge, tools, and multimodal inputs. Drawing on Section 5.4.4, name the knowledge sources (Foundry IQ), tools (direct or MCP), and multimodal inputs (Content Understanding) your agent needs, and justify each against a specific requirement. State whether your design is a single agent or a multi-agent team, and if multi-agent, which orchestration pattern coordinates it (Section 5.3).

Initial architecture sketch. Briefly describe (or diagram) how the pieces connect: which capability handles which part of the workflow, how data flows, and where the user interacts with the agent. This is a conceptual architecture, not a detailed implementation plan.

As you develop your proposal, revisit the feasibility criteria from Section 5.4.5. Can the capabilities you selected actually do what you need? Do you have a realistic sense of the data requirements? Is the scope achievable? Remember that the full design document and recorded walkthrough come later, in Chapter 8 — the goal here is a focused, defensible scope.

You will review two classmates' proposals and provide constructive feedback. Consider: Is the problem well-defined or too vague? Are the target users clearly identified? Do the chosen capabilities match the stated requirements, or are there gaps? Is the single-vs-multi-agent decision justified, and does the orchestration choice make sense? Is the scope realistic?

Effective feedback is specific and actionable. Instead of "this is good," explain why it works: "Grounding the agent in a Foundry IQ knowledge base of policy documents is strong because the proposal requires cited answers from a fixed document corpus." Instead of "this needs work," explain what to improve: "The proposal lists Content Understanding but never says what images or documents the agent would interpret — naming a specific multimodal input would strengthen this section."