For Windows and Azure teams, the important distinction is between a useful architecture and a proven return on investment. Several of the components discussed here are preview features, and several headline gains come from Microsoft’s own tests under defined conditions—not from independently replicated production deployments. Context engineering can reduce unnecessary token use and potentially improve an agent’s decisions, but it also introduces retrieval, security, governance, testing, and operating-cost questions that cannot be ignored.
Why context has become an enterprise-agent problem
An enterprise agent may need product documentation, SharePoint files, OneLake data, internal policies, customer history, and access to a wide collection of APIs. The naive design is to put as much as possible into the model’s prompt: a long block of retrieved documents, every available tool definition, and a large instruction set covering every workflow.
That approach has obvious cost implications because prompt tokens and repeated tool descriptions consume model input. But the quality problem matters just as much. Research on long-context models has found that performance can decline when relevant material is placed in the middle of a lengthy input. That does not prove that every shorter Foundry prompt will be better. It does establish that longer context is a trade-off, not a universal remedy for model errors.
Context engineering is the effort to make the available information more selective and task-specific. In practice, that means retrieving evidence rather than pasting whole repositories into a prompt; discovering an appropriate tool rather than declaring hundreds of tools at startup; reusing procedures where they apply; and keeping only durable information in persistent memory.
The approach is especially relevant to organizations building agents around Windows, Microsoft 365, Azure, Fabric, or line-of-business systems. Those environments often have both rich data estates and strict access boundaries. An agent that sees too little context can be unhelpful; one that sees too much may be expensive, confused, or over-privileged.
Foundry IQ: retrieval is not just a search box
Foundry IQ provides reusable, multi-source knowledge bases built on Azure AI Search’s agentic retrieval capabilities. Its retrieval design can plan and decompose a query, execute parallel searches, semantically rerank results, and return grounded content with citations. Importantly for enterprise deployment, it can enforce supported-source permissions using the caller’s Microsoft Entra identity.
That last property is more important than it may first appear. A knowledge system is not secure merely because its source repository is secure. The retrieval path also needs to honor who is asking. Teams should test whether a user who lacks access to a SharePoint document, for example, is unable to obtain its contents indirectly through the agent.
The documented knowledge-source scope should also be described precisely. Foundry IQ lists indexed sources including Azure Blob Storage, OneLake, SharePoint, and existing search indexes, alongside remote SharePoint and web sources. It should not be assumed that every Microsoft data product is a direct knowledge-base source. Fabric IQ, for example, is documented separately as a server-side tool connection for agents rather than as proof of a direct Foundry IQ knowledge-base integration. A broad claim covering Work IQ, Fabric IQ, web content, Blob Storage, SharePoint, OneLake, and Azure SQL in one directly connected knowledge base is not established by the available documentation.
Microsoft reports encouraging test results: replacing single-shot retrieval-augmented generation with a knowledge base improved evidence recall by up to 46%, while combining a smaller agent model with agentic retrieval improved recall by up to 54%. Microsoft also reported 34% retrieval-token savings from fewer retrieval-tool calls.
Those figures are useful design signals, but they are not a purchase-order forecast. They are Microsoft-reported measurements, not independently replicated customer savings. Retrieval quality will depend on content freshness, chunking and indexing choices, access-control configuration, query mix, evaluation method, model selection, and whether an organization’s documents contain clear answerable evidence in the first place.
There is also an economic wrinkle: Foundry IQ depends materially on Azure AI Search. Query-planning model use can be optional, but parallel subquery processing requires it. Azure AI Search consumption and Azure OpenAI usage can therefore both affect the total bill. Reducing tokens in one step does not guarantee lower end-to-end cost if it causes additional searches, planning calls, or downstream tool activity.
Toolboxes tackle a different kind of prompt bloat
Knowledge retrieval answers the question, “What information should the agent see?” Tool management answers, “What actions should the agent be able to take?” These are related problems, but they require different controls.
Foundry Toolboxes can consolidate tools behind a single endpoint compatible with the Model Context Protocol, or MCP. They provide central controls for authentication, policy, versioning, and changes that consuming agent code can pick up without a code change. That can simplify operations where separate teams own HR, IT service management, finance, customer support, and infrastructure tools.
Tool Search is the key context-saving feature. Rather than placing the entire tool catalog in the initial model context, it exposes two meta-tools: tool_search and call_tool. The agent can search for a relevant capability and then invoke the selected tool. Microsoft’s stated goal is to prevent a large catalog from crowding out the information needed to reason about the user’s actual request.
Microsoft reported that Tool Search reduced token use by more than 60% with 50 tools and by more than 97% with 1,000 tools. The latter result needs careful framing. It came from a ToolRet ablation against a baseline that exposed every tool up front. ToolRet itself addresses a real research challenge—finding useful tools among large catalogs where tasks may need multiple tools—but the more-than-97% number is not a universal average across enterprise deployments. It is also not evidence of independently verified customer cost reductions.
A tool-search layer can create new failure modes. Search metadata must be accurate enough that the agent can find the appropriate action. Tool names, descriptions, authorization requirements, and side-effect warnings become operationally significant data. A poorly described “close account” tool may be overlooked; a poorly constrained one may be selected when a read-only lookup was intended.
For Windows administrators, this argues for least privilege and a deliberate separation between read, write, and destructive actions. A catalog can be large without being broad in authority. Sensitive functions—such as changing identity settings, revoking access, deleting cloud resources, or modifying production configurations—should retain explicit authorization and confirmation controls outside the model’s natural-language reasoning.
Skills can standardize procedures, but they are not magic compression
Foundry Skills are centrally versioned reusable procedures that can be attached to a Toolbox and discovered by MCP-compatible clients. They offer a way to publish repeatable operating instructions, such as an approved escalation sequence or a policy-sensitive customer-support workflow, instead of embedding a separate copy of that procedure into every agent.
The implementation detail matters. In the documented preview model, MCP clients discover attached skills through resource listing and then read the skill content through a resource request. This supports a composable design, but it does not prove that every client or agent runtime automatically loads full instructions only after determining a skill is relevant. That behavior remains client- and runtime-dependent in the supplied evidence.
Organizations should therefore test actual prompt composition rather than assume that attaching a large skill library has no context cost. They should also apply conventional change management: version procedures, review them with the teams that own the policy, test them against realistic requests, and keep an audit trail of which version influenced an action.
Tool Search and toolbox-based Skill discovery are documented as preview capabilities. That makes controlled adoption especially important. A pilot should include fallback behavior if the service changes, an MCP client cannot interpret a resource as expected, or a version update modifies an established workflow.
Persistent memory is separate from conversation history
One of the easiest architectural mistakes is to treat conversation history and Foundry memory as the same service. They are not.
A Foundry conversation persists the history of turns. Foundry memory, by contrast, is a managed long-term memory feature designed to let an agent retain information across sessions. The preview feature supports persistent memory stores, per-user scoping and isolation, procedural-memory configuration, default time-to-live settings, and retention controls.
This distinction has direct privacy and compliance consequences. Conversation history may be appropriate for carrying a current support discussion from one turn to the next. Long-term memory needs a more explicit answer to harder questions: What is durable? Which user or tenant owns it? When does it expire? Can it be corrected or removed? Is a preference, a personal detail, or an inferred fact suitable for retention at all?
Microsoft reported approximately 5% improvement on STATE-Bench and τ-bench when procedural memory was enabled. This is a meaningful indicator that stored procedures can help consistency, but it is not a blanket claim of 5% better performance in production. STATE-Bench contains 450 tasks in synthetic travel, customer-support, and shopping domains, and its creators caution that its LLM-generated datasets are for research use. τ-bench evaluates dynamic interactions with domain-specific APIs and policy guidelines, which is valuable but still distinct from a company’s production systems and users.
The practical message is to treat procedural memory as governed operational knowledge, not as an opaque store of everything an agent has ever observed.
Agent Optimizer requires a human-controlled release process
Foundry Agent Optimizer can evaluate a baseline agent and generate candidate instructions, Skills, tool descriptions, and model-selection configurations. It then evaluates and ranks candidates using a selected dataset. Microsoft describes it as a preview capability, and the customer must select and apply the chosen candidate before redeploying a hosted agent.
That is optimization assistance, not an unattended self-improving production loop. It is a valuable distinction for change control. An automatically generated prompt or tool description can improve a benchmark while creating unsafe behavior on an unrepresented task, weakening an important policy constraint, or selecting a costlier model configuration.
Evaluation itself can have consequences. During optimization, external APIs, databases, and third-party tools may execute repeatedly. Without test endpoints, mocks, or carefully limited accounts, that can create charges, trigger state changes, consume quotas, or hit rate limits. A help-desk test might create real tickets; a commerce test might place or modify orders; an infrastructure test might alter resources.
A responsible release path is therefore straightforward: use representative evaluation cases, isolate tools where possible, review candidate changes, test them in a controlled environment, and monitor post-deployment results. Evaluation datasets should include permission boundaries, ambiguous requests, stale information, failed tool calls, and situations where the safest answer is to ask for confirmation or decline an action.
What a measured rollout looks like
The strongest case for context engineering is not “fewer tokens at any cost.” It is a disciplined effort to make agents more reliable, economical, and governable as their information and action surfaces grow.
Start by measuring a baseline: prompt-token volume, retrieval calls, tool-selection success, task completion, latency, user corrections, access-control failures, and total cost including search and tool execution. Then change one layer at a time. Introduce a knowledge base for a bounded document set; test Tool Search on a catalog where tool descriptions have been reviewed; apply durable-memory retention rules before enabling persistence; and treat optimizer output as a proposed change requiring approval.
For enterprise Windows environments, the most concrete payoff may be architectural rather than headline benchmark gains. Centralized tool authentication and versioning can reduce integration sprawl. Entra-aware retrieval can help preserve existing document permissions. Versioned procedures can make agent behavior more consistent. But all three benefits depend on careful identity design, content governance, test coverage, and operational ownership.
Microsoft’s context-engineering components point toward a practical future for enterprise agents: smaller, better-targeted working contexts rather than ever-larger prompts. The evidence supports that direction. It does not yet justify assuming universal savings, automatic quality gains, or fully autonomous optimization—especially while several of the most consequential features remain in preview.