The agent helps researchers explore questions beyond a trial’s predefined protocol. It examines data, generates and executes analysis code, revises its approach, and returns summaries and visualizations. Its previous production stack combined Azure Container Apps, PostgreSQL conversation storage, and Container Apps dynamic sessions for sandboxed execution.
What Foundry takes over—and what it does not
Microsoft Learn describes hosted agents as customer-controlled code packaged into container images. Developers retain their framework, orchestration, tools, and runtime behavior; Foundry supplies the surrounding endpoint, agent identity, session infrastructure, scaling, and lifecycle management. Each session receives a dedicated VM-isolated sandbox.
That changes the unit of infrastructure planning. Instead of configuring a replica count, teams allocate resources for individual sessions, with compute provisioned as sessions become active. Microsoft documents a configurable idle timeout of 2–240 minutes, defaulting to 15 minutes. Compute is deprovisioned after that timeout, while persisted session state can be restored; sessions are permanently deleted after 30 days of inactivity. These are service parameters, not measurements from Novo’s deployment.
Managed scaling also does not mean cost planning disappears. CPU and memory allocations apply per session, and runtime billing reflects resources consumed across active sessions. Oversizing an individual sandbox therefore becomes more expensive as concurrency grows.
One important implementation distinction deserves attention: session continuity is not conversation continuity. Microsoft’s session-management documentation separates the sandbox and filesystem from message history:
- With the Responses protocol, conversation threading uses a conversation identifier or
previous_response_id. - With Invocations, application code manages conversation history.
- Reusing
agent_session_idalone does not replay earlier messages to the model.
For a migration, the practical implication is straightforward: test both whether the agent remembers the discussion and whether it can still access the files produced earlier. Those are separate acceptance checks, not interchangeable signs of success.
Private networking needs two decisions
Microsoft’s networking documentation adds a crucial qualification to the migration story: a private endpoint alone does not isolate outbound traffic. Under the public-egress model, it restricts inbound access while the agent’s outbound connections remain public. Foundry also offers customer-provided and Microsoft-managed virtual-network models.
For organizations bringing their own virtual network, subnet planning belongs before account creation. Microsoft says network injection cannot be added to an existing account, and the assigned subnet cannot subsequently be resized. All projects in the account share its capacity.
The documented sizing guidance is:
| Subnet size | Planning meaning |
|---|---|
| /27 | Supported minimum; workload capacity and upgrade headroom still need validation |
| /24 or larger | Recommended production starting point where combined project demand or upgrades require substantial address capacity |
A larger subnet does not raise the separate session quota. Project networking components and upgrade infrastructure also consume addresses, so subnet size is not simply a researcher headcount translated into IP addresses.
The lesson: “private” is an architecture, not a checkbox.
Better traces can create another sensitive data store
Novo’s team reports retaining both Langfuse and Application Insights tracing during the transition. It also used Azure Verified Modules for repeatable infrastructure, while warning that generated dependencies and database throughput still required review.
Foundry tracing uses OpenTelemetry and stores traces in connected Azure Monitor Application Insights resources. Microsoft documents that captured information can include prompts, model outputs, tool activity, token usage, latency, and errors. In a clinical-data workflow, that makes telemetry part of the data-protection design—not merely an engineering convenience.
Microsoft’s data-handling guidance says tracing is enabled by connecting an Application Insights resource. Access and retention depend on the connected monitoring resources; disconnecting tracing stops new collection but does not erase previously retained records. The guidance calls for minimizing personal data, excluding secrets, and controlling access.
For troubleshooting, Microsoft recommends consistent trace attributes and correlation with evaluation-run identifiers. If traces are missing, the documented checks include the Application Insights connection, telemetry-query permissions, and agent instrumentation. A dashboard cannot diagnose an interaction it never received.
Keep the quality benchmark steady
Novo preserved backend interfaces so its existing evaluation framework could exercise the hosted implementation. The team continued using the same evaluation set and graders, including expert-provided reference answers. Microsoft’s account does not publish comparative scores, costs, latency figures, or clinical outcomes.
The strongest transferable lesson is therefore methodological: change the runtime without simultaneously changing the yardstick. Keeping evaluation inputs and grading stable makes a migration easier to assess; traces can then help explain individual failures rather than substitute for outcome testing.
Novo’s experience supports a narrower, more credible conclusion than “managed agents solve production.” Hosted infrastructure changes who operates the machinery. Teams still need to establish that the agent preserves its state correctly, reaches only approved systems, protects its telemetry, and produces acceptable answers. Microsoft’s own hosted-agent documentation explicitly leaves application testing, data-flow decisions, and appropriate security and quality controls with the customer.