The project was detailed in a Nature paper published September 16 and reported separately by Nature and The Register. Paper2Agent takes a manuscript and its associated research outputs, finds and configures the underlying code, converts selected functions into Model Context Protocol tools, and tests them against the work’s reported outputs. A compatible client such as Claude Code, Codex or Gemini CLI can then connect to the generated server and invoke those methods conversationally.
The important qualification is buried beneath the “turn any paper into an agent” framing: Paper2Agent does not make a prose-only paper reproducible. It relies on accessible source code, datasets or examples, working dependencies, and enough documentation to reconstruct the environment. Its own large-scale evaluation found that only 74 of 100 sampled computational-biology papers could be converted successfully without human intervention. That 26 percent failure rate is less a knock on the tool than a useful measurement of the software debt that still sits behind much published computational research.
An MCP server, not a smarter PDF reader
A conventional AI assistant given a paper and Git repository has to infer how the project fits together every time a user asks for an analysis. Paper2Agent’s approach is to do the setup work once: identify the project’s core workflows, wrap them as MCP tools, test them, then expose the validated functions through a standardized interface.
That makes a material difference to how an agent behaves. A model can summarize a paper from text alone, but it cannot demonstrate that it knows the paper’s analysis pipeline unless it can execute that pipeline against supplied inputs. The Paper2Agent authors describe the resulting system as a “virtual corresponding author,” but the implementation is closer to an agent-ready software package containing documentation, workflows, data resources and callable code.
The distinction matters for anybody who has tried to reproduce an academic repository on Windows, Linux or a managed research workstation. The obstacle is rarely understanding the abstract. It is resolving a dependency frozen years ago, locating an undocumented input format, finding the notebook cell that establishes the actual order of operations, or discovering that a tutorial relies on a file missing from the repository.
Paper2Agent uses specialist sub-agents for environment setup, extraction and test validation. The resulting MCP server can include tools for executing the paper’s method, resources for accessing the manuscript and supplementary material, and prompts that describe multi-step workflows. In principle, this lets a scientist request an analysis without manually wiring together Python packages, scripts and parameters.
For Windows users, that does not eliminate the platform problem; it moves it into a generated project environment. The project’s GitHub documentation lists Python and Git as baseline requirements and notes that a target repository may also need R, native command-line tools, GPUs, APIs or other data dependencies. A successful generated agent should record tested versions and usage instructions, but the compatibility burden remains real.
The validation layer is the project’s real contribution
Paper2Agent’s strongest technical claim is not that it uses MCP. Many tools now expose capabilities over the protocol. Its claim is that generated tools are tested against the source paper’s reported figures and results before they are kept in the finished server.
The Nature paper says the system iteratively runs, diagnoses and repairs tests during conversion. A tool is included only after it produces expected files, numerical outputs within specified tolerances and figures sufficiently close to the reference results. The authors say this reduces code hallucination by locking a tested implementation instead of asking an LLM to invent fresh analysis code at query time.
That is a better failure model than simply pasting a paper into a chat window. It gives users something inspectable: the specific functions, environment and tests that produced an answer. It also provides a potentially valuable practical test for research publishers and software maintainers. If an automated system cannot install a paper’s repository, run its official tutorial and recreate representative output, the research may be understandable but is not yet operationally reproducible.
The validation does have limits. Passing a test means the generated workflow matched designated examples, not that every scientific conclusion in a paper is correct. It also does not establish that the workflow will behave correctly on a new institution’s data, a different operating system, a revised upstream library or a dataset that falls outside the original method’s assumptions.
Nature’s reporting highlighted this gap in a revealing example. A Paper2Agent version of Google DeepMind’s AlphaGenome work identified a different candidate causal gene in an analysis of a cholesterol-related DNA variant than the original paper emphasized. The research team argued that the underlying data supported both possibilities. That is an intriguing use case for re-evaluation, but it is also precisely why generated conclusions still need domain-expert review rather than automatic acceptance.
The benchmark is promising, but narrower than the headline
The researchers evaluated Paper2Agent on three groups: 100 computational-biology papers from bioRxiv, 26 data- and discovery-focused papers, and 10 non-biology computational projects. The 74 successful conversions from the 100-paper biology sample were used to generate 300 tutorial-based benchmark questions.
On those questions, the authors report that Paper2Agent using Claude Sonnet 4 reached 91.2 percent accuracy, compared with 80.3 percent for Claude Code given direct access to the same paper and repository. They also report lower per-query cost and latency for the prepared Paper2Agent servers than for direct repository access.
Those are useful results, but they should be read as a comparison between two agent workflows under the team’s evaluation design, not as proof that agentified papers are generally reliable. The paper’s primary validation is concentrated in computational biology and in projects with code repositories that could at least be processed by the pipeline. The system was also built and demonstrated using Claude Code, even though the generated MCP servers are intended to work with other agent clients.
In other words, Paper2Agent’s headline is broad, while its evidence is more specific: it is a promising automation workflow for computational research with reasonably accessible software artifacts. A paper describing a wet-lab experiment, an inaccessible proprietary dataset, a clinical study with restricted records, or an analysis whose real procedure exists only in a researcher’s head cannot be transformed into a trustworthy executable agent by extracting text alone.
Stanford itself acknowledges a related limitation. Its announcement notes that manuscripts omit failed experiments and judgment calls made during experimental work, requiring human authors to add context through interaction with the paper agent. That admission is important. Reproducibility is not solely a packaging problem, and a clean MCP interface cannot recreate details that were never recorded.
Treat generated paper agents as code you must trust
Paper2Agent is open source under the MIT license, and its repository provides installation guidance for Claude Code and Codex. But adoption should follow normal software-supply-chain discipline, especially in enterprise research environments.
The system’s purpose is to take external research artifacts, assemble a working environment and expose code through an AI client with shell access. That creates a much larger trust boundary than a read-only literature assistant. A paper agent may download repositories, install packages, execute scripts, access mounted files, call APIs and produce outputs that look authoritative because they came from a named scientific method.
Administrators considering internal use should isolate conversion jobs, review generated MCP tool definitions and dependency manifests, restrict network egress where possible, and avoid giving the agent credentials broader than the specific workflow needs. Generated servers should be treated like any other externally sourced research code: scan them, pin dependencies, log execution, and promote them from a sandbox only after review.
Data handling deserves equal attention. The Register reported that James Zou said an MCP server can run locally, but that sensitive information would still be sent to the LLM backend connected to Paper2Agent unless the user excludes it. That makes the phrase “run locally” insufficient for protected health information, confidential research data or enterprise datasets. Local execution can protect the code and data path only if the model endpoint, telemetry settings and agent tools are also governed appropriately.
A useful new incentive for reproducible research
Paper2Agent may have its biggest effect before a scientist ever chats with one of its outputs. If research groups begin treating agentification as a release target, they will have a concrete reason to publish runnable examples, explicit environment definitions, stable data schemas and tests that regenerate central figures.
That is a healthier standard than asking readers to accept that a GitHub link exists somewhere beneath a paper. The projects that fail Paper2Agent’s conversion process expose the same weaknesses that frustrate human users: missing documentation, incomplete codebases and environments that cannot be rebuilt.
For now, Paper2Agent is best understood as an experimental bridge between a paper and the software required to use it. Its 74-of-100 conversion result shows that the bridge can work, but also that a substantial share of supposedly reusable research code still has no safe, repeatable path from publication to execution.