# agentic-api Roadmap

`agentic-api`, also referred to as vLLM Agentic API, is the stateful agentic
API layer for [vLLM core](https://github.com/vllm-project/vllm). It is optimized
for vLLM core and implemented in Rust. Its job is to own stateful agentic APIs,
including Responses and Messages, and to orchestrate server-side tool execution
for tool calls generated by vLLM core.

This roadmap uses:

- **vLLM core** for [`vllm-project/vllm`](https://github.com/vllm-project/vllm),
  the inference engine and model-serving project.
- **vLLM Agentic API** for this repository, `vllm-project/agentic-api`.

vLLM Agentic API should let vLLM core focus on inference while this repository
handles API state, continuation, tool-call execution, and the higher-level
orchestration needed by stateful agentic applications.

## Project Goals

### 1. Responses API Hydration

The first project goal is reliable state hydration for the Responses API.

This means supporting `previous_response_id` so clients can continue from a
stored response without replaying the full conversation history. The server must
persist the model-visible items needed for continuation, rehydrate them in the
right order, append new input, and send the resulting request to vLLM core.

Core work:

- Store Responses API state needed for continuation.
- Rehydrate prior input, assistant output, tool calls, and tool outputs.
- Preserve OpenAI-compatible request and response shapes.
- Support streaming and non-streaming Responses API flows.
- Keep storage semantics explicit and testable.

### 2. Codex Support Through Responses

The second project goal is to support Codex through the Responses API.

Codex depends on stateful continuation, tool-call preservation, and a broad set
of agentic tool and response shapes. `agentic-api` should accept Codex-compatible
Responses traffic, preserve the fields Codex needs, and execute or return tool
calls according to the ownership model for each tool.

Core work:

- Preserve Codex-used tool declarations and response item shapes.
- Support client-owned tools, gateway-owned tools, and provider-owned tools.
- Implement the tool calls and API behavior needed for practical Codex sessions.

### 3. Server-Side Tool Execution

The project should execute server-side tool calls generated by vLLM core.

When vLLM core produces a tool call that is owned by the gateway, vLLM Agentic
API should dispatch it, capture the result, append the tool output, and continue
the agentic loop. The tool system should make ownership explicit so each tool
call has a clear execution path.

Tool ownership means:

- **Gateway-owned tools** are executed by vLLM Agentic API.
- **Client-owned tools** are preserved and returned to the client for execution.
- **Provider-owned tools** are passed through to vLLM core or another upstream
  provider surface; vLLM Agentic API does not execute them.

Ownership should be resolved from the request, the configured tool registry, and
the tool type. vLLM Agentic API should only execute tools that resolve to a
configured gateway-owned handler. Unknown, unsupported, or ambiguous tool shapes
are preserved and returned or passed through; they are never executed by default.

Requests may opt into parallel tool calling for gateway-owned built-in tools: a
single turn can invoke the same built-in tool more than once (for example, two
web searches). Agentic API forwards that model-generation preference upstream,
then executes emitted gateway calls through a bounded, configurable window.
Calls to different tool names can overlap; calls to the same name overlap only
when that handler declares it safe. Results retain model call order and are all
appended before continuing the agentic loop
([#181](https://github.com/vllm-project/agentic-api/issues/181)).

Initial and expected tool areas include:

- File search.
- Web search.
- Web fetch.
- Computer use.
- Shell tools.
- MCP-backed tools and resources.
- Function tools owned by the gateway.

### 4. Messages API

The third major API goal is Messages.

Messages should build on the same persistence, rehydration, and execution
foundations as Responses, while exposing the API surface expected by agentic
clients that organize state around message objects.

Messages should adapt into shared execution primitives for items, tool calls,
and tool outputs instead of forking a Responses-specific execution loop.

Core work:

- Define the Messages API state model.
- Map Messages state into the shared execution and storage primitives.
- Preserve compatibility with Responses where the APIs overlap.
- Add tests that cover continuation and tool-call behavior through Messages.

### 5. Broader Stateful Agentic APIs

After Responses, Codex support, and Messages, the project should add the
remaining stateful APIs needed by core agentic applications.

This should be driven by concrete client needs and vLLM core integration points,
not by adding broad abstractions ahead of use. Each new API should reuse the
common storage, continuation, and tool-execution foundations wherever possible.

## Near-Term Focus

The near-term focus is to make the Rust implementation dependable and easy to
extend.

- Keep orchestration logic reusable across standalone and gateway deployments.
- Keep HTTP serving, gateway integration, and tool execution thin around the
  shared stateful agentic API implementation.
- Keep the implementation Rust-first and optimized around vLLM core's inference
  surface.
- Treat model aliasing and routing as deployment concerns that may live at the
  standalone server or gateway edge; they are not part of tool execution
  semantics.
- Expand compatibility tests for Responses, Codex-shaped traffic, tool calls,
  streaming, and continuation.
- Document behavior as it stabilizes, especially where compatibility or tool
  ownership rules are subtle.

## Longer-Term Direction

Longer-term work should improve production readiness and performance without
moving inference responsibilities out of vLLM core.

- Production storage backends, retention policy, and compaction.
- Observability for request lifecycle, tool execution, and continuation.
- Gateway integration with Praxis and other deployment surfaces.
- Cached prefix continuation and other latency optimizations where vLLM core
  owns rendering, tokenization, and KV-cache execution.
- Coordination with vLLM core and llm-d where token identity, prefix routing,
  and renderer boundaries matter.

## Non-Goals For Now

These are intentionally out of scope until the core APIs and execution model are
stable.

- Reimplementing tokenization, chat templates, or model-specific rendering in
  this repository.
- Building a broad plugin platform before the built-in execution model is
  settled.
- Adding stateful agentic APIs without a concrete client or vLLM core
  integration need.
- Moving vLLM core inference responsibilities into vLLM Agentic API.