GitHub
BUILT FOR AGENTIC INFERENCE

Your agent harness.
Now on vLLM.

vLLM Agentic API is the agentic server that lets you run Codex and Claude Code on top of vLLM.

Everything you need for agentic inference: conversation state, built-in tools, and multi-turn execution with the open models you choose.

Get started View on GitHub
APACHE 2.0BUILT IN RUSTPOWERED BY vLLM
THE AGENTIC STACK01 — 03
Codex
Claude Code
Your app
HTTP · SSE · WebSockets
Agentic APITHE APPLICATION LAYER
RUST
State Tools Execution
Reason → call tools → continue↻
MODEL INFERENCE
vLLMINFERENCE ENGINE

Open models. Your GPUs.

(SGLang and Dynamo also supported)

You own every layer.
One application layer.Conversations+Built-in tools+Multi-turn execution+WebSockets
BUILT FOR THE AGENT LOOP

Inference is the start.
Keep the agent going.

Agentic inference brings model calls, built-in tool execution, and conversation state together. vLLM serves the model; Agentic API coordinates the loop.

01 / CONTEXT

A conversation that carries forward.

Keep item history and response state on the server. Continue with previous_response_id, without rebuilding the transcript on every call.

response_01→response_02→response_03
02 / ACTION

Tools, with the server in the loop.

Execute built-in web search and tools from Model Context Protocol (MCP) servers on the gateway. Tool call outputs flow back to the model so it can take the next step.

web_search MCP toolsGATEWAY-EXECUTED
03 / EXECUTION

One request.
Multiple inference rounds.

Let the gateway coordinate model calls and built-in tools until the response is ready or the client needs to act.

REASON → ACT → CONTINUE
04 / TRANSPORT

Stay connected.
Keep streaming.

Use HTTP for requests, server-sent events (SSE) for streamed responses, and WebSockets for interactive Responses API clients.

HTTPSSEWS
05 / INFERENCE

Open models.
Your choice.

Connect to compatible open models served by vLLM. Choose the model and tool-calling configuration for your workload.

EXPLORE SETUP

Clear execution boundaries. Codex and Claude Code still execute their own shell and editor tools. Agentic API runs the tools assigned to the gateway.

ACROSS THE OPEN INFERENCE STACK

Bring your serving stack.

Built for vLLM, with recorded integration tests for SGLang and NVIDIA Dynamo covering streaming, multi-turn state, and client-executed function calls.

SGLang

Connect Agentic API to SGLang’s Responses endpoint. The gateway stores response history and carries it into the next turn.

TESTED CONFIGURATIONSGLang 0.5.18 · Qwen3-8B
Setup and test coverage

NVIDIA Dynamo

Point Agentic API at the Dynamo frontend to add conversation state and continuation to your distributed inference stack.

TESTED CONFIGURATIONDynamo 1.4.1 · vLLM worker · GPT-OSS 20B
Setup and test coverage
FROM MODEL TO AGENT

Same workflow.
Your infrastructure.

Point your coding agent at Agentic API. Keep your familiar workflow, with an open model behind it.

  1. 01
    Serve your model

    Run a tool-capable model with vLLM. The example uses an upstream on port 5050.

  2. 02
    Install Agentic API

    Get the release from crates.io. A Python package is coming to PyPI, with source builds available for development.

  3. 03
    Launch your client

    Install Codex or Claude Code, then choose your client and served model.

Read the full setup guide
QUICKSTARTbash
Install from crates.io
cargo install agentic-server --locked

Install the released gateway and agentic CLI. Requires Rust and Cargo.

Launch Codex
agentic run codex \
  --upstream http://127.0.0.1:5050 \
  --model Qwen/Qwen3-30B-A3B-FP8
Use the model ID served by your vLLM instance.
BUILT IN THE OPEN

Better agents.
Built together.

Meet the people building the application layer for open-model agents. There’s room for your ideas, your fixes, and your next contribution.

Join us in #sig-agentic-api on the vLLM Slack.

Meet the team See contributors

People helping make Agentic API better.

Make your first contribution