vLLM Tool Parsers — Navigation Map

This skill points to the right source file, template, or GH issue. The source code is authoritative — read it. Do not paraphrase from this skill when the actual file is available.

Where things live

Assume a local vllm-project/vllm checkout is accessible. Every reference below is relative to that repo root.

If the operator's question is "what does parser X do" — read `vllm/tool_parsers/X_tool_parser.py`.

Except for the 19 names on the unified-parser path (v0.32.0): there the tool_parsers/ file is a stub of a few lines and the logic lives in vllm/parser/<model>.py. The CLI name → real file map, with the version each name moved, is in references/parser-index.md (rows written a.py → vllm/parser/b.py).

Recent names NOT on this path — dots (DotsToolParser), hy_v4 (HYV4ToolParser), muse_glimmer (MuseGlimmerToolParser) and k2_horizon (K2HorizonToolParser). None imports a *ParserToolAdapter, so each is an ordinary standalone tool parser with no paired reasoning adapter. Also off-path at v0.32.0: qwen3_json (an alias of hermes's Hermes2ProToolParser) and hf (ResponseTemplateToolParser, see the CLI contract). Registry total: 54 names at v0.32.0 (51 at v0.30.0 and v0.31.0).

`_engine_` in the filename is not the marker. glm47_moe_tool_parser.py, kimi_k2_tool_parser.py, minimax_m2_tool_parser.py and mistral_tool_parser.py have ordinary names and are still stubs. The test is whether the file imports an adapter: grep -l "ParserToolAdapter" vllm/tool_parsers/*.py (17 files at v0.32.0, 12 at v0.30.0; ling3_tool_parser.py imports its adapter from vllm.parser.ling3, so grepping registered_adapters misses it). Exception: cohere_command_tool_parser.py (cohere_command3/cohere_command4) is a stub too but imports no adapter — it's composed by name-check in vllm/parser/parser_manager.py instead, so this grep won't find it. Its parsing lives in vllm/parser/cohere_command.py via the external cohere_melody package — not a vLLM runtime dependency: pip install cohere-melody into the image (CI pins 0.14.0), or the parser raises ImportError at load. (kimi_k3 goes through the same name-check but its tool_parsers/ file is the real implementation, not a stub.)

This is the same refactor described in vllm-reasoning-parsers — a single per-model parser now backs both the tool and reasoning adapters. Consequence: a grammar change to vllm/parser/qwen3.py moves tool and reasoning behaviour at once — they are no longer independent surfaces for those models.

The CLI contract

Two flags, both required together for auto tool choice:

vllm serve <model> --enable-auto-tool-choice --tool-call-parser <name> [--chat-template <path>]

• --enable-auto-tool-choice alone → TypeError: --enable-auto-tool-choice requires --tool-call-parser (see cli_args.py).
• --tool-call-parser alone → legal. Parser still runs for tool_choice="required" and named, and on Responses API.
• v0.32.0+: a parser the tokenizer cannot serve (e.g. mistral without [TOOL_CALLS] in the vocab) fails startup with TypeError: --tool-call-parser X cannot be used with the tokenizer of '<model>'. Before v0.32.0 the server came up, /health returned 200, and every chat request returned 500.
• --tool-call-parser hf (v0.32.0+) parses from the checkpoint's response_template in tokenizer_config.json; startup fails without it. Cannot be combined with another tool or reasoning parser; rejects strict tools, required/named tool_choice and parallel_tool_calls=false.
• --tool-strict-level auto|function|parameter (v0.31.0+, default auto) raises the structural-tag floor server-side: function constrains the call envelope (markup + name) on every request with tools; parameter also pins every tool's argument schema as if strict: true. VLLM_ENFORCE_STRICT_TOOL_CALLING=false overrides it. Read docs/features/tool_calling.md.
• No `auto` sentinel. Name a concrete parser.
• Ship a parser change as a --tool-parser-plugin file or in a rebuilt image, never by editing site-packages inside running pods: the pod drifts from its image, the edit is lost on restart or reschedule, and there is no rollback.
• --tool-parser-plugin <path.py> → third-party file that calls @ToolParserManager.register_module("name"). v0.30.0+: the value can also be a dotted module name importable from site-packages (e.g. an installed pip package) — import_plugin() tries importlib.import_module(value) first, falling back to file-path import only on ModuleNotFoundError (vllm/utils/import_utils.py). Same for --reasoning-parser-plugin.
• --reasoning-parser is independent but several tool parsers assume a </think> has closed — match them (see "Reasoning-parser pairing" below).
• Chat template often matters. Each parser has a reference Jinja at examples/tool_chat_template_<family>.jinja. Wrong template → model never emits the sentinels the parser expects.

Parser → model family index

Use this to pick the CLI name. Then read the parser file and the matching Jinja for details — the wrapping tokens, streaming strategy, and quirks live there, not here.

The table is not authoritative — verify against the deployed tree:

grep -oE '^\s+"[a-z0-9_-]+": \(' vllm/tool_parsers/__init__.py | tr -d ' ":('   # CLI names only (54 at v0.32.0)
ls examples/tool_chat_template_*.jinja            # lists shipped templates
ls vllm/tool_parsers/*_tool_parser.py              # lists source files

Framework contract (mental model)

• ToolParser subclass implements extract_tool_calls (non-streaming, stateless) and extract_tool_calls_streaming (stateful, per-delta). See vllm/tool_parsers/abstract_tool_parser.py.
• Serving-layer invariants (guaranteed to the streaming method):
• current_text == previous_text + delta_text
• current_token_ids == previous_token_ids + delta_token_ids
• Deltas may span multiple tokens.
• Four state fields the parser MUST maintain:
• prev_tool_call_arr: list[dict] — at stream end get_remaining_unstreamed_args() takes the last entry's "arguments", subtracts what streamed_args_for_tool already sent, and appends the rest to the final delta's last tool call — only if that final delta carries tool_calls.
• current_tool_id: int — starts -1, increments per call.
• current_tool_name_sent: bool — flip True once name flushed for current tool.
• streamed_args_for_tool: list[str] — cumulative args already emitted per tool index. Append on every flush: if it is not a prefix of the full args, the tail is silently dropped; if it lags, the tail re-sends text already streamed.
• Streaming finish_reason is tool_calls iff at least one returned DeltaMessage carried tool_calls (serving's tools_streamed[i]), the request did not name a tool, and (v0.31.0+) the engine stopped on stop — a max_tokens-truncated stream keeps length. A parser that only buffers and never emits a DeltaToolCall ends on stop, whatever prev_tool_call_arr holds.
• Optional: adjust_request(request) — set skip_special_tokens=False, inject grammar, etc. supports_required_and_named: bool = True — flip False if the output shape breaks guided JSON; required/named then get no JSON-schema injection and the parser must extract calls from its native format.
• Return-value contract for streaming: None = "consumed, nothing to emit"; DeltaMessage(content=...) = pass-through; DeltaMessage(tool_calls=[DeltaToolCall(...)]) = tool progress.

Align new parsers with the parse_delta shape (RFC #11522 and its merged follow-on PRs #39446, #39728, #38755) rather than copying older HACKs.

Reasoning-parser pairing

Several tool parsers gate on a reasoning-end sentinel. Mismatched pair = tool parser sees reasoning text as content, misses sentinels, or emits from inside <think>. Table below is a pointer — verify by reading the tool-parser file for the adjust_request/is_reasoning_end interaction.

Sibling skill: vllm-reasoning-parsers — defer there for reasoning-side questions.

Diagnostic playbook

Work down this list in order on any broken tool call.

1. Both flags set? Check serve command for --enable-auto-tool-choice --tool-call-parser <name>.
2. Right parser for the model? Cross-check with the table above AND with vllm/tool_parsers/__init__.py registry.
3. Chat template matches? Inspect the actual template bytes — don't trust the filename. See step 7. Parser and template are a pair: switching --tool-call-parser between flavours (llama4_json ↔ llama4_pythonic, JSON ↔ pythonic) without switching --chat-template makes the model emit a shape the parser never matches — calls arrive as plain content. A --chat-template value that cannot be opened fails startup (looks like a file path, but it failed to be opened) — unless it contains {, } or a newline, in which case vLLM uses the string itself as the template — a mistyped path containing { silently becomes a literal template. Check the path is readable inside the container before rollout.
4. Reasoning parser paired? If the model has <think> / </seed:think> / harmony channels, --reasoning-parser must match. Check vllm/reasoning/ for the registry.
5. Streaming vs non-streaming? Grep the parser file: if extract_tool_calls_streaming returns None unconditionally or raises NotImplementedError, streaming isn't supported (e.g. phi4_mini_json, openai).
6. `finish_reason` = `stop` when a call was expected? The parser never returned a DeltaMessage with tool_calls during the stream (or the request named a tool, where stop is correct). Compare with "stream": false on the same prompt: tool calls there but not in the stream = a streaming-path bug in that parser.
7. Raw model output vs what parser sees. Bypass the parser: call /v1/completions (no tool parser) with the chat-rendered prompt — tokenizer.apply_chat_template(messages, tools=tools, tokenize=False, add_generation_prompt=True) with the deployed template. Keep "skip_special_tokens": false: the default true strips special-token sentinels (<|tool_call|>, [TOOL_CALLS], …) from text, which looks exactly like "model never emitted them". Dump raw text and non-ASCII codepoints:

   curl -sS $VLLM/v1/completions -H 'content-type: application/json' \
     -d '{"model":"...","prompt":"...","max_tokens":200,"skip_special_tokens":false}' \
     | python3 -c "import sys,json,unicodedata; t=json.load(sys.stdin)['choices'][0]['text']; \
         print(repr(t)); [print(hex(ord(ch)), unicodedata.name(ch,'?'), repr(ch)) for ch in t[:2000] if ord(ch)>0x7E]"

Read the result with the table below.

1. vLLM version vs known bugs. See "Bug archaeology" below.
2. Custom template? Diff the deployed Jinja against examples/tool_chat_template_<family>.jinja. Character-level. Full-width vs ASCII Unicode gotchas bite here.

Done when steps 1–7 each have a pass/fail answer backed by evidence (serve args, registry name, template bytes, raw /v1/completions text), and every failing step has a fix named:

Bug archaeology (don't trust static lists — search)

# Open bugs affecting a specific parser
gh search issues --repo vllm-project/vllm "<parser-name> tool" --state open \
  --json title,url,state,updatedAt --limit 20

# Recently merged fix PRs
gh search prs --repo vllm-project/vllm "<parser-name>" --state merged \
  --json title,url,mergedAt --limit 20

# Streaming-specific
gh search issues --repo vllm-project/vllm "tool parser streaming <parser-name>" \
  --json title,url,state,updatedAt --limit 20

# Broad sweep if parser name unknown
gh search issues --repo vllm-project/vllm "extract_tool_calls_streaming" --limit 30

On finding a referenced issue/PR, read it directly (gh issue view N --repo vllm-project/vllm --comments). Never paraphrase from a cached memory — the fix may have landed since.

In-tree HACKs to recognize

grep -rn "HACK" vllm/tool_parsers/
grep -rn "TODO" vllm/tool_parsers/
grep -rn "prev_tool_call_arr = \[{\"arguments\": {}}\]" vllm/tool_parsers/

The prev_tool_call_arr = [{"arguments": {}}] plant is the old "force finish_reason=tool_calls" workaround; finish_reason no longer reads that field, and where it once forced tool_calls it produced a call with empty arguments — hiding the failed parse instead of fixing it. Do not copy it into a new parser. At v0.32.0 exactly five parsers carry it: pythonic, llama4_pythonic, olmo3, lfm2, minicpm5xml — the pythonic family plus the two that reuse its flush shape. (mistral does not; it populates prev_tool_call_arr properly.) Write a new parser to the parse_delta shape (RFC #11522) instead.

Writing a custom parser

Copy the existing parser closest in format.

# Match the target shape to an existing parser
grep -lE "<your_sentinel>" vllm/tool_parsers/   # sometimes the sentinel itself is in the code
ls vllm/tool_parsers/                            # scan names by format

For the skeleton + detailed checklist see references/custom-parser-plugin.md. Key points:

• Register with @ToolParserManager.register_module(["name"]) (decorator path is lazy — plugin loader imports the file).
• Launch: --tool-parser-plugin /abs/path/file.py --tool-call-parser name.
• Closest starting point for most models: vllm/tool_parsers/pythonic_tool_parser.py (kwargs Python syntax), hermes_tool_parser.py (JSON-in-tags), or dots_tool_parser.py (standalone streaming XML <invoke>/<parameter>; step3p5_tool_parser.py is an engine stub at v0.32.0+).
• Must honor the four state fields above. Read vllm/tool_parsers/abstract_tool_parser.py for exact signatures.
• Read tests/tool_parsers/common_tests.py — reviewers expect this harness.

Before upstreaming, read AGENTS.md at the repo root for the duplicate-work + PR-description policy, and vllm/tool_parsers/AGENTS.md (v0.32.0+): new in-tree models go in vllm/parser/ as a unified Parser plus a thin adapter; ToolParser is kept as legacy for plugins.

References in this skill

Last verified: 2026-10-18 against vLLM v0.32.0. See references/sources.md for per-reference probe details and timestamps.