MiniCPM5 tool parser by przepeck · Pull Request #4373 · openvinotoolkit/model_server

przepeck · 2026-07-13T13:05:11Z

🛠 Summary

https://jira.devtools.intel.com/browse/CVS-187940
Adding tool parser for MiniCPM5, based on PR#4339

MiniCPM5 - int4

unary results:

streaming results:

MiniCPM5 - int8

unary results(without reasoning):

unary results(with reasoning):

Adding autodetection for this parser and reasoning parser.
Minja is currently unsupported for this model.

🧪 Checklist

Unit tests added.
The documentation updated.
Change follows security best practices.
``

MiniCPM5 emits XML-style tool calls (<function name="..."><param name="...">value</param></function>) which OVMS had no parser for, so /v3/chat/completions left the XML in message.content with empty tool_calls. Add a minicpm5 tool parser modeled on the qwen3coder XML parser: - new src/llm/io_processing/minicpm5/ tool parser (unary + streaming state machine; attribute-style tags; <think>...</think> stripping; schema-driven argument typing) - register "minicpm5" in output_parser.cpp and supported parser names - bazel target + unit tests Relates to #4268. Prototype for maintainer coordination; not yet a PR. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

Harden the prototype's test coverage: - streaming: char-by-char and every-fragment-size feeds must yield the same tool calls as a single-chunk parse (single call, two calls, leading content + <think> block) - edge cases that must not crash or emit spurious calls: unterminated <function>, empty <param> value, stray '<' in plain prose All 15 Minicpm5 impl/stream tests pass locally. Relates to #4268. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

…easoning, add tests (#4268) Responds to review feedback on the MiniCPM5 tool parser: - Extract helpers shared with qwen3coder: trimNewline, jsonTypeOf, enforceStringValue and Python-bool normalization move to io_processing/utils; parseToolSchema/createToolsParametersTypesMap move to base_output_parser (which already owns the ParameterType/ToolsSchemas types). Both parsers now use the shared versions; qwen3coder behavior is unchanged. - Split the long parseUntilStateChange switch into per-state handler methods. - Stop handling <think> in the tool parser; reasoning is now the reasoning parser's job (reuse Qwen3ReasoningParser via reasoning_parser: qwen3, same <think> tags), so reasoning is preserved as reasoning_content. - Tests: drive the public parser streaming path (name delta then arguments delta) and add complex-content cases (embedded quotes/backslashes, code with braces/newlines, JSON-looking string values, special/unicode chars). The special-token-tag handling (requiresStreamingWithSpecialTokens) is deferred to a follow-up; it needs the model to confirm which tags are special tokens and is coupled to the reasoning parser's matching flag. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

…rser (#4268) MiniCPM5-1B emits its tool-call tags as special tokens (<function id=18, </function>=19, <param>=20, </param>=21 in the openbmb/MiniCPM5-1B tokenizer), which the text streamer skips by default — so the tag-scanning parser never saw them during streaming. Override requiresStreamingWithSpecialTokens() to return true so they are preserved. The framework requires a paired reasoning parser to agree on that flag. Since MiniCPM5's <think>/</think> are ordinary (non-special) tokens, the Qwen3 reasoning logic still applies verbatim; add a thin Minicpm5ReasoningParser subclass that only flips the flag, and register "minicpm5" as a reasoning parser. Tests updated to pair minicpm5+minicpm5 and assert the flag. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

Addresses review feedback: no MiniCPM5 tokenizer exists in the test assets. The parser operates on decoded text (tokenizer.decode) via plain string/XML matching, never token ids or vocab-specific special tokens, so any tokenizer that round-trips the test strings via encode()+decode() is safe to use here. Signed-off-by: exzile <joeypongallo@gmail.com>

Addresses review feedback: Minicpm5ReasoningParser previously inherited Qwen3ReasoningParser's text-based parse() verbatim and only overrode requiresStreamingWithSpecialTokens(). MiniCPM5's <think>/</think> tags are real dedicated special tokens in its tokenizer (ids 8 and 9), not incidental text, so parse() now locates the reasoning segment by scanning generatedTokens for these token ids directly -- mirroring the existing Gemma4ReasoningParser pattern -- instead of searching decoded text for the tag substrings. requiresStreamingWithSpecialTokens() remains true: MiniCPM5's tool-call tags (<function>, <param>, ...) are also special tokens, and the framework requires the paired tool/reasoning parser to agree on this setting (same as Gemma4). Test fix: the existing reasoning-parser test encoded '<think>...</think>' text with the Qwen3-8B stand-in tokenizer used elsewhere in this test file (no MiniCPM5 tokenizer test asset exists). Qwen3-8B has its own, different ids for those strings (151667/151668), so the new token-id based parse() would not have found them. The test now assembles generatedTokens directly with the real ids (8/9) as delimiters and uses the stand-in tokenizer only to encode/decode the surrounding free-form text, which round-trips correctly regardless of whose vocab it is. Signed-off-by: exzile <joeypongallo@gmail.com>

…soning Self-review fix: parse() previously decoded the content spans (outside the <think>...</think> boundary) with skip_special_tokens(true), copied from Gemma4ReasoningParser's pattern. That's safe for Gemma4 because its tool parser reads raw token ids, but MiniCPM5's tool parser is text-based -- Minicpm5ToolParserImpl matches the literal "<function"/"<param" substrings in decoded content. Those tags are themselves special tokens in MiniCPM5's tokenizer (<function = id 18, <param = id 20, both "special": true), so skip_special_tokens(true) silently erased them from content before the tool parser ever ran, breaking tool-call extraction whenever a reasoning block was present. Content spans now decode with skip_special_tokens(false), matching the framework's own default full-decode behavior. Only the extracted reasoning text keeps skip_special_tokens(true), since it's a human-facing string not matched against tags downstream. Also documented the existing find-first-occurrence limitation shared with Gemma4ReasoningParser (a stray </think> before any <think> would hide a later well-formed pair). Signed-off-by: exzile <joeypongallo@gmail.com>

…verage Addresses review feedback: the Minicpm5ToolParserImplStreamTest section (streamInFragments-based) only asserted that the aggregated final result is fragmentation-invariant, not that the actual user-facing streaming deltas are surfaced correctly. Retitled/documented that section to make its scope explicit and point to Minicpm5PublicStreamTest (already added in an earlier review round, commit 869484c) for the delta-timing concern, which drives the real Minicpm5ToolParser::parseChunk API and asserts on the actual emitted name/arguments deltas. That existing coverage was single-call only, so added EmitsDeltasForTwoSequentialCalls: verifies each of two sequential tool calls gets exactly one name delta and one arguments delta, in order, each carrying the correct (distinct, increasing) tool_call index -- not just that the final parse result is correct. Signed-off-by: exzile <joeypongallo@gmail.com>

…into przepeck/minicpm5

Copilot

Pull request overview

Adds MiniCPM5 support to OVMS LLM output parsing by introducing a MiniCPM5 XML-style tool-call parser and a paired reasoning parser, wiring both into OutputParser selection and supported-parser validation, and adding unit/streaming tests plus model-prep script support.

Changes:

Implemented minicpm5 tool parser + reasoning parser and registered them in the parser factory/validation lists.
Refactored/shared schema + JSON/string helper utilities to support attribute/tag-style tool parsers.
Added a comprehensive MiniCPM5 output-parser test suite and updated LLM model preparation script for the MiniCPM5 tokenizer.

Reviewed changes

Copilot reviewed 14 out of 14 changed files in this pull request and generated 6 comments.

Show a summary per file

File	Description
src/test/llm/output_parsers/minicpm5_output_parser_test.cpp	Adds unary + streaming tests for MiniCPM5 tool calls and reasoning handling.
src/llm/io_processing/utils.hpp	Exposes shared helper APIs for tool parser implementations.
src/llm/io_processing/utils.cpp	Implements shared helper utilities (newline trim, JSON type, boolean normalization, quote normalization).
src/llm/io_processing/qwen3coder/qwen3coder_tool_parser.cpp	Switches qwen3coder to shared helpers (reduces duplication).
src/llm/io_processing/parser_config_validation.cpp	Registers `minicpm5` as supported tool + reasoning parser.
src/llm/io_processing/output_parser.cpp	Wires `minicpm5` parser implementations into the factory.
src/llm/io_processing/minicpm5/minicpm5_tool_parser.hpp	Declares MiniCPM5 tool parser + state-machine impl.
src/llm/io_processing/minicpm5/minicpm5_tool_parser.cpp	Implements MiniCPM5 XML tool-call parsing and streaming deltas.
src/llm/io_processing/minicpm5/minicpm5_reasoning_parser.hpp	Declares MiniCPM5 reasoning parser.
src/llm/io_processing/minicpm5/minicpm5_reasoning_parser.cpp	Implements token-based reasoning extraction and streaming reasoning deltas.
src/llm/io_processing/base_output_parser.hpp	Exposes shared schema→type-map helpers for tool parsers.
src/llm/io_processing/base_output_parser.cpp	Implements shared schema→type-map helpers.
src/llm/BUILD	Adds Bazel target for MiniCPM5 parsers and updates deps.
prepare_llm_models.sh	Adds MiniCPM5 tokenizer conversion/download step.

przepeck · 2026-07-14T13:32:13Z

Model's BFCL accuracy, without reasoning.
With reasoning it's ~ 40%, it has tendency to loop and returning arguments as object with the function name of "arguments".

handling case indicated by copilot, when the whole tool call is being delivered in one chunk

Copilot

Pull request overview

Copilot reviewed 14 out of 14 changed files in this pull request and generated 12 comments.

Copilot

Pull request overview

Copilot reviewed 21 out of 21 changed files in this pull request and generated 4 comments.

atobiszei · 2026-07-16T14:42:53Z

+    toolSchemas(toolSchemas),
+    streamParser(this->toolsParametersTypes) {}
+
+const std::vector<int64_t> Minicpm5ToolParser::removeReasoningTokens(const std::vector<int64_t>& generatedTokens) {


Why we need to remove reasoning tokens from nside tool parser?

We are using tokens, from const std::vector<int64_t>& generatedTokens, it's const so we cannot remove reasoning tokens from this vector in reasoning parser, that means we need to remove it in tool parser's content. It's smelly WA, but I cannot think about anything better then refactor of BaseOutputParser. Miłosz knows about this issue and it will be probably solved in next iteration of our parser's refactor

exzile and others added 17 commits June 28, 2026 22:42

Merge branch 'main' into feature/minicpm5-tool-parser

5c6e4a7

Merge branch 'main' into feature/minicpm5-tool-parser

e9a9d86

Merge branch 'main' into feature/minicpm5-tool-parser

2243ea4

Merge remote-tracking branch 'minifork/feature/minicpm5-tool-parser' …

b77dd79

…into przepeck/minicpm5

save

d7a00ec

fix content with reasoning and without tool calls

00c099a

streaming tests

dc115a9

fixing array and object parsing

4376744

styles

db2abda

Copilot AI review requested due to automatic review settings July 13, 2026 13:05

Copilot started reviewing on behalf of przepeck July 13, 2026 13:05 View session

Copilot AI reviewed Jul 13, 2026

View reviewed changes

przepeck added 4 commits July 14, 2026 12:36

fix reasoning, tests and special tokens in content

2f966cd

reasoning fixes

9c4cf6c

styles

941b157

Merge branch 'main' into przepeck/minicpm5

baf4831

przepeck added 2 commits July 14, 2026 17:34

copilot review

a60410e

handling case indicated by copilot, when the whole tool call is being delivered in one chunk

style

88bfa43

przepeck requested review from atobiszei, Copilot and dkalinowski July 15, 2026 08:10

Copilot AI reviewed Jul 15, 2026

View reviewed changes

Copilot started reviewing on behalf of przepeck July 15, 2026 08:37 View session

przepeck added 5 commits July 15, 2026 12:42

typos

a02c8dd

copilot review

52706ea

styles

59ad38d

Merge branch 'main' into przepeck/minicpm5

053ff35

windows model download

645a29e

dtrawins requested a review from Copilot July 16, 2026 12:24

Copilot started reviewing on behalf of dtrawins July 16, 2026 12:24 View session

autodetection of tool and reasoning parser

24ca369

Copilot AI reviewed Jul 16, 2026

View reviewed changes

Comment thread src/llm/io_processing/minicpm5/minicpm5_tool_parser.cpp

Comment thread src/llm/io_processing/utils.cpp

Comment thread src/test/llm/output_parsers/minicpm5_output_parser_test.cpp Outdated

Comment thread src/test/llm/output_parsers/minicpm5_output_parser_test.cpp Outdated

dtrawins added this to the 2026.3 milestone Jul 16, 2026

copilot's review

3ee10e9

atobiszei reviewed Jul 16, 2026

View reviewed changes

Uh oh!

Conversation

przepeck commented Jul 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🛠 Summary

MiniCPM5 - int4

MiniCPM5 - int8

🧪 Checklist

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

przepeck commented Jul 14, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

atobiszei Jul 16, 2026

Choose a reason for hiding this comment

Uh oh!

przepeck Jul 16, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

przepeck commented Jul 13, 2026 •

edited

Loading