feat(cli): document pages/s in gw chat stats; rename tokens to toks - #217
Merged
Conversation
Report pages and pages/s in the gw chat summary footer for document inputs, parsing gateway OCR wrappers or counting local PDF pages when pypdfium2 is available. Rename the token count label from tokens to toks to match the existing toks/s throughput metric. Co-authored-by: sudeep <sudeep@vlm.run>
Co-authored-by: sudeep <sudeep@vlm.run>
Remove pypdfium2 fallback for page counts. Pages and pages_per_sec are parsed from <document pages="N"> in gateway output for stream, --no-stream, and --json modes. Co-authored-by: sudeep <sudeep@vlm.run>
Co-authored-by: sudeep <sudeep@vlm.run>
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
|
|
||
| def _parse_document_pages_from_content(content: str) -> Optional[int]: | ||
| """Sum ``pages`` attributes from ``<document pages="N">`` wrappers in OCR output.""" | ||
| matches = _DOCUMENT_PAGES_RE.findall(content) |
Contributor
There was a problem hiding this comment.
🟡 Document-like OCR text inflates page metrics
When OCR text contains <document pages="N">, _parse_document_pages_from_content counts it as another wrapper. The footer and JSON then overstate pages and throughput.
Prompt for agents
The page parser in vlmrun/cli/_cli/gateway.py uses findall across the complete model-generated response. OCR page bodies can themselves contain text matching <document pages="N">, so those matches are summed with the gateway's real document wrappers. Parse only wrapper elements that occur at valid document boundaries, while preserving support for multiple top-level wrappers and avoiding matches inside page content. Add a regression test containing one real wrapper and a document-like tag in its OCR text.
Was this helpful? React with 👍 or 👎 to provide feedback.
Use dim not-bold Rich styling instead of dim white for the stats footer. Round elapsed time to integer seconds (e.g. 15s) for a cleaner total. Co-authored-by: sudeep <sudeep@vlm.run>
glm-ocr wraps OCR output in <document num_pages="N"> rather than pages="N". Accept both attribute names when reporting pages/s. Co-authored-by: sudeep <sudeep@vlm.run>
glm-ocr wraps URL-backed PDFs with num_pages (not pages) and optional <page> blocks. Broaden the OCR output parser for single quotes, attributes split across streaming chunks, and page-tag fallback. Add gw chat tests that assert pages metrics for http(s) document inputs. Co-authored-by: sudeep <sudeep@vlm.run>
Restore dim white (not bold) for the stats footer and panel subtitle. Drop the blue rule style on the streaming stats line so the summary stays neutral. Co-authored-by: sudeep <sudeep@vlm.run>
Display page metrics as `5 pages · 2.00 pages/s` instead of `pages: 5 · pages/s: 2.00` to match other throughput labels. Co-authored-by: sudeep <sudeep@vlm.run>
Co-authored-by: sudeep <sudeep@vlm.run>
jeremyipark
approved these changes
Sep 2, 2026
1 task
jeremyipark
pushed a commit
that referenced
this pull request
Sep 2, 2026
<!-- CURSOR_AGENT_PR_BODY_BEGIN --> ## Summary Fixes `toks/s` to measure from first streamed token (excluding TTFT), rebased onto main so PR #217's `pages` / `pages/s` stats are preserved. - `_drain_stream` returns `generation_s` (first token → stream end) - `_stats_parts` uses `generation_s` for `toks/s`, total `latency_s` for `pages/s` and the `Xs` footer - Page stats from #217 (`_parse_document_pages_from_content`, footer + `--json`) unchanged ## Why pages might not appear Page metrics are **parsed from OCR response text**, not from the local PDF: - Needs `<document pages="N">` / `num_pages="N"` or countable `<page>` tags (glm-ocr markdown) - Raw JSON OCR output (`--method detect` / `ocr`) has no document wrapper → no pages in footer ## Test plan - [x] `pytest -sv tests/test_gateway.py` (120 passed; 1 streaming footer test can flake on narrow terminals when `toks/s` is huge) <!-- CURSOR_AGENT_PR_BODY_END --> <div><a href="https://cursor.com/agents/bc-58c0d343-1ef3-46d2-8ce5-e2704dc70d00?cursor_ref=pr_footer&cursor_cta=open_in_web"><picture><source media="(prefers-color-scheme: dark)" srcset="https://cursor.com/assets/images/open-in-web-dark.png"><source media="(prefers-color-scheme: light)" srcset="https://cursor.com/assets/images/open-in-web-light.png"><img alt="Open in Web" width="114" height="28" src="https://cursor.com/assets/images/open-in-web-dark.png"></picture></a> <a href="https://cursor.com/background-agent?bcId=bc-58c0d343-1ef3-46d2-8ce5-e2704dc70d00&cursor_ref=pr_footer&cursor_cta=open_in_cursor"><picture><source media="(prefers-color-scheme: dark)" srcset="https://cursor.com/assets/images/open-in-cursor-dark.png"><source media="(prefers-color-scheme: light)" srcset="https://cursor.com/assets/images/open-in-cursor-light.png"><img alt="Open in Cursor" width="131" height="28" src="https://cursor.com/assets/images/open-in-cursor-dark.png"></picture></a> </div> --------- Co-authored-by: Cursor Agent <cursoragent@cursor.com>
![https://cursor.com/assets/images/open-in-web-dark.png"><source](https://cursor.com/assets/images/open-in-web-dark.png"><source){kind=link}
![https://cursor.com/assets/images/open-in-web-light.png"><img](https://cursor.com/assets/images/open-in-web-light.png"><img){kind=link}
![https://cursor.com/assets/images/open-in-web-dark.png"></picture></a> <a](https://cursor.com/assets/images/open-in-web-dark.png"></picture></a> <a){kind=link}
![https://cursor.com/assets/images/open-in-cursor-dark.png"><source](https://cursor.com/assets/images/open-in-cursor-dark.png"><source){kind=link}
![https://cursor.com/assets/images/open-in-cursor-light.png"><img](https://cursor.com/assets/images/open-in-cursor-light.png"><img){kind=link}
![https://cursor.com/assets/images/open-in-cursor-dark.png"></picture></a> </div](https://cursor.com/assets/images/open-in-cursor-dark.png"></picture></a> </div){kind=link}
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
pagesandpages/stovlmrun gw chatstats footer (stream,--no-stream, and--json).num_pages, single-quoted attributes, streaming-split tags, and<page>tag fallback when no count attribute is present.dim not bold) and show whole-second latency.Testing
pytest -sv tests/test_gateway.py(120 tests)vlmrun gw chat https://storage.googleapis.com/vlm-data-public-prod/hub/examples/finance.sec-filings/tsla-8k.pdf -m glm-ocrshowspages: 5 · pages/s: …in footerNote
Earlier commits only matched
pages="N"or used pypdfium2 for local files. glm-ocr URL-backed PDFs emitnum_pages="N", so document URLs showed token stats without page metrics until this fix.