Repository navigation
Resolve paper titles that differ from the catalog by a typo - #29
Merged
Merged
Conversation
Production chat asked get_paper_info for "BEIR: A Heterogeneous Benchmark ..." and got not_found, although the catalog holds the arXiv spelling "Heterogenous" (2104.08663) and keyword search returns it first. A full title (24+ normalized characters) now also resolves to a unique search result whose normalized title matches at a 0.95 similarity ratio with the same numbers, so typos and variant spellings resolve while sequels, other editions, and short names do not. Several near matches stay ambiguous. Replaying last week's 144 failed title lookups against the live catalog resolves BEIR and "DINO: Emerging Properties in Self-Supervised Vision Transformers" with no false matches; the rest are papers outside the catalog. Bumps pwc-mcp to 0.2.8. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
Production chat audit (2026-10-15):
get_paper_info("BEIR: A Heterogeneous Benchmark for Zero-shot Evaluation of Information Retrieval Models")returnednot_found. The catalog holds the arXiv spelling "Heterogenous" (2104.08663), which keyword search returns first, but title resolution only accepts exact titles or a short name.What
near_title(): for a full title (≥ 24 normalized characters), a search result counts as a near match when the normalized titles have aSequenceMatcherratio ≥ 0.95 and the same numbers (so "Llama 3" ≠ "Llama 4", and other editions or years don't match)._resolve_paperfalls back to a unique near match after exact and short-name matches. Several near matches → the existing "ambiguous" error. URL slugs never use it.Validation
uv run pytest -q testsinmcp_server: 71 passed, including new BEIR, ambiguous-twin, and short-name/edition rejection tests.Chat-related: stops after review. It ships via
pwc-deploy <env> <sha> --component mcponly with explicit authorization.🤖 Generated with Claude Code