Skip to content

Sync chat's vendored PwC MCP skill to pwc-mcp 0.2.7 - #517

Merged
NielsRogge merged 2 commits into
mainfrom
claude/sync-mcp-skill-0.2.7
Oct 15, 2026
Merged

NielsRogge merged 2 commits into
mainfrom
claude/sync-mcp-skill-0.2.7

Conversation

@NielsRogge

@NielsRogge NielsRogge commented Oct 14, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

  • Vendors huggingface/pwc-cli mcp_server/SKILL.md at a2fc9d0 (Use the public-metadata API for papers outside the catalog pwc-cli#28, pwc-mcp 0.2.7) into backend/chat_sandbox_worker/PWC_MCP_SKILL.md, and updates the pin comment in mcp_research_prompt.ts.
  • New skill text: papers not indexed on Papers With Code return in_catalog: false, a note, and available_at links (via Serve public metadata for papers outside the catalog #508's public-metadata endpoint), and the agent must cite those sources rather than a PwC paper page. It also brings the 0.2.1–0.2.7 tool docs (hybrid default search, paginated get_benchmark, search on task/method lists, etc.).
  • Additional change: the new skill tells the agent to use get_paper_evaluations. The live MCP server already exposes that tool (confirmed with tools/list on paperswithcode.co/mcp), but the chat allowlist didn't include it, so the prompt would have named a tool the chat can't call. This PR adds it to the gateway TOOL_NAMES and Pi PWC_MCP_TOOLS (20 → 21 tools), maps pwc paper evaluations to it, and updates the catalog-usage prompt and docs/chat-pwc-mcp.md. Because discovery fails closed, this must not ship to an environment whose MCP server lacks get_paper_evaluations; production already has it.
  • No sync workflow or hash check pins this file. sync-pwc-cli-skill.yml only syncs PWC_CLI_SKILL.md, and test_pi_prompt_assets.py compares the published file against the vendored one, not against a fixed hash.

Tests

  • pytest backend/tests/chat_gateway/test_pwc_mcp.py backend/tests/test_pi_prompt_assets.py backend/tests/chat_sandbox/test_prompts_and_catalogs.py: 29 passed.
  • On the wider set (chat_gateway/, external_pi/test_runtime.py, paper_edits/test_policy.py), 32 tests fail in test_turn_fence.py and external_pi/test_runtime.py. The same set fails on a clean origin/main checkout in this environment.
  • pi_worker npm test: 199 pass. 2 tests fail ("native MCP …" web search), and they fail the same way on origin/main.
  • I checked by hand that adaptMcpSkill renders pwc_get_paper_evaluations and the available_at guidance.

Chat-related: do not merge or deploy without explicit approval.

🤖 Generated with Claude Code


Note

Medium Risk
Expands the fixed MCP allowlist and prompt contract; discovery fails closed if an environment’s MCP server lacks get_paper_evaluations, and off-catalog citation rules change agent behavior for missing papers.

Overview
Updates the vendored PwC MCP skill to pwc-mcp v0.2.7 (new upstream pin in mcp_research_prompt.ts) so chat prompts match current tool behavior: hybrid default search, paginated get_benchmark, search on task/method lists, softer handling of unknown leaderboard metrics, and richer paper lookup (short titles, off-catalog in_catalog: false with available_at links instead of PwC paper URLs).

get_paper_evaluations is added to the gateway and Pi tool allowlists (20 → 21 tools) with CLI mapping and catalog-usage text, so agents are not instructed to call a tool discovery would reject. Prompt adaptation drops upstream guidance to pass "next" before the first read_paper read, since chat resolves cursors differently.

Docs and tests reflect the new tool count and prompt strings (available_at, pwc_get_paper_evaluations).

Reviewed by Cursor Bugbot for commit 60b5700. Bugbot is set up for automated code reviews on this repo. Configure here.

Vendor huggingface/pwc-cli mcp_server/SKILL.md at a2fc9d0 (PR #28). Papers
not indexed on Papers With Code now come back with in_catalog=false, a note,
and available_at links, and the skill tells the agent to cite those sources
rather than a Papers With Code paper page.

The new skill also directs the agent to get_paper_evaluations, which the live
MCP server already exposes, so allowlist it in the gateway and Pi worker and
map `pwc paper evaluations` to it; otherwise the prompt would name a tool the
chat cannot call.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using default effort and found 2 potential issues.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 76df4c5. Configure here.

Comment thread backend/chat_sandbox_worker/pi_worker/src/mcp_research_prompt.ts Outdated
Comment thread backend/chat_sandbox_worker/PWC_MCP_SKILL.md
The catalog overlay and state-of-the-art procedure now describe
pwc_get_paper_info as compact by default, with include_resources for
artifacts, pwc_get_paper_evaluations for benchmark results, and
in_catalog=false papers cited via available_at. adaptMcpSkill drops the
upstream advice to pass cursor "next" on a first read_paper, which chat
resolves from a saved token and rejects when none exists.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@NielsRogge
NielsRogge merged commit 3ba85d7 into main Oct 15, 2026
4 checks passed
@NielsRogge
NielsRogge deleted the claude/sync-mcp-skill-0.2.7 branch October 15, 2026 07:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant