Skip to content

chore: prepare packages for npm registry publishing#125

Merged
jithin23-kv merged 9 commits into
KeyValueSoftwareSystems:masterfrom
arunSunnyKVS:feat/npm-publish-setup
Jun 29, 2026
Merged

chore: prepare packages for npm registry publishing#125
jithin23-kv merged 9 commits into
KeyValueSoftwareSystems:masterfrom
arunSunnyKVS:feat/npm-publish-setup

Conversation

@arunSunnyKVS

@arunSunnyKVS arunSunnyKVS commented Jun 26, 2026

Copy link
Copy Markdown
Collaborator

Problem

The three runner packages (`@keyvaluesystems/agent-opfor-cli`, `@keyvaluesystems/agent-opfor-mcp`, `@keyvaluesystems/agent-opfor-sdk`) could not be published to npm. Three blockers:

  1. All runners declared `@keyvaluesystems/agent-opfor-core` as a runtime dependency — a wildcard resolved by npm workspaces locally but unresolvable from an external registry. Core is intentionally not published.
  2. The `evaluators/`, `suites/`, and `skills/` data directories were not included in runner packages. After installing from npm, `getRepoRoot()` in `evaluatorsLayout.ts` resolved to the wrong path, so evaluator discovery would fail at runtime.
  3. None of the scoped packages had `publishConfig: { access: "public" }`, which is required for `@scope/*` packages on npm.

Solution

Bundle core inline instead of resolving it at runtime

  • CLI and MCP runners use esbuild to produce a single `dist/index.js` that inlines all of core's compiled output. No runtime npm resolution of core needed.
  • SDK uses tsup (wraps esbuild + tsc declaration emit) to produce `dist/index.js` + `dist/index.d.ts`. tsup is used instead of esbuild because SDK consumers need TypeScript type declarations.
  • Core moved from `dependencies` → `devDependencies` in all three runners.

Ship data files with each runner

  • `prepack` scripts copy `evaluators/`, `suites/`, `atlas-data/`, and (for MCP/SDK) `data/` into each runner directory before packing.
  • `postpack` scripts clean them up after.
  • All three runners' `files` arrays updated to include these directories.

Fix `getRepoRoot()` path resolution

  • Added a one-level-up check so bundled runners (bundle at `dist/`) find data at the package root (`../`).
  • Changed the existence probe from `evaluators/` to `evaluators/agent/` to avoid a false-positive against `core/src/evaluators/` (a TypeScript source directory, not the data directory).

Pre-publish hardening

  • Added `publishConfig: { access: "public" }` to all three runners.
  • Clean builds: `rm -rf dist` before esbuild runs; `clean: true` in tsup config.
  • Created `README.md` for CLI and MCP runners (SDK already had one).
  • Added `LICENSE` copy in all three `prepack` scripts.
  • SDK `types.ts` now defines all types inline (no re-exports from core) so the published `.d.ts` has zero `@keyvaluesystems/agent-opfor-core` references.

Checklist

  • `npm run build` passes
  • `npm run typecheck` passes
  • `npm run lint` passes
  • `npm pack --dry-run` on all three runners shows correct file layout with no core references
  • `npm run build:catalog:check` passes (if evaluators or suites changed)
  • Tested against a local target (if evaluator added or changed)
  • No secrets, `.env`, or `.opfor/` artifacts committed
  • PR title follows `: `

Evaluator checklist (skip if no evaluator added)

N/A — no evaluators changed.

Summary by CodeRabbit

  • New Features

    • Updated package names and references across the CLI, SDK, MCP server, browser extension, and related docs.
    • Added bundled packaging and release support for published packages.
    • Added smoke testing for published package installs.
  • Bug Fixes

    • Improved package loading so runtime data works in both packaged and monorepo layouts.
    • Made ATLAS validation fail gracefully with a warning instead of stopping loading.

@coderabbitai

coderabbitai Bot commented Jun 26, 2026

Copy link
Copy Markdown

Review Change Stack

Warning

Review limit reached

@jithin23-kv, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 2 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 5c41b012-c016-49b1-92db-21cf55428067

📥 Commits

Reviewing files that changed from the base of the PR and between 73eb946 and c602203.

📒 Files selected for processing (1)
  • AGENTS.md

Walkthrough

Package names and workspace references were updated across the repo. Core packaging now stages runtime assets, resolves installed-package data paths, and adds a tarball smoke test.

Changes

Scoped package migration

Layer / File(s) Summary
Core packaging and smoke test
core/package.json, .gitignore, core/src/autonomous/*, core/src/browser.ts, core/src/catalog/*, core/src/config/*, core/src/report/types.ts, core/src/standards/atlas.ts, scripts/smoke-pack.ts
core package metadata, runtime asset lookup, ATLAS validation, repository layout probing, and tarball smoke validation are updated for installed-package use.
CLI imports and packaging
runners/cli/package.json, runners/cli/scripts/bundle.mjs, runners/cli/README.md, runners/cli/src/..., runners/cli/tests/*, runners/cli/ui/package.json
CLI package metadata, bundling, command/UI imports, and tests switch to the new core namespace and package names.
SDK imports and examples
runners/sdk/package.json, runners/sdk/README.md, runners/sdk/tsup.config.ts, runners/sdk/src/..., runners/sdk/tests/*, docs/sdk.md
SDK package metadata, build config, source imports/type references, tests, and examples switch to the new scoped package names.
MCP package and bootstrap
runners/mcp/package.json, runners/mcp/scripts/bundle.mjs, runners/mcp/README.md, runners/mcp/src/index.ts
MCP package metadata and bootstrap imports switch to the new scoped core package names.
Repo references and workflows
.github/*, AGENTS.md, CONTRIBUTING.md, README.md, docs/browser-extension.md, docs/cli.md, docs/mcp.md, package.json, runners/extension/*, tests/e2e/*
Repo templates, workflows, docs, root scripts, extension metadata, and e2e package names switch to the new scoped names.

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~45 minutes

Suggested reviewers

  • jithin23-kv
🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 15.79% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title is concise, specific, and accurately summarizes the main publishing-prep changes.
Description check ✅ Passed The description follows the required template and includes the problem, solution, checklist, and evaluator section.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@arunSunnyKVS arunSunnyKVS changed the title feat: npm publish setup chore: prepare packages for npm registry publishing Jun 26, 2026

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (2)
core/package.json (1)

89-94: 📐 Maintainability & Code Quality | 🔵 Trivial

Clarify cross-platform scope of prepack hooks

The prepack script uses POSIX-only commands (cp -r, mkdir -p, rm -rf) that will fail on Windows (cmd.exe/PowerShell). All CI workflows (.github/workflows/*.yml) execute on ubuntu-latest, so automated publishing is safe. However, if maintaining expects to publish locally from Windows, this will block release.

  • If local publishing on Windows is not in scope, add a note in CONTRIBUTING.md or README.md stating that publishing requires a POSIX environment.
  • If Windows support is expected, replace the shell commands with a cross-platform Node script or use shx/cpy as proposed.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@core/package.json` around lines 89 - 94, The prepack/postpack hooks in the
package.json scripts rely on POSIX-only shell commands, so clarify the intended
publishing environment. If Windows local publishing is not supported, add
documentation in CONTRIBUTING.md or README.md stating that the publish flow must
run in a POSIX shell; if it is supported, replace the current prepack/postpack
command chain with a cross-platform implementation using a Node script or
helpers like shx/cpy.
core/src/catalog/loadEvaluatorCatalog.ts (1)

119-124: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Make the ATLAS warning actionable.

The new fallback is fine, but the emitted warning still doesn't tell the user what to do next. Include the concrete remediation in the log itself (for example: bundle/install the ATLAS data correctly, or set OPFOR_VALIDATE_ATLAS=0 if they intentionally want to skip validation).

Suggested change
-      log.warn(`Skipping ATLAS technique-id validation: ${msg.split("\n")[0]}`);
+      log.warn(
+        `Skipping ATLAS technique-id validation: ${msg.split("\n")[0]}. ` +
+          `Ensure the ATLAS data is packaged with `@keyvaluesystems/agent-opfor-core`, ` +
+          `or set OPFOR_VALIDATE_ATLAS=0 to disable this validation.`
+      );

Based on learnings, error messages must be actionable and tell the user what to fix, not just that something failed.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@core/src/catalog/loadEvaluatorCatalog.ts` around lines 119 - 124, The warning
in loadEvaluatorCatalog’s ATLAS validation catch block is too vague and should
tell users what to do next. Update the log message emitted by the catch path so
it includes an explicit remediation, such as bundling/installing the ATLAS data
correctly or setting OPFOR_VALIDATE_ATLAS=0 to intentionally skip validation.
Keep the existing fallback behavior in loadEvaluatorCatalog, but make the log
text actionable by referencing the ATLAS data availability and the
OPFOR_VALIDATE_ATLAS flag directly.

Source: Learnings

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@core/package.json`:
- Around line 93-94: The package staging in the prepack/postpack scripts is not
idempotent: if a prior pack is interrupted, the existing staged directories will
cause cp -r to nest content instead of replacing it. Update the prepack flow in
package.json to clear the staging targets before copying, so evaluators, suites,
skills, data, and atlas-data are always recreated cleanly and cannot accumulate
nested directories after a failed pack.

---

Nitpick comments:
In `@core/package.json`:
- Around line 89-94: The prepack/postpack hooks in the package.json scripts rely
on POSIX-only shell commands, so clarify the intended publishing environment. If
Windows local publishing is not supported, add documentation in CONTRIBUTING.md
or README.md stating that the publish flow must run in a POSIX shell; if it is
supported, replace the current prepack/postpack command chain with a
cross-platform implementation using a Node script or helpers like shx/cpy.

In `@core/src/catalog/loadEvaluatorCatalog.ts`:
- Around line 119-124: The warning in loadEvaluatorCatalog’s ATLAS validation
catch block is too vague and should tell users what to do next. Update the log
message emitted by the catch path so it includes an explicit remediation, such
as bundling/installing the ATLAS data correctly or setting
OPFOR_VALIDATE_ATLAS=0 to intentionally skip validation. Keep the existing
fallback behavior in loadEvaluatorCatalog, but make the log text actionable by
referencing the ATLAS data availability and the OPFOR_VALIDATE_ATLAS flag
directly.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: cd2fbac3-711d-40bc-b80d-a70342e594bf

📥 Commits

Reviewing files that changed from the base of the PR and between 4ee277a and 5e2b43f.

⛔ Files ignored due to path filters (2)
  • package-lock.json is excluded by !**/package-lock.json
  • runners/cli/ui/package-lock.json is excluded by !**/package-lock.json
📒 Files selected for processing (61)
  • .github/ISSUE_TEMPLATE/bug_report.yml
  • .github/workflows/ci.yml
  • .github/workflows/release.yml
  • .gitignore
  • AGENTS.md
  • CONTRIBUTING.md
  • README.md
  • core/package.json
  • core/src/autonomous/index.ts
  • core/src/autonomous/knowledge/load.ts
  • core/src/autonomous/lib/types.ts
  • core/src/browser.ts
  • core/src/catalog/loadEvaluatorCatalog.ts
  • core/src/config/evaluatorsLayout.ts
  • core/src/report/types.ts
  • core/src/standards/atlas.ts
  • docs/browser-extension.md
  • docs/cli.md
  • docs/mcp.md
  • docs/sdk.md
  • package.json
  • runners/cli/package.json
  • runners/cli/src/commands/hunt.ts
  • runners/cli/src/commands/run.ts
  • runners/cli/src/commands/setup.ts
  • runners/cli/src/ui/bridge.ts
  • runners/cli/src/ui/demoData.ts
  • runners/cli/src/ui/server.ts
  • runners/cli/src/ui/snapshot.ts
  • runners/cli/tests/fork.test.ts
  • runners/cli/tests/http.test.ts
  • runners/cli/tests/knowledge.test.ts
  • runners/cli/tests/leads.test.ts
  • runners/cli/tests/observe.test.ts
  • runners/cli/tests/progress.test.ts
  • runners/cli/tests/recordFinding.test.ts
  • runners/cli/tests/runlog.test.ts
  • runners/cli/tests/sendToTarget.test.ts
  • runners/cli/ui/package.json
  • runners/extension/README.md
  • runners/extension/package.json
  • runners/mcp/package.json
  • runners/mcp/src/index.ts
  • runners/sdk/README.md
  • runners/sdk/package.json
  • runners/sdk/src/catalog.ts
  • runners/sdk/src/hunt.ts
  • runners/sdk/src/run.ts
  • runners/sdk/src/types.ts
  • runners/sdk/tests/catalog.test.ts
  • runners/sdk/tests/hunt.test.ts
  • runners/sdk/tests/opfor-class.test.ts
  • runners/sdk/tests/report.test.ts
  • runners/sdk/tests/run.test.ts
  • scripts/smoke-pack.ts
  • tests/e2e/agents/customer-support/Dockerfile
  • tests/e2e/agents/customer-support/package.json
  • tests/e2e/agents/vanilla-chat/Dockerfile
  • tests/e2e/agents/vanilla-chat/package.json
  • tests/e2e/agents/vulnerable-memory/package.json
  • tests/e2e/mcp/vulnerable-server/package.json

Comment thread core/package.json Outdated
arunSunnyKVS and others added 6 commits June 29, 2026 15:23
- Remove private:true from @agent-opfor/core so it can be published
- Add prepack/postpack to core to bundle data dirs (evaluators, suites, skills)
- Update getRepoRoot() to detect published vs monorepo context via existsSync
- Pin @agent-opfor/core from "*" to "^0.9.0" in cli, mcp, sdk runners
- Wire release-please config-file and manifest-file in release workflow

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The scope rename updated each test agent's package.json name but left
their package-lock.json self-name stale. Sync the lockfile name fields
to @keyvaluesystems/agent-opfor-* so each lock matches its package.json.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@core/package.json`:
- Line 6: The core package is still marked private, which prevents publishing
the package. Remove the private flag from the package.json for
`@keyvaluesystems/agent-opfor-core` so the publish flow can proceed, and keep the
rest of the package metadata unchanged.

In `@runners/sdk/package.json`:
- Around line 16-28: The SDK package manifest is staging catalog assets
incompletely, so add the missing skills/ tree to the package contents. Update
the packaging logic in runners/sdk/package.json, specifically the prepack and
matching postpack cleanup, so the published tarball includes skills/ alongside
data/, evaluators/, suites/, and atlas-data/. This should align with
runners/sdk/src/catalog.ts and loadSkillCatalog so installed SDKs can resolve
skill-catalog calls.

In `@runners/sdk/src/types.ts`:
- Around line 351-449: The SDK types refactor removed a previously exported
public type, so restore `UnifiedRunReport` in `runners/sdk/src/types.ts`
alongside the other inlined exports. Make sure the `types.ts` surface still
re-exports or defines `UnifiedRunReport` from the same SDK entrypoint so
existing consumers keep compiling without changing imports.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 8ccfe33e-74c3-4424-bcfa-e4e4a8641e95

📥 Commits

Reviewing files that changed from the base of the PR and between 5e2b43f and 66eaea6.

⛔ Files ignored due to path filters (5)
  • package-lock.json is excluded by !**/package-lock.json
  • tests/e2e/agents/customer-support/package-lock.json is excluded by !**/package-lock.json
  • tests/e2e/agents/vanilla-chat/package-lock.json is excluded by !**/package-lock.json
  • tests/e2e/agents/vulnerable-memory/package-lock.json is excluded by !**/package-lock.json
  • tests/e2e/mcp/vulnerable-server/package-lock.json is excluded by !**/package-lock.json
📒 Files selected for processing (15)
  • core/package.json
  • core/src/config/evaluatorsLayout.ts
  • package.json
  • runners/cli/README.md
  • runners/cli/package.json
  • runners/cli/scripts/bundle.mjs
  • runners/cli/src/index.ts
  • runners/mcp/README.md
  • runners/mcp/package.json
  • runners/mcp/scripts/bundle.mjs
  • runners/mcp/src/index.ts
  • runners/sdk/README.md
  • runners/sdk/package.json
  • runners/sdk/src/types.ts
  • runners/sdk/tsup.config.ts
💤 Files with no reviewable changes (2)
  • runners/cli/src/index.ts
  • runners/mcp/src/index.ts
✅ Files skipped from review due to trivial changes (4)
  • runners/sdk/tsup.config.ts
  • runners/cli/README.md
  • runners/mcp/README.md
  • runners/sdk/README.md
🚧 Files skipped from review as they are similar to previous changes (1)
  • core/src/config/evaluatorsLayout.ts

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Caution

Inline review comments failed to post. This is likely due to GitHub's internal server error or limits when posting large numbers of comments. If you are seeing this consistently it is likely a permissions issue. Please check "Moderation" -> "Code review limits" under your organization settings.

Actionable comments posted: 3

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@core/package.json`:
- Line 6: The core package is still marked private, which prevents publishing
the package. Remove the private flag from the package.json for
`@keyvaluesystems/agent-opfor-core` so the publish flow can proceed, and keep the
rest of the package metadata unchanged.

In `@runners/sdk/package.json`:
- Around line 16-28: The SDK package manifest is staging catalog assets
incompletely, so add the missing skills/ tree to the package contents. Update
the packaging logic in runners/sdk/package.json, specifically the prepack and
matching postpack cleanup, so the published tarball includes skills/ alongside
data/, evaluators/, suites/, and atlas-data/. This should align with
runners/sdk/src/catalog.ts and loadSkillCatalog so installed SDKs can resolve
skill-catalog calls.

In `@runners/sdk/src/types.ts`:
- Around line 351-449: The SDK types refactor removed a previously exported
public type, so restore `UnifiedRunReport` in `runners/sdk/src/types.ts`
alongside the other inlined exports. Make sure the `types.ts` surface still
re-exports or defines `UnifiedRunReport` from the same SDK entrypoint so
existing consumers keep compiling without changing imports.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 8ccfe33e-74c3-4424-bcfa-e4e4a8641e95

📥 Commits

Reviewing files that changed from the base of the PR and between 5e2b43f and 66eaea6.

⛔ Files ignored due to path filters (5)
  • package-lock.json is excluded by !**/package-lock.json
  • tests/e2e/agents/customer-support/package-lock.json is excluded by !**/package-lock.json
  • tests/e2e/agents/vanilla-chat/package-lock.json is excluded by !**/package-lock.json
  • tests/e2e/agents/vulnerable-memory/package-lock.json is excluded by !**/package-lock.json
  • tests/e2e/mcp/vulnerable-server/package-lock.json is excluded by !**/package-lock.json
📒 Files selected for processing (15)
  • core/package.json
  • core/src/config/evaluatorsLayout.ts
  • package.json
  • runners/cli/README.md
  • runners/cli/package.json
  • runners/cli/scripts/bundle.mjs
  • runners/cli/src/index.ts
  • runners/mcp/README.md
  • runners/mcp/package.json
  • runners/mcp/scripts/bundle.mjs
  • runners/mcp/src/index.ts
  • runners/sdk/README.md
  • runners/sdk/package.json
  • runners/sdk/src/types.ts
  • runners/sdk/tsup.config.ts
💤 Files with no reviewable changes (2)
  • runners/cli/src/index.ts
  • runners/mcp/src/index.ts
✅ Files skipped from review due to trivial changes (4)
  • runners/sdk/tsup.config.ts
  • runners/cli/README.md
  • runners/mcp/README.md
  • runners/sdk/README.md
🚧 Files skipped from review as they are similar to previous changes (1)
  • core/src/config/evaluatorsLayout.ts
🛑 Comments failed to post (3)
core/package.json (1)

6-6: 🎯 Functional Correctness | 🔴 Critical | ⚡ Quick win

Drop private: true from the publishable core package.

Line 6 still marks @keyvaluesystems/agent-opfor-core as private, so npm publish will refuse to ship it. That blocks the release path this PR is setting up.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@core/package.json` at line 6, The core package is still marked private, which
prevents publishing the package. Remove the private flag from the package.json
for `@keyvaluesystems/agent-opfor-core` so the publish flow can proceed, and keep
the rest of the package metadata unchanged.
runners/sdk/package.json (1)

16-28: 🗄️ Data Integrity & Integration | 🟠 Major | ⚡ Quick win

Publish skills/ with the SDK package.

runners/sdk/src/catalog.ts imports loadSkillCatalog, but this package only stages data/, evaluators/, suites/, and atlas-data/. Installed SDK tarballs will miss the skills/ tree and fail on skill-catalog calls.

Suggested manifest/script fix
   "files": [
     "dist/",
     "data/",
     "evaluators/",
     "suites/",
+    "skills/",
     "atlas-data/"
   ],
@@
-    "prepack": "cp -r ../../evaluators ./evaluators && cp -r ../../suites ./suites && cp -r ../cli/data ./data && mkdir -p ./atlas-data && cp ../../third_party/atlas-data/dist/ATLAS.yaml ./atlas-data/ATLAS.yaml && cp ../../LICENSE ./LICENSE",
-    "postpack": "rm -rf ./evaluators ./suites ./data ./atlas-data ./LICENSE"
+    "prepack": "cp -r ../../evaluators ./evaluators && cp -r ../../suites ./suites && cp -r ../../skills ./skills && cp -r ../cli/data ./data && mkdir -p ./atlas-data && cp ../../third_party/atlas-data/dist/ATLAS.yaml ./atlas-data/ATLAS.yaml && cp ../../LICENSE ./LICENSE",
+    "postpack": "rm -rf ./evaluators ./suites ./skills ./data ./atlas-data ./LICENSE"
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

    "dist/",
    "data/",
    "evaluators/",
    "suites/",
    "skills/",
    "atlas-data/"
  ],
  "scripts": {
    "dev": "tsx src/index.ts",
    "build": "tsup",
    "typecheck": "tsc -p tsconfig.json --noEmit",
    "test": "tsx --test tests/*.test.ts",
    "prepack": "cp -r ../../evaluators ./evaluators && cp -r ../../suites ./suites && cp -r ../../skills ./skills && cp -r ../cli/data ./data && mkdir -p ./atlas-data && cp ../../third_party/atlas-data/dist/ATLAS.yaml ./atlas-data/ATLAS.yaml && cp ../../LICENSE ./LICENSE",
    "postpack": "rm -rf ./evaluators ./suites ./skills ./data ./atlas-data ./LICENSE"
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@runners/sdk/package.json` around lines 16 - 28, The SDK package manifest is
staging catalog assets incompletely, so add the missing skills/ tree to the
package contents. Update the packaging logic in runners/sdk/package.json,
specifically the prepack and matching postpack cleanup, so the published tarball
includes skills/ alongside data/, evaluators/, suites/, and atlas-data/. This
should align with runners/sdk/src/catalog.ts and loadSkillCatalog so installed
SDKs can resolve skill-catalog calls.
runners/sdk/src/types.ts (1)

351-449: 🗄️ Data Integrity & Integration | 🟠 Major | ⚡ Quick win

Keep exporting UnifiedRunReport from the SDK surface.

This refactor inlines several core types but drops UnifiedRunReport from runners/sdk/src/types.ts. That turns a namespace/publishing update into a breaking compile-time API removal for existing SDK consumers.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@runners/sdk/src/types.ts` around lines 351 - 449, The SDK types refactor
removed a previously exported public type, so restore `UnifiedRunReport` in
`runners/sdk/src/types.ts` alongside the other inlined exports. Make sure the
`types.ts` surface still re-exports or defines `UnifiedRunReport` from the same
SDK entrypoint so existing consumers keep compiling without changing imports.

@jithin23-kv jithin23-kv force-pushed the feat/npm-publish-setup branch from 66eaea6 to 4dae060 Compare June 29, 2026 10:37
@jithin23-kv jithin23-kv self-assigned this Jun 29, 2026
@jithin23-kv jithin23-kv self-requested a review June 29, 2026 11:17

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🧹 Nitpick comments (1)
scripts/smoke-pack.ts (1)

36-39: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Preserve command output and make these failures actionable.

Right now the smoke test collapses failures into generic messages, and the catch path drops execFileSync's stderr/stdout. When npm pack, npm install, or the clean-install probe fails, the operator does not get the failing command, output, or a fix path.

Suggested change
-function fail(msg: string): never {
-  console.error(`\n❌ Smoke test FAILED: ${msg}`);
+function fail(msg: string, fix: string): never {
+  console.error(`\n❌ Smoke test FAILED: ${msg}\n→ ${fix}`);
   process.exit(1);
 }
@@
-  if (!tarball) fail("could not determine packed tarball filename");
+  if (!tarball) {
+    fail(
+      "npm pack did not print a tarball filename.",
+      "Run `npm pack` in `core/` and inspect the final stdout line plus any `prepack` output."
+    );
+  }
@@
-  if (failures.length > 0) fail(`empty results for: ${failures.join(", ")}`);
+  if (failures.length > 0) {
+    fail(
+      `Smoke pack loaded empty results for: ${failures.join(", ")}`,
+      "Check `core/package.json` `files`/`exports` and confirm the runtime data directories are included in the tarball."
+    );
+  }
@@
 } catch (e: unknown) {
-  const msg = e instanceof Error ? e.message : String(e);
-  fail(msg);
+  const stderr =
+    typeof e === "object" && e && "stderr" in e
+      ? String((e as { stderr?: string | Buffer }).stderr ?? "").trim()
+      : "";
+  const msg = stderr || (e instanceof Error ? e.message : String(e));
+  fail(
+    msg,
+    "Inspect the failing npm/node command output above and verify `core/package.json` packaging hooks and exported runtime files."
+  );
 }

As per coding guidelines, “Write actionable error messages that tell the user how to fix the problem, not just that something failed.”

Also applies to: 53-54, 97-102

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@scripts/smoke-pack.ts` around lines 36 - 39, The smoke test failure handling
in fail() and the execFileSync catch paths is too generic and drops useful
stdout/stderr, so make these errors actionable by preserving the failing command
plus captured output when npm pack, npm install, or the clean-install probe
fails. Update the failure reporting in smoke-pack.ts so the catch block passes
through execFileSync diagnostics and the fail helper includes a clear fix hint
or next step, using the existing fail() and command-execution logic to locate
the changes.

Source: Coding guidelines

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@AGENTS.md`:
- Line 27: The fenced block in AGENTS.md is missing a language tag, which causes
markdownlint to flag it. Update that repo-tree fence to use a text language
identifier so the block is treated as plain text and passes lint; keep the
change limited to the fenced block in the docs.

In `@runners/sdk/src/types.ts`:
- Around line 351-449: The SDK public type surface in types.ts now omits
UnifiedRunReport, which breaks existing consumers importing it from the package.
Add the UnifiedRunReport type back into this file alongside the other inlined
public types and ensure it is exported from the SDK surface, using the existing
types.ts export section as the location to restore it. If you intentionally do
not want to expose it, then treat this as a breaking change and update the
package versioning/release notes accordingly.

---

Nitpick comments:
In `@scripts/smoke-pack.ts`:
- Around line 36-39: The smoke test failure handling in fail() and the
execFileSync catch paths is too generic and drops useful stdout/stderr, so make
these errors actionable by preserving the failing command plus captured output
when npm pack, npm install, or the clean-install probe fails. Update the failure
reporting in smoke-pack.ts so the catch block passes through execFileSync
diagnostics and the fail helper includes a clear fix hint or next step, using
the existing fail() and command-execution logic to locate the changes.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: ba4e6f24-b3ca-4173-84f1-b9ff8ed503f5

📥 Commits

Reviewing files that changed from the base of the PR and between 66eaea6 and 73eb946.

⛔ Files ignored due to path filters (6)
  • package-lock.json is excluded by !**/package-lock.json
  • runners/cli/ui/package-lock.json is excluded by !**/package-lock.json
  • tests/e2e/agents/customer-support/package-lock.json is excluded by !**/package-lock.json
  • tests/e2e/agents/vanilla-chat/package-lock.json is excluded by !**/package-lock.json
  • tests/e2e/agents/vulnerable-memory/package-lock.json is excluded by !**/package-lock.json
  • tests/e2e/mcp/vulnerable-server/package-lock.json is excluded by !**/package-lock.json
📒 Files selected for processing (67)
  • .github/ISSUE_TEMPLATE/bug_report.yml
  • .github/workflows/ci.yml
  • .github/workflows/release.yml
  • .gitignore
  • AGENTS.md
  • CONTRIBUTING.md
  • README.md
  • core/package.json
  • core/src/autonomous/index.ts
  • core/src/autonomous/knowledge/load.ts
  • core/src/autonomous/lib/types.ts
  • core/src/browser.ts
  • core/src/catalog/loadEvaluatorCatalog.ts
  • core/src/config/evaluatorsLayout.ts
  • core/src/report/types.ts
  • core/src/standards/atlas.ts
  • docs/browser-extension.md
  • docs/cli.md
  • docs/mcp.md
  • docs/sdk.md
  • package.json
  • runners/cli/README.md
  • runners/cli/package.json
  • runners/cli/scripts/bundle.mjs
  • runners/cli/src/commands/hunt.ts
  • runners/cli/src/commands/run.ts
  • runners/cli/src/commands/setup.ts
  • runners/cli/src/index.ts
  • runners/cli/src/ui/bridge.ts
  • runners/cli/src/ui/demoData.ts
  • runners/cli/src/ui/server.ts
  • runners/cli/src/ui/snapshot.ts
  • runners/cli/tests/fork.test.ts
  • runners/cli/tests/http.test.ts
  • runners/cli/tests/knowledge.test.ts
  • runners/cli/tests/leads.test.ts
  • runners/cli/tests/observe.test.ts
  • runners/cli/tests/progress.test.ts
  • runners/cli/tests/recordFinding.test.ts
  • runners/cli/tests/runlog.test.ts
  • runners/cli/tests/sendToTarget.test.ts
  • runners/cli/ui/package.json
  • runners/extension/README.md
  • runners/extension/package.json
  • runners/mcp/README.md
  • runners/mcp/package.json
  • runners/mcp/scripts/bundle.mjs
  • runners/mcp/src/index.ts
  • runners/sdk/README.md
  • runners/sdk/package.json
  • runners/sdk/src/catalog.ts
  • runners/sdk/src/hunt.ts
  • runners/sdk/src/run.ts
  • runners/sdk/src/types.ts
  • runners/sdk/tests/catalog.test.ts
  • runners/sdk/tests/hunt.test.ts
  • runners/sdk/tests/opfor-class.test.ts
  • runners/sdk/tests/report.test.ts
  • runners/sdk/tests/run.test.ts
  • runners/sdk/tsup.config.ts
  • scripts/smoke-pack.ts
  • tests/e2e/agents/customer-support/Dockerfile
  • tests/e2e/agents/customer-support/package.json
  • tests/e2e/agents/vanilla-chat/Dockerfile
  • tests/e2e/agents/vanilla-chat/package.json
  • tests/e2e/agents/vulnerable-memory/package.json
  • tests/e2e/mcp/vulnerable-server/package.json
💤 Files with no reviewable changes (1)
  • runners/cli/src/index.ts
✅ Files skipped from review due to trivial changes (40)
  • runners/cli/ui/package.json
  • runners/sdk/tests/opfor-class.test.ts
  • runners/mcp/README.md
  • runners/cli/tests/knowledge.test.ts
  • runners/sdk/tests/hunt.test.ts
  • core/src/browser.ts
  • docs/cli.md
  • .gitignore
  • .github/ISSUE_TEMPLATE/bug_report.yml
  • runners/cli/tests/progress.test.ts
  • runners/extension/README.md
  • docs/browser-extension.md
  • core/package.json
  • runners/cli/tests/leads.test.ts
  • runners/cli/tests/runlog.test.ts
  • runners/cli/tests/recordFinding.test.ts
  • runners/cli/tests/http.test.ts
  • core/src/report/types.ts
  • runners/sdk/src/catalog.ts
  • runners/sdk/README.md
  • tests/e2e/agents/customer-support/Dockerfile
  • runners/mcp/scripts/bundle.mjs
  • tests/e2e/agents/vanilla-chat/package.json
  • core/src/autonomous/index.ts
  • tests/e2e/agents/customer-support/package.json
  • runners/extension/package.json
  • runners/cli/src/ui/bridge.ts
  • runners/sdk/tests/run.test.ts
  • CONTRIBUTING.md
  • README.md
  • runners/sdk/src/hunt.ts
  • core/src/autonomous/lib/types.ts
  • runners/mcp/src/index.ts
  • tests/e2e/agents/vulnerable-memory/package.json
  • tests/e2e/mcp/vulnerable-server/package.json
  • docs/mcp.md
  • runners/cli/README.md
  • runners/cli/src/commands/setup.ts
  • docs/sdk.md
  • runners/sdk/src/run.ts
🚧 Files skipped from review as they are similar to previous changes (23)
  • runners/cli/tests/fork.test.ts
  • runners/sdk/tests/catalog.test.ts
  • .github/workflows/release.yml
  • runners/cli/src/ui/demoData.ts
  • core/src/autonomous/knowledge/load.ts
  • tests/e2e/agents/vanilla-chat/Dockerfile
  • runners/cli/src/commands/run.ts
  • runners/cli/src/ui/snapshot.ts
  • runners/cli/src/ui/server.ts
  • runners/cli/scripts/bundle.mjs
  • runners/cli/src/commands/hunt.ts
  • runners/cli/tests/observe.test.ts
  • .github/workflows/ci.yml
  • runners/cli/tests/sendToTarget.test.ts
  • runners/sdk/tsup.config.ts
  • runners/sdk/tests/report.test.ts
  • runners/sdk/package.json
  • core/src/standards/atlas.ts
  • core/src/config/evaluatorsLayout.ts
  • core/src/catalog/loadEvaluatorCatalog.ts
  • package.json
  • runners/cli/package.json
  • runners/mcp/package.json

Comment thread AGENTS.md Outdated
Comment thread runners/sdk/src/types.ts
@jithin23-kv jithin23-kv merged commit 852fcd4 into KeyValueSoftwareSystems:master Jun 29, 2026
7 checks passed
@jithin23-kv jithin23-kv deleted the feat/npm-publish-setup branch June 29, 2026 12:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants