Skip to content

fix(rosettify-prompts): reject duplicate suite IDs - #362

Merged
isolomatov-gd merged 1 commit into
griddynamics:mainfrom
shin4141:codex/v32-rosetta-352-suite-ids
Sep 29, 2026
Merged

isolomatov-gd merged 1 commit into
griddynamics:mainfrom
shin4141:codex/v32-rosetta-352-suite-ids

Conversation

@shin4141

@shin4141 shin4141 commented Sep 29, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Fixes #352.

Reject repeated suites[].id values during config parsing, before benchmark runs. The change keeps variant IDs scoped to their suites and documents the uniqueness rule in the package README.

Problem and expected behavior

Reports group runs by (suiteId, variantId). The previous parser accepted two suites with the same ID. With the same variant ID in both, each summary included both suites' runs and the first sample reply appeared in both sections. Issue #352 explicitly asks for repeated suite IDs to be rejected before execution; it does not ask the report to invent another identity for them.

Change

  • Check suite IDs for duplicates in parseConfig and report the duplicated ID.
  • Add a regression for two suites with the same ID and a control showing that distinct suites may reuse a variant ID.
  • State the suite-ID uniqueness requirement in src/rosettify-prompts/README.md.

This is a three-file change with 28 added lines and no removed lines. Configs that previously used duplicate suite IDs must rename those suites before running again.

Validation and limits

  • Before dependency installation, an exact-file snapshot reproduced the original mixed summaries and repeated first reply without an API call. The new config regression failed on unmodified main while the other 21 focused tests passed; the patch passed all 22 using locally available compatibility dependencies. This was an earlier, limited check, separate from the lockfile validation below.
  • On a full checkout of the submitted commit cb0fe59ee4969fe222831e41a693cdf796e17e18, npm ci installed the package lockfile's Zod 3.25.76, Vitest 4.1.10, and TypeScript 5.9.3. With Node 26.4.0 and npm 11.17.0, npm test passed all 98 tests in 9 files; npm run typecheck and npm run build passed. The worktree remained clean after validation, and the commit's three-file diff matched the proposed patch exactly.
  • The root src/validate-types.sh passed for the installed package. The repository's full venv/bin/python scripts/pre_commit.py also passed: hooks build, standard and lightweight plugin sync, Python type validation and tests, hooks tests (1,441 in 35 files), and rosettify-prompts tests (98 in 9 files). Its script skipped the unrelated curiocity, rosettify, and rosettify-plugins TypeScript checks/tests because their node_modules were not installed; this is not a claim of repository-wide coverage.
  • Public CI for this same head passed: CI Rosettify Prompts ran npm ci, typecheck, build, and tests on Node 24; CodeQL Advanced succeeded.

The initial sandboxed npm request could not resolve the registry. A host-side CLI with a temporary cache completed the lockfile install; npm warned that esbuild and fsevents lifecycle scripts were not approved in that environment, but the checks above passed. No Anthropic or other paid model API was called.

AI assistance

Codex assisted with the private investigation, patch, and validation. The submitter authorized the DCO sign-off using the same identity as prior Rosetta contributions and remains responsible for the final diff.

Signed-off-by: Shin <128954611+shin4141@users.noreply.github.com>
@shin4141
shin4141 marked this pull request as ready for review September 29, 2026 03:19

@isolomatov-gd isolomatov-gd left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@shin4141 thank you. The duplicate-ID check sits at the right place, and both tests prove the rule. Approved.

@isolomatov-gd
isolomatov-gd merged commit 7dd0298 into griddynamics:main Sep 29, 2026
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[ROSETTA] Reject duplicate rosettify-prompts suite IDs

2 participants