Skip to content

docs(testing): map CCTV and imported vector coverage - #498

Merged
kixelated merged 4 commits into
mainfrom
quest/m3/confidence/testing-vectors-rows
Nov 8, 2026
Merged

kixelated merged 4 commits into
mainfrom
quest/m3/confidence/testing-vectors-rows

Conversation

@kixelated

@kixelated kixelated commented Nov 7, 2026 •

Copy link
Copy Markdown
Collaborator

Finishes quest/m3/confidence/testing-vectors-rows.md.

Problem

The coverage table omits the CCTV and aws-lc/BoringSSL suites. Readers cannot tell which primitives or CPU tiers those vectors check.

Approach

  • Map every suite's actual test consumers into the primitive rows, including key encodings, and explain partial upstream coverage and backend-specific tests.
  • Distinguish the 10,000 default ML-KEM accumulation cases from the million-case native, denied-tier, and portable runs in build.yml.
  • Extend admin/testing to require the integration suite tests and the aws_lc, boringssl, and cctv unit tests in the Vectors column, and reject stale test references.
  • Remove the completed quest and its epic link. Preserve the portable tier and mid::portable paths.

Impact

  • No public API, wire, cryptographic behavior, vector data, or workflow change.

Alternatives

  • Map the suites by their test consumers instead of duplicating every upstream filename. The consumers show which operations actually run in mixed algorithm files and reuse the table's existing function-reference format.

Choices made without asking

  • Check consumer names, following the existing trace-test prefix convention, rather than add a second file inventory or parse Rust control flow.
  • Document the existing slow-suite scope; do not expand CI or change cryptography.

Validation

  • quest gate just ci origin/main: passed (coverage map, 18 benchmark-tool tests, quest structure, shell and Nix checks). The scoped gate selected no Rust build.
  • python3 .tmp/check-testing.py: passed six negative probes for omitted integration/unit consumers, stale names, and the wrong column, then the complete table.
  • quest check --base origin/main and git diff --check: passed.
  • Compared against origin/main: all tier rows and every non-vector primitive column are unchanged.
  • After merging origin/main (conflict-free): quest gate nix develop --command just check passed, before and after the review fix.
  • Local review via the Claude subscription gateway, recorded under ## Review: round 1 found mod.rs vector tests named with a stray mod segment, fixed in aab9402; a probe adding cctv_probe to rind/src/mid/mod.rs and listing mid::cctv_probe fails before the fix and passes after. Round 2 reports no findings.
  • Coverage statements traced to rind/tests/{aws_lc,cctv}.rs, their unit-test consumers, build.yml, x86-cpu.yml, sh/armv7.sh, and sh/portable.sh.

Caused by

  • moq-dev/rind#481 - added the imported suites while the coverage-table PR was open, leaving these rows for this quest.

Follow-ups

None.

Models

  • Size: XS
  • Implementer: Codex, gpt-6.1-sol, xhigh effort

(Written by GPT-6.1 Sol with xhigh effort)

Review

Rewritten by quest review after each round.

  • Reviewer: Claude Code, claude-opus-5-5, low effort
  • Session: 63d9f44f-4e16-4d12-9ebd-16c340a01f10, round 2
  • Head: aab9402
  • Base: origin/main at ea5e690
  • Score: 9/10, The one finding is fixed, and the only remaining change is a one-line correction to the CI checker.
  • Findings: none
  • The last round's findings, at aadbbc7:
    • Fixed: medium, admin/testing:149: Expected test names for mod.rs files keep a `mod` segment that table entries never have; At admin/testing:149 the script now uses the file's parent directory when the file is `mod.rs`, so a test in `a/b/mod.rs` is expected as `a::b::FN`, which matches the table entry.

kixelated and others added 2 commits November 7, 2026 14:25
Co-Authored-By: GPT-6.1 Sol <noreply@openai.com>
@kixelated kixelated changed the title TESTING.md lists the CCTV and imported vector sets docs(testing): map CCTV and imported vector coverage Nov 7, 2026
@kixelated
kixelated marked this pull request as ready for review November 8, 2026 19:08
@coderabbitai

coderabbitai Bot commented Nov 8, 2026

Copy link
Copy Markdown

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 26 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available.

Learn how review limits work.

Review configuration:

Run configuration

  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: c16526a5-ddda-4c8f-831a-71a4dba5f5c4
Commits

Reviewing files that changed from the base of the PR and between ea5e690 and aab9402.

Files selected for processing: 4

  • TESTING.md
  • admin/testing
  • quest/m3/confidence/README.md
  • quest/m3/confidence/testing-vectors-rows.md
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ No new issues found.

Reviewed changes

Reviewed the full PR: the TESTING.md vector-coverage documentation, the admin/testing checker extension, and the quest bookkeeping. Every factual claim was traced against the code and CI recipes, and admin/testing was run locally (passes, exit 0).

  • TESTING.md "Vector suites" section: maps rind/tests/aws_lc.rs and rind/tests/cctv.rs plus the aws_lc/boringssl/cctv-prefixed unit tests to what they check, including the ML-KEM accumulation counts (10,000 default, 1,000,000 with SLOW_TESTS=1, matching high::kem::cctv_accumulated).
  • Primitives table rows: each primitive's Vectors column now names its imported-suite consumers — all nine aws_lc:: tests, cctv::ed25519_edge_cases, and the prefixed unit tests exist where the table says they do; the Poly1305 and P-256 point-addition backend gating matches the cfg attributes in low/mod.rs and low/x86_64/ecp_nistz256.rs.
  • Slow-tier scope statements: verified against build.yml (nightly SLOW_TESTS=1 native release, denied x86_64/aarch64 tiers, and just portable), sh/armv7.sh and x86-cpu.yml (release without SLOW_TESTS, i.e. default 10,000).
  • admin/testing: the new vector_suites/vector_tests checks validate aws_lc::FN/cctv::FN tokens against rind/tests/{aws_lc,cctv}.rs, require every prefixed unit test to appear in the Vectors column, reject tokens placed in the wrong column, and the mod.rs path-derivation fix in aab9402 is correct.
  • Quest bookkeeping: removes the completed quest file and its epic link from quest/m3/confidence/README.md.

Pullfrog  | View workflow run | Using glm-5.3 | 𝕏

@kixelated
kixelated added this pull request to the merge queue Nov 8, 2026
Merged via the queue into main with commit 077605e Nov 8, 2026
12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant