Skip to content

test(perf): add Employment separation buyer-path workload - #317

Draft
seonghobae wants to merge 160 commits into
fix/hire-identity-type-integrityfrom
perf/employment-separation-buyer-path
Draft

test(perf): add Employment separation buyer-path workload#317
seonghobae wants to merge 160 commits into
fix/hire-identity-type-integrityfrom
perf/employment-separation-buyer-path

Conversation

@seonghobae

@seonghobae seonghobae commented Sep 13, 2026

Copy link
Copy Markdown
Contributor

Scope

This Draft PR owns executable performance-measurement and evidence-contract mechanics for #316 without changing People production semantics. It remains stacked on active People owner #64 rather than being retargeted merely to manufacture CI evidence.

Buyer-path workload

  • Uses right-cleared, non-synthetic fixture data bound to exact candidate SHA, clearance, preparation, prepared-state, and resource-evidence references.
  • Requires 1,000 ordinary profile records and 100 contention pairs.
  • Exercises the real POST /v1/employment-separations shape with governed workforce_admin purpose.
  • Runs exactly one profile per invocation: first_commit, replay, rejection, or contention.
  • Uses version-controlled k6 constant-arrival-rate load models. Ordinary profiles are currently 20 RPS × 50 s with 20 preallocated / 80 max VUs; contention is 10 pair-iterations/s × 10 s with 20 preallocated / 80 max VUs. These are reproducibility inputs, not capacity claims.
  • Applies p(95)<=20 only to ordinary first_commit; replay/rejection/contention remain separate distributions. The result records p50/p95/p99/max and exact sample cardinality.
  • Measures direct-client elapsed time as k6 blocked + duration; k6 v2.2.0 nests TCP/TLS acquisition inside blocked, so adding connecting and tls_handshaking again would double-count cold-connection latency.

Execution and local evidence integrity

  • Pins ghcr.io/grafana/k6 v2.2.0 by OCI digest sha256:9bd01d6941fca969cb61bb57d2da5ee9b385fe2aa8881df3798c196564d6ace6, verifies upstream container UID 12345, and runs with --pull=never.
  • Rejects caller-supplied k6 CLI/load overrides.
  • Requires a clean checkout whose HEAD equals ORGMETRA_PERFORMANCE_TARGET_SHA, streams git archive of that exact commit into an ephemeral Podman image, and mounts that image read-only at /workspace; the live host working tree is not the executed workload.
  • Stages the result in a private per-run directory, rejects stale destinations, publishes with a no-clobber hard link, and checks source/target device+inode+size+SHA-256 across publication.
  • Binds raw fixture/result/runtime-evidence bytes by SHA-256 and strict UTF-8 before JSON interpretation; runtime evidence independently records observed service SHA, load model, deployment/observer references, CPU/RSS, DB-pool behavior, and zero residual run-scoped HTTP/DB/transaction/socket/worker/pool state.
  • The runner still emits ORGMETRA_PERFORMANCE_RESULT_SHA256=<digest>, but that value is now explicitly structural evidence only. A value copied from caller-visible stdout is not an authenticated commercial trust root.

Commercial acceptance trust boundary — fail closed

CodeRabbit found a valid P1 on exact 108f1c0ee70eebfb0fa7d391aecb49a7927d912d: the attempted result-digest handoff still accepted the digest as a caller-supplied CLI argument. A caller able to replace the result could also calculate a replacement digest and construct matching runtime evidence. Equality between caller-controlled bytes and caller-controlled digest is integrity checking, not authentication.

The product-local lane therefore no longer claims that the digest token closes commercial provenance. The current repair is deliberately fail closed:

  • d2ab0b057dd81610c53326fe60a96f9a3b7595da adds the RED contract: replacing both result identity and local digest must not produce commercial acceptance.
  • 5b8b49746c148240e3078745c714d33533aa2951 adds an explicit authenticated-evidence gate.
  • 51da6578d279af8e25e0238f47d15e2a1d5b80af places that gate at the commercial acceptance entry point before pathname, digest, runtime, fixture, or latency evidence can be converted into a receipt. The CLI no longer accepts a caller-supplied runner digest as authority.
  • b6968520ef191fe8a3fd84d3cd5d4dc23a778281 corrects the runner documentation: its emitted digest is local structural evidence, not an immutable handoff authority.

The authenticated owner path is now central .github#2162, perf: attest exact product performance evidence before commercial acceptance. It requires an organization-owned immutable reusable workflow that re-hashes inert sealed result/runtime/fixture artifacts, binds exact caller repository/source SHA/run/artifact identity, signs a versioned performance-evidence predicate with GitHub/Sigstore, and verifies the signer workflow/OIDC/source/subject before a product may emit a positive commercial receipt. .github#1228 is a prerequisite warning: the existing reusable exact-artifact workflow's github.workflow_sha pattern is not a sound cross-repository reusable-workflow identity and must not be copied into this owner path.

Until the released central contract is consumed here, employment_separation_acceptance_check.mjs intentionally terminates non-zero with the .github#2162 owner requirement. Local validators remain useful for structural/schema/cardinality/latency/runtime consistency tests, but cannot establish commercial acceptance.

Verified earlier finding/repair lineage

The stack already repaired, test-first and ordinary-forward, false-GREEN/false-RED paths involving production-incompatible Idempotency-Key fixtures, hidden prepared DB state, wrong conflict response shape, mixed-profile load, missing p99, truncated samples, undersized cardinality, self-declared load models, coordinated omission, lossy UTF-8 artifacts, omitted transport time, double-counted TCP/TLS, substitutable k6 runner provenance, caller-controlled k6 CLI/load intensity, unbound checkout bytes, live-working-tree TOCTOU, stale-result reuse, non-root mount ownership, and staged-summary replacement during publication. The latest authenticated-attestation finding supersedes the earlier claim that a leaf-local digest token alone was a durable acceptance boundary.

Current authority

Exact performance head: b6968520ef191fe8a3fd84d3cd5d4dc23a778281.
Exact stacked base: #64 c0ba482c57a77820128f259b4a43c06df643c0f3.

No hosted exact-head k6 execution, production-equivalent deployment, authenticated performance attestation, runtime-cleanup observation, measured p50/p95/p99/max, protected status, or release status is claimed. A real buyer RED/GREEN still requires the exact-candidate ASGI composition, sanctioned Keyverse/auth path, production-equivalent PostgreSQL/pool settings, right-cleared prepared records, pinned k6 runtime, independent runtime/load observer, resource/cleanup evidence, and the authenticated .github#2162 evidence boundary. #316 remains open.

@coderabbitai

coderabbitai Bot commented Sep 13, 2026

Copy link
Copy Markdown

Review Change StackReview Change Stack

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

  • Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch perf/employment-separation-buyer-path

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

Copy link
Copy Markdown
Contributor Author

@coderabbitai review

Please review the exact current head for performance-evidence false-GREEN risk: right-cleared/non-synthetic fixture enforcement, cross-profile Employment isolation, contention semantics, bearer handling, p95 threshold scoping, result provenance, and any k6 runtime incompatibility. This PR must not claim measured acceptance.

@coderabbitai

coderabbitai Bot commented Sep 13, 2026

Copy link
Copy Markdown
🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -euo pipefail

echo '=== HEAD / PR FILES ==='
git rev-parse HEAD
git diff --name-status HEAD^ HEAD
git diff --check HEAD^ HEAD

echo '=== REPOSITORY MAP ==='
fd -HI -t f | rg -i '(k6|perf|performance|employment|fixture|workload|package\.json|readme|requirements)'

echo '=== JAVASCRIPT/TYPECRIPT OUTLINES ==='
while IFS= read -r file; do
  echo "--- $file"
  ast-grep outline "$file" --items all 2>/dev/null || true
done < <(fd -HI -t f -e js -e mjs -e cjs -e ts | rg -i '(k6|perf|performance|employment|fixture|workload)')

echo '=== DIFF ==='
git diff --find-renames --find-copies --unified=80 HEAD^ HEAD

Length of output: 3432


🏁 Script executed:

#!/bin/bash
set -euo pipefail

echo '=== OUTLINE: WORKLOAD ==='
ast-grep outline tests/performance/employment_separation_buyer_path.js --items all || true

echo '=== OUTLINE: FIXTURE CONTRACT ==='
ast-grep outline tests/performance/employment_separation_fixture_contract.mjs --items all || true

echo '=== OUTLINE: FIXTURE CONTRACT TEST ==='
ast-grep outline tests/performance/employment_separation_fixture_contract.test.mjs --items all || true

echo '=== WORKLOAD ==='
cat -n tests/performance/employment_separation_buyer_path.js

echo '=== FIXTURE CONTRACT ==='
cat -n tests/performance/employment_separation_fixture_contract.mjs

echo '=== FIXTURE CONTRACT TEST ==='
cat -n tests/performance/employment_separation_fixture_contract.test.mjs

echo '=== RELATED PERFORMANCE POLICY ==='
cat -n docs/adr/0009-performance-criterion-observation-scope.md

echo '=== ALL SHARED-CONTRACT IMPORTS AND CALL SITES ==='
rg -n -C 4 --glob '*.mjs' --glob '*.js' \
  '(from .*\bemployment_separation_fixture_contract\b|require\(.+employment_separation_fixture_contract|validateFixture|fixtureIdentity|candidateSha|resourceEvidence)' \
  tests services packages

Length of output: 37674


🏁 Script executed:

#!/bin/bash
set -euo pipefail

echo '=== PACKAGE METADATA ==='
cat -n package.json

echo '=== PERFORMANCE DOCUMENTATION AND EXECUTION REFERENCES ==='
rg -n -C 5 -i \
  '(employment[-_ ]separation.*(performance|buyer|k6)|ORGMETRA_PERFORMANCE_|k6 run|performance-result|candidate_sha|right_cleared)' \
  README.md docs tests package.json .github 2>/dev/null || true

echo '=== SERVICE ROUTE CONTRACT ==='
rg -n -C 8 --glob '*.py' --glob '*.md' --glob '*.json' \
  '(/v1/employment-separations|employment-separations|separation_conflict|X-Actor-Reference|X-Purpose-Code|X-Tenant-Reference)' \
  services/people-api packages tests | head -n 500

echo '=== K6 VERSION PINNING OR CONTAINER CONFIGURATION ==='
fd -HI -t f | rg -i '(dockerfile|compose|k6|tool-versions|mise|asdf|package-lock|pnpm-lock|yarn\.lock|github/workflows)'
rg -n -C 3 -i '(grafana/k6|k6(\s|:|@)|xk6)' .github README.md package.json Dockerfile docker-compose.yml 2>/dev/null || true

echo '=== CURRENT HEAD ID REFERENCES ==='
rg -n -C 3 '(2046bc4bea97cf8e0a11694fe3b707c23d5273df|ORGMETRA_PERFORMANCE_TARGET_SHA|candidate_sha)' .

Length of output: 54948


@seonghobae Review found one false-GREEN risk.

  • P1 — Require completion of every configured iteration before accepting the p95 result. tests/performance/employment_separation_buyer_path.js Lines 56-87 set maxDuration: "30m" for all shared-iterations scenarios. If a scenario reaches that limit, k6 can stop it before it processes every fixture record. The first-commit trend can then satisfy p(95)<=20 from the completed subset. The result envelope contains the k6 summary, but the workload has no threshold or explicit completion check for the required 1,000 first-commit, replay, and rejection records or the 100 contention pairs. Add per-scenario completed-iteration assertions, or fail handleSummary when the completed counts differ from the configured counts. This prevents timeout-truncated samples from appearing as an acceptance result.

The fixture contract otherwise enforces the requested controls: right-cleared and non-synthetic flags, full-SHA format and fixture/runtime SHA equality, non-empty evidence references, cross-profile Employment isolation, contention command equivalence except for idempotency key, runtime bearer credentials, and a first-commit-only p95 trend. The workload does not contain a measured acceptance claim.

⚠️ Action not completed

Head commit changed.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

Copy link
Copy Markdown
Contributor Author

Finding verification on current stack: the initial performance fixture validator accepted short idempotency keys, while the live People contract requires 16–200 visible ASCII characters. A real run with those fixtures would have returned request-validation 400s and produced a false performance RED. Repaired ordinary-forward in b5c25443713b769f6945dec62a300b97593a290e, then aligned the fixture with the remaining separation HTTP/application preconditions in 2d11255b2867e86ad587a74674133167c8652f6a + 093e8b200bff732a3dd67e9b76bfed10bb0add40. The final workload head 6dc5e2806b566ca8952eaf2d5bd65de6fa41d836 also keeps the target endpoint out of emitted evidence. Local contract result is now 9/9 PASS; this is harness evidence only, not a p95 claim.

Copy link
Copy Markdown
Contributor Author

@coderabbitai review

Please review exact head 6dc5e2806b566ca8952eaf2d5bd65de6fa41d836. Focus on false-GREEN/false-RED risk, k6 runtime compatibility, concurrency semantics, request-contract parity, credential/result leakage, sample cardinality, and whether the p95 threshold is scoped to the ordinary buyer path without hiding contention.

@coderabbitai

coderabbitai Bot commented Sep 13, 2026

Copy link
Copy Markdown

Tip

For best results, initiate chat on the files or code changes.

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -euo pipefail

target='6dc5e2806b566ca8952eaf2d5bd65de6fa41d836'
base='d9cc516d54b4642f59fe126c331a19945fdf75f1'

echo '=== VERIFY EXACT HEAD ==='
git rev-parse HEAD
git cat-file -t "$target"
git merge-base --is-ancestor "$base" "$target" && echo "base_is_ancestor=yes"
git diff --check "$base" "$target"

echo '=== CHANGED FILES ==='
git diff --name-status "$base" "$target"

echo '=== JAVASCRIPT OUTLINES ==='
git diff --name-only "$base" "$target" | while IFS= read -r f; do
  case "$f" in
    *.js|*.mjs|*.cjs)
      echo "--- $f"
      ast-grep outline "$f" --items all || true
      ;;
  esac
done

echo '=== PERFORMANCE HARNESS FILES AT TARGET ==='
fd -HI -t f . tests 2>/dev/null | rg 'tests/performance/(employment_separation|.*performance)' || true

echo '=== TARGET DIFF ==='
git diff --find-renames --unified=100 "$base" "$target" -- tests/performance docs package.json

echo '=== RELATED PRODUCTION BOUNDARY LOCATIONS ==='
rg -n -C 5 --glob '*.py' \
  '(Idempotency-Key|X-Actor-Reference|X-Purpose-Code|X-Tenant-Reference|Bearer |employment-separations|separation_reason|evidence_version|confirmation_reference)' \
  services packages tests | head -n 1000

echo '=== EXECUTION / VERSION REFERENCES ==='
rg -n -C 4 --glob '*.md' --glob '*.json' --glob '*.yml' --glob '*.yaml' --glob '*.js' --glob '*.mjs' \
  '(k6 run|grafana/k6|shared-iterations|maxDuration|handleSummary|thresholds|ORGMETRA_PERFORMANCE|candidate_sha|resource_evidence)' \
  README.md docs tests package.json .github 2>/dev/null || true

Length of output: 146825


🏁 Script executed:

#!/bin/bash
set -euo pipefail

echo '=== PEOPLE SEPARATION SOURCE MAP ==='
rg -n -l --glob '*.py' '(employment.separation|employment-separation|separation_conflict|separation_effective_on)' services/people-api | while IFS= read -r f; do
  echo "--- $f"
  ast-grep outline "$f" --items all 2>/dev/null || true
done

echo '=== PEOPLE SEPARATION HTTP AND COMMAND IMPLEMENTATIONS ==='
rg -n -C 12 --glob '*.py' \
  '(def .*separation|class .*Separation|/v1/employment-separations|separation_conflict|Idempotency-Key|idempotency.*(16|200)|evidence_version_code|confirmation_reference)' \
  services/people-api/src services/people-api/tests | head -n 1600

echo '=== PERFORMANCE CONTRACT TEST COMPLETE ==='
cat -n tests/performance/employment_separation_fixture_contract.test.mjs

echo '=== K6-SPECIFIC API PATTERNS IN REPOSITORY ==='
rg -n -C 5 --glob '*.js' --glob '*.mjs' \
  '(handleSummary|execution|iterationInTest|http\.batch|shared-iterations)' . 2>/dev/null

echo '=== COMMIT HISTORY FOR HARNESS ==='
git log --format='%H%n%s%n%b%n---' --reverse d9cc516d54b4642f59fe126c331a19945fdf75f1..6dc5e2806b566ca8952eaf2d5bd65de6fa41d836 -- tests/performance

Length of output: 210212


@seonghobae Review found one false-GREEN risk in exact head 6dc5e2806b566ca8952eaf2d5bd65de6fa41d836.

  • P1 — Fail the result when a scenario does not complete its configured iterations. tests/performance/employment_separation_buyer_path.js Lines 56-87 sets maxDuration: "30m" for each shared-iterations scenario. k6 can stop a scenario at that limit before it processes every fixture record. The first-commit trend can then satisfy p(95)<=20 from only the completed subset. The existing checks and employment_separation_unexpected_response thresholds only validate executed requests. They do not prove completion of 1,000 first-commit, replay, and rejection records, or 100 contention pairs. Add a per-scenario completion counter with an exact-count threshold, or make handleSummary() fail when each completed count differs from its configured iteration count.

The remaining reviewed controls are consistent with the stated harness scope. The fixture validator enforces right-cleared non-synthetic input, candidate-SHA equality, resource-evidence presence, cross-profile Employment isolation, and contention pair equivalence except for idempotency key. The workload supplies the bearer token only from the runtime environment. The result envelope does not explicitly contain the target endpoint. The p(95)<=20 threshold applies only to the ordinary first-commit trend. Contention remains a separate reported distribution with a 200 plus 409 outcome check.

You are interacting with an AI system.

Copy link
Copy Markdown
Contributor Author

@coderabbitai review

Please review exact head beff0210c24f5d179f8a6717fbfeb5881f7c5365. In addition to runtime/k6 and false-GREEN/false-RED risk, verify that replay/rejection prepared-state provenance is explicit enough to prevent hidden pre-materialization claims and that no endpoint/credential material is emitted into result evidence.

@coderabbitai

coderabbitai Bot commented Sep 13, 2026

Copy link
Copy Markdown

Tip

For best results, initiate chat on the files or code changes.

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -euo pipefail

target='beff0210c24f5d179f8a6717fbfeb5881f7c5365'
base='d9cc516d54b4642f59fe126c331a19945fdf75f1'

echo '=== EXACT-HEAD VERIFICATION ==='
printf 'checked_out='
git rev-parse HEAD
git cat-file -t "$target"
git merge-base --is-ancestor "$base" "$target" && echo 'base_is_ancestor=yes'
git diff --check "$base" "$target"

echo '=== CHANGED FILES ==='
git diff --name-status "$base" "$target"

echo '=== PERFORMANCE FILES AT TARGET ==='
git ls-tree -r --name-only "$target" | rg '^tests/performance/|(^|/)(package\.json|README\.md)$|^docs/' | rg -i '(employment|separation|performance|buyer|fixture|workload|package\.json|readme)' || true

echo '=== JAVASCRIPT OUTLINES ==='
git diff --name-only "$base" "$target" | while IFS= read -r file; do
  case "$file" in
    *.js|*.mjs|*.cjs)
      echo "--- $file"
      ast-grep outline "$file" --items all 2>/dev/null || true
      ;;
  esac
done

echo '=== HARNESS DIFF ==='
git diff --find-renames --unified=100 "$base" "$target" -- tests/performance docs package.json README.md

echo '=== CURRENT HARNESS SOURCES ==='
for file in \
  tests/performance/employment_separation_buyer_path.js \
  tests/performance/employment_separation_fixture_contract.mjs \
  tests/performance/employment_separation_fixture_contract.test.mjs
do
  if git cat-file -e "$target:$file" 2>/dev/null; then
    echo "--- $file"
    git show "$target:$file" | cat -n
  fi
done

echo '=== PEOPLE REQUEST-BOUNDARY IMPLEMENTATION ==='
rg -n -C 10 --glob '*.py' \
  '(Idempotency-Key|X-Actor-Reference|X-Purpose-Code|X-Tenant-Reference|Authorization|Bearer |/v1/employment-separations|separation_reason|evidence_version|confirmation_reference)' \
  services/people-api packages 2>/dev/null | head -n 1800 || true

echo '=== K6 RUNTIME/PATTERN REFERENCES ==='
rg -n -C 6 --glob '*.js' --glob '*.mjs' --glob '*.md' --glob '*.json' --glob '*.yml' --glob '*.yaml' \
  '(shared-iterations|maxDuration|handleSummary|execution|iterationInTest|Trend|Counter|thresholds|k6 run|grafana/k6|ORGMETRA_PERFORMANCE|candidate_sha|prepared_state|preparation_protocol|resource_evidence)' \
  tests docs README.md package.json .github 2>/dev/null || true

echo '=== RESULT-EVIDENCE FIELD AND SECRET/ENDPOINT AUDIT ==='
rg -n -C 5 --glob '*.js' --glob '*.mjs' \
  '(handleSummary|JSON\.stringify|result|envelope|endpoint|target|base_url|authorization|bearer|token|ORGMETRA_PERFORMANCE)' \
  tests/performance 2>/dev/null || true

echo '=== FIXTURE/PROFILE CROSS-REFERENCE ==='
rg -n -C 5 --glob '*.js' --glob '*.mjs' \
  '(first_commit|replay|rejection|contention|employment_id|employment_reference|profile_precondition|prepared_state|preparation_protocol)' \
  tests/performance 2>/dev/null || true

Length of output: 203259


🏁 Script executed:

#!/bin/bash
set -euo pipefail

target='beff0210c24f5d179f8a6717fbfeb5881f7c5365'
log='/tmp/coderabbit-shell-logs/shell-output-5yYGg9'

echo '=== EXACT HEAD AND DIFF STAT ==='
git rev-parse HEAD
git show -s --format='%H%n%P%n%s' "$target"
git diff --stat d9cc516d54b4642f59fe126c331a19945fdf75f1 "$target"
git diff --name-status 6dc5e2806b566ca8952eaf2d5bd65de6fa41d836 "$target"

echo '=== COMPLETE TARGET: WORKLOAD ==='
git show "$target:tests/performance/employment_separation_buyer_path.js" | nl -ba

echo '=== COMPLETE TARGET: FIXTURE CONTRACT ==='
git show "$target:tests/performance/employment_separation_fixture_contract.mjs" | nl -ba

echo '=== COMPLETE TARGET: CONTRACT TESTS ==='
git show "$target:tests/performance/employment_separation_fixture_contract.test.mjs" | nl -ba

echo '=== PREVIOUS COMMAND: KEY SECTIONS ==='
rg -n -C 3 '^=== EXACT-HEAD|^checked_out=|^base_is_ancestor|^=== CHANGED FILES|^=== HARNESS DIFF|^=== CURRENT HARNESS SOURCES' "$log" || true

echo '=== TARGET: ALL PERFORMANCE OUTPUT SINKS AND ENVIRONMENT USE ==='
git grep -n -E '(handleSummary|console\.|JSON\.stringify|__ENV|http\.(post|batch)|open\()' "$target" -- tests/performance || true

echo '=== TARGET: DOCUMENTED K6 INVOCATION ==='
git grep -n -i -E '(k6 run|ORGMETRA_PERFORMANCE_|employment-separation-performance-result)' "$target" -- README.md docs tests package.json .github 2>/dev/null || true

Length of output: 36295


@seonghobae Review found one false-GREEN risk in exact head beff0210c24f5d179f8a6717fbfeb5881f7c5365.

  • P1 — Fail an incomplete scenario before accepting its latency result. tests/performance/employment_separation_buyer_path.js Lines 56-87 configure shared-iterations with maxDuration: "30m". k6 can stop a scenario when this duration expires. The first-commit trend can then pass p(95)<=20 from fewer than the required 1,000 records. The current checks and unexpected-response thresholds only validate requests that ran. They do not prove that all first-commit, replay, rejection, and contention iterations completed. Add exact per-profile completion counters and thresholds, or validate the completed iteration counts in handleSummary() before emitting an acceptance result.

The prepared-state provenance is structurally explicit. The fixture requires preparation_protocol_reference, prepared_state_evidence_reference, and fixed preconditions for replay and rejection. The result envelope preserves those fields. Source review cannot verify that an external reference proves the declared database state.

The result envelope does not explicitly emit baseUrl or bearerToken. The bearer token comes only from ORGMETRA_PERFORMANCE_BEARER_TOKEN. The p95 threshold applies only to employment_separation_first_commit_duration_ms. The contention distribution remains separate.

k6 runtime execution was not available in this environment. The source uses standard k6 module imports, shared-iterations, exec.scenario.iterationInTest, http.batch, and handleSummary patterns.

You are interacting with an AI system.

Copy link
Copy Markdown
Contributor Author

@coderabbitai review

Please review exact head 2ef2f39449bcaa0ce8a077f43649af4a70a8e0e3, focusing on false-GREEN/false-RED risks in the published response contract, single-profile isolation, contention result classification, k6 threshold semantics, and evidence provenance. This is a stacked Draft on #64; do not infer protected/released status.

@coderabbitai

coderabbitai Bot commented Sep 13, 2026

Copy link
Copy Markdown

Tip

For best results, initiate chat on the files or code changes.

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -euo pipefail

target='2ef2f39449bcaa0ce8a077f43649af4a70a8e0e3'
base='d9cc516d54b4642f59fe126c331a19945fdf75f1'

echo '=== EXACT HEAD ==='
printf 'checked_out='
git rev-parse HEAD
printf 'target_type='
git cat-file -t "$target"
git merge-base --is-ancestor "$base" "$target" && echo 'base_is_ancestor=yes'
git diff --check "$base" "$target"
git show -s --format='target=%H%nparents=%P%nsubject=%s' "$target"

echo '=== CHANGED FILES ==='
git diff --name-status "$base" "$target"

echo '=== PERFORMANCE SOURCE OUTLINES ==='
for file in \
  tests/performance/employment_separation_buyer_path.js \
  tests/performance/employment_separation_fixture_contract.mjs \
  tests/performance/employment_separation_fixture_contract.test.mjs
do
  if git cat-file -e "$target:$file" 2>/dev/null; then
    echo "--- $file"
    git show "$target:$file" > "/tmp/$(basename "$file")"
    ast-grep outline "/tmp/$(basename "$file")" --items all 2>/dev/null || true
  fi
done

echo '=== WORKLOAD ==='
git show "$target:tests/performance/employment_separation_buyer_path.js" | nl -ba

echo '=== FIXTURE CONTRACT ==='
git show "$target:tests/performance/employment_separation_fixture_contract.mjs" | nl -ba

echo '=== CONTRACT TESTS ==='
git show "$target:tests/performance/employment_separation_fixture_contract.test.mjs" | nl -ba

echo '=== HARNESS DIFF ==='
git diff --find-renames --unified=100 "$base" "$target" -- \
  tests/performance docs README.md package.json

echo '=== RESPONSE BOUNDARY ==='
rg -n -C 12 --glob '*.py' \
  '(/v1/employment-separations|separation_conflict|replayed|recorded_at|employment_id|Idempotency-Key|X-Actor-Reference|X-Purpose-Code|X-Tenant-Reference)' \
  services packages tests | head -n 1800 || true

echo '=== K6, THRESHOLD, AND RESULT-EVIDENCE AUDIT ==='
git grep -n -E \
  '(shared-iterations|maxDuration|thresholds|Trend|Counter|handleSummary|JSON\.stringify|__ENV|http\.(post|batch)|iterationInTest|baseUrl|bearer|token|endpoint|candidate_sha|prepared_state|preparation_protocol|resource_evidence)' \
  "$target" -- tests/performance docs README.md package.json 2>/dev/null || true

echo '=== TARGET PERFORMANCE INVOCATION REFERENCES ==='
git grep -n -i -E \
  '(k6 run|ORGMETRA_PERFORMANCE_PROFILE|first_commit|replay|rejection|contention|employment-separation-performance-result)' \
  "$target" -- README.md docs tests package.json .github 2>/dev/null || true

Length of output: 202926


🏁 Script executed:

#!/bin/bash
set -euo pipefail

target='2ef2f39449bcaa0ce8a077f43649af4a70a8e0e3'

echo '=== EXACT HEAD SUMMARY ==='
git show -s --format='target=%H%nparents=%P%nsubject=%s' "$target"
git diff --name-status 'd9cc516d54b4642f59fe126c331a19945fdf75f1' "$target"

echo '=== WORKLOAD 1-240 ==='
git show "$target:tests/performance/employment_separation_buyer_path.js" | nl -ba | sed -n '1,240p'

echo '=== RUN CONTRACT ==='
git show "$target:tests/performance/employment_separation_run_contract.mjs" | nl -ba

echo '=== RESPONSE CONTRACT ==='
git show "$target:tests/performance/employment_separation_response_contract.mjs" | nl -ba

echo '=== FIXTURE CONTRACT 1-280 ==='
git show "$target:tests/performance/employment_separation_fixture_contract.mjs" | nl -ba | sed -n '1,280p'

echo '=== RESPONSE AND RUN CONTRACT TESTS ==='
for file in \
  tests/performance/employment_separation_response_contract.test.mjs \
  tests/performance/employment_separation_run_contract.test.mjs
do
  echo "--- $file"
  git show "$target:$file" | nl -ba
done

echo '=== PUBLISHED SEPARATION ERROR ENVELOPE ==='
sed -n '300,385p' services/people-api/src/orgmetra_people_api/separation_http.py | nl -ba -v300

echo '=== K6 PACKAGE / VERSION DECLARATIONS ==='
fd -HI -t f -a 'package.json' -a 'Dockerfile*' -a 'docker-compose*.yml' -a 'docker-compose*.yaml' -a '*.tool-versions' -a 'mise.toml' -a '.github/workflows' | while IFS= read -r file; do
  echo "--- $file"
  rg -n -i -C 3 '(k6|grafana/k6|xk6)' "$file" || true
done

Length of output: 33949


@seonghobae Review found one false-GREEN risk in exact head 2ef2f39449bcaa0ce8a077f43649af4a70a8e0e3.

  • P1 — Require all configured iterations before a profile result can pass. tests/performance/employment_separation_buyer_path.js Lines 67-98 use shared-iterations with maxDuration: "30m". k6 can stop the selected scenario when that duration expires. The existing checks and employment_separation_unexpected_response thresholds validate only requests that completed. Therefore, first_commit can satisfy p(95)<=20 from fewer than the required 1,000 records. The same condition can hide incomplete replay, rejection, or contention coverage. Add an exact completion counter and threshold for the selected profile. Alternatively, make handleSummary() fail when completed iterations do not equal the configured fixture-record count. For contention, the required completed count is the configured pair count, not the number of HTTP requests.

The reviewed response contract matches the published boundary. First-commit and replay require 200, the requested Employment identity, a UUID terminal-version identity, a valid UTC recorded_at, and the expected replayed value. Rejection and contention conflicts require 409 with error: "separation_conflict".

The selected-profile design prevents replay, rejection, and contention traffic from entering the first-commit duration trend. The p95 threshold exists only for first_commit. Contention remains separately measured and requires one governed non-replay success plus one governed conflict per pair.

The fixture contract requires right-cleared, non-synthetic data, exact candidate-SHA binding, explicit preparation and prepared-state references, resource evidence, minimum profile cardinality, and Employment isolation across profiles. The result envelope does not emit the endpoint or bearer token.

This is a Draft stacked on #64. This review does not infer protected, released, or measured acceptance status. k6 runtime execution was not available.

You are interacting with an AI system.

Copy link
Copy Markdown
Contributor Author

@coderabbitai review

Please review exact current head 38c6ee346bc21416a48c287facf0d72fd667caa8. The earlier fresh-summary repair exposed a mount-permission defect: pinned upstream k6 v2.2.0 declares USER 12345, while private host-owned fixture/staging paths are not reliably accessible under rootless Podman's default user mapping. Current head verifies the OCI image Config.User is 12345, explicitly runs as 12345:12345, and uses --userns=keep-id:uid=12345,gid=12345 so the invoking host owner maps to the non-root k6 identity. Please check for rootless/rootful/remote Podman incompatibility, evidence-permission weakening, user-namespace mistakes, and any remaining stale-result or mount TOCTOU false-GREEN path. No buyer p95 or protected acceptance is claimed.

@coderabbitai

coderabbitai Bot commented Sep 13, 2026

Copy link
Copy Markdown

Tip

For best results, initiate chat on the files or code changes.

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -euo pipefail

target='38c6ee346bc21416a48c287facf0d72fd667caa8'
previous='2ded2f0cdac52d0db5f481b4c97c118df8c9057c'
base='c0ba482c57a77820128f259b4a43c06df643c0f3'

echo '=== EXACT HEAD ==='
printf 'checked_out='; git rev-parse HEAD
git cat-file -t "$target"
git merge-base --is-ancestor "$base" "$target" && echo 'stacked_base_is_ancestor=yes'
git show -s --format='target=%H%nparents=%P%nsubject=%s' "$target"
git diff --check "$base" "$target"

echo '=== DELTA SINCE PUBLICATION REVIEW ==='
git diff --name-status "$previous" "$target"
git diff --find-renames --unified=140 "$previous" "$target" -- tests/performance docs README.md package.json .github

echo '=== PERFORMANCE FILE MAP ==='
git ls-tree -r --name-only "$target" -- tests/performance | sort

echo '=== OUTLINES ==='
for file in \
  tests/performance/run_employment_separation_benchmark.sh \
  tests/performance/employment_separation_buyer_path.js \
  tests/performance/employment_separation_acceptance_check.mjs \
  tests/performance/employment_separation_acceptance_contract.mjs \
  tests/performance/employment_separation_run_contract.mjs \
  tests/performance/employment_separation_k6_runtime_contract.mjs \
  tests/performance/employment_separation_k6_evidence_contract.mjs
do
  if git cat-file -e "$target:$file" 2>/dev/null; then
    echo "--- $file"
    git show "$target:$file" > "/tmp/$(basename "$file")"
    ast-grep outline "/tmp/$(basename "$file")" --items all 2>/dev/null || true
  fi
done

echo '=== RUNNER ==='
git show "$target:tests/performance/run_employment_separation_benchmark.sh" | nl -ba

echo '=== ACCEPTANCE AND WORKLOAD BOUNDARIES ==='
for file in \
  tests/performance/employment_separation_buyer_path.js \
  tests/performance/employment_separation_acceptance_check.mjs \
  tests/performance/employment_separation_acceptance_contract.mjs \
  tests/performance/employment_separation_run_contract.mjs \
  tests/performance/employment_separation_k6_runtime_contract.mjs \
  tests/performance/employment_separation_k6_evidence_contract.mjs
do
  echo "--- $file"
  git show "$target:$file" | nl -ba
done

echo '=== FOCUSED REGRESSIONS ==='
git ls-tree -r --name-only "$target" -- tests/performance | \
  rg '(runner|benchmark|publication|summary|workload|provenance|k6.*(runtime|evidence)|acceptance.*(binding|contract|cardinality|load)).*test\.(mjs|js)$' | \
  while IFS= read -r file; do
    echo "--- $file"
    git show "$target:$file" | nl -ba
  done

echo '=== USERNS, PODMAN, PERMISSION, AND PUBLICATION AUDIT ==='
git grep -n -E -i \
  '(userns|keep-id|--user |Config\.User|12345|podman (info|image|run|import)|remote|rootless|rootful|mount|--mount|--volume|umask|chmod|chown|mktemp|summary_run_file|SUMMARY_FILE|test -[efLs]|ln |sha256|stat|open|O_NOFOLLOW|fixture|staging|/workspace|--pull=never)' \
  "$target" -- tests/performance docs README.md package.json .github 2>/dev/null || true

echo '=== NON-TEST EXECUTION REFERENCES ==='
git grep -n -E \
  '(run_employment_separation_benchmark|ORGMETRA_PERFORMANCE_SUMMARY_FILE|employment_separation_acceptance_check)' \
  "$target" -- . ':!tests/performance/*test.mjs' 2>/dev/null || true

Length of output: 203682


🤖 web_search executed:

Podman documentation for --userns=keep-id:uid=12345,gid=12345: behavior for rootless versus rootful Podman, bind mounts and host file ownership, and limitations when using a remote Podman client. Also document podman image inspect Config.User behavior.

Length of output: 17085


🏁 Script executed:

#!/bin/bash
set -euo pipefail

target='38c6ee346bc21416a48c287facf0d72fd667caa8'
previous='2ded2f0cdac52d0db5f481b4c97c118df8c9057c'

echo '=== RUNNER, COMPLETE ==='
git show "$target:tests/performance/run_employment_separation_benchmark.sh" | nl -ba

echo '=== USER-MAPPING DELTA ==='
git diff --unified=100 "$previous" "$target" -- \
  tests/performance/run_employment_separation_benchmark.sh \
  tests/performance/employment_separation_k6_runtime_contract.test.mjs

echo '=== RUNNER-RELATED TEST FILES ==='
git ls-tree -r --name-only "$target" -- tests/performance | rg '(runner|benchmark).*test\.(mjs|js)$' | while IFS= read -r file; do
  echo "--- $file"
  git show "$target:$file" | nl -ba
done

echo '=== REMOTE AND PODMAN CONFIGURATION REFERENCES ==='
git grep -n -i -E '(CONTAINER_HOST|CONTAINER_CONNECTION|podman.*remote|remote.*podman|PODMAN_USERNS|containers\.conf|--remote|podman info)' \
  "$target" -- tests/performance docs README.md .github package.json 2>/dev/null || true

echo '=== SUMMARY PUBLICATION REFERENCES ==='
git grep -n -E '(summary_run_file|summary_target|ln "\$\{summary_run_file\}"|O_NOFOLLOW|stat |sha256sum|shasum|mktemp -d)' \
  "$target" -- tests/performance/run_employment_separation_benchmark.sh tests/performance 2>/dev/null || true

Length of output: 21741


@seonghobae Review found one remaining source-level false-GREEN path in exact head 38c6ee346bc21416a48c287facf0d72fd667caa8.

  • P1 — Publish the exact staged artifact that the runner validates. tests/performance/run_employment_separation_benchmark.sh validates ${summary_run_file} at Lines 122-124. It later gives the pathname to ln at Line 126. A process with the invoking user's authority can replace the regular staged file, replace it with a different regular file, or modify its contents after the validation and before ln resolves the path. The private mktemp directory prevents unrelated users from entering the directory. It does not protect against the benchmark owner.

    The new keep-id mapping makes the k6 process and the invoking host owner intentionally share access to the private staging directory. Therefore, it does not close this same-owner publication race. The runner can publish bytes that it did not validate after k6 exits. The later acceptance checks validate consistency of supplied artifacts, but they do not prove that the published artifact was the validated post-run artifact.

    Open the staged file once with O_RDONLY | O_NOFOLLOW. Verify through that file descriptor that it is a non-empty regular file. Publish bytes from that same descriptor into an exclusively created destination. If the hard-link design must remain, compare the source inode, device, size, and digest before and after linking. Fail if any value changes. Add a regression that replaces the staged pathname between validation and publication.

The user-mapping repair is otherwise correct for its supported topology.

  • The runner verifies Config.User equals 12345.
  • The benchmark invocation explicitly uses --user=12345:12345.
  • --userns=keep-id:uid=12345,gid=12345 maps the invoking rootless host owner to the k6 UID/GID.
  • The fixture remains read-only.
  • The staging directory remains private. The runner does not make fixture or staging paths world-writable.
  • The immutable workload image remains mounted at /workspace. The live repository is not mounted.

Rootful Podman is not a false-GREEN path. Podman documents that keep-id is unsupported when the caller is root. The benchmark then fails before it can publish a result. Add an explicit rootless-local preflight if a clearer operational failure is required.

Remote Podman is also not explicitly supported or rejected. Remote execution does not provide the local host-path ownership assumption used by this runner. A commercial run should use a local rootless Podman engine unless the runner adds a remote-engine evidence and mount contract.

I did not execute Podman, k6, or repository tests. This review does not claim measured buyer p95 acceptance, hosted execution, protected status, or release status.

You are interacting with an AI system.

Copy link
Copy Markdown
Contributor Author

Current authority receipt after CodeRabbit P1 repair: exact head 44dedef4412ac0c49e3f8facda25a0b85f3a9af3, stacked base #64 c0ba482c57a77820128f259b4a43c06df643c0f3, open/Draft/mergeable.

CodeRabbit correctly found that predecessor 38c6ee34... validated ${summary_run_file} and then resolved that pathname again in ln, leaving a same-owner check-to-link race. Test-first e17f371fffe0e5f5cd58e410562c2ff018ca1fba requires device/inode/size and SHA-256 binding across publication. Causal repair c0a2b0c7748e31c91a9cd507e88a35322a906ed5 captures source device/inode/size plus digest before the no-clobber hard link, re-reads source and published-target identity/digest afterward, removes the caller-visible target and fails if any value changes, disappears, or diverges. 44dedef... fixes the regression matcher so the executable contract matches the multi-line fail-closed check. The prior non-root 12345:12345/keep-id mount ownership repair is retained.

This receipt supersedes the prior body’s volatile exact-head line; the durable scope and earlier lineage remain unchanged. No hosted exact-head workflow, Podman/k6 buyer run, measured p50/p95/p99/max, protected integration, or release is claimed.

Copy link
Copy Markdown
Contributor Author

@coderabbitai review

Please review exact current head 44dedef4412ac0c49e3f8facda25a0b85f3a9af3, focusing on the repaired summary-publication P1. The runner now records staged source device/inode/size and SHA-256, performs the no-clobber hard link, then requires both source and target to retain/equal that identity and digest; any disappearance/divergence removes the caller-visible target and fails closed. Please look for remaining replacement/modification races, symlink/hard-link edge cases, partial publication, same-owner TOCTOU, or false-GREEN evidence paths. The prior USER 12345/rootless keep-id and immutable workload-image controls remain. No buyer p95 or protected acceptance is claimed.

@coderabbitai

coderabbitai Bot commented Sep 13, 2026

Copy link
Copy Markdown

Tip

For best results, initiate chat on the files or code changes.

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -euo pipefail

target='44dedef4412ac0c49e3f8facda25a0b85f3a9af3'
previous='38c6ee346bc21416a48c287facf0d72fd667caa8'
base='c0ba482c57a77820128f259b4a43c06df643c0f3'

echo '=== EXACT HEAD ==='
printf 'checked_out='; git rev-parse HEAD
git cat-file -t "$target"
git merge-base --is-ancestor "$base" "$target" && echo 'stacked_base_is_ancestor=yes'
git show -s --format='target=%H%nparents=%P%nsubject=%s' "$target"
git diff --check "$base" "$target"

echo '=== PUBLICATION-REPAIR DELTA ==='
git diff --name-status "$previous" "$target"
git diff --find-renames --unified=160 "$previous" "$target" -- tests/performance docs README.md package.json .github

echo '=== FOCUSED OUTLINES ==='
for file in \
  tests/performance/run_employment_separation_benchmark.sh \
  tests/performance/employment_separation_acceptance_check.mjs \
  tests/performance/employment_separation_acceptance_contract.mjs
do
  if git cat-file -e "$target:$file" 2>/dev/null; then
    echo "--- $file"
    git show "$target:$file" > "/tmp/$(basename "$file")"
    ast-grep outline "/tmp/$(basename "$file")" --items all 2>/dev/null || true
  fi
done

echo '=== RUNNER ==='
git show "$target:tests/performance/run_employment_separation_benchmark.sh" | nl -ba

echo '=== RUNNER/PUBLICATION REGRESSIONS ==='
git ls-tree -r --name-only "$target" -- tests/performance | \
  rg '(runner|benchmark|publication|summary|workload|provenance).*test\.(mjs|js)$' | \
  while IFS= read -r file; do
    echo "--- $file"
    git show "$target:$file" | nl -ba
  done

echo '=== PUBLICATION, PATH, AND CLEANUP AUDIT ==='
git grep -n -E \
  '(summary_run_file|summary_target|ORGMETRA_PERFORMANCE_SUMMARY_FILE|mktemp|umask|trap|stat |sha256sum|shasum|ln |rm -f|rm -rf|test -[efLs]|\\[\\[ -e|--out|handleSummary|podman run|--mount|--volume|userns|keep-id|--user=|/workspace)' \
  "$target" -- tests/performance docs README.md package.json .github 2>/dev/null || true

echo '=== ACCEPTANCE ARTIFACT BINDINGS ==='
for file in \
  tests/performance/employment_separation_acceptance_check.mjs \
  tests/performance/employment_separation_acceptance_contract.mjs \
  tests/performance/employment_separation_runtime_evidence_artifact.mjs
do
  if git cat-file -e "$target:$file" 2>/dev/null; then
    echo "--- $file"
    git show "$target:$file" | nl -ba
  fi
done

echo '=== NON-TEST ENTRY POINTS ==='
git grep -n -E \
  '(run_employment_separation_benchmark|ORGMETRA_PERFORMANCE_SUMMARY_FILE|employment_separation_acceptance_check)' \
  "$target" -- . ':!tests/performance/*test.mjs' 2>/dev/null || true

Length of output: 58226


@seonghobae Review found one remaining summary-publication race in exact head 44dedef4412ac0c49e3f8facda25a0b85f3a9af3.

  • P1 — Do not return success after the final target-path validation without protecting the published artifact. tests/performance/run_employment_separation_benchmark.sh Lines 132-140 re-stat and re-hash ${summary_target}, then return success. A process with the invoking owner’s authority can replace, unlink, or modify ${summary_target} after Line 135 and before the conditional completes. The script then exits successfully and leaves the replacement at the caller-visible path.

    The current checks detect changes that occur before the final target digest is read. They cannot detect a replacement after that read. This still permits a stale or substituted artifact to appear as the successful run output.

    Publish into a directory that the competing process cannot modify, or retain and verify an open descriptor for the published artifact through the completion boundary. If the caller-visible path must remain writable by the invoking owner, document that the runner can only guarantee the artifact identity at its final verification instant. The post-run acceptance workflow should consume and bind the verified artifact bytes immediately, rather than trust the pathname later. Add a regression that replaces the target after the final target digest capture and verifies that the runner cannot report success with the replacement present.

The current repair closes the earlier source-path race before the final verification:

  • The runner rejects a pre-existing target, including a symlink.
  • k6 writes only to a private staging directory.
  • The runner requires a non-empty, non-symlink regular staged file.
  • The runner records the staged file device, inode, size, and SHA-256 before ln.
  • The runner requires the staged file and linked target to retain the same identity and digest after linking.
  • The runner removes the target when this comparison detects divergence.
  • The workload remains an immutable image mount.
  • The k6 container remains pinned, non-root, and mapped with keep-id.

I did not execute repository tests, Podman, or k6. This source review does not claim measured buyer p95 acceptance, hosted execution, protected status, or release status.

You are interacting with an AI system.

Copy link
Copy Markdown
Contributor Author

Current authority receipt: exact head 2ba21d88aa3c9837fa08d54554d2e94edf04b6f6, stacked base #64 c0ba482c57a77820128f259b4a43c06df643c0f3, open/Draft/mergeable.

The CodeRabbit P1 repair at c0a2b0c... is now backed by an executable same-owner race regression, not only source-shape assertions. 2ba21d88... creates a fake pinned-Podman boundary and intercepts the publication ln to replace the staged summary after pre-link identity/digest capture. The canonical runner must detect the post-link source/target digest divergence, remove the caller-visible target, and exit non-zero. The test keeps all fixtures outside the repository and derives ORGMETRA_PERFORMANCE_TARGET_SHA from the checked-out exact HEAD.

No hosted exact-head workflow or production-equivalent k6 run is claimed yet; the prior body’s volatile exact-head line is superseded by this receipt.

Copy link
Copy Markdown
Contributor Author

@coderabbitai review

Please review exact current head 2ba21d88aa3c9837fa08d54554d2e94edf04b6f6, specifically the repaired summary-publication P1 and the new executable race regression. The runner binds staged source device/inode/size and SHA-256 before the no-clobber hard link, rechecks source and published-target identity/digest afterward, removes the target and fails on any divergence. The new test intercepts publication by replacing the staged summary between pre-link validation and ln, and requires the real wrapper to fail closed with no caller-visible artifact. Please look for remaining same-owner replacement/modification races, symlink/hard-link edge cases, partial publication, fake-test blind spots, or false-GREEN evidence paths. Existing pinned k6 OCI, immutable workload image, USER 12345/rootless keep-id, right-cleared fixture, and load-model controls remain unchanged. No hosted buyer p95, protected integration, or release is claimed.

@coderabbitai

coderabbitai Bot commented Sep 13, 2026

Copy link
Copy Markdown

Tip

For best results, initiate chat on the files or code changes.

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -euo pipefail

target='2ba21d88aa3c9837fa08d54554d2e94edf04b6f6'
previous='44dedef4412ac0c49e3f8facda25a0b85f3a9af3'
base='c0ba482c57a77820128f259b4a43c06df643c0f3'

echo '=== EXACT HEAD ==='
printf 'checked_out='; git rev-parse HEAD
printf 'target_type='; git cat-file -t "$target"
git merge-base --is-ancestor "$base" "$target" && echo 'stacked_base_is_ancestor=yes'
git show -s --format='target=%H%nparents=%P%nsubject=%s' "$target"
git diff --check "$base" "$target"

echo '=== DELTA SINCE PRIOR PUBLICATION REVIEW ==='
git diff --name-status "$previous" "$target"
git diff --find-renames --unified=180 "$previous" "$target" -- tests/performance docs README.md package.json .github

echo '=== PERFORMANCE FILE MAP ==='
git ls-tree -r --name-only "$target" -- tests/performance | sort

echo '=== OUTLINES ==='
for file in \
  tests/performance/run_employment_separation_benchmark.sh \
  tests/performance/employment_separation_runner_contract.test.mjs \
  tests/performance/employment_separation_benchmark_runner.test.mjs \
  tests/performance/employment_separation_acceptance_check.mjs \
  tests/performance/employment_separation_acceptance_contract.mjs \
  tests/performance/employment_separation_runtime_evidence_artifact.mjs
do
  if git cat-file -e "$target:$file" 2>/dev/null; then
    echo "--- $file"
    git show "$target:$file" > "/tmp/$(basename "$file")"
    ast-grep outline "/tmp/$(basename "$file")" --items all 2>/dev/null || true
  fi
done

echo '=== COMPLETE RUNNER ==='
git show "$target:tests/performance/run_employment_separation_benchmark.sh" | nl -ba

echo '=== RUNNER/PUBLICATION REGRESSIONS ==='
git ls-tree -r --name-only "$target" -- tests/performance | \
  rg '(runner|benchmark|publication|summary|workload|provenance).*test\.(mjs|js)$' | \
  while IFS= read -r file; do
    echo "--- $file"
    git show "$target:$file" | nl -ba
  done

echo '=== SUMMARY PATH, RACE, LINK, AND CLEANUP AUDIT ==='
git grep -n -E \
  '(summary_run_file|summary_target|ORGMETRA_PERFORMANCE_SUMMARY_FILE|mktemp|umask|trap|stat |sha256sum|shasum|ln |rm -f|rm -rf|test -[efLs]|\\[\\[ -e|--out|handleSummary|rename|open|O_NOFOLLOW|flock|mkdir|chmod|chown)' \
  "$target" -- tests/performance docs README.md package.json .github 2>/dev/null || true

echo '=== ARTIFACT BINDING AND EXECUTION BOUNDARY ==='
for file in \
  tests/performance/employment_separation_acceptance_check.mjs \
  tests/performance/employment_separation_acceptance_contract.mjs \
  tests/performance/employment_separation_runtime_evidence_artifact.mjs \
  tests/performance/employment_separation_buyer_path.js
do
  if git cat-file -e "$target:$file" 2>/dev/null; then
    echo "--- $file"
    git show "$target:$file" | nl -ba
  fi
done

echo '=== NON-TEST ENTRY POINTS ==='
git grep -n -E \
  '(run_employment_separation_benchmark|ORGMETRA_PERFORMANCE_SUMMARY_FILE|employment_separation_acceptance_check)' \
  "$target" -- . ':!tests/performance/*test.mjs' 2>/dev/null || true

Length of output: 66185


@seonghobae Review found one remaining summary-publication false-GREEN path in exact head 2ba21d88aa3c9837fa08d54554d2e94edf04b6f6.

  • P1 — Do not report successful publication after the final target hash without a protected consumption boundary. In tests/performance/run_employment_separation_benchmark.sh, Lines 132-135 re-read the staged path and the published target path. Lines 136-143 compare the captured values and then exit successfully.

    A process with the invoking owner’s authority can replace, unlink, or modify ${summary_target} after the hash capture at Line 135. The comparison then uses the already captured identity and digest. The wrapper can exit with status zero while the caller-visible path contains different bytes.

    The new executable regression is useful. It replaces ${summary_run_file} before ln. The post-link source and target checks detect that replacement. It does not exercise replacement of ${summary_target} after the final target digest capture.

    A pathname in a caller-writable directory cannot provide a durable same-owner integrity guarantee after the wrapper exits. Use a protected publication directory and invoke the acceptance check immediately on the verified bytes. Alternatively, make the runner return a verified file descriptor or immutable content digest to the consuming acceptance process. The acceptance receipt must bind the bytes that it reads, not a target pathname that can change later.

    Add a regression that replaces ${summary_target} after summary_target_digest is captured. The regression should prove that the runner does not report success with a substituted caller-visible artifact. If this cannot be guaranteed for a caller-owned directory, document the boundary and require immediate acceptance consumption from the verified artifact.

The new regression closes the earlier staged-source replacement path.

  • It invokes the real wrapper with a fake Podman executable.
  • It intercepts ln and replaces the staged file after pre-link identity and digest capture.
  • The wrapper detects the divergence.
  • The wrapper removes the caller-visible target.
  • The test requires a non-zero exit status.

The pinned k6 OCI boundary, immutable workload image, private staging directory, non-root 12345:12345 mapping, right-cleared fixture validation, and result/runtime artifact bindings remain unchanged at source level.

I did not execute repository tests, Podman, or k6. This review does not claim a hosted run, measured buyer latency, protected integration, or release status.

You are interacting with an AI system.

Copy link
Copy Markdown
Contributor Author

@coderabbitai review

Please review exact head 2ba21d88aa3c9837fa08d54554d2e94edf04b6f6, with particular attention to the staged-summary identity binding and the executable replacement regression added after 38c6ee346bc21416a48c287facf0d72fd667caa8. Treat this as harness/provenance review only; no buyer-performance or hosted-GREEN claim is requested.

@coderabbitai

coderabbitai Bot commented Sep 13, 2026

Copy link
Copy Markdown

Tip

For best results, initiate chat on the files or code changes.

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -euo pipefail

target='2ba21d88aa3c9837fa08d54554d2e94edf04b6f6'
previous='38c6ee346bc21416a48c287facf0d72fd667caa8'
base='c0ba482c57a77820128f259b4a43c06df643c0f3'

echo '=== EXACT COMMIT ==='
printf 'checked_out='; git rev-parse HEAD
printf 'target_type='; git cat-file -t "$target"
git merge-base --is-ancestor "$base" "$target" && echo 'stacked_base_is_ancestor=yes'
git show -s --format='target=%H%nparents=%P%nsubject=%s' "$target"
git diff --check "$base" "$target"

echo '=== REPAIR DELTA ==='
git diff --name-status "$previous" "$target"
git diff --find-renames --unified=160 "$previous" "$target" -- tests/performance docs README.md package.json .github

echo '=== PERFORMANCE FILE MAP ==='
git ls-tree -r --name-only "$target" -- tests/performance | sort

echo '=== OUTLINES ==='
for file in \
  tests/performance/run_employment_separation_benchmark.sh \
  tests/performance/employment_separation_benchmark_runner.test.mjs \
  tests/performance/employment_separation_runner_contract.test.mjs \
  tests/performance/employment_separation_acceptance_check.mjs \
  tests/performance/employment_separation_acceptance_contract.mjs \
  tests/performance/employment_separation_runtime_evidence_artifact.mjs
do
  if git cat-file -e "$target:$file" 2>/dev/null; then
    echo "--- $file"
    git show "$target:$file" > "/tmp/$(basename "$file")"
    ast-grep outline "/tmp/$(basename "$file")" --items all 2>/dev/null || true
  fi
done

echo '=== RUNNER ==='
git show "$target:tests/performance/run_employment_separation_benchmark.sh" | nl -ba

echo '=== RUNNER/PUBLICATION TESTS ==='
git ls-tree -r --name-only "$target" -- tests/performance | \
  rg '(runner|benchmark|publication|summary|workload|provenance).*test\.(mjs|js)$' | \
  while IFS= read -r file; do
    echo "--- $file"
    git show "$target:$file" | nl -ba
  done

echo '=== ACCEPTANCE AND ARTIFACT CONSUMPTION ==='
for file in \
  tests/performance/employment_separation_acceptance_check.mjs \
  tests/performance/employment_separation_acceptance_contract.mjs \
  tests/performance/employment_separation_runtime_evidence_artifact.mjs \
  tests/performance/employment_separation_buyer_path.js
do
  if git cat-file -e "$target:$file" 2>/dev/null; then
    echo "--- $file"
    git show "$target:$file" | nl -ba
  fi
done

echo '=== PATH / IDENTITY / RACE AUDIT ==='
git grep -n -E \
  '(summary_run_file|summary_target|ORGMETRA_PERFORMANCE_SUMMARY_FILE|mktemp|umask|trap|stat |sha256sum|shasum|ln |rm -f|rm -rf|test -[efLs]|rename|open|O_NOFOLLOW|flock|mkdir|chmod|chown|readFile|readFileSync|performance_result_sha256|runtime_evidence_sha256)' \
  "$target" -- tests/performance docs README.md package.json .github 2>/dev/null || true

Length of output: 110474


@seonghobae Review found one remaining summary-publication false-GREEN path in exact head 2ba21d88aa3c9837fa08d54554d2e94edf04b6f6.

  • P1 — Do not report successful publication after the final target-path verification without a protected consumption boundary. In tests/performance/run_employment_separation_benchmark.sh, Lines 132-135 re-read the staged source and ${summary_target} identity and digest. Lines 136-143 compare the captured values and return success.

    A process with the invoking owner’s authority can replace, unlink, or modify ${summary_target} after Line 135. The runner can then exit successfully while the caller-visible path contains different bytes. The acceptance CLI correctly hashes the bytes it later reads, but that does not prove that a consumer reads the artifact that the runner verified.

    The new executable regression is valid and closes the staged-source race. tests/performance/employment_separation_k6_runtime_contract.test.mjs replaces ${summary_run_file} before ln. The runner detects the changed source digest, removes ${summary_target}, and fails. The regression does not exercise replacement of ${summary_target} after the final target digest capture.

    A caller-writable pathname cannot provide a durable same-owner integrity guarantee after the wrapper exits. Publish into a protected directory and invoke acceptance on the verified bytes immediately. Alternatively, make the runner provide an immutable content digest or a verified file descriptor to the acceptance consumer. Add a regression that replaces ${summary_target} after summary_target_digest is captured and verify that the wrapper cannot report success with substituted output.

The staged-summary identity binding otherwise works as intended at source level.

  • The runner rejects a pre-existing target, including a symlink.
  • k6 writes only to a private per-run staging directory.
  • The runner requires a non-empty regular staged artifact.
  • The runner binds staged device, inode, size, and SHA-256 before ln.
  • The runner rechecks both paths after publication and removes the target on detected divergence.
  • The immutable workload image and pinned non-root k6 runner controls remain unchanged.

I did not execute repository tests, Podman, or k6. This harness and provenance review does not claim measured buyer performance, hosted GREEN, protected integration, or release status.

You are interacting with an AI system.

Copy link
Copy Markdown
Contributor Author

@coderabbitai review

Please review exact head 108f1c0ee70eebfb0fa7d391aecb49a7927d912d against the P1 you identified on 2ba21d88.... The repaired boundary no longer treats a caller-writable pathname as durable evidence: the runner emits the verified result SHA-256 as an immutable handoff token, the commercial acceptance CLI requires that token and re-hashes the raw result bytes before runtime/result validation, and an executable CLI regression substitutes the result bytes after the runner digest and requires fail-closed rejection. Please check for any remaining way to obtain an accepted receipt from bytes other than those bound by the runner digest. No hosted k6 or buyer-p95 claim is requested.

@coderabbitai

coderabbitai Bot commented Sep 13, 2026

Copy link
Copy Markdown

Tip

For best results, initiate chat on the files or code changes.

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -euo pipefail

target='108f1c0ee70eebfb0fa7d391aecb49a7927d912d'
previous='2ba21d88aa3c9837fa08d54554d2e94edf04b6f6'
base='c0ba482c57a77820128f259b4a43c06df643c0f3'

echo '=== EXACT HEAD ==='
printf 'checked_out='; git rev-parse HEAD
printf 'target_type='; git cat-file -t "$target"
git merge-base --is-ancestor "$base" "$target" && echo 'stacked_base_is_ancestor=yes'
git show -s --format='target=%H%nparents=%P%nsubject=%s' "$target"
git diff --check "$base" "$target"

echo '=== DELTA FROM PRIOR P1 HEAD ==='
git diff --name-status "$previous" "$target"
git diff --find-renames --unified=140 "$previous" "$target" -- tests/performance docs README.md package.json .github

echo '=== PERFORMANCE FILE MAP ==='
git ls-tree -r --name-only "$target" -- tests/performance | sort

echo '=== OUTLINES ==='
for file in \
  tests/performance/run_employment_separation_benchmark.sh \
  tests/performance/employment_separation_acceptance_check.mjs \
  tests/performance/employment_separation_acceptance_contract.mjs \
  tests/performance/employment_separation_runtime_evidence_artifact.mjs \
  tests/performance/employment_separation_result_artifact.mjs \
  tests/performance/employment_separation_buyer_path.js
do
  if git cat-file -e "$target:$file" 2>/dev/null; then
    echo "--- $file"
    git show "$target:$file" > "/tmp/$(basename "$file")"
    ast-grep outline "/tmp/$(basename "$file")" --items all 2>/dev/null || true
  fi
done

echo '=== RUNNER AND ACCEPTANCE SOURCES ==='
for file in \
  tests/performance/run_employment_separation_benchmark.sh \
  tests/performance/employment_separation_acceptance_check.mjs \
  tests/performance/employment_separation_acceptance_contract.mjs \
  tests/performance/employment_separation_runtime_evidence_artifact.mjs \
  tests/performance/employment_separation_result_artifact.mjs \
  tests/performance/employment_separation_buyer_path.js
do
  if git cat-file -e "$target:$file" 2>/dev/null; then
    echo "--- $file"
    git show "$target:$file" | nl -ba
  fi
done

echo '=== DIGEST-HANDOFF AND ACCEPTANCE REGRESSIONS ==='
git ls-tree -r --name-only "$target" -- tests/performance | \
  rg '(digest|handoff|acceptance.*(contract|binding|provenance|edge)|runtime_evidence|result_artifact|runner|benchmark).*test\.(mjs|js)$' | \
  while IFS= read -r file; do
    echo "--- $file"
    git show "$target:$file" | nl -ba
  done

echo '=== RESULT DIGEST, RAW BYTE, CLI, AND PATH AUDIT ==='
git grep -n -E \
  '(ORGMETRA_PERFORMANCE_RESULT_SHA256|performance_result_sha256|runner.*digest|result.*digest|sha256|createHash|readFile|readFileSync|process\.argv|runtime_evidence|summary_target|SUMMARY_FILE|acceptance_check|validateEmploymentSeparationAcceptance|JSON\.parse|TextDecoder|utf)' \
  "$target" -- tests/performance docs README.md package.json .github 2>/dev/null || true

echo '=== NON-TEST ENTRY POINTS ==='
git grep -n -E \
  '(run_employment_separation_benchmark|employment_separation_acceptance_check|ORGMETRA_PERFORMANCE_RESULT_SHA256)' \
  "$target" -- . ':!tests/performance/*test.mjs' 2>/dev/null || true

Length of output: 166565


@seonghobae Review found one remaining source-level false-GREEN path in exact head 108f1c0ee70eebfb0fa7d391aecb49a7927d912d.

  • P1 — Authenticate or independently bind the runner digest handoff token. tests/performance/employment_separation_acceptance_check.mjs Line 9 accepts runnerResultSha256 as a caller-supplied fourth CLI argument. Line 18 proves only that this supplied value hashes resultPath bytes. It does not prove that the value is the ORGMETRA_PERFORMANCE_RESULT_SHA256 value emitted by tests/performance/run_employment_separation_benchmark.sh at Line 149.

    A caller can replace the result artifact, construct matching runtime evidence, calculate the replacement artifact SHA-256, and pass that replacement digest as the fourth argument. The CLI then passes the digest check and the existing runtime/result binding checks. The new regression keeps the original digest while replacing the result. It does not test replacement of both the result and the caller-supplied digest.

    The runner digest removes trust in the mutable pathname only if the digest token has an integrity-protected transfer path. Bind the token to an independently authenticated runtime-evidence artifact, or have a trusted orchestrator capture and attest the runner output. The acceptance CLI must verify that attestation before it accepts the result bytes. Add a regression that substitutes both the result bytes and the CLI digest, and require rejection.

The repair closes the prior pathname-only replacement path when the runner token remains intact.

  • The runner emits a SHA-256 digest after successful publication checks.
  • The CLI requires exactly four inputs after the script name.
  • The CLI hashes raw Buffer bytes before JSON parsing.
  • The CLI rejects malformed digest values.
  • The CLI rejects changed result bytes when the original runner digest is supplied.
  • Runtime evidence still binds the raw result bytes, fixture bytes, candidate SHA, approved load model, and pinned k6 identity.

I did not execute repository tests, Podman, or k6. This source review does not claim hosted execution, measured buyer latency, protected integration, or release status.

You are interacting with an AI system.

Copy link
Copy Markdown
Contributor Author

@coderabbitai review

Please review exact head b6968520ef191fe8a3fd84d3cd5d4dc23a778281 against your P1 on 108f1c0e.... The leaf-local runner digest is no longer treated as authenticated authority. The commercial acceptance entry point now fails closed before reading caller result/runtime/fixture or digest evidence and names central owner .github#2162; the runner comment demotes its digest to structural evidence only. A RED contract explicitly requires that substituting both result identity and a locally chosen digest cannot restore commercial acceptance. Please check whether any source-level path in this PR still emits a positive commercial acceptance receipt without the future authenticated central attestation contract. No hosted k6 or p95 claim is requested.

@coderabbitai

coderabbitai Bot commented Sep 13, 2026

Copy link
Copy Markdown

Tip

For best results, initiate chat on the files or code changes.

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -euo pipefail

target='b6968520ef191fe8a3fd84d3cd5d4dc23a778281'
previous='108f1c0ee70eebfb0fa7d391aecb49a7927d912d'
base='c0ba482c57a77820128f259b4a43c06df643c0f3'

echo '=== EXACT COMMIT ==='
printf 'checked_out='; git rev-parse HEAD
printf 'target_type='; git cat-file -t "$target"
git merge-base --is-ancestor "$base" "$target" && echo 'stacked_base_is_ancestor=yes'
git show -s --format='target=%H%nparents=%P%nsubject=%s' "$target"
git diff --check "$base" "$target"

echo '=== DELTA FROM PRIOR DIGEST-HANDOFF HEAD ==='
git diff --name-status "$previous" "$target"
git diff --find-renames --unified=160 "$previous" "$target" -- tests/performance docs README.md package.json .github

echo '=== PERFORMANCE FILE MAP ==='
git ls-tree -r --name-only "$target" -- tests/performance | sort

echo '=== OUTLINES ==='
for file in \
  tests/performance/employment_separation_acceptance_check.mjs \
  tests/performance/employment_separation_acceptance_contract.mjs \
  tests/performance/employment_separation_result_artifact.mjs \
  tests/performance/employment_separation_runtime_evidence_artifact.mjs \
  tests/performance/run_employment_separation_benchmark.sh
do
  if git cat-file -e "$target:$file" 2>/dev/null; then
    echo "--- $file"
    git show "$target:$file" > "/tmp/$(basename "$file")"
    ast-grep outline "/tmp/$(basename "$file")" --items all 2>/dev/null || true
  fi
done

echo '=== ACCEPTANCE ENTRY POINTS AND RELATED SOURCES ==='
for file in \
  tests/performance/employment_separation_acceptance_check.mjs \
  tests/performance/employment_separation_acceptance_contract.mjs \
  tests/performance/employment_separation_result_artifact.mjs \
  tests/performance/employment_separation_runtime_evidence_artifact.mjs \
  tests/performance/run_employment_separation_benchmark.sh
do
  if git cat-file -e "$target:$file" 2>/dev/null; then
    echo "--- $file"
    git show "$target:$file" | nl -ba
  fi
done

echo '=== ACCEPTANCE / ATTESTATION / RED-CASE TESTS ==='
git ls-tree -r --name-only "$target" -- tests/performance | \
  rg '(acceptance|attest|digest|handoff|result_artifact|runtime_evidence).*test\.(mjs|js)$' | \
  while IFS= read -r file; do
    echo "--- $file"
    git show "$target:$file" | nl -ba
  done

echo '=== ALL POSITIVE-RECEIPT AND ACCEPTANCE PATHS ==='
git grep -n -E -i \
  '(commercial.*accept|acceptance.*receipt|receipt|commercial_acceptance|validateEmploymentSeparationAcceptance|acceptance_check|authenticated.*(evidence|attestation)|attestation|ORGMETRA_PERFORMANCE_RESULT_SHA256|runnerResultSha256|process\.exit|console\.log|JSON\.stringify)' \
  "$target" -- . ':!node_modules' ':!dist' ':!build' 2>/dev/null || true

echo '=== CALL SITES OUTSIDE TESTS ==='
git grep -n -E \
  '(employment_separation_acceptance_check|validateEmploymentSeparationAcceptance|ORGMETRA_PERFORMANCE_RESULT_SHA256)' \
  "$target" -- . ':!tests/performance/*test.mjs' 2>/dev/null || true

Length of output: 145837


@seonghobae Review found one remaining source-level commercial-acceptance bypass in exact head b6968520ef191fe8a3fd84d3cd5d4dc23a778281.

  • P1 — Do not let the local structural validator emit a positive commercial acceptance result. tests/performance/employment_separation_acceptance_check.mjs correctly calls requireAuthenticatedPerformanceEvidence() at Line 12 before it reads any caller path or parses any evidence. The CLI therefore fails closed. However, tests/performance/employment_separation_acceptance_contract.mjs exports validateEmploymentSeparationAcceptance() at Line 330. It returns { accepted: true, ... } at Lines 343-350 without the authenticated-evidence gate.

    Any caller can import that module and submit self-consistent result, runtime, and fixture bytes. The function then produces the same positive acceptance-shaped object that the CLI would have emitted before this repair. The existing tests do this directly. This bypasses .github#2162 because the gate exists only in the CLI wrapper.

    Keep the local validator for structural validation, but make its result non-commercial. For example, rename it to a structural-evidence validator and return structurally_valid: true, not accepted: true. Alternatively, enforce the authenticated-evidence gate within every API that can emit a commercial acceptance receipt. Add a regression that imports the local validator with mutually consistent replacement artifacts and proves that it cannot produce a positive commercial receipt before the central attestation contract exists.

The CLI repair correctly closes the prior caller-controlled digest path.

  • The gate always throws before CLI argument validation and before readFile() calls.
  • The gate message identifies ContextualWisdomLab/.github#2162.
  • The CLI no longer accepts a runner digest argument.
  • The RED regression rejects both a caller-supplied digest and replacement result/runtime paths with a locally chosen digest.
  • The benchmark runner now documents ORGMETRA_PERFORMANCE_RESULT_SHA256 as structural evidence only.

I did not execute repository tests, Podman, or k6. This review does not claim measured buyer p95, hosted execution, protected acceptance, or release status.


✏️ Learnings added
Learnt from: seonghobae
URL: https://github.com/ContextualWisdomLab/Orgmetra/pull/317

Timestamp: 2026-09-13T12:46:51.062Z
Learning: For the Employment separation performance harness, local result, runtime, fixture, and digest validation is structural evidence only. A positive commercial acceptance receipt requires the organization-owned authenticated performance-evidence attestation contract tracked by ContextualWisdomLab/.github#2162.

Note: Learnings are effective only in the context of similar code segments. To apply general coding guidelines, please configure review instructions. You can manage existing learnings in the app.

You are interacting with an AI system.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant