You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Make admin-console review findings demonstrate a reachable broken contract before asking for a change. Distinguish an intentionally narrowed workflow from a regression, without exempting API mismatches, concealed failures, stale responses, or authorization defects.
Clarify that an API read used as a browser assertion's oracle, or a stronger assertion of an existing contract, does not by itself require another typed smoke scenario. Genuinely new deployed-platform contracts still belong in typed space_smoke.
Limit coverage complaints to a missing clause of the changed behavior's claimed contract. The reviewer must check existing coverage, identify an independent oracle and a wrong result that would pass, and avoid tests of smoke tests or configured-value mirrors.
Keep the PR-description check blocking for missing information, but accept equivalent headings rather than enforcing exact spelling. Require severity, risk, and AI disclosure as distinct facts, and refer to "validation evidence" consistently in the existing regression-test advisory check. Privacy checks, approval-risk limits, sensitive-path overrides, and all check modes remain unchanged.
Validation
The configuration passes the official CodeRabbit JSON schema.
git diff --check and the repository's Gitleaks pre-commit hook pass.
A structural comparison with the baseline confirms that only the UI instructions, a new console-loop instruction block, the PR-description check's instruction text, and the regression advisory's reference to "validation evidence" change. Modes, privacy checks, approval rules, tools, sensitive paths, and unrelated instructions are unchanged.
CodeRabbit's final configuration preview reported no findings on the final configuration-only diff. A fresh seed-builder replay had identified inconsistent pre-merge wording; the final revision corrects it.
Fresh live replay results are recorded below. All four replay-only PRs are closed without merging.
No application code changes and no new application tests. Schema validation proves configuration shape, not review quality.
Calibration method and iteration
Replayed the server-search/owner-email changes at historical revision 301a232eecea417ca1fa3400c60399b301dc870b, and the seed-builder changes at 3b67af9adc83046647613a14ef6fbfb38f033334, on their original bases.
The search baseline and seed-builder baseline already did not reproduce the historical duplicate-test and missing-rebuild complaints. Do not interpret this calibration as a measured reduction in false positives.
The second candidate produced a configured-destination mirror request. The third revision therefore applies the existing test-economy configuration stop rule and distinguishes a genuinely missing claimed contract from an arbitrary stronger test.
The fresh search replay reported a concrete readiness-wait issue, not a demand for a duplicate typed scenario or a configured-URL mirror.
The fresh seed-builder replay retained findings for non-idempotent sandbox retries and the runtime-PID readiness race. It also caught inconsistent check wording: severity and risk are now separate required facts, AI disclosure is explicitly identified, and the regression advisory accepts validation evidence under equivalent headings.
The replayed UI and console-loop instructions are identical to this PR's final review instructions. Only pre-merge wording changed after these replays. Their application content is unchanged from the baseline replays. Full application CI on the replay-only PRs was intentionally cancelled; those drafts are not integration candidates and must not merge.
The fresh seed-builder run did not rediscover every baseline finding. These small, stochastic, shared-repository runs do not establish precision or recall rates, or prove that all legitimate findings are retained.
Preview output is non-authoritative and does not exercise canonical approval/request-changes status. No claim is made that the PR-description custom check has been live-validated through a canonical review using unmerged configuration.
Severity and risk
Severity: does not apply; review-policy tuning.
Risk: ambiguous regressions could receive less feedback. The instructions retain concrete correctness and security checks, distinguish intentional behavior from verified defects, and preserve the existing test-placement rules. Revert this commit to restore the previous instructions.
AI assistance: Perplexity Computer drafted the configuration, this description, and the calibration procedure.
Automatic approval is disabled by the configured approval policy for this pull request. Review comments and requests for changes continue normally.
Commit: 108cad91d09f.
No actionable comments were generated in the recent review.
Walkthrough
The review configuration adds UI and console-loop checks for contracts, coverage gaps, and specific failure cases. The pull-request description check accepts equivalent headings and split sections while requiring summary, validation evidence, severity, risk, and AI-assistance disclosure. The bug-fix check now refers to validation evidence rather than a specific section.
Possibly related PRs
perplexityai/SPACE#1428: Introduces admin-console review instructions that this PR extends with checks for broken contracts and stale responses.
Merge Risk: Minimal · up to 108ca
No actionable merge-blocking issue is established; the configuration is mergeable after normal checks.
Architecture risk: Low · up to 108ca
The changed surface does not map to a changed system, dependency edge, entrypoint, or external dependency.
Changed systems: None identified.
Architecture concerns
No architecture-level concerns identified.
Review details
Before / after behavior
observed — Modified behavior in .coderabbit.yaml: The ui/** guidance replaces a short list of UI checks with instructions to trace component callers, render conditions, data sources, and API contracts; evaluate removed controls against replacement paths and supported states; and flag request/API mismatches, concealed read failures, stale-response overwrites, and browser-only authorization. It adds scripts/console-loop/** guidance to assess distinct contracts against existing coverage, specify a missing clause, independent oracle, and concrete failure before reporting a gap, and flag vacuous assertions, missing oracle data, unconditional compatibility skips, and racing fixed sleeps.
observed — Modified behavior in .coderabbit.yaml: The description check no longer requires fixed headings or a specific AI-assistance line format. It requires substantive summary, validation evidence (including “CI on this head” as an option), severity or an explicit non-applicability statement, a separate risk statement, and an explicit AI-assistance disclosure; equivalent headings and split sections are accepted. It checks for presence and names missing information, without assessing quality or whether reported validation ran.
observed — Modified behavior in .coderabbit.yaml: The bug-fix check now refers to “validation evidence” naming a regression test rather than requiring the test to be named in the “Validation section.”
Pre-merge checks: 5
Check name
Status
Explanation
Title check
The title identifies the configuration area and a main change. It meets the title format requirements.
Description Follows The Pr Template
The description includes a summary, reported validation results, a severity statement, a risk statement, and an AI assistance disclosure.
Bug Fixes Name A Regression Test
The PR changes only .coderabbit.yaml. It does not fix application behavior under crates/, shim/, or ui/, so this check does not apply.
Weakened Safeguards Are Justified
The PR gives a specific reason for relaxing the description check: it requires the template information without enforcing exact headings. The check remains blocking. The diff does not remove or weaken…
Public-Safe Content
The diff, title, and description contain no identified content prohibited by the check. The diff changes only .coderabbit.yaml review instructions.
Full details: Weakened Safeguards Are Justified
Explanation
The PR gives a specific reason for relaxing the description check: it requires the template information without enforcing exact headings. The check remains blocking. The diff does not remove or weaken tests, assertions, lints, or CI steps.
Recent review info
Automatic approval policy
Commit: 108cad91d09f.
Setting
Effective value
Source
Automatic approvals
Disabled
Override 1, Override 4
Merge risk
Low
Override 2
Security risk
Not set
Global configuration
Matching overrides (numbered as in the resolved configuration):
Any matching disabled override withholds approval, regardless of risk. Otherwise, the strictest explicitly set limit wins for each score. Limits omitted by all matching overrides inherit the global configuration.
Not set means no limit was configured after inheritance. Use @coderabbitai configuration to inspect the resolved settings and their sources.
Reviewing files that changed from the base of the PR and between 953e802 and 108cad9.
Files selected for processing: 1
.coderabbit.yaml
Included review availability: This review used your included allowance. Your plan provides up to 12 included reviews per hour; 11 remain after this review.
[!WARNING]
This isolated full review uses the current CodeRabbit configuration from this draft pull request. Its output is non-authoritative and does not update canonical review state or pull request status.
The reason will be displayed to describe this comment to others. Learn more.
Warning
Configuration preview: This review used the unmerged CodeRabbit configuration from this draft pull request. It is non-authoritative and does not update canonical review state or pull request status.
Reviewing files that changed from the base of the PR and between 953e802 and 61f46ca.
Files selected for processing: 1
.coderabbit.yaml
Included review availability: This review used your included allowance. Your plan provides up to 12 included reviews per hour; 9 remain after this review.
[!WARNING]
This isolated full review uses the current CodeRabbit configuration from this draft pull request. Its output is non-authoritative and does not update canonical review state or pull request status.
The reason will be displayed to describe this comment to others. Learn more.
Warning
Configuration preview: This review used the unmerged CodeRabbit configuration from this draft pull request. It is non-authoritative and does not update canonical review state or pull request status.
Reviewing files that changed from the base of the PR and between 953e802 and 726cde3.
Files selected for processing: 1
.coderabbit.yaml
Included review availability: This review used your included allowance. Your plan provides up to 12 included reviews per hour; 9 remain after this review.
[!WARNING]
This isolated full review uses the current CodeRabbit configuration from this draft pull request. Its output is non-authoritative and does not update canonical review state or pull request status.
The reason will be displayed to describe this comment to others. Learn more.
Warning
Configuration preview: This review used the unmerged CodeRabbit configuration from this draft pull request. It is non-authoritative and does not update canonical review state or pull request status.
Reviewing files that changed from the base of the PR and between 953e802 and 480b4f3.
Files selected for processing: 1
.coderabbit.yaml
Included review availability: This review used your included allowance. Your plan provides up to 12 included reviews per hour; 9 remain after this review.
[!WARNING]
This isolated full review uses the current CodeRabbit configuration from this draft pull request. Its output is non-authoritative and does not update canonical review state or pull request status.
The reason will be displayed to describe this comment to others. Learn more.
Warning
Configuration preview: This review used the unmerged CodeRabbit configuration from this draft pull request. It is non-authoritative and does not update canonical review state or pull request status.
Reviewing files that changed from the base of the PR and between 953e802 and 108cad9.
Files selected for processing: 1
.coderabbit.yaml
Included review availability: This review used your included allowance. Your plan provides up to 12 included reviews per hour; 11 remain after this review.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Summary
Make admin-console review findings demonstrate a reachable broken contract before asking for a change. Distinguish an intentionally narrowed workflow from a regression, without exempting API mismatches, concealed failures, stale responses, or authorization defects.
Clarify that an API read used as a browser assertion's oracle, or a stronger assertion of an existing contract, does not by itself require another typed smoke scenario. Genuinely new deployed-platform contracts still belong in typed
space_smoke.Limit coverage complaints to a missing clause of the changed behavior's claimed contract. The reviewer must check existing coverage, identify an independent oracle and a wrong result that would pass, and avoid tests of smoke tests or configured-value mirrors.
Keep the PR-description check blocking for missing information, but accept equivalent headings rather than enforcing exact spelling. Require severity, risk, and AI disclosure as distinct facts, and refer to "validation evidence" consistently in the existing regression-test advisory check. Privacy checks, approval-risk limits, sensitive-path overrides, and all check modes remain unchanged.
Validation
git diff --checkand the repository's Gitleaks pre-commit hook pass.Calibration method and iteration
301a232eecea417ca1fa3400c60399b301dc870b, and the seed-builder changes at3b67af9adc83046647613a14ef6fbfb38f033334, on their original bases.Severity and risk
Severity: does not apply; review-policy tuning.
Risk: ambiguous regressions could receive less feedback. The instructions retain concrete correctness and security checks, distinguish intentional behavior from verified defects, and preserve the existing test-placement rules. Revert this commit to restore the previous instructions.
AI assistance: Perplexity Computer drafted the configuration, this description, and the calibration procedure.