Skip to content

fix: reduce review noise and clarify agent communication - #1657

Merged
tmchow merged 11 commits into
mainfrom
tmchow/review-autonomy-plain-language
Sep 10, 2026
Merged

fix: reduce review noise and clarify agent communication#1657
tmchow merged 11 commits into
mainfrom
tmchow/review-autonomy-plain-language

Conversation

@tmchow

@tmchow tmchow commented Sep 8, 2026

Copy link
Copy Markdown
Collaborator

Summary

Reviews should ask users about choices that need their judgment and explain those choices plainly. Agents now investigate findings, discard weak or settled concerns, and resolve authorized technical work before returning it to the user.

Document review can apply proven corrections needed to carry out concrete decisions already made in the document. It preserves explicit edit restrictions and unresolved product choices. Findings from another model need independent local support before silent application. Choosing a remedy and having permission to apply it remain separate checks.

Planning and implementation callers reassess returned findings instead of forwarding them unchanged. ce-pov supports consequential questions selectively and returns missing context to its caller. Shared writing guidance removes internal jargon while preserving technical terms, evidence, and qualifications.

Validation

  • Fresh Claude Code Fable 5.1 low and Codex CLI Astra low runs corrected concrete document defects without approval requests and respected report-only and product-choice boundaries.
  • In a focused reassessment of a larger completed review, Fable reduced eight decision items to two; Astra returned no user decisions. Fable still retained avoidable decisions and weak findings. These results demonstrate improvement, not consistently noise-free review.
  • Code review through ce-work corrected the ownership guard and passed the relevant 403/404 tests on both models. Simplification-before-review ordering was not fully established by those runs.
  • A focused code-review replay reproduced rejected preferences returning as findings before the fix. Afterward, both models discarded the preference, retained a real ownership defect, and preserved review-only permissions.
  • A bounded document-review replay reproduced rejected residuals inflating the count from zero to two. Afterward, both models returned zero and preserved the one genuine customer-commitment question.
  • After integrating current main and the review fixes, all 4,022 tests passed. Release metadata and strict plugin validation passed.

Detailed scenarios, iterations, and limitations include the earlier unsuccessful runs.

Security Disclosure

Expands default document-edit authority to proven corrections required by existing decisions. Explicit edit restrictions, protected user choices, and independent-review requirements remain. No new shell execution, credential handling, or dependencies.

Agent Disclosure

  • Model: Codex CLI · GPT-6

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Sep 8, 2026

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review Completed 2026-09-10T01:16:27.785006Z 9505738 New commits
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 37bfe17102

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread tests/skill-eval-cell/calibration-scenarios.ts Outdated
@tmchow
tmchow force-pushed the tmchow/review-autonomy-plain-language branch from 1ef0408 to 2f91f39 Compare September 9, 2026 07:09

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 2f91f391c1

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread skills/ce-doc-review/references/subagent-template.md
Require demonstrated consequences before retaining review findings and let
document review apply proven corrections that fulfill existing decisions.
Keep unresolved user choices and explicit edit restrictions protected, and
make callers resolve supported technical work before presenting a handoff.

Add behavioral fixtures and record cross-host evaluation results, including
remaining Fable calibration limits. Full suite: 3,915 tests passed; release
and plugin validation passed.
Preserve review calibration alongside upstream agent cleanup and blocked
handoff handling. Keep both behavioral scenario sets. Move task visibility
unchanged into the required scope reference to preserve the Codex prompt
budget, with an always-loaded pointer.

Full suite: 3,913 passed; the two prompt-budget failures were corrected and
passed on targeted rerun. Release and strict plugin validation passed.
Independent source review confirmed both parent contracts remain intact.
Behavioral replay was not repeated for this mechanical reconciliation and
verbatim relocation into an already-required reference.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: fc3000110e

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread tests/skill-eval-cell/calibration-scenarios.ts Outdated
tmchow and others added 2 commits September 9, 2026 16:44
Use the main-reachable pre-change contract for the completed-review handoff scenario so fresh checkouts can run its pre arm after squash merge. Historical evaluation reports retain their original refs.

Validation: catalog 16 passed. Full suite 3913 passed, 2 failed; both affected test files passed on targeted rerun (100 tests).
Merge origin/main (#1664 docs site, #1549 Compound Packs) into the review
calibration branch.

One textual conflict in skills/ce-code-review/references/scope.md: main
extended the Stage 1b signals paragraph with `declared_packs`; this branch
appended a Task Visibility section after it. Both are kept, main's paragraph
first. The other nine shared files (guides, dispatch-reviewers, finish-review,
doc-review subagent template, review contract test) auto-merged into disjoint
hunks and were read for semantic fit: pack-rule contradictions enter as
ordinary findings and pass the same admission and edit-authority gates this
branch tightened, and nothing on main reintroduces the collection or residual
gate language this branch removed.

Release validation passed. Full suite: 4,008 passed; 13 timeouts were all in
ce-work-unit-workspace-fallback, which neither side touched, and that file
passes in isolation (15/15). Strict plugin validation is left to CI (no
`claude` binary here).

Co-authored-by: Kieran Klaassen <kieranklaassen@users.noreply.github.com>

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 968c79007a

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread skills/ce-code-review/references/finish-review.md Outdated
Keep upstream attributable-terminal collection and local review-noise
filtering together. Preserve cleanup, incomplete-review protection, and
separate host waiting and result delivery. Align collection assertions
with the upstream contract while retaining admission checks.

Validation: 4,022 tests passed; release metadata and strict plugin
validation passed. Independent source comparison verified both parents'
intent. No new behavioral replay for this mechanical reconciliation of
existing collection contracts.
Remove the synthesis exception that reinserted preference-only settled conflicts into the primary report. Keep genuine defects actionable and preserve the boundary against reversing settled decisions during local apply.

A bounded Fable baseline reproduced advisory resurrection; Fable and Astra post-change routing discarded the preference while retaining the ownership defect. All 4022 tests pass; release and plugin validation pass.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 6d4a5b4218

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread skills/ce-doc-review/references/synthesis-and-presentation.md
Comment thread docs/guides/ce-work.md
Replace preference-only FYI examples with verified advisory benefits and reconcile caller guides with the existing outcome-based completion gate. Preserve synthesis authority and caller behavior.

Bounded Fable and Astra rendering checks retained only the useful FYI; the pre-change Fable run already did so. All 4022 tests and release/plugin validation pass.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 6cf05fc0c7

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread skills/ce-doc-review/references/document-intake.md Outdated
Comment thread skills/ce-doc-review/references/synthesis-and-presentation.md
Comment thread skills/ce-doc-review/references/review-output-template.md Outdated
Comment thread skills/ce-plan/references/plan-handoff.md Outdated
@tmchow
tmchow merged commit 16c2b97 into main Sep 10, 2026
8 of 9 checks passed
@github-actions github-actions Bot mentioned this pull request Sep 9, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants