Skip to content

depends-on a submit_pull_request_review output is never confirmed: a dependent output (e.g. create_check_run) fails after the review was posted #75293

Closed

Description

@ekbritecore

Analysis

On gh-aw v1.2.0, an agent that calls submit_pull_request_review and then create_check_run with depends-on pointing at the review's output_id always ends with a failed safe_outputs job, although the review was posted.

Observed (pi engine, handler-managed outputs): the safe_outputs job log shows

Submitting PR review on <repo>#<n>: event=COMMENT, comments=0, bodyLength=...
Created PR review #...: https://github.com/<repo>/pull/<n>#pullrequestreview-...
✓ PR review submitted for <repo>#<n>
=== Processing Summary ===  Status: partial_success  Successful: 1  Failed: 1
##[error]1 safe output(s) failed:
  - create_check_run: Safe-output prerequisite did not complete successfully   (E099)

and agent_output.json holds the two items, the second with "depends-on": "<output_id of the first>". Runs where the agent omits depends-on on the same two calls succeed.

Reading safe_output_dependencies.cjs: checkSafeOutputDependencies accepts a prerequisite only when isConfirmedDependencySuccess(record) is true, which rejects a result that is staged, queued, buffered or deferred. The review comments are buffered and the review is submitted once at the end, so the review's record presumably never counts as a confirmed success (I did not trace which outcome field it carries), and every dependent output fails after the real effect happened. depends-on is advertised on every safe-output tool ("Execute only after every prerequisite succeeds"), so the agent has no hint that this one cannot be depended on.

Impact: a green review plus a red job; with create_check_run as the run's only visible result, the check also shows as failed or missing.

Implementation Plan

  1. Decide whether a submitted review can be a dependency. If yes, mark the review's outcome confirmed once the review exists (after the buffered submission), and let dependents wait for it (allowWaiting) instead of failing.
  2. If no, do not offer depends-on on tools that cannot be depended on (reject it at call time with a clear error, as is already done for noop, report_incomplete and similar), so the agent learns it immediately.
  3. Either way, say so in the safe-outputs reference (which outputs can be dependencies, and that review comments and the review are submitted together at the end).

Acceptance Criteria

  • An agent that submits a review and then calls create_check_run (or any dependent output) with depends-on the review either succeeds, or gets an error at the tool call, never a post-hoc safe_outputs failure after the review was posted.
  • A regression test covers a review followed by a dependent output.
  • The reference page lists which outputs support depends-on.

Finish with make agent-finish.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions