Skip to content

Nondeterministic ticket selection when a PR description references more than 3 issues #2421

@dellch

Description

@dellch

Git provider (optional)

None

System Info (optional)

No response

Issues details

Description

extract_ticket_links_from_pr_description() in pr_agent/tools/ticket_pr_compliance_check.py accumulates GitHub issue URLs in a set, then enforces the 3-ticket cap by slicing a list built from that set:

github_tickets = set()
...
if len(github_tickets) > 3:
    github_tickets = set(list(github_tickets)[:3])
...
return list(github_tickets)

Because a set has no defined iteration order, list(github_tickets)[:3] selects an arbitrary 3 of the referenced issues, and the order can vary between runs. When a PR description references more than 3 issues, which issues get fetched for review context is therefore nondeterministic.

Impact

  • The ticket context fed to the model can differ run-to-run for the same PR, making reviews inconsistent and harder to debug.
  • Inconsistent with the Jira key extraction in the same file (find_jira_tickets), which already preserves first-seen order.

Expected behavior

Selection should be deterministic — keep the first 3 issues in the order they appear in the PR description.

Steps to reproduce

  1. Open a PR whose description references more than 3 issues, e.g. Fixes #5, #1, #9, #2, #7.
  2. Run the review tool with ticket analysis enabled (require_ticket_analysis_review).
  3. Observe that the 3 issues fetched for context are an arbitrary subset and can change across runs.

Suggested fix

Track URLs in first-seen order while de-duplicating (a seen set plus an ordered list), then apply the cap by slicing the ordered list — mirroring what find_jira_tickets() already does. This keeps behavior identical for ≤3 issues and only changes the >3 cap case.

Notes

This was originally surfaced by the Qodo review bot on #2420 (a separate Jira-support PR), but the function predates that PR and is unrelated to Jira,

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions