Skip to content

FEAT: Drill into paired Scenario outcome conversations #3389

Open

Description

@romanlutz

Is your feature request related to a problem? Please describe.

The Paired outcomes panel on a Scenario run reports transitions such as one baseline failure becoming an attack success, but there is no way to inspect which objective and conversations produced that transition. Aggregate counts alone make it difficult to investigate the differences.

Describe the solution you'd like

On the Scenario run results page:

  • Make each paired-outcome transition row expandable on demand and collapsible again.
  • Expanding a row should load the matching baseline/attack pairs, rather than eagerly loading all conversations with the initial comparison.
  • Show the objective or logical seed and useful context for both sides of each pair, including their outcomes. Provide clear navigation to the relevant baseline and attack conversations using the existing conversation routes and Scenario context.
  • Keep the selected groups, latest top-level attempt rules, and decided-pair population consistent with the displayed transition count. Reuse backend pairing semantics rather than calculating a separate cohort in the GUI.
  • Support bounded loading or pagination for large categories, keyboard-accessible expansion/navigation, and clear loading, empty, and retryable error states. Users must be able to collapse an expanded category again.

For example, expanding Failure -> Success: 1 should reveal the matching pair and let the user inspect the baseline conversation and the successful attack conversation.

Describe alternatives you've considered, if relevant

Keeping aggregate counts only requires users to locate matching results manually. A single conversation link on a category would not work when it contains multiple pairs. Loading every conversation with the initial comparison would add unnecessary work for large Scenario runs. On-demand row expansion lets users investigate the transition they care about without that upfront cost.

Additional context

Follow-up to merged PR #3337. This belongs to Scenario run results, not general attack-result analytics: the saved run plan identifies the baseline, attack groups, and matched seed population.

Scope/coverage refresh, October 31, 2026: assigned to shashank03-dev and covered by open #3391. The aggregate comparison panel, backend pairing/counts, result Details, saved-conversation routes, Scenario context, and transcript/media renderer already exist. Extend the existing comparison with cohort-consistent paged pair rows; do not build another results table, conversation viewer, or saved message-piece paging layer.

For example, a compound result without its own conversation needs a clearly labelled link to the appropriate saved child conversation while keeping the parent outcome and the aggregate pair population authoritative. It must not count that child as a new pair.

Coordinate #3391 with the results/UI stack, particularly #3643's proposed removal/replacement of the Paired outcomes panel. That unmerged presentation proposal does not retire this agreed drill-down requirement. Scope 10A/#3602/#3611 supplies general parent/child navigation; scope 10B supplies bounded saved transcript reads. Neither duplicates this issue's baseline/attack cohort matching.

Activity

  1. shashank03-dev commented on Oct 20, 2026

    @shashank03-dev
    Contributor

    Happy to take this on! Could you assign it to me? 😄

  2. changed the title [-][BLOCKED] FEAT: Drill into paired Scenario outcome conversations[/-] [+]FEAT: Drill into paired Scenario outcome conversations[/+] on Oct 28, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions