Skip to content

test: add complex visual workflow campaign - #284

Merged
abrichr merged 11 commits into
mainfrom
codex/complex-visual-workflow-benchmark
Aug 6, 2026
Merged

test: add complex visual workflow campaign#284
abrichr merged 11 commits into
mainfrom
codex/complex-visual-workflow-benchmark

Conversation

@abrichr

@abrichr abrichr commented Aug 5, 2026

Copy link
Copy Markdown
Member

Summary

Adds a reusable complex visual workflow campaign that combines pixel-only multi-window interaction with email, CSV/spreadsheet, document handling, branches, loops, and independent effect verification.

Campaign

  • 27 trials across healthy and fault conditions
  • 12 expected VERIFIED outcomes
  • 15 expected safe halts
  • wrong entity, ambiguity, focus theft, stale frame, partial render, display drift, reconnect, and commit-timeout
  • 0 silent incorrect successes
  • 0 over-halts
  • 0 model calls

The public change contains the general harness and synthetic fixture only. It does not publish tuned thresholds, grown failure data, or customer recipes.

Verification

  • focused Pytest campaign tests
  • direct 27-trial campaign
  • Ruff
  • source-boundary validation

@abrichr
abrichr merged commit 1082192 into main Aug 6, 2026
2 checks passed
@abrichr
abrichr deleted the codex/complex-visual-workflow-benchmark branch August 6, 2026 11:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant