Skip to content

Add Fraud Triage environment: transaction risk-decisioning RL task - #926

Merged
burtenshaw merged 24 commits into
openenv:mainfrom
atharvsatpute:add-fraud-triage-env
Oct 27, 2026
Merged

burtenshaw merged 24 commits into
openenv:mainfrom
atharvsatpute:add-fraud-triage-env

Conversation

@atharvsatpute

@atharvsatpute atharvsatpute commented Jul 6, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Adds a Fraud Triage environment: a sequential transaction risk-decisioning
RL task where an agent must approve/flag/escalate/block synthetic
transactions. Fills a gap in the existing finance environments (finrl_env
covers stock trading, finqa_env covers document QA) — neither covers
transaction-level risk decisioning.

Type of Change

  • Bug fix
  • New feature
  • Breaking change
  • Documentation
  • New environment
  • Refactoring

Alignment Checklist

Before submitting, verify:

  • I have read .claude/docs/PRINCIPLES.md and this PR aligns with our principles
  • I have checked .claude/docs/INVARIANTS.md and no invariants are violated
  • I have run /pre-submit-pr (or bash .claude/hooks/lint.sh and tests) and addressed all issues

RFC Status

  • Not required (bug fix, docs, minor refactoring)
  • RFC exists: #___
  • RFC needed (will create before merge)

Test Plan

Ran the existing test suite locally:

All 4 tests pass:

  • test_reset_returns_valid_observation
  • test_approve_and_block_actions_return_reward
  • test_episode_terminates_after_episode_length_steps
  • test_state_tracks_confusion_matrix_counts

Also manually verified end-to-end with a heuristic policy (amount z-score

  • new-device/new-merchant thresholds) achieving 100% fraud recall over a
    200-transaction episode, confirming the reward signal and episode
    termination behave as intended.

Environment follows existing Gym-style conventions (see connect4_env) —
Pydantic wire types, WebSocket-based client, no MCP tool exposure, rewards
computed entirely server-side inside step().

Claude Code Review

N/A


Note

Low Risk
Self-contained new environment and documentation with synthetic data only; no changes to core OpenEnv runtime, auth, or existing environments.

Overview
Adds a new fraud_triage_env OpenEnv package for sequential payment fraud triage: agents see one synthetic transaction at a time (risk features only; labels hidden) and choose APPROVE / FLAG / ESCALATE / BLOCK.

Server logic streams transactions from a lightweight TransactionGenerator, scores decisions with asymmetric rewards (heavy penalty for missed fraud, smaller cost for false positives, extra cost for escalate), and tracks episode confusion-matrix stats in state. Wiring follows the usual pattern: Pydantic models, FraudTriageEnv WebSocket client, FastAPI app via create_app, openenv.yaml, Docker image, and pyproject.toml.

Docs add fraud_triage to the environments index and toctree, plus a dedicated guide and env README. Tests cover seeded replay, reward/counter behavior, config validation, episode termination, and WebSocket reset/step/state parity with the client parsers.

Reviewed by Cursor Bugbot for commit d9e54c5. Bugbot is set up for automated code reviews on this repo. Configure here.

@cursor cursor Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Want higher recall? High effort reviews run extra passes and find more bugs. A team admin can switch effort levels in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 2ccb2a3. Configure here.

@atharvsatpute

Copy link
Copy Markdown
Contributor Author

Hi everyone, I've addressed the main items and the PR is ready. Could a maintainer please trigger the automated tests? (Note: I already checked the Bugbot output and everything looks good on that end). Thanks for your time!

@burtenshaw

Copy link
Copy Markdown
Collaborator

Thanks! Could you also deploy this space on hugging face for review of the runtime?

@bot-ci-comment

Copy link
Copy Markdown

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

@burtenshaw

Copy link
Copy Markdown
Collaborator

Runtime review on 0062719: the environment's Dockerfile built successfully, and the packaged server passed health, typed-client reset, a 200-step episode, exact termination, state-counter checks, post-terminal rejection without mutation, and deterministic seeded replay. The temporary container was removed and its absence verified. Image ID: sha256:468241ba1670e4efdeb9e7d682768ac4a6f43256781dcbc86175bed98e640873. This is local container evidence; no Hugging Face Space was published.

The 20 focused tests also pass. The docs now state the simplified reward semantics explicitly: FLAG and BLOCK have equal payoff, while ESCALATE adds a cost and provides no extra evidence.

@burtenshaw burtenshaw left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed 0062719. The post-terminal reward bug and current reset/step API incompatibilities are fixed, action values are validated, and seeded resets replay the full observation stream. All 20 focused tests and the packaged Docker server's 200-step typed-client smoke pass, including terminal rejection, state immutability, replay, and container removal. Required dependency lock and docs/catalog entries are present. The simplified FLAG/BLOCK/ESCALATE economics are now documented accurately. No remaining blocker found; no public Hugging Face deployment was performed.

@burtenshaw

Copy link
Copy Markdown
Collaborator

Refreshed onto main 07f26b0 after #1198 and wired FraudTriageState into the app in both import layouts. All 50 focused environment/State-wiring tests and touched-file lint pass at d9e54c5.

Rebuilt the actual Dockerfile from this head and verified the packaged app's source hash. The live local container passed health, /schema and /state with all eight subclass fields, typed-client reset, all 200 steps, exact termination, rejection of an extra step without state mutation, and deterministic replay. Container removal was verified. Image: sha256:4d90eeb8d103f061996af3fb90166c4a0300260f9e7c6cfdda6d5de5655878c4. Fresh CI is running.

@burtenshaw burtenshaw left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed the #1198 integration: FraudTriageState is imported in both package and standalone paths and passed to create_app. Independent validation passed 50 tests; HTTP /schema and /state returned 200 and retained all eight State fields. Touched-file lint passed.

@burtenshaw

Copy link
Copy Markdown
Collaborator

Reviewed and validated current head 0c2e23bf68844f3c48ca601f4baf69cb3b029296 against main 9b2c0bea: 20 scoped tests pass. Tests cover seeded replay, all action/reward cases, terminal-step rejection, and WebSocket serialization/client parsing. Touched lint passes. No Docker build was run locally. Refreshed CI and the final merge decision remain with the coordinating maintainer.

@burtenshaw
burtenshaw merged commit 8d9d66c into openenv:main Oct 27, 2026
13 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants