Skip to content

fix: violation record on exception - #148

Open
prakhar-singh1928 wants to merge 1 commit into
0.1.xfrom
fix/violation-record-on-exception
Open

fix: violation record on exception#148
prakhar-singh1928 wants to merge 1 commit into
0.1.xfrom
fix/violation-record-on-exception

Conversation

@prakhar-singh1928

Copy link
Copy Markdown
Contributor

Summary

Fixes the silent gap in PluginResult.executions when violations_as_exceptions=True — the denying plugin's ControlExecutionRecord was never written, making it impossible to identify which control blocked an invocation from telemetry alone.

Closes: #147


Changes

  • cpex/framework/manager.py_run_serial_phase: The except PluginViolationError block now appends a ControlExecutionRecord for the denying plugin (status=COMPLETED, effective_allow=False, matched=True, applied=True, error_code and reason from the violation) before re-raising. Previously this path was a bare raise with the comment "propagate immediately, no record needed".

  • cpex/framework/manager.pyexecute: The entire pipeline dispatch is now wrapped in a try/except PluginViolationError that attaches list(executions) to pve.executions before propagating. Callers catching PluginViolationError can now inspect pve.executions to see the full execution chain including the blocking plugin.

  • cpex/framework/errors.pyPluginViolationError: Added executions: list[ControlExecutionRecord] | None = None attribute. Defaults to None (backward compatible, no signature change). Set to the accumulated records list by execute() before the exception reaches the caller.

  • tests/unit/cpex/framework/test_execution_records.py: Two new tests directly assert pve.executions on the raised exception:

    • test_executions_on_violation_exception_single_plugin — one plugin denies; asserts 1 record with correct fields.
    • test_executions_on_violation_exception_two_plugin_chain — allow + deny chain; asserts 2 records in order, the second having effective_allow=False and the correct error_code.

Checks

  • make lint passes
  • make test passes
  • CHANGELOG updated (if user-facing)

Notes

No changes required on the ContextForge side. The existing except PluginViolationError workaround in tool_service.py (ctl_acc.mark_denied(hook="pre/post")) continues to work unchanged — pve.executions is purely additive.

…lationError

When violations_as_exceptions=True and a plugin denies an invocation,
the except PluginViolationError handler in _run_serial_phase previously
re-raised immediately without appending a ControlExecutionRecord for
the denying plugin to the executions accumulator.

This meant PluginResult.executions only contained records for plugins
that ran *before* the denial, making it impossible for consumers to
identify which specific control blocked the invocation from telemetry.

Changes:
- cpex/framework/manager.py (_run_serial_phase): Record the denying
  plugin's ControlExecutionRecord (status=COMPLETED, effective_allow=False,
  matched=True, applied=True, error_code from violation) before re-raising
  PluginViolationError.
- cpex/framework/manager.py (execute): Wrap the entire pipeline dispatch
  in a try/except PluginViolationError that attaches the accumulated
  executions list to the exception via pve.executions before propagating.
- cpex/framework/errors.py (PluginViolationError): Add executions attribute
  (list[ControlExecutionRecord] | None, default None) so callers catching
  PluginViolationError can inspect the full execution chain including
  the denying plugin.
- tests/unit/cpex/framework/test_execution_records.py: Add two new tests:
  - test_executions_on_violation_exception_single_plugin: asserts
    pve.executions contains exactly one record with correct fields.
  - test_executions_on_violation_exception_two_plugin_chain: asserts
    pve.executions contains two records (allow + deny) with correct fields.

Fixes: cpex#130
Reported-by: ContextForge tool_service.py:6628-6635, PR IBM/mcp-context-forge#6003
@prakhar-singh1928
prakhar-singh1928 changed the base branch from main to 0.1.x July 31, 2026 12:49
@araujof araujof changed the title Fix/violation record on exception fix: violation record on exception Jul 31, 2026
@araujof araujof added bug Something isn't working Python 0.1.x labels Jul 31, 2026
@araujof araujof added this to CPEX Jul 31, 2026
@github-project-automation github-project-automation Bot moved this to Backlog in CPEX Jul 31, 2026
@araujof araujof added this to the 0.1.3 milestone Jul 31, 2026
@araujof araujof moved this from Backlog to In review in CPEX Jul 31, 2026

@terylt terylt left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @prakhar-singh1928 , Thanks for the PR. Overall it's good. Here are some suggested changes:

[Medium — incomplete fix] The same gap remains for CONCURRENT-mode plugins

execute_plugin raises PluginViolationError for both SEQUENTIAL and CONCURRENT modes
when violations_as_exceptions=True (manager.py:1177, :1194-1204). But the deny record
is only appended in _run_serial_phase. In the concurrent path:

  • execute_plugin raises inside the task;
  • _tagged only catches PluginTimeoutError (manager.py:1028), so the PVE escapes the
    as_completed loop before the record-append at manager.py:451 and before the
    task-cancellation at :474;
  • the new outer handler then attaches an executions list that is missing the denying
    concurrent plugin
    — and the pending sibling tasks are never cancelled (pre-existing
    leak).

So a caller using violations_as_exceptions=True with any concurrent plugin still hits the
exact bug #147 describes. Recommend either (a) handling PVE in _tagged/the concurrent loop
symmetrically, or (b) explicitly scoping this PR to serial and noting the concurrent gap as
a follow-up.

Why it's serial-only: the record-writing code lives solely in
_run_serial_phase's exception handler. Concurrent plugins are dispatched in a separate
asyncio-task block that writes its record by inspecting a returned result
(manager.py:451). Because execute_plugin raises rather than returns for concurrent
denials under violations_as_exceptions=True, the exception unwinds past that append — that
concurrent-halt branch is effectively dead code in this mode.

[Low — traceability] Issue-number mismatch

The PR says "Closes #147", but every code comment, the errors.py docstring, and the test
section header reference cpex#130. #130 is the closed umbrella feature ("structured
control execution records"); #147 is the actual bug. The inline references should point at
#147.

[Nit] errors.py import is heavier than needed

errors.py already imports from cpex.framework.models at runtime
(PluginErrorModel, PluginViolation), so there's no circular-import risk in adding
ControlExecutionRecord to that existing line. The added from __future__ import annotations + TYPE_CHECKING guard works but is more machinery than required. Harmless
either way.

[Note — pre-existing] FAF records absent on the exception path

On a normal (non-exception) sequential halt, _build_halt_result schedules fire-and-forget
plugins and their records land in executions; on the violations_as_exceptions=True raise
path, FAF plugins don't fire, so pve.executions won't include them. Not introduced here,
but since the PR makes executions authoritative on the exception path, a one-line note
would keep consumers from assuming parity between the two halt shapes.


Test coverage

Good for the serial case. Gaps:

  • No concurrent-mode test (which would surface the Medium issue above).
  • No regression guard that the non-exception halt path still attaches executions
    correctly.
  • Consider asserting pve.executions is a distinct list from the manager's internal
    accumulator (the list(...) copy), so a later mutation can't leak back.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

0.1.x bug Something isn't working Python

Projects

Status: In review

Development

Successfully merging this pull request may close these issues.

[BUG]: PluginResult.executions missing ControlExecutionRecord for denying plugin when violations_as_exceptions=True

3 participants