Skip to content

feat(expectation): add LOGGING expectation support for collectors (#8756) - #374

Open
Yann (impolitepanda) wants to merge 1 commit into
mainfrom
feat/8746_logging-expectation-support
Open

Yann (impolitepanda) wants to merge 1 commit into
mainfrom
feat/8746_logging-expectation-support

Conversation

@impolitepanda

Copy link
Copy Markdown
Member

Proposed changes

  • Add the LOGGING expectation type: LoggingExpectation model (labels Logged / Not Logged), and LOGGING in the expectation, signature and contract enums, so injectors can offer it later (ADR-018 §4.4).
  • Fetch LOGGING expectations from the dedicated route GET /injects/expectations/logging/{source_id} (logging_expectations_for_source, logging_expectation_models_for_source). An older platform answers 404: it is treated as "no LOGGING expectation", with one warning (ADR-018 §4.2).
  • expectations_models_for_source maps DETECTION and PREVENTION explicitly and skips any other type with a warning, instead of building a PreventionExpectation. No change for existing collectors: /assets/{sourceId} only returns Prevention and Detection expectations.
  • CollectorDaemon declares the expectation types the collector supports (collector_supported_expectation_types) at registration and on every ping. Default: DETECTION + PREVENTION, so LOGGING is opt-in per collector. update_supported_expectation_types changes the list at runtime, e.g. to withdraw LOGGING when the log source becomes unusable.
  • Signatures fix: ExtraSignatureData.get_extra handles logging and returns {} for unknown types. Before, nmap, nuclei and netexec would fail the whole signature payload, Detection signatures included, as soon as an inject carried a LOGGING expectation.
  • Traces accept the optional inject_expectation_trace_match_type (marker, source_ip, target_ip).
  • Typing and refactor, not strictly required: the list check of ai_expectations_for_source moves to a shared helper (same messages, covered by the existing tests), the expectation models are built by one typed helper, and expectations_assets_for_source is annotated as returning a list, which it always did.

Testing Instructions

  1. coverage run -m pytest then coverage report -m --fail-under=80.
  2. isort --check-only ., black --check ., flake8 --ignore=E,W .
  3. Further testing will have to be done once feat(logging): collector API and supported expectation types (#8742) openaev#8811 will be merged, and probably once Splunk dev will be done as well.
  4. No regression should happen

Related issues

Checklist

  • I consider the submitted work as finished
  • [-] I tested the code for its functionality (but mocked only)
  • I wrote test cases for the relevant uses case
  • I added/update the relevant documentation (either on github or on notion)
  • Where necessary I refactored code to improve the overall quality

Further comments

Copilot AI balanced review requested due to automatic review settings October 29, 2026 17:12
@github-actions github-actions Bot added the filigran team Item from the Filigran team. label Oct 29, 2026

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

No active review issues were identified in the logging-expectation support and collector capability changes.

No open issues 🎉

What changed in this PR

This PR adds client-library support for LOGGING expectations, preparing collectors and injectors to use the platform’s new logging workflow while keeping existing collectors opted out by default.

Changes:

  • Adds logging models, enums, signature extras, and an optional trace match field.
  • Adds dedicated logging retrieval with a one-time warning on 404, plus explicit model mapping that skips unsupported types.
  • Declares collector capabilities at registration and on pings, supports runtime updates, and adds regression tests.
File Description
test/​signatures/​test_signature_manager_build_payload.py Tests logging signature groups and extra-data handling.
test/​daemons/​test_collector_daemon.py Tests capability defaults, validation, registration, and runtime updates.
test/​contracts/​test_contract_expectations.py Tests logging contract serialization.
test/​apis/​inject_expectation/​test_inject_expectation.py Tests logging retrieval, 404 fallback, and model filtering.
test/​apis/​expectation/​test_expectation.py Tests logging result labels and success updates.
pyoaev/​signatures/​types.py Adds the logging signature expectation type.
pyoaev/​signatures/​models.py Supports logging extras and returns empty extras for unknown types.
pyoaev/​daemons/​collector_daemon.py Declares and updates supported expectation types.
pyoaev/​contracts/​contract_config.py Adds logging to contract types and related documentation.
pyoaev/​apis/​inject_expectation/​model/​expectation.py Defines the logging model and result labels.
pyoaev/​apis/​inject_expectation/​model/​__init__.py Exports the logging model.
pyoaev/​apis/​inject_expectation/​inject_expectation.py Adds logging retrieval and shared response validation and model construction.
pyoaev/​apis/​inject_expectation_trace.py Accepts the optional trace match type.
pyoaev/​apis/​collector.py Allows supported expectation types in collector registration.

Review effort: Balanced

optional=(
# Signal that matched, for a LOGGING expectation: "marker",
# "source_ip" or "target_ip". Left out for the other types.
"inject_expectation_trace_match_type",

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

would a small enum (marker, source_ip, target_ip) help collectors here? The platform silently drops an unknown value, so a typo would go unnotice

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A quality of life, and a little warning/question about the default _supported_expectation_types.

But globally a good contribution, thanks for the work done on that PR!

LGTM.

def expectations_assets_for_source(
self, source_id: str, expiration_time: int = None, **kwargs: Any
) -> Dict[str, Any]:
) -> List[Dict[str, Any]]:

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nitpick (non-blocking): Now that this is annotated List[Dict] and _as_list_of_dicts exists, could /assets use it too? A malformed body would then raise OpenAEVParsingError instead of an AttributeError in _models_from_dicts. Fine as a follow-up.

Comment on lines +84 to +88
self._supported_expectation_types = _validate_expectation_types(
DEFAULT_SUPPORTED_EXPECTATION_TYPES
if supported_expectation_types is None
else supported_expectation_types
)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

question: Is it intentional that every collector (including payload/vulnerability collectors like mitre-attack or nvd-nist-cve) always sends the legacy default?

gonna need to ensure that those collectors properly set supported expectation_types !

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

filigran team Item from the Filigran team.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants