test(ffe): cover billed source precedence

[leoromanovsky]

Motivation

Feature Flags delivery has billing implications. These tests prove that new applications do not poll before provider access, existing Remote Configuration adopters are grandfathered, explicit precedence is honored, and kill-switch or invalid configurations cannot create phantom CDN or FFE_FLAGS traffic.

flowchart TD
  Matrix["Configuration precedence matrix"] --> SDK["Node.js or Java tracer"]
  SDK -->|"agentless"| CDN["Mock CDN request counter"]
  SDK -->|"remote_config"| RC["Agent FFE_FLAGS capability and product"]
  SDK -->|"disabled"| Silent["No Feature Flags delivery traffic"]
  CDN --> Assert["Assertions observe both billing surfaces"]
  RC --> Assert
  Silent --> Assert

Loading

Changes

flowchart LR
  Cases["10 source-selection cases"] --> Env["Canonical and legacy env inputs"]
  Env --> Controller["Parametric FFE controller"]
  Controller --> Boundary["/ffe/start provider-access boundary"]
  Boundary --> Mock["Mock agentless backend"]
  Controller --> Agent["Agent RC state"]
  Generic["Generic endpoint parity"] --> Noop["Ready no-op provider"]
  Dynamic["Generic RC capability test"] --> Explicit["Explicit remote_config opt-in"]
  Explicit --> Agent

Loading

• Extend the mocked agentless backend with observable configuration requests.
• Cover default agentless activation, explicit billed-source opt-ins, DD_FEATURE_FLAGS_ENABLED=false, and the legacy migration window.
• Exercise explicit-source-over-legacy precedence and fail-closed invalid/offline behavior.
• Make the Java and Node.js parametric controllers access the real provider for Feature Flags configuration tests while retaining the ready no-op provider for unrelated endpoint-parity tests.
• Make the generic dynamic-configuration suite explicitly select remote_config when it validates the Feature Flags RC capability.

Decisions

• Assertions observe both delivery surfaces: mock CDN requests and Agent Remote Configuration capabilities/products.
• Default agentless delivery must remain silent until /ffe/start crosses the application provider-access boundary.
• DD_FEATURE_FLAGS_ENABLED=false must suppress both CDN polling and the Feature Flags RC subscription.
• Explicit or grandfathered Remote Configuration must never fall back to CDN delivery.
• Generic RC tests that expect the FFE_FLAGS capability must explicitly opt into that billed source.
• The generic /ffe/start parity contract remains unchanged outside Feature Flags configuration tests.

Validation

• ./format.sh --check: passed.
• Node source-selection matrix against the latest stacked tracer: 10 passing.
• Node generic /ffe/start plus explicit RC capability completeness: 2 passing.
• Java source-selection matrix plus generic /ffe/start against the latest stacked agent: 11 passing.

[github-actions]

CODEOWNERS have been resolved as:

tests/parametric/test_dynamic_configuration.py                          @DataDog/system-tests-core @DataDog/apm-sdk-capabilities
tests/parametric/test_ffe/test_configuration_sources.py                 @DataDog/feature-flagging-and-experimentation-sdk @DataDog/system-tests-core
tests/test_the_test/test_mock_ffe_agentless_backend.py                  @DataDog/feature-flagging-and-experimentation-sdk @DataDog/system-tests-core
utils/build/docker/java/parametric/src/main/java/com/datadoghq/trace/controller/FeatureFlagEvaluatorController.java  @DataDog/apm-java @DataDog/asm-java @DataDog/system-tests-core
utils/build/docker/nodejs/parametric/server.js                          @DataDog/dd-trace-js @DataDog/system-tests-core
utils/docker_fixtures/_mock_ffe_agentless_backend.py                    @DataDog/feature-flagging-and-experimentation-sdk @DataDog/system-tests-core

[datadog-datadog-prod-us1-2]

🎉 All green!

🧪 All tests passed
❄️ No new flaky tests detected

_{This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 993b2b1 | Docs | Datadog PR Page | Give us feedback!}