Skip to content

[Bug]: Method-reference factory wrappers bypass exclusive groups and enable both OTel views #856

Open

Description

@zhongkechen

Expected Behavior

Registering both Execution and Invocation OTel views should fail with a diagnostic naming both views, including when either factory is wrapped with a method reference or delegating lambda. A wrapper should not allow both mutually exclusive views to run. Keep single-view configurations and unrelated plugins valid.

Actual Behavior

Direct factories are rejected with:

Conflicting plugin factories ExecutionOtelPlugin and InvocationOtelPlugin in exclusive group 'durable-otel-view'. Configure only one plugin from this group.

Replacing the execution factory with executionFactory::createPlugin makes the same configuration succeed. The wrapper has getExclusiveGroup() == null, so both views receive lifecycle hooks.

A local, single-step execution exports 10 spans, including two Workflow spans and three duplicated (traceId, spanId) identities: DurableExecutionRoot, the step, and Workflow. The two step records have the same identity but different parents: Workflow versus Invocation. The two attempt spans have distinct IDs. The execution succeeds and the step body runs once; the duplication is in instrumentation.

This was reproduced with either view wrapped and both registration orders (four cases), using an in-memory exporter and no propagated parent. The existing 13 OtelViewRegistrationTest cases also pass.

Steps to Reproduce

Add this JUnit test under otel-plugin/src/test/java/software/amazon/lambda/durable/ and run it with the module's existing test dependencies. No AWS calls or deployment are needed.

package software.amazon.lambda.durable;

import static org.junit.jupiter.api.Assertions.*;

import io.opentelemetry.sdk.testing.exporter.InMemorySpanExporter;
import io.opentelemetry.sdk.trace.SdkTracerProvider;
import io.opentelemetry.sdk.trace.export.SimpleSpanProcessor;
import java.util.concurrent.atomic.AtomicInteger;
import org.junit.jupiter.api.Test;
import software.amazon.lambda.durable.model.ExecutionStatus;
import software.amazon.lambda.durable.otel.ExecutionOtelPlugin;
import software.amazon.lambda.durable.otel.InvocationOtelPlugin;
import software.amazon.lambda.durable.otel.OtelPluginConfig;
import software.amazon.lambda.durable.plugin.DurableExecutionPluginFactory;
import software.amazon.lambda.durable.testing.LocalDurableTestRunner;

class OtelFactoryWrapperTest {
    @Test
    void wrappedFactoryBypassesExclusivity() {
        var exporter = InMemorySpanExporter.create();
        var options = OtelPluginConfig.builder()
                .contextExtractor(() -> null).enableMdc(false).build();
        var executionFactory = ExecutionOtelPlugin.factory(
                SdkTracerProvider.builder()
                        .addSpanProcessor(SimpleSpanProcessor.create(exporter)), options);
        var invocationFactory = InvocationOtelPlugin.factory(
                SdkTracerProvider.builder()
                        .addSpanProcessor(SimpleSpanProcessor.create(exporter)), options);

        assertThrows(IllegalArgumentException.class, () -> DurableConfig.builder()
                .withPlugins(executionFactory, invocationFactory).build());
        assertTrue(exporter.getFinishedSpanItems().isEmpty());

        DurableExecutionPluginFactory wrapper = executionFactory::createPlugin;
        assertNull(wrapper.getExclusiveGroup());
        var config = DurableConfig.builder()
                .withPlugins(wrapper, invocationFactory).build(); // Unexpectedly accepted.
        var calls = new AtomicInteger();
        var runner = LocalDurableTestRunner.create(String.class,
                (input, ctx) -> ctx.step("greet", String.class, stepCtx -> {
                    calls.incrementAndGet();
                    return input;
                }), config);
        assertEquals(ExecutionStatus.SUCCEEDED, runner.runUntilComplete("hello").getStatus());
        assertEquals(1, calls.get());
        var spans = exporter.getFinishedSpanItems();
        assertEquals(10, spans.size());
        assertEquals(2, spans.stream().filter(s -> s.getName().equals("Workflow")).count());
        assertEquals(7, spans.stream()
                .map(s -> s.getTraceId() + ":" + s.getSpanId()).distinct().count());
    }
}
mvn -pl otel-plugin -am test -Dtest=OtelFactoryWrapperTest \
  -Dsurefire.failIfNoSpecifiedTests=false

The assertions above describe the faulty behavior. The desired regression test should instead require rejection before both views become active.

SDK Version

Source at 4b1aa82b0d898fea495cda45760ce3737fbb341e, with SDK, testing SDK, and OTel plugin built together; Maven version 3.0.0.

Java Version

21 (Amazon Corretto 21.0.11), macOS arm64.

Is this a regression?

Yes: the previously enforced one-view invariant regressed during the per-invocation factory migration in PR #782, merged 2026-10-21 at 00:48:17 UTC (October 20 in PDT), commit 953c3771d4374a8bfeaa8653b133a8fa83d624dd. The literal method-reference factory registration is a new API surface; it was not a supported 2.x snippet.

Additional Context

This is a remaining wrapper case related to the closed exclusivity issue #758. Direct registration is now rejected; this report exercises the factory-based API through a method reference.

The immediate workaround is to pass the original factory. If a wrapper is required, explicitly forward getExclusiveGroup() and the factory identity used in diagnostics.

Related SDK issues

Regression history

  • Original fix: PR #766, merged 2026-10-07 at 18:09:21 UTC, commit 95a63f3a9db5d327620d76fb95b8372f53db6eaa, closed [Bug]: enabling both OTel views is accepted without mutual-exclusion validation #758. PluginRunner inspected the actual plugin class and superclass annotations. A helper returning the actual OTel instance retained its exclusive group.
  • Last release retaining that protection: v2.3.0. Its PluginRunner is identical to the immediate pre-migration main version.
  • Reintroduction: PR #782 / 953c3771 replaced actual-instance annotation inspection with factory-only getExclusiveGroup() validation, whose default is null. The legacy annotation now explicitly says SDK 3.x does not inspect it. No returned-instance check replaces the removed protection.
  • First public GitHub release affected by the new factory-wrapper path: v3.0.0, published 2026-10-26 at 21:33:41 UTC. Its PluginRunner and factory interface match the introducing migration, and those paths are unchanged through the reproduced revision.

The history is established by source/API, tag, and GitHub release comparisons; the runtime reproduction above uses the current investigated source, not rebuilt 2.x artifacts. The precise regression is weakened enforcement for actual plugin types behind factories. Arbitrarily wrapping a plugin instance itself in an unannotated forwarding implementation is a separate case; this does not claim the old API prevented every possible metadata-stripping wrapper. All dates in this history are UTC.

V2 verification: wrapper scope clarification

The released 2.3.0 tag (6a1d8dd273ceec72beb178bc8da6bfc9d6a1aa48) was built and tested with matching core/testing/OTel 2.3.0 artifacts:

  • Direct instances and a helper or real ServiceLoader provider returning the actual OTel instance are rejected before spans are exported.
  • A custom unannotated ForwardingPlugin implements DurableExecutionPlugin that delegates the public hooks to the real OTel instance bypasses validation. This was reproduced for either wrapped view, both registration orders, and through a real ServiceLoader provider returning that wrapper. Execution succeeds, the step runs once, and the exporter receives ten spans, two Workflow records, and three duplicated identities (root, step, Workflow). Attempt IDs are distinct. Seven focused JUnit cases passed.
  • The v2.3.0 validator traverses the registered object's class/superclass annotations; it does not follow a wrapper's delegate.
  • Released Maven 2.2.1 JARs were tested separately: direct registration is accepted, with one step call, two Workflow records, eight spans total, and duplicate step/Workflow identities. The original guard first shipped in 2.3.0; 2.2.0 and 2.2.1 sources have no exclusivity guard.

Therefore, 2.3.0 is not a universally wrapper-safe baseline. The #782 attribution above applies to the new 3.x factory-method-reference path. A separate unannotated plugin-instance/hook-wrapper path already existed in v2. The original #766 fix protected direct actual-instance registration but did not make arbitrary delegating plugin implementations preserve the delegate's annotation.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingneeds-triageIssue needs triagepkg:pluginsPlugin interface, possibly impacting all plugins

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions