Skip to content

fix: don't call methods on mock objects while recording - #329

Merged
dividedmind merged 2 commits into
masterfrom
fix/dont-call-methods-on-mocks
Sep 18, 2026
Merged

dividedmind merged 2 commits into
masterfrom
fix/dont-call-methods-on-mocks

Conversation

@dividedmind

Copy link
Copy Markdown
Contributor

Recording a value called toString() on it. When the value is a mock, that call is an invocation as far as the mocking framework is concerned, and mocking frameworks keep thread-local state between one call and the next. Mockito collects argument matchers on a stack and binds them to the next invocation it sees, and when() applies to the most recent invocation. A toString() from the agent lands in the middle of that handshake, so Mockito attributes the matcher or the stubbing to the agent's call instead of the test's.

The result is that the stubbing a test set up silently never applies, and the stubbed method returns the Java type default. A test that does

when(service.getData()).thenReturn("expectedValue");

fails with "expected: but was:" under the agent and passes without it. Where all arguments use matchers, Mockito reports "2 matchers expected, 1 recorded" instead.

Recognize mocks and record a placeholder rather than asking them for a value. Detection goes through the mocking framework's own API, because the class name isn't a reliable signal: Mockito's inline mock maker (the default since 5.0) mocks a class by retransforming it, so a mock of com.example.Foo is an instance of com.example.Foo. The lookup is resolved against the class loader of the object being inspected and cached per class, so it costs nothing when no mocking framework is on the class path.

The new test project has a control task that runs the same tests with no agent attached, so a failure under the agent is attributable.

Assisted-by: Claude:claude-opus-5

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

MockDetector can return invalid null lookup results and cannot reliably detect mocks across isolated class loaders.

Get a fresh assessment by requesting another Copilot review.

Pull request overview

This PR prevents AppMap recording from invoking methods on Mockito mocks and adds integration coverage.

Changes:

  • Adds reflective Mockito detection with a [mocked] placeholder.
  • Adds Mockito regression tests with agent and control runs.
  • Adds standalone Gradle/Bats test-project configuration.
File summaries
File Summary
agent/test/mockito/src/test/java/com/example/mockito/MockRecordingTest.java Tests Mockito stubbing and recording behavior.
agent/test/mockito/src/main/java/com/example/mockito/Service.java Provides the instrumented test fixture.
agent/test/mockito/src/main/java/com/example/mockito/Calculator.java Defines the mocked collaborator.
agent/test/mockito/settings.gradle Defines the Gradle project.
agent/test/mockito/mockito.bats Runs control and agent integration tests.
agent/test/mockito/build.gradle Configures test tasks and dependencies.
agent/test/mockito/appmap.yml Configures recorded packages.
agent/test/mockito/.gitignore Ignores generated files.
agent/src/main/java/com/appland/appmap/util/MockDetector.java Detects Mockito mocks reflectively.
agent/src/main/java/com/appland/appmap/output/v1/Value.java Records placeholders for mocks.
Review details

Suppressed comments (2)

agent/src/main/java/com/appland/appmap/util/MockDetector.java:47

  • getClassLoader() == null does not prove an object cannot be a Mockito mock: Mockito's inline mock maker can instrument bootstrap-loaded concrete types such as java.util.ArrayList. This early return causes Value.freeze() to call toString() on those mocks, reintroducing the matcher-state corruption; use a fallback loader that can resolve Mockito and add a regression test for a bootstrap-loaded mock.
      if (loader == null) {
        // Bootstrap classes are never mocks.
        return null;

agent/src/main/java/com/appland/appmap/util/MockDetector.java:54

  • The API lookup is limited to the inspected object's class loader. With an isolated application loader, an inline mock can be an instance of an application class while Mockito is visible only from the test/context loader; this Class.forName then fails, and the cached null makes every later freeze call toString() on that mock. Please add a compatible context/system/agent-loader fallback instead of assuming the object loader can see Mockito.
        Method mockingDetails = Class.forName("org.mockito.Mockito", false, loader)
            .getMethod("mockingDetails", Object.class);
        Method isMock = Class.forName("org.mockito.MockingDetails", false, loader)
            .getMethod("isMock");
  • Files reviewed: 10/10 changed files
  • Comments generated: 1
  • Review effort level: Lite

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread agent/src/main/java/com/appland/appmap/util/MockDetector.java Outdated
@dividedmind
dividedmind force-pushed the fix/dont-call-methods-on-mocks branch from 0849838 to 20bc8b5 Compare September 16, 2026 15:12
Recording a value called toString() on it. When the value is a mock,
that call is an invocation as far as the mocking framework is concerned,
and mocking frameworks keep thread-local state between one call and the
next. Mockito collects argument matchers on a stack and binds them to
the next invocation it sees, and when() applies to the most recent
invocation. A toString() from the agent lands in the middle of that
handshake, so Mockito attributes the matcher or the stubbing to the
agent's call instead of the test's.

The result is that the stubbing a test set up silently never applies,
and the stubbed method returns the Java type default. A test that does

    when(service.getData()).thenReturn("expectedValue");

fails with "expected:<expectedValue> but was:<null>" under the agent and
passes without it. Where all arguments use matchers, Mockito reports
"2 matchers expected, 1 recorded" instead.

Recognize mocks and record a placeholder rather than asking them for a
value. Detection goes through the mocking framework's own API, because
the class name isn't a reliable signal: Mockito's inline mock maker
(the default since 5.0) mocks a class by retransforming it, so a mock
of com.example.Foo is an instance of com.example.Foo. The lookup is
resolved against the class loader of the object being inspected and
cached per class, so it costs nothing when no mocking framework is on
the class path.

The new test project has a control task that runs the same tests with
no agent attached, so a failure under the agent is attributable.

Assisted-by: Claude:claude-opus-5
@dividedmind
dividedmind force-pushed the fix/dont-call-methods-on-mocks branch from 20bc8b5 to 6bcd7dd Compare September 16, 2026 19:01
@dividedmind
dividedmind marked this pull request as ready for review September 17, 2026 19:23
@dividedmind
dividedmind requested review from kgilpin and a lite review from Copilot September 17, 2026 19:23

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

The functional change is well-contained and backed by targeted regression tests; remaining feedback is limited to a minor documentation correction.

Review details
  • Files reviewed: 10/10 changed files
  • Comments generated: 1
  • Review effort level: Lite

Comment thread agent/src/main/java/com/appland/appmap/util/MockDetector.java
Update comment to clarify Mockito's mock-detection behavior.

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
@dividedmind
dividedmind merged commit 4f09699 into master Sep 18, 2026
6 checks passed
@dividedmind
dividedmind deleted the fix/dont-call-methods-on-mocks branch September 18, 2026 17:22
@dividedmind

Copy link
Copy Markdown
Contributor Author

🎉 This PR is included in version 1.30.2 🎉

The release is available on:

Your semantic-release bot 📦🚀

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants