Skip to content

"ValueError: Pytest terminal summary report not found" when debugging pytester test that uses runpytest_subprocess on python 3.14 #25581

Open
Open
"ValueError: Pytest terminal summary report not found" when debugging pytester test that uses runpytest_subprocess on python 3.14#25581
Assignees
eleanorjboyd
Labels
bugIssue identified by VS Code Team member as probable bug

Description

@DetachHead
DetachHead
opened on Nov 11, 2025

Type: Bug

Behaviour

when using pytest's pytester fixture to test my pytest plugin, my tests fail when debugging the test, but pass when running the tests normally. this only seems to occur on python 3.14. it works fine on python 3.13

tested with both pytest 8.4.0 and 9.0.0, same behavior.

Steps to reproduce:

  1. create the following files:
# tests/test_foo.py

from pytest import Pytester

def test_foo(pytester: Pytester):
    pytester.makepyfile("def test_foo(): ...")
    result = pytester.runpytest_subprocess()
    result.assert_outcomes(passed=1)
# tests/conftest.py

pytest_plugins = ["pytester"]
// .vscode/settings.json
{
    "python.testing.pytestArgs": [
        "tests"
    ],
    "python.testing.unittestEnabled": false,
    "python.testing.pytestEnabled": true
}
  1. click the test explorer and click the run button on test_foo (it passes)
  2. right click the test and select "Debug Test" (it fails)

debug console output

python 3.14 

Running pytest with args: ['-p', 'vscode_pytest', '--rootdir=/home/me/projects/test', '--capture=no', '/home/me/projects/test/tests/test_foo.py::test_foo']
============================= test session starts ==============================
platform linux -- Python 3.14.0, pytest-8.4.2, pluggy-1.6.0
rootdir: /home/me/projects/test
configfile: pyproject.toml
collected 1 item

tests/test_foo.py running: /home/me/projects/test/.venv/bin/python -mpytest --basetemp=/tmp/pytest-of-me/pytest-36/test_foo0/runpytest-0
     in: /tmp/pytest-of-me/pytest-36/test_foo0
0.00s - Debugger warning: It seems that frozen modules are being used, which may
0.00s - make the debugger miss breakpoints. Please pass -Xfrozen_modules=off
0.00s - to python to disable frozen modules.
0.00s - Note: Debugging will proceed. Set PYDEVD_DISABLE_FILE_VALIDATION=1 to disable this validation.
No module named pytest
F

=================================== FAILURES ===================================
___________________________________ test_foo ___________________________________

pytester = <Pytester PosixPath('/tmp/pytest-of-me/pytest-36/test_foo0')>

    def test_foo(pytester: Pytester):
        pytester.makepyfile("def test_addition(): ...")
        result = pytester.runpytest_subprocess()
>       result.assert_outcomes(passed=1)

/home/me/projects/test/tests/test_foo.py:6:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
/home/me/projects/test/.venv/lib/python3.14/site-packages/_pytest/pytester.py:566: in parseoutcomes
    return self.parse_summary_nouns(self.outlines)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

cls = <class '_pytest.pytester.RunResult'>, lines = []

    @classmethod
    def parse_summary_nouns(cls, lines) -> dict[str, int]:
        """Extract the nouns from a pytest terminal summary line.

        It always returns the plural noun for consistency::

            ======= 1 failed, 1 passed, 1 warning, 1 error in 0.13s ====

        Will return ``{"failed": 1, "passed": 1, "warnings": 1, "errors": 1}``.
        """
        for line in reversed(lines):
            if rex_session_duration.search(line):
                outcomes = rex_outcome.findall(line)
                ret = {noun: int(count) for (count, noun) in outcomes}
                break
        else:
>           raise ValueError("Pytest terminal summary report not found")
E           ValueError: Pytest terminal summary report not found

/home/me/projects/test/.venv/lib/python3.14/site-packages/_pytest/pytester.py:584: ValueError
=========================== short test summary info ============================
FAILED tests/test_foo.py::test_foo - ValueError: Pytest terminal summary repo...
============================== 1 failed in 0.91s ===============================

python 3.13.8 

Running pytest with args: ['-p', 'vscode_pytest', '--rootdir=/home/me/projects/test', '--capture=no', '/home/me/projects/test/tests/test_foo.py::test_foo']
============================= test session starts ==============================
platform linux -- Python 3.13.8, pytest-8.4.2, pluggy-1.6.0
rootdir: /home/me/projects/test
configfile: pyproject.toml
collected 1 item

tests/test_foo.py running: /home/me/projects/test/.venv/bin/python -mpytest --basetemp=/tmp/pytest-of-me/pytest-37/test_foo0/runpytest-0
     in: /tmp/pytest-of-me/pytest-37/test_foo0
============================= test session starts ==============================
platform linux -- Python 3.13.8, pytest-8.4.2, pluggy-1.6.0
0.00s - Debugger warning: It seems that frozen modules are being used, which may
0.00s - make the debugger miss breakpoints. Please pass -Xfrozen_modules=off
0.00s - to python to disable frozen modules.
0.00s - Note: Debugging will proceed. Set PYDEVD_DISABLE_FILE_VALIDATION=1 to disable this validation.

rootdir: /tmp/pytest-of-me/pytest-37/test_foo0
collected 1 item

test_foo.py .                                                            [100%]

============================== 1 passed in 0.01s ===============================
.

============================== 1 passed in 1.54s ===============================

note the "No module named pytest" that gets printed when running with python 3.14, that looks sus.

debug console output

Diagnostic data

Output for Python in the Output panel (View→Output, change the drop-down the upper-right of the Output panel to Python) 

2025-11-12 01:25:53.146 [info] arg: --rootdir already exists in args, not adding.
2025-11-12 01:25:53.147 [info] Attempting to use temp directory for test ids file, file name: test-ids-c7f03da52498b119f6b8.txt
2025-11-12 01:25:53.147 [info] Environment variables set for pytest execution: PYTHONPATH=/home/me/.vscode-oss/extensions/ms-python.python-2025.16.0-universal/python_files, TEST_RUN_PIPE=/run/user/1000/python-test-results-274bda5030eacd4c916c, RUN_TEST_IDS_PIPE=/run/user/1000/test-ids-c7f03da52498b119f6b8.txt
2025-11-12 01:25:53.147 [info] Running DEBUG pytest with arguments: --rootdir=/home/me/projects/test,--capture=no for workspace /home/me/projects/test 

2025-11-12 01:25:53.148 [info] Using configuration in workspace
2025-11-12 01:25:55.958 [info] Test Result named pipe /run/user/1000/python-test-results-274bda5030eacd4c916c  cancelled

Extension version: 2025.16.0
VSCodium version: VSCodium 1.105.17075 (14bd1561ce547502e6ff1968090dc18c49160aab, 2025-10-21T20:24:03.344Z)
OS version: Linux x64 6.8.0-87-generic
Modes:

  • Python version (& distribution if applicable, e.g. Anaconda): 3.14.0
  • Type of virtual environment used (e.g. conda, venv, virtualenv, etc.): Venv
  • Value of the python.languageServer setting: None

Activity

  1. github-actions
    added
    triage-neededNeeds assignment to the proper sub-team
    on Nov 11, 2025

  2. eleanorjboyd commented on Nov 11, 2025

    @eleanorjboyd
    Eleanor Boyd (eleanorjboyd)
    on Nov 11, 2025
    Member

    if you instead create a launch.json config and try running pytest that way does it work?

  3. Eleanor Boyd (eleanorjboyd)
    self-assigned this
    on Nov 11, 2025
  4. Eleanor Boyd (eleanorjboyd)
    added
    info-neededIssue requires more information from poster
    on Nov 11, 2025

  5. DetachHead commented on Nov 12, 2025

    @DetachHead
    DetachHead
    on Nov 12, 2025
    Author

    same thing seems to happen with the following launch.json:

    {
  "version": "0.2.0",
  "configurations": [
    {
      "name": "pytest with args",
      "type": "debugpy",
      "request": "launch",
      "module": "pytest",
      "args": "${command:pickArgs}",
      "justMyCode": false
    }
  ]
}
  6. github-actions
    removed
    info-neededIssue requires more information from poster
    on Nov 12, 2025
  7. Eleanor Boyd (eleanorjboyd)
    added
    bugIssue identified by VS Code Team member as probable bug
    on Dec 18, 2025
  8. DetachHead
    added 2 commits that reference this issue on Jan 29, 2026

    pin python version to 3.13 due to microsoft/vscode-python#25581

    047723b

    pin python version to 3.13 due to microsoft/vscode-python#25581

    9fd0f32
  9. Anthony Kim (anthonykim1)
    removed
    triage-neededNeeds assignment to the proper sub-team
    on Mar 13, 2026

  10. DetachHead commented on May 13, 2026

    @DetachHead
    DetachHead
    on May 13, 2026
    Author

    this is now even more broken:

    • no errors are printed to the debug console output anymore:

      Running pytest with args: ['-p', 'vscode_pytest', '--rootdir=/mnt/projects/test', '--capture=no', '/mnt/projects/test/tests/test_foo.py::test_foo']
============================= test session starts ==============================
platform linux -- Python 3.13.8, pytest-9.0.3, pluggy-1.6.0
rootdir: /mnt/projects/test
configfile: pyproject.toml
collected 1 item

tests/test_foo.py running: /mnt/projects/test/.venv/bin/python -mpytest --basetemp=/tmp/pytest-of-me/pytest-39/test_foo0/runpytest-0
     in: /tmp/pytest-of-me/pytest-39/test_foo0

    • the issue is no longer unique to python 3.14, it's now broken on every python version i tested with (as far back as 3.10, idk about lower versions)

    this means i don't have a workaround and can no longer debug my tests at all.

    Version: 1.119.1
    Commit: 974500e64f0d1cfdf7c9821a2a51c2cb3bf0e561
    Date: 2026-05-12T01:20:22+09:00
    Electron: 39.8.8
    ElectronBuildId: 13870025
    Chromium: 142.0.7444.265
    Node.js: 22.22.1
    V8: 14.2.231.22-electron.0
    OS: Linux x64 6.8.0-111-generic

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

bugIssue identified by VS Code Team member as probable bug

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions