Repository navigation
runTests can't find tests in a file #25662
Description
The runTests tool cannot find any tests in my E2E tests file (e2e.py), even though it does have many tests. It might be because I named it "e2e.py" instead of "e2e_test.py" or "test_e2e.py", the standard pytest test discovery naming. That's intentional, as I always run it separately from the fast pytest tests.
Here's the file:
https://github.com/Azure-Samples/azure-search-openai-demo/blob/main/tests/e2e.py
GitHub Copilot logs are below.
Version: 1.104.0-insider (Universal)
Commit: e296bdfe0313d571a6f58399b22afd199f6da454
Date: 2025-09-08T05:26:36.685Z
Electron: 37.3.1
ElectronBuildId: 12259562
Chromium: 138.0.7204.235
Node.js: 22.18.0
V8: 13.8.258.31-electron.0
OS: Darwin arm64 23.6.0
Steps to Reproduce:
- Open https://github.com/Azure-Samples/azure-search-openai-demo/
- Use "fixer" mode in GitHub Copilot
- Ask it to run the E2E playwright tests
Tool Call - call_IU2uqaWgR9kLS0x409vkP0n3__vscode-1757360813940
Request
id : call_IU2uqaWgR9kLS0x409vkP0n3__vscode-1757360813940
tool : runTests
args : {
"files": [
"/Users/pamelafox/azure-search-openai-demo/tests/e2e.py"
]
}
Response
No tests found in the files. Ensure the correct absolute paths are passed to the tool.
Activity
AnasJadoon31 commentedon Sep 10, 2025
pamelafox commentedon Sep 10, 2025
Muhammad Anas (@AnasJadoon31) I know all of that. (Was that LLM-generated?) My issue is about runTests tool, and whether it can find tests when explicitly given a filename that does not fit the test-discovery naming pattern. The pytest command can do so, so it seems reasonable that runTests tool could be extended to do so as well.
eleanorjboyd commentedon Sep 12, 2025
Hi Pamela Fox (@pamelafox)! If you run pytest from your terminal can it find and run these tests? The test behavior should behave just like the terminal so if you are able to get it running and confirm the args are the same between the terminal and running in VS Code then this will be a bug I can investigate. Thanks!
pamelafox commentedon Sep 12, 2025
Eleanor Boyd (@eleanorjboyd) I can run
pytest tests/e2e.pyand it runs them correctly. It won't find them if I just saypytestsince I haven't named the file with test_/_testeleanorjboyd commentedon Sep 12, 2025
ok that makes sense- so you would need the tool to provide run args when it does it call since it would need to run
pytest tests/e2e.py. I will investigate model being able to provide args for the python test runs and how this could work since I don't think it is supported at all right nowwiperpaul commentedon Oct 16, 2025
Hey I also wanted to chime in here, i'm on version (1.105.1); I also had to tell chat to explicitly use the runtests tool otherwise it was trying to use the pytest terminal commands.
I also found it discovered the tests better after I added some explicit settings in settings.json. A follow up task used this in a few instances. The test blade and discovery was already configured for pytest and working before this.
I ran this a few times in different repos and sometimes a "Discovering tests..." step runs and performs a search for the test folders but often it just runs without any file paths or with the path of file you want to run the tests for.