Skip to content

Prefer for loops over parametrized tests in AGENTS.md - #4150

Open
kevinjqliu wants to merge 1 commit into
apache:mainfrom
kevinjqliu:kevinjqliu-agents-md-prefer-loops-over-parametrize
Open

kevinjqliu wants to merge 1 commit into
apache:mainfrom
kevinjqliu:kevinjqliu-agents-md-prefer-loops-over-parametrize

Conversation

@kevinjqliu

Copy link
Copy Markdown
Contributor

Rationale for this change

Adds an AGENTS.md testing guideline to prefer a for loop in a single test over @pytest.mark.parametrize. Each parametrized case is a separate pytest item with its own collection, setup, and reporting overhead, which adds up across the suite.

Measured locally with the project's tests/conftest.py (pytest -m unit -q, trivial bucket-transform assertion as the test body):

Cases parametrize for loop
1,000 ~0.30s ~0.01s
10,000 ~2.95s ~0.01s

Roughly 0.3ms of overhead per case. About 2,258 of the 4,022 collected unit tests (56%) are parametrized cases.

Are these changes tested?

Documentation-only change. prek hooks (markdownlint, codespell, end-of-file, trailing whitespace) passed on commit.

Are there any user-facing changes?

No.

Each parametrized case adds per-item pytest overhead (~0.3ms measured), which slows the suite as cases accumulate.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@kevinjqliu
kevinjqliu marked this pull request as ready for review October 24, 2026 18:32
Copilot AI balanced review requested due to automatic review settings October 24, 2026 18:32
@kevinjqliu

Copy link
Copy Markdown
Contributor Author

I noticed that a lot of new PRs tend to use @pytest.mark.parametrize when a for loop can suffice (my own generated PRs included 😄 )

Hopefully this nudge all the agents to do the right thing

@kevinjqliu
kevinjqliu requested review from Fokko and geruh October 24, 2026 18:33

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

The documentation-only change matches the stated intent and has no unresolved review findings.

No open issues 🎉

What changed in this PR

This documentation-only PR updates contributor testing guidance to reduce pytest overhead without changing runtime behavior.

Changes:

  • Prefer loops within a single test over parametrization.
  • Require input values in assertion messages to identify failing cases.
  • Reserve parametrization for cases needing different fixtures.
File Description
AGENTS.md Adds guidance on loops, assertion messages, and when to use parametrization.

Review effort: Balanced

@rambleraptor

Copy link
Copy Markdown
Collaborator

Can we add something about using multiple tests if the parameters are sufficiently complicated?

I've seen a number of tests that parameterize on 3-4+ very complicated fixtures. Even a for loop makes reading them a mess. The loss in readability doesn't make up for the few lines of code savings.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants