Skip to content

Try not simplifying unions during unification - #22035

Merged
ilevkivskyi merged 4 commits into
python:masterfrom
ilevkivskyi:try-no-simplify
Sep 25, 2026
Merged

ilevkivskyi merged 4 commits into
python:masterfrom
ilevkivskyi:try-no-simplify

Conversation

@ilevkivskyi

Copy link
Copy Markdown
Member

On one hand, this will likely degrade inference quality in some cases, but on the other hand this makes some core numeric libraries like pandas ~2x faster to type-check. Reasons why I think this should rarely affect users:

  • This only has effect when there is a redundant union item in a function annotation itself (as opposite to some type that we inferred, which may be redundant quite often).
  • This only affects subtyping for generic callables, which is rare in user code (unlike 3rd party libraries, where we don't show errors anyway).

Let's see how the primer looks.

@github-actions

This comment has been minimized.

@ilevkivskyi

Copy link
Copy Markdown
Member Author

Hm, primer output looks good, but performance improvement in primer for numerical libraries is much more modest that I see locally, only ~20% FWIW I think it is still worth it. cc @JukkaL @hauntsaninja

@github-actions

This comment has been minimized.

@JukkaL JukkaL left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! Type checking some numeric code is definitely quite slow currently, so optimizations like these are valuable.

Comment thread mypy/subtypes.py Outdated
type.ret_type, target.ret_type, return_constraint_direction
)
constraints.extend(c)
type_state.keep_unions = keep_unions_old

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wrap this around try/finally so that the new value won't stick around?

@github-actions

Copy link
Copy Markdown
Contributor

Diff from mypy_primer, showing the effect of this PR on open source code:

hydra-zen (https://github.com/mit-ll-responsible-ai/hydra-zen)
- src/hydra_zen/typing/_builds_overloads.py:229: error: Overloaded function implementation does not accept all possible parameters of signature 3  [misc]
- src/hydra_zen/typing/_builds_overloads.py:229: error: Overloaded function implementation does not accept all possible parameters of signature 5  [misc]
- src/hydra_zen/typing/_builds_overloads.py:229: error: Overloaded function implementation does not accept all possible parameters of signature 7  [misc]
- src/hydra_zen/typing/_builds_overloads.py:229: error: Overloaded function implementation does not accept all possible parameters of signature 9  [misc]
- src/hydra_zen/typing/_builds_overloads.py:468: error: Overloaded function implementation does not accept all possible parameters of signature 2  [misc]
- src/hydra_zen/typing/_builds_overloads.py:468: error: Overloaded function implementation does not accept all possible parameters of signature 4  [misc]
- src/hydra_zen/typing/_builds_overloads.py:468: error: Overloaded function implementation does not accept all possible parameters of signature 6  [misc]
- src/hydra_zen/typing/_builds_overloads.py:468: error: Overloaded function implementation does not accept all possible parameters of signature 8  [misc]
- src/hydra_zen/typing/_builds_overloads.py:468: error: Overloaded function implementation does not accept all possible parameters of signature 10  [misc]
- src/hydra_zen/typing/_builds_overloads.py:654: error: Overloaded function implementation does not accept all possible parameters of signature 1  [misc]
- src/hydra_zen/typing/_builds_overloads.py:654: error: Overloaded function implementation does not accept all possible parameters of signature 5  [misc]

@ilevkivskyi
ilevkivskyi merged commit 309f227 into python:master Sep 25, 2026
29 of 39 checks passed
@ilevkivskyi
ilevkivskyi deleted the try-no-simplify branch September 25, 2026 00:59
@ilevkivskyi

Copy link
Copy Markdown
Member Author

Btw OoC I tried to compare simple mypy -c "import pandas", and it went down from 3.2 sec on v2.3.1 (compiled) to 2.4 sec on current master (compiled), so I think it was definitely worth it after all.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants