Skip to content

Supprt Python 3.15 with breaking changes. - #945

Draft
junkmd wants to merge 8 commits into
enthought:mainfrom
junkmd:py315_support
Draft

junkmd wants to merge 8 commits into
enthought:mainfrom
junkmd:py315_support

Conversation

@junkmd

@junkmd junkmd commented Sep 21, 2026

Copy link
Copy Markdown
Collaborator

fixes: #894 #938

1. Summary

This PR adds official support for Python 3.15 to comtypes.
It resolves internal CPython ABI/struct layout incompatibilities and updates the code generator to handle Python 3.15's changes to enum.IntFlag.

Warning

Breaking Change:
Enumerations containing negative member values are now generated as enum.IntEnum instead of enum.IntFlag. This avoids Python 3.15 range-masking behavior and mathematically inconsistent bitflag definitions, but may affect user code performing bitwise operations or IntFlag type checks on those enums.

2. Motivation & Background

  1. Python 3.15 PyCArgObject Layout Change:

    • In Python 3.15, the internal layout of PyCArgObject in ctypes changed (tag changed from c_char to c_char_p, and size changed from c_int to c_ssize_t).
    • Without an adaptive bridge, low-level offset calculations in comtypes.util fail or cause memory access violations on Python 3.15.
  2. Python 3.15 enum.IntFlag Negative Member Masking:

    • In Python 3.15, IntFlag reinterprets negative member values by masking them into the positive bit domain rather than preserving their negative literal values (e.g., -1).
    • Bitwise flag semantics on negative integer constants are conceptually flawed.
    • Generated friendly modules previously cast all enums unconditionally to IntFlag, causing negative constants (such as MsiInstallState.msiInstallStateUnknown = -1) to corrupt their literal values on Python 3.15+.

3. Breaking Changes & Migration Guide

Description of the Incompatibility

  • Previous behavior: All enums generated in friendly modules (comtypes.gen.<mod>) inherited from enum.IntFlag.
  • New behavior:
    • Enums containing at least one negative member value inherit from enum.IntEnum.
    • Enums containing only non-negative values continue to inherit from enum.IntFlag.

Impact on Your Code

  • Bitwise Operations: Bitwise operations (|, &, ^, ~) on negative-valued enums (e.g., MsiInstallState) will no longer return an enum instance or may raise a TypeError depending on usage.
  • Type Checking: Code relying on isinstance(val, enum.IntFlag) or issubclass(EnumType, enum.IntFlag) will return False for enums with negative values.

Migration Action

  • If your code uses bitwise operations on enums, ensure the enum does not represent distinct negative status/sentinel codes (e.g., check against IntEnum values using equality == rather than bitwise masking &).
  • Update type annotations or assertions that explicitly assumed enum.IntFlag.

4. Key Changes

Core & Code Generator

  • comtypes.tools.codegenerator.namespaces.EnumerationNamespaces:
    • Inspect member values to decide whether to derive from IntEnum (if negative values exist) or IntFlag.
    • Dynamically import IntEnum and/or IntFlag based on whether each is used.
  • comtypes.util:
    • Dynamically configure PyCArgObject fields based on sys.version_info >= (3, 15) (_TAG_TYPE = c_char_p, _SIZE_TYPE = c_ssize_t).

Tests & Documentation

  • comtypes.test.test_client: Added test_enum_base_classes to assert correct base class assignment (IntEnum vs IntFlag) based on member sign.
  • comtypes.test.test_util: Removed temporary skip logic for Python 3.15 alpha/beta.
  • docs/source/client.rst: Updated documentation with a Changed in version 1.5.0 callout detailing the IntEnum / IntFlag distinction and rationale.

CI & Build Infrastructure

  • Updated .github/workflows/autotest.yml to include Python 3.15 in the test matrix.

5. Verification & Testing

  • Test suite executed cleanly with cache invalidation (comtypes.clear_cache).
  • Verified PyCArgObject offset calculation on Python 3.15.
  • Verified code generation outputs expected IntEnum for negative members and IntFlag for positive members.
  • CI workflow runs successfully across Python 3.9 through 3.15.

@junkmd junkmd added this to the 1.5.0 / Support Python 3.15 milestone Sep 21, 2026
@codecov-commenter

codecov-commenter commented Sep 21, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 89.05%. Comparing base (177a190) to head (7b9468c).

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #945      +/-   ##
==========================================
+ Coverage   88.98%   89.05%   +0.07%     
==========================================
  Files         140      140              
  Lines       13676    13691      +15     
==========================================
+ Hits        12169    12193      +24     
+ Misses       1507     1498       -9     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@junkmd junkmd linked an issue Sep 21, 2026 that may be closed by this pull request
…gative values.

This commit enhances the code generator to correctly determine the base
class for enumerations.

* `comtypes/tools/codegenerator/namespaces.py`:
  - Introduce `to_enums` method to generate Python `enum` classes.
  - If an enumeration contains negative values, it will be generated as
    `IntEnum` (e.g., `MsiInstallState`).
  - If an enumeration contains only non-negative values, it will be
    generated as `IntFlag` (e.g., `OLE_TRISTATE`).

* `comtypes/tools/codegenerator/codegenerator.py`:
  - Update the import statement to dynamically import `IntEnum` and `IntFlag`
    based on their usage.
  - Utilize the new `to_enums` method for enum generation.

* `comtypes/test/test_client.py`:
  - Add `test_enum_base_classes` to verify the correct generation of
    `IntEnum` and `IntFlag` for enums based on their value ranges.

This addresses issue for Python 3.15+ compatibility where `IntFlag` might
truncate negative values.
Remove the temporary logic that skipped tests on Python 3.15 alpha/beta
versions. This logic was previously added to avoid `RuntimeError` during
import due to `PyCArgObject` layout changes.
Update `util` to accommodate changes in the internal `PyCArgObject` structure
introduced in Python 3.15.

The `tag` field was changed from `c_char` to `c_char_p`, and the `size` field
was changed from `c_int` to `c_ssize_t`. This change uses a version-based
bridge to maintain backward compatibility with older Python versions.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Support Python 3.15 PyCArgObject layout changes in comtypes.util Adapting to Python 3.15+ IntFlag changes for negative members.

2 participants