diff --git a/skills/mypyc-migrate/SKILL.md b/skills/mypyc-migrate/SKILL.md new file mode 100644 index 0000000000000..3a927872e51a4 --- /dev/null +++ b/skills/mypyc-migrate/SKILL.md @@ -0,0 +1,883 @@ +--- +name: mypyc-migrate +description: Migrate Python source files to be compiled with mypyc. + Useful for migrating existing code and newly written Python code. + Make changes that are needed for compatibility and to avoid major + performance bottlenecks, but the code won't be fully optimized. +--- + +# Migrate Python Code to be Compiled with Mypyc + +## Summary of Key Changes + +* In target files, perform these simple refactorings (don't modify behavior): + * Annotate all constants in module top level and class bodies using + `Final` annotation. Import `Final` from `typing`. + * Annotate all class variables using `ClassVar` imported from `typing`. + Prefer `Final` to `ClassVar` when possible, but don't use both. + * Replace uses of the `six` package with the modern Python 3 ways of + doing things. + * Add `@final` class decorator to internal/helper classes that conform to + these rules, if the class clearly isn't intended to be subclassed: + * They don't have any subclasses in the codebase. + * They aren't an ABC or a protocol. + * They aren't an exception class. + * They don't subclass any stdlib class (other than `object`). + * Refactor classes defined within functions into top-level classes with + an underscore name prefix to mark them as internal. Mypyc doesn't support + classes defined within functions. + * Refactor classes defined within classes into top-level classes. If other + code refers to the class by name, you can add an alias to class body + like `NestedClass: Final = _NestedClass`. + * Use `@mypyc_attr(native_class=False)` (import from `mypy_extensions`) + for classes that have non-trivial metaclasses (`ABCMeta` is trivial, + and also `GenericMeta`). + * Also make class non-native if class has an unsupported base class from + `stdlib` or a third-party library (see + https://mypyc.readthedocs.io/en/stable/native_classes.html#inheritance). +* Check for code patterns that won't compile or will behave differently when + compiled, and fix them (see "Code That Won't Compile or Will Break" below). +* Apply the simple performance improvements described in "Cheap Performance + Wins" below. + +## Basic Examples + +### Annotate Constants + +Module-level constants should be annotated: + +``` +MAX_ITEMS = 100 +``` + +Add `Final` annotation: + +``` +from typing import Final + +MAX_ITEMS: Final = 100 +``` + +Class-level constants should also be annotated: + +``` +class Foo: + MAX_ITEMS = 100 +``` + +Add `Final`: + +``` +from typing import Final + +class Foo: + MAX_ITEMS: Final = 100 +``` + +### Class Variables + +Class variables should always be annotated, since otherwise mypyc treats +them as instance attributes that take space in each instance: + +``` +class User: + next_id = 0 + + def __init__(self) -> None: + self.id = User.next_id + User.next_id += 1 +``` + +Annotate it like this: + +``` +from typing import ClassVar + +class User: + next_id: ClassVar = 0 + + def __init__(self) -> None: + self.id = User.next_id + User.next_id += 1 +``` + +### Six Usage + +Example of removing use of `six`: + +``` +import six + +def foo(x: object) -> bool: + return isinstance(x, six.string_types) +``` + +Use `str` instead of `six.string_types`: + +``` +def foo(x: object) -> bool: + return isinstance(x, str) +``` + +Another example of `six` usage: + +``` +def f(d: dict[str, int]) -> None: + for x in six.itervalues(d): + print(x) +``` + +Just use the `values()` method: + +``` +def f(d: dict[str, int]) -> None: + for x in d.values(): + print(x) +``` + +### Add @final Decorator + +Assume class definitions like this: + +``` +class Node: + def __init__(self, name: str, value: int) -> None: + self.name = name + self.value = value +``` + +Now we can mark `Node` as final, assuming there are no subclasses +anywhere in the repository, since it looks like a simple utility +class that is unlikely to be subclassed: + +``` +from typing import final + +@final +class Node: + ... +``` + +Another example: + +``` +class MyError(Exception): + pass +``` + +`MyError` shouldn't be final, since it's an exception class and also +subclasses a stdlib class. It doesn't help to make these final. + +### Class Nested within Function + +Mypyc doesn't support classes defined within functions, so move them to +the module top level: + +``` +def process(items: list[str]) -> list[str]: + class Visitor: + def __init__(self) -> None: + self.seen: set[str] = set() + + def visit(self, item: str) -> bool: + if item in self.seen: + return False + self.seen.add(item) + return True + + v = Visitor() + return [item for item in items if v.visit(item)] +``` + +Move the class to the top level and add an underscore prefix to the name +to mark it as internal (also add `@final` if the rules above allow it): + +``` +from typing import final + +@final +class _Visitor: + def __init__(self) -> None: + self.seen: set[str] = set() + + def visit(self, item: str) -> bool: + if item in self.seen: + return False + self.seen.add(item) + return True + + +def process(items: list[str]) -> list[str]: + v = _Visitor() + return [item for item in items if v.visit(item)] +``` + +If the nested class refers to local variables or arguments of the enclosing +function, it can't be moved as is. Pass the values explicitly to the +constructor and store them as attributes instead: + +``` +def make_filter(prefix: str) -> Callable[[str], bool]: + class PrefixFilter: + def __call__(self, s: str) -> bool: + return s.startswith(prefix) + + return PrefixFilter() +``` + +Make `prefix` an explicit attribute: + +``` +from typing import Callable, final + +@final +class _PrefixFilter: + def __init__(self, prefix: str) -> None: + self.prefix = prefix + + def __call__(self, s: str) -> bool: + return s.startswith(self.prefix) + + +def make_filter(prefix: str) -> Callable[[str], bool]: + return _PrefixFilter(prefix) +``` + +If the captured variable is reassigned after the class is defined (in the +enclosing function or via `nonlocal` in a method), the class sees the +updated value. Preserve these semantics, for example by storing a +reference to a shared mutable object instead of a copy of the value. +If this gets complicated, leave the class nested and mention it in +your summary. + +Don't move the class if it's intentionally created anew on each call, +for example if it's built with a base class or class attributes that +depend on function arguments, or if the code relies on each call producing +a distinct class object. Also check whether anything depends on the class +name (e.g. `__name__` or `__qualname__` used in reprs, logging, pickling or +test assertions), since moving and renaming the class changes these. + +## Code That Won't Compile or Will Break + +Before migrating, check the target files for these patterns. Many of them +need a search across the whole repository (including tests), since the +problem is often in code that uses the migrated module, not in the module +itself. If a fix would change behavior, or would require non-trivial +refactoring, mention it in your summary instead of guessing. + +### Type Check Errors + +The target files must type check cleanly with mypy, since mypyc won't compile +code with type errors. Don't use `# type: ignore` to silence errors in +compiled code unless there is no alternative, as it can result in incorrect +compiled code. + +### Unsupported Mypy Options + +Mypyc refuses to compile code if these mypy options are used: + +* `--no-strict-optional` (`strict_optional = False` in a config file) +* `--no-strict-bytes` (`strict_bytes = False` in a config file) + +Check the mypy configuration (`mypy.ini`, `setup.cfg`, `pyproject.toml`), +including per-module sections that match the target files, and inline +`# mypy: no-strict-optional` comments in the target files. If these options +are used, the target files must type check without them. This can +produce new type errors that need to be fixed, such as missing `None` in +types (`str` instead of `str | None`), or `bytearray` or `memoryview` values +used where `bytes` is expected. + +### `if TYPE_CHECKING` and `if MYPY` Blocks + +Mypy treats `TYPE_CHECKING` (and a module-level `MYPY = False` constant) as +always true, so it considers any code that only runs when the condition is +false at runtime unreachable. Mypyc compiles such unreachable code into code +that raises `RuntimeError: Reached allegedly unreachable code!`. In a +compiled module, these all fail at runtime (at import time if at module +level): + +``` +if TYPE_CHECKING: + from foo import Foo +else: + Foo = object # RuntimeError (even if the else block is just "pass") + +if not TYPE_CHECKING: + setup_runtime_hooks() # RuntimeError + +def f() -> str: + if TYPE_CHECKING: + return "a" + else: + return "b" # RuntimeError when f() is called +``` + +An `if TYPE_CHECKING:` block without an `else` branch works as expected: it +isn't executed at runtime. This is the recommended way to import names +only used in annotations, e.g. to break an import cycle. Annotations that use +these names are still checked at runtime: + +``` +from __future__ import annotations + +from typing import TYPE_CHECKING + +if TYPE_CHECKING: + from foo import Foo + +def process(x: Foo) -> None: # OK + ... +``` + +Fix code that runs when the condition is false at runtime: + +* Remove `else` branches that only define placeholder names for annotations + (such as `Foo = object` above). Use `from __future__ import annotations` or + string literal annotations instead, so that the names aren't needed at + runtime. +* Move other code in `else` branches and `if not TYPE_CHECKING:` blocks out + of the conditional, so that it runs unconditionally. If the `if + TYPE_CHECKING:` branch defines the same name (e.g. a more precise + type for mypy), you may need to restructure the code. If it's not + clear how to do this without changing behavior, mention it in your summary. +* Replace `if MYPY:` with `if TYPE_CHECKING:` (import it from `typing`), and + remove the `MYPY = False` definition. + +### Annotations Are Enforced at Runtime + +Compiled code checks argument, return and attribute types at runtime, +and raises `TypeError` if a value has the wrong type. Code that worked +despite an inaccurate annotation may fail after compilation: + +``` +def greet(name: str) -> str: + if name is None: + return "Hello!" + return f"Hello, {name}!" + +greet(None) # TypeError when compiled: str object expected; got None +``` + +If callers can pass `None` (e.g. from non-compiled or untyped code), fix +the annotation: + +``` +def greet(name: str | None) -> str: + ... +``` + +Look for signs of inaccurate annotations, such as `is None` checks on values +whose type doesn't include `None`, or `isinstance` checks against types that +aren't allowed by the annotation. + +### Assigning `int` Values to `float` Variables + +Mypy allows an `int` value to be used where a `float` is expected, but +`int` and `float` values have different runtime representations in compiled +code. Assigning an `int` value to a variable or attribute declared as +`float` is a compile error, even in the initial assignment: + +``` +def average(values: list[float], n: int) -> float: + total: float = 0 # Error + for v in values: + total += v + if n == 0: + total = n # Error + return total / max(n, 1) +``` + +Use a float literal, or convert explicitly using `float(...)`: + +``` +def average(values: list[float], n: int) -> float: + total = 0.0 + for v in values: + total += v + if n == 0: + total = float(n) + return total / max(n, 1) +``` + +The same applies to attributes, such as `self.ratio: float = 1` in +`__init__`. Passing an `int` value as a `float` argument or returning it from +a function with a `float` return type works. + +### Surrogate Code Points in String Literals + +String literals can't contain lone surrogate code points (U+D800 to U+DFFF), +such as `"\ud800"`. Use `chr(...)`, or compare against the integer code point +using `ord(...)`: + +``` +from typing import Final + +HIGH_SURROGATE: Final = chr(0xD800) # Instead of "\ud800" + +def is_high_surrogate(c: str) -> bool: + return 0xD800 <= ord(c) <= 0xDBFF # Instead of "\ud800" <= c <= "\udbff" +``` + +### Conditional Function and Class Definitions + +Defining the same function or class in multiple branches of an `if` +statement is a compile error: + +``` +if HAS_FAST_IMPL: + def process(data: bytes) -> bytes: + return fast_process(data) +else: + def process(data: bytes) -> bytes: # Error + return slow_process(data) +``` + +Define the function once, and move the condition inside it: + +``` +def process(data: bytes) -> bytes: + if HAS_FAST_IMPL: + return fast_process(data) + else: + return slow_process(data) +``` + +A single definition guarded by a runtime condition compiles, but the function +is always defined, even if the condition is false. Look out for code that +checks for the existence of the function (e.g. using `hasattr` or +`globals()`). + +Checks against `sys.version_info` and `sys.platform` are fine, since mypy +evaluates them statically. A `try`/`except ImportError` fallback definition +also works. + +### `if __name__ == "__main__"` + +The `__name__` of a compiled module is never `"__main__"`, and compiled +modules can't be run using `python -m`. Move the main block to a separate +script that isn't compiled and that imports the module: + +``` +# mod.py (compiled) +def main() -> None: + ... + +# mod_main.py (not compiled) +from mod import main + +if __name__ == "__main__": + main() +``` + +### Async Generators + +Async generators (`async def` functions that contain `yield`) aren't +supported by mypyc. This includes functions decorated with +`@asynccontextmanager`. Rewrite them as classes that implement the async +iterator protocol (`__aiter__` and `__anext__`) or the async context +manager protocol (`__aenter__` and `__aexit__`). Keep the original function +as a thin wrapper, so that callers don't need to change. + +An async generator used as an async iterator: + +``` +from typing import AsyncIterator + +async def countdown(n: int) -> AsyncIterator[int]: + while n > 0: + yield n + n -= 1 +``` + +Rewrite as a class with `__aiter__` and `__anext__`. Local variables that are +preserved between `yield`s become attributes, and `StopAsyncIteration` is +raised at the end of iteration: + +``` +from typing import AsyncIterator, final + +@final +class _Countdown: + def __init__(self, n: int) -> None: + self.n = n + + def __aiter__(self) -> "_Countdown": + return self + + async def __anext__(self) -> int: + if self.n <= 0: + raise StopAsyncIteration + n = self.n + self.n -= 1 + return n + + +def countdown(n: int) -> AsyncIterator[int]: + return _Countdown(n) +``` + +An async context manager: + +``` +from contextlib import asynccontextmanager +from typing import AsyncIterator + +@asynccontextmanager +async def connect(name: str) -> AsyncIterator[str]: + print("open") + try: + yield name + finally: + print("close") +``` + +Rewrite as a class with `__aenter__` and `__aexit__`. Code before `yield` +goes into `__aenter__`, and the yielded value is returned from it. Code +after `yield` (or in a `finally` block) goes into `__aexit__`: + +``` +from contextlib import AbstractAsyncContextManager +from types import TracebackType +from typing import final + +@final +class _Connection: + def __init__(self, name: str) -> None: + self.name = name + + async def __aenter__(self) -> str: + print("open") + return self.name + + async def __aexit__( + self, + exc_type: type[BaseException] | None, + exc: BaseException | None, + tb: TracebackType | None, + ) -> None: + print("close") + + +def connect(name: str) -> AbstractAsyncContextManager[str]: + return _Connection(name) +``` + +If the original function catches exceptions raised within the `async with` +block (an `except` clause around `yield`), handle them in `__aexit__` using +`exc`, and return `True` from `__aexit__` if the exception should be +suppressed (declare the return type as `bool` in this case). If the async +generator is complex (e.g. it has multiple `yield` statements in different +places), mention it in your summary instead of rewriting it. + +### Monkey Patching and Mocking + +Functions and classes in compiled modules can't be monkey patched. Calls +within the same compilation unit are bound at compile time, so replacing a +module attribute has no effect on compiled callers, and replacing methods of +native classes fails: + +``` +# In a test +# Compiled code calling fetch() isn't affected +with mock.patch("mylib.util.fetch") as m: + ... + +# Error: can't set attributes of native class +with mock.patch.object(Client, "send"): + ... + +Client.send = fake_send # Error +``` + +Search the tests (and other code) for `mock.patch`, `patch.object`, +`monkeypatch.setattr` and direct assignments that target the migrated +modules. Possible fixes: + +* Inject the dependency explicitly (e.g. as an argument or an attribute) + instead of patching it. +* Patch something outside the compiled modules. +* Make the class non-native using `@mypyc_attr(native_class=False)`, if + methods of the class need to be patched (this is slower). + +### Undeclared Attributes + +Native classes only support attributes that are assigned in the class body +or in methods of the class (similar to `__slots__`). Setting other +attributes from outside the class raises `AttributeError`: + +``` +class Request: + def __init__(self, url: str) -> None: + self.url = url + +req = Request("...") +req.retry_count = 0 # Error at runtime +``` + +Declare the attribute in the class body (or initialize it in `__init__`): + +``` +class Request: + retry_count: int = 0 + + def __init__(self, url: str) -> None: + self.url = url +``` + +Also look for `setattr()` calls with attribute names that aren't known in +advance and uses of `obj.__dict__` or `vars(obj)`, since instances of native +classes don't have a `__dict__`. Make the class non-native if these are +needed. + +### Deleting Attributes + +Attributes of native classes can't be deleted by default. List any attributes +that are deleted using `del` in `__deletable__` in the class body: + +``` +class Cache: + value: int | None + + __deletable__ = ["value"] + + def clear(self) -> None: + del self.value +``` + +### Unsupported Dunder Methods + +`__getattribute__`, `__delattr__` and `__index__` don't work in native +classes. Make classes that define them non-native. + +### Descriptors + +Native classes only support `@property`, `@staticmethod` and `@classmethod` +descriptors. Other descriptors, such as `functools.cached_property` or custom +descriptor classes used as class attributes, require a non-native class, or +the code must be rewritten. For example, `cached_property` can be replaced +with a property that stores the value in an attribute: + +``` +class Config: + @cached_property + def settings(self) -> dict[str, str]: + return load_settings() +``` + +Rewrite as: + +``` +class Config: + def __init__(self) -> None: + self._settings: dict[str, str] | None = None + + @property + def settings(self) -> dict[str, str]: + if self._settings is None: + self._settings = load_settings() + return self._settings +``` + +### Class Decorators + +Classes that use class decorators other than `@dataclass`, +`@attr.s(auto_attribs=True)`, `@trait` and `@mypyc_attr` are silently +compiled as non-native classes (similar to classes with unsupported +metaclasses). This is slow but works. Mark them explicitly using +`@mypyc_attr(native_class=False)` to make this clear. + +Conversely, you can use `@mypyc_attr(native_class=True)` for important +classes to make it a compile error if the class can't be native. + +### Multiple Inheritance + +Native classes only support single inheritance, with the exception of trait +types. Mixin classes can often be turned into traits using `@trait` (import +from `mypy_extensions`). Traits must come after the regular base class: + +``` +from mypy_extensions import trait + +@trait +class LoggingMixin: + def log(self, msg: str) -> None: + print(msg) + +class Service(BaseService, LoggingMixin): # Trait must come last + ... +``` + +If this doesn't work, make the class non-native. + +### Pickling and Copying + +Instances of native classes can't be pickled or copied using `copy.copy` or +`copy.deepcopy` if `__init__` has required arguments. Search for uses of +`pickle`, `copy` (and libraries that pickle objects, such as +`multiprocessing`) involving migrated classes, and use +`@mypyc_attr(serializable=True)` for these classes: + +``` +from mypy_extensions import mypyc_attr + +@mypyc_attr(serializable=True) +class Job: + def __init__(self, job_id: int) -> None: + self.job_id = job_id +``` + +### Subclassing in Non-compiled Code + +By default, native classes can't be subclassed in non-compiled code, or in +another compilation unit. Search for subclasses of migrated classes outside +the migrated files, including test doubles and fakes in tests. Use +`@mypyc_attr(allow_interpreted_subclasses=True)` for these classes (and don't +make them `@final`): + +``` +from mypy_extensions import mypyc_attr + +@mypyc_attr(allow_interpreted_subclasses=True) +class Handler: + def handle(self, item: str) -> None: + ... +``` + +Also don't mark methods `@final` if they are overridden in non-compiled +subclasses, since compiled code will ignore the overrides. + +### Class Attributes Overridden in Non-compiled Subclasses + +If a non-compiled subclass assigns a new value to an attribute in the class +body, compiled code in the base class won't see the new value, since a class +attribute without a `ClassVar` annotation is an instance attribute with a +default value in a native class: + +``` +# Compiled +@mypyc_attr(allow_interpreted_subclasses=True) +class Base: + timeout = 10 + + def get_timeout(self) -> int: + return self.timeout + +# Not compiled +class Child(Base): + timeout = 30 + +Child().timeout # 30 +Child().get_timeout() # 10 (!) +``` + +Search for subclasses that override class attributes. If the attribute is +never assigned via an instance, annotate it with `ClassVar` in the base class +(but not `Final`, since that prevents overriding): + +``` +from typing import ClassVar + +@mypyc_attr(allow_interpreted_subclasses=True) +class Base: + timeout: ClassVar = 10 +``` + +Otherwise the subclass can assign the attribute in `__init__` instead +(`self.timeout = 30`). + +The same problem affects instance attributes that are assigned in methods +of the base class, if a non-compiled subclass defines a class attribute with +the same name. Compiled code and non-compiled code then see different +values, and compiled code may fail with `AttributeError` if the attribute +was never assigned (normal Python would fall back to the class attribute): + +``` +# Compiled +@mypyc_attr(allow_interpreted_subclasses=True) +class Connection: + def __init__(self) -> None: + self.retries = 3 + + def get_retries(self) -> int: + return self.retries + +@mypyc_attr(allow_interpreted_subclasses=True) +class Job: + result: str # Only assigned in run() + + def get_result(self) -> str: + return self.result + + def run(self) -> None: + self.result = "done" + +# Not compiled +class FastConnection(Connection): + retries = 1 # Class attribute shadows an instance attribute + +c = FastConnection() +c.retries # 1 (would be 3 in normal Python) +c.get_retries() # 3 + +class DefaultJob(Job): + result = "pending" + +DefaultJob().get_result() # AttributeError (would be "pending") +``` + +Fix these by assigning the value in the subclass `__init__` (after calling +`super().__init__()`), instead of using a class attribute: + +``` +class FastConnection(Connection): + def __init__(self) -> None: + super().__init__() + self.retries = 1 +``` + +For attributes that are only sometimes assigned, such as `Job.result`, it +may be simpler to give the attribute a default value in the base class and +set it in the subclass `__init__`. + +### Reassigned Final Values + +References to `Final` module-level constants and class attributes are +replaced with the value at compile time, when the value is known during +compilation. If other code or tests reassign a constant (e.g. +`config.MAX_RETRIES = 0` or `monkeypatch.setattr(config, "MAX_RETRIES", 0)`), +compiled code won't see the new value. Don't make these `Final`; leave +them as regular variables. + +## Cheap Performance Wins + +These changes are simple and don't change behavior, but they can make a big +difference in compiled code. + +### Don't Cache Bound Methods in Local Variables + +Caching a method in a local variable is a common CPython optimization, but it +makes compiled code slower, since the call can no longer be bound at compile +time: + +``` +def squares(n: int) -> list[int]: + a = [] + append = a.append # Slow in compiled code + for i in range(n): + append(i * i) + return a +``` + +Call the method directly: + +``` +def squares(n: int) -> list[int]: + a = [] + for i in range(n): + a.append(i * i) + return a +``` + +The same applies to caching global functions or module attributes in local +variables or default argument values (e.g. `def f(x, _len=len)`) — refer to +them directly instead. diff --git a/skills/mypyc-optimize/SKILL.md b/skills/mypyc-optimize/SKILL.md new file mode 100644 index 0000000000000..9c9e320eb17d0 --- /dev/null +++ b/skills/mypyc-optimize/SKILL.md @@ -0,0 +1,1157 @@ +--- +name: mypyc-optimize +description: Optimize Python source files that are compiled with mypyc (or + will be compiled), or identify performance bottlenecks and optimization + opportunities in them without making changes. Also apply the + performance-related changes from the mypyc-migrate skill, and improve + type annotations so that mypyc can generate faster code. +--- + +# Optimize Python Code Compiled with Mypyc + +This skill builds on the mypyc-migrate skill (`../mypyc-migrate/SKILL.md`). +Read that skill first; it has examples and details for the steps below. + +This skill can also be used to only identify bottlenecks and optimization +opportunities, without changing the code. In this case, use the +techniques below (profiling, inspecting generated code, and so on) to find +issues, and report them to the user, ordered by expected impact. + +Don't change behavior in significant ways. If an improvement would require +non-trivial refactoring or might change behavior, mention it in your summary +instead. + +## Workflow + +Optimize incrementally: + +1. Find the hot functions using a profile of a realistic workload (see + "Profiling"), unless the user already told you what to optimize. If + no profile is available, make educated guesses about likely hot + functions (e.g. error handling is likely not hot). +2. Make one change, or a small set of related changes, at a time. +3. Run the project's tests after each change, including tests that run + the compiled code, if there are any. If running the tests or + benchmarks is slow, you can make changes in larger batches to make + the process faster. +4. Measure whether the change helps (see "Measure Performance"). Revert + changes that don't clearly help, unless they are simple or also make + the code cleaner (such as adding missing annotations). Keep simple + changes even if their effect can't be measured: many simple changes + are helpful, but the impact of each one in isolation is often too small + to measure. If no benchmark is available, first try to show the impact + using extracted synthetic microbenchmarks that simulate the actual + code. If this isn't practical, assume that changes that follow the + instructions below (including their conditions) improve performance. + +In your summary, list the changes you made and the measured effect, if +any. Also list changes that you tried but reverted since they didn't help, +and potential bottlenecks that you found but didn't fix (see "Report Other +Potential Bottlenecks"). + +## Steps from the Migrate Skill + +In the target files, make sure that all the changes described in the migrate +skill have been applied. These are especially important for performance: + +* Annotate all constants in module top level and class bodies using `Final` + (import from `typing`). Reading `Final` values is much faster than reading + regular module-level variables. Don't make a value `Final` if any code or + test reassigns it. +* Annotate all class variables using `ClassVar` (import from `typing`), + unless they are already `Final`. Without `ClassVar`, mypyc treats a class + variable as an instance attribute, which takes space in each instance. +* Add `@final` to classes that aren't intended to be subclassed (see the + rules in the migrate skill). +* Move classes nested within functions or other classes to the module top + level. +* Don't cache bound methods or global functions in local variables. + +Also go through the compatibility checks in "Code That Won't Compile or Will +Break" in the migrate skill, and fix any issues. + +## Improve Type Annotations + +Precise type annotations let mypyc use fast, specialized operations. Values +with type `Any` and values in functions without annotations use slow, +generic operations. + +* Add type annotations to functions that have missing annotations, including + missing return type annotations (use `-> None` if a function doesn't return + a value). A function without any annotations isn't type checked by mypy + (when using default strictness options), and mypyc compiles it using generic + operations. +* Replace `Any` types with precise types where possible. For example, use + `list[str]` or `dict[str, int]` instead of `Any`, `list[Any]` or + `dict[str, Any]` when the item types are known. Prefer concrete types such + as `list[...]` over abstract types such as `Sequence[...]` when the concrete + type is known. +* Annotate variables that get their value from untyped code (for example, + a library without type annotations), so that the variable doesn't have type + `Any`. + +Only use a type if you are confident that it's correct. Compiled code checks +types at runtime and raises `TypeError` if a value has the wrong type (see +"Annotations Are Enforced at Runtime" in the migrate skill). Check callers, +including tests and non-compiled code, to make sure that they never pass +values of other types, such as `None`. Adding annotations to a function that +previously had none also causes mypy to type check its body; fix any new type +errors that this produces. + +## Measure Performance + +The speedup from the optimizations below varies a lot, depending on the +code, and some changes don't help at all in a particular case. Measure +whether a change helps before keeping it, especially if it makes the code +more verbose or harder to read. + +If the target code is self-contained enough, write a throwaway benchmark +script that imports the compiled modules and runs a realistic workload +(for example, the main entry point on representative input). Run it before +and after each change (or set of related changes, and don't run it too +often if the benchmark runs slowly). Make sure each benchmark run isn't +very slow -- since these are synthetic workloads, they may not show very +minor performance changes reliably. + +If the code isn't self-contained (for example, it needs a server, a +database, or a large application to run), extract the hot functions into a +separate module and write microbenchmarks that simulate the behavior of +the original functions. Keep the same type annotations and similar data +(types, sizes and shapes of the values), since these determine what code +mypyc generates. Use these microbenchmarks as synthetic proxies for the +performance of the real code, for example to decide which of several ways +of writing a function is fastest. + +When benchmarking: + +* Compile benchmarks in a temporary directory, since `mypyc` writes build + files to the `build/` directory under the current working directory. +* Check that the benchmarked module was actually compiled + (`mod.__file__` should end in `.so` or `.pyd`), and that mypyc reported + no errors. +* Run the benchmarked code many times, or in a loop, so that each + measurement takes at least several milliseconds, and take the best (or + the median) of several runs using `time.perf_counter()`. Timings can be + noisy, so only trust differences that are clearly larger than the + variation between runs. +* Don't leave throwaway benchmarks in the project, unless asked to. + +Benchmarks can also be profiled to find where time is spent (see +"Profiling"). + +## Profiling + +Profile the code (or a benchmark) to find the hot functions before +optimizing, and to check where time is spent after a change. Use a native +(sampling) profiler that shows C-level frames. Good options to try, if +they are available: + +* `py-spy record --native -o profile.svg -- python3 bench.py` (native + frames are supported on Linux and Windows) +* `perf` on Linux (`perf record -g python3 bench.py`, then `perf report`) +* On macOS: Instruments (Time Profiler), `samply`, or `sample` +* On Windows: the Visual Studio profiler, or Windows Performance Recorder + and Analyzer + +If the profiled workload seems realistic enough, focus most optimization +effort on the functions where the profile shows the most time is spent +(including time spent in functions they call). Optimizing code that +barely shows up in the profile rarely makes a noticeable difference, and +makes the code more verbose for little gain. The simple, mechanical steps +from the migrate skill can still be applied everywhere. + +Don't use `cProfile` or `profile`. They don't produce native frames, and +don't see calls within compiled code, so the results are misleading for +compiled modules. + +Compiled functions have names with the prefix `CPyDef_` in profiles (e.g. +`CPyDef_mod___parse` for function `parse` in module `mod`). Functions with +the prefix `CPyPy_` are wrappers used when compiled functions are called +from non-compiled code. + +Call stacks can be incomplete if CPython or the compiled extensions were +built without frame pointers. A custom CPython build (and compiled modules) +with `-fno-omit-frame-pointer`, or a profiler mode that uses DWARF debug +information for stack unwinding, can give better results. Look up the +current recommended approach for the profiler and Python version used. + +## Inspect Generated Code + +Looking at the code mypyc generates can reveal bottlenecks that aren't +obvious from the source code, such as unexpected `Any` types that cause +slow, generic operations to be used. + +Mypyc writes the generated intermediate representation (IR) to +`build/ops.txt` and the generated C code to `build/__native*.c`. The IR is +easier to read than C. Signs of generic operations include calls to +C API functions such as `PyObject_GetAttr`, `CPyObject_GetAttr`, +`PyObject_GetItem`, `PyObject_Vectorcall`, `PyObject_GetIter`, +`PyIter_Next` or `PyNumber_Add` (and other `PyNumber_` functions) in a +hot function. Specialized operations have names such as `CPyTagged_Add` +(for `int` values) or `CPyList_GetItem`, or use native attribute access. +Common causes of generic operations include values with type `Any`, +missing annotations, non-native classes, and values that come from +non-compiled modules. Fix these using the other techniques in this skill, +such as annotating variables that get their value from untyped code. + +### Operations with Fast Implementations + +The mypyc documentation lists the operations that have fast, +specialized implementations for each primitive type. Operations that +aren't listed use generic operations. Check these when choosing between +alternative ways of writing a hot function: + +* `mypyc/doc/native_operations.rst` (native classes and functions) +* `mypyc/doc/int_operations.rst`, `float_operations.rst` and + `bool_operations.rst` +* `mypyc/doc/str_operations.rst`, `bytes_operations.rst` and + `bytearray_operations.rst` +* `mypyc/doc/list_operations.rst`, `dict_operations.rst`, + `set_operations.rst`, `frozenset_operations.rst` and + `tuple_operations.rst` + +These files are in the mypy repository, and they are also available at +https://mypyc.readthedocs.io/. For example, a `str` method that isn't +listed in `str_operations.rst` is called using a generic method call. + +## Compile Related Modules Together + +All modules compiled in a single mypyc invocation (a single `mypyc` +command, or a single `mypycify(...)` call in `setup.py`) form a +*compilation unit*. Calls to functions, methods and classes within a +compilation unit use fast, direct calls, and `Final` values are inlined. +References to other compilation units use slow, generic operations, and +are about as slow as references to non-compiled modules. Using +`separate=True` with `mypycify` still keeps all the modules in the same +compilation unit. + +If modules that frequently call each other are compiled using separate +invocations, suggest compiling them together, or report this to the +user. Similarly, if a hot function frequently calls a module that isn't +compiled, consider compiling that module too (after applying the +migrate skill to it). Don't change the build configuration without +asking the user, since it may affect packaging and build times. + +## Keep Classes Native + +Native classes are much faster than regular Python classes (non-native +classes). Making an important class native is often the biggest single +improvement. A class is silently compiled as a non-native class if it uses +an unsupported metaclass or class decorator (see "Class Decorators" in the +migrate skill). Where possible, remove the cause. + +For example, a class decorator that registers the class can often be +replaced with an explicit call: + +``` +@register +class JsonHandler: # Non-native, due to the class decorator + ... +``` + +Call `register` after the class definition instead: + +``` +class JsonHandler: # Native + ... + +register(JsonHandler) +``` + +This is only equivalent if the decorator returns the original class +(instead of a wrapper or a different class). Check the implementation of +the decorator. + +If a performance-critical class has non-trivial base classes (other than +`object`, `ABC`, `Generic[...]` or native classes in the same project), a +metaclass other than `ABCMeta`, or class decorators, add +`@mypyc_attr(native_class=True)` (import from `mypy_extensions`), so that the +class can't accidentally become non-native later. Don't add it to other +classes, as they are unlikely to become non-native. + +### Replace Dataclasses with Regular Classes + +Dataclasses are supported, but they aren't as efficient as regular native +classes. Consider replacing performance-critical dataclasses with regular +classes: + +``` +from dataclasses import dataclass + +@dataclass(frozen=True) +class Point: + x: int + y: int + label: str = "" +``` + +Define `__init__` and `__repr__` explicitly. For a frozen dataclass, declare +the attributes as `Final` in `__init__`, since they can't be assigned to after +initialization: + +``` +from typing import Final, final + +@final +class Point: + def __init__(self, x: int, y: int, label: str = "") -> None: + self.x: Final = x + self.y: Final = y + self.label: Final = label + + def __repr__(self) -> str: + return f"Point(x={self.x!r}, y={self.y!r}, label={self.label!r})" +``` + +Dataclasses also generate other methods, such as `__eq__` (and `__hash__` +for frozen dataclasses, and ordering methods if `order=True`). If any code +compares instances, uses them as dictionary keys or in sets, or sorts them, +define these methods explicitly, as otherwise the behavior changes. Also +check for uses of `dataclasses` functions such as `dataclasses.replace`, +`dataclasses.asdict` or `dataclasses.fields` on the class. Don't replace the +dataclass if these are used. + +## Avoid Mutable Module-level Variables + +Reading and writing a module-level variable that isn't `Final` requires a +dictionary lookup in the module namespace, which is slow. If a mutable +module-level variable is used frequently (for example, a counter or a cache +updated using `global`), store the state in an attribute of a `Final` +instance of a native class: + +``` +_calls = 0 +_errors = 0 + +def record(ok: bool) -> None: + global _calls, _errors + _calls += 1 + if not ok: + _errors += 1 +``` + +Use a `Final` object instead: + +``` +from typing import Final, final + +@final +class _Stats: + def __init__(self) -> None: + self.calls = 0 + self.errors = 0 + +_stats: Final = _Stats() + +def record(ok: bool) -> None: + _stats.calls += 1 + if not ok: + _stats.errors += 1 +``` + +Only do this for internal (often underscore-prefixed) variables that +aren't accessed from other modules or tests, since the variables will no +longer be available as module attributes. + +## Avoid Optional Primitive Types in Performance-critical Code + +`int`, `float`, `bool` and fixed-length tuple values are unboxed in +compiled code, but union types such as `int | None` are always boxed +(heap-allocated), and operations on them are slower. If `None` is only used +as a marker for a missing value, and there is an otherwise unused value +that can be used as a marker instead, use it in internal code: + +``` +class Parser: + def __init__(self) -> None: + self.error_pos: int | None = None # Boxed + + def has_error(self) -> bool: + return self.error_pos is not None +``` + +Use `-1` to represent a missing position: + +``` +class Parser: + def __init__(self) -> None: + self.error_pos = -1 # -1 if no error; unboxed + + def has_error(self) -> bool: + return self.error_pos >= 0 +``` + +Only do this if the attribute or variable isn't part of a public interface +and all uses can be updated, and document the marker value in a comment. + +## Use Native Classes Instead of Dictionaries as Records + +Dictionaries with a fixed set of string keys are often used as records. +Creating a native class instance and accessing its attributes is much +faster (about 4x faster than `dict[str, Any]` and 2.5x faster than a +`TypedDict` in a benchmark): + +``` +def make_loc(name: str, line: int) -> dict[str, Any]: + return {"name": name, "line": line} + +def next_line(loc: dict[str, Any]) -> int: + return loc["line"] + 1 +``` + +Use a native class instead: + +``` +from typing import final + +@final +class Loc: + def __init__(self, name: str, line: int) -> None: + self.name = name + self.line = line + +def make_loc(name: str, line: int) -> Loc: + return Loc(name, line) + +def next_line(loc: Loc) -> int: + return loc.line + 1 +``` + +This also gives precise attribute types instead of `Any`. Only do this if +you can find and update all code that uses the dictionaries. Don't do this +if the dictionaries are part of a public interface, if keys are added or +removed dynamically, or if the dictionaries are passed to code that +requires a dictionary (for example, `json.dumps`, `**` unpacking, iteration +over keys, or `.get()` with a default). + +## Dictionary Lookups + +### Use a Single `dict.get` Instead of `in` Followed by Indexing + +Checking for a key using `in` and then indexing performs two dictionary +lookups. A single `get` is faster if the key is usually present: + +``` +if key in counts: + total += counts[key] +``` + +Use `get` instead: + +``` +n = counts.get(key) +if n is not None: + total += n +``` + +This is only equivalent if the dictionary never contains `None` values. +If the key is usually missing, `in` alone is slightly faster, so only do +this if the key is expected to be present in most cases. + +### Replace `defaultdict` with a Plain `dict` + +Mypyc has fast, specialized operations for `dict`, but `defaultdict` is +a subclass that doesn't get all of these optimizations. Replacing an +internal `defaultdict` with a plain `dict` is somewhat faster: + +``` +from collections import defaultdict + +counts: defaultdict[str, int] = defaultdict(int) +groups: defaultdict[str, list[int]] = defaultdict(list) +for i, key in enumerate(keys): + counts[key] += 1 + groups[key].append(i) +``` + +Use `get` and `setdefault` with a plain `dict` instead: + +``` +counts: dict[str, int] = {} +groups: dict[str, list[int]] = {} +for i, key in enumerate(keys): + counts[key] = counts.get(key, 0) + 1 + groups.setdefault(key, []).append(i) +``` + +Mypyc special-cases `setdefault` with an empty `[]`, `{}` or `set()` +default, so that the empty collection is only created if the key is +missing. Other default values are evaluated on each call, even if the key +is present. + +Only do this if the dictionary is internal (it's not returned to callers +or stored in a public attribute), since code that reads a `defaultdict` +may rely on missing keys getting a default value. Check all places where +the dictionary is indexed. The benefit is small, so prioritize this only +in very hot code. + +## Return Multiple Values Using Fixed-length Tuples + +Fixed-length tuple types such as `tuple[int, str]` are unboxed in +compiled code: returning and unpacking them doesn't allocate an object. +Other ways of returning multiple values are much slower. In a benchmark, +returning two `int` values as a `tuple[int, int]` was about 7x faster than +returning a native class instance, 8x faster than `tuple[int, ...]`, and +40x faster than a `NamedTuple`. + +Named tuples are particularly inefficient for this, since creating a named +tuple instance is slow in compiled code (about 7x slower than creating a +native class instance in a benchmark). Accessing items by name, by index or +by unpacking is fairly fast (about as fast as with a regular tuple), but +still slower than accessing native class attributes: + +``` +from typing import NamedTuple + +class QR(NamedTuple): + q: int + r: int + +def div(a: int, b: int) -> QR: + return QR(a // b, a % b) + +q, r = div(x, 7) +``` + +Use a fixed-length tuple type instead: + +``` +def div(a: int, b: int) -> tuple[int, int]: + return a // b, a % b + +q, r = div(x, 7) +``` + +Only do this for internal functions where all callers can be updated, +since callers may access items by name (such as `div(x, 7).q`), and +named tuples have a different `repr`. If there are more than a few items, +or the names are important for readability, use a small native class +instead (see below). Use a precise, fixed-length tuple type in annotations +instead of `tuple[int, ...]` when the length is known. + +### Replace Named Tuples with Native Classes + +A simple named tuple that is used like a record (items are accessed by +name) can be replaced with a native class with `Final` attributes. In a +benchmark, creating instances was about 7x faster and accessing attributes +about 3.5x faster. This helps most if many instances are created: + +``` +from typing import NamedTuple + +class Span(NamedTuple): + start: int + end: int +``` + +Use a native class instead: + +``` +from typing import Final, final + +@final +class Span: + def __init__(self, start: int, end: int) -> None: + self.start: Final = start + self.end: Final = end + + def __repr__(self) -> str: + return f"Span(start={self.start!r}, end={self.end!r})" +``` + +Only do this if the tuple behavior isn't needed. Check that no code +unpacks the values (`start, end = span`), indexes or iterates over them, +compares them using `==` or `<`, hashes them (uses them as dictionary keys +or in sets), passes them where a tuple is expected, or uses named tuple +methods such as `_replace` or `_asdict`. If equality or hashing is needed, +define `__eq__` and `__hash__` explicitly. + +## Initialize Attributes Before `self` Escapes in `__init__` + +Mypyc analyzes `__init__` methods to find attributes that are always +initialized. Reading these attributes doesn't need a check for an undefined +value (which would raise `AttributeError`), which makes reads faster. The +analysis stops once `self` may escape: `self` is passed to a function, a +method is called on `self`, or a property of `self` is accessed. Attributes +assigned after this point aren't considered always defined. Calls that +don't involve `self` are fine. + +``` +class Connection: + def __init__(self, host: str) -> None: + self.host = host + self.register() # 'self' escapes + self.retries = 0 # Not always defined +``` + +Assign attributes before `self` escapes, when this doesn't change behavior: + +``` +class Connection: + def __init__(self, host: str) -> None: + self.host = host + self.retries = 0 # Always defined + self.register() +``` + +A call to `super().__init__()` only makes `self` escape if the base class +`__init__` makes `self` escape (for example, if it calls a method of +`self`). In this case, initialize attributes defined in the subclass before +calling `super().__init__()`, if the values don't depend on `self`: + +``` +class Base: + def __init__(self, name: str) -> None: + self.name = name + self.setup() # 'self' escapes + +class Child(Base): + def __init__(self, name: str) -> None: + # Always defined, since assigned before super().__init__() + self.count = 0 + super().__init__(name) +``` + +Only reorder assignments if the code that runs after `self` escapes (the +called methods, including methods overridden in subclasses, and the base +class `__init__`) doesn't read or assign the attributes being moved, and if +the assigned values don't depend on anything that is initialized later. + +The analysis is only performed for native classes where all subclasses are +known at compile time. It's not performed for classes with +`allow_interpreted_subclasses=True` or `serializable=True`, for example. +An attribute defined in a base class is only always defined if it's always +defined in all subclasses. + +## Use `is` to Compare Enum Values + +Comparing an enum value using `is` can be much faster than `==`, when the +value has an optional type such as `Color | None` (about 6x faster in a +benchmark), or when the enum class is defined in a module that isn't compiled +together with the target files. If both operands have the same enum type and +the enum is compiled, `==` is already fast. + +``` +def is_red(c: Color | None) -> bool: + return c == Color.RED # Slow +``` + +Use `is` (and `is not` instead of `!=`): + +``` +def is_red(c: Color | None) -> bool: + return c is Color.RED # Fast +``` + +This changes behavior for `IntEnum`, `StrEnum` and other enums that +inherit from a type such as `int` or `str`, since `==` also matches plain +`int` or `str` values, and for enums that define `__eq__`. Only use `is` +with these if you are sure that the other operand is always an enum member. + +## Use Integer Code Points Instead of One-character Strings + +When processing strings character by character, operating on integer code +points is much faster in compiled code than operating on one-character +strings (often 5x to 15x faster in benchmarks). `ord(s[i])` is a fast +operation when `s` has type `str`, and `ord("x")` of a string literal is a +compile-time constant: + +``` +def count_separators(s: str) -> int: + n = 0 + for c in s: + if c == "," or c == ";": + n += 1 + return n + +def count_digits(s: str) -> int: + n = 0 + for c in s: + if "0" <= c <= "9": + n += 1 + return n +``` + +Use `ord(...)` instead: + +``` +def count_separators(s: str) -> int: + n = 0 + for i in range(len(s)): + c = ord(s[i]) + if c == ord(",") or c == ord(";"): + n += 1 + return n + +def count_digits(s: str) -> int: + n = 0 + for i in range(len(s)): + c = ord(s[i]) + if ord("0") <= c <= ord("9"): + n += 1 + return n +``` + +For `str` methods such as `c.isdigit()`, `c.isspace()`, `c.isalpha()`, +`c.isalnum()`, `c.upper()` and `c.lower()` on a one-character string, use the +corresponding functions in `librt.strings` that take a code point (see "Use +librt" below), e.g. `isdigit(ord(s[i]))`. Note that `librt.strings.toupper` +and `tolower` leave code points unchanged if the result would have multiple +code points (e.g. `"ß".upper()` is `"SS"`). + +To build a string from code points, use `librt.strings.StringWriter` +(`w.append(c)`). + +These changes make code slower when it's not compiled, so only do this in +modules that are compiled. + +## Move Nested Functions to Module Level or Methods + +Each time the enclosing function is called, a nested function (or a +lambda) must be allocated, together with an environment object for the +captured variables. This is relatively slow, and it adds up if the +enclosing function is called often. Module-level functions and methods +don't have this overhead: + +``` +def total(a: list[int], k: int) -> int: + def weight(x: int) -> int: + return x * k + 1 + + t = 0 + for x in a: + t += weight(x) + return t +``` + +Move the nested function to the module level, and pass captured variables +as arguments: + +``` +def _weight(x: int, k: int) -> int: + return x * k + 1 + +def total(a: list[int], k: int) -> int: + t = 0 + for x in a: + t += _weight(x, k) + return t +``` + +If the nested function is within a method and uses `self`, make it a +method instead (e.g. `self._weight(x)`). + +Only do this if the nested function is simple enough (it captures only a +few variables, and it doesn't assign captured variables using +`nonlocal`), or if it's very performance-critical. Otherwise the +refactored code may be harder to read. + +If the function is only used as a callback, such as +`sorted(a, key=lambda x: x[0])`, or passed as an argument with type +`Callable[...]`, it's called using a generic operation either way. In +these cases, the only benefit is avoiding the allocation, so move the +function only if the enclosing function is called often. + +## Use `vec` Instead of `list` for Packed Item Types + +`vec` (from `librt.vecs`) is a growable array type that stores items of +certain types, such as `i64`, `i32`, `u8`, `float` and `bool`, in a packed +binary representation, without boxing. Building, iterating over, and +indexing a `vec` with a packed item type is much faster than with a +`list[int]` or `list[float]` in compiled code, and uses much less memory. +Consider using a `vec` instead of an internal list of integers or floats +in performance-critical code: + +``` +class Offsets: + def __init__(self) -> None: + self.offsets: list[int] = [] + + def add(self, x: int) -> None: + self.offsets.append(x) + + def total(self) -> int: + t = 0 + for x in self.offsets: + t += x + return t +``` + +Use `vec[i64]` instead: + +``` +from librt.vecs import append, vec +from mypy_extensions import i64 + +class Offsets: + def __init__(self) -> None: + self.offsets = vec[i64]() + + def add(self, x: i64) -> None: + self.offsets = append(self.offsets, x) + + def total(self) -> i64: + t: i64 = 0 + for x in self.offsets: + t += x + return t +``` + +Only do this if the list is internal and changing it doesn't change a +public API. Check all uses of the list, including in tests and +non-compiled code. Be careful to preserve behavior: + +* The length of a `vec` value can't be changed. `append`, `extend`, + `remove` and `pop` are functions that return a new value, which must be + assigned back (as in `add` above). If other code holds a reference to + the same list (for example, it was passed to another object that also + appends to it), the references no longer share changes. Don't convert + lists that are shared like this. +* `vec` isn't a full sequence type, and it doesn't support all `list` + operations and methods (such as `sort`). Don't pass it to code that + expects a `list`. +* Integer values must fit in the item type (e.g. 64 bits for `i64`). + Arithmetic on native integer types isn't checked for overflow. Use + native integer types for related local variables too (such as `t` + above), since conversions between `int` and native integers have a + cost. +* In free-threaded Python builds, `vec` gives fewer thread safety + guarantees than `list`. + +Avoid union types that include vecs, such as `vec[i64] | vec[float]`, +in performance-critical code. Operations on these use slow, generic +operations, which can make the code much slower than with a `list`. +Optional types such as `vec[i64] | None` are also slower than `vec[i64]`, +even after narrowing using `is not None`. If `None` is only used to mean +"no items", use an empty `vec` instead (an empty `vec` is cheap, since it +doesn't allocate a buffer). + +Check the `librt.vecs` documentation (`mypyc/doc/librt_vecs.rst` in the +mypy repository) first, and only use `vec` if the project depends on +`librt` (see "Use librt"). The benefit is much smaller for other item +types, such as `str`, which aren't packed, and there is no benefit in +non-compiled code. + +## Generators + +Generators are efficient in compiled code. Don't replace a generator with +a function that returns a `list`, even if the result is always fully +consumed, as building the list is usually slower. + +An exception is a generator that yields values of a type that `vec` (from +`librt.vecs`) stores in a packed representation, such as `i64` or `float`. +Returning a `vec` can be faster than a generator, especially if the caller +also uses native integer types: + +``` +from typing import Iterator + +def evens(a: list[int]) -> Iterator[int]: + for x in a: + if x % 2 == 0: + yield x +``` + +Return a `vec` instead: + +``` +from librt.vecs import append, vec +from mypy_extensions import i64 + +def evens(a: list[int]) -> vec[i64]: + result = vec[i64]() + for x in a: + if x % 2 == 0: + result = append(result, x) + return result +``` + +This doesn't help for other item types, such as `vec[str]`. Only do this +for internal functions where all callers can be updated, if the values fit +in the fixed-size type, and if the project depends on `librt` (see "Use +librt"). Check the `librt.vecs` documentation first, since `vec` isn't a +full sequence type. Measure the change using a benchmark (see "Measure +Performance"). + +## Replace `@contextmanager` with a Class + +`contextlib.contextmanager` is implemented in non-compiled Python code, +and entering and exiting the context manager involves several slow +operations, even if the decorated generator function is compiled. This is +significant if the `with` statement is on a hot path and the context +manager does little work: + +``` +from contextlib import contextmanager +from typing import Iterator + +@contextmanager +def nested(s: State) -> Iterator[None]: + s.depth += 1 + try: + yield + finally: + s.depth -= 1 +``` + +Use a native class with `__enter__` and `__exit__` methods instead. Use +precise types for the `__exit__` arguments (instead of `*args`), as this +may be faster: + +``` +from types import TracebackType +from typing import final + +@final +class nested: + def __init__(self, s: State) -> None: + self.s = s + + def __enter__(self) -> None: + self.s.depth += 1 + + def __exit__( + self, + typ: type[BaseException] | None, + exc: BaseException | None, + tb: TracebackType | None, + ) -> None: + self.s.depth -= 1 +``` + +The `with nested(s):` statements don't need to be changed. Be careful to +preserve behavior: + +* Code after `yield` in a `finally:` block runs whether or not there was + an exception, and so does `__exit__`. Code after `yield` that isn't + in a `finally:` block only runs if there was no exception, so in + `__exit__`, only run it if `typ is None`. +* If the generator catches exceptions raised at `yield` (using + `except`), the exception is suppressed unless it's re-raised. Return + `True` from `__exit__` to suppress an exception, and make the return + type `bool`. +* If the generator yields a value (used as `with nested(s) as x:`), + return it from `__enter__`. + +## Replace itertools and functools with Plain Code + +Mypyc compiles `for` loops over lists, `range(...)` and other primitive +types into fast code, but `itertools` and `functools` functions are called +through the generic Python API. Replacing them with plain loops and calls +is often 2x to 10x faster in performance-critical code (and 20x faster in a +benchmark where a callback created using `functools.partial` was replaced +with a direct call): + +``` +import functools +import itertools +import operator + +def total(a: list[int], b: list[int]) -> int: + t = 0 + for x in itertools.chain(a, b): + t += x + return t + +def grid_sum(a: list[int], b: list[int]) -> int: + t = 0 + for x, y in itertools.product(a, b): + t += x * y + return t + +def deltas(a: list[int]) -> int: + t = 0 + for x, y in itertools.pairwise(a): + t += y - x + return t + +def sum_all(a: list[int]) -> int: + return functools.reduce(operator.add, a, 0) + +def triple_all(a: list[int]) -> int: + return apply_all(functools.partial(scale, 3), a) +``` + +Use plain loops, indexing and lambdas instead: + +``` +def total(a: list[int], b: list[int]) -> int: + t = 0 + for x in a: + t += x + for x in b: + t += x + return t + +def grid_sum(a: list[int], b: list[int]) -> int: + t = 0 + for x in a: + for y in b: + t += x * y + return t + +def deltas(a: list[int]) -> int: + t = 0 + for i in range(1, len(a)): + t += a[i] - a[i - 1] + return t + +def sum_all(a: list[int]) -> int: + t = 0 + for x in a: + t += x + return t + +def triple_all(a: list[int]) -> int: + return apply_all(lambda x: scale(3, x), a) +``` + +If the function that receives the callback is simple and in the same +module, calling `scale` directly in a loop instead is even faster. + +Be careful to preserve behavior: + +* Replacing `itertools.product` with nested loops is only equivalent if the + inner iterable can be iterated multiple times (`product` makes a copy of + each iterable first). +* Indexing only works for sequences such as lists, not for arbitrary + iterables or iterators. +* `functools.partial` evaluates its arguments once when it's created, but a + lambda evaluates the expressions each time it's called. If an argument is + a variable that may be reassigned later, or an expression with side + effects, assign it to a local variable first. +* A `partial` object has attributes such as `func` and `args`, and a + different `repr`. Check that no code depends on these. + +Only do this in modules that are compiled, as the plain code can be slower +when it's not compiled. + +The speedup depends on the function, the types involved and how much work +each iteration does. For functions not covered above, or if you aren't sure +which replacement to use (e.g. a lambda, a small native class with +`__call__`, or inlining the call into a loop), write a short +microbenchmark that compares the variants first (see "Measure +Performance"), and only refactor if there is a clear win. + +## Use librt + +The `librt` package has faster alternatives to some standard library +features, optimized for compiled code: + +* `librt.strings.StringWriter`: build a `str` (faster than `io.StringIO` + or `"".join(list_of_parts)`). +* `librt.strings.BytesWriter`: build a `bytes` object (faster than + `io.BytesIO`, `bytearray` or `b"".join(...)`). +* `librt.base64`: `b64encode`, `b64decode` and related functions (faster + than the `base64` module). +* `librt.random`: pseudorandom numbers (faster than the `random` module). +* `librt.time.time()`: faster than `time.time()`. +* `librt.threading.Lock`: faster than `threading.Lock`. +* `librt.vecs.vec`: a growable array type with a packed representation + for item types such as `i64` and `float` (see "Use `vec` Instead of + `list` for Packed Item Types"). + +Example of using `StringWriter`: + +``` +def join_items(items: list[str]) -> str: + parts = [] + for s in items: + parts.append(s) + parts.append(",") + return "".join(parts) +``` + +Use `StringWriter` instead: + +``` +from librt.strings import StringWriter + +def join_items(items: list[str]) -> str: + w = StringWriter() + for s in items: + w.write(s) + w.append(ord(",")) # ord(...) of a literal is a compile-time constant + return w.getvalue() +``` + +These aren't always drop-in replacements. Check the librt documentation +(`mypyc/doc/librt*.rst` in the mypy repository, or +https://mypyc.readthedocs.io/) before replacing anything. Examples of +differences: + +* The `librt.base64` decode functions behave differently for malformed data, + and don't support the `altchars` and `validate` arguments. +* `librt.random` produces different values than `random` for the same seed, + and isn't suitable for cryptographic use (neither is `random`). +* Calls to `librt.time.time()` can't be monkey patched in tests. +* `librt.threading.Lock` doesn't support timeouts, subclassing or use with + `threading.Condition`. + +Only use `librt` if the project already depends on it (or you add it as a +dependency). It's installed together with mypy, but compiled code +deployed without mypy needs an explicit dependency. + +## Tune Garbage Collection + +Compiled code often allocates objects faster than non-compiled code, so a +larger fraction of CPU time can be spent in the CPython garbage collector +(GC). Only consider tuning the GC if the GC shows up as expensive in a CPU +profile of a realistic workload (look for CPython functions related to +garbage collection, which usually have names that start with `gc_`). + +Options include making collections less frequent (`gc.set_threshold`), +moving long-lived objects out of GC tracking after startup +(`gc.freeze()`), or disabling the GC during an allocation-heavy phase +(`gc.disable()` followed by `gc.enable()`). The effects vary a lot between +Python versions, since the GC implementation changes between releases, +and some of these can make performance worse or increase memory use. +Measure the effect using a realistic workload. + +GC settings affect the whole process, so they belong in the application +entry point, not in library code. Suggest the change to the user instead +of making it, unless you were asked to tune the application as a whole. + +## Report Other Potential Bottlenecks + +Some code patterns are slow in compiled code, but replacing them often +requires non-trivial tradeoffs, such as changing an API, losing +flexibility, or duplicating code. Don't change these. Instead, report them +to the user as potential bottlenecks if they are on a hot path, and +explain what could be done: + +* Calls to functions that take `*args` or `**kwargs`, and calls that use + `*args` or `**kwargs` to pass arguments. These use slow, generic calls + instead of fast, direct calls. Explicit parameters would be faster. +* Functions decorated using wrapper decorators (decorators other than + special-cased ones such as `@property`, `@staticmethod` and + `@classmethod`), for example decorators used for logging, + tracing, caching or validation. Calls to decorated functions go through + the wrapper using generic calls, and often also use `*args` and + `**kwargs`, which can make each call much slower than calling an + undecorated function. Possible fixes include calling an undecorated + helper from the hot path, or removing the decorator where it's not + needed. If the decorator runs code before and after each call (such as + for tracing or timing), a `with` statement in the function body that + uses a native context manager class (see "Replace `@contextmanager` + with a Class") can be much faster: + + ``` + @traced + def parse(s: str) -> Node: + ... + ``` + + Use a `with` statement instead (`tracing` is a native class with + `__enter__` and `__exit__` methods): + + ``` + def parse(s: str) -> Node: + with tracing("parse"): + ... + ``` + + This is only equivalent if the decorator doesn't change the arguments + or the return value, and if nothing depends on the function being + wrapped (for example, by checking for attributes added by the + decorator).