Skip to content

Commit cf69000

Browse files
authored
gh-139122: Reimplement base UUID type, uuid4(), and uuid7() in C (#139123)
The C implementation considerably boosts the performance of the key UUID operations: ------------------------------------ Operation Speedup ------------------------------------ uuid4() generation 15.01x uuid7() generation 29.64x UUID from string 6.76x UUID from bytes 5.16x str(uuid) conversion 6.66x ------------------------------------ Summary of changes: * The UUID type is reimplemented in C in its entirety. * The pure-Python is kept around and is used of the C implementation isn't available for some reason. * Both implementations are tested extensively; additional tests are added to ensure that the C implementation of the type follows the pure Python implementation fully. * The Python implementation stores UUID values as int objects. The C implementation stores them as `uint8_t[16]` array. * The C implementation has faster hash() implementation but also caches the computed hash value to speedup cases when UUIDs are used as set/dict keys. * The C implementation has a freelist to make new UUID object instantiation as fast as possible. * uuid4() and uuid7() are now implmented in C. The most performance boost (10x) comes from overfetching entropy to decrease the number of _PyOS_URandom() calls. On its own it's a safe optimization with the edge case that Unix fork needs to be explicitly handled. We do that by comparing the current PID to the PID of when the random buffer was populated. * Portions of code are coming from my implementation of faster UUID in gel-python [1]. I did use AI during the development, but basically had to rewrite the code it generated to be more idiomatic and efficient. * The benchmark can be found here [2]. * This PR makes Python UUID operations as fast as they are in NodeJS and Bun runtimes. [1] https://github.com/MagicStack/py-pgproto/blob/b8109fb311a59f30f9947567a13508da9a776564/uuid.pyx [2] https://gist.github.com/1st1/f03e816f34a61e4d46c78ff98baf4818
1 parent 239808c commit cf69000

13 files changed

Lines changed: 2842 additions & 63 deletions

‎Doc/library/uuid.rst‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -69,6 +69,11 @@ which relays any information about the UUID's safety, using this enumeration:
6969
variant and version number set according to :rfc:`9562`, overriding bits in the
7070
given *hex*, *bytes*, *bytes_le*, *fields*, or *int*.
7171

72+
.. versionchanged:: next
73+
Passing an object other than :class:`bytes` as *bytes* or *bytes_le*
74+
now consistently raises :exc:`TypeError`. Previously, some invalid inputs
75+
raised :exc:`AssertionError` or :exc:`ValueError`.
76+
7277
Comparison of UUID objects are made by way of comparing their
7378
:attr:`UUID.int` attributes. Comparison with a non-UUID object
7479
raises a :exc:`TypeError`.

‎Doc/whatsnew/3.16.rst‎

Lines changed: 40 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -852,6 +852,18 @@ unicodedata
852852
* The Unicode database has been updated to Unicode 18.0.0.
853853

854854

855+
uuid
856+
----
857+
858+
* :class:`uuid.UUID`, :func:`uuid.uuid4`, and :func:`uuid.uuid7` now have
859+
C implementations, with a pure Python fallback.
860+
In microbenchmarks, UUIDv4 and UUIDv7 generation are approximately 15 and
861+
30 times faster, respectively, than the pure Python implementations.
862+
UUID construction and string conversion are also faster; see
863+
:ref:`whatsnew316-uuid-optimizations` for the benchmark results.
864+
(Contributed by Yury Selivanov in :gh:`139123`.)
865+
866+
855867
xml
856868
---
857869

@@ -960,6 +972,28 @@ re
960972
:gh:`152056`.)
961973

962974

975+
.. _whatsnew316-uuid-optimizations:
976+
977+
uuid
978+
----
979+
980+
* C implementations accelerate UUID generation, construction, and string
981+
conversion. Microbenchmarks comparing the C and pure Python implementations
982+
measured the following speedups:
983+
984+
================================ =======
985+
Operation Speedup
986+
================================ =======
987+
:func:`uuid.uuid4` generation 15.01x
988+
:func:`uuid.uuid7` generation 29.64x
989+
:class:`uuid.UUID` from a string 6.76x
990+
:class:`uuid.UUID` from bytes 5.16x
991+
:func:`str` conversion of a UUID 6.66x
992+
================================ =======
993+
994+
(Contributed by Yury Selivanov in :gh:`139123`.)
995+
996+
963997
Removed
964998
========
965999

@@ -1158,6 +1192,12 @@ that may require changes to your code.
11581192
:exc:`TypeError`.
11591193
(Contributed by Serhiy Storchaka in :gh:`152587`.)
11601194

1195+
* :class:`uuid.UUID` now consistently raises :exc:`TypeError` when the
1196+
*bytes* or *bytes_le* argument is not a :class:`bytes` object.
1197+
Previously, some invalid inputs raised :exc:`AssertionError` or
1198+
:exc:`ValueError`.
1199+
(Contributed by Yury Selivanov in :gh:`139123`.)
1200+
11611201
* :mod:`xml.dom.minidom` now raises :exc:`~xml.dom.InvalidCharacterErr`
11621202
for a name which is not a valid XML name,
11631203
:exc:`~xml.dom.WrongDocumentErr`

‎Include/internal/pycore_global_objects_fini_generated.h‎

Lines changed: 6 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎Include/internal/pycore_global_strings.h‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -349,6 +349,7 @@ struct _Py_global_strings {
349349
STRUCT_FOR_ID(byte_size)
350350
STRUCT_FOR_ID(byteorder)
351351
STRUCT_FOR_ID(bytes)
352+
STRUCT_FOR_ID(bytes_le)
352353
STRUCT_FOR_ID(bytes_per_sep)
353354
STRUCT_FOR_ID(c_call)
354355
STRUCT_FOR_ID(c_exception)
@@ -543,6 +544,7 @@ struct _Py_global_strings {
543544
STRUCT_FOR_ID(hash_name)
544545
STRUCT_FOR_ID(header)
545546
STRUCT_FOR_ID(headers)
547+
STRUCT_FOR_ID(hex)
546548
STRUCT_FOR_ID(hi)
547549
STRUCT_FOR_ID(hook)
548550
STRUCT_FOR_ID(host)
@@ -576,13 +578,15 @@ struct _Py_global_strings {
576578
STRUCT_FOR_ID(insert_comments)
577579
STRUCT_FOR_ID(insert_pis)
578580
STRUCT_FOR_ID(instructions)
581+
STRUCT_FOR_ID(int)
579582
STRUCT_FOR_ID(intern)
580583
STRUCT_FOR_ID(intersection)
581584
STRUCT_FOR_ID(interval)
582585
STRUCT_FOR_ID(io)
583586
STRUCT_FOR_ID(is_compress)
584587
STRUCT_FOR_ID(is_raw)
585588
STRUCT_FOR_ID(is_running)
589+
STRUCT_FOR_ID(is_safe)
586590
STRUCT_FOR_ID(is_struct)
587591
STRUCT_FOR_ID(isatty)
588592
STRUCT_FOR_ID(isinstance)
@@ -756,6 +760,7 @@ struct _Py_global_strings {
756760
STRUCT_FOR_ID(query)
757761
STRUCT_FOR_ID(queuetype)
758762
STRUCT_FOR_ID(quotetabs)
763+
STRUCT_FOR_ID(random_func)
759764
STRUCT_FOR_ID(raw)
760765
STRUCT_FOR_ID(read)
761766
STRUCT_FOR_ID(read1)
@@ -870,6 +875,7 @@ struct _Py_global_strings {
870875
STRUCT_FOR_ID(threading)
871876
STRUCT_FOR_ID(throw)
872877
STRUCT_FOR_ID(time)
878+
STRUCT_FOR_ID(time_func)
873879
STRUCT_FOR_ID(timeout)
874880
STRUCT_FOR_ID(timer)
875881
STRUCT_FOR_ID(times)

‎Include/internal/pycore_pylifecycle.h‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -96,8 +96,8 @@ extern const char* _Py_gitversion(void);
9696
// Export for '_asyncio' shared extension
9797
PyAPI_FUNC(int) _Py_IsInterpreterFinalizing(PyInterpreterState *interp);
9898

99-
/* Random */
100-
extern int _PyOS_URandom(void *buffer, Py_ssize_t size);
99+
// Export for '_uuid' shared extension
100+
PyAPI_FUNC(int) _PyOS_URandom(void *buffer, Py_ssize_t size);
101101

102102
// Export for '_random' shared extension
103103
PyAPI_FUNC(int) _PyOS_URandomNonblock(void *buffer, Py_ssize_t size);

‎Include/internal/pycore_runtime_init_generated.h‎

Lines changed: 6 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎Include/internal/pycore_unicodeobject_generated.h‎

Lines changed: 6 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

0 commit comments

Comments
 (0)