gh-87613: Use AC vectorcall for bytes and bytearray

[cmaloney]

Speeds up common construction of both objects.

edit: Dropped the no-copy unique object path as it is not sound as noted by PIeter

Benchmark

Generally faster: the fixed per-call overhead drops by about 3–9 ns, which shows most on small inputs. Copy-dominated sizes (1MiB) are unchanged.

Run on an Apple M3 (macOS): PGO+LTO, Apple clang 21, --with-tail-call-interp. Base is 85c03d4 (merge-base with main).
(note: AI Written, but I think sufficiently representative)

"""bytes() and bytearray() construction.

temp/ passes a unique temporary (adopted, no copy); named/ binds the same
source to a name first so it is always copied.
"""
import pyperf

runner = pyperf.Runner()

runner.timeit('call/bytes()', 'bytes()')
runner.timeit('call/bytes(int)', 'bytes(16)')
runner.timeit('call/bytes(bytes)', "bytes(b'abcdefghabcdefgh')")
runner.timeit("call/bytes(str, 'ascii')", "bytes('abcdefgh', 'ascii')")
runner.timeit('call/bytearray()', 'bytearray()')
runner.timeit('call/bytearray(int)', 'bytearray(16)')
runner.timeit('call/bytearray(bytes)', "bytearray(b'abcdefghabcdefgh')")
runner.timeit("call/bytearray(str, 'ascii')",
              "bytearray('abcdefgh', 'ascii')")

for size_name, size in (('16B', 16), ('4KiB', 4096), ('1MiB', 1 << 20)):
    # Slicing in the statement makes a fresh unique temporary.
    setup = f"b = b'x' * {size}; ba = bytearray(b)"
    runner.timeit(f'temp/bytes(bytearray) {size_name}',
                  'bytes(ba[1:])', setup)
    runner.timeit(f'named/bytes(bytearray) {size_name}',
                  'x = ba[1:]; bytes(x); del x', setup)
    runner.timeit(f'temp/bytearray(bytes) {size_name}',
                  'bytearray(b[1:])', setup)
    runner.timeit(f'named/bytearray(bytes) {size_name}',
                  'x = b[1:]; bytearray(x); del x', setup)
    runner.timeit(f'temp/bytearray(bytearray) {size_name}',
                  'bytearray(ba[1:])', setup)
    runner.timeit(f'named/bytearray(bytearray) {size_name}',
                  'x = ba[1:]; bytearray(x); del x', setup)

Benchmark	main	patch
call/bytes()	11.5 ns	8.26 ns: 1.39x faster
call/bytes(bytes)	32.4 ns	23.3 ns: 1.39x faster
call/bytes(str, 'ascii')	34.9 ns	25.5 ns: 1.37x faster
call/bytes(int)	31.8 ns	23.7 ns: 1.34x faster
call/bytearray(int)	38.1 ns	29.9 ns: 1.27x faster
call/bytearray(bytes)	40.7 ns	32.6 ns: 1.25x faster
call/bytearray(str, 'ascii')	44.0 ns	36.3 ns: 1.21x faster
temp/bytes(bytearray) 16B	62.1 ns	52.9 ns: 1.17x faster
temp/bytearray(bytes) 16B	57.1 ns	49.2 ns: 1.16x faster
named/bytes(bytearray) 16B	63.7 ns	55.0 ns: 1.16x faster
named/bytearray(bytes) 16B	59.2 ns	51.1 ns: 1.16x faster
call/bytearray()	20.9 ns	18.2 ns: 1.15x faster
temp/bytearray(bytearray) 16B	63.5 ns	55.4 ns: 1.15x faster
named/bytearray(bytearray) 16B	65.5 ns	57.3 ns: 1.14x faster
named/bytes(bytearray) 4KiB	186 ns	173 ns: 1.08x faster
temp/bytes(bytearray) 4KiB	185 ns	173 ns: 1.07x faster
named/bytearray(bytearray) 4KiB	191 ns	179 ns: 1.07x faster
temp/bytearray(bytearray) 4KiB	187 ns	178 ns: 1.05x faster
temp/bytearray(bytes) 4KiB	183 ns	176 ns: 1.04x faster
named/bytearray(bytearray) 1MiB	29.5 us	28.4 us: 1.04x faster
named/bytearray(bytes) 4KiB	185 ns	179 ns: 1.04x faster
named/bytes(bytearray) 1MiB	30.7 us	29.7 us: 1.03x faster
Geometric mean	(ref)	1.14x faster

Benchmark hidden because not significant (4): named/bytearray(bytes) 1MiB, temp/bytes(bytearray) 1MiB, temp/bytearray(bytes) 1MiB, temp/bytearray(bytearray) 1MiB

The unique-temporary adoption (no copy) was split out to a follow-up after review found it unsafe when a C caller still holds a borrowed reference (see review thread). The temp/ rows now match named/.

• Issue: Generate vectorcall code to parse arguments using Argument Clinic #87613

[cmaloney]

note: bytearray is an immutable type so I don't think this can be overridden easily to do arbitrary other things. Can export and direct call into the C if that is preferred.

>>> a = bytearray()
>>> def tb_test(self):
...     return b"abc"
...     
>>> a.take_bytes = tb_test
Traceback (most recent call last):
  File "<python-input-4>", line 1, in <module>
    a.take_bytes = tb_test
    ^^^^^^^^^^^^
AttributeError: 'bytearray' object attribute 'take_bytes' is read-only

[eendebakpt]

The vectorcall changes look good. But the optimizations need a second look. For example:

>>> import bisect
>>> lst = []
>>> bisect.insort(lst, bytearray(b'abc')[1:], key=bytes)
>>> lst
[bytearray(b'')]      # main: [bytearray(b'bc')]

For that reason maybe split the PR?

[cmaloney]

Splitting the PR

It looks like this gets broken because a lot of the stack uses "borrowed" references which make it look like it's uniquely referenced when it is still in use in multiple paths.... Still want to do the no-copy optimization and the similar one with iconcat but clearly still need some different tools.

[vstinner]

LGTM

[vstinner]

The commit message of the merged change says:

| temp/bytearray(bytearray) 1MiB  | 515 us  | 14.9 us: 34.53x faster |
| temp/bytearray(bytes) 1MiB      | 508 us  | 14.8 us: 34.27x faster |
| temp/bytes(bytearray) 1MiB      | 507 us  | 14.9 us: 34.05x faster |

This speedup was with the "avoid copy" optimization using PyUnstable_Object_IsUniqueReferencedTemporary() which was not merged at the end, no? So these benchmark results are outdated, no?

[vstinner]

Nice optimization!

[cmaloney]

The benchmarks in the PR are up to date with the final version, the commit message ones are old