Repository navigation
Avoid removed Py_FileSystemDefaultEncoding in load_library - #287
Open
ozeranskii wants to merge 2 commits into
Open
ozeranskii wants to merge 2 commits into
ozeranskii wants to merge 2 commits into
Conversation
ozeranskii
added a commit
to ozeranskii/cffi
that referenced
this pull request
Oct 18, 2026
This was referenced Oct 18, 2026
ozeranskii
added a commit
to ozeranskii/cffi
that referenced
this pull request
Oct 18, 2026
ozeranskii
force-pushed
the
fix-py316-fsdefaultencoding
branch
from
October 18, 2026 18:47
9159b58 to
c12ae81
Compare
ozeranskii
added a commit
to ozeranskii/cffi
that referenced
this pull request
Oct 19, 2026
ozeranskii
force-pushed
the
fix-py316-fsdefaultencoding
branch
from
October 19, 2026 17:59
c12ae81 to
46d7a1e
Compare
CPython 3.16 removed Py_FileSystemDefaultEncoding from the headers, so _cffi_backend.c no longer compiled. Convert the file name with PyUnicode_FSConverter instead, which uses the filesystem encoding, rejects embedded null characters and is available on all supported versions. The encoded name is kept in *p_temp, which the callers already release.
ozeranskii
force-pushed
the
fix-py316-fsdefaultencoding
branch
from
October 19, 2026 18:36
46d7a1e to
38dd2f6
Compare
Author
|
@mattip, could you approve the CI workflows for the latest push? The first run failed only in |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
CPython 3.16 removed
Py_FileSystemDefaultEncodingfrom the headers (python/cpython#158020), so_cffi_backend.cno longer compiles there:This replaces the
"et"conversion inb_do_dlopenwith"O&"+PyUnicode_FSConverter. That converter encodes with the interpreter's filesystem encoding, rejects embedded null characters, releases its result if a later argument fails to parse, and has the same behaviour on every supported version (3.10+), so no version checks are needed. TheNone, cdata and Windows wide-char paths are unchanged.PyUnicode_EncodeFSDefault, as suggested in #86, would be shorter. It does not check for embedded nulls, though, soload_library("/usr/lib/libc.so.6\0x")would silently load the truncated path. Today that raisesTypeError; with this change it raisesValueError: embedded null byte, likeopen(). The new test covers this and the existing rejection of bytes. Non-ASCII file names are already covered bytesting/cffi1/test_re_python.pyandtesting/cffi0/test_ownlib.py.Addresses the
Py_FileSystemDefaultEncodingpart of #180 (_PyLong_Signremains). Refs #86.Tested on macOS arm64 with
python setup.py build_ext -i && pytest src/c/test_c.pyon 3.10 through 3.15t (unchanged: 225 passed, 6 skipped) and on 3.16.0a1 / 3.16.0a1t (previously failed to build, now 225 passed, 6 skipped). I have not tested on Windows, where astrgoes through the separatedlopenWinWbranch, which this does not touch. That branch already truncates at an embedded null, so the test skips that check on win32. This PR does not add 3.16 to CI, since neither cibuildwheel nor setup-python ship 3.16 yet. I'm happy to send that as a follow-up once they do.