Skip to content

s3torchconnectorclient: Add version 1.5.0 - #2965

Draft
luhenry wants to merge 3 commits into
mainfrom
s3torchconnectorclient
Draft

luhenry wants to merge 3 commits into
mainfrom
s3torchconnectorclient

Conversation

@luhenry

@luhenry luhenry commented Oct 13, 2026 •

Copy link
Copy Markdown
Member

Builds the PyO3/setuptools-rust native extension wrapping AWS's Mountpoint-S3 Rust client (and its statically-linked AWS Common Runtime) that backs the s3torchconnector PyTorch package. Upstream publishes no riscv64 wheel.

Mirrors upstream's wheels.yml, minus its CodeBuild runner matrix and AWS-credentialed steps.

Differs from upstream

  • Dropped the CentOS7 devtoolset/llvm-toolset packages - this image's own gcc is new enough
  • rustup installs the Rust toolchain - the image ships none
  • license-files points at LICENSE.* instead of the ORT-generated THIRD-PARTY-LICENSES - never checked into the repo

Matrix: cp312/cp313/cp314 only - s3torchconnector (its own test dependency) needs torch, which has no riscv64 wheel below cp312

Testing

  • Drops the four CI_*-prefixed S3 integration/e2e commands - need real AWS credentials this CI has none of
  • Runs the client's and connector's own unit suites instead, both backed by the mock S3 client

License: Wheel statically links the AWS Common Runtime (aws-lc, s2n-tls, aws-c-common, aws-c-cal, aws-c-io, aws-c-compression, aws-c-http, aws-c-sdkutils, aws-c-auth, aws-checksums, aws-c-s3; all Apache-2.0); the build collects each one's licence text since upstream's own THIRD-PARTY-LICENSES generation step is not reproducible from a plain checkout.

Patches

  • 0001-ship-the-licences-of-the-statically-linked-dependencies.patch - Upstream-Status: To upstream. Points license-files at the per-dependency files the build now collects; reproduces off riscv64 too.

Local build skipped: this host has under 2.5 GB free, too tight to vendor the AWS-LC/s2n-tls source tree cargo needs; CI is the first real build.

luhenry added a commit that referenced this pull request Oct 13, 2026
@luhenry
luhenry marked this pull request as draft October 13, 2026 09:26
@github-actions

Copy link
Copy Markdown
Contributor
PR Preview Action v1.8.1

QR code for preview link

🚀 View preview at
https://riseproject-dev.github.io/python-wheels/pr-preview/pr-2965/

Built to branch gh-pages at 2026-10-13 13:22 UTC.
Preview will be ready when the GitHub Pages deployment is complete.

The cargo registry keeps crate sources one level below registry/src
(index.crates.io-<hash>/<crate>), so the -maxdepth 1 lookup found
nothing and the copy failed on '/LICENSE'. The mountpoint-s3-crt-sys
crate also ships no top-level LICENSE, only one per vendored CRT
library, so drop that copy.

mountpoint-s3-crt-sys generates its bindings with bindgen at build
time, which needs libclang: install clang/clang-devel as upstream's
own manylinux_2_28 before-all does.
…esis deadlines in tests

The wheel built and repaired on every interpreter, but installing its
own [test] extra failed: PyPI's newest Pillow (13.0.0) has no riscv64
wheel, so pip picked its sdist over the registry's 12.3.0 wheel and the
build died on the missing libjpeg headers. PIP_ONLY_BINARY=pillow makes
the registry wheel the only candidate.

cibuildwheel's container does not inherit the runner's CI variable, so
Hypothesis keeps its default 200ms per-example deadline, which the
connector's unit suite already exceeds intermittently on x86_64
(test_s3_client_custom_config). CI=true selects Hypothesis' built-in ci
profile, which drops the deadline and the too_slow health check.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant