Release v0.1.0 - #19
Merged
Merged
Release v0.1.0 #19
Conversation
Minimal buildable PyO3 extension that imports successfully:
- src/lib.rs: PyO3 module exposing __version__ from Cargo.toml
- python/pyharfrust/__init__.py: re-exports __version__
- tests/test_import.py: verifies import and version attribute
- pyproject.toml: maturin backend, dev deps in dependency-groups,
pyright config for native module resolution
- Cargo.toml: renamed to pyharfrust/_pyharfrust, harfrust 0.5.2 + pyo3 0.28.3
- .pre-commit-config.yaml: formatting on commit, clippy/tests on push
- .gitignore: trimmed to project-relevant patterns
- uv.lock and Cargo.lock committed for reproducible builds
- github workflow checks for rust and python
Full Phase 1 foundation for the HarfRust Python bindings: a buildable PyO3/maturin package that can be installed, imported, and validated in CI. It sets up the Rust crate, Python packaging metadata, initial extension module wiring, import smoke test, lockfiles, and developer automation (pre-commit + CI) so future shaping APIs can be added on a stable base. ### What’s included - Rust crate scaffold for the extension module (`Cargo.toml`, `src/lib.rs`, `Cargo.lock`) - Python package scaffold and import surface (`python/pyharfrust/__init__.py`, `pyproject.toml`) - Initial test coverage for import/version smoke check (`tests/test_import.py`) - Project ignore and local dev configuration updates (`.gitignore`, `.cargo/config.toml`) - CI pipeline for formatting, linting, build, and tests (`.github/workflows/ci.yml`) - Pre-commit hooks for local quality gates (`.pre-commit-config.yaml`) - Dependency lockfile for reproducible Python tooling (`uv.lock`) ### Validation - Extension builds with `maturin develop` - Import test passes with `pytest` - Rust formatting/lint/test checks and Python lint/format/test checks are wired in CI
Expose harfrust's configuration types as Python classes with string parsing, properties, equality, and hashing. Script validates input as 4-letter ISO 15924 tags rather than inheriting harfrust's permissive parsing. All error messages include the offending input.
## Summary
Gives Python users the building blocks needed to configure text shaping:
Direction, Script, Language, Feature, and Variation. These are the types
that will flow into Buffer and Font in later phases.
Each type is constructed from a string using the same syntax as
harfbuzz/harfrust (e.g. `Feature("+kern")`, `Variation("wght=700")`,
`Direction("rtl")`), making the API familiar to anyone coming from the
harfbuzz ecosystem. Script enforces valid 4-letter ISO 15924 tags rather
than silently accepting garbage input.
This is an intentional Python API divergence from core harfrust's
lenient script parsing, so invalid script tags fail fast with a clear
ValueError.
## Test plan
- 46 passing tests covering construction, string round-tripping,
equality, hashing, error messages, and edge cases for all five types
- `maturin develop && pytest tests/ -v`
Wrap hr_shape::shape() and hr_shape::run_from_args() as Python functions, stripping trailing newlines from hr-shape output. Add .tests file regression infrastructure with bundled test fonts and 6 verified test cases covering Latin, RTL, Indic, variable fonts, and mark attachment.
- Drop redundant closures in shape.rs to satisfy clippy -D warnings. - Use a context manager for .tests file reading. Scaffold a pytest 'external' marker so Tier 2 cases (from HARFRUST_SOURCE) can be filtered via -m external / -m "not external" once the Tier 2 parser lands.
Python can now shape text end-to-end through pyharfrust, and the project has a regression-test foundation built around the same `.tests` file format that harfbuzz and harfrust use upstream. ## What's conceptually done - **Shaping is callable from Python.** `pyharfrust.shape(font, text, options)` and `pyharfrust.run_from_args([...])` expose hr-shape's high-level entry points as ordinary Python functions. This is deliberately the *thin* path — the textual shape report, not a structured buffer API — so we have a known-good oracle to test richer bindings against as they get added. - **A regression harness that speaks HarfBuzz's native test format.** Tests are driven by `.tests` files, parametrized through a pytest fixture. Each line becomes one test case, so expanding coverage is a data change, not a code change. - **Hermetic bundled suite in-repo.** A small set of fonts plus a curated `bundled.tests` file lives alongside the tests so CI always has something to run without reaching outside the repo. Cases cover Latin, RTL reordering, Indic reordering, variable-font axis selection, and mark attachment — a deliberately diverse set chosen to catch regressions in the major shaping paths. - **Hook for a future external suite.** The conftest already branches on a `HARFRUST_SOURCE` env var and registers a pytest `external` marker, so when the parser for the upstream harfrust suite lands it can be wired in without restructuring the harness. ## Why this shape The binding is intentionally minimal here. The goal wasn't to design the Pythonic shaping API — that comes later — it was to prove the build/link path works end-to-end from Python down to hr-shape, and to stand up the regression scaffolding before adding surface area, so future work lands against a real test suite instead of ad-hoc scripts.
Introduces an opt-in regression mode that shapes every test case in a local harfrust checkout through pyharfrust.shape and compares against the expected output embedded in harfrust's own tests. Enabled by setting HARFRUST_SOURCE; default-denied in pytest config so local dev and the main-branch CI stay fast. CI runs Tier 2 only on push/PR targeting the release branch, against harfrust pinned at commit efdae31 (0.5.2) to match the hr-shape dep.
The `_collect_external_cases()` function was incorrectly parsing `.tests` files from harfrust's `tests/custom/` directory. These files are source inputs for harfrust's test generator (`gen-shaping-tests.py`), not actual test cases to run. They contain expected values that may intentionally differ from harfrust's current behavior, as noted in the files themselves: "the expected values for the shaping process will be ignored and sometimes wrong." This caused false test failures, such as the `--language=pl` BigCaslon test which expects `cacute.polish` but harfrust outputs `cacute` (confirmed by running hr-shape CLI directly). Now we only parse the generated `.rs` files in `tests/shaping/`, which represent the actual tests that `cargo test` runs in harfrust. Test count: 6145 → 6139 external cases (removed 6 invalid cases from .tests)
Get a high-confidence signal that `pyharfrust.shape` stays faithful to
upstream harfrust output — not just on the handful of cases we bundle,
but on the full shaping corpus harfrust itself uses to guard against
regressions.
Bundled Tier 1 tests are fast and self-contained, but they cover a tiny
slice of scripts and features. Every harfrust version bump is a chance
for subtle output drift that our ~6 bundled cases would never catch.
Tier 2 closes that gap.
## Approach
**Opt-in, not always-on.** A harfrust checkout is a heavy dependency
(fonts, generated test files, a specific commit pin), and the suite
parametrizes into ~6 k cases. Making it the default would punish every
local `pytest` run and every `main`-branch CI job for a signal that only
matters at release time.
The mechanism:
- Set `HARFRUST_SOURCE=/path/to/harfrust` to enable.
- Cases are marked `@pytest.mark.external` at collection time.
- `pyproject.toml` default-denies via `addopts = "-m 'not external'"`,
so Tier 2 only runs when the marker filter is explicitly overridden
(`pytest -m external` or `-m ""`).
- A `MIN_EXTERNAL_CASES` floor ensures a silently-broken parser can't
pass by collecting zero cases.
**Parse, don't re-run.** harfrust's corpus lives in generated
`tests/shaping/*.rs` files containing literal `shape(...)` assertions.
We extract the `(font, text, options, expected)` tuples with a targeted
regex (handles `\u{XXXX}` escapes and line continuations), then compare
against `pyharfrust.shape` output. We intentionally skip the
`tests/custom/*.tests` source files — those are generator inputs that
may contain stale expected values.
**CI pinned to the shipped version.** Tier 2 runs on push/PR targeting
`release`, against harfrust checked out at commit `efdae31` (the 0.5.2
tag, matching the `hr-shape` version in `Cargo.toml`). Bumping the dep
means bumping the SHA in the same PR — corpus and shaper stay in
lockstep. Main-branch CI is unchanged and still fast.
## Why regex over an AST
The `.rs` files are machine-generated with a rigid, uniform
`shape("...", "...", "..."), "..."` shape. A full Rust parser
(tree-sitter-rust, syn) would be overkill, add a heavy dep, and gain
nothing on content this regular. If harfrust's generator format ever
changes materially, the regex will fail loudly (via the min-case guard)
rather than silently drift.
## Out of scope
- Vendoring the harfrust corpus into this repo (too large, and would
defeat the point of testing against upstream).
- Automating the harfrust SHA bump alongside `hr-shape` — manual step,
intentional coupling.
Update ROADMAP.md to use the pyharfrust package name (python/pyharfrust/ and pyharfrust.shape()) and point README.md at ROADMAP.md instead of the renamed PLAN.md.
Update ROADMAP.md to use the pyharfrust package name (python/pyharfrust/ and pyharfrust.shape()) and point README.md at ROADMAP.md instead of the renamed PLAN.md.
Pythonic Buffer class backed by Option<UnicodeBuffer>, which establishes the consumption pattern for Phase 5 shape() without yet exposing a consumer. Covers the operations needed to drive shaping end-to-end: text accumulation, direction/script/language properties, guess_segment_properties, context setters, and clear.
Adds a Buffer test covering set_not_found_variation_selector_glyph, which was exposed but previously unexercised. Marks the Direction label match inside Buffer.__repr__ with a comment pointing at the equivalent (private) match in PyDirection::__repr__, so the next edit to either keeps them in sync. Expands the TestContext header to spell out why pre/post context tests only assert that the call doesn't raise.
- Adds `Buffer` — the Pythonic entry point for feeding text into the shaping pipeline. - Wraps harfrust's `UnicodeBuffer` behind `Option<UnicodeBuffer>` so the consumption pattern (`ValueError` on use-after-shape) is wired in before any consumer exists. Phase 5's `Font.shape()` will call the `pub(crate) take_inner` / `restore` hooks without further refactor. - Exposes the operations Phase 5 will need: `add_str`, `add(codepoint, cluster=0)`, `clear`, `reset_clusters`, `guess_segment_properties`, `reserve`,`set_pre_context` / `set_post_context`, and `direction` / `script` / `language` properties. `language` returns `None` when unset, matching upstream. ## Out of scope - The actual buffer-consumption code path — has to wait for `Font.shape()` in Phase 5. ## Test plan - [x] `cargo fmt --all --check` - [x] `cargo clippy --all -- -D warnings` - [x] `cargo test --all` - [x] `uv run maturin develop` - [x] `uv run ruff check && uv run ruff format --check` - [x] `uv run pytest` — 81 passed (25 new, 56 pre-existing)
- Added `Font` class for loading and managing font data, including methods for creating instances from file paths and byte arrays. - Introduced `GlyphBuffer`, `GlyphInfo`, and `GlyphPosition` classes to handle glyph shaping and positioning, providing a structured way to access glyph information. - Updated `__init__.py` to include new classes in the module's public API. - Enhanced `Buffer` class with methods for handling glyph data and serialization. - Added comprehensive tests for the new font and glyph functionalities, ensuring correct behavior and integration with existing components.
- Updated `test_glyph_info_fields` to assert additional properties of glyph information, including cluster values for multiple glyphs. - Introduced `test_from_bytes_matches_from_path` to verify that font data loaded from bytes matches that loaded from a file path. - Adjusted error handling in `test_features_wrong_type` to ensure proper feature validation.
This PR adds the object-oriented shaping API: load fonts from path or bytes, configure axis variations and point size, shape a `Buffer`, and work with the resulting `GlyphInfo` / `GlyphPosition` data. Buffers can be recycled through `GlyphBuffer.clear()`, and serialized output can be matched to the high-level `shape()` helper for the cases we cover in tests. ## What’s included - **`Font`** — load via path (`Font(path)`) or `Font.from_bytes`, optional `face_index`, `units_per_em`, `set_variations` (string or `Variation` list), `set_point_size`, and `shape(buffer, features=…)` with string or `Feature` list. - **`GlyphBuffer`**, **`GlyphInfo`**, **`GlyphPosition`** — length, index access, iteration as `(info, pos)` pairs, `glyph_infos` / `glyph_positions` getters, and `serialize(font)` for HarfRust-style glyph strings. - **Safety / UX** — clear errors when the buffer’s direction is still invalid before shaping, and explicit consumption semantics for `Buffer` (after `shape`) and `GlyphBuffer` (after `clear()`) to match the owned-container design. - **Tests** — construction, shaping, iteration, feature and variation options, point size, serialize parity with `pyharfrust.shape`, and buffer consumption / recycling.
Ship `__init__.pyi` covering every public type and function with property signatures, exception notes, and `Sequence[Feature] | str | None` unions that match the runtime accepted shapes. Add an empty `py.typed` marker so type checkers pick up the stubs from an installed wheel — maturin already includes both files from `python-source` without further config. Suppress pyright on the five negative-type test lines that intentionally pass wrong types to verify TypeError. Rewrite the README around installation, the two-tier API (high-level `shape()` and the `Font` / `Buffer` / `GlyphBuffer` object API), variable fonts, buffer recycling, and the error taxonomy.
…Buffer behavior. Added example for `run_from_args(argv)` to demonstrate full CLI parity with `hb-shape`. Updated description of `GlyphBuffer.clear()` to specify its consumption behavior and error handling.
This pull request introduces comprehensive improvements and clarifications to the Python API and documentation for the `pyharfrust` package. The main changes include the addition of a detailed installation and usage guide in the `README.md`, the introduction of a complete type stub file (`__init__.pyi`) for type checking and editor support, and minor test adjustments to suppress type checker warnings. **Documentation and API improvements:** * Added a detailed installation, quick start, and advanced usage guide to `README.md`, covering building from source, high-level and object-oriented shaping APIs, configuration types, variable fonts, buffer recycling, and error handling. * Added a complete PEP 561 type stub file `python/pyharfrust/__init__.pyi`, describing all public classes, methods, properties, and exceptions, enabling type checkers and editors to offer full API support.
New `release.yml`: tag-push (`v*`) builds wheels for Linux x86_64/aarch64, macOS x86_64/aarch64, and Windows x64 across CPython 3.11/3.12/3.13, plus an sdist. The `publish` job uploads to PyPI via OIDC trusted publishing and uses `skip-existing` so re-runs are idempotent. The `gate` job rejects any tag whose SHA isn't the current tip of `release`. Combined with branch protection requiring `ci` to pass on `release`, that turns "tip of release" into a hard guarantee that full CI (Tier 1 + Tier 2) ran on the exact SHA being released — so the release workflow doesn't re-run tests itself. Add a `release-source-gate` job to `ci.yml` that fails any PR targeting `release` whose head ref isn't `main`. GitHub branch protection has no "restrict source branch" setting, so we enforce it as a required check. `RELEASING.md` documents guarantees (branch protection rules, PyPI trusted publisher) and the per-release procedure, plus recovery from a lost tag race.
This pull request introduces a robust and secure release process by adding new workflows and documentation to enforce stricter controls over how releases are created and published. The changes ensure that releases are only made from well-tested code, prevent accidental or unauthorized modifications to the release branch, and provide clear instructions for maintainers. Key changes include: **Release Process Enforcement:** - Added a `release-source-gate` job to `.github/workflows/ci.yml` to ensure that pull requests to the `release` branch can only come from the `main` branch, blocking all other sources. - Introduced a new `.github/workflows/release.yml` workflow, which: - Ensures release tags only point to the tip of the `release` branch via a gating job. - Builds and uploads Python wheels for Linux, macOS, and Windows (across multiple architectures and Python versions), as well as a source distribution. - Publishes artifacts to PyPI using trusted publishing with OIDC, and only after all builds succeed. **Documentation:** - Added a comprehensive `RELEASING.md` file detailing the release process, the guarantees provided by the new checks and workflows, and troubleshooting steps for common issues.
- Modified `.pre-commit-config.yaml` to enforce checks on both `main` and `release` branches. - Updated `.github/workflows/ci.yml` to pin the `harfrust` repository reference to a specific commit for consistency in CI testing.
- Modified `.pre-commit-config.yaml` to enforce checks on both `main` and `release` branches. - Updated `.github/workflows/ci.yml` to pin the `harfrust` repository reference to a specific commit for consistency in CI testing.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Changes
This is the first release of
pyharfrust. It provides a complete, IDE-typed Python binding for the harfrust shapingengine.
Core API
shape(font_path, text, options)andrun_from_args(argv)matching thehb-shapeCLI.Font(file or bytes),Buffer,GlyphBuffer,GlyphInfo,GlyphPosition. Buffers are consumed byshape()and recyclable viaGlyphBuffer.clear().Font.set_variations()andFont.set_point_size().Value types
Direction,Script,Language,Feature,Variationwith string-based construction matchinghb-shapesyntax.Packaging
__init__.pyi,py.typed).release-source-gatePR check, OIDC trusted publishing — seeRELEASING.md.Known limitations (deferred)
GlyphBuffer.__getitem__doesn't accept slices.GlyphInfoandGlyphPositiondon't support value equality.GlyphBuffer.serializedoesn't acceptSerializeFlags.Font.set_named_instance(index)for variable fonts.FontRefis re-parsed inside everyFont.shape()/Font.units_per_emcall (perf, not correctness).Test plan
release-source-gate).releaseperRELEASING.mdto trigger wheel build + PyPI publish.pyharfrust==0.1.0installs and imports on a clean venv.