Repository navigation
release: v0.3.0 - #155
Merged
Merged
release: v0.3.0#155
Conversation
#28, #29, #45 and #46 ran on a Tesla T4 on 2026-11-18 before they merged; the chain bump (#40) retired the thrift advisory (#4, #54); #31 landed with #148's fail-fast. Three lines remain: double-buffered transfers, TLS for the cluster ports (#30, whose private-network half shipped) and a runner for gpu.yml (#38). Signed-off-by: Vyncint Ng <115854244+vyncint@users.noreply.github.com>
gpu.yml, docs/RELEASING.md step 0, the test-cuda-device comment and the README's CUDA line named the 2026-09-06 session as the only device run. 0.3.0's CUDA changes ran on a T4 on 2026-11-18, on the fix branches before they merged. RELEASING now asks a release with no gpu runner to say exactly what ran (ADR-0012) rather than only 'compiled-only'. Signed-off-by: Vyncint Ng <115854244+vyncint@users.noreply.github.com>
[workspace.package] and every internal version in [workspace.dependencies]; the lockfile's nine oxidelake entries follow. Signed-off-by: Vyncint Ng <115854244+vyncint@users.noreply.github.com>
STATUS: the header describes 0.3.0; the matrix records the 2026-11-18 T4 session (on the fix branches before they merged, not the tagged commit) in the CUDA kernel, hash join, aggregation and operator rows, which moved out of the compiled-only and needs-GPU columns; a Known limitations section names #142 (NaN pruning, apache/datafusion#15812) and #148 (Ballista 55's lost-worker rollback) with their workarounds; two decision bullets that later ones superseded (cudarc without the NVRTC probe, spill accounting before #116) are gone. README: the status line describes 0.3.0. Signed-off-by: Vyncint Ng <115854244+vyncint@users.noreply.github.com>
CHANGELOG: [Unreleased] becomes [0.3.0] - 2026-11-18, with an introduction (what the release is, how its CUDA paths were verified, the two known upstream issues) and the link definitions repointed. Its sections are consolidated to one heading each. The fix PRs each added a '### Fixed' heading of their own, so main had seven, and seven bullets had landed under Fixed that their fix branches filed under Changed: two marked Breaking (TelemetrySnapshot.operator_totals, #102) and API or default changes (BufferLayout::padded_len, #90, #46, the CUDA aggregate's read-back, #134, #148). They are under Changed now, as the preamble promises for every breaking change. No line of text was added, lost or duplicated by the move (checked). Signed-off-by: Vyncint Ng <115854244+vyncint@users.noreply.github.com>
…e hub The witnesses in tests/cluster.rs count the Gpu*Exec operators the process-wide telemetry hub registered from an id on, and the file's tests run in parallel threads of one process. On the release PR's macOS runner, the_static_planner_runs_lowered_nodes_correctly counted two GpuHashJoinExec of 2019 rows each where its COUNT(*) join ran one: distributed_results_equal_embedded_results was running the same t ⋈ r join at the same moment. The rows were right; the witness was not. Every test that runs queries on a cluster now holds one tokio mutex for its whole run (tokio's, because the guard lives across awaits). Three local runs of --test cluster: 7 passed each, about 8 s, up from 2 s. Signed-off-by: Vyncint Ng <115854244+vyncint@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Release 0.3.0, per
docs/RELEASING.md. Main has taken every fix (#135, #137–#141, #143–#147, #149–#154); this PR changes no code.Release mechanics
[workspace.package], every internal version, and the lockfile's nine entries.[Unreleased]becomes[0.3.0] - 2026-11-18, with an introduction and repointed link definitions..github/scripts/extract-changelog.sh 0.3.0prints the section: 810 lines, 51.5 KB.CHANGELOG repair. Every fix PR added a
### Fixedheading of its own, so main had seven. Seven bullets also sat under Fixed although their fix branches filed them under Changed: two marked Breaking (TelemetrySnapshot.operator_totals, #102), plus API and default changes (BufferLayout::padded_len, #90, #46, the CUDA aggregate's read-back, #134, #148). Each kind now has one heading, and those bullets are under Changed, as the CHANGELOG's preamble promises for every breaking change. A script checked that no line of text was added, lost or duplicated.Docs truth pass
STATUS.md:NaNis present apache/datafusion#15812) and A worker lost mid-query can make a cluster return duplicated rows with exit 0, or hang (upstream Ballista 55 rollback) #148 (Ballista 55's lost-worker rollback), each with its workaround.docs/roadmap.md: Phase 8 ticks the seven lines 0.3.0 closed (Match Arrow totalOrder float comparison in the CUDA filter kernel and add NaN and -0.0 rows to conformance #28, Stream and partition GpuAggregateExec instead of collecting the whole input (audit P4) #29, Persist the CUDA join table across probe batches and size group tables by distinct keys #45, Detect Int64 SUM overflow on the CUDA aggregate path instead of wrapping #46, Test worker loss, expose Ballista retry settings and handle SIGTERM in scheduler and worker #31, Prepare the Ballista 55 / DataFusion 55 / arrow 59 chain bump ahead of the release that fixes thrift #40, thrift CVE-2026-43868 via parquet 58 — fix is parquet 59, blocked on Ballista > 54 #4/Retire the thrift advisory acceptance from deny.toml, SECURITY.md and STATUS.md once parquet >= 59 is in the lock #54). Three remain: double-buffered transfers, TLS for the cluster ports (Production cluster mode needs TLS/auth or an enforced private-network posture #30) and agpurunner (Add an on-demand CUDA execution job and make it a release prerequisite #38).gpu.yml,docs/RELEASING.mdand thetest-cuda-devicecomment no longer call 2026-09-06 the only device run. RELEASING now asks a release with nogpurunner to say exactly what ran (ADR-0012), not just "compiled-only".Before tagging
The stress workflow passed on main 726fe6d, on both Ubuntu and macOS. It runs
cargo test --release -p oxidelake-tui, and neither that crate nor its one internal dependency,oxidelake-core, has changed since.cargo-semver-checks(0.50, forced to patch so every break is listed) on this tree against the published 0.2.0 found four breaks in three crates:GpuOperator::hash_jointakes a fifth parameter (oxidelake-compute, A skewed join aborts the process: GpuHashJoinExec materialises every match of a probe batch into one output batch #95);TelemetrySnapshothas a new public field,operator_totals(oxidelake-core, /metrics repeats identical series once per exec instance and its counters go backwards when operators are evicted #102);HashJoinArgslost itsleft_key/right_keyfields and gainedtable/matches/first/rows(oxidelake-device, Persist the CUDA join table across probe batches and size group tables by distinct keys #45/The CUDA hash join is quadratic in a build key's duplicate count #108).All four are under Changed, with migrations where a caller needs one. The other six crates need no update. A 0.x minor bump allows them, so release.yml's own semver job (which infers minor) passes. The DataFusion 55 coupling, which semver-checks cannot see, is declared in Changed as well.
Locally:
crate-metadata,ci-scripts(47 tests),release-scriptsand zizmor pass. CI runs the full gate on this PR.One test fix. On this PR's first CI run, the macOS job failed
the_static_planner_runs_lowered_nodes_correctly. Its telemetry witness counted twoGpuHashJoinExecof 2019 rows each where itsCOUNT(*)join ran one, becausedistributed_results_equal_embedded_resultswas running the same join in a parallel thread of the same process. The rows were right, so this was a race in the test, not in the product. Every test intests/cluster.rsthat runs cluster queries now holds one tokio mutex for its whole run. A flake there would also stallrelease.yml's CI gate before publishing.CUDA, per RELEASING step 0. No
gpurunner is registered (#38), sogpu.ymlcannot run, and the release notes say exactly what did run instead: a Tesla T4 (driver 595.71.05, CUDA 13.2) on 2026-11-18, three passes, on the fix branches before they merged. All of them passed:--include-ignoredunderOXIDE_BACKEND=cuda, including conformance,join_timingandaggregate_timing;special_floatsandvector_null_elementson the device.