ci: fail release if any expected artifacts are missing#14
Conversation
Port the artifact-completeness gate from lemonade-sdk/llama.cpp#16 to this repo's release job, adapted to its build matrix: - cuda: 8 sm_* variants on ubuntu x64 + windows x64, plus sm_121-only on ubuntu arm64 (GB10) — 17 files total. - rocm-therock: TheRock nightly ROCm 7.13.0 on windows + ubuntu. - rocm-hipsdk: official/legacy HIP SDK ROCm on windows (7.1.1) + ubuntu (7.2.1) — kept separate from rocm-therock since they're different SDKs, not just different platforms of the same backend. - cpu: ubuntu + windows avx2 builds. - metal: macos-14 + macos-latest builds. Each family is all-or-nothing: if any artifact in a family is missing, every artifact for that family is dropped from the release so a release never contains a partial/inconsistent set (e.g. CUDA split across sm_* or platforms). Other families are unaffected by a gap in one. A release with zero artifacts overall is never created. Also brings the release/upload mechanism up to parity with llama.cpp's: an idempotent create-or-get-release step (safe to re-run after a partial failure) and a retrying, resumable asset upload, replacing the old anzz1/action-create-release + github-script@v3 flow. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Without always(), GitHub Actions' default skip-if-a-needed-job-failed behavior meant a single failed build job (e.g. one CUDA sm_* matrix leg, several of which use fail-fast: false) could skip the release job outright, before the per-family completeness check ever ran — silently producing no release instead of a correct partial one. The other issue found in the equivalent llama.cpp fix (a hard-fail step that could block unrelated families) doesn't apply here: this workflow has no cross-job merge step comparable to llama.cpp's Windows CPU-into- other-zips step, so there's nothing else that needs the same treatment. Follow-up to #14. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
|
Pushed 03e249d: applied one of the two fixes from the follow-up review on lemonade-sdk/llama.cpp#16.
|
stable-diffusion.cpp's Windows CUDA build fails with: fatal error C1128: number of sections exceeded object file format limit: compile with /bigobj src/stable-diffusion.cpp combined with the CUDA backend's generated code exceeds MSVC's default COFF section limit per object file. The windows-latest-cmake job already works around the same class of issue with -DCMAKE_CXX_FLAGS='/bigobj'; apply the same flag to the CUDA build.
fl0rianr
left a comment
There was a problem hiding this comment.
This is moving in the right direction and I like the strict all-or-nothing checks per backend family, especially for CUDA.
I would still not merge this yet because the release job still has normal needs: on all build jobs and no always() as stated on the other PR. So again suggesting:
if: ${{ always() && (github.event_name == 'schedule' || github.event.inputs.create_release == 'true') }}
Then the artifact check can decide which complete families to publish.
One more subtle edge case: because the release is now reused and existing uploaded assets are skipped, a previous partial release could keep stale/partial assets from a family that is dropped in the current run. If a family is dropped locally, matching existing release assets for that family should probably be deleted too, or at least this limitation should be documented.
The release is reused across runs against the same tag (idempotent create-or-get). If a family (e.g. rocm-therock) was complete and published in an earlier run but the current run finds it incomplete and drops it locally, its assets from that earlier run were never removed — the release would keep looking complete for a family this run couldn't actually reproduce. The upload step now reconciles existing release assets against the current ./artifact file set: anything not fully uploaded (a broken partial upload, as before) or not present in this run's file set (stale leftovers from a dropped family) gets deleted before uploading. Addresses review feedback on #14. The always() fix requested in the same review was already applied in 03e249d. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
|
Both addressed:
|
Same fix as lemonade-sdk/stable-diffusion.cpp#14: the release is reused across runs against the same tag (idempotent create-or-get). If a family (e.g. rocm) was complete and published in an earlier run but the current run finds it incomplete and drops it locally, its assets from that earlier run were never removed — the release would keep looking complete for a family this run couldn't actually reproduce. The upload step now reconciles existing release assets against the current ./release file set: anything not fully uploaded (a broken partial upload, as before) or not present in this run's file set (stale leftovers from a dropped family) gets deleted before uploading. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
fl0rianr
left a comment
There was a problem hiding this comment.
This now matches the intended release behavior for me.
The all-or-nothing family logic is exactly what I like. Thanks!
ci(windows-cuda): pass /bigobj to fix C1128 build failure
Port the artifact-completeness gate from lemonade-sdk/llama.cpp#16 to this repo's release job, adapted to its build matrix: - cuda: 8 sm_* variants on ubuntu x64 + windows x64, plus sm_121-only on ubuntu arm64 (GB10) — 17 files total. - rocm-therock: TheRock nightly ROCm 7.13.0 on windows + ubuntu. - rocm-hipsdk: official/legacy HIP SDK ROCm on windows (7.1.1) + ubuntu (7.2.1) — kept separate from rocm-therock since they're different SDKs, not just different platforms of the same backend. - cpu: ubuntu + windows avx2 builds. - metal: macos-14 + macos-latest builds. Each family is all-or-nothing: if any artifact in a family is missing, every artifact for that family is dropped from the release so a release never contains a partial/inconsistent set (e.g. CUDA split across sm_* or platforms). Other families are unaffected by a gap in one. A release with zero artifacts overall is never created. Also brings the release/upload mechanism up to parity with llama.cpp's: an idempotent create-or-get-release step (safe to re-run after a partial failure) and a retrying, resumable asset upload, replacing the old anzz1/action-create-release + github-script@v3 flow. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Without always(), GitHub Actions' default skip-if-a-needed-job-failed behavior meant a single failed build job (e.g. one CUDA sm_* matrix leg, several of which use fail-fast: false) could skip the release job outright, before the per-family completeness check ever ran — silently producing no release instead of a correct partial one. The other issue found in the equivalent llama.cpp fix (a hard-fail step that could block unrelated families) doesn't apply here: this workflow has no cross-job merge step comparable to llama.cpp's Windows CPU-into- other-zips step, so there's nothing else that needs the same treatment. Follow-up to #14. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
The release is reused across runs against the same tag (idempotent create-or-get). If a family (e.g. rocm-therock) was complete and published in an earlier run but the current run finds it incomplete and drops it locally, its assets from that earlier run were never removed — the release would keep looking complete for a family this run couldn't actually reproduce. The upload step now reconciles existing release assets against the current ./artifact file set: anything not fully uploaded (a broken partial upload, as before) or not present in this run's file set (stale leftovers from a dropped family) gets deleted before uploading. Addresses review feedback on #14. The always() fix requested in the same review was already applied in 03e249d. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…e-artifacts' into fix/validate-release-artifacts
Summary
Ports the artifact-completeness gate from lemonade-sdk/llama.cpp#16 to this repo's release job, adapted to its build matrix. Previously the release job had no check at all — it created a release and uploaded whatever showed up in
./artifact, so a build failure anywhere could silently ship a release missing entire backends.Changes
Each backend family is checked as an all-or-nothing unit; if any artifact in a family is missing, every artifact for that family is dropped from the release (other families are unaffected), and the job still fails so the gap is visible:
sm_*variants on ubuntu x64 + windows x64, plussm_121-only on ubuntu arm64 (GB10) — 17 files total.rocm-therocksince they're different SDKs, not just different platforms of the same backend.A release with zero artifacts overall is never created.
Also brings the release/upload mechanism up to parity with llama.cpp's: an idempotent create-or-get-release step (safe to re-run after a partial failure) and a retrying, resumable asset upload, replacing the old
anzz1/action-create-release+github-script@v3flow.Test plan