Repository navigation
Conversation
69884b5 to
74c41cb
Compare
74c41cb to
1a60761
Compare
Upstream renamed the repository to NVIDIA/hyperflare-inference, split the CLI into flare-cli/ over a flare/ library, and renamed the runtime package to the runtime core throughout (helios::package to helios::core, runtime_package.py to runtime_core.py, package_storage to core_storage), alongside Kimi K3.1 preparation work and a FlashInfer pin move. Keeping the integration branch on main is what lets the Spark checkouts keep testing the open PRs against current code. Nine files conflicted. docs/roce-ring.md, helios/src/execution/roce.rs: keep the ABI 11 runtime (roce-abi-11) in upstream's core wording. helios/src/core/ conversion.rs and heliopause/src/export/conversion.rs: keep the convert progress reporting (NVIDIA#223) on the renamed module and crate paths; the primary name filter already matched upstream's new primary_payload helper. ignition/preparation/runtime_core.py: keep the pre-export low-floor listing on the renamed RuntimeCore. The MiMo and DeepSeek V4 recipes, bootstraps, GLM's reseal report, and tests/recipes/test_deepseek4_hosts.py take upstream's core_storage parameter name through our multi-host code paths; tests/preparation/test_low_window.py imports from runtime_core and CoreStorage. Validation on this box: flare-cli type-checks; Helios core::conversion tests pass (50); heliopause export tests pass (172); the Python suites for the touched recipes, low window, toolchain and bmm pass apart from the Qwen, Kimi and Muse metadata cases that fail identically on unmodified main. Not run on GB10; the Sparks need uv sync and a flare-cli rebuild after pulling. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Design-Signed-Off-By: Claude Fable 5.1 (Claude Code); design-review=cfcf Signed-off-by: Kyonjova <kyonjova@pm.me>
The convert progress rows still said Verify package after the core rename; the phase and its documentation now say Verify core, matching the rebased NVIDIA#223. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Design-Signed-Off-By: Claude Fable 5.1 (Claude Code); design-review=cfcf Signed-off-by: Kyonjova <kyonjova@pm.me>
1a60761 to
6cb5d88
Compare
original-el8
left a comment
There was a problem hiding this comment.
This addresses a concrete usability problem and merges cleanly into main at 91b2e85. The existing conversion validation, staging, and publication paths remain in place. Two reporting corrections are needed: the successful verification phase claims signature authentication that conversion does not perform, and the write counters mix logical payload bytes with physical container bytes.
Validation at 6cb5d88: all 50 core::conversion tests passed with cargo test --locked --lib --features runtime-core,server core::conversion::. An additional local fixture test flipped one byte of the core signature, confirmed that protection::verify rejected it, then successfully packed and unpacked it while both operations printed [ OK ] Verify core. signatures. That same test reproduced the byte-count discrepancies noted inline. This is a misleading new status message, not a newly introduced authentication bypass: keyless conversion already intentionally preserves signatures without authenticating them.
The added round-trip test checks output bytes but supplies StartupProgress::silent() to both paths; please also cover the observable status text and progress accounting. I did not repeat the reported 178 GB GPU/package-serving validation, and the tests above ran on the PR head rather than the merged tree.
| } | ||
| let payloads = verified.payloads()?; | ||
| payloads.verify()?; | ||
| phase.finish("signatures"); |
There was a problem hiding this comment.
[P2] Describe the checks actually performed, rather than reporting verified signatures
Verified::open validates metadata, body commitments, and payload closure; it does not authenticate signatures. In a local fixture I flipped one byte of core_signature and confirmed protection::verify rejected the envelope, but pack_with and unpack_with both succeeded and printed [ OK ] Verify core. signatures. This new success message can therefore tell an operator that an invalid signature passed verification. Use an accurate completion detail such as metadata and commitments here and at the corresponding unpack site, and correct the PR description. Preserve the existing keyless conversion contract; this does not require adding publisher trust or signature authentication to conversion.
| let mut writer = Writer::new( | ||
| Reporting::new(file, |bytes| { | ||
| written.set(written.get() + bytes); | ||
| phase.advance_bytes(bytes); |
There was a problem hiding this comment.
[P3] Use the same byte accounting for totals and progress increments
The expected total sums tensor payload lengths, but Reporting counts every underlying write, including the safetensors header and alignment padding. With the existing fixture, the logical total was 1,218 bytes while the packed writer advanced by 24,605 bytes, so completed can exceed total and the bar can reach 100% prematurely. Unpack also mixes conventions: companions advance only slice.len(), while the primary writer includes container overhead; its final report said 16,627 bytes written for files totaling 24,819 bytes. Choose one consistent measure for total, increments, and the final detail in both operations, and exercise the actual reported counters in a test.
… core `ignition convert` printed nothing until it finished. Converting a two-host core of about 178 GB to packed form hashes every payload and copies every byte, so the command stayed silent for minutes with no sign of whether it was verifying, writing, or stuck, while `ignition export` reports each of its native steps as a phase row with elapsed time and byte counts. Conversion also read the stored payloads twice before writing: opening the verified core hashed every payload, and pack and unpack then hashed the same payloads again. Reporting verification as its own phase requires the hashing to happen in one place, and the second pass checked nothing the first had not. Native pack and unpack gain `pack_with` and `unpack_with`, which take a `StartupProgress` and report four phases. `Verify core` validates the metadata structure, the commitment digests, and the presence of every selected payload, and finishes with the detail `metadata and commitments`. It authenticates no signature: conversion needs no key, and the loader authenticates signatures before serving. `Verify stored data` hashes each located payload once against its signed identity and counts payload bytes. `Write packed core` and `Write core companions` count every byte written to the destination. A forwarding writer reports each write of the primary container and of each companion, and the expected total comes from the layout planner that `Writer::finish` itself uses, so the total includes the header length field, the padded header, and the alignment padding as well as the payloads. The last phase publishes the destination. The convert binding passes the standard error row reporter that export uses. `pack`, `unpack`, and their selected variants keep their signatures and report silently; the cache, combine, and promote paths are unchanged. In unit-test builds a progress observation also records the final snapshot of each completed phase; other builds retain the latest snapshot only. `docs/cli-design.md` states what each phase checks and counts. The core format, the metadata, and the bytes the container writer emits are unchanged. A terminal shows a redrawing bar for each phase and a redirected log gets a progress line every ten seconds. A finished write phase reports a total, a completed count, and a detail that all equal the size of the files it produced. Verification reads the stored payloads once instead of twice: on a DGX Spark a core of 180.5 GB verified in 47 s, where a core of 178.1 GB took 45 s and 47 s in the two passes of an earlier revision. On an x86 host without a GPU, `cargo test --locked --lib --features runtime-core,server core::conversion::` passes its 52 tests. One packs and unpacks a signed fixture under an observed reporter and asserts each phase's label and detail, that the stored-data total and completed count equal the located payload bytes, and that each write phase's total, completed count, and detail equal the size on disk of the packed file or of the unpacked directory. A second repeats the write assertions for tensors whose lengths force alignment padding. A third confirms that the reporting forms write the same bytes as the silent forms. A container written from five tensors with misaligned lengths, and four companion headers, hashed identically before and after the writer moved to the shared planner. On a DGX Spark (GB10), a build that contains this change converted a two-rank GLM 5.3 Flash export to packed form. `Verify core` finished in under 0.1 s with the detail `metadata and commitments`, `Verify stored data` hashed 180,523,902,823 source bytes in 47.2 s, `Write packed core` wrote 180,528,104,976 bytes in 146.7 s, and publication took 6.9 s. The command's summary line reported the same 180,528,104,976 bytes. The earlier two-pass timing comes from a different core, a two-rank MiMo V2.6 Flash RL export of 178,122,320,431 source bytes, so the comparison is between cores of similar size and not a repeat of one conversion. Unpack to directory form at that size, and conversion of a core whose signature is invalid, have not been run on hardware. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com> Design-Signed-Off-By: Claude Fable 5.2 (Claude Code); design-review=ed22 Signed-off-by: Kyonjova <kyonjova@pm.me>
6cb5d88 to
81f949a
Compare
|
Both corrections are in 81f949a, rebased onto main at 5fd4070. [P2] Verification detail. Pack and unpack now finish [P3] Byte accounting. Both write phases now count physical bytes throughout. Companions report every write, as the primary already did, and the expected total comes from the layout planner that Tests. A test packs and unpacks the signed fixture under an observed reporter and asserts each phase's label, detail, total, and completed count against the located payload bytes and the sizes on disk. A second covers tensors that force alignment padding. Unit-test builds of One further change to review: conversion hashed the stored payloads twice, once while opening the verified core and once before writing, which is why the earlier Spark run showed 45 s under On a DGX Spark, converting a two-rank GLM 5.3 Flash export at this revision took under 0.1 s in Not run: unpack to directory form at that size, and a conversion of a core with an invalid signature. |
original-el8
left a comment
There was a problem hiding this comment.
Reviewed 81f949a against main at 5fd4070. Both findings from my previous review are resolved, and I found no remaining blocker. The branch merges cleanly.
Pack and unpack now report metadata and commitments without claiming signature authentication. Write totals, increments, and final details consistently include container headers and alignment padding. Removing the duplicate hash pass preserves verification of the same stored payload set before staging or writing begins.
Validation on the PR head:
cargo test --locked --manifest-path helios/Cargo.toml --lib --features runtime-core,server core::: 424 passed, 8 ignored.- The same configuration with
startup::: 5 passed. - Two additional local regression tests passed. One corrupts the core signature, confirms that
protection::verifyrejects it, and packs/unpacks while asserting the accurate verification detail and physical write counts: 24,605 packed bytes and 24,819 unpacked bytes. The other compares the shared writer byte-for-byte with main's writer across multiple tensor counts and alignment boundaries, including companion headers.
I did not repeat the reported 180 GB conversion or package-serving validation. The additional regression tests were local review checks and are not committed to this branch.
Approving; this supersedes my earlier request for changes.
ignition convertprinted nothing until it finished. Converting a two-host core of about 178 GB to packed form hashes every payload and copies every byte, so the command stayed silent for minutes with no sign of whether it was verifying, writing, or stuck, whileignition exportreports each of its native steps as a phase row with elapsed time and byte counts. Conversion also read the stored payloads twice before writing: opening the verified core hashed every payload, and pack and unpack then hashed the same payloads again. Reporting verification as its own phase requires the hashing to happen in one place, and the second pass checked nothing the first had not.What changed
Native pack and unpack gain
pack_withandunpack_with, which take aStartupProgressand report four phases:Verify corevalidates the metadata structure, the commitment digests, and the presence of every selected payload, and finishes with the detailmetadata and commitments. It authenticates no signature: conversion needs no key, and the loader authenticates signatures before serving.Verify stored datahashes each located payload once against its signed identity and counts payload bytes. Before this change the payloads were hashed twice.Write packed coreorWrite core companionscounts every byte written to the destination. A forwarding writer reports each write of the primary container and of each companion. The expected total comes from the layout planner thatWriter::finishitself uses, so it includes the header length field, the padded header, and the alignment padding as well as the payloads.The convert binding passes the standard error row reporter that export uses.
pack,unpack, and their selected variants keep their signatures and report silently; the cache, combine, and promote paths are unchanged. In unit-test builds a progress observation also records the final snapshot of each completed phase, which the tests read; other builds retain the latest snapshot only.docs/cli-design.mdstates what each phase checks and counts. The core format, the metadata, and the bytes the container writer emits are unchanged.Result
A terminal shows a redrawing bar for each phase and a redirected log gets a progress line every ten seconds. A finished write phase reports a total, a completed count, and a detail that all equal the size of the files it produced. Verification reads the stored payloads once instead of twice: on a DGX Spark a core of 180.5 GB verified in 47 s, where a core of 178.1 GB took 45 s and 47 s in the two passes of an earlier revision.
Evidence
x86 host without a GPU:
cargo test --locked --lib --features runtime-core,server core::conversion::passes its 52 tests on this branch rebased onto main at 5fd4070.A container written from five tensors with misaligned lengths, and four companion headers, hashed identically before and after the writer moved to the shared planner.
DGX Spark (GB10): a build that contains this change converted a two-rank GLM 5.3 Flash export to packed form and printed:
The earlier two-pass timing comes from a different core, a two-rank MiMo V2.6 Flash RL export of 178,122,320,431 source bytes converted by a revision of this change that still hashed twice (45 s and 47 s). The comparison is between cores of similar size, not a repeat of one conversion.
Not run on hardware: unpack to directory form at that size, and conversion of a core whose signature is invalid.
🤖 Generated with Claude Code