Skip to content

fix: bound large schematic NBT imports - #129

Merged
madawei2699 merged 1 commit into
mainfrom
codex/issue128-bounded-large-nbt
Sep 30, 2026
Merged

madawei2699 merged 1 commit into
mainfrom
codex/issue128-bounded-large-nbt

Conversation

@madawei2699

Copy link
Copy Markdown
Contributor

Summary

  • Reproduce the Probe B failure: the default prismarine-nbt@2.8.0 / protodef@1.19.0 parser rejects BlockData length 16,777,216 at the 0xffffff guard.
  • Add bounded NBT preflight, compressed/uncompressed input caps, a 16,777,216-cell volume bound, 24 MiB BlockData bound, 1,000,000-item cap for other NBT collections, and a 1,000,000 occupied-block cap for the large-array path.
  • Only after preflight passes, use the supported parseUncompressed(data, "big", { noArraySizeCheck: true }) option for the oversized top-level BlockData case.
  • Add the exact MinePilot Probe B fixture, regression/safety-bound tests, and import benchmark script.

Probe B evidence

  • Dimensions: 512×64×512; bounding volume: 16,777,216 cells.
  • Occupied blocks: 997,632; BlockData: exactly 16,777,216 bytes; compressed .schem: 107,799 bytes.
  • Import time: 540.9 ms; peak RSS after import: 902,381,568 bytes (860.6 MiB).
  • Round-trip verification: dimensions and occupied count match; every imported occupied coordinate and material matches the deterministic source geometry.
  • Exact-position verification took 17.0 ms and did not raise peak RSS beyond the importer high-water mark.
  • This is PASS_HEAVY: import succeeds, but peak memory remains substantial. MinePilot's measured export remains about 5.81 s / 925 MiB; exporter was not changed.

Safety boundary and limitation

Large imports are bounded to 16 MiB compressed input, 32 MiB uncompressed NBT, 16,777,216 cells, 24 MiB BlockData, 1,000,000 total items across other NBT collections, 100,000 tags, depth 64, and 1,000,000 occupied blocks. Larger volume, payload, or occupied output remains rejected. The parser option is not applied to arbitrary NBT: a structural preflight checks declared lengths before parsing.

Verification

  • pnpm build
  • pnpm --filter @i365dev/craftdag-importer-schem test — 12 passed
  • pnpm --filter @i365dev/craftdag-core test — 191 passed
  • pnpm --filter @i365dev/craftdag-exporter-schem test — 7 passed
  • pnpm lint
  • node scripts/bench-probe-b-512.mjs

Closes #128.

Copy link
Copy Markdown
Contributor Author

Release follow-up required

The implementation/CI evidence is sufficient for the measured Probe B blocker, but this PR intentionally leaves @i365dev/craftdag-importer-schem at 0.2.5 (and importerVersion at 0.2.5).

After this PR merges, a release PR must bump the importer to a new patch version (expected 0.2.6, unless the repository release baseline has advanced), update runtime/version metadata consistently, pack/test it, and publish it.

The current publish workflow skips versions already present on npm, so triggering publish while the package remains 0.2.5 would not deliver this fix.

Only after the new importer version is published should MinePilot update its pinned dependency and rerun the missing Probe B 512 stages.

@madawei2699
madawei2699 merged commit cddfb2c into main Sep 30, 2026
1 check passed
@madawei2699
madawei2699 deleted the codex/issue128-bounded-large-nbt branch September 30, 2026 09:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Large schematic import: support BlockData > 0xffffff with bounded NBT parsing

2 participants