From caf69c9d3912893ade4a617607314c05321edfad Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Eric=20Meadows-J=C3=B6nsson?= Date: Thu, 1 Oct 2026 21:28:50 +0200 Subject: [PATCH] Document build_env metadata and reproducible tarballs Add the optional build_env metadata field: the names and versions of the tools that created the package tarball. Hex records its version there with hexpm/hex#1252, so the version needed to rebuild a package and compare checksums can be read from the package alone. Add recommendations to the package tarball spec for creating tarballs whose bytes only depend on the files and the tool versions: entry order, timestamps, owners, modes, NFC names, the gzip header and the order of metadata keys. They describe what hex_core does since hexpm/hex_core#228. They aren't requirements, and different tools aren't expected to produce the same bytes as each other. --- package_metadata.md | 7 +++++++ package_tarball.md | 12 ++++++++++++ 2 files changed, 19 insertions(+) diff --git a/package_metadata.md b/package_metadata.md index 78e51a8..baf3e2f 100644 --- a/package_metadata.md +++ b/package_metadata.md @@ -94,6 +94,13 @@ All keys are strings. character. Intended for paths a package deliberately ships secret-like values in, such as test fixtures. + + `build_env (kvlist(string => string)) (optional)` + + Names and versions of the tools that created the package tarball, for + example `hex`, `rebar3_hex` or `gleam`. Creating a tarball from the same + files with the same tool versions gives the same tarball. Clients don't + use it when fetching or building the package. + ### Optional dependencies An optional dependency will only be used if a package higher up the dependency chain also depends on it (only if that the dependency is not defined as optional as well). diff --git a/package_tarball.md b/package_tarball.md index fcec467..0bce9d6 100644 --- a/package_tarball.md +++ b/package_tarball.md @@ -25,3 +25,15 @@ The package tarball contains the following files: This checksum is called the "inner checksum" and is deprecated in favor of the "outer checksum" which is the SHA-256 checksum of the entirety of the tarball file. When fetching packages the checksums should be verified against checksums stored in the registry. See `Release.inner_checksum` and `Release.outer_checksum` in the [protobuf definitions](registry/package.proto). + +## Reproducible tarballs + +The following is recommended for tools that create package tarballs, so that creating a tarball from the same files with the same tool versions gives the same bytes. None of it is required, and different tools aren't expected to produce the same bytes as each other. + + * Write the entries of contents.tar.gz sorted by name, and the outer tarball's files in the order VERSION, CHECKSUM, metadata.config, contents.tar.gz. + * Write file names in Unicode Normalization Form C (NFC). + * Set the modification, access and change times of every entry to 2000-01-01T00:00:00Z, and the user and group IDs to 0. + * Write regular files with mode 0644, or 0755 when the file is executable, directories with mode 0755 and symbolic links with mode 0777. Only write directory entries for empty directories. + * Write the gzip header of contents.tar.gz without a file name and with MTIME and OS set to 0. + * Write the keys of metadata.config, and the file names in its `files` field, in sorted order. + * Record the names and versions of the tools in the `build_env` field of the [package metadata](package_metadata.md).