Generate feature-accurate CycloneDX SBOMs from cargo build logs (gen-sbom-log / gen-sbom-log-dir) #7

Problem

cargo cyclonedx builds its component list from the Cargo dependency graph for whatever features it was invoked with, not what was actually compiled. Optional, feature-gated dependencies show up in the SBOM even when they were never compiled, and dependencies that were compiled can be missing.

Concrete regression found in dnp3 1.7.0: the Windows, macOS and x86_64/i686/aarch64-gnu FFI libraries are built with --no-default-features --features serial,tls-aws-lc, but packaging runs cargo cyclonedx -f json --target $t with no feature flags. The SBOM therefore lists ring (not compiled) and omits aws-lc-rs / aws-lc-sys (compiled and linked). Verified against a real --message-format json build log for x86_64-unknown-linux-gnu. aws-lc-* are also absent from dnp3's allowed.json, so they were never reviewed, and the shipped third-party-licenses.txt is wrong for those targets.

Principle: BOM scope always comes from the actual build log. 0.2.0 did this for the human-readable report (gen-licenses-log*). We have no machine-readable equivalent, so we cannot publish a correct CycloneDX SBOM next to our releases, which downstream OEMs increasingly ask for (e.g. EU CRA). An SBOM that over- or under-lists is worse than none.

Goal

1. Add gen-sbom-log / gen-sbom-log-dir that emit a CycloneDX JSON SBOM from the same validated inventory that gen-licenses-log* uses, so the SBOM and third-party-licenses.txt always agree.
2. Classify every compiled crate as runtime (linked/shipped) or build-time automatically from the dependency graph, and check allowed.json against that classification instead of using build_only as a hand-maintained filter (supersedes explore if we can remove build deps #6).

Use the existing cyclonedx-bom dependency (0.8, which serializes up to spec 1.5) for the data model and serialization.

Inventory model (the core of this work)

Membership: the build log

• The set of crates in scope is exactly the compiler-artifact messages in the build log(s). cargo metadata is used only to resolve and classify them, never to add crates.
• Include fresh: true artifacts (cached units still ship).
• Fail closed:
  • every compiled registry crate must be present in the metadata;
  • each log must end with build-finished / success: true;
  • each log must contain an artifact for --root-package;
  • reject any artifact built with profile.test == true (the log must come from a release build of the product, not cargo test / --all-targets).
• First-party crates (path+ ids) are matched to workspace_members by name, because the absolute path differs across build environments (e.g. cross, see 0.2.1). Parse the id per Cargo's pkgid spec: the name is omitted when it equals the directory name (path+file:///…/dnp3/dnp3#1.7.0). A path+ crate that is not a workspace member is an error.
• Check source identity: a crate whose config entry says crates.io must come from the crates.io registry; git / alternate-registry sources are an error unless explicitly approved.

Role: the metadata dependency graph

Starting from --root-package, walk metadata.resolve, only through crates that compiled:

• runtime (scope: required): reachable via normal (kind: null) edges, without passing through a proc-macro crate.
• build-time (scope: excluded): reachable only via build-dependency edges or through a proc-macro (includes the proc-macro's own dependencies, e.g. syn, quote; these are not inherently build-only and may be runtime elsewhere).
• Dev edges are never followed.
• Compiled but unreachable from the root → error (the build was wider than the product, e.g. missing -p).
• A crate that is runtime anywhere is runtime (runtime dominates).
• Optional-edge activation: cargo metadata unifies features across host/target, unlike resolver 2. An edge for an optional dependency only counts if the parent's compiled features (from its compiler-artifact message) enable it. This removes spurious runtime classifications and also gives correct SBOM dependency edges.
• Per target, then merge: -dir mode reads one log and one metadata file per target subdirectory, classifies each target independently, then unions the results. Metadata must be generated with the same --filter-platform <target> and feature flags as the build. (A single shared -m metadata file remains accepted for compatibility.)

Embedded build-time crates

Some build-dependencies copy their own source into the product, so the graph calls them build-time but their code ships:

• sfio-tracing-ffi / sfio-tokio-ffi: dnp3-ffi/build.rs writes their include_str!'d implementation files into the library.
• oo-bindgen: include_str!s copy/*.rs into the JNI library and Helpers.cs, BindingLibraryLoader.java and C++ snippets into the generated bindings.

A config entry may set "embedded": true, which forces that crate (only, not its dependency closure) to runtime.

allowed.json: approval, not classification

allowed.json exists so that every crate entering the build is explicitly approved and its usage is understood. The graph classifies; the config is asserted against it.

Same three keys (format stays compatible; internally normalize into a model that tracks ownership, license and usage separately so the format can evolve later):

key	meaning
third_party	open-source licensed, license reviewed (regardless of author, so Step Function's MIT/Apache crates such as rx509, scursor, sfio-rustls-config, oo-bindgen belong here)
vendor	licensed to the customer by Step Function under the commercial agreement (e.g. sfio-promise, sfio-tokio-ffi, sfio-tracing-ffi); needs a license name and a URL to the license text, not just an SCM URL
build_only	approved to execute at build time; distribution/license review deferred

Rules (log-based commands only):

1. Every compiled non-workspace crate must be in exactly one list.
2. A runtime crate must be in third_party or vendor. A runtime crate in build_only is an error. This catches a build tool becoming linked in.
3. A build-time crate may be in any list (approval for linking covers build use; the config is shared across products and targets).
4. embedded: true forces runtime as above.
5. Workspace members are first-party automatically and need no entry.
6. License drift check: every reviewed license in a third_party entry must appear in the crate's declared license expression for the compiled version; otherwise error. Catches upstream relicensing without pinning approvals to versions.

Outputs

License report (gen-licenses-log*)

Lists exactly the runtime third_party crates. This changes the output of gen-licenses-log*: build-time crates (proc-macros and their dependencies, etc.) drop out, and runtime crates missing from the old output now fail approval. Call this out in the CHANGELOG. gen-licenses / gen-licenses-dir (cargo-cyclonedx input) are unchanged and deprecated.

SBOM (gen-sbom-log, gen-sbom-log-dir)

CLI mirrors the log commands:

• gen-sbom-log: -g/--log-path, -m/--metadata-path, -c/--config-path
• gen-sbom-log-dir: -l/--list-dir, -g/--log-file, --metadata-file (per subdirectory) or -m/--metadata-path (shared), -c/--config-path
• both: --root-package <name> (workspace member; version from metadata), optional --lockfile <Cargo.lock>, output to stdout.

One SBOM per product root (dnp3-ffi and dnp3-ffi-java are separate SBOMs, matching the separate license files).

Content:

• Spec version 1.5.
• metadata.component: root package (library, name, version, purl). metadata.tools: allow-list + version.
• components: every included crate (runtime and build-time, all three lists, plus first-party crates other than the root):
  • type: library, name, version, scope (required / excluded)
  • purl: pkg:cargo/<name>@<version>, with vcs_url qualifier for git sources; used as bom-ref
  • licenses: the crate's declared SPDX expression from metadata (what scanners expect; the reviewed licenses are validated against it by rule 6). vendor: a LicenseRef-… with name and URL to the license text
  • hashes: SHA-256 from the matching Cargo.lock checksum when --lockfile is given, matched by full source identity (this is the .crate archive hash); omitted for path/git
  • vendor: SCM URL as an external reference
• dependencies: from the activated graph edges, filtered to included components.
• Reproducible: components and dependencies sorted; timestamp from SOURCE_DATE_EPOCH when set; serialNumber random by default, with an option to derive a deterministic UUID (v5 over the content) or omit it.

Invariant: the set of scope: required components whose config list is third_party equals the crate set of the license report from the same inputs. Both come from one function.

Tests

Fixture-based (small build log + metadata + config), covering:

• an optional dependency present in metadata but not in the log → absent (the serialport / ring regression)
• the invariant above
• schema validation of the output against CycloneDX 1.5 (cyclonedx-bom validation and/or CycloneDX CLI in CI)
• path+ first-party id from a different environment, including the name-omitted form
• crate that is both build-time and runtime → runtime
• proc-macro and its deps → build-time
• inactive optional edge between two compiled crates (host/target feature split) → not runtime
• embedded crate → runtime, its dependencies stay build-time
• build-time crate becoming runtime while listed in build_only → error
• truncated log / success: false / missing root artifact / test-profile artifact → error
• unreachable compiled crate → error
• alternate source for an approved name → error
• declared license no longer containing the reviewed license → error
• multi-target union where a crate is runtime on one target and build-time on another

Constraints

• Workspace lints (unsafe_code = forbid, warnings = deny, unreachable_pub = deny, …).
• No network access at runtime; all inputs are local files.
• Bump allow-list to 0.3.0, CHANGELOG entry (including the gen-licenses-log* output change and deprecation of the cargo-cyclonedx commands), tag 0.3.0.

Consumer integration (follow-up in consuming repos)

1. dnp3, rodbus, *-no-serial: drop cargo cyclonedx. In each build job, produce build.json with cargo build/cross build … --message-format json and metadata.json with cargo metadata --format-version 1 --filter-platform <target> plus the same feature flags, next to the produced binary. In packaging, run gen-licenses-log-dir and gen-sbom-log-dir per product root, and attach <pkg>-<version>.cdx.json to the GitHub release next to third-party-licenses.txt.
2. Update each allowed.json: move Step Function open-source crates from vendor to third_party, add aws-lc-rs / aws-lc-sys (dnp3), add new build tools (e.g. jobserver, pkg-config), mark sfio-tokio-ffi, sfio-tracing-ffi, oo-bindgen as embedded.

Out of scope

• VEX documents
• SBOM signing or attestation
• SPDX output format
• Inventory of native/system components beyond the crates that bundle them (document as a known limitation)
• Recording build provenance (command, host, revision) and hashing shipped binaries