Skip to content

feat(config): add AllTargetsFiles to invalidate all targets on trigger file change - #286

Merged
yushan8 merged 4 commits into
mainfrom
yushan/all-targets-config-base
Aug 20, 2026
Merged

feat(config): add AllTargetsFiles to invalidate all targets on trigger file change#286
yushan8 merged 4 commits into
mainfrom
yushan/all-targets-config-base

Conversation

@yushan8

@yushan8 yushan8 commented Aug 19, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Adds RepositoryConfig.AllTargetsFiles: a list of repo-relative file paths that, when their content hash differs between two revisions, cause GetChangedTargets to report every target in the second revision as changed (distance 0)
  • File hashes are computed from git ls-tree during graph computation in targethasher.FromProto, stored as sidecar data in Metadata.AllTargetsFileHashes, and compared at diff time in the controller
  • Intended for files that affect the build globally but are invisible to the target graph (toolchain definitions, CI config, Bazel wrapper scripts)
  • Gob-format graphs carry this field automatically via gob serialization

Data flow

git ls-tree → HashConfig.KnownSourceHashes
  → targethasher.FromProto filters by AllTargetsFiles
  → Result.AllTargetsFileHashes
  → mapper → entity.Metadata.AllTargetsFileHashes
  → stored in graph blob (gob auto-serialization)
  → read back at comparison time in controller
  → if any hash differs → all targets reported as changed

Example config

repository:
  - remote: "git@github.com:org/repo.git"
    all_targets_files:
      - .bazelrc
      - .bazelversion
      - tools/bazel
      - rules/go/sdk.bzl

Test plan

  • allTargetsFileChanged unit tests — nil/empty/matching/differing hash scenarios (table-driven)
  • compareTargetGraphs gob-path test — trigger fires → all targets changed
  • compareTargetGraphs gob-path test — matching hashes → normal diff
  • Mapper propagation tests — Result → Metadata, including via ResultToGraphChunks
  • All existing tests pass

Stack: TGB format support in PR #285

yushan8 and others added 3 commits August 17, 2026 21:18
…r file change

Some files affect the build globally but are invisible to the target
graph (toolchain definitions, CI config, Bazel wrapper scripts). Add
RepositoryConfig.AllTargetsFiles: file hashes for listed paths are
computed from git ls-tree during graph computation, stored as sidecar
data in Metadata.AllTargetsFileHashes, and compared at diff time.

When any configured file's hash differs between the two revisions,
GetChangedTargets reports every target in the second revision as changed
(distance 0) instead of running the normal target-graph diff.

This mirrors the Java TargetHasher/FastTargetComparer.resolveCTC pattern:
hashes travel as sidecar data alongside the graph, checked post-fetch.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
SplitMetadata builds fresh Metadata chunks and never carried the sidecar
map, so the AllTargetsFiles trigger silently never fired on graphs
written via the orchestrator's real chunking path (only the direct
ResultToTargetGraph call, used by tests, preserved it).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…rgethasher

The hash extraction belongs in targethasher.FromProto where it has
direct access to HashConfig.KnownSourceHashes, rather than in the
graphrunner which was duplicating the lookup. AllTargetsFiles is now
a field on HashConfig, and FromProto populates Result.AllTargetsFileHashes
after the inner fromProto returns.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@yushan8
yushan8 marked this pull request as ready for review August 19, 2026 18:24
@yushan8
yushan8 requested review from a team as code owners August 19, 2026 18:24
gofmt had left ExcludedRegex/UseBzlmod/AllTargetsFiles misaligned.
Also adds a unit test exercising the AllTargetsFileHashes extraction
in FromProto directly (previously only covered indirectly via
mapper tests).
@yushan8
yushan8 merged commit 2a8097d into main Aug 20, 2026
10 checks passed
yushan8 added a commit that referenced this pull request Aug 20, 2026
## Summary

Adds TGB (Target Graph Binary) format support for
`AllTargetsFileHashes`, stacked on #286.

- New column 22 (`ALL_TARGETS_FILE_HASHES`) — length-prefixed key-value
string pairs, zstd compressed
- Forward/backward compatible: old readers skip unknown column IDs, new
readers on old blobs get nil (no version bump needed)
- `Reader.AllTargetsFileHashes()` lazy accessor decodes only column 22
(cheap, no full graph decode)
- Controller TGB comparison path checks AllTargetsFileHashes before
dispatching to normal TGB diff
- `MergeChunks` propagates AllTargetsFileHashes from metadata chunks
before TGB encoding

### Files changed

- `internal/tgb/format.go` — column ID 22
- `internal/tgb/encode.go` — `encodeStringMap` + column emission
- `internal/tgb/reader.go` — `decodeStringMap` +
`AllTargetsFileHashes()` accessor + `DecodeGraph` integration
- `internal/tgbdiff/adapt.go` — `MergeChunks` propagation fix
- `controller/getchangedtargets.go` — `allTargetsChangedFromTGB()`, TGB
path wiring
- `controller/getchangedtargets_tgb_test.go` — trigger/no-trigger tests
- `internal/tgb/tgb_test.go` — round-trip tests

## Test plan

- [x] TGB round-trip tests — AllTargetsFileHashes survives encode/decode
(present/absent/empty)
- [x] TGB Reader.AllTargetsFileHashes() accessor test — lazy
single-column decode
- [x] TGB controller test — trigger fires → all targets changed, skips
normal TGB diff
- [x] TGB controller test — matching hashes → normal TGB diff proceeds
- [x] All existing TGB tests pass

**Stack:** depends on #286

---------

Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants