A rule may reach the content its repository pins: files.reach = "pinned" (ADR 0012)

CONTRIBUTING.md

@@ -92,7 +92,7 @@ The records under `docs/adr/` stay as history and are cited as they are. No new
 one is added: the `adr-freeze` rule in `policy/principles.toml` refuses any
 other file under that directory, and its message names the remedy.
 [ADR 0012](docs/adr/0012-a-rule-may-reach-the-content-its-repository-pins.md)
-stands at Proposed until the owner rules it.
+is Accepted and implemented as `files.reach`.
 
 ## Working on the engine
 

docs/REFERENCE.md

@@ -60,7 +60,7 @@ does not run.**
 
 | keys | vocabulary | runs it |
 |---|---|---|
-| `files.*` | ripgrep scoping — `glob`, `multiline`, `fixed_strings` — and `min_selected`, the floor under what the scoping leaves | `uphold scan` |
+| `files.*` | ripgrep scoping — `glob`, `multiline`, `fixed_strings` — `min_selected`, the floor under what the scoping leaves, and `reach`, `"repository"` (the default) or `"pinned"` for the content the repository's submodules pin | `uphold scan` |
 | `git.hooks` | githooks(5) names — `pre-commit`, `commit-msg`, `pre-merge-commit`, `pre-push`, `manual` | `uphold guard --stage <hook>` |
 | `command.before` | the command line as typed — `"gh pr create"`, `"git push"` | `uphold shim <command>` |
 
@@ -346,6 +346,62 @@ counts links and `require_any_anchor` counts anchors. A `links-resolve` rule may
 hold both; they fail differently, and a glob typo is caught by the one that
 counts files.
 
+### A rule may reach the content its repository pins
+
+`files.reach` says whose tracked files a rule selects from. `"repository"` is
+this repository's own, and is what an absent key means. `"pinned"` adds the
+content of every submodule the index pins, selected under its mount path:
+
+```toml
+[rule.member-readmes-name-an-owner]
+require_regexp = '(?m)^Owner: '
+message = "every member says who owns it"
+files.glob = ["/*/README.md"]
+files.reach = "pinned"
+files.min_selected = 1
+```
+
+The pins are the gitlinks in the index (mode `160000` in `git ls-files -s`),
+and each member is asked for its own `git ls-files` inside it. A member that
+pins members of its own is followed the same way, at every depth. Not
+`git ls-files --recurse-submodules`: that follows git's active-submodule
+filter, and a rule that claims the pinned content must not pass over part of it
+without a word.
+
+- **A pinned mount that cannot be read is exit `2`**: not checked out, or
+  checked out and marked inactive by `submodule.<name>.active` or
+  `submodule.active`. The message names the mount and
+  `git submodule update --init <path>`, which checks it out and marks it active.
+  A CI checkout without `--recurse-submodules` goes red on a pinned rule, on
+  purpose. A repository that pins nothing reads the same as at
+  `"repository"`.
+- **Paths are mount-prefixed everywhere**: findings, path baselines and size
+  baselines key on `member/sub/file`, and `files.include` may name a directory
+  inside a mount. `files.min_selected` counts the prefixed files.
+- **The globs are the superproject's.** `include`, `exclude` and `glob` keep
+  their gitignore meaning, rooted at the superproject: `/vendor.txt` is the
+  superproject's own file, and `vendor.txt` matches at any depth, inside mounts
+  too.
+- **A member's `.gitattributes` is asked inside the member**, so a file it
+  declares `-text` is skipped and listed like the superproject's own. A member
+  whose attributes cannot be asked is exit `2`, as the superproject's is.
+- **A link in a member's Markdown is the member's.** A leading `/` resolves
+  against the member's root, and a link leaving the member is outside the
+  repository, for `links-resolve` and `anchors-resolve` alike.
+- **Nothing the member declares about policy is read**: not its policy file,
+  not its excludes. The rule is the superproject's, and so is the verdict.
+
+The direction is one-way. A member never borrows upward: run in a member with
+no policy of its own, uphold stops at the member's root rather than loading the
+superproject's. A repository may judge, downward, the content it pins, and
+reports it under the mount path.
+
+`files.reach` is refused on a guard built-in's `[rule.files]`, for the reason
+`min_selected` is: there it scopes the bytes a hook is about to record, and no
+path it is handed lies inside a mount. `uphold rules --effective --json` carries
+`"reach": "pinned"` on a rule that declares it, and nothing on one that does
+not.
+
 ### A rule may not be about its own declaration
 
 A policy file is a tracked file, so a rule's `regexp` and `require_regexp` are
@@ -407,6 +463,11 @@ it. In a directory git has no index for, the tree is walked instead with **no**
 ignore file consulted, which selects a superset of what would be tracked.
 Over-reporting is the direction a checker may fail in; hiding a file is not.
 
+**A submodule is its own repository**, so its content is not among a rule's
+files: the gitlink is a pointer, and the scan passes over it. A rule that
+declares `files.reach = "pinned"` claims that content too. See
+[A rule may reach the content its repository pins](#a-rule-may-reach-the-content-its-repository-pins).
+
 A path a rule selected and could not open — an unstaged deletion, a sparse
 checkout, a directory this process may not enter — is **named on stderr and is
 exit `2`**, after every other rule has reported. It is not dropped from the

docs/adr/0012-a-rule-may-reach-the-content-its-repository-pins.md

@@ -1,6 +1,6 @@
 # ADR 0012: a rule may reach the content its repository pins
 
-Status: Proposed
+Status: Accepted
 
 `uphold scan` reads what `git ls-files -z` lists (`index_bytes`,
 `src/selection.rs:150`) and drops every gitlink on purpose

src/config.rs

@@ -239,6 +239,35 @@ pub(crate) struct Files {
     /// shape `trivial_comments = false` is refused for.
     #[serde(default)]
     pub min_selected: Option<u64>,
+    /// Whose tracked files the selection is drawn from: this repository's own
+    /// (`"repository"`, and what an absent key means), or those plus the
+    /// content of every repository its index pins (`"pinned"`).
+    ///
+    /// An `Option` rather than a defaulted value so that a rule written before
+    /// the field existed serializes exactly as it did, into the bundled-set
+    /// lock and everywhere else a rule is printed.
+    #[serde(default)]
+    pub reach: Option<Reach>,
+}
+
+/// How far down a rule's selection reaches.
+///
+/// `Repository` is what every rule did before the field existed: the files
+/// this repository's index lists, a gitlink skipped as another repository's
+/// content. `Pinned` is a claim about the content the repository pins as well:
+/// each gitlink in the index is asked for its own tracked files, which are
+/// selected under the mount path. The pin is what makes the content this
+/// repository's business -- a member never borrows upward, and a repository
+/// may judge, downward, what it pins.
+#[derive(Debug, Clone, Copy, Default, PartialEq, Eq, Deserialize, Serialize)]
+#[serde(rename_all = "lowercase")]
+pub(crate) enum Reach {
+    /// This repository's own tracked files.
+    #[default]
+    Repository,
+    /// This repository's own tracked files and those of every repository it
+    /// pins, at every depth, under their mount paths.
+    Pinned,
 }
 
 /// The five stage names a rule may run at: four of git's own, spelled as

src/config/rule.rs

@@ -17,7 +17,7 @@ use std::sync::OnceLock;
 
 use serde::{Deserialize, Serialize};
 
-use super::{CommandWhere, Files, Git, Origin};
+use super::{CommandWhere, Files, Git, Origin, Reach};
 use crate::error::{Fatal, Result};
 use crate::text::{Judged, Seam};
 
@@ -1322,6 +1322,12 @@ impl Rule {
             .unwrap_or_else(|| DEFAULTS.get_or_init(Files::default))
     }
 
+    /// How far this rule's selection reaches, `Repository` where the rule
+    /// does not say.
+    pub(crate) fn reach(&self) -> Reach {
+        self.files().reach.unwrap_or_default()
+    }
+
     /// Whether this rule searches files at all. Absent `files.*` keys are the
     /// answer, not a default to fill in.
     pub(crate) const fn reads_files(&self) -> bool {
@@ -1537,6 +1543,7 @@ impl Rule {
 
         self.validate_prose(check)?;
         self.validate_selection_floor()?;
+        self.validate_reach()?;
 
         // The label is resolved at load, so a typo is a refusal here and not a
         // rule that fails every file it selects.
@@ -1859,6 +1866,30 @@ impl Rule {
         )
     }
 
+    /// The `files.reach` half of [`Rule::validate`]: a reach needs a selection
+    /// for it to widen.
+    ///
+    /// Refused where `min_selected` is refused and for the same reason: a
+    /// guard built-in's `[rule.files]` scopes the bytes git is about to record
+    /// one path at a time, and nothing it is handed lies inside a mount. A
+    /// `"pinned"` there would read as a claim on the pinned content that no
+    /// seam ever makes.
+    fn validate_reach(&self) -> Result<()> {
+        if self.reach() == Reach::Repository || self.selection_is_counted() {
+            return Ok(());
+        }
+        Err(Fatal::new(format!(
+            "rule {:?}: `files.reach = \"pinned\"` widens the selection `uphold scan` builds \
+             to the repositories this one pins, and built-in {:?} runs at a git hook over the \
+             bytes git is about to record -- its `files.*` keys scope that guard one path at a \
+             time, and no path it is handed lies inside a mount. On this rule the field would \
+             be read by nothing. The built-ins the scan selects for are {}",
+            self.id,
+            self.builtin().unwrap_or_default(),
+            crate::guard::SCAN_BUILTINS.join(", ")
+        )))
+    }
+
     /// The `seams` half of [`Rule::validate`]: the list names seams this
     /// rule's kind can run at, beside the table that makes it published-text
     /// rule at all.

src/git.rs

@@ -60,10 +60,30 @@ pub(crate) fn try_run(root: &Path, args: &[&str]) -> Result<Option<String>> {
 pub(crate) fn try_run_elsewhere(directory: &Path, args: &[&str]) -> Result<Option<String>> {
     let mut command = crate::shim::inner_tool("git");
     command.current_dir(directory);
+    answer(elsewhere(&mut command), args)
+}
+
+/// A git command aimed at a repository other than the hooked one, with the
+/// hooked repository's environment taken away.
+///
+/// The one place that decides what "elsewhere" strips, so a caller that has to
+/// build its own `Command` -- one that speaks to git over stdin, which
+/// [`try_run_elsewhere`] does not -- strips the same list rather than a copy.
+pub(crate) fn elsewhere(command: &mut std::process::Command) -> &mut std::process::Command {
     for name in REPOSITORY_ENVIRONMENT {
         command.env_remove(name);
     }
-    answer(&mut command, args)
+    command
+}
+
+/// Whether a gitlink's mount holds a checkout: a `.git` file or directory is
+/// what `git submodule update --init` leaves there, and an uninitialised mount
+/// is an empty directory, or no directory at all.
+///
+/// Asked by every reader that follows a pin into the member, so "not checked
+/// out" means one thing across them.
+pub(crate) fn is_checked_out(mount: &Path) -> bool {
+    mount.join(".git").symlink_metadata().is_ok()
 }
 
 fn answer(command: &mut std::process::Command, args: &[&str]) -> Result<Option<String>> {

src/main.rs

@@ -637,7 +637,7 @@ fn scan_command(arguments: &[OsString]) -> Result<Exit> {
     };
 
     let policy = config::load(&root, &policy_path)?;
-    let scanner = scan::Scan::new(&root, &policy);
+    let scanner = scan::Scan::new(&root, &policy)?;
     let failures = scanner.run()?;
     for failure in &failures {
         failure.print();
@@ -967,6 +967,12 @@ struct EffectiveRule<'rule> {
     /// whose only place is `command.before` reconciled green in a repository
     /// where nothing runs it. The loader knows; it says so here.
     seams: Vec<&'static str>,
+    /// `"pinned"` where the rule's selection reaches the content this
+    /// repository pins. Only there, for the reason `overridden` gives: the
+    /// entry of every rule that reads only its own repository is what it was
+    /// before the field existed.
+    #[serde(skip_serializing_if = "Option::is_none")]
+    reach: Option<config::Reach>,
     /// Which fields of an inherited rule an override changed. Only where one
     /// did, so the entry of every rule no override touches is what it was
     /// before the field existed.
@@ -1009,11 +1015,16 @@ fn effective_rules_command(as_json: bool) -> Result<Exit> {
             // Which parts of an inherited rule this policy changed, so a
             // reworded message or a widened selection reads as local rather
             // than as the set's.
+            let reach = if rule.reach() == config::Reach::Pinned {
+                "  [reach: pinned]"
+            } else {
+                ""
+            };
             if rule.overridden.is_empty() {
-                println!("  {}  ({at})", rule.id);
+                println!("  {}  ({at}){reach}", rule.id);
             } else {
                 println!(
-                    "  {}  ({at})  [override: {}]",
+                    "  {}  ({at}){reach}  [override: {}]",
                     rule.id,
                     rule.overridden.join(", ")
                 );
@@ -1029,6 +1040,7 @@ fn effective_rules_command(as_json: bool) -> Result<Exit> {
             id: &rule.id,
             git_hooks: rule.hooks(),
             seams: rule.seams(),
+            reach: (rule.reach() == config::Reach::Pinned).then_some(config::Reach::Pinned),
             overridden: &rule.overridden,
         })
         .collect();