diff --git a/.agents/docs/2026-09-27-eight-reports-by-home-and-one-optimisation-plan.md b/.agents/docs/2026-09-27-eight-reports-by-home-and-one-optimisation-plan.md new file mode 100644 index 000000000..50460b55b --- /dev/null +++ b/.agents/docs/2026-09-27-eight-reports-by-home-and-one-optimisation-plan.md @@ -0,0 +1,1417 @@ +--- +subject: triage +status: active +--- + +# Eight reports after 2026.9.27.1: what each one is, where it belongs, and one optimisation plan + +**Status:** active, revision 4 (2026-09-27). Every decision is settled (§12). +Nothing described here has been implemented. + +- **Revision 1** routed the reports and asked seven questions. +- **Revision 2** recorded the reviewer's first answers: + - D1 was settled by measurement. + - D2 and D3 were revised to satisfy both mcpp's specifications and the industry + norm. + - Two items the reviewer raised were added: progress (§9.1) and the index floor + (§9.2). + - The first self-review was recorded. +- **Revision 3** records the remaining answers and a second self-review (§13.4): + - The reviewer pointed out that `-p` is declared as `--package `, so D1 + resolves the package identity first. + - D2, D3, D8 and D9 are accepted. +- **Revision 4** records two changes, both made during implementation: + - The reviewer added #726, with its pull request #727 (W13, §9.3), and directed + that the whole round land in #727 (D10). + - The split (W7) moves to the last stage (§11). + +**Basis.** Engine code was read at `b439fd97` (origin/main, mcpp 2026.9.27.1). The +reports cite `52549fbb`, which predates the decomposition of +`src/build/prepare.cppm`; every line number below is that of `b439fd97`. + +- A statement marked *measured* was run on Linux x86_64 with released binaries of + `xim-x-mcpp`, using the reports' own fixtures where they give one: + - 2026.9.27.1 and 2026.9.26.1 in general; + - 2026.9.18.1 for the index floor (§9.2). +- A statement marked *read* names a file and a line, and was not executed. +- A statement marked *inferred* is labelled as such. + +Nothing was run on Windows or macOS. + +## 0. Scope, and the rules applied + +The numbers 717 to 725 are eight reports and one release pull request (#719, +merged; it is not a report). One report is excluded: #721 is labelled +`upstream-bug` and is a GCC 16.1 defect, and upstream issues are out of scope for +this round. The remaining seven reports contain eleven separable items, because +#724 carries four and #725 carries two. + +Every item is assigned exactly one home, and each assignment states why the cheaper +home does not suffice. The rules applied are the following: + +1. **Home, in order of preference.** The order is usage, then a project-local + plugin, then the official plugin repository, then ecosystem data, then the + engine. The order filters features, not defects: a silent drop, a lying record + or a wrong diagnostic is the engine's to fix wherever it appears. +2. **The engine names no tool** (SPEC-007 §0). When a plugin meets a gap that is + general, the engine closes it with a general mechanism. The plugin does not + work around it, and the engine does not learn Qt. +3. **No slot for workarounds, and no rule that nothing enforces.** An invariant is + enforced by construction, by a property test over every row, or by a negative + test. It is not enforced by a list of exceptions or by a sentence in a document. +4. **A new condition is first tried in the existing condition syntax.** SPEC-004 + §6 requires `[target..
]` before any new syntax is discussed. +5. **Planning does not change the project** (SPEC-005 R2.1, R2.5). + `emit build-database` writes nothing into the project and runs no action. + +None of the seven reports is a usage question as a whole. Two of them contain a +usage reading, and this record answers it: + +- **#720.** A maintainer comment reads the report as a build dependency used by the + package body. The fixture refutes that reading: `app/main.cpp` imports nothing, + and only `build.mcpp` imports `repro.rules` (measured). +- **#717.** No current spelling expresses the report's need (measured, §6). + +Two further items come from the reviewer rather than from a report: download +progress (§9.1) and the index floor (§9.2). The same rules route them. A third, +#726 with its pull request #727, was added by the reviewer on 2026-09-27 as part +of the same round (§9.3), and the whole round lands in #727. + +## 1. The ledger + +| Item | What it is | Kind | Home | Verdict | Order | +|---|---|---|---|---|---| +| #725 | A rooted workspace built as itself carries no workspace context to its path dependencies. Its members are loaded as strangers, and 2026.9.26.1 silently ignored `[workspace.dependencies]` | defect (older than #714; #714 made it loud) | engine | fix | W1 | +| #725 `-p` | `-p, --package ` promises a package, while the resolvers and docs/07 match only a member's directory. mcppls's members are named `mcppls-base` in `modules/base` | defect (measured) | engine | `-p` resolves the package identity first, and keeps the directory spellings as a fallback | W1 | +| #720 | A host-module package's lib root is compiled before the siblings it imports | defect | engine | fix | W2 | +| #724 §1 | A rule-claimed device source is a C++ compile unit of the plan. It appears in S1 and also in `mcpp build`'s own `compile_commands.json`, and `build.ninja` carries a dead edge for it | defect | engine | fix | W3 | +| #724 A | A failed build program is reported as unclaimed device sources, and the true diagnostic is discarded | defect (SPEC-005 R5.2) | engine | fix | W4 | +| #724 B | `emit` writes `/.mcpp/.xlings.json` into the project | defect (SPEC-005 R2.1) | engine | fix | W5 | +| #723 | Two deploys of one identical generated file are refused at planning | defect: the check compares a proxy (a source path), not content | engine | fix differently from all three proposed options | W6 | +| #722 | Split the longest phase functions | internal | engine | do, in the same round (stage 2) | W7 | +| #717 | `dialect_cxxflags` under a target condition | feature: a grammar gap | engine (manifest) | do; decline the build-program directive | W8 | +| #718 | CRT for the LLVM row on the MSVC ABI | gap: an existing key is not consumed by one row, and the rows' defaults disagree | engine | do with the existing keys; `toolchain-coupled` is the default for every MSVC-ABI row; no new keys; defer the debug CRT | W9 | +| #724 §2 | Describe generated outputs, and give the path the build writes | feature | engine and S1 (mcppls repository) | do, as one record | W10 | +| #724 §2.3 | Run side-effect-free generators under `emit` | feature | not the engine | decline | none | +| review | An index that requires a newer mcpp prints `error: ... [E0006]` at the start of a run that then succeeds | defect (measured) | engine | a closing tip at most, and only when the run refreshed an index | W12 | +| review | Library, git and index acquisitions show no progress, while toolchains do | gap | engine; xlings if its `update_packages` emits no events | one renderer, more producers; non-terminal output without `\r` | W11 | +| #726 | On Windows an xlings invocation left the registry's shim directory in front of the process `PATH`, kept `XLINGS_HOME` set for the rest of the run, and started xlings in the project's directory | defect (measured on GalTranslPP's Windows CI) | engine | fix, as pull request #727 proposes (§9.3) | W13 | +| #721 | GCC 16.1 ICE | upstream | not in scope for this round | excluded | none | + +Everything is one round and one release, in one pull request (#727), in three +stages: + +- **Stage 1** holds the defects: W1 to W6, W12 and W13. +- **Stage 2** holds the features: W8 to W11. +- **Stage 3** is #722 (W7). + +§11 gives the order and its reasons, including why the split moved from the +second stage to the last. + +## 2. #725: a member is a member however the build is rooted + +### 2.1 What was measured + +- **2026.9.27.1**, using the report's fixture exactly: `mcpp build` at `root/` + fails with "no workspace declares 'cmdline'" and exits with 2. +- **2026.9.26.1:** the same build resolves `cmdline → v0.0.2` and finishes. +- **2026.9.26.1 was not correct.** With only `[workspace.dependencies] cmdline = + "0.0.1"` changed, 2026.9.26.1 still resolves and locks `cmdline@0.0.2`. The + member's `cmdline.workspace = true` was never merged against the workspace. It + fell through as an unconstrained dependency, which happened to resolve to the + version the report used. + +So #714 did not break a working feature. It added the refusal that SPEC-004 §9 +item 9 requires (`unresolved_workspace_dependency_error`, `src/project.cppm:404`), +and that refusal exposed a gap older than itself. The fix remains first in the +order, because the refusal now blocks builds that used to succeed. Among them is +mcpp-language-server's nightly run against the latest mcpp. + +### 2.2 Cause (read) + +The workspace context is not set in the rooted-workspace branch: + +- `src/build/prepare/manifest.cpp:227-283` sets `state.wsManifest` and + `state.runtimeWorkspaceRoot` only in the branch that switches to a member. That + branch is taken for `-p`, or for a virtual workspace. +- A rooted workspace built as itself takes the branch at `:266-268`. That branch + merges the root's *own* `workspace = true` entries and sets neither field. + +The dependency loader then decides membership from exactly those two fields: + +- `src/build/prepare/graph.cpp:1999-2003` computes `depIsMember` from them. With + both fields empty, `depIsMember` is false. +- A member reached through the root package's `[dependencies] a = { path = "a" }` + is therefore loaded as an ordinary path dependency. +- It receives none of the three inheritances that SPEC-004 §9 item 1 requires: + `[workspace.package]`, `[workspace.build]` and `x.workspace = true`. + +The `depIsMember` block is byte-identical in 2026.9.26.1. The comment at +`graph.cpp:1987` states the intended invariant: "A MEMBER IS A MEMBER HOWEVER IT IS +REACHED". + +**Silent twin (inferred from the same fields).** In this position a member also +loses `[workspace.package]` and `[workspace.build]`: + +- A member that omits `package.version` because the workspace supplies it is + refused for a missing version. +- A member receives none of the workspace's build flags. + +The report shows only the loudest of these three losses. + +**The `-p` symptom is a second defect: the option promises a package and the +resolvers answer a directory** (measured with the layout of mcpp-language-server, +read from its repository). + +- **The report's own fixture does not show it.** In that fixture, `mcpp build -p a` + at the root succeeds, because the member's directory and its package are both + named `a`. +- **mcppls's layout does show it.** mcppls declares `members = + ["modules/base", ...]`. Its member packages are named `mcppls-base` and + `mcppls-platform`, and the root package depends on them under those names. +- **Which spellings work.** In a fixture with this shape, `-p base` and + `-p modules/base` succeed. `-p mcppls-base` answers "workspace member + 'mcppls-base' not found in [workspace].members". +- **What the interface promises.** Every command declares the option as + `-p, --package `, "the named workspace member" (`src/cli.cppm:377`, `:421`, + `:521`, `:686`). +- **What the code and docs/07 do instead.** Both resolvers match only a member's + directory basename or its path string (`src/build/prepare/manifest.cpp:230-243`, + `src/project.cppm:486-494`). They agree with each other, and no subcommand + matches a package name. docs/07 §5.3, in both languages, documents that + directory behaviour. + +The option's name, SPEC-001's identity and the root's own `[dependencies]` keys +all name the package. The resolvers and docs/07 name a directory. The fix is +§2.3 item 2. + +### 2.3 Fix + +1. **The workspace context.** The workspace context is a property of where the + manifest lives, not of the branch that was taken. When the root manifest has + `[workspace]`, the context is set before any dependency is loaded, whichever + branch follows: + - `wsManifest` is the root manifest. + - `runtimeWorkspaceRoot` is the root directory. + + The member-switch branch keeps setting the same two values. SPEC-004 §9 item 1 + gains the missing position: "a path dependency of the rooted workspace's own + package". + + The context carries the inheritance of SPEC-004 §9 item 1 and nothing more. + `[toolchain]`, `[target.]` and `[indices]` stay root-position keys + (§9 item 10), and a member reached by path does not take them. The criterion + in §2.4 checks this. + + **Compatibility note.** Such members now receive `[workspace.build]` and + `[workspace.package]` as SPEC-004 §9 item 1 always required. Their compile + commands can therefore change, and the CHANGELOG says so. +2. **`-p` means the package, as its name says.** The two resolvers become one + function, and the reviewer confirmed this reading (D1). A value is resolved in + this order: + 1. **Qualified name.** A value equal to a member's qualified name + (`.`, SPEC-001) selects that member. + 2. **Package name.** Otherwise, a value equal to a member's package name + selects it. When two members share that name under different namespaces, + the value is refused, and the message names both qualified names. + 3. **Directory or path.** Otherwise, a value equal to a member's path in + `[workspace] members`, or to its directory basename, selects that member. + These are the spellings docs/07 documents today, and they are kept, so a + script written against them keeps working. Two members can share a + directory basename, as `apps/core` and `libs/core` do. Today the first such + member in `members` is selected silently. It stays selected, and a warning + now names the others and their paths. + + A value can match one member at step 2 and a different member at step 3, for + example a member whose directory is named like another member's package. The + step-2 member is selected, because the option names a package. A warning names + the other member and its path spelling. This changes which member is selected + in such a layout, and the CHANGELOG says so. + + The "not found" message lists every member with its package name and its path. + docs/07 §5.3, in both languages, and the option's help text state the order. + +### 2.4 Criterion + +The criterion is an e2e test on a rooted workspace: + +- The root has both `[workspace]` and `[package]`. +- The root package reaches member `a` by `path`. +- `a` uses `x.workspace = true` for a dependency pinned in + `[workspace.dependencies]` to a version that is not the latest. +- `a` omits `package.version`, and `[workspace.package]` supplies it. +- The workspace declares a `[toolchain]` that `a`'s standalone build would not + choose. +- A plain `mcpp build` locks the pinned version. +- A plain `mcpp build` compiles `a` with the root's toolchain, and `a`'s commands + carry `[workspace.build]`'s flags. + +The non-latest pin also catches the accidental pass seen in 2026.9.26.1. The test +fails on both 2026.9.26.1 and 2026.9.27.1. Existing coverage misses the shape: +e2e 770 builds a member through `-p` on a virtual workspace, and the unit test only +formats the error. + +For `-p`, the same fixture gives member `a` the directory `modules/base` and the +package name `ws-base`: + +- `-p ws-base`, `-p base` and `-p modules/base` select the same member. +- Suppose a second member's directory basename is `ws-base`. Then `-p ws-base` + selects the package `ws-base` and warns, naming the second member's path. +- Two members named `ws-common` under different namespaces make `-p ws-common` + refused. Each qualified name selects its own member. + +`-p ws-base` fails on 2026.9.27.1. + +## 3. #720: the lib root is a node of its package's order + +### 3.1 What was measured + +The report's fixture, run on Linux with 2026.9.27.1, gives `fatal error: module +'repro.helper' not found`. Changing only `lib.path` to `src/helper.cppm` makes the +build pass. The ordering does not depend on the platform. + +### 3.2 Cause (read) + +- `src/build/prepare/features.cpp:976-1093` orders a host-module package's units. + It resolves the lib root (`:992`) and pushes it first (`:995`) before reading + any of its imports. +- It sorts only the *other* units topologically (`:1032-1089`, excluding the root + at `:1040`) and appends them after the root. The comment at `:1051` states the + assumption: "the lib root [is] already ahead of every entry here". +- `src/build/build_program.cppm:1385-1425` compiles the list in order, and each + unit sees only the BMIs of the units ahead of it. + +The topological sort came from `d4a83244` (2026.9.8.1, #589). That change replaced +an alphabetical order, and the old order had made mcpp-plugins fold everything into +its lib roots. The change took "the root is first" as an axiom. The axiom holds +until a lib root imports a sibling. + +### 3.3 Fix, silent twins and criterion + +- **Fix.** The root is an ordinary node of the same sort. Its name and its imports + are read the way the other units' are, and the existing diagnostic for a missing + lib root is unchanged. With no import edges from the root, the order is + byte-for-byte today's, because the sort keeps the original order as its stable + order. +- **Silent twins.** None were found. Ordinary dependency builds order units through + the scanner and ninja, `emit` does not compile host modules, and host tools are + full sub-builds. +- **Criterion.** An e2e test uses the report's fixture, with an explicit `lib.path` + and with the conventional lib root. It fails on 2026.9.27.1. +- **Ecosystem follow-up.** This is optional and not required. mcpp-plugins may + split the monolithic lib roots of its rule packages, which were grown to work + around the old order. + +## 4. #724: the build database and rule-generated files + +### 4.1 §1: a device source is not a compile unit + +**Facts (read; measured on the fixture of `examples/12-a-new-device-language`):** + +- `classify()` gives `SourceKind::Device` to an extension a rule claims + (`src/build/prepare/features.cpp:590-645`, `:765-780`). +- `src/build/plan.cppm:1848-1880` turns *every* graph unit into a `CompileUnit`, + including device units. +- `is_implementation_source` (`plan.cppm:741`) keeps device units out of link + inputs. Even so, `build.ninja` still carries a `cxx_object` edge for the device + file, and nothing references that edge (measured). +- `unit_invocations` (`src/build/compile_commands.cppm:433`) excludes NASM units + only. It feeds both the S1 document (`build_database.cppm:369`) and `mcpp + build`'s own `compile_commands.json` (`compile_commands.cppm:505`). Both list the + device source with `g++ … -c .toy` (measured). + +The defect is therefore not specific to `emit`. The plan states that a device +source is a C++ compile unit, and three consumers read the plan. + +**Fix.** The plan does not make a device-kind source a compile unit. This is +decided once, where units become `CompileUnit`s, so that ninja, the compile +database and S1 agree without each needing its own filter. The source remains in +`watch`, because the sources glob matches it. SPEC-005 R3.7 names device sources +beside NASM units. + +**Criterion.** A fixture with a device source is used: + +- `build.ninja` has no `cxx_object` edge whose input is the device source. +- Neither `compile_commands.json` nor the S1 document lists it. + +The criterion fails on 2026.9.27.1. + +### 4.2 Side finding A: the true diagnostic is discarded + +**Facts (read):** + +- When the build program fails, `src/build/prepare/target_side.cpp:1611-1625` does + two things: + - records `MCPP_BUILD_DATABASE_PROGRAM_FAILED` in `planNotes`; + - applies none of the program's directives. +- The device-source check (`:1767-1805`) runs anyway, for every package. With no + actions recorded, every device source is reported as orphaned, and the check + returns an error. +- `phase9_target_side` fails, so `prepare/driver.cpp:44` returns before `phase13` + copies `planNotes`. The recorded diagnostic is lost. +- `cmd_build.cppm:402-408` then reports `MCPP_BUILD_DATABASE_PLAN_FAILED` with the + orphan text. +- `hasProgram` (`:1784`) is true when a `build.mcpp` exists. The text therefore + says "`build.mcpp` ran" about a program that never ran successfully. + +SPEC-005 R5.2 already specifies the intended behaviour. A package whose build +program failed is described without that program's directives, and it carries +`PROGRAM_FAILED`. + +**Fix.** + +1. A check whose premise is a build program's directives does not run for a + package whose program failed in this pass. The package is then described as + R5.2 says, and the member keeps its one error diagnostic, `PROGRAM_FAILED`. It + is not failed for a symptom of that error. +2. Notes recorded before a phase fails are carried on the failure path. Today they + are lost on *every* failure after phase 9, not only on this one. +3. `hasProgram` means that the program ran and succeeded. + +**Criterion.** The fixture is a package that declares a device source and whose +`build.mcpp` does not compile: + +- `emit` reports `PROGRAM_FAILED`, with `path` set to `build.mcpp`. +- `emit` mentions no device source. +- The package is described. + +The criterion fails on 2026.9.27.1. + +### 4.3 Side finding B: `emit` writes into the project + +**Facts (read):** + +- `ensure_project_index_dir` (`src/build/config.cppm:929-975`) writes + `/.mcpp/.xlings.json` when the project declares `[xlings]` payloads or + custom repositories. +- The call site (`src/build/prepare/xlings.cpp:432-441`) uses the private work + root only when `runtimeSelection.ownerRoot == workRoot`. +- `ownerRoot` is the project root regardless of `emit`'s private `work_dir` + (`src/xlings/runtime_selection.cppm:51`). A project with `[xlings] deps`, which + every rules-qt consumer has, is therefore written to during planning. + +The reporter measured this behaviour. This record did not, because the effect +needs a fixture with payloads. + +**Why the criterion missed it.** e2e 688 asserts R2.1 with a digest of the project +tree. Its fixture declares no `[xlings]` payloads, so the branch that writes is +never taken. The criterion lacked the case that mattered. + +**Fix and criterion.** + +- Under a private `work_dir`, the file is written where the sibling branch already + writes it: the private root. +- e2e 688's tree digest is repeated on a fixture with `[xlings] deps`. A stub + xlings, as in e2e 733, is enough and needs no network. + +### 4.4 §2: generated outputs + +**Facts (read and measured):** + +- The plan holds each action's `id`, `role`, argv, inputs and outputs + (`modules/manifest/src/types.cppm:487-551`). The S1 document carries none of + them. +- `emit` plans under `$MCPP_HOME/cache/build-database/` + (`cmd_build.cppm:396`, `prepare/manifest.cpp:344`), as R2.1 requires, and action + outputs resolve there. +- `prepare_actions` (`modules/buildmcpp/src/directives.cppm:1324-1384`) writes an + empty placeholder for a compilable output that does not exist yet, under both + `build` and `emit`. A header gets no placeholder. + - Under `emit`, `moc_*.cpp` is therefore 0 bytes and `ui_*.h` is absent. + - The TU entries for the placeholders point at empty files. +- One generated file has two paths (measured): `/target/.build-mcpp/out/…` + under `build`, and `/target/.build-mcpp/out/…` under `emit`. The #699 + design fixed these path spaces (D2, D6) and did not treat generated outputs, so + this is a new gap, not a reversal. + +**The three asks:** + +1. **Describe the generated outputs: do.** This is general, the data exists, and + it names no tool. +2. **Say where a build put them: do, but as a fact of the plan rather than of the + filesystem.** The record gives the path that `mcpp build` of the same + configuration writes, whether or not the file exists now. + - A statement of existence would be stale the moment the user builds, and + `emit`'s `watch` set does not cover `target/`. + - The consumer checks existence itself and watches the path. + - The mapping from the private tree to the project's tree then stays inside + mcpp, which was the reporter's concern. +3. **Run "side-effect-free" generators under `emit`: decline.** + - R2.5 exists so that planning cannot fail on construction, and cannot be slowed + by it. + - A per-action purity claim is a promise the engine cannot verify, so it would + be a rule that nothing enforces. + - The generators are payloads that may not be installed at planning time. + SPEC-007 R8.3's criterion is a plan made without the payloads. + - Because the record in ask 1 carries the argv, a consumer that wants generated + files for a project that was never built can run the command itself into its + own cache, with its user's consent. The decision stays with the party that + owns the consent. + +**Proposal (W10).** + +- **One record for each action output that a set can see.** This covers + source-role outputs and every include directory under the output tree. Each + record carries: + - the planned path and the path the build writes; + - the action `id`, the role, the inputs and the argv; + - whether the output is a compilable source, a header or a directory. +- **Where the shape is defined.** The shape is written into S1, in the mcppls + repository, as an addition to profile 0.2.0. S1-11.2-1 lets S1 consumers ignore + unknown fields. +- **Where it does not go.** `compile_commands.json` must not carry the record: + clangd and clang-tidy reject a database that contains one unknown key (measured + 2026-09-26 during the #699 design). +- **Placeholders.** A TU whose source is a placeholder is identified as generated + through the record. No new `ide.role` is introduced. + +**Criterion (W10).** The fixture is the device-language example +(`examples/12-a-new-device-language`), extended with an action that generates a +header into an include directory: + +- The S1 document from `emit` carries a record for the header. The record gives + the planned path, the build's path, the action `id`, the inputs and the argv. +- A following `mcpp build` writes the header at exactly the build's path. +- `compile_commands.json` carries no such field. + +The first check fails on 2026.9.27.1, which emits no record. + +## 5. #723: one destination, one content + +### 5.1 Facts (read) + +- `resolve_runtime_contract` (`src/build/plan.cppm:877-996`) merges every package's + deploy entries into one list for the whole plan. +- `add_deploy` (`plan.cppm:1436-1456`) refuses two different normalised source + *paths* for one destination. +- Deploys become `stage_file` edges (`src/build/ninja_backend.cppm:2978`), and + those edges run `mcpp stage`. `mcpp stage` already compares bytes (`same_content` + in `src/build/stage.cppm`) so that it skips identical writes. +- `mcpp pack` reads the placed files by destination (`src/pack/pipeline.cppm:470`), + after the build. + +The collision is therefore not specific to `artifacts` (inferred from the merge). +Any two packages of one graph whose build programs deploy one generated file name +collide in the same way. An example is a Qt library dependency and its Qt consumer +when both ask rules-qt for translations. + +### 5.2 The invariant, and the four options + +The invariant is that a destination holds one content. The present check tests a +proxy for it, one source path, because at planning time a generated source has no +content yet. + +| Option | What it gets wrong | Verdict | +|---|---|---| +| 1. The consumer takes precedence, with a note | When the contents differ, the artifact program runs with a file it was not built for. A note is then the only record | reject | +| 2. Actions with equal commands and inputs produce the same file | A heuristic about tools. A tool whose output depends on its output path or working directory breaks it, and the engine cannot know which tools do. It also covers only pairs of actions | reject | +| 3. A per-edge exclusion list | A slot for exceptions. The author must know which files collide, and a list written today hides a real divergence tomorrow | reject | +| 4. Check the invariant where the contents exist | none of the above | **adopt** | + +Option 4 works as follows: + +- Two or more source paths for one destination become one `stage_file` edge, with + every source as an input. +- `mcpp stage` places the file when all sources are byte-identical, using the + comparison it already implements. +- Otherwise it fails, naming every source and the destination. + +The plan no longer refuses at planning. `emit` no longer fails on the collision, +since it runs no deploy. + +**One destination, one writer (found in self-review).** Three mechanisms write +into a program's directory today, and only the first goes through `add_deploy`: + +1. **Declared deploys.** These are `[runtime] deploy` and the `deploy` directive. +2. **The toolchain's runtime DLLs under `toolchain-coupled`.** These are staged by + `flags.cppm:1515-1560`, which lets a declared deploy of the same name win, with + a diagnostic. +3. **R4.3's placement after a PE link.** It reads the program's import table and + copies DLLs from the runtime search directories (`ninja_backend.cppm:1917-1937`, + `pack.cppm:1378`). It never consults the deploy list, so a DLL of the same name + can be written by a second, unrelated edge. + +W6 makes the merged deploy list the single authority for a destination: + +- **Declared deploys.** Among them, the content check above applies. +- **Toolchain runtime DLLs.** A declared deploy keeps outranking them, with the + existing diagnostic, because an explicit statement outranks a derived default. +- **R4.3 placement.** It skips a name that the list already places. It compares + the contents and reports a difference, rather than writing over the file. + +W9's default staging of the redistributable (§7.3) depends on this rule. Without +it, a plugin's runtime search directory that ships its own `vcruntime140.dll` +would race with the staged copy. + +### 5.3 Cost and criterion + +- **Code.** `DeployFile` carries a list of sources; `mcpp stage` accepts several; + `add_deploy` merges instead of refusing; the edge lists every source. Consumers + keep keying on the destination. +- **Plugin option.** rules-qt could deploy `qtbase_.qm` straight from the SDK + when one catalog suffices, which also removes an `lconvert` action. This is an + optimisation of the plugin, not the answer, because two independent plugins can + produce the same file. +- **Criterion.** + - Setup: two packages of one graph each generate the same bytes into their own + output directory and deploy them to the same name. + - Expected: the build succeeds and one file is placed. + - Control with different bytes: the build fails, naming both sources. + - The criterion fails on 2026.9.27.1 at planning. +- **Criterion for one writer (Windows leg).** + - Setup: a PE program's runtime search directory holds a DLL whose name the + deploy list also places, with different bytes. + - Expected: after the build the program's directory holds the listed file, and + the build reports the difference. + - On 2026.9.27.1 the file present depends on which edge ran last. + +## 6. #717: a graph-wide flag under a target condition + +### 6.1 Facts + +- **Measured:** `[target.linux.build] dialect_cxxflags = ["-DX717"]` produces + "unsupported key 'dialect_cxxflags' (ignored)", and the flag reaches no command. +- **`BuildInputs` versus `dialectCxxflags` (read):** + - `BuildInputs` (`modules/manifest/src/types.cppm:342`) holds the additive, + per-package inputs that a condition may carry. + - `dialectCxxflags` (`types.cppm:932`) is graph-wide. Only the root's value is + read (`src/build/prepare/scan.cpp:202-216`). + - It reaches the std BMI and the scan (`target_side.cpp:1923-1928`), the + translation units (`plan.cppm:1393`) and the fingerprint + (`prepare_inputs.cppm:568-571`, `cache_key.cppm:323`). + - `[workspace.build] dialect_cxxflags` is prepended into the member being + built, and into every member pulled in as a path dependency + (`inherit_workspace_build`, `src/project.cppm:327`). +- **A dependency's own value (read).** A dependency's `[build] dialect_cxxflags` + is parsed and enters that dependency's fingerprint (`prepare_inputs.cppm:663`), + but it reaches no command. +- **Precedents (read).** + - `[target..abi]` for `threads` and `exceptions` is a graph-wide switch + under a condition, with only the root's value rendered (`toml.cppm:3476-3501`). + - `[target.] cxx_runtime` and `linkage` are a second precedent. +- **Promotion list (read).** `-fms-runtime-lib` is not in the promotion list + (`types.cppm:2310-2325`), and that is correct. + +### 6.2 Assessment and design + +The need is general: a graph-wide switch that exists only on some targets. No +current spelling expresses it, so its home is the manifest grammar. + +SPEC-004 §3.1 and §6 fix the spelling. The key lives in `[build]`, so its +conditional form is `[target..build] dialect_cxxflags`. The engine keeps +`BuildInputs` separate from graph-wide keys. The conditional section is parsed into +two destinations: + +- the package's additive inputs; +- a conditional graph-wide list. + +The rules for the graph-wide list are these: + +- **Who contributes.** Only the root of the build contributes: the command's + package, or the member that `-p` selects. The order is `[workspace.build]`, the + root's `[build]`, then each matching `[target..build]` in manifest + order. Entries are appended, as `cxxflags` are. +- **Which target decides.** The resolved target decides, including a host build's + host row (SPEC-004 §4.6). A build program and a host tool are sub-builds with + their own root, so the program's host std BMI is not changed after the fact, + which the report requires. +- **Dependencies.** A dependency's graph-wide keys still reach no command. SPEC-004 + §9 item 10 adds `dialect_cxxflags`, with its conditional form, to the + root-position keys. This makes today's behaviour a stated rule. There is no + diagnostic, as for `[toolchain]`: a dependency legitimately declares these keys + for its own builds as a root. +- **Caching.** The resolved list enters the fingerprint and the std BMI key through + the variable already used. +- **A key enters a fingerprint only where it reaches a command (found in + self-review).** + - **Today.** A dependency's own `dialect_cxxflags` enters that dependency's + fingerprint (`prepare_inputs.cppm:663`) although it reaches no command. + `inherit_workspace_build` (`src/project.cppm:327`) also prepends + `[workspace.build] dialect_cxxflags` into every member pulled in as a path + dependency. W1 therefore makes more members carry an inert value, and each + such member would be rebuilt once for nothing. + - **Change.** W8 removes a package's own graph-wide keys from its fingerprint + contribution. Every package's cache key already carries the resolved + graph-wide list (`language.dialect_flags`, `cache_key.cppm:323`), and that + list is what reaches the commands. + +**The build-program directive (`mcpp::dialect_cxxflag`): decline.** + +- The condition is a target predicate, which the manifest states as data. A program + earns its place only when the decision needs something only a program can + compute, and no such need is shown. +- A directive would give a build program authority over the std BMI and over every + package's translation units, and would need an authority rule for dependency + programs. +- The report leaves this choice to the maintainers. + +After W9, the report's motivating flag is no longer written by the project at all. +W8 remains useful for other target-specific graph-wide flags. + +**Criterion.** On a Linux host: + +- `[target.linux.build] dialect_cxxflags` reaches the std BMI, scan and TU + commands. +- `[target.windows.build]` reaches none of them. +- Switching between the two rebuilds the std BMI (A, then B, then A). + +The criterion fails on 2026.9.27.1. + +## 7. #718: the CRT on the MSVC ABI + +### 7.1 Facts (read; the closure comment of #649 E10) + +- **Which dialect a row gets.** `dialect_for` returns the MSVC dialect only for cl + (`modules/toolchain-model/src/dialect.cppm:260`). The LLVM row, which is clang++ + targeting `*-windows-msvc`, gets the GNU dialect. +- **What each row emits.** The CRT block (`src/build/flags.cppm:1045-1053`, and its + mirror for the std module in `prepare/scan.cpp:690-702`) emits nothing for the + LLVM row, so clang links `libcmt` by default. cl gets `/MD` by default and `/MT` + on request. +- **The shared derivation.** `msvc_wants_static_crt(linkage, cxxRuntime)` + (`dialect.cppm:160`) is the single derivation of #422. The translation units and + the std BMI both use it. +- **What E10 records.** E10 (`src/build/distribution.cppm:725-742`) records the + LLVM row as `self-contained`. It downgrades an explicit request for any other + value, with a warning that names `msvc@system`. +- **Rows mcpp drives.** mcpp drives no clang-cl row, so the report's clang-cl + column has nothing to apply to. +- **Debug CRT.** No debug CRT exists anywhere. +- **A flag given only at compile time does not reach the link.** This was measured + for the E10 record (`2026-09-16-646-649-four-issues-by-home.md` §4.5). With + `-fms-runtime-lib=dll` at compile time, the objects carry + `--dependent-lib=msvcrt`, yet the clang driver's link step still passes + `-defaultlib:libcmt`. The clang driver chooses the CRT separately at compile time + and at link time. + +### 7.2 The default: two questions, and the norm that answers each + +A default CRT answers two questions that mcpp's vocabulary already separates: + +1. **Which CRT is compiled against.** Static or dynamic; this is a matter of ABI. +2. **Where the DLLs come from at run time.** This is a matter of deployment. + +**The industry norm answers the first question with the dynamic CRT.** The +compiler drivers default to the static CRT when no flag is given. Every +mainstream build system and package manager for Windows overrides that and +defaults to the dynamic CRT: + +| Party | Default | +|---|---| +| cl.exe, clang-cl, the clang++ driver, with no flag | static (`/MT`, `libcmt`) | +| Visual Studio project templates | `/MD` (Release), `/MDd` (Debug) | +| CMake 3.15+ (policy CMP0091) | `MultiThreaded$<$:Debug>DLL`, which gives `/MD`, and `/MDd` for Debug | +| Meson (`b_vscrt = from_buildtype`) | `/MD`, and `/MDd` for the debug build type | +| Cargo and rustc (`*-pc-windows-msvc`) | dynamic; `+crt-static` opts into static | +| vcpkg's default triplet `x64-windows`; Conan's `compiler.runtime` | dynamic | +| Qt's official binaries | `/MD` | + +The reasons are structural, not a matter of taste: + +- **Every object in an image must agree.** Every object and prebuilt library linked + into one image must use the same CRT: the `RuntimeLibrary` mismatch check fails + the link with LNK2038. The prebuilt ecosystem ships `/MD`, which is the case of + GalTranslPP with Qt and vcpkg. +- **Each `/MT` image has its own CRT state.** Under `/MT`, each DLL carries its own + heap, `FILE*` table, `errno` and locale. Memory or CRT objects that cross a DLL + boundary are therefore unsound. A program made of several images needs one + process-wide CRT. Examples are `dependency_linkage = "shared"` and plugins + loaded at run time. + +**mcpp's own norm answers the second question.** `cxx_runtime` defaults to +`self-contained`, "portable by default": a built artifact runs on a machine where +nothing was installed (docs/20). On the MSVC ABI, mcpp's three values already +split the two questions: + +| Value | CRT | Run time | +|---|---|---| +| `self-contained` | `/MT` | nothing outside the image | +| `toolchain-coupled` | `/MD` | the toolset's own `vcruntime140*.dll` and `msvcp140*.dll` are staged beside the artifact | +| `host-coupled` | `/MD` | the target has the redistributable installed | + +`ucrtbase.dll` is a component of Windows 10 and later. Microsoft permits +app-local deployment of the redistributable files from `VC\Redist`. A +`toolchain-coupled` artifact therefore runs on a clean Windows 10+ machine. + +**The answer: `toolchain-coupled` is the default for every row whose target is +the MSVC ABI.** This covers cl and clang++ alike, because the CRT is a property of +the target ABI and not of the compiler. The default satisfies both norms: the +dynamic CRT that the ecosystem is built against, and an artifact that runs +without an installer. + +The choice keeps the existing shape of mcpp's defaults: docs/20 already lets the +default of the shared-library role depend on the hazard of the target format. +Here the default depends on the ABI, because the hazard of a CRT per image is +the ABI's. + +No new key is added. `linkage` and `cxx_runtime` state static versus dynamic, and +also where the DLLs come from, which the proposed `msvc_crt_linkage` would not. A +second key would be a second spelling of one fact. + +**Two deliberate departures from the industry norm, each with its reason:** + +1. **The redistributable is staged by default.** The industry relies on an + installer or a central redistributable instead. mcpp's promise is that the + built directory runs as it is. The cost is a few DLLs, about 1 to 2 MB, beside + each program. `host-coupled` removes them. +2. **The dev profile does not select the debug CRT.** CMake, Meson and Visual + Studio select `/MDd` for Debug. mcpp's dev profile states debug information, + not a different ABI: + - The debug CRT changes `_ITERATOR_DEBUG_LEVEL` for every prebuilt library. + - Its DLLs may not be redistributed. + - The report itself asks that `debug = true` not imply it. + + The debug CRT stays deferred, as an opt-in axis to be designed when a consumer + needs it. + +### 7.3 Change (W9) + +- **Which rows receive the model, and where it reaches.** Every MSVC-ABI row + receives the CRT model: + - cl spells it `/MT` or `/MD`; + - clang++ spells it `-fms-runtime-lib=static` or `-fms-runtime-lib=dll`. + + The word reaches the translation units, the std and std.compat BMIs, and the + link command, because of the fact in §7.1. It comes from one helper, and it + enters the fingerprint and the std BMI key. MinGW (`*-windows-gnu`) is not the + MSVC ABI and is unaffected. +- **Default.** The undeclared contract on the MSVC ABI resolves to + `toolchain-coupled` for every role. The contract is whole-project on this ABI: + docs/20 already refuses a per-role override there. docs/20's per-format table today + says only "PE (Windows)" for its `self-contained` shared-library default. It is + amended to say that this row applies to the GNU ABI (MinGW). With this change the record and the flags agree: today the cl row's + default records `host-coupled` while emitting `/MD`. +- **Explicit values.** + - `self-contained`, or `linkage = "static"`: `/MT`. + - `host-coupled`: `/MD` without staging. + - `toolchain-coupled`: `/MD` with staging. + + `msvc_wants_static_crt` keeps its inputs. +- **What is staged, and from where.** The existing mechanism is used, and no + second one is added. + - **The mechanism.** The planning-time `toolchain-coupled` staging + (`flags.cppm:1515-1560`) produces `stage_file` edges. The same directory is put + on the `mcpp run` and `mcpp test` search path. + - **The source.** The source directory is the resolved toolset's + `VC\Redist\MSVC\\\Microsoft.VC*.CRT`. It is carried as its own + toolchain field. + - cl's `linkRuntimeDirs` holds exactly that directory today + (`src/toolchain/msvc.cppm:1573`). + - On the LLVM row the field comes from the row's `sysroot` resolution. The + LLVM row's `linkRuntimeDirs` holds LLVM's own directories + (`src/toolchain/clang.cppm:190`), so copying it would stage the wrong files. + - **The gate.** The staging gate at `flags.cppm:1522` asks whether the target is + the MSVC ABI and the toolset has a redistributable directory. Today it asks + whether the compiler is cl. +- **A row whose toolset has no redistributable directory.** Whether `xim:msvc` + carries one is yet to be measured. On such a row the default is `host-coupled`, + and `resolution.json` records it. It is a property of the row, stated once in + docs/20, not a warning on every build. An explicit `toolchain-coupled` on such a row + is refused, naming the missing directory. It is never downgraded: an explicit + statement that cannot be met is an error, as in the pack rule below. +- **`mcpp pack`.** The default mode (`vendored`) carries the staged DLLs. + - An explicit `--mode system` resolves a *defaulted* contract to `host-coupled`: + an explicit choice outranks a default. + - Only an explicit `toolchain-coupled` together with `--mode system` is refused, + as today. +- **E10.** Every MSVC-ABI row now emits a CRT model, so E10's degrade path applies + to no row and is removed. A value that a row cannot deliver is refused, as + stated above. e2e 703 is inverted. +- **Free-form CRT words (D3).** Every MSVC-ABI build now states its CRT, so a CRT + word in `cxxflags` or `dialect_cxxflags` (`-fms-runtime-lib=*`, `/MD`, `/MT`, + `/MDd`, `/MTd`) is always a second statement. The engine never lets the last + word win. + - A word that agrees with the resolved model is warned as redundant. + GalTranslPP's `-fms-runtime-lib=dll` therefore keeps building. + - A word that disagrees is refused. The message names the word, the key, and + the value that corresponds to the word. +- **Upgrade.** + - **cl-row projects.** Their compile commands are unchanged; programs gain the + staged DLLs. + - **LLVM-row programs.** They move from the static to the dynamic CRT, and + their std BMI is rebuilt once. A project that links `/MT` prebuilt libraries + fails with LNK2038 and states `cxx_runtime = "self-contained"`. + - **Announcement.** The CHANGELOG and docs/20 carry an "Upgrading" note, as + docs/20 already does for the 2026.8.16 change. +- **Debug CRT.** Deferred, as stated in §7.2. + +**Criteria.** + +- **Unit property.** It covers every MSVC-ABI row × {undeclared, `self-contained`, + `toolchain-coupled`, `host-coupled`, `linkage = "static"`}. Each combination + yields exactly one CRT word, spelled for its driver. The word is equal in the TU, + the std BMI and the link command. +- **Windows leg: imports and staging.** + - The default LLVM-row program imports `vcruntime140.dll`, and the file is + staged beside it. + - The program runs from the build directory with the Visual Studio directories + removed from `PATH`. + - `self-contained` imports none of these DLLs. +- **Windows leg: switching.** Switching between the default and `self-contained` + rebuilds the std BMI (A, then B, then A). +- **Windows leg: pack.** `mcpp pack` in its default mode includes the DLLs. + `--mode system` succeeds and records `host-coupled`. + +## 8. #722: phase functions + +The item is internal, so its home is the engine repository, and it is done in +this round as stage 2 (D7). It follows stage 1, because those fixes touch `manifest.cpp`, `graph.cpp`, +`features.cpp`, `target_side.cpp` and `plan.cpp`. A mechanical split first would +turn each defect fix into a rebase. A split afterwards leaves each fix a small, +reviewable diff, and the features W8 to W11 then land in the smaller functions. + +The criteria the report states apply unchanged: + +- the golden fixtures stay byte-identical; +- AddressSanitizer runs with `detect_stack_use_after_return=1`; +- no new interface unit enters `mcpp.build.prepare`'s import chain. + +The proposed gate on function length is adopted only in a form that parses: +clang-tidy `readability-function-size`, run over a compile database that mcpp +produces for its own LLVM leg. A line-counting script over brace heuristics is a +substring criterion. The gate is part of W7's acceptance, and it fails before +the split, because functions of 1,000 to 2,300 lines exist. If the parsing form +cannot be built, the function limit is removed from the acceptance. It is not kept +as a sentence that nothing checks (rule 3). The file gate stays in either case. + +## 9. Items raised in review + +These items do not come from the eight reports. The reviewer raised them on +2026-09-27, and they are routed by the same rules as the reports. + +### 9.1 One progress mechanism for every acquisition (W11) + +**The observation.** A toolchain download shows progress. A download of a library +or of an xlings dependency appears to show none. + +**Read.** One producer and one renderer already exist, and most paths use them: + +- **The producer.** `xlings interface install_packages` streams NDJSON + `download_progress` events. xlings builds these events from the same + `DownloadProgressRenderer` that draws its own bars + (`openxlings/xlings src/core/xim/commands.cpp:811-833`). +- **The renderer.** mcpp parses the events in `InstallProgressHandler` and draws + them with `ui::DownloadProgress` (`src/fetcher/progress.cppm:208-263`, + `src/ui.cppm:148-162`, `:623-682`). + +The following table lists every path by which mcpp acquires remote content: + +| Path | Progress today | +|---|---| +| Toolchain, runtime payload (glibc, openkal), host tools (`resolve_xpkg_path`) | the shared bar | +| Library packages from the index (`graph_load.cpp:848-877`, global and project scope) | the shared bar | +| `[xlings]` and `[feature-xlings]` payloads (`prepare/fetch.cpp:186-471`) | the shared bar | +| Index refresh: automatic (`refresh_policy.cppm:161-164`, `xlings.cppm:2168-2171`), and first use of a custom index (`prepare/xlings.cpp:558-560`) | one static status line, then silence: the bare `xlings update` CLI runs with `quiet = true` | +| Explicit `mcpp index update` | xlings's unstructured text, reprinted line by line | +| `git` dependencies (`fetch.cpp:127-140`, `graph.cpp:1911-1944`) | none: the output is captured whole and shown only on failure | +| The sandbox bootstrap (`xlings.cppm:1584-1738`) | a spinner on a TTY; the direct `xlings install` output is discarded | + +**Reading of the observation (inferred, to be measured).** On the paths read, +library and payload installs do draw the shared bar. The silence the reviewer saw +most likely has one of three sources: + +- the index refresh that precedes a library's first resolution, which can take + many seconds and shows one static line; +- a `git` dependency; +- a payload whose install downloads through a channel that emits no + `download_progress` event. + +W11 therefore starts with a measurement: one cold run of every row of the table +above, recording what the terminal shows. The measurement also confirms that the +rows marked as sharing the bar really draw it. + +**A defect found on the way (read).** `ProgressBar` writes `\r` and ANSI erase +sequences whether or not the stream is a terminal (`src/ui.cppm:440-475`); only +colour is gated. CI logs therefore collect one line per repaint. The bootstrap +spinner is the one place that gates on `is_tty()` (`xlings.cppm:1621`). + +**Checked, not a defect.** `emit build-database` redirects mcpp's stdout to +stderr during planning (`StdoutToStderr`, `src/cli/cmd_build.cppm:377`). A bar +drawn during planning therefore cannot corrupt the document on stdout (SPEC-005 +R1.3). + +**Design.** The design adds no second UI. Every acquisition becomes a producer of +the one event shape that `ui::DownloadProgress` already renders: an item, a label, +bytes done and bytes total, or a phase percentage when bytes are unknown. + +| Path | Producer | Home | +|---|---|---| +| Index refresh | the `update_packages` capability of `xlings interface`, which exists (`src/capabilities.cpp:182`), in place of the bare CLI. Whether it emits `download_progress` for an index sync is to be measured. If it emits none, xlings emits it for the index artifact and the git sync. | mcpp; xlings if the events are missing | +| `git` dependencies | git's own `--progress` download phase (`Receiving objects: 45% ...`), read as it is redrawn: the streaming runner gains an opt-in rule that a lone carriage return also ends a line | mcpp | +| Sandbox bootstrap | the shared bar, in indeterminate form, in place of the hand-drawn terminal-only spinner. The direct CLI stays first: its comment records that it is the more reliable path for large payloads, and the order is not what hid the progress | mcpp | +| A payload whose install emits no events | the payload's install uses xlings's downloader, which emits them | ecosystem data (the payload's recipe) or xlings, per case | + +Rendering follows one rule per output mode: + +| Mode | Rendering | +|---|---| +| Terminal (TTY) | the live bar, as today | +| Not a terminal | one line when an item starts, with its size when known, and one line when it finishes, with its duration; no `\r` and no ANSI. This mode fixes the defect above. | +| `--quiet` | nothing | +| Machine output (`--format json`, `emit`) | nothing on the document stream. Progress remains narration on stderr and is not part of the envelope. | + +The end-of-run notices of W12 (§9.2) go through the same reporter, as the run's +closing lines. + +**Criteria.** + +- **Unit: the git progress parser.** Recorded git stderr is parsed into the + expected events. +- **Unit: the renderer in non-terminal mode.** It is rendered into a buffer, which + contains no `\r` or `ESC` byte and exactly one start line and one finish line + per item. +- **e2e.** A `git` dependency is fetched with stderr redirected to a file. The file + holds the start and finish lines and no `\r`. This criterion fails on + 2026.9.27.1, which prints nothing for the clone. +- **Measurement.** The measurement matrix above is recorded in this document + before and after the change. + +### 9.2 An index that requires a newer mcpp is reported as an error (W12) + +**Measured.** The binary is mcpp 2026.9.18.1, and the index's `min_mcpp` is +2026.9.18.3. `mcpp build` of a project with one index dependency prints, before +anything else: + +``` +error: index requires mcpp >= 2026.9.18.3 but this is mcpp 2026.9.18.1 [E0006] + ... + Details: mcpp explain E0006 (override for debugging: MCPP_INDEX_FLOOR=ignore) Upgrade: 'xlings update mcpp' (recommended default installer) +``` + +The build then resolves its dependency, compiles, finishes, and exits with 0. The +line labelled `error:` described no error of the command. The message also lacks +a line break before the appended upgrade line. + +**Read.** + +- **The read site.** `read_identity_verified_xpkg_lua` + (`src/pm/package_fetcher.cppm:698`) calls `check_index_floor` on the first + descriptor read from each index. On a violation it calls `mcpp::ui::error` and + returns no descriptor. This is the "start of the run" in the observation. +- **The fact is also recorded.** `check_index_floor` + (`src/pm/index_contract.cppm:236-262`) records the fact in a process registry. +- **The failure path already explains itself.** A lookup that then fails carries + the cause through `unusable_index_hint()` (`package_fetcher.cppm:410`, + `prepare/fetch.cpp:487`). That hint currently says "See the [E0006] error + above". +- **The refresh guard.** `update_index` (`src/xlings/xlings.cppm:2046-2080`) keeps + the previous snapshot when a refreshed index requires a newer mcpp. It prints a + `Kept` status line at that moment, which is mid-run. +- **The two defects.** The guard itself is correct. The label and the timing of + the notice are not. + +**The rule this follows.** The rule is already written down, and this path breaks +it: an index is data and mcpp is a program, and `min_mcpp` is a routing signal, +not a termination signal (the 2026-08-03 records on the index floor). A run that +succeeds has no error to report. + +**Change.** + +1. **The read site prints nothing.** The floor violation is recorded, as today, + and the lookup still does not use that tree. +2. **A run that fails** carries the cause in its failure message, as + `unusable_index_hint()` already does. The hint now states the E0006 text + itself instead of pointing at an error above it. +3. **A run that succeeds** prints at most one line at its end, and only when this + run refreshed or attempted to refresh an index and that index requires a newer + mcpp. The refresh may be `mcpp index update` or an automatic refresh. The line + has the form `tip: the package index now requires mcpp >= X; this run used the + previous index. Upgrade: xlings update mcpp`. A run that did not touch the + index says nothing. The tip moves the guard's mid-run `Kept` status to the end + of the run. +4. **Under `--format json`**, the fact is a diagnostic of severity `note`. It + never changes the exit code (SPEC-003). +5. **`mcpp doctor` reports the state. This is new work, not existing + behaviour.** Today `doctor_report` (`src/doctor.cppm:187-790`) never consults + the unusable-index registry. `src/doctor.cppm:1337` is the text of + `mcpp explain E0006`, not a check. W12 adds a doctor check that lists every + index in `unusable_indexes()` with its floor. Without that check, removing the + error at the read site would leave the state visible only in a failing run. +6. **The missing line break** in `e0006_message` is added. + +**Criteria.** Each is an e2e test with a path index whose `index.toml` +`min_mcpp` is above the binary's version: + +- A build that needs no package from that index prints no `error:` line and exits + with 0. +- The same build with an index refresh in the run prints exactly one `tip:` line, + and it is the last line of the output. +- A build that needs a package only that index serves fails, and its last error + names E0006. This is e2e 185's existing assertion, kept. +- `mcpp doctor` lists that index and its floor. + +The first and last criteria fail on 2026.9.27.1: the read site is unchanged at +`b439fd97`, and `doctor_report` has no such check. + +### 9.3 #726: an xlings invocation on Windows leaves the process as it found it (W13) + +**Observation (measured on GalTranslPP's Windows CI).** A `vcpkg install` action +failed with `'C:\Program' is not recognized ...` in a build that installed a +toolchain or a payload itself; a second build passed. + +**Cause (read, and confirmed by isolation runs in #726).** On POSIX the xlings +command prefix carries `cd `, `XLINGS_HOME` and `PATH`, and nothing reaches +the build. On Windows two differences existed: + +1. **`build_command_prefix` changed the process.** + - It set `XLINGS_HOME` and prepended `/registry/subos/default/bin` + to the process `PATH`, and nothing restored either. Three copies of the + directory were observed. + - ninja and every action then found `xim:llvm`'s `cl`, `link`, `lib` and `rc` + shims in front of MSVC's tools, and vcpkg's MSVC detection met them. +2. **xlings started in mcpp's working directory.** + - xlings chooses project mode by walking up to a `.xlings.json`, so in a + project that pins its mcpp there, the registry's xlings adopted the project. + - It wrote the shims of mcpp's toolchain and payloads into the project's + SubOS, whose shims resolve against the user's home. + +**Change (pull request #727).** + +- **`ScopedInvocationEnv` owns the whole invocation environment on Windows.** It + saves, applies and restores `XLINGS_HOME`, the scope variables and `PATH`, + newest first. +- **The Windows command starts in the home.** It is + `cd /d "" && ""`, as the POSIX one starts with `cd`. +- **Building a command changes nothing.** The unit tests check that across three + guarded invocations. + +**Review of #727 (this record).** + +- **Guards.** Every use of `build_command_prefix` is in `src/xlings/xlings.cppm`, + and each runs under a guard. +- **No concurrency.** mcpp never runs two xlings invocations at once, so the + process-wide save and restore cannot interleave. The one thread near an + invocation is the direct install's worker, which runs a single command while + the caller's guard is held. +- **One defect found.** The pull request moved `mcpp.toml` to 2026.9.27.2 but not + `MCPP_VERSION` (`modules/versioning/src/version.cppm`). Its Linux job failed on + the version check. The round moves both places together at release time. +- **An interaction with W11.** W11 routes the index refresh through + `xlings interface update_packages`. That command is built by + `build_command_prefix` inside `update_index_unguarded`, which holds the guard, + so the refresh's environment does not reach the build either. + +**Criteria.** + +- `tests/unit/test_xlings.cpp`: + - `NeitherPathNorTheHomeOutlivesTheInvocation`; + - `TheWindowsPrefixStartsInTheHome`. +- The Windows legs of the pull request's CI. +- GalTranslPP's Windows CI against the branch, which is a real project with a + `.xlings.json` at its root and 22 vcpkg ports. + +## 10. What is not done, and why + +| Proposal | Source | Why not | +|---|---|---| +| `mcpp::dialect_cxxflag` in build programs | #717 | A target predicate is data; the directive would give programs authority over the whole graph (§6.2) | +| Auto-promote `-fms-runtime-lib` into the dialect flags | #717 | The CRT belongs to `linkage` and `cxx_runtime` (§7.3) | +| New keys `msvc_crt_linkage` and `msvc_crt_variant` | #718 | A second spelling of an existing fact (§7.2) | +| The debug CRT in the dev profile, as CMake, Meson and Visual Studio do | #718 | The dev profile states debug information, not a different ABI; deferred as an opt-in axis (§7.2) | +| Keeping the static CRT as the LLVM row's default | #718 | It is ABI-incompatible with the prebuilt ecosystem and gives each DLL its own CRT state (§7.2) | +| Consumer precedence, action equality, exclusion lists | #723 | §5.2 | +| Run generators under `emit` | #724 | §4.4 | +| A function-length gate written as a text heuristic | #722 | §8 | +| A second progress UI for the paths that show none | review | One renderer exists; the missing paths become its producers (§9.1) | +| Any work on the GCC 16.1 ICE | #721 | upstream; out of scope for this round | + +## 11. Order of work + +The reviewer decided that #722 is done in the same round (D7) and that the round +lands in pull request #727 (D10). The round is therefore one pull request and one +release, in three stages. The stages are ordered so that each diff stays small and +reviewable. + +| Stage | Step | Content | Specs and docs | +|---|---|---|---| +| 1 | W1 | #725: the workspace context for a rooted workspace; `-p` resolves the package identity | SPEC-004 §9 item 1; docs 07 §5.3 (en, zh); the `-p` help text | +| 1 | W2 | #720: the lib root in the sort | docs 31 (a lib root may import siblings) | +| 1 | W3 | #724 §1: a device source is not a compile unit | SPEC-005 R3.7 | +| 1 | W4 | #724 A: the failed program's diagnostic survives | SPEC-005 R5.2 amended: a check whose premise is a build program's directives does not run for a package whose program failed | +| 1 | W5 | #724 B: no project write under `emit` | SPEC-005 R2.1 (unchanged) | +| 1 | W6 | #723: one destination, one content, checked when staging; one writer per destination | SPEC-007 R4.2 and R4.3 | +| 1 | W12 | the index floor is a closing tip, not an error; a doctor check; W12 introduces the reporter's list of closing notices | docs 09 (and its doctor section), docs 50 (`note` severity) | +| 1 | W13 | #726: an xlings invocation on Windows leaves the process environment as it found it and starts in the home (#727) | CHANGELOG | +| 2 | W8 | #717: conditional graph-wide dialect flags | SPEC-004 §3.1, §9 item 10 | +| 2 | W9 | #718: the CRT model by ABI; `toolchain-coupled` is the MSVC-ABI default | docs 20 and 04; SPEC-006 (the row's CRT) | +| 2 | W10 | #724 §2: the generated-output record | S1 addition (mcppls), SPEC-005 §3 | +| 2 | W11 | one progress mechanism; the producers per path | docs 09 | +| 3 | W7 | #722: split the phase functions | none | + +The order has three reasons: + +1. **Stage 1 comes first because it fixes defects on the present layout.** W1 is + first in it, because it blocks existing builds, including mcppls's nightly run. +2. **Stage 2 holds the features.** Their order is fixed by three dependencies: + - W8 precedes W9, because W9's rule for free-form CRT words reads the + `dialect_cxxflags` that W8 makes conditional. + - W10 waits for the S1 text (D6) and builds on W3. + - W11 attaches progress rendering to the reporter that W12 introduced. +3. **Stage 3 splits the phase functions last.** + - **Revised during implementation.** Revision 3 placed the split between the + defects and the features. Implementation reversed that, for two reasons: + - The features run as parallel tasks, and each touches `src/build/prepare/`. + A split that moves every phase function while they are written would + turn each of their diffs into a conflict inside moved code, which a merge + cannot carry. + - As the last step, the split is a purely mechanical change against a + fixed tree. That is exactly the case its criterion is built for. + - **Its criterion.** The criterion of byte-identical output compares the split + with the unsplit code at the same commit of the round. The golden fixtures + are regenerated from that commit, and the features and defects are then + already in them. + +Every defect and feature step (W1 to W6 and W8 to W13) has a criterion that fails +on 2026.9.27.1 and passes after the change. W7 is a refactor, and its invariants +hold before and after by construction: the golden fixtures stay byte-identical, +and ASan stays clean. Its one criterion that fails before is the function-size +gate (§8). + +**Compatibility notes for the CHANGELOG.** + +- **W1.** Members reached as path dependencies of a rooted workspace receive + `[workspace.build]` and `[workspace.package]`. `-p` resolves a package name or + a qualified name first, and the directory spellings second. A value that is one + member's package name and another member's directory now selects the package, + with a warning. A package name shared by two members under different namespaces + is refused, and the message gives both qualified names. This is the third + refusal the round adds. +- **W6.** A deploy collision is reported when staging, and identical files no + longer collide. +- **W9.** LLVM-row programs on the MSVC ABI move to the dynamic CRT, with the + redistributable staged beside them. cl-row programs keep `/MD` and gain the + staged DLLs. `cxx_runtime = "self-contained"` restores the static CRT. + - **Two manifests that build today are refused afterwards**, each with the + one-line fix in its message: + - a free-form CRT word that contradicts the resolved model, for example + `-fms-runtime-lib=static` on the LLVM row; + - an explicit `toolchain-coupled` on a row whose toolset has no + redistributable directory. Today E10 downgrades that request with a warning. +- **W12.** E0006 is no longer printed as an error by a run that succeeds. + `mcpp doctor` lists any index this mcpp cannot read. + +**Other repositories.** + +- **openxlings/xlings.** It is involved only if the W11 measurement shows that + `update_packages` emits no `download_progress` for an index sync. The xlings + release then precedes the mcpp release, and the mcpp release pull request + carries the xlings pin. +- **mcpp-language-server.** + - The S1 addition for W10 (D6). + - After the release, its nightly run is green again (W1), and it may read the + generated-output record. +- **GalTranslPP.** After the release it needs: + - no CRT flag at all, because W9's default is its choice; + - the updater back on an `artifacts` edge (W6); + - `lib.path` may point at `gpp-build.ixx` (W2). +- **mcpp-plugins.** No change is required. Two changes are optional: splitting + the lib roots (W2), and the SDK-catalog deploy in rules-qt (W6). + +## 12. Decisions + +Every decision is settled (revision 3, 2026-09-27). + +| | Decision | State | +|---|---|---| +| D1 | #725: the workspace context is set from where the manifest lives. `-p` resolves the qualified name, then the package name, then the directory spellings (§2.3) | accepted. The reviewer pointed out that the option is `--package ` and asked for the reasonable, specification-conforming form | +| D2 | #718: `toolchain-coupled` (`/MD` with the redistributable staged) is the default for every MSVC-ABI row, cl and clang++ alike | accepted | +| D3 | #718: a free-form CRT word that agrees with the model is warned as redundant; one that disagrees is refused | accepted | +| D4 | #717: `[target..build] dialect_cxxflags`, root-only and appended; the directive is declined | accepted | +| D5 | #723: a content check when staging replaces the planning refusal, for every deploy; the merged deploy list is the single authority for a destination | accepted (the one-writer rule was added by the self-review, §5.2) | +| D6 | #724 §2: mcpp proposes the record's shape, and the mcppls maintainers write it into S1 before W10 lands | accepted | +| D7 | #722 in the same round | accepted: one release, with #722 as stage 2 (§11) | +| D8 | W12: no error at the read site; the cause goes into a failure message; a closing tip only when the run refreshed an index; a new `mcpp doctor` check | accepted | +| D9 | W11: measure every acquisition path first, then make the missing paths producers of the one renderer; non-terminal output without `\r` | accepted | +| D10 | The whole round lands in pull request #727, together with #726's fix, and the combined change is verified as one | given by the reviewer on 2026-09-27 | + +## 13. Self-review of the whole plan + +**Method.** The self-review was done in two passes on revision 2: + +1. **The author's pass.** The author read the plan against the specifications and + the code, looking for interactions between steps. +2. **An independent adversarial review.** A second reader checked the steps + against each other, against SPEC-001 to SPEC-007 and docs 10, 20 and 50, for + upgrade cliffs, and for criteria that cannot fail. It spot-checked the + citations behind W1, W3, W6, W9 and W12. + +The citations behind those five steps were confirmed. Every finding below changed +the text, in the section named. + +### 13.1 Findings, and what changed + +| # | Finding | Found by | Change | Where | +|---|---|---|---|---| +| 1 | The plan said `mcpp doctor` already reports an unreadable index. It does not: the cited line is the text of `mcpp explain E0006`. Removing the error at the read site would have left the state visible only in a failing run | reviewer | W12 adds a doctor check | §9.2 | +| 2 | Three mechanisms write into a program's directory, and only declared deploys go through `add_deploy`. R4.3's placement ignores the deploy list, so W9's default staging of `vcruntime140.dll` could race with a plugin directory that ships its own copy | reviewer and author | the merged deploy list is the single authority for a destination; R4.3 skips a name the list places | §5.2 | +| 3 | The first text of W9 staged the redistributable through R4.3, from `linkRuntimeDirs`. On the LLVM row that field holds LLVM's own directories | author | the existing planning-time staging is kept; the toolset's redistributable directory becomes its own field | §7.3 | +| 4 | For a toolset without a redistributable directory, the text left open whether an explicit `toolchain-coupled` would be downgraded, which is E10's shape again | reviewer and author | a defaulted contract resolves to `host-coupled`, recorded once; an explicit one is refused | §7.3 | +| 5 | W4 changes what SPEC-005 R5.2 means for checks that depend on a build program's directives; the plan called this "an example" | author | R5.2 is named as amended | §11 | +| 6 | W1 could have carried root-position keys (`[toolchain]`, `[target.]`, `[indices]`) to members reached by path | author | the scope is stated, and the criterion checks the member's toolchain and flags | §2.3, §2.4 | +| 7 | A dependency's inert `dialect_cxxflags` enters its fingerprint. W1 would have made more members carry one, and each such member would be rebuilt once for nothing | reviewer | W8: a key enters a fingerprint only where it reaches a command | §6.2 | +| 8 | Refusing an ambiguous `-p` would break a command that works today | reviewer | revised in revision 3 after the reviewer's D1 answer. The option names a package, so the package identity is resolved first; directory spellings remain a fallback, and the one layout where the two disagree selects the package with a warning. A short name shared under two namespaces is refused, since no earlier behaviour existed there to preserve | §2.3 | +| 9 | "Every criterion fails before the change" is false for a refactor. W7 also allowed the function limit to survive as prose | reviewer | W7's failing criterion is the parsed size gate; without the gate, the limit is dropped rather than written down | §8, §11 | +| 10 | The plan cited docs/20 as already scoping the PE shared-library default to the GNU ABI; the table says only "PE (Windows)" | reviewer | stated as a planned amendment | §7.3 | +| 11 | Removing device units from the plan's compile units could break a consumer that pairs them with graph units by position | author | checked. The only index uses are within the list (`plan.cppm:1914`, `prepare/plan.cpp:1790`, `:1832`). The dependency cache collects a package's units by path, so a dependency with device sources changes its artifact set; the implementation verifies this | §4.1 | +| 12 | A progress bar drawn during `emit` could corrupt the document on stdout | author | checked: `emit` redirects stdout to stderr while planning (`cmd_build.cppm:377`) | §9.1 | + +### 13.2 Properties the plan keeps, checked across all steps + +- **One home for each item, and the engine names no tool.** + - W6 compares bytes. + - W10 describes actions without naming their tools. + - W9 speaks of ABIs and toolset directories. + - W11 renders events from any producer. + + No knowledge of Qt, vcpkg or CMake enters the engine. The only step outside the + engine is xlings's progress events, and only if a measurement shows they are + missing. +- **No slot for exceptions.** + - W6 replaces a proxy check with the invariant. + - W9 has one derivation and one word for each driver. + - W12 removes a false error instead of silencing it. + - W7's limit is either parsed or dropped. +- **An explicit statement outranks a default wherever a default is introduced or + changed:** + - W9's pack mode, and its rows without a redistributable directory; + - W6's declared deploys over the toolchain's derived staging. +- **Planning stays pure.** + - W5 removes the one write into the project. + - W10 describes generated outputs and runs nothing. + - W3 and W4 change only what is described and reported. +- **An index is data.** After W12 the floor appears in exactly three places: + - the cause in a failing run; + - a closing tip in a run that refreshed an index; + - `mcpp doctor`. + + It is never an error in a run that succeeds. +- **Upgrades are announced.** + - Every behaviour change is listed in §11's compatibility notes, with its + one-line remedy. + - Three refusals are added, and all three are named there: + - W9: a contradicting CRT word; + - W9: an explicit `toolchain-coupled` without a redistributable directory; + - W1: a package name that is ambiguous across namespaces. + - The only selection that changes is W1's `-p`, where the option's own name + decides. The change is warned and announced. +- **The order holds.** + - W12 lands before W11, which reuses its reporter. + - W8 lands before W9, whose rule reads W8's conditional flags. + - W10 follows W3 and waits for the S1 text. + - W7 splits code that stage 1 has already changed. + +### 13.3 Owed before the release, not open in the design + +- **The Windows leg:** the criteria of §7.3 (W9) and §5.3 (W6's one writer), and + a `mcpp test` run with no Visual Studio directory on `PATH`. +- **Whether `xim:msvc` carries `VC\Redist`.** +- **The acquisition matrix of W11**, before and after, and whether xlings's + `update_packages` emits `download_progress` for an index sync. +- **The S1 text for W10**, from the mcppls maintainers (D6). + +### 13.4 Second self-review (revision 3) + +**Method.** After the reviewer's answers, the author read the whole document again +from beginning to end. The pass checked three things: + +1. every section against the settled decisions; +2. every step against its criterion and its specification change; +3. the counts and claims in the summaries against the sections they summarise. + +**What changed.** + +| # | Finding | Change | Where | +|---|---|---|---| +| 1 | W10 had no criterion | the generated-header record is checked against the path a following `mcpp build` writes | §4.4 | +| 2 | The one-writer rule of W6 had no criterion, and SPEC-007 R4.3 was not listed as changed | a Windows-leg criterion; R4.3 added to §11 | §5.3, §11 | +| 3 | The `mcpp doctor` check of W12 had no criterion | a criterion, which fails on 2026.9.27.1 | §9.2 | +| 4 | The package-first `-p` of D1 adds a refusal: a short name shared under two namespaces. §13.2 said no other refusal was added | the refusal is listed in §11 and counted in §13.2 | §11, §13.2 | +| 5 | Duplicate directory basenames are resolved silently to the first member today. Refusing them would break working commands | the first member stays selected, and a warning names the others | §2.3 | +| 6 | §13.2 listed `-p` under "an explicit statement outranks a default". The resolution order is a question of the option's documented meaning, not of a default | the item moved to the upgrade bullet | §13.2 | +| 7 | §6.1 said `[workspace.build] dialect_cxxflags` reaches only the root member. `inherit_workspace_build` also prepends it into every member pulled in as a path dependency | corrected; the cache-key claim of the fingerprint rule is cited (`cache_key.cppm:323`) | §6.1, §6.2 | +| 8 | E10's removal was stated as "every row delivers every value", which the refusal for a row without a redistributable directory contradicts | reworded | §7.3 | +| 9 | The scope counted eleven items for "#724 carries four" alone. The basis omitted the 2026.9.18.1 binary used for §9.2 | corrected | §0, basis | + +**Checked and unchanged.** + +- **Decisions against sections.** Every settled decision (§12) matches the + section it names. +- **Ledger against order.** Each of W1 to W12 appears in both the ledger (§1) and + the order (§11). +- **Criteria.** Every step W1 to W6 and W8 to W12 now has a criterion that fails + on 2026.9.27.1. W7's failing criterion is its parsed size gate. +- **Routing.** No step moved between homes in this pass. + +### 13.5 Readings during implementation (revision 4) + +Each reading below corrected the design or the implementation; none left an open +question in the round. + +| # | Reading | Consequence | Where | +|---|---|---|---| +| 1 | The stray `[1/7] awesome::xim.lua` text on xlings's NDJSON stream comes neither from xlings's code nor from libxpkg's. It comes from the `pkgindex-build.lua` script that ships inside an index repository, which libxpkg runs with the real `io` library | xlings captures fd 1 around every interface capability (`platform::StdoutCapture`) and turns a recognised `[i/n]` line into a `progress` event; the fix does not depend on the one script (openxlings/xlings#625) | §9.1 | +| 2 | The refresh's `progress` events are per repository (`index_sync`) and per descriptor file (`index_rebuild`) | mcpp draws one bar per phase, and one per repository for a sync, so that output off a terminal is a few lines, not one per file | §9.1 | +| 3 | A differential run of the Linux e2e suite, fresh binary against 2026.9.27.1 under the same environment, found one regression: e2e 797 re-ran the DLL placement on every build after the first | The plan's deploy set reads runtime search directories that a `prepare` action fills, so it differs between the first plan and the second. W6's first form wrote the deploy names into the `place-dlls` command, which then changed. `place-dlls` now reads the other writer's DLLs from the directory at run time, and its command is the one 2026.9.27.1 wrote | §5.2 | +| 4 | SPEC-005 R5.2 allows a failed member exactly one error diagnostic | W4 folds the notes recorded before a phase failure into that one diagnostic's message instead of adding diagnostics | §4.2 | +| 5 | Matching `[target.]` tables merge in the lexical order of the selector text, not in manifest order. The TOML table carries no key order, so SPEC-004's "manifest order" cannot be implemented by a conforming reader | Out of this round: SPEC-004 §3.1.1 is marked partially implemented and mcpp#728 tracks the rule | §6.2 | +| 6 | #727's own change moved `mcpp.toml` to 2026.9.27.2 but not `MCPP_VERSION`, which failed four CI jobs | The round moves both places together; the release is 2026.9.28.1 | §9.3 | +| 7 | xlings's `update` CLI and `interface update_packages` are one function (`xim::cmd_update`) | Routing the refresh through the interface changes no behaviour of the refresh itself | §9.1 | +| 8 | Windows CI's e2e default row is the LLVM row: e2e 703 asserts rather than skips | e2e 814's skip branch for other rows is a safety net, not the path CI takes | §7.3 | + +### 13.6 Global review and the first CI run (revision 5) + +Three read-only reviews covered the whole change from three angles +(specification and architecture; stability and cross-platform; compatibility, +user experience and simplicity). The first CI run of the integrated branch +covered every platform. The table lists what each found and what changed. + +| # | Source | Finding | Change | +|---|---|---|---| +| 1 | CI, Windows | e2e 190 and 191 relinked `bin/`; on an MSVC-ABI row `bin/` also holds the staged redistributable DLLs (§7), so the first file was a DLL and no response file was written | the tests select the program by name | +| 2 | CI, Windows | e2e 811: the plan-time scan of runtime search directories added the dependency's `libmathkit.dll` as a second source of the declared deploy, and `mcpp stage` refused the two different files | a DLL a search directory offers yields to a declared destination (SPEC-007 R4.3); the difference is warned at planning, because a successful build discards edge output; e2e 818 carries 811's case on Linux through the mingw cross toolchain | +| 3 | CI, Linux (GCC) | e2e 807: a `build.mcpp` that imports only a build rule compiled in the project root and could not find the rule's BMI in `gcm.cache`. The defect predates the round; the fixture is the first to reach it on GCC | a build program that imports any module compiles in the build directory | +| 4 | CI, Linux | e2e 205: W5 wrote the runtime-environment half of `.xlings.json` at `workRoot`, which for a workspace member is not the runtime's owner | two roots again; only `plan_only` redirects the owner's half to the planning directory | +| 5 | Specification review | `ide.generated` file records went to the bare package set only, so a test set whose unit includes a generated header carried none | every set of the package names the outputs (R3.12 states it) | +| 6 | Stability review | `check_crt_word` read the root's `cxxflags` only; a dependency's word reaches its own units after the graph's | every package's `cxxflags` is checked; a dependency's agreeing word is not warned, because the replacing key is the root's | +| 7 | Compatibility review | `/MDd` and `/MTd` were bucketed with `/MD` and `/MT`, so an agreeing debug word was called redundant | a debug CRT word is refused: the model has no debug axis, and the std module and the link use the release CRT | +| 8 | Compatibility review | the dependency cache key had no CRT axis; switching `cxx_runtime` within one release could be served objects of the other model | the key carries the CRT word from the helper the flag builder uses | +| 9 | Stability review | deploy destinations compared case-sensitively on PE | `DeployFile::is_destination` folds case on a PE target, used by both deploy comparisons | +| 10 | Stability review | an index refresh that failed mid-run drew its bar as done | the renderer finishes the open bar as not completed; an automatic refresh that exhausts its retries warns | +| 11 | Compatibility review | the floor tip named neither the version nor the install-aware upgrade | the guard records each refreshed tree's `min_mcpp`; one helper gives the advice E0006 gives | +| 12 | Stability and compatibility reviews | `-p` compared member paths as strings; the help omitted the qualified form; the newest redistributable was picked by string order; a missing deploy source read as a disagreement | each corrected, with a unit test | + +**Not changed, with the reason.** The second NDJSON `download_progress` parser +predates the round (the round only extracted it into one helper for the two +xlings-side readers); unifying it with the fetcher's is a refactor across a +module boundary that the round does not need. A UNC `MCPP_HOME` fails loudly under +`cd /d`, which is the correct shape for an unsupported home, not a silent one. + +**The method's blind spot.** The differential run of §13.5 item 3 compared the +fresh binary with the released one in one environment. e2e 205 and 807 failed +there on both binaries, for reasons of the environment (an unset `MCPP_HOME`; +the machine's default toolchain), so the differential read them as unchanged. +A test that fails on both sides measures nothing about the change; its reading +has to come from an environment in which it passes, which here is CI. + +**#722, completed after the first CI run.** The first pass split the seven named +functions and three more the gate found (10 findings at the base, 6 after it). +The second pass split the six that remained (`step6_provision_host_tools`, +`loadVersionDep`, `phase11_scan`, `step1_target_and_static_overrides`, +`phase2_define_toolchain_resolver`, `step9_kernel_abi_interfaces_and_requirements`), +each with a phase-local context struct where several of its steps share locals. +Readings on the final tree: the gate reports no function over 400 lines; the +seven fixtures are byte-identical to the pre-split binary; AddressSanitizer +reports nothing over the fixtures and a 20-script e2e subset; the unit suite +passes. The gate was wired into `ci-linux.yml`'s LLVM toolchain job, after the +step that builds mcpp with llvm@20.1.7, and crashed there: that build has +never completed (libc++ 20's `std` module hides `directory_iterator`'s +comparison), and the step reads the resolution line rather than the build's +exit status, so the job reports success on `main` too. The step came out again, +the gate's finding filter was narrowed to diagnostic lines (the crash dump named +the check and every file and had read as a finding), and #729 records the step +and the wiring. The gate resolves clang-tidy from `xim:llvm-tools` at the +version that wrote the compile database, because it reads that clang's BMIs, +and is run by hand after `mcpp build --toolchain llvm@22.1.8`. The merge +of the first pass took `plan.cpp` to 2,505 lines, over the file gate; the +records half of P13 (`mcpp.lock`, `resolution.json`) moved to `records.cpp`, +byte-identical on the same fixtures. diff --git a/.agents/docs/2026-09-27-eight-reports-implementation-plan.md b/.agents/docs/2026-09-27-eight-reports-implementation-plan.md new file mode 100644 index 000000000..0dc6dc285 --- /dev/null +++ b/.agents/docs/2026-09-27-eight-reports-implementation-plan.md @@ -0,0 +1,115 @@ +--- +subject: plan +status: active +--- + +# Eight reports after 2026.9.27.1: implementation plan + +This record implements `2026-09-27-eight-reports-by-home-and-one-optimisation-plan.md` +(the design, revision 3, all decisions settled). The design fixes what is built. This +record fixes the following: + +- the order; +- which files each task owns; +- the repositories involved and the order of their releases; +- how each step is verified. + +## 1. Readings that shaped the plan + +- **xlings emits no progress events for an index sync (measured).** The command is + `xlings interface update_packages --args '{}'` (xlings 2026.9.27.1). + - It emits heartbeats and one result, and no progress event. + - It also writes its terminal progress text (`[1/7] awesome::xim.lua` followed by + an erase sequence) onto the NDJSON stream. That text is not JSON, and the + xlings interface protocol (`docs/spec/interface-ndjson-v1.md`) does not allow + it. + - W11 therefore needs an xlings change and an xlings release before the mcpp + release. +- **mcpp-language-server.** speak-agent has read access only, so its S1 change is + proposed from a fork. +- **The next e2e number is 805.** Unit tests live in `tests/unit/`. + +## 2. Repositories and their order + +| Order | Repository | Pull request | Content | Release | +|---|---|---|---|---| +| 1 | openxlings/xlings | one | interface mode emits `progress` events for an index sync and keeps terminal text off the NDJSON stream | yes; the date version of the day | +| 2 | Sunrisepeak/mcpp-language-server | one, from a fork | S1: the generated-output record (design §4.4, D6) | no, a specification only | +| 3 | mcpp-community/mcpp | one: #727, which also carries #726's fix (W13) | W1 to W13, docs, specs, CHANGELOG, version, xlings pin | yes | +| 4 | openxlings/xim-pkgindex | the bot's bump pull request | mcpp's new version | merged by a maintainer account | +| 5 | mcpplibs/mcpp-index | one, if its CI pin or `latest_mcpp` must move | index consumer pins | no release; the index publishes on merge | + +The mcpp pull request pins the xlings release of row 1 (`kXlingsVersion`), and the +release pull request carries that pin. + +## 3. Tasks, owners and dependencies + +The work uses one integration branch, `feat/eight-reports`, in the worktree +`mcpp-eight`. Each task has its own worktree, branched from the integration +branch, and is merged back when its criteria pass. + +| Task | Steps | Files owned (smallest hunks elsewhere) | Depends on | +|---|---|---|---| +| T1 | W1 | `src/build/prepare/manifest.cpp`, `src/project.cppm` (member resolution), `src/cli.cppm` (`-p` help), `docs/07` (en, zh), SPEC-004 §9, `tests/unit/test_workspace_inheritance.cpp`, e2e 805 and 806 | none | +| T2 | W2 | `src/build/prepare/features.cpp` (host-module unit order), e2e 807 | none | +| T3 | W3, W4, W5 | `src/build/plan.cppm` (the unit loop only), `src/build/prepare/target_side.cpp` (the device-source check), `src/build/prepare/driver.cpp`, `src/build/prepare/xlings.cpp` (the project index file), `src/cli/cmd_build.cppm` (the emit failure path), SPEC-005, e2e 688 extended, e2e 808 and 809 | none | +| T4 | W6 | `src/build/plan.cppm` (`add_deploy` only), `src/build/stage.cppm`, `src/cli/cmd_build.cppm` (`cmd_stage` only), `src/build/ninja_backend.cppm` (the stage and `place_dlls` edges), `src/pack/pack.cppm` (`place_runtime_dlls`), SPEC-007 R4.2 and R4.3, e2e 810 and 811 | none | +| T5 | W12 | `src/pm/package_fetcher.cppm`, `src/pm/index_contract.cppm`, `src/xlings/xlings.cppm` (`update_index` reporting), `src/ui.cppm` (closing notices), `src/doctor.cppm`, `docs/09` and `docs/50`, e2e 185 updated, e2e 812 | none | +| T6 | W7 | `src/build/prepare/*.cpp` (phase functions), `.github/tools/` (the size gate), `tests/unit/test_prepare_helpers.cpp` | every other task merged (the last step; see the design, §11) | +| T7 | W8 | `modules/manifest/src/toml.cppm`, `modules/manifest/src/types.cppm`, `src/build/prepare/scan.cpp` and `target_side.cpp` (dialect resolution), `src/build/prepare_inputs.cppm`, SPEC-004 §3.1 and §9, e2e 813 | T1 to T5 merged | +| T8 | W9 | `modules/toolchain-model/src/dialect.cppm`, `src/build/flags.cppm`, `src/build/prepare/scan.cpp` (std-module CRT), `src/build/distribution.cppm`, the toolchain redistributable field (`src/toolchain/msvc.cppm`, the LLVM row's sysroot resolution), `src/pack/pack.cppm` (contract), `docs/20` and `docs/04`, unit tests, e2e 814 (Windows) | T1 to T5 merged; its free-form word rule reads T7's list at merge | +| T9 | W10 | `src/build/build_database.cppm`, SPEC-005 §3, e2e 815 | T3; the S1 text (Sunrisepeak/mcpp-language-server#28) | +| T10 | W11 | `src/ui.cppm` (terminal and non-terminal rendering), `src/xlings/xlings.cppm` (index refresh through the interface), the git fetch in `src/build/prepare/fetch.cpp` and `graph.cpp`, the sandbox bootstrap, `docs/09`, unit tests, e2e 816 | T5; the xlings change (X1), with its release before mcpp's | +| X1 | xlings | `openxlings/xlings`: the interface event stream for `update_packages` | none | +| L1 | mcppls | `docs/specs` S1 addition | none | + +T1 to T5, X1 and L1 have no dependency on one another. At most three subagents run +at once. The author takes T2 and the merges, and runs the integration build and +the full test suites. + +**Rules for parallel work.** These come from the 2026-09-12 and 2026-09-26 records. + +- **No global configuration change.** No task changes `~/.mcpp/config.toml` or any + other global configuration. A toolchain is selected per fixture or per command. +- **No broad `pkill -f`.** No task kills processes by a broad `pkill -f` pattern. +- **One build per worktree.** No two builds run in one worktree at once. +- **Clean up after merging.** A merged task's `target/` is removed. + +## 4. Verification + +**Per task.** + +- The fresh binary passes `mcpp test` and the task's own e2e scripts. +- Each new criterion is also run with the fix removed, and must then fail. + +**Integration.** + +- A full `mcpp test`. +- The e2e suite on Linux, through `tests/e2e/run_all.sh` with the fresh binary. +- The golden fixtures of the #719 decomposition. +- CI on every platform through the one pull request. + +**After the release.** + +- **A sandbox.** `xlings subos new eight`, then `xlings subos use eight --sandbox + --cmd ...`, with both mcpp and xlings on the CN mirror. The sandbox installs the + released mcpp by its release path and runs one probe per step. The probe is + passed in as base64, and each probe directory is removed at the start of its + section. +- **A control.** The same script runs against 2026.9.27.1, where exactly the fixed + criteria must fail. +- **The index ecosystem.** mcpp-index's validation sweep runs against the new + release. + +## 5. Release + +The version is the date version of the release day. The xlings pin moves to the +xlings release of row 1. + +1. After the release workflow starts, every archive and its sidecar are uploaded + to GitCode with the local `gtc` as soon as each appears on the GitHub release. +2. Each GitCode asset is verified by a GET with a byte comparison. +3. The xim-pkgindex bump pull request is merged with the maintainer account, and + its state is read back afterwards. +4. The release is complete when `pkgs/m/mcpp.lua` on the index's `main` has + `latest` pointing at the release. diff --git a/.agents/docs/README.md b/.agents/docs/README.md index 483783b11..c7b063685 100644 --- a/.agents/docs/README.md +++ b/.agents/docs/README.md @@ -18,7 +18,7 @@ superseded_by: 2026-09-07-....md # when status is superseded --- ``` -311 records. +313 records. ## By subject @@ -56,6 +56,7 @@ Records that declare one. Everything else is listed by date below. ### plan +- [Eight reports after 2026.9.27.1: implementation plan](2026-09-27-eight-reports-implementation-plan.md) — active - [#690: implementation plan](2026-09-25-issue-690-implementation-plan.md) — landed - [工具链选择与载荷可信度：实施计划](2026-09-24-toolchain-selection-implementation-plan.md) — landed - [openkal 生态：完整性收尾与验收方案](2026-09-21-openkal-ecosystem-completion-and-acceptance.md) — active @@ -82,6 +83,7 @@ Records that declare one. Everything else is listed by date below. ### triage +- [Eight reports after 2026.9.27.1: what each one is, where it belongs, and one optimisation plan](2026-09-27-eight-reports-by-home-and-one-optimisation-plan.md) — active - [#685、#687 与 Windows clang 的 MSVC STL：三个问题的归属，以及工具链载荷的规范化](2026-09-24-685-687-msvc-stl-and-toolchain-payloads.md) — landed - [运行时绑定方案 v3:让 mcpp 真正安装它所声明的运行时](2026-09-17-runtime-binding-multi-repo-plan.md) — landed - [#662:目标侧由依赖图提供时，编译器的隐式头文件搜索仍指向宿主](2026-09-17-issue-662-graph-target-header-isolation-plan.md) — active @@ -102,6 +104,8 @@ Records that declare one. Everything else is listed by date below. ### 2026-09 +- [Eight reports after 2026.9.27.1: implementation plan](2026-09-27-eight-reports-implementation-plan.md) — active +- [Eight reports after 2026.9.27.1: what each one is, where it belongs, and one optimisation plan](2026-09-27-eight-reports-by-home-and-one-optimisation-plan.md) — active - [The compile database, `emit build-database`, and #701/#702: triage against the specifications, and one design](2026-09-26-compile-database-and-issue-699-design.md) — landed - [Issues #693 to #696: triage against mcpp's contracts, and one repair plan](2026-09-25-issues-693-696-triage-and-repair-plan.md) — landed - [Workspace inheritance, flag scoping and the published form: a unified repair plan (#690)](2026-09-25-issue-690-workspace-build-inheritance-consistency.md) — landed diff --git a/.github/actions/bootstrap-mcpp/action.yml b/.github/actions/bootstrap-mcpp/action.yml index 2784d94b3..1eb3332aa 100644 --- a/.github/actions/bootstrap-mcpp/action.yml +++ b/.github/actions/bootstrap-mcpp/action.yml @@ -25,7 +25,7 @@ inputs: # `package.name`, so one of the two was simply unreachable — and which one # depended on the machine, which is why CI failed on `compat:lua` on # Windows and `mcpplibs.capi:lua` on Linux. Never pin below that. - default: '2026.9.27.1' + default: '2026.9.28.1' cache-target: description: also restore/save target/ (build artifacts + BMIs) required: false diff --git a/.github/actions/setup-macos-llvm/action.yml b/.github/actions/setup-macos-llvm/action.yml index b70e47469..f62c6a249 100644 --- a/.github/actions/setup-macos-llvm/action.yml +++ b/.github/actions/setup-macos-llvm/action.yml @@ -15,7 +15,7 @@ inputs: # Floor imposed by the index, not a routine bump — see # .github/actions/bootstrap-mcpp/action.yml for why 0.4.69 is required # (two packages named `lua` in one repo need openxlings/xlings#381). - default: '2026.9.27.1' + default: '2026.9.28.1' image: description: > The runner label the job runs on (macos-15, xcode-27). It is part of the diff --git a/.github/tools/check_function_sizes.sh b/.github/tools/check_function_sizes.sh new file mode 100755 index 000000000..6f8f45137 --- /dev/null +++ b/.github/tools/check_function_sizes.sh @@ -0,0 +1,184 @@ +#!/usr/bin/env bash +# +# Guard: no function under the prepare.cppm decomposition grows past ~400 +# lines (mcpp-community/mcpp#722, T6 of the 2026-09-27 round). +# +# WHY +# +# check_file_lengths.sh caps each FILE at 2,500 lines. It says nothing about +# a single FUNCTION inside a file that stays under the cap while one phase +# function alone climbs back past a thousand lines and closes back over the +# ~180-local shape prepare.cppm was split to remove in the first place (see +# that script's own header, and the layout comment atop src/build/prepare.cppm). +# #722 split the seven functions that had grown past ~400 lines into +# sub-steps named after the sections their own banners already used; this +# gate is what keeps a phase function from quietly growing back into one. +# +# THE RULE +# +# Every function defined in a file directly under src/build/prepare/ (or in +# src/build/prepare.cppm itself) stays at or under LINE_THRESHOLD lines, as +# clang-tidy's readability-function-size check counts them (its own count, +# not a text-heuristic line counter -- a brace-counting or regex-based +# stand-in cannot tell a function's extent from a `{`/`}` pair inside a +# string literal or a designated initializer, both common in this codebase's +# std::format calls and manifest structs; see .agents/docs/ +# 2026-09-27-eight-reports-by-home-and-one-optimisation-plan.md §8). +# +# WHAT THIS NEEDS +# +# A compile database that names BMIs explicitly (-fmodule-file=...), which +# only a build actually produces: `mcpp build --toolchain llvm@22.1.8` writes +# compile_commands.json at the project root. This script does not build it: +# the caller runs that build first. check_file_lengths.sh needs no such +# division because it reads the tree. +# +# NOT IN CI YET. The only CI job that builds mcpp with clang (ci-linux.yml, +# "toolchain: musl + llvm", llvm@20.1.7) does not produce a complete build: +# libc++ 20's `std` module does not make directory_iterator's comparison +# visible, and that step reads the resolution line rather than the build's +# exit status. Over the partial database clang-tidy crashes. The gate is wired +# in once a CI job builds mcpp with clang (mcpp-community/mcpp#729); until +# then it is run by hand after `mcpp build --toolchain llvm@22.1.8`. +# +# clang-tidy itself is not part of the plain xim:llvm payload mcpp resolves +# for `--toolchain llvm@...` (measured: xim-x-llvm/22.1.8/bin has clang, +# clang-scan-deps and the LLVM binutils, no clang-tidy). It ships in the +# sibling package `xim:llvm-tools` at the same version -- resolved and +# searched for under the xlings package store; install it with +# `xlings install xim:llvm-tools@` +# if this script cannot find it. +# +# Usage: bash .github/tools/check_function_sizes.sh [repo_dir] + +set -uo pipefail + +REPO_DIR="${1:-$(pwd)}" +cd "$REPO_DIR" || { echo "FAIL: cannot cd to $REPO_DIR" >&2; exit 1; } + +LINE_THRESHOLD=400 +DIR="src/build/prepare" +PRIMARY="src/build/prepare.cppm" +CDB="compile_commands.json" + +[ -d "$DIR" ] || { echo "FAIL: $DIR does not exist -- this guard has gone stale" >&2; exit 1; } + +if [ ! -f "$CDB" ]; then + cat >&2 </bin/clang++`) and looked up in either xlings store. +cdb_llvm_version() { + grep -o 'xim-x-llvm/[0-9][0-9.]*/bin/clang' "$CDB" 2>/dev/null | head -1 \ + | sed 's|xim-x-llvm/\([0-9.]*\)/bin/clang|\1|' +} +find_clang_tidy() { # $1 = the llvm version + local root + for root in "${MCPP_HOME:-$HOME/.mcpp}/registry/data/xpkgs" "$HOME/.xlings/data/xpkgs"; do + [ -x "$root/xim-x-llvm-tools/$1/bin/clang-tidy" ] \ + && { echo "$root/xim-x-llvm-tools/$1/bin/clang-tidy"; return 0; } + done + return 1 +} + +if [ -n "${CLANG_TIDY:-}" ]; then + [ -x "$CLANG_TIDY" ] || { echo "FAIL: CLANG_TIDY=$CLANG_TIDY is not executable" >&2; exit 1; } +else + LLVM_VERSION="$(cdb_llvm_version)" + CLANG_TIDY="$( [ -n "$LLVM_VERSION" ] && find_clang_tidy "$LLVM_VERSION" )" || { + cat >&2 <} +EOF + exit 1 + } +fi + +# The files this database actually has entries for, restricted to the +# decomposition's own directory (plus the primary interface, if it is ever +# given its own compiled entry point -- it has none today, since it defines +# only declarations and inline exports; the loop below tolerates that). +mapfile -t FILES < <(python3 - "$CDB" "$DIR" "$PRIMARY" <<'PYEOF' +import json, sys +cdb_path, dirname, primary = sys.argv[1], sys.argv[2], sys.argv[3] +with open(cdb_path) as f: + entries = json.load(f) +seen = set() +for e in entries: + path = e["file"] + if f"/{dirname}/" in path or path.endswith(f"/{primary}"): + seen.add(path) +for p in sorted(seen): + print(p) +PYEOF +) + +if [ "${#FILES[@]}" -eq 0 ]; then + echo "FAIL: $CDB has no entry under $DIR -- was it built with a matching source tree?" >&2 + exit 1 +fi + +echo "checking ${#FILES[@]} file(s) with $CLANG_TIDY (LineThreshold=$LINE_THRESHOLD)..." + +OUT="$(mktemp)" +trap 'rm -f "$OUT"' EXIT + +# No --warnings-as-errors: the only check enabled is readability-function-size +# itself, and a finding in json.hpp (bundled third-party, reached through one +# of these files' imports) would then make clang-tidy exit non-zero on every +# run regardless of this decomposition's own state -- exactly the ambiguity +# the "diagnostic tool problem" branch below exists to catch, and it cannot +# tell the two apart from an exit code alone. The `relevant` filter is the +# sole pass/fail signal; clang-tidy's own exit code is read only as a sign +# that the tool itself failed to run (a bad compile command, a crash), which +# a plain warning never produces. +"$CLANG_TIDY" \ + --checks='-*,readability-function-size' \ + --config="{CheckOptions: {readability-function-size.LineThreshold: '$LINE_THRESHOLD'}}" \ + -p "$REPO_DIR" \ + "${FILES[@]}" > "$OUT" 2>&1 +rc=$? + +# Only findings inside the decomposition's own directory gate the build: a +# bundled third-party header (e.g. modules/libs/src/json/json.hpp) reached +# through one of these files' imports is not this decomposition's to fix. +# A finding is a diagnostic line, which ends with the bracketed check name; a +# crash dump also names the check (in its program arguments) together with +# every file path, and must not read as a finding. +relevant=$(grep -E '\[readability-function-size\]$' "$OUT" | grep -F -e "/$DIR/" -e "/$(basename "$PRIMARY")" || true) + +if [ -n "$relevant" ]; then + echo "$relevant" >&2 + echo >&2 + echo "FAIL: function(s) over $LINE_THRESHOLD lines under $DIR -- see above." >&2 + echo " Split at the sub-section boundaries its own banners already name" >&2 + echo " (mcpp-community/mcpp#722's own method), the way phase13_finish," >&2 + echo " phase4b_graph_worklist, phase6_features_and_host_tools and" >&2 + echo " phase9_target_side were split." >&2 + exit 1 +fi + +if [ "$rc" -ne 0 ]; then + echo "FAIL: clang-tidy exited $rc with no readability-function-size finding under $DIR" >&2 + echo " (a diagnostic tool problem, not a function-size one -- see the log):" >&2 + cat "$OUT" >&2 + exit 1 +fi + +echo "ok: no function under $DIR (or $PRIMARY) exceeds $LINE_THRESHOLD lines" +exit 0 diff --git a/.github/workflows/bootstrap-macos.yml b/.github/workflows/bootstrap-macos.yml index d1ac92b6d..9ae64d582 100644 --- a/.github/workflows/bootstrap-macos.yml +++ b/.github/workflows/bootstrap-macos.yml @@ -17,7 +17,7 @@ jobs: # Dormant (workflow_dispatch only), but kept in step with the rest — # check_version_pins.sh holds it there. Floor: 0.4.69, below which the # index cannot resolve two packages that share a short name. - XLINGS_VERSION: '2026.9.27.1' + XLINGS_VERSION: '2026.9.28.1' steps: - uses: actions/checkout@v4 diff --git a/.github/workflows/ci-fresh-install.yml b/.github/workflows/ci-fresh-install.yml index 30f695682..7957fe434 100644 --- a/.github/workflows/ci-fresh-install.yml +++ b/.github/workflows/ci-fresh-install.yml @@ -152,7 +152,7 @@ jobs: env: XLINGS_NON_INTERACTIVE: '1' run: | - curl -fsSL https://raw.githubusercontent.com/openxlings/xlings/main/tools/other/quick_install.sh | bash -s v2026.9.27.1 + curl -fsSL https://raw.githubusercontent.com/openxlings/xlings/main/tools/other/quick_install.sh | bash -s v2026.9.28.1 echo "$HOME/.xlings/subos/current/bin" >> "$GITHUB_PATH" - name: Install mcpp and config mirror @@ -312,7 +312,7 @@ jobs: - name: Install xlings + mcpp run: | - curl -fsSL https://raw.githubusercontent.com/openxlings/xlings/main/tools/other/quick_install.sh | bash -s v2026.9.27.1 + curl -fsSL https://raw.githubusercontent.com/openxlings/xlings/main/tools/other/quick_install.sh | bash -s v2026.9.28.1 # Deliberately NOT writing to $GITHUB_PATH here. On container # images that declare no PATH in their config (opensuse/ # tumbleweed), appending a single dir to GITHUB_PATH makes the @@ -403,7 +403,7 @@ jobs: # (older ones carry minos=15 and refuse to start). # v0.4.51+: in-process sha256 — this image has no sha256sum # binary, so pinned fetches failed before it. - curl -fsSL https://raw.githubusercontent.com/openxlings/xlings/main/tools/other/quick_install.sh | bash -s v2026.9.27.1 + curl -fsSL https://raw.githubusercontent.com/openxlings/xlings/main/tools/other/quick_install.sh | bash -s v2026.9.28.1 echo "$HOME/.xlings/subos/current/bin" >> "$GITHUB_PATH" - name: Install mcpp and config mirror diff --git a/.github/workflows/ci-linux-e2e.yml b/.github/workflows/ci-linux-e2e.yml index adfb4bbcc..d3f7d8627 100644 --- a/.github/workflows/ci-linux-e2e.yml +++ b/.github/workflows/ci-linux-e2e.yml @@ -384,7 +384,7 @@ jobs: - name: Bootstrap xlings + released mcpp run: | - curl -fsSL https://raw.githubusercontent.com/openxlings/xlings/main/tools/other/quick_install.sh | bash -s v2026.9.27.1 + curl -fsSL https://raw.githubusercontent.com/openxlings/xlings/main/tools/other/quick_install.sh | bash -s v2026.9.28.1 export PATH="$HOME/.xlings/subos/current/bin:$PATH" xlings update xlings install mcpp -y -g diff --git a/.github/workflows/cross-build-test.yml b/.github/workflows/cross-build-test.yml index fd048e898..080f46b7c 100644 --- a/.github/workflows/cross-build-test.yml +++ b/.github/workflows/cross-build-test.yml @@ -135,7 +135,7 @@ jobs: # release assets were uploaded in a broken state (records present, # blobs missing → 404 on GET); re-uploaded clean. The stale-INDEX # half is handled by the marker-clear below. - XLINGS_VERSION: '2026.9.27.1' + XLINGS_VERSION: '2026.9.28.1' run: | tarball="xlings-${XLINGS_VERSION}-linux-x86_64.tar.gz" bash "$GITHUB_WORKSPACE/.github/tools/fetch_release.sh" \ @@ -289,7 +289,7 @@ jobs: - name: Bootstrap mcpp via xlings env: XLINGS_NON_INTERACTIVE: '1' - XLINGS_VERSION: '2026.9.27.1' + XLINGS_VERSION: '2026.9.28.1' run: | tarball="xlings-${XLINGS_VERSION}-linux-x86_64.tar.gz" bash "$GITHUB_WORKSPACE/.github/tools/fetch_release.sh" \ diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index 987c78bb9..bca387dfb 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -96,7 +96,7 @@ jobs: # Pin xlings to a known-good version. The upstream install # script always grabs `latest` (no version override), so we # download + self-install manually to avoid broken releases. - XLINGS_VERSION: '2026.9.27.1' + XLINGS_VERSION: '2026.9.28.1' run: | if [ ! -x "$HOME/.xlings/subos/default/bin/xlings" ]; then tarball="xlings-${XLINGS_VERSION}-linux-x86_64.tar.gz" @@ -314,7 +314,7 @@ jobs: - name: Bootstrap mcpp via xlings env: XLINGS_NON_INTERACTIVE: '1' - XLINGS_VERSION: '2026.9.27.1' + XLINGS_VERSION: '2026.9.28.1' run: | tarball="xlings-${XLINGS_VERSION}-linux-x86_64.tar.gz" bash "$GITHUB_WORKSPACE/.github/tools/fetch_release.sh" \ @@ -385,7 +385,7 @@ jobs: # below are pinned to the same version as XLINGS_VERSION; they are # NOT interpolated from it, so check_version_pins.sh scans for them # explicitly (they were absent from the old lock-step comment). - XLA="xlings-2026.9.27.1-linux-aarch64.tar.gz" + XLA="xlings-2026.9.28.1-linux-aarch64.tar.gz" # NOT fetch_release.sh: this asset is OPTIONAL and the `if` is the # point — an arch with no prebuilt xlings must fall through quietly, # while the helper retries a 404 five times before giving up. The one @@ -394,9 +394,9 @@ jobs: # cover it. if curl -fsSL --retry 3 --retry-delay 2 --retry-all-errors \ --connect-timeout 20 --max-time 600 -o "/tmp/$XLA" \ - "https://github.com/openxlings/xlings/releases/download/v2026.9.27.1/$XLA"; then + "https://github.com/openxlings/xlings/releases/download/v2026.9.28.1/$XLA"; then tar -xzf "/tmp/$XLA" -C /tmp - XLBIN=$(find /tmp/xlings-2026.9.27.1-linux-aarch64 -path '*/bin/xlings' -type f | head -1) + XLBIN=$(find /tmp/xlings-2026.9.28.1-linux-aarch64 -path '*/bin/xlings' -type f | head -1) if [ -n "$XLBIN" ]; then mkdir -p "$STAGING/$WRAPPER/registry/bin" cp "$XLBIN" "$STAGING/$WRAPPER/registry/bin/xlings" @@ -474,7 +474,7 @@ jobs: - name: Bootstrap mcpp via xlings env: XLINGS_NON_INTERACTIVE: '1' - XLINGS_VERSION: '2026.9.27.1' + XLINGS_VERSION: '2026.9.28.1' run: | if [ ! -x "$HOME/.xlings/subos/default/bin/xlings" ]; then WORK=$(mktemp -d) @@ -657,7 +657,7 @@ jobs: shell: bash env: XLINGS_NON_INTERACTIVE: '1' - XLINGS_VERSION: '2026.9.27.1' + XLINGS_VERSION: '2026.9.28.1' run: | # Captured before the `cd` below, in POSIX form: this step never # returns to the workspace, and GITHUB_WORKSPACE is a backslash diff --git a/CHANGELOG.md b/CHANGELOG.md index 0fc06010e..186dcd463 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -3,6 +3,92 @@ > 本文件追踪 `mcpp-community/mcpp` 公开仓的版本演进。 > 格式参考 [Keep a Changelog](https://keepachangelog.com/zh-CN/1.1.0/)。 +## [2026.9.28.1] - 2026-09-28 + +本版本合入 #717、#718、#720、#722、#723、#724、#725 与 #726 的修复与特性。设计与实施记录见 +`.agents/docs/2026-09-27-eight-reports-by-home-and-one-optimisation-plan.md` 与 +`.agents/docs/2026-09-27-eight-reports-implementation-plan.md`。 + +### 缺陷修复 + +- **带 `[package]` 的工作空间根的 `path` 依赖(#725)。** 这样到达的成员此前不被识别为成员: + 2026.9.26.1 忽略其 `[workspace.dependencies]` 中钉住的版本,2026.9.27.1 拒绝其 + `workspace = true`。现在工作空间的上下文由清单所在的位置决定,这样的成员按 SPEC-004 §9 第 1 条 + 继承 `[workspace.package]`、`[workspace.build]` 与 `workspace = true`;`[toolchain]`、 + `[target.]`、`[indices]` 仍只属于根。 +- **`-p, --package ` 按包的身份解析(#725)。** 顺序为限定名、包名,然后是成员的路径或 + 目录名(此前唯一的写法,保留)。同一个包名在两个命名空间下出现时拒绝并给出两个限定名;一个值 + 既是某成员的包名又是另一成员的目录名时选择前者并警告。 +- **宿主模块包的 lib root 参与本包单元的导入排序(#720)。** 导入同包其他单元的 lib root 此前 + 先于被导入者编译而失败。不导入同包单元的包顺序不变。 +- **规则认领的设备源不是编译单元(#724)。** 它此前出现在 S1 文档与 `mcpp build` 自己的 + `compile_commands.json` 中,带一条 C++ 编译命令,`build.ninja` 也带一条无人引用的边。 +- **构建程序失败时,其诊断得以保留(#724)。** 以构建程序的指令为前提的检查(设备源的认领)不再 + 对构建程序已失败的包运行,规划失败路径也保留已记录的说明;此前报告的是「设备源无人编译」。 +- **`emit build-database` 不写入工程目录(#724)。** 声明了 `[xlings]` 载荷的工程此前会得到 + `.mcpp/.xlings.json`(SPEC-005 R2.1)。 +- **一个放置目标一份内容、一个写入者(#723)。** 同一目标的多个来源在放置时逐字节核对,相同则 + 放置一份,不同则失败并点名全部来源;此前在规划时即被拒绝,即使内容相同。链接后放置 DLL 的步骤 + 不覆盖另一写入者放在程序旁的文件;规划时在运行时搜索目录中找到的同名 DLL 让位于声明的放置, + 内容不同时在规划时警告(SPEC-007 R4.2、R4.3)。PE 目标上目的地的比较不区分大小写。 +- **只导入构建规则的 `build.mcpp` 在 GCC 下可以编译。** 它此前在工程根目录中编译,找不到位于 + 构建目录 `gcm.cache` 中的规则 BMI;现在导入任一模块的构建程序都在构建目录中编译。 +- **要求更新 mcpp 的索引不再报告为错误。** 读取处不再打印 `error: ... [E0006]`;失败的运行在 + 使其停止的消息中给出 E0006;刷新了索引而遇到下限的运行在最后打印一行 `tip:`,信封中为说明 + `MCPP_INDEX_REQUIRES_NEWER_MCPP`;`mcpp self doctor` 列出当前 mcpp 不满足其下限的索引。 +- **Windows 上一次 xlings 调用只作用于 xlings 子进程(#726)。** 此前的 Windows 实现在两处与 + POSIX 不一致: + - 每次调用把 registry 的 `subos/default/bin` 加到进程 `PATH` 的最前面,并设置进程级的 + `XLINGS_HOME`,调用后不恢复。同一次构建安装过载荷后,ninja 与每个动作都先找到 `xim:llvm` + 注册的 `cl`、`link`、`lib`、`rc` shim,vcpkg 对宿主三元组的编译器检测因此失败。 + - xlings 在 mcpp 的工作目录中运行,从那里向上找到工程的 `.xlings.json` 而进入工程模式,把 mcpp + 工具链与载荷的 shim 写进工程的 SubOS。 + + 现在 `ScopedInvocationEnv` 在调用期间应用 `XLINGS_HOME`、作用域变量与 `PATH` 前缀并在调用后 + 全部恢复;命令以 `cd /d "" &&` 开头,与 POSIX 前缀中的 `cd` 相同。 + +### 特性 + +- **条件化的 `dialect_cxxflags`(#717)。** `[target..build] dialect_cxxflags` 在命中的 + 目标上把参数加入全图的方言参数:标准库 BMI、扫描与每个编译单元。只读取构建的根包;依赖包自己的 + 全图键不进入其指纹。 +- **MSVC ABI 的 CRT 模型(#718)。** CRT 是目标 ABI 的性质:cl 以 `/MD`、`/MT`,clang++ 以 + `-fms-runtime-lib=dll`、`static` 表达同一模型,并到达编译、std BMI 与链接。MSVC ABI 的默认契约为 + `toolchain-coupled`:动态 CRT,工具集的 `vcruntime140.dll`、`msvcp140.dll` 放到程序旁。 + `cxx_runtime` 与 `linkage` 之外不增加新键;手写的 CRT 参数与模型一致时提示冗余,矛盾时拒绝; + 依赖包 `[build] cxxflags` 中矛盾的 CRT 参数同样拒绝;调试 CRT 参数(`/MDd` 等)总被拒绝。依赖 + 的全局缓存键包含 CRT 模型。 +- **构建数据库描述规则生成的文件(#724)。** S1 集合的 `ide.generated` 列出生成的文件与目录、 + 同一组选择下 `mcpp build` 写入的路径以及生成它的步骤(S1 0.3.0)。命令仍不运行任何 action。 +- **统一的下载进度。** 工具链与载荷的安装、索引中的库包、`[xlings]` 载荷、索引刷新、`git` 依赖的 + 克隆与沙箱的首次引导由同一个渲染器报告。标准输出不是终端时,每一项只打印开始与结束两行,不含 + 回车与擦除序列。索引刷新经 `xlings interface update_packages` 进行,需要 xlings 2026.9.28.1 的 + 进度事件才逐步显示。 + +### 内部 + +- **`src/build/prepare/` 的阶段函数按其小节拆分(#722)。** 代码逐字移动,不改变语句顺序;七个 + 夹具的 `resolution.json`、`build.ninja` 与构建数据库输出与拆分前逐字节相同。该目录下没有超过 + 400 行的函数:`.github/tools/check_function_sizes.sh` 以 clang-tidy 的 `readability-function-size` + 在以 LLVM 构建得到的编译数据库上检查,拆分前报告 10 处;CI 中尚无能完整构建 mcpp 的 clang 任务, + 接入见 #729。`mcpp.lock` 与 `resolution.json` 的写入移入 `records.cpp`。 +- **xlings 固定版本为 2026.9.28.1。** 该版本的 interface 协议为 1.2:`update_packages` 按阶段发出 + 进度事件,interface 能力运行期间写到标准输出的文本不再混入事件流(openxlings/xlings#625)。 + +### 兼容性 + +- **成员的编译命令可能改变。** 经 `path` 依赖到达、带 `[package]` 的工作空间根的成员,现在收到 + `[workspace.build]` 与 `[workspace.package]`。 +- **Windows 上 LLVM 行的程序改用动态 CRT。** 它们现在导入 `vcruntime140.dll` 等,文件放在程序旁; + 写 `cxx_runtime = "self-contained"` 可恢复静态 CRT。cl 行的编译参数不变,程序旁多出这些 DLL。 +- **新增四种拒绝。** 每一种都在消息中给出一行修法: + - 手写的 CRT 参数与解析出的模型矛盾; + - 在没有 redistributable 目录的行上显式写 `toolchain-coupled`; + - 在两个命名空间下都有成员的包名上使用 `-p`; + - 手写的调试 CRT 参数,以及依赖包中与模型矛盾的 CRT 参数。 +- **`-p` 的值同时是一个成员的包名与另一个成员的目录名时,选择前者并警告。** 此前按目录名选择。 +- **S1 profile 版本为 0.3.0。** 0.2.0 的消费方忽略新字段。 + ## [2026.9.27.1] - 2026-09-27 ### 缺陷修复(#704、#705、#710、#712 至 #716) diff --git a/docs/04-mcpp-toml.md b/docs/04-mcpp-toml.md index 6a7c249e7..1d368ee98 100644 --- a/docs/04-mcpp-toml.md +++ b/docs/04-mcpp-toml.md @@ -138,6 +138,26 @@ The check reads the **effective** flags, so it fires for the same flag written i `[profile.] cxxflags` or in a `[target.…]` block. It does not fire when nothing in the graph imports `std`, where the flag is an ordinary per-unit option that works. +`dialect_cxxflags` also accepts the conditional form `[target..build] +dialect_cxxflags` *(mcpp 2026.9.28.1+)*, for a dialect switch that exists only on some targets: + +```toml +[build] +dialect_cxxflags = ["-fno-exceptions"] + +[target.windows.build] +dialect_cxxflags = ["-D_HAS_EXCEPTIONS=0"] +``` + +Unlike an ordinary build input, `dialect_cxxflags` is graph-wide (SPEC-004 §9 item 10), so only +the root of the build contributes it: the command's own package, or the member `-p` selects. +Entries are appended in this order — `[workspace.build]`, the root's own `[build]`, then each +matching `[target..build]` in manifest order — and the resolved list is what reaches +the std BMI prebuild, the module scan and every translation unit, on the root and on every +dependency alike. A dependency's own `dialect_cxxflags`, conditional or not, reaches no command: +a package legitimately declares it for the build it does when it is the root of one, which is why +it is not warned about. + ### 2.2 `[targets.]` — Build Targets ```toml @@ -1471,9 +1491,11 @@ is a table of exactly two strings. `from` is relative to the declaring package's root, and `to` is relative to the executable's directory, where `"."` means that directory itself. Both are separated by `/` on every host, and neither may be absolute, name a drive, or contain an empty, `.` or `..` component; an entry -that does is refused, and the refusal names its index. Two sources for one -destination are refused naming the destination, while one file name in two -directories is not a collision. `deploy` is a key of its own rather than a table +that does is refused, and the refusal names its index. Two or more sources for +one destination merge into a single copy, placed once every source is +byte-identical (mcpp#723); they are refused at build time, naming every source +and the destination, when they are not. One file name in two directories is +not a collision. `deploy` is a key of its own rather than a table form of `deploy_files`, because a descriptor reader that predates it meets `{` inside `deploy_files` and does not terminate, whereas it skips a `runtime` key it does not know. `mcpp pack` stages the files of both keys at the same relative diff --git a/docs/07-workspace.md b/docs/07-workspace.md index 1e9179d10..9b587aad8 100644 --- a/docs/07-workspace.md +++ b/docs/07-workspace.md @@ -350,14 +350,28 @@ mcpp searches upward from the current directory; if it finds an `mcpp.toml` cont ### 5.3 The `-p, --package` Option -`-p` works with `build`, `test`, `run`, and other commands to select the target member. Its value is either the last path segment of a member's directory name or the full relative path: +`-p` works with `build`, `test`, `run`, and other commands to select the target +member. Its value is resolved in one order, because the option names a +*package*: + +1. a member's qualified name, `.` (only meaningful for a + member that declares a namespace); +2. otherwise, a member's bare `package.name` — refused, naming every match, if + two or more members share it; +3. otherwise, a member's path as written in `[workspace] members`, or its + directory's last segment (the historical spellings, kept as a fallback). ```bash -mcpp build -p server # matches apps/server +mcpp build -p server # matches apps/server (by directory or package name) mcpp test -p core # matches libs/core mcpp run -p server -- --port 8080 ``` +A value that is one member's package name and a different member's directory +selects the member named by the package, with a warning naming the other one — +the option promises a package, so an exact package-name match outranks a +directory that merely happens to share the spelling. + `--workspace` (on `build` and `test`) is the fan-out form: it acts on **every** member. `mcpp test --workspace` reports each member separately and continues past a failing member, exiting non-zero if any member failed — ideal as a single, diff --git a/docs/09-commands-by-scenario.md b/docs/09-commands-by-scenario.md index 5538ccc10..21261d0bd 100644 --- a/docs/09-commands-by-scenario.md +++ b/docs/09-commands-by-scenario.md @@ -187,12 +187,46 @@ $ mcpp index status absent after a refresh is a propagation question, not a naming one — indices reach clients as artifacts rather than git clones. +An index can require a newer mcpp than the one running (`index.toml` +`min_mcpp`). That is not an error of the run (2026.9.28.1+). A refresh that +brings in such an index keeps the previous copy, and the run ends with one line: + +``` +tip: the refreshed package index `mcpplibs` requires a newer mcpp; this run used the previous index. It requires mcpp >= 2026.10.1.1; this is mcpp 2026.9.28.1. Upgrade: xlings update mcpp +``` + +A run that did not refresh an index says nothing about it. A run that fails +because a package is served only by such an index names E0006 in the message +that stops it. `mcpp self doctor` lists every index whose floor the running mcpp +does not meet. Earlier versions printed the E0006 text as `error:` at the start +of any run that read such an index, including runs that then succeeded. + `--offline` (or `MCPP_OFFLINE=1`) forbids the network for one invocation and fails rather than fetch. `--locked` fails when resolution differs from `mcpp.lock` instead of rewriting it, which is the shape a CI job wants. `mcpp index pin ` records a commit for a custom index in `mcpp.toml`; `mcpp index unpin` removes it. +## Download progress + +Every acquisition is reported by one renderer (2026.9.28.1+): + +- a toolchain or payload install; +- a library package from an index; +- an `[xlings]` payload; +- an index refresh; +- the clone of a `git` dependency; +- the sandbox's first-run tools. + +On a terminal each item is a bar drawn in place. When stdout is not a terminal, +as in a CI log or a pipe, each item prints one line when it starts, with its +size when known, and one line when it finishes, with its duration. That output +carries no carriage return and no erase sequence. `--quiet` prints neither. + +An index refresh is reported step by step when the xlings that mcpp drives +emits progress events for it (xlings 2026.9.28.1+). With an older xlings it +shows its status line and finishes silently, as before. + ## Validating a descriptor before publishing `mcpp xpkg parse` reads a descriptor with the resolver's own grammar, so what diff --git a/docs/20-toolchains.md b/docs/20-toolchains.md index 30eb8c67b..e6303bdff 100644 --- a/docs/20-toolchains.md +++ b/docs/20-toolchains.md @@ -499,7 +499,10 @@ It is a **compatibility floor declaration**, not a payload binding like `glibc@2.39` on Linux — `ucrtbase.dll` is a Windows component and mcpp neither ships nor substitutes it. -**CRT model.** `/MD` (host-coupled) by default; `/MT` when either +**CRT model.** `/MD` by default, with the toolset's own redistributable staged +beside the artifact (`toolchain-coupled` — see [On the MSVC +runtime](#on-the-msvc-runtime) below for the full model, which applies to `cl` +and to clang++ on this ABI alike); `/MT` when either ```toml [target.x86_64-windows-msvc] @@ -1131,7 +1134,7 @@ loaded *into* a process that already has a C++ runtime. |---|---|---| | ELF (Linux, …) | `toolchain-coupled` | ELF has one global symbol namespace and the first definition loaded wins. A `.so` that statically embedded libstdc++ **exports** it, and the executable linking that library binds *its* `std::` references there — its own `self-contained` contract silently becomes a no-op, and its C++ runtime is whichever build of that library happens to load. | | Mach-O | `self-contained` | the mechanism there is already `-load_hidden`, i.e. hidden visibility, so dyld never unifies those symbols; and toolchain-coupled is not available on macOS at all (see the note below). | -| PE (Windows) | `self-contained` | PE has no global symbol namespace — imports resolve per-DLL by name, so a DLL's private runtime cannot be picked up by anything else. | +| PE, GNU ABI (MinGW) | `self-contained` | PE has no global symbol namespace — imports resolve per-DLL by name, so a DLL's private runtime cannot be picked up by anything else. The MSVC ABI's own default is a separate rule — see [On the MSVC runtime](#on-the-msvc-runtime) below. | Setting `shared = "self-contained"` on ELF is supported and does exactly what it says: the library embeds the runtime. mcpp additionally passes @@ -1214,28 +1217,80 @@ artifact than the manifest asked for. ### On the MSVC runtime -The CRT model is the mechanism here, and it is a **whole-project** switch: cl -bakes `_MSVC_MT`/`_MSVC_MD` into the one `std` module a project builds, so a -per-role contract that disagrees with the project's cannot be honoured and is -reported rather than ignored. - -| value | meaning on MSVC | -|---|---| -| `self-contained` | `/MT` — the static CRT. `linkage = "static"` selects the same thing from the libc axis. | -| `host-coupled` (default under `/MD`) | the target provides `vcruntime140.dll` / `msvcp140.dll` — i.e. Visual Studio or the redistributable is installed there. | -| `toolchain-coupled` | the toolset's **own** copy of those DLLs travels with the artifact. | - -`toolchain-coupled` is worth spelling out, because the obvious reading is -wrong. `ucrtbase.dll` *is* a Windows component (since Windows 10) and mcpp -never ships it. `vcruntime140.dll` and `msvcp140.dll` are **not**: every MSVC -toolset carries them under `VC\Redist\MSVC\\\`, exactly the -way a gcc payload carries `libstdc++.so`. Under this contract mcpp stages them -beside the artifact — which is what makes a default `/MD` build runnable on a -machine that has only the pinned toolset and no Visual Studio at all. +The CRT model is a property of the **target ABI**, not of the compiler: `cl` +and clang++ targeting `*-windows-msvc` (the `llvm` row) receive the *same* +model, each spelling it for its own driver. It is also a **whole-project** +switch: `cl` bakes `_MSVC_MT`/`_MSVC_MD` into the one `std` module a project +builds, so a per-role contract that disagrees with the project's cannot be +honoured and is reported rather than ignored. + +| value | meaning on the MSVC ABI | `cl` spelling | clang++ spelling | +|---|---|---|---| +| `self-contained` (or `linkage = "static"`) | the static CRT | `/MT` | `-fms-runtime-lib=static` | +| `toolchain-coupled` (**default**) | the dynamic CRT, with the toolset's own copy of `vcruntime140.dll`/`msvcp140.dll` staged beside the artifact | `/MD` | `-fms-runtime-lib=dll` | +| `host-coupled` | the dynamic CRT, with nothing staged — the target provides those DLLs itself (Visual Studio, or the redistributable installer) | `/MD` | `-fms-runtime-lib=dll` | + +**`toolchain-coupled` is the default**, whatever `cxx_runtime` says, for every +role. This is worth spelling out, because the obvious reading of "portable by +default" is wrong here: `ucrtbase.dll` *is* a Windows component (since Windows +10) and mcpp never ships it, but `vcruntime140.dll` and `msvcp140.dll` are +**not** — every MSVC toolset carries them under +`VC\Redist\MSVC\\\`, exactly the way a gcc payload carries +`libstdc++.so`. Under this contract mcpp stages them beside the artifact +(on the `mcpp build` output directory) and puts the same directory on the +`mcpp run`/`mcpp test` search path — which is what makes the default build +runnable on a machine that has only the pinned toolset and no Visual Studio +at all. + +A resolved toolset that carries no `VC\Redist\MSVC` directory (measured on +some `msvc@system` installs) cannot deliver `toolchain-coupled`. The +undeclared default then resolves to `host-coupled` instead, silently — this is +a property of the row, stated once here, not a warning on every build of it. +An **explicit** `cxx_runtime = "toolchain-coupled"` on such a row is refused, +naming the missing directory: an explicit statement a toolset cannot meet is +an error, never a silent downgrade. The debug CRT (`vcruntime140d.dll` and friends, under `debug_nonredist\`) is -never staged: it may not be redistributed. - +never staged: it may not be redistributed, and mcpp's `dev` profile does not +select it — it states debug information, not a different CRT. That axis stays +deferred until a consumer needs it. + +Combining `toolchain-coupled` or `host-coupled` with `/MT` (`linkage = +"static"`, or `self-contained`) is a contradiction rather than a missing +feature — a static CRT leaves no DLL to couple to — so it is reported and +resolved to `self-contained`. `mcpp pack` enforces the other half: a mode +that bundles nothing (`--mode static` or `--mode system`) together with an +*explicit* `toolchain-coupled` cannot deliver it and refuses; `--mode system` +on a project that never stated a contract resolves the default to +`host-coupled` instead, since an explicit mode outranks a default. + +**A free-form CRT word is always a second statement.** Every MSVC-ABI build +now states its own CRT, so a literal `/MT`, `/MD`, `/MTd`, `/MDd` or +`-fms-runtime-lib=*` (either dash) in `[build] cxxflags` or `dialect_cxxflags` +can never be the only voice. One that **agrees** with the resolved model is +warned as redundant, naming the key (`cxx_runtime` or `linkage`) to write +instead; one that **contradicts** it is refused, naming the word, the key it +was found in, and the value it corresponds to. A dependency's `[build] +cxxflags` are checked too, since they reach that package's own units: a +contradicting word is refused, naming the package, and an agreeing one is +not warned, because `cxx_runtime` is the root's key. A debug word (`/MTd`, +`/MDd`, `-fms-runtime-lib=*_dbg`) is always refused: the model has no debug +axis, and the standard library module and the link use the release CRT. The +engine never lets the last word on the command line decide silently. + +> **Upgrading to 2026.9.28.1?** `cl`-row projects are unchanged apart from +> gaining the staged DLLs beside their programs. **LLVM-row programs move +> from the static to the dynamic CRT**: before this release clang++ on the +> MSVC ABI received no model at all and linked `libcmt` regardless of +> `cxx_runtime`; now it receives the same model `cl` does, defaulting to +> `toolchain-coupled`. A project that links a prebuilt `/MT` library on this +> row now fails to link (`LNK2038`, a CRT mismatch) and should state +> `cxx_runtime = "self-contained"` to restore the static CRT it had before. +> Two manifests that built before this release are refused after it: a +> free-form CRT word that contradicts the resolved model, and an explicit +> `toolchain-coupled` on a row whose toolset ships no redistributable — see +> above for both. +> > **Upgrading from 2026.8.15 or earlier?** This key used to be **inert** on the > MSVC ABI — it reported `not implemented for the MSVC runtime yet` and every > value fell back to `/MD`. Since 2026.8.16 it is honoured, so a manifest that @@ -1244,23 +1299,6 @@ never staged: it may not be redistributed. > model, and the switch is silent because the value was always valid. A project > that set it while the key did nothing should re-confirm the intended value. -Combining it with `/MT` is a contradiction rather than a missing feature — a -static CRT leaves no DLL to couple to — so it is reported and resolved to -`self-contained`. `mcpp pack` enforces the other half: a mode that bundles -nothing (`--mode system`, `--mode static`) cannot deliver `toolchain-coupled` -and refuses. - -**Clang on the MSVC ABI** (the `llvm` row of `x86_64-windows-msvc`, mcpp -2026.9.16.1+ for the record). The table above describes `cl.exe`, the one -compiler mcpp passes a CRT model to. Clang on the MSVC ABI speaks the GNU dialect -and receives no model, and its driver links the static CRT (`-defaultlib:libcmt`): -a program built on this row imports no `vcruntime140.dll`, `msvcp140.dll` or -`api-ms-win-crt-*`, and each DLL carries its own CRT. The row is therefore -`self-contained` whatever `cxx_runtime` says, `resolution.json` records it so, and -an explicit `host-coupled` or `toolchain-coupled` prints that the row does not -deliver it. A project that needs the dynamic CRT on the MSVC ABI builds with -`msvc@system`. - **Scope.** The contract governs the C++ runtime only. Static **libc** is a separate axis (`linkage = "static"` / `--static`, e.g. a musl target), and the deployment floor is a third — `macos_deployment_target` in `[package]` for Apple targets, diff --git a/docs/50-machine-output.md b/docs/50-machine-output.md index 5c0a5fc7c..c03eae927 100644 --- a/docs/50-machine-output.md +++ b/docs/50-machine-output.md @@ -420,6 +420,7 @@ a program classifying the outcome reads `reason`: | `lld-required-absent` | the target links through lld directly and the resolved toolchain payload ships none | | `host-tool-toolchain` | `build.mcpp` under a cross `--target` needs a resolvable HOST toolchain and none is set | | `std-module-precompile` | the standard library's module could not be precompiled for this configuration | +| `msvc-redist-unavailable` | an explicit `cxx_runtime = "toolchain-coupled"` on an MSVC-ABI row whose toolset has no redistributable directory to stage *(2026.9.28.1+)* | | `other` | a refusal whose branch has not been given a token yet | **One token is also printed by `mcpp build` itself.** @@ -453,7 +454,7 @@ writes nothing into the project. `data` is: | field | | |---|---| -| `spec` | `{"name": "s1", "version": "0.2.0"}`, or `{"name": "compile-commands"}` with `--spec compile-commands` | +| `spec` | `{"name": "s1", "version": "0.3.0"}` (0.2.0 before 2026.9.28.1), or `{"name": "compile-commands"}` with `--spec compile-commands` | | `database` | the document of that specification: an S1 build database, or the entries `mcpp build --configure-only` writes to `compile_commands.json` | | `watch` | the inputs whose change can change the document: paths and glob patterns relative to the workspace root, or absolute paths | | `inputs-fingerprint` | `fnv1a:<16 hex digits>`, a digest of those inputs, the mcpp version and the selectors | @@ -463,6 +464,13 @@ whatever it would print to `` instead. The content of the document, the no-write guarantee and the `watch` rules are [SPEC-005](specs/build-database.md). +Since 2026.9.28.1 an S1 set also names the files that its package's build +program generates, in `ide.generated` (S1 0.3.0 section 7.2). The command runs no +action, so such a file does not exist yet in the directory the document names. +Each entry gives the path a `mcpp build` of the same selection writes +(`build-path`) and the step that writes it, so that an editor can say "generated, +build once" instead of reporting a missing header. + `emit` plans every selected member on its own (#699 item 1): one member's planning failure does not cost its siblings'. Outside a project, or when every selected member fails to plan, the envelope omits `data` and exits 1, @@ -505,6 +513,7 @@ fails there still fails the build. | `MCPP_BUILD_DATABASE_STD_UNIT_UNDESCRIBED` | warning | no standard-library build command names its module source, so that unit is not listed | | `MCPP_BUILD_DATABASE_HOST_TOOL_DEFERRED` | note | a requested host tool is not in the tool store and is not built by the command; the plan names the path it will be published at (2026.9.27.1+; replaces the 2026.9.26.2 warning `MCPP_BUILD_DATABASE_HOST_TOOL_UNBUILT`) | | `MCPP_BUILD_DATABASE_PROGRAM_FAILED` | error | a build program failed; its package is described without its directives | +| `MCPP_INDEX_REQUIRES_NEWER_MCPP` | note | an index refreshed by this run requires a newer mcpp; the previous copy was kept or restored, or none is usable (2026.9.28.1+; the same notice a terminal run prints as its closing `tip:` line) | `--protocol-version` declares `init-mcpp-home`, `read-project`, `network`, `write-global-cache` and `exec-build-script` for the command, and never diff --git a/docs/specs/README.md b/docs/specs/README.md index 6bfb1f504..3d1743998 100644 --- a/docs/specs/README.md +++ b/docs/specs/README.md @@ -33,10 +33,10 @@ | [SPEC-001](package-identity.md) | 包身份(`package.namespace` / `package.name`)、`[dependencies]` 选择器与匹配机制 | 评审中 v1.1 | 2026-08-03 | mcpp >= 0.0.106 | | [SPEC-002](target-side.md) | 目标侧模型与能力声明(`mcpp:` 保留命名空间、五层、三条规则) | 评审中 v1.0 | 2026-08-24 | mcpp >= 2026.8.24.2 | | [SPEC-003](exit-codes.md) | 退出码契约(分类、语义、稳定性承诺) | 评审中 v1.0 | 2026-09-01 | mcpp >= 2026.9.1.1 | -| [SPEC-004](manifest-semantics.md) | `mcpp.toml` 的平面划分、条件化形状、解析轴与命名规约 | 草案 v1.7 | 2026-09-26 | 条件化形状 mcpp >= 2026.8.29.1;目标轴 mcpp >= 2026.9.6.4;`linkage` 默认值 mcpp >= 2026.9.15.2;链接 flag 的词读法 mcpp >= 2026.9.26.2 | -| [SPEC-005](build-database.md) | 构建数据库:`mcpp emit build-database` 的内容、取值规则与不写工程目录的保证 | 评审中 v1.3 | 2026-09-26 | mcpp >= 2026.9.15.1;v1.3 条款 mcpp >= 2026.9.26.2 | -| [SPEC-006](toolchain-management.md) | 工具链管理:身份、来源、选择与载荷契约 | 草案 v0.2 | 2026-09-24 | 逐条标注;已实现条款 mcpp >= 2026.9.24.1 | -| [SPEC-007](build-plugins.md) | 构建插件:配置、施工与校验的分工,运行时与规划期的义务 | 草案 v0.2 | 2026-09-26 | 逐条标注;mcpp >= 2026.9.26.2 | +| [SPEC-004](manifest-semantics.md) | `mcpp.toml` 的平面划分、条件化形状、解析轴与命名规约 | 草案 v1.9 | 2026-09-28 | 条件化形状 mcpp >= 2026.8.29.1;目标轴 mcpp >= 2026.9.6.4;`linkage` 默认值 mcpp >= 2026.9.15.2;链接 flag 的词读法 mcpp >= 2026.9.26.2;条件化的 `dialect_cxxflags` 与 `-p` 的包身份 mcpp >= 2026.9.28.1 | +| [SPEC-005](build-database.md) | 构建数据库:`mcpp emit build-database` 的内容、取值规则与不写工程目录的保证 | 评审中 v1.5 | 2026-09-28 | mcpp >= 2026.9.15.1;v1.3 条款 mcpp >= 2026.9.26.2;v1.4 条款 mcpp >= 2026.9.27.1;v1.5 条款 mcpp >= 2026.9.28.1 | +| [SPEC-006](toolchain-management.md) | 工具链管理:身份、来源、选择与载荷契约 | 草案 v0.3 | 2026-09-28 | 逐条标注;已实现条款 mcpp >= 2026.9.24.1;§3.7 mcpp >= 2026.9.28.1 | +| [SPEC-007](build-plugins.md) | 构建插件:配置、施工与校验的分工,运行时与规划期的义务 | 草案 v0.4 | 2026-09-28 | 逐条标注;mcpp >= 2026.9.26.2;v0.3 条款 mcpp >= 2026.9.27.1;v0.4 条款 mcpp >= 2026.9.28.1 | ## 文档约定 diff --git a/docs/specs/build-database.md b/docs/specs/build-database.md index aa3ff1b4f..3988e7b5c 100644 --- a/docs/specs/build-database.md +++ b/docs/specs/build-database.md @@ -4,13 +4,13 @@ |---|---| | 规范编号 | SPEC-005 | | 标题 | mcpp 输出的构建数据库:内容、取值规则与不写工程目录的保证 | -| 状态 | 评审中 v1.4 | -| 版本 | 1.4 | -| 最后修改 | 2026-09-26 | -| 对应实现 | mcpp >= 2026.9.15.1;v1.3 修改的 R2.5、R3.7、R3.8、R4.1、R5.2 为 mcpp >= 2026.9.26.2;v1.4 修改的 R2.5 为 mcpp >= 2026.9.27.1 | +| 状态 | 评审中 v1.5 | +| 版本 | 1.5 | +| 最后修改 | 2026-09-28 | +| 对应实现 | mcpp >= 2026.9.15.1;v1.3 修改的 R2.5、R3.7、R3.8、R4.1、R5.2 为 mcpp >= 2026.9.26.2;v1.4 修改的 R2.5 为 mcpp >= 2026.9.27.1;v1.5 修改的 R3.7、R3.12、R5.1、R5.2 为 mcpp >= 2026.9.28.1 | | 相关设计文档 | `.agents/docs/2026-09-14-636-build-database-and-the-latest-xlings.md`
`.agents/docs/2026-09-26-compile-database-and-issue-699-design.md` | | 相关 issue | #636, #648, #655, #699, #702, #707 | -| 依据的外部规范 | S1「C++ Build Database: IDE Profile」profile 0.2.0 与 S2 0.2.0 §3.4,取自 https://github.com/Sunrisepeak/lsp-mcpp-private 提交 `b82859d`(schema 自提交 `28ecd6e` 起未变);S2 0.3.0 §3.4 的部分回答(S2-3.4-12、S2-3.4-13,Sunrisepeak/mcpp-language-server#25);JSON Compilation Database | +| 依据的外部规范 | S1「C++ Build Database: IDE Profile」profile 0.3.0(§7.2 的 `generated`,Sunrisepeak/mcpp-language-server#28;此前为 0.2.0)与 S2 0.2.0 §3.4,取自 https://github.com/Sunrisepeak/lsp-mcpp-private 提交 `b82859d`(schema 自提交 `28ecd6e` 起未变);S2 0.3.0 §3.4 的部分回答(S2-3.4-12、S2-3.4-13,Sunrisepeak/mcpp-language-server#25);JSON Compilation Database | ## 0. 适用范围 @@ -100,7 +100,11 @@ mcpp 输出的 S1 文档满足 S1 等级 2,不输出 `ide.options`。等级 3 ### 3.3 翻译单元 -- **R3.7** 除 NASM 单元外,构建计划中的每个编译单元是一个翻译单元。`source`、 +- **R3.7** 除 NASM 单元与规则声明的设备源文件(`SourceKind::Device`)外,构建计划中 + 的每个编译单元是一个翻译单元;两者都不在 S1 文档与 `compile_commands.json` 中 + 出现,但原因不同——NASM 单元是构建计划的编译单元,只是被逐出翻译单元的集合; + 设备源文件从不是构建计划的编译单元(引擎对其扩展名没有编译规则,能编译它的只有 + 包自己的构建程序,通过一个动作),因而也从不进入这一集合。`source`、 `work-directory`、`arguments`、`object` 与 `compile_commands.json` 中对应条目的 `file`、`directory`、`arguments`、`output` 取自同一条记录,因而逐字相同。 `work-directory` 是编译器实际运行的目录——即输出目录 @@ -148,6 +152,21 @@ mcpp 输出的 S1 文档满足 S1 等级 2,不输出 `ide.options`。等级 3 命令中找不到该源文件时,不列出该单元,并输出警告 `MCPP_BUILD_DATABASE_STD_UNIT_UNDESCRIBED`。**已实现** +### 3.5 生成的文件 + +- **R3.12** 一个集合的 `ide.generated`(S1 0.3.0 §7.2)列出该集合所属包的构建程序以 + `role = "source"` 的 action 生成的每一个输出,以及该集合的单元以 `-I` 命名、位于规划 + 目录的 `target/.build-mcpp` 之下的每一个目录;包的测试集合与其普通集合一样列出这些输出, + 因为不经预处理无法知道哪些单元包含一个头文件。每一项给出 `path`(本文档中的路径)、 + `build-path`(同一组选择器下 `mcpp build` 写入的路径:把规划目录换成工程根,文件存在 + 与否都给出)与 `kind`。一个输出同时是该集合某个单元的 `source` 时 `kind` 为 `source`, + 否则为 `header`;目录为 `directory`。文件一项另有 `generator`:action 的 `id`、`inputs`、 + 作为 `arguments` 的命令,以及 `work-directory`(action 声明的 `cwd`,未声明时为构建 + 目录)。没有这样的输出与目录的集合不带该字段。该字段不进入 `--spec compile-commands` + 的文档,因为 JSON Compilation Database 的读者拒绝未知的键。命令不运行任何 action + (R2.5);由消费方决定是否在其用户同意时运行 `generator`。**已实现** + (mcpp >= 2026.9.28.1,mcpp#724) + ## 4. `--spec compile-commands` - **R4.1** 文档为 `mcpp build --configure-only` 在同一组选择器下写入 @@ -158,7 +177,7 @@ mcpp 输出的 S1 文档满足 S1 等级 2,不输出 `ide.options`。等级 3 ## 5. 信封 - **R5.1** `kind` 为 `mcpp.build-database`,`kindVersion` 为 1。`data` 含 `spec` - (`{"name": "s1", "version": "0.2.0"}` 或 `{"name": "compile-commands"}`)、 + (`{"name": "s1", "version": "0.3.0"}` 或 `{"name": "compile-commands"}`)、 `database`、`watch` 与 `inputs-fingerprint`。**已实现** - **R5.2** 命令独立规划每一个被选中的成员:一个成员规划失败只影响它自己,不影响 其余成员的集合(#699 第 1 项)。规划失败的成员不贡献任何集合,只贡献一条 `error` @@ -170,7 +189,10 @@ mcpp 输出的 S1 文档满足 S1 等级 2,不输出 `ide.options`。等级 3 的成员中,构建程序失败的包被描述为不含该程序产生的指令(清单自身的配置、工具链、 模块图与标准库单元仍照常描述),`diagnostics` 另有一条 `error`, `MCPP_BUILD_DATABASE_PROGRAM_FAILED`,`path` 为该包的 `build.mcpp`;后续失败若是 - 由缺失的指令引起,则按前一条规则使整个成员失败。只要 `diagnostics` 中有一条 + 由缺失的指令引起,则按前一条规则使整个成员失败。一项检查若以构建程序的指令为 + 前提(例如"每个设备源文件都被某个动作消费"),对本轮构建程序失败的包不运行: + 该包已经带着这一条 `PROGRAM_FAILED` 诊断被描述,不应因指令缺失这一后果本身被 + 判成第二个失败,把真正的诊断挤出信封。只要 `diagnostics` 中有一条 `error`,退出码就是 1,无论 `data` 是否出现。**已实现**(离线诊断码: mcpp >= 2026.9.16.1;成员独立规划、`path` 与构建程序失败的描述:mcpp >= 2026.9.26.2) - **R5.3** 信封的 `effects` 为 `read-project` 与 `write-global-cache`,运行了构建程序时 @@ -203,3 +225,4 @@ mcpp 输出的 S1 文档满足 S1 等级 2,不输出 `ide.options`。等级 3 | 1.2 | 2026-09-17 | R3.7 陈述 `arguments` 的每一项是编译器收到的参数,单元 flag 按 SPEC-004 §8 的词列出(#655)。 | | 1.3 | 2026-09-26 | R2.5:`emit` 下构建失败的宿主工具是警告。R3.7:`work-directory` 是输出目录,模块接口单元的 `arguments` 带语言 flag。R3.8:标准库单元的 `provides` 指向 std 缓存中的 BMI,工具链带 `build-id`。R4.1:compile-commands 文档包含标准库单元(S1-12-1)。R5.2:成员各自规划,构建程序失败的包不带其指令地被描述(#699,#702)。 | | 1.4 | 2026-09-26 | R2.5:命令不构建宿主工具;工具库中没有的工具被推迟,输出说明 `MCPP_BUILD_DATABASE_HOST_TOOL_DEFERRED`,取代 1.3 的警告 `MCPP_BUILD_DATABASE_HOST_TOOL_UNBUILT`(#707)。 | +| 1.5 | 2026-09-28 | R3.7:规则声明的设备源不是编译单元,不进入 S1 与 `compile_commands.json`(#724)。新增 R3.12:集合的 `ide.generated` 列出规则生成的文件与目录,给出构建写入的路径与生成它的步骤,S1 0.3.0(#724,Sunrisepeak/mcpp-language-server#28)。R5.1:S1 版本为 0.3.0。R5.2:以构建程序的指令为前提的检查不对其构建程序已失败的包运行,失败路径保留已记录的说明(#724)。 | diff --git a/docs/specs/build-plugins.md b/docs/specs/build-plugins.md index fead3f5ab..ceb641cc9 100644 --- a/docs/specs/build-plugins.md +++ b/docs/specs/build-plugins.md @@ -4,10 +4,10 @@ |---|---| | 规范编号 | SPEC-007 | | 标题 | 构建插件:配置、施工与校验的分工,运行时与规划期的义务 | -| 状态 | 草案 v0.3 | -| 版本 | 0.3 | -| 最后修改 | 2026-09-27 | -| 对应实现 | 逐条标注。未注明版本的「已实现」条款对应 mcpp >= 2026.9.26.1;注明 mcpp#702 的条款对应 mcpp >= 2026.9.26.2;注明 mcpp#707、#708、#709、#711 的条款对应 mcpp >= 2026.9.27.1 | +| 状态 | 草案 v0.4 | +| 版本 | 0.4 | +| 最后修改 | 2026-09-28 | +| 对应实现 | 逐条标注。未注明版本的「已实现」条款对应 mcpp >= 2026.9.26.1;注明 mcpp#702 的条款对应 mcpp >= 2026.9.26.2;注明 mcpp#707、#708、#709、#711 的条款对应 mcpp >= 2026.9.27.1;注明 mcpp#723 的条款对应 mcpp >= 2026.9.28.1 | | 相关设计文档 | `.agents/docs/2026-09-26-compile-database-and-issue-699-design.md`(§5) | | 相关 issue | mcpp#699、mcpp#701、mcpp#702、mcpp#703、mcpp#707、mcpp#708、mcpp#709、mcpp#711 | | 使用文档 | [docs/30 - build.mcpp](../30-build-mcpp.md)、[docs/31 - 编写规则包](../31-authoring-a-rule-package.md) | @@ -140,12 +140,21 @@ 从不作为 `-L`)、`mcpp run` 的加载路径、`mcpp pack` 的闭包搜索与运行时校验,并把依赖包的 声明传到消费方的可执行文件。(**已实现**,mcpp#702) - **R4.2** 一个在配置时已知的文件需要位于程序旁的某个相对位置时(Qt 的平台插件、Vulkan 的 - ICD 清单),插件用 `deploy`。(**已实现**,协议 11) + ICD 清单),插件用 `deploy`。(**已实现**,协议 11)两个或更多来源为同一目的地各自声明 + `deploy` 时,规划期不再把它当作错误拒绝:被声明的来源此时可能尚未生成,其内容无法比较。 + 引擎把它们合并为一条施工边,将每个来源都列为该边的输入;施工时(`mcpp stage`)逐字节核对 + 这些来源,字节相同则放置,不同则该边失败,消息点名每一个来源与该目的地。(**已实现**, + mcpp#723) - **R4.3** Windows 的可执行文件没有运行路径。`mcpp run` 通过 `PATH` 使用运行时搜索目录, `mcpp pack` 把闭包需要的 DLL 放到程序旁(**已实现**)。链接之后,引擎把程序直接或间接导入的、 位于其运行时搜索目录中的非系统 DLL 放到程序旁,使从构建目录直接启动的程序同样能找到它们; 闭包的求解与 `mcpp pack` 相同,DLL 在其目录中被替换后下一次构建再次放置。(**已实现**, - mcpp#702) + mcpp#702)一个目的地只有一个写者:本条的放置以 R4.2 与工具链耦合运行时 DLL(`toolchain- + coupled`)合并而成的部署清单为唯一权威,**禁止**写入该清单已经放置的名字。遇到清单已放置 + 的名字时,本条只比较该名字现有文件与运行时搜索目录中同名文件的字节,相同则不作声张,不同 + 则以警告点名这一差异,**禁止**覆盖清单已放置的文件。规划时在运行时搜索目录中找到的 DLL + 同样是推导出的来源,让位于清单中声明的同名目的地,差异由本条的放置报告。(**已实现**, + mcpp#723) - **R4.4** 插件**禁止**在 `link_flag` 中写运行路径(`-Wl,-rpath,...`),**必须**使用 R4.1。 (作者义务) @@ -198,4 +207,5 @@ |---|---|---| | 0.1 | 2026-09-26 | 首版草案(mcpp#699、#701、#702、#703)。 | | 0.3 | 2026-09-27 | 随 mcpp 2026.9.27.1:新增 R3.8(action 的 `env` 与 `cwd`,协议 13,mcpp#708);R5.3 改为规划不构建宿主工具、缺失的工具以 note 推迟(mcpp#707);新增 R6.3(特性的 `tools`,mcpp#709)与 R6.4(`artifacts` 与 `${mcpp.artifact:}`,mcpp#711)。 | +| 0.4 | 2026-09-28 | 随 mcpp 2026.9.28.1:R4.2 同一目标的多个来源在放置时按内容核对,相同则放置一份,不同则失败并点名全部来源;R4.3 一个目标一个写入者,链接后的放置不覆盖另一写入者放在程序旁的文件(mcpp#723)。 | | 0.2 | 2026-09-26 | 随 mcpp 2026.9.26.2 落地:R1.3 的警告、R2.1 的 `runtime_search_dir`、R2.4、R3.3 的 `prepare`(目录须含文件;链接边等待所有 `prepare`)、R3.5、R3.6、R4.1、R4.3、R5.2、R5.3 标为已实现。 | diff --git a/docs/specs/manifest-semantics.md b/docs/specs/manifest-semantics.md index 91da1bd6d..950104557 100644 --- a/docs/specs/manifest-semantics.md +++ b/docs/specs/manifest-semantics.md @@ -5,11 +5,11 @@ | **规范编号** | SPEC-004 | | **标题** | `mcpp.toml` 的平面划分、条件化形状、解析轴与命名规约 | | **状态** | **草案(Draft)** | -| **版本** | 1.8 | -| **最后修改** | 2026-09-27 | +| **版本** | 1.9 | +| **最后修改** | 2026-09-28 | | **最低实现版本** | 条件化形状:mcpp **2026.8.29.1**(`[target..build-dependencies]` 起齐备);目标轴:mcpp **2026.9.6.4** | | **作者/维护** | mcpp-community | -| **相关设计文档** | `.agents/docs/2026-09-07-mcpp-toml-unified-semantics-design.md`
`.agents/docs/2026-06-04-manifest-schema-ownership.md`
`.agents/docs/2026-09-03-xlings-workspace-as-the-one-table.md`
`.agents/docs/2026-09-25-issue-690-workspace-build-inheritance-consistency.md` | +| **相关设计文档** | `.agents/docs/2026-09-07-mcpp-toml-unified-semantics-design.md`
`.agents/docs/2026-06-04-manifest-schema-ownership.md`
`.agents/docs/2026-09-03-xlings-workspace-as-the-one-table.md`
`.agents/docs/2026-09-25-issue-690-workspace-build-inheritance-consistency.md`
`.agents/docs/2026-09-27-eight-reports-by-home-and-one-optimisation-plan.md` | | **相关使用文档** | [docs/04 —— mcpp.toml 字段参考](../04-mcpp-toml.md) | ## 规范用语 @@ -81,6 +81,15 @@ Principle)规定,本规范不重复它,只在 §6 引用并补充一条。 **状态:已实现**(上列 section)。 +`[build]` 之下的 `dialect_cxxflags` 是图级联的方言开关,不是逐包可叠加的构建输入 +(`BuildConfig::dialectCxxflags`;§9 第 10 条)。写在 `[target..build]` +之下时,它接受与本节其它键相同的条件形状,但按图级联规则解析而不是按包解析: +只有一次构建的根(命令的包,或 `-p` 选中的成员)对这个列表贡献,依赖包自己声明 +的这个键不到达任何命令。向量按 `[workspace.build]`、根的 `[build]`、再到每个 +命中的 `[target..build]`(按清单顺序)追加,如同一个可叠加的构建输入。 + +**状态:已实现(mcpp 2026.9.28.1)。** + ### 3.1.1 条件声明替换同一身份的无条件声明 在 `` 命中的行上,`[target..dependencies]` 中某个身份的声明 @@ -104,7 +113,7 @@ Principle)规定,本规范不重复它,只在 §6 引用并补充一条。 与 `linkage` 并存、一行同时陈述 `kind` 与 `linkage`、`linkage` 写在程序目标上,均**必须** 被拒绝;按行合并时后命中的陈述替换先前的陈述,无论两者各是 `kind` 还是 `linkage`。 -**状态:已实现**(mcpp 2026.9.14.2;`linkage` 为 2026.9.15.2)。 +**状态:部分实现**(mcpp 2026.9.14.2;`linkage` 为 2026.9.15.2)。多个命中的条件表的先后:实现按选择器文本的字典序合并,而不是按清单中的位置,因为 TOML 的表不带键的顺序;该条款待 mcpp#728 修订。 ### 3.2 门可以嵌进条件 @@ -434,10 +443,15 @@ mcpp 2026.9.26.2,#703)。** ## 9. 工作空间继承与构建需求的作用域 1. 工作空间成员**必须**恰好接收一次 `[workspace.package]`、`[workspace.build]` 与 - `x.workspace = true` 条目的继承,无论它是命令构建的包、另一个成员的 `path` 依赖、 - 通过 `git` 引用的托管在 git 上的工作空间的成员,还是索引包归档内的成员(描述符的 - `mcpp` 字段指向该成员的清单)。后两种情况按该成员所在仓库或归档的工作空间根继承, - 相对路径以该根为锚点;在归档内查找工作空间根时**禁止**越出该版本的安装根。 + `x.workspace = true` 条目的继承,无论它是命令构建的包、带 `[package]` 的工作空间根 + 自己的 `path` 依赖所到达的成员、另一个成员的 `path` 依赖、通过 `git` 引用的托管在 + git 上的工作空间的成员,还是索引包归档内的成员(描述符的 `mcpp` 字段指向该成员的 + 清单)。后两种情况按该成员所在仓库或归档的工作空间根继承,相对路径以该根为锚点; + 在归档内查找工作空间根时**禁止**越出该版本的安装根。工作空间的上下文(它是哪个 + 工作空间、工作空间根在哪里)取决于清单**在哪里**,与命令走的是哪条分支无关——带 + `[package]` 的工作空间根按自身构建时,同样要在解析任何依赖之前建立这一上下文。 + `-p`/`--package` 首先按成员的包身份(限定名 `.`,其次是裸包名) + 为其命名,目录路径与目录名是回落拼法。 2. 向量按工作空间、成员、命中的 `[target..build]` 的顺序追加;`defines` 按 §8 的集合语义合并。标量仅在成员未**声明**该键时取工作空间的值。 3. 继承**必须**在 `defines` 展开之前、在清单被固定进构建图之前完成。实现**必须**拒绝 @@ -462,10 +476,14 @@ mcpp 2026.9.26.2,#703)。** `workspace = true` 条目。 10. `[toolchain]`、`[target.]` 与 `[indices]` 是根位置的键:它们为整个依赖图选择 编译器、目标行与索引,因此只在成员作为一次构建的根时继承。作为宿主工具构建的成员是其 - 子构建的根,同样继承这三项(§10.1)。 + 子构建的根,同样继承这三项(§10.1)。`[build] dialect_cxxflags`(及其条件形式 + `[target..build] dialect_cxxflags`)同样是根位置的键:它是 §3.1 + 所述的图级联方言开关,只在包作为一次构建的根时被渲染并到达命令。与前三项相同, + 一个包声明它不被诊断——一个依赖包为自己将来作为根的构建合法地声明这些键,这一条 + 只是把已有行为写成明文规则。 **状态:已实现(第 1 至 7 条 mcpp 2026.9.25.1;第 8 至 10 条 mcpp 2026.9.27.1,mcpp#713、 -#714、#710)。** +#714、#710;第 10 条的 `dialect_cxxflags` 为 mcpp 2026.9.28.1,#717)。** ## 10. 依赖的程序 @@ -520,3 +538,4 @@ mcpp 2026.9.26.2,#703)。** | 1.6 | 2026-09-25 | 工作空间继承与构建需求的作用域(mcpp 2026.9.25.1,#690):§8 补 `defines` 的集合语义;新增 §9 与 §7 第 11 至 14 条判据。 | | 1.7 | 2026-09-26 | §8 的读法扩展到 `ldflags` 与构建程序的链接指令(mcpp 2026.9.26.2,#703):`$ORIGIN` 原样到达链接器;§7 补第 15 条判据。 | | 1.8 | 2026-09-27 | mcpp 2026.9.27.1:§4.5 的版本位按 xlings 文法回答(#712);新增 §4.6 宿主构建读取宿主三元组的行(#704);§9 补第 8 至 10 条(#713、#714、#710);新增 §10 依赖的程序:`tools`、特性的 `tools`、`artifacts`(#709、#711);§7 补第 16 至 20 条判据。 | +| 1.9 | 2026-09-28 | mcpp 2026.9.28.1:§9 第 1 条补上带 `[package]` 的工作空间根自己的 `path` 依赖所到达的成员,`-p` 先按包的身份解析(#725);§3.1 接受 `[target..build] dialect_cxxflags`,§9 第 10 条把它列为根位置的键(#717);§3.1.1 的状态改为部分实现,多个命中的条件表的先后见 mcpp#728。 | diff --git a/docs/specs/toolchain-management.md b/docs/specs/toolchain-management.md index 958e3d481..cc7d14656 100644 --- a/docs/specs/toolchain-management.md +++ b/docs/specs/toolchain-management.md @@ -4,11 +4,11 @@ |---|---| | 规范编号 | SPEC-006 | | 标题 | 工具链管理:身份、来源、选择与载荷契约 | -| 状态 | 草案 v0.2 | -| 最后修改 | 2026-09-24 | +| 状态 | 草案 v0.3 | +| 最后修改 | 2026-09-28 | | 对应实现 | 逐条标注;标为「已实现」的条款对应 mcpp >= 2026.9.24.1。标为「未实现」的条款计划与下一批 LLVM 工具链一同落地,届时按实测修订本规范 | | 相关设计文档 | `.agents/docs/2026-09-24-toolchain-selection-and-payload-trust-design.md`、`.agents/docs/2026-09-24-685-687-msvc-stl-and-toolchain-payloads.md` | -| 相关 issue | mcpp#685、mcpp#687 | +| 相关 issue | mcpp#685、mcpp#687、mcpp#718 | | 使用文档 | [docs/20 - 工具链](../zh/20-toolchains.md)、[docs/32 - 编写载荷](../zh/32-authoring-a-payload.md)、[docs/91 - 工具链内部](../zh/91-toolchain-internals.md) | 本规范定义 mcpp 对工具链的命名、选择和使用方式,以及一个工具链载荷在发布前必须满足的条件。 @@ -119,6 +119,26 @@ MSVC ABI 目标上:SDK 以 `ucrt@<版本>` 进入运行时身份;clang 行的 to 描述产物的属性(最低系统版本、三元组中的版本段)**必须**按目标判定,与宿主无关; 只有在宿主上执行的编译(build.mcpp)按宿主判定。macOS 的 deployment target 在任何宿主上都按目标解析与施加。 +### 3.7 MSVC ABI 目标的 CRT 模型 已实现 + +在 `*-windows-msvc` 目标上,CRT 模型(静态或动态)是目标 ABI 的属性,而非某一个编译器的属性: +`cl.exe` 与以该 ABI 为目标的 clang 行**必须**接收同一个模型,分别以各自驱动的拼写(`/MT`/`/MD`, +`-fms-runtime-lib=static`/`=dll`)发给编译单元、`std`/`std.compat` BMI 与链接命令。 + +- 未声明的契约在该 ABI 上,对每个角色都**必须**解析为 `toolchain-coupled`:动态 CRT,并将所选 + toolset 自带的 `vcruntime140.dll`/`msvcp140.dll` 等文件置于产物旁。 +- `self-contained`,或 `linkage = "static"`,**必须**解析为静态 CRT。 +- `host-coupled` **必须**解析为动态 CRT,且不放置文件。 +- 所选 toolset 不带 `VC\Redist\MSVC\<版本>\<架构>\Microsoft.VC*.CRT` 目录时,未声明的契约**必须** + 静默解析为 `host-coupled`;显式声明的 `toolchain-coupled` **必须**被拒绝,并指出缺失的目录—— + 这是行的一个属性,不因某一次构建而降级。 +- `[build] cxxflags` 或 `dialect_cxxflags` 中出现的自由拼写 CRT 词(`/MT`、`/MD`、`-fms-runtime-lib=*` + 等)与已解析的模型一致时**应当**被警告为冗余;不一致时**必须**被拒绝,消息**必须**指出该词、 + 所在的键与该词对应的值。依赖包的 `[build] cxxflags` 同样检查,因为它们作用于该包自己的单元: + 不一致时**必须**被拒绝并指出该包;一致时不警告,因为替代它的键 `cxx_runtime` 只属于根。 + 调试 CRT 词(`/MTd`、`/MDd`、`-fms-runtime-lib=*_dbg`)**必须**被拒绝:模型不表达调试 CRT, + 标准库模块与链接使用发布版 CRT。 + --- ## 4. 载荷契约 @@ -228,3 +248,4 @@ xim-pkgindex 的准入脚本 `verify-toolchain.sh` 对一个载荷归档做一 |---|---|---| | v0.1 | 2026-09-24 | 初版草案:身份与写法、来源与选择(含 MSVC ABI 目标的 sysroot)、载荷契约、构建、验收、发布顺序 | | v0.2 | 2026-09-24 | 随 mcpp 2026.9.24.1 更新实现状态:§2.3、§2.4、§3.1 至 §3.6 已实现;§4.2、§6.4 部分实现;§2.2 更正:不带族的 `system` 被拒绝 | +| v0.3 | 2026-09-28 | 随 mcpp 2026.9.28.1:新增 §3.7,MSVC ABI 的 CRT 模型是目标 ABI 的性质,cl 与 clang++ 同样收到,默认 `toolchain-coupled`(mcpp#718)。 | diff --git a/docs/zh/04-mcpp-toml.md b/docs/zh/04-mcpp-toml.md index 035279b4c..749a3b451 100644 --- a/docs/zh/04-mcpp-toml.md +++ b/docs/zh/04-mcpp-toml.md @@ -150,6 +150,25 @@ error: `-fno-exceptions` changes the language dialect, but the `import std` BMI 没有任何单元 import `std` 时不会触发，此时该标志只是一个正常生效的按 单元选项。 +`dialect_cxxflags` 也接受条件形式 `[target..build] dialect_cxxflags` +*(mcpp 2026.9.28.1+)*，用来表达一个只在部分目标上存在的方言开关： + +```toml +[build] +dialect_cxxflags = ["-fno-exceptions"] + +[target.windows.build] +dialect_cxxflags = ["-D_HAS_EXCEPTIONS=0"] +``` + +与普通的构建输入不同，`dialect_cxxflags` 是图级联的（SPEC-004 §9 第 10 条）， +所以只有这次构建的根贡献它：命令直接构建的那个包，或 `-p` 选中的成员。条目 +按这个顺序追加——`[workspace.build]`、根自己的 `[build]`、再到每个命中的 +`[target..build]`（按清单顺序）——解出的列表到达 std BMI 的预构建、 +模块扫描与每一个翻译单元，对根和对每个依赖一视同仁。依赖包自己声明的 +`dialect_cxxflags`，无论是否带条件，都不到达任何命令：一个包为自己将来作为 +根的构建合法地声明它，这也是它不被诊断的原因。 + ### 2.2 `[targets.]` —— 构建目标 ```toml @@ -1401,9 +1420,10 @@ Link intent 把各个发现阶段分开处理： 两个字符串组成的表。`from` 相对声明它的包根目录，`to` 相对可执行 文件所在目录，`"."` 意味着那个目录本身。两者在每个宿主上都以 `/` 分隔，都不能是绝对路径、不能命名一个驱动器、不能含有空、`.` 或 `..` -组成部分；不满足的条目会被拒绝，拒绝信息点名它的索引。两个来源指向 -同一个目的地会被拒绝并点名那个目的地，而一个文件名出现在两个不同 -目录下不算冲突。`deploy` 是一个独立的键，而不是 `deploy_files` 的 +组成部分；不满足的条目会被拒绝，拒绝信息点名它的索引。两个或更多 +来源指向同一个目的地会合并为一次拷贝，字节相同则放置（mcpp#723）； +不同则在施工时失败，点名每一个来源与该目的地。一个文件名出现在两个 +不同目录下不算冲突。`deploy` 是一个独立的键，而不是 `deploy_files` 的 表格形式，因为一个早于它出现的描述符读取器，遇到 `deploy_files` 里的 `{` 会无法终止，而它会跳过一个不认识的 `runtime` 键。`mcpp pack` 把 这两个键指向的文件，以打包出的可执行文件为参照，拷贝到同样的相对 diff --git a/docs/zh/07-workspace.md b/docs/zh/07-workspace.md index 18a0fb0d9..06203ccc9 100644 --- a/docs/zh/07-workspace.md +++ b/docs/zh/07-workspace.md @@ -332,15 +332,23 @@ mcpp 从当前目录向上搜索；若发现某个 `mcpp.toml` 含 `[workspace]` ### 5.3 `-p, --package` 选项 -`-p` 可用于 `build`、`test`、`run` 等命令，指定目标成员。参数值可以是成员目录名 -的最后一段，也可以是完整相对路径： +`-p` 可用于 `build`、`test`、`run` 等命令，指定目标成员。选项名说的是**包**， +参数值按下述顺序解析： + +1. 成员的限定名 `.`（只有声明了 namespace 的成员才有这个拼法）； +2. 否则，成员裸的 `package.name`——如果两个以上成员共享它，拒绝并点名每一个匹配； +3. 否则，成员在 `[workspace] members` 里写的路径，或其目录名的最后一段（历史拼法， + 作为回落保留）。 ```bash -mcpp build -p server # matches apps/server +mcpp build -p server # matches apps/server（按目录或包名） mcpp test -p core # matches libs/core mcpp run -p server -- --port 8080 ``` +参数值若既是某个成员的包名，又是另一个成员的目录，选中包名所命名的那个成员，并给 +出警告点名另一个成员——选项名的是包，包名的精确匹配压过恰好同名的目录。 + `--workspace`（用于 `build` 与 `test`）是扇出形式：作用于**每个**成员。 `mcpp test --workspace` 逐成员分别汇报，遇失败继续，只要有任一成员失败就非零 退出——很适合作为"一个测试众多库的工作空间"单条、无需 shell 的 CI 步骤。 diff --git a/docs/zh/09-commands-by-scenario.md b/docs/zh/09-commands-by-scenario.md index cc93afd00..8861bd934 100644 --- a/docs/zh/09-commands-by-scenario.md +++ b/docs/zh/09-commands-by-scenario.md @@ -173,11 +173,42 @@ $ mcpp index status 是传播问题而不是命名问题 —— 索引以 artifact 而非 git clone 的形式到达 客户端。 +索引可以要求比正在运行的 mcpp 更新的版本（`index.toml` 的 `min_mcpp`）。 +这不是本次运行的错误（2026.9.28.1+）。一次刷新若取回这样的索引，会保留先前 +的副本，运行以一行结尾： + +``` +tip: the refreshed package index `mcpplibs` requires a newer mcpp; this run used the previous index. It requires mcpp >= 2026.10.1.1; this is mcpp 2026.9.28.1. Upgrade: xlings update mcpp +``` + +没有刷新索引的运行不提及它。因某个包只由这样的索引提供而失败的运行，在使它 +停止的消息中给出 E0006。`mcpp self doctor` 列出正在运行的 mcpp 不满足其下限 +的每一个索引。更早的版本在读到这样的索引时，把 E0006 的文字作为 `error:` +打印在运行开头，其后成功的运行也是如此。 + `--offline`（或 `MCPP_OFFLINE=1`）在单次调用中禁止网络，宁可失败也不拉取。 `--locked` 在解析结果与 `mcpp.lock` 不一致时失败，而不是改写它，这正是 CI 作业需要的形状。`mcpp index pin ` 把一个自定义索引的某个 commit 记进 `mcpp.toml`;`mcpp index unpin` 移除它。 +## 下载进度 + +每一次获取都由同一个渲染器报告（2026.9.28.1+）： + +- 工具链或载荷的安装； +- 来自索引的库包； +- `[xlings]` 载荷； +- 索引刷新； +- `git` 依赖的克隆； +- 沙箱首次运行时的工具。 + +在终端上，每一项是一个原地重绘的进度条。标准输出不是终端时（例如 CI 日志或 +管道），每一项在开始时打印一行（已知时带上大小），结束时打印一行（带上耗时）。 +这样的输出不含回车符，也不含擦除序列。`--quiet` 两者都不打印。 + +当 mcpp 驱动的 xlings 为索引刷新发出进度事件时（xlings 2026.9.28.1+），索引刷新 +逐步报告。较旧的 xlings 下，它显示其状态行，然后安静地结束，与以前相同。 + ## 发布前校验描述符 `mcpp xpkg parse` 用解析器自己的文法读一个描述符，所以它报告的就是解析时 diff --git a/docs/zh/20-toolchains.md b/docs/zh/20-toolchains.md index 5199c8932..82f6a5682 100644 --- a/docs/zh/20-toolchains.md +++ b/docs/zh/20-toolchains.md @@ -1058,7 +1058,7 @@ libc++.a/libc++abi.a/libunwind.a。更低的 macOS 下限（11–13）需要一 |---|---|---| | ELF（Linux 等） | `toolchain-coupled` | ELF 只有一个全局符号命名空间，先加载的定义胜出。一个静态内嵌了 libstdc++ 的 `.so` 会把它**导出**，链接该库的可执行文件于是把自己的 `std::` 引用绑定到那里——它自己的 `self-contained` 契约会静默变成空操作，它的 C++ 运行时变成碰巧加载到的那一份该库。 | | Mach-O | `self-contained` | 那里的机制本来就是 `-load_hidden`，即隐藏可见性，dyld 因此从不归一这些符号；而且 macOS 上根本没有 toolchain-coupled 这一档（见下文注记）。 | -| PE(Windows) | `self-contained` | PE 没有全局符号命名空间——导入按 DLL 逐个按名解析，一个 DLL 的私有运行时不可能被别的东西捡走。 | +| PE，GNU ABI(MinGW) | `self-contained` | PE 没有全局符号命名空间——导入按 DLL 逐个按名解析，一个 DLL 的私有运行时不可能被别的东西捡走。MSVC ABI 自己的默认值是另一条规则——见下文[在 MSVC 运行时上](#在-msvc-运行时上)。 | 在 ELF 上显式写 `shared = "self-contained"` 是支持的，而且就是字面意思： 库会内嵌运行时。此时 mcpp 会额外为标准库归档传递 @@ -1134,27 +1134,69 @@ cxx_runtime = { shared = "self-contained" } ### 在 MSVC 运行时上 -这里的机制就是 CRT 模型，而它是一个**整个工程**级的开关：cl 会把 +CRT 模型是**目标 ABI** 的属性，不是编译器的属性：`cl` 与以 +`*-windows-msvc` 为目标的 clang++（`llvm` 行）接收**同一个**模型，各自以 +自己驱动的拼写发出。它同时是一个**整个工程**级的开关：`cl` 会把 `_MSVC_MT`/`_MSVC_MD` 烘进一个工程唯一构建的那份 `std` 模块，所以一个与 工程不一致的按角色契约无法被兑现，会被报出来，而不是被忽略。 -| 取值 | 在 MSVC 上的含义 | -|---|---| -| `self-contained` | `/MT`——静态 CRT。`linkage = "static"` 从 libc 那根轴选中的是同一件事。 | -| `host-coupled`（`/MD` 下的默认值） | 由目标机器提供 `vcruntime140.dll` / `msvcp140.dll`——即那台机器装了 Visual Studio 或对应的 redistributable。 | -| `toolchain-coupled` | toolset **自带**的那份 DLL 跟着产物一起走。 | +| 取值 | 在 MSVC ABI 上的含义 | `cl` 的拼写 | clang++ 的拼写 | +|---|---|---|---| +| `self-contained`(或 `linkage = "static"`) | 静态 CRT | `/MT` | `-fms-runtime-lib=static` | +| `toolchain-coupled`(**默认值**) | 动态 CRT，toolset 自带的 `vcruntime140.dll`/`msvcp140.dll` 会被放到产物旁边 | `/MD` | `-fms-runtime-lib=dll` | +| `host-coupled` | 动态 CRT，不放置任何文件——由目标机器自己提供这些 DLL(Visual Studio,或 redistributable 安装程序) | `/MD` | `-fms-runtime-lib=dll` | -`toolchain-coupled` 值得说清楚，因为直觉上的理解是错的。`ucrtbase.dll` -**是**一个 Windows 组件（Windows 10 起），mcpp 从不分发它；而 -`vcruntime140.dll` 与 `msvcp140.dll` **不是**：每个 MSVC toolset 都在 +**`toolchain-coupled` 是默认值**，不论 `cxx_runtime` 写了什么，对每个角色 +皆然。这一点值得说清楚，因为「默认即可移植」这个直觉在这里是错的： +`ucrtbase.dll` **是**一个 Windows 组件(Windows 10 起)，mcpp 从不分发它； +而 `vcruntime140.dll` 与 `msvcp140.dll` **不是**：每个 MSVC toolset 都在 `VC\Redist\MSVC\\\` 下带着它们，和一个 gcc 载荷带着 -`libstdc++.so` 是同一件事。在这份契约下，mcpp 会把它们放到产物旁边——这 -正是让一次默认的 `/MD` 构建，能在一台只装了被钉住的 toolset、完全没有 +`libstdc++.so` 是同一件事。在这份契约下，mcpp 会把它们放到产物旁边(在 +`mcpp build` 的产出目录里)，并把同一个目录放上 `mcpp run`/`mcpp test` +的搜索路径——这正是让默认构建，能在一台只装了被钉住的 toolset、完全没有 Visual Studio 的机器上运行起来的原因。 -调试版 CRT（`debug_nonredist\` 下的 `vcruntime140d.dll` 等）永远不会被 -放进去：它不可再分发。 - +一个不带 `VC\Redist\MSVC` 目录的 toolset(在某些 `msvc@system` 安装上实测 +存在)无法兑现 `toolchain-coupled`。此时未声明的默认值会静默解析为 +`host-coupled`——这是这一行的一个属性，在此说明一次，不是每次构建都打印 +的警告。在这样的行上**显式**写 `cxx_runtime = "toolchain-coupled"` 会被 +拒绝，并指出缺失的目录：一个 toolset 兑现不了的显式声明是一个错误，绝不 +是一次静默降级。 + +调试版 CRT(`debug_nonredist\` 下的 `vcruntime140d.dll` 等)永远不会被 +放进去：它不可再分发，而且 mcpp 的 `dev` profile 不会选中它——那个 +profile 表达的是调试信息，不是另一个 CRT。这根轴留待有消费者需要时再设计。 + +把 `toolchain-coupled` 或 `host-coupled` 和 `/MT`(`linkage = "static"`, +或 `self-contained`)一起写是一处**矛盾**，而不是缺功能——一份静态 CRT +根本没有 DLL 可以耦合——所以它会被报出来，并落回 `self-contained`。 +`mcpp pack` 兜底另一半:一个什么都不打包的模式(`--mode static` 或 +`--mode system`)配上一个**显式**的 `toolchain-coupled` 兑现不了，会直接 +拒绝；而 `--mode system` 用在一个从未声明契约的工程上，会把默认值解析为 +`host-coupled`——一个显式的 mode 胜过一个默认值。 + +**自由拼写的 CRT 词永远是第二次声明。** 每个 MSVC ABI 构建现在都会声明 +自己的 CRT,所以 `[build] cxxflags` 或 `dialect_cxxflags` 里出现的字面 +`/MT`、`/MD`、`/MTd`、`/MDd` 或 `-fms-runtime-lib=*`(两种短横线拼写皆 +可)永远不能是唯一的声音。与已解析的模型**一致**的会被警告为冗余，并指 +出应当改写的键(`cxx_runtime` 或 `linkage`)；**不一致**的会被拒绝，消息 +指出该词、它所在的键，以及它对应的取值。依赖包的 `[build] cxxflags` +同样检查，因为它们作用于该包自己的单元：不一致的会被拒绝并指出该包， +一致的不警告，因为 `cxx_runtime` 是根的键。调试 CRT 词(`/MTd`、`/MDd`、 +`-fms-runtime-lib=*_dbg`)总被拒绝：模型没有调试这一轴，标准库模块与链接 +使用的是发布版 CRT。引擎绝不让命令行上最后一个词静默胜出。 + +> **升级到 2026.9.28.1?** `cl` 行的工程不受影响，只是程序旁多了被放置 +> 的 DLL。**LLVM 行的程序会从静态 CRT 换到动态 CRT**：这次发布之前， +> MSVC ABI 上的 clang++ 收不到任何模型，总是链接 `libcmt`，与 +> `cxx_runtime` 无关；现在它收到与 `cl` 相同的模型，默认解析为 +> `toolchain-coupled`。一个在这一行链接 `/MT` 预构建库的工程，现在会链接 +> 失败(`LNK2038`,一处 CRT 不一致)，应当写 +> `cxx_runtime = "self-contained"` 以恢复它此前的静态 CRT。有两类 +> manifest 在这次发布前能构建、之后会被拒绝:一个与已解析模型矛盾的自由 +> 拼写 CRT 词，以及在一个 toolset 不带 redistributable 的行上显式写 +> `toolchain-coupled`——两者见上文。 +> > **从 2026.8.15 或更早版本升级时的变化。** 这个键在 MSVC ABI 上曾经是 > **空操作**——它会报 `not implemented for the MSVC runtime yet`，写任何 > 值都会退回 `/MD`。自 2026.8.16 起它真的会生效，于是一份从那个年代带着 @@ -1163,21 +1205,6 @@ Visual Studio 的机器上运行起来的原因。 > 这个值一直是合法的，这次切换是**静默**的。如果一个工程是在这个键尚未 > 生效时写下它的，应当重新确认所需的取值。 -把它和 `/MT` 一起写是一处**矛盾**，而不是缺功能——一份静态 CRT 根本没有 -DLL 可以耦合——所以它会被报出来，并落回 `self-contained`。另一半由 -`mcpp pack` 兜底：一个什么都不打包的模式（`--mode system`、 -`--mode static`）兑现不了 `toolchain-coupled`，会直接拒绝。 - -**MSVC ABI 上的 clang**（`x86_64-windows-msvc` 的 `llvm` 行，记录自 mcpp -2026.9.16.1 起）。上表描述的是 `cl.exe`，mcpp 只向它传递 CRT 模型。MSVC -ABI 上的 clang 使用 GNU 方言，收不到任何模型，它的驱动链接静态 CRT -(`-defaultlib:libcmt`)：这一行构建出的程序不会导入 -`vcruntime140.dll`、`msvcp140.dll` 或 `api-ms-win-crt-*`，每个 DLL 各自 -带着自己的 CRT。因此这一行无论 `cxx_runtime` 写什么都是 -`self-contained`，`resolution.json` 如实记录这一点，显式写 -`host-coupled` 或 `toolchain-coupled` 会打印这一行兑现不了它。需要在 -MSVC ABI 上使用动态 CRT 的工程，应当用 `msvc@system` 构建。 - **边界。** 该契约只管辖 C++ 运行时。静态 **libc** 是另一根轴 (`linkage = "static"` / `--static`，例如一个 musl target)，部署下限是 第三根轴——Apple target 用 `[package]` 里的 diff --git a/docs/zh/50-machine-output.md b/docs/zh/50-machine-output.md index caae0be3a..e8fdee48a 100644 --- a/docs/zh/50-machine-output.md +++ b/docs/zh/50-machine-output.md @@ -393,6 +393,7 @@ replaced}` —— `origin` 与构建的状态行使用的是同一句话 | `lld-required-absent` | 目标直接通过 lld 链接，而解析出的工具链载荷不带 lld | | `host-tool-toolchain` | 一个交叉 `--target` 下的 `build.mcpp` 需要一个可解析的**宿主**工具链，而一个都没有配置 | | `std-module-precompile` | 标准库的模块在这个配置下无法被预编译 | +| `msvc-redist-unavailable` | 在 MSVC ABI 的行上显式写了 `cxx_runtime = "toolchain-coupled"`，而该行的工具集没有可放置的 redistributable 目录 *(2026.9.28.1+)* | | `other` | 一个尚未被赋予记号的拒绝分支 | **其中一个记号也由 `mcpp build` 自己打印。** `interface-not-provided` 会 @@ -423,7 +424,7 @@ mcpp emit build-database [--spec s1|compile-commands] --format json | 字段 | | |---|---| -| `spec` | `{"name": "s1", "version": "0.2.0"}`；使用 `--spec compile-commands` 时为 `{"name": "compile-commands"}` | +| `spec` | `{"name": "s1", "version": "0.3.0"}`（2026.9.28.1 之前为 0.2.0）；使用 `--spec compile-commands` 时为 `{"name": "compile-commands"}` | | `database` | 该规范对应的文档：一份 S1 构建数据库，或者 `mcpp build --configure-only` 写入 `compile_commands.json` 的那些条目 | | `watch` | 一旦发生变化就可能改变这份文档的输入：相对工作区根目录的路径与 glob，或绝对路径 | | `inputs-fingerprint` | `fnv1a:<16 位十六进制>`，对上述输入、mcpp 版本与选择器求出的摘要 | @@ -432,6 +433,11 @@ mcpp emit build-database [--spec s1|compile-commands] --format json 写入 ``。文档的内容、不写入项目目录这条保证，以及 `watch` 的规则，见 [SPEC-005](../specs/build-database.md)。 +自 2026.9.28.1 起，S1 的集合还在 `ide.generated` 中列出其包的构建程序生成的文件 +（S1 0.3.0 §7.2）。命令不运行任何 action，所以这样的文件在文档所指的目录中尚不存在。 +每一项给出同一组选择下 `mcpp build` 写入的路径（`build-path`）以及写出它的步骤， +编辑器因此可以说明“这是生成的文件，构建一次即可”，而不是报告缺少头文件。 + `emit` 独立规划每一个被选中的成员（#699 第 1 项）：一个成员的规划失败不会 连累它的兄弟成员。不在项目中，或者被选中的成员全部规划失败时，信封省略 `data` 并以 1 退出，每个失败的成员各带一条诊断：不在项目中是 @@ -466,6 +472,7 @@ mcpp emit build-database [--spec s1|compile-commands] --format json | `MCPP_BUILD_DATABASE_STD_UNIT_UNDESCRIBED` | 警告 | 没有任何标准库构建命令点名它的模块源文件，该单元因此不被列出 | | `MCPP_BUILD_DATABASE_HOST_TOOL_DEFERRED` | 说明 | 被请求的宿主工具不在工具库中，命令不构建它；计划给出它将被发布到的路径（2026.9.27.1+；取代 2026.9.26.2 的警告 `MCPP_BUILD_DATABASE_HOST_TOOL_UNBUILT`） | | `MCPP_BUILD_DATABASE_PROGRAM_FAILED` | 错误 | 构建程序失败；它所属的包被描述为不含它产生的指令 | +| `MCPP_INDEX_REQUIRES_NEWER_MCPP` | 说明 | 本次运行刷新的某个索引要求更新的 mcpp；先前的副本被保留或恢复，或者没有可用的副本（2026.9.28.1+；终端运行以结尾的 `tip:` 行打印同一条说明） | `--protocol-version` 为这条命令声明 `init-mcpp-home`、`read-project`、 `network`、`write-global-cache` 与 `exec-build-script`，从不声明 diff --git a/mcpp.toml b/mcpp.toml index e70ad04ef..7d81aec89 100644 --- a/mcpp.toml +++ b/mcpp.toml @@ -1,6 +1,6 @@ [package] name = "mcpp" -version = "2026.9.27.1" +version = "2026.9.28.1" description = "Modern C++ build & package management tool" license = "Apache-2.0" authors = ["mcpp-community"] diff --git a/modules/manifest/src/toml.cppm b/modules/manifest/src/toml.cppm index 8d298d10b..3b02a298e 100644 --- a/modules/manifest/src/toml.cppm +++ b/modules/manifest/src/toml.cppm @@ -3712,6 +3712,11 @@ std::expected parse_string(std::string_view content, "belongs in [build].sources")); } read_list("sources", cc.inputs.sources); + // #717: a GRAPH-WIDE switch, not a per-package build input -- + // read into `cc.dialectCxxflags` (a ConditionalConfig member of + // its own), never into `cc.inputs`. See that member's comment + // for why it is kept apart from BuildInputs. + read_list("dialect_cxxflags", cc.dialectCxxflags); // #296: package-level macros are a build input like any other, // so the cfg axis carries them too — a platform-only macro // (`[target.'cfg(windows)'.build] defines = ["USE_WIN32"]`) @@ -3746,14 +3751,15 @@ std::expected parse_string(std::string_view content, cc.inputs.globFlags)) return std::unexpected(error(origin, *err)); } - // The conditional axis carries BuildInputs and nothing else, so - // its vocabulary is exactly that struct's members — a key - // outside it (`static_stdlib`, `target`, a profile knob) is not - // conditionable and would otherwise vanish without a word, the - // #296 failure mode. MUST stay in sync with the reads above and - // with types.cppm's BuildInputs. + // The conditional axis carries BuildInputs, plus the one + // graph-wide member `dialect_cxxflags` (#717; ConditionalConfig, + // not BuildInputs), so its vocabulary is exactly those members — + // a key outside them (`static_stdlib`, `target`, a profile knob) + // is not conditionable and would otherwise vanish without a + // word, the #296 failure mode. MUST stay in sync with the reads + // above and with types.cppm's BuildInputs and ConditionalConfig. static constexpr std::string_view kKnownConditionalBuildKeys[] = { - "cflags", "cxxflags", "defines", "flags", + "cflags", "cxxflags", "defines", "dialect_cxxflags", "flags", "include_dirs", "include_dirs_after", "ldflags", "private_include_dirs", "sources", "std-module-flags", }; diff --git a/modules/manifest/src/types.cppm b/modules/manifest/src/types.cppm index 4e9f94a82..1a3cc555e 100644 --- a/modules/manifest/src/types.cppm +++ b/modules/manifest/src/types.cppm @@ -1441,6 +1441,23 @@ struct ConditionalConfig { // `[target..abi] exceptions` -- see BuildConfig::abiExceptions. bool abiExceptions = false; bool abiExceptionsDeclared = false; + // `[target..build] dialect_cxxflags` -- a GRAPH-WIDE switch under a + // target condition (issue #717, design 2026-09-27 §6). Like `abiThreads`/ + // `abiExceptions` above, only the ROOT's value is rendered; unlike them it + // is ADDITIVE rather than scalar, so there is no "declared" pair -- an + // absent section contributes nothing, exactly as an absent entry in a list + // does everywhere else. + // + // Deliberately NOT a member of `inputs` (BuildInputs): that type is the + // additive, PER-PACKAGE surface a condition may carry (types.cppm's + // BuildInputs comment), and this key is graph-wide -- a dependency's own + // conditional row must reach no command, the same rule its unconditional + // `[build] dialect_cxxflags` already follows. Recorded here for EVERY + // package (merge_conditional_config runs for each one), and merged into + // `BuildConfig::dialectCxxflags` -- the one variable every consumer (the + // scan, the std BMI prebuild, every TU, the fingerprint) already reads -- + // only for the package that is the root of this build. + std::vector dialectCxxflags; // `[target.] requires_abi = { ... }` -- design 2026-09-12 (the UI // framework record) section 2.6, A6: a requirement can sit on the target // axis, because the sources it gates (`[target..build] sources`) are @@ -1534,7 +1551,12 @@ inline bool is_empty(const ConditionalConfig& c) { && c.targetKinds.empty() && c.xlings.empty() && !c.abiThreadsDeclared && !c.abiExceptionsDeclared && !c.requiresAbiThreads && !c.requiresAbiExceptions - && c.featureRequiresAbiThreads.empty() && c.featureRequiresAbiExceptions.empty(); + && c.featureRequiresAbiThreads.empty() && c.featureRequiresAbiExceptions.empty() + // #717: `dialect_cxxflags` is a member of ConditionalConfig, not of + // `c.inputs` (BuildInputs), so `is_empty(c.inputs)` above does not see + // it -- a section carrying only this key would otherwise be dropped + // before it is ever evaluated, the same failure #296 was filed for. + && c.dialectCxxflags.empty(); } // `[lib]` — library "root" interface convention. diff --git a/modules/platform/src/process.cppm b/modules/platform/src/process.cppm index 446a77bcb..3dd90aca0 100644 --- a/modules/platform/src/process.cppm +++ b/modules/platform/src/process.cppm @@ -261,11 +261,17 @@ int run_streaming(std::string_view command, // Stdout and stderr share the pipe, as they do for every captured deadline // run; a caller that parses stdout redirects stderr in the command itself. // Returns -1 when the launcher could not run the command at all. +// +// `split_on_cr` also ends a line at a carriage return that is not part of a +// CRLF pair. A command that redraws one progress line in place (git with +// `--progress`) then reaches `on_line` once per redraw, as it happens, rather +// than once per finished phase. int run_streaming_bounded(std::string_view command, std::function on_line, std::chrono::milliseconds total, std::chrono::milliseconds idle, - bool* timed_out); + bool* timed_out, + bool split_on_cr = false); // Run `command`, passing stdout/stderr through to the terminal. // Optionally captures stdout into `output` if non-null. @@ -931,7 +937,8 @@ BoundedOutcome dispatch_bounded( std::string_view windowsCommandLine = {}, std::chrono::milliseconds idle = std::chrono::milliseconds{0}, bool ownGroup = false, - const std::function* on_line = nullptr) + const std::function* on_line = nullptr, + bool splitOnCr = false) { BoundedOutcome outcome; @@ -957,7 +964,8 @@ BoundedOutcome dispatch_bounded( struct StreamCtx { std::string* buffer; const std::function* on_line; - } streamCtx{ &outcome.output, on_line }; + bool splitOnCr; + } streamCtx{ &outcome.output, on_line, splitOnCr }; using Sink = void (*)(void*, const char*, unsigned long); const Sink sink = !capture ? nullptr : on_line == nullptr @@ -967,8 +975,22 @@ BoundedOutcome dispatch_bounded( : +[](void* ctx, const char* data, unsigned long len) { auto* c = static_cast(ctx); c->buffer->append(data, len); + // The end of the next line: a newline, or with `splitOnCr` a + // carriage return followed by anything but a newline. A carriage + // return that is the last byte so far waits for the next chunk, + // which decides whether it was half of a CRLF. + auto next_end = [c]() -> std::size_t { + const auto nl = c->buffer->find('\n'); + if (!c->splitOnCr) return nl; + for (std::size_t i = 0; i < c->buffer->size() && i < nl; ++i) { + if ((*c->buffer)[i] != '\r') continue; + if (i + 1 == c->buffer->size()) return std::string::npos; + if ((*c->buffer)[i + 1] != '\n') return i; + } + return nl; + }; std::size_t pos; - while ((pos = c->buffer->find('\n')) != std::string::npos) { + while ((pos = next_end()) != std::string::npos) { std::string_view line{c->buffer->data(), pos}; while (!line.empty() && line.back() == '\r') line.remove_suffix(1); (*c->on_line)(line); @@ -1067,7 +1089,8 @@ int run_streaming_bounded(std::string_view command, std::function on_line, std::chrono::milliseconds total, std::chrono::milliseconds idle, - bool* timed_out) + bool* timed_out, + bool split_on_cr) { if (timed_out) *timed_out = false; if (command.empty() || (total.count() <= 0 && idle.count() <= 0)) return -1; @@ -1079,7 +1102,7 @@ int run_streaming_bounded(std::string_view command, on_line ? std::move(on_line) : [](std::string_view) {}; auto r = dispatch_bounded(argv, {}, {}, total, /*capture=*/true, windows_shell_command_line(sealed), idle, - /*ownGroup=*/true, &sink); + /*ownGroup=*/true, &sink, split_on_cr); if (!r.supported) return -1; if (timed_out) *timed_out = r.timed_out; return r.exit_code; diff --git a/modules/toolchain-model/src/dialect.cppm b/modules/toolchain-model/src/dialect.cppm index b721d3889..d83bf9783 100644 --- a/modules/toolchain-model/src/dialect.cppm +++ b/modules/toolchain-model/src/dialect.cppm @@ -162,6 +162,48 @@ constexpr bool msvc_wants_static_crt(std::string_view linkage, return linkage == "static" || cxxRuntime == "self-contained"; } +// THE CRT MODEL WORD, SPELLED FOR WHICHEVER DRIVER `tc` IS. +// +// Every MSVC-ABI row now receives the model — cl and clang++ targeting +// `*-windows-msvc` alike (#649 E10, #718) — because the CRT is a property of +// the target ABI, not of the compiler. `dialect_for` still gives clang the +// GNU dialect (its `-I`/`-D`/... spellings are unchanged), so this is NOT +// `msvc_crt_flag(dialect_for(tc), ...)`: the GNU dialect's own +// `staticRuntime` is `-static`, full static linking, a different axis this +// function must not be confused with. +// +// cl.exe /MT | /MD +// clang++ *-windows-msvc -fms-runtime-lib=static | -fms-runtime-lib=dll +// every other row "" (MinGW links the MSVC CRT to no row at all) +// +// ONE HELPER, so the translation units (flags.cppm), the std and std.compat +// BMIs (stdmod.cppm via clang.cppm/msvc.cppm) and the link command +// (flags.cppm's `LinkShape::PeLld` branch) cannot spell three different +// answers to the same question — which is exactly how E10 arose: a flag +// given only at compile time does not reach clang's link step, which chooses +// `-defaultlib:` on its own (measured, `2026-09-16-646-649-four-issues-by- +// home.md` §4.5). +std::string msvc_abi_crt_word(const Toolchain& tc, bool staticCrt); + +// A FREE-FORM CRT WORD IS ALWAYS A SECOND STATEMENT (D3, #718). +// +// Every MSVC-ABI build now states its own CRT model, so a literal spelling of +// it in `[build] cxxflags` or `dialect_cxxflags` — cl's `/MT`/`/MD`(`d`) or +// clang's `-fms-runtime-lib=*` (either dash) — can never be the only voice: +// either it repeats what mcpp already resolved, or it contradicts it, and the +// engine must never let the last word on the command line decide silently. +// +// `word` is one token already read out of a flags list (SPEC-004 §8's +// per-word reading). `key` names where the caller found it, so the message +// can point back at it. Returns nullopt for a spelling this function does not +// recognise as a CRT word — it says nothing about flags outside this axis. +struct CrtWordVerdict { + bool contradicts; + std::string message; +}; +std::optional check_crt_word(std::string_view word, + bool staticCrt, + std::string_view key); // The two dialect rows, reachable without a Toolchain. Exposed so the MSVC // row — which no build reaches until the cl.exe backend lands — can still be @@ -265,6 +307,14 @@ const CommandDialect& dialect_for(const Toolchain& tc) { const CommandDialect& gnu_dialect() { return kGnuDialect; } const CommandDialect& msvc_dialect() { return kMsvcDialect; } +std::string msvc_abi_crt_word(const Toolchain& tc, bool staticCrt) { + if (tc.compiler == CompilerId::MSVC) + return std::string(msvc_crt_flag(msvc_dialect(), staticCrt)); + if (is_msvc_target(tc)) + return staticCrt ? "-fms-runtime-lib=static" : "-fms-runtime-lib=dll"; + return {}; +} + std::string lib_flag_for(const CommandDialect& d, std::string_view name) { // Two shapes, one table entry: `{}` marks where the name goes, which is // a prefix position for GNU and a suffix position for MSVC. @@ -283,4 +333,56 @@ std::string std_flag_for(const CommandDialect& d, return std::format("{}{}", d.stdPrefix, canonical); } +std::optional check_crt_word(std::string_view word, + bool staticCrt, + std::string_view key) { + // Both dash conventions: cl.exe accepts `-MD` exactly as it does `/MD`, + // and a manifest that targets more than one driver from one list (SPEC- + // 004 §8) should not have to spell the word twice. + std::optional wantsStatic; + if (word == "/MT" || word == "-MT" || word == "/MTd" || word == "-MTd" + || word == "-fms-runtime-lib=static" + || word == "-fms-runtime-lib=static_dbg") { + wantsStatic = true; + } else if (word == "/MD" || word == "-MD" || word == "/MDd" || word == "-MDd" + || word == "-fms-runtime-lib=dll" + || word == "-fms-runtime-lib=dll_dbg") { + wantsStatic = false; + } else { + return std::nullopt; + } + + // A DEBUG CRT WORD IS NEVER REDUNDANT. The model states static or dynamic + // and nothing about debug (that axis is deferred, docs/20), while the std + // module and the link are built against the release CRT it resolved. A + // debug word therefore compiles its units against a CRT nothing else uses. + const bool debugWord = word.ends_with("d") || word.ends_with("_dbg"); + if (debugWord) { + return CrtWordVerdict{true, std::format( + "`{}` in {} asks for a debug CRT, which the CRT model does not " + "express; the standard library module and the link use {}. Remove " + "the flag", + word, key, staticCrt ? "the static CRT, /MT" : "the dynamic CRT, /MD")}; + } + std::string_view wordValue = + *wantsStatic ? "the static CRT (/MT)" : "the dynamic CRT (/MD)"; + if (*wantsStatic == staticCrt) { + return CrtWordVerdict{false, std::format( + "`{}` in {} agrees with the CRT model this build already " + "resolved and says nothing new. Write {} instead if {} should " + "stay an explicit statement, and drop the flag", + word, key, + staticCrt ? "`cxx_runtime = \"self-contained\"` (or `linkage = \"static\"`)" + : "`cxx_runtime = \"toolchain-coupled\"` (or `\"host-coupled\"`)", + wordValue)}; + } + return CrtWordVerdict{true, std::format( + "`{}` in {} asks for {}, which contradicts the CRT model this build " + "already resolved ({}). Every MSVC-ABI row now states its own CRT, " + "so the word can never be the only voice: remove it, or change " + "`cxx_runtime`/`linkage` to match", + word, key, wordValue, + staticCrt ? "the static CRT, /MT" : "the dynamic CRT, /MD")}; +} + } // namespace mcpp::toolchain diff --git a/modules/toolchain-model/src/model.cppm b/modules/toolchain-model/src/model.cppm index a6bf1046a..27f359dc4 100644 --- a/modules/toolchain-model/src/model.cppm +++ b/modules/toolchain-model/src/model.cppm @@ -282,6 +282,20 @@ struct Toolchain { std::string msvcOrigin; // "system" | "managed" std::string msvcProduct; // for the one printed line std::filesystem::path windowsSdkRoot; // /10 or the payload + // THE TOOLSET'S OWN REDISTRIBUTABLE CRT DIRECTORY + // (`VC\Redist\MSVC\\\Microsoft.VC*.CRT`), on an MSVC-ABI row of + // EITHER driver — cl.exe or clang++ targeting `*-windows-msvc`. + // + // Its own field rather than a reading of `linkRuntimeDirs`, because that + // field means something else on the two rows: for cl it happens to hold + // exactly this directory, but for clang it holds the LLVM payload's OWN + // runtime directories (`discover_link_runtime_dirs`, clang.cppm), and + // reading it for staging there would copy LLVM's files into a Windows + // program's `bin/`. Empty when the resolved toolset ships no + // redistributable (`msvc_abi_default_contract` then defaults to + // host-coupled rather than toolchain-coupled, and an explicit + // toolchain-coupled contract is refused, naming this field empty). + std::filesystem::path msvcRedistDir; // Something about HOW this toolchain was resolved that the user has to be // told, but which is not a failure. Non-empty ⇒ the caller MUST surface it. // diff --git a/modules/versioning/src/version.cppm b/modules/versioning/src/version.cppm index c07002110..b25a0082d 100644 --- a/modules/versioning/src/version.cppm +++ b/modules/versioning/src/version.cppm @@ -31,6 +31,6 @@ import std; export namespace mcpp { -inline constexpr std::string_view MCPP_VERSION = "2026.9.27.1"; +inline constexpr std::string_view MCPP_VERSION = "2026.9.28.1"; } // namespace mcpp diff --git a/src/build/build_database.cppm b/src/build/build_database.cppm index efbc618cb..ffdad9b56 100644 --- a/src/build/build_database.cppm +++ b/src/build/build_database.cppm @@ -41,7 +41,8 @@ import mcpp.toolchain.stdmod; export namespace mcpp::build::database { -inline constexpr std::string_view kProfileVersion = "0.2.0"; +// 0.3.0: a set carries `generated` (S1 section 7.2, mcpp#724). +inline constexpr std::string_view kProfileVersion = "0.3.0"; inline constexpr std::string_view kStdSetName = "mcpp:std"; // One planned member of the document. @@ -271,6 +272,10 @@ struct SetData { std::string familyName; std::string kind; nlohmann::json units = nlohmann::json::array(); + // S1 section 7.2: the files this plan generates and the set's units + // compile or include, and the generated include directories they name. + nlohmann::json generated = nlohmann::json::array(); + std::set generatedDirs; }; } // namespace @@ -365,6 +370,22 @@ Rendered render(std::span members, return it->second; }; + // GENERATED FILES (S1 0.3.0 section 7.2, mcpp#724). A planning pass + // runs no action (SPEC-005 R2.5), so a file a rule generates is absent + // from the directory this document names until a build writes it. The + // plan knows each generating step; the document states it, together + // with the path the build of the same configuration writes, so that a + // reader can tell "not built yet" from "missing" without knowing how + // the planning directory maps onto the project's own `target/`. + const auto planRoot = member.workDir.empty() ? ctx.projectRoot : member.workDir; + const auto generatedTree = (planRoot / "target" / ".build-mcpp").lexically_normal(); + auto build_path = [&](const std::filesystem::path& p) { + if (!member.workDir.empty()) + if (auto rel = relative_to(p, member.workDir); rel && !rel->empty()) + return native_string(ctx.projectRoot / std::filesystem::path(*rel)); + return native_string(p); + }; + const auto flags = mcpp::build::compute_flags(ctx.plan); auto invocations = mcpp::build::unit_invocations(ctx.plan, flags); if (!toolchains.contains(tcId)) @@ -378,6 +399,10 @@ Rendered render(std::span members, : "library"; auto& set = set_for(member.setPrefix + package + (isTest ? ":test" : ""), package, kind); + for (auto const* dirs : {&cu.localIncludeDirs, &cu.localIncludeDirsAfter}) + for (auto const& dir : *dirs) + if (relative_to(dir, generatedTree)) + set.generatedDirs.insert(native_string(dir.lexically_normal())); nlohmann::json provides = nlohmann::json::object(); if (!cu.providesModule.empty()) provides[cu.providesModule] = ""; nlohmann::json requires_ = nlohmann::json::array(); @@ -443,6 +468,53 @@ Rendered render(std::span members, // (prepare.cppm, onto BuildContext::planNotes) and reaches `r.notes` // through the unconditional copy below, with every other plan note. + { + using Role = mcpp::manifest::BuildAction::Role; + for (auto const& a : ctx.plan.actions) { + if (a.role != Role::Source || a.outputs.empty()) continue; + const std::string package = a.packageName.empty() ? rootName : a.packageName; + nlohmann::json inputs = nlohmann::json::array(); + for (auto const& in : a.inputs) + inputs.push_back(native_string(std::filesystem::path(in).lexically_normal())); + const nlohmann::json generator{ + {"id", a.id}, + {"inputs", std::move(inputs)}, + {"arguments", a.command}, + {"work-directory", native_string(a.cwd.empty() + ? ctx.plan.outputDir + : std::filesystem::path(a.cwd))}, + }; + // SPEC-005 R3.12: every set of the package lists what its build + // program generates, its test set included, since the units + // that include a header are not known without preprocessing. + // `kind` is per set: `source` where a unit of that set is + // compiled from the file. + for (auto& [name, set] : groups) { + if (set.familyName != package) continue; + for (auto const& out : a.outputs) { + const auto p = std::filesystem::path(out).lexically_normal(); + const auto path = native_string(p); + const bool isUnit = std::ranges::any_of(set.units, [&](auto const& u) { + return u.value("source", "") == path; + }); + set.generated.push_back(nlohmann::json{ + {"path", path}, + {"build-path", build_path(p)}, + {"kind", isUnit ? "source" : "header"}, + {"generator", generator}, + }); + } + } + } + for (auto& [name, set] : groups) + for (auto const& dir : set.generatedDirs) + set.generated.push_back(nlohmann::json{ + {"path", dir}, + {"build-path", build_path(std::filesystem::path(dir))}, + {"kind", "directory"}, + }); + } + for (auto const& name : order) { auto& set = groups.at(name); nlohmann::json visible = nlohmann::json::array(); @@ -459,6 +531,8 @@ Rendered render(std::span members, {"kind", set.kind}, }}, }; + if (!set.generated.empty()) + setJson["ide"]["generated"] = std::move(set.generated); split_baseline(setJson); sets.push_back(std::move(setJson)); } diff --git a/src/build/build_program.cppm b/src/build/build_program.cppm index 5e917239a..7be7e4054 100644 --- a/src/build/build_program.cppm +++ b/src/build/build_program.cppm @@ -1536,11 +1536,13 @@ std::expected run_build_program( } mcpp::ui::info("build.mcpp", "compiling"); // GCC resolves imported BMIs via gcm.cache/ relative to the compile cwd, so - // any compile that imports a module — `mcpp`, `std`, or both — has to run - // from bdir, where they were staged. One condition, not two: a build.mcpp - // that imports only std needs exactly the same cwd as one that imports - // only mcpp. Otherwise the project root is fine. - const bool needsBmiCwd = usesModule || stdStagedInBdir; + // any compile that imports a module — `mcpp`, `std`, a build rule's host + // module, or any mix — has to run from bdir, where they were staged or + // compiled. One condition: a build.mcpp that imports only a rule needs + // exactly the same cwd as one that imports only mcpp (a rule-only program + // compiled in the project root and failed with "failed to read compiled + // module", e2e 807 under GCC). Otherwise the project root is fine. + const bool needsBmiCwd = usesModule || stdStagedInBdir || !env.hostModules.empty(); std::string compileCwd = needsBmiCwd ? bdir.string() : root.string(); auto cres = mcpp::platform::process::capture_exec(compileArgv, compileEnv, compileCwd); diff --git a/src/build/cache_key.cppm b/src/build/cache_key.cppm index 984074f0f..dbca7f03a 100644 --- a/src/build/cache_key.cppm +++ b/src/build/cache_key.cppm @@ -55,6 +55,7 @@ import mcpp.libs.json; import mcpp.manifest; import mcpp.modgraph.scanner; import mcpp.toolchain.detect; +import mcpp.toolchain.dialect; import mcpp.toolchain.fingerprint; import mcpp.toolchain.linkmodel; import mcpp.toolchain.triple; @@ -199,6 +200,13 @@ struct BuildAxes { // Decided by `make_plan` (BuildPlan::needsPic) and passed in, so the // compiler flag and this key read the same bit. bool pic = false; + // The CRT model word on the MSVC ABI (`/MT`, `-fms-runtime-lib=dll`, ...), + // empty off it. Whole-build like `pic`: every object of the graph is + // compiled against it, and an object of one CRT in an image of the other + // is a second CRT state. Derived by the same helper and from the same two + // root keys as the flag builder (flags.cppm), so that switching + // `cxx_runtime` cannot be served an object compiled for the other model. + std::string crt; }; // Axes D/E/F for one package. @@ -330,6 +338,7 @@ nlohmann::json to_json(const BuildAxes& b, const PackageAxes& p) { {"lto", b.lto}, {"strip", b.strip}, {"pic", b.pic}, + {"crt", b.crt}, }; j["package"] = { {"index", p.indexName}, @@ -378,6 +387,7 @@ std::string key_hex(const BuildAxes& b, const PackageAxes& p) { put(s, "lto", b.lto ? "1" : "0"); put(s, "strip", b.strip ? "1" : "0"); put(s, "pic", b.pic ? "1" : "0"); + put(s, "crt", b.crt); // D put(s, "index", p.indexName); put(s, "pkg", p.packageName); @@ -410,6 +420,9 @@ BuildAxes build_axes(const mcpp::toolchain::Toolchain& tc, { BuildAxes b; b.pic = needsPic; + b.crt = mcpp::toolchain::msvc_abi_crt_word( + tc, mcpp::toolchain::msvc_wants_static_crt( + rootManifest.buildConfig.linkage, rootManifest.buildConfig.cxxRuntime)); b.compilerId = std::string(tc.compiler_name()); b.compilerVersion = tc.version; // Same rule the whole-project fingerprint uses: prefer the declared driver diff --git a/src/build/distribution.cppm b/src/build/distribution.cppm index 6819ded9d..139ecde9a 100644 --- a/src/build/distribution.cppm +++ b/src/build/distribution.cppm @@ -250,12 +250,38 @@ Contract default_contract(Role r, Format f) { return Contract::SelfContained; } +// THE MSVC-ABI WHOLE-PROJECT DEFAULT (§7.2, §7.3, #718), used in place of +// `default_contract` for EVERY role there. The CRT is a per-ABI switch, not a +// per-role judgement about a format's hazard the way `default_contract`'s PE +// cell is for MinGW: one contract answers for every role, because cl bakes +// `_MSVC_MT`/`_MSVC_MD` into the one std module a whole project shares. +// +// static CRT (`/MT`) SelfContained — no DLL dependency to couple to. +// dynamic CRT, redist found ToolchainCoupled — `/MD`, with the toolset's +// own vcruntime140.dll/msvcp140.dll staged +// beside the artifact (portable by default). +// dynamic CRT, no redist HostCoupled — nothing to stage; the row is as +// capable as it has ever been (#649 E10), and +// this is RECORDED rather than degraded with a +// warning (a property of the row, not of one +// build). +Contract msvc_abi_default_contract(bool staticCrt, bool hasRedist) { + if (staticCrt) return Contract::SelfContained; + return hasRedist ? Contract::ToolchainCoupled : Contract::HostCoupled; +} + // What a manifest states about the C++ runtime, read once for every role. struct ContractStatement { std::string_view cxxRuntime; // `cxx_runtime = "..."` or its `default` std::string_view cxxRuntimeTests; // `cxx_runtime = { tests = "..." }` std::string_view cxxRuntimeShared; // `cxx_runtime = { shared = "..." }` bool staticStdlib = true; + // The MSVC-ABI whole-project default (above), or nullopt off that ABI. + // When set, it replaces `default_contract` for every role — see + // `role_contracts`. Computed by the caller from `msvc_wants_static_crt` + // and the resolved toolset's redistributable directory, because those are + // toolchain facts this module does not otherwise see. + std::optional msvcAbiDefault; }; // Which images of a build load a C++ shared library the build itself makes: @@ -309,16 +335,22 @@ std::optional stated_shared_library_contract(std::string_view cxxRunti RoleContracts role_contracts(const ContractStatement& s, Format f, CxxSharedLoad load) { RoleContracts c; + // The MSVC-ABI default replaces `default_contract` for EVERY role passed + // through it below — the CRT model has no per-role judgement to make. + const Contract distributableDefault = s.msvcAbiDefault.value_or( + default_contract(Role::Distributable, f)); + const Contract sharedDefault = s.msvcAbiDefault.value_or( + default_contract(Role::SharedLibrary, f)); c.programStated = !s.cxxRuntime.empty() || !s.staticStdlib; c.program = parse_contract(s.cxxRuntime).value_or( - s.staticStdlib ? default_contract(Role::Distributable, f) : Contract::HostCoupled); + s.staticStdlib ? distributableDefault : Contract::HostCoupled); c.intermediate = c.program; c.testsStated = c.programStated || !s.cxxRuntimeTests.empty(); c.tests = parse_contract(s.cxxRuntimeTests).value_or(c.program); c.sharedStated = c.programStated || !s.cxxRuntimeShared.empty(); c.shared = stated_shared_library_contract(s.cxxRuntime, s.cxxRuntimeShared, s.staticStdlib, f) - .value_or(default_contract(Role::SharedLibrary, f)); + .value_or(sharedDefault); // ONE PROCESS, ONE C++ RUNTIME (#646 F3a). // @@ -383,13 +415,6 @@ struct MechanismInput { // decoration. Derived by `msvc_wants_static_crt`, which is also what // emits the flag. bool msvcStaticCrt = false; - // MSVC STL only: does mcpp pass a CRT model (`/MT` or `/MD`) to this - // compiler? True for cl.exe. FALSE FOR CLANG ON THE MSVC ABI: that driver - // speaks the GNU dialect, mcpp emits no runtime flag for it, and clang - // then links the static CRT (`-defaultlib:libcmt`, measured with the - // 22.1.8 driver). The table must report the model the compiler was given, - // not the model `cl.exe` would have been given (#649 E10). - bool msvcCrtModelEmitted = true; // Toolchain capability id: "libstdc++", "libc++", or an MSVC STL spelling. std::string_view stdlibId; Format format = Format::Elf; @@ -712,91 +737,69 @@ Mechanism resolve(const MechanismInput& in) { // ---------------------------------------------------------------- PE case Format::Pe: { if (!detail::is_libstdcxx(in.stdlibId)) { - // MSVC STL (cl.exe, or clang on the MSVC ABI). The CRT model is - // the mechanism here, and it is a whole-project switch: /MT is - // self-contained (no vcruntime DLL dependency), /MD is - // host-coupled. `msvcStaticCrt` is that switch, already derived - // by whoever emits the flag — so what this table reports and what - // cl was actually told cannot disagree. + // MSVC STL (cl.exe, or clang on the MSVC ABI). Every MSVC-ABI row + // now receives a CRT model — one helper (`msvc_abi_crt_word`) + // spells `/MT`/`/MD` for cl and `-fms-runtime-lib=static`/`=dll` + // for clang++ alike, reaching the TUs, the std/std.compat BMIs + // and the link command — so E10 is gone: there is no row left + // that the table must record as receiving none. // - // No unit flags: the model is a COMPILE flag on every TU, not - // something added to the link line. + // No unit flags: the model is a compile (and link) flag, not + // something added to this table's link-flag string. // - // CLANG ON THE MSVC ABI IS GIVEN NO MODEL, so the table records - // the one its driver chooses. The rows below were written for - // cl.exe, and for this row they recorded `host-coupled` beside an - // artifact that imports no vcruntime DLL at all (#649 E10). The - // artifact is left as it is; the record, and an explicit request - // the row does not deliver, now say what it is. - if (!in.msvcCrtModelEmitted) { + // `in.requested` already reflects the MSVC-ABI whole-project + // default (`msvc_abi_default_contract`, resolved before this + // table runs): an undeclared row without a redistributable + // directory arrives here as HostCoupled already, and an EXPLICIT + // `toolchain-coupled` a toolset cannot deliver is refused before + // compiling (`prepare/plan.cpp`), not silently degraded here — + // the same rule the pack mode contradiction follows. + if (in.msvcStaticCrt) { m.effective = Contract::SelfContained; if (in.requested != Contract::SelfContained && in.explicitRequest) { + // A static CRT leaves NO DLL to couple to, whichever + // coupled value was asked for — a genuine contradiction, + // not a missing mechanism. Only reachable from a + // per-role override or `linkage = "static"` beside an + // explicit `cxx_runtime`: a project-level `cxx_runtime = + // "self-contained"` would already have set this role's + // `requested` to SelfContained too. m.degraded = true; m.diagnostic = std::format( - "cxx_runtime = \"{}\" is not delivered for clang on the " - "MSVC ABI: mcpp passes this driver no CRT model, and clang " - "links the static CRT (libcmt) by default. Use msvc@system " - "for the dynamic CRT; using self-contained", - to_string(in.requested)); + "cxx_runtime = \"{}\" cannot apply to a project " + "compiled with the static CRT (/MT, or linkage = " + "\"static\"): there is no vcruntime140.dll/" + "msvcp140.dll dependency left to couple to; using " + "self-contained", to_string(in.requested)); } return m; } - m.effective = in.msvcStaticCrt ? Contract::SelfContained - : Contract::HostCoupled; - if (in.requested == Contract::SelfContained && !in.msvcStaticCrt) { + // Dynamic CRT (/MD). ToolchainCoupled is delivered exactly when + // requested — the row without a redistributable never reaches + // this table asking for it undeclared, and an explicit request it + // cannot meet is a planning-time refusal, not this table's to + // degrade. + if (in.requested == Contract::ToolchainCoupled) { + m.effective = Contract::ToolchainCoupled; + m.deployToolchainRuntime = true; + } else { + m.effective = Contract::HostCoupled; + } + if (in.requested == Contract::SelfContained && in.explicitRequest) { // Asked for, not delivered. Only reachable from a per-ROLE // override, because a project-level one would have set - // msvcStaticCrt — so name that, instead of the old "not - // implemented", which stopped being true and had already - // been contradicted by flags.cppm emitting /MT for - // `linkage = "static"`. - m.degraded = in.explicitRequest; - m.diagnostic = in.explicitRequest - ? "on the MSVC runtime the CRT model is a whole-project " - "property — one std module is built per project and cl " - "bakes _MSVC_MT/_MSVC_MD into it, so a single role " - "cannot differ. Move it to [build] cxx_runtime = " - "\"self-contained\" (or linkage = \"static\") to apply " - "it everywhere; using host-coupled here" - : ""; - } else if (in.requested == Contract::ToolchainCoupled) { - // THIS USED TO BE A FLAT REFUSAL, and the sentence it refused - // with was half true: - // - // "…has no meaning for the MSVC runtime (it ships with the - // OS/redistributable, not with the toolchain)" - // - // True of `ucrtbase.dll`, which IS an OS component since - // Win10. NOT true of `vcruntime140.dll` / `msvcp140.dll`, - // which are the toolset's own and sit inside every MSVC - // toolset ever shipped: - // - // VC\Redist\MSVC\\\Microsoft.VC.CRT\*.dll - // - // That is the same relationship gcc has to libstdc++.so, so it - // takes the same contract — and refusing it left a hole in the - // matrix that had a real cost: the default `/MD` artifact - // depends on DLLs a machine with only a managed toolset does - // not have, and there was no spelling that made them travel. - // - // `/MT` is the one case that stays a degradation, and it is a - // genuine contradiction rather than a missing mechanism: a - // static CRT leaves NO DLL to couple to. Say which one won. - if (in.msvcStaticCrt) { - m.effective = Contract::SelfContained; - m.degraded = true; - m.diagnostic = - "cxx_runtime = \"toolchain-coupled\" cannot apply to a " - "project compiled with the static CRT (/MT): there is " - "no vcruntime140.dll/msvcp140.dll dependency left to " - "couple to. Drop linkage = \"static\" (or the " - "project-wide self-contained contract) if the toolset's " - "CRT should travel beside the artifact instead; using " - "self-contained"; - } else { - m.effective = Contract::ToolchainCoupled; - m.deployToolchainRuntime = true; - } + // `msvcStaticCrt` — so name that: the CRT model is a + // whole-project property, one std module is built per + // project and cl bakes _MSVC_MT/_MSVC_MD into it, so a + // single role cannot differ. + m.degraded = true; + m.diagnostic = std::format( + "on the MSVC runtime the CRT model is a whole-project " + "property — one std module is built per project and cl " + "bakes _MSVC_MT/_MSVC_MD into it, so a single role " + "cannot differ. Move it to [build] cxx_runtime = " + "\"self-contained\" (or linkage = \"static\") to apply " + "it everywhere; using {} here", to_string(m.effective)); } return m; } diff --git a/src/build/flags.cppm b/src/build/flags.cppm index 9b21fdccf..2b9bc774e 100644 --- a/src/build/flags.cppm +++ b/src/build/flags.cppm @@ -95,6 +95,13 @@ struct CompileFlags { // The contract each role actually got (after any degradation). std::array contractByRole{}; + // Was the Distributable role's contract WRITTEN, or is + // `contractByRole[Distributable]` a default `role_contracts` picked + // (`msvc_abi_default_contract` on the MSVC ABI)? `mcpp pack` reads this + // to tell "the manifest asked for toolchain-coupled" from "nobody asked + // and the row happened to default to it" — only the first survives an + // explicit `--mode system` (#718). + bool programCxxRuntimeStated = false; // macOS + self-contained: link units need the initializer-ordering shim // object prepended to their inputs (issue #336). bool needsStreamInitShim = false; @@ -605,6 +612,18 @@ CompileFlags compute_flags(const BuildPlan& plan) { const bool isMsvcDialect = (d.id == "msvc"); + // THE CRT MODEL WORD, for the driver `plan.toolchain` actually is (#718). + // Computed once so the compile line below, the link line + // (`LinkShape::PeLld`) and the mechanism table's record + // (`mi.msvcStaticCrt`) all read the SAME two facts: whether the project + // wants the static CRT (`msvc_wants_static_crt`, unchanged since #422), + // and how that driver spells it (`msvc_abi_crt_word`, new for #718 — + // empty off the MSVC ABI, where it is inert). + const bool msvcAbiWantsStaticCrt = mcpp::toolchain::msvc_wants_static_crt( + plan.manifest.buildConfig.linkage, plan.manifest.buildConfig.cxxRuntime); + const std::string msvcAbiCrtWord = mcpp::toolchain::msvc_abi_crt_word( + plan.toolchain, msvcAbiWantsStaticCrt); + // PIC is a GNU concept and a property of the TARGET FORMAT: PE code is // position independent by design (base relocations), and clang rejects the // flag outright — `unsupported option '-fPIC' for target @@ -1041,15 +1060,18 @@ CompileFlags compute_flags(const BuildPlan& plan) { // model — /MD by default, /MT when either knob asks for the static CRT // (portable-by-default is impossible on MSVC-ABI; /MT at least removes // the vcruntime DLL dep). + // + // CLANG ON THE MSVC ABI TAKES THE SAME WORD, spelled for its own driver + // (#718): `msvcAbiCrtWord` is empty for every non-MSVC-ABI row, so this + // `else if` adds nothing there. `d.alwaysFlags` is cl-only (`/nologo + // /EHsc /utf-8`) and stays out of the clang branch — nothing about those + // three flags is MSVC-ABI-specific. std::string msvc_base; if (isMsvcDialect) { msvc_base = std::format(" {}", d.alwaysFlags); - // ONE derivation, shared with the std module build — see - // `msvc_wants_static_crt` in mcpp.toolchain.dialect and #422. - msvc_base += std::format(" {}", mcpp::toolchain::msvc_crt_flag( - d, mcpp::toolchain::msvc_wants_static_crt( - plan.manifest.buildConfig.linkage, - plan.manifest.buildConfig.cxxRuntime))); + msvc_base += std::format(" {}", msvcAbiCrtWord); + } else if (mcpp::toolchain::is_msvc_target(plan.toolchain)) { + msvc_base = std::format(" {}", msvcAbiCrtWord); } // User link flags: `[build] ldflags`, the `link_flag` and `link_lib` @@ -1133,11 +1155,15 @@ CompileFlags compute_flags(const BuildPlan& plan) { opt_flag, pic_flag, compile_toolchain_flags, b_flag); // MSVC compiles C with cl.exe too; /std: for C uses cN spellings — skip // the C standard flag there (cl defaults are fine for the C entry TUs). + // + // `msvc_base` rides the GNU branch too: on the LLVM row it holds the CRT + // word (`-fms-runtime-lib=*`) and nothing else, since `d.alwaysFlags` is + // only ever set for the msvc dialect. Empty everywhere else, as before. f.cc = isMsvcDialect ? std::format("{}{}{}{}", msvc_base, opt_flag, compile_toolchain_flags, b_flag) - : std::format("{}{}{}{}{}{}", d.stdPrefix, c_std, opt_flag, pic_flag, - compile_toolchain_flags, b_flag); + : std::format("{}{}{}{}{}{}{}", msvc_base, d.stdPrefix, c_std, opt_flag, + pic_flag, compile_toolchain_flags, b_flag); // GAS assembly (.S/.s via the C driver): the asm-safe subset — no -std // (C-only) and no -O (meaningless), but PIC stays (.S sources gate on @@ -1218,12 +1244,23 @@ CompileFlags compute_flags(const BuildPlan& plan) { .program = mcpp::build::image_loads_cxx_shared_library(plan, LinkUnit::Binary), .tests = mcpp::build::image_loads_cxx_shared_library(plan, LinkUnit::TestBinary), }; + // The MSVC-ABI whole-project default (#718, §7.3): a row without a + // redistributable directory defaults to host-coupled rather than + // toolchain-coupled, silently — nothing to stage, and no per-build + // warning for a property of the row. Off that ABI, nullopt leaves + // every other format's own per-role defaults untouched. + const std::optional msvcAbiDefault = + mcpp::toolchain::is_msvc_target(plan.toolchain) + ? std::optional(dist::msvc_abi_default_contract( + msvcAbiWantsStaticCrt, !plan.toolchain.msvcRedistDir.empty())) + : std::nullopt; const dist::RoleContracts contracts = dist::role_contracts( dist::ContractStatement{ .cxxRuntime = bc.cxxRuntime, .cxxRuntimeTests = bc.cxxRuntimeTests, .cxxRuntimeShared = bc.cxxRuntimeShared, .staticStdlib = bc.staticStdlib, + .msvcAbiDefault = msvcAbiDefault, }, format, cxxSharedLoad); @@ -1279,11 +1316,7 @@ CompileFlags compute_flags(const BuildPlan& plan) { // other one fails inside the ucrt headers (#422). The mechanism table // needs to know so it can say that out loud rather than silently // ignoring a role override. - mi.msvcStaticCrt = mcpp::toolchain::msvc_wants_static_crt( - bc.linkage, bc.cxxRuntime); - // The model is emitted only for the `msvc` dialect (above); clang on - // the MSVC ABI receives none and links the static CRT (#649 E10). - mi.msvcCrtModelEmitted = isMsvcDialect; + mi.msvcStaticCrt = msvcAbiWantsStaticCrt; mi.mingw = isMingwTc; mi.macosFloor = !macosDeploymentTarget.empty(); // READ from the one value prepare resolved. The SDK being located for @@ -1471,6 +1504,12 @@ CompileFlags compute_flags(const BuildPlan& plan) { const bool explicitBase = contracts.programStated; const bool explicitTests = contracts.testsStated; const bool explicitShared = contracts.sharedStated; + // Read by `mcpp pack` (#718): an explicit `--mode system` outranks a + // DEFAULTED toolchain-coupled contract (resolved to host-coupled + // instead of a mode contradiction), but not a stated one — the same + // "explicit outranks a default" rule the pack mode / cxx_runtime + // refusal already follows for an explicit request no toolset can meet. + f.programCxxRuntimeStated = explicitBase; // Report a role's degradation only if this build HAS that role. // @@ -1513,25 +1552,25 @@ CompileFlags compute_flags(const BuildPlan& plan) { "{} target: {}", dist::to_string(role), r.diagnostic)); } if (wantsToolchainRuntime) { - // `linkRuntimeDirs` is the toolset's own redistributable CRT - // directory and nothing else on this toolchain — `enrich_toolchain - // _from_cl` puts exactly `vc_redist_dir()` there. Guarded on the - // compiler anyway: the field means "the toolchain's private - // runtime" for every provider, and on gcc it holds libstdc++'s - // directory, which has no business being copied into a PE tree. - if (plan.toolchain.compiler == mcpp::toolchain::CompilerId::MSVC) { + // THE GATE IS THE ABI AND A REDISTRIBUTABLE DIRECTORY, NOT THE + // COMPILER (#718). `msvcRedistDir` is its own field, set for cl + // AND for clang++ on the MSVC ABI alike (`enrich_toolchain_from_cl` + // / `bind_msvc_sysroot`) — unlike `linkRuntimeDirs`, which on the + // LLVM row holds the LLVM payload's OWN runtime directories and + // must not be searched here: copying those into a Windows + // program's `bin/` would stage the wrong files. + if (mcpp::toolchain::is_msvc_target(plan.toolchain) + && !plan.toolchain.msvcRedistDir.empty()) { std::vector sources; std::error_code ec; - for (auto const& dir : plan.toolchain.linkRuntimeDirs) { - for (auto const& e : - std::filesystem::directory_iterator(dir, ec)) { - if (!e.is_regular_file(ec)) continue; - auto ext = e.path().extension().string(); - std::ranges::transform(ext, ext.begin(), - [](unsigned char c) { return std::tolower(c); }); - if (ext != ".dll") continue; - sources.push_back(e.path()); - } + for (auto const& e : std::filesystem::directory_iterator( + plan.toolchain.msvcRedistDir, ec)) { + if (!e.is_regular_file(ec)) continue; + auto ext = e.path().extension().string(); + std::ranges::transform(ext, ext.begin(), + [](unsigned char c) { return std::tolower(c); }); + if (ext != ".dll") continue; + sources.push_back(e.path()); } // Directory order is not a stable input: this list reaches // build.ninja, and a graph that differs between two runs of @@ -1545,18 +1584,21 @@ CompileFlags compute_flags(const BuildPlan& plan) { // the toolset's copy is a different program than the one // the manifest describes. auto clash = std::ranges::find_if(plan.runtimeDeployFiles, - [&](auto const& d) { return d.dest == dest; }); + [&](auto const& d) { return d.is_destination(dest, /*peTarget=*/true); }); if (clash != plan.runtimeDeployFiles.end()) { - if (clash->source.lexically_normal() - != src.lexically_normal()) + if (std::ranges::none_of(clash->sources, + [&](auto const& s) { + return s.lexically_normal() + == src.lexically_normal(); + })) f.diagnostics.push_back(std::format( "toolchain-coupled would stage '{}' beside the " "artifact, but this project already deploys " "'{}' there; keeping the project's file", - src.string(), clash->source.string())); + src.string(), clash->sources.front().string())); continue; } - f.toolchainRuntimeDeploy.push_back({src, dest}); + f.toolchainRuntimeDeploy.push_back({{src}, dest}); } } } @@ -1792,9 +1834,20 @@ CompileFlags compute_flags(const BuildPlan& plan) { // from these rather than from the machine; empty without a toolset. const auto msvcSysroot = mcpp::toolchain::render_tokens(lm.msvc_driver_tokens(ninjaEsc)); - f.ld = std::format("{} -fuse-ld=lld{}{}{}{}", full_static, msvcSysroot, - link_intent_ld, user_ldflags, link_extra); - f.ldC = f.ld; // no C++ runtime token on this line + // THE CRT WORD REACHES THIS LINE TOO (#649 E10, #718). The clang + // driver chooses `-defaultlib:` at LINK time independently of what + // the objects were compiled with — measured with `-fms-runtime- + // lib=dll` at compile time only: the objects carried `--dependent- + // lib=msvcrt`, and the link step still passed `-defaultlib:libcmt`. + // `msvcAbiCrtWord` is the same word the compile line above carries, + // so a link run without recompiling (an incremental relink) cannot + // ask the driver for a different CRT than the objects already have. + const std::string msvcAbiCrtLd = + msvcAbiCrtWord.empty() ? std::string{} : (" " + msvcAbiCrtWord); + f.ld = std::format("{}{} -fuse-ld=lld{}{}{}{}", full_static, + msvcAbiCrtLd, msvcSysroot, link_intent_ld, + user_ldflags, link_extra); + f.ldC = f.ld; // the CRT model applies to C TUs on this ABI too } else if (linkShape == LinkShape::AppleSdk) { // macOS. The C++ runtime itself is decided by the contract table above // (dist::Format::MachO) and rides unit_ldflags; what is left here is diff --git a/src/build/ninja_backend.cppm b/src/build/ninja_backend.cppm index c1abc0040..d826dd31e 100644 --- a/src/build/ninja_backend.cppm +++ b/src/build/ninja_backend.cppm @@ -1935,6 +1935,13 @@ std::string emit_ninja_string(const BuildPlan& plan) { for (auto const& d : plan.linkIntent.runtimeSearchDirs) dirs += " " + ninja_command_word(d.string()); append("rule place_dlls\n"); + // One destination, one writer (SPEC-007 R4.2/R4.3, #723): `place-dlls` + // decides for itself which DLLs beside the program are another + // writer's (see cmd_place_dlls), so the command line carries no list. + // A list here changed whenever the plan's deploy set did -- and that + // set reads runtime search directories a `prepare` action fills, so it + // differs between the first plan and the second, and every build after + // the first re-ran the placement (e2e 797). append(" command = $mcpp place-dlls --output $out --depfile $out.d $in" + dirs + "\n"); append(" depfile = $out.d\n"); append(" deps = gcc\n"); @@ -2975,10 +2982,20 @@ std::string emit_ninja_string(const BuildPlan& plan) { // previous `mcpp run` gets the skip-if-equivalent treatment instead of a // hard "cannot copy" failure. // Inert on RPATH platforms where the merged deploy list is empty. + // + // SPEC-007 R4.2 (#723): a destination with more than one source (two + // packages of this graph each generated the same file) becomes ONE edge + // with every source as an input, not one edge per source. `mcpp stage` + // is where they are checked against each other's bytes — planning cannot, + // because a generated source may not exist yet. A destination with + // exactly one source (every project before this feature, and most + // packages after it) emits the exact same line as always: the loop below + // reduces to the one-word case with no change in spelling. for (auto const& d : deployFiles) { - append(std::format("build {} : stage_file {}\n", - escape_ninja_path(d.dest), - escape_ninja_path(d.source))); + std::string ins; + for (auto const& s : d.sources) ins += " " + escape_ninja_path(s); + append(std::format("build {} : stage_file{}\n", + escape_ninja_path(d.dest), ins)); } if (!deployFiles.empty()) append("\n"); diff --git a/src/build/plan.cppm b/src/build/plan.cppm index a6cc965cc..fd174e139 100644 --- a/src/build/plan.cppm +++ b/src/build/plan.cppm @@ -437,10 +437,41 @@ struct BuildPlan { // byte-for-byte unchanged; only a Windows prebuilt-DLL package (or a test // that ships a .dll) populates it. dest is relative to outputDir. struct DeployFile { - std::filesystem::path source; // absolute source DLL + // Absolute source paths. Usually one; more than one means two or more + // packages of this graph each generate a file for this destination + // (SPEC-007 R4.2) — an `artifacts` dependency and its consumer asking + // one plugin for the same translation catalog is the case #723 was + // filed for. Planning no longer refuses this: at planning time a + // generated source may not exist yet, so its content cannot be + // compared. `mcpp stage` (mcpp.build.stage) is where the sources are + // actually checked, at build time, against each other's bytes. + std::vector sources; std::filesystem::path dest; // relative to outputDir, e.g. bin/libopenblas.dll + + // Whether `other` names this destination. On a PE target the + // comparison folds case: the file systems a Windows program runs from + // and its loader both do, so `Foo.DLL` and `foo.dll` are one file + // there, whatever the build host's file system does. + bool is_destination(const std::filesystem::path& other, bool peTarget) const { + if (!peTarget) return dest == other; + auto fold = [](std::string s) { + std::ranges::transform(s, s.begin(), + [](unsigned char c) { return static_cast(std::tolower(c)); }); + return s; + }; + return fold(dest.generic_string()) == fold(other.generic_string()); + } }; std::vector runtimeDeployFiles; + // A DLL a runtime search directory offers under a name the deploy list + // declares (SPEC-007 R4.3). The declared file is placed; the planning + // caller compares the two and warns on a difference, because the output + // of the post-link placement edge is not shown on a successful build. + struct ShadowedDll { + std::filesystem::path declared; // the deploy list's source + std::filesystem::path offered; // the search directory's file + }; + std::vector shadowedSearchDirDlls; // Aggregated host-runtime requirements from dependency packages' // [runtime] metadata. Capability/provider-driven — no platform special-casing // in mcpp: providers (e.g. compat.glx-runtime) declare these per platform. @@ -1430,39 +1461,58 @@ make_plan(const mcpp::manifest::Manifest& manifest, // `toDir` is a `runtime.deploy` destination, relative to the executable's // directory; empty and "." both mean that directory itself, which is where - // every `deploy_files` entry goes. The collision check keys on the full - // relative destination, so two files of one name in two directories do not - // collide, and two sources for one destination still do. + // every `deploy_files` entry goes. The check keys on the full relative + // destination, so two files of one name in two directories do not + // collide. + // + // Two sources for one destination are no longer refused HERE (SPEC-007 + // R4.2, #723): at planning time a generated source may not exist yet, so + // its content cannot be compared. Both stay as inputs of the one + // `stage_file` edge this destination becomes (ninja_backend.cppm), and + // `mcpp stage` (mcpp.build.stage) is where the invariant — one + // destination, one content — is actually checked, once the sources exist. + // A source already listed for this destination (the ordinary case: the + // same file reached through two graph edges) is not duplicated. + // On a PE target a destination is compared without case + // (DeployFile::is_destination). + const bool peTarget = targetTriple.empty() ? bool(mcpp::platform::is_windows) + : targetTriple.is_pe(); auto add_deploy = [&](const std::filesystem::path& source, - std::string_view toDir = {}) - -> std::optional { + std::string_view toDir = {}) { const auto normalized = source.lexically_normal(); auto destDir = std::filesystem::path("bin"); if (!toDir.empty() && toDir != ".") destDir /= std::filesystem::path(toDir); const auto dest = destDir / source.filename(); auto existing = std::ranges::find_if(plan.runtimeDeployFiles, - [&](auto const& value) { return value.dest == dest; }); + [&](auto const& value) { return value.is_destination(dest, peTarget); }); if (existing != plan.runtimeDeployFiles.end()) { - if (existing->source.lexically_normal() != normalized) { - return std::format( - "runtime deploy collision: '{}' and '{}' both target '{}'", - existing->source.string(), normalized.string(), dest.string()); - } - return std::nullopt; + if (std::ranges::find(existing->sources, normalized) + == existing->sources.end()) + existing->sources.push_back(normalized); + return; } - plan.runtimeDeployFiles.push_back({normalized, dest}); - return std::nullopt; + plan.runtimeDeployFiles.push_back({{normalized}, dest}); }; // Structured deploy files are explicit and platform-neutral. Legacy // library_dirs keeps its one-train DLL discovery behavior below. for (auto const& source : plan.linkIntent.deployFiles) { - if (auto collision = add_deploy(source)) - return std::unexpected(std::move(*collision)); + add_deploy(source); } for (auto const& entry : plan.linkIntent.deploy) { - if (auto collision = add_deploy(entry.from, entry.to)) - return std::unexpected(std::move(*collision)); + add_deploy(entry.from, entry.to); } + // A DLL found in a runtime search directory is a derived source: it + // yields to a destination the lists above declare (SPEC-007 R4.3, one + // destination, one writer), as the toolset's staged runtime does + // (flags.cppm). The post-link placement compares the two files and warns + // on a difference. Two search directories offering one name are two + // derived sources of one destination and are checked by `mcpp stage`. + const auto declaredCount = plan.runtimeDeployFiles.size(); + auto declared = [&](const std::filesystem::path& dest) -> const BuildPlan::DeployFile* { + for (auto const& d : std::span{plan.runtimeDeployFiles}.first(declaredCount)) + if (d.is_destination(dest, peTarget)) return &d; + return nullptr; + }; for (auto const& dir : plan.linkIntent.runtimeSearchDirs) { std::error_code dirEc; if (!std::filesystem::is_directory(dir, dirEc)) continue; @@ -1472,8 +1522,11 @@ make_plan(const mcpp::manifest::Manifest& manifest, std::ranges::transform(ext, ext.begin(), [](unsigned char c){ return std::tolower(c); }); if (ext != ".dll") continue; - if (auto collision = add_deploy(entry.path())) - return std::unexpected(std::move(*collision)); + if (auto const* d = declared(std::filesystem::path("bin") / entry.path().filename())) { + plan.shadowedSearchDirDlls.push_back({d->sources.front(), entry.path()}); + continue; + } + add_deploy(entry.path()); } } // The same private runtime directories embedded as executable RUNPATH are @@ -1483,6 +1536,15 @@ make_plan(const mcpp::manifest::Manifest& manifest, for (auto const& dir : tc.linkRuntimeDirs) { append_unique_path(plan.runtimeLibraryDirs, dir); } + // The MSVC toolset's own redistributable CRT (#718), on the `mcpp run`/ + // `mcpp test` search path exactly as the staged copy is beside the + // artifact (`flags.cppm`'s toolchain-coupled staging) — the two are the + // same directory for both the cl.exe row (already inside + // `linkRuntimeDirs` above, so this is a harmless duplicate there) and the + // LLVM row (where `linkRuntimeDirs` holds LLVM's own directories instead + // and would otherwise never mention it). + if (!tc.msvcRedistDir.empty()) + append_unique_path(plan.runtimeLibraryDirs, tc.msvcRedistDir); // The private glibc payload exists here for ONE reason: a dlopen()'d // library, whose own DT_NEEDED closure never consults the main // executable's RUNPATH, must still resolve the same libc the executable @@ -1844,8 +1906,24 @@ make_plan(const mcpp::manifest::Manifest& manifest, }; // 1. Compile units in topological order + // + // A device-kind source (`SourceKind::Device`) is not one of them. The + // engine has no compile rule for it — it is compiled, if at all, by the + // package's build program through an action (`mcpp::action`), never by a + // `cxx_object`/`c_object` edge — so turning it into a `CompileUnit` here + // gave every consumer of `plan.compileUnits` a unit nothing runs: a dead + // `cxx_object` edge in `build.ninja` with no consumer, and an entry in + // `compile_commands.json` and the S1 document naming a compiler that never + // ran on the file (design 2026-09-27 §4.1, mcpp#724). Fixed at the source, + // once, so ninja, the compile database and S1 agree without each needing + // its own filter for this kind. The file itself still reaches `watch` + // (build_database.cppm expands the sources glob directly, not through + // `plan.compileUnits`), and it still reaches the package's build program + // through `MCPP_DEVICE_SOURCES` (features.cpp), which is the one thing + // that does compile it. for (auto idx : topoOrder) { auto& u = graph.units[idx]; + if (u.kind == mcpp::SourceKind::Device) continue; CompileUnit cu; cu.source = u.path; cu.packageName = u.packageName; diff --git a/src/build/prepare.cppm b/src/build/prepare.cppm index a3a368338..8cbc3a356 100644 --- a/src/build/prepare.cppm +++ b/src/build/prepare.cppm @@ -25,7 +25,8 @@ // dependencies' build programs. // target_side.cpp P9, P10 -- the target side and each dependency's link form. // scan.cpp P11, P12 -- the module scan, validation, fingerprint. -// plan.cpp P13 -- the BuildContext, mcpp.lock, resolution.json. +// plan.cpp P13 -- the BuildContext. +// records.cpp P13 -- mcpp.lock and resolution.json. // config.cpp, options.cpp, toolchain_env.cpp, fetch.cpp // the helpers the phases share: manifest merges and // feature requests; invocation options; target rows, @@ -725,4 +726,27 @@ prepare_build(bool print_fingerprint, bool includeDevDeps = false, std::vector extraTargets = {}, BuildOverrides overrides = {}); +// The PlanNotes a failed call recorded before the phase that failed it. +// +// On success, `prepare_build` copies `PrepareState::planNotes` into +// `BuildContext::planNotes` (phase13_finish) — but on failure it returns only +// `r.error()`, a plain string, and the `PrepareState` that held the notes is a +// local of `prepare_build` and is gone the moment it returns. A note recorded +// by an earlier phase (`MCPP_BUILD_DATABASE_PROGRAM_FAILED`, say) was +// therefore lost on every failure of a later phase, not only the one that +// motivated this (design 2026-09-27 §4.2, mcpp#724 side finding A, fix item +// 2): under `emit build-database`'s `plan_only`, a member the caller could +// otherwise describe the ordinary way (R5.2) instead reported nothing but the +// later phase's own message. +// +// Same per-run-sink discipline as `mcpp::build::refusal` (refusal.cppm), and +// for the same reason: widening `prepare_build`'s return type would touch +// every caller of `.error()` to carry something only the failure path of one +// caller (`emit`) reads. Written immediately before prepare_build's own +// failing return, from the state that failure saw; read by the caller that +// turns that failure into diagnostics. `take` reads and clears, so neither a +// later failure of the SAME call nor a later, unrelated call inherits a stale +// set of notes. +export std::vector take_notes_on_failure(); + } // namespace mcpp::build diff --git a/src/build/prepare/config.cpp b/src/build/prepare/config.cpp index 948d39e61..4fe182e2b 100644 --- a/src/build/prepare/config.cpp +++ b/src/build/prepare/config.cpp @@ -335,6 +335,22 @@ void merge_conditional_config(mcpp::manifest::Manifest& m, m.buildConfig.abiExceptions = cc.abiExceptions; m.buildConfig.abiExceptionsDeclared = true; } + // `[target..build] dialect_cxxflags` (#717): recorded for every + // package, like the abi switches above, but APPENDED rather than + // replaced -- there is no "last matching section wins" here, because + // the key is additive by design (design 2026-09-27 §6: "entries are + // appended, as cxxflags are"). This is the SAME iteration this loop + // already performs in manifest order, so a package's matching rows + // land after its own unconditional `[build] dialect_cxxflags` + // (already in `m.buildConfig.dialectCxxflags` from the initial parse) + // in exactly the declared order. Only the root's resulting list is + // ever read downstream; a dependency's is inert on that dependency's + // own manifest and excluded from its fingerprint contribution + // (prepare_inputs.cppm). + if (!cc.dialectCxxflags.empty()) + m.buildConfig.dialectCxxflags.insert(m.buildConfig.dialectCxxflags.end(), + cc.dialectCxxflags.begin(), + cc.dialectCxxflags.end()); // `[target.] requires_abi` / `.feature-requires-abi` (A6): a // requirement on the TARGET axis, unioned in -- not overwritten -- // because more than one matching selector may ask for the same diff --git a/src/build/prepare/driver.cpp b/src/build/prepare/driver.cpp index b044c4f9c..40fadda9a 100644 --- a/src/build/prepare/driver.cpp +++ b/src/build/prepare/driver.cpp @@ -24,6 +24,22 @@ import mcpp.platform; namespace mcpp::build { +namespace { +// `thread_local` for the same reason `mcpp::build::refusal`'s sink is +// (refusal.cppm): `prepare_build` recurses for nested host sub-builds on the +// calling thread, and a failure of the INNER call must not leave notes behind +// for an outer call that goes on to succeed. Cleared at the top of every +// `prepare_build` call and on its success path, so only a call that is +// itself failing can leave something here for its caller to take. +thread_local std::vector g_notesOnFailure; +} // namespace + +std::vector take_notes_on_failure() { + auto notes = std::move(g_notesOnFailure); + g_notesOnFailure.clear(); + return notes; +} + std::expected prepare_build(bool print_fingerprint, bool includeDevDeps, @@ -32,18 +48,29 @@ prepare_build(bool print_fingerprint, PrepareState state(print_fingerprint, includeDevDeps, std::move(extraTargets), std::move(overrides)); pending_flag_words_notes().clear(); + g_notesOnFailure.clear(); + + // Every early return below carries `state.planNotes` as they stood at the + // failing phase, so a caller whose only handle on the failure is + // `.error()` (a plain string) can still read what an earlier phase + // recorded — see `take_notes_on_failure`'s declaration in prepare.cppm. + auto fail = [&](std::string message) -> std::unexpected { + g_notesOnFailure = state.planNotes; + return std::unexpected(std::move(message)); + }; - if (auto r = phase0_manifest_and_workspace(state); !r) return std::unexpected(r.error()); - if (auto r = phase1_toolchain_spec_and_axes(state); !r) return std::unexpected(r.error()); - if (auto r = phase2_define_toolchain_resolver(state); !r) return std::unexpected(r.error()); - if (auto r = phase3_xlings_before_graph(state); !r) return std::unexpected(r.error()); - if (auto r = phase4a_graph_load(state); !r) return std::unexpected(r.error()); - if (auto r = phase4b_graph_worklist(state); !r) return std::unexpected(r.error()); - if (auto r = phase5_toolchain_after_graph(state); !r) return std::unexpected(r.error()); - if (auto r = phase6_features_and_host_tools(state); !r) return std::unexpected(r.error()); - if (auto r = phase9_target_side(state); !r) return std::unexpected(r.error()); - if (auto r = phase11_scan(state); !r) return std::unexpected(r.error()); + if (auto r = phase0_manifest_and_workspace(state); !r) return fail(r.error()); + if (auto r = phase1_toolchain_spec_and_axes(state); !r) return fail(r.error()); + if (auto r = phase2_define_toolchain_resolver(state); !r) return fail(r.error()); + if (auto r = phase3_xlings_before_graph(state); !r) return fail(r.error()); + if (auto r = phase4a_graph_load(state); !r) return fail(r.error()); + if (auto r = phase4b_graph_worklist(state); !r) return fail(r.error()); + if (auto r = phase5_toolchain_after_graph(state); !r) return fail(r.error()); + if (auto r = phase6_features_and_host_tools(state); !r) return fail(r.error()); + if (auto r = phase9_target_side(state); !r) return fail(r.error()); + if (auto r = phase11_scan(state); !r) return fail(r.error()); + g_notesOnFailure.clear(); return phase13_finish(state); } diff --git a/src/build/prepare/features.cpp b/src/build/prepare/features.cpp index f4905860a..cec91d432 100644 --- a/src/build/prepare/features.cpp +++ b/src/build/prepare/features.cpp @@ -61,7 +61,28 @@ import mcpp.wire; // Severity, for PlanNote (#699 item 2, E3) namespace mcpp::build { -std::expected phase6_features_and_host_tools(PrepareState& state) { +// STEP FUNCTIONS (mcpp#722 / T6), one per section phase6's own banners +// already named. Statements moved verbatim; `aggregatedRequest` (used by +// two of these sections) is promoted from a local lambda to a file-scope +// function of PrepareState&, the same treatment #719 gave every closure +// that a later phase needed. + +static std::pair, bool> +aggregatedRequest(PrepareState& state, std::size_t depPkgIndex) { + std::vector feats; + bool anyEdge = false, anyDefault = false; + for (auto const& edge : state.dependencyEdges) { + if (edge.dependencyPackageIndex != depPkgIndex) continue; + anyEdge = true; + if (edge.defaultFeatures) anyDefault = true; + for (auto const& f : edge.requestedFeatures) + if (std::find(feats.begin(), feats.end(), f) == feats.end()) + feats.push_back(f); + } + return { std::move(feats), anyEdge ? anyDefault : true }; +} + +static void step6_check_version_floors_closure(PrepareState& state) { // ─── Feature activation (Cargo-style, additive) ──────────────────── // activated(pkg) = pkg.[features].default ∪ features requested for it // (root: --features; deps: the root dep spec's `features = [...]`). @@ -191,7 +212,9 @@ std::expected phase6_features_and_host_tools(PrepareState& st } return std::nullopt; }; - { +} + +static std::expected step6_activate_features(PrepareState& state) { auto sanitize = [](std::string f) { for (auto& c : f) c = std::isalnum(static_cast(c)) @@ -525,23 +548,9 @@ std::expected phase6_features_and_host_tools(PrepareState& st // activation AGREE with resolution (mergeActiveFeatureDeps, which reads // the true per-edge spec): a transitive dep's requested features and its // consumer's `default-features = false` are no longer silently dropped. - auto aggregatedRequest = [&](std::size_t depPkgIndex) - -> std::pair, bool> { - std::vector feats; - bool anyEdge = false, anyDefault = false; - for (auto const& edge : state.dependencyEdges) { - if (edge.dependencyPackageIndex != depPkgIndex) continue; - anyEdge = true; - if (edge.defaultFeatures) anyDefault = true; - for (auto const& f : edge.requestedFeatures) - if (std::find(feats.begin(), feats.end(), f) == feats.end()) - feats.push_back(f); - } - return { std::move(feats), anyEdge ? anyDefault : true }; - }; for (std::size_t i = 1; i < state.packages.size(); ++i) { auto& pname = state.packages[i].manifest.package.name; - auto [req, depDefaultFeatures] = aggregatedRequest(i); + auto [req, depDefaultFeatures] = aggregatedRequest(state, i); if (!req.empty() && !state.packages[i].manifest.featuresMap.empty()) { for (auto& f : req) { if (state.packages[i].manifest.featuresMap.contains(f)) continue; @@ -563,6 +572,10 @@ std::expected phase6_features_and_host_tools(PrepareState& st feature_closure(state.packages[i].manifest, req, depDefaultFeatures); } + return {}; +} + +static std::expected step6_device_extensions_and_rules(PrepareState& state) { // ─── Device extensions a rule dependency declared ────────────────── // // A rule package states which device extensions it compiles, on the @@ -818,7 +831,10 @@ std::expected phase6_features_and_host_tools(PrepareState& st bc.ruleModules = std::move(applies); } state.activeFeaturesByPackage.resize(state.packages.size()); + return {}; +} +static std::expected step6_xlings_workspace_from_graph(PrepareState& state) { // ── The GRAPH's `[xlings.workspace]`, provisioned BEFORE build.mcpp ── // // Same ordering rule as the host-tool block directly below, and for the @@ -870,7 +886,11 @@ std::expected phase6_features_and_host_tools(PrepareState& st } } } + return {}; +} +static std::expected>, std::string> +step6_host_module_registration(PrepareState& state) { // ── #355: HOST tool provisioning ──────────────────────────────────── // // Runs AFTER feature activation (a tool target's gate is a feature) and @@ -886,7 +906,6 @@ std::expected phase6_features_and_host_tools(PrepareState& st // main build: the sub-build may use the tool package's own toolchain, // its own profile, and its own resolution — none of it has to agree // with the consumer. - { // Aggregate off the authoritative edge graph, exactly like feature // activation — a transitive consumer's request must not be // silently dropped (#242/#243). @@ -992,14 +1011,17 @@ std::expected phase6_features_and_host_tools(PrepareState& st auto rel = mcpp::manifest::resolve_lib_root_path( depPkg.manifest, depPkg.root); auto iface = depPkg.root / rel; - push(iface, prov::host_module_name(iface, pkg.name)); + auto rootName = prov::host_module_name(iface, pkg.name); // A missing lib root is reported as such by build_host_module, // and that has to stay the diagnostic. Enumerating the listed // units first would let one of them collide with the missing // root's fallback name and report a collision between a file // and a file that does not exist. std::error_code ec; - if (!std::filesystem::exists(iface, ec)) return out; + if (!std::filesystem::exists(iface, ec)) { + push(iface, std::move(rootName)); + return out; + } std::set matched, dropped; for (auto const& g : depPkg.manifest.buildConfig.sources) { @@ -1034,7 +1056,21 @@ std::expected phase6_features_and_host_tools(PrepareState& st std::string name; std::vector imports; }; + // The lib root is the first node of the same sort (mcpp#720). + // Placing it ahead of the sort assumed that it imports no + // other unit of its package; a root that does was compiled + // before the unit it imports and failed with "module not + // found". As the first node it is still emitted first whenever + // it imports nothing of its own package, so the order of every + // package that built before is unchanged. std::vector pending; + { + std::ifstream is(root); + std::stringstream buf; + if (is) buf << is.rdbuf(); + pending.push_back({root, std::move(rootName), + prov::declared_imports(buf.str())}); + } for (auto const& f : matched) { // std::set: sorted if (dropped.contains(f)) continue; if (std::filesystem::equivalent(f, root, ec)) continue; @@ -1049,9 +1085,9 @@ std::expected phase6_features_and_host_tools(PrepareState& st } // Only names this package itself declares constrain anything. - // `import std;` and the lib root are already ahead of every - // entry here, and a name from another package is ordered by the - // cross-package DFS below rather than by this sort. + // `import std;` is ahead of every entry here, and a name from + // another package is ordered by the cross-package DFS below + // rather than by this sort. std::map byName; for (std::size_t i = 0; i < pending.size(); ++i) byName.emplace(pending[i].name, i); @@ -1250,426 +1286,539 @@ std::expected phase6_features_and_host_tools(PrepareState& st mcpp::build::tool_store::kMaxDepth, state.overrides.tool_chain)); } - for (auto const& [depIdx, wanted] : toolRequests) { - auto& depPkg = state.packages[depIdx]; - const auto& depName = depPkg.manifest.package.name; - std::string depShort = depName; - if (auto dot = depName.rfind('.'); - dot != std::string::npos && dot + 1 < depName.size()) - depShort = depName.substr(dot + 1); + return toolRequests; +} - for (auto const& toolName : wanted) { - // The target must exist and be a binary. Naming the - // alternatives matters: the consumer wrote a string, and a - // typo is the likeliest cause. - // - // #622 A3: deliberately still `Binary`, not `is_program()`. - // A host tool is exec'd directly ON THE BUILD MACHINE - // during THIS build, so it is "literally an executable - // link" — the question this site was already asking — and - // an `app` whose row form happened to be a library (never - // the host row in practice, but the check would be a - // silent trap if the host itself were ever Android) could - // not stand in for it. A build-time tool is declared - // `kind = "bin"`; that is what the word means here. - const mcpp::manifest::Target* tgt = nullptr; - std::string binList; - for (auto const& t : depPkg.manifest.targets) { - if (t.kind != mcpp::manifest::Target::Binary) continue; - if (!binList.empty()) binList += ", "; - binList += t.name; - if (t.name == toolName) tgt = &t; - } - if (!tgt) { - // A package may declare a bin target on some platforms - // only. When the request came from a LIBRARY rather - // than from the user, the user cannot edit it away, so - // point at the knob that library needs (#359 D3a). - return std::unexpected(std::format( - "dependency '{}' has no `kind = \"bin\"` target named " - "'{}' (requested via tools = [...]).\n" - " available bin targets: [{}]\n" - " If the requesting package is a library, it can " - "scope the request per platform with\n" - " [target.'cfg(...)'.feature-deps.].", - depName, toolName, - binList.empty() ? std::string("none") : binList)); - } +// A phase-local struct passed by reference to the steps of ONE requested +// host tool, the same way WorklistItemCtx (graph.cpp) is passed to the steps +// of one worklist item and TargetSideGather (target_side.cpp) to the steps +// of one target. Each field is a local the original single-function loop +// body declared once and read again in a later part of the same tool's +// provisioning. +struct HostToolCtx { + std::size_t depIdx = 0; + std::string toolName; + std::string depName; + std::string depShort; + const mcpp::manifest::Target* tgt = nullptr; + prov::Provision want{}; + std::string toolSource; + std::vector closure; + std::string toolTcSpec; + mcpp::build::tool_store::Key key; + std::filesystem::path cacheRoot; + std::filesystem::path entry; + std::filesystem::path binOut; + BuildOverrides sub; + std::filesystem::path goal; + std::filesystem::path subOutputDir; +}; - // #359: every consumer that can SEE this tool gets it, not - // just the one whose edge asked for it. The bare spelling - // is emitted only where the namespace ladder binds the tail - // to this package — otherwise two libraries re-exporting a - // same-tailed tool would decide the winner by append order. - // The spellings are `publishedNamesFor`'s, so a tool is - // addressed by exactly the names its directory is. - const prov::Provision want{ prov::Kind::Tool, depIdx, toolName }; - auto record = [&](const std::filesystem::path& p) { - for (std::size_t c = 0; c < state.provisionGraph.visible.size(); ++c) { - if (!state.provisionGraph.visible[c].contains(want)) continue; - auto& v = state.toolEnvByConsumer[c]; - std::vector vars; - for (auto const& n : state.publishedNamesFor(depIdx, state.bareBindingsFor(c))) { - auto var = mcpp::build::tool_store::env_var_name(n, toolName); - if (std::ranges::find(vars, var) != vars.end()) continue; - vars.push_back(var); - v.emplace_back(std::move(var), p.string()); - } - } - }; +// The body of the `record` closure the single-function version of this step +// captured per requested tool. It is called from four sites below (an +// override, a store cache hit, a deferred plan-only tool, and a freshly +// built one), so it is a named helper taking the context rather than a +// per-tool closure. +static void step6_record_tool_provision(PrepareState& state, HostToolCtx& ctx, + const std::filesystem::path& p) { + for (std::size_t c = 0; c < state.provisionGraph.visible.size(); ++c) { + if (!state.provisionGraph.visible[c].contains(ctx.want)) continue; + auto& v = state.toolEnvByConsumer[c]; + std::vector vars; + for (auto const& n : state.publishedNamesFor(ctx.depIdx, state.bareBindingsFor(c))) { + auto var = mcpp::build::tool_store::env_var_name(n, ctx.toolName); + if (std::ranges::find(vars, var) != vars.end()) continue; + vars.push_back(var); + v.emplace_back(std::move(var), p.string()); + } + } +} - // Escape hatch first: it is the cheapest resolution and the - // one a user reaches for precisely when building is not an - // option. Deliberately not part of the store key — see - // tool_store.cppm. - if (auto ovr = mcpp::build::tool_store::find_override( - *state.m, depName, depShort, toolName)) { - if (!std::filesystem::exists(*ovr)) { - return std::unexpected(std::format( - "tool override for '{}:{}' points at '{}', which " - "does not exist", depName, toolName, ovr->string())); - } - mcpp::ui::info("Tool", std::format( - "{}:{} → {} (override)", depName, toolName, ovr->string())); - record(*ovr); - continue; - } +static std::expected +step6_resolve_tool_target(PrepareState& state, HostToolCtx& ctx) { + auto& depPkg = state.packages[ctx.depIdx]; + // The target must exist and be a binary. Naming the + // alternatives matters: the consumer wrote a string, and a + // typo is the likeliest cause. + // + // #622 A3: deliberately still `Binary`, not `is_program()`. + // A host tool is exec'd directly ON THE BUILD MACHINE + // during THIS build, so it is "literally an executable + // link" — the question this site was already asking — and + // an `app` whose row form happened to be a library (never + // the host row in practice, but the check would be a + // silent trap if the host itself were ever Android) could + // not stand in for it. A build-time tool is declared + // `kind = "bin"`; that is what the word means here. + std::string binList; + for (auto const& t : depPkg.manifest.targets) { + if (t.kind != mcpp::manifest::Target::Binary) continue; + if (!binList.empty()) binList += ", "; + binList += t.name; + if (t.name == ctx.toolName) ctx.tgt = &t; + } + if (!ctx.tgt) { + // A package may declare a bin target on some platforms + // only. When the request came from a LIBRARY rather + // than from the user, the user cannot edit it away, so + // point at the knob that library needs (#359 D3a). + return std::unexpected(std::format( + "dependency '{}' has no `kind = \"bin\"` target named " + "'{}' (requested via tools = [...]).\n" + " available bin targets: [{}]\n" + " If the requesting package is a library, it can " + "scope the request per platform with\n" + " [target.'cfg(...)'.feature-deps.].", + ctx.depName, ctx.toolName, + binList.empty() ? std::string("none") : binList)); + } + return {}; +} - // A TOOL WHOSE OWN BUILD REQUESTS IT AGAIN IS REFUSED AT - // THE FIRST REPETITION (#649 E6). The depth bound below - // caught it only after four nested sub-builds, with the - // same prefix repeated four times and no word about which - // edge asked. The edge is the one whose request reached - // this package in THIS graph. - const std::string toolSource = std::format( - "{}|{}", depPkg.root.lexically_normal().generic_string(), toolName); - if (std::ranges::find(state.overrides.tool_chain_sources, toolSource) - != state.overrides.tool_chain_sources.end()) { - std::string askedBy; - for (auto const& edge : state.dependencyEdges) { - if (edge.dependencyPackageIndex != depIdx) continue; - if (std::ranges::find(edge.requestedTools, toolName) - == edge.requestedTools.end()) continue; - if (edge.consumerPackageIndex < state.packages.size()) { - askedBy = mcpp::build::qualified_package_name( - state.packages[edge.consumerPackageIndex].manifest); - break; - } - } - return std::unexpected(std::format( - "the host tool '{}:{}' is requested by its own build: " - "{} -> {}:{}.\n" - " The request comes from '{}', which the tool's " - "sub-build resolves with the feature or dependency that " - "asks for the tool.\n" - " fix: the tool's own graph must not activate " - "that request (a feature it does not enable, or a " - "`[target..feature-deps]` row it does not match).", - depName, toolName, - state.overrides.tool_chain.empty() ? "root" : state.overrides.tool_chain, - depName, toolName, - askedBy.empty() ? std::string("a package of its graph") : askedBy)); - } +// Returns true when the tool is already resolved (an escape-hatch override +// found and recorded), in which case the caller's per-tool work is done. +static std::expected +step6_check_tool_override(PrepareState& state, HostToolCtx& ctx) { + // Escape hatch first: it is the cheapest resolution and the + // one a user reaches for precisely when building is not an + // option. Deliberately not part of the store key — see + // tool_store.cppm. + if (auto ovr = mcpp::build::tool_store::find_override( + *state.m, ctx.depName, ctx.depShort, ctx.toolName)) { + if (!std::filesystem::exists(*ovr)) { + return std::unexpected(std::format( + "tool override for '{}:{}' points at '{}', which " + "does not exist", ctx.depName, ctx.toolName, ovr->string())); + } + mcpp::ui::info("Tool", std::format( + "{}:{} → {} (override)", ctx.depName, ctx.toolName, ovr->string())); + step6_record_tool_provision(state, ctx, *ovr); + return true; + } + return false; +} - // Build it. The feature set is the tool package's own - // defaults PLUS the target's required_features — in a tool - // sub-build the target is what was ASKED FOR, so its - // requirements are inputs rather than a gate. (Same field, - // opposite resolution direction; docs/05 says so.) - std::vector feats = tgt->requiredFeatures; - auto closure = feature_closure(depPkg.manifest, feats, true); - - // WHICH COMPILER BUILDS THE TOOL IS DECIDED HERE, ONCE - // (#710). The key used to record this build's host - // toolchain while the sub-build chose its own -- the tool - // package's `[toolchain]`, else the global default -- so an - // entry could name gcc 15.1 over a binary gcc 16.1 had - // produced, and a member tool built for a consumer used a - // different compiler than `mcpp build -p `. The - // choice is `--toolchain` when given, else the tool - // package's own (its workspace's, for a member), else the - // compiler this build compiles its build programs with. It - // is handed to the sub-build as an override and recorded in - // the key, so the two cannot disagree. - std::string toolTcSpec; - if (const char* e = std::getenv("MCPP_TOOLCHAIN"); e && *e) - toolTcSpec = e; - else if (auto own = host_tool_declared_toolchain( - depPkg.manifest, depPkg.root, kCurrentPlatform)) - toolTcSpec = *own; - std::string compilerIdentity; - if (toolTcSpec.empty()) { - auto hostTc = state.host_tc_for_build_program(); - if (!hostTc) return std::unexpected(hostTc.error()); - toolTcSpec = state.host_spec_for_build_program(); - compilerIdentity = std::format("{}|{}|{}", - hostTc->second.label(), hostTc->second.version, - hostTc->first.string()); - } else { - compilerIdentity = "spec|" + toolTcSpec; - } +static std::expected +step6_check_tool_self_request(PrepareState& state, HostToolCtx& ctx) { + auto& depPkg = state.packages[ctx.depIdx]; + // A TOOL WHOSE OWN BUILD REQUESTS IT AGAIN IS REFUSED AT + // THE FIRST REPETITION (#649 E6). The depth bound below + // caught it only after four nested sub-builds, with the + // same prefix repeated four times and no word about which + // edge asked. The edge is the one whose request reached + // this package in THIS graph. + ctx.toolSource = std::format( + "{}|{}", depPkg.root.lexically_normal().generic_string(), ctx.toolName); + if (std::ranges::find(state.overrides.tool_chain_sources, ctx.toolSource) + != state.overrides.tool_chain_sources.end()) { + std::string askedBy; + for (auto const& edge : state.dependencyEdges) { + if (edge.dependencyPackageIndex != ctx.depIdx) continue; + if (std::ranges::find(edge.requestedTools, ctx.toolName) + == edge.requestedTools.end()) continue; + if (edge.consumerPackageIndex < state.packages.size()) { + askedBy = mcpp::build::qualified_package_name( + state.packages[edge.consumerPackageIndex].manifest); + break; + } + } + return std::unexpected(std::format( + "the host tool '{}:{}' is requested by its own build: " + "{} -> {}:{}.\n" + " The request comes from '{}', which the tool's " + "sub-build resolves with the feature or dependency that " + "asks for the tool.\n" + " fix: the tool's own graph must not activate " + "that request (a feature it does not enable, or a " + "`[target..feature-deps]` row it does not match).", + ctx.depName, ctx.toolName, + state.overrides.tool_chain.empty() ? "root" : state.overrides.tool_chain, + ctx.depName, ctx.toolName, + askedBy.empty() ? std::string("a package of its graph") : askedBy)); + } + return {}; +} - mcpp::build::tool_store::Key key; - key.indexName = depIdx >= 1 && depIdx - 1 < state.dep_cache_identities.size() - ? state.dep_cache_identities[depIdx - 1].indexName - : std::string(mcpp::pm::kDefaultNamespace); - key.packageName = depName; - // THE VERSION IDENTIFIES THE SOURCES ONLY FOR AN INDEX - // PACKAGE. A `git` package is keyed by its commit and a - // `path` package by a stamp of its tree, because both - // change under an unchanged version and the store then - // serves a binary built from sources that no longer exist - // (#630, item 6; measured 2026-09-08 with examples/12). - // The same rule applies to every upstream below. - auto source_keyed_version = [&](std::size_t pkgIdx) { - const auto& man = state.packages[pkgIdx].manifest.package; - std::string v = man.version; - if (pkgIdx >= 1 && pkgIdx - 1 < state.dep_cache_identities.size()) { - const auto& id = state.dep_cache_identities[pkgIdx - 1]; - if (id.sourceKind == "git" && !id.sourceRef.empty()) - v += "+git." + id.sourceRef; - else if (id.sourceKind == "path") - v += "+path." + mcpp::build::tool_store::tree_stamp( - id.sourceRef.empty() ? state.packages[pkgIdx].root - : std::filesystem::path(id.sourceRef)); - } - return v; - }; - key.version = source_keyed_version(depIdx); - key.targetName = toolName; - key.hostTriple = mcpp::toolchain::triple::host_triple().str(); - key.compilerIdentity = compilerIdentity; - key.profile = "release"; - key.features = closure; - std::ranges::sort(key.features); - // The tool package's TRANSITIVE dependency closure, not just - // its direct edges. Direct-only would be enough for index - // packages (a frozen version cannot change its own deps), - // but a path dependency can: bump something two levels down - // and the tool's direct list is unchanged, so a stale binary - // stays in the store — a silently wrong artifact. - for (auto up : dg::transitive_dependencies(state.dependencyEdges, depIdx)) - key.upstreamKeys.push_back(std::format("{}@{}", - state.packages[up].manifest.package.name, - source_keyed_version(up))); - std::ranges::sort(key.upstreamKeys); - - const auto cacheRoot = mcpp::home::cache_root(); - const auto entry = mcpp::build::tool_store::entry_dir(cacheRoot, key); - const auto exeSuffix = std::string(mcpp::platform::exe_suffix); - const auto binOut = mcpp::build::tool_store::bin_path( - entry, toolName, exeSuffix); - - if (mcpp::build::tool_store::entry_valid(entry, key, toolName, - exeSuffix)) { - record(binOut); - continue; - } +// Returns true when the tool is already resolved (a valid store entry, or a +// plan-only deferral), in which case the caller's per-tool work is done. +static std::expected +step6_resolve_tool_key(PrepareState& state, HostToolCtx& ctx) { + auto& depPkg = state.packages[ctx.depIdx]; + // Build it. The feature set is the tool package's own + // defaults PLUS the target's required_features — in a tool + // sub-build the target is what was ASKED FOR, so its + // requirements are inputs rather than a gate. (Same field, + // opposite resolution direction; docs/05 says so.) + std::vector feats = ctx.tgt->requiredFeatures; + ctx.closure = feature_closure(depPkg.manifest, feats, true); - // PLANNING BUILDS NO TOOL (SPEC-005 R2.5, v1.4; #707). - // `emit build-database` describes a build; it does not - // perform one (R2.2), and a tool sub-build is a whole - // compile of another package, with its own prepare - // actions -- measured on a fresh store, a single `emit` - // compiled the tool and ran the tool package's `prepare` - // action. A tool already in the store is used as above. One - // that is not is deferred: the build program receives the - // path the tool will be published at (`binOut`, fixed - // before anything is built), which is the answer it gets - // after a successful build, and a note names the tool. A - // build program that must RUN the tool while configuring - // meets the same missing file it meets when the tool fails - // to build (SPEC-007 R5.3), so no new contract follows. - if (state.overrides.plan_only) { - state.planNotes.push_back({"MCPP_BUILD_DATABASE_HOST_TOOL_DEFERRED", - std::format("host tool '{}' of package '{}' is not in " - "the tool store and is not built while " - "planning; the plan names the path it will " - "be published at: {}", - toolName, depName, binOut.string()), - mcpp::wire::Severity::Note}); - record(binOut); - continue; - } + // WHICH COMPILER BUILDS THE TOOL IS DECIDED HERE, ONCE + // (#710). The key used to record this build's host + // toolchain while the sub-build chose its own -- the tool + // package's `[toolchain]`, else the global default -- so an + // entry could name gcc 15.1 over a binary gcc 16.1 had + // produced, and a member tool built for a consumer used a + // different compiler than `mcpp build -p `. The + // choice is `--toolchain` when given, else the tool + // package's own (its workspace's, for a member), else the + // compiler this build compiles its build programs with. It + // is handed to the sub-build as an override and recorded in + // the key, so the two cannot disagree. + if (const char* e = std::getenv("MCPP_TOOLCHAIN"); e && *e) + ctx.toolTcSpec = e; + else if (auto own = host_tool_declared_toolchain( + depPkg.manifest, depPkg.root, kCurrentPlatform)) + ctx.toolTcSpec = *own; + std::string compilerIdentity; + if (ctx.toolTcSpec.empty()) { + auto hostTc = state.host_tc_for_build_program(); + if (!hostTc) return std::unexpected(hostTc.error()); + ctx.toolTcSpec = state.host_spec_for_build_program(); + compilerIdentity = std::format("{}|{}|{}", + hostTc->second.label(), hostTc->second.version, + hostTc->first.string()); + } else { + compilerIdentity = "spec|" + ctx.toolTcSpec; + } - mcpp::ui::status("Building", std::format( - "host tool {}:{} from {} v{} (once per package source and " - "host toolchain)", depName, toolName, depName, - depPkg.manifest.package.version)); + ctx.key.indexName = ctx.depIdx >= 1 && ctx.depIdx - 1 < state.dep_cache_identities.size() + ? state.dep_cache_identities[ctx.depIdx - 1].indexName + : std::string(mcpp::pm::kDefaultNamespace); + ctx.key.packageName = ctx.depName; + // THE VERSION IDENTIFIES THE SOURCES ONLY FOR AN INDEX + // PACKAGE. A `git` package is keyed by its commit and a + // `path` package by a stamp of its tree, because both + // change under an unchanged version and the store then + // serves a binary built from sources that no longer exist + // (#630, item 6; measured 2026-09-08 with examples/12). + // The same rule applies to every upstream below. + auto source_keyed_version = [&](std::size_t pkgIdx) { + const auto& man = state.packages[pkgIdx].manifest.package; + std::string v = man.version; + if (pkgIdx >= 1 && pkgIdx - 1 < state.dep_cache_identities.size()) { + const auto& id = state.dep_cache_identities[pkgIdx - 1]; + if (id.sourceKind == "git" && !id.sourceRef.empty()) + v += "+git." + id.sourceRef; + else if (id.sourceKind == "path") + v += "+path." + mcpp::build::tool_store::tree_stamp( + id.sourceRef.empty() ? state.packages[pkgIdx].root + : std::filesystem::path(id.sourceRef)); + } + return v; + }; + ctx.key.version = source_keyed_version(ctx.depIdx); + ctx.key.targetName = ctx.toolName; + ctx.key.hostTriple = mcpp::toolchain::triple::host_triple().str(); + ctx.key.compilerIdentity = compilerIdentity; + ctx.key.profile = "release"; + ctx.key.features = ctx.closure; + std::ranges::sort(ctx.key.features); + // The tool package's TRANSITIVE dependency closure, not just + // its direct edges. Direct-only would be enough for index + // packages (a frozen version cannot change its own deps), + // but a path dependency can: bump something two levels down + // and the tool's direct list is unchanged, so a stale binary + // stays in the store — a silently wrong artifact. + for (auto up : dg::transitive_dependencies(state.dependencyEdges, ctx.depIdx)) + ctx.key.upstreamKeys.push_back(std::format("{}@{}", + state.packages[up].manifest.package.name, + source_keyed_version(up))); + std::ranges::sort(ctx.key.upstreamKeys); - BuildOverrides sub; - sub.project_root = depPkg.root; - // Never the package root: it is shared across projects and - // may be read-only. This is the reason work_dir exists. - // - // Scratch is keyed on the CONSUMING project, not shared: - // the store is GLOBAL, so two projects can want the same - // tool at once. A single `/build` would have them - // writing one ninja tree concurrently, and whichever - // finished first would `remove_all` it out from under the - // other. The published binary is what gets shared; the - // scratch is not. - // - // Hashed rather than random so a re-run reuses its own - // scratch (ninja stays incremental if the publish step - // never got to delete it). - // - // Beside the entries rather than inside one: every - // directory name of the entry is repeated in each object - // path the sub-build writes, and on Windows those paths - // crossed the 260-character limit (mcpp#641, item 3). - sub.work_dir = mcpp::build::tool_store::scratch_dir( - cacheRoot, entry, state.workRoot); - sub.target_triple = ""; // HOST — the whole point - sub.toolchain = toolTcSpec; - sub.profile = "release"; - sub.cache_mode = state.overrides.cache_mode; - sub.tool_depth = state.overrides.tool_depth + 1; - sub.tool_chain_sources = state.overrides.tool_chain_sources; - sub.tool_chain_sources.push_back(toolSource); - // The PRISTINE manifest the resolver produced for this - // package — `packages[depIdx].manifest` is a copy that - // feature activation has already mutated, and re-activating - // on top of it would fold the same feature sources in - // twice. A `compat` (Form B) package has no mcpp.toml on - // disk at all, so without this the sub-build could not read - // a manifest for it in the first place. - // - // UNMERGED, because the sub-build targets the HOST: the - // resolver merged this manifest's conditional sections for - // the consumer's target, and the sub-build merges them for - // its own (#690, F12). - if (depIdx >= 1 && depIdx - 1 < state.dep_manifests.size() - && state.dep_manifests[depIdx - 1]) { - auto const& dep = *state.dep_manifests[depIdx - 1]; - sub.preloaded_manifest = dep.beforeConditionalMerge - ? dep.beforeConditionalMerge - : std::make_shared(dep); - } - sub.inherited_runtime_selection = std::make_shared< - const mcpp::xlings::runtime::RuntimeSelection>( - state.runtimeSelection); - sub.inherited_runtime_binding = std::make_shared< - const mcpp::platform::runtime::RuntimeBinding>( - state.runtimeBindingSnapshot); - sub.tool_chain = state.overrides.tool_chain.empty() - ? std::format("root → {}:{}", depName, toolName) - : std::format("{} → {}:{}", state.overrides.tool_chain, depName, - toolName); - for (auto const& f : closure) { - if (!sub.features.empty()) sub.features += ","; - sub.features += f; - } + ctx.cacheRoot = mcpp::home::cache_root(); + ctx.entry = mcpp::build::tool_store::entry_dir(ctx.cacheRoot, ctx.key); + const auto exeSuffix = std::string(mcpp::platform::exe_suffix); + ctx.binOut = mcpp::build::tool_store::bin_path( + ctx.entry, ctx.toolName, exeSuffix); - // #359 (D3b): a sub-build failure must be attributable and - // REPRODUCIBLE. The Windows tool sub-build has been failing - // on three abseil TUs since #355 and is still unlocated, - // because what reached the log was a one-line summary with - // no scratch path, no chain, and — on the ninja branch below - // — a filtered view of the inner output. Naming the scratch - // directory is what lets a maintainer re-run the exact inner - // build; MCPP_TOOL_BUILD_VERBOSE turns off the filtering. - auto subContext = [&] { - return std::format( - "\n chain: {}\n sub-build scratch: {}\n" - " re-run it directly: mcpp build -p {} --release\n" - " (set MCPP_TOOL_BUILD_VERBOSE=1 for the inner " - "build's unfiltered output)", - sub.tool_chain, sub.work_dir.string(), - depPkg.root.string()); - }; - auto subCtx = prepare_build(/*print_fingerprint=*/false, - /*includeDevDeps=*/false, - /*extraTargets=*/{}, sub); - if (!subCtx) { - return std::unexpected(std::format( - "building host tool '{}:{}' failed: {}{}", - depName, toolName, subCtx.error(), subContext())); - } + if (mcpp::build::tool_store::entry_valid(ctx.entry, ctx.key, ctx.toolName, + exeSuffix)) { + step6_record_tool_provision(state, ctx, ctx.binOut); + return true; + } - // Build ONLY the requested target (#274 gave the backend - // explicit goals) — a tool request must not drag the whole - // package's other artifacts along. - std::filesystem::path goal; - for (auto const& lu : subCtx->plan.linkUnits) { - if (lu.targetName == toolName) { goal = lu.output; break; } - } - if (goal.empty()) { - return std::unexpected(std::format( - "host tool '{}:{}' produced no link unit — its " - "required_features may not be satisfiable on this " - "platform", depName, toolName)); - } + // PLANNING BUILDS NO TOOL (SPEC-005 R2.5, v1.4; #707). + // `emit build-database` describes a build; it does not + // perform one (R2.2), and a tool sub-build is a whole + // compile of another package, with its own prepare + // actions -- measured on a fresh store, a single `emit` + // compiled the tool and ran the tool package's `prepare` + // action. A tool already in the store is used as above. One + // that is not is deferred: the build program receives the + // path the tool will be published at (`binOut`, fixed + // before anything is built), which is the answer it gets + // after a successful build, and a note names the tool. A + // build program that must RUN the tool while configuring + // meets the same missing file it meets when the tool fails + // to build (SPEC-007 R5.3), so no new contract follows. + if (state.overrides.plan_only) { + state.planNotes.push_back({"MCPP_BUILD_DATABASE_HOST_TOOL_DEFERRED", + std::format("host tool '{}' of package '{}' is not in " + "the tool store and is not built while " + "planning; the plan names the path it will " + "be published at: {}", + ctx.toolName, ctx.depName, ctx.binOut.string()), + mcpp::wire::Severity::Note}); + step6_record_tool_provision(state, ctx, ctx.binOut); + return true; + } - auto be = mcpp::build::make_ninja_backend(); - mcpp::build::BuildOptions bopt; - bopt.ninjaTargets = { goal.generic_string() }; - // Unfiltered inner output on demand: the filter drops - // ninja's own progress and command echoes, which is right - // for a normal build and wrong when the question is "what - // did the inner build actually do". - if (const char* v = std::getenv("MCPP_TOOL_BUILD_VERBOSE"); - v && *v && std::string_view(v) != "0") - bopt.verbose = true; - auto br = be->build(subCtx->plan, bopt); - if (!br) { - auto diag = br.error().diagnosticOutput; - if (diag.empty()) - diag = "(the inner build produced no diagnostic " - "output; re-run with MCPP_TOOL_BUILD_VERBOSE=1)"; - return std::unexpected(std::format( - "building host tool '{}:{}' failed: {}{}\n{}", - depName, toolName, br.error().message, - subContext(), diag)); - } - if (br->exitCode != 0) { - return std::unexpected(std::format( - "building host tool '{}:{}' failed (exit {}){}", - depName, toolName, br->exitCode, subContext())); - } + return false; +} - // Publish into the store: build out of place, then move — - // the same discipline mcpp.build.stage follows, so a - // concurrent consumer never observes a half-written entry. - std::error_code cpEc; - auto produced = subCtx->plan.outputDir / goal; - if (!std::filesystem::exists(produced, cpEc)) { - return std::unexpected(std::format( - "host tool '{}:{}' built but '{}' is missing", - depName, toolName, produced.string())); - } - std::filesystem::create_directories(binOut.parent_path(), cpEc); - auto tmp = binOut; - tmp += ".tmp"; - std::filesystem::remove(tmp, cpEc); - std::filesystem::copy_file(produced, tmp, - std::filesystem::copy_options::overwrite_existing, cpEc); - if (cpEc) { - return std::unexpected(std::format( - "staging host tool '{}:{}' failed: {}", - depName, toolName, cpEc.message())); - } - std::filesystem::permissions(tmp, - std::filesystem::perms::owner_exec - | std::filesystem::perms::group_exec - | std::filesystem::perms::others_exec, - std::filesystem::perm_options::add, cpEc); - std::filesystem::rename(tmp, binOut, cpEc); - if (cpEc) { - return std::unexpected(std::format( - "publishing host tool '{}:{}' failed: {}", - depName, toolName, cpEc.message())); - } - mcpp::build::tool_store::write_entry(entry, key); - // The sub-build tree is large (protoc is several hundred - // objects) and the key covers every input, so a hit never - // needs it again. Removes only THIS consumer's scratch. - std::filesystem::remove_all(sub.work_dir, cpEc); - record(binOut); +static std::expected +step6_build_tool(PrepareState& state, HostToolCtx& ctx) { + auto& depPkg = state.packages[ctx.depIdx]; + mcpp::ui::status("Building", std::format( + "host tool {}:{} from {} v{} (once per package source and " + "host toolchain)", ctx.depName, ctx.toolName, ctx.depName, + depPkg.manifest.package.version)); + + auto& sub = ctx.sub; + sub.project_root = depPkg.root; + // Never the package root: it is shared across projects and + // may be read-only. This is the reason work_dir exists. + // + // Scratch is keyed on the CONSUMING project, not shared: + // the store is GLOBAL, so two projects can want the same + // tool at once. A single `/build` would have them + // writing one ninja tree concurrently, and whichever + // finished first would `remove_all` it out from under the + // other. The published binary is what gets shared; the + // scratch is not. + // + // Hashed rather than random so a re-run reuses its own + // scratch (ninja stays incremental if the publish step + // never got to delete it). + // + // Beside the entries rather than inside one: every + // directory name of the entry is repeated in each object + // path the sub-build writes, and on Windows those paths + // crossed the 260-character limit (mcpp#641, item 3). + sub.work_dir = mcpp::build::tool_store::scratch_dir( + ctx.cacheRoot, ctx.entry, state.workRoot); + sub.target_triple = ""; // HOST — the whole point + sub.toolchain = ctx.toolTcSpec; + sub.profile = "release"; + sub.cache_mode = state.overrides.cache_mode; + sub.tool_depth = state.overrides.tool_depth + 1; + sub.tool_chain_sources = state.overrides.tool_chain_sources; + sub.tool_chain_sources.push_back(ctx.toolSource); + // The PRISTINE manifest the resolver produced for this + // package — `packages[depIdx].manifest` is a copy that + // feature activation has already mutated, and re-activating + // on top of it would fold the same feature sources in + // twice. A `compat` (Form B) package has no mcpp.toml on + // disk at all, so without this the sub-build could not read + // a manifest for it in the first place. + // + // UNMERGED, because the sub-build targets the HOST: the + // resolver merged this manifest's conditional sections for + // the consumer's target, and the sub-build merges them for + // its own (#690, F12). + if (ctx.depIdx >= 1 && ctx.depIdx - 1 < state.dep_manifests.size() + && state.dep_manifests[ctx.depIdx - 1]) { + auto const& dep = *state.dep_manifests[ctx.depIdx - 1]; + sub.preloaded_manifest = dep.beforeConditionalMerge + ? dep.beforeConditionalMerge + : std::make_shared(dep); + } + sub.inherited_runtime_selection = std::make_shared< + const mcpp::xlings::runtime::RuntimeSelection>( + state.runtimeSelection); + sub.inherited_runtime_binding = std::make_shared< + const mcpp::platform::runtime::RuntimeBinding>( + state.runtimeBindingSnapshot); + sub.tool_chain = state.overrides.tool_chain.empty() + ? std::format("root → {}:{}", ctx.depName, ctx.toolName) + : std::format("{} → {}:{}", state.overrides.tool_chain, ctx.depName, + ctx.toolName); + for (auto const& f : ctx.closure) { + if (!sub.features.empty()) sub.features += ","; + sub.features += f; + } + + // #359 (D3b): a sub-build failure must be attributable and + // REPRODUCIBLE. The Windows tool sub-build has been failing + // on three abseil TUs since #355 and is still unlocated, + // because what reached the log was a one-line summary with + // no scratch path, no chain, and — on the ninja branch below + // — a filtered view of the inner output. Naming the scratch + // directory is what lets a maintainer re-run the exact inner + // build; MCPP_TOOL_BUILD_VERBOSE turns off the filtering. + auto subContext = [&] { + return std::format( + "\n chain: {}\n sub-build scratch: {}\n" + " re-run it directly: mcpp build -p {} --release\n" + " (set MCPP_TOOL_BUILD_VERBOSE=1 for the inner " + "build's unfiltered output)", + sub.tool_chain, sub.work_dir.string(), + depPkg.root.string()); + }; + auto subCtx = prepare_build(/*print_fingerprint=*/false, + /*includeDevDeps=*/false, + /*extraTargets=*/{}, sub); + if (!subCtx) { + return std::unexpected(std::format( + "building host tool '{}:{}' failed: {}{}", + ctx.depName, ctx.toolName, subCtx.error(), subContext())); + } + + // Build ONLY the requested target (#274 gave the backend + // explicit goals) — a tool request must not drag the whole + // package's other artifacts along. + std::filesystem::path goal; + for (auto const& lu : subCtx->plan.linkUnits) { + if (lu.targetName == ctx.toolName) { goal = lu.output; break; } + } + if (goal.empty()) { + return std::unexpected(std::format( + "host tool '{}:{}' produced no link unit — its " + "required_features may not be satisfiable on this " + "platform", ctx.depName, ctx.toolName)); + } + + auto be = mcpp::build::make_ninja_backend(); + mcpp::build::BuildOptions bopt; + bopt.ninjaTargets = { goal.generic_string() }; + // Unfiltered inner output on demand: the filter drops + // ninja's own progress and command echoes, which is right + // for a normal build and wrong when the question is "what + // did the inner build actually do". + if (const char* v = std::getenv("MCPP_TOOL_BUILD_VERBOSE"); + v && *v && std::string_view(v) != "0") + bopt.verbose = true; + auto br = be->build(subCtx->plan, bopt); + if (!br) { + auto diag = br.error().diagnosticOutput; + if (diag.empty()) + diag = "(the inner build produced no diagnostic " + "output; re-run with MCPP_TOOL_BUILD_VERBOSE=1)"; + return std::unexpected(std::format( + "building host tool '{}:{}' failed: {}{}\n{}", + ctx.depName, ctx.toolName, br.error().message, + subContext(), diag)); + } + if (br->exitCode != 0) { + return std::unexpected(std::format( + "building host tool '{}:{}' failed (exit {}){}", + ctx.depName, ctx.toolName, br->exitCode, subContext())); + } + + // `goal` and the sub-build's output directory outlive this step: the + // publish step below re-derives `produced` from them, the same way this + // function derived it the first time in the single-function version. + ctx.goal = goal; + ctx.subOutputDir = subCtx->plan.outputDir; + return {}; +} + +static std::expected +step6_publish_tool(PrepareState& state, HostToolCtx& ctx) { + // Publish into the store: build out of place, then move — + // the same discipline mcpp.build.stage follows, so a + // concurrent consumer never observes a half-written entry. + std::error_code cpEc; + auto produced = ctx.subOutputDir / ctx.goal; + if (!std::filesystem::exists(produced, cpEc)) { + return std::unexpected(std::format( + "host tool '{}:{}' built but '{}' is missing", + ctx.depName, ctx.toolName, produced.string())); + } + std::filesystem::create_directories(ctx.binOut.parent_path(), cpEc); + auto tmp = ctx.binOut; + tmp += ".tmp"; + std::filesystem::remove(tmp, cpEc); + std::filesystem::copy_file(produced, tmp, + std::filesystem::copy_options::overwrite_existing, cpEc); + if (cpEc) { + return std::unexpected(std::format( + "staging host tool '{}:{}' failed: {}", + ctx.depName, ctx.toolName, cpEc.message())); + } + std::filesystem::permissions(tmp, + std::filesystem::perms::owner_exec + | std::filesystem::perms::group_exec + | std::filesystem::perms::others_exec, + std::filesystem::perm_options::add, cpEc); + std::filesystem::rename(tmp, ctx.binOut, cpEc); + if (cpEc) { + return std::unexpected(std::format( + "publishing host tool '{}:{}' failed: {}", + ctx.depName, ctx.toolName, cpEc.message())); + } + mcpp::build::tool_store::write_entry(ctx.entry, ctx.key); + // The sub-build tree is large (protoc is several hundred + // objects) and the key covers every input, so a hit never + // needs it again. Removes only THIS consumer's scratch. + std::filesystem::remove_all(ctx.sub.work_dir, cpEc); + step6_record_tool_provision(state, ctx, ctx.binOut); + return {}; +} + +// Drives one requested tool through resolution, override/cycle checks, +// store lookup, sub-build and publish -- the sequence the single-function +// version ran as one pass down its loop body, with each `continue` above +// becoming an early `return {}` here (there is no next iteration inside a +// per-tool function; the outer loop in step6_provision_host_tools moves on +// on its own). +static std::expected +step6_provision_one_tool(PrepareState& state, HostToolCtx& ctx) { + if (auto r = step6_resolve_tool_target(state, ctx); !r) + return std::unexpected(r.error()); + + // #359: every consumer that can SEE this tool gets it, not + // just the one whose edge asked for it. The bare spelling + // is emitted only where the namespace ladder binds the tail + // to this package — otherwise two libraries re-exporting a + // same-tailed tool would decide the winner by append order. + // The spellings are `publishedNamesFor`'s, so a tool is + // addressed by exactly the names its directory is. + ctx.want = prov::Provision{ prov::Kind::Tool, ctx.depIdx, ctx.toolName }; + + auto overridden = step6_check_tool_override(state, ctx); + if (!overridden) return std::unexpected(overridden.error()); + if (*overridden) return {}; + + if (auto r = step6_check_tool_self_request(state, ctx); !r) + return std::unexpected(r.error()); + + auto resolved = step6_resolve_tool_key(state, ctx); + if (!resolved) return std::unexpected(resolved.error()); + if (*resolved) return {}; + + if (auto r = step6_build_tool(state, ctx); !r) + return std::unexpected(r.error()); + + return step6_publish_tool(state, ctx); +} + +static std::expected +step6_provision_host_tools(PrepareState& state, + const std::map>& toolRequests) { + for (auto const& [depIdx, wanted] : toolRequests) { + auto& depPkg = state.packages[depIdx]; + const auto& depName = depPkg.manifest.package.name; + std::string depShort = depName; + if (auto dot = depName.rfind('.'); + dot != std::string::npos && dot + 1 < depName.size()) + depShort = depName.substr(dot + 1); + + for (auto const& toolName : wanted) { + HostToolCtx ctx; + ctx.depIdx = depIdx; + ctx.toolName = toolName; + ctx.depName = depName; + ctx.depShort = depShort; + if (auto r = step6_provision_one_tool(state, ctx); !r) + return std::unexpected(r.error()); } } - } + return {}; +} + +static std::expected step6_dependency_build_programs(PrepareState& state) { // ── G2: dependency build.mcpp (Cargo build.rs model) ──────────────── // Runs AFTER feature activation (the env contract exposes the dep's // active features) and BEFORE the modgraph scan (generated sources @@ -1692,7 +1841,7 @@ std::expected phase6_features_and_host_tools(PrepareState& st // Same edge-graph aggregation as feature activation above, so a // dep build.mcpp sees the SAME active feature set the dep is built // with (incl. transitive requests / default-features opt-out). - auto [req, depDefaultFeatures] = aggregatedRequest(i); + auto [req, depDefaultFeatures] = aggregatedRequest(state, i); auto dirSafe = [](std::string s) { for (auto& c : s) if (c == '/' || c == '\\' || c == ':') c = '_'; return s; @@ -1768,6 +1917,12 @@ std::expected phase6_features_and_host_tools(PrepareState& st pkg.manifest.package.name, r.error()), mcpp::wire::Severity::Error, (pkg.root / "build.mcpp").string()}); + // Same reason as the root's mirror of this in + // target_side.cpp: a later check whose premise is this + // program's directives (the device-source check) must be + // able to tell this package apart from one with no program + // at all. + state.programFailedPackages.insert(pkg.root.string()); continue; } return std::unexpected(std::format( @@ -1856,7 +2011,10 @@ std::expected phase6_features_and_host_tools(PrepareState& st // first pass (above) ran before features were activated. Idempotent: // include-dir/flag propagation is unique-append. state.computeUsageRequirements(); + return {}; +} +static std::expected step6_capability_binding(PrepareState& state) { // ─── Capability binding (Stage 3) ────────────────────────────────── // For each required capability, bind exactly one provider from the // graph. Deterministic: an explicit [capabilities] pin wins; otherwise @@ -1990,7 +2148,6 @@ std::expected phase6_features_and_host_tools(PrepareState& st } // exactly one → bound implicitly. } - } // The package that supplies the C++ layer when the graph does, as an index // into `packages`. Recorded where the provider is found so that the check @@ -2011,6 +2168,24 @@ std::expected phase6_features_and_host_tools(PrepareState& st // of the same build). Before this existed, only the second reader was // written, and it derived the set itself — which is how the two could // describe different worlds. + return {}; +} + +std::expected phase6_features_and_host_tools(PrepareState& state) { + step6_check_version_floors_closure(state); + + if (auto r = step6_activate_features(state); !r) return std::unexpected(r.error()); + if (auto r = step6_device_extensions_and_rules(state); !r) return std::unexpected(r.error()); + if (auto r = step6_xlings_workspace_from_graph(state); !r) return std::unexpected(r.error()); + + auto toolRequests = step6_host_module_registration(state); + if (!toolRequests) return std::unexpected(toolRequests.error()); + if (auto r = step6_provision_host_tools(state, *toolRequests); !r) + return std::unexpected(r.error()); + + if (auto r = step6_dependency_build_programs(state); !r) return std::unexpected(r.error()); + if (auto r = step6_capability_binding(state); !r) return std::unexpected(r.error()); + return {}; } diff --git a/src/build/prepare/fetch.cpp b/src/build/prepare/fetch.cpp index a06404a0f..748d6aa06 100644 --- a/src/build/prepare/fetch.cpp +++ b/src/build/prepare/fetch.cpp @@ -126,11 +126,40 @@ std::string git_cache_head(const std::filesystem::path& gitRoot) { // directory" — a second, different error that says nothing about the first. mcpp::platform::process::RunResult run_with_network_retry( std::string_view command, - const std::function& between) { + const std::function& between, + std::string_view progressLabel) { mcpp::platform::process::RunResult r{}; mcpp::platform::env::note_network_access(); // the envelope's `effects` (#648 A4) + // A clone runs for as long as the repository takes to arrive, and its + // output was captured whole, so a large one showed nothing until it + // finished. With a label, git's download phase is drawn with the renderer + // every other acquisition uses (W11), each redraw read as it happens; the + // output is still kept whole for the failure message. A clone that writes + // nothing for fifteen minutes is stopped as stalled. + auto run_once = [&]() -> mcpp::platform::process::RunResult { + if (progressLabel.empty()) return mcpp::platform::process::capture(command); + mcpp::platform::process::RunResult out; + std::optional bar; + bool timedOut = false; + out.exit_code = mcpp::platform::process::run_streaming_bounded(command, + [&](std::string_view line) { + out.output.append(line).push_back('\n'); + auto g = mcpp::fetcher::parse_git_progress(line); + if (!g || g->phase != "Receiving objects") return; + if (!bar) bar.emplace("Fetching", progressLabel); + bar->update(g->percent); + }, + std::chrono::milliseconds{0}, std::chrono::minutes{15}, &timedOut, + /*split_on_cr=*/true); + if (bar) { + if (out.exit_code == 0 && !timedOut) bar->finish(); + else bar->finish_failed(progressLabel); + } + if (timedOut && out.exit_code == 0) out.exit_code = 124; + return out; + }; for (int attempt = 1; attempt <= 3; ++attempt) { - r = mcpp::platform::process::capture(command); + r = run_once(); if (r.exit_code == 0) return r; if (between) between(); if (attempt < 3) diff --git a/src/build/prepare/graph.cpp b/src/build/prepare/graph.cpp index 1a3815962..1f271c914 100644 --- a/src/build/prepare/graph.cpp +++ b/src/build/prepare/graph.cpp @@ -59,432 +59,104 @@ import mcpp.project; namespace mcpp::build { -std::expected phase4b_graph_worklist(PrepareState& state) { - - // #634, X: every request that reached a package, as the requester wrote - // it, for the `graph` section of resolution.json. Kept apart from - // `dependencyEdges`, which merges two requests of one consumer for one - // dependency into one edge; the record has to keep both keys, because two - // keys over one identity (A2) and the table a declaration came from (A1) - // are what it exists to show. - // The link form each dependency takes and the facts it was decided from, - // by package index. COMPUTED ONCE, before the root build program runs, so - // that program can read the answer (#642 E2); APPLIED after the scan, where - // it always was. Every reader below reads this, never a second resolution. - // #355: consumer package index → (env var, absolute path) for each host - // tool that consumer requested. Filled by the provisioning pass below; - // read by BOTH build.mcpp call sites (the dependency loop and the root), - // which is why it lives out here rather than inside the resolution block. - // #355 step 5: consumer package index → (logical module name, interface - // path) for each dependency that offers HOST build rules. Same fan-out - // shape as toolEnvByConsumer, and read by the same two call sites. - // The same providers by INDEX, and the reason they are needed twice. - // - // A rule's code runs inside its CONSUMER's build program, so - // `mcpp::xpkg_dir("cuda-nvcc")` is asked there -- while the payload that - // answers it was declared by the RULE, under `[feature-xlings.]`, which - // is where it belongs: which packages a device compiler needs is the - // rule's knowledge and no project should have to rediscover it. - // - // The graph pass already INSTALLS what a dependency declares. Only the - // answer was missing: `fillXpkgDirs` read one manifest, so the address was - // fetched, unpacked, and then unreachable from the only code that wanted - // it -- a failure that reads as "the toolkit is not installed" while it - // sits on disk. - // - // The set is the host-module providers rather than every dependency: the - // code that can call `xpkg_dir` in this build program is the consumer's - // own `build.mcpp` plus exactly the rule modules compiled into it. - // #359: who can see which build-time provision. Computed once by the - // provisioning pass below (a fixpoint over `dependencyEdges`, the same - // shape as computeUsageRequirements) and read by every consumer of the - // three env channels above. Declared here because `fillDepDirs` closes - // over it and is defined long before the pass runs; every call site is - // after it. - // The spellings a given consumer may address a provider by. The qualified - // name always works; the bare tail only when the namespace ladder binds it - // to exactly this package FOR THIS CONSUMER. Scoped per consumer rather - // than globally because two packages sharing a tail only collide inside an - // environment that contains both. - state.bareBindingsFor = [&](std::size_t consumer) { - std::vector fqns; - if (consumer < state.provisionGraph.visible.size()) - for (auto const& pr : state.provisionGraph.visible[consumer]) { - if (pr.provider >= state.packages.size()) continue; - auto const& n = state.packages[pr.provider].manifest.package.name; - if (std::find(fqns.begin(), fqns.end(), n) == fqns.end()) - fqns.push_back(n); - } - return prov::bind_bare_names(fqns); - }; - // THE NAMES UNDER WHICH ONE PROVIDER IS PUBLISHED TO ONE CONSUMER, derived - // once for every channel (#647 E4.3). The manifest's `name`, the qualified - // `namespace.name` when the manifest writes the two apart, and the bare - // tail where the namespace ladder binds it to this provider for this - // consumer. `dep_dir`/`dep_linkage` and `dep_bin` used to derive this list - // separately; #642 added the qualified spelling to the first and the second - // kept publishing `MCPP_DEP_INSTALLER_BIN_*` alone for a package written - // `namespace = "spike"`, `name = "installer"`, so - // `dep_bin("spike.installer", ...)` read nothing. - state.publishedNamesFor = - [&](std::size_t provider, - const std::map& bind) { - std::vector out; - auto const& manifest = state.packages[provider].manifest; - auto const& canon = manifest.package.name; - out.push_back(canon); - if (auto qualified = mcpp::build::qualified_package_name(manifest); - qualified != canon) - out.push_back(std::move(qualified)); - if (auto tail = prov::tail_of(canon); tail != canon) { - auto it = bind.find(tail); - if (it != bind.end() && it->second.owner == canon) - out.push_back(std::move(tail)); - } - return out; - }; - - // A package whose DECLARED targets are all programs (#649 E6). See the - // worklist, where such a package is not walked into a consumer's graph. - state.isProgramOnlyPackage = [](const mcpp::manifest::Manifest& pm) { - if (pm.targetsInferred || pm.targets.empty()) return false; - return std::ranges::none_of(pm.targets, [](const mcpp::manifest::Target& t) { - return t.kind == mcpp::manifest::Target::Library - || t.kind == mcpp::manifest::Target::SharedLibrary; - }); - }; - // A package some edge asked for programs to SHIP (mcpp#711). Its programs - // are linked in this plan, so it is scanned and configured here like any - // library dependency, even when every target it declares is a program. - state.isArtifactPackage = [&](std::size_t i) { - return std::ranges::any_of(state.dependencyEdges, [&](const DependencyEdge& e) { - return e.dependencyPackageIndex == i && !e.requestedArtifacts.empty(); - }); - }; - // Compiled in this plan: not a package of programs, or one whose programs - // this plan ships. - state.compilesHere = [&](std::size_t i) { - return i == 0 || !state.isProgramOnlyPackage(state.packages[i].manifest) || state.isArtifactPackage(i); - }; - auto parseVisibility = [](std::string_view visibility) { +// STEP FUNCTIONS (mcpp#722 / T6). The preamble closures below that +// captured only `state`, or nothing, are ordinary file-scope functions: +// statements moved verbatim, only their header changed (a name and a +// return type in place of `auto x = [&](...) {`, and an explicit +// `PrepareState& state` parameter where the body used to capture it). +// Internal linkage: these names belong to this file, not to +// mcpp.build.prepare's surface. + +static mcpp::modgraph::DependencyVisibility parseVisibility(std::string_view visibility) { if (visibility == "private") return mcpp::modgraph::DependencyVisibility::Private; if (visibility == "interface") return mcpp::modgraph::DependencyVisibility::Interface; return mcpp::modgraph::DependencyVisibility::Public; - }; +} - auto packageIndexForConsumer = [&](std::size_t consumerDepIndex) { +static std::size_t packageIndexForConsumer(std::size_t consumerDepIndex) { if (consumerDepIndex == kMainConsumer) return std::size_t{0}; return consumerDepIndex + 1; - }; - - state.appendUniquePath = - [](std::vector& dirs, - const std::filesystem::path& dir) -> bool - { - if (std::find(dirs.begin(), dirs.end(), dir) != dirs.end()) return false; - dirs.push_back(dir); - return true; - }; +} - state.appendUniquePaths = - [&](std::vector& dirs, - const std::vector& additions) -> bool - { - bool changed = false; - for (auto const& dir : additions) { - changed = state.appendUniquePath(dirs, dir) || changed; +static std::vector expandIncludeDirs( + PrepareState& state, + const std::filesystem::path& packageRoot, + const mcpp::manifest::Manifest& manifest) +{ + std::vector dirs; + for (auto const& inc : manifest.buildConfig.includeDirs) { + if (inc.is_absolute()) { + // Native spelling: a TOML `C:/SDL2/include` stays mixed on + // MSVC and leaks into the CDB's -I otherwise. Direct + // make_preferred — no generic_string round trip, which can + // throw for names the ANSI codepage cannot spell (mcpp#230). + auto n = inc; + n.make_preferred(); + state.appendUniquePath(dirs, std::move(n)); + continue; + } + for (auto& dir : mcpp::modgraph::expand_dir_glob( + packageRoot, inc.generic_string())) { + state.appendUniquePath(dirs, dir); + } } - return changed; - }; + return dirs; +} - // "Which compile-visible channels a build.mcpp directive lands in" is a - // property of the DIRECTIVE TABLE, not of this call site, so both the mark - // and the fold now live with the table in mcpp.build.directives. This pair - // used to be defined here and was already incomplete — the comment it - // replaced admitted that link/source residues stayed at the call sites, - // which is the #242 two-derivations shape. - // - // The fold is PRIVATE by design (Cargo discipline — a build-time program - // must not widen the package's public interface): privateBuild only, never - // publicUsage. The after-dirs ride the typed #249 channel, which owns the - // per-dialect degradations (cl.exe /I, NASM -I). - using DirectiveMark = mcpp::build::directives::Mark; - state.markDirectiveTail = [](const mcpp::manifest::Manifest& mm) { - return mcpp::build::directives::mark(mm); - }; - state.foldDirectiveTailIntoPrivateBuild = - [](mcpp::modgraph::PackageRoot& pkg, const mcpp::manifest::Manifest& ran, - const DirectiveMark& t) - { - mcpp::build::directives::fold_private_tail(pkg.privateBuild, ran, t); - }; + // #249: same glob expansion for `include_dirs_after` (the -idirafter + // channel — searched after the toolchain's system dirs). +static std::vector expandIncludeDirsAfter( + PrepareState& state, + const std::filesystem::path& packageRoot, + const mcpp::manifest::Manifest& manifest) +{ + std::vector dirs; + for (auto const& inc : manifest.buildConfig.includeDirsAfter) { + if (inc.is_absolute()) { + auto n = inc; + n.make_preferred(); + state.appendUniquePath(dirs, std::move(n)); + continue; + } + for (auto& dir : mcpp::modgraph::expand_dir_glob( + packageRoot, inc.generic_string())) { + state.appendUniquePath(dirs, dir); + } + } + return dirs; +} - // mcpp#241: the (name → dir) pairs a package's build.mcpp receives as - // MCPP_DEP__DIR. ONE owner: the dependency loop and the root call - // site had drifted into two near-identical copies of this, and #355 was - // about to add a third. Each dependency is emitted under BOTH its - // canonical name and its namespace-stripped tail, so - // `mcpp::dep_dir("compat.zlib")` and `mcpp::dep_dir("zlib")` both resolve - // regardless of which spelling the author used in `deps`. - // - // #359: the set is now the consumer's VISIBLE provisions rather than its - // direct edges, so a re-exported dependency's directory reaches it too. - // That is what makes a rule package able to find data files belonging to a - // dependency the user never declared — protoc's well-known .proto files - // are exactly such a directory, and `grpcgen` reads them through dep_dir. - // - // The bare tail is emitted only when the namespace ladder binds it here. - // Emitting it unconditionally was safe while only the root's own - // declarations reached build.mcpp; with re-export, two packages that never - // heard of each other can share a tail and the later emplace_back would - // silently win. - // The xlings half of fillDepDirs. Same question ("where did my declared - // dependency's payload land"), different namespace and store layout, so it - // cannot ride the mcpp dependency channel — but it must be an INTERFACE on - // the build.mcpp side for the same reason that one is: a program that - // reconstructs the store path is coupled to internals mcpp is free to - // change. See mcpp::build::hostprogram::xpkg_dir. - // Which dependency supplied the runner, for the exactly-one-provider - // error below. A name rather than a bool: the message has to name both. - // ONE PROVIDER PER RUNNER NAME. `runner` has had this rule since #544; - // a NAMED runner inherits it per name, because a board may legitimately - // supply `flash` while a different package supplies `monitor`. + // The same expansion for `private_include_dirs`, so a private entry may be + // a glob and still name exactly the directories it expands to. +static std::vector expandPrivateIncludeDirs( + PrepareState& state, + const std::filesystem::path& packageRoot, + const mcpp::manifest::Manifest& manifest) +{ + std::vector dirs; + for (auto const& inc : manifest.buildConfig.privateIncludeDirs) { + if (inc.is_absolute()) { + auto n = inc; + n.make_preferred(); + state.appendUniquePath(dirs, std::move(n)); + continue; + } + for (auto& dir : mcpp::modgraph::expand_dir_glob( + packageRoot, inc.generic_string())) { + state.appendUniquePath(dirs, dir); + } + } + return dirs; +} - state.fillXpkgDirs = [&](mcpp::build::BuildProgramEnv& e, - const mcpp::manifest::Manifest& owner, - std::size_t consumer) { - // `[feature-xlings.]` is provisioned when `` is active, so it has - // to be answerable here too. Before this, a tool a feature declared was - // downloaded and installed and then `mcpp::xpkg_dir` returned "" for it - // — the build program was told to declare a package it had already - // declared, which is a diagnostic pointing at the wrong file. - // - // The set is taken from the SAME env the caller already computed, so - // "which features are on" is answered once. Installation stays the - // filter below: a declared address whose payload is absent answers "", - // which is what a `when = "dev"` entry looks like to a consumer. - std::vector declared = owner.xlings.deps; - for (auto const& f : e.features) - if (auto it = owner.xlings.featureDeps.find(f); - it != owner.xlings.featureDeps.end()) - for (auto const& address : it->second) - if (std::ranges::find(declared, address) == declared.end()) - declared.push_back(address); - // …and what the rule packages compiled INTO this build program - // declared. Their own active features, not the consumer's: the - // consumer asked for `features = ["rules-cuda"]` on the edge, and that - // is what decides which of the rule's `[feature-xlings]` tables apply. - if (auto pit = state.hostModuleProvidersByConsumer.find(consumer); - pit != state.hostModuleProvidersByConsumer.end()) { - for (auto q : pit->second) { - if (q >= state.packages.size()) continue; - auto const& pm = state.packages[q].manifest; - auto want = [&](const std::string& address) { - if (std::ranges::find(declared, address) == declared.end()) - declared.push_back(address); - }; - for (auto const& address : pm.xlings.deps) want(address); - const auto& pf = q < state.activeFeaturesByPackage.size() - ? state.activeFeaturesByPackage[q] : std::vector{}; - for (auto const& f : pf) - if (auto it = pm.xlings.featureDeps.find(f); - it != pm.xlings.featureDeps.end()) - for (auto const& address : it->second) want(address); - } - } - if (declared.empty()) return; - auto cfg = state.get_cfg(true); - if (!cfg) return; - auto xlEnv = mcpp::config::make_xlings_env(**cfg); - std::set answered; - for (auto const& raw : declared) { - // THE VERSION THIS BUILD INSTALLED, NOT THE ONE THIS MANIFEST - // WROTE. Both statements are about one package, and only one - // version of it exists on disk; answering from the local spelling - // is how a rule package could declare `>=8.5.0`, have the project's - // exact pin installed instead, and then be told nothing is there. - // `xlingsWinner` is empty only before the split has run, and every - // caller of this lambda runs after it — the fallback keeps that a - // fact about ordering rather than a crash. - const auto key = mcpp::xlings::addrset::package_key(raw); - if (!answered.insert(key).second) continue; - auto wit = state.xlingsWinner.find(key); - const std::string spec = wit == state.xlingsWinner.end() ? raw : wit->second; - auto ref = mcpp::xlings::paths::parse_xpkg_ref(spec); - auto dir = mcpp::xlings::paths::xpkg_payload(xlEnv, ref); - if (!dir) continue; // declared but not installed: "" is the answer - // Namespaced first — it is the exact spelling, and the bare form - // below must not shadow it (the receiver keeps the first value it - // is given for a name). - e.xpkgDirs.emplace_back( - mcpp::build::xpkg_env_var(ref.ns, ref.name), dir->string()); - e.xpkgDirs.emplace_back( - mcpp::build::xpkg_env_var("", ref.name), dir->string()); - } - }; - - // `linkForms` (#642 E2): when given, each dependency that has a resolved - // library form is also offered under exactly the names its directory is, - // so `dep_linkage(n)` answers for every `n` that `dep_dir(n)` answers for. - // Only the root's program passes it; see the root call site for why. - state.fillDepDirs = [&](mcpp::build::BuildProgramEnv& e, std::size_t consumer, - const std::map* linkForms = nullptr) { - if (consumer >= state.provisionGraph.visible.size()) return; - auto bind = state.bareBindingsFor(consumer); - for (auto const& [tail, b] : bind) { - if (auto note = prov::contest_note(tail, b); !note.empty()) - mcpp::diag::warning("provisions/ambiguous", note); - } - for (auto const& pr : state.provisionGraph.visible[consumer]) { - if (pr.kind != prov::Kind::DepDir) continue; - if (pr.provider >= state.packages.size()) continue; - auto const& depPkg = state.packages[pr.provider]; - auto const& canon = depPkg.manifest.package.name; - const std::string* form = nullptr; - if (linkForms) - if (auto f = linkForms->find(pr.provider); f != linkForms->end()) - form = &f->second; - // Every spelling of `publishedNamesFor`: the manifest's name, the - // qualified name a manifest writing `namespace = "ns"` and - // `name = "fw"` is addressed by (#642: the framework's rule asks - // `dep_linkage("huxerui.huxerui")`), and the bound tail. - for (auto const& n : state.publishedNamesFor(pr.provider, bind)) { - e.depDirs.emplace_back(n, depPkg.root); - if (form) e.depLinkages.emplace_back(n, *form); - } - } - }; - - // A declared build-graph node's Source outputs must be visible to the - // scan, so they are materialized as placeholders and joined to the source - // set here — the same two lists `generated=` feeds, for the same reason - // (the scanner walks the legacy modules.sources mirror). ninja overwrites - // the placeholder before the compile edge runs, because that compile - // depends on the action's output. - state.adoptActionOutputs = [](mcpp::manifest::Manifest& mm, - const std::filesystem::path& pkgRoot, - std::size_t firstNewAction) { - if (firstNewAction >= mm.buildConfig.actions.size()) return; - std::vector fresh( - mm.buildConfig.actions.begin() - + static_cast(firstNewAction), - mm.buildConfig.actions.end()); - // The package that DECLARED the outputs classifies them: a dependency - // generating a `.ixx` asks its own manifest, not the root project's. - // Built once per package, not once per output — and BEFORE - // `prepare_actions`, which needs the same table to decide which - // outputs get a placeholder (a header does not; see mcpp#534). - const auto pkgExtTable = - mcpp::extension_table_for(mm.buildConfig.moduleExtensions, - mm.buildConfig.deviceExtensions); - mcpp::build::directives::prepare_actions(fresh, pkgRoot, pkgExtTable); - std::copy(fresh.begin(), fresh.end(), - mm.buildConfig.actions.begin() - + static_cast(firstNewAction)); - for (auto const& a : fresh) { - if (a.role != mcpp::manifest::BuildAction::Role::Source) continue; - for (auto const& o : a.outputs) { - if (o.find("${mcpp.") != std::string::npos) continue; - // Companion outputs (protoc's .pb.h next to its .pb.cc) are - // produced by the edge but are NOT translation units. - if (!mcpp::build::directives::is_compilable_output(o, pkgExtTable)) - continue; - mm.buildConfig.sources.push_back(o); - mm.modules.sources.push_back(o); - } - } - }; - - - state.appendUniqueFlags = - [](std::vector& flags, - const std::vector& additions) -> bool - { - bool changed = false; - for (auto const& f : additions) { - if (std::find(flags.begin(), flags.end(), f) != flags.end()) continue; - flags.push_back(f); - changed = true; - } - return changed; - }; - - auto expandIncludeDirs = - [&](const std::filesystem::path& packageRoot, - const mcpp::manifest::Manifest& manifest) - { - std::vector dirs; - for (auto const& inc : manifest.buildConfig.includeDirs) { - if (inc.is_absolute()) { - // Native spelling: a TOML `C:/SDL2/include` stays mixed on - // MSVC and leaks into the CDB's -I otherwise. Direct - // make_preferred — no generic_string round trip, which can - // throw for names the ANSI codepage cannot spell (mcpp#230). - auto n = inc; - n.make_preferred(); - state.appendUniquePath(dirs, std::move(n)); - continue; - } - for (auto& dir : mcpp::modgraph::expand_dir_glob( - packageRoot, inc.generic_string())) { - state.appendUniquePath(dirs, dir); - } - } - return dirs; - }; - - // #249: same glob expansion for `include_dirs_after` (the -idirafter - // channel — searched after the toolchain's system dirs). - auto expandIncludeDirsAfter = - [&](const std::filesystem::path& packageRoot, - const mcpp::manifest::Manifest& manifest) - { - std::vector dirs; - for (auto const& inc : manifest.buildConfig.includeDirsAfter) { - if (inc.is_absolute()) { - auto n = inc; - n.make_preferred(); - state.appendUniquePath(dirs, std::move(n)); - continue; - } - for (auto& dir : mcpp::modgraph::expand_dir_glob( - packageRoot, inc.generic_string())) { - state.appendUniquePath(dirs, dir); - } - } - return dirs; - }; - - // The same expansion for `private_include_dirs`, so a private entry may be - // a glob and still name exactly the directories it expands to. - auto expandPrivateIncludeDirs = - [&](const std::filesystem::path& packageRoot, - const mcpp::manifest::Manifest& manifest) - { - std::vector dirs; - for (auto const& inc : manifest.buildConfig.privateIncludeDirs) { - if (inc.is_absolute()) { - auto n = inc; - n.make_preferred(); - state.appendUniquePath(dirs, std::move(n)); - continue; - } - for (auto& dir : mcpp::modgraph::expand_dir_glob( - packageRoot, inc.generic_string())) { - state.appendUniquePath(dirs, dir); - } - } - return dirs; - }; - - auto makePackageRoot = - [&](const std::filesystem::path& packageRoot, - const mcpp::manifest::Manifest& manifest) - -> std::expected - { - // THE SNAPSHOT READS A NORMALISED MANIFEST; IT DOES NOT NORMALISE ONE. +static std::expected makePackageRoot( + PrepareState& state, + const std::filesystem::path& packageRoot, + const mcpp::manifest::Manifest& manifest) +{ + // THE SNAPSHOT READS A NORMALISED MANIFEST; IT DOES NOT NORMALISE ONE. // // Every merge that feeds a package's build inputs (workspace // inheritance, the conditional `[target..build]` sections) runs @@ -510,8 +182,8 @@ std::expected phase4b_graph_worklist(PrepareState& state) { pkg.manifest = manifest; pkg.usageResolved = true; - pkg.privateBuild.includeDirs = expandIncludeDirs(packageRoot, manifest); - pkg.privateBuild.includeDirsAfter = expandIncludeDirsAfter(packageRoot, manifest); + pkg.privateBuild.includeDirs = expandIncludeDirs(state, packageRoot, manifest); + pkg.privateBuild.includeDirsAfter = expandIncludeDirsAfter(state, packageRoot, manifest); pkg.privateBuild.cflags = manifest.buildConfig.cflags; pkg.privateBuild.cxxflags = manifest.buildConfig.cxxflags; // NOT `= privateBuild` ANY MORE — a package may now say which of @@ -530,7 +202,7 @@ std::expected phase4b_graph_worklist(PrepareState& state) { // published because it is not literally equal to `musl/src/include`. { const auto privateExpanded = - expandPrivateIncludeDirs(packageRoot, manifest); + expandPrivateIncludeDirs(state, packageRoot, manifest); for (auto const& d : pkg.privateBuild.includeDirs) if (std::ranges::find(privateExpanded, d) == privateExpanded.end()) pkg.publicUsage.includeDirs.push_back(d); @@ -564,21 +236,16 @@ std::expected phase4b_graph_worklist(PrepareState& state) { pkg.publicUsage.includeDirsAfter = pkg.privateBuild.includeDirsAfter; pkg.linkUsage.ldflags = manifest.buildConfig.ldflags; return pkg; - }; - - { - auto rootPackage = makePackageRoot(*state.root, *state.m); - if (!rootPackage) return std::unexpected(rootPackage.error()); - state.packages[0] = std::move(*rootPackage); - } +} - auto recordDependencyEdge = - [&](std::size_t consumerDepIndex, - std::size_t dependencyPackageIndex, - const mcpp::manifest::DependencySpec& spec, - bool buildOnly, - const std::string& writtenKey) - { +static void recordDependencyEdge( + PrepareState& state, + std::size_t consumerDepIndex, + std::size_t dependencyPackageIndex, + const mcpp::manifest::DependencySpec& spec, + bool buildOnly, + const std::string& writtenKey) +{ const auto consumerPackageIndex = packageIndexForConsumer(consumerDepIndex); if (consumerPackageIndex >= state.packages.size() || dependencyPackageIndex >= state.packages.size()) { @@ -665,74 +332,17 @@ std::expected phase4b_graph_worklist(PrepareState& state) { .reexport = spec.reexport, .buildOnly = buildOnly, }); - }; +} - state.computeUsageRequirements = [&] { - bool changed = true; - while (changed) { - changed = false; - for (auto const& edge : state.dependencyEdges) { - if (edge.consumerPackageIndex >= state.packages.size() - || edge.dependencyPackageIndex >= state.packages.size()) { - continue; - } - auto& consumer = state.packages[edge.consumerPackageIndex]; - auto const& dependency = state.packages[edge.dependencyPackageIndex]; - // A package of programs publishes no usage requirements to its - // consumers (#649 E6): nothing of it is compiled or linked here. - if (edge.dependencyPackageIndex > 0 - && state.isProgramOnlyPackage(dependency.manifest)) continue; - - if (edge.visibility == mcpp::modgraph::DependencyVisibility::Private - || edge.visibility == mcpp::modgraph::DependencyVisibility::Public) { - changed = state.appendUniquePaths(consumer.privateBuild.includeDirs, - dependency.publicUsage.includeDirs) - || changed; - // #249: after-dirs ride the same edges but keep their - // after-ness — consumers receive them as -idirafter, - // never upgraded to -I. - changed = state.appendUniquePaths(consumer.privateBuild.includeDirsAfter, - dependency.publicUsage.includeDirsAfter) - || changed; - // Interface defines (a dependency's active-feature `defines`) - // ride the same edges as include dirs: they must reach the - // consumer's own TUs so header-only switches like - // EIGEN_USE_BLAS take effect where the headers are used. - changed = state.appendUniqueFlags(consumer.privateBuild.cflags, - dependency.publicUsage.cflags) - || changed; - changed = state.appendUniqueFlags(consumer.privateBuild.cxxflags, - dependency.publicUsage.cxxflags) - || changed; - } - if (edge.visibility == mcpp::modgraph::DependencyVisibility::Public - || edge.visibility == mcpp::modgraph::DependencyVisibility::Interface) { - changed = state.appendUniquePaths(consumer.publicUsage.includeDirs, - dependency.publicUsage.includeDirs) - || changed; - changed = state.appendUniquePaths(consumer.publicUsage.includeDirsAfter, - dependency.publicUsage.includeDirsAfter) - || changed; - changed = state.appendUniqueFlags(consumer.publicUsage.cflags, - dependency.publicUsage.cflags) - || changed; - changed = state.appendUniqueFlags(consumer.publicUsage.cxxflags, - dependency.publicUsage.cxxflags) - || changed; - } - } - } - }; - - auto normalizeDepLdflag = [](const std::filesystem::path& depRoot, - const std::string& flag) { - auto absolute_path = [&](std::string_view raw) { - std::filesystem::path p{std::string(raw)}; - // A loader token stays as written; see the predicate. - if (p.is_absolute() || mcpp::build::is_loader_relative_search_path(raw)) - return p; - return depRoot / p; - }; +static std::string normalizeDepLdflag(const std::filesystem::path& depRoot, + const std::string& flag) { + auto absolute_path = [&](std::string_view raw) { + std::filesystem::path p{std::string(raw)}; + // A loader token stays as written; see the predicate. + if (p.is_absolute() || mcpp::build::is_loader_relative_search_path(raw)) + return p; + return depRoot / p; + }; if (flag.starts_with("-L") && flag.size() > 2) { return "-L" + absolute_path(std::string_view(flag).substr(2)).string(); @@ -745,12 +355,13 @@ std::expected phase4b_graph_worklist(PrepareState& state) { } return flag; - }; +} - auto propagateLinkFlags = [&](const std::filesystem::path& depRoot, - const mcpp::manifest::Manifest& depManifest) - -> std::vector - { +static std::vector propagateLinkFlags( + PrepareState& state, + const std::filesystem::path& depRoot, + const mcpp::manifest::Manifest& depManifest) +{ // Word by word (SPEC-004 §8, #703): a search path is made absolute // per word, and each word is written back as an element that reads as // exactly that word, so the consumer's renderer reads the dependency's @@ -763,21 +374,20 @@ std::expected phase4b_graph_worklist(PrepareState& state) { added.push_back(std::move(normalized)); } return added; - }; +} - auto removeLinkFlags = [&](const std::vector& flags) { +static void removeLinkFlags(PrepareState& state, const std::vector& flags) { auto& ldflags = state.m->buildConfig.ldflags; for (auto const& flag : flags) { auto pos = std::find(ldflags.begin(), ldflags.end(), flag); if (pos != ldflags.end()) ldflags.erase(pos); } - }; +} - auto package_source_files = []( +static std::expected, std::string> package_source_files( const std::filesystem::path& srcRoot, const mcpp::manifest::Manifest& depManifest) - -> std::expected, std::string> - { +{ // Resolve the source globs against the original root, falling // back to the convention default if the manifest didn't set any. std::vector globs = depManifest.modules.sources; @@ -808,7 +418,7 @@ std::expected phase4b_graph_worklist(PrepareState& state) { srcRoot.string(), globs.size())); } return sourceFiles; - }; +} // Stage a dep's source files into a fresh directory, rewriting their // module / import declarations against `rename`. Used by the multi- @@ -839,12 +449,12 @@ std::expected phase4b_graph_worklist(PrepareState& state) { // source and is not itself staged, verbatim: rewriting applies to module // declarations, and a header has none. Directories with no staged source // are not visited, so this stays proportional to what is being staged. - auto stage_with_rewrite = [&](const std::filesystem::path& srcRoot, - const std::filesystem::path& dstRoot, - const mcpp::manifest::Manifest& depManifest, - const std::map& rename) - -> std::expected - { +static std::expected stage_with_rewrite( + const std::filesystem::path& srcRoot, + const std::filesystem::path& dstRoot, + const mcpp::manifest::Manifest& depManifest, + const std::map& rename) +{ std::error_code ec; std::filesystem::create_directories(dstRoot, ec); if (ec) return std::unexpected(std::format( @@ -896,12 +506,12 @@ std::expected phase4b_graph_worklist(PrepareState& state) { ec.clear(); } return {}; - }; +} - auto declared_modules_for = [&](const std::filesystem::path& srcRoot, - const mcpp::manifest::Manifest& depManifest) - -> std::expected, std::string> - { +static std::expected, std::string> declared_modules_for( + const std::filesystem::path& srcRoot, + const mcpp::manifest::Manifest& depManifest) +{ auto sources = package_source_files(srcRoot, depManifest); if (!sources) return std::unexpected(sources.error()); std::vector modules; @@ -919,18 +529,48 @@ std::expected phase4b_graph_worklist(PrepareState& state) { "mangle: package '{}' declares no named C++ module to rewrite", depManifest.package.name)); return modules; - }; +} - // Stage 2a — feature-activated optional dependencies. Defined as local - // lambdas (NOT file-scope functions): keeping their std::map instantiations - // inside this implementation unit avoids polluting the exported module BMI, - // which otherwise trips a GCC-16 modules bug ("failed to load pendings for - // __normal_iterator") when other modules import std. - auto activateFeatures = [](const mcpp::manifest::Manifest& pm, - const std::vector& requested, - bool seedDefault = true) { + // Stage 2a — feature-activated optional dependencies. Static file-scope + // functions (mcpp#722 / T6 split), not local lambdas as originally + // written: the GCC 16 modules bug this comment used to warn about + // ("failed to load pendings for __normal_iterator") is triggered by an + // EXPORTED declaration's reachable set including a std::map + // instantiation; a `static` function has no external linkage and is + // never reachable from mcpp.build.prepare's exported interface, so it + // cannot pollute the BMI the bug reads from. Verified by a full build + // (mcpp itself, GCC 16.1): every consumer of this module still + // compiles clean. +static std::vector activateFeatures( + const mcpp::manifest::Manifest& pm, + const std::vector& requested, + bool seedDefault = true) { return feature_closure(pm, requested, seedDefault); // single shared implementation - }; +} + +static std::string dependencySourceOf(const mcpp::manifest::DependencySpec& s) { + if (s.inheritWorkspace) return std::string("workspace = true"); + if (s.isPath()) { + auto norm = std::filesystem::path(s.path).lexically_normal().generic_string(); + while (norm.size() > 1 && norm.back() == '/') norm.pop_back(); + return std::format("path = \"{}\"", norm); + } + if (s.isGit()) + return std::format("git = \"{}\", {} = \"{}\"", s.git, + s.gitRefKind.empty() ? "rev" : s.gitRefKind, s.gitRev); + return std::format("version = \"{}\"", s.version); +} + + // What the comparison is made on. The message shows the declaration as it + // was written; the judgement drops the whitespace inside a constraint, so + // the two declarations are compared on what they mean. +static std::string dependencySourceKey(const mcpp::manifest::DependencySpec& s) { + auto spelled = dependencySourceOf(s); + if (!s.inheritWorkspace && !s.isPath() && !s.isGit()) + std::erase_if(spelled, [](char c) { return c == ' ' || c == '\t'; }); + return spelled; +} + // Merge a manifest's active feature-deps into its `dependencies` map so the // worklist below pulls them like any normal dep. A top-level dep of the same // key is never overwritten; deps declared only under a feature appear only @@ -954,31 +594,10 @@ std::expected phase4b_graph_worklist(PrepareState& state) { // `">= 1.2.0"` and `">=1.2.0"` are one constraint and the manifest that // spells them differently built on 2026.9.15.2. A gate added for #647 E4.2 // must refuse a restatement that names another source, and nothing else. - auto dependencySourceOf = [](const mcpp::manifest::DependencySpec& s) { - if (s.inheritWorkspace) return std::string("workspace = true"); - if (s.isPath()) { - auto norm = std::filesystem::path(s.path).lexically_normal().generic_string(); - while (norm.size() > 1 && norm.back() == '/') norm.pop_back(); - return std::format("path = \"{}\"", norm); - } - if (s.isGit()) - return std::format("git = \"{}\", {} = \"{}\"", s.git, - s.gitRefKind.empty() ? "rev" : s.gitRefKind, s.gitRev); - return std::format("version = \"{}\"", s.version); - }; - // What the comparison is made on. The message shows the declaration as it - // was written; the judgement drops the whitespace inside a constraint, so - // the two declarations are compared on what they mean. - auto dependencySourceKey = [&](const mcpp::manifest::DependencySpec& s) { - auto spelled = dependencySourceOf(s); - if (!s.inheritWorkspace && !s.isPath() && !s.isGit()) - std::erase_if(spelled, [](char c) { return c == ' ' || c == '\t'; }); - return spelled; - }; - auto mergeActiveFeatureDeps = [&](mcpp::manifest::Manifest& pm, - const std::vector& requested, - bool seedDefault = true) - -> std::expected { +static std::expected mergeActiveFeatureDeps( + mcpp::manifest::Manifest& pm, + const std::vector& requested, + bool seedDefault = true) { if (pm.featureDeps.empty()) return {}; for (auto& f : activateFeatures(pm, requested, seedDefault)) { auto it = pm.featureDeps.find(f); @@ -1027,7 +646,7 @@ std::expected phase4b_graph_worklist(PrepareState& state) { } } return {}; - }; +} // #243: dep/feat forwarding. When a resolved package's feature F is active, // it may forward features to its dependencies (Cargo `[features] F = @@ -1037,10 +656,11 @@ std::expected phase4b_graph_worklist(PrepareState& state) { // spec.features) and activation (recordDependencyEdge stores spec.features on // the P->D edge, which aggregatedRequest unions and apply() activates). // Transitive forwarding rides the BFS forward edge (root -> mid -> leaf). - auto injectForwards = [](const mcpp::manifest::Manifest& parent, - const std::vector& parentActive, - const std::string& childKey, - mcpp::manifest::DependencySpec& childSpec) { +static void injectForwards( + const mcpp::manifest::Manifest& parent, + const std::vector& parentActive, + const std::string& childKey, + mcpp::manifest::DependencySpec& childSpec) { if (parent.featureForwards.empty()) return; for (auto const& f : parentActive) { auto it = parent.featureForwards.find(f); @@ -1052,23 +672,10 @@ std::expected phase4b_graph_worklist(PrepareState& state) { childSpec.features.push_back(depFeat); } } - }; - // #243: a forward whose active feature targets a dependency that is not - // declared is a manifest bug — name it instead of silently dropping. Only - // active features' forwards are checked (lazy, like the - // unknown-requested-feature gate at ~2875). - // - // THE VALIDATOR ASKS WHAT THE FORWARD LANGUAGE DEFINES: IS THE KEY DECLARED - // IN ANY DEPENDENCY TABLE OF THIS MANIFEST, ON ANY ROW, UNDER ANY FEATURE - // (#647 E4.1). It used to look in `dependencies` and `devDependencies` - // only, while `injectForwards` applies a forward to the build-dependency - // edge as well, so a forward along `[build-dependencies]` was applied and - // reported as undeclared in the same run, and `--strict` refused a build - // whose forward had worked. A key declared only for another row, or only - // under an inactive feature, is declared: on this row the forward reaches - // no edge and does nothing, which is what a portable manifest means by it. - auto declaresDependencyKey = [](const mcpp::manifest::Manifest& pm, - const std::string& key) { +} + +static bool declaresDependencyKey(const mcpp::manifest::Manifest& pm, + const std::string& key) { auto inFeatureDeps = [&](const auto& byFeature) { for (auto const& [f, deps] : byFeature) if (deps.contains(key)) return true; @@ -1083,11 +690,27 @@ std::expected phase4b_graph_worklist(PrepareState& state) { || inFeatureDeps(cc.featureDeps)) return true; return false; - }; - auto validateForwards = [&](const mcpp::manifest::Manifest& parent, - const std::vector& parentActive, - std::string_view parentName) - -> std::expected { +} + + // #243: a forward whose active feature targets a dependency that is not + // declared is a manifest bug — name it instead of silently dropping. Only + // active features' forwards are checked (lazy, like the + // unknown-requested-feature gate at ~2875). + // + // THE VALIDATOR ASKS WHAT THE FORWARD LANGUAGE DEFINES: IS THE KEY DECLARED + // IN ANY DEPENDENCY TABLE OF THIS MANIFEST, ON ANY ROW, UNDER ANY FEATURE + // (#647 E4.1). It used to look in `dependencies` and `devDependencies` + // only, while `injectForwards` applies a forward to the build-dependency + // edge as well, so a forward along `[build-dependencies]` was applied and + // reported as undeclared in the same run, and `--strict` refused a build + // whose forward had worked. A key declared only for another row, or only + // under an inactive feature, is declared: on this row the forward reaches + // no edge and does nothing, which is what a portable manifest means by it. +static std::expected validateForwards( + PrepareState& state, + const mcpp::manifest::Manifest& parent, + const std::vector& parentActive, + std::string_view parentName) { for (auto const& f : parentActive) { auto it = parent.featureForwards.find(f); if (it == parent.featureForwards.end()) continue; @@ -1103,93 +726,19 @@ std::expected phase4b_graph_worklist(PrepareState& state) { } } return {}; - }; - - // Pull the root package's active feature-deps into its dependency set before - // seeding, so `mcpp build --features X` resolves X's optional deps. - state.rootReq = parse_feature_request(state.overrides.features); - if (auto fm = mergeActiveFeatureDeps(*state.m, state.rootReq); !fm) - return std::unexpected(fm.error()); - // #243: the root's active features may forward features to its direct deps. - std::vector rootActive = feature_closure(*state.m, state.rootReq, true); - if (auto fe = validateForwards(*state.m, rootActive, state.m->package.name); !fe) - return std::unexpected(fe.error()); - state.activeFeaturesByPackage.assign(1, rootActive); - - // `--features /` (#649 E8): a forward of the root, - // applied to the edges exactly as a `[features]` forward is and checked - // against the same tables. Named whether or not the root declares - // `[features]`: the token cannot be a macro of the root, so there is no - // "pure macro usage" to preserve for it. - std::vector> cliForwards; - for (auto const& tok : feature_forward_request_tokens(state.overrides.features)) { - auto fwd = mcpp::pm::split_feature_forward_token(tok); - std::string msg; - if (!fwd) - msg = std::format("--features requests '{}', which names neither a " - "feature nor `/`", tok); - else if (!declaresDependencyKey(*state.m, fwd->first)) - msg = std::format("--features requests '{}', and no dependency table " - "of '{}' declares '{}'", tok, state.m->package.name, - fwd->first); - if (!msg.empty()) { - if (state.overrides.strict) return std::unexpected(msg); - mcpp::diag::warning("features/request", msg); - continue; - } - cliForwards.push_back(std::move(*fwd)); - } - auto injectCliForwards = [&](const std::string& childKey, - mcpp::manifest::DependencySpec& childSpec) { - for (auto const& [depKey, depFeat] : cliForwards) - if (depKey == childKey - && std::ranges::find(childSpec.features, depFeat) - == childSpec.features.end()) - childSpec.features.push_back(depFeat); - }; - - // Seed the worklist from the main manifest. Dev-deps only when the - // caller wants them; they're never propagated transitively. - const std::string mainPkgLabel = state.m->package.name; - for (auto& [n, s] : state.m->dependencies) { - auto req = s; - injectForwards(*state.m, rootActive, n, req); - injectCliForwards(n, req); - state.worklist.push_back({n, req, mainPkgLabel, req.version, kMainConsumer, {}}); - } - if (state.includeDevDeps) { - for (auto& [n, s] : state.m->devDependencies) { - auto req = s; - injectForwards(*state.m, rootActive, n, req); - injectCliForwards(n, req); - state.worklist.push_back({n, req, mainPkgLabel + " (dev-dep)", - req.version, kMainConsumer, {}, /*devOnly=*/true}); - } - } - // `[build-dependencies]`. Parsed since 0.0.x, merged across workspace - // members, conditionalised by target predicate — and until now read by - // nothing that made a decision, so writing it produced a manifest that - // loaded, no diagnostic, and no effect. Seeded here, and unlike dev-deps - // it IS walked transitively: a build dependency's own dependencies are - // what make it work, and they inherit its build-only nature. - for (auto& [n, s] : state.m->buildDependencies) { - auto req = s; - injectForwards(*state.m, rootActive, n, req); - injectCliForwards(n, req); - state.worklist.push_back({n, req, mainPkgLabel + " (build-dep)", - req.version, kMainConsumer, {}, /*devOnly=*/false, - /*buildOnly=*/true}); - } +} // `ResolvedRecord::sourceRef` for a given declaration — see the field's // comment. Computed from what was AUTHORED, not from a network round // trip: a `branch` reference is compared by name here, and the two // clones it may eventually resolve to are a question `resolveSemver`-style // ANSWERING code, not this IDENTITY code, would have to ask. - auto sourceRefOf = [&](const std::string& kind, - const mcpp::manifest::DependencySpec& s, - const std::filesystem::path& resolveRoot, - const std::string& originalConstraint) -> std::string { +static std::string sourceRefOf( + PrepareState& state, + const std::string& kind, + const mcpp::manifest::DependencySpec& s, + const std::filesystem::path& resolveRoot, + const std::string& originalConstraint) { if (kind == "git") { return std::format("{}#{}={}", s.git, s.gitRefKind, s.gitRev); } @@ -1204,11 +753,32 @@ std::expected phase4b_graph_worklist(PrepareState& state) { // "version": the constraint as authored; empty means unconstrained, // matching `addrset::unify`'s treatment of a bare-name claim. return originalConstraint.empty() ? std::string("*") : originalConstraint; - }; +} - while (!state.worklist.empty()) { - auto item = std::move(state.worklist.front()); - state.worklist.pop_front(); +// The worklist's per-item locals that cross a step boundary within one +// iteration (mcpp#722 / T6) -- the PrepareState pattern one level deeper: +// a phase-local struct passed by reference to the steps of ONE worklist +// item, the way the phase itself is passed PrepareState. Each field is +// a local the original single-function loop body declared once and read +// again in a later part of the same iteration. +struct WorklistItemCtx { + WorkItem item; + std::string sourceKind; + ResolvedKey key; + // The commit a `git` dependency resolved to, carried out of the clone + // branch below for the cache identity. + std::string sourceCommit; + // The repository member a `git` dependency selected; empty for the + // repository's root package (#649 E7). + std::string gitMember; + std::filesystem::path gitMemberCloneRoot; + std::filesystem::path dep_root; + std::optional dep_manifest; +}; + +static std::expected +step4b_resolve_identity(PrepareState& state, WorklistItemCtx& ctx) { + auto& item = ctx.item; const auto& name = item.name; auto& spec = item.spec; @@ -1273,26 +843,20 @@ std::expected phase4b_graph_worklist(PrepareState& state) { return std::unexpected(r.error()); } - ResolvedKey key{ + auto& key = ctx.key; + key = ResolvedKey{ spec.namespace_, spec.shortName.empty() ? name : spec.shortName, }; - const std::string sourceKind = + auto& sourceKind = ctx.sourceKind; + sourceKind = spec.isPath() ? "path" : spec.isGit() ? "git" : "version"; - // The commit a `git` dependency resolved to, carried out of the clone - // branch below for the cache identity. - std::string sourceCommit; - // The repository member a `git` dependency selected; empty for the - // repository's root package (#649 E7). - std::string gitMember; - std::filesystem::path gitMemberCloneRoot; - // A second key over a source that is already resolved takes the // identity resolved there; its manifest is not loaded again. if (sourceKind != "version") { - const auto source = sourceRefOf(sourceKind, spec, item.resolveRoot, + const auto source = sourceRefOf(state, sourceKind, spec, item.resolveRoot, item.originalConstraint); // A key naming another package of the same repository is that // member, not a second key over the root's identity (#649 E7). @@ -1325,112 +889,18 @@ std::expected phase4b_graph_worklist(PrepareState& state) { } } - if (auto it = state.resolved.find(key); it != state.resolved.end()) { - // A package is dev-only until some non-dev consumer wants it. Order - // of arrival must not decide, so this is an AND over every request. - it->second.devOnly = it->second.devOnly && item.devOnly; - // Conflict detection: a KIND clash (`path`/`git`/`version` differ). - // Rows 4 and 5 of the decision table in the 2026-09-13-630 record - // §2.2. Two non-root requesters keep the outright refusal (row - // 5); when the root is a party, its declaration wins instead - // (row 4) — a whole-graph choice of WHICH checkout an identity - // resolves to is exactly the kind of decision - // `DependencySpec::linkage` already reserves to the root's own - // edges (dep_spec.cppm). - if (it->second.source != sourceKind) { - const bool existingIsRoot = it->second.fromRoot; - const bool incomingIsRoot = item.consumerDepIndex == kMainConsumer; + return {}; +} - if (!existingIsRoot && !incomingIsRoot) { - return std::unexpected(std::format( - "dependency '{}{}{}' is requested as both a {} dep " - "(by '{}') and a {} dep (by '{}'). Pick one.\n" - " declare '{}{}{}' in the root to settle it.", - key.ns, key.ns.empty() ? "" : ".", key.shortName, - it->second.source, it->second.requestedBy, - sourceKind, item.requestedBy, - key.ns, key.ns.empty() ? "" : ".", key.shortName)); - } - if (incomingIsRoot && !existingIsRoot) { - // FIFO SEEDING MAKES THIS UNREACHABLE. Every root-declared - // identity is pushed onto `worklist` before this loop - // starts; a transitive dependency's request is pushed - // onto the BACK of the same deque while the loop runs. - // The root's own entry for any identity is therefore - // always dequeued — and resolved — before any - // dependency's request for that identity can arrive. If - // this branch is ever reached, the invariant broke - // upstream (the seed reordered, or a new seed source was - // added after the loop starts): refusing and naming the - // invariant is safer than silently letting whichever side - // arrived first win, which is the accident #630 reports. - return std::unexpected(std::format( - "internal: dependency '{}{}{}': the root's " - "declaration arrived after '{}' had already resolved " - "it. This is unreachable under first-in-first-out " - "worklist seeding; please report this as an mcpp " - "engine defect.", - key.ns, key.ns.empty() ? "" : ".", key.shortName, - it->second.requestedBy)); - } +static std::expected +step4b_identity_version_merge(PrepareState& state, WorklistItemCtx& ctx, + std::map::iterator it) { + auto& item = ctx.item; + auto& name = item.name; + auto& spec = item.spec; + auto& key = ctx.key; + auto& sourceKind = ctx.sourceKind; - // The root already holds this identity (existingIsRoot); the - // incoming, non-root declaration is overridden. When the - // OVERRIDDEN declaration is a version requirement, it is - // still a promise about the graph and is checked against - // what the root's checkout actually is — the same - // Holds/Violated test `addrset::unify` runs for a tool pin - // (address_set.cppm). - if (sourceKind == "version") { - const std::string winnerVersion = it->second.source == "version" - ? it->second.version - : (it->second.depIndex < state.dep_manifests.size() - ? state.dep_manifests[it->second.depIndex]->package.version - : std::string{}); - auto req = mcpp::version_req::parse_req(item.originalConstraint); - auto ver = mcpp::version_req::parse_version(winnerVersion); - // An unparseable requirement or checkout version is - // reported as an override below rather than refused: a - // refusal manufactured from ignorance is worse than the - // silent override it would be preventing (the same - // reasoning `addrset::check` states for an unparseable - // spelling). - if (req && ver && !mcpp::version_req::matches(*req, *ver)) { - return std::unexpected(std::format( - "'{}{}{}' is pinned to {} (version {}) by '{}', " - "and '{}' requires {}.\n" - " One checkout of a package is used, so the " - "two cannot both hold.\n" - " fix: relax the requirement, or point the " - "root's pin at a checkout satisfying it.", - key.ns, key.ns.empty() ? "" : ".", key.shortName, - it->second.sourceRef, winnerVersion, - it->second.requestedBy, - item.requestedBy, item.originalConstraint)); - } - } - - mcpp::diag::warning("dependency/source-override", std::format( - "'{}{}{}' is declared as a {} dep (by '{}', {}) and as a " - "{} dep (by '{}', {}); the root's declaration wins.", - key.ns, key.ns.empty() ? "" : ".", key.shortName, - it->second.source, it->second.requestedBy, it->second.sourceRef, - sourceKind, item.requestedBy, - sourceKind == "version" ? item.originalConstraint - : sourceRefOf(sourceKind, spec, - item.resolveRoot, - item.originalConstraint)), - std::format("declare '{}{}{}' in the root to choose the other.", - key.ns, key.ns.empty() ? "" : ".", key.shortName)); - - if (it->second.depIndex + 1 < state.packages.size()) { - recordDependencyEdge(item.consumerDepIndex, - it->second.depIndex + 1, - spec, item.buildOnly, name); - } - continue; - } - if (sourceKind == "version" && it->second.version != spec.version) { // SemVer merge attempt: AND-combine the two original // constraint strings and ask the index for a single version // satisfying both. Same-major caret/tilde/exact pairs that @@ -1592,12 +1062,12 @@ std::expected phase4b_graph_worklist(PrepareState& state) { .sourceKind = "version", }); const auto depPackageIndex = state.packages.size(); - auto secPackage = makePackageRoot(secStage, *state.dep_manifests.back()); + auto secPackage = makePackageRoot(state, secStage, *state.dep_manifests.back()); if (!secPackage) return std::unexpected(secPackage.error()); state.packages.push_back(std::move(*secPackage)); - recordDependencyEdge(item.consumerDepIndex, depPackageIndex, + recordDependencyEdge(state, item.consumerDepIndex, depPackageIndex, spec, item.buildOnly, name); - auto linkFlagsAdded = propagateLinkFlags(secStage, *state.dep_manifests.back()); + auto linkFlagsAdded = propagateLinkFlags(state, secStage, *state.dep_manifests.back()); ResolvedKey mangledKey{key.ns, mangledPackage}; state.resolved[mangledKey] = ResolvedRecord{ @@ -1621,7 +1091,7 @@ std::expected phase4b_graph_worklist(PrepareState& state) { std::format("{} v{} ↔ v{} → {} (cross-major fallback)", moduleName, it->second.version, spec.version, mangledModule)); - continue; + return {}; } // Combine the constraint strings so future merges AND with @@ -1637,10 +1107,10 @@ std::expected phase4b_graph_worklist(PrepareState& state) { if (*merged == it->second.version) { // The existing pin already satisfies the new constraint — // no re-fetch needed; just record this consumer edge. - recordDependencyEdge(item.consumerDepIndex, + recordDependencyEdge(state, item.consumerDepIndex, it->second.depIndex + 1, spec, item.buildOnly, name); - continue; + return {}; } // Merged version differs from the previously-pinned one. @@ -1686,18 +1156,18 @@ std::expected phase4b_graph_worklist(PrepareState& state) { newManifest.package.sourceProvenance = std::format( "index+{}@{}", state.cache_index_name(key.ns), *merged); - removeLinkFlags(it->second.linkFlagsAdded); - auto linkFlagsAdded = propagateLinkFlags(newRoot, newManifest); + removeLinkFlags(state, it->second.linkFlagsAdded); + auto linkFlagsAdded = propagateLinkFlags(state, newRoot, newManifest); // Replace in dep_manifests + packages. depIndex is the slot // in dep_manifests; packages = [main, dep_0, dep_1, …], so // packages[depIndex+1] is the same dep. *state.dep_manifests[it->second.depIndex] = std::move(newManifest); auto mergedPackage = - makePackageRoot(newRoot, *state.dep_manifests[it->second.depIndex]); + makePackageRoot(state, newRoot, *state.dep_manifests[it->second.depIndex]); if (!mergedPackage) return std::unexpected(mergedPackage.error()); state.packages[it->second.depIndex + 1] = std::move(*mergedPackage); - recordDependencyEdge(item.consumerDepIndex, + recordDependencyEdge(state, item.consumerDepIndex, it->second.depIndex + 1, spec, item.buildOnly, name); @@ -1718,7 +1188,127 @@ std::expected phase4b_graph_worklist(PrepareState& state) { child_spec.version, it->second.depIndex, {}, item.devOnly}); } - continue; + return {}; + return {}; +} + +static std::expected +step4b_handle_already_resolved(PrepareState& state, WorklistItemCtx& ctx, + std::map::iterator it) { + auto& item = ctx.item; + auto& name = item.name; + auto& spec = item.spec; + auto& key = ctx.key; + auto& sourceKind = ctx.sourceKind; + + // A package is dev-only until some non-dev consumer wants it. Order + // of arrival must not decide, so this is an AND over every request. + it->second.devOnly = it->second.devOnly && item.devOnly; + // Conflict detection: a KIND clash (`path`/`git`/`version` differ). + // Rows 4 and 5 of the decision table in the 2026-09-13-630 record + // §2.2. Two non-root requesters keep the outright refusal (row + // 5); when the root is a party, its declaration wins instead + // (row 4) — a whole-graph choice of WHICH checkout an identity + // resolves to is exactly the kind of decision + // `DependencySpec::linkage` already reserves to the root's own + // edges (dep_spec.cppm). + if (it->second.source != sourceKind) { + const bool existingIsRoot = it->second.fromRoot; + const bool incomingIsRoot = item.consumerDepIndex == kMainConsumer; + + if (!existingIsRoot && !incomingIsRoot) { + return std::unexpected(std::format( + "dependency '{}{}{}' is requested as both a {} dep " + "(by '{}') and a {} dep (by '{}'). Pick one.\n" + " declare '{}{}{}' in the root to settle it.", + key.ns, key.ns.empty() ? "" : ".", key.shortName, + it->second.source, it->second.requestedBy, + sourceKind, item.requestedBy, + key.ns, key.ns.empty() ? "" : ".", key.shortName)); + } + if (incomingIsRoot && !existingIsRoot) { + // FIFO SEEDING MAKES THIS UNREACHABLE. Every root-declared + // identity is pushed onto `worklist` before this loop + // starts; a transitive dependency's request is pushed + // onto the BACK of the same deque while the loop runs. + // The root's own entry for any identity is therefore + // always dequeued — and resolved — before any + // dependency's request for that identity can arrive. If + // this branch is ever reached, the invariant broke + // upstream (the seed reordered, or a new seed source was + // added after the loop starts): refusing and naming the + // invariant is safer than silently letting whichever side + // arrived first win, which is the accident #630 reports. + return std::unexpected(std::format( + "internal: dependency '{}{}{}': the root's " + "declaration arrived after '{}' had already resolved " + "it. This is unreachable under first-in-first-out " + "worklist seeding; please report this as an mcpp " + "engine defect.", + key.ns, key.ns.empty() ? "" : ".", key.shortName, + it->second.requestedBy)); + } + + // The root already holds this identity (existingIsRoot); the + // incoming, non-root declaration is overridden. When the + // OVERRIDDEN declaration is a version requirement, it is + // still a promise about the graph and is checked against + // what the root's checkout actually is — the same + // Holds/Violated test `addrset::unify` runs for a tool pin + // (address_set.cppm). + if (sourceKind == "version") { + const std::string winnerVersion = it->second.source == "version" + ? it->second.version + : (it->second.depIndex < state.dep_manifests.size() + ? state.dep_manifests[it->second.depIndex]->package.version + : std::string{}); + auto req = mcpp::version_req::parse_req(item.originalConstraint); + auto ver = mcpp::version_req::parse_version(winnerVersion); + // An unparseable requirement or checkout version is + // reported as an override below rather than refused: a + // refusal manufactured from ignorance is worse than the + // silent override it would be preventing (the same + // reasoning `addrset::check` states for an unparseable + // spelling). + if (req && ver && !mcpp::version_req::matches(*req, *ver)) { + return std::unexpected(std::format( + "'{}{}{}' is pinned to {} (version {}) by '{}', " + "and '{}' requires {}.\n" + " One checkout of a package is used, so the " + "two cannot both hold.\n" + " fix: relax the requirement, or point the " + "root's pin at a checkout satisfying it.", + key.ns, key.ns.empty() ? "" : ".", key.shortName, + it->second.sourceRef, winnerVersion, + it->second.requestedBy, + item.requestedBy, item.originalConstraint)); + } + } + + mcpp::diag::warning("dependency/source-override", std::format( + "'{}{}{}' is declared as a {} dep (by '{}', {}) and as a " + "{} dep (by '{}', {}); the root's declaration wins.", + key.ns, key.ns.empty() ? "" : ".", key.shortName, + it->second.source, it->second.requestedBy, it->second.sourceRef, + sourceKind, item.requestedBy, + sourceKind == "version" ? item.originalConstraint + : sourceRefOf(state, sourceKind, spec, + item.resolveRoot, + item.originalConstraint)), + std::format("declare '{}{}{}' in the root to choose the other.", + key.ns, key.ns.empty() ? "" : ".", key.shortName)); + + if (it->second.depIndex + 1 < state.packages.size()) { + recordDependencyEdge(state, item.consumerDepIndex, + it->second.depIndex + 1, + spec, item.buildOnly, name); + } + return {}; + } + if (sourceKind == "version" && it->second.version != spec.version) { + if (auto r = step4b_identity_version_merge(state, ctx, it); !r) + return std::unexpected(r.error()); + return {}; } // SAME kind, possibly DIFFERENT reference: two `git` declarations // of different rev/tag/branch, or two `path` declarations of @@ -1731,7 +1321,7 @@ std::expected phase4b_graph_worklist(PrepareState& state) { // order"). if (sourceKind != "version") { const std::string incomingRef = - sourceRefOf(sourceKind, spec, item.resolveRoot, item.originalConstraint); + sourceRefOf(state, sourceKind, spec, item.resolveRoot, item.originalConstraint); if (incomingRef != it->second.sourceRef) { const bool existingIsRoot = it->second.fromRoot; const bool incomingIsRoot = item.consumerDepIndex == kMainConsumer; @@ -1776,14 +1366,25 @@ std::expected phase4b_graph_worklist(PrepareState& state) { // consumers can need the same dep's public surface even though // the dep itself is fetched/scanned once. if (it->second.depIndex + 1 < state.packages.size()) { - recordDependencyEdge(item.consumerDepIndex, + recordDependencyEdge(state, item.consumerDepIndex, it->second.depIndex + 1, spec, item.buildOnly, name); } - continue; - } + return {}; + return {}; +} + +static std::expected +step4b_acquire_dependency_source(PrepareState& state, WorklistItemCtx& ctx) { + auto& item = ctx.item; + auto& name = item.name; + auto& spec = item.spec; + auto& key = ctx.key; + auto& sourceCommit = ctx.sourceCommit; + auto& gitMember = ctx.gitMember; + auto& gitMemberCloneRoot = ctx.gitMemberCloneRoot; - std::filesystem::path dep_root; + auto& dep_root = ctx.dep_root; if (spec.isPath()) { // Path-based: resolve relative to the consumer's root dir. @@ -1920,7 +1521,7 @@ std::expected phase4b_graph_worklist(PrepareState& state) { // on a different one than the project. auto cloneCmd = (spec.gitRefKind == "branch" && !fromLock) ? std::format( - "git clone --depth 1 --branch {} {} {} && " + "git clone --progress --depth 1 --branch {} {} {} && " "git -C {} checkout --quiet {} 2>&1", mcpp::platform::shell::quote(spec.gitRev), mcpp::platform::shell::quote(spec.git), @@ -1928,7 +1529,7 @@ std::expected phase4b_graph_worklist(PrepareState& state) { mcpp::platform::shell::quote(gitRoot.string()), mcpp::platform::shell::quote(resolvedGitRev)) : std::format( - "git clone {} {} && git -C {} checkout --quiet {} 2>&1", + "git clone --progress {} {} && git -C {} checkout --quiet {} 2>&1", mcpp::platform::shell::quote(spec.git), mcpp::platform::shell::quote(gitRoot.string()), mcpp::platform::shell::quote(gitRoot.string()), @@ -1937,7 +1538,7 @@ std::expected phase4b_graph_worklist(PrepareState& state) { // callback is removing between attempts. auto r = run_with_network_retry(cloneCmd, [&] { std::filesystem::remove_all(gitRoot, ec); - }); + }, spec.git); if (r.exit_code != 0) { std::filesystem::remove_all(gitRoot, ec); return std::unexpected(std::format( @@ -1961,7 +1562,7 @@ std::expected phase4b_graph_worklist(PrepareState& state) { sourceCommit = resolvedGitRev; dep_root = gitRoot; state.gitCloneBySource.try_emplace( - sourceRefOf("git", spec, item.resolveRoot, item.originalConstraint), + sourceRefOf(state, "git", spec, item.resolveRoot, item.originalConstraint), GitClone{ gitRoot, spec.git, spec.gitRefKind, spec.gitRev }); if (auto member = state.gitMemberDeclaring(gitRoot, key)) { gitMember = *member; @@ -1977,7 +1578,7 @@ std::expected phase4b_graph_worklist(PrepareState& state) { // - Version dep: delegate to loadVersionDep — the index entry's // `mcpp` field decides where mcpp.toml lives (StringPath / // TableBody / default lookup). - std::optional dep_manifest; + auto& dep_manifest = ctx.dep_manifest; if (spec.isPath() || spec.isGit()) { if (!std::filesystem::exists(dep_root / "mcpp.toml")) { return std::unexpected(std::format( @@ -2054,24 +1655,36 @@ std::expected phase4b_graph_worklist(PrepareState& state) { dep_root = std::move(loaded->first); dep_manifest = std::move(loaded->second); } + return {}; +} + +static std::expected +step4b_finalize_dependency(PrepareState& state, WorklistItemCtx& ctx) { + auto& item = ctx.item; + auto& name = item.name; + auto& spec = item.spec; + auto& key = ctx.key; + auto& sourceKind = ctx.sourceKind; + auto& sourceCommit = ctx.sourceCommit; + auto& gitMember = ctx.gitMember; // Name match via compat::resolve_package_name — handles both // canonical (explicit namespace field) and legacy (dotted name) // forms transparently. { auto resolved = mcpp::pm::compat::resolve_package_name( - dep_manifest->package.name, dep_manifest->package.namespace_); + ctx.dep_manifest->package.name, ctx.dep_manifest->package.namespace_); const std::string& expectedShort = spec.shortName.empty() ? name : spec.shortName; const bool nameOk = resolved.shortName == expectedShort - || dep_manifest->package.name == expectedShort - || dep_manifest->package.name == + || ctx.dep_manifest->package.name == expectedShort + || ctx.dep_manifest->package.name == mcpp::pm::compat::qualified_name(spec.namespace_, expectedShort); if (!nameOk) { return std::unexpected(std::format( "dependency '{}' resolved to package '{}' (mismatch with declared name '{}')", - name, dep_manifest->package.name, expectedShort)); + name, ctx.dep_manifest->package.name, expectedShort)); } } @@ -2081,14 +1694,14 @@ std::expected phase4b_graph_worklist(PrepareState& state) { // resolved as `mcpplibs.fw` while every reader that builds a name from // the manifest saw `huxdemo.fw`, and a second edge written // `huxdemo.fw` put the same sources into the build twice. - const bool namespaceDeclared = !dep_manifest->package.namespace_.empty(); + const bool namespaceDeclared = !ctx.dep_manifest->package.namespace_.empty(); const std::string manifestPath = sourceKind == "version" ? std::string{} - : (dep_root / "mcpp.toml").lexically_normal().generic_string(); + : (ctx.dep_root / "mcpp.toml").lexically_normal().generic_string(); if (sourceKind != "version" && namespaceDeclared) { auto declaredName = mcpp::pm::compat::resolve_package_name( - dep_manifest->package.name, dep_manifest->package.namespace_); - ResolvedKey declared{ dep_manifest->package.namespace_, + ctx.dep_manifest->package.name, ctx.dep_manifest->package.namespace_); + ResolvedKey declared{ ctx.dep_manifest->package.namespace_, declaredName.shortName }; if (!(declared == key)) { state.reportAdoption(item.requestedBy, name, key, declared, manifestPath); @@ -2107,7 +1720,7 @@ std::expected phase4b_graph_worklist(PrepareState& state) { item.spec.legacyCandidateSearch = false; item.spec.legacyDottedKey = false; state.worklist.push_front(std::move(item)); - continue; + return {}; } key = declared; } @@ -2118,26 +1731,26 @@ std::expected phase4b_graph_worklist(PrepareState& state) { // answered it; otherwise two indices containing the same short name // collapse in runtime provenance even though resolution distinguished // them correctly. - if (dep_manifest->package.namespace_.empty()) { - dep_manifest->package.namespace_ = key.ns.empty() + if (ctx.dep_manifest->package.namespace_.empty()) { + ctx.dep_manifest->package.namespace_ = key.ns.empty() ? std::string(mcpp::pm::kDefaultNamespace) : key.ns; } if (sourceKind == "version") { - dep_manifest->package.sourceProvenance = std::format( + ctx.dep_manifest->package.sourceProvenance = std::format( "index+{}@{}", state.cache_index_name(key.ns), spec.version); } else if (sourceKind == "git") { - dep_manifest->package.sourceProvenance = std::format( + ctx.dep_manifest->package.sourceProvenance = std::format( "git+{}#{}={}", spec.git, spec.gitRefKind, spec.gitRev); } else { - dep_manifest->package.sourceProvenance = - "path+" + dep_root.lexically_normal().generic_string(); + ctx.dep_manifest->package.sourceProvenance = + "path+" + ctx.dep_root.lexically_normal().generic_string(); } // Stage 2a: merge this dependency's active feature-deps into its own // dependency set before its children are pushed, so a dep's feature can // transitively pull a provider. `spec.features` = features the consumer // requested for this dep. - if (auto fm = mergeActiveFeatureDeps(*dep_manifest, spec.features, + if (auto fm = mergeActiveFeatureDeps(*ctx.dep_manifest, spec.features, spec.defaultFeatures); !fm) return std::unexpected(fm.error()); @@ -2149,16 +1762,16 @@ std::expected phase4b_graph_worklist(PrepareState& state) { // application `z.o`), compiled its sources in the consumer's build, and // made a tool that depends on the package declaring it a cycle of the // consumer's graph although the two builds never meet. - const bool depProgramOnly = state.isProgramOnlyPackage(*dep_manifest) + const bool depProgramOnly = state.isProgramOnlyPackage(*ctx.dep_manifest) && spec.artifacts.empty(); auto linkFlagsAdded = depProgramOnly ? std::vector{} - : propagateLinkFlags(dep_root, *dep_manifest); + : propagateLinkFlags(state, ctx.dep_root, *ctx.dep_manifest); // Move the manifest into stable storage so we can later look it up // by depIndex (the SemVer merger needs to overwrite the slot). state.dep_manifests.push_back( - std::make_unique(std::move(*dep_manifest))); + std::make_unique(std::move(*ctx.dep_manifest))); state.dep_cache_identities.push_back({ .indexName = state.cache_index_name(key.ns), .packageName = name, @@ -2167,21 +1780,21 @@ std::expected phase4b_graph_worklist(PrepareState& state) { : state.dep_manifests.back()->package.version, .sourceKind = sourceKind, .sourceRef = sourceKind == "git" ? sourceCommit - : sourceKind == "path" ? dep_root.string() + : sourceKind == "path" ? ctx.dep_root.string() : std::string{}, }); const auto depPackageIndex = state.packages.size(); - auto depPackage = makePackageRoot(dep_root, *state.dep_manifests.back()); + auto depPackage = makePackageRoot(state, ctx.dep_root, *state.dep_manifests.back()); if (!depPackage) return std::unexpected(depPackage.error()); state.packages.push_back(std::move(*depPackage)); - recordDependencyEdge(item.consumerDepIndex, depPackageIndex, spec, + recordDependencyEdge(state, item.consumerDepIndex, depPackageIndex, spec, item.buildOnly, name); // Record this dep as resolved so future encounters of the same // (ns, name) hit the fast path (skip / merge / conflict). if (sourceKind != "version") { state.identityBySource.emplace( - sourceRefOf(sourceKind, spec, item.resolveRoot, item.originalConstraint) + sourceRefOf(state, sourceKind, spec, item.resolveRoot, item.originalConstraint) + (gitMember.empty() ? std::string{} : "#member=" + gitMember), key); state.declaringManifest[key] = DeclaringManifest{ manifestPath, namespaceDeclared }; @@ -2191,7 +1804,7 @@ std::expected phase4b_graph_worklist(PrepareState& state) { .constraint = sourceKind == "version" ? item.originalConstraint : "", .requestedBy = item.requestedBy, .source = sourceKind, - .sourceRef = sourceRefOf(sourceKind, spec, item.resolveRoot, + .sourceRef = sourceRefOf(state, sourceKind, spec, item.resolveRoot, item.originalConstraint), .fromRoot = item.consumerDepIndex == kMainConsumer, .devOnly = item.devOnly, @@ -2203,7 +1816,7 @@ std::expected phase4b_graph_worklist(PrepareState& state) { // dev-dependencies are intentionally NOT walked — those are // private to the dep's test runs, not part of its public ABI. // A package of programs is not walked at all; see `depProgramOnly`. - if (depProgramOnly) continue; + if (depProgramOnly) return {}; const std::string thisDepLabel = std::format( "{}{}{}@{}", key.ns, @@ -2219,14 +1832,14 @@ std::expected phase4b_graph_worklist(PrepareState& state) { // with resolution. auto depActive = feature_closure(*state.dep_manifests.back(), spec.features, spec.defaultFeatures); - if (auto fe = validateForwards(*state.dep_manifests.back(), depActive, + if (auto fe = validateForwards(state, *state.dep_manifests.back(), depActive, state.dep_manifests.back()->package.name); !fe) return std::unexpected(fe.error()); for (auto& [child_name, child_spec] : state.dep_manifests.back()->dependencies) { auto childReq = child_spec; injectForwards(*state.dep_manifests.back(), depActive, child_name, childReq); state.worklist.push_back({child_name, childReq, thisDepLabel, - childReq.version, selfIdx, dep_root, + childReq.version, selfIdx, ctx.dep_root, item.devOnly, item.buildOnly}); } // A dependency's own `[build-dependencies]` — the only channel through @@ -2245,11 +1858,13 @@ std::expected phase4b_graph_worklist(PrepareState& state) { injectForwards(*state.dep_manifests.back(), depActive, child_name, childReq); state.worklist.push_back({child_name, childReq, thisDepLabel + " (build-dep)", - childReq.version, selfIdx, dep_root, + childReq.version, selfIdx, ctx.dep_root, item.devOnly, /*buildOnly=*/true}); } - } + return {}; +} +static std::expected step4b_cycle_check(PrepareState& state) { // ONE PLACE DETECTS A CYCLE OF PACKAGES, AND IT IS HERE, WHERE THE GRAPH // IS RESOLVED (#649 E6). The build-cache key walk was the only reader that // noticed, and it runs for the global cache only, so the same manifest was @@ -2297,6 +1912,530 @@ std::expected phase4b_graph_worklist(PrepareState& state) { "own tool sub-build.", path)); } } + return {}; +} + +static void step4b_define_lookup_closures(PrepareState& state) { + state.bareBindingsFor = [&](std::size_t consumer) { + std::vector fqns; + if (consumer < state.provisionGraph.visible.size()) + for (auto const& pr : state.provisionGraph.visible[consumer]) { + if (pr.provider >= state.packages.size()) continue; + auto const& n = state.packages[pr.provider].manifest.package.name; + if (std::find(fqns.begin(), fqns.end(), n) == fqns.end()) + fqns.push_back(n); + } + return prov::bind_bare_names(fqns); + }; + // THE NAMES UNDER WHICH ONE PROVIDER IS PUBLISHED TO ONE CONSUMER, derived + // once for every channel (#647 E4.3). The manifest's `name`, the qualified + // `namespace.name` when the manifest writes the two apart, and the bare + // tail where the namespace ladder binds it to this provider for this + // consumer. `dep_dir`/`dep_linkage` and `dep_bin` used to derive this list + // separately; #642 added the qualified spelling to the first and the second + // kept publishing `MCPP_DEP_INSTALLER_BIN_*` alone for a package written + // `namespace = "spike"`, `name = "installer"`, so + // `dep_bin("spike.installer", ...)` read nothing. + state.publishedNamesFor = + [&](std::size_t provider, + const std::map& bind) { + std::vector out; + auto const& manifest = state.packages[provider].manifest; + auto const& canon = manifest.package.name; + out.push_back(canon); + if (auto qualified = mcpp::build::qualified_package_name(manifest); + qualified != canon) + out.push_back(std::move(qualified)); + if (auto tail = prov::tail_of(canon); tail != canon) { + auto it = bind.find(tail); + if (it != bind.end() && it->second.owner == canon) + out.push_back(std::move(tail)); + } + return out; + }; + + // A package whose DECLARED targets are all programs (#649 E6). See the + // worklist, where such a package is not walked into a consumer's graph. + state.isProgramOnlyPackage = [](const mcpp::manifest::Manifest& pm) { + if (pm.targetsInferred || pm.targets.empty()) return false; + return std::ranges::none_of(pm.targets, [](const mcpp::manifest::Target& t) { + return t.kind == mcpp::manifest::Target::Library + || t.kind == mcpp::manifest::Target::SharedLibrary; + }); + }; + // A package some edge asked for programs to SHIP (mcpp#711). Its programs + // are linked in this plan, so it is scanned and configured here like any + // library dependency, even when every target it declares is a program. + state.isArtifactPackage = [&](std::size_t i) { + return std::ranges::any_of(state.dependencyEdges, [&](const DependencyEdge& e) { + return e.dependencyPackageIndex == i && !e.requestedArtifacts.empty(); + }); + }; + // Compiled in this plan: not a package of programs, or one whose programs + // this plan ships. + state.compilesHere = [&](std::size_t i) { + return i == 0 || !state.isProgramOnlyPackage(state.packages[i].manifest) || state.isArtifactPackage(i); + }; + + + state.appendUniquePath = + [](std::vector& dirs, + const std::filesystem::path& dir) -> bool + { + if (std::find(dirs.begin(), dirs.end(), dir) != dirs.end()) return false; + dirs.push_back(dir); + return true; + }; + + state.appendUniquePaths = + [&](std::vector& dirs, + const std::vector& additions) -> bool + { + bool changed = false; + for (auto const& dir : additions) { + changed = state.appendUniquePath(dirs, dir) || changed; + } + return changed; + }; + + // "Which compile-visible channels a build.mcpp directive lands in" is a + // property of the DIRECTIVE TABLE, not of this call site, so both the mark + // and the fold now live with the table in mcpp.build.directives. This pair + // used to be defined here and was already incomplete — the comment it + // replaced admitted that link/source residues stayed at the call sites, + // which is the #242 two-derivations shape. + // + // The fold is PRIVATE by design (Cargo discipline — a build-time program + // must not widen the package's public interface): privateBuild only, never + // publicUsage. The after-dirs ride the typed #249 channel, which owns the + // per-dialect degradations (cl.exe /I, NASM -I). + using DirectiveMark = mcpp::build::directives::Mark; + state.markDirectiveTail = [](const mcpp::manifest::Manifest& mm) { + return mcpp::build::directives::mark(mm); + }; + state.foldDirectiveTailIntoPrivateBuild = + [](mcpp::modgraph::PackageRoot& pkg, const mcpp::manifest::Manifest& ran, + const DirectiveMark& t) + { + mcpp::build::directives::fold_private_tail(pkg.privateBuild, ran, t); + }; + + // mcpp#241: the (name → dir) pairs a package's build.mcpp receives as + // MCPP_DEP__DIR. ONE owner: the dependency loop and the root call + // site had drifted into two near-identical copies of this, and #355 was + // about to add a third. Each dependency is emitted under BOTH its + // canonical name and its namespace-stripped tail, so + // `mcpp::dep_dir("compat.zlib")` and `mcpp::dep_dir("zlib")` both resolve + // regardless of which spelling the author used in `deps`. + // + // #359: the set is now the consumer's VISIBLE provisions rather than its + // direct edges, so a re-exported dependency's directory reaches it too. + // That is what makes a rule package able to find data files belonging to a + // dependency the user never declared — protoc's well-known .proto files + // are exactly such a directory, and `grpcgen` reads them through dep_dir. + // + // The bare tail is emitted only when the namespace ladder binds it here. + // Emitting it unconditionally was safe while only the root's own + // declarations reached build.mcpp; with re-export, two packages that never + // heard of each other can share a tail and the later emplace_back would + // silently win. + // The xlings half of fillDepDirs. Same question ("where did my declared + // dependency's payload land"), different namespace and store layout, so it + // cannot ride the mcpp dependency channel — but it must be an INTERFACE on + // the build.mcpp side for the same reason that one is: a program that + // reconstructs the store path is coupled to internals mcpp is free to + // change. See mcpp::build::hostprogram::xpkg_dir. + +} + +static std::expected +step4b_define_provisioning_closures(PrepareState& state) { + // Which dependency supplied the runner, for the exactly-one-provider + // error below. A name rather than a bool: the message has to name both. + // ONE PROVIDER PER RUNNER NAME. `runner` has had this rule since #544; + // a NAMED runner inherits it per name, because a board may legitimately + // supply `flash` while a different package supplies `monitor`. + + state.fillXpkgDirs = [&](mcpp::build::BuildProgramEnv& e, + const mcpp::manifest::Manifest& owner, + std::size_t consumer) { + // `[feature-xlings.]` is provisioned when `` is active, so it has + // to be answerable here too. Before this, a tool a feature declared was + // downloaded and installed and then `mcpp::xpkg_dir` returned "" for it + // — the build program was told to declare a package it had already + // declared, which is a diagnostic pointing at the wrong file. + // + // The set is taken from the SAME env the caller already computed, so + // "which features are on" is answered once. Installation stays the + // filter below: a declared address whose payload is absent answers "", + // which is what a `when = "dev"` entry looks like to a consumer. + std::vector declared = owner.xlings.deps; + for (auto const& f : e.features) + if (auto it = owner.xlings.featureDeps.find(f); + it != owner.xlings.featureDeps.end()) + for (auto const& address : it->second) + if (std::ranges::find(declared, address) == declared.end()) + declared.push_back(address); + // …and what the rule packages compiled INTO this build program + // declared. Their own active features, not the consumer's: the + // consumer asked for `features = ["rules-cuda"]` on the edge, and that + // is what decides which of the rule's `[feature-xlings]` tables apply. + if (auto pit = state.hostModuleProvidersByConsumer.find(consumer); + pit != state.hostModuleProvidersByConsumer.end()) { + for (auto q : pit->second) { + if (q >= state.packages.size()) continue; + auto const& pm = state.packages[q].manifest; + auto want = [&](const std::string& address) { + if (std::ranges::find(declared, address) == declared.end()) + declared.push_back(address); + }; + for (auto const& address : pm.xlings.deps) want(address); + const auto& pf = q < state.activeFeaturesByPackage.size() + ? state.activeFeaturesByPackage[q] : std::vector{}; + for (auto const& f : pf) + if (auto it = pm.xlings.featureDeps.find(f); + it != pm.xlings.featureDeps.end()) + for (auto const& address : it->second) want(address); + } + } + if (declared.empty()) return; + auto cfg = state.get_cfg(true); + if (!cfg) return; + auto xlEnv = mcpp::config::make_xlings_env(**cfg); + std::set answered; + for (auto const& raw : declared) { + // THE VERSION THIS BUILD INSTALLED, NOT THE ONE THIS MANIFEST + // WROTE. Both statements are about one package, and only one + // version of it exists on disk; answering from the local spelling + // is how a rule package could declare `>=8.5.0`, have the project's + // exact pin installed instead, and then be told nothing is there. + // `xlingsWinner` is empty only before the split has run, and every + // caller of this lambda runs after it — the fallback keeps that a + // fact about ordering rather than a crash. + const auto key = mcpp::xlings::addrset::package_key(raw); + if (!answered.insert(key).second) continue; + auto wit = state.xlingsWinner.find(key); + const std::string spec = wit == state.xlingsWinner.end() ? raw : wit->second; + auto ref = mcpp::xlings::paths::parse_xpkg_ref(spec); + auto dir = mcpp::xlings::paths::xpkg_payload(xlEnv, ref); + if (!dir) continue; // declared but not installed: "" is the answer + // Namespaced first — it is the exact spelling, and the bare form + // below must not shadow it (the receiver keeps the first value it + // is given for a name). + e.xpkgDirs.emplace_back( + mcpp::build::xpkg_env_var(ref.ns, ref.name), dir->string()); + e.xpkgDirs.emplace_back( + mcpp::build::xpkg_env_var("", ref.name), dir->string()); + } + }; + + // `linkForms` (#642 E2): when given, each dependency that has a resolved + // library form is also offered under exactly the names its directory is, + // so `dep_linkage(n)` answers for every `n` that `dep_dir(n)` answers for. + // Only the root's program passes it; see the root call site for why. + state.fillDepDirs = [&](mcpp::build::BuildProgramEnv& e, std::size_t consumer, + const std::map* linkForms = nullptr) { + if (consumer >= state.provisionGraph.visible.size()) return; + auto bind = state.bareBindingsFor(consumer); + for (auto const& [tail, b] : bind) { + if (auto note = prov::contest_note(tail, b); !note.empty()) + mcpp::diag::warning("provisions/ambiguous", note); + } + for (auto const& pr : state.provisionGraph.visible[consumer]) { + if (pr.kind != prov::Kind::DepDir) continue; + if (pr.provider >= state.packages.size()) continue; + auto const& depPkg = state.packages[pr.provider]; + auto const& canon = depPkg.manifest.package.name; + const std::string* form = nullptr; + if (linkForms) + if (auto f = linkForms->find(pr.provider); f != linkForms->end()) + form = &f->second; + // Every spelling of `publishedNamesFor`: the manifest's name, the + // qualified name a manifest writing `namespace = "ns"` and + // `name = "fw"` is addressed by (#642: the framework's rule asks + // `dep_linkage("huxerui.huxerui")`), and the bound tail. + for (auto const& n : state.publishedNamesFor(pr.provider, bind)) { + e.depDirs.emplace_back(n, depPkg.root); + if (form) e.depLinkages.emplace_back(n, *form); + } + } + }; + + // A declared build-graph node's Source outputs must be visible to the + // scan, so they are materialized as placeholders and joined to the source + // set here — the same two lists `generated=` feeds, for the same reason + // (the scanner walks the legacy modules.sources mirror). ninja overwrites + // the placeholder before the compile edge runs, because that compile + // depends on the action's output. + state.adoptActionOutputs = [](mcpp::manifest::Manifest& mm, + const std::filesystem::path& pkgRoot, + std::size_t firstNewAction) { + if (firstNewAction >= mm.buildConfig.actions.size()) return; + std::vector fresh( + mm.buildConfig.actions.begin() + + static_cast(firstNewAction), + mm.buildConfig.actions.end()); + // The package that DECLARED the outputs classifies them: a dependency + // generating a `.ixx` asks its own manifest, not the root project's. + // Built once per package, not once per output — and BEFORE + // `prepare_actions`, which needs the same table to decide which + // outputs get a placeholder (a header does not; see mcpp#534). + const auto pkgExtTable = + mcpp::extension_table_for(mm.buildConfig.moduleExtensions, + mm.buildConfig.deviceExtensions); + mcpp::build::directives::prepare_actions(fresh, pkgRoot, pkgExtTable); + std::copy(fresh.begin(), fresh.end(), + mm.buildConfig.actions.begin() + + static_cast(firstNewAction)); + for (auto const& a : fresh) { + if (a.role != mcpp::manifest::BuildAction::Role::Source) continue; + for (auto const& o : a.outputs) { + if (o.find("${mcpp.") != std::string::npos) continue; + // Companion outputs (protoc's .pb.h next to its .pb.cc) are + // produced by the edge but are NOT translation units. + if (!mcpp::build::directives::is_compilable_output(o, pkgExtTable)) + continue; + mm.buildConfig.sources.push_back(o); + mm.modules.sources.push_back(o); + } + } + }; + + + state.appendUniqueFlags = + [](std::vector& flags, + const std::vector& additions) -> bool + { + bool changed = false; + for (auto const& f : additions) { + if (std::find(flags.begin(), flags.end(), f) != flags.end()) continue; + flags.push_back(f); + changed = true; + } + return changed; + }; + + + + + + { + auto rootPackage = makePackageRoot(state, *state.root, *state.m); + if (!rootPackage) return std::unexpected(rootPackage.error()); + state.packages[0] = std::move(*rootPackage); + } + + + state.computeUsageRequirements = [&] { + bool changed = true; + while (changed) { + changed = false; + for (auto const& edge : state.dependencyEdges) { + if (edge.consumerPackageIndex >= state.packages.size() + || edge.dependencyPackageIndex >= state.packages.size()) { + continue; + } + auto& consumer = state.packages[edge.consumerPackageIndex]; + auto const& dependency = state.packages[edge.dependencyPackageIndex]; + // A package of programs publishes no usage requirements to its + // consumers (#649 E6): nothing of it is compiled or linked here. + if (edge.dependencyPackageIndex > 0 + && state.isProgramOnlyPackage(dependency.manifest)) continue; + + if (edge.visibility == mcpp::modgraph::DependencyVisibility::Private + || edge.visibility == mcpp::modgraph::DependencyVisibility::Public) { + changed = state.appendUniquePaths(consumer.privateBuild.includeDirs, + dependency.publicUsage.includeDirs) + || changed; + // #249: after-dirs ride the same edges but keep their + // after-ness — consumers receive them as -idirafter, + // never upgraded to -I. + changed = state.appendUniquePaths(consumer.privateBuild.includeDirsAfter, + dependency.publicUsage.includeDirsAfter) + || changed; + // Interface defines (a dependency's active-feature `defines`) + // ride the same edges as include dirs: they must reach the + // consumer's own TUs so header-only switches like + // EIGEN_USE_BLAS take effect where the headers are used. + changed = state.appendUniqueFlags(consumer.privateBuild.cflags, + dependency.publicUsage.cflags) + || changed; + changed = state.appendUniqueFlags(consumer.privateBuild.cxxflags, + dependency.publicUsage.cxxflags) + || changed; + } + if (edge.visibility == mcpp::modgraph::DependencyVisibility::Public + || edge.visibility == mcpp::modgraph::DependencyVisibility::Interface) { + changed = state.appendUniquePaths(consumer.publicUsage.includeDirs, + dependency.publicUsage.includeDirs) + || changed; + changed = state.appendUniquePaths(consumer.publicUsage.includeDirsAfter, + dependency.publicUsage.includeDirsAfter) + || changed; + changed = state.appendUniqueFlags(consumer.publicUsage.cflags, + dependency.publicUsage.cflags) + || changed; + changed = state.appendUniqueFlags(consumer.publicUsage.cxxflags, + dependency.publicUsage.cxxflags) + || changed; + } + } + } + }; + + return {}; +} + +std::expected phase4b_graph_worklist(PrepareState& state) { + + // #634, X: every request that reached a package, as the requester wrote + // it, for the `graph` section of resolution.json. Kept apart from + // `dependencyEdges`, which merges two requests of one consumer for one + // dependency into one edge; the record has to keep both keys, because two + // keys over one identity (A2) and the table a declaration came from (A1) + // are what it exists to show. + // The link form each dependency takes and the facts it was decided from, + // by package index. COMPUTED ONCE, before the root build program runs, so + // that program can read the answer (#642 E2); APPLIED after the scan, where + // it always was. Every reader below reads this, never a second resolution. + // #355: consumer package index → (env var, absolute path) for each host + // tool that consumer requested. Filled by the provisioning pass below; + // read by BOTH build.mcpp call sites (the dependency loop and the root), + // which is why it lives out here rather than inside the resolution block. + // #355 step 5: consumer package index → (logical module name, interface + // path) for each dependency that offers HOST build rules. Same fan-out + // shape as toolEnvByConsumer, and read by the same two call sites. + // The same providers by INDEX, and the reason they are needed twice. + // + // A rule's code runs inside its CONSUMER's build program, so + // `mcpp::xpkg_dir("cuda-nvcc")` is asked there -- while the payload that + // answers it was declared by the RULE, under `[feature-xlings.]`, which + // is where it belongs: which packages a device compiler needs is the + // rule's knowledge and no project should have to rediscover it. + // + // The graph pass already INSTALLS what a dependency declares. Only the + // answer was missing: `fillXpkgDirs` read one manifest, so the address was + // fetched, unpacked, and then unreachable from the only code that wanted + // it -- a failure that reads as "the toolkit is not installed" while it + // sits on disk. + // + // The set is the host-module providers rather than every dependency: the + // code that can call `xpkg_dir` in this build program is the consumer's + // own `build.mcpp` plus exactly the rule modules compiled into it. + // #359: who can see which build-time provision. Computed once by the + // provisioning pass below (a fixpoint over `dependencyEdges`, the same + // shape as computeUsageRequirements) and read by every consumer of the + // three env channels above. Declared here because `fillDepDirs` closes + // over it and is defined long before the pass runs; every call site is + // after it. + // The spellings a given consumer may address a provider by. The qualified + // name always works; the bare tail only when the namespace ladder binds it + // to exactly this package FOR THIS CONSUMER. Scoped per consumer rather + // than globally because two packages sharing a tail only collide inside an + // environment that contains both. + step4b_define_lookup_closures(state); + + if (auto r = step4b_define_provisioning_closures(state); !r) + return std::unexpected(r.error()); + + // Pull the root package's active feature-deps into its dependency set before + // seeding, so `mcpp build --features X` resolves X's optional deps. + state.rootReq = parse_feature_request(state.overrides.features); + if (auto fm = mergeActiveFeatureDeps(*state.m, state.rootReq); !fm) + return std::unexpected(fm.error()); + // #243: the root's active features may forward features to its direct deps. + std::vector rootActive = feature_closure(*state.m, state.rootReq, true); + if (auto fe = validateForwards(state, *state.m, rootActive, state.m->package.name); !fe) + return std::unexpected(fe.error()); + state.activeFeaturesByPackage.assign(1, rootActive); + + // `--features /` (#649 E8): a forward of the root, + // applied to the edges exactly as a `[features]` forward is and checked + // against the same tables. Named whether or not the root declares + // `[features]`: the token cannot be a macro of the root, so there is no + // "pure macro usage" to preserve for it. + std::vector> cliForwards; + for (auto const& tok : feature_forward_request_tokens(state.overrides.features)) { + auto fwd = mcpp::pm::split_feature_forward_token(tok); + std::string msg; + if (!fwd) + msg = std::format("--features requests '{}', which names neither a " + "feature nor `/`", tok); + else if (!declaresDependencyKey(*state.m, fwd->first)) + msg = std::format("--features requests '{}', and no dependency table " + "of '{}' declares '{}'", tok, state.m->package.name, + fwd->first); + if (!msg.empty()) { + if (state.overrides.strict) return std::unexpected(msg); + mcpp::diag::warning("features/request", msg); + continue; + } + cliForwards.push_back(std::move(*fwd)); + } + auto injectCliForwards = [&](const std::string& childKey, + mcpp::manifest::DependencySpec& childSpec) { + for (auto const& [depKey, depFeat] : cliForwards) + if (depKey == childKey + && std::ranges::find(childSpec.features, depFeat) + == childSpec.features.end()) + childSpec.features.push_back(depFeat); + }; + + // Seed the worklist from the main manifest. Dev-deps only when the + // caller wants them; they're never propagated transitively. + const std::string mainPkgLabel = state.m->package.name; + for (auto& [n, s] : state.m->dependencies) { + auto req = s; + injectForwards(*state.m, rootActive, n, req); + injectCliForwards(n, req); + state.worklist.push_back({n, req, mainPkgLabel, req.version, kMainConsumer, {}}); + } + if (state.includeDevDeps) { + for (auto& [n, s] : state.m->devDependencies) { + auto req = s; + injectForwards(*state.m, rootActive, n, req); + injectCliForwards(n, req); + state.worklist.push_back({n, req, mainPkgLabel + " (dev-dep)", + req.version, kMainConsumer, {}, /*devOnly=*/true}); + } + } + // `[build-dependencies]`. Parsed since 0.0.x, merged across workspace + // members, conditionalised by target predicate — and until now read by + // nothing that made a decision, so writing it produced a manifest that + // loaded, no diagnostic, and no effect. Seeded here, and unlike dev-deps + // it IS walked transitively: a build dependency's own dependencies are + // what make it work, and they inherit its build-only nature. + for (auto& [n, s] : state.m->buildDependencies) { + auto req = s; + injectForwards(*state.m, rootActive, n, req); + injectCliForwards(n, req); + state.worklist.push_back({n, req, mainPkgLabel + " (build-dep)", + req.version, kMainConsumer, {}, /*devOnly=*/false, + /*buildOnly=*/true}); + } + + + while (!state.worklist.empty()) { + WorklistItemCtx ctx; + ctx.item = std::move(state.worklist.front()); + state.worklist.pop_front(); + + if (auto r = step4b_resolve_identity(state, ctx); !r) + return std::unexpected(r.error()); + + if (auto it = state.resolved.find(ctx.key); it != state.resolved.end()) { + if (auto r = step4b_handle_already_resolved(state, ctx, it); !r) + return std::unexpected(r.error()); + continue; + } + + if (auto r = step4b_acquire_dependency_source(state, ctx); !r) + return std::unexpected(r.error()); + if (auto r = step4b_finalize_dependency(state, ctx); !r) + return std::unexpected(r.error()); + } + + if (auto r = step4b_cycle_check(state); !r) return std::unexpected(r.error()); state.computeUsageRequirements(); diff --git a/src/build/prepare/graph_load.cpp b/src/build/prepare/graph_load.cpp index 59fb4ea80..08a5532ed 100644 --- a/src/build/prepare/graph_load.cpp +++ b/src/build/prepare/graph_load.cpp @@ -50,7 +50,15 @@ import mcpp.project; namespace mcpp::build { -std::expected phase4a_graph_load(PrepareState& state) { +// STEP FUNCTIONS (mcpp#722 / T6): the closures phase4a_graph_load +// assigns onto `state` (each captures only `state`) are split into two +// groups; `LoadedDep` is hoisted here so it stays visible to +// `state.loadVersionDep`, which is defined further down, in the +// orchestrator itself (see the file's own comment for why it is not +// split further). +using LoadedDep = std::pair; + +static void step4a_define_split_and_identity_closures(PrepareState& state) { // The features each package ends up built with, index-aligned with // `packages`. Recorded at activation because the passes that run after it // — `[feature-xlings]` provisioning among them — otherwise have no way to @@ -250,12 +258,9 @@ std::expected phase4a_graph_load(PrepareState& state) { s.version = std::move(*resolved); return {}; }; +} - // Acquire a version-source dep at a specific pinned version. Used both - // by the first-time walk and by the SemVer merger when a re-fetch at a - // different version is needed. Returns the dep's effective root (where - // mcpp.toml lives) and a fully loaded manifest. - using LoadedDep = std::pair; +static void step4a_define_candidate_selection_closures(PrepareState& state) { // Identity-first candidate probe. A candidate is DISAMBIGUATED by the // DECLARED (namespace, name) of whatever descriptor the index holds — never // by whether a canonically-named file `..lua` happens to exist on @@ -594,67 +599,95 @@ std::expected phase4a_graph_load(PrepareState& state) { spec.candidates = std::move(candidates); return {}; }; +} - // 0.0.10+: loadVersionDep accepts structured (ns, shortName) for - // namespace-aware lookup. depName is the map key (qualified or bare), - // kept for install() target formatting and error messages. +// A phase-local struct passed by reference to the steps of ONE +// loadVersionDep call -- the same pattern WorklistItemCtx (graph.cpp) and +// HostToolCtx (features.cpp) use for the steps of one worklist item / one +// requested tool. loadVersionDep is recursive (a preinstall hook may call +// state.loadVersionDep again for one of its own dependencies below), so this +// struct is local to one call's stack frame, not shared across calls. +struct LoadVersionDepCtx { + std::string depName; + std::string ns; + std::string shortName; + std::string version; + const mcpp::pm::IndexSpec* idxSpec = nullptr; + bool useProjectEnv = false; + std::optional luaContent; + std::optional installed; +}; + +// The body of the `readLuaContent` closure the single-function version of +// this step captured per call. Used at two points below (the initial read, +// and the re-check after a fresh install), so it is a named helper rather +// than a per-call closure. `state.get_cfg` is memoized (state.cfg_opt), so +// reconstructing `fetcher` here costs nothing beyond the first call. +static std::optional +step4a_read_lua_content(PrepareState& state, LoadVersionDepCtx& ctx) { + auto cfg = state.get_cfg(true); + if (!cfg) return std::nullopt; // already validated once at closure entry + mcpp::fetcher::Fetcher fetcher(**cfg); + if (ctx.idxSpec && ctx.idxSpec->is_local()) { + auto indexPath = mcpp::config::resolve_project_index_path(*state.root, *ctx.idxSpec); + return mcpp::fetcher::Fetcher::read_xpkg_lua_from_path( + indexPath, ctx.ns, ctx.shortName); + } + if (ctx.idxSpec && !ctx.idxSpec->is_builtin()) { + return mcpp::fetcher::Fetcher::read_xpkg_lua_from_project_data( + *state.root, ctx.ns, ctx.shortName); + } + return fetcher.read_xpkg_lua(ctx.ns, ctx.shortName); +} - state.loadVersionDep = [&](const std::string& depName, - const std::string& ns, - const std::string& shortName, - const std::string& version) - -> std::expected - { - auto cfg = state.get_cfg(true); - if (!cfg) return std::unexpected(cfg.error()); - mcpp::fetcher::Fetcher fetcher(**cfg); +// The body of the `findRawInstalled` closure. Used at two points below (the +// initial completeness probe, and again after a fresh install), so it is a +// named helper rather than a per-call closure. +static std::optional +step4a_find_raw_installed(PrepareState& state, LoadVersionDepCtx& ctx) { + auto cfg = state.get_cfg(true); + if (!cfg) return std::nullopt; // already validated once at closure entry + mcpp::fetcher::Fetcher fetcher(**cfg); + if (ctx.useProjectEnv) { + if (auto p = mcpp::fetcher::Fetcher::install_path_from_project_data( + *state.root, ctx.ns, ctx.shortName, ctx.version)) { + return p; + } + } + return fetcher.install_path(ctx.ns, ctx.shortName, ctx.version); +} + +// The body of the `markInstalled` closure, called once below after a fresh +// install completes. +static void step4a_mark_installed(const std::filesystem::path& p) { + mcpp::fallback::mark_install_complete(p); +} +static std::expected +step4a_load_version_dep_locate(PrepareState& state, LoadVersionDepCtx& ctx) { // ─── Routing: check if this dep's namespace maps to a custom index ── - auto* idxSpec = state.findIndexForNs(ns); + ctx.idxSpec = state.findIndexForNs(ctx.ns); - const bool useProjectEnv = idxSpec && !idxSpec->is_builtin(); + ctx.useProjectEnv = ctx.idxSpec && !ctx.idxSpec->is_builtin(); - auto readLuaContent = [&]() -> std::optional { - if (idxSpec && idxSpec->is_local()) { - auto indexPath = mcpp::config::resolve_project_index_path(*state.root, *idxSpec); - return mcpp::fetcher::Fetcher::read_xpkg_lua_from_path( - indexPath, ns, shortName); - } - if (idxSpec && !idxSpec->is_builtin()) { - return mcpp::fetcher::Fetcher::read_xpkg_lua_from_project_data( - *state.root, ns, shortName); - } - return fetcher.read_xpkg_lua(ns, shortName); - }; - - auto luaContent = readLuaContent(); - if (idxSpec && idxSpec->is_local() && !luaContent) { - auto indexPath = mcpp::config::resolve_project_index_path(*state.root, *idxSpec); + ctx.luaContent = step4a_read_lua_content(state, ctx); + if (ctx.idxSpec && ctx.idxSpec->is_local() && !ctx.luaContent) { + auto indexPath = mcpp::config::resolve_project_index_path(*state.root, *ctx.idxSpec); return std::unexpected(with_index_cause(std::format( "dependency '{}': not found in local index at '{}'", - depName, indexPath.string()))); + ctx.depName, indexPath.string()))); } - auto findRawInstalled = [&]() -> std::optional { - if (useProjectEnv) { - if (auto p = mcpp::fetcher::Fetcher::install_path_from_project_data( - *state.root, ns, shortName, version)) { - return p; - } - } - return fetcher.install_path(ns, shortName, version); - }; - auto installedLayoutMatchesIndex = [&](const std::filesystem::path& verRoot) -> bool { - if (!luaContent) return false; + if (!ctx.luaContent) return false; - auto field = mcpp::manifest::extract_mcpp_field(*luaContent); + auto field = mcpp::manifest::extract_mcpp_field(*ctx.luaContent); if (field.kind == mcpp::manifest::McppField::StringPath) { return !mcpp::modgraph::expand_glob(verRoot, field.value).empty(); } if (field.kind == mcpp::manifest::McppField::TableBody) { auto dm = mcpp::manifest::synthesize_from_xpkg_lua( - *luaContent, shortName, version, *state.targetPlatform); + *ctx.luaContent, ctx.shortName, ctx.version, *state.targetPlatform); if (!dm) return false; for (auto const& [generatedPath, _] : dm->buildConfig.generatedFiles) { if (!generatedPath.empty()) return true; @@ -684,10 +717,10 @@ std::expected phase4a_graph_load(PrepareState& state) { // and records the new revision. A payload with no xlings record at // all is judged by the marker alone, as before. const int recipeRevision = [&] { - if (!luaContent) return 0; + if (!ctx.luaContent) return 0; for (auto const& e : mcpp::manifest::list_xpkg_version_entries( - *luaContent, *state.targetPlatform)) - if (e.version == version) return e.revision; + *ctx.luaContent, *state.targetPlatform)) + if (e.version == ctx.version) return e.revision; return 0; }(); auto revisionIsCurrent = [&](const std::filesystem::path& p) { @@ -695,12 +728,12 @@ std::expected phase4a_graph_load(PrepareState& state) { if (!installed || *installed == recipeRevision) return true; mcpp::log::verbose("fetcher", std::format( "{}@{}: installed revision {}, descriptor revision {}; reinstalling", - depName, version, *installed, recipeRevision)); + ctx.depName, ctx.version, *installed, recipeRevision)); return false; }; auto findCompleteInstalled = [&]() -> std::optional { - auto p = findRawInstalled(); + auto p = step4a_find_raw_installed(state, ctx); if (!p) return std::nullopt; if (!revisionIsCurrent(*p)) return std::nullopt; if (mcpp::fallback::is_install_complete(*p)) return p; @@ -712,23 +745,32 @@ std::expected phase4a_graph_load(PrepareState& state) { return std::nullopt; }; - auto markInstalled = [&](const std::filesystem::path& p) { - mcpp::fallback::mark_install_complete(p); - }; - // For custom indices, try project-level xlings data roots first. // Existing directories without the mcpp completion marker are treated // as stale/incomplete on this active resolve path and reinstalled. - std::optional installed = findCompleteInstalled(); + ctx.installed = findCompleteInstalled(); + + return {}; +} + +static std::expected +step4a_load_version_dep_fetch(PrepareState& state, LoadVersionDepCtx& ctx) { + auto cfg = state.get_cfg(true); + if (!cfg) return std::unexpected(cfg.error()); + mcpp::fetcher::Fetcher fetcher(**cfg); + auto const& depName = ctx.depName; + auto const& ns = ctx.ns; + auto const& shortName = ctx.shortName; + auto const& version = ctx.version; // #278 masking guard. The hard INV-NAME check lives on the install path // below, so a machine that already has the package from an older index // snapshot keeps building. That asymmetry is exactly the trap the issue // names — local green, clean CI red — so make it visible here instead of // letting it stay silent. - if (installed && luaContent) { + if (ctx.installed && ctx.luaContent) { if (auto violation = mcpp::manifest:: - xpkg_name_form_violation_from_lua(*luaContent)) { + xpkg_name_form_violation_from_lua(*ctx.luaContent)) { mcpp::ui::warning(std::format( "dependency '{}': {}\n" " resolving from the already-installed copy; a clean " @@ -737,12 +779,12 @@ std::expected phase4a_graph_load(PrepareState& state) { } } - if (!installed) { - if (luaContent) { - auto field = mcpp::manifest::extract_mcpp_field(*luaContent); + if (!ctx.installed) { + if (ctx.luaContent) { + auto field = mcpp::manifest::extract_mcpp_field(*ctx.luaContent); if (field.kind == mcpp::manifest::McppField::TableBody) { auto depManifest = mcpp::manifest::synthesize_from_xpkg_lua( - *luaContent, shortName, version, *state.targetPlatform); + *ctx.luaContent, shortName, version, *state.targetPlatform); if (!depManifest) { return std::unexpected(std::format( "dependency '{}': {}", depName, depManifest.error().format())); @@ -802,11 +844,11 @@ std::expected phase4a_graph_load(PrepareState& state) { // `mcpp::manifest::xpkg_wire_address` for why splitting the two // sources is the bug it is. auto wireAddr = mcpp::manifest::xpkg_wire_address( - luaContent ? std::string_view(*luaContent) : std::string_view{}, + ctx.luaContent ? std::string_view(*ctx.luaContent) : std::string_view{}, ns, shortName); - if (luaContent) { + if (ctx.luaContent) { if (auto violation = mcpp::manifest:: - xpkg_name_form_violation_from_lua(*luaContent)) { + xpkg_name_form_violation_from_lua(*ctx.luaContent)) { return std::unexpected(std::format( "dependency '{}': {}", depName, *violation)); } @@ -846,7 +888,7 @@ std::expected phase4a_graph_load(PrepareState& state) { into += (into.empty() ? "" : "\n ") + std::string("xlings: ") + line; }; auto install_one = [&](std::string target) -> std::expected { - if (useProjectEnv) { + if (ctx.useProjectEnv) { // Project/custom-index deps install into the project-local // xlings data root (so a package's install hook can find // sibling packages from the same index). The NDJSON @@ -952,7 +994,7 @@ std::expected phase4a_graph_load(PrepareState& state) { // global xlings home), any child error text we captured, plus // a hint about the known ≥2-repo xlings resolution gap. The // real fix lives in openxlings/xlings; this only surfaces WHY. - auto xlingsJson = (useProjectEnv + auto xlingsJson = (ctx.useProjectEnv ? (state.workRoot / ".mcpp") : (*cfg)->xlingsHome()) / ".xlings.json"; @@ -975,23 +1017,32 @@ std::expected phase4a_graph_load(PrepareState& state) { return std::unexpected(std::move(diag)); } // After install, check project data first for custom index packages. - installed = findRawInstalled(); - if (!installed) return std::unexpected(std::format( + ctx.installed = step4a_find_raw_installed(state, ctx); + if (!ctx.installed) return std::unexpected(std::format( "package '{}@{}' install path missing after fetch", depName, version)); - markInstalled(*installed); + step4a_mark_installed(*ctx.installed); } - std::filesystem::path verRoot = *installed; + + return {}; +} + +static std::expected +step4a_load_version_dep_read_manifest(PrepareState& state, LoadVersionDepCtx& ctx) { + auto const& depName = ctx.depName; + auto const& shortName = ctx.shortName; + auto const& version = ctx.version; + std::filesystem::path verRoot = *ctx.installed; // Route xpkg.lua reading through the appropriate index. - if (!luaContent) { - luaContent = readLuaContent(); + if (!ctx.luaContent) { + ctx.luaContent = step4a_read_lua_content(state, ctx); } - if (!luaContent) return std::unexpected(with_index_cause(std::format( + if (!ctx.luaContent) return std::unexpected(with_index_cause(std::format( "dependency '{}': index entry not found in local clone", depName))); - auto field = mcpp::manifest::extract_mcpp_field(*luaContent); + auto field = mcpp::manifest::extract_mcpp_field(*ctx.luaContent); // 0.0.6+: read explicit namespace from xpkg lua if present. - auto luaNs = mcpp::manifest::extract_xpkg_namespace(*luaContent); + auto luaNs = mcpp::manifest::extract_xpkg_namespace(*ctx.luaContent); std::optional manifest; std::filesystem::path effRoot = verRoot; @@ -1032,7 +1083,7 @@ std::expected phase4a_graph_load(PrepareState& state) { if (auto r = loadFrom(matches.front()); !r) return std::unexpected(r.error()); } else if (field.kind == mcpp::manifest::McppField::TableBody) { auto dm = mcpp::manifest::synthesize_from_xpkg_lua( - *luaContent, shortName, version, *state.targetPlatform); + *ctx.luaContent, shortName, version, *state.targetPlatform); if (!dm) return std::unexpected(std::format( "dependency '{}': {}", depName, dm.error().format())); warn_unknown_xpkg_keys(*dm, depName); @@ -1107,8 +1158,48 @@ std::expected phase4a_graph_load(PrepareState& state) { if (state.abiThreadsRendered) state.add_once(manifest->buildConfig.cflags, "-pthread"); return std::pair{effRoot, std::move(*manifest)}; +} + +std::expected phase4a_graph_load(PrepareState& state) { + step4a_define_split_and_identity_closures(state); + step4a_define_candidate_selection_closures(state); + + // 0.0.10+: loadVersionDep accepts structured (ns, shortName) for + // namespace-aware lookup. depName is the map key (qualified or bare), + // kept for install() target formatting and error messages. + // + // The body itself is split into the three steps a call passes through in + // sequence (locate an already-installed copy, fetch one if none is + // installed, then read its manifest) over a LoadVersionDepCtx that holds + // this call's parameters and the locals more than one step reads -- + // step4a_load_version_dep_locate, step4a_load_version_dep_fetch and + // step4a_load_version_dep_read_manifest above. This lambda stays the + // entry point with the same signature, so `state.loadVersionDep`'s + // callers, including its own recursive call for a preinstall hook's + // dependencies, do not change. + state.loadVersionDep = [&](const std::string& depName, + const std::string& ns, + const std::string& shortName, + const std::string& version) + -> std::expected + { + auto cfg = state.get_cfg(true); + if (!cfg) return std::unexpected(cfg.error()); + + LoadVersionDepCtx ctx; + ctx.depName = depName; + ctx.ns = ns; + ctx.shortName = shortName; + ctx.version = version; + + if (auto r = step4a_load_version_dep_locate(state, ctx); !r) + return std::unexpected(r.error()); + if (auto r = step4a_load_version_dep_fetch(state, ctx); !r) + return std::unexpected(r.error()); + return step4a_load_version_dep_read_manifest(state, ctx); }; return {}; } + } // namespace mcpp::build diff --git a/src/build/prepare/manifest.cpp b/src/build/prepare/manifest.cpp index 9ee6aec32..62cb68f8e 100644 --- a/src/build/prepare/manifest.cpp +++ b/src/build/prepare/manifest.cpp @@ -45,6 +45,116 @@ import mcpp.project; namespace mcpp::build { +// STEP FUNCTION (mcpp#722 / T6 follow-on): phase0's own "Workspace +// handling" section, extracted verbatim. +static std::expected step0_workspace_handling(PrepareState& state) { + // ─── Workspace handling ──────────────────────────────────────────── + // If the manifest has [workspace] and is a virtual workspace (no [package]), + // or if -p filter is set, switch to the target member's manifest. + if (state.m->workspace.present) { + std::string targetMember; + + if (!state.overrides.package_filter.empty()) { + // `-p `: the package identity first, the member's + // directory as a fallback -- one resolver shared with every + // other `-p`/`--package` command + // (mcpp::project::resolve_member_dir, #725). + auto matched = mcpp::project::resolve_member_dir( + *state.m, *state.root, state.overrides.package_filter); + if (!matched) return std::unexpected(matched.error()); + targetMember = matched->lexically_relative(*state.root).generic_string(); + } else if (state.m->package.name.empty()) { + // Virtual workspace: find a member with a program target ("is + // this the program", #622 A3's `is_program()`, so a member whose + // only target is `kind = "app"` is picked exactly as one whose + // target is `bin` is), or use last member. + for (auto& mp : state.m->workspace.members) { + auto memberDir = *state.root / mp; + auto mm = mcpp::manifest::load(memberDir / "mcpp.toml", + {.insideWorkspace = true}); + if (!mm) continue; + for (auto& t : mm->targets) { + if (t.is_program()) { + targetMember = mp; + break; + } + } + if (!targetMember.empty()) break; + } + if (targetMember.empty() && !state.m->workspace.members.empty()) { + targetMember = state.m->workspace.members.back(); + } + } + // else: rooted workspace with [package] — build root normally. Its own + // `x.workspace = true` entries name its own [workspace.dependencies]. + // The workspace context is set here too (#725): it is a property of + // where the manifest lives, not of the branch that was taken, so a + // member this package reaches through its OWN `path` dependencies + // (`depIsMember`, graph.cpp) is recognised as a member and receives + // `[workspace.package]`, `[workspace.build]` and `x.workspace = true` + // the same way a sibling's `path` dependency does. + else if (state.m->workspace.present) { + state.runtimeWorkspaceRoot = *state.root; + state.wsManifest = *state.m; + mcpp::project::merge_workspace_deps(*state.m, *state.m, *state.root); + } + + if (!targetMember.empty()) { + auto memberDir = *state.root / targetMember; + if (!std::filesystem::exists(memberDir / "mcpp.toml")) { + return std::unexpected(std::format( + "workspace member '{}' has no mcpp.toml", targetMember)); + } + state.runtimeWorkspaceRoot = *state.root; + state.wsManifest = std::move(*state.m); // preserve workspace manifest + auto memberManifest = mcpp::manifest::load(memberDir / "mcpp.toml", + {.insideWorkspace = true}); + if (!memberManifest) return std::unexpected(std::format( + "workspace member '{}': {}", targetMember, + memberManifest.error().format())); + state.m = std::move(*memberManifest); + + // ONE call, not a hand-copied list. `*root` is still the WORKSPACE + // root here (the `root = memberDir` reassignment below has not + // happened yet), which is what a relative `[indices].path` or + // `[workspace.dependencies] path` was written against (#224). + mcpp::project::inherit_workspace_config(*state.m, *state.wsManifest, *state.root); + if (auto bad = mcpp::project::workspace_inheritance_error(*state.m, memberDir)) + return std::unexpected(*bad); + + mcpp::ui::status("Workspace", std::format("building member '{}'", targetMember)); + state.root = memberDir; + } + } else { + // Not at workspace root: inside a member, the loader above has + // already inherited (#224 anchoring included). Only the workspace is + // recorded here, for the membership test of this member's own `path` + // dependencies. + if (state.effective && state.effective->member) { + state.runtimeWorkspaceRoot = state.effective->workspaceRoot; + state.wsManifest = std::move(*state.effective->workspace); + } else if (state.overrides.preloaded_manifest) { + auto wsRoot = mcpp::project::find_workspace_root(*state.root); + if (!wsRoot.empty()) { + if (auto wsm = mcpp::manifest::load(wsRoot / "mcpp.toml"); + wsm && wsm->workspace.present) { + state.runtimeWorkspaceRoot = wsRoot; + state.wsManifest = std::move(*wsm); + } + } + // A preloaded manifest was inherited at its dependency load site, + // which gives a member everything but the root-position keys. This + // build IS rooted at it (a host-tool sub-build), so it takes those + // too, from the workspace that lists it (#710). + if (state.wsManifest + && mcpp::project::is_workspace_member(*state.wsManifest, state.runtimeWorkspaceRoot, *state.root)) + mcpp::project::inherit_workspace_root_position( + *state.m, *state.wsManifest, state.runtimeWorkspaceRoot); + } + } + return {}; +} + std::expected phase0_manifest_and_workspace(PrepareState& state) { // A refusal decided early and released late. `host_can_serve` answers // "does a payload on this machine produce this target", which is knowable @@ -221,106 +331,8 @@ std::expected phase0_manifest_and_workspace(PrepareState& sta state.root->string(), state.m->package.name, state.root->string())); } - // ─── Workspace handling ──────────────────────────────────────────── - // If the manifest has [workspace] and is a virtual workspace (no [package]), - // or if -p filter is set, switch to the target member's manifest. - if (state.m->workspace.present) { - std::string targetMember; - - if (!state.overrides.package_filter.empty()) { - // -p : find matching member by directory basename or path - for (auto& mp : state.m->workspace.members) { - auto basename = std::filesystem::path(mp).filename().string(); - if (basename == state.overrides.package_filter || mp == state.overrides.package_filter) { - targetMember = mp; - break; - } - } - if (targetMember.empty()) { - return std::unexpected(std::format( - "workspace member '{}' not found in [workspace].members", - state.overrides.package_filter)); - } - } else if (state.m->package.name.empty()) { - // Virtual workspace: find a member with a program target ("is - // this the program", #622 A3's `is_program()`, so a member whose - // only target is `kind = "app"` is picked exactly as one whose - // target is `bin` is), or use last member. - for (auto& mp : state.m->workspace.members) { - auto memberDir = *state.root / mp; - auto mm = mcpp::manifest::load(memberDir / "mcpp.toml", - {.insideWorkspace = true}); - if (!mm) continue; - for (auto& t : mm->targets) { - if (t.is_program()) { - targetMember = mp; - break; - } - } - if (!targetMember.empty()) break; - } - if (targetMember.empty() && !state.m->workspace.members.empty()) { - targetMember = state.m->workspace.members.back(); - } - } - // else: rooted workspace with [package] — build root normally. Its own - // `x.workspace = true` entries name its own [workspace.dependencies]. - else if (state.m->workspace.present) - mcpp::project::merge_workspace_deps(*state.m, *state.m, *state.root); - - if (!targetMember.empty()) { - auto memberDir = *state.root / targetMember; - if (!std::filesystem::exists(memberDir / "mcpp.toml")) { - return std::unexpected(std::format( - "workspace member '{}' has no mcpp.toml", targetMember)); - } - state.runtimeWorkspaceRoot = *state.root; - state.wsManifest = std::move(*state.m); // preserve workspace manifest - auto memberManifest = mcpp::manifest::load(memberDir / "mcpp.toml", - {.insideWorkspace = true}); - if (!memberManifest) return std::unexpected(std::format( - "workspace member '{}': {}", targetMember, - memberManifest.error().format())); - state.m = std::move(*memberManifest); + if (auto r = step0_workspace_handling(state); !r) return std::unexpected(r.error()); - // ONE call, not a hand-copied list. `*root` is still the WORKSPACE - // root here (the `root = memberDir` reassignment below has not - // happened yet), which is what a relative `[indices].path` or - // `[workspace.dependencies] path` was written against (#224). - mcpp::project::inherit_workspace_config(*state.m, *state.wsManifest, *state.root); - if (auto bad = mcpp::project::workspace_inheritance_error(*state.m, memberDir)) - return std::unexpected(*bad); - - mcpp::ui::status("Workspace", std::format("building member '{}'", targetMember)); - state.root = memberDir; - } - } else { - // Not at workspace root: inside a member, the loader above has - // already inherited (#224 anchoring included). Only the workspace is - // recorded here, for the membership test of this member's own `path` - // dependencies. - if (state.effective && state.effective->member) { - state.runtimeWorkspaceRoot = state.effective->workspaceRoot; - state.wsManifest = std::move(*state.effective->workspace); - } else if (state.overrides.preloaded_manifest) { - auto wsRoot = mcpp::project::find_workspace_root(*state.root); - if (!wsRoot.empty()) { - if (auto wsm = mcpp::manifest::load(wsRoot / "mcpp.toml"); - wsm && wsm->workspace.present) { - state.runtimeWorkspaceRoot = wsRoot; - state.wsManifest = std::move(*wsm); - } - } - // A preloaded manifest was inherited at its dependency load site, - // which gives a member everything but the root-position keys. This - // build IS rooted at it (a host-tool sub-build), so it takes those - // too, from the workspace that lists it (#710). - if (state.wsManifest - && mcpp::project::is_workspace_member(*state.wsManifest, state.runtimeWorkspaceRoot, *state.root)) - mcpp::project::inherit_workspace_root_position( - *state.m, *state.wsManifest, state.runtimeWorkspaceRoot); - } - } if (auto bad = mcpp::project::unresolved_workspace_dependency_error(*state.m, *state.root)) return std::unexpected(*bad); diff --git a/src/build/prepare/plan.cpp b/src/build/prepare/plan.cpp index 2813db92a..34e157a1e 100644 --- a/src/build/prepare/plan.cpp +++ b/src/build/prepare/plan.cpp @@ -1,6 +1,6 @@ // plan.cpp -- P13: the BuildContext: the plan, prebuilt dependencies, -// assembly units, Windows resources, the global cache, mcpp.lock and -// resolution.json. +// assembly units, Windows resources and the global cache. mcpp.lock and +// resolution.json are written by records.cpp. module mcpp.build.prepare; import :state; @@ -9,6 +9,7 @@ import mcpp.build.prepare_inputs; import std; import mcpp.diag; +import mcpp.build.stage; import mcpp.build.refusal; import mcpp.build.version_floor; import mcpp.home; @@ -22,6 +23,7 @@ import mcpp.toolchain.hostflags; // the compile-token producer the package std import mcpp.toolchain.cppfly; import mcpp.toolchain.detect; import mcpp.toolchain.dialect; +import mcpp.toolchain.model; // is_msvc_target — the MSVC-ABI default (#718) import mcpp.toolchain.fingerprint; import mcpp.toolchain.registry; import mcpp.toolchain.linkmodel; @@ -76,23 +78,17 @@ import mcpp.bmi_cache; namespace mcpp::build { -std::expected phase13_finish(PrepareState& state) { - BuildContext ctx; - ctx.strict = state.overrides.strict; - ctx.manifest = *state.m; - ctx.tc = *state.tc; - ctx.fp = state.fp; - ctx.runtimeSelection = state.runtimeSelection; - ctx.runtimeBinding = state.runtimeBindingSnapshot; - ctx.profile = state.effectiveProfile; - ctx.activeFeatureRequest = state.overrides.features; - ctx.compilerChoice = { std::string(tc_origin_name(state.tcOrigin)), - state.graphCompilerRequiredBy, - state.graphCompilerReplaced.empty() ? state.pinReplacedDefault - : state.graphCompilerReplaced }; - ctx.cacheMode = state.cacheMode; - ctx.projectRoot= *state.root; - ctx.outputDir = target_dir(*state.tc, state.fp, state.workRoot); + +// SUB-STEPS (mcpp#722 / T6). Each function below is one section of +// phase13_finish, named for what its own banner already called it, +// extracted verbatim: statements moved, not reordered or rewritten. Every +// step takes the same (PrepareState&, BuildContext&) pair phase13_finish +// held locally, called in the original order from the slimmed-down +// phase13_finish at the bottom of this file. Internal linkage: these +// names are this file's own, not part of mcpp.build.prepare's surface; +// the two records steps live in records.cpp and are declared in `:state`. + +static std::expected step13_source_packages(PrepareState& state, BuildContext& ctx) { { std::error_code ec; const bool firstPlan = !std::filesystem::exists(ctx.outputDir / "build.ninja", ec); @@ -167,6 +163,10 @@ std::expected phase13_finish(PrepareState& state) { } ctx.depSourceRoots = std::move(roots); } + return {}; +} + +static std::expected step13_runner_and_xlings(PrepareState& state, BuildContext& ctx) { // Where a runner may find the programs this project declared (#544). The // same resolution `fillXpkgDirs` hands to build programs, kept as // directories rather than env vars because the reader is mcpp's own @@ -258,6 +258,10 @@ std::expected phase13_finish(PrepareState& state) { } } } + return {}; +} + +static std::expected step13_prebuilt_check(PrepareState& state, BuildContext& ctx) { // ─── Prebuilt dependencies: check before planning to link them ───── // // Here rather than at each place a dependency manifest is loaded, because @@ -297,7 +301,10 @@ std::expected phase13_finish(PrepareState& state) { return std::unexpected(ok.error()); } } + return {}; +} +static std::expected step13_link_forms(PrepareState& state, BuildContext& ctx) { // ── #519: the form each dependency takes, APPLIED ────────────────────── // // The answers were computed before the root build program (see there). @@ -352,12 +359,34 @@ std::expected phase13_finish(PrepareState& state) { t.kind = mcpp::manifest::Target::SharedLibrary; } } + return {}; +} +static std::expected step13_make_plan(PrepareState& state, BuildContext& ctx) { auto planResult = mcpp::build::make_plan(*state.m, *state.tc, state.fp, state.scan.graph, state.report.topoOrder, state.packages, *state.root, ctx.outputDir, state.stdBmiPath, state.stdObjectPath, state.storeRoots); if (!planResult) return std::unexpected(planResult.error()); ctx.plan = std::move(*planResult); + // SPEC-007 R4.3: a declared deploy outranks a search directory's file of + // the same name, and a difference between the two is said here, where + // the user sees it (the post-link placement edge says it only under -v). + // A declared source that an action writes is left to that edge: at + // planning it may still hold the previous build's bytes. + std::set actionOutputs; + for (auto const& a : ctx.plan.actions) + for (auto const& o : a.outputs) + actionOutputs.insert(std::filesystem::path(o).lexically_normal()); + for (auto const& s : ctx.plan.shadowedSearchDirDlls) { + if (actionOutputs.contains(s.declared.lexically_normal())) continue; + std::error_code ec; + if (std::filesystem::is_regular_file(s.declared, ec) + && !mcpp::build::stage::same_content(s.declared, s.offered)) + mcpp::diag::warning("build/deploy-shadows-search-dir", std::format( + "'{}' is placed by this project's deploy list; the runtime " + "search directories also offer a different '{}', which is " + "not used", s.declared.string(), s.offered.string())); + } // Resolved far above, where the dependency graph first exists. It is // attached here rather than threaded through `make_plan` because nothing // that function does depends on it: the flag assembly that does reads the @@ -405,7 +434,10 @@ std::expected phase13_finish(PrepareState& state) { add_std_unit(state.tc->stdCompatSource, sm.compatCommands, sm.compatObjectPath, sm.compatBmiPath, "std.compat", {"std"}); } + return {}; +} +static std::expected step13_cxx_private_runtime(PrepareState& state, BuildContext& ctx) { // A DEPENDENCY'S C++ SHARED LIBRARY OVER A C++ RUNTIME THAT IS A PACKAGE // (#641, item 5). // @@ -507,7 +539,10 @@ std::expected phase13_finish(PrepareState& state) { providerName, provider.package.version, constrained, staticRemedy)); } } + return {}; +} +static std::expected step13_cxx_process_runtime(PrepareState& state, BuildContext& ctx) { // ONE PROCESS, ONE C++ RUNTIME; ONE STATIC PACKAGE, ONE IMAGE (#646). // // Both are decided by `make_plan` and the contract table; this is where a @@ -526,14 +561,53 @@ std::expected phase13_finish(PrepareState& state) { .tests = mcpp::build::image_loads_cxx_shared_library( ctx.plan, mcpp::build::LinkUnit::TestBinary), }; + // The MSVC-ABI whole-project default (#718, §7.3) — read the same way + // flags.cppm does, so the record this check reads and the flags a + // build actually emits cannot disagree about which contract an + // undeclared row resolved to. + const std::optional msvcAbiDefault = + mcpp::toolchain::is_msvc_target(*state.tc) + ? std::optional(dist::msvc_abi_default_contract( + mcpp::toolchain::msvc_wants_static_crt( + bc.linkage, bc.cxxRuntime), + !state.tc->msvcRedistDir.empty())) + : std::nullopt; const auto contracts = dist::role_contracts( dist::ContractStatement{ .cxxRuntime = bc.cxxRuntime, .cxxRuntimeTests = bc.cxxRuntimeTests, .cxxRuntimeShared = bc.cxxRuntimeShared, .staticStdlib = bc.staticStdlib, + .msvcAbiDefault = msvcAbiDefault, }, format, load); + // A ROW WITHOUT A REDISTRIBUTABLE DIRECTORY CANNOT DELIVER AN + // EXPLICIT `toolchain-coupled`, AND SAYS SO BEFORE COMPILING. + // + // The undeclared case is silent (`msvc_abi_default_contract` already + // resolved it to host-coupled above); an explicit statement that + // cannot be met is an error, never a downgrade with a warning — the + // same rule `mcpp pack`'s mode contradiction follows. + if (mcpp::toolchain::is_msvc_target(*state.tc) + && state.tc->msvcRedistDir.empty()) { + struct { dist::Contract c; bool stated; std::string_view role; } rows[] = { + {contracts.program, contracts.programStated, "distributable"}, + {contracts.tests, contracts.testsStated, "test"}, + {contracts.shared, contracts.sharedStated, "shared-library"}, + }; + for (auto const& r : rows) { + if (r.c != dist::Contract::ToolchainCoupled || !r.stated) continue; + refusal::record(refusal::Code::MsvcRedistUnavailable); + return std::unexpected(std::format( + "cxx_runtime = \"toolchain-coupled\" cannot be delivered " + "for the {} target: this MSVC toolset carries no " + "VC\\Redist\\MSVC directory to stage vcruntime140.dll / " + "msvcp140.dll from.\n" + " Use cxx_runtime = \"host-coupled\", or a toolset " + "that ships its redistributable.", + r.role)); + } + } // F3a. A stated self-contained program over a coupled C++ shared // library of this build: the program would carry a static C++ runtime // and the library would load a shared one. The unstated case needs no @@ -658,7 +732,10 @@ std::expected phase13_finish(PrepareState& state) { "= \"shared\" }}", first)); } } + return {}; +} +static void step13_graph_and_schedule(PrepareState& state, BuildContext& ctx) { // The module graph outlives the plan for one consumer: `mcpp pack`, which // has to know which units are INTERFACE (published as source) and which // are implementation (published only as an object). The plan flattens that @@ -748,7 +825,9 @@ std::expected phase13_finish(PrepareState& state) { if (state.tc->compiler == mcpp::toolchain::CompilerId::GCC && !state.overrides.plan_only) ctx.plan.gccCleanSpecs = mcpp::toolchain::write_clean_link_specs( state.tc->binaryPath, ctx.outputDir); +} +static std::expected step13_build_graph_actions(PrepareState& state, BuildContext& ctx) { // ── Declared build-graph nodes → the plan ─────────────────────────────── // // Collected here rather than inside make_plan because the engine-variable @@ -1105,7 +1184,10 @@ std::expected phase13_finish(PrepareState& state) { } ctx.plan.stdCompatBmiPath = state.stdCompatBmiPath; ctx.plan.stdCompatObjectPath = state.stdCompatObjectPath; + return {}; +} +static std::expected step13_assembly_units(PrepareState& state, BuildContext& ctx) { // Clang: discover clang-scan-deps for P1689 dyndep scanning. if (mcpp::toolchain::is_clang(*state.tc)) { if (auto sd = mcpp::toolchain::clang::find_scan_deps(*state.tc)) { @@ -1196,7 +1278,10 @@ std::expected phase13_finish(PrepareState& state) { ctx.plan.nasmPath = *nasmBin; } } + return {}; +} +static std::expected step13_windows_resources(PrepareState& state, BuildContext& ctx) { // ─── Windows resources: [resources] → a tracked link input (mcpp#365) ── // // Four rules, in this order: @@ -1536,7 +1621,10 @@ std::expected phase13_finish(PrepareState& state) { if (trip.is_pe()) if (auto r = plan_resources(); !r) return std::unexpected(r.error()); } + return {}; +} +static std::expected step13_dependency_cache(PrepareState& state, BuildContext& ctx) { // ─── Global dependency cache: per-package keys, hit → stage edges ── // // Every index package gets a key over the axes that actually reach its @@ -1865,154 +1953,10 @@ std::expected phase13_finish(PrepareState& state) { } } // ────────────────────────────────────────────────────────────────── + return {}; +} - // Write/update mcpp.lock for any version-based deps that succeeded. - // Path deps are intentionally NOT locked — their source is local filesystem. - // - // mcpp#363: the version entries come from `resolved` — what the walk - // actually picked — not from `m->dependencies`, which still holds the - // constraint the user wrote and only covers DIRECT deps. Reading the input - // instead of the output made the lock record `^1.92.8` (a range locks - // nothing) and omit the transitive graph entirely. Git entries deliberately - // stay on `m->dependencies`: their lock line is read back as a resolution - // anchor (#329), keyed by the root manifest's map key, and that contract is - // unchanged here. - { - mcpp::lockfile::Lockfile lock; - lock.schemaVersion = 2; - - // The lock key for a dep the ROOT declares is the map key it declared - // it under (`compat.imgui`, `gtest`) — that is the key #329's git anchor - // lookup uses, and changing it would silently unpin every branch dep. - // A dep reached only transitively has no such key, so it is written - // under its fully-qualified identity. - auto lock_name_for = [&](const ResolvedKey& k) -> std::string { - for (auto const& [n, s] : state.m->dependencies) { - const std::string sn = s.shortName.empty() ? n : s.shortName; - if (s.namespace_ == k.ns && sn == k.shortName) return n; - } - return mcpp::pm::compat::qualified_name(k.ns, k.shortName); - }; - - // Lock custom index shas from manifest [indices] section. - for (auto const& [idxName, spec] : state.m->indices) { - if (spec.is_local() || spec.is_builtin()) continue; - mcpp::lockfile::LockedIndex li; - li.name = idxName; - li.url = spec.url; - li.rev = spec.rev; // may be empty if not yet resolved - lock.indices.push_back(std::move(li)); - } - - // Git deps: root-declared only, unchanged (see the note above). - for (auto const& [name, spec] : state.m->dependencies) { - if (!spec.isGit()) continue; - mcpp::lockfile::LockedPackage lp; - lp.name = name; - lp.version = spec.gitRev; - auto gitIt = state.root_git_lock_identities.find(name); - if (gitIt == state.root_git_lock_identities.end()) { - lp.source = std::format("git+{}#{}={}", - spec.git, spec.gitRefKind, spec.gitRev); - lp.hash = "fnv1a:" + mcpp::toolchain::hash_string(lp.source); - } else { - lp.source = gitIt->second.source; - lp.hash = gitIt->second.hash; - } - lock.packages.push_back(std::move(lp)); - } - - // Version deps: the whole resolved graph, at the versions actually - // chosen. `resolved` is an ordered map, so the file is deterministic. - for (auto const& [key, rec] : state.resolved) { - if (rec.source != "version") continue; // path / git handled elsewhere - if (rec.version.empty()) continue; - // See ResolvedRecord::devOnly: `mcpp test` resolves dev-deps and - // `mcpp build` does not, so writing them would make the file depend - // on which command ran last. - if (rec.devOnly) continue; - mcpp::lockfile::LockedPackage lp; - lp.name = lock_name_for(key); - lp.namespace_ = key.ns; - lp.version = rec.version; - // Use the namespace and resolved version as the source identifier. - // For custom indices, include the index name for traceability. - auto sourceIndex = lp.namespace_.empty() - ? std::string(mcpp::pm::kDefaultNamespace) - : lp.namespace_; - lp.source = std::format("index+{}@{}", sourceIndex, lp.version); - // Use a deterministic hash based on namespace + name + version. - // A future PR can replace this with a real content hash from the - // xpkg.lua's declared sha256 or from the install plan. - // - // NOT `std::hash`: its output is implementation-defined - // (MSVC FNV-1a, libstdc++/libc++ MurmurHash), so the same dependency - // used to hash differently on Windows and Linux while the `fnv1a:` - // prefix claimed otherwise. `index_package_digest` is FNV-1a on - // every host. - lp.hash = mcpp::pm::index_package_digest(sourceIndex, lp.name, lp.version); - lock.packages.push_back(std::move(lp)); - } - if (!lock.packages.empty() || !lock.indices.empty()) { - auto lockPath = state.workRoot / "mcpp.lock"; - // `--locked` ASSERTS THAT THIS RESOLUTION IS THE RECORDED ONE. - // - // The file has always been written after the walk and never read - // back as a constraint; its own header says so ("does not yet pin - // future builds"). Making it an input to resolution is a change to - // the resolver. Making it an ASSERTION is not, and it is the half - // that reproducibility actually needs: a release build, a CI job or - // an audit can demand that what resolved today is what was recorded, - // and find out when it is not. - // - // THE FAILURE NAMES THE DIFFERENCE. "The lock is out of date" is - // true and useless; which package moved, from which version to - // which, is what the reader does something about. - if (mcpp::platform::env::get("MCPP_LOCKED").value_or("") == "1") { - auto prior = mcpp::lockfile::load(lockPath); - if (!prior) { - return std::unexpected(std::format( - "--locked was given and there is no readable mcpp.lock at {}\n" - " Run the same command without --locked once to record " - "this resolution, then commit mcpp.lock.", - lockPath.string())); - } - auto key = [](const mcpp::lockfile::LockedPackage& p) { - return p.namespace_.empty() ? p.name - : p.namespace_ + "." + p.name; - }; - std::map was, now; - for (auto const& p : prior->packages) was[key(p)] = p.version; - for (auto const& p : lock.packages) now[key(p)] = p.version; - std::vector drift; - for (auto const& [k, v] : now) { - auto it = was.find(k); - if (it == was.end()) drift.push_back(k + " " + v + " (not in the lock)"); - else if (it->second != v) drift.push_back(k + " " + it->second + " -> " + v); - } - for (auto const& [k, v] : was) - if (!now.contains(k)) drift.push_back(k + " " + v + " (no longer resolved)"); - if (!drift.empty()) { - std::string msg = "--locked was given and this resolution " - "differs from mcpp.lock:"; - for (auto const& d : drift) msg += "\n " + d; - msg += "\n Re-run without --locked to update the lock, " - "or pin the dependency that moved."; - return std::unexpected(msg); - } - } - (void)mcpp::lockfile::write(lock, lockPath); - } - - // Same data, second consumer: the "Compiling v" banner. - // It reads this rather than re-deriving from the manifest, so the banner - // and the lock cannot disagree about what was built. - for (auto const& [key, rec] : state.resolved) { - if (rec.source != "version" || rec.version.empty()) continue; - ctx.resolvedVersions[lock_name_for(key)] = rec.version; - } - } - +static std::expected step13_runtime_provider_overrides(PrepareState& state, BuildContext& ctx) { // Apply [runtime.] provider = "" overrides. Canonical // identity wins; the old short spelling is accepted only when it denotes // exactly one provider. A same-short-name collision is never guessed. @@ -2055,7 +1999,10 @@ std::expected phase13_finish(PrepareState& state) { return pr.capability.starts_with(capKey) && pr.provider == selected; }); } + return {}; +} +static std::expected step13_abi_enforcement(PrepareState& state, BuildContext& ctx) { // Capability-driven ABI enforcement, dimensional (see src/toolchain/abi.cppm // and .agents/docs/2026-06-27-abi-compat-model-single-pr-design.md). Each // dependency may constrain specific toolchain dimensions via `abi:` @@ -2090,235 +2037,10 @@ std::expected phase13_finish(PrepareState& state) { mm.need)); } } + return {}; +} - // Per-build resolution manifest: the durable, provider-neutral facts that - // `mcpp why runtime` interprets without resolving again or probing the - // current host. The post-link validator replaces `validation.pending` - // with the exact artifact verdict produced at the link seam. - { - const std::string tcAbi = - ctx.tc.targetTriple.find("musl") != std::string::npos ? "musl" - : ctx.tc.stdlibId == "libc++" ? "libc++" - : ctx.tc.compiler == mcpp::toolchain::CompilerId::MSVC ? "msvc" - : "glibc"; - auto package_json = [](const mcpp::manifest::PackageId& id) { - return nlohmann::json{ - {"canonical", id.canonical()}, - {"namespace", id.namespace_}, - {"name", id.name}, - {"version", id.version}, - {"source", id.sourceProvenance}, - }; - }; - auto path_array = [](auto const& paths) { - nlohmann::json values = nlohmann::json::array(); - for (auto const& path : paths) - values.push_back(path.lexically_normal().generic_string()); - return values; - }; - nlohmann::json j; - j["schema_version"] = 2; - j["toolchain"] = { - {"spec", ctx.tc.label()}, {"abi", tcAbi}, - {"triple", ctx.tc.targetTriple}, {"stdlib", ctx.tc.stdlibId}, - }; - nlohmann::json dirs = nlohmann::json::array(); - for (auto& d : ctx.plan.runtimeLibraryDirs) dirs.push_back(d.string()); - nlohmann::json legacyCaps = nlohmann::json::array(); - nlohmann::json providers = nlohmann::json::array(); - for (auto& [cap, prov] : ctx.plan.runtimeProviders) - { - legacyCaps.push_back({{"capability", cap}, - {"provider", prov.canonical()}}); - providers.push_back({{"capability", cap}, - {"provider", package_json(prov)}}); - } - nlohmann::json requirements = nlohmann::json::array(); - for (auto const& requirement : ctx.plan.runtimeRequirements) { - requirements.push_back({ - {"kind", requirement.kind}, - {"value", requirement.value}, - {"phase", requirement.phase}, - {"requester", package_json(requirement.requester)}, - {"required", requirement.required}, - }); - } - nlohmann::json artifacts = nlohmann::json::array(); - for (auto const& artifact : ctx.plan.runtimeArtifacts) { - artifacts.push_back({ - {"role", artifact.role}, - {"provider", package_json(artifact.provider)}, - {"path", artifact.path.lexically_normal().generic_string()}, - {"provenance", artifact.provenance}, - {"abi", artifact.abi}, - {"digest", artifact.digest}, - {"host_fingerprint", artifact.hostFingerprint}, - // A requirement must land on a THING, and the thing must be - // the one that was declared. mcpp already enforces this for - // the private libc; recording it per artifact makes a stale - // binding visible instead of leaving `providers:` naming - // something nobody checked. - {"identity", std::string( - mcpp::build::runtime_validation::to_string( - mcpp::build::runtime_validation - ::artifact_identity_verdict(artifact)))}, - }); - } - nlohmann::json binding = nlohmann::json::parse( - mcpp::platform::runtime::serialize_runtime_binding( - ctx.plan.runtimeBinding), nullptr, false); - if (binding.is_discarded()) binding = nlohmann::json::object(); - - // ASKED OF THE PARSED TRIPLE, with the substring test kept only for a - // spelling `parse` rejects. This field is the SECOND copy of a - // derivation `mcpp.build.dist::format_for` already owns, and it had - // the same defect: mcpp's canonical `aarch64-macos` contains neither - // "apple" nor "darwin", so an explicit `--target aarch64-macos` - // recorded `"elf"` while the native build on the same machine recorded - // `"macho"` -- one report contradicting the other about one machine. - std::string format = "elf"; - if (auto t = mcpp::toolchain::triple::parse(ctx.tc.targetTriple)) { - format = std::string(mcpp::toolchain::triple::to_string(t->object_format())); - std::ranges::transform(format, format.begin(), - [](unsigned char c) { return std::tolower(c); }); - if (format == "mach-o") format = "macho"; - } else { - auto triple = ctx.tc.targetTriple; - std::ranges::transform(triple, triple.begin(), - [](unsigned char c) { return std::tolower(c); }); - const bool pe = triple.find("windows") != std::string::npos - || triple.find("mingw") != std::string::npos; - const bool macho = triple.find("darwin") != std::string::npos - || triple.find("apple") != std::string::npos; - format = pe ? "pe" : macho ? "macho" : "elf"; - } - // The ORDERED run-time search closure with provenance. Order is - // semantics here, not presentation: it is what the loader will walk, - // and the mutable SubOS farm sitting last is the invariant that keeps - // libc resolving from the pinned payload. Recorded so "why does my GL - // program find its driver" is answerable without readelf, and so a - // regression in the ordering is visible to CI and to `mcpp why`. - nlohmann::json closure = nlohmann::json::array(); - for (auto const& dir : ctx.plan.runtimeSearch) { - closure.push_back({ - {"path", dir.path.generic_string()}, - {"origin", std::string( - mcpp::platform::search::to_string(dir.origin))}, - {"machine_local", - mcpp::platform::search::is_machine_local(dir.origin)}, - }); - } - nlohmann::json search = { - {"format", format}, - {"link_library", format == "pe" ? "libpath" : "library_path"}, - {"transitive_needed", format == "elf" ? "rpath_link" : "none"}, - {"runtime", format == "pe" ? "deploy" - : format == "macho" ? "loader_rpath" : "runpath"}, - {"closure", closure}, - }; - // #418 — the contract each ROLE actually got, after any downgrade. - // - // `CompileFlags::contractByRole` was written and never read: a valuable - // observation with no way out of the process. Since #414 the shared - // library role can legitimately end up on a different contract from the - // binaries beside it, so "which one did my .so actually get?" is a - // question a user has, and the only answer available was to run - // `readelf` and infer. - // - // Recorded as the RESOLVED value, not the requested one — a request - // that was downgraded is exactly the case worth being able to see. - // `compute_flags` is pure in the plan; prepare does not otherwise hold - // the result, and threading it through just for this would widen a - // signature for one field. - const auto roleFlags = mcpp::build::compute_flags(ctx.plan); - nlohmann::json contracts = nlohmann::json::object(); - for (std::size_t i = 0; i < mcpp::build::dist::kRoleCount; ++i) { - contracts[std::string(mcpp::build::dist::to_string( - static_cast(i)))] = - std::string(mcpp::build::dist::to_string(roleFlags.contractByRole[i])); - } - - // #634, X: the resolved dependency graph. One entry per package, the - // root first: its identity as `runtime` records identities, every - // request that reached it with the key as written and the table that - // declared it, and for a library the link form with its reason. It is - // what `mcpp why deps` prints, and what a test of a resolution rule - // reads instead of a warning's wording. - { - nlohmann::json graphPackages = nlohmann::json::array(); - for (std::size_t i = 0; i < state.packages.size(); ++i) - graphPackages.push_back(state.graph_package_entry(i, /*forBuildProgram=*/false)); - j["graph"] = { {"packages", std::move(graphPackages)} }; - } - - j["runtime"] = { - {"cxx_runtime_by_role", contracts}, - {"library_dirs", dirs}, - {"dlopen_libs", ctx.plan.runtimeDlopenLibs}, - {"capabilities", legacyCaps}, - {"binding", binding}, - {"requirements", requirements}, - {"artifacts", artifacts}, - {"providers", providers}, - {"link_intent", { - {"libraries", ctx.plan.linkIntent.libraries}, - {"link_library_dirs", - path_array(ctx.plan.linkIntent.linkLibraryDirs)}, - {"transitive_needed_dirs", - path_array(ctx.plan.linkIntent.transitiveNeededDirs)}, - {"runtime_search_dirs", - path_array(ctx.plan.linkIntent.runtimeSearchDirs)}, - {"frameworks", ctx.plan.linkIntent.frameworks}, - {"deploy_files", path_array(ctx.plan.linkIntent.deployFiles)}, - {"deploy", [&] { - auto a = nlohmann::json::array(); - for (auto const& d : ctx.plan.linkIntent.deploy) - a.push_back({{"from", d.from.generic_string()}, - {"to", d.to}}); - return a; - }()}, - }}, - {"search", search}, - {"validation", { - {"status", format == "elf" ? "pending" : "not_exercised"}, - {"source", "post_link"}, - {"artifacts", nlohmann::json::array()}, - }}, - }; - // THE MSVC SYSROOT OF THE CLANG ROW: which toolset and SDK the build - // compiled against, and where each came from. Absent on every other - // row, so a reader can tell "not this row" from "not recorded". - if (!ctx.plan.toolchain.msvcToolsDir.empty()) { - const auto& tcr = ctx.plan.toolchain; - j["msvc_toolset"] = { - {"version", tcr.msvcToolsVersion}, - {"origin", tcr.msvcOrigin}, - {"product", tcr.msvcProduct}, - {"root", tcr.msvcToolsDir.generic_string()}, - }; - j["windows_sdk"] = { - {"version", tcr.windowsSdkVersion}, - {"root", tcr.windowsSdkRoot.generic_string()}, - }; - } - std::error_code ec; - std::filesystem::create_directories(ctx.plan.outputDir, ec); - auto path = ctx.plan.outputDir / "resolution.json"; - auto tmp = path; - tmp += ".tmp"; - if (std::ofstream js(tmp); js) { - js << j.dump(2) << "\n"; - js.close(); - std::filesystem::rename(tmp, path, ec); - if (ec) { - ec.clear(); - std::filesystem::remove(path, ec); - ec.clear(); - std::filesystem::rename(tmp, path, ec); - } - } - } - +static std::expected step13_empty_link_check(PrepareState& state, BuildContext& ctx) { // ── A link unit with no inputs is not a build (mcpp#533) ──────────────── // // Checked HERE, last, because objects arrive from three places and each @@ -2365,6 +2087,44 @@ std::expected phase13_finish(PrepareState& state) { lu.targetName, kindName, lu.output.generic_string(), lu.targetName)); } + return {}; +} + +std::expected phase13_finish(PrepareState& state) { + BuildContext ctx; + ctx.strict = state.overrides.strict; + ctx.manifest = *state.m; + ctx.tc = *state.tc; + ctx.fp = state.fp; + ctx.runtimeSelection = state.runtimeSelection; + ctx.runtimeBinding = state.runtimeBindingSnapshot; + ctx.profile = state.effectiveProfile; + ctx.activeFeatureRequest = state.overrides.features; + ctx.compilerChoice = { std::string(tc_origin_name(state.tcOrigin)), + state.graphCompilerRequiredBy, + state.graphCompilerReplaced.empty() ? state.pinReplacedDefault + : state.graphCompilerReplaced }; + ctx.cacheMode = state.cacheMode; + ctx.projectRoot= *state.root; + ctx.outputDir = target_dir(*state.tc, state.fp, state.workRoot); + + if (auto r = step13_source_packages(state, ctx); !r) return std::unexpected(r.error()); + if (auto r = step13_runner_and_xlings(state, ctx); !r) return std::unexpected(r.error()); + if (auto r = step13_prebuilt_check(state, ctx); !r) return std::unexpected(r.error()); + if (auto r = step13_link_forms(state, ctx); !r) return std::unexpected(r.error()); + if (auto r = step13_make_plan(state, ctx); !r) return std::unexpected(r.error()); + if (auto r = step13_cxx_private_runtime(state, ctx); !r) return std::unexpected(r.error()); + if (auto r = step13_cxx_process_runtime(state, ctx); !r) return std::unexpected(r.error()); + step13_graph_and_schedule(state, ctx); + if (auto r = step13_build_graph_actions(state, ctx); !r) return std::unexpected(r.error()); + if (auto r = step13_assembly_units(state, ctx); !r) return std::unexpected(r.error()); + if (auto r = step13_windows_resources(state, ctx); !r) return std::unexpected(r.error()); + if (auto r = step13_dependency_cache(state, ctx); !r) return std::unexpected(r.error()); + if (auto r = step13_lockfile(state, ctx); !r) return std::unexpected(r.error()); + if (auto r = step13_runtime_provider_overrides(state, ctx); !r) return std::unexpected(r.error()); + if (auto r = step13_abi_enforcement(state, ctx); !r) return std::unexpected(r.error()); + step13_resolution_json(state, ctx); + if (auto r = step13_empty_link_check(state, ctx); !r) return std::unexpected(r.error()); ctx.planNotes = std::move(state.planNotes); return ctx; diff --git a/src/build/prepare/records.cpp b/src/build/prepare/records.cpp new file mode 100644 index 000000000..b4e77baeb --- /dev/null +++ b/src/build/prepare/records.cpp @@ -0,0 +1,464 @@ +// records.cpp -- P13, the records half: mcpp.lock and resolution.json. + +module mcpp.build.prepare; +import :state; + +import mcpp.build.prepare_inputs; + +import std; +import mcpp.diag; +import mcpp.build.stage; +import mcpp.build.refusal; +import mcpp.build.version_floor; +import mcpp.home; +import mcpp.platform.axis; +import mcpp.libs.json; +import mcpp.log; +import mcpp.manifest; +import mcpp.source_kind; +import mcpp.toolchain.clang; +import mcpp.toolchain.hostflags; // the compile-token producer the package std module reuses +import mcpp.toolchain.cppfly; +import mcpp.toolchain.detect; +import mcpp.toolchain.dialect; +import mcpp.toolchain.model; // is_msvc_target — the MSVC-ABI default (#718) +import mcpp.toolchain.fingerprint; +import mcpp.toolchain.registry; +import mcpp.toolchain.linkmodel; +// For `resolve_version_match` / `list_installed_versions`: a bare compiler +// family named by the dependency graph resolves to a concrete version through +// exactly the path `mcpp toolchain default ` uses. +import mcpp.toolchain.lifecycle; +import mcpp.toolchain.stdmod; +import mcpp.toolchain.post_install; +import mcpp.toolchain.abi; +import mcpp.toolchain.triple; +import mcpp.build.linkage_form; // #519 — which form each dependency takes +import mcpp.build.plan; +import mcpp.build.schedule.policy; +import mcpp.build.flags; // compute_flags — the per-role contracts (#418) +import mcpp.build.distribution; // dist::Role / dist::Contract to_string +import mcpp.platform.capacity; // the host fallback handed to schedule::decide +import mcpp.build.graph_shape; // #407: the graph says which mode wrote it +import mcpp.build.runtime_validation; // declared artifact -> identity verdict +import mcpp.build.cache_key; +import mcpp.pack.abi_tag; // the tag a prebuilt dependency is checked against +import mcpp.pack.prebuilt; // …and the check itself +import mcpp.pack.stage_tree; // where `${mcpp.stage_dir}` points, and its manifest +import mcpp.build.build_program; +import mcpp.build.resources; // #365 Windows resources: synthesise / scan / find rc +import mcpp.build.backend; // BuildOptions for the tool sub-build +import mcpp.build.ninja; // make_ninja_backend — driving that sub-build +import mcpp.lockfile; +import mcpp.config; +import mcpp.xlings; +import mcpp.runtime.binding; +import mcpp.platform.runtime_search; +import mcpp.toolchain.post_install; +import mcpp.platform; +import mcpp.build.runner_lookup; +import mcpp.fetcher; +import mcpp.fetcher.progress; +import mcpp.pm.resolver; +import mcpp.pm.index_spec; +import mcpp.pm.index_contract; +import mcpp.pm.index_route; +import mcpp.pm.index_refresh; +import mcpp.pm.mangle; +import mcpp.pm.compat; +import mcpp.pm.dep_spec; +import mcpp.pm.dependency_selector; +import mcpp.pm.lock_io; +import mcpp.version_req; +import mcpp.ui; +import mcpp.log; +import mcpp.bmi_cache; + +namespace mcpp::build { + +// The two files a build writes about itself, cut out of plan.cpp at a step +// boundary (check_file_lengths.sh). The statements are the ones phase13_finish +// runs, called from it in the same order, and are declared in the `:state` +// partition (state.cppm) as the phases are. + +std::expected step13_lockfile(PrepareState& state, BuildContext& ctx) { + // Write/update mcpp.lock for any version-based deps that succeeded. + // Path deps are intentionally NOT locked — their source is local filesystem. + // + // mcpp#363: the version entries come from `resolved` — what the walk + // actually picked — not from `m->dependencies`, which still holds the + // constraint the user wrote and only covers DIRECT deps. Reading the input + // instead of the output made the lock record `^1.92.8` (a range locks + // nothing) and omit the transitive graph entirely. Git entries deliberately + // stay on `m->dependencies`: their lock line is read back as a resolution + // anchor (#329), keyed by the root manifest's map key, and that contract is + // unchanged here. + { + mcpp::lockfile::Lockfile lock; + lock.schemaVersion = 2; + + // The lock key for a dep the ROOT declares is the map key it declared + // it under (`compat.imgui`, `gtest`) — that is the key #329's git anchor + // lookup uses, and changing it would silently unpin every branch dep. + // A dep reached only transitively has no such key, so it is written + // under its fully-qualified identity. + auto lock_name_for = [&](const ResolvedKey& k) -> std::string { + for (auto const& [n, s] : state.m->dependencies) { + const std::string sn = s.shortName.empty() ? n : s.shortName; + if (s.namespace_ == k.ns && sn == k.shortName) return n; + } + return mcpp::pm::compat::qualified_name(k.ns, k.shortName); + }; + + // Lock custom index shas from manifest [indices] section. + for (auto const& [idxName, spec] : state.m->indices) { + if (spec.is_local() || spec.is_builtin()) continue; + mcpp::lockfile::LockedIndex li; + li.name = idxName; + li.url = spec.url; + li.rev = spec.rev; // may be empty if not yet resolved + lock.indices.push_back(std::move(li)); + } + + // Git deps: root-declared only, unchanged (see the note above). + for (auto const& [name, spec] : state.m->dependencies) { + if (!spec.isGit()) continue; + mcpp::lockfile::LockedPackage lp; + lp.name = name; + lp.version = spec.gitRev; + auto gitIt = state.root_git_lock_identities.find(name); + if (gitIt == state.root_git_lock_identities.end()) { + lp.source = std::format("git+{}#{}={}", + spec.git, spec.gitRefKind, spec.gitRev); + lp.hash = "fnv1a:" + mcpp::toolchain::hash_string(lp.source); + } else { + lp.source = gitIt->second.source; + lp.hash = gitIt->second.hash; + } + lock.packages.push_back(std::move(lp)); + } + + // Version deps: the whole resolved graph, at the versions actually + // chosen. `resolved` is an ordered map, so the file is deterministic. + for (auto const& [key, rec] : state.resolved) { + if (rec.source != "version") continue; // path / git handled elsewhere + if (rec.version.empty()) continue; + // See ResolvedRecord::devOnly: `mcpp test` resolves dev-deps and + // `mcpp build` does not, so writing them would make the file depend + // on which command ran last. + if (rec.devOnly) continue; + mcpp::lockfile::LockedPackage lp; + lp.name = lock_name_for(key); + lp.namespace_ = key.ns; + lp.version = rec.version; + // Use the namespace and resolved version as the source identifier. + // For custom indices, include the index name for traceability. + auto sourceIndex = lp.namespace_.empty() + ? std::string(mcpp::pm::kDefaultNamespace) + : lp.namespace_; + lp.source = std::format("index+{}@{}", sourceIndex, lp.version); + // Use a deterministic hash based on namespace + name + version. + // A future PR can replace this with a real content hash from the + // xpkg.lua's declared sha256 or from the install plan. + // + // NOT `std::hash`: its output is implementation-defined + // (MSVC FNV-1a, libstdc++/libc++ MurmurHash), so the same dependency + // used to hash differently on Windows and Linux while the `fnv1a:` + // prefix claimed otherwise. `index_package_digest` is FNV-1a on + // every host. + lp.hash = mcpp::pm::index_package_digest(sourceIndex, lp.name, lp.version); + lock.packages.push_back(std::move(lp)); + } + if (!lock.packages.empty() || !lock.indices.empty()) { + auto lockPath = state.workRoot / "mcpp.lock"; + // `--locked` ASSERTS THAT THIS RESOLUTION IS THE RECORDED ONE. + // + // The file has always been written after the walk and never read + // back as a constraint; its own header says so ("does not yet pin + // future builds"). Making it an input to resolution is a change to + // the resolver. Making it an ASSERTION is not, and it is the half + // that reproducibility actually needs: a release build, a CI job or + // an audit can demand that what resolved today is what was recorded, + // and find out when it is not. + // + // THE FAILURE NAMES THE DIFFERENCE. "The lock is out of date" is + // true and useless; which package moved, from which version to + // which, is what the reader does something about. + if (mcpp::platform::env::get("MCPP_LOCKED").value_or("") == "1") { + auto prior = mcpp::lockfile::load(lockPath); + if (!prior) { + return std::unexpected(std::format( + "--locked was given and there is no readable mcpp.lock at {}\n" + " Run the same command without --locked once to record " + "this resolution, then commit mcpp.lock.", + lockPath.string())); + } + auto key = [](const mcpp::lockfile::LockedPackage& p) { + return p.namespace_.empty() ? p.name + : p.namespace_ + "." + p.name; + }; + std::map was, now; + for (auto const& p : prior->packages) was[key(p)] = p.version; + for (auto const& p : lock.packages) now[key(p)] = p.version; + std::vector drift; + for (auto const& [k, v] : now) { + auto it = was.find(k); + if (it == was.end()) drift.push_back(k + " " + v + " (not in the lock)"); + else if (it->second != v) drift.push_back(k + " " + it->second + " -> " + v); + } + for (auto const& [k, v] : was) + if (!now.contains(k)) drift.push_back(k + " " + v + " (no longer resolved)"); + if (!drift.empty()) { + std::string msg = "--locked was given and this resolution " + "differs from mcpp.lock:"; + for (auto const& d : drift) msg += "\n " + d; + msg += "\n Re-run without --locked to update the lock, " + "or pin the dependency that moved."; + return std::unexpected(msg); + } + } + (void)mcpp::lockfile::write(lock, lockPath); + } + + // Same data, second consumer: the "Compiling v" banner. + // It reads this rather than re-deriving from the manifest, so the banner + // and the lock cannot disagree about what was built. + for (auto const& [key, rec] : state.resolved) { + if (rec.source != "version" || rec.version.empty()) continue; + ctx.resolvedVersions[lock_name_for(key)] = rec.version; + } + } + return {}; +} + +void step13_resolution_json(PrepareState& state, BuildContext& ctx) { + // Per-build resolution manifest: the durable, provider-neutral facts that + // `mcpp why runtime` interprets without resolving again or probing the + // current host. The post-link validator replaces `validation.pending` + // with the exact artifact verdict produced at the link seam. + { + const std::string tcAbi = + ctx.tc.targetTriple.find("musl") != std::string::npos ? "musl" + : ctx.tc.stdlibId == "libc++" ? "libc++" + : ctx.tc.compiler == mcpp::toolchain::CompilerId::MSVC ? "msvc" + : "glibc"; + auto package_json = [](const mcpp::manifest::PackageId& id) { + return nlohmann::json{ + {"canonical", id.canonical()}, + {"namespace", id.namespace_}, + {"name", id.name}, + {"version", id.version}, + {"source", id.sourceProvenance}, + }; + }; + auto path_array = [](auto const& paths) { + nlohmann::json values = nlohmann::json::array(); + for (auto const& path : paths) + values.push_back(path.lexically_normal().generic_string()); + return values; + }; + nlohmann::json j; + j["schema_version"] = 2; + j["toolchain"] = { + {"spec", ctx.tc.label()}, {"abi", tcAbi}, + {"triple", ctx.tc.targetTriple}, {"stdlib", ctx.tc.stdlibId}, + }; + nlohmann::json dirs = nlohmann::json::array(); + for (auto& d : ctx.plan.runtimeLibraryDirs) dirs.push_back(d.string()); + nlohmann::json legacyCaps = nlohmann::json::array(); + nlohmann::json providers = nlohmann::json::array(); + for (auto& [cap, prov] : ctx.plan.runtimeProviders) + { + legacyCaps.push_back({{"capability", cap}, + {"provider", prov.canonical()}}); + providers.push_back({{"capability", cap}, + {"provider", package_json(prov)}}); + } + nlohmann::json requirements = nlohmann::json::array(); + for (auto const& requirement : ctx.plan.runtimeRequirements) { + requirements.push_back({ + {"kind", requirement.kind}, + {"value", requirement.value}, + {"phase", requirement.phase}, + {"requester", package_json(requirement.requester)}, + {"required", requirement.required}, + }); + } + nlohmann::json artifacts = nlohmann::json::array(); + for (auto const& artifact : ctx.plan.runtimeArtifacts) { + artifacts.push_back({ + {"role", artifact.role}, + {"provider", package_json(artifact.provider)}, + {"path", artifact.path.lexically_normal().generic_string()}, + {"provenance", artifact.provenance}, + {"abi", artifact.abi}, + {"digest", artifact.digest}, + {"host_fingerprint", artifact.hostFingerprint}, + // A requirement must land on a THING, and the thing must be + // the one that was declared. mcpp already enforces this for + // the private libc; recording it per artifact makes a stale + // binding visible instead of leaving `providers:` naming + // something nobody checked. + {"identity", std::string( + mcpp::build::runtime_validation::to_string( + mcpp::build::runtime_validation + ::artifact_identity_verdict(artifact)))}, + }); + } + nlohmann::json binding = nlohmann::json::parse( + mcpp::platform::runtime::serialize_runtime_binding( + ctx.plan.runtimeBinding), nullptr, false); + if (binding.is_discarded()) binding = nlohmann::json::object(); + + // ASKED OF THE PARSED TRIPLE, with the substring test kept only for a + // spelling `parse` rejects. This field is the SECOND copy of a + // derivation `mcpp.build.dist::format_for` already owns, and it had + // the same defect: mcpp's canonical `aarch64-macos` contains neither + // "apple" nor "darwin", so an explicit `--target aarch64-macos` + // recorded `"elf"` while the native build on the same machine recorded + // `"macho"` -- one report contradicting the other about one machine. + std::string format = "elf"; + if (auto t = mcpp::toolchain::triple::parse(ctx.tc.targetTriple)) { + format = std::string(mcpp::toolchain::triple::to_string(t->object_format())); + std::ranges::transform(format, format.begin(), + [](unsigned char c) { return std::tolower(c); }); + if (format == "mach-o") format = "macho"; + } else { + auto triple = ctx.tc.targetTriple; + std::ranges::transform(triple, triple.begin(), + [](unsigned char c) { return std::tolower(c); }); + const bool pe = triple.find("windows") != std::string::npos + || triple.find("mingw") != std::string::npos; + const bool macho = triple.find("darwin") != std::string::npos + || triple.find("apple") != std::string::npos; + format = pe ? "pe" : macho ? "macho" : "elf"; + } + // The ORDERED run-time search closure with provenance. Order is + // semantics here, not presentation: it is what the loader will walk, + // and the mutable SubOS farm sitting last is the invariant that keeps + // libc resolving from the pinned payload. Recorded so "why does my GL + // program find its driver" is answerable without readelf, and so a + // regression in the ordering is visible to CI and to `mcpp why`. + nlohmann::json closure = nlohmann::json::array(); + for (auto const& dir : ctx.plan.runtimeSearch) { + closure.push_back({ + {"path", dir.path.generic_string()}, + {"origin", std::string( + mcpp::platform::search::to_string(dir.origin))}, + {"machine_local", + mcpp::platform::search::is_machine_local(dir.origin)}, + }); + } + nlohmann::json search = { + {"format", format}, + {"link_library", format == "pe" ? "libpath" : "library_path"}, + {"transitive_needed", format == "elf" ? "rpath_link" : "none"}, + {"runtime", format == "pe" ? "deploy" + : format == "macho" ? "loader_rpath" : "runpath"}, + {"closure", closure}, + }; + // #418 — the contract each ROLE actually got, after any downgrade. + // + // `CompileFlags::contractByRole` was written and never read: a valuable + // observation with no way out of the process. Since #414 the shared + // library role can legitimately end up on a different contract from the + // binaries beside it, so "which one did my .so actually get?" is a + // question a user has, and the only answer available was to run + // `readelf` and infer. + // + // Recorded as the RESOLVED value, not the requested one — a request + // that was downgraded is exactly the case worth being able to see. + // `compute_flags` is pure in the plan; prepare does not otherwise hold + // the result, and threading it through just for this would widen a + // signature for one field. + const auto roleFlags = mcpp::build::compute_flags(ctx.plan); + nlohmann::json contracts = nlohmann::json::object(); + for (std::size_t i = 0; i < mcpp::build::dist::kRoleCount; ++i) { + contracts[std::string(mcpp::build::dist::to_string( + static_cast(i)))] = + std::string(mcpp::build::dist::to_string(roleFlags.contractByRole[i])); + } + + // #634, X: the resolved dependency graph. One entry per package, the + // root first: its identity as `runtime` records identities, every + // request that reached it with the key as written and the table that + // declared it, and for a library the link form with its reason. It is + // what `mcpp why deps` prints, and what a test of a resolution rule + // reads instead of a warning's wording. + { + nlohmann::json graphPackages = nlohmann::json::array(); + for (std::size_t i = 0; i < state.packages.size(); ++i) + graphPackages.push_back(state.graph_package_entry(i, /*forBuildProgram=*/false)); + j["graph"] = { {"packages", std::move(graphPackages)} }; + } + + j["runtime"] = { + {"cxx_runtime_by_role", contracts}, + {"library_dirs", dirs}, + {"dlopen_libs", ctx.plan.runtimeDlopenLibs}, + {"capabilities", legacyCaps}, + {"binding", binding}, + {"requirements", requirements}, + {"artifacts", artifacts}, + {"providers", providers}, + {"link_intent", { + {"libraries", ctx.plan.linkIntent.libraries}, + {"link_library_dirs", + path_array(ctx.plan.linkIntent.linkLibraryDirs)}, + {"transitive_needed_dirs", + path_array(ctx.plan.linkIntent.transitiveNeededDirs)}, + {"runtime_search_dirs", + path_array(ctx.plan.linkIntent.runtimeSearchDirs)}, + {"frameworks", ctx.plan.linkIntent.frameworks}, + {"deploy_files", path_array(ctx.plan.linkIntent.deployFiles)}, + {"deploy", [&] { + auto a = nlohmann::json::array(); + for (auto const& d : ctx.plan.linkIntent.deploy) + a.push_back({{"from", d.from.generic_string()}, + {"to", d.to}}); + return a; + }()}, + }}, + {"search", search}, + {"validation", { + {"status", format == "elf" ? "pending" : "not_exercised"}, + {"source", "post_link"}, + {"artifacts", nlohmann::json::array()}, + }}, + }; + // THE MSVC SYSROOT OF THE CLANG ROW: which toolset and SDK the build + // compiled against, and where each came from. Absent on every other + // row, so a reader can tell "not this row" from "not recorded". + if (!ctx.plan.toolchain.msvcToolsDir.empty()) { + const auto& tcr = ctx.plan.toolchain; + j["msvc_toolset"] = { + {"version", tcr.msvcToolsVersion}, + {"origin", tcr.msvcOrigin}, + {"product", tcr.msvcProduct}, + {"root", tcr.msvcToolsDir.generic_string()}, + }; + j["windows_sdk"] = { + {"version", tcr.windowsSdkVersion}, + {"root", tcr.windowsSdkRoot.generic_string()}, + }; + } + std::error_code ec; + std::filesystem::create_directories(ctx.plan.outputDir, ec); + auto path = ctx.plan.outputDir / "resolution.json"; + auto tmp = path; + tmp += ".tmp"; + if (std::ofstream js(tmp); js) { + js << j.dump(2) << "\n"; + js.close(); + std::filesystem::rename(tmp, path, ec); + if (ec) { + ec.clear(); + std::filesystem::remove(path, ec); + ec.clear(); + std::filesystem::rename(tmp, path, ec); + } + } + } +} + +} // namespace mcpp::build diff --git a/src/build/prepare/scan.cpp b/src/build/prepare/scan.cpp index 88be45488..dfee5da34 100644 --- a/src/build/prepare/scan.cpp +++ b/src/build/prepare/scan.cpp @@ -22,6 +22,7 @@ import mcpp.toolchain.hostflags; // the compile-token producer the package std import mcpp.toolchain.cppfly; import mcpp.toolchain.detect; import mcpp.toolchain.dialect; +import mcpp.toolchain.model; // is_msvc_target — the MSVC-ABI default (#718) import mcpp.toolchain.fingerprint; import mcpp.toolchain.registry; import mcpp.toolchain.linkmodel; @@ -50,7 +51,11 @@ import mcpp.log; namespace mcpp::build { -std::expected phase11_scan(PrepareState& state) { +// Scans this graph's sources and validates the result, returning whether the +// graph (or one of its extra targets) imports `std` -- the single value +// several of the steps below need, computed here because it depends on the +// scan this step performs. +static std::expected step11_scan_sources(PrepareState& state) { // mcpp#225 (E2): observability marker for the source-discovery phase — // `mcpp run`'s fast path (build_run_target/try_fast_run in execute.cppm) @@ -105,8 +110,11 @@ std::expected phase11_scan(PrepareState& state) { return std::unexpected(msg); } - bool needsStdModule = graph_or_targets_import_std(state.scan.graph, *state.m, *state.root); + return graph_or_targets_import_std(state.scan.graph, *state.m, *state.root); +} +static std::expected +step11_dependency_standard_scope_check(PrepareState& state) { // A DEPENDENCY THAT DECLARED A HIGHER STANDARD THAN THE GRAPH IS BUILT AT. // // A C++ module graph has ONE standard — cross-level BMIs are hard @@ -133,42 +141,44 @@ std::expected phase11_scan(PrepareState& state) { // declaring c++26 compiles perfectly well at c++23 whenever it happens not // to use a C++26 construct, and that is a working configuration today for // anyone who wrote the key aspirationally. `--strict` promotes it. - { - const auto graphLevel = state.m->cppStandard.level; - for (std::size_t i = 1; i < state.packages.size(); ++i) { - auto const& pkg = state.packages[i]; - if (!pkg.manifest.package.standardDeclared) continue; - // A C++-layer provider's declaration IS applied, to every unit of - // it that neither provides nor imports a module (`make_plan`), so - // "is not applied" would be false for exactly the package whose - // sources need the level. Its module units stay at the graph's - // level, as every module unit does. - if (mcpp::manifest::cxx_layer_implementation_standard(pkg.manifest)) - continue; - // The scope gate. A package whose root is under a store directory - // arrived from an index and its declaration was written by a - // descriptor generator, not by the person reading this diagnostic. - if (mcpp::build::path_is_under_any(pkg.root, state.storeRoots)) - continue; - auto declared = mcpp::manifest::normalize_cpp_standard( - pkg.manifest.package.standard); - if (!declared || declared->level <= graphLevel) continue; - mcpp::diag::degraded( - "build/standard", - std::format("dependency `{}` declares standard = \"{}\", and " - "this graph is built at {}", - pkg.manifest.package.name, - declared->canonical, state.m->cppStandard.canonical), - "a C++ module graph has one standard, so the dependency's " - "declaration is not applied and its sources are compiled at the " - "graph's level", - std::format( - "raise the consumer's standard to \"{}\", or declare it " - "once for every member:\n\n [workspace.package]\n " - "standard = \"{}\"", declared->canonical, declared->canonical)); - } + const auto graphLevel = state.m->cppStandard.level; + for (std::size_t i = 1; i < state.packages.size(); ++i) { + auto const& pkg = state.packages[i]; + if (!pkg.manifest.package.standardDeclared) continue; + // A C++-layer provider's declaration IS applied, to every unit of + // it that neither provides nor imports a module (`make_plan`), so + // "is not applied" would be false for exactly the package whose + // sources need the level. Its module units stay at the graph's + // level, as every module unit does. + if (mcpp::manifest::cxx_layer_implementation_standard(pkg.manifest)) + continue; + // The scope gate. A package whose root is under a store directory + // arrived from an index and its declaration was written by a + // descriptor generator, not by the person reading this diagnostic. + if (mcpp::build::path_is_under_any(pkg.root, state.storeRoots)) + continue; + auto declared = mcpp::manifest::normalize_cpp_standard( + pkg.manifest.package.standard); + if (!declared || declared->level <= graphLevel) continue; + mcpp::diag::degraded( + "build/standard", + std::format("dependency `{}` declares standard = \"{}\", and " + "this graph is built at {}", + pkg.manifest.package.name, + declared->canonical, state.m->cppStandard.canonical), + "a C++ module graph has one standard, so the dependency's " + "declaration is not applied and its sources are compiled at the " + "graph's level", + std::format( + "raise the consumer's standard to \"{}\", or declare it " + "once for every member:\n\n [workspace.package]\n " + "standard = \"{}\"", declared->canonical, declared->canonical)); } + return {}; +} +static std::expected +step11_dialect_flag_reaches_std_prebuild(PrepareState& state, bool needsStdModule) { // A DIALECT FLAG THAT REACHES EVERY TU AND NOT THE `import std` PREBUILD // IS A BUILD THAT CANNOT SUCCEED, AND MCPP KNOWS IT BEFORE COMPILING. // @@ -253,7 +263,71 @@ std::expected phase11_scan(PrepareState& state) { }())); } } + return {}; +} + +static std::expected +step11_msvc_crt_word_check(PrepareState& state) { + // A FREE-FORM CRT WORD IS ALWAYS A SECOND STATEMENT ON THE MSVC ABI (D3, + // #718). Every MSVC-ABI build now states its own CRT model, so a literal + // `/MT`/`/MD`(`d`) or `-fms-runtime-lib=*` in `[build] cxxflags` or + // `dialect_cxxflags` can never be the only voice: agreeing repeats a + // fact already resolved (warned, naming the key to write instead); + // disagreeing is refused before compiling, naming the word, the key and + // the value it corresponds to. + // + // NOT gated on `needsStdModule`: the CRT model is a link-time fact for + // every MSVC-ABI build, with or without `import std`. + // + // EVERY PACKAGE'S `cxxflags`, because each reaches its own package's + // units after the graph's flags and would compile them against another + // CRT: one image, two CRTs. `dialect_cxxflags` is the root's alone (a + // dependency's reaches no command). `cxx_runtime` and `linkage` are + // root-level keys, so a dependency's agreeing word is not warned: the key + // the warning would name cannot be written there, and the word changes + // nothing. Its contradicting word is refused like the root's. + if (mcpp::toolchain::is_msvc_target(*state.tc)) { + const bool wantsStatic = mcpp::toolchain::msvc_wants_static_crt( + state.m->buildConfig.linkage, state.m->buildConfig.cxxRuntime); + for (std::size_t i = 0; i < state.packages.size(); ++i) { + auto const& pkg = state.packages[i]; + const bool isRoot = i == 0; + auto check_words = [&](std::span list, + std::string_view key) + -> std::expected { + for (auto const& w : list) { + auto verdict = mcpp::toolchain::check_crt_word( + w, wantsStatic, key); + if (!verdict) continue; + if (verdict->contradicts) + return std::unexpected(verdict->message); + // Redundant, not degraded: the engine does exactly what + // it would have done without the flag. + if (isRoot) + mcpp::diag::warning("build/msvc-crt-word", + verdict->message); + } + return {}; + }; + const auto cxxflagsWords = + mcpp::manifest::flag_words(pkg.manifest.buildConfig.cxxflags); + const auto cxxflagsKey = isRoot + ? std::string("[build] cxxflags") + : std::format("the [build] cxxflags of dependency '{}'", + pkg.manifest.package.name); + if (auto r = check_words(cxxflagsWords, cxxflagsKey); !r) + return std::unexpected(r.error()); + if (isRoot) + if (auto r = check_words(pkg.manifest.buildConfig.dialectCxxflags, + "[build] dialect_cxxflags"); !r) + return std::unexpected(r.error()); + } + } + return {}; +} +static std::expected +step11_package_std_module_source(PrepareState& state) { // A standard library that came from a PACKAGE brings its own module // source, because the compiler cannot be asked for one it does not have. // @@ -458,7 +532,11 @@ std::expected phase11_scan(PrepareState& state) { state.tc->stdModuleFlags = flags; break; } + return {}; +} +static std::expected +step11_apple_sdk_cxx_runtime(PrepareState& state, bool needsStdModule) { // AN APPLE CROSS TARGET WITHOUT A GRAPH C++ RUNTIME LINKS THE SDK'S // libc++ (the Mach-O cell in distribution.cppm), AND THE HEADERS FOLLOW // THE RUNTIME. The payload's `std.cppm` and headers describe libc++ 22; @@ -501,7 +579,11 @@ std::expected phase11_scan(PrepareState& state) { "llvm.compiler-rt-builtins = \"22.1.8.5\" beside it)"); } } + return {}; +} +static std::expected +step11_std_module_availability_gate(PrepareState& state, bool needsStdModule) { if (needsStdModule && !state.tc->hasImportStd) { // A freestanding target reaches here for a reason the generic message // gets wrong. Nothing is missing from the toolchain — libc++'s std @@ -598,7 +680,10 @@ std::expected phase11_scan(PrepareState& state) { mcpp::manifest::cpp_standard_level_name(state.tc->importStdMinLevel), state.m->package.standard)); } + return {}; +} +static void step11_compute_fingerprint(PrepareState& state) { // Compute fingerprint (no lockfile in M1 → empty hash) mcpp::toolchain::FingerprintInputs fpi; fpi.toolchain = *state.tc; @@ -682,7 +767,10 @@ std::expected phase11_scan(PrepareState& state) { fpi.dependencyLockHash = ""; // M2 fpi.stdBmiHash = ""; // updated after stdmod build (chicken/egg ok for M1) state.fp = mcpp::toolchain::compute_fingerprint(fpi); +} +static std::expected +step11_prebuild_std_module(PrepareState& state, bool needsStdModule) { // Pre-build std module only when the source graph actually imports it. if (needsStdModule) { // The std BMI must be compiled with the SAME dialect set its @@ -690,16 +778,14 @@ std::expected phase11_scan(PrepareState& state) { // a std BMI built without it structurally lacks std::meta). Both // pieces were already in the fingerprint; this fixes the COMMAND // construction the fingerprint promised (stdFlagAndDialect above). - // #422: the CRT model reaches the std module too. Derived from the - // SAME expression the project's TUs use (flags.cppm), through the one - // helper, so the two cannot drift. A GNU dialect yields "-static" or "" - // here, and the gcc and clang std module builders do not read it, so - // their commands are unchanged; clang on the MSVC ABI is given no CRT - // model at all (see `MechanismInput::msvcCrtModelEmitted`). - const auto& stdDialect = mcpp::toolchain::dialect_for(*state.tc); - const auto stdCrt = mcpp::toolchain::msvc_crt_flag( - stdDialect, mcpp::toolchain::msvc_wants_static_crt( - state.m->buildConfig.linkage, state.m->buildConfig.cxxRuntime)); + // #422/#718: the CRT model reaches the std module too, on EVERY + // MSVC-ABI row now (cl and clang++ targeting `*-windows-msvc` alike). + // `msvc_abi_crt_word` is the SAME helper flags.cppm uses for the + // project's TUs, so the two cannot drift; it is empty off the MSVC + // ABI, where the gcc and clang std module builders leave it unread. + const auto stdCrt = mcpp::toolchain::msvc_abi_crt_word( + *state.tc, mcpp::toolchain::msvc_wants_static_crt( + state.m->buildConfig.linkage, state.m->buildConfig.cxxRuntime)); // Whether THIS build's resolved toolchain targets macOS — the same // target-not-host discriminator `min_platform_version` uses, parsed // locally because `tc` (not a `triple::Triple`) is what is in scope @@ -771,6 +857,35 @@ std::expected phase11_scan(PrepareState& state) { if (described) state.describedStdModule = std::move(*described); } } + return {}; +} + +std::expected phase11_scan(PrepareState& state) { + auto needsStdModule = step11_scan_sources(state); + if (!needsStdModule) return std::unexpected(needsStdModule.error()); + + if (auto r = step11_dependency_standard_scope_check(state); !r) + return std::unexpected(r.error()); + + if (auto r = step11_dialect_flag_reaches_std_prebuild(state, *needsStdModule); !r) + return std::unexpected(r.error()); + + if (auto r = step11_msvc_crt_word_check(state); !r) + return std::unexpected(r.error()); + + if (auto r = step11_package_std_module_source(state); !r) + return std::unexpected(r.error()); + + if (auto r = step11_apple_sdk_cxx_runtime(state, *needsStdModule); !r) + return std::unexpected(r.error()); + + if (auto r = step11_std_module_availability_gate(state, *needsStdModule); !r) + return std::unexpected(r.error()); + + step11_compute_fingerprint(state); + + if (auto r = step11_prebuild_std_module(state, *needsStdModule); !r) + return std::unexpected(r.error()); if (state.print_fingerprint) { std::println("Toolchain: {}", state.tc->label()); @@ -783,5 +898,4 @@ std::expected phase11_scan(PrepareState& state) { return {}; } - } // namespace mcpp::build diff --git a/src/build/prepare/state.cppm b/src/build/prepare/state.cppm index 3ea84182c..b242da3e1 100644 --- a/src/build/prepare/state.cppm +++ b/src/build/prepare/state.cppm @@ -462,6 +462,16 @@ struct PrepareState { std::vector> abiRequiresExceptions; std::map> capExclusive; std::map> deviceSourcesByPackage; + // Keyed like `deviceSourcesByPackage`, by `pkg.root.string()` (root + // package included: `packages[0].root == *root`). Holds a package whose + // build program failed IN THIS PASS, under `plan_only` (`emit + // build-database`) — the one case a failed program does not already end + // the whole call (SPEC-005 R5.2, #699 item 2, E3). A check whose premise + // is that program's directives must not run for such a package: with no + // directives applied, every premise reads as unmet, which is a symptom of + // the recorded `MCPP_BUILD_DATABASE_PROGRAM_FAILED`, not a second defect + // (design 2026-09-27 §4.2, mcpp#724 side finding A). + std::set programFailedPackages; std::function()> checkVersionFloors; mcpp::targetside::TargetSide resolvedTargetSide; std::optional cxxLayerProviderIndex; @@ -499,6 +509,9 @@ std::expected phase6_features_and_host_tools(PrepareState& st std::expected phase9_target_side(PrepareState& state); std::expected phase11_scan(PrepareState& state); std::expected phase13_finish(PrepareState& state); +// P13's records half (records.cpp), called by phase13_finish. +std::expected step13_lockfile(PrepareState& state, BuildContext& ctx); +void step13_resolution_json(PrepareState& state, BuildContext& ctx); // ── Helpers the phases share, defined in the files named below ───────────── @@ -561,9 +574,12 @@ std::string min_platform_version(const mcpp::manifest::Manifest& m, // fetch.cpp: git remotes, network retries, xlings addresses and their provisioning std::string git_cache_head(const std::filesystem::path& gitRoot); +// `progressLabel`, when given, draws git's `--progress` download phase as +// one bar labelled with it (W11); the command must pass `--progress`. mcpp::platform::process::RunResult run_with_network_retry( std::string_view command, - const std::function& between = {}); + const std::function& between = {}, + std::string_view progressLabel = {}); std::vector applicable_xlings_addresses(const mcpp::manifest::Manifest& man, const std::vector& activeFeatures, diff --git a/src/build/prepare/target_side.cpp b/src/build/prepare/target_side.cpp index d06b55f32..2b4b35de2 100644 --- a/src/build/prepare/target_side.cpp +++ b/src/build/prepare/target_side.cpp @@ -59,29 +59,30 @@ import mcpp.wire; // Severity, for PlanNote (#699 item 2, E3) namespace mcpp::build { -std::expected phase9_target_side(PrepareState& state) { - // ── THE TARGET SIDE, RESOLVED ONCE ─────────────────────────────────────── - // - // HERE AND NOT EARLIER, AND THAT IS THE WHOLE POINT. - // - // mcpp serves two ways of supplying a target's platform interface, C - // library and C++ runtime, and the moment each becomes knowable is - // opposite: a prebuilt directory is known before dependency resolution, a - // set of packages only after it. Until now three separate derivations ran - // at the earlier moment and guessed the later answer — the family name in - // this file, `graphTargetSide` in flags, `graphCxxRuntime` in the contract - // — and they disagreed on the case none of them was written for. Measured: - // - // ld64.lld: error: …/lib/x86_64-unknown-linux-gnu/libc++.so: - // unhandled file type - // - // for a pure C program crossed to macOS, whose graph supplies a C library - // and no C++ runtime at all. - // - // Placing the resolution after capability binding and before the root - // build.mcpp means every later consumer reads one value, and a build - // program can be told what was resolved rather than re-deriving it. - { +// STEP FUNCTIONS (mcpp#722 / T6), one per section phase9's own banners +// already named. Statements moved verbatim; two scoping braces that +// wrapped several sections at once (no matching close inside any one +// of them) are dropped as redundant, the same treatment graph.cpp and +// features.cpp needed for their own such wrappers. + +// Hoisted from a local declaration inside phase9_target_side (mcpp#722 / +// T6): `byLayer`'s element type, needed by the struct that now carries +// gather state across step boundaries. +struct TargetSideCandidate { mcpp::targetside::Provider p; bool direct; std::size_t index = 0; }; + +// The locals phase9_target_side's first section (candidate gathering) +// used to declare and every later section still reads: the PrepareState +// pattern, one level deeper, for one phase's own steps. +struct TargetSideGather { + mcpp::targetside::Inputs in; + std::map> byLayer; + std::vector requirements; +}; + +static std::expected +step9_gather_target_side_candidates(PrepareState& state) { + namespace tsd = mcpp::targetside; + TargetSideGather gather; namespace tsd = mcpp::targetside; // Scan the graph once for every layer. A package declares the layer it @@ -107,10 +108,8 @@ std::expected phase9_target_side(PrepareState& state) { // publishes an include set the WHOLE build must see (see // `targetSideUsage` below), and reaching that package by name would be // a second lookup of something already in hand. - struct Candidate { tsd::Provider p; bool direct; std::size_t index = 0; }; - std::map> byLayer; - std::vector requirements; - + using Candidate = TargetSideCandidate; + const auto& rootDeps = state.m->dependencies; auto is_direct = [&](std::string_view name) { for (auto const& [k, _] : rootDeps) { @@ -180,7 +179,7 @@ std::expected phase9_target_side(PrepareState& state) { p.hasStdModule = !pkg.manifest.stdModule.empty(); p.cAbiDecl = pkg.manifest.cAbiDecl; - auto& slot = byLayer[static_cast(decl->layer)]; + auto& slot = gather.byLayer[static_cast(decl->layer)]; // A package may carry both spellings during the transition, and // the array order is the author's, not a preference. Two entries // from the SAME package are one supplier; the current spelling @@ -212,12 +211,12 @@ std::expected phase9_target_side(PrepareState& state) { // read — as an error for the root and a warning for a // dependency — so it is skipped rather than refused twice. if (!parsed || !*parsed) continue; - requirements.push_back({ pkgId, (*parsed)->layer, + gather.requirements.push_back({ pkgId, (*parsed)->layer, (*parsed)->interfaceName }); } } - for (auto const& [layerInt, slot] : byLayer) { + for (auto const& [layerInt, slot] : gather.byLayer) { if (slot.size() < 2) continue; tsd::Conflict c; c.layer = static_cast(layerInt); @@ -230,12 +229,12 @@ std::expected phase9_target_side(PrepareState& state) { auto provider_of = [&](tsd::CapLayer want) -> std::optional { - auto it = byLayer.find(static_cast(want)); - if (it == byLayer.end() || it->second.empty()) return std::nullopt; + auto it = gather.byLayer.find(static_cast(want)); + if (it == gather.byLayer.end() || it->second.empty()) return std::nullopt; return it->second.front().p; }; - tsd::Inputs in; + auto& in = gather.in; if (state.tc) { if (auto tt = mcpp::toolchain::triple::parse(state.tc->targetTriple)) { in.llvmTriple = tt->llvm_triple( @@ -367,6 +366,13 @@ std::expected phase9_target_side(PrepareState& state) { } } + return gather; +} + +static std::expected +step9_resolve_and_realise_cabi(PrepareState& state, TargetSideGather& gather) { + namespace tsd = mcpp::targetside; + auto& in = gather.in; state.resolvedTargetSide = tsd::resolve(in); state.targetSideResolved = true; @@ -608,7 +614,12 @@ std::expected phase9_target_side(PrepareState& state) { // line is assembled after it. A std BMI built against a different C // library than its importers is what e2e 181 catches. if (state.tc) state.tc->cAbiPrebuilt = state.resolvedTargetSide.cAbi.prebuilt(); + return {}; +} +static std::expected +step9_target_side_include_broadcast(PrepareState& state, TargetSideGather& gather) { + namespace tsd = mcpp::targetside; // ── The target side's include set is a property of the BUILD ───────── // // IT WAS ALREADY COMPUTED, AND IT REACHED EXACTLY ONE TRANSLATION @@ -651,8 +662,8 @@ std::expected phase9_target_side(PrepareState& state) { std::set layerProviderIndices; auto note_layer = [&](tsd::CapLayer which, const tsd::Layer& resolved) { if (!resolved.fromGraph()) return; - auto it = byLayer.find(static_cast(which)); - if (it != byLayer.end() && !it->second.empty()) { + auto it = gather.byLayer.find(static_cast(which)); + if (it != gather.byLayer.end() && !it->second.empty()) { layerProviderIndices.insert(it->second.front().index); if (which == tsd::CapLayer::CxxAbi) state.cxxLayerProviderIndex = it->second.front().index; @@ -778,7 +789,11 @@ std::expected phase9_target_side(PrepareState& state) { state.appendUniqueFlags(p.privateBuild.asmflags, state.tc->cEnvBuiltinsTokens); } } + return {}; +} +static std::expected +step9_kernel_abi_interface_enumeration(PrepareState& state) { // INTERFACE ENUMERATION — THE RESOLUTION-TIME HALF OF THE CAPABILITY // MODEL (design 2026-09-20 §5.5; openkal SPEC 0.14 §3.3, §6.2). // @@ -800,113 +815,116 @@ std::expected phase9_target_side(PrepareState& state) { // writes `[kernel-abi]` reaches neither loop below, so this addition // changes no command line and no diagnostic for every project built // before it. - { - // THE LIST COMES FROM THE PACKAGE THAT RESOLVED AS THE LAYER, NOT - // FROM THE FIRST ONE IN THE GRAPH THAT STATED ONE. A graph may - // carry more than one candidate for a layer — a workspace member - // beside a dependency, a second implementation reached through a - // feature that did not activate — and only one of them is the - // provider this build resolved. Reading whichever came first in - // `packages` would compare a consumer's requirements against an - // implementation the build is not using, which is a wrong answer - // rather than a missing one. - std::vector providedInterfaces; - std::string providerId; - for (auto& pkg : state.packages) { - if (pkg.manifest.kernelAbiProvidesInterfaces.empty()) continue; - // `impl` is `name@version`; the name is what precedes the - // separator. A substring test would match `openkal` against - // `openkal-linux@0.15.0` and read one implementation's list - // as another's. - if (!state.resolvedTargetSide.kernelAbi.impl.empty()) { - auto const& impl = state.resolvedTargetSide.kernelAbi.impl; - const auto at = impl.find('@'); - const auto implName = at == std::string::npos - ? impl : impl.substr(0, at); - if (implName != pkg.manifest.package.name) continue; - } - providedInterfaces = pkg.manifest.kernelAbiProvidesInterfaces; - providerId = pkg.manifest.package.name; - break; + // THE LIST COMES FROM THE PACKAGE THAT RESOLVED AS THE LAYER, NOT + // FROM THE FIRST ONE IN THE GRAPH THAT STATED ONE. A graph may + // carry more than one candidate for a layer — a workspace member + // beside a dependency, a second implementation reached through a + // feature that did not activate — and only one of them is the + // provider this build resolved. Reading whichever came first in + // `packages` would compare a consumer's requirements against an + // implementation the build is not using, which is a wrong answer + // rather than a missing one. + std::vector providedInterfaces; + std::string providerId; + for (auto& pkg : state.packages) { + if (pkg.manifest.kernelAbiProvidesInterfaces.empty()) continue; + // `impl` is `name@version`; the name is what precedes the + // separator. A substring test would match `openkal` against + // `openkal-linux@0.15.0` and read one implementation's list + // as another's. + if (!state.resolvedTargetSide.kernelAbi.impl.empty()) { + auto const& impl = state.resolvedTargetSide.kernelAbi.impl; + const auto at = impl.find('@'); + const auto implName = at == std::string::npos + ? impl : impl.substr(0, at); + if (implName != pkg.manifest.package.name) continue; } - // A REQUIREMENT NOBODY ANSWERED IS SAID SO, because otherwise - // "yes" and "never asked" are the same reading. - // - // Three situations exist and two of them build: the provider - // states a list and it contains the requirement (build); it - // states a list and does not (refuse, below); it states nothing - // at all (build, and until this note, in silence). The third is - // deliberate --- `provides-interfaces` is younger than the - // implementations that exist, and a graph that has not adopted it - // must keep building --- but a consumer reading a green build - // cannot tell it from the first. One line closes that, and it - // costs nothing to a graph where the provider does declare. - std::size_t uncheckedRequirements = 0; - for (auto& pkg : state.packages) { - const auto& need = pkg.manifest.kernelAbiRequiresInterfaces; - if (need.empty()) continue; - if (providerId.empty()) { - uncheckedRequirements += need.size(); - continue; - } - auto missing = mcpp::targetside::interfaces_not_provided( - need, providedInterfaces); - if (missing.empty()) continue; - refusal::record(refusal::Code::InterfaceNotProvided); - std::string names; - for (auto const& mI : missing) { - names += "\n "; - names += mI; - } - // THE CODE IS PRINTED, THE WAY E0006 IS, BECAUSE SOMETHING - // READS THIS. A refusal that only a person can recognise - // forces every machine consumer to match prose --- and prose - // that a package's own compile error could coincidentally - // contain. The mcpp-index compatibility measurement - // distinguishes "this graph does not supply what the member - // asked for" from "the member did not build" on exactly this - // token, and that distinction decides whether a member counts - // against a compatibility figure. - // THE LABEL SAYS WHICH IMPLEMENTATION WAS RESOLVED, NOT - // "provided by". The missing names are listed immediately - // above it, and `provided by fakekernel` under `openkal.space` - // reads as the statement that fakekernel provides it --- the - // exact opposite of what this refusal is about. Read once, - // rendered, which is the only way that kind of defect is - // visible: every assertion on this message matches an - // identifier inside it, and an identifier is in the right - // place under either wording. - return std::unexpected(std::format( - "'{}' requires interfaces the resolved implementation does " - "not provide. [interface-not-provided]{}\n" - " the resolved implementation is {} ({} interface{}), " - "and none of those listed above is among them.\n" - " This is refused before anything is compiled " - "because dependency resolution is the earliest time the " - "question can be answered. Select an implementation that " - "provides them, or remove them from [kernel-abi] " - "requires-interfaces in '{}'.", - pkg.manifest.package.name, names, providerId, - providedInterfaces.size(), - providedInterfaces.size() == 1 ? "" : "s", - pkg.manifest.package.name)); + providedInterfaces = pkg.manifest.kernelAbiProvidesInterfaces; + providerId = pkg.manifest.package.name; + break; + } + // A REQUIREMENT NOBODY ANSWERED IS SAID SO, because otherwise + // "yes" and "never asked" are the same reading. + // + // Three situations exist and two of them build: the provider + // states a list and it contains the requirement (build); it + // states a list and does not (refuse, below); it states nothing + // at all (build, and until this note, in silence). The third is + // deliberate --- `provides-interfaces` is younger than the + // implementations that exist, and a graph that has not adopted it + // must keep building --- but a consumer reading a green build + // cannot tell it from the first. One line closes that, and it + // costs nothing to a graph where the provider does declare. + std::size_t uncheckedRequirements = 0; + for (auto& pkg : state.packages) { + const auto& need = pkg.manifest.kernelAbiRequiresInterfaces; + if (need.empty()) continue; + if (providerId.empty()) { + uncheckedRequirements += need.size(); + continue; } - - if (uncheckedRequirements > 0) { - // THE IMPLEMENTATION IS NAMED FROM THE RESOLVED LAYER, not - // from whichever package happened to be first: the note has - // to say WHOSE silence this is, or a reader cannot act on it. - const auto& impl = state.resolvedTargetSide.kernelAbi.impl; - mcpp::ui::info("note", std::format( - "kernel-abi interfaces: {} states none, {} requirement{} " - "unchecked", - impl.empty() ? std::string("the resolved implementation") - : impl, - uncheckedRequirements, - uncheckedRequirements == 1 ? "" : "s")); + auto missing = mcpp::targetside::interfaces_not_provided( + need, providedInterfaces); + if (missing.empty()) continue; + refusal::record(refusal::Code::InterfaceNotProvided); + std::string names; + for (auto const& mI : missing) { + names += "\n "; + names += mI; } + // THE CODE IS PRINTED, THE WAY E0006 IS, BECAUSE SOMETHING + // READS THIS. A refusal that only a person can recognise + // forces every machine consumer to match prose --- and prose + // that a package's own compile error could coincidentally + // contain. The mcpp-index compatibility measurement + // distinguishes "this graph does not supply what the member + // asked for" from "the member did not build" on exactly this + // token, and that distinction decides whether a member counts + // against a compatibility figure. + // THE LABEL SAYS WHICH IMPLEMENTATION WAS RESOLVED, NOT + // "provided by". The missing names are listed immediately + // above it, and `provided by fakekernel` under `openkal.space` + // reads as the statement that fakekernel provides it --- the + // exact opposite of what this refusal is about. Read once, + // rendered, which is the only way that kind of defect is + // visible: every assertion on this message matches an + // identifier inside it, and an identifier is in the right + // place under either wording. + return std::unexpected(std::format( + "'{}' requires interfaces the resolved implementation does " + "not provide. [interface-not-provided]{}\n" + " the resolved implementation is {} ({} interface{}), " + "and none of those listed above is among them.\n" + " This is refused before anything is compiled " + "because dependency resolution is the earliest time the " + "question can be answered. Select an implementation that " + "provides them, or remove them from [kernel-abi] " + "requires-interfaces in '{}'.", + pkg.manifest.package.name, names, providerId, + providedInterfaces.size(), + providedInterfaces.size() == 1 ? "" : "s", + pkg.manifest.package.name)); } + if (uncheckedRequirements > 0) { + // THE IMPLEMENTATION IS NAMED FROM THE RESOLVED LAYER, not + // from whichever package happened to be first: the note has + // to say WHOSE silence this is, or a reader cannot act on it. + const auto& impl = state.resolvedTargetSide.kernelAbi.impl; + mcpp::ui::info("note", std::format( + "kernel-abi interfaces: {} states none, {} requirement{} " + "unchecked", + impl.empty() ? std::string("the resolved implementation") + : impl, + uncheckedRequirements, + uncheckedRequirements == 1 ? "" : "s")); + } + return {}; +} + +static std::expected +step9_layering_and_requirement_checks(PrepareState& state, TargetSideGather& gather) { + namespace tsd = mcpp::targetside; if (auto why = tsd::check_layering(state.resolvedTargetSide)) { refusal::record(refusal::Code::LayerOrdering); return std::unexpected(*why); @@ -920,7 +938,7 @@ std::expected phase9_target_side(PrepareState& state) { // now means the project stated its own compiler, and the remedy has to // name that statement rather than a global default it is not using. if (auto why = tsd::check_requirements( - state.resolvedTargetSide, requirements, + state.resolvedTargetSide, gather.requirements, tc_origin_is_user_explicit(state.tcOrigin) ? tc_origin_name(state.tcOrigin) : std::string_view{})) { refusal::record(refusal::Code::LayerRequirement); @@ -943,7 +961,11 @@ std::expected phase9_target_side(PrepareState& state) { refusal::record(refusal::Code::HostCannotServe); return std::unexpected(state.unservedTargetDiagnosis); } + return {}; +} +static void +step9_pin_and_linkage_diagnostics(PrepareState& state) { // THE TARGET AND THE COMPILER ARE NOT BOUND TOGETHER, AND THE // TARGET ROW'S CONVENTION IS A FALLBACK RATHER THAN A RULE. // @@ -1035,7 +1057,11 @@ std::expected phase9_target_side(PrepareState& state) { "target's system comes from the dependency graph: those " "packages are compiled into this build as objects, and there " "is no shared object to link against. The artifact is static."); +} +static std::expected +step9_same_os_check_and_report(PrepareState& state) { + namespace tsd = mcpp::targetside; // Reported, and reported HERE rather than recorded in a manifest field. // // A line a project writes states an intention, and it goes stale the @@ -1122,7 +1148,11 @@ std::expected phase9_target_side(PrepareState& state) { } mcpp::ui::info("Target", tsd::format_report( state.resolvedTargetSide, reportedTargetName, mcpp::log::is_verbose())); + return {}; +} +static std::expected +step9_platform_sdk_closure_visibility(PrepareState& state) { // CLOSURE VISIBILITY — design §6. Distinct from the five-layer // report above: a platform dependency is not a LAYER (no engine // vocabulary names it, and `mcpp.targetside` — the pure, layer-only @@ -1177,8 +1207,26 @@ std::expected phase9_target_side(PrepareState& state) { : "platform dependencies", joined)); } - } + return {}; +} + +static std::expected +step9_kernel_abi_interfaces_and_requirements(PrepareState& state, TargetSideGather& gather) { + if (auto r = step9_kernel_abi_interface_enumeration(state); !r) + return std::unexpected(r.error()); + if (auto r = step9_layering_and_requirement_checks(state, gather); !r) + return std::unexpected(r.error()); + + step9_pin_and_linkage_diagnostics(state); + + if (auto r = step9_same_os_check_and_report(state); !r) + return std::unexpected(r.error()); + + return step9_platform_sdk_closure_visibility(state); +} + +static std::expected step9_layer_conditional_config(PrepareState& state) { // ── L1b: conditional sections whose predicate names a target-side layer ── // // The second half of the conditional axis, and it runs HERE for the same @@ -1255,6 +1303,10 @@ std::expected phase9_target_side(PrepareState& state) { } } + return {}; +} + +static std::expected step9_dependency_link_forms(PrepareState& state) { // ── #519: which FORM does each dependency take in this build ──────────── // // The decision itself lives in `mcpp.build.linkage_form`, which is a pure, @@ -1368,6 +1420,10 @@ std::expected phase9_target_side(PrepareState& state) { } } + return {}; +} + +static void step9_define_graph_package_entry_closure(PrepareState& state) { // ── The resolved graph, one derivation for two readers (#634 X, #647 E1) ── // // `resolution.json`'s `graph` section and the document the root build @@ -1441,6 +1497,9 @@ std::expected phase9_target_side(PrepareState& state) { return entry; }; +} + +static std::expected step9_root_build_program(PrepareState& state) { // ── L3: ROOT build.mcpp (moved after dependency resolution, design §3.1 // item 4) ──────────────────────────────────────────────────────────────── // Runs HERE — after dep resolution + feature activation (so the contract @@ -1622,6 +1681,10 @@ std::expected phase9_target_side(PrepareState& state) { state.planNotes.push_back({"MCPP_BUILD_DATABASE_PROGRAM_FAILED", bp.error(), mcpp::wire::Severity::Error, (*state.root / "build.mcpp").string()}); + // Named so the device-source check below (and anything else whose + // premise is this program's directives) can tell a package whose + // program failed apart from one that simply has no program. + state.programFailedPackages.insert(state.root->string()); } if (bp) { // THE SAME RULE THE DEPENDENCIES ARE HELD TO, WITH THE ROOT AS A PARTY. @@ -1737,7 +1800,10 @@ std::expected phase9_target_side(PrepareState& state) { state.m->runtimeConfig.linkIntent.runtimeSearchDirs.end()); } } + return {}; +} +static std::expected step9_device_sources_reach_an_action(PrepareState& state) { // ── Every device source must reach some action ───────────────────────── // // A device-kind file is the one source the engine has no compile rule for. @@ -1764,8 +1830,18 @@ std::expected phase9_target_side(PrepareState& state) { // condition an action needs anyway -- one that compiles a file it does not // declare as an input does not rerun when that file changes -- so a rule // that satisfies it is a rule that rebuilds correctly. + // + // THE PREMISE OF THIS CHECK IS THE BUILD PROGRAM'S DIRECTIVES: an action + // consuming a device source is one such directive. A package whose program + // failed in this pass (`plan_only`, above) applied none of them, so every + // device source would read as an orphan -- not a second defect, only the + // shape the first one takes here. Such a package already carries its one + // diagnostic, `MCPP_BUILD_DATABASE_PROGRAM_FAILED`; this check does not run + // for it, exactly as SPEC-005 R5.2 now states (design 2026-09-27 §4.2, + // mcpp#724 side finding A). for (std::size_t i = 0; i < state.packages.size(); ++i) { auto const& pkg = state.packages[i]; + if (state.programFailedPackages.contains(pkg.root.string())) continue; auto dit = state.deviceSourcesByPackage.find(pkg.root.string()); if (dit == state.deviceSourcesByPackage.end() || dit->second.empty()) continue; auto const& mm = (i == 0) ? *state.m : pkg.manifest; @@ -1781,6 +1857,9 @@ std::expected phase9_target_side(PrepareState& state) { orphans += " " + rel + "\n"; if (orphans.empty()) continue; std::error_code hasEc; + // The `programFailedPackages` skip above means this package's program, + // if it has one, ran and succeeded — `exists(build.mcpp)` here can no + // longer be true of a program that merely started and failed. const bool hasProgram = std::filesystem::exists(pkg.root / "build.mcpp", hasEc) || !pkg.manifest.buildConfig.ruleModules.empty(); refusal::record(refusal::Code::DeviceSourceUnconsumed); @@ -1804,6 +1883,10 @@ std::expected phase9_target_side(PrepareState& state) { " drop them from `[build] sources`.")); } + return {}; +} + +static std::expected step9_rerun_input_prepare_dir(PrepareState& state) { // ── R1.3: a re-run input inside a `prepare` directory (SPEC-007 §3) ───── // // A build program's re-run set is declared BEFORE anything is built @@ -1929,4 +2012,24 @@ std::expected phase9_target_side(PrepareState& state) { return {}; } +std::expected phase9_target_side(PrepareState& state) { + auto gather = step9_gather_target_side_candidates(state); + if (!gather) return std::unexpected(gather.error()); + if (auto r = step9_resolve_and_realise_cabi(state, *gather); !r) + return std::unexpected(r.error()); + if (auto r = step9_target_side_include_broadcast(state, *gather); !r) + return std::unexpected(r.error()); + if (auto r = step9_kernel_abi_interfaces_and_requirements(state, *gather); !r) + return std::unexpected(r.error()); + + if (auto r = step9_layer_conditional_config(state); !r) return std::unexpected(r.error()); + if (auto r = step9_dependency_link_forms(state); !r) return std::unexpected(r.error()); + step9_define_graph_package_entry_closure(state); + if (auto r = step9_root_build_program(state); !r) return std::unexpected(r.error()); + if (auto r = step9_device_sources_reach_an_action(state); !r) return std::unexpected(r.error()); + if (auto r = step9_rerun_input_prepare_dir(state); !r) return std::unexpected(r.error()); + + return {}; +} + } // namespace mcpp::build diff --git a/src/build/prepare/toolchain.cpp b/src/build/prepare/toolchain.cpp index 111164196..1ad3bd57a 100644 --- a/src/build/prepare/toolchain.cpp +++ b/src/build/prepare/toolchain.cpp @@ -53,7 +53,12 @@ import mcpp.ui; namespace mcpp::build { -std::expected phase1_toolchain_spec_and_axes(PrepareState& state) { +// STEP FUNCTIONS (mcpp#722 / T6), split at the points where phase1's +// own banners mark a new concern: the closures phase1 assigns onto +// `state` (each captures only `state`), the target/static override +// resolution, and the device axis plus the L1 conditional-section merge. + +static std::expected step1_define_early_toolchain_closures(PrepareState& state) { // ─── Toolchain resolution (docs/21) ──────────────────────────────── // // THE WHOLE CHAIN, in the order it is applied. It was documented twice, as @@ -410,18 +415,545 @@ std::expected phase1_toolchain_spec_and_axes(PrepareState& st // this row the author wrote down, so it replaces `[toolchain]` and the // global default; `--toolchain` and a consumer's decision for a host tool // are statements about THIS invocation and keep precedence over it. - auto apply_target_section = [&](const mcpp::manifest::TargetEntry& e) { - if (!e.toolchain.empty() && !state.tcFromCommandLine && !state.tcFromConsumer) { - state.tcSpec = e.toolchain; - state.tcOrigin = TcOrigin::TargetSection; + return {}; +} + +// A phase-local struct passed by reference to the steps that resolve one +// `--target` / manifest-target request -- the same pattern WorklistItemCtx +// (graph.cpp) and HostToolCtx (features.cpp) use. Each field is a local the +// original single-function body declared once (inside its +// `if (!target_triple.empty())` block) and read again in a later part of the +// same request's resolution. +struct TargetOverrideCtx { + std::string requestedSpelling; + std::optional parsed; + mcpp::toolchain::triple::RequestResolution req; + // The `[target.]` section this request matched, or null. A + // pointer rather than the map iterator the single-function version held, + // since iterator validity is not this struct's business to reason about + // and the callers only ever read `->second`. + const mcpp::manifest::TargetEntry* sectionEntry = nullptr; + bool hasExplicitSection = false; + bool hasToolchainOverride = false; + const mcpp::toolchain::triple::TargetInfo* known = nullptr; +}; + +// The body of the `apply_target_section` closure the single-function version +// of this step captured. Used at two points -- an explicit `[target.]` +// section for the resolved request, and the host's own row on a build with no +// `--target` -- so it is a named helper rather than a per-call closure. +static void step1_apply_target_section(PrepareState& state, + const mcpp::manifest::TargetEntry& e) { + if (!e.toolchain.empty() && !state.tcFromCommandLine && !state.tcFromConsumer) { + state.tcSpec = e.toolchain; + state.tcOrigin = TcOrigin::TargetSection; + } + if (!e.linkage.empty()) state.m->buildConfig.linkage = e.linkage; + // #336: a per-target C++ runtime contract overrides the project + // default, so "self-contained everywhere except this triple" is + // expressible without touching the cfg() input channel. + if (!e.cxxRuntime.empty()) state.m->buildConfig.cxxRuntime = e.cxxRuntime; +} + +static std::expected +step1_resolve_target_triple_request(PrepareState& state, TargetOverrideCtx& ctx) { + namespace triple = mcpp::toolchain::triple; + // THE SPELLING THE PROJECT WROTE, KEPT FOR EVERY DIAGNOSTIC BELOW. + // `state.overrides.target_triple` is canonicalised further down, and until + // this variable existed the refusals quoted the canonical form: + // `--target aarch64-linux` produced "target 'aarch64-linux-gnu' is + // registered but not yet supported", a string the reader never typed + // and cannot find in their own command. + ctx.requestedSpelling = state.overrides.target_triple; + ctx.parsed = triple::parse(state.overrides.target_triple); + + // THE REQUEST IS COMPLETED FROM THE VOCABULARY BEFORE ANYTHING + // READS IT, AND THE ORDER RELATIVE TO THE `[target.X]` LOOKUP IS PART + // OF THE CONTRACT. + // + // `parse` fills a missing env segment lexically so the identity stays + // total — `x86_64-linux` IS `x86_64-linux-gnu`, and a unit test says so. + // Every gate below then asked about the filled value instead of about + // the request. See `triple::resolve_request` for the two measurements. + // + // The lookup that follows keys on `parsed->str()`, so completing after + // it would match sections against a triple this build is not going to + // use. A project wanting the `planned` row keeps its escape hatch by + // WRITING the segment: `--target aarch64-linux-gnu` skips completion + // entirely, because a written segment is a request rather than a gap. + if (ctx.parsed) { + ctx.req = triple::resolve_request(*ctx.parsed); + ctx.parsed = ctx.req.triple; + } + + // [target.X] lookup is spelling-independent: a section keyed + // `x86_64-w64-mingw32` matches `--target x86_64-windows-gnu` and + // vice versa. Unparseable keys/inputs compare exactly (escape hatch). + auto it = state.m->targetOverrides.find(state.overrides.target_triple); + if (it == state.m->targetOverrides.end() && ctx.parsed) { + for (auto o = state.m->targetOverrides.begin(); + o != state.m->targetOverrides.end(); ++o) { + if (auto k = triple::parse(o->first); + k && k->str() == ctx.parsed->str()) { it = o; break; } } - if (!e.linkage.empty()) state.m->buildConfig.linkage = e.linkage; - // #336: a per-target C++ runtime contract overrides the project - // default, so "self-contained everywhere except this triple" is - // expressible without touching the cfg() input channel. - if (!e.cxxRuntime.empty()) state.m->buildConfig.cxxRuntime = e.cxxRuntime; - }; + } + ctx.hasExplicitSection = it != state.m->targetOverrides.end(); + ctx.sectionEntry = ctx.hasExplicitSection ? &it->second : nullptr; + ctx.hasToolchainOverride = ctx.hasExplicitSection + && !it->second.toolchain.empty(); + + ctx.known = ctx.parsed ? triple::find_known_target(*ctx.parsed) : nullptr; + + return {}; +} + +static std::expected +step1_validate_target_tier(PrepareState& state, TargetOverrideCtx& ctx) { + namespace triple = mcpp::toolchain::triple; + // Validation: a typo must never silently fall through to the host + // toolchain (the worst failure mode — you think you cross-compiled). + // An explicit [target.X] section is the escape hatch for custom + // triples outside the vocabulary. + // Several rows serve this (arch, os) and the lexical default names none + // of them, so there is nothing to complete the request WITH. Refusing + // and listing them is the only honest answer; picking one would be an + // invented convention. No group has this shape today — the rule is here + // so the first one that does gets a diagnosis rather than a guess. + if (ctx.parsed && ctx.req.ambiguous && !ctx.hasExplicitSection) { + std::string opts; + for (auto s : ctx.req.supported) { + if (!opts.empty()) opts += ", "; + opts += std::string(s); + } + refusal::record(refusal::Code::AmbiguousRequest); + return std::unexpected(std::format( + "target '{}' does not say which C library, and several are " + "supported here.\n" + " candidates: {}\n" + " Name one of them.", + ctx.requestedSpelling, opts)); + } + if (!ctx.known && !ctx.hasExplicitSection) { + // "UNKNOWN" IS A CLAIM ABOUT THE VOCABULARY, AND IT WAS FALSE FOR + // A WHOLE arch+os FAMILY. + // + // Measured on 2026.8.26.1: `--target riscv64-linux` reported + // `unknown target 'riscv64-linux'` while `riscv64-linux-musl` was + // sitting in `kKnownTargets` as `planned`. The lexical fill had + // produced `riscv64-linux-gnu` — a row that genuinely does not + // exist — and the gate reported on the fill. + // + // A non-empty sibling group means the family IS registered, so this + // is the planned refusal wearing the wrong word. It names the row + // that exists, which is also the one the reader would have to write + // to opt in. + if (!ctx.req.siblings.empty()) { + std::string rows; + for (auto s : ctx.req.siblings) { + if (!rows.empty()) rows += ", "; + rows += std::string(s); + } + refusal::record(refusal::Code::TierPlanned); + return std::unexpected(std::format( + "target '{}' is registered but not yet supported (planned) — " + "no toolchain is published for it yet.\n" + " registered rows for this system: {}\n" + " An explicit [target.] toolchain override can " + "opt in early.", + ctx.requestedSpelling, rows)); + } + auto sug = triple::did_you_mean(ctx.requestedSpelling); + refusal::record(refusal::Code::UnknownTarget); + return std::unexpected(std::format( + "unknown target '{}'{}\n" + " known targets: `mcpp toolchain list`; a custom triple needs an\n" + " explicit [target.{}] section in mcpp.toml", + ctx.requestedSpelling, + sug ? std::format(" — did you mean '{}'?", *sug) : "", + ctx.requestedSpelling)); + } + if (ctx.known && ctx.known->tier == "planned" && !ctx.hasToolchainOverride) { + refusal::record(refusal::Code::TierPlanned); + // The subject is what the user wrote. When completion filled a + // segment, both are shown — otherwise the sentence is about a + // string that appears nowhere in their command. + const std::string subject = + ctx.requestedSpelling == ctx.parsed->str() + ? std::format("'{}'", ctx.requestedSpelling) + : std::format("'{}' (which resolves to '{}')", + ctx.requestedSpelling, ctx.parsed->str()); + return std::unexpected(std::format( + "target {} is registered but not yet supported (planned) — " + "no toolchain is published for it yet.\n" + " An explicit [target.{}] toolchain override can opt in early.", + subject, ctx.parsed->str())); + } + return {}; +} + +static std::expected +step1_apple_sdk_check(PrepareState& state, TargetOverrideCtx& ctx) { + // AN APPLE SDK IS LOCATED, SO ITS ABSENCE IS KNOWN NOW. + // + // REFUSED HERE AND NOT WITH THE TOOLCHAIN, which is a decision about + // WHEN rather than about the message. The iOS rows need the machine's + // iPhoneOS or iPhoneSimulator SDK, and that is knowable before any + // payload is resolved -- so a machine without Xcode used to download + // a 700 MB compiler and then be told the thing it was missing was not + // the compiler. + // + // AND UNLIKE `host_can_serve` BELOW, THIS IS NOT DEFERRED. That + // refusal waits for the dependency graph because a package can supply + // a target's C library and platform interface. An Apple SDK is not + // redistributable, so no package supplies it: there is nothing a later + // line could learn that would change this answer. + // + // The escape hatch that opens the tier gate does NOT open this one. + // Declaring a toolchain says which compiler; it says nothing about + // where the headers and stub libraries are, and every compiler needs + // them. + if (ctx.parsed && ctx.parsed->is_ios()) { + const auto which = ctx.parsed->is_ios_simulator() + ? mcpp::platform::macos::sdk_iphonesim + : mcpp::platform::macos::sdk_iphoneos; + state.appleSdkLocated = mcpp::platform::macos::sdk_path(which); + // AN UNSET FLOOR IS THE LOCATED SDK'S VERSION, READ RATHER THAN + // LEFT TO THE DRIVER. `docs/20` promised that an unversioned + // triple meant the SDK's own default; measured on macos-15 with + // Xcode 16.4, clang given `arm64-apple-ios` with no version + // refused thread-local storage for the target, which libc++abi + // uses, so the default it chose was older than any SDK on the + // machine. The version `xcrun` reports for the located SDK is the + // one the SDK was made for, and it enters the manifest here so + // that the fingerprint slot, the effective triple and every + // report read one value. + if (state.appleSdkLocated && state.m->buildConfig.iosDeploymentTarget.empty()) { + if (auto v = mcpp::platform::macos::sdk_version(which)) { + state.m->buildConfig.iosDeploymentTarget = *v; + state.iosFloorFromSdk = true; + } + } + if (!state.appleSdkLocated) { + // A CODE, BECAUSE THE MATRIX COMPARES REASONS AND NOT ONLY + // OUTCOMES. A refusal with no code is recorded as `other`, + // which `check_matrix_reasons.sh` refuses on the ground that + // it freezes an unnamed branch into the expected table. + refusal::record(refusal::Code::AppleSdkAbsent); + return std::unexpected(std::format( + "target {} needs the {} SDK, which this machine does not " + "provide.\n" + " It is not redistributable, so mcpp LOCATES it " + "rather than installing it: `xcrun --sdk {} " + "--show-sdk-path` must answer, which needs Xcode on macOS " + "(not the Command Line Tools alone -- those ship the " + "macOS SDK only).\n" + " Check `xcode-select -p`, and note that the " + "compiler is not what is missing: these rows pin " + "`xim:llvm`, which every other Apple row also uses.", + ctx.parsed->str(), which, which)); + } + } + return {}; +} + +static std::expected +step1_wasm_shared_lib_check(PrepareState& state, TargetOverrideCtx& ctx) { + namespace triple = mcpp::toolchain::triple; + // A `shared` TARGET NAMES A LINK CONTRACT THIS ENGINE DOES NOT RENDER. + // + // `-sSIDE_MODULE` is a different Emscripten link mode from the + // ordinary one (one static image, `artifact_naming`'s `.js`+`.wasm` + // pair) and mcpp emits no flag for it. Falling through to the + // ordinary link would still WRITE a `.so`-shaped file — the fallback + // naming's `sharedLibExt` is empty, so the linker would be asked for + // an empty-named output — so this is caught here, by NAME, rather + // than reached as an obscure link failure. + // + // REFUSED HERE AND NOT AT PLAN TIME, same reasoning as the Apple SDK + // check above: `parsed` and the manifest's own target list are both + // already known, resolving neither an emsdk payload nor any other + // toolchain, so an offline build (no emsdk installed) gets this + // sentence instead of downloading the SDK first. + if (ctx.parsed && ctx.parsed->object_format() + == triple::ObjectFormat::Wasm) { + for (auto const& t : state.m->targets) { + if (t.kind != mcpp::manifest::Target::SharedLibrary) continue; + return std::unexpected(std::format( + "[targets.{}] kind = \"shared\" is not supported on " + "wasm32-emscripten: a side module needs -sSIDE_MODULE, " + "which mcpp does not render", + t.name)); + } + } + return {}; +} + +static void +step1_host_can_serve_check(PrepareState& state, TargetOverrideCtx& ctx) { + namespace triple = mcpp::toolchain::triple; + // Known, supported — and IMPOSSIBLE ON THIS HOST. + // + // Without this the target falls through to the host toolchain and the + // build SUCCEEDS, which is the failure the check above calls the worst + // one, arriving through a different door. Measured on Linux: + // + // $ mcpp build --target x86_64-windows-msvc + // Resolved gcc@16.1.0 → x86_64-windows-msvc → …/xim-x-gcc/bin/g++ + // Finished dev [unoptimized + debuginfo] in 0.07s + // $ ls target/ + // x86_64-linux-gnu/ ← an ELF, reported as a Windows build + // + // The vocabulary tier says "mcpp supports this target"; it never said + // "this machine can produce it". `host_can_serve` is the answer to the + // second question and lives beside the payload resolution it has to + // agree with. + // + // The escape hatch stays open on purpose: an explicit `[target.X]` + // toolchain override means the author is supplying the cross toolchain + // themselves, and mcpp's payload matrix has no standing to refuse it. + // DIAGNOSED HERE, REPORTED LATER, AND THE DIFFERENCE IS THE POINT. + // + // Whether a payload on this machine produces this target is knowable + // now. Whether anything ELSE produces it is not: a dependency can + // supply the target's platform interface and C library, and the + // dependency graph does not exist yet at this line. Refusing here + // therefore answered a narrower question than the one it claimed — + // measured, a project that only had to add a dependency was told its + // machine could not build the target at all. + // + // The refusal is kept in full, because it is right whenever nothing + // supplies the target side, which remains the ordinary case. It is + // carried to where the graph is known and released there. Nothing + // between here and there consumes the answer: what follows is toolchain + // and dependency resolution, and a target no payload serves resolves to + // a driver that simply will not be asked to emit anything. + // + // The escape hatch stays open on purpose: an explicit `[target.X]` + // toolchain override means the author is supplying the cross toolchain + // themselves, and mcpp's payload matrix has no standing to refuse it. + if (ctx.known && ctx.known->tier != "planned" && !ctx.hasToolchainOverride + && ctx.parsed + && !mcpp::toolchain::host_can_serve(*ctx.parsed)) { + std::string servable; + for (auto const& info : triple::known_targets()) { + auto t = triple::parse(info.canonical); + if (!t || info.tier == "planned") continue; + if (!mcpp::toolchain::host_can_serve(*t)) continue; + if (!servable.empty()) servable += ", "; + servable += t->str(); + } + state.unservedTargetDiagnosis = std::format( + "target '{}' cannot be built on this host.\n" + " No toolchain payload here produces it, and nothing in " + "the dependency graph\n" + " supplies its system side.\n" + " this host can build with the payload alone: {}\n" + " To build it anyway, depend on a package that implements " + "the target's system\n" + " (its kernel interface and C library), or supply your own " + "cross toolchain with\n" + " an explicit [target.{}] toolchain = \"…\" section.", + ctx.parsed->str(), + servable.empty() ? "(nothing — `mcpp toolchain list`)" : servable, + ctx.parsed->str()); + } +} + +static void +step1_capture_display_and_canonicalize(PrepareState& state, TargetOverrideCtx& ctx) { + // CAPTURED BEFORE CANONICALISATION, BECAUSE CANONICALISATION IS + // EXACTLY WHAT DESTROYS IT. + // + // `str()` renders the filled-in identity, so `x86_64-linux` becomes + // `x86_64-linux-gnu` here and every later `parse` of that string reports + // an env segment the project never wrote. The request has to be taken + // from the ONLY triple that still knows the difference: this one. + if (ctx.parsed && ctx.parsed->envExplicit) state.requestedCAbi = ctx.parsed->env; + // AND THE SPELLING THE PROJECT USED, FOR THE REPORT ONLY. + // + // The canonical form is the identity — the output directory, the cache + // key, the subject of a `cfg()` — and it must stay filled. The REPORT is + // a different thing: it says what was asked for and what resolved, and + // heading it `x86_64-linux-gnu` above a line reading `c-abi musl` states + // a contradiction the build does not actually contain. A project that + // declined to name a C library is shown as having declined. + if (ctx.parsed && !ctx.parsed->envExplicit && !ctx.parsed->env.empty()) { + auto asWritten = *ctx.parsed; + asWritten.env.clear(); + state.targetDisplayName = asWritten.str(); + } + + // Canonical from here on: cfg evaluation, spec attachment and the + // target/ output directory all see one spelling. + if (ctx.parsed) state.overrides.target_triple = ctx.parsed->str(); + if (ctx.hasExplicitSection) step1_apply_target_section(state, *ctx.sectionEntry); +} + +static std::expected +step1_target_row_pin_and_capability_check(PrepareState& state, TargetOverrideCtx& ctx) { + // Convention from the vocabulary table (triple.cppm): the target's + // pinned toolchain (host-awareness — native musl-gcc vs triple-named + // cross, winlibs mingw vs Linux-hosted cross — lives in the payload + // mapping, not here) and its default linkage. GCC 16 pin rationale: + // GCC 15 drops module template instantiations at link (remediation + // doc A2; packages shipped 2026-07-08/09, GitHub+GitCode). + // A convention, not an instruction: on the Windows-GNU first-run path + // this is what turns the seeded target into `gcc@16.1.0`. + // + // It must not fire when it would overrule a toolchain the user wrote + // down. The pin is mcpp's own default for a target row — `gcc@16.1.0` + // for Windows-GNU, because the mingw payload is what supplies that + // target's headers and C library — and an explicit `[toolchain]` line + // is not a default. This is the promise the no-Visual-Studio fallback + // is built on: mcpp revises its own defaults, never yours. + // + // HOW THE TARGET WAS NAMED IS NOT PART OF THE QUESTION, and it used to + // be. The guard read `targetFromGlobalDefault && user_explicit`, so a + // target given on the command line disabled it — and then the row's pin + // replaced a toolchain the project had stated. Measured 2026-08-23: + // `--target x86_64-windows-gnu` with an explicit `llvm@22.1.8` resolved + // `x86_64-w64-mingw32-g++`, and gcc cannot compile libc++'s std module. + // + // A project that means to use a different compiler for a pinned target + // is stating something about its own build, and a project whose target + // side comes from its dependency graph is the ordinary reason to do so: + // the payload the row names supplies headers and a C library that such + // a project does not use. The narrower reading of this guard was + // patched with an openkal-specific exception; stating the rule + // correctly removes the need for one. + // RECORDED, NOT APPLIED. The convention answers "which payload + // supplies this target's C library", and whether it is needed depends on + // whether the dependency graph supplies one instead. That is knowable + // only after resolution, so the decision waits for + // `resolve_target_toolchain` and only the candidate is kept here. + if (ctx.known && !ctx.known->pin.empty() && ctx.parsed + && !ctx.parsed->pin_is_capability()) { + state.targetRowPin = std::string(ctx.known->pin); + state.targetRowName = ctx.parsed->str(); + } + if (ctx.known && !ctx.hasToolchainOverride && !ctx.known->pin.empty() + && !tc_origin_is_user_explicit(state.tcOrigin)) { + state.targetPinCandidate = std::string(ctx.known->pin); + state.targetPinIsCapability = ctx.parsed && ctx.parsed->pin_is_capability(); + } + // A USER'S EXPLICIT TOOLCHAIN OVERRIDES A CONVENTION, NOT A + // CAPABILITY — AND UNTIL THIS LINE IT OVERRODE BOTH. + // + // The block above deliberately steps aside for an explicit + // `[toolchain] default`: a hosted row's pin says "this payload supplies + // the target's C library", and an author who names their own compiler + // has said they will supply it instead. A bare-metal row's pin says + // something the author cannot override — the table's own words: "the + // pin is llvm on every host because clang/lld are cross-compilers by + // construction". A host g++ does not emit riscv64 whatever anyone + // declares. + // + // Measured 2026-08-26: + // + // [toolchain] default = "gcc@16.1.0" + // $ mcpp build --target riscv64-none-elf + // g++: error: unrecognized argument in option '-mabi=lp64d' + // g++: note: valid arguments to '-mabi=' are: ms sysv + // + // — a message about an option, for a decision made here. Refusing at + // the decision costs one line; the alternative is a compiler complaining + // about flags the reader never wrote. + if (ctx.known && ctx.parsed && ctx.parsed->pin_is_capability() + && tc_origin_is_user_explicit(state.tcOrigin) && state.tcSpec.has_value()) { + auto declared = mcpp::toolchain::parse_toolchain_spec(*state.tcSpec); + // WHICH DECLARATIONS THE ROW ACCEPTS IS THE ROW'S PIN, NOT A FIXED + // FAMILY. + // + // This asked `family != Llvm`, which was right while every + // capability-pinned row pinned llvm. `wasm32-emscripten` pins + // `emsdk@6.0.9`, and emsdk NORMALISES to the llvm family -- `em++` + // is clang -- so a declared `llvm@22.1.8` passed this gate, was + // never refused, and resolved the generic llvm payload for a target + // it cannot emit. The condition is now the pin's own family, which + // is the question the row was always answering. + const auto pinFamily = [&]() -> std::optional { + if (ctx.known->pin.empty()) return mcpp::toolchain::Family::Llvm; + if (auto ps = mcpp::toolchain::parse_toolchain_spec( + std::string(ctx.known->pin))) + return ps->family; + return std::nullopt; + }(); + const bool declaredMatchesPin = + declared && pinFamily && declared->family == *pinFamily + // An emsdk row is llvm-family, so the family alone cannot + // separate `emsdk@6.0.9` from `llvm@22.1.8`. The pin's own + // spelling is what does. + && (ctx.known->pin.empty() + || state.tcSpec->find(ctx.known->pin.substr(0, ctx.known->pin.find('@'))) + != std::string::npos); + if (declared && !declaredMatchesPin) { + // THE REASON TRAVELS WITH THE ROW. The rows refuse for the + // same rule and NOT for the same reason, and one sentence + // covering all of them would be wrong about the others: a + // PE+musl target is not bare metal, a wasm target is neither, + // and a reader told the wrong one stops reading. + // + // Measured before the third arm existed: `--target + // wasm32-emscripten` with a declared gcc was refused correctly + // and explained with "No gcc payload emits a PE with a musl C + // library", which is a true sentence about a different row. + // + // IT HAPPENED AGAIN, AND ADDING AN ARM IS ONLY HALF THE FIX. + // Android became a capability row and this chain still had + // three arms, so a declared `llvm@22.1.8` against + // `aarch64-linux-android` was refused correctly and explained + // with the PE+musl sentence -- the identical wrong answer the + // paragraph above records for wasm, reached the same way: by a + // fourth case falling into a final `else` that was written as + // the third case's answer. + // + // So the last arm now NAMES ITS OWN ROW and the fallthrough is + // generic. A capability added later gets a sentence that is + // merely unspecific instead of one that is false, and the + // refusal still names the pin either way. + std::string_view why = ctx.parsed->is_freestanding() + ? "A freestanding target has no per-host cross payload: " + "clang and lld are\n" + " cross-compilers by construction and gcc is not." + : ctx.parsed->is_wasm() + ? "Nothing but Emscripten emits WebAssembly: `em++` is a " + "clang whose target,\n" + " sysroot and JavaScript glue all come from its own " + "payload." + : ctx.parsed->is_android() + ? "An Android target needs bionic, not just an aarch64 or " + "x86_64 back end:\n" + " its headers, its per-API-level stubs and its " + "loader path are inside the\n" + " NDK, and no package adds them to another compiler." + : (ctx.parsed->is_pe() && ctx.parsed->is_musl()) + ? "No gcc payload emits a PE with a musl C library — the " + "mingw payload emits\n" + " PE with the MinGW CRT, which is the separate " + "`-gnu` row." + : "This row's toolchain is the only one that can emit the " + "target at all."; + refusal::record(refusal::Code::CapabilityPin); + return std::unexpected(std::format( + "target '{}' cannot be emitted by '{}'.\n" + " {}\n" + " The row names `{}` as a capability rather than as a " + "preference, so\n" + " this one line is not a convention you can override.\n" + " remove the `[toolchain]` line for this target, or set " + "it to `{}`.", + ctx.parsed->str(), *state.tcSpec, why, + ctx.known->pin.empty() ? std::string_view("llvm") : ctx.known->pin, + ctx.known->pin.empty() ? std::string_view("llvm") : ctx.known->pin)); + } + } + if (ctx.known && ctx.known->defaultStatic && state.m->buildConfig.linkage.empty()) + state.m->buildConfig.linkage = "static"; + return {}; +} + +static std::expected step1_target_and_static_overrides(PrepareState& state) { // ─── --target / --static overrides ────────────────────────────────── // Target-axis default resolution when no --target flag was passed: // [build] target (project default, ≙ cargo build.target) > @@ -441,469 +973,19 @@ std::expected phase1_toolchain_spec_and_axes(PrepareState& st // the known-target vocabulary, then apply the manifest [target.] // override and the vocabulary-table convention (pin + default linkage). if (!state.overrides.target_triple.empty()) { - namespace triple = mcpp::toolchain::triple; - // THE SPELLING THE PROJECT WROTE, KEPT FOR EVERY DIAGNOSTIC BELOW. - // `state.overrides.target_triple` is canonicalised further down, and until - // this variable existed the refusals quoted the canonical form: - // `--target aarch64-linux` produced "target 'aarch64-linux-gnu' is - // registered but not yet supported", a string the reader never typed - // and cannot find in their own command. - const std::string requestedSpelling = state.overrides.target_triple; - auto parsed = triple::parse(state.overrides.target_triple); - - // THE REQUEST IS COMPLETED FROM THE VOCABULARY BEFORE ANYTHING - // READS IT, AND THE ORDER RELATIVE TO THE `[target.X]` LOOKUP IS PART - // OF THE CONTRACT. - // - // `parse` fills a missing env segment lexically so the identity stays - // total — `x86_64-linux` IS `x86_64-linux-gnu`, and a unit test says so. - // Every gate below then asked about the filled value instead of about - // the request. See `triple::resolve_request` for the two measurements. - // - // The lookup that follows keys on `parsed->str()`, so completing after - // it would match sections against a triple this build is not going to - // use. A project wanting the `planned` row keeps its escape hatch by - // WRITING the segment: `--target aarch64-linux-gnu` skips completion - // entirely, because a written segment is a request rather than a gap. - triple::RequestResolution req; - if (parsed) { - req = triple::resolve_request(*parsed); - parsed = req.triple; - } - - // [target.X] lookup is spelling-independent: a section keyed - // `x86_64-w64-mingw32` matches `--target x86_64-windows-gnu` and - // vice versa. Unparseable keys/inputs compare exactly (escape hatch). - auto it = state.m->targetOverrides.find(state.overrides.target_triple); - if (it == state.m->targetOverrides.end() && parsed) { - for (auto o = state.m->targetOverrides.begin(); - o != state.m->targetOverrides.end(); ++o) { - if (auto k = triple::parse(o->first); - k && k->str() == parsed->str()) { it = o; break; } - } - } - bool hasExplicitSection = it != state.m->targetOverrides.end(); - bool hasToolchainOverride = hasExplicitSection - && !it->second.toolchain.empty(); - - const triple::TargetInfo* known = - parsed ? triple::find_known_target(*parsed) : nullptr; - - // Validation: a typo must never silently fall through to the host - // toolchain (the worst failure mode — you think you cross-compiled). - // An explicit [target.X] section is the escape hatch for custom - // triples outside the vocabulary. - // Several rows serve this (arch, os) and the lexical default names none - // of them, so there is nothing to complete the request WITH. Refusing - // and listing them is the only honest answer; picking one would be an - // invented convention. No group has this shape today — the rule is here - // so the first one that does gets a diagnosis rather than a guess. - if (parsed && req.ambiguous && !hasExplicitSection) { - std::string opts; - for (auto s : req.supported) { - if (!opts.empty()) opts += ", "; - opts += std::string(s); - } - refusal::record(refusal::Code::AmbiguousRequest); - return std::unexpected(std::format( - "target '{}' does not say which C library, and several are " - "supported here.\n" - " candidates: {}\n" - " Name one of them.", - requestedSpelling, opts)); - } - if (!known && !hasExplicitSection) { - // "UNKNOWN" IS A CLAIM ABOUT THE VOCABULARY, AND IT WAS FALSE FOR - // A WHOLE arch+os FAMILY. - // - // Measured on 2026.8.26.1: `--target riscv64-linux` reported - // `unknown target 'riscv64-linux'` while `riscv64-linux-musl` was - // sitting in `kKnownTargets` as `planned`. The lexical fill had - // produced `riscv64-linux-gnu` — a row that genuinely does not - // exist — and the gate reported on the fill. - // - // A non-empty sibling group means the family IS registered, so this - // is the planned refusal wearing the wrong word. It names the row - // that exists, which is also the one the reader would have to write - // to opt in. - if (!req.siblings.empty()) { - std::string rows; - for (auto s : req.siblings) { - if (!rows.empty()) rows += ", "; - rows += std::string(s); - } - refusal::record(refusal::Code::TierPlanned); - return std::unexpected(std::format( - "target '{}' is registered but not yet supported (planned) — " - "no toolchain is published for it yet.\n" - " registered rows for this system: {}\n" - " An explicit [target.] toolchain override can " - "opt in early.", - requestedSpelling, rows)); - } - auto sug = triple::did_you_mean(requestedSpelling); - refusal::record(refusal::Code::UnknownTarget); - return std::unexpected(std::format( - "unknown target '{}'{}\n" - " known targets: `mcpp toolchain list`; a custom triple needs an\n" - " explicit [target.{}] section in mcpp.toml", - requestedSpelling, - sug ? std::format(" — did you mean '{}'?", *sug) : "", - requestedSpelling)); - } - if (known && known->tier == "planned" && !hasToolchainOverride) { - refusal::record(refusal::Code::TierPlanned); - // The subject is what the user wrote. When completion filled a - // segment, both are shown — otherwise the sentence is about a - // string that appears nowhere in their command. - const std::string subject = - requestedSpelling == parsed->str() - ? std::format("'{}'", requestedSpelling) - : std::format("'{}' (which resolves to '{}')", - requestedSpelling, parsed->str()); - return std::unexpected(std::format( - "target {} is registered but not yet supported (planned) — " - "no toolchain is published for it yet.\n" - " An explicit [target.{}] toolchain override can opt in early.", - subject, parsed->str())); - } - // AN APPLE SDK IS LOCATED, SO ITS ABSENCE IS KNOWN NOW. - // - // REFUSED HERE AND NOT WITH THE TOOLCHAIN, which is a decision about - // WHEN rather than about the message. The iOS rows need the machine's - // iPhoneOS or iPhoneSimulator SDK, and that is knowable before any - // payload is resolved -- so a machine without Xcode used to download - // a 700 MB compiler and then be told the thing it was missing was not - // the compiler. - // - // AND UNLIKE `host_can_serve` BELOW, THIS IS NOT DEFERRED. That - // refusal waits for the dependency graph because a package can supply - // a target's C library and platform interface. An Apple SDK is not - // redistributable, so no package supplies it: there is nothing a later - // line could learn that would change this answer. - // - // The escape hatch that opens the tier gate does NOT open this one. - // Declaring a toolchain says which compiler; it says nothing about - // where the headers and stub libraries are, and every compiler needs - // them. - if (parsed && parsed->is_ios()) { - const auto which = parsed->is_ios_simulator() - ? mcpp::platform::macos::sdk_iphonesim - : mcpp::platform::macos::sdk_iphoneos; - state.appleSdkLocated = mcpp::platform::macos::sdk_path(which); - // AN UNSET FLOOR IS THE LOCATED SDK'S VERSION, READ RATHER THAN - // LEFT TO THE DRIVER. `docs/20` promised that an unversioned - // triple meant the SDK's own default; measured on macos-15 with - // Xcode 16.4, clang given `arm64-apple-ios` with no version - // refused thread-local storage for the target, which libc++abi - // uses, so the default it chose was older than any SDK on the - // machine. The version `xcrun` reports for the located SDK is the - // one the SDK was made for, and it enters the manifest here so - // that the fingerprint slot, the effective triple and every - // report read one value. - if (state.appleSdkLocated && state.m->buildConfig.iosDeploymentTarget.empty()) { - if (auto v = mcpp::platform::macos::sdk_version(which)) { - state.m->buildConfig.iosDeploymentTarget = *v; - state.iosFloorFromSdk = true; - } - } - if (!state.appleSdkLocated) { - // A CODE, BECAUSE THE MATRIX COMPARES REASONS AND NOT ONLY - // OUTCOMES. A refusal with no code is recorded as `other`, - // which `check_matrix_reasons.sh` refuses on the ground that - // it freezes an unnamed branch into the expected table. - refusal::record(refusal::Code::AppleSdkAbsent); - return std::unexpected(std::format( - "target {} needs the {} SDK, which this machine does not " - "provide.\n" - " It is not redistributable, so mcpp LOCATES it " - "rather than installing it: `xcrun --sdk {} " - "--show-sdk-path` must answer, which needs Xcode on macOS " - "(not the Command Line Tools alone -- those ship the " - "macOS SDK only).\n" - " Check `xcode-select -p`, and note that the " - "compiler is not what is missing: these rows pin " - "`xim:llvm`, which every other Apple row also uses.", - parsed->str(), which, which)); - } - } - // A `shared` TARGET NAMES A LINK CONTRACT THIS ENGINE DOES NOT RENDER. - // - // `-sSIDE_MODULE` is a different Emscripten link mode from the - // ordinary one (one static image, `artifact_naming`'s `.js`+`.wasm` - // pair) and mcpp emits no flag for it. Falling through to the - // ordinary link would still WRITE a `.so`-shaped file — the fallback - // naming's `sharedLibExt` is empty, so the linker would be asked for - // an empty-named output — so this is caught here, by NAME, rather - // than reached as an obscure link failure. - // - // REFUSED HERE AND NOT AT PLAN TIME, same reasoning as the Apple SDK - // check above: `parsed` and the manifest's own target list are both - // already known, resolving neither an emsdk payload nor any other - // toolchain, so an offline build (no emsdk installed) gets this - // sentence instead of downloading the SDK first. - if (parsed && parsed->object_format() - == triple::ObjectFormat::Wasm) { - for (auto const& t : state.m->targets) { - if (t.kind != mcpp::manifest::Target::SharedLibrary) continue; - return std::unexpected(std::format( - "[targets.{}] kind = \"shared\" is not supported on " - "wasm32-emscripten: a side module needs -sSIDE_MODULE, " - "which mcpp does not render", - t.name)); - } - } - // Known, supported — and IMPOSSIBLE ON THIS HOST. - // - // Without this the target falls through to the host toolchain and the - // build SUCCEEDS, which is the failure the check above calls the worst - // one, arriving through a different door. Measured on Linux: - // - // $ mcpp build --target x86_64-windows-msvc - // Resolved gcc@16.1.0 → x86_64-windows-msvc → …/xim-x-gcc/bin/g++ - // Finished dev [unoptimized + debuginfo] in 0.07s - // $ ls target/ - // x86_64-linux-gnu/ ← an ELF, reported as a Windows build - // - // The vocabulary tier says "mcpp supports this target"; it never said - // "this machine can produce it". `host_can_serve` is the answer to the - // second question and lives beside the payload resolution it has to - // agree with. - // - // The escape hatch stays open on purpose: an explicit `[target.X]` - // toolchain override means the author is supplying the cross toolchain - // themselves, and mcpp's payload matrix has no standing to refuse it. - // DIAGNOSED HERE, REPORTED LATER, AND THE DIFFERENCE IS THE POINT. - // - // Whether a payload on this machine produces this target is knowable - // now. Whether anything ELSE produces it is not: a dependency can - // supply the target's platform interface and C library, and the - // dependency graph does not exist yet at this line. Refusing here - // therefore answered a narrower question than the one it claimed — - // measured, a project that only had to add a dependency was told its - // machine could not build the target at all. - // - // The refusal is kept in full, because it is right whenever nothing - // supplies the target side, which remains the ordinary case. It is - // carried to where the graph is known and released there. Nothing - // between here and there consumes the answer: what follows is toolchain - // and dependency resolution, and a target no payload serves resolves to - // a driver that simply will not be asked to emit anything. - // - // The escape hatch stays open on purpose: an explicit `[target.X]` - // toolchain override means the author is supplying the cross toolchain - // themselves, and mcpp's payload matrix has no standing to refuse it. - if (known && known->tier != "planned" && !hasToolchainOverride - && parsed - && !mcpp::toolchain::host_can_serve(*parsed)) { - std::string servable; - for (auto const& info : triple::known_targets()) { - auto t = triple::parse(info.canonical); - if (!t || info.tier == "planned") continue; - if (!mcpp::toolchain::host_can_serve(*t)) continue; - if (!servable.empty()) servable += ", "; - servable += t->str(); - } - state.unservedTargetDiagnosis = std::format( - "target '{}' cannot be built on this host.\n" - " No toolchain payload here produces it, and nothing in " - "the dependency graph\n" - " supplies its system side.\n" - " this host can build with the payload alone: {}\n" - " To build it anyway, depend on a package that implements " - "the target's system\n" - " (its kernel interface and C library), or supply your own " - "cross toolchain with\n" - " an explicit [target.{}] toolchain = \"…\" section.", - parsed->str(), - servable.empty() ? "(nothing — `mcpp toolchain list`)" : servable, - parsed->str()); - } - // CAPTURED BEFORE CANONICALISATION, BECAUSE CANONICALISATION IS - // EXACTLY WHAT DESTROYS IT. - // - // `str()` renders the filled-in identity, so `x86_64-linux` becomes - // `x86_64-linux-gnu` here and every later `parse` of that string reports - // an env segment the project never wrote. The request has to be taken - // from the ONLY triple that still knows the difference: this one. - if (parsed && parsed->envExplicit) state.requestedCAbi = parsed->env; - // AND THE SPELLING THE PROJECT USED, FOR THE REPORT ONLY. - // - // The canonical form is the identity — the output directory, the cache - // key, the subject of a `cfg()` — and it must stay filled. The REPORT is - // a different thing: it says what was asked for and what resolved, and - // heading it `x86_64-linux-gnu` above a line reading `c-abi musl` states - // a contradiction the build does not actually contain. A project that - // declined to name a C library is shown as having declined. - if (parsed && !parsed->envExplicit && !parsed->env.empty()) { - auto asWritten = *parsed; - asWritten.env.clear(); - state.targetDisplayName = asWritten.str(); - } - - // Canonical from here on: cfg evaluation, spec attachment and the - // target/ output directory all see one spelling. - if (parsed) state.overrides.target_triple = parsed->str(); - - if (hasExplicitSection) apply_target_section(it->second); - // Convention from the vocabulary table (triple.cppm): the target's - // pinned toolchain (host-awareness — native musl-gcc vs triple-named - // cross, winlibs mingw vs Linux-hosted cross — lives in the payload - // mapping, not here) and its default linkage. GCC 16 pin rationale: - // GCC 15 drops module template instantiations at link (remediation - // doc A2; packages shipped 2026-07-08/09, GitHub+GitCode). - // A convention, not an instruction: on the Windows-GNU first-run path - // this is what turns the seeded target into `gcc@16.1.0`. - // - // It must not fire when it would overrule a toolchain the user wrote - // down. The pin is mcpp's own default for a target row — `gcc@16.1.0` - // for Windows-GNU, because the mingw payload is what supplies that - // target's headers and C library — and an explicit `[toolchain]` line - // is not a default. This is the promise the no-Visual-Studio fallback - // is built on: mcpp revises its own defaults, never yours. - // - // HOW THE TARGET WAS NAMED IS NOT PART OF THE QUESTION, and it used to - // be. The guard read `targetFromGlobalDefault && user_explicit`, so a - // target given on the command line disabled it — and then the row's pin - // replaced a toolchain the project had stated. Measured 2026-08-23: - // `--target x86_64-windows-gnu` with an explicit `llvm@22.1.8` resolved - // `x86_64-w64-mingw32-g++`, and gcc cannot compile libc++'s std module. - // - // A project that means to use a different compiler for a pinned target - // is stating something about its own build, and a project whose target - // side comes from its dependency graph is the ordinary reason to do so: - // the payload the row names supplies headers and a C library that such - // a project does not use. The narrower reading of this guard was - // patched with an openkal-specific exception; stating the rule - // correctly removes the need for one. - // RECORDED, NOT APPLIED. The convention answers "which payload - // supplies this target's C library", and whether it is needed depends on - // whether the dependency graph supplies one instead. That is knowable - // only after resolution, so the decision waits for - // `resolve_target_toolchain` and only the candidate is kept here. - if (known && !known->pin.empty() && parsed - && !parsed->pin_is_capability()) { - state.targetRowPin = std::string(known->pin); - state.targetRowName = parsed->str(); - } - if (known && !hasToolchainOverride && !known->pin.empty() - && !tc_origin_is_user_explicit(state.tcOrigin)) { - state.targetPinCandidate = std::string(known->pin); - state.targetPinIsCapability = parsed && parsed->pin_is_capability(); - } - // A USER'S EXPLICIT TOOLCHAIN OVERRIDES A CONVENTION, NOT A - // CAPABILITY — AND UNTIL THIS LINE IT OVERRODE BOTH. - // - // The block above deliberately steps aside for an explicit - // `[toolchain] default`: a hosted row's pin says "this payload supplies - // the target's C library", and an author who names their own compiler - // has said they will supply it instead. A bare-metal row's pin says - // something the author cannot override — the table's own words: "the - // pin is llvm on every host because clang/lld are cross-compilers by - // construction". A host g++ does not emit riscv64 whatever anyone - // declares. - // - // Measured 2026-08-26: - // - // [toolchain] default = "gcc@16.1.0" - // $ mcpp build --target riscv64-none-elf - // g++: error: unrecognized argument in option '-mabi=lp64d' - // g++: note: valid arguments to '-mabi=' are: ms sysv - // - // — a message about an option, for a decision made here. Refusing at - // the decision costs one line; the alternative is a compiler complaining - // about flags the reader never wrote. - if (known && parsed && parsed->pin_is_capability() - && tc_origin_is_user_explicit(state.tcOrigin) && state.tcSpec.has_value()) { - auto declared = mcpp::toolchain::parse_toolchain_spec(*state.tcSpec); - // WHICH DECLARATIONS THE ROW ACCEPTS IS THE ROW'S PIN, NOT A FIXED - // FAMILY. - // - // This asked `family != Llvm`, which was right while every - // capability-pinned row pinned llvm. `wasm32-emscripten` pins - // `emsdk@6.0.9`, and emsdk NORMALISES to the llvm family -- `em++` - // is clang -- so a declared `llvm@22.1.8` passed this gate, was - // never refused, and resolved the generic llvm payload for a target - // it cannot emit. The condition is now the pin's own family, which - // is the question the row was always answering. - const auto pinFamily = [&]() -> std::optional { - if (known->pin.empty()) return mcpp::toolchain::Family::Llvm; - if (auto ps = mcpp::toolchain::parse_toolchain_spec( - std::string(known->pin))) - return ps->family; - return std::nullopt; - }(); - const bool declaredMatchesPin = - declared && pinFamily && declared->family == *pinFamily - // An emsdk row is llvm-family, so the family alone cannot - // separate `emsdk@6.0.9` from `llvm@22.1.8`. The pin's own - // spelling is what does. - && (known->pin.empty() - || state.tcSpec->find(known->pin.substr(0, known->pin.find('@'))) - != std::string::npos); - if (declared && !declaredMatchesPin) { - // THE REASON TRAVELS WITH THE ROW. The rows refuse for the - // same rule and NOT for the same reason, and one sentence - // covering all of them would be wrong about the others: a - // PE+musl target is not bare metal, a wasm target is neither, - // and a reader told the wrong one stops reading. - // - // Measured before the third arm existed: `--target - // wasm32-emscripten` with a declared gcc was refused correctly - // and explained with "No gcc payload emits a PE with a musl C - // library", which is a true sentence about a different row. - // - // IT HAPPENED AGAIN, AND ADDING AN ARM IS ONLY HALF THE FIX. - // Android became a capability row and this chain still had - // three arms, so a declared `llvm@22.1.8` against - // `aarch64-linux-android` was refused correctly and explained - // with the PE+musl sentence -- the identical wrong answer the - // paragraph above records for wasm, reached the same way: by a - // fourth case falling into a final `else` that was written as - // the third case's answer. - // - // So the last arm now NAMES ITS OWN ROW and the fallthrough is - // generic. A capability added later gets a sentence that is - // merely unspecific instead of one that is false, and the - // refusal still names the pin either way. - std::string_view why = parsed->is_freestanding() - ? "A freestanding target has no per-host cross payload: " - "clang and lld are\n" - " cross-compilers by construction and gcc is not." - : parsed->is_wasm() - ? "Nothing but Emscripten emits WebAssembly: `em++` is a " - "clang whose target,\n" - " sysroot and JavaScript glue all come from its own " - "payload." - : parsed->is_android() - ? "An Android target needs bionic, not just an aarch64 or " - "x86_64 back end:\n" - " its headers, its per-API-level stubs and its " - "loader path are inside the\n" - " NDK, and no package adds them to another compiler." - : (parsed->is_pe() && parsed->is_musl()) - ? "No gcc payload emits a PE with a musl C library — the " - "mingw payload emits\n" - " PE with the MinGW CRT, which is the separate " - "`-gnu` row." - : "This row's toolchain is the only one that can emit the " - "target at all."; - refusal::record(refusal::Code::CapabilityPin); - return std::unexpected(std::format( - "target '{}' cannot be emitted by '{}'.\n" - " {}\n" - " The row names `{}` as a capability rather than as a " - "preference, so\n" - " this one line is not a convention you can override.\n" - " remove the `[toolchain]` line for this target, or set " - "it to `{}`.", - parsed->str(), *state.tcSpec, why, - known->pin.empty() ? std::string_view("llvm") : known->pin, - known->pin.empty() ? std::string_view("llvm") : known->pin)); - } - } - if (known && known->defaultStatic && state.m->buildConfig.linkage.empty()) - state.m->buildConfig.linkage = "static"; + TargetOverrideCtx ctx; + if (auto r = step1_resolve_target_triple_request(state, ctx); !r) + return std::unexpected(r.error()); + if (auto r = step1_validate_target_tier(state, ctx); !r) + return std::unexpected(r.error()); + if (auto r = step1_apple_sdk_check(state, ctx); !r) + return std::unexpected(r.error()); + if (auto r = step1_wasm_shared_lib_check(state, ctx); !r) + return std::unexpected(r.error()); + step1_host_can_serve_check(state, ctx); + step1_capture_display_and_canonicalize(state, ctx); + if (auto r = step1_target_row_pin_and_capability_check(state, ctx); !r) + return std::unexpected(r.error()); } // A HOST BUILD READS ITS OWN ROW (#704). `[target.]` is looked up // by the triple the build produces, and a build without `--target` @@ -915,7 +997,7 @@ std::expected phase1_toolchain_spec_and_axes(PrepareState& st // the host build a target build and turn the row's env segment into a // requested C library. else if (auto* hostRow = find_target_entry(*state.m, mcpp::toolchain::triple::host_triple())) - apply_target_section(*hostRow); + step1_apply_target_section(state, *hostRow); if (state.overrides.force_static) state.m->buildConfig.linkage = "static"; // #254: everything compiled INTO this build is resolved for the TARGET — @@ -930,6 +1012,11 @@ std::expected phase1_toolchain_spec_and_axes(PrepareState& st // canonicalized. Reading it before that point would silently fall back to // the host for any project that sets its target in the manifest rather // than on the command line. + return {}; +} + + +static std::expected step1_device_axis_and_layer_merge(PrepareState& state) { // ── The device axis, resolved ONCE ──────────────────────────────────── // // `--accel` / `--no-accel` over `[build] accel`. `--no-accel` arrives as the @@ -1079,13 +1166,36 @@ std::expected phase1_toolchain_spec_and_axes(PrepareState& st // Guards the one recursive call below. Set before the call so the second // pass cannot reach it, whatever else changed in between. state.targetPassDone = false; + return {}; +} + +std::expected phase1_toolchain_spec_and_axes(PrepareState& state) { + if (auto r = step1_define_early_toolchain_closures(state); !r) return std::unexpected(r.error()); + if (auto r = step1_target_and_static_overrides(state); !r) return std::unexpected(r.error()); + if (auto r = step1_device_axis_and_layer_merge(state); !r) return std::unexpected(r.error()); return {}; } -std::expected phase2_define_toolchain_resolver(PrepareState& state) { - state.resolve_target_toolchain = [&]() -> std::expected { - std::optional parsedSpec; +// A phase-local struct passed by reference to the steps of ONE call to +// `state.resolve_target_toolchain` -- the same pattern WorklistItemCtx +// (graph.cpp) and HostToolCtx (features.cpp) use for the steps of one +// worklist item / one requested tool. Only the parsed spec and the Windows +// installed-toolset probe outlive the branch that computes them; every other +// local below (the first-run defaults, the explicit-spec resolution's own +// payload/frontend locals, and so on) is read only within the one step that +// declares it and stays a plain local there, exactly as it was in the single +// function this splits. +struct ToolchainResolveCtx { + std::optional parsedSpec; + // Windows only (see step2_parse_toolchain_spec); resolvable on every + // platform so the struct itself has one shape. + std::optional installedPin; + std::vector installedPinNotes; +}; + +static std::expected +step2_parse_toolchain_spec(PrepareState& state, ToolchainResolveCtx& ctx) { auto tcOriginAxis = mcpp::toolchain::Origin::Managed; if (state.tcSpec.has_value() && *state.tcSpec != "system") { // A parse FAILURE is not the same as an unparseable spec being @@ -1095,14 +1205,14 @@ std::expected phase2_define_toolchain_resolver(PrepareState& auto s = mcpp::toolchain::parse_toolchain_spec(*state.tcSpec); if (!s) return std::unexpected(std::format( "{} = '{}': {}", state.tcSpecSource(), *state.tcSpec, s.error())); - parsedSpec = std::move(*s); - tcOriginAxis = mcpp::toolchain::origin_of(*parsedSpec); + ctx.parsedSpec = std::move(*s); + tcOriginAxis = mcpp::toolchain::origin_of(*ctx.parsedSpec); } // ASSIGNED, NOT DECLARED. `host_tc_for_build_program` reads it and is // defined outside this lambda, so the declaration lives in the enclosing // scope; the value is still decided here, where the spec is parsed. state.tcSpecIsMsvc = - parsedSpec && tcOriginAxis == mcpp::toolchain::Origin::SystemMsvc; + ctx.parsedSpec && tcOriginAxis == mcpp::toolchain::Origin::SystemMsvc; // A PINNED TOOLSET THIS MACHINE ALREADY HAS IS USED WHERE IT IS. // @@ -1110,24 +1220,28 @@ std::expected phase2_define_toolchain_resolver(PrepareState& // of that version unpacks the same installer payloads Visual Studio does, // so an installed copy is the same toolset without a download. `xim:` // opts out: it asks for the package, whose SDK is pinned with it. - std::optional installedPin; - std::vector installedPinNotes; if constexpr (mcpp::platform::is_windows) { - if (parsedSpec && !state.tcSpecIsMsvc - && parsedSpec->family == mcpp::toolchain::Family::Msvc - && !parsedSpec->ecosystemOnly && !parsedSpec->version.empty()) - installedPin = mcpp::toolchain::msvc::system_installation_matching( - parsedSpec->version, mcpp::toolchain::msvc::ToolsetNeeds{}, - &installedPinNotes); + if (ctx.parsedSpec && !state.tcSpecIsMsvc + && ctx.parsedSpec->family == mcpp::toolchain::Family::Msvc + && !ctx.parsedSpec->ecosystemOnly && !ctx.parsedSpec->version.empty()) + ctx.installedPin = mcpp::toolchain::msvc::system_installation_matching( + ctx.parsedSpec->version, mcpp::toolchain::msvc::ToolsetNeeds{}, + &ctx.installedPinNotes); } + return {}; +} - if (installedPin) { - for (auto const& n : installedPinNotes) mcpp::ui::info("note", n); - state.explicit_compiler = installedPin->clPath; +static void +step2_use_installed_pin(PrepareState& state, ToolchainResolveCtx& ctx) { + for (auto const& n : ctx.installedPinNotes) mcpp::ui::info("note", n); + state.explicit_compiler = ctx.installedPin->clPath; mcpp::ui::info("Resolved", std::format( - "{} → msvc {} (installed: {})", parsedSpec->display(), - installedPin->display_version(), installedPin->clPath.string())); - } else if (state.tcSpecIsMsvc) { + "{} → msvc {} (installed: {})", ctx.parsedSpec->display(), + ctx.installedPin->display_version(), ctx.installedPin->clPath.string())); +} + +static std::expected +step2_use_system_msvc(PrepareState& state) { if (!mcpp::platform::is_windows) { return std::unexpected(std::format( "toolchain '{}' is only available on Windows hosts", *state.tcSpec)); @@ -1140,8 +1254,12 @@ std::expected phase2_define_toolchain_resolver(PrepareState& mcpp::ui::info("Resolved", std::format( "msvc@system → msvc {} ({})", inst->display_version(), inst->clPath.string())); - } else if (parsedSpec) { - auto spec = parsedSpec; + return {}; +} + +static std::expected +step2_resolve_explicit_spec(PrepareState& state, ToolchainResolveCtx& ctx) { + auto spec = ctx.parsedSpec; if (spec->version.empty()) { return std::unexpected(std::format( "{} = '{}' is invalid; expected '@'", @@ -1291,7 +1409,11 @@ std::expected phase2_define_toolchain_resolver(PrepareState& mcpp::fetcher::make_path_ctx(&**state.get_cfg(true), *state.root)), chosenBy)); } - } else if (state.tcSpec.has_value() && *state.tcSpec == "system") { + return {}; +} + +static std::expected +step2_system_toolchain_refusal(PrepareState& state) { // REFUSED. THE COMPILER IS THE ONE AXIS THAT IS NOT THE PROJECT'S TO // TAKE FROM THE HOST. // @@ -1338,8 +1460,10 @@ std::expected phase2_define_toolchain_resolver(PrepareState& " Host LIBRARIES are a separate question and are not refused " "— a project may link them and owns the result.", kCurrentPlatform, kCurrentPlatform)); - } else if (mcpp::platform::env::offline_mode() - || mcpp::platform::env::no_auto_install()) { +} + +static std::expected +step2_offline_refusal(PrepareState& state) { // CI / offline / test opt-out: hard-error instead of silently // pulling ~800 MB of toolchain. Preserves the original M5.5 // contract for environments that need it. @@ -1390,7 +1514,10 @@ std::expected phase2_define_toolchain_resolver(PrepareState& " {}", pins::kSuggestGccMusl, pins::kFirstRunLinuxOther, release)); } - } else { +} + +static std::expected +step2_first_run_auto_install(PrepareState& state) { // First-run UX: no project-level [toolchain], no global default, // and the user just ran `mcpp build` (or similar). Auto-install // the platform's canonical default so the user gets a working @@ -1510,8 +1637,11 @@ std::expected phase2_define_toolchain_resolver(PrepareState& // persists BOTH axes, and persisting only the target leaves // `mcpp toolchain list` disagreeing with what the build used. state.firstRunNeedsTargetPass = !state.overrides.target_triple.empty(); - } + return {}; +} +static void +step2_windows_gnu_first_run_persist(PrepareState& state) { // Windows first run that got diverted to winlibs GCC: announce it and // persist BOTH axes, so the next invocation is silent and // `mcpp toolchain list` shows the same pair the build actually used. @@ -1544,49 +1674,10 @@ std::expected phase2_define_toolchain_resolver(PrepareState& } state.tcOrigin = TcOrigin::FirstRun; } +} - // AND NOW RESOLVE FOR THE TARGET, IF ONE WAS ASKED FOR. - // - // The first-run branch above answers "this machine has no toolchain, give - // it one", and the answer is a HOST payload; `--target` was never read - // there. On a machine that had never built anything, - // `mcpp build --target x86_64-windows-gnu` therefore installed a native - // gcc and compiled Windows sources with it — measured in CI 2026-08-25: - // - // First run no toolchain configured — installing gcc@16.1.0 … - // Resolved gcc@16.1.0 → …/xim-x-gcc/16.1.0/bin/g++ - // ↑ no target in the path - // - // against the same command where one already existed: - // - // Resolved gcc@16.1.0 → x86_64-windows-gnu → …/mingw-cross-gcc/… - // - // REUSES THE PATH THAT ALREADY KNOWS HOW rather than repeating it. The - // default just chosen is the spec; mapping a spec plus a target onto a - // payload (installing it if absent — `autoInstall` was always true there) - // is what the top of this function does. Depth is one: the second pass - // takes the `tcSpec.has_value()` branch the first run just made true. - // ONE-SHOT, AND THE FLAG IS SET BEFORE THE CALL, NOT AFTER. - // - // This line sits OUTSIDE the first-run branch — it has to, because the - // Windows block just above sets the target itself — so it is evaluated on - // every pass. The first version relied on `firstRunNeedsTargetPass` being - // false on the second pass; it is a captured variable that nothing - // resets, so every pass recursed again. Measured in a consumer's CI as - // the same `Resolved` line four times and then - // - // ##[error]Process completed with exit code 139 - // - // — SIGSEGV, a stack that ran out. A recursion whose termination depends - // on state the recursive call does not change is not a depth-one - // recursion, however its comment reads. - if (!state.targetPassDone - && (state.firstRunNeedsTargetPass - || (state.windowsGnuFirstRun && state.tcSpec.has_value()))) { - state.targetPassDone = true; - return state.resolve_target_toolchain(); - } - +static std::expected +step2_detect_toolchain(PrepareState& state) { auto detected = mcpp::toolchain::detect( state.explicit_compiler, state.runtimePayload, state.runtimeBindingSnapshot.contractHash); if (!detected) return std::unexpected(detected.error().message); @@ -1599,7 +1690,11 @@ std::expected phase2_define_toolchain_resolver(PrepareState& // SILENTLY is indistinguishable from one that was never set. if (!state.tc->resolutionNote.empty()) mcpp::ui::info("note", state.tc->resolutionNote); + return {}; +} +static std::expected +step2_retarget_for_retargetable_driver(PrepareState& state) { // ── A retargetable driver has to be TOLD what it is targeting ──────── // // `tc.targetTriple` comes from `-dumpmachine`, and for every cross target @@ -1925,7 +2020,11 @@ std::expected phase2_define_toolchain_resolver(PrepareState& } } } + return {}; +} +static std::expected +step2_bind_msvc_toolset(PrepareState& state) { // THE MSVC TOOLSET OF THE CLANG ROW, chosen once and recorded before the // runtime identity below reads its SDK version. See bind_msvc_sysroot. if (state.tc->compiler == mcpp::toolchain::CompilerId::Clang @@ -1936,7 +2035,11 @@ std::expected phase2_define_toolchain_resolver(PrepareState& if (auto ok = check_cl_row_sysroot(*state.tc, *state.m); !ok) return std::unexpected(ok.error()); } + return {}; +} +static void +step2_windows_runtime_identity(PrepareState& state) { // The Windows runtime identity, flowing BACK into the contract. // // Everything else about the runtime is known before a toolchain is @@ -1953,7 +2056,10 @@ std::expected phase2_define_toolchain_resolver(PrepareState& state.runtimeBindingSnapshot, state.tc->windowsSdkVersion); state.tc->runtimeContractHash = state.runtimeBindingSnapshot.contractHash; } +} +static std::expected +step2_msvc_abi_without_msvc_repair(PrepareState& state) { // ── Targeting the MSVC ABI without a usable MSVC ───────────────────── // // One judgement, one place. This used to be two separate concerns and @@ -2070,21 +2176,98 @@ std::expected phase2_define_toolchain_resolver(PrepareState& if (!redetected) return std::unexpected(redetected.error().message); state.tc = std::move(*redetected); } + return {}; +} - // For musl-gcc the toolchain is fully self-contained - // (`/x86_64-linux-musl/{include,lib}` is its own sysroot). - // musl-gcc's `-dumpmachine` reports `x86_64-linux-musl`. - bool isMuslTc = mcpp::toolchain::is_musl_target(*state.tc); - - // A musl toolchain only really makes sense with static linkage — - // dynamic-musl binaries depend on a system /lib/ld-musl-x86_64.so.1 - // that most distros don't ship. Default linkage to "static" when - // the resolved toolchain is musl, unless the user has already opted - // out via `--static` or [target.].linkage. (There is no - // [build].linkage — the parser only reads it under a target section.) - if (isMuslTc && state.m->buildConfig.linkage.empty()) { - state.m->buildConfig.linkage = "static"; +static void +step2_musl_default_static_linkage(PrepareState& state) { + // For musl-gcc the toolchain is fully self-contained + // (`/x86_64-linux-musl/{include,lib}` is its own sysroot). + // musl-gcc's `-dumpmachine` reports `x86_64-linux-musl`. + bool isMuslTc = mcpp::toolchain::is_musl_target(*state.tc); + + // A musl toolchain only really makes sense with static linkage — + // dynamic-musl binaries depend on a system /lib/ld-musl-x86_64.so.1 + // that most distros don't ship. Default linkage to "static" when + // the resolved toolchain is musl, unless the user has already opted + // out via `--static` or [target.].linkage. (There is no + // [build].linkage — the parser only reads it under a target section.) + if (isMuslTc && state.m->buildConfig.linkage.empty()) { + state.m->buildConfig.linkage = "static"; + } +} + +std::expected phase2_define_toolchain_resolver(PrepareState& state) { + state.resolve_target_toolchain = [&]() -> std::expected { + ToolchainResolveCtx ctx; + if (auto r = step2_parse_toolchain_spec(state, ctx); !r) return std::unexpected(r.error()); + + if (ctx.installedPin) { + step2_use_installed_pin(state, ctx); + } else if (state.tcSpecIsMsvc) { + if (auto r = step2_use_system_msvc(state); !r) return std::unexpected(r.error()); + } else if (ctx.parsedSpec) { + if (auto r = step2_resolve_explicit_spec(state, ctx); !r) return std::unexpected(r.error()); + } else if (state.tcSpec.has_value() && *state.tcSpec == "system") { + if (auto r = step2_system_toolchain_refusal(state); !r) return std::unexpected(r.error()); + } else if (mcpp::platform::env::offline_mode() + || mcpp::platform::env::no_auto_install()) { + if (auto r = step2_offline_refusal(state); !r) return std::unexpected(r.error()); + } else { + if (auto r = step2_first_run_auto_install(state); !r) return std::unexpected(r.error()); + } + + step2_windows_gnu_first_run_persist(state); + + // AND NOW RESOLVE FOR THE TARGET, IF ONE WAS ASKED FOR. + // + // The first-run branch above answers "this machine has no toolchain, give + // it one", and the answer is a HOST payload; `--target` was never read + // there. On a machine that had never built anything, + // `mcpp build --target x86_64-windows-gnu` therefore installed a native + // gcc and compiled Windows sources with it — measured in CI 2026-08-25: + // + // First run no toolchain configured — installing gcc@16.1.0 … + // Resolved gcc@16.1.0 → …/xim-x-gcc/16.1.0/bin/g++ + // ↑ no target in the path + // + // against the same command where one already existed: + // + // Resolved gcc@16.1.0 → x86_64-windows-gnu → …/mingw-cross-gcc/… + // + // REUSES THE PATH THAT ALREADY KNOWS HOW rather than repeating it. The + // default just chosen is the spec; mapping a spec plus a target onto a + // payload (installing it if absent — `autoInstall` was always true there) + // is what the top of this function does. Depth is one: the second pass + // takes the `tcSpec.has_value()` branch the first run just made true. + // ONE-SHOT, AND THE FLAG IS SET BEFORE THE CALL, NOT AFTER. + // + // This line sits OUTSIDE the first-run branch — it has to, because the + // Windows block just above sets the target itself — so it is evaluated on + // every pass. The first version relied on `firstRunNeedsTargetPass` being + // false on the second pass; it is a captured variable that nothing + // resets, so every pass recursed again. Measured in a consumer's CI as + // the same `Resolved` line four times and then + // + // ##[error]Process completed with exit code 139 + // + // — SIGSEGV, a stack that ran out. A recursion whose termination depends + // on state the recursive call does not change is not a depth-one + // recursion, however its comment reads. + if (!state.targetPassDone + && (state.firstRunNeedsTargetPass + || (state.windowsGnuFirstRun && state.tcSpec.has_value()))) { + state.targetPassDone = true; + return state.resolve_target_toolchain(); } + + if (auto r = step2_detect_toolchain(state); !r) return std::unexpected(r.error()); + if (auto r = step2_retarget_for_retargetable_driver(state); !r) return std::unexpected(r.error()); + if (auto r = step2_bind_msvc_toolset(state); !r) return std::unexpected(r.error()); + step2_windows_runtime_identity(state); + if (auto r = step2_msvc_abi_without_msvc_repair(state); !r) return std::unexpected(r.error()); + step2_musl_default_static_linkage(state); + return {}; }; diff --git a/src/build/prepare/toolchain_env.cpp b/src/build/prepare/toolchain_env.cpp index c4b16d42d..4e1e5bf56 100644 --- a/src/build/prepare/toolchain_env.cpp +++ b/src/build/prepare/toolchain_env.cpp @@ -194,6 +194,10 @@ bind_msvc_sysroot(mcpp::toolchain::Toolchain& tc, tc.msvcToolsVersion = choice->version; tc.msvcOrigin = origin; tc.msvcProduct = choice->product; + // The toolset's own redistributable CRT (#718), reached from the + // sysroot rather than from a cl.exe path — this row runs no cl.exe. + tc.msvcRedistDir = msvc::vc_redist_dir_for_tools_dir( + choice->toolsDir, tt->arch); if (sdk.sdk) { tc.windowsSdkRoot = sdk.sdk->root; tc.windowsSdkVersion = sdk.sdk->version; diff --git a/src/build/prepare/xlings.cpp b/src/build/prepare/xlings.cpp index 5b5adcd1c..7edc35065 100644 --- a/src/build/prepare/xlings.cpp +++ b/src/build/prepare/xlings.cpp @@ -54,7 +54,11 @@ import mcpp.log; namespace mcpp::build { -std::expected phase3_xlings_before_graph(PrepareState& state) { +// STEP FUNCTION (mcpp#722 / T6 follow-on): materializing root +// generated_files, the host-toolchain closures phase3 assigns onto +// state, and the index-refresh section, extracted verbatim. +static std::expected +step3_define_host_tc_closures_and_refresh_index(PrepareState& state) { // Sysroot comes from the toolchain payload itself (GCC -print-sysroot, // Clang clang++.cfg). mcpp does not override it — the payload is @@ -348,6 +352,13 @@ std::expected phase3_xlings_before_graph(PrepareState& state) } } } + return {}; +} + +std::expected phase3_xlings_before_graph(PrepareState& state) { + if (auto r = step3_define_host_tc_closures_and_refresh_index(state); !r) + return std::unexpected(r.error()); + // Set up project-level .mcpp/ directory for custom indices and/or the // [xlings] build environment (L-1). This creates .mcpp/.xlings.json with @@ -429,7 +440,17 @@ std::expected phase3_xlings_before_graph(PrepareState& state) penv.workspace.emplace_back(entry.target, entry.pin()); } } - if (state.runtimeSelection.ownerRoot == state.workRoot) { + // Two halves, two roots. The custom-indices half belongs to + // `state.workRoot`, where this invocation writes. The runtime- + // environment half (`penv`: deps/subos/workspace) belongs to the + // runtime's owner, `runtimeSelection.ownerRoot`: the workspace + // root when a member builds (e2e 205), the project root otherwise. + // Under `plan_only` (`emit build-database`) nothing is written + // into the project (SPEC-005 R2.1, mcpp#724 side finding B, e2e + // 817), so the owner's half goes to the planning directory too. + const auto& runtimeRoot = state.overrides.plan_only + ? state.workRoot : state.runtimeSelection.ownerRoot; + if (runtimeRoot == state.workRoot) { mcpp::config::ensure_project_index_dir( **cfg2, state.workRoot, state.m->indices, penv); } else { @@ -438,7 +459,7 @@ std::expected phase3_xlings_before_graph(PrepareState& state) **cfg2, state.workRoot, state.m->indices, {}); if (materializeRootRuntime) mcpp::config::ensure_project_index_dir( - **cfg2, state.runtimeSelection.ownerRoot, {}, penv); + **cfg2, runtimeRoot, {}, penv); } // `[xlings] deps` are DECLARED above and, until now, nothing diff --git a/src/build/prepare_inputs.cppm b/src/build/prepare_inputs.cppm index 78ef7cc11..0cfaaf068 100644 --- a/src/build/prepare_inputs.cppm +++ b/src/build/prepare_inputs.cppm @@ -517,8 +517,16 @@ std::filesystem::path target_dir(const mcpp::toolchain::Toolchain& tc, // Exported so the "every build-variant knob is in here" invariant is machine- // checkable: the profile knobs were absent for a long time precisely because // nothing could assert on this string. +// `includeDialectFlags`: false when this call serves the PER-PACKAGE +// fingerprint loop (`canonical_package_build_metadata` below) for a package +// that is not necessarily this build's root. `dialectCxxflags` is graph-wide +// (types.cppm's BuildConfig::dialectCxxflags): only the root's value reaches +// any command, so only the root's value may enter the fingerprint, and only +// once (#717 design 2026-09-27 §6.2). The default keeps this the SAME call +// the direct root-only call site below already makes. std::string canonical_compile_flags(const mcpp::manifest::Manifest& m, - bool targetIsMacos = false) { + bool targetIsMacos = false, + bool includeDialectFlags = true) { std::string s; s += "-std="; s += m.package.standard; s += " -fmodules"; @@ -565,9 +573,19 @@ std::string canonical_compile_flags(const mcpp::manifest::Manifest& m, } // Explicit [build] dialect_cxxflags (auto-promoted ones are already in // cxxflags above) — they change every BMI in the graph. - for (auto const& flag : m.buildConfig.dialectCxxflags) { - s += " dialect:"; - s += flag; + // + // GATED: this is graph-wide (only the root's value reaches a command, + // BuildConfig::dialectCxxflags's own comment), so it belongs in the + // fingerprint only where `m` is known to be the root -- the direct call + // below, not the per-package loop of `canonical_package_build_metadata`, + // which calls this for every dependency too (#717 design §6.2, finding 7: + // a dependency's own value used to enter ITS fingerprint although it + // reaches no command). + if (includeDialectFlags) { + for (auto const& flag : m.buildConfig.dialectCxxflags) { + s += " dialect:"; + s += flag; + } } for (auto const& flag : m.buildConfig.ldflags) { s += " ldflag:"; @@ -663,8 +681,18 @@ std::string canonical_package_build_metadata( // packages[0] is the root, whose flags `canonical_compile_flags` // already folds; serialising it twice is harmless and keeps this loop // one rule rather than one rule and an exception. + // + // EXCEPT for `dialect_cxxflags` (#717 design §6.2, finding 7): that + // key is graph-wide, so a dependency's own value must not enter ITS + // fingerprint contribution, and the root's must enter the fingerprint + // exactly once -- through the DIRECT root-only call this function's + // caller already makes on the root manifest (`canonical_compile_flags + // (*state.m, ...)`, scan.cpp), not through this per-package loop, + // where `includeDialectFlags = false` for every entry including the + // root. s += ' '; - s += canonical_compile_flags(pkg.manifest, targetIsMacos); + s += canonical_compile_flags(pkg.manifest, targetIsMacos, + /*includeDialectFlags=*/false); // The level a C++-layer provider compiles its implementation units at // (`make_plan`). Appended only when there is one, so every other // output directory keeps its identity. diff --git a/src/build/refusal.cppm b/src/build/refusal.cppm index 0a96cd539..cebf2ff4f 100644 --- a/src/build/refusal.cppm +++ b/src/build/refusal.cppm @@ -149,6 +149,12 @@ enum class Code { // `[build] platform-dependencies = "refuse"` and a package in the graph // brings a platform SDK dependency (design §6). PlatformDependency, + // An explicit `cxx_runtime = "toolchain-coupled"` on an MSVC-ABI row + // whose resolved toolset carries no `VC\Redist\MSVC` directory (#718). + // Distinct from every other CRT-model message: the contract is not + // downgraded here, because an explicit statement a toolset cannot meet is + // an error, not a default to fall back from. + MsvcRedistUnavailable, Other, // a refusal that has not been given a code yet }; @@ -193,6 +199,8 @@ constexpr std::string_view name(Code c) { return "c-env-verification-mismatch"; case Code::InterfaceNotProvided: return "interface-not-provided"; case Code::PlatformDependency: return "platform-dependency"; + case Code::MsvcRedistUnavailable: + return "msvc-redist-unavailable"; case Code::Other: return "other"; } return "other"; diff --git a/src/build/stage.cppm b/src/build/stage.cppm index cba4bad79..7ee0b56eb 100644 --- a/src/build/stage.cppm +++ b/src/build/stage.cppm @@ -75,6 +75,30 @@ std::expected stage_file(const std::filesystem::path& const std::filesystem::path& dst, const StageOptions& opts = {}); +// Publish one destination that may have more than one source (SPEC-007 R4.2, +// mcpp#723): two or more packages of one graph can each generate a +// byte-identical file and deploy it under the same name, and the plan no +// longer refuses that at planning time — a generated source may not exist yet +// when the plan is built, so its content cannot be compared there. This is +// where the invariant is actually checked, because by build time every source +// exists: every source is compared against the first by content +// (`same_content`, never masking the way `bmi_equivalent` does — a deploy +// target is an ordinary file, not a BMI with a compiler-embedded clock), and +// the first is staged when they all agree. Disagreement is refused, naming +// every source and the destination, so the message points at every producer +// instead of an arbitrary pair. +// +// `srcs.size() == 1` is not a special case bolted on top: it takes the same +// path as every other count, with the comparison loop simply empty, and ends +// up calling `stage_file` on that one source exactly as before this function +// existed. This is what keeps a project with one source per destination — +// every project before this feature — emitting the same `mcpp stage` +// invocation it always has. +std::expected stage_files( + const std::vector& srcs, + const std::filesystem::path& dst, + const StageOptions& opts = {}); + // Byte-for-byte comparison (exported for tests). False when either file is // unreadable or the sizes differ. bool same_content(const std::filesystem::path& a, const std::filesystem::path& b); @@ -337,4 +361,36 @@ std::expected stage_file(const std::filesystem::path& return std::unexpected(StageError{failure_message(src, dst, last)}); } +std::expected stage_files( + const std::vector& srcs, + const std::filesystem::path& dst, + const StageOptions& opts) +{ + if (srcs.empty()) { + return std::unexpected(StageError{ + std::format("staging '{}' requires at least one source", dst.string())}); + } + // A source that is not there is named as such: the step that should have + // written it failed or does not exist, which is not a disagreement. + if (srcs.size() > 1) + for (auto const& s : srcs) { + std::error_code ec; + if (!std::filesystem::is_regular_file(s, ec)) + return std::unexpected(StageError{std::format( + "a source of '{}' does not exist: {}", dst.string(), s.string())}); + } + for (std::size_t i = 1; i < srcs.size(); ++i) { + if (!same_content(srcs.front(), srcs[i])) { + std::string list; + for (auto const& s : srcs) list += std::format("\n {}", s.string()); + return std::unexpected(StageError{std::format( + "two or more sources disagree for one destination\n" + " destination: {}\n" + " sources:{}", + dst.string(), list)}); + } + } + return stage_file(srcs.front(), dst, opts); +} + } // namespace mcpp::build::stage diff --git a/src/cli.cppm b/src/cli.cppm index 7df677d55..6c1ccedd7 100644 --- a/src/cli.cppm +++ b/src/cli.cppm @@ -375,7 +375,7 @@ int run(int argc, char** argv) { .option(cl::Option("static").help( "Force static linking (-static). On Linux, prefer pairing with --target -linux-musl")) .option(cl::Option("package").short_name('p').takes_value().value_name("NAME") - .help("Build only the named workspace member")) + .help("Build only the named workspace member (namespace.name or package name, then directory)")) .option(cl::Option("profile").takes_value().value_name("NAME") .help("Build profile: dev (default) | release | dist | <[profile.*] name>")) .option(cl::Option("release").help("Shorthand for --profile release")) @@ -419,7 +419,7 @@ int run(int argc, char** argv) { .option(cl::Option("target-triple").takes_value().value_name("TRIPLE") .help("Alias for --target")) .option(cl::Option("package").short_name('p').takes_value().value_name("NAME") - .help("Run only the named workspace member (single-member; no --workspace fan-out)")) + .help("Run only the named workspace member (namespace.name or package name, then directory; single-member, no --workspace fan-out)")) // DECLARED ON THE THREE COMMANDS THAT BUILD BEFORE THEY ACT, AS ON // `build`. The value has always reached them: the pre-parse loop // above publishes it as MCPP_TOOLCHAIN for every command, and @@ -519,7 +519,7 @@ int run(int argc, char** argv) { .option(cl::Option("strict") .help("Treat manifest schema warnings (unknown feature/platform) as errors")) .option(cl::Option("package").short_name('p').takes_value().value_name("NAME") - .help("Run tests only for the named workspace member")) + .help("Run tests only for the named workspace member (namespace.name or package name, then directory)")) .option(cl::Option("toolchain").takes_value().value_name("SPEC") .help("Build the tests with this toolchain for one invocation, e.g. llvm@22.1.8")) .option(cl::Option("cache").takes_value().value_name("MODE") @@ -684,7 +684,7 @@ int run(int argc, char** argv) { .help("Describe the variant built for no accelerator")) .option(cl::Option("static").help("Describe the build with --static")) .option(cl::Option("package").short_name('p').takes_value().value_name("NAME") - .help("Describe only the named workspace member")) + .help("Describe only the named workspace member (namespace.name or package name, then directory)")) .option(cl::Option("profile").takes_value().value_name("NAME") .help("Build profile: dev (default) | release | dist | <[profile.*] name>")) .option(cl::Option("release").help("Shorthand for --profile release")) @@ -1304,6 +1304,9 @@ int run(int argc, char** argv) { return 2; } app.run(*parsed); + // The run's closing notices come last, after everything the command + // printed, whatever its exit status (see mcpp::ui::add_closing_notice). + mcpp::ui::print_closing_notices(); return action_rc; } diff --git a/src/cli/cmd_build.cppm b/src/cli/cmd_build.cppm index e26940cbc..df5dc6716 100644 --- a/src/cli/cmd_build.cppm +++ b/src/cli/cmd_build.cppm @@ -281,6 +281,12 @@ export int cmd_emit_build_database(const mcpplibs::cmdline::ParsedArgs& parsed) const mcpp::build::BuildOverrides ov = overrides_from_selectors(parsed); std::vector diagnostics; + // The run's closing notices belong in the envelope as `note` diagnostics; + // taken here, they are not printed again as `tip:` lines at exit. + auto take_closing_notes = [&] { + for (auto& n : mcpp::ui::take_closing_notices()) + diagnostics.push_back({std::move(n.code), Severity::Note, std::move(n.message)}); + }; auto publish = [&](const std::string& text) -> int { if (!outputPath) { std::print("{}", text); return 0; } const std::filesystem::path out{*outputPath}; @@ -316,6 +322,7 @@ export int cmd_emit_build_database(const mcpplibs::cmdline::ParsedArgs& parsed) mcpp::wire::severity_name(d.severity), d.message); return 1; } + take_closing_notes(); const auto text = mcpp::wire::to_json(mcpp::wire::Envelope{ .kind = "mcpp.build-database", .effects = {Effect::ReadProject}, @@ -400,10 +407,28 @@ export int cmd_emit_build_database(const mcpplibs::cmdline::ParsedArgs& parsed) includeDevDeps, std::move(discovered->targets), mo); if (!ctx) { + // A wholly-failed member contributes exactly one `error` + // diagnostic, `path` its `mcpp.toml` (SPEC-005 R5.2) — that + // invariant is kept exactly, so a note an earlier phase + // recorded (most importantly + // `MCPP_BUILD_DATABASE_PROGRAM_FAILED`) is folded into THIS + // diagnostic's own message instead of becoming a diagnostic of + // its own. Without it, a later phase's failure that follows + // from the missing directives (SPEC-005 R5.2's own words) read + // as a single, unexplained symptom, and the actual cause — + // recorded, then discarded the moment `prepare_build` returned + // — never reached the reader (design 2026-09-27 §4.2, mcpp#724 + // side finding A, fix item 2). + std::string message = member.empty() ? ctx.error() + : std::format("{}: {}", member, ctx.error()); + for (auto const& note : mcpp::build::take_notes_on_failure()) + message += note.path.empty() + ? std::format("\n earlier in this pass, {}: {}", + note.code, note.message) + : std::format("\n earlier in this pass, {} ({}): {}", + note.code, note.path, note.message); diagnostics.push_back({plan_failure_code(), Severity::Error, - member.empty() ? ctx.error() - : std::format("{}: {}", member, ctx.error()), - memberPath}); + std::move(message), memberPath}); failedMemberRoots.push_back(memberRoot); continue; } @@ -482,6 +507,7 @@ export int cmd_emit_build_database(const mcpplibs::cmdline::ParsedArgs& parsed) if (const auto rc = publish(document.dump(2) + "\n"); rc != 0) return rc; return hasError ? 1 : 0; } + take_closing_notes(); std::vector effects{Effect::ReadProject, Effect::WriteGlobalCache}; if (ranBuildPrograms) effects.push_back(Effect::ExecBuildScript); nlohmann::json specJson{{"name", spec}}; @@ -906,19 +932,25 @@ export int cmd_dyndep(const mcpplibs::cmdline::ParsedArgs& parsed) { } // Invoked by ninja during build (stage_file rule): -// mcpp stage --output +// mcpp stage --output ... // // Publishes a cache-owned artifact (std BMI, std.o, runtime DLL) into the // build directory. See mcpp.build.stage for the semantics — in particular why // an already-equivalent destination is left untouched (#311). +// +// More than one source (SPEC-007 R4.2, mcpp#723) means two or more packages +// of this graph deploy the same destination; `stage_files` places it when +// every source is byte-identical and otherwise fails, naming every source +// and the destination. One source — every invocation before this feature — +// takes the exact path it always has. export int cmd_stage(const mcpplibs::cmdline::ParsedArgs& parsed) { std::filesystem::path outPath = parsed.option_or_empty("output").value(); if (outPath.empty()) { std::println(stderr, "error: --output required"); return 2; } - if (parsed.positional_count() != 1) { - std::println(stderr, "error: stage requires exactly one source path"); + if (parsed.positional_count() < 1) { + std::println(stderr, "error: stage requires at least one source path"); return 2; } @@ -931,9 +963,13 @@ export int cmd_stage(const mcpplibs::cmdline::ParsedArgs& parsed) { if (!verify.empty()) opts.verify = mcpp::build::stage::parse_verify(verify); - auto r = mcpp::build::stage::stage_file( - mcpp::platform::fs::extended_length(std::filesystem::path{parsed.positional(0)}), - mcpp::platform::fs::extended_length(outPath), opts); + std::vector sources; + for (std::size_t i = 0; i < parsed.positional_count(); ++i) + sources.push_back(mcpp::platform::fs::extended_length( + std::filesystem::path{parsed.positional(i)})); + + auto r = mcpp::build::stage::stage_files( + sources, mcpp::platform::fs::extended_length(outPath), opts); if (!r) { std::println(stderr, "error: {}", r.error().message); return 1; diff --git a/src/cli/cmd_publish.cppm b/src/cli/cmd_publish.cppm index c5a16be19..a92b4d52e 100644 --- a/src/cli/cmd_publish.cppm +++ b/src/cli/cmd_publish.cppm @@ -36,11 +36,18 @@ export int cmd_publish(const mcpplibs::cmdline::ParsedArgs& parsed) { parsed.is_flag_set("dry-run"), parsed.is_flag_set("allow-dirty")); } -// `mcpp place-dlls --output --depfile ...` -- the -// edge that follows a Windows program's link when its plan has runtime search -// directories (mcpp.pack's `place_runtime_dlls`, SPEC-007 R4.3). Internal: -// only a generated build.ninja names it, and it runs on whatever host builds, -// because it reads the program's import table rather than asking a loader. +// `mcpp place-dlls --output --depfile ...` +// -- the edge that follows a Windows program's link when its plan has runtime +// search directories (mcpp.pack's `place_runtime_dlls`, SPEC-007 R4.3). +// Internal: only a generated build.ninja names it, and it runs on whatever +// host builds, because it reads the program's import table rather than asking +// a loader. +// +// The command line names no DLL. Which DLLs beside the program belong to +// another writer (a declared deploy, the toolchain's staged runtime) is +// decided below, from the program's directory, the stamp and the search +// directories, so that the command does not change when the plan's deploy set +// does (SPEC-007 R4.2/R4.3). // // The depfile names every DLL placed, so ninja runs the edge again when one of // them changes in its directory; the stamp is the edge's only declared output, @@ -50,7 +57,8 @@ export int cmd_place_dlls(const mcpplibs::cmdline::ParsedArgs& parsed) { const std::filesystem::path stamp{parsed.option_or_empty("output").value()}; const std::filesystem::path depfile{parsed.option_or_empty("depfile").value()}; if (stamp.empty() || depfile.empty() || parsed.positional_count() < 1) { - std::println(stderr, "error: place-dlls requires --output, --depfile and a program"); + std::println(stderr, + "error: place-dlls requires --output, --depfile and a program"); return 2; } const std::filesystem::path program{parsed.positional(0)}; @@ -67,12 +75,43 @@ export int cmd_place_dlls(const mcpplibs::cmdline::ParsedArgs& parsed) { for (std::string line; std::getline(prev, line);) if (!line.empty()) placedBefore.push_back(line); } - auto placed = mcpp::pack::place_runtime_dlls(program, dirs, placedBefore); + // ONE DESTINATION, ONE WRITER (SPEC-007 R4.3, #723). A DLL already beside + // the program that this edge did not place, and that a runtime search + // directory also offers, is another writer's: a declared deploy or the + // toolchain's staged runtime, both completed before the link this edge + // follows. It is never overwritten; `place_runtime_dlls` compares it with + // the directory's copy and warns on a difference. A DLL only the + // program's directory holds (a library the project built there) is not + // one this edge could write, and stays an ordinary member of the closure. + // Decided here, from the directories, so the edge's command does not + // change when the plan's deploy set does. + std::vector placedByOthers; + { + std::error_code dirEc; + const auto here = program.has_parent_path() ? program.parent_path() + : std::filesystem::path("."); + for (auto const& e : std::filesystem::directory_iterator(here, dirEc)) { + if (!e.is_regular_file(dirEc)) continue; + auto ext = e.path().extension().string(); + std::ranges::transform(ext, ext.begin(), + [](unsigned char c) { return static_cast(std::tolower(c)); }); + if (ext != ".dll") continue; + const auto name = e.path().filename().string(); + if (std::ranges::find(placedBefore, name) != placedBefore.end()) continue; + const bool offered = std::ranges::any_of(dirs, [&](auto const& d) { + std::error_code fe; + return std::filesystem::is_regular_file(d / name, fe); + }); + if (offered) placedByOthers.push_back(name); + } + } + auto placed = mcpp::pack::place_runtime_dlls(program, dirs, placedBefore, placedByOthers); if (!placed) { std::println(stderr, "error: {}", placed.error().message); return 1; } for (auto const& n : placed->notes) std::println("note: {}", n); + for (auto const& w : placed->warnings) std::println(stderr, "warning: {}", w); // The depfile syntax ninja reads (`deps = gcc`): a space and `#` are // escaped with a backslash, and `$` is doubled. diff --git a/src/doctor.cppm b/src/doctor.cppm index f57a9bf66..2e9629b07 100644 --- a/src/doctor.cppm +++ b/src/doctor.cppm @@ -32,6 +32,8 @@ import mcpp.platform.process; import mcpp.platform.env; import mcpp.runtime.elf; import mcpp.pm.index_refresh; // staleness_note for `mcpp why deps` +import mcpp.pm.index_contract; // the floor each index declares +import mcpp.version; // MCPP_VERSION, compared with each floor import mcpp.project; import mcpp.toolchain.detect; import mcpp.toolchain.msvc; @@ -367,6 +369,35 @@ export int doctor_report() { (*cfg).xlingsBinary.string())); } ok(std::format("default index = '{}'", (*cfg).defaultIndex)); + + // Every index tree this mcpp reads, against the floor it declares. + // A run never reports this as an error (an index is data, and a run + // that resolves what it needs has nothing to report), so doctor is + // where the state is visible without a failing run. + mcpp::ui::status("Checking", "package indexes"); + const auto dataRoot = mcpp::xlings::paths::index_data( + mcpp::config::make_xlings_env(*cfg)); + std::error_code idxEc; + bool anyIndex = false; + for (auto const& entry : std::filesystem::directory_iterator(dataRoot, idxEc)) { + if (!entry.is_directory(idxEc)) continue; + if (!std::filesystem::exists(entry.path() / "pkgs", idxEc)) continue; + anyIndex = true; + const auto name = entry.path().filename().string(); + auto contract = mcpp::pm::read_index_contract(entry.path()); + if (!contract || contract->minMcpp.empty()) { + ok(std::format("index '{}' declares no mcpp floor", name)); + } else if (mcpp::pm::index_usable(entry.path())) { + ok(std::format("index '{}' requires mcpp >= {}", name, contract->minMcpp)); + } else { + warn(std::format( + "index '{}' requires mcpp >= {}, and this is mcpp {}; the " + "packages it serves cannot be resolved until mcpp is " + "upgraded (xlings update mcpp). See `mcpp explain E0006`", + name, contract->minMcpp, mcpp::MCPP_VERSION)); + } + } + if (!anyIndex) ok("no package index synced yet"); } mcpp::ui::status("Checking", "cache health"); diff --git a/src/fetcher/progress.cppm b/src/fetcher/progress.cppm index 351981744..02d75bb58 100644 --- a/src/fetcher/progress.cppm +++ b/src/fetcher/progress.cppm @@ -191,6 +191,30 @@ std::vector to_ui_download_files(const std::vector return out; } +// One line of git's `--progress` output: the phase and its percentage. +// `remote: Counting objects: 40% (2/5)` and `Receiving objects: 100% (5/5), +// 1.20 MiB | 800.00 KiB/s, done.` both parse; the phase has no `remote: ` +// prefix. Any other line (`Cloning into ...`, an error) yields nullopt. +export struct GitProgress { + std::string phase; + std::size_t percent = 0; +}; +export std::optional parse_git_progress(std::string_view line) { + if (line.starts_with("remote: ")) line.remove_prefix(8); + const auto colon = line.find(':'); + if (colon == std::string_view::npos || colon == 0) return std::nullopt; + const auto pct = line.find('%', colon); + if (pct == std::string_view::npos) return std::nullopt; + auto begin = pct; + while (begin > colon + 1 && line[begin - 1] >= '0' && line[begin - 1] <= '9') --begin; + if (begin == pct) return std::nullopt; + GitProgress g; + g.phase = std::string(line.substr(0, colon)); + std::from_chars(line.data() + begin, line.data() + pct, g.percent); + if (g.percent > 100) g.percent = 100; + return g; +} + // Adapter from `mcpp::config::BootstrapProgress` (xlings download_progress // event) to the centralized download renderer. Used by load_or_init() during // the one-time sandbox bootstrap (xim:patchelf, xim:ninja + transitive deps). diff --git a/src/pack/pack.cppm b/src/pack/pack.cppm index 1cc46363e..abb43932f 100644 --- a/src/pack/pack.cppm +++ b/src/pack/pack.cppm @@ -387,16 +387,26 @@ ClosureRead read_closure(const ClosureReadInput& in); // copy this function put beside the program is resolved again from its // directory rather than taken for a file of the program's own; `notes` names // each DLL that more than one directory offers, with the one the search order -// chose. +// chose; `warnings` names a DLL this function did NOT place because +// `placedByOthers` already claims that name, together with the difference +// found (SPEC-007 R4.2/R4.3: one destination, one writer, #723). struct RuntimeDllPlacement { std::vector sources; std::vector names; std::vector notes; + std::vector warnings; }; +// `placedByOthers` names the DLLs another writer already put directly beside +// `program` (a declared deploy, or the toolchain's staged runtime; the caller +// determines them from the directory, see cmd_place_dlls): a name in it is +// never written here. When the resolved import differs from what is already +// there, the difference is reported in `warnings` rather than silently kept +// or silently overwritten. std::expected place_runtime_dlls(const std::filesystem::path& program, const std::vector& searchDirs, - const std::vector& placedBefore = {}); + const std::vector& placedBefore = {}, + const std::vector& placedByOthers = {}); // Build a Plan from already-resolved inputs. Caller is expected to have // already run `mcpp build` (or equivalent) and pass the resulting @@ -1377,7 +1387,8 @@ make_tarball(const std::filesystem::path& stagingRoot, std::expected place_runtime_dlls(const std::filesystem::path& program, const std::vector& searchDirs, - const std::vector& placedBefore) + const std::vector& placedBefore, + const std::vector& placedByOthers) { const auto programDir = program.parent_path(); auto same_dir = [](const std::filesystem::path& a, const std::filesystem::path& b) { @@ -1385,6 +1396,15 @@ place_runtime_dlls(const std::filesystem::path& program, if (std::filesystem::equivalent(a, b, ec)) return true; return a.lexically_normal() == b.lexically_normal(); }; + // PE names are case-insensitive, as the loader treats them. + auto lower = [](std::string_view s) { + std::string l(s); + std::ranges::transform(l, l.begin(), + [](unsigned char c) { return std::tolower(c); }); + return l; + }; + std::set deployedNames; + for (auto const& n : placedByOthers) deployedNames.insert(lower(n)); ClosureReadInput in; in.object = program; @@ -1392,7 +1412,18 @@ place_runtime_dlls(const std::filesystem::path& program, in.searchDirs.push_back(programDir.empty() ? std::filesystem::path(".") : programDir); for (auto const& d : searchDirs) if (!same_dir(d, in.searchDirs.front())) in.searchDirs.push_back(d); + // A name the deploy list places is, by the time this edge runs, already + // sitting in the program's own directory (that edge is an order-only + // dependency of the link this edge reads). Left alone, the closure + // resolution below would find that very copy in `searchDirs.front()` and + // treat the name as already resolved, so the runtime search directories' + // copy — the one to compare against — would never be looked at. Folding + // `placedByOthers` into `notInFirstDir` forces resolution from the OTHER + // search directories instead, exactly as it already does for a name THIS + // function placed on a previous run. in.notInFirstDir = placedBefore; + in.notInFirstDir.insert(in.notInFirstDir.end(), + placedByOthers.begin(), placedByOthers.end()); const auto read = read_closure(in); // The program itself is the one object the caller chose, so a program that @@ -1406,6 +1437,25 @@ place_runtime_dlls(const std::filesystem::path& program, RuntimeDllPlacement out; for (auto const& m : read.members) { if (same_dir(m.source.parent_path(), in.searchDirs.front())) continue; + + // SPEC-007 R4.2/R4.3: one destination, one writer. A name the merged + // deploy list already places beside this program is that list's + // file, not this mechanism's — `add_deploy`'s content check (`mcpp + // stage`) is the authority for it. This loop never writes over it; it + // only compares and, on a real difference, warns. + if (deployedNames.contains(lower(m.name))) { + const auto existing = programDir / m.name; + std::error_code ec; + if (std::filesystem::is_regular_file(existing, ec) + && !mcpp::build::stage::same_content(m.source, existing)) { + out.warnings.push_back(std::format( + "'{}' is placed by this project's deploy list; the runtime " + "search directories also offer a different '{}', which was " + "not used", existing.string(), m.source.string())); + } + continue; + } + auto staged = mcpp::build::stage::stage_file(m.source, programDir / m.name); if (!staged) return std::unexpected(Error{std::format( diff --git a/src/pack/pipeline.cppm b/src/pack/pipeline.cppm index 4f850fcb0..e2e9749d0 100644 --- a/src/pack/pipeline.cppm +++ b/src/pack/pipeline.cppm @@ -457,7 +457,21 @@ export PackOutcome build_and_pack(Options opts, bool modeFromUser, flags.contractByRole[static_cast( mcpp::build::dist::Role::Distributable)] == mcpp::build::dist::Contract::ToolchainCoupled; + // AN EXPLICIT `--mode system` OUTRANKS A DEFAULTED CONTRACT (#718, + // §7.3): the MSVC-ABI default is toolchain-coupled, and asking for + // `--mode system` on a project that never wrote `cxx_runtime` down is + // an explicit choice for host-coupled, not a contradiction — the + // contradiction is reserved for a manifest that SAID + // toolchain-coupled and a mode that bundles nothing (checked below, + // unchanged). + if (opts.mode == mcpp::pack::Mode::None + && opts.carryToolchainRuntime + && !flags.programCxxRuntimeStated) { + opts.carryToolchainRuntime = false; + } opts.toolchainRuntimeDirs = ctx->plan.toolchain.linkRuntimeDirs; + if (!ctx->plan.toolchain.msvcRedistDir.empty()) + opts.toolchainRuntimeDirs.push_back(ctx->plan.toolchain.msvcRedistDir); // Where a third-party dependency's shared library may be found. Both // channels, because they answer for different things: the runtime // library dirs are what `mcpp run` puts on the loader's path, and the diff --git a/src/pm/index_contract.cppm b/src/pm/index_contract.cppm index 8018da906..1cc429028 100644 --- a/src/pm/index_contract.cppm +++ b/src/pm/index_contract.cppm @@ -52,6 +52,11 @@ floor_violation(std::string_view minMcpp, std::string_view ownVersion); // running binary. std::string e0006_message(std::string violation, bool distroManaged); +// The closing half of an index-floor notice: the version the index asks for +// (omitted when unknown) and the upgrade command for this install layout, +// the same advice E0006 gives (`distro_managed_install`). +std::string index_floor_upgrade_advice(std::string_view minMcpp); + // Pure predicate — no reporting, no registration, no dedup. For callers that // need to ask "would this tree be usable?" without the side effects of // check_index_floor (the refresh guard asks it twice per refresh). @@ -170,7 +175,8 @@ std::string e0006_message(std::string violation, bool distroManaged) if (pos != std::string::npos) violation.replace(pos, kInstallShUpgrade.size(), kDistroUpgrade); } - // Append the recommended installer note to every layout. + // Append the recommended installer note to every layout, on its own line. + if (!violation.empty() && violation.back() != '\n') violation += '\n'; violation += kXlingsUpgrade; return violation; } @@ -189,6 +195,17 @@ bool distro_managed_install() #endif } +std::string index_floor_upgrade_advice(std::string_view minMcpp) +{ + const std::string_view how = distro_managed_install() + ? "update the mcpp-bin package with your AUR helper" + : "xlings update mcpp"; + return minMcpp.empty() + ? std::format("Upgrade: {}", how) + : std::format("It requires mcpp >= {}; this is mcpp {}. Upgrade: {}", + minMcpp, mcpp::MCPP_VERSION, how); +} + bool index_usable(const std::filesystem::path& indexRoot) { if (const char* v = std::getenv("MCPP_INDEX_FLOOR"); @@ -221,14 +238,22 @@ std::string unusable_index_hint() { auto& reg = unusable_registry(); if (reg.empty()) return {}; // Name the index, not just the fact: with several repos configured, "an - // index was too new" leaves the reader guessing which one to act on. + // index was too new" leaves the reader guessing which one to act on. The + // E0006 text travels here, in the message that stops the run, because the + // read site no longer prints it (see read_identity_verified_xpkg_lua). std::string s = "note: this resolve ran with an index this mcpp cannot read:\n"; + std::vector texts; for (auto& u : reg) { s += " " + u.root.string() + "\n"; + if (std::ranges::find(texts, u.message) == texts.end()) + texts.push_back(u.message); + } + for (auto const& t : texts) { + s += t; + if (!t.empty() && t.back() != '\n') s += '\n'; } - s += " Packages served by it were reported as not found. See the " - "[E0006] error above,\n" - " or run `mcpp explain E0006`."; + s += " Packages served by it were reported as not found. " + "Run `mcpp explain E0006` for the details."; return s; } diff --git a/src/pm/index_snapshot.cppm b/src/pm/index_snapshot.cppm index 1b1f60fc6..f6fcab1d8 100644 --- a/src/pm/index_snapshot.cppm +++ b/src/pm/index_snapshot.cppm @@ -124,6 +124,9 @@ struct GuardOutcome { std::vector recovered; // Index trees left unusable — nothing local could serve them. std::vector stillUnusable; + // The `min_mcpp` of each refreshed tree found unusable, read before a + // rollback replaces it, so a notice can name the version it asks for. + std::map requiredMcpp; bool degraded() const { return !rolledBack.empty() || !recovered.empty() || !stillUnusable.empty(); @@ -329,6 +332,8 @@ int guarded_refresh(const std::filesystem::path& dataRoot, prune(dataRoot, dir); continue; } + if (auto c = mcpp::pm::read_index_contract(dir)) + out.requiredMcpp[dir] = c->minMcpp; auto wasUsable = usableBefore.find(dir); if (wasUsable != usableBefore.end() && wasUsable->second) { // Got worse. This is the case the invariant exists for. diff --git a/src/pm/package_fetcher.cppm b/src/pm/package_fetcher.cppm index e6286e624..9f6fe5e4f 100644 --- a/src/pm/package_fetcher.cppm +++ b/src/pm/package_fetcher.cppm @@ -693,12 +693,17 @@ read_identity_verified_xpkg_lua(const std::filesystem::path& pkgsDir, // Index→client version contract: the tree carries its own floor // (/index.toml min_mcpp). Checked here — the single choke // point every transport converges on (artifact snapshot, git clone, - // [indices] path, CI-restored cache). Loud once per index; the - // resolve then fails as not-found with the cause already printed. - if (auto violation = mcpp::pm::check_index_floor(pkgsDir.parent_path())) { - mcpp::ui::error(*violation); + // [indices] path, CI-restored cache). + // + // NOTHING IS PRINTED HERE. The fact is recorded, and this tree answers no + // lookup. Whether it matters is decided later: a lookup that then fails + // carries the cause in its own message (unusable_index_hint), and a run + // that succeeds has no error to report. The read site used to print the + // E0006 text as `error:` at the start of runs that went on to resolve every + // package from another tree and exit 0. + if (mcpp::pm::index_marked_unusable(pkgsDir.parent_path()) + || mcpp::pm::check_index_floor(pkgsDir.parent_path())) return std::nullopt; - } std::error_code ec; if (!std::filesystem::exists(pkgsDir, ec)) return std::nullopt; diff --git a/src/project.cppm b/src/project.cppm index c7738791e..4410247ec 100644 --- a/src/project.cppm +++ b/src/project.cppm @@ -12,6 +12,7 @@ export module mcpp.project; import std; import mcpp.manifest; +import mcpp.ui; namespace mcpp::project { @@ -468,35 +469,173 @@ load_effective_manifest(const std::filesystem::path& dir) { return EffectiveManifest{ std::move(*m), std::move(*ws), wsRoot, true }; } +// One declared workspace member, with the identity its own manifest states. +// +// #725: `-p, --package ` promises a package (SPEC-001's `(namespace, +// name)`), so resolving it needs each member's OWN declaration, not only the +// directory `[workspace] members` names it with. +export struct WorkspaceMember { + std::string memberPath; // as written in [workspace] members + std::filesystem::path dir; // rootDir / memberPath + std::string namespace_; // "" when undeclared, or unreadable + std::string name; // "" when the manifest could not be + // read -- such a member still matches + // by directory or path (step 3 below) +}; + +// A member's qualified name the way SPEC-001 spells one: `.`. +// Meaningless (and not attempted by the resolver below) when the member +// declares no namespace -- SPEC-001 §3.1: an unnamespaced package's bare name +// already IS its full identity, so there is no separate dotted spelling to +// match against. +export std::string qualified_member_name(const WorkspaceMember& m) { + return m.namespace_.empty() ? m.name : m.namespace_ + "." + m.name; +} + +// Every declared member, with its own package identity. Loaded the way a +// member is always loaded (`insideWorkspace = true`, so an omitted +// `package.version` is not refused here); a member whose manifest fails to +// parse keeps an empty name rather than aborting the listing, because `-p` +// still owes an answer about the members that DO parse. +export std::vector +workspace_members(const mcpp::manifest::Manifest& rootManifest, + const std::filesystem::path& rootDir) { + std::vector out; + out.reserve(rootManifest.workspace.members.size()); + for (auto& mp : rootManifest.workspace.members) { + WorkspaceMember wm; + wm.memberPath = mp; + wm.dir = rootDir / mp; + if (auto mm = mcpp::manifest::load(wm.dir / "mcpp.toml", {.insideWorkspace = true})) { + wm.namespace_ = mm->package.namespace_; + wm.name = mm->package.name; + } + out.push_back(std::move(wm)); + } + return out; +} + // Resolve which member directory a workspace command acts on, for the -// single-member case. Shares the match rule (basename OR member path) with -// prepare_build's member switch, so `build -p X` and `test -p X` agree. +// single-member case. Shared by every `-p`/`--package` reader (build, test, +// run, emit, and prepare_build's own root-level switch, #725), so +// `build -p X` and `test -p X` always agree on what X means. +// +// A value is resolved in this order, because the option names a package +// (SPEC-001) and a directory is a fallback spelling docs/07 §5.3 has always +// documented: +// 1. a member's qualified name, `.` (only attempted for a +// member that declares a namespace -- see `qualified_member_name`); +// 2. otherwise, a member's bare package name -- refused, naming every +// match's qualified name, when two or more members share it; +// 3. otherwise, a member's path as written in `[workspace] members`, or its +// directory's last segment. Two members sharing a directory basename +// keep today's first-match selection (a script written against it keeps +// working), with a warning naming the others and their paths. +// A value that is one member's package name (step 2) and a different +// member's directory (step 3) selects the step-2 member, with a warning +// naming the other member and its path. +// // Returns: // - the member dir when `package_filter` names a member, // - empty path when no switch applies (not a workspace, or a rooted // workspace with no filter → act on the root package), -// - error when the filter names an unknown member, or a *virtual* -// workspace is addressed with no filter (the caller must -// pick a member with -p or fan out with --workspace). +// - error when the filter names no member, names more than one, +// or a *virtual* workspace is addressed with no filter +// (the caller must pick a member with -p or fan out with +// --workspace). export std::expected resolve_member_dir(const mcpp::manifest::Manifest& rootManifest, const std::filesystem::path& rootDir, std::string_view package_filter) { if (!rootManifest.workspace.present) return std::filesystem::path{}; - if (!package_filter.empty()) { - for (auto& mp : rootManifest.workspace.members) { - auto basename = std::filesystem::path(mp).filename().string(); - if (basename == package_filter || mp == package_filter) - return rootDir / mp; + if (package_filter.empty()) { + if (rootManifest.package.name.empty()) { + return std::unexpected(std::string( + "virtual workspace: specify -p or --workspace")); } + return std::filesystem::path{}; // rooted workspace, no filter → root + } + + auto members = workspace_members(rootManifest, rootDir); + + // Step 1: the qualified name. Unique by construction (SPEC-001 §3.3 asks + // a single index to keep `(namespace, name)` unique; two workspace + // members sharing one is a manifest defect this resolver does not + // adjudicate), so the first hit is taken without a warning. + for (auto const& m : members) + if (!m.namespace_.empty() && qualified_member_name(m) == package_filter) + return m.dir; + + // Step 2: the bare package name, ignoring namespace. + std::vector byName; + for (std::size_t i = 0; i < members.size(); ++i) + if (!members[i].name.empty() && members[i].name == package_filter) + byName.push_back(i); + if (byName.size() > 1) { + std::string names; + for (auto i : byName) + names += (names.empty() ? "" : ", ") + qualified_member_name(members[i]); return std::unexpected(std::format( - "workspace member '{}' not found in [workspace].members", package_filter)); + "-p '{}' is ambiguous: it is the package name of {} members ({}). " + "Write the qualified name (.) to select one.", + package_filter, byName.size(), names)); + } + + // Step 3: the directory path or basename `[workspace] members` writes. + // The path is compared as a path, so `./modules/base/`, and on Windows + // `modules\base`, name the member written `modules/base`. + auto as_path = [](std::string_view s) { + auto p = std::filesystem::path(s).lexically_normal(); + return (!p.has_filename() && p.has_parent_path()) ? p.parent_path() : p; + }; + const auto filterPath = as_path(package_filter); + std::vector byPath; + for (std::size_t i = 0; i < members.size(); ++i) { + auto basename = members[i].dir.filename().string(); + if (basename == package_filter || as_path(members[i].memberPath) == filterPath) + byPath.push_back(i); + } + + if (byName.size() == 1) { + const auto& picked = members[byName.front()]; + if (!byPath.empty() && byPath.front() != byName.front()) { + const auto& other = members[byPath.front()]; + mcpp::ui::warning(std::format( + "-p '{}' is the package name of member '{}' ({}) and also the " + "directory of member '{}' ({}); the package is selected. Write " + "'{}' to select the other member.", + package_filter, picked.memberPath, qualified_member_name(picked), + other.memberPath, qualified_member_name(other), other.memberPath)); + } + return picked.dir; + } + + if (!byPath.empty()) { + if (byPath.size() > 1) { + std::string others; + for (std::size_t k = 1; k < byPath.size(); ++k) + others += std::format("{}'{}'", others.empty() ? "" : ", ", + members[byPath[k]].memberPath); + mcpp::ui::warning(std::format( + "-p '{}' matches more than one member's directory; '{}' is " + "selected (the first listed in [workspace] members). Also " + "matched: {}.", + package_filter, members[byPath.front()].memberPath, others)); + } + return members[byPath.front()].dir; } - if (rootManifest.package.name.empty()) { - return std::unexpected(std::string( - "virtual workspace: specify -p or --workspace")); + + std::string list; + for (auto const& m : members) { + auto label = m.name.empty() ? std::string("?") + : (m.namespace_.empty() ? m.name + : std::format("{} ({})", m.name, + qualified_member_name(m))); + list += std::format("\n {} at '{}'", label, m.memberPath); } - return std::filesystem::path{}; // rooted workspace, no filter → root package + return std::unexpected(std::format( + "workspace member '{}' not found. [workspace] members:{}", + package_filter, list)); } } // namespace mcpp::project diff --git a/src/toolchain/clang.cppm b/src/toolchain/clang.cppm index 1ca760217..0586f7999 100644 --- a/src/toolchain/clang.cppm +++ b/src/toolchain/clang.cppm @@ -23,11 +23,17 @@ void enrich_toolchain(Toolchain& tc, const std::string& envPrefix); std::filesystem::path std_bmi_path(const std::filesystem::path& cacheDir); std::filesystem::path staged_std_bmi_path(const std::filesystem::path& outputDir); +// `crtFlag` (#718): empty off the MSVC ABI; on it, `-fms-runtime-lib=static` +// or `=dll` (`mcpp.toolchain.dialect::msvc_abi_crt_word`), applied to BOTH +// commands below — the precompile step needs it as much as the codegen step +// does, since the CRT choice also selects which `` declarations +// this compile sees. std::vector std_module_build_commands(const Toolchain& tc, const std::filesystem::path& cacheDir, const std::filesystem::path& bmiPath, std::string_view sysrootFlag, - std::string_view cppStandardFlag); + std::string_view cppStandardFlag, + std::string_view crtFlag = {}); std::optional find_libcxx_std_compat_source( const std::filesystem::path& cxx_binary, @@ -41,7 +47,8 @@ std::vector std_compat_build_commands(const Toolchain& tc, const std::filesystem::path& bmiPath, const std::filesystem::path& stdBmiPath, std::string_view sysrootFlag, - std::string_view cppStandardFlag); + std::string_view cppStandardFlag, + std::string_view crtFlag = {}); // Locate clang-scan-deps in the same bin/ directory as clang++. @@ -240,8 +247,14 @@ std::vector std_module_build_commands(const Toolchain& tc, const std::filesystem::path& cacheDir, const std::filesystem::path& bmiPath, std::string_view sysrootFlag, - std::string_view cppStandardFlag) { + std::string_view cppStandardFlag, + std::string_view crtFlag) { auto relBmi = std::filesystem::relative(bmiPath, cacheDir).string(); + // #718: rendered as its own token, with a leading space, so an empty + // `crtFlag` (every non-MSVC-ABI row) leaves both commands byte-identical + // to what they were before this parameter existed. + const std::string crtToken = + crtFlag.empty() ? std::string{} : std::format(" {}", crtFlag); // A PACKAGE-PROVIDED std MODULE REPLACES THE TOOLCHAIN'S SYSROOT FLAGS // RATHER THAN BEING APPENDED TO THEM. // @@ -343,20 +356,22 @@ std::vector std_module_build_commands(const Toolchain& tc, // the cause is a branch keyed on which machine is doing the building. return { std::format( - "{} {}{}{}{} " + "{} {}{}{}{}{} " "--precompile {} -o {}", tc.binaryPath.string(), cppStandardFlag, + crtToken, ixxFlags, sysrootFlag, precompileFlags, mcpp::xlings::shq(tc.stdModuleSource.string()), mcpp::xlings::shq(absBmi)), std::format( - "{} {}{}{} " + "{} {}{}{}{} " "{} -c -o {}", tc.binaryPath.string(), cppStandardFlag, + crtToken, sysrootFlag, codegenFlags, mcpp::xlings::shq(absBmi), @@ -365,23 +380,25 @@ std::vector std_module_build_commands(const Toolchain& tc, #else return { std::format( - "cd {} && {}{} {} -Wno-reserved-module-identifier{}{} " + "cd {} && {}{} {}{} -Wno-reserved-module-identifier{}{} " "--precompile {} -o {} 2>&1", mcpp::xlings::shq(cacheDir.string()), mcpp::toolchain::compiler_env_prefix(tc), mcpp::xlings::shq(tc.binaryPath.string()), cppStandardFlag, + crtToken, sysrootFlag, precompileFlags, mcpp::xlings::shq(tc.stdModuleSource.string()), mcpp::xlings::shq(relBmi)), std::format( - "cd {} && {}{} {} -Wno-reserved-module-identifier{}{} " + "cd {} && {}{} {}{} -Wno-reserved-module-identifier{}{} " "{} -c -o std.o 2>&1", mcpp::xlings::shq(cacheDir.string()), mcpp::toolchain::compiler_env_prefix(tc), mcpp::xlings::shq(tc.binaryPath.string()), cppStandardFlag, + crtToken, sysrootFlag, codegenFlags, mcpp::xlings::shq(relBmi)) @@ -487,10 +504,15 @@ std::vector std_compat_build_commands(const Toolchain& tc, const std::filesystem::path& bmiPath, const std::filesystem::path& stdBmiPath, std::string_view sysrootFlag, - std::string_view cppStandardFlag) + std::string_view cppStandardFlag, + std::string_view crtFlag) { auto relBmi = std::filesystem::relative(bmiPath, cacheDir).string(); auto relStdBmi = std::filesystem::relative(stdBmiPath, cacheDir).string(); + // #718: see `std_module_build_commands` — empty keeps both commands + // byte-identical off the MSVC ABI. + const std::string crtToken = + crtFlag.empty() ? std::string{} : std::format(" {}", crtFlag); // THE SAME REPLACEMENT THE `std` BUILDER MAKES, FOR THE SAME REASON. // // `std.compat` is a second module over the SAME library, and it therefore @@ -545,23 +567,25 @@ std::vector std_compat_build_commands(const Toolchain& tc, auto absStdBmi = (cacheDir / relStdBmi).string(); auto absObj = (cacheDir / "std.compat.o").string(); return { - std::format("{}{} {} -Wno-reserved-module-identifier{}{} " + std::format("{}{} {}{} -Wno-reserved-module-identifier{}{} " "-fmodule-file=std={} " "--precompile {} -o {} 2>&1", mcpp::toolchain::compiler_env_prefix(tc), mcpp::xlings::shq(tc.binaryPath.string()), cppStandardFlag, + crtToken, sysrootFlag, precompileFlags, absStdBmi, mcpp::xlings::shq(tc.stdCompatSource.string()), mcpp::xlings::shq(absBmi)), - std::format("{}{} {} -Wno-reserved-module-identifier{}{} " + std::format("{}{} {}{} -Wno-reserved-module-identifier{}{} " "-fmodule-file=std={} " "{} -c -o {} 2>&1", mcpp::toolchain::compiler_env_prefix(tc), mcpp::xlings::shq(tc.binaryPath.string()), cppStandardFlag, + crtToken, sysrootFlag, codegenFlags, absStdBmi, diff --git a/src/toolchain/msvc.cppm b/src/toolchain/msvc.cppm index 1e814f40e..0761ef56c 100644 --- a/src/toolchain/msvc.cppm +++ b/src/toolchain/msvc.cppm @@ -385,6 +385,17 @@ bool msvc_available_here(const std::filesystem::path& pkgsDir); std::filesystem::path vc_redist_dir(const std::filesystem::path& clPath, std::string_view arch = "x64"); +// The same directory, reached from the row's SYSROOT rather than from a +// cl.exe path: the LLVM row (clang++ targeting `*-windows-msvc`) runs no +// cl.exe of its own, and its `Toolchain::linkRuntimeDirs` holds the LLVM +// payload's own runtime directories, not this one (clang.cppm). `toolsDir` +// is `Toolchain::msvcToolsDir` (`/Tools/MSVC/`), the field +// `bind_msvc_sysroot` already resolves for this row; `archGnu` is the +// target triple's GNU-spelled architecture ("x86_64", "aarch64", "i686"), +// mapped here to the "x64"/"arm64"/"x86" spelling `vc_redist_dir` takes. +std::filesystem::path vc_redist_dir_for_tools_dir( + const std::filesystem::path& toolsDir, std::string_view archGnu); + // Synthesize the environment cl.exe/link.exe need — what vcvars would set, // derived directly from the located VC tools + SDK (no vcvarsall.bat run): // INCLUDE = \include; \Include\\{ucrt,um,shared,winrt} @@ -1125,6 +1136,31 @@ constexpr std::string_view sdk_lib_arch = "x64"; namespace { +// Whether version directory name `a` is newer than `b`, compared by +// dot-separated components, numerically where both are numbers (`14.9` is +// older than `14.10`) and as text otherwise. An empty name is the oldest. +bool newer_version(std::string_view a, std::string_view b) { + auto next = [](std::string_view& s) { + const auto dot = s.find('.'); + const auto part = s.substr(0, dot); + s = dot == std::string_view::npos ? std::string_view{} : s.substr(dot + 1); + return part; + }; + auto number = [](std::string_view p) -> std::optional { + unsigned long long n = 0; + const auto r = std::from_chars(p.data(), p.data() + p.size(), n); + if (p.empty() || r.ec != std::errc{} || r.ptr != p.data() + p.size()) return std::nullopt; + return n; + }; + while (!a.empty() || !b.empty()) { + const auto pa = next(a), pb = next(b); + const auto na = number(pa), nb = number(pb); + if (na && nb) { if (*na != *nb) return *na > *nb; } + else if (pa != pb) return pa > pb; + } + return false; +} + // Highest version dir under `root/Include` that actually carries the UCRT // headers; `want` (from WindowsSdkVersion) wins if it is one of them. std::optional pick_sdk_in(const std::filesystem::path& root, @@ -1143,7 +1179,7 @@ std::optional pick_sdk_in(const std::filesystem::path& root, auto v = e.path().filename().string(); if (!usable(e.path(), v)) continue; if (!want.empty() && v == want) return WindowsSdk{root, v}; - if (v > best) best = v; + if (newer_version(v, best)) best = v; } if (best.empty()) return std::nullopt; return WindowsSdk{root, best}; @@ -1460,11 +1496,13 @@ std::vector std_compat_build_commands( ref, crtFlag) }; } -std::filesystem::path vc_redist_dir(const std::filesystem::path& clPath, - std::string_view arch) { - // /Tools/MSVC//bin/Host//cl.exe → up 6 from the arch dir - auto vc = clPath.parent_path(); - for (int i = 0; i < 6 && !vc.empty(); ++i) vc = vc.parent_path(); +namespace { + +// The scan shared by both spellings of "where is this toolset's redist": +// given the `` root (the parent of `Tools` and `Redist` alike), the +// newest `Redist\MSVC\\\Microsoft.VC*.CRT` directory. +std::filesystem::path vc_redist_dir_under(const std::filesystem::path& vc, + std::string_view arch) { std::error_code ec; auto redist = vc / "Redist" / "MSVC"; if (!std::filesystem::is_directory(redist, ec)) return {}; @@ -1485,7 +1523,7 @@ std::filesystem::path vc_redist_dir(const std::filesystem::path& clPath, continue; if (c.path().string().find("debug_nonredist") != std::string::npos) continue; - if (auto ver = v.path().filename().string(); ver > bestVer) { + if (auto ver = v.path().filename().string(); newer_version(ver, bestVer)) { bestVer = ver; best = c.path(); } @@ -1494,6 +1532,29 @@ std::filesystem::path vc_redist_dir(const std::filesystem::path& clPath, return best; } +} // namespace + +std::filesystem::path vc_redist_dir(const std::filesystem::path& clPath, + std::string_view arch) { + // /Tools/MSVC//bin/Host//cl.exe → up 6 from the arch dir + auto vc = clPath.parent_path(); + for (int i = 0; i < 6 && !vc.empty(); ++i) vc = vc.parent_path(); + return vc_redist_dir_under(vc, arch); +} + +std::filesystem::path vc_redist_dir_for_tools_dir( + const std::filesystem::path& toolsDir, std::string_view archGnu) { + // toolsDir = /Tools/MSVC/ → up 3 reaches , the same root + // `vc_redist_dir` reaches by walking up from a cl.exe path. + auto vc = toolsDir.parent_path() // Tools/MSVC + .parent_path() // Tools + .parent_path(); // + std::string_view arch = "x64"; + if (archGnu == "aarch64") arch = "arm64"; + else if (archGnu == "i686" || archGnu == "x86") arch = "x86"; + return vc_redist_dir_under(vc, arch); +} + std::expected enrich_toolchain_from_cl(Toolchain& tc) { auto banner = capture_cl_banner(tc.binaryPath); auto parsed = parse_cl_banner(banner); @@ -1571,6 +1632,11 @@ std::expected enrich_toolchain_from_cl(Toolchain& tc) { if (auto redist = vc_redist_dir(tc.binaryPath, parsed->second); !redist.empty()) { tc.linkRuntimeDirs.push_back(redist); + // Also its own field (#718): `msvc_abi_default_contract` and the + // staging gate read THIS rather than `linkRuntimeDirs`, which on the + // LLVM row holds LLVM's own directories instead — one name for "the + // toolset's redistributable" that means the same thing on both rows. + tc.msvcRedistDir = redist; } return {}; } diff --git a/src/toolchain/stdmod.cppm b/src/toolchain/stdmod.cppm index 7340f355c..84fded325 100644 --- a/src/toolchain/stdmod.cppm +++ b/src/toolchain/stdmod.cppm @@ -343,8 +343,13 @@ std::expected derive_std_module( isMsvc ? mcpp::toolchain::msvc::std_module_build_commands( tc, cacheDir, cpp_standard_flag, msvc_crt_flag) : is_clang(tc) + // #718: `msvc_crt_flag` is empty off the MSVC ABI (every row + // this build serves except the LLVM one), so passing it here + // unconditionally changes nothing for a clang row that targets + // anything else. ? mcpp::toolchain::clang::std_module_build_commands( - tc, cacheDir, d.bmiPath, sysroot_flag, cpp_standard_flag) + tc, cacheDir, d.bmiPath, sysroot_flag, cpp_standard_flag, + msvc_crt_flag) : mcpp::toolchain::gcc::std_module_build_commands( tc, cacheDir, sysroot_flag, cpp_standard_flag); if (!tc.stdCompatSource.empty()) { @@ -354,7 +359,8 @@ std::expected derive_std_module( } else if (is_clang(tc)) { auto compatBmi = mcpp::toolchain::clang::std_compat_bmi_path(cacheDir); d.compatCommands = mcpp::toolchain::clang::std_compat_build_commands( - tc, cacheDir, compatBmi, d.bmiPath, sysroot_flag, cpp_standard_flag); + tc, cacheDir, compatBmi, d.bmiPath, sysroot_flag, + cpp_standard_flag, msvc_crt_flag); } } d.metadata = metadata_for(tc, cpp_standard, cpp_standard_flag, diff --git a/src/ui.cppm b/src/ui.cppm index b61e5e1da..8c3961644 100644 --- a/src/ui.cppm +++ b/src/ui.cppm @@ -41,6 +41,21 @@ void finished(std::string_view profile, std::chrono::milliseconds elapsed, void warning(std::string_view message); void error(std::string_view message); +// Closing notices: advisories that concern the run as a whole rather than the +// step that noticed them, such as a refreshed package index that requires a +// newer mcpp. They are printed once, after the command's own output, as `tip:` +// lines on stderr, so that they are the last thing a reader sees and never +// interleave with a build's progress. A command that writes a machine-readable +// envelope takes them first and reports them as `note` diagnostics instead. +// A notice never changes the exit status. +struct ClosingNotice { + std::string code; // a stable code for machine output + std::string message; // one line; no trailing newline +}; +void add_closing_notice(std::string code, std::string message); +std::vector take_closing_notices(); +void print_closing_notices(); + // Multi-line Rust-style diagnostic (M4 #8.1). // Renders as: // @@ -89,6 +104,15 @@ void flush(); void set_line_buffered(); // --- progress bar (single-line, \r-rewritten) --- +// +// ONE RENDERER, TWO OUTPUT MODES. On a terminal the bar is redrawn in place. +// When stdout is not a terminal (a CI log, a pipe, a file) it prints one line +// when the item starts, with its size when known, and one line when it +// finishes, with its duration: no `\r`, no erase sequence, no repaint per +// frame. The mode follows stdout; `set_live_progress` overrides it for tests. +void set_live_progress(bool live); +bool live_progress(); + class ProgressBar { public: ProgressBar(std::string_view verb, std::string_view label); @@ -114,15 +138,25 @@ public: // Finish: replaces progress with final-state line. void finish(); void finish_with(std::string_view final_message); + // Finish an item that did not complete: the line says so rather than + // reporting it done. + void finish_failed(std::string_view final_message); private: void render_line(std::size_t percent, const std::string& info_text); void render_line_swept(std::size_t frame, const std::string& info_text); + // Not a terminal: one start line, then one finish line. + void announce(std::size_t total_bytes); + void finish_plain(std::string_view final_message); + std::string verb_; std::string label_; std::chrono::steady_clock::time_point lastDraw_; - bool finished_ = false; + std::chrono::steady_clock::time_point start_; + bool finished_ = false; + bool announced_ = false; + std::size_t lastBytes_ = 0; }; // --- download progress (centralized) --- @@ -154,6 +188,8 @@ public: void update(std::span files, double elapsed_sec); void finish(); // finish the active bar if any (idempotent) + // Finish the active bar as not completed: the run that fed it failed. + void finish_failed(); private: std::optional bar_; @@ -195,6 +231,8 @@ namespace { bool g_color = false; bool g_quiet = false; bool g_inited = false; +// -1: follow stdout; 0 / 1: set by set_live_progress. +int g_liveOverride = -1; constexpr std::string_view kReset = "\033[0m"; constexpr std::string_view kBold = "\033[1m"; @@ -240,6 +278,14 @@ void disable_color() { g_color = false; } bool is_color_enabled() { return g_color; } void set_quiet(bool q) { g_quiet = q; } + +void set_live_progress(bool live) { g_liveOverride = live ? 1 : 0; } + +bool live_progress() { + if (g_liveOverride >= 0) return g_liveOverride == 1; + static const bool tty = mcpp::platform::terminal::is_tty(); + return tty; +} bool is_quiet() { return g_quiet; } void flush() { std::fflush(stdout); } @@ -330,6 +376,36 @@ void error(std::string_view message) { } } +namespace { +std::vector& closing_notices() { + static std::vector notices; + return notices; +} +} // namespace + +void add_closing_notice(std::string code, std::string message) { + auto& all = closing_notices(); + for (auto const& n : all) + if (n.message == message) return; + all.push_back({std::move(code), std::move(message)}); +} + +std::vector take_closing_notices() { + return std::exchange(closing_notices(), {}); +} + +void print_closing_notices() { + auto notices = take_closing_notices(); + if (g_quiet) return; + init(); + for (auto const& n : notices) { + if (g_color) + std::println(stderr, "{}{}tip:{} {}", kBold, kCyan, kReset, n.message); + else + std::println(stderr, "tip: {}", n.message); + } +} + void plain(std::string_view message) { if (g_quiet) return; std::println("{}", message); @@ -513,9 +589,29 @@ void draw_status_line(std::string_view verb, const std::string& label, ProgressBar::ProgressBar(std::string_view verb, std::string_view label) : verb_(verb), label_(label), - lastDraw_(std::chrono::steady_clock::now() - std::chrono::seconds(1)) + lastDraw_(std::chrono::steady_clock::now() - std::chrono::seconds(1)), + start_(std::chrono::steady_clock::now()) {} +void ProgressBar::announce(std::size_t total_bytes) { + if (announced_) return; + announced_ = true; + if (total_bytes > 0) + info(verb_, std::format("{} ({})", label_, fmt_bytes(total_bytes))); + else + info(verb_, label_); +} + +void ProgressBar::finish_plain(std::string_view final_message) { + const auto secs = std::chrono::duration( + std::chrono::steady_clock::now() - start_).count(); + if (lastBytes_ > 0) + info(verb_, std::format("{} done, {} in {:.1f}s", final_message, + fmt_bytes(lastBytes_), secs)); + else + info(verb_, std::format("{} done in {:.1f}s", final_message, secs)); +} + ProgressBar::~ProgressBar() { if (!finished_) finish(); } @@ -554,6 +650,7 @@ void ProgressBar::render_line_swept(std::size_t frame, const std::string& info_t void ProgressBar::update(std::size_t percent) { if (g_quiet || finished_) return; + if (!live_progress()) { announce(0); return; } auto now = std::chrono::steady_clock::now(); if (now - lastDraw_ < std::chrono::milliseconds(80) && percent < 100) return; lastDraw_ = now; @@ -563,6 +660,7 @@ void ProgressBar::update(std::size_t percent) { void ProgressBar::update_bytes(std::size_t current, std::size_t total, double elapsed_sec) { if (g_quiet || finished_) return; + if (!live_progress()) { announce(total); lastBytes_ = current; return; } auto now = std::chrono::steady_clock::now(); auto pct = total ? (current * 100 / total) : 0; if (pct > 100) pct = 100; @@ -585,6 +683,7 @@ void ProgressBar::update_bytes(std::size_t current, std::size_t total, void ProgressBar::update_indeterminate(std::size_t current_bytes, double elapsed_sec) { if (g_quiet || finished_) return; + if (!live_progress()) { announce(0); lastBytes_ = current_bytes; return; } auto now = std::chrono::steady_clock::now(); // Same ~80ms throttle as update_bytes(); there is no "100%" early-out here // because there is no known total. @@ -607,6 +706,11 @@ void ProgressBar::finish() { if (finished_) return; finished_ = true; if (g_quiet) return; + if (!live_progress()) { + announce(0); + finish_plain(label_); + return; + } // Clear the line and re-emit as a static info line. std::print("\r\033[2K"); info(verb_, label_); @@ -616,10 +720,26 @@ void ProgressBar::finish_with(std::string_view final_message) { if (finished_) return; finished_ = true; if (g_quiet) return; + if (!live_progress()) { + announce(0); + finish_plain(final_message); + return; + } std::print("\r\033[2K"); info(verb_, final_message); } +void ProgressBar::finish_failed(std::string_view final_message) { + if (finished_) return; + finished_ = true; + if (g_quiet) return; + if (live_progress()) std::print("\r\033[2K"); + else announce(0); + const auto secs = std::chrono::duration( + std::chrono::steady_clock::now() - start_).count(); + info(verb_, std::format("{} did not complete ({:.1f}s)", final_message, secs)); +} + // --- DownloadProgress --- DownloadProgress::~DownloadProgress() { finish(); } @@ -630,6 +750,12 @@ void DownloadProgress::finish() { active_.clear(); } +void DownloadProgress::finish_failed() { + if (bar_) bar_->finish_failed(active_); + bar_.reset(); + active_.clear(); +} + void DownloadProgress::update(std::span files, double elapsed_sec) { if (files.empty()) return; diff --git a/src/xlings/xlings.cppm b/src/xlings/xlings.cppm index a870069ea..9cd3d3f65 100644 --- a/src/xlings/xlings.cppm +++ b/src/xlings/xlings.cppm @@ -19,6 +19,7 @@ import mcpp.pm.index_contract; import mcpp.pm.index_snapshot; import mcpp.platform; import mcpp.log; +import mcpp.ui; // closing notices of the refresh guard import mcpp.home; import mcpp.xpkg_version; import mcpp.libs.json; @@ -111,7 +112,7 @@ namespace pinned { // no output (mcpp#693), and under an MCPP_HOME outside it the xlings mcpp // vendors could not initialise its sandbox. It now declares the UTF-8 code // page, as mcpp.exe does. - inline constexpr std::string_view kXlingsVersion = "2026.9.27.1"; + inline constexpr std::string_view kXlingsVersion = "2026.9.28.1"; inline constexpr std::string_view kNasmVersion = "3.02"; } @@ -304,6 +305,18 @@ std::string shq_meta(std::string_view s); // XLINGS_HOME='' '' std::string build_command_prefix(const Env& env); +// The Windows spelling of that prefix, compiled on every host so that it is +// tested from any: `cd /d "" && ""`. The variables are applied +// by ScopedInvocationEnv instead. +// +// THE WORKING DIRECTORY IS THE HOME ON BOTH PLATFORMS (#726). xlings enters +// project mode by walking up from its working directory to a `.xlings.json`. +// Started from a project that has one -- a project whose own mcpp is pinned +// there -- the registry's xlings adopted that project and wrote the shims of +// mcpp's toolchain and payloads (`cl`, `link`, `cmake`, ...) into the +// project's SubOS, whose `bin` is on PATH wherever the project's shell is. +std::string windows_command_prefix(const Env& env); + // THE ENVIRONMENT OF ONE XLINGS INVOCATION (#614), decided once. Each entry is // a variable, its value, and whether it is present at all. Global mode is an // absent XLINGS_PROJECT_DIR, because xlings resolves its subos scope from that @@ -325,13 +338,19 @@ struct InvocationVar { }; std::vector invocation_env(const Env& env); -// Applies the scope half of `invocation_env` to this process for the guard's -// lifetime on Windows, and restores the prior value when the guard ends. On -// POSIX the command prefix carries it and the guard does nothing. Every -// function that runs a command built by `build_command_prefix` holds one while -// the command runs, so a project directory set for one invocation does not -// reach the processes mcpp starts afterwards. XLINGS_HOME and the PATH prefix -// keep their process-wide lifetime. +// Applies `invocation_env` and the sandbox's `bin` in front of PATH to this +// process for the guard's lifetime on Windows, and restores every prior value +// when the guard ends. On POSIX the command prefix carries them and the guard +// does nothing. Every function that runs a command built by +// `build_command_prefix` holds one while the command runs. +// +// NOTHING OUTLIVES THE INVOCATION (#726). The process environment after an +// xlings invocation is the one before it, so a build that installed a payload +// starts ninja with the environment a build that installed nothing does. The +// PATH prefix used to stay: the sandbox's `bin` holds the shims `xim:llvm` +// registers on Windows (`cl`, `link`, `lib`, `rc`), and every action of a +// first build then met them in front of MSVC's tools -- vcpkg's compiler +// detection failed there while the second build passed. class ScopedInvocationEnv { public: explicit ScopedInvocationEnv(const Env& env); @@ -1293,38 +1312,45 @@ std::vector invocation_env(const Env& env) { ScopedInvocationEnv::ScopedInvocationEnv(const Env& env) { if constexpr (mcpp::platform::is_windows) { - // Every variable but XLINGS_HOME is scope: applied for the guard's - // lifetime and restored after it. XLINGS_HOME keeps the process-wide - // lifetime `build_command_prefix` gives it. - for (auto const& var : invocation_env(env)) { - if (var.name == "XLINGS_HOME") continue; + auto save = [this](const std::string& name) { Saved s; - s.name = var.name; - if (auto prior = mcpp::platform::env::get(var.name)) { + s.name = name; + if (auto prior = mcpp::platform::env::get(name)) { s.hadPrevious = true; s.previous = *prior; } saved_.push_back(s); + }; + for (auto const& var : invocation_env(env)) { + save(var.name); if (var.present) mcpp::platform::env::set(var.name, var.value); else mcpp::platform::env::unset(var.name); } + save("PATH"); + mcpp::platform::windows::prepend_path(paths::sandbox_bin(env).string()); } } ScopedInvocationEnv::~ScopedInvocationEnv() { - for (auto const& s : saved_) { - if (s.hadPrevious) mcpp::platform::env::set(s.name, s.previous); - else mcpp::platform::env::unset(s.name); + // Newest first, so a variable saved twice ends at its oldest value. + for (auto it = saved_.rbegin(); it != saved_.rend(); ++it) { + if (it->hadPrevious) mcpp::platform::env::set(it->name, it->previous); + else mcpp::platform::env::unset(it->name); } } +std::string windows_command_prefix(const Env& env) { + return std::format("cd /d {} && {}", + mcpp::platform::shell::quote_windows(env.home.string()), + mcpp::platform::shell::quote_windows(env.binary.string())); +} + std::string build_command_prefix(const Env& env) { auto xvmBin = paths::sandbox_bin(env).string(); if constexpr (mcpp::platform::is_windows) { - // The scope variable is applied by the caller's ScopedInvocationEnv. - mcpp::platform::env::set("XLINGS_HOME", env.home.string()); - mcpp::platform::windows::prepend_path(xvmBin); - return env.binary.string(); + // The environment is applied by the caller's ScopedInvocationEnv and + // restored after the command; building a command changes nothing. + return windows_command_prefix(env); } else { // `env` takes its `-u` operands before its assignments. std::string unset, assign; @@ -1579,6 +1605,129 @@ std::vector stderr_error_tail(std::string_view text, std::size_t li return {tail.begin(), tail.end()}; } +namespace { +// The `files[]` of one NDJSON `download_progress` data event, in the order +// xlings reports them; nullopt when the line carries none. Shared by the +// sandbox bootstrap and the index refresh, which render the same event. +std::optional download_progress_of(std::string_view line) { + LineScan ls{line}; + auto p = line.find("\"files\":["); + if (p == std::string_view::npos) return std::nullopt; + p += 9; + + BootstrapProgress prog; + prog.elapsedSec = ls.find_num("elapsedSec"); + + while (p < line.size()) { + while (p < line.size() && (line[p] == ' ' || line[p] == '\n' + || line[p] == ',')) ++p; + if (p >= line.size() || line[p] == ']') break; + if (line[p] != '{') break; + int depth = 0; + auto start = p; + bool in_string = false; + for (; p < line.size(); ++p) { + char c = line[p]; + if (in_string) { + if (c == '\\' && p + 1 < line.size()) { ++p; continue; } + if (c == '"') in_string = false; + continue; + } + if (c == '"') in_string = true; + else if (c == '{') ++depth; + else if (c == '}') { if (--depth == 0) { ++p; break; } } + } + LineScan fl{line.substr(start, p - start)}; + BootstrapFile f; + f.name = fl.find_str("name"); + f.downloadedBytes = fl.find_num("downloadedBytes"); + f.totalBytes = fl.find_num("totalBytes"); + f.started = fl.find_bool("started"); + f.finished = fl.find_bool("finished"); + if (!f.name.empty()) prog.files.push_back(std::move(f)); + } + if (prog.files.empty()) return std::nullopt; + return prog; +} + +// The events of one `xlings interface update_packages` run, rendered with the +// renderer every other acquisition uses (mcpp::ui::ProgressBar and +// DownloadProgress). A `progress` event carries a phase and a percentage; +// one bar is drawn per phase (xlings reports `index_sync`, one event per +// repository, and `index_rebuild`, one event per descriptor file), so a +// refresh prints a few lines off a terminal rather than one per file. A +// `download_progress` data event is an index artifact being fetched. A line +// that is not an event is not rendered: an xlings that predates structured +// index progress printed its own terminal text on this stream, and the bar +// is what replaces it. +class IndexRefreshRenderer { +public: + void line(std::string_view text) { + LineScan ls{text}; + const auto kind = ls.find_str("kind"); + if (kind == "result") { + resultExit_ = static_cast(ls.find_num("exitCode")); + } else if (kind == "progress") { + const auto phase = ls.find_str("phase"); + if (phase.empty()) return; + // A sync step names its repository at the end of its message + // (`syncing index repo 2/5: mcpplibs`); each repository is its + // own bar, labelled with that name. + std::string subject; + if (phase == "index_sync") { + const auto message = ls.find_str("message"); + if (auto colon = message.rfind(": "); colon != std::string::npos) + subject = message.substr(colon + 2); + } + const auto key = subject.empty() ? phase : phase + "/" + subject; + if (!bar_ || key != label_) { + if (bar_) bar_->finish(); + shown_ = subject.empty() ? phase_label(phase) + : std::format("package index {}", subject); + bar_.emplace("Updating", shown_); + label_ = key; + } + const auto pct = std::clamp(ls.find_num("percent"), 0.0, 100.0); + bar_->update(static_cast(pct)); + } else if (kind == "data" && ls.find_str("dataKind") == "download_progress") { + auto prog = download_progress_of(text); + if (!prog) return; + if (bar_) { bar_->finish(); bar_.reset(); label_.clear(); } + std::vector files; + for (auto const& f : prog->files) + files.push_back({f.name, + static_cast(f.downloadedBytes), + static_cast(f.totalBytes), + f.started, f.finished}); + download_.update(files, prog->elapsedSec); + } + } + // The exit code the result event carried, or -1 when none arrived. + int result_exit() const { return resultExit_; } + // The run failed or was stopped: the open bar says the step did not + // complete rather than reporting it done. + void fail() { failed_ = true; } + ~IndexRefreshRenderer() { + if (bar_) failed_ ? bar_->finish_failed(shown_) : bar_->finish(); + failed_ ? download_.finish_failed() : download_.finish(); + } + +private: + static std::string phase_label(std::string_view phase) { + if (phase == "index_sync") return "package index (sync)"; + if (phase == "index_rebuild") return "package index (rebuild)"; + return std::format("package index ({})", phase); + } + + std::optional bar_; + std::string label_; // the phase (and repository) drawn + std::string shown_; // the bar's label + bool failed_ = false; + mcpp::ui::DownloadProgress download_; + int resultExit_ = -1; +}; +} // namespace + // ─── install_with_progress ────────────────────────────────────────── int install_with_progress(const Env& env, std::string_view target, @@ -1615,10 +1764,11 @@ int install_with_progress(const Env& env, std::string_view target, // The direct install redirects all output to the null device, so it // produces zero feedback — on a slow/network-bound first run this - // looks frozen. Run the blocking std::system() on a worker thread and - // paint an in-place elapsed-time spinner on stderr while it runs. - // Only when interactive (not quiet, stderr/stdout is a TTY). - const bool showSpinner = !quiet && mcpp::platform::terminal::is_tty(); + // looks frozen. Run the blocking command on a worker thread and draw + // an elapsed-time bar while it runs, with the renderer every other + // acquisition uses (W11): redrawn in place on a terminal, one start + // and one finish line otherwise, nothing under --quiet. + const bool showSpinner = !quiet; mcpp::platform::env::note_network_access(); std::atomic done{false}; @@ -1637,29 +1787,21 @@ int install_with_progress(const Env& env, std::string_view target, }); if (showSpinner) { - constexpr std::string_view frames = "⠋⠙⠹⠸⠼⠴⠦⠧⠇⠏"; - // Each braille frame is 3 bytes in UTF-8. - constexpr std::size_t kFrameBytes = 3; - const std::size_t nFrames = frames.size() / kFrameBytes; + mcpp::ui::ProgressBar bar("Installing", target); const auto start = std::chrono::steady_clock::now(); - std::size_t i = 0; while (!done.load(std::memory_order_acquire)) { - auto elapsed = std::chrono::duration_cast( - std::chrono::steady_clock::now() - start).count(); - std::print(stderr, "\r {} installing {} ({}s)\x1b[K", - frames.substr((i % nFrames) * kFrameBytes, kFrameBytes), - target, elapsed); - std::fflush(stderr); - ++i; + bar.update_indeterminate(0, std::chrono::duration( + std::chrono::steady_clock::now() - start).count()); std::this_thread::sleep_for(std::chrono::milliseconds(200)); } - // Clear the spinner line. - std::print(stderr, "\r\x1b[K"); - std::fflush(stderr); + worker.join(); + if (directRc == 0) bar.finish(); + else bar.finish_failed(target); + if (directRc == 0) return 0; + } else { + worker.join(); + if (directRc == 0) return 0; } - - worker.join(); - if (directRc == 0) return 0; } // Fallback: NDJSON interface path (provides progress callbacks). @@ -1685,42 +1827,7 @@ int install_with_progress(const Env& env, std::string_view target, if (ls.find_str("dataKind") != "download_progress") return; if (!cb) return; - auto p = line.find("\"files\":["); - if (p == std::string_view::npos) return; - p += 9; - - BootstrapProgress prog; - prog.elapsedSec = ls.find_num("elapsedSec"); - - while (p < line.size()) { - while (p < line.size() && (line[p] == ' ' || line[p] == '\n' - || line[p] == ',')) ++p; - if (p >= line.size() || line[p] == ']') break; - if (line[p] != '{') break; - int depth = 0; - auto start = p; - bool in_string = false; - for (; p < line.size(); ++p) { - char c = line[p]; - if (in_string) { - if (c == '\\' && p + 1 < line.size()) { ++p; continue; } - if (c == '"') in_string = false; - continue; - } - if (c == '"') in_string = true; - else if (c == '{') ++depth; - else if (c == '}') { if (--depth == 0) { ++p; break; } } - } - LineScan fl{line.substr(start, p - start)}; - BootstrapFile f; - f.name = fl.find_str("name"); - f.downloadedBytes = fl.find_num("downloadedBytes"); - f.totalBytes = fl.find_num("totalBytes"); - f.started = fl.find_bool("started"); - f.finished = fl.find_bool("finished"); - if (!f.name.empty()) prog.files.push_back(std::move(f)); - } - if (!prog.files.empty()) cb(prog); + if (auto prog = download_progress_of(line)) cb(*prog); }; bool idleTimedOut = false; @@ -2066,25 +2173,34 @@ int update_index(const Env& env, bool quiet) { // Report ONLY when the guard had to act. The common path — refresh keeps // the index readable — must stay silent, or the notice becomes noise that // users learn to skip past, which is the same as not printing it. + // + // A floor is not an error of the run (an index is data; mcpp is the + // program). Each case is one closing notice, printed after the command's + // own output (mcpp::ui::add_closing_notice), and only because this run + // refreshed the index. `quiet` governs the refresh's own narration, not + // these: they are the one thing the refresh has to say. + (void)quiet; + auto advice = [&](const std::filesystem::path& dir) { + auto it = out.requiredMcpp.find(dir); + return mcpp::pm::index_floor_upgrade_advice( + it == out.requiredMcpp.end() ? std::string_view{} : std::string_view{it->second}); + }; for (auto& dir : out.rolledBack) { - print_status("Kept", std::format( - "previous index for `{}` — the refreshed one requires a newer mcpp", - dir.filename().string())); - if (!quiet) { - std::println(" Your build continues to work with the packages " - "it already describes."); - std::println(" Upgrade to pick up newer packages: xlings update mcpp"); - } + mcpp::ui::add_closing_notice("MCPP_INDEX_REQUIRES_NEWER_MCPP", std::format( + "the refreshed package index `{}` requires a newer mcpp; this run " + "used the previous index. {}", dir.filename().string(), advice(dir))); } for (auto& dir : out.recovered) { - print_status("Restored", std::format( - "index `{}` from a local snapshot this mcpp can read", - dir.filename().string())); + mcpp::ui::add_closing_notice("MCPP_INDEX_REQUIRES_NEWER_MCPP", std::format( + "the package index `{}` was restored from a local snapshot this mcpp " + "can read; the published index requires a newer mcpp. {}", + dir.filename().string(), advice(dir))); } for (auto& dir : out.stillUnusable) { - mcpp::log::verbose("index", std::format( - "index `{}` requires a newer mcpp and no local snapshot is usable", - dir.filename().string())); + mcpp::ui::add_closing_notice("MCPP_INDEX_REQUIRES_NEWER_MCPP", std::format( + "the package index `{}` requires a newer mcpp and no earlier copy is " + "usable; packages it serves cannot be resolved. {}", + dir.filename().string(), advice(dir))); } return rc; } @@ -2119,7 +2235,15 @@ int update_index_unguarded(const Env& env, bool quiet) { return 0; } - std::string cmd = build_command_prefix(env) + " update 2>&1"; + // Through the NDJSON interface, as installs are (W11): the refresh reports + // its steps as events and is drawn by the same renderer, instead of the + // bare CLI's terminal text, which was either reprinted verbatim or, for + // the automatic refresh, discarded, so a refresh of many seconds showed + // nothing. The interface's `update_packages` and the CLI's `update` are + // one function in xlings (xim::cmd_update). + std::string cmd = std::format("{} interface update_packages --args {} {} {}", + build_command_prefix(env), shq_meta("{}"), mcpp::platform::null_redirect, + mcpp::platform::is_windows ? "(refreshBound), - std::chrono::milliseconds{0}, &timedOut); + { + IndexRefreshRenderer renderer; + rc = mcpp::platform::process::run_streaming_bounded(cmd, + [&renderer](std::string_view line) { renderer.line(line); }, + std::chrono::duration_cast(refreshBound), + std::chrono::milliseconds{0}, &timedOut); + if (rc == 0 && renderer.result_exit() > 0) rc = renderer.result_exit(); + if (rc != 0 || timedOut) renderer.fail(); + } if (rc == 0 && !timedOut) { mark_known_indexes_refreshed(env); return 0; } // A refresh that exceeded its bound is not retried: the retries exist for // a transient failure that ends, and a connection that never answers @@ -2158,8 +2285,13 @@ int update_index_unguarded(const Env& env, bool quiet) { std::this_thread::sleep_for(std::chrono::seconds(delay)); } } - mcpp::log::verbose("index", std::format( - "index update failed after {} attempts (rc {})", kMaxAttempts, rc)); + // Said, not only logged, to a caller that refreshes on its own (`quiet`: + // the TTL refresh before a build), which continues with the local index; + // `mcpp index update` reports the failure itself. + if (quiet) + std::println(stderr, + "warning: the package index refresh failed after {} attempts (exit {}); " + "continuing with the local index", kMaxAttempts, rc); return rc; } } // namespace diff --git a/tests/e2e/190_link_rspfile_newlines.sh b/tests/e2e/190_link_rspfile_newlines.sh index 821a63b4d..41fa677fc 100755 --- a/tests/e2e/190_link_rspfile_newlines.sh +++ b/tests/e2e/190_link_rspfile_newlines.sh @@ -66,7 +66,10 @@ echo " ok: link rules declare rspfile_content = \$in_newline" # 2. Observable: ninja keeps the response file under -d keeprsp, and it holds # one object per line rather than all of them on the first. bdir=$(dirname "$ninja_file") -bin_rel=$(cd "$bdir" && ls bin/ 2>/dev/null | head -1) +# The program by its name: on an MSVC-ABI row bin/ also holds the toolset's +# runtime DLLs staged beside it (docs/20, toolchain-coupled), and relinking +# one of those names re-runs a copy, which writes no response file. +bin_rel=$(cd "$bdir" && ls bin/ 2>/dev/null | grep -E '^multi(\.exe)?$' | head -1) [ -n "$bin_rel" ] || { echo "FAIL: no linked binary to inspect"; exit 1; } (cd "$bdir" && rm -f "bin/$bin_rel" && ninja -d keeprsp "bin/$bin_rel" > /dev/null 2>&1) \ || { echo "FAIL: relink under -d keeprsp"; exit 1; } diff --git a/tests/e2e/191_link_scale.sh b/tests/e2e/191_link_scale.sh index bf61cb9a0..d68414af2 100755 --- a/tests/e2e/191_link_scale.sh +++ b/tests/e2e/191_link_scale.sh @@ -97,7 +97,10 @@ echo " ok: built $((N + 1)) objects" ninja_file=$(find target -name build.ninja | head -1) [ -n "$ninja_file" ] || { echo "FAIL: no build.ninja"; exit 1; } bdir=$(dirname "$ninja_file") -bin_rel=$(cd "$bdir" && ls bin/ 2>/dev/null | head -1) +# The program by its name: on an MSVC-ABI row bin/ also holds the toolset's +# runtime DLLs staged beside it (docs/20, toolchain-coupled), and relinking +# one of those names re-runs a copy, which writes no response file. +bin_rel=$(cd "$bdir" && ls bin/ 2>/dev/null | grep -E '^scale(\.exe)?$' | head -1) [ -n "$bin_rel" ] || { echo "FAIL: nothing was linked"; exit 1; } # Relink with the response file kept, so its real contents can be inspected — diff --git a/tests/e2e/52_local_path_namespaced_index.sh b/tests/e2e/52_local_path_namespaced_index.sh index 37794fbcb..7b231895e 100755 --- a/tests/e2e/52_local_path_namespaced_index.sh +++ b/tests/e2e/52_local_path_namespaced_index.sh @@ -125,7 +125,8 @@ if [[ "${1:-}" == "self" && "${2:-}" == "init" ]]; then exit 0 fi -if [[ "${1:-}" == "update" ]]; then +# The index refresh: the bare CLI before 2026.9.28.1, the NDJSON interface since. +if [[ "${1:-}" == "update" || ( "${1:-}" == "interface" && "${2:-}" == "update_packages" ) ]]; then printf 'update\n' > "${FAKE_XLINGS_UPDATE_LOG:?}" exit 0 fi diff --git a/tests/e2e/58_preinstall_mcpp_deps_for_hooks.sh b/tests/e2e/58_preinstall_mcpp_deps_for_hooks.sh index d598e2788..0328c85a5 100644 --- a/tests/e2e/58_preinstall_mcpp_deps_for_hooks.sh +++ b/tests/e2e/58_preinstall_mcpp_deps_for_hooks.sh @@ -96,7 +96,8 @@ if [[ "${1:-}" == "self" && "${2:-}" == "init" ]]; then exit 0 fi -if [[ "${1:-}" == "update" ]]; then +# The index refresh: the bare CLI before 2026.9.28.1, the NDJSON interface since. +if [[ "${1:-}" == "update" || ( "${1:-}" == "interface" && "${2:-}" == "update_packages" ) ]]; then exit 0 fi diff --git a/tests/e2e/60_stale_xpkg_cache_reinstall.sh b/tests/e2e/60_stale_xpkg_cache_reinstall.sh index 4c9fc7fee..f16d649d4 100644 --- a/tests/e2e/60_stale_xpkg_cache_reinstall.sh +++ b/tests/e2e/60_stale_xpkg_cache_reinstall.sh @@ -53,7 +53,8 @@ if [[ "${1:-}" == "self" && "${2:-}" == "init" ]]; then exit 0 fi -if [[ "${1:-}" == "update" ]]; then +# The index refresh: the bare CLI before 2026.9.28.1, the NDJSON interface since. +if [[ "${1:-}" == "update" || ( "${1:-}" == "interface" && "${2:-}" == "update_packages" ) ]]; then exit 0 fi diff --git a/tests/e2e/646_runtime_deploy_places_files_in_a_directory.sh b/tests/e2e/646_runtime_deploy_places_files_in_a_directory.sh index dc1426823..08290830d 100755 --- a/tests/e2e/646_runtime_deploy_places_files_in_a_directory.sh +++ b/tests/e2e/646_runtime_deploy_places_files_in_a_directory.sh @@ -9,8 +9,11 @@ # file beside the executable; a test binary finds the same layout; # 2. a dependency's entry resolves `from` against the dependency and lands in # the consumer's bin/; -# 3. one file name in two directories is not a collision, and two sources for -# one destination are refused naming the destination; +# 3. one file name in two directories is not a collision, and two DIFFERENT +# sources for one destination are refused at BUILD TIME (mcpp#723, +# SPEC-007 R4.2), naming every source and the destination -- planning no +# longer refuses this, because at planning time neither source may exist +# yet to compare; # 4. a destination that leaves the executable's directory is refused naming # the entry. set -e @@ -91,15 +94,44 @@ CPP rm -rf tests echo "test layout OK" -# ── 3. Two sources for one destination ──────────────────────────────────── +# ── 3. Two DIFFERENT sources for one destination ────────────────────────── +# `assets/layers/lvp_icd.json` ("layer manifest") and the dependency's +# `share/vulkan/icd.d/lvp_icd.json` ("library_path": ...) disagree, so the +# merged destination is refused -- at BUILD time (`mcpp stage`), not at +# planning: mcpp#723 merges the two sources into one stage edge instead of +# refusing the second one at `add_deploy`. write_manifest '{ from = "assets/layers/lvp_icd.json", to = "vulkan/icd.d" }' if "$MCPP" build > collision.log 2>&1; then - fail "two sources for bin/vulkan/icd.d/lvp_icd.json were accepted" collision.log + fail "two different sources for bin/vulkan/icd.d/lvp_icd.json were accepted" collision.log fi -grep -Eq "runtime deploy collision: .* both target 'bin.vulkan.icd\.d.lvp_icd\.json'" collision.log \ - || fail "the collision is not refused naming the destination" collision.log +grep -q "sources disagree" collision.log \ + || fail "the refusal does not say the sources disagree" collision.log +grep -q "vulkan/icd.d/lvp_icd.json" collision.log \ + || fail "the refusal does not name the destination" collision.log +grep -q "layers/lvp_icd.json" collision.log \ + || fail "the refusal does not name the root's source" collision.log +grep -q "icd/share/vulkan/icd.d/lvp_icd.json" collision.log \ + || fail "the refusal does not name the dependency's source" collision.log echo "collision OK" +# The SAME shape with IDENTICAL bytes is not a collision at all: two sources +# for one destination merge into one stage edge, and the build succeeds +# (mcpp#723). The source must be named `lvp_icd.json` too -- the destination +# filename is the source's own filename, not `to`. +mkdir -p assets/icd2 +cp "$TMP/icd/share/vulkan/icd.d/lvp_icd.json" assets/icd2/lvp_icd.json +write_manifest '{ from = "assets/icd2/lvp_icd.json", to = "vulkan/icd.d" }' +"$MCPP" build > merge.log 2>&1 || fail "identical bytes for one destination were refused" merge.log +grep -q 'library_path' "$bin/vulkan/icd.d/lvp_icd.json" 2>/dev/null \ + || fail "the merged destination does not carry the shared bytes" merge.log +G=$(find target -name build.ninja | head -1) +STAGE_LINES=$(grep -c "^build .*vulkan/icd\.d/lvp_icd\.json : stage_file" "$G" 2>/dev/null || true) +[ "$STAGE_LINES" -eq 1 ] \ + || fail "expected exactly one stage_file edge, found $STAGE_LINES" "$G" +grep "^build .*vulkan/icd\.d/lvp_icd\.json : stage_file" "$G" | grep -qF "icd2/lvp_icd.json" \ + || fail "the merged edge does not list the root's second source" "$G" +echo "identical-bytes merge OK" + # ── 4. A destination outside the executable's directory ─────────────────── write_manifest '{ from = "assets/readme.txt", to = "../outside" }' if "$MCPP" build > escape.log 2>&1; then diff --git a/tests/e2e/688_emit_build_database.sh b/tests/e2e/688_emit_build_database.sh index 70a9a59e7..489d38151 100755 --- a/tests/e2e/688_emit_build_database.sh +++ b/tests/e2e/688_emit_build_database.sh @@ -110,8 +110,8 @@ assert e["kindVersion"] == 1, e["kindVersion"] assert "write-project" not in e["effects"], e["effects"] assert "read-project" in e["effects"], e["effects"] d = e["data"] -assert d["spec"] == {"name": "s1", "version": "0.2.0"}, d["spec"] -assert d["database"]["ide"]["profile-version"] == "0.2.0" +assert d["spec"] == {"name": "s1", "version": "0.3.0"}, d["spec"] +assert d["database"]["ide"]["profile-version"] == "0.3.0" assert d["inputs-fingerprint"].startswith("fnv1a:"), d["inputs-fingerprint"] EOF "$MCPP" --protocol-version > "$OUT/proto.json" diff --git a/tests/e2e/703_the_llvm_row_on_the_msvc_abi_records_its_static_crt.sh b/tests/e2e/703_the_llvm_row_on_the_msvc_abi_records_its_static_crt.sh index a5017bc22..d1235cedf 100755 --- a/tests/e2e/703_the_llvm_row_on_the_msvc_abi_records_its_static_crt.sh +++ b/tests/e2e/703_the_llvm_row_on_the_msvc_abi_records_its_static_crt.sh @@ -1,18 +1,28 @@ #!/usr/bin/env bash # requires: windows -# 703 -- the C++ runtime record of clang on the MSVC ABI (#649 E10). +# 703 -- the C++ runtime record of clang on the MSVC ABI (#649 E10, closed by +# #718). Kept at this number: the filename is now inexact (the row's DEFAULT +# is no longer the static CRT), and the fuller Windows leg of #718 lives at +# 814; this file keeps testing the one thing it always tested — what +# `resolution.json` records for the llvm row's C++ runtime contract. # -# mcpp emits a CRT model (`/MT` or `/MD`) only for cl.exe. clang on the MSVC ABI -# receives none, and its driver links the static CRT (`-defaultlib:libcmt`), so -# the program imports no vcruntime or ucrt DLL. The contract table was written -# for cl.exe and recorded `host-coupled` for that artifact; an explicit -# `cxx_runtime = "host-coupled"` was recorded and not delivered, with no word. -# The record now states `self-contained`, and the explicit request prints that -# the row does not deliver it. Nothing about the artifact changes. +# mcpp used to emit a CRT model (`/MT` or `/MD`) only for cl.exe. clang on the +# MSVC ABI received none, its driver linked the static CRT +# (`-defaultlib:libcmt`) regardless of what `cxx_runtime` said, and the table +# recorded `self-contained` for every request while an explicit `host-coupled` +# or `toolchain-coupled` printed that the row did not deliver it. # -# Read only when the default toolchain on this runner is the llvm row: a runner -# whose default is msvc@system prints that and asserts nothing, because the -# cl.exe cells are unchanged and covered elsewhere. +# Every MSVC-ABI row now receives the SAME model cl.exe does, spelled +# `-fms-runtime-lib=static`/`=dll` — one helper, `msvc_abi_crt_word` — reaching +# the compile line, the std/std.compat BMIs and the link command alike. The +# row's DEFAULT is therefore `toolchain-coupled` (the dynamic CRT, with the +# toolset's own vcruntime140.dll/msvcp140.dll staged beside the artifact), +# and an explicit `host-coupled` or `toolchain-coupled` is now delivered +# rather than degraded. +# +# Read only when the default toolchain on this runner is the llvm row: a +# runner whose default is msvc@system prints that and asserts nothing, +# because the cl.exe cells are unchanged and covered elsewhere. set -e TMP=$(mktemp -d) @@ -45,14 +55,14 @@ write_app '' cd app "$MCPP" build > default.log 2>&1 || fail "the default build failed" default.log if ! grep -q "Resolved llvm@" default.log; then - echo "READING E10: the default toolchain here is not the llvm row: $(grep -m1 'Resolved' default.log)" - echo "PASS: 703 the llvm row on the MSVC ABI records its static CRT (not the llvm row; nothing to assert)" + echo "READING #718: the default toolchain here is not the llvm row: $(grep -m1 'Resolved' default.log)" + echo "PASS: 703 the llvm row on the MSVC ABI records its default CRT (not the llvm row; nothing to assert)" exit 0 fi contract=$(record_of) -echo "READING E10 default record: distributable=$contract" -[ "$contract" = "self-contained" ] \ - || fail "the llvm row recorded '$contract' for a program linked with the static CRT" default.log +echo "READING #718 default record: distributable=$contract" +[ "$contract" = "toolchain-coupled" ] \ + || fail "the llvm row's undeclared default recorded '$contract', not toolchain-coupled" default.log cd .. write_app 'cxx_runtime = "host-coupled"' @@ -60,11 +70,22 @@ cd app rm -rf target "$MCPP" build > host.log 2>&1 || fail "the host-coupled build failed" host.log grep -q 'is not delivered for clang on the MSVC ABI' host.log \ - || fail "an explicit host-coupled request on the llvm row printed nothing" host.log + && { fail "an explicit host-coupled request still prints the old E10 message" host.log; } || true +contract=$(record_of) +echo "READING #718 host-coupled record: distributable=$contract" +[ "$contract" = "host-coupled" ] \ + || fail "an explicit host-coupled request recorded '$contract', not host-coupled" host.log + +cd .. +write_app 'cxx_runtime = "self-contained"' +cd app +rm -rf target +"$MCPP" build > self.log 2>&1 || fail "the self-contained build failed" self.log contract=$(record_of) -echo "READING E10 host-coupled record: distributable=$contract" +echo "READING #718 self-contained record: distributable=$contract" [ "$contract" = "self-contained" ] \ - || fail "the undelivered request was recorded as '$contract'" host.log -echo "ok: the llvm row records the static CRT it links, and says an explicit dynamic request is not delivered" + || fail "an explicit self-contained request recorded '$contract'" self.log + +echo "ok: the llvm row's default is toolchain-coupled, and an explicit host-coupled or self-contained is delivered exactly as recorded" -echo "PASS: 703 the llvm row on the MSVC ABI records its static CRT" +echo "PASS: 703 the llvm row on the MSVC ABI records its default CRT" diff --git a/tests/e2e/732_xlings_refresh_is_bounded_and_owned.sh b/tests/e2e/732_xlings_refresh_is_bounded_and_owned.sh index b65625aac..d4c669b0f 100755 --- a/tests/e2e/732_xlings_refresh_is_bounded_and_owned.sh +++ b/tests/e2e/732_xlings_refresh_is_bounded_and_owned.sh @@ -33,8 +33,8 @@ bounded() { # mkdir -p "$TMP/bin" "$TMP/home" cat > "$TMP/bin/xlings" <<'EOF' #!/usr/bin/env bash -case "${1:-}" in - update) echo "$$" > "${STUB_PID:?}"; exec sleep 1000 ;; +case "${1:-}:${2:-}" in + update:*|interface:update_packages) echo "$$" > "${STUB_PID:?}"; exec sleep 1000 ;; esac exit 0 EOF diff --git a/tests/e2e/733_envelope_reports_observed_network_access.sh b/tests/e2e/733_envelope_reports_observed_network_access.sh index 9fb4f1989..33a7b2bbb 100755 --- a/tests/e2e/733_envelope_reports_observed_network_access.sh +++ b/tests/e2e/733_envelope_reports_observed_network_access.sh @@ -50,7 +50,7 @@ effects() { python3 -c 'import json,sys; d=json.load(open(sys.argv[1])); print(" STUB_LOG="$TMP/online.log" "$MCPP" emit build-database --format json > online.json 2> online.err || true [ -s online.json ] || fail "A: no envelope" online.err -grep -q '^update' "$TMP/online.log" 2>/dev/null || fail "A: the plan did not refresh the index" online.err +grep -qE '^update|^interface update_packages' "$TMP/online.log" 2>/dev/null || fail "A: the plan did not refresh the index" online.err effects online.json > online.txt head -1 online.txt | grep -qw network || fail "A: a plan that refreshed the index reports no network effect" online.txt diff --git a/tests/e2e/734_auto_refresh_false_governs_first_index_sync.sh b/tests/e2e/734_auto_refresh_false_governs_first_index_sync.sh index 44cd442ed..4953ef96d 100755 --- a/tests/e2e/734_auto_refresh_false_governs_first_index_sync.sh +++ b/tests/e2e/734_auto_refresh_false_governs_first_index_sync.sh @@ -55,7 +55,7 @@ set -e [ "$rc" -ne 0 ] || fail "A: the build succeeded with an unsynced custom index" build.log grep -q "mcpp index update" build.log || fail "A: the refusal does not name mcpp index update" build.log grep -q "auto_refresh = false" build.log || fail "A: the refusal does not name the setting" build.log -if grep -q '^update' "$TMP/stub.log" 2>/dev/null; then +if grep -qE '^update|^interface update_packages' "$TMP/stub.log" 2>/dev/null; then fail "B: the custom index was synced although auto_refresh = false" "$TMP/stub.log" fi echo "PASS: 734 auto_refresh = false governs the first custom index sync" diff --git a/tests/e2e/805_rooted_workspace_reaches_its_own_path_dependency.sh b/tests/e2e/805_rooted_workspace_reaches_its_own_path_dependency.sh new file mode 100755 index 000000000..d994349e6 --- /dev/null +++ b/tests/e2e/805_rooted_workspace_reaches_its_own_path_dependency.sh @@ -0,0 +1,190 @@ +#!/usr/bin/env bash +# requires: gcc python3 +# 805 -- a rooted workspace (root [package] + [workspace]) built as itself +# carries its workspace context to a member reached through its OWN [dependencies] +# path entry (mcpp#725). +# +# Before the fix, `state.wsManifest` / `state.runtimeWorkspaceRoot` were left +# unset on this exact branch (src/build/prepare/manifest.cpp), so +# `depIsMember` (src/build/prepare/graph.cpp) read the member as a stranger: +# +# A. its `x.workspace = true` entry, pinned in [workspace.dependencies] to a +# version that is NOT the latest, was either refused (2026.9.27.1) or +# resolved as an unconstrained dependency that happened to land on the +# latest version anyway (2026.9.26.1) -- the pin must be honoured; +# B. its omitted `package.version`, supplied by [workspace.package], made the +# manifest parser itself refuse the member (it was not loaded +# `insideWorkspace`); +# C. it received none of [workspace.build]'s flags. +# +# The dependency is served from a project-local path index and pre-extracted +# into the private xlings data directory `install_path_from_project_data` +# reads, so the build touches no network: only the PINNED version (0.0.1) is +# pre-extracted, and the index's "latest", 0.0.2, is not -- so an unpinned +# resolution fails loudly (a network fetch of an https://example.invalid URL) +# rather than silently passing, which is what let 2026.9.26.1 look correct. +set -e + +TMP=$(mktemp -d) +trap 'rm -rf "$TMP"' EXIT +fail() { echo "FAIL: $1"; [ -n "${2:-}" ] && cat "$2"; exit 1; } + +export MCPP_HOME="$TMP/mcpp-home" +source "$(dirname "$0")/_inherit_toolchain.sh" + +# A gcc payload already installed on this machine, pinned explicitly at the +# workspace root so the whole graph is compiled with a toolchain member `a`'s +# own (nonexistent) declaration would not choose. The HIGHEST installed +# version, not merely the first: an old gcc (13.x) has no `-fmodules` at all, +# and this fixture needs C++23 modules to build. +GCC_VER="" +if [ -d "$MCPP_HOME/registry/data/xpkgs/xim-x-gcc" ]; then + GCC_VER="$(ls -1 "$MCPP_HOME/registry/data/xpkgs/xim-x-gcc" | sort -V | tail -1)" +fi +[ -n "$GCC_VER" ] || fail "no gcc payload found under \$MCPP_HOME to pin [toolchain]" + +mkdir -p "$TMP/ws" && cd "$TMP/ws" + +# ── a project-local index serving two versions of one package ────────────── +mkdir -p local-index/pkgs/f +cat > local-index/pkgs/f/fx.pinned.lua <<'EOF' +package = { + spec = "1", + namespace = "fx", + name = "fx.pinned", + description = "fixture package pinned to a non-latest version (#725)", + licenses = {"MIT"}, + type = "package", + xpm = { + linux = { + ["0.0.1"] = { + url = "https://example.invalid/fx-pinned-0.0.1.tar.gz", + sha256 = "0000000000000000000000000000000000000000000000000000000000000000", + }, + ["0.0.2"] = { + url = "https://example.invalid/fx-pinned-0.0.2.tar.gz", + sha256 = "0000000000000000000000000000000000000000000000000000000000000000", + }, + }, + }, + mcpp = { + language = "c++23", + import_std = false, + sources = { "src/pinned.cppm" }, + targets = { ["pinned"] = { kind = "lib" } }, + deps = {}, + }, +} +EOF + +# Only the PINNED version (0.0.1) is pre-extracted -- 0.0.2 ("latest") is +# deliberately absent, so resolving the wrong version fails loudly instead of +# silently succeeding (the 2026.9.26.1 shape). +mkdir -p ".mcpp/.xlings/data/xpkgs/fx.pinned/0.0.1/src" +cat > ".mcpp/.xlings/data/xpkgs/fx.pinned/0.0.1/src/pinned.cppm" <<'EOF' +export module pinned; +export int pinned_value() { return 1; } +EOF + +# ── the rooted workspace: root has BOTH [package] and [workspace] ────────── +mkdir -p src a/src +cat > mcpp.toml < src/main.cpp <<'EOF' +import a; +int main() { return a_value() == 2 ? 0 : 1; } +EOF + +# `a` is reached ONLY through the root's own [dependencies] path entry, never +# through `-p`: it omits `package.version` (supplied by [workspace.package]), +# and its `x.workspace = true` entry names a package this workspace's own +# [indices] resolves. +cat > a/mcpp.toml <<'EOF' +[package] +name = "a" + +[dependencies.fx] +pinned = { workspace = true } + +[targets.a] +kind = "lib" + +[build] +sources = ["src/a.cppm"] +EOF +cat > a/src/a.cppm <<'EOF' +export module a; +import pinned; +#if !defined(WS_FLAG) +#error "a.cppm: [workspace.build] cxxflags did not reach this member (#725)" +#endif +export int a_value() { return pinned_value() + WS_FLAG; } +EOF + +"$MCPP" build > build.log 2>&1 || fail "a rooted workspace's own path dependency did not build" build.log + +# ── B: `a` was recognised as a member (no missing-version refusal happened) ─ +grep -qE "missing required field 'package.version'" build.log \ + && fail "a's omitted package.version was not supplied by [workspace.package]" build.log + +# ── A: the pinned, non-latest version was locked and used ────────────────── +grep -q 'version = "0.0.1"' mcpp.lock \ + || fail "mcpp.lock does not record the pinned version 0.0.1" mcpp.lock +grep -q 'version = "0.0.2"' mcpp.lock \ + && fail "mcpp.lock records 0.0.2 -- the pin (not \"latest\") should have won" mcpp.lock + +# ── C, and the shared root-position toolchain: read compile_commands.json ── +python3 - "$GCC_VER" > cdb_check.log 2>&1 <<'EOF' || fail "compile_commands.json did not show a's inherited flags/toolchain" cdb_check.log +import json, sys +gcc_ver = sys.argv[1] +entries = json.load(open("compile_commands.json")) +def find(suffix): + for e in entries: + if e["file"].replace("\\", "/").endswith(suffix): + return e + sys.exit(f"no compile_commands.json entry ending in {suffix}") +a_entry = find("a/src/a.cppm") +main_entry = find("src/main.cpp") +if "-DWS_FLAG=1" not in a_entry["arguments"]: + sys.exit("[workspace.build] cxxflags did not reach a.cppm's compile command") +a_cxx = a_entry["arguments"][0] +main_cxx = main_entry["arguments"][0] +if a_cxx != main_cxx: + sys.exit(f"member 'a' and the root were not compiled with the same " + f"toolchain ({a_cxx!r} vs {main_cxx!r})") +if gcc_ver not in a_cxx: + sys.exit(f"member 'a' was not compiled with the workspace's " + f"[toolchain] gcc@{gcc_ver} ({a_cxx!r})") +EOF + +"$MCPP" run > run.log 2>&1 || fail "the program did not return 0 (pinned_value() + WS_FLAG != 2)" run.log + +echo "PASS: 805_rooted_workspace_reaches_its_own_path_dependency" diff --git a/tests/e2e/806_dash_p_resolves_the_package_first.sh b/tests/e2e/806_dash_p_resolves_the_package_first.sh new file mode 100755 index 000000000..9266f0f1e --- /dev/null +++ b/tests/e2e/806_dash_p_resolves_the_package_first.sh @@ -0,0 +1,95 @@ +#!/usr/bin/env bash +# requires: gcc +# 806 -- `-p, --package ` resolves a member's package identity first, and +# its directory (docs/07 §5.3's historical spellings) only as a fallback +# (mcpp#725). Before the fix, both resolvers (src/build/prepare/manifest.cpp, +# src/project.cppm resolve_member_dir) matched only a member's directory +# basename or its `[workspace] members` path -- the option's own name, +# `--package`, promised a package that neither ever read. +set -e + +TMP=$(mktemp -d) +trap 'rm -rf "$TMP"' EXIT +fail() { echo "FAIL: $1"; [ -n "${2:-}" ] && cat "$2"; exit 1; } + +export MCPP_HOME="$TMP/mcpp-home" +source "$(dirname "$0")/_inherit_toolchain.sh" + +mkdir -p "$TMP/ws" && cd "$TMP/ws" + +member() { # member [] + local dir="$1" name="$2" ns="${3:-}" + mkdir -p "$dir" + { + echo "[package]" + [ -n "$ns" ] && echo "namespace = \"$ns\"" + echo "name = \"$name\"" + echo "version = \"0.1.0\"" + echo + echo "[targets.$name]" + echo "kind = \"lib\"" + echo + echo "[build]" + echo "sources = [\"x.cpp\"]" + } > "$dir/mcpp.toml" + echo "int ${name//[^a-zA-Z0-9_]/_}_x() { return 0; }" > "$dir/x.cpp" +} + +# `modules/base`'s package is `ws-base`; `ns1/common` and `ns2/common` share +# the bare package name `ws-common` under two different namespaces. +member "modules/base" "ws-base" +member "ns1/common" "ws-common" "ns1" +member "ns2/common" "ws-common" "ns2" +cat > mcpp.toml <<'EOF' +[workspace] +members = ["modules/base", "ns1/common", "ns2/common"] +EOF + +# ── package name, directory basename, and full path all select the member ── +for filter in ws-base base modules/base; do + rm -rf modules/base/compile_commands.json + "$MCPP" build -p "$filter" > "sel-$(basename "$filter").log" 2>&1 \ + || fail "-p $filter did not build" "sel-$(basename "$filter").log" + [ -f modules/base/compile_commands.json ] \ + || fail "-p $filter did not select modules/base" + grep -qi 'warning' "sel-$(basename "$filter").log" \ + && fail "-p $filter warned when no other member could conflict" \ + "sel-$(basename "$filter").log" +done +echo "ok: package name, basename and path all select the one member" + +# ── two members share a bare name under different namespaces ─────────────── +"$MCPP" build -p ws-common > ambiguous.log 2>&1 \ + && fail "-p ws-common should have been refused as ambiguous" ambiguous.log +grep -q 'ns1.ws-common' ambiguous.log \ + || fail "the ambiguity refusal must name 'ns1.ws-common'" ambiguous.log +grep -q 'ns2.ws-common' ambiguous.log \ + || fail "the ambiguity refusal must name 'ns2.ws-common'" ambiguous.log + +"$MCPP" build -p ns1.ws-common > qns1.log 2>&1 || fail "-p ns1.ws-common did not build" qns1.log +[ -f ns1/common/compile_commands.json ] || fail "-p ns1.ws-common did not select ns1/common" +"$MCPP" build -p ns2.ws-common > qns2.log 2>&1 || fail "-p ns2.ws-common did not build" qns2.log +[ -f ns2/common/compile_commands.json ] || fail "-p ns2.ws-common did not select ns2/common" +echo "ok: the bare name is refused; each qualified name selects its own member" + +# ── a second member's directory basename collides with the first's package ─ +member "dup/ws-base" "other" +cat > mcpp.toml <<'EOF' +[workspace] +members = ["modules/base", "ns1/common", "ns2/common", "dup/ws-base"] +EOF +rm -rf modules/base/compile_commands.json + +"$MCPP" build -p ws-base > name-vs-dir.log 2>&1 \ + || fail "-p ws-base did not build once a same-named directory existed" name-vs-dir.log +[ -f modules/base/compile_commands.json ] \ + || fail "-p ws-base must still select the package 'ws-base' (modules/base)" +[ -f dup/ws-base/compile_commands.json ] \ + && fail "-p ws-base must not have built dup/ws-base" +grep -qi 'warning' name-vs-dir.log \ + || fail "-p ws-base must warn once a directory shares its spelling" name-vs-dir.log +grep -q 'dup/ws-base' name-vs-dir.log \ + || fail "the warning must name the other member's path ('dup/ws-base')" name-vs-dir.log +echo "ok: the package name outranks another member's directory, with a warning" + +echo "PASS: 806_dash_p_resolves_the_package_first" diff --git a/tests/e2e/807_a_lib_root_imports_its_own_package.sh b/tests/e2e/807_a_lib_root_imports_its_own_package.sh new file mode 100755 index 000000000..103d933ce --- /dev/null +++ b/tests/e2e/807_a_lib_root_imports_its_own_package.sh @@ -0,0 +1,87 @@ +#!/usr/bin/env bash +# requires: gcc +# 807_a_lib_root_imports_its_own_package.sh — a host-module package's lib root +# may import another unit of its own package (mcpp#720). +# +# The units of a host-module package are compiled in a list, and each unit sees +# only the BMIs of the units ahead of it. The list is ordered by what each unit +# imports. Until mcpp#720 the lib root was placed at the head of the list before +# the order was computed, so a lib root that imports a sibling was compiled +# first and failed with "module 'repro.helper' not found", while the same +# package built when `lib.path` named the sibling instead. +# +# The lib root is now the first node of the same sort: it is still emitted +# first whenever it imports nothing of its own package, so every package that +# built before keeps its order. +set -e + +TMP=$(mktemp -d) +trap "rm -rf $TMP" EXIT +cd "$TMP" + +make_rules() { # $1 = directory, $2 = "explicit" | "convention" + local dir="$1" form="$2" + mkdir -p "$dir/src" + { + printf '[package]\nnamespace = "repro"\nname = "rules"\nversion = "0.1.0"\n\n' + if [[ "$form" == explicit ]]; then + printf '[lib]\npath = "src/rules.cppm"\n\n' + fi + printf '[build]\nsources = ["src/*.cppm"]\n\n[targets.rules]\nkind = "lib"\n' + } > "$dir/mcpp.toml" + cat > "$dir/src/rules.cppm" <<'EOF' +export module repro.rules; +import repro.helper; +export int answer() { return helper_answer(); } +EOF + cat > "$dir/src/helper.cppm" <<'EOF' +export module repro.helper; +export int helper_answer() { return 42; } +EOF +} + +make_app() { # $1 = directory + local dir="$1" + mkdir -p "$dir" + cat > "$dir/mcpp.toml" <<'EOF' +[package] +namespace = "repro" +name = "app" +version = "0.1.0" + +[build-dependencies] +"repro.rules" = { path = "../rules", host-module = true } + +[build] +sources = ["main.cpp"] + +[targets.app] +kind = "bin" +main = "main.cpp" +EOF + cat > "$dir/build.mcpp" <<'EOF' +import repro.rules; +int main() { return answer() == 42 ? 0 : 1; } +EOF + printf 'int main() { return 0; }\n' > "$dir/main.cpp" +} + +for form in explicit convention; do + rm -rf "$form" + mkdir "$form" + make_rules "$form/rules" "$form" + make_app "$form/app" + ( cd "$form/app" && "$MCPP" build > build.log 2>&1 ) || { + cat "$form/app/build.log" + echo "FAIL: $form lib root: a lib root that imports its own package's unit did not build" + exit 1 + } + grep -q "build.mcpp running" "$form/app/build.log" || { + cat "$form/app/build.log" + echo "FAIL: $form lib root: the build program did not run" + exit 1 + } + echo "ok: $form lib root imports its own package's unit" +done + +echo "PASS: 807_a_lib_root_imports_its_own_package" diff --git a/tests/e2e/808_device_source_is_not_a_compile_unit.sh b/tests/e2e/808_device_source_is_not_a_compile_unit.sh new file mode 100755 index 000000000..b64fe5825 --- /dev/null +++ b/tests/e2e/808_device_source_is_not_a_compile_unit.sh @@ -0,0 +1,84 @@ +#!/usr/bin/env bash +# requires: gcc +# 808 -- a rule-claimed device source is not a C++ compile unit of the plan +# (mcpp#724 §1, design 2026-09-27 §4.1). +# +# `SourceKind::Device` names a file the engine has no compile rule for: it is +# compiled, if at all, by the package's build program through an action, never +# by a `cxx_object` edge. Until now the plan turned every graph unit into a +# `CompileUnit`, device units included, so `build.ninja` carried a dead +# `cxx_object` edge for the device source and both `compile_commands.json` and +# the S1 document (`emit build-database`) listed it with a compiler command +# that never ran on the file. Criteria, on examples/12's own fixture (a `.toy` +# kernel a rule claims and turns into generated C++): +# A. `build.ninja` has no `cxx_object` edge whose input is the device source. +# B. `compile_commands.json` lists no entry for it. +# C. The S1 document (`emit build-database --format json`) lists no unit for +# it, in any set. +# D. `--spec compile-commands` agrees with B. +# E. The build still succeeds and the kernel still reaches the program: the +# fix must not stop the device source from being compiled BY THE RULE. +set -e + +SRC="$(cd "$(dirname "$0")/../.." && pwd)/examples/12-a-new-device-language" +[[ -d "$SRC" ]] || { echo "FAIL: $SRC is missing"; exit 1; } + +TMP=$(mktemp -d) +trap "rm -rf $TMP" EXIT +cp -r "$SRC" "$TMP/ex" +find "$TMP/ex" -maxdepth 3 -type d -name target -exec rm -rf {} + 2>/dev/null || true +find "$TMP/ex" -maxdepth 3 -name mcpp.lock -delete 2>/dev/null || true + +cd "$TMP/ex/app" + +# ── A, B: build.ninja and compile_commands.json ───────────────────────────── +"$MCPP" build --configure-only > configure.log 2>&1 \ + || { cat configure.log; echo "FAIL: configure-only did not succeed"; exit 1; } + +ninja_file="$(find target -name build.ninja | head -1)" +[[ -n "$ninja_file" ]] || { echo "FAIL: no build.ninja generated"; exit 1; } + +if grep -qE '^build [^:]*: cxx_object [^|]*answer\.toy' "$ninja_file"; then + grep -nE '^build [^:]*: cxx_object [^|]*answer\.toy' "$ninja_file" + echo "FAIL: A: build.ninja carries a cxx_object edge for the device source" + exit 1 +fi +echo "ok: A, no cxx_object edge for the device source" + +[[ -f compile_commands.json ]] || { echo "FAIL: no compile_commands.json"; exit 1; } +if grep -q 'answer\.toy' compile_commands.json; then + echo "FAIL: B: compile_commands.json lists the device source" + exit 1 +fi +echo "ok: B, compile_commands.json lists no entry for the device source" + +# ── C: the S1 document ─────────────────────────────────────────────────── +"$MCPP" emit build-database --format json > s1.json 2> s1.err \ + || { cat s1.err; echo "FAIL: emit build-database failed"; exit 1; } +python3 - s1.json <<'EOF' || { echo "FAIL: C"; exit 1; } +import json, sys +db = json.load(open(sys.argv[1]))["data"]["database"] +for s in db["sets"]: + for u in s["translation-units"]: + if u["source"].endswith("answer.toy"): + print("device source listed in set", s["name"], u["source"]) + sys.exit(1) +EOF +echo "ok: C, the S1 document lists no unit for the device source" + +# ── D: --spec compile-commands ─────────────────────────────────────────── +"$MCPP" emit build-database --spec compile-commands > cc.json 2> cc.err \ + || { cat cc.err; echo "FAIL: emit --spec compile-commands failed"; exit 1; } +if grep -q 'answer\.toy' cc.json; then + echo "FAIL: D: --spec compile-commands lists the device source" + exit 1 +fi +echo "ok: D, --spec compile-commands agrees" + +# ── E: the fix must not stop the rule from compiling the kernel ──────────── +"$MCPP" build > build.log 2>&1 || { cat build.log; echo "FAIL: E: the build failed"; exit 1; } +out="$("$MCPP" run 2>&1 | tail -1)" +[[ "$out" == *"= 42"* ]] || { echo "FAIL: E: the kernel did not reach the program: '$out'"; exit 1; } +echo "ok: E, the device source is still compiled by the rule and the build still runs" + +echo "PASS: 808 a device source is not a compile unit" diff --git a/tests/e2e/809_failed_build_program_hides_no_device_source.sh b/tests/e2e/809_failed_build_program_hides_no_device_source.sh new file mode 100755 index 000000000..0d4e69cdc --- /dev/null +++ b/tests/e2e/809_failed_build_program_hides_no_device_source.sh @@ -0,0 +1,101 @@ +#!/usr/bin/env bash +# requires: python3 +# 809 -- a package that declares a device source and whose build.mcpp does not +# compile is described by `emit build-database` with its one real diagnostic, +# `MCPP_BUILD_DATABASE_PROGRAM_FAILED`, not with the device-source orphan +# refusal (mcpp#724 side finding A, design 2026-09-27 §4.2, SPEC-005 R5.2 +# amended). +# +# `rules` declares a feature whose `device_extensions`/`rule_module` classify +# `.dev` as a device source when active; `app809` activates it and lists a +# `.dev` file in `[build] sources`, and its own `build.mcpp` is invalid C++. +# The build program never runs, so it applies none of its directives — no +# action claims the device source, which used to read as an orphan and fail +# the whole member (a check whose premise is the program's directives ran +# anyway). Criteria: +# A. exit status 1. +# B. `data` is present, and `app809` is described (its manifest, toolchain +# and module graph do not depend on the failed program). +# C. the one diagnostic is `MCPP_BUILD_DATABASE_PROGRAM_FAILED`, severity +# error, `path` "build.mcpp". +# D. that diagnostic does not mention the device source or the orphan text. +set -e + +TMP=$(mktemp -d) +trap "rm -rf $TMP" EXIT +fail() { echo "FAIL: $1"; shift; for f in "$@"; do echo "--- $f ---"; cat "$f" 2>/dev/null; done; exit 1; } +PY=python3 + +mkdir -p "$TMP/rules/src" "$TMP/app/src/kernels" + +cat > "$TMP/rules/mcpp.toml" <<'EOF' +[package] +name = "rules" +namespace = "t809" +version = "0.1.0" + +[features] +default = [] + +[features.x] +sources = ["src/rules.cppm"] +rule_module = "t809.rules" +device_extensions = [".dev"] +EOF +cat > "$TMP/rules/src/rules.cppm" <<'EOF' +export module t809.rules; +export namespace t809::rules { +inline bool noop() { return true; } +} +EOF + +cd "$TMP/app" +cat > mcpp.toml <<'EOF' +[package] +name = "app809" +version = "0.1.0" + +[language] +standard = "c++23" + +[dependencies] +t809.rules = { path = "../rules", features = ["x"] } + +[build] +sources = ["src/*.cpp", "src/kernels/*.dev"] +EOF +cat > src/main.cpp <<'EOF' +int main() { return 0; } +EOF +echo "not a real kernel" > src/kernels/k.dev +# Invalid C++: the build program never runs, and applies none of its +# directives -- the same shape 789 uses for its `nocompile` leg. +cat > build.mcpp <<'EOF' +int main() { this is not valid c++ } +EOF + +set +e +"$MCPP" emit build-database --format json > out.json 2> out.err +rc=$? +set -e +[ "$rc" = 1 ] || fail "A: exit status $rc, expected 1" out.err out.json +echo "ok: A, exit status 1" + +"$PY" - out.json <<'EOF' || fail "B/C/D: the envelope" out.json +import json, sys +e = json.load(open(sys.argv[1])) +assert "data" in e, e +sets = {s["name"]: s for s in e["data"]["database"]["sets"]} +assert "app809" in sets, sets +diags = e["diagnostics"] +assert len(diags) == 1, diags +diag = diags[0] +assert diag["code"] == "MCPP_BUILD_DATABASE_PROGRAM_FAILED", diag +assert diag["severity"] == "error", diag +assert diag["path"] == "build.mcpp", diag +assert "k.dev" not in diag["message"], diag["message"] +assert "device sources that no action compiles" not in diag["message"], diag["message"] +EOF +echo "ok: B, the package is described; C, the one diagnostic is PROGRAM_FAILED with path build.mcpp; D, it mentions no device source" + +echo "PASS: 809 a failed build program hides no device source" diff --git a/tests/e2e/810_two_packages_deploy_the_same_bytes_to_one_name.sh b/tests/e2e/810_two_packages_deploy_the_same_bytes_to_one_name.sh new file mode 100755 index 000000000..d9b5ac0ad --- /dev/null +++ b/tests/e2e/810_two_packages_deploy_the_same_bytes_to_one_name.sh @@ -0,0 +1,173 @@ +#!/usr/bin/env bash +# requires: gcc elf +# 810_two_packages_deploy_the_same_bytes_to_one_name.sh -- mcpp#723, SPEC-007 +# R4.2: two or more source paths for one deploy destination become ONE stage +# edge, placed when every source is byte-identical, refused at build time +# (naming every source and the destination) otherwise. +# +# The fixture is the shape #723 was filed for, without naming any plugin: a +# path dependency and its consumer each run their own build.mcpp action that +# generates a file under their own MCPP_OUT_DIR, and each deploys it to the +# SAME name (`mcpp::deploy`, docs/30-build-mcpp.md). Before this change, +# `add_deploy` (src/build/plan.cppm) refused two different source PATHS for +# one destination even when the bytes are identical, and `mcpp emit +# build-database` failed the same way. +# +# Criteria: +# 1. identical bytes: the build succeeds, and one file is placed at +# bin/shared/shared.bin; +# 2. build.ninja carries exactly one `stage_file` edge for that destination, +# with BOTH sources as inputs; +# 3. `mcpp emit build-database` succeeds on the same project (it used to +# fail at planning); +# 4. different bytes: the build fails, naming both sources and the +# destination -- not at planning, at staging. +set -e + +TMP=$(mktemp -d) +trap "rm -rf $TMP" EXIT + +fail() { echo "FAIL: $1"; shift; for f in "$@"; do echo "--- $f ---"; cat "$f" 2>/dev/null; done; exit 1; } + +MCPP="${MCPP:-mcpp}" + +write_dep() { # $1 = the res.txt content + mkdir -p "$TMP/dep/src" + cat > "$TMP/dep/mcpp.toml" <<'TOML' +[package] +name = "dep" +version = "0.1.0" + +[targets.dep] +kind = "lib" +TOML + cat > "$TMP/dep/src/lib.cppm" <<'CPP' +export module dep; +export int dep_value() { return 1; } +CPP + cat > "$TMP/dep/gen.sh" <<'SH' +#!/usr/bin/env bash +set -e +cp "$1" "$2" +SH + chmod +x "$TMP/dep/gen.sh" + printf '%s\n' "$1" > "$TMP/dep/res.txt" + cat > "$TMP/dep/build.mcpp" <<'CPP' +import mcpp; +#include +int main() { + const std::string root = mcpp::manifest_dir(); + const std::string src = root + "/res.txt"; + const std::string out = std::string(mcpp::out_dir()) + "/gen/shared.bin"; + + mcpp::action a; + a.id = "dep-gen-shared"; + a.role = "source"; + a.arg((root + "/gen.sh").c_str()).arg(src.c_str()).arg(out.c_str()) + .input(src.c_str()) + .output(out.c_str()) + .submit(); + + mcpp::deploy(out.c_str(), "shared"); + return 0; +} +CPP +} + +write_app() { # $1 = the res.txt content + mkdir -p "$TMP/app/src" + cat > "$TMP/app/mcpp.toml" <<'TOML' +[package] +name = "app" +version = "0.1.0" + +[dependencies] +dep = { path = "../dep" } +TOML + cat > "$TMP/app/src/main.cpp" <<'CPP' +import dep; +int main() { return dep_value() == 1 ? 0 : 1; } +CPP + cat > "$TMP/app/gen.sh" <<'SH' +#!/usr/bin/env bash +set -e +cp "$1" "$2" +SH + chmod +x "$TMP/app/gen.sh" + printf '%s\n' "$1" > "$TMP/app/res.txt" + cat > "$TMP/app/build.mcpp" <<'CPP' +import mcpp; +#include +int main() { + const std::string root = mcpp::manifest_dir(); + const std::string src = root + "/res.txt"; + const std::string out = std::string(mcpp::out_dir()) + "/gen/shared.bin"; + + mcpp::action a; + a.id = "app-gen-shared"; + a.role = "source"; + a.arg((root + "/gen.sh").c_str()).arg(src.c_str()).arg(out.c_str()) + .input(src.c_str()) + .output(out.c_str()) + .submit(); + + mcpp::deploy(out.c_str(), "shared"); + return 0; +} +CPP +} + +find_graph() { find target -name build.ninja | head -1; } + +# ── 1, 2, 3: identical bytes ──────────────────────────────────────────────── +write_dep 'shared payload' +write_app 'shared payload' + +cd "$TMP/app" +"$MCPP" build > build.log 2>&1 || fail "the build failed on identical bytes" build.log +DEPLOYED=$(find target -path '*/bin/shared/shared.bin' | head -1) +[ -n "$DEPLOYED" ] || fail "bin/shared/shared.bin was not placed" build.log +grep -qx "shared payload" "$DEPLOYED" \ + || fail "the placed file does not carry the shared bytes" "$DEPLOYED" +echo "PASS: identical bytes place one file" + +G=$(find_graph) +[ -n "$G" ] || fail "no build.ninja" build.log +STAGE_LINES=$(grep -c "^build .*shared/shared\.bin : stage_file" "$G" || true) +[ "$STAGE_LINES" -eq 1 ] \ + || fail "expected exactly one stage_file edge for shared/shared.bin, found $STAGE_LINES" "$G" +STAGE_LINE=$(grep "^build .*shared/shared\.bin : stage_file" "$G") +# The dependency's own build.mcpp runs in the CONSUMING project's tree +# (`target/.build-mcpp/deps/dep@/...`, 111_dep_build_mcpp.sh), so its +# source is distinguished by that path segment; the app's own source is the +# sibling under `target/.build-mcpp/out/...` with no `deps/` segment. +INPUTS=$(echo "$STAGE_LINE" | sed 's/^.*: stage_file //') +[ "$(echo "$INPUTS" | wc -w)" -eq 2 ] \ + || fail "expected exactly two inputs on the stage_file edge, got: $INPUTS" "$G" +echo "$INPUTS" | tr ' ' '\n' | grep -qF "deps/dep@" \ + || fail "the edge does not list the dependency's source" "$G" +echo "$INPUTS" | tr ' ' '\n' | grep -v "deps/dep@" | grep -q "\.build-mcpp/out/gen/shared\.bin$" \ + || fail "the edge does not also list the app's own source" "$G" +echo "PASS: one stage_file edge lists both sources" + +"$MCPP" emit build-database > emit.log 2>&1 || fail "emit build-database failed on identical bytes" emit.log +echo "PASS: emit build-database succeeds" + +# ── 4: different bytes -- refused at build time, naming both sources ─────── +write_app 'a different payload' +touch src/main.cpp # past the whole-project no-op fast path; see e2e 139 +set +e +"$MCPP" build > collision.log 2>&1 +rc=$? +set -e +[ "$rc" -ne 0 ] || fail "different bytes for one destination were accepted" collision.log +grep -q "disagree" collision.log \ + || fail "the refusal does not say the sources disagree" collision.log +grep -qF "deps/dep@" collision.log \ + || fail "the refusal does not name the dependency's source" collision.log +grep -q '\.build-mcpp/out/gen/shared\.bin' collision.log \ + || fail "the refusal does not name the app's own source" collision.log +grep -q "shared.bin" collision.log || fail "the refusal does not name the destination" collision.log +echo "PASS: different bytes are refused at build time, naming both sources" + +echo "PASS: 810_two_packages_deploy_the_same_bytes_to_one_name" diff --git a/tests/e2e/811_place_dlls_never_overwrites_a_declared_deploy.sh b/tests/e2e/811_place_dlls_never_overwrites_a_declared_deploy.sh new file mode 100755 index 000000000..156d23cc2 --- /dev/null +++ b/tests/e2e/811_place_dlls_never_overwrites_a_declared_deploy.sh @@ -0,0 +1,126 @@ +#!/usr/bin/env bash +# requires: windows +# 811_place_dlls_never_overwrites_a_declared_deploy.sh -- mcpp#723 self-review, +# SPEC-007 R4.2/R4.3: one destination, one writer. +# +# `mcpp place-dlls` (SPEC-007 R4.3) places, beside a linked PE program, every +# non-system DLL it imports transitively from a runtime search directory. It +# used to do this unconditionally, so a declared deploy (`[runtime] deploy`) +# naming the same file could be overwritten by whichever edge ran last. This +# fixture gives `app`'s runtime search directory the REAL `libmathkit.dll` +# (which `app.exe` imports) and, through a plain declared deploy, a DIFFERENT +# file under the very same name. The deploy list is the single authority for +# that destination: `place-dlls` must leave the declared file alone and warn +# about the difference instead of silently choosing one or the other. +# +# Modelled on 794_a_windows_program_finds_a_dll_through_runtime_search_dir.sh, +# which this reuses for the "how does a program come to import a DLL it did +# not declare as a dependency" half of the fixture. +set -e +source "$(dirname "$0")/_host_path.sh" + +TMP=$(mktemp -d) +trap "rm -rf $TMP" EXIT +cd "$TMP" + +mkdir -p mathkit/src +cat > mathkit/src/mathkit.cppm <<'EOF' +export module mathkit; +export namespace mk { int answer(); } +EOF +cat > mathkit/src/impl.cpp <<'EOF' +module mathkit; +namespace mk { int answer() { return 42; } } +extern "C" __declspec(dllexport) int mk_answer_extern() { return mk::answer(); } +EOF +cat > mathkit/mcpp.toml <<'EOF' +[package] +name = "mathkit" +version = "0.1.0" +[toolchain] +windows = "gcc@16.1.0" +[build] +sources = ["src/*.cppm", "src/*.cpp"] +[targets.mathkit] +kind = "shared" +EOF + +MCPP="${MCPP:-mcpp}" +( cd mathkit && "$MCPP" build > build.log 2>&1 ) || { + cat mathkit/build.log; echo "FAIL: mathkit build failed"; exit 1; } +DLL="$(find mathkit/target -name 'libmathkit.dll' | head -1)" +IMP="$(find mathkit/target -name 'libmathkit.dll.a' | head -1)" +[[ -n "$DLL" && -n "$IMP" ]] || { find mathkit/target -type f; echo "FAIL: mathkit did not produce a DLL + import library"; exit 1; } +DLLDIR_HOST="$(host_path "$(cd "$(dirname "$DLL")" && pwd)")" +LIBDIR_HOST="$(host_path "$(cd "$(dirname "$IMP")" && pwd)")" + +mkdir -p app/src +cat > app/src/main.cpp <<'EOF' +extern "C" int mk_answer_extern(); +int main() { return mk_answer_extern() == 42 ? 0 : 1; } +EOF + +# A file named EXACTLY like the real DLL app.exe will import, with different +# bytes -- `deploy`'s destination filename is the source's own filename +# (docs/04 SS2.11), so the source itself must be named `libmathkit.dll`. +mkdir -p app/deploy +printf 'not the real DLL\n' > app/deploy/libmathkit.dll + +cat > app/mcpp.toml <<'EOF' +[package] +name = "app" +version = "0.1.0" +[toolchain] +windows = "gcc@16.1.0" +[targets.app] +kind = "bin" +main = "src/main.cpp" + +[runtime] +deploy = [ { from = "deploy/libmathkit.dll", to = "." } ] +EOF + +cat > app/build.mcpp < build.log 2>&1 || { cat build.log; echo "FAIL: app build failed"; exit 1; } +EXE="$(find target -name 'app.exe' | head -1)" +[[ -n "$EXE" ]] || { cat build.log; echo "FAIL: no app.exe produced"; exit 1; } +BINDIR="$(dirname "$EXE")" + +# ── 1. the declared deploy's file stays; place-dlls did not overwrite it ─── +[[ -f "$BINDIR/libmathkit.dll" ]] || { echo "FAIL: libmathkit.dll is missing beside app.exe"; exit 1; } +CONTENT="$(cat "$BINDIR/libmathkit.dll")" +[[ "$CONTENT" == "not the real DLL" ]] || { + echo "FAIL: libmathkit.dll beside app.exe is not the declared deploy's file (got: $CONTENT)" + exit 1; } + +# ── 2. the build reports the difference ───────────────────────────────────── +grep -qi "libmathkit.dll" build.log || { + cat build.log + echo "FAIL: the build does not mention libmathkit.dll at all" + exit 1; } +grep -q "warning:" build.log || { + cat build.log + echo "FAIL: the build does not warn about the difference" + exit 1; } + +# ── 3. a second build (place-dlls re-running) leaves the same file in place ─ +"$MCPP" build > build2.log 2>&1 || { cat build2.log; echo "FAIL: second build failed"; exit 1; } +CONTENT2="$(cat "$BINDIR/libmathkit.dll")" +[[ "$CONTENT2" == "not the real DLL" ]] || { + echo "FAIL: a second build replaced the declared deploy's file (got: $CONTENT2)" + exit 1; } + +echo "PASS: 811_place_dlls_never_overwrites_a_declared_deploy" diff --git a/tests/e2e/812_an_index_floor_is_a_closing_tip.sh b/tests/e2e/812_an_index_floor_is_a_closing_tip.sh new file mode 100755 index 000000000..0a6651093 --- /dev/null +++ b/tests/e2e/812_an_index_floor_is_a_closing_tip.sh @@ -0,0 +1,72 @@ +#!/usr/bin/env bash +# requires: unix-shell +# 812_an_index_floor_is_a_closing_tip.sh — an index that requires a newer mcpp +# is not an error of the run. +# +# An index is data and mcpp is the program; `min_mcpp` routes, it does not +# terminate. Criteria: +# A. A refresh that brings in a tree requiring a newer mcpp succeeds; the +# guard keeps the previous tree, and the run ends with exactly one `tip:` +# line, which is its last line of output. No `error:` line is printed. +# B. `mcpp self doctor` lists an index whose floor this mcpp does not meet, +# with the floor, and points at E0006. +set -e +source "$(dirname "$0")/_host_path.sh" + +TMP=$(mktemp -d) +trap 'rm -rf "$TMP"' EXIT +fail() { echo "FAIL: $1"; [ -n "${2:-}" ] && cat "$2"; exit 1; } + +export MCPP_HOME="$TMP/home" +source "$(dirname "$0")/_inherit_toolchain.sh" + +DATA="$MCPP_HOME/registry/data/mcpplibs" +rm -rf "$DATA" +mkdir -p "$DATA/pkgs/f" +printf '[index]\nspec = "1"\nmin_mcpp = "0.0.1"\n' > "$DATA/index.toml" +printf 'package = { spec = "1", name = "fixture", type = "package" }\n' > "$DATA/pkgs/f/fixture.lua" + +# The stub stands in for xlings: an index update rewrites the tree so that it +# requires a newer mcpp than any release, which is what a published floor bump +# looks like to an older client. Every other call succeeds and does nothing. +mkdir -p "$TMP/bin" +cat > "$TMP/bin/xlings" < "$DATA/index.toml" + case " \$* " in *" interface "*) echo '{"kind":"result","exitCode":0}' ;; esac + ;; +esac +exit 0 +EOF +chmod +x "$TMP/bin/xlings" +STUB_HOST="$(host_path "$TMP/bin/xlings")" +{ + grep -v '^binary' "$MCPP_HOME/config.toml" 2>/dev/null | sed '/^\[xlings\]/d' + printf '\n[xlings]\nbinary = "%s"\n' "$STUB_HOST" +} > "$TMP/config.toml" +mv "$TMP/config.toml" "$MCPP_HOME/config.toml" + +# ── A. the refresh ends with one tip, and no error ────────────────────────── +cd "$TMP" +"$MCPP" index update > a.out 2> a.err || fail "A: the refresh failed" a.err +grep -q '^error:' a.err && fail "A: a refresh that kept a usable index printed an error" a.err +[ "$(grep -c '^tip:' a.err)" = 1 ] || fail "A: expected exactly one tip line" a.err +tail -1 a.err | grep -q '^tip: .*requires a newer mcpp' \ + || fail "A: the tip is not the last line of the run" a.err +grep -q '^tip: .*requires mcpp >= 9999.9.9.9' a.err \ + || fail "A: the tip does not name the version the index asks for" a.err +grep -q 'min_mcpp = "0.0.1"' "$DATA/index.toml" \ + || fail "A: the guard did not keep the previous, usable tree" "$DATA/index.toml" +echo "ok: A. a floor bump seen by a refresh is one closing tip" + +# ── B. doctor reports the state ───────────────────────────────────────────── +printf '[index]\nspec = "1"\nmin_mcpp = "9999.9.9.9"\n' > "$DATA/index.toml" +"$MCPP" self doctor > b.out 2>&1 || true +grep -q "index 'mcpplibs' requires mcpp >= 9999.9.9.9" b.out \ + || fail "B: doctor does not list the index this mcpp cannot read" b.out +grep -q 'E0006' b.out || fail "B: doctor does not point at E0006" b.out +echo "ok: B. doctor lists an index whose floor this mcpp does not meet" + +echo "PASS: 812_an_index_floor_is_a_closing_tip" diff --git a/tests/e2e/813_conditional_dialect_cxxflags_root_only.sh b/tests/e2e/813_conditional_dialect_cxxflags_root_only.sh new file mode 100755 index 000000000..a8ee12ddf --- /dev/null +++ b/tests/e2e/813_conditional_dialect_cxxflags_root_only.sh @@ -0,0 +1,224 @@ +#!/usr/bin/env bash +# requires: gcc +# 813_conditional_dialect_cxxflags_root_only.sh -- #717, design 2026-09-27 §6: +# `[target..build] dialect_cxxflags` is a graph-wide switch under a +# target condition. Before the fix (measured on 2026.9.27.1), the key inside +# `[target.linux.build]` was reported as "unsupported key 'dialect_cxxflags' +# (ignored)" and reached no command at all. +# +# Four properties, each with its own scenario below: +# +# 1. a matching row reaches the std BMI prebuild, the module scan and every +# translation unit, in the order [build] (root, unconditional) then the +# matching [target..build] (§6.2: "entries are appended"); +# 2. a selector that does not match the resolved target contributes nothing; +# 3. switching a manifest between a matching and a non-matching row rebuilds +# the std BMI, and switching back reuses the earlier std-module cache +# entry rather than rebuilding it a second time (A, then B, then A); +# 4. a DEPENDENCY's own `dialect_cxxflags` (conditional or not) reaches no +# command, and toggling it does not change the OUTPUT DIRECTORY +# fingerprint of the graph it belongs to (the graph-wide keys must be +# excluded from a package's own per-package fingerprint contribution, +# design §6.2 finding 7). +set -e + +TMP=$(mktemp -d) +trap "rm -rf $TMP" EXIT + +fail() { echo "FAIL: $1"; shift; for f in "$@"; do echo "--- $f ---"; cat "$f" 2>/dev/null; done; exit 1; } + +export MCPP_HOME="$TMP/mcpp-home" +source "$(dirname "$0")/_inherit_toolchain.sh" + +std_module_json_for() { # $1 = a substring the std_flag field must contain + grep -rl -- "$1" "$MCPP_HOME/build-cache/v1/std" 2>/dev/null | head -1 +} + +echo "== 813/1: a matching row reaches the std BMI, the scan and every TU, after the root's own [build] ==" + +mkdir -p "$TMP/one/src" +cd "$TMP/one" +cat > mcpp.toml <<'TOML' +[package] +name = "app717a" +version = "0.1.0" +standard = "c++23" + +[build] +dialect_cxxflags = ["-DBASE717"] + +[targets.app717a] +kind = "bin" +main = "src/main.cpp" + +[target.linux.build] +dialect_cxxflags = ["-DX717A"] +TOML +printf 'import std;\nint main() { std::println("ok"); return 0; }\n' > src/main.cpp + +"$MCPP" build --toolchain gcc@16.1.0 > build1.log 2>&1 || fail "the build with a matching [target.linux.build] row failed" build1.log +ninja1=$(find target -name build.ninja | head -1) +[ -n "$ninja1" ] || fail "no build.ninja produced" build1.log + +cxxflags_line=$(grep -E '^cxxflags' "$ninja1" | head -1) +echo "$cxxflags_line" | grep -qF -- '-DX717A' \ + || fail "the row's flag is not in build.ninja's global cxxflags (reaches neither scan nor TUs)" "$ninja1" +echo "$cxxflags_line" | grep -qF -- '-DBASE717' \ + || fail "the root's own unconditional flag is missing from cxxflags" "$ninja1" +# Order: the root's own [build] precedes the matching conditional row. +base_pos=$(echo "$cxxflags_line" | grep -bo -- '-DBASE717' | head -1 | cut -d: -f1) +cond_pos=$(echo "$cxxflags_line" | grep -bo -- '-DX717A' | head -1 | cut -d: -f1) +[ "$base_pos" -lt "$cond_pos" ] \ + || fail "the conditional row's flag does not follow the root's own [build] flag" "$ninja1" +echo " ok: build.ninja's cxxflags carries -DBASE717 then -DX717A (reaches the scan and every TU)" + +grep -qF -- '-DX717A' compile_commands.json \ + || fail "-DX717A is missing from compile_commands.json" compile_commands.json +echo " ok: the flag reaches compile_commands.json" + +stdA=$(std_module_json_for '-DX717A') +[ -n "$stdA" ] || fail "no std-module.json records a std_flag containing -DX717A" build1.log +grep -qF -- '-DBASE717' "$stdA" \ + || fail "the std module's cached prebuild command is missing the root's unconditional flag" "$stdA" +echo " ok: the std BMI prebuild's recorded command carries both flags ($stdA)" + +echo "== 813/2: a selector that does not match the resolved target contributes nothing ==" + +mkdir -p "$TMP/two/src" +cd "$TMP/two" +cat > mcpp.toml <<'TOML' +[package] +name = "app717b" +version = "0.1.0" +standard = "c++23" + +[build] +dialect_cxxflags = ["-DBASE717"] + +[targets.app717b] +kind = "bin" +main = "src/main.cpp" + +[target.windows.build] +dialect_cxxflags = ["-DX717A"] +TOML +printf 'import std;\nint main() { std::println("ok"); return 0; }\n' > src/main.cpp + +"$MCPP" build --toolchain gcc@16.1.0 > build2.log 2>&1 || fail "the build with a non-matching [target.windows.build] row failed" build2.log +ninja2=$(find target -name build.ninja | head -1) +grep -qF -- '-DX717A' "$ninja2" compile_commands.json \ + && fail "-DX717A from a non-matching [target.windows.build] row reached a Linux build" "$ninja2" +grep -E '^cxxflags' "$ninja2" | grep -qF -- '-DBASE717' \ + || fail "the root's own unconditional flag is missing" "$ninja2" +echo " ok: the non-matching row's flag reaches neither build.ninja nor compile_commands.json" + +echo "== 813/3: switching between a matching and a non-matching row rebuilds the std BMI (A, then B, then A) ==" + +mkdir -p "$TMP/cyc/src" +cd "$TMP/cyc" +write_cyc() { # $1 = the selector ("linux" or "windows") + cat > mcpp.toml < src/main.cpp + +write_cyc linux +rm -rf target compile_commands.json +"$MCPP" build --toolchain gcc@16.1.0 > cyc-a1.log 2>&1 || fail "build A (matching) failed" cyc-a1.log +grep -qF -- '-DCYC717' "$(find target -name build.ninja | head -1)" \ + || fail "build A did not carry -DCYC717" cyc-a1.log +stdA1=$(std_module_json_for '-DCYC717') +[ -n "$stdA1" ] || fail "no std-module.json for build A" cyc-a1.log +echo " ok: build A (linux, matching) carries -DCYC717 ($stdA1)" + +# `target/` (and the project's own compile_commands.json) is removed between +# steps so that only ONE build.ninja / compile_commands.json ever exists at a +# time -- this scenario's subject is the SHARED std BMI cache under +# $MCPP_HOME/build-cache (content-addressed, independent of the project's own +# output directory), not the project's own directory reuse, which 813/4 below +# checks on its own terms. +write_cyc windows +rm -rf target compile_commands.json +"$MCPP" build --toolchain gcc@16.1.0 > cyc-b.log 2>&1 || fail "build B (non-matching) failed" cyc-b.log +grep -qF -- '-DCYC717' "$(find target -name build.ninja | head -1)" compile_commands.json \ + && fail "build B (windows, non-matching on a Linux host) still carries -DCYC717" cyc-b.log +echo " ok: build B (windows, non-matching) carries no -DCYC717 -- a different std BMI" + +write_cyc linux +rm -rf target compile_commands.json +"$MCPP" build --toolchain gcc@16.1.0 > cyc-a2.log 2>&1 || fail "build A2 (matching again) failed" cyc-a2.log +grep -qF -- '-DCYC717' "$(find target -name build.ninja | head -1)" \ + || fail "build A2 did not carry -DCYC717" cyc-a2.log +stdA2=$(std_module_json_for '-DCYC717') +[ -n "$stdA2" ] || fail "no std-module.json for build A2" cyc-a2.log +[ "$stdA2" = "$stdA1" ] \ + || fail "switching back to the matching row did not reuse build A's std BMI cache entry" "$stdA1" "$stdA2" +echo " ok: switching back to linux reuses build A's std BMI cache entry exactly (A, then B, then A)" + +echo "== 813/4: a dependency's own dialect_cxxflags reaches no command and does not move the graph's fingerprint ==" + +mkdir -p "$TMP/dep717/src" "$TMP/four/src" +cat > "$TMP/dep717/mcpp.toml" <<'TOML' +[package] +name = "dep717" +version = "0.1.0" +standard = "c++23" + +[targets.dep717] +kind = "lib" + +[build] +dialect_cxxflags = ["-DDEPFLAG_V1"] +TOML +printf 'export module dep717;\nexport int value() { return 42; }\n' > "$TMP/dep717/src/dep717.cppm" + +cd "$TMP/four" +cat > mcpp.toml <<'TOML' +[package] +name = "app717d" +version = "0.1.0" +standard = "c++23" + +[dependencies] +dep717 = { path = "../dep717" } + +[targets.app717d] +kind = "bin" +main = "src/main.cpp" +TOML +printf 'import dep717;\nint main() { return value() == 42 ? 0 : 1; }\n' > src/main.cpp + +"$MCPP" build --toolchain gcc@16.1.0 > build4a.log 2>&1 || fail "the first build (dependency carries -DDEPFLAG_V1) failed" build4a.log +ninjas4a=$(find target -name build.ninja) +[ "$(echo "$ninjas4a" | wc -l)" -eq 1 ] || fail "more than one build.ninja after the first build" build4a.log +grep -qF -- '-DDEPFLAG_V1' "$ninjas4a" compile_commands.json \ + && fail "the dependency's own dialect_cxxflags reached a command" "$ninjas4a" +echo " ok: the dependency's -DDEPFLAG_V1 reaches neither build.ninja nor compile_commands.json" + +sed -i 's/-DDEPFLAG_V1/-DDEPFLAG_V2/' "$TMP/dep717/mcpp.toml" +"$MCPP" build --toolchain gcc@16.1.0 > build4b.log 2>&1 || fail "the second build (dependency's flag changed) failed" build4b.log +ninjas4b=$(find target -name build.ninja) +if [ "$(echo "$ninjas4b" | wc -l)" -ne 1 ]; then + echo "build.ninja files after the second build:"; echo "$ninjas4b" + fail "changing the dependency's own dialect_cxxflags left a second output directory behind (the fingerprint moved)" build4b.log +fi +if [ "$ninjas4b" != "$ninjas4a" ]; then + echo "first build.ninja: $ninjas4a"; echo "second build.ninja: $ninjas4b" + fail "changing the dependency's own dialect_cxxflags moved the graph's output directory" build4b.log +fi +grep -qF -- '-DDEPFLAG_V2' "$ninjas4b" compile_commands.json \ + && fail "the dependency's changed dialect_cxxflags reached a command" "$ninjas4b" +echo " ok: the same output directory is reused; the dependency's own key never entered the graph's fingerprint" + +echo "PASS: 813" diff --git a/tests/e2e/814_the_llvm_row_defaults_to_the_dynamic_crt.sh b/tests/e2e/814_the_llvm_row_defaults_to_the_dynamic_crt.sh new file mode 100755 index 000000000..b455382f1 --- /dev/null +++ b/tests/e2e/814_the_llvm_row_defaults_to_the_dynamic_crt.sh @@ -0,0 +1,214 @@ +#!/usr/bin/env bash +# requires: windows +# 814_the_llvm_row_defaults_to_the_dynamic_crt.sh -- #718: the CRT model is a +# property of the MSVC ABI, not of the compiler. clang++ targeting +# `x86_64-windows-msvc` (the LLVM row) used to receive no CRT model at all and +# always linked the static CRT (#649 E10, inverted at 703); it now receives +# the same model cl.exe does, spelled `-fms-runtime-lib=static`/`=dll`, and +# `toolchain-coupled` (the dynamic CRT, with the toolset's own +# vcruntime140.dll/msvcp140.dll staged beside the artifact) is the default. +# +# This checks the LLVM row's half of that: the default program imports +# vcruntime140.dll and runs from a clean PATH because the file is staged +# beside it; `cxx_runtime = "self-contained"` restores the static CRT and the +# import disappears; switching between the two keeps both std BMIs valid +# (neither clobbers the other); and `mcpp pack` carries the DLL by default +# while `--mode system` records host-coupled instead of refusing. +# +# Read only when the default toolchain here is the llvm row (703's own +# convention): a runner whose default is msvc@system prints that and asserts +# nothing, because the cl.exe cells are covered by the unit property test +# (NinjaBackendPeRuntime.CrtWordIsOneWordEqualOnCompileAndLink) and by +# 180/181. +set -e + +MCPP_HOME="${MCPP_HOME:-$HOME/.mcpp}" +TMP=$(mktemp -d) +trap "rm -rf $TMP" EXIT +fail() { echo "FAIL: $1"; shift; for f in "$@"; do echo "--- $f ---"; cat "$f" 2>/dev/null; done; exit 1; } + +OBJDUMP=$(ls "${MCPP_HOME}/registry/data/xpkgs/xim-x-llvm"/*/bin/llvm-objdump.exe 2>/dev/null | head -1) +[[ -x "$OBJDUMP" ]] || OBJDUMP=$(command -v llvm-objdump 2>/dev/null || true) +[[ -x "$OBJDUMP" ]] || OBJDUMP=$(command -v llvm-objdump.exe 2>/dev/null || true) +[[ -x "$OBJDUMP" ]] || OBJDUMP=$(command -v objdump 2>/dev/null || true) +[[ -x "$OBJDUMP" ]] || OBJDUMP=$(command -v objdump.exe 2>/dev/null || true) +imports_of() { # $1 = exe path -- the DLL names on its PE import table + [[ -x "$OBJDUMP" ]] || return 0 + "$OBJDUMP" -p "$1" 2>/dev/null | grep -i "DLL Name" +} + +cd "$TMP" +mkdir -p app/src +cat > app/src/main.cpp <<'CPP' +import std; +int main() { std::println("crt-abi-ok"); return 0; } +CPP + +write_app() { # $1 = extra [build] lines + cat > app/mcpp.toml < default.log 2>&1 || fail "the default build failed" default.log +if ! grep -q "Resolved llvm@" default.log; then + echo "READING #718: the default toolchain here is not the llvm row: $(grep -m1 'Resolved' default.log)" + echo "PASS: 814 the llvm row defaults to the dynamic CRT (not the llvm row; nothing to assert)" + exit 0 +fi + +EXE=$(find target -name "app.exe" -path "*/bin/*" | head -1) +[[ -n "$EXE" ]] || fail "no exe produced" default.log +BIN_DIR=$(dirname "$EXE") + +imports=$(imports_of "$EXE") +echo "default imports:"; echo "$imports" +echo "$imports" | grep -iq "vcruntime140" \ + || fail "the default LLVM-row program imports no vcruntime140.dll" default.log +[[ -f "$BIN_DIR/vcruntime140.dll" ]] \ + || fail "vcruntime140.dll was not staged beside the exe (found: $(ls "$BIN_DIR"))" default.log + +# Runs with the toolset removed from PATH: only the staged copy may serve it. +run_out=$(cd "$BIN_DIR" && PATH="/usr/bin:/c/Windows/System32" ./app.exe 2>&1) \ + || fail "the default program did not run with the VS directories off PATH" <(echo "$run_out") +[[ "$run_out" == *"crt-abi-ok"* ]] || fail "unexpected run output: $run_out" + +contract=$(record_of) +echo "READING #718 default record: distributable=$contract" +[ "$contract" = "toolchain-coupled" ] \ + || fail "the llvm row's undeclared default recorded '$contract', not toolchain-coupled" default.log + +STD_CACHE="$MCPP_HOME/build-cache/v1/std" +dyn_bmis=$(grep -rl -- '-fms-runtime-lib=dll' "$STD_CACHE" 2>/dev/null | wc -l | tr -d ' ') +[ "${dyn_bmis:-0}" -ge 1 ] \ + || fail "no std BMI command recorded the dynamic CRT word" default.log + +echo "ok: the default llvm-row program imports and stages vcruntime140.dll, and runs with a clean PATH" + +cd .. +write_app 'cxx_runtime = "self-contained"' +cd app +rm -rf target +"$MCPP" build > self-contained.log 2>&1 || fail "the self-contained build failed" self-contained.log +EXE=$(find target -name "app.exe" -path "*/bin/*" | head -1) +[[ -n "$EXE" ]] || fail "no exe produced (self-contained)" self-contained.log + +imports=$(imports_of "$EXE") +echo "self-contained imports:"; echo "$imports" +(echo "$imports" | grep -iqE "vcruntime|msvcp") \ + && { fail "self-contained still imports a CRT DLL" <(echo "$imports"); } || true +[[ -f "$(dirname "$EXE")/vcruntime140.dll" ]] \ + && { fail "self-contained staged vcruntime140.dll, which it should not need" self-contained.log; } || true + +contract=$(record_of) +echo "READING #718 self-contained record: distributable=$contract" +[ "$contract" = "self-contained" ] \ + || fail "self-contained recorded '$contract'" self-contained.log + +static_bmis=$(grep -rl -- '-fms-runtime-lib=static' "$STD_CACHE" 2>/dev/null | wc -l | tr -d ' ') +[ "${static_bmis:-0}" -ge 1 ] \ + || fail "no std BMI command recorded the static CRT word" self-contained.log + +echo "ok: self-contained restores the static CRT and imports no vcruntime/msvcp DLL" + +# Switching back (A, B, A): the dynamic-CRT identity must still be valid and +# reusable, not clobbered by the intervening static build. +cd .. +write_app '' +cd app +rm -rf target +"$MCPP" build > default-again.log 2>&1 || fail "the second default build failed" default-again.log +EXE=$(find target -name "app.exe" -path "*/bin/*" | head -1) +imports=$(imports_of "$EXE") +echo "$imports" | grep -iq "vcruntime140" \ + || fail "switching back to the default lost the dynamic CRT" default-again.log +echo "ok: switching A, B, A keeps both std BMI identities valid" + +# `mcpp pack`: the default mode carries the DLL. An explicit `--mode system` +# on a DEFAULTED (never-declared) `toolchain-coupled` contract outranks the +# default and resolves it to host-coupled instead of refusing — the +# contradiction stays reserved for an EXPLICIT `toolchain-coupled` (unchanged, +# and not this fixture's row: it never wrote `cxx_runtime` down). +# +# `resolution.json` is a property of the BUILD, not of one `pack` invocation, +# so the observable here is the package's own contents: `--format dir` avoids +# needing an unzip step to look inside. +MARKER="$TMP/marker"; touch "$MARKER" +pack_out=$("$MCPP" pack 2>&1) || fail "the default pack failed" <(echo "$pack_out") +DIST=$(find target/dist -maxdepth 1 -mindepth 1 -newer "$MARKER" | head -1) +[[ -n "$DIST" ]] || fail "no pack output produced" <(echo "$pack_out"; find target/dist) +{ [[ -d "$DIST" ]] && find "$DIST" -iname "vcruntime140.dll" | grep -q .; } \ + || unzip -l "$DIST" 2>/dev/null | grep -qi "vcruntime140.dll" \ + || fail "the default pack did not carry vcruntime140.dll" <(echo "$pack_out"; echo "$DIST") + +touch "$MARKER" +system_out=$("$MCPP" pack --mode system --format dir 2>&1) \ + || fail "--mode system refused a defaulted (never-declared) toolchain-coupled contract" \ + <(echo "$system_out") +SYSTEM_DIST=$(find target/dist -maxdepth 1 -mindepth 1 -newer "$MARKER" | head -1) +[[ -n "$SYSTEM_DIST" ]] || fail "--mode system produced no output" <(echo "$system_out") +find "$SYSTEM_DIST" -iname "vcruntime140.dll" | grep -q . \ + && { fail "--mode system still bundled vcruntime140.dll for a defaulted (host-coupled) contract" \ + <(echo "$system_out"; find "$SYSTEM_DIST"); } || true + +echo "ok: mcpp pack carries the DLL by default, and --mode system resolves the defaulted contract to host-coupled instead of refusing" + +# A dependency's `[build] cxxflags` reach its own units after the graph's +# flags, so a CRT word there would compile them against the other CRT: one +# image, two CRTs. A contradicting word is refused, naming the dependency; +# an agreeing one is accepted without a warning (`cxx_runtime` is the root's). +cd "$TMP" +mkdir -p dep/src +printf 'export module dep;\nexport int dep_value() { return 7; }\n' > dep/src/dep.cppm +write_dep() { # $1 = the dependency's cxxflags word + cat > dep/mcpp.toml < user/src/main.cpp +cat > user/mcpp.toml <<'TOML' +[package] +name = "user" +version = "0.1.0" + +[dependencies] +dep = { path = "../dep" } +TOML +cd user +write_dep_here() { (cd .. && write_dep "$1"); } +write_dep_here "-fms-runtime-lib=static" +if "$MCPP" build > dep-static.log 2>&1; then + fail "a dependency's contradicting CRT word was not refused" dep-static.log +fi +grep -q "dependency 'dep'" dep-static.log && grep -q -- "-fms-runtime-lib=static" dep-static.log \ + || fail "the refusal does not name the dependency and the word" dep-static.log +write_dep_here "-fms-runtime-lib=dll" +"$MCPP" build > dep-dll.log 2>&1 || fail "a dependency's agreeing CRT word was refused" dep-dll.log +if grep -q "agrees with the CRT model" dep-dll.log; then + fail "a dependency's agreeing CRT word was warned" dep-dll.log +fi +echo "ok: a dependency's contradicting CRT word is refused by name; an agreeing one is accepted silently" + +echo "PASS: 814 the llvm row defaults to the dynamic CRT" diff --git a/tests/e2e/815_the_database_describes_what_a_rule_generates.sh b/tests/e2e/815_the_database_describes_what_a_rule_generates.sh new file mode 100755 index 000000000..1881f0a6b --- /dev/null +++ b/tests/e2e/815_the_database_describes_what_a_rule_generates.sh @@ -0,0 +1,99 @@ +#!/usr/bin/env bash +# requires: unix-shell python3 +# 815_the_database_describes_what_a_rule_generates.sh — the S1 document names +# the files a build program's actions generate (S1 0.3.0 section 7.2, mcpp#724). +# +# `emit build-database` plans in a directory of its own and runs no action +# (SPEC-005 R2.1, R2.5), so a header an action generates is absent from the +# include directory the units' arguments name. The plan knows the generating +# step; the document states it, with the path a `mcpp build` of the same +# configuration writes. Criteria: +# A. the package's set carries `ide.generated` entries: the header, with its +# generator (id, inputs, arguments), and the generated include directory; +# the package's test set, whose unit includes the same header, carries +# the header too (SPEC-005 R3.12: every set of the package); +# B. each entry's `build-path` is under the project's own `target/`, and a +# following `mcpp build` writes the header at exactly that path; +# C. compile_commands.json carries no such field. +set -e + +TMP=$(mktemp -d) +trap 'rm -rf "$TMP"' EXIT +fail() { echo "FAIL: $1"; [ -n "${2:-}" ] && cat "$2"; exit 1; } +cd "$TMP" +mkdir -p proj/src proj/templates proj/tests +cd proj + +cat > mcpp.toml <<'EOF' +[package] +name = "gendb" +version = "0.1.0" +EOF +cat > templates/answer.h.in <<'EOF' +#pragma once +inline int generated_answer() { return 42; } +EOF +cat > src/main.cpp <<'EOF' +#include "answer.h" +int main() { return generated_answer() == 42 ? 0 : 1; } +EOF +cat > tests/test_answer.cpp <<'EOF' +#include "answer.h" +int main() { return generated_answer() == 42 ? 0 : 1; } +EOF +cat > build.mcpp <<'EOF' +import std; +import mcpp; +int main() { + const std::string gen = std::string(mcpp::out_dir()) + "/gen"; + const std::string in = std::string(mcpp::manifest_dir()) + "/templates/answer.h.in"; + const std::string out = gen + "/answer.h"; + mcpp::action a; + a.id = "gen:answer"; + a.role = mcpp::roles::source; + a.arg("cp").arg(in.c_str()).arg(out.c_str()) + .input(in.c_str()) + .output(out.c_str()) + .submit(); + mcpp::include_dir(gen.c_str()); +} +EOF + +"$MCPP" emit build-database --format json > db.json 2> db.err || fail "the plan failed" db.err +python3 - "$PWD" > check.out 2>&1 <<'PY' || { cat check.out; exit 1; } +import json, os, sys +root = os.path.realpath(sys.argv[1]) +env = json.load(open("db.json")) +sets = env["data"]["database"]["sets"] +gen = [g for s in sets for g in s.get("ide", {}).get("generated", [])] +headers = [g for g in gen if g["kind"] == "header"] +dirs = [g for g in gen if g["kind"] == "directory"] +assert headers, f"A: no generated header entry in {gen}" +h = headers[0] +assert h["path"].endswith(os.path.join("gen", "answer.h")), h +assert h["generator"]["id"] == "gen:answer", h +assert any(i.endswith("answer.h.in") for i in h["generator"]["inputs"]), h +assert h["generator"]["arguments"][0] == "cp", h +assert dirs and dirs[0]["path"].endswith("gen"), f"A: no generated directory entry in {gen}" +by_set = {s["name"]: s.get("ide", {}).get("generated", []) for s in sets} +assert "gendb:test" in by_set, f"A: no test set among {sorted(by_set)}" +for name in ("gendb", "gendb:test"): + assert any(g["kind"] == "header" and g["generator"]["id"] == "gen:answer" for g in by_set[name]), \ + f"A: set {name} does not name the generated header: {by_set[name]}" +for g in (h, dirs[0]): + bp = os.path.realpath(g["build-path"]) + assert bp.startswith(os.path.join(root, "target") + os.sep), f"B: {g['build-path']} is not under {root}/target" +open("build-path.txt", "w").write(h["build-path"]) +print("ok") +PY +echo "ok: A. the set names the generated header, its step, and the generated directory" + +"$MCPP" build > build.log 2>&1 || fail "the build failed" build.log +[ -f "$(cat build-path.txt)" ] || fail "B: the build did not write the header at the stated build-path" build.log +echo "ok: B. a build writes the header at the stated build-path" + +"$MCPP" emit build-database --spec compile-commands > cdb.json 2> cdb.err || fail "the compile database failed" cdb.err +if grep -q '"generated"' cdb.json; then fail "C: compile_commands.json carries a generated field" cdb.json; fi +echo "ok: C. compile_commands.json carries no generated field" + +echo "PASS: 815_the_database_describes_what_a_rule_generates" diff --git a/tests/e2e/816_a_git_dependency_reports_its_fetch.sh b/tests/e2e/816_a_git_dependency_reports_its_fetch.sh new file mode 100755 index 000000000..1996a3614 --- /dev/null +++ b/tests/e2e/816_a_git_dependency_reports_its_fetch.sh @@ -0,0 +1,50 @@ +#!/usr/bin/env bash +# requires: unix-shell +# 816_a_git_dependency_reports_its_fetch.sh — fetching a git dependency is +# reported with the renderer every other acquisition uses (W11). +# +# The clone's output used to be captured whole and shown only on failure, so a +# large repository printed nothing until it had arrived. The download phase is +# now drawn as a bar. When stdout is not a terminal the bar is one line when the +# item starts and one line when it finishes, with no carriage return and no +# erase sequence, so a CI log is not filled with repaints. Criteria: +# A. the build prints a `Fetching ` start line and a finish line; +# B. the captured output contains no carriage return and no ESC byte. +set -e + +TMP=$(mktemp -d) +trap 'rm -rf "$TMP"' EXIT +fail() { echo "FAIL: $1"; [ -n "${2:-}" ] && cat "$2"; exit 1; } + +mkdir -p "$TMP/repo/src" && cd "$TMP/repo" +cat > mcpp.toml <<'EOF' +[package] +namespace = "probe816" +name = "gdep" +version = "0.1.0" + +[targets.gdep] +kind = "lib" +EOF +printf 'export module probe816.gdep;\nexport int gdep_value() { return 7; }\n' > src/gdep.cppm +git init -q -b main . && git add -A \ + && git -c user.email=e2e@mcpp -c user.name=e2e commit -qm init + +mkdir -p "$TMP/consumer/src" && cd "$TMP/consumer" +cat > mcpp.toml < src/main.cpp + +"$MCPP" build > build.log 2>&1 || fail "the build failed" build.log +grep -q "Fetching file://$TMP/repo" build.log || fail "A: no start line for the fetch" build.log +grep -q "Fetching file://$TMP/repo.* done" build.log || fail "A: no finish line for the fetch" build.log +if LC_ALL=C grep -q $'\r' build.log; then fail "B: the log carries a carriage return" build.log; fi +if LC_ALL=C grep -q $'\x1b' build.log; then fail "B: the log carries an ESC byte" build.log; fi +echo "ok: A, B" +echo "PASS: 816_a_git_dependency_reports_its_fetch" diff --git a/tests/e2e/817_emit_writes_no_xlings_json_under_private_work_dir.sh b/tests/e2e/817_emit_writes_no_xlings_json_under_private_work_dir.sh new file mode 100755 index 000000000..32c37be3f --- /dev/null +++ b/tests/e2e/817_emit_writes_no_xlings_json_under_private_work_dir.sh @@ -0,0 +1,123 @@ +#!/usr/bin/env bash +# requires: unix-shell python3 +# 817 -- `emit build-database` writes nothing into a project that declares +# `[xlings] deps` (mcpp#724 side finding B, design 2026-09-27 §4.3, SPEC-005 +# R2.1). +# +# e2e 688 already repeats a project-tree digest across `emit build-database`, +# but its fixture declares no `[xlings]` payloads, so the branch that writes +# `/.mcpp/.xlings.json` (`ensure_project_index_dir`, reached through +# `src/build/prepare/xlings.cpp`) is never taken there -- the criterion missed +# the case that mattered. This repeats it on a fixture that does declare +# `[xlings] deps`, with a stub xlings as in e2e 733, so it needs no network. +# +# The call site used the private `work_dir` (`emit`'s planning cache under +# `$MCPP_HOME/cache/build-database/`) for the custom-indices half of that +# file only when `runtimeSelection.ownerRoot == workRoot`; `ownerRoot` is +# always the real project root, so under `emit`'s private work_dir the +# runtime-environment half (deps/subos/workspace) went to the project instead. +# Criteria: +# A. the project tree is byte-identical before and after `emit +# build-database`. +# B. `/.mcpp/.xlings.json` does not exist afterwards. +# C. the envelope's `effects` do not include `write-project`. +# D. the private work directory DOES gain a `.xlings.json` naming the +# declared dependency -- so the fix is "written at the private root", +# not "never written at all", and xlings can still resolve it there. +set -e + +TMP=$(mktemp -d) # the measured tree +OUT=$(mktemp -d) # everything this script writes, outside the measured tree +cleanup() { rm -rf "$TMP" "$OUT"; } +trap cleanup EXIT +fail() { echo "FAIL: $1"; shift; for f in "$@"; do echo "--- $f ---"; cat "$f" 2>/dev/null; done; exit 1; } +PY=python3 + +export MCPP_HOME="$OUT/home" +source "$(dirname "$0")/_inherit_toolchain.sh" + +mkdir -p "$OUT/bin" +cat > "$OUT/bin/xlings" <<'EOF' +#!/usr/bin/env bash +echo "$*" >> "${STUB_LOG:?}" +exit 0 +EOF +chmod +x "$OUT/bin/xlings" +{ + grep -v '^binary' "$MCPP_HOME/config.toml" 2>/dev/null | sed '/^\[xlings\]/d' + printf '\n[xlings]\nbinary = "%s"\n' "$OUT/bin/xlings" +} > "$OUT/config.toml" +mv "$OUT/config.toml" "$MCPP_HOME/config.toml" + +mkdir -p "$TMP/proj/src" +cat > "$TMP/proj/mcpp.toml" <<'EOF' +[package] +name = "proj817" +version = "0.1.0" + +[xlings] +deps = ["definitely-not-a-real-package-817"] +EOF +echo 'int main() { return 0; }' > "$TMP/proj/src/main.cpp" +cd "$TMP/proj" + +tree_digest() { + "$PY" - "$TMP" <<'EOF' +import hashlib, os, sys +root = sys.argv[1] +h = hashlib.sha256() +for dirpath, dirnames, filenames in os.walk(root): + dirnames.sort() + rel = os.path.relpath(dirpath, root) + h.update(("D " + rel + "\n").encode()) + for name in sorted(filenames): + with open(os.path.join(dirpath, name), "rb") as f: + h.update(("F " + os.path.join(rel, name) + " ").encode() + hashlib.sha256(f.read()).hexdigest().encode() + b"\n") +print(h.hexdigest()) +EOF +} + +before=$(tree_digest) + +STUB_LOG="$OUT/xlings.log" "$MCPP" emit build-database --format json \ + > "$OUT/env.json" 2> "$OUT/env.err" \ + || fail "emit build-database exited non-zero" "$OUT/env.err" "$OUT/env.json" + +after=$(tree_digest) + +# ── A ────────────────────────────────────────────────────────────────────── +[ "$before" = "$after" ] || fail "A: the project tree changed" +echo "ok: A, the project tree is byte-identical before and after" + +# ── B ────────────────────────────────────────────────────────────────────── +[ ! -e "$TMP/proj/.mcpp/.xlings.json" ] \ + || fail "B: emit wrote /.mcpp/.xlings.json" +echo "ok: B, no .mcpp/.xlings.json in the project" + +# ── C ────────────────────────────────────────────────────────────────────── +"$PY" - "$OUT/env.json" <<'EOF' || fail "C: the envelope" "$OUT/env.json" +import json, sys +e = json.load(open(sys.argv[1])) +assert "write-project" not in e["effects"], e["effects"] +EOF +echo "ok: C, the envelope reports no write-project effect" + +# ── D ────────────────────────────────────────────────────────────────────── +work_dir=$("$PY" -c ' +import json, sys +db = json.load(open(sys.argv[1]))["data"]["database"] +for s in db["sets"]: + for u in s["translation-units"]: + norm = u["object"].replace("\\", "/") + if "/target/" in norm: + print(norm.split("/target/")[0]) + sys.exit(0) +sys.exit(1) +' "$OUT/env.json") || fail "D: could not recover the private work directory" "$OUT/env.json" +found=$(find "$work_dir" -name ".xlings.json" 2>/dev/null | head -1) +[ -n "$found" ] || fail "D: no .xlings.json under the private work directory $work_dir" +grep -q 'definitely-not-a-real-package-817' "$found" \ + || fail "D: the private .xlings.json does not name the declared dependency" "$found" +echo "ok: D, the runtime environment is written at the private root instead, where xlings resolves it from" + +echo "PASS: 817 emit writes no .xlings.json into the project" diff --git a/tests/e2e/818_a_declared_deploy_outranks_a_search_dir_dll_cross.sh b/tests/e2e/818_a_declared_deploy_outranks_a_search_dir_dll_cross.sh new file mode 100644 index 000000000..3719ebff7 --- /dev/null +++ b/tests/e2e/818_a_declared_deploy_outranks_a_search_dir_dll_cross.sh @@ -0,0 +1,107 @@ +#!/usr/bin/env bash +# requires: mingw-cross +# 818_a_declared_deploy_outranks_a_search_dir_dll_cross.sh -- the Linux-hosted +# stand-in for 811 (SPEC-007 R4.2/R4.3, mcpp#723): one destination, one +# writer. 811 runs on a Windows runner only; the property does not depend on +# the host, because both the plan-time scan of the runtime search directories +# and the post-link `place-dlls` edge run on whatever host builds, so this +# builds the same fixture for `x86_64-windows-gnu` with the cross toolchain. +# +# `app` declares a deploy of a file named `libmathkit.dll` with bytes of its +# own, and names, as a runtime search directory, the directory that holds the +# real `libmathkit.dll` that `app.exe` imports. Criteria: +# 1. the build succeeds: the DLL the scan finds is a derived source and +# yields to the declared destination, instead of meeting it at `mcpp +# stage` as a second source with different bytes; +# 2. the declared file is the one beside the program; +# 3. the build warns about the difference, naming the DLL; +# 4. a second build leaves the declared file in place. +set -e +source "$(dirname "$0")/_host_path.sh" + +TMP=$(mktemp -d) +trap "rm -rf $TMP" EXIT +cd "$TMP" +fail() { echo "FAIL: $1"; shift; for f in "$@"; do echo "--- $f ---"; cat "$f" 2>/dev/null; done; exit 1; } + +TRIPLE=x86_64-windows-gnu +MCPP="${MCPP:-mcpp}" + +mkdir -p mathkit/src +cat > mathkit/src/mathkit.cppm <<'EOF' +export module mathkit; +export namespace mk { int answer(); } +EOF +cat > mathkit/src/impl.cpp <<'EOF' +module mathkit; +namespace mk { int answer() { return 42; } } +extern "C" int mk_answer_extern() { return mk::answer(); } +EOF +cat > mathkit/mcpp.toml <<'EOF' +[package] +name = "mathkit" +version = "0.1.0" +[build] +sources = ["src/*.cppm", "src/*.cpp"] +[targets.mathkit] +kind = "shared" +EOF +( cd mathkit && "$MCPP" build --target "$TRIPLE" > build.log 2>&1 ) \ + || fail "mathkit build failed" mathkit/build.log +DLL="$(find mathkit/target -name 'libmathkit.dll' | head -1)" +IMP="$(find mathkit/target -name 'libmathkit.dll.a' | head -1)" +[[ -n "$DLL" && -n "$IMP" ]] || fail "mathkit did not produce a DLL and an import library" +DLLDIR=$(realpath "$(dirname "$DLL")") +LIBDIR=$(realpath "$(dirname "$IMP")") + +mkdir -p app/src app/deploy +cat > app/src/main.cpp <<'EOF' +extern "C" int mk_answer_extern(); +int main() { return mk_answer_extern() == 42 ? 0 : 1; } +EOF +printf 'not the real DLL\n' > app/deploy/libmathkit.dll +cat > app/mcpp.toml <<'EOF' +[package] +name = "app" +version = "0.1.0" +[targets.app] +kind = "bin" +main = "src/main.cpp" + +[runtime] +deploy = [ { from = "deploy/libmathkit.dll", to = "." } ] +EOF +cat > app/build.mcpp < build.log 2>&1 \ + || fail "1: the build failed; the search directory's DLL met the declared deploy as a second source" build.log +EXE="$(find target/$TRIPLE -name 'app.exe' | head -1)" +[[ -n "$EXE" ]] || fail "no app.exe produced" build.log +BINDIR="$(dirname "$EXE")" +echo "ok: 1. the build succeeds" + +[[ "$(cat "$BINDIR/libmathkit.dll" 2>/dev/null)" == "not the real DLL" ]] \ + || fail "2: the file beside app.exe is not the declared deploy's" build.log +echo "ok: 2. the declared file is the one beside the program" + +grep -q "warning:.*libmathkit.dll\|libmathkit.dll.*differ" build.log \ + || grep -A3 "warning:" build.log | grep -q "libmathkit.dll" \ + || fail "3: the build does not warn about the difference, naming libmathkit.dll" build.log +echo "ok: 3. the build warns about the difference" + +"$MCPP" build --target "$TRIPLE" > build2.log 2>&1 || fail "the second build failed" build2.log +[[ "$(cat "$BINDIR/libmathkit.dll")" == "not the real DLL" ]] \ + || fail "4: a second build replaced the declared file" build2.log +echo "ok: 4. a second build leaves the declared file in place" + +echo "PASS: 818_a_declared_deploy_outranks_a_search_dir_dll_cross" diff --git a/tests/unit/test_build_stage.cpp b/tests/unit/test_build_stage.cpp index 3768c7ae4..98ada9888 100644 --- a/tests/unit/test_build_stage.cpp +++ b/tests/unit/test_build_stage.cpp @@ -191,3 +191,118 @@ TEST(BuildStage, VerifyModeParsing) { EXPECT_EQ(parse_verify("nonsense"), Verify::Content); EXPECT_EQ(parse_verify(""), Verify::Content); } + +// ── SPEC-007 R4.2 (mcpp#723): a destination with more than one source ────── + +TEST(BuildStageFiles, OneSourceIsIdenticalToStageFile) { + // The whole point of `stage_files`: a project with exactly one source per + // destination -- every project before this feature -- must not notice a + // difference. This is not asserted by string-matching a ninja line (that + // is `test_ninja_backend.cpp`'s job); here it is the outcome that must + // match `stage_file`'s exactly. + Tmp tmp; + auto src = tmp.path / "src.bin"; + auto dst = tmp.path / "dst.bin"; + write_file(src, "payload"); + + auto r = stage_files({src}, dst, no_retry()); + ASSERT_TRUE(r.has_value()) << r.error().message; + EXPECT_TRUE(r->copied); + EXPECT_EQ(read_file(dst), "payload"); +} + +TEST(BuildStageFiles, AgreeingSourcesArePlaced) { + Tmp tmp; + auto a = tmp.path / "a.bin"; + auto b = tmp.path / "b.bin"; + auto c = tmp.path / "c.bin"; + auto dst = tmp.path / "dst.bin"; + write_file(a, "shared bytes"); + write_file(b, "shared bytes"); + write_file(c, "shared bytes"); + + auto r = stage_files({a, b, c}, dst, no_retry()); + ASSERT_TRUE(r.has_value()) << r.error().message; + EXPECT_TRUE(r->copied); + EXPECT_EQ(read_file(dst), "shared bytes"); +} + +TEST(BuildStageFiles, AnAlreadyEquivalentDestinationIsNotTouched) { + // The #311 property (BuildStage.EquivalentDestinationIsNotTouched) must + // survive going through the multi-source entry point too. + Tmp tmp; + auto a = tmp.path / "a.bin"; + auto b = tmp.path / "b.bin"; + auto dst = tmp.path / "dst.bin"; + write_file(a, "same-bytes"); + write_file(b, "same-bytes"); + write_file(dst, "same-bytes"); + auto before = std::filesystem::file_time_type::clock::now() - std::chrono::hours{2}; + std::filesystem::last_write_time(dst, before); + auto recorded = std::filesystem::last_write_time(dst); + + auto r = stage_files({a, b}, dst, no_retry()); + ASSERT_TRUE(r.has_value()); + EXPECT_FALSE(r->copied); + EXPECT_EQ(std::filesystem::last_write_time(dst), recorded); +} + +TEST(BuildStageFiles, DisagreeingSourcesFailNamingEveryOneAndTheDestination) { + Tmp tmp; + auto a = tmp.path / "qtbase_zh_CN.qm"; + auto b = tmp.path / "other" / "qtbase_zh_CN.qm"; + auto dst = tmp.path / "bin" / "translations" / "qt_zh_CN.qm"; + write_file(a, "catalog A"); + write_file(b, "catalog B"); + + auto r = stage_files({a, b}, dst, no_retry()); + ASSERT_FALSE(r.has_value()); + EXPECT_NE(r.error().message.find(a.string()), std::string::npos) + << r.error().message; + EXPECT_NE(r.error().message.find(b.string()), std::string::npos) + << r.error().message; + EXPECT_NE(r.error().message.find(dst.string()), std::string::npos) + << r.error().message; + // Refused, not silently resolved to either source: the destination is + // untouched (no partial write from a rejected attempt). + EXPECT_FALSE(std::filesystem::exists(dst)); +} + +TEST(BuildStageFiles, ThreeSourcesWhereOnlyTheLastDisagreesAreAllNamed) { + // A pairwise comparison against only the first neighbour would miss a + // mismatch between the second and third; every source must be compared. + Tmp tmp; + auto a = tmp.path / "a.bin"; + auto b = tmp.path / "b.bin"; + auto c = tmp.path / "c.bin"; + auto dst = tmp.path / "dst.bin"; + write_file(a, "X"); + write_file(b, "X"); + write_file(c, "Y"); + + auto r = stage_files({a, b, c}, dst, no_retry()); + ASSERT_FALSE(r.has_value()); + EXPECT_NE(r.error().message.find(c.string()), std::string::npos) + << r.error().message; +} + +TEST(BuildStageFiles, AMissingSourceIsNamedAsMissingNotAsADisagreement) { + Tmp tmp; + auto a = tmp.path / "a.bin"; + auto b = tmp.path / "never-written.bin"; + auto dst = tmp.path / "dst.bin"; + write_file(a, "X"); + + auto r = stage_files({a, b}, dst, no_retry()); + ASSERT_FALSE(r.has_value()); + EXPECT_NE(r.error().message.find("does not exist"), std::string::npos) << r.error().message; + EXPECT_NE(r.error().message.find(b.string()), std::string::npos) << r.error().message; + EXPECT_EQ(r.error().message.find("disagree"), std::string::npos) << r.error().message; + EXPECT_FALSE(std::filesystem::exists(dst)); +} + +TEST(BuildStageFiles, EmptySourceListIsAnError) { + Tmp tmp; + auto r = stage_files({}, tmp.path / "dst.bin", no_retry()); + ASSERT_FALSE(r.has_value()); +} diff --git a/tests/unit/test_cache_key.cpp b/tests/unit/test_cache_key.cpp index bcec3a7df..1bb3ae339 100644 --- a/tests/unit/test_cache_key.cpp +++ b/tests/unit/test_cache_key.cpp @@ -628,6 +628,34 @@ TEST(CacheKey, PicIsRecordedInEntryJsonSoAHitCanBeAudited) { EXPECT_EQ(j["profile"]["pic"], true); } +// The CRT model on the MSVC ABI is whole-build like PIC (#718): switching +// `cxx_runtime` must not be served an object compiled against the other CRT. +// Off the MSVC ABI the axis is empty, so no other entry's key moves. +TEST(CacheKey, TheMsvcCrtModelChangesTheKeyOnlyOnTheMsvcAbi) { + auto key_for = [](std::string triple, mcpp::toolchain::CompilerId id, + std::string cxxRuntime) { + mcpp::toolchain::Toolchain tc; + tc.compiler = id; + tc.targetTriple = std::move(triple); + mcpp::manifest::Manifest m; + m.buildConfig.cxxRuntime = std::move(cxxRuntime); + auto b = ck::build_axes(tc, m, "-std=c++23", {}, {}); + return std::pair{b.crt, ck::key_hex(b, pkg())}; + }; + using mcpp::toolchain::CompilerId; + for (auto id : {CompilerId::Clang, CompilerId::MSVC}) { + auto [crtStatic, keyStatic] = key_for("x86_64-pc-windows-msvc", id, "self-contained"); + auto [crtDynamic, keyDynamic] = key_for("x86_64-pc-windows-msvc", id, "toolchain-coupled"); + EXPECT_FALSE(crtStatic.empty()); + EXPECT_NE(crtStatic, crtDynamic); + EXPECT_NE(keyStatic, keyDynamic); + } + auto [crtA, keyA] = key_for("x86_64-linux-gnu", CompilerId::GCC, "self-contained"); + auto [crtB, keyB] = key_for("x86_64-linux-gnu", CompilerId::GCC, "toolchain-coupled"); + EXPECT_TRUE(crtA.empty()); + EXPECT_TRUE(crtB.empty()); +} + TEST(CacheKey, PicDefaultsOffSoExistingEntriesKeepTheirIdentity) { ck::BuildAxes fresh; EXPECT_FALSE(fresh.pic); diff --git a/tests/unit/test_distribution.cpp b/tests/unit/test_distribution.cpp index 34d96f255..25e247950 100644 --- a/tests/unit/test_distribution.cpp +++ b/tests/unit/test_distribution.cpp @@ -215,6 +215,52 @@ TEST(Distribution, MingwParity) { EXPECT_EQ(dist::resolve(in).unitFlags, " -static"); } +// #718 §7.2/§7.3: the MSVC-ABI whole-project default. Undeclared resolves to +// toolchain-coupled when the toolset has a redistributable, and to +// host-coupled — silently — when it does not; `linkage = "static"` or an +// explicit `self-contained` both mean the static CRT regardless. +TEST(Distribution, MsvcAbiDefaultContractRow) { + EXPECT_EQ(dist::msvc_abi_default_contract(/*staticCrt=*/false, /*hasRedist=*/true), + dist::Contract::ToolchainCoupled); + EXPECT_EQ(dist::msvc_abi_default_contract(/*staticCrt=*/false, /*hasRedist=*/false), + dist::Contract::HostCoupled); + EXPECT_EQ(dist::msvc_abi_default_contract(/*staticCrt=*/true, /*hasRedist=*/true), + dist::Contract::SelfContained); + EXPECT_EQ(dist::msvc_abi_default_contract(/*staticCrt=*/true, /*hasRedist=*/false), + dist::Contract::SelfContained) + << "a static CRT needs no redistributable to be self-contained"; +} + +// `role_contracts` takes the SAME default for every role once +// `ContractStatement::msvcAbiDefault` is set — a per-role judgement about a +// format's hazard (the way MinGW's PE cell has one) has no place on this ABI. +TEST(Distribution, RoleContractsUsesTheMsvcAbiDefaultForEveryRole) { + dist::ContractStatement s; + s.msvcAbiDefault = dist::Contract::ToolchainCoupled; + auto c = dist::role_contracts(s, dist::Format::Pe, {}); + EXPECT_EQ(c.program, dist::Contract::ToolchainCoupled); + EXPECT_EQ(c.intermediate, dist::Contract::ToolchainCoupled); + EXPECT_EQ(c.tests, dist::Contract::ToolchainCoupled); + EXPECT_EQ(c.shared, dist::Contract::ToolchainCoupled); + EXPECT_FALSE(c.programStated); + EXPECT_FALSE(c.sharedStated); + + // An explicit statement still outranks the default, on this ABI as on + // every other. + s.cxxRuntime = "host-coupled"; + auto explicitC = dist::role_contracts(s, dist::Format::Pe, {}); + EXPECT_EQ(explicitC.program, dist::Contract::HostCoupled); + EXPECT_TRUE(explicitC.programStated); + + // Off the MSVC ABI (`msvcAbiDefault` unset), the format's own per-role + // defaults are untouched — SharedLibrary still gets the ELF-only + // toolchain-coupled default, not the MSVC-ABI one. + dist::ContractStatement elf; + auto elfC = dist::role_contracts(elf, dist::Format::Elf, {}); + EXPECT_EQ(elfC.program, dist::Contract::SelfContained); + EXPECT_EQ(elfC.shared, dist::Contract::ToolchainCoupled); +} + // MSVC's self-contained form IS the /MT runtime, and mcpp emits it — the // switch is `msvcStaticCrt`, derived once by `msvc_wants_static_crt` from the // two manifest keys that mean the same physical thing on this ABI. @@ -899,38 +945,47 @@ TEST(Distribution, TheOlderSpellingStatesEveryRole) { } // --------------------------------------------------------------------------- -// #649 E10 -- clang on the MSVC ABI is given no CRT model, and its driver links -// the static CRT (`-defaultlib:libcmt`). The table recorded `host-coupled` -// beside an artifact that imports no vcruntime DLL. It now records what the -// row delivers, and an explicit request the row does not deliver says so. -TEST(Distribution, ClangOnTheMsvcAbiRecordsTheStaticCrtItsDriverLinks) { +// #649 E10, closed by #718. Clang on the MSVC ABI used to be given no CRT +// model at all (`MechanismInput::msvcCrtModelEmitted`), so the table recorded +// `host-coupled` beside an artifact that imported no vcruntime DLL. Every +// MSVC-ABI row now receives the SAME model — one helper, `msvc_abi_crt_word`, +// spells it for cl and for clang++ alike — so the mechanism table has no cell +// left that distinguishes a driver: it reads only `msvcStaticCrt`, +// `requested` and `explicitRequest`, none of which name a compiler. That is +// the property that made the field removable rather than merely unused; a +// row without a redistributable directory is a planning-time refusal for an +// explicit `toolchain-coupled` (prepare/plan.cpp), not a table cell, and an +// undeclared one is resolved to `host-coupled` before this table ever runs +// (`dist::msvc_abi_default_contract`). +TEST(Distribution, NoMechanismCellDistinguishesClangFromClOnTheMsvcAbi) { dist::MechanismInput in; - in.format = dist::Format::Pe; - in.stdlibId = "msvc"; - in.msvcCrtModelEmitted = false; - - for (auto requested : {dist::Contract::SelfContained, - dist::Contract::HostCoupled, - dist::Contract::ToolchainCoupled}) { - in.requested = requested; - in.explicitRequest = false; - auto quiet = dist::resolve(in); - EXPECT_EQ(quiet.effective, dist::Contract::SelfContained); - EXPECT_FALSE(quiet.degraded); - EXPECT_FALSE(quiet.deployToolchainRuntime); - EXPECT_TRUE(quiet.unitFlags.empty()); - } + in.format = dist::Format::Pe; + in.stdlibId = "msvc"; + // /MD, requesting (and delivering) the toolchain's own copy. + in.msvcStaticCrt = false; in.explicitRequest = true; - in.requested = dist::Contract::HostCoupled; - auto undelivered = dist::resolve(in); - EXPECT_EQ(undelivered.effective, dist::Contract::SelfContained); - EXPECT_TRUE(undelivered.degraded); - EXPECT_NE(undelivered.diagnostic.find("not delivered"), std::string::npos) - << undelivered.diagnostic; - EXPECT_NE(undelivered.diagnostic.find("libcmt"), std::string::npos) - << undelivered.diagnostic; + in.requested = dist::Contract::ToolchainCoupled; + auto coupled = dist::resolve(in); + EXPECT_EQ(coupled.effective, dist::Contract::ToolchainCoupled); + EXPECT_TRUE(coupled.deployToolchainRuntime); + EXPECT_FALSE(coupled.degraded); + EXPECT_TRUE(coupled.diagnostic.empty()); - in.requested = dist::Contract::SelfContained; - EXPECT_FALSE(dist::resolve(in).degraded); + // /MT, requesting (and delivering) the static CRT. + in.msvcStaticCrt = true; + in.requested = dist::Contract::SelfContained; + auto selfContained = dist::resolve(in); + EXPECT_EQ(selfContained.effective, dist::Contract::SelfContained); + EXPECT_FALSE(selfContained.degraded); + EXPECT_TRUE(selfContained.diagnostic.empty()); + + // The undeclared default: quiet, whichever way `msvcStaticCrt` reads. + in.explicitRequest = false; + in.msvcStaticCrt = false; + in.requested = dist::Contract::HostCoupled; + auto quiet = dist::resolve(in); + EXPECT_EQ(quiet.effective, dist::Contract::HostCoupled); + EXPECT_FALSE(quiet.degraded); + EXPECT_TRUE(quiet.diagnostic.empty()); } diff --git a/tests/unit/test_index_contract.cpp b/tests/unit/test_index_contract.cpp index 3163a0137..09cbe77f9 100644 --- a/tests/unit/test_index_contract.cpp +++ b/tests/unit/test_index_contract.cpp @@ -84,3 +84,26 @@ TEST(IndexContract, ReadContractRoundTrip) { std::filesystem::remove_all(dir); EXPECT_FALSE(mcpp::pm::read_index_contract(dir).has_value()); } + +// The upgrade note appended to E0006 starts on its own line. It was appended +// directly after the "Details:" line and read as one line. +// The closing tip of a refresh names the version the index asks for, and +// falls back to the advice alone when the contract could not be read. +TEST(IndexContract, FloorUpgradeAdviceNamesTheVersionWhenKnown) { + auto known = mcpp::pm::index_floor_upgrade_advice("9999.9.9.9"); + EXPECT_NE(known.find("requires mcpp >= 9999.9.9.9"), std::string::npos) << known; + EXPECT_NE(known.find("Upgrade: "), std::string::npos) << known; + auto unknown = mcpp::pm::index_floor_upgrade_advice(""); + EXPECT_EQ(unknown.find(">="), std::string::npos) << unknown; + EXPECT_TRUE(unknown.starts_with("Upgrade: ")) << unknown; +} + +TEST(IndexContract, E0006UpgradeNoteStartsOnItsOwnLine) { + auto violation = mcpp::pm::floor_violation("2026.8.3.3", "2026.7.28.2"); + ASSERT_TRUE(violation.has_value()); + auto text = mcpp::pm::e0006_message(*violation, /*distroManaged=*/false); + auto at = text.find(" Upgrade: 'xlings update mcpp'"); + ASSERT_NE(at, std::string::npos) << text; + ASSERT_GT(at, 0u); + EXPECT_EQ(text[at - 1], '\n') << text; +} diff --git a/tests/unit/test_manifest.cpp b/tests/unit/test_manifest.cpp index c468418eb..a147f55cc 100644 --- a/tests/unit/test_manifest.cpp +++ b/tests/unit/test_manifest.cpp @@ -764,6 +764,136 @@ static_stdlib = true << "the message must be generated from the same list the check uses"; } +// #717: `dialect_cxxflags` under `[target..build]` used to be +// reported as "unsupported key 'dialect_cxxflags' (ignored)" and reached no +// command (measured on 2026.9.27.1). It is now accepted, without a warning, +// and parsed into `ConditionalConfig::dialectCxxflags` -- a member of its own, +// NOT of `BuildInputs` (`.inputs`), because the key is graph-wide rather than +// a per-package additive input (design 2026-09-27 §6.2). +TEST(Manifest, ConditionalDialectCxxflagsAcceptedAndKeptOffBuildInputs) { + constexpr auto src = R"( +[package] +name = "x" +version = "0.1.0" +[target.'cfg(linux)'.build] +cxxflags = ["-DORDINARY"] +dialect_cxxflags = ["-DDIALECT_A", "-DDIALECT_B"] +)"; + auto m = mcpp::manifest::parse_string(src); + ASSERT_TRUE(m.has_value()) << m.error().format(); + EXPECT_TRUE(m->schemaWarnings.empty()) + << (m->schemaWarnings.empty() ? "" : m->schemaWarnings[0]); + ASSERT_EQ(m->conditionalConfigs.size(), 1u); + auto const& cc = m->conditionalConfigs[0]; + ASSERT_EQ(cc.dialectCxxflags.size(), 2u); + EXPECT_EQ(cc.dialectCxxflags[0], "-DDIALECT_A"); + EXPECT_EQ(cc.dialectCxxflags[1], "-DDIALECT_B"); + // An ordinary per-package build input written in the SAME section still + // lands on BuildInputs, as always -- only `dialect_cxxflags` is diverted. + ASSERT_EQ(cc.inputs.cxxflags.size(), 1u); + EXPECT_EQ(cc.inputs.cxxflags[0], "-DORDINARY"); +} + +// The negative control for the positive test above: a section carrying ONLY +// `dialect_cxxflags` must still be recorded, the same emptiness-gate rule +// `ConditionalSectionWithOnlyDefinesIsRecorded` states for `defines`. +TEST(Manifest, ConditionalSectionWithOnlyDialectCxxflagsIsRecorded) { + constexpr auto src = R"( +[package] +name = "x" +version = "0.1.0" +[target.linux.build] +dialect_cxxflags = ["-DONLY_DIALECT"] +)"; + auto m = mcpp::manifest::parse_string(src); + ASSERT_TRUE(m.has_value()) << m.error().format(); + EXPECT_TRUE(m->schemaWarnings.empty()); + ASSERT_EQ(m->conditionalConfigs.size(), 1u); + ASSERT_EQ(m->conditionalConfigs[0].dialectCxxflags.size(), 1u); + EXPECT_EQ(m->conditionalConfigs[0].dialectCxxflags[0], "-DONLY_DIALECT"); +} + +// The resolution rules: only a matching row contributes, a non-matching one +// contributes nothing, and the order is the root's own `[build]` first, then +// each matching `[target..build]` in manifest order -- the same +// technique `tests/unit/test_abi.cpp`'s `RequiresAbiOnTargetAxis` tests use +// (`merge_conditional_config` against a resolved triple, no toolchain needed). +namespace manifest_dialect { + +mcpp::manifest::Manifest merged_for(std::string_view src, std::string_view triple) { + auto m = mcpp::manifest::parse_string(src); + EXPECT_TRUE(m.has_value()) << (m.has_value() ? "" : m.error().format()); + if (!m) return {}; + mcpp::build::merge_conditional_config(*m, mcpp::build::cfgpred::context_for(triple)); + return *m; +} + +} // namespace manifest_dialect + +TEST(ConditionalDialectCxxflags, RootBuildPrecedesTheMatchingRow) { + constexpr auto src = R"( +[package] +name = "x" +version = "0.1.0" +[build] +dialect_cxxflags = ["-DBASE"] +[target.linux.build] +dialect_cxxflags = ["-DLINUX_ONE"] +[target.windows.build] +dialect_cxxflags = ["-DWINDOWS_ONLY"] +)"; + auto lin = manifest_dialect::merged_for(src, "x86_64-unknown-linux-gnu"); + ASSERT_EQ(lin.buildConfig.dialectCxxflags.size(), 2u); + EXPECT_EQ(lin.buildConfig.dialectCxxflags[0], "-DBASE"); + EXPECT_EQ(lin.buildConfig.dialectCxxflags[1], "-DLINUX_ONE"); +} + +// Two DIFFERENT selectors that both match the same resolved triple both +// contribute -- this is not "last matching wins" (that rule belongs to a +// conditional REPLACING a scalar or a dependency identity, §3.1.1); every +// matching row is APPENDED, as `cxxflags` already is. The relative order +// between two conditional rows is not asserted here: `m.conditionalConfigs` +// is populated from the underlying `Table` (`std::map`, ordered by selector +// text), not by source position, which is a property of the parser this +// feature inherits rather than one it introduces or could change. +TEST(ConditionalDialectCxxflags, TwoMatchingSelectorsBothContribute) { + constexpr auto src = R"( +[package] +name = "x" +version = "0.1.0" +[build] +dialect_cxxflags = ["-DBASE"] +[target.linux.build] +dialect_cxxflags = ["-DLINUX_ONE"] +[target.'cfg(arch = "x86_64")'.build] +dialect_cxxflags = ["-DX86"] +)"; + auto lin = manifest_dialect::merged_for(src, "x86_64-unknown-linux-gnu"); + ASSERT_EQ(lin.buildConfig.dialectCxxflags.size(), 3u); + EXPECT_EQ(lin.buildConfig.dialectCxxflags[0], "-DBASE") + << "the root's own unconditional [build] value always leads: it is " + "set before the conditional-merge loop runs, regardless of which " + "matching row the loop visits first"; + auto const& flags = lin.buildConfig.dialectCxxflags; + EXPECT_NE(std::find(flags.begin(), flags.end(), "-DLINUX_ONE"), flags.end()); + EXPECT_NE(std::find(flags.begin(), flags.end(), "-DX86"), flags.end()); +} + +TEST(ConditionalDialectCxxflags, ASelectorThatDoesNotMatchContributesNothing) { + constexpr auto src = R"( +[package] +name = "x" +version = "0.1.0" +[build] +dialect_cxxflags = ["-DBASE"] +[target.windows.build] +dialect_cxxflags = ["-DWINDOWS_ONLY"] +)"; + auto lin = manifest_dialect::merged_for(src, "x86_64-unknown-linux-gnu"); + ASSERT_EQ(lin.buildConfig.dialectCxxflags.size(), 1u); + EXPECT_EQ(lin.buildConfig.dialectCxxflags[0], "-DBASE"); +} + // #540: `[features]` was the one structured section with no schema check at // all, so a misplaced `include_dirs` inside a feature built successfully with // zero diagnostics — while the identical misplacement in `[build]` or diff --git a/tests/unit/test_ninja_backend.cpp b/tests/unit/test_ninja_backend.cpp index 8ca03473e..7508a3c32 100644 --- a/tests/unit/test_ninja_backend.cpp +++ b/tests/unit/test_ninja_backend.cpp @@ -1082,6 +1082,17 @@ TEST(NinjaBackend, McppBinaryIsBoundEvenWithoutDyndep) { EXPECT_NE(ninja.find("\nmcpp = "), std::string::npos) << ninja; } +// SPEC-007 R4.2: one destination, one writer. On a PE target two spellings +// that differ only in case are one file where the program runs, so they are +// one destination; elsewhere they are two. +TEST(DeployDestination, FoldsCaseOnlyOnAPeTarget) { + const mcpp::build::BuildPlan::DeployFile d{{"/a/Foo.DLL"}, "bin/Foo.DLL"}; + EXPECT_TRUE(d.is_destination("bin/Foo.DLL", /*peTarget=*/false)); + EXPECT_FALSE(d.is_destination("bin/foo.dll", /*peTarget=*/false)); + EXPECT_TRUE(d.is_destination("bin/foo.dll", /*peTarget=*/true)); + EXPECT_FALSE(d.is_destination("bin/sub/foo.dll", /*peTarget=*/true)); +} + TEST(NinjaBackend, StdArtifactsAndRuntimeDllsUseTheStageRule) { auto plan = minimal_plan(); plan.toolchain.compiler = mcpp::toolchain::CompilerId::Clang; @@ -1090,7 +1101,7 @@ TEST(NinjaBackend, StdArtifactsAndRuntimeDllsUseTheStageRule) { plan.stdObjectPath = "/cache/bmi/fp/std.o"; plan.stdCompatBmiPath = "/cache/bmi/fp/pcm.cache/std.compat.pcm"; plan.stdCompatObjectPath = "/cache/bmi/fp/std.compat.o"; - plan.runtimeDeployFiles.push_back({"/pkg/lib/libfoo.dll", "bin/libfoo.dll"}); + plan.runtimeDeployFiles.push_back({{"/pkg/lib/libfoo.dll"}, "bin/libfoo.dll"}); auto ninja = emit_ninja_string(plan); @@ -1166,7 +1177,7 @@ TEST(NinjaBackend, DeployedDllsAreOrderOnlyInputsOfTheLink) { // The linker reads the import library, never the deployed DLL, so a DLL // that changes (or a deploy entry a later plan adds) must not relink. auto plan = program_plan("x86_64-w64-windows-gnu", false); - plan.runtimeDeployFiles.push_back({"/pkg/lib/libfoo.dll", "bin/libfoo.dll"}); + plan.runtimeDeployFiles.push_back({{"/pkg/lib/libfoo.dll"}, "bin/libfoo.dll"}); auto ninja = emit_ninja_string(plan); auto link = ninja.find("build bin/app.exe"); ASSERT_NE(link, std::string::npos) << ninja; @@ -1517,6 +1528,7 @@ BuildPlan msvc_plan_with_redist(const FakeRedistDir& redist, plan.toolchain.binaryPath = "cl.exe"; plan.toolchain.targetTriple = "x86_64-pc-windows-msvc"; plan.toolchain.linkRuntimeDirs = {redist.path}; + plan.toolchain.msvcRedistDir = redist.path; plan.manifest.buildConfig.cxxRuntime = std::string(cxxRuntime); plan.linkUnits.push_back({ .targetName = "app", @@ -1541,7 +1553,8 @@ TEST(NinjaBackendPeRuntime, ToolchainCoupledStagesTheToolsetCrtBesideTheExe) { EXPECT_EQ(d.dest.parent_path(), std::filesystem::path("bin")) << "a DLL must land in the same directory as the .exe: " << d.dest.string(); - EXPECT_EQ(d.source.extension(), ".dll") << d.source.string(); + ASSERT_EQ(d.sources.size(), 1u); + EXPECT_EQ(d.sources.front().extension(), ".dll") << d.sources.front().string(); } auto ninja = emit_ninja_string(plan); @@ -1569,16 +1582,20 @@ TEST(NinjaBackendPeRuntime, ToolchainCoupledStagesTheToolsetCrtBesideTheExe) { } TEST(NinjaBackendPeRuntime, HostCoupledStagesNothing) { - // The DEFAULT Windows build. Staging DLLs unasked would change what every - // existing project ships, and `host-coupled` is a promise that the machine - // provides them — keeping a copy beside the artifact contradicts it. + // An EXPLICIT `host-coupled`: the machine is a promise that it provides + // the DLLs, and keeping a copy beside the artifact would contradict it. FakeRedistDir redist; auto plan = msvc_plan_with_redist(redist, "host-coupled"); EXPECT_TRUE(compute_flags(plan).toolchainRuntimeDeploy.empty()); + // The BARE default (#718): `toolchain-coupled` is now the MSVC-ABI + // default for every role, so a project that never mentioned + // `cxx_runtime` stages the toolset's redistributable exactly as an + // explicit `toolchain-coupled` would — see + // `ToolchainCoupledStagesTheToolsetCrtBesideTheExe`. auto bare = msvc_plan_with_redist(redist, ""); - EXPECT_TRUE(compute_flags(bare).toolchainRuntimeDeploy.empty()) - << "a project that never mentioned cxx_runtime gained staged DLLs"; + EXPECT_EQ(compute_flags(bare).toolchainRuntimeDeploy.size(), 3u) + << "the undeclared MSVC-ABI default no longer stages the redistributable"; } TEST(NinjaBackendPeRuntime, AProjectsOwnDeployFileOutranksTheToolsets) { @@ -1589,7 +1606,7 @@ TEST(NinjaBackendPeRuntime, AProjectsOwnDeployFileOutranksTheToolsets) { FakeRedistDir redist; auto plan = msvc_plan_with_redist(redist, "toolchain-coupled"); plan.runtimeDeployFiles.push_back( - {"/vendor/msvcp140.dll", std::filesystem::path("bin") / "msvcp140.dll"}); + {{"/vendor/msvcp140.dll"}, std::filesystem::path("bin") / "msvcp140.dll"}); auto flags = compute_flags(plan); for (auto const& d : flags.toolchainRuntimeDeploy) @@ -1614,6 +1631,70 @@ TEST(NinjaBackendPeRuntime, AnElfToolchainNeverStagesItsRuntimeDirs) { EXPECT_TRUE(compute_flags(plan).toolchainRuntimeDeploy.empty()); } +// #718: every MSVC-ABI row x {undeclared, self-contained, toolchain-coupled, +// host-coupled, linkage=static} yields exactly one CRT word, spelled for its +// own driver, and that word is the SAME on the compile line (`f.cxx`, +// `f.cc`) and — for the LLVM row — the link line (`f.ld`, `f.ldC`). This is +// the exact property #649 E10 broke: a compile-only flag the clang driver's +// link step never saw, so it chose `-defaultlib:libcmt` on its own. cl's +// link line is `link.exe` directly (`LinkShape::MsvcLinkExe`) and carries no +// compile flag at all — the CRT is baked into the objects it is given — so +// only the LLVM row's link line is asked to repeat the word. +TEST(NinjaBackendPeRuntime, CrtWordIsOneWordEqualOnCompileAndLink) { + FakeRedistDir redist; + struct Row { + std::string_view label; + mcpp::toolchain::CompilerId compiler; + std::string_view staticWord, dynamicWord; + }; + Row rows[] = { + {"cl", mcpp::toolchain::CompilerId::MSVC, "/MT", "/MD"}, + {"llvm", mcpp::toolchain::CompilerId::Clang, "-fms-runtime-lib=static", + "-fms-runtime-lib=dll"}, + }; + struct Case { std::string_view linkage, cxxRuntime; bool wantsStatic; }; + Case cases[] = { + {"", "", false}, // undeclared -> toolchain-coupled + {"", "self-contained", true}, + {"", "toolchain-coupled", false}, + {"", "host-coupled", false}, + {"static", "", true}, + }; + for (auto& row : rows) { + for (auto& c : cases) { + auto plan = msvc_plan_with_redist(redist, c.cxxRuntime); + plan.toolchain.compiler = row.compiler; + plan.manifest.buildConfig.linkage = std::string(c.linkage); + auto flags = compute_flags(plan); + auto want = c.wantsStatic ? row.staticWord : row.dynamicWord; + auto other = c.wantsStatic ? row.dynamicWord : row.staticWord; + SCOPED_TRACE(std::format("{} linkage='{}' cxx_runtime='{}'", + row.label, c.linkage, c.cxxRuntime)); + + EXPECT_EQ(count_occurrences(flags.cxx, want), 1u) << flags.cxx; + EXPECT_EQ(count_occurrences(flags.cc, want), 1u) << flags.cc; + EXPECT_EQ(flags.cxx.find(other), std::string::npos); + EXPECT_EQ(flags.cc.find(other), std::string::npos); + + // `link_shape` (flags.cppm) picks `LinkShape::PeLld` — the + // branch that carries this word on the link line — only when + // `current_link_host()` is Windows, a HOST fact fixed at mcpp's + // own compile time. A Linux-built test binary always resolves + // this plan's link line through the ELF/Generic branch instead, + // whatever the plan's TARGET triple says, so the link-line half + // of this property is checkable only on a Windows-built mcpp — + // see the Windows e2e leg (814) for that half. + if constexpr (mcpp::platform::is_windows) { + if (row.compiler == mcpp::toolchain::CompilerId::Clang) { + EXPECT_EQ(count_occurrences(flags.ld, want), 1u) << flags.ld; + EXPECT_EQ(flags.ld.find(other), std::string::npos); + EXPECT_EQ(flags.ld, flags.ldC); + } + } + } + } +} + // ── link_failure_advice ────────────────────────────────────────────────────── // // A freestanding link that reaches for `operator new` fails naming a mangled diff --git a/tests/unit/test_pack_closure.cpp b/tests/unit/test_pack_closure.cpp index f00ab170e..aafd6cb67 100644 --- a/tests/unit/test_pack_closure.cpp +++ b/tests/unit/test_pack_closure.cpp @@ -285,6 +285,86 @@ TEST(PackClosurePe, ForceBundleReachesTheSystemList) { EXPECT_TRUE(with.platform.empty()); } +// ── SPEC-007 R4.2/R4.3: one destination, one writer (#723 self-review) ── +// +// `place_runtime_dlls` is the function behind the `place-dlls` ninja edge. +// These fixtures give it a real PE program and a real search directory (the +// PE reader is not a fake), and a `placedByDeploy` list, which is the only +// part `ninja_backend.cppm` computes from the plan — everything else here is +// this function's own decision. + +namespace { + +std::string read_bytes(const std::filesystem::path& p) { + std::ifstream is(p, std::ios::binary); + return std::string{std::istreambuf_iterator(is), {}}; +} + +} // namespace + +TEST(PlaceRuntimeDlls, ANameTheDeployListPlacesIsNeitherOverwrittenNorPlaced) { + Tree t; + auto exe = t.write("bin/app.exe", pe_importing({"foo.dll"})); + // The deploy list's own file, already beside the program (as it would be + // by the time this edge runs — an order-only dependency of the link). + t.write("bin/foo.dll", pe_importing({}) + "DEPLOYED"); + // A different "foo.dll" the runtime search directory also offers. + t.write("deps/foo.dll", pe_importing({}) + "SEARCHDIR"); + + auto r = mcpp::pack::place_runtime_dlls( + exe, {t.root / "deps"}, /*placedBefore=*/{}, /*placedByDeploy=*/{"foo.dll"}); + ASSERT_TRUE(r.has_value()) << r.error().message; + EXPECT_TRUE(r->names.empty()) << "the deploy list's name was placed a second time"; + EXPECT_TRUE(r->sources.empty()); + ASSERT_EQ(r->warnings.size(), 1u); + EXPECT_NE(r->warnings[0].find("foo.dll"), std::string::npos) << r->warnings[0]; + // Not overwritten: the file beside the program is still the deploy list's. + EXPECT_EQ(read_bytes(t.root / "bin" / "foo.dll"), pe_importing({}) + "DEPLOYED"); +} + +TEST(PlaceRuntimeDlls, ANameTheDeployListPlacesWithMatchingBytesIsSilent) { + Tree t; + auto exe = t.write("bin/app.exe", pe_importing({"foo.dll"})); + const auto bytes = pe_importing({}) + "SAME"; + t.write("bin/foo.dll", bytes); + t.write("deps/foo.dll", bytes); + + auto r = mcpp::pack::place_runtime_dlls( + exe, {t.root / "deps"}, {}, {"foo.dll"}); + ASSERT_TRUE(r.has_value()) << r.error().message; + EXPECT_TRUE(r->names.empty()); + EXPECT_TRUE(r->warnings.empty()); +} + +TEST(PlaceRuntimeDlls, TheDeployListNameIsMatchedCaseInsensitively) { + Tree t; + auto exe = t.write("bin/app.exe", pe_importing({"Foo.DLL"})); + // The deploy list itself may spell the name differently (it named a + // destination file, not this program's import table entry); the match + // against `placedByDeploy` must not depend on which case either side used. + t.write("bin/Foo.DLL", pe_importing({}) + "DEPLOYED"); + t.write("deps/Foo.DLL", pe_importing({}) + "SEARCHDIR"); + + auto r = mcpp::pack::place_runtime_dlls( + exe, {t.root / "deps"}, {}, {"foo.dll"}); + ASSERT_TRUE(r.has_value()) << r.error().message; + EXPECT_TRUE(r->names.empty()) + << "PE names are case-insensitive; the deploy list's entry should still match"; +} + +TEST(PlaceRuntimeDlls, ANameNotInTheDeployListIsPlacedAsBefore) { + Tree t; + auto exe = t.write("bin/app.exe", pe_importing({"foo.dll"})); + t.write("deps/foo.dll", pe_importing({}) + "PAYLOAD"); + + auto r = mcpp::pack::place_runtime_dlls(exe, {t.root / "deps"}); + ASSERT_TRUE(r.has_value()) << r.error().message; + EXPECT_EQ(r->names, (Names{"foo.dll"})); + EXPECT_EQ(r->sources, (std::vector{t.root / "deps" / "foo.dll"})); + EXPECT_TRUE(r->warnings.empty()); + EXPECT_EQ(read_bytes(t.root / "bin" / "foo.dll"), pe_importing({}) + "PAYLOAD"); +} + // ── Mach-O ─────────────────────────────────────────────────────────────── TEST(PackClosureMachO, AnRpathDylibBesideTheProgramIsAMemberTransitively) { diff --git a/tests/unit/test_pm_package_fetcher.cpp b/tests/unit/test_pm_package_fetcher.cpp index 38f7d1c0f..46694e3d7 100644 --- a/tests/unit/test_pm_package_fetcher.cpp +++ b/tests/unit/test_pm_package_fetcher.cpp @@ -2,6 +2,7 @@ import std; import mcpp.pm.package_fetcher; +import mcpp.pm.index_contract; // Regression for the compat.zlib vs upstream bare zlib.lua collision. // @@ -276,3 +277,33 @@ TEST(PmPackageFetcher, ReadSeededIndexReposToleratesArtifactFields) { std::filesystem::remove_all(project); } + +// A tree whose index.toml requires a newer mcpp answers no lookup, and the +// read prints nothing. The fact is recorded for the message of a run that then +// fails (unusable_index_hint), which carries the E0006 text itself; a run that +// resolves every package elsewhere has no error to report. Before this, the +// read site printed the E0006 text as `error:` at the start of such runs. +TEST(PackageFetcher, AnIndexThatRequiresANewerMcppIsRecordedNotPrinted) { + auto root = make_tempdir("mcpp-floor-read"); + write_file(root / "index.toml", + "[index]\nspec = \"1\"\nmin_mcpp = \"9999.9.9.9\"\n"); + write_file(root / "pkgs" / "f" / "floorpkg.lua", + "package = { spec = \"1\", name = \"floorpkg\", type = \"package\" }\n"); + + testing::internal::CaptureStderr(); + auto first = mcpp::pm::Fetcher::read_xpkg_lua_from_path(root, "floorns", "floorpkg"); + auto second = mcpp::pm::Fetcher::read_xpkg_lua_from_path(root, "floorns", "floorpkg"); + auto err = testing::internal::GetCapturedStderr(); + + EXPECT_FALSE(first.has_value()); + EXPECT_FALSE(second.has_value()) << "a tree recorded as unusable answers no later lookup"; + EXPECT_TRUE(err.empty()) << err; + EXPECT_TRUE(mcpp::pm::index_marked_unusable(root)); + + auto hint = mcpp::pm::unusable_index_hint(); + EXPECT_NE(hint.find("E0006"), std::string::npos) << hint; + EXPECT_NE(hint.find("9999.9.9.9"), std::string::npos) << hint; + EXPECT_EQ(hint.find("error above"), std::string::npos) << hint; + mcpp::pm::reset_unusable_indexes_for_test(); + std::filesystem::remove_all(root); +} diff --git a/tests/unit/test_progress_render.cpp b/tests/unit/test_progress_render.cpp new file mode 100644 index 000000000..45a8e73e2 --- /dev/null +++ b/tests/unit/test_progress_render.cpp @@ -0,0 +1,105 @@ +#include + +import std; +import mcpp.ui; +import mcpp.fetcher.progress; +import mcpp.platform.process; + +// W11: one renderer for every acquisition, with a terminal mode (redrawn in +// place) and a plain mode (one start line and one finish line per item). + +TEST(GitProgress, ParsesThePhaseAndThePercentage) { + auto r = mcpp::fetcher::parse_git_progress( + "Receiving objects: 45% (450/1000), 1.20 MiB | 800.00 KiB/s"); + ASSERT_TRUE(r.has_value()); + EXPECT_EQ(r->phase, "Receiving objects"); + EXPECT_EQ(r->percent, 45u); + + auto remote = mcpp::fetcher::parse_git_progress("remote: Counting objects: 100% (5/5), done."); + ASSERT_TRUE(remote.has_value()); + EXPECT_EQ(remote->phase, "Counting objects"); + EXPECT_EQ(remote->percent, 100u); + + EXPECT_FALSE(mcpp::fetcher::parse_git_progress("Cloning into 'x'...").has_value()); + EXPECT_FALSE(mcpp::fetcher::parse_git_progress("fatal: repository not found").has_value()); + EXPECT_FALSE(mcpp::fetcher::parse_git_progress("").has_value()); +} + +TEST(ProgressBarPlain, OneStartLineAndOneFinishLineWithoutRepaints) { + mcpp::ui::disable_color(); + mcpp::ui::set_live_progress(false); + testing::internal::CaptureStdout(); + { + mcpp::ui::ProgressBar bar("Fetching", "example"); + bar.update_bytes(10, 100, 0.1); + bar.update_bytes(50, 100, 0.5); + bar.update_bytes(100, 100, 1.0); + bar.finish(); + } + auto out = testing::internal::GetCapturedStdout(); + EXPECT_EQ(out.find('\r'), std::string::npos) << out; + EXPECT_EQ(out.find('\x1b'), std::string::npos) << out; + EXPECT_EQ(std::ranges::count(out, '\n'), 2) << out; + EXPECT_NE(out.find("Fetching example ("), std::string::npos) << out; + EXPECT_NE(out.find("done"), std::string::npos) << out; +} + +TEST(ProgressBarPlain, AFailedItemIsNotReportedDone) { + mcpp::ui::disable_color(); + mcpp::ui::set_live_progress(false); + testing::internal::CaptureStdout(); + { + mcpp::ui::ProgressBar bar("Installing", "xim:example"); + bar.update_indeterminate(0, 0.2); + bar.finish_failed("xim:example"); + } + auto out = testing::internal::GetCapturedStdout(); + EXPECT_NE(out.find("did not complete"), std::string::npos) << out; + EXPECT_EQ(out.find(" done"), std::string::npos) << out; +} + +TEST(DownloadProgressPlain, AnInterruptedDownloadIsNotReportedDone) { + mcpp::ui::disable_color(); + mcpp::ui::set_live_progress(false); + testing::internal::CaptureStdout(); + { + mcpp::ui::DownloadProgress dl; + const mcpp::ui::DownloadFile f{"xim-index.tar.gz", 10, 100, true, false}; + dl.update(std::span{&f, 1}, 0.2); + dl.finish_failed(); + } + auto out = testing::internal::GetCapturedStdout(); + EXPECT_NE(out.find("xim-index.tar.gz did not complete"), std::string::npos) << out; + EXPECT_EQ(out.find(" done"), std::string::npos) << out; +} + +TEST(ProgressBarLive, RedrawsInPlace) { + mcpp::ui::disable_color(); + mcpp::ui::set_live_progress(true); + testing::internal::CaptureStdout(); + { + mcpp::ui::ProgressBar bar("Fetching", "example"); + bar.update(100); + bar.finish(); + } + auto out = testing::internal::GetCapturedStdout(); + mcpp::ui::set_live_progress(false); + EXPECT_NE(out.find('\r'), std::string::npos) << out; +} + +#if !defined(_WIN32) +TEST(StreamingBounded, ACarriageReturnEndsALineOnlyWhenAsked) { + const auto cmd = std::string("printf 'a\\rb\\r\\nc\\n'"); + std::vector plain, split; + bool timedOut = false; + mcpp::platform::process::run_streaming_bounded(cmd, + [&](std::string_view l) { plain.emplace_back(l); }, + std::chrono::milliseconds{0}, std::chrono::seconds{30}, &timedOut); + mcpp::platform::process::run_streaming_bounded(cmd, + [&](std::string_view l) { split.emplace_back(l); }, + std::chrono::milliseconds{0}, std::chrono::seconds{30}, &timedOut, + /*split_on_cr=*/true); + EXPECT_EQ(plain, (std::vector{"a\rb", "c"})); + EXPECT_EQ(split, (std::vector{"a", "b", "c"})); +} +#endif diff --git a/tests/unit/test_toolchain_dialect.cpp b/tests/unit/test_toolchain_dialect.cpp index 0beecb8e8..209a29582 100644 --- a/tests/unit/test_toolchain_dialect.cpp +++ b/tests/unit/test_toolchain_dialect.cpp @@ -152,3 +152,120 @@ TEST(MingwModel, TargetPredicate) { EXPECT_FALSE(is_mingw_target(make_tc(CompilerId::GCC, "x86_64-linux-gnu"))); EXPECT_FALSE(is_mingw_target(make_tc(CompilerId::Clang, "x86_64-pc-windows-msvc"))); } + +// ─── #718: the CRT model reaches every MSVC-ABI row, cl and clang++ alike ── + +// The property the whole design rests on: one helper, `msvc_abi_crt_word`, +// spells the SAME two-state switch (static/dynamic CRT) for whichever driver +// `tc` is, and says nothing for a row that is not the MSVC ABI at all. +TEST(MsvcAbiCrtWord, SpelledForEachDriverAndEmptyOffTheAbi) { + auto cl = make_tc(CompilerId::MSVC, "x86_64-pc-windows-msvc"); + EXPECT_EQ(msvc_abi_crt_word(cl, /*staticCrt=*/true), "/MT"); + EXPECT_EQ(msvc_abi_crt_word(cl, /*staticCrt=*/false), "/MD"); + + auto llvmRow = make_tc(CompilerId::Clang, "x86_64-pc-windows-msvc"); + EXPECT_EQ(msvc_abi_crt_word(llvmRow, /*staticCrt=*/true), + "-fms-runtime-lib=static"); + EXPECT_EQ(msvc_abi_crt_word(llvmRow, /*staticCrt=*/false), + "-fms-runtime-lib=dll"); + + // Neither the MSVC ABI: MinGW links the MSVC CRT to no row at all, and a + // plain Linux row has no CRT axis of this kind either. + EXPECT_TRUE(msvc_abi_crt_word( + make_tc(CompilerId::GCC, "x86_64-w64-mingw32"), true).empty()); + EXPECT_TRUE(msvc_abi_crt_word( + make_tc(CompilerId::Clang, "x86_64-linux-gnu"), false).empty()); + EXPECT_TRUE(msvc_abi_crt_word( + make_tc(CompilerId::GCC, "x86_64-linux-gnu"), true).empty()); +} + +// Every MSVC-ABI row x {undeclared, self-contained, toolchain-coupled, +// host-coupled, linkage=static} yields exactly one CRT word, spelled for its +// own driver — the property property test the design asks for at the level +// this pure helper can state it at (§7.3's criterion). "undeclared" and +// "toolchain-coupled" share a word (`/MD`/`-fms-runtime-lib=dll`) because the +// undeclared default and an explicit `toolchain-coupled` both compile +// against the dynamic CRT; they differ only in whether the redistributable +// is staged, which is a distribution.cppm/flags.cppm concern this helper does +// not carry. +TEST(MsvcAbiCrtWord, EveryContractResolvesToExactlyOneWordPerRow) { + struct Row { Toolchain tc; std::string_view label; }; + Row rows[] = { + {make_tc(CompilerId::MSVC, "x86_64-pc-windows-msvc"), "cl"}, + {make_tc(CompilerId::Clang, "x86_64-pc-windows-msvc"), "llvm"}, + }; + struct Case { + std::string_view linkage, cxxRuntime; + bool wantsStatic; + }; + Case cases[] = { + {"", "", false}, // undeclared + {"", "self-contained", true}, + {"", "toolchain-coupled", false}, + {"", "host-coupled", false}, + {"static", "", true}, + }; + for (auto& row : rows) { + for (auto& c : cases) { + const bool staticCrt = msvc_wants_static_crt(c.linkage, c.cxxRuntime); + EXPECT_EQ(staticCrt, c.wantsStatic) + << row.label << " linkage='" << c.linkage + << "' cxx_runtime='" << c.cxxRuntime << "'"; + auto word = msvc_abi_crt_word(row.tc, staticCrt); + EXPECT_FALSE(word.empty()) << row.label; + // Exactly one of the two spellings for this driver, and it is + // the SAME word `flags.cppm`'s compile line and `scan.cpp`'s std + // BMI mirror both call this helper for — one derivation, so a TU + // and its std BMI cannot disagree by construction. + if (row.tc.compiler == CompilerId::MSVC) { + EXPECT_EQ(word, staticCrt ? "/MT" : "/MD") << row.label; + } else { + EXPECT_EQ(word, staticCrt ? "-fms-runtime-lib=static" + : "-fms-runtime-lib=dll") << row.label; + } + } + } +} + +// D3: a free-form CRT word is always a second statement once the model is +// resolved for every MSVC-ABI row. +TEST(CheckCrtWord, AgreeingWordIsRedundantContradictingWordIsRefused) { + // Recognised spellings, both dash conventions and both drivers. + for (auto* word : {"/MT", "-MT", "-fms-runtime-lib=static"}) { + auto agree = check_crt_word(word, /*staticCrt=*/true, "[build] cxxflags"); + ASSERT_TRUE(agree.has_value()) << word; + EXPECT_FALSE(agree->contradicts) << word; + // The key it names states the same model. + EXPECT_NE(agree->message.find("self-contained"), std::string::npos) << agree->message; + auto disagree = check_crt_word(word, /*staticCrt=*/false, "[build] cxxflags"); + ASSERT_TRUE(disagree.has_value()) << word; + EXPECT_TRUE(disagree->contradicts) << word; + EXPECT_NE(disagree->message.find(word), std::string::npos) << disagree->message; + EXPECT_NE(disagree->message.find("[build] cxxflags"), std::string::npos) + << disagree->message; + } + for (auto* word : {"/MD", "-MD", "-fms-runtime-lib=dll"}) { + auto agree = check_crt_word(word, /*staticCrt=*/false, "dialect_cxxflags"); + ASSERT_TRUE(agree.has_value()) << word; + EXPECT_FALSE(agree->contradicts) << word; + EXPECT_NE(agree->message.find("toolchain-coupled"), std::string::npos) << agree->message; + EXPECT_EQ(agree->message.find("self-contained"), std::string::npos) << agree->message; + auto disagree = check_crt_word(word, /*staticCrt=*/true, "dialect_cxxflags"); + ASSERT_TRUE(disagree.has_value()) << word; + EXPECT_TRUE(disagree->contradicts) << word; + } + // A debug CRT word is refused under either model: the model has no debug + // axis, and the std module and the link use the release CRT. + for (auto* word : {"/MTd", "-MTd", "/MDd", "-MDd", + "-fms-runtime-lib=static_dbg", "-fms-runtime-lib=dll_dbg"}) { + for (bool staticCrt : {true, false}) { + auto v = check_crt_word(word, staticCrt, "[build] cxxflags"); + ASSERT_TRUE(v.has_value()) << word; + EXPECT_TRUE(v->contradicts) << word; + EXPECT_NE(v->message.find("debug CRT"), std::string::npos) << v->message; + } + } + // A word this axis does not recognise says nothing about it. + EXPECT_FALSE(check_crt_word("-O2", true, "[build] cxxflags").has_value()); + EXPECT_FALSE(check_crt_word("/EHsc", false, "[build] cxxflags").has_value()); +} diff --git a/tests/unit/test_toolchain_msvc.cpp b/tests/unit/test_toolchain_msvc.cpp index bfbac1376..b454428de 100644 --- a/tests/unit/test_toolchain_msvc.cpp +++ b/tests/unit/test_toolchain_msvc.cpp @@ -479,6 +479,37 @@ TEST(MsvcRedist, AToolsetWithoutARedistIsNotAnError) { EXPECT_TRUE(msvc::vc_redist_dir(t.clPath, "x64").empty()); } +// #718: the LLVM row reaches the SAME redistributable from its sysroot's +// tools directory, since it runs no cl.exe of its own to derive one from. +TEST(MsvcRedist, TheSameDirectoryIsReachableFromTheToolsDirAlone) { + FakeRedist t{"14.44.35207", "14.44.35112"}; + // /Tools/MSVC//bin/Hostx64/x64/cl.exe -> up 4 reaches the tools + // dir `Toolchain::msvcToolsDir` carries (/Tools/MSVC/). + auto toolsDir = t.clPath.parent_path().parent_path().parent_path().parent_path(); + auto fromCl = msvc::vc_redist_dir(t.clPath, "x64"); + auto fromTools = msvc::vc_redist_dir_for_tools_dir(toolsDir, "x86_64"); + ASSERT_FALSE(fromCl.empty()); + EXPECT_EQ(fromCl, fromTools); +} + +TEST(MsvcRedist, ToolsDirArchMapping) { + // The GNU-spelled triple architecture (what a Toolchain carries) maps to + // the msvc spelling `vc_redist_dir` takes. + FakeRedist t{"14.44.35207", "14.44.35112"}; + t.add("14.44.35112", "arm64", "Microsoft.VC143.CRT", "vcruntime140.dll"); + t.add("14.44.35112", "x86", "Microsoft.VC143.CRT", "vcruntime140.dll"); + auto toolsDir = t.clPath.parent_path().parent_path().parent_path().parent_path(); + EXPECT_FALSE(msvc::vc_redist_dir_for_tools_dir(toolsDir, "aarch64").empty()); + EXPECT_FALSE(msvc::vc_redist_dir_for_tools_dir(toolsDir, "i686").empty()); + EXPECT_FALSE(msvc::vc_redist_dir_for_tools_dir(toolsDir, "x86_64").empty()); + // An architecture this toolset was never given a redist for finds none — + // it does not fall back to a different one. + std::error_code ec; + auto vc = toolsDir.parent_path().parent_path().parent_path(); // + std::filesystem::remove_all(vc / "Redist" / "MSVC" / "14.44.35112" / "arm64", ec); + EXPECT_TRUE(msvc::vc_redist_dir_for_tools_dir(toolsDir, "aarch64").empty()); +} + TEST(MsvcSdk, HeadersWithoutImportLibsIsNotAnAnswer) { // The half that used to pass. `Include//ucrt/corecrt.h` is there and // `Lib/` is not, which is exactly what a managed windows-sdk payload diff --git a/tests/unit/test_ui_closing_notices.cpp b/tests/unit/test_ui_closing_notices.cpp new file mode 100644 index 000000000..2783d2e5d --- /dev/null +++ b/tests/unit/test_ui_closing_notices.cpp @@ -0,0 +1,43 @@ +#include + +import std; +import mcpp.ui; + +// Closing notices are advisories about the run as a whole. They are printed +// once, after the command's own output, as `tip:` lines on stderr; a command +// that writes an envelope takes them first and reports them as notes. + +TEST(ClosingNotices, AreDeduplicatedAndTakenOnce) { + (void)mcpp::ui::take_closing_notices(); + mcpp::ui::add_closing_notice("CODE_A", "the index needs a newer mcpp"); + mcpp::ui::add_closing_notice("CODE_A", "the index needs a newer mcpp"); + mcpp::ui::add_closing_notice("CODE_B", "a second notice"); + auto taken = mcpp::ui::take_closing_notices(); + ASSERT_EQ(taken.size(), 2u); + EXPECT_EQ(taken[0].code, "CODE_A"); + EXPECT_EQ(taken[1].message, "a second notice"); + EXPECT_TRUE(mcpp::ui::take_closing_notices().empty()); +} + +TEST(ClosingNotices, PrintAsTipLinesOnStderrAndThenAreGone) { + (void)mcpp::ui::take_closing_notices(); + mcpp::ui::disable_color(); + mcpp::ui::add_closing_notice("CODE", "upgrade to see newer packages"); + testing::internal::CaptureStderr(); + mcpp::ui::print_closing_notices(); + mcpp::ui::print_closing_notices(); + auto err = testing::internal::GetCapturedStderr(); + EXPECT_EQ(err, "tip: upgrade to see newer packages\n"); +} + +TEST(ClosingNotices, QuietPrintsNothingButStillConsumes) { + (void)mcpp::ui::take_closing_notices(); + mcpp::ui::add_closing_notice("CODE", "not shown under --quiet"); + mcpp::ui::set_quiet(true); + testing::internal::CaptureStderr(); + mcpp::ui::print_closing_notices(); + auto err = testing::internal::GetCapturedStderr(); + mcpp::ui::set_quiet(false); + EXPECT_TRUE(err.empty()) << err; + EXPECT_TRUE(mcpp::ui::take_closing_notices().empty()); +} diff --git a/tests/unit/test_workspace_inheritance.cpp b/tests/unit/test_workspace_inheritance.cpp index 62b7ec9cf..74f4d5a52 100644 --- a/tests/unit/test_workspace_inheritance.cpp +++ b/tests/unit/test_workspace_inheritance.cpp @@ -256,3 +256,167 @@ TEST(HostToolToolchain, AMemberToolReadsItsWorkspaceToolchain) { std::error_code ec; fs::remove_all(root, ec); } + +// #725. `-p, --package ` resolves the package identity first, and the +// directory spellings docs/07 §5.3 has always documented second +// (mcpp::project::resolve_member_dir). One resolver serves every `-p` +// reading command; the e2e halves (the workspace-context repair itself, and +// `-p` on a real build) are tests/e2e/805_… and tests/e2e/806_…. +namespace resolve_member { + +namespace fs = std::filesystem; + +struct Fixture { + fs::path root; + + explicit Fixture(std::string_view tag) { + root = fs::temp_directory_path() + / std::format("mcpp-725-{}-{:x}", tag, std::random_device{}()); + } + ~Fixture() { + std::error_code ec; + fs::remove_all(root, ec); + } + Fixture(const Fixture&) = delete; + + void write(const fs::path& rel, std::string_view text) { + auto p = root / rel; + fs::create_directories(p.parent_path()); + std::ofstream(p) << text; + } + // `path` is written into `mcpp.toml` as the manifest's own [package] + // fields; `dir` is the directory it lives in, relative to `root`. + void member(std::string_view dir, std::string_view name, + std::string_view ns = "") { + std::string toml = "[package]\n"; + if (!ns.empty()) toml += std::format("namespace = \"{}\"\n", ns); + toml += std::format("name = \"{}\"\n", name); + write(fs::path(dir) / "mcpp.toml", toml); + } + mcpp::manifest::Manifest root_manifest(std::string_view membersToml) { + write("mcpp.toml", std::format( + "[package]\nname = \"root\"\nversion = \"0.1.0\"\n\n{}", membersToml)); + auto m = mcpp::manifest::load(root / "mcpp.toml"); + EXPECT_TRUE(m.has_value()) << (m ? "" : m.error().format()); + return m ? std::move(*m) : mcpp::manifest::Manifest{}; + } +}; + +// A member's own package name selects it, its full path does, and so does +// its directory's last segment (docs/07 §5.3's historical spellings, kept as +// a fallback). +TEST(ResolveMember, PackageNamePathAndBasenameAllSelectTheOneMember) { + Fixture f("single"); + f.member("modules/base", "ws-base"); + auto ws = f.root_manifest("[workspace]\nmembers = [\"modules/base\"]\n"); + + std::vector filters{"ws-base", "base", "modules/base", + "./modules/base", "modules/base/"}; +#if defined(_WIN32) + filters.push_back("modules\\base"); +#endif + for (std::string_view filter : filters) { + SCOPED_TRACE(std::string(filter)); + testing::internal::CaptureStderr(); + auto m = mcpp::project::resolve_member_dir(ws, f.root, filter); + auto warned = testing::internal::GetCapturedStderr(); + ASSERT_TRUE(m.has_value()) << (m ? "" : m.error()); + EXPECT_EQ(*m, f.root / "modules/base"); + EXPECT_TRUE(warned.empty()) << warned; + } +} + +// Two members declare the same package name under different namespaces: +// the bare name is refused (naming both qualified names), and each +// qualified name selects its own member without any ambiguity. +TEST(ResolveMember, SameNameUnderTwoNamespacesRefusesTheBareNameOnly) { + Fixture f("dup-ns"); + f.member("a", "ws-common", "ns1"); + f.member("b", "ws-common", "ns2"); + auto ws = f.root_manifest("[workspace]\nmembers = [\"a\", \"b\"]\n"); + + auto bare = mcpp::project::resolve_member_dir(ws, f.root, "ws-common"); + ASSERT_FALSE(bare.has_value()); + EXPECT_NE(bare.error().find("ns1.ws-common"), std::string::npos) << bare.error(); + EXPECT_NE(bare.error().find("ns2.ws-common"), std::string::npos) << bare.error(); + + auto qa = mcpp::project::resolve_member_dir(ws, f.root, "ns1.ws-common"); + ASSERT_TRUE(qa.has_value()) << qa.error(); + EXPECT_EQ(*qa, f.root / "a"); + + auto qb = mcpp::project::resolve_member_dir(ws, f.root, "ns2.ws-common"); + ASSERT_TRUE(qb.has_value()) << qb.error(); + EXPECT_EQ(*qb, f.root / "b"); +} + +// A value that is one member's package name and a different member's +// directory selects the package -- the option names a package -- and warns, +// naming the other member and its path. +TEST(ResolveMember, PackageNameOutranksAnotherMembersDirectoryAndWarns) { + Fixture f("name-vs-dir"); + f.member("modules/base", "ws-base"); + f.member("ws-base", "other"); + auto ws = f.root_manifest( + "[workspace]\nmembers = [\"modules/base\", \"ws-base\"]\n"); + + testing::internal::CaptureStderr(); + auto m = mcpp::project::resolve_member_dir(ws, f.root, "ws-base"); + auto warned = testing::internal::GetCapturedStderr(); + ASSERT_TRUE(m.has_value()) << (m ? "" : m.error()); + EXPECT_EQ(*m, f.root / "modules/base"); + EXPECT_NE(warned.find("modules/base"), std::string::npos) << warned; + EXPECT_NE(warned.find("ws-base"), std::string::npos) << warned; +} + +// Two members share a directory basename (no package-name collision): the +// first in `[workspace] members` is still selected (a script written against +// it keeps working), and a warning now names the other and its path. +TEST(ResolveMember, DuplicateBasenameKeepsFirstMatchAndWarns) { + Fixture f("dup-basename"); + f.member("apps/core", "coreapp"); + f.member("libs/core", "corelib"); + auto ws = f.root_manifest("[workspace]\nmembers = [\"apps/core\", \"libs/core\"]\n"); + + testing::internal::CaptureStderr(); + auto m = mcpp::project::resolve_member_dir(ws, f.root, "core"); + auto warned = testing::internal::GetCapturedStderr(); + ASSERT_TRUE(m.has_value()) << (m ? "" : m.error()); + EXPECT_EQ(*m, f.root / "apps/core"); + EXPECT_NE(warned.find("libs/core"), std::string::npos) << warned; +} + +// A filter matching nothing lists every member with its package name and its +// path. +TEST(ResolveMember, NotFoundListsEveryMemberByNameAndPath) { + Fixture f("not-found"); + f.member("a", "widget", "acme"); + f.member("b", "gadget"); + auto ws = f.root_manifest("[workspace]\nmembers = [\"a\", \"b\"]\n"); + + auto m = mcpp::project::resolve_member_dir(ws, f.root, "no-such-member"); + ASSERT_FALSE(m.has_value()); + EXPECT_NE(m.error().find("acme.widget"), std::string::npos) << m.error(); + EXPECT_NE(m.error().find("'a'"), std::string::npos) << m.error(); + EXPECT_NE(m.error().find("gadget"), std::string::npos) << m.error(); + EXPECT_NE(m.error().find("'b'"), std::string::npos) << m.error(); +} + +// No filter on a rooted workspace acts on the root package; no filter on a +// virtual one is refused. Unaffected by #725, kept here as the resolver's +// baseline. +TEST(ResolveMember, NoFilterActsOnRootedRootAndRefusesVirtual) { + Fixture f("no-filter"); + f.member("a", "a"); + auto rooted = f.root_manifest("[workspace]\nmembers = [\"a\"]\n"); + auto onRoot = mcpp::project::resolve_member_dir(rooted, f.root, ""); + ASSERT_TRUE(onRoot.has_value()) << onRoot.error(); + EXPECT_TRUE(onRoot->empty()); + + auto virt = mcpp::manifest::parse_string("[workspace]\nmembers = [\"a\"]\n"); + ASSERT_TRUE(virt.has_value()) << virt.error().format(); + auto onVirtual = mcpp::project::resolve_member_dir(*virt, f.root, ""); + ASSERT_FALSE(onVirtual.has_value()); + EXPECT_NE(onVirtual.error().find("--workspace"), std::string::npos); +} + +} // namespace resolve_member diff --git a/tests/unit/test_xlings.cpp b/tests/unit/test_xlings.cpp index 95691533a..f7abc8069 100644 --- a/tests/unit/test_xlings.cpp +++ b/tests/unit/test_xlings.cpp @@ -599,7 +599,6 @@ TEST(XlingsInvocationEnv, GlobalModeIsAnAbsentProjectDirectory) { TEST(XlingsInvocationEnv, TheProcessEnvironmentIsUnchangedAfterwards) { namespace env = mcpp::platform::env; - // Held so that what the Windows prefix sets process-wide is restored too. env::ScopedEnv keepPath("PATH", env::get("PATH")); env::ScopedEnv keepHome("XLINGS_HOME", env::get("XLINGS_HOME")); env::ScopedEnv prior("XLINGS_PROJECT_DIR", std::string("prior-project")); @@ -626,6 +625,49 @@ TEST(XlingsInvocationEnv, TheProcessEnvironmentIsUnchangedAfterwards) { EXPECT_EQ(env::get("XLINGS_PROJECT_DIR"), std::optional("prior-project")); } +// #726: the sandbox's `bin` in front of PATH and XLINGS_HOME reach the xlings +// child only. They used to stay in the process on Windows, so every action of a +// build that had installed a payload found `xim:llvm`'s `cl`, `link` and `lib` +// shims in front of MSVC's tools, while a build that installed nothing did not. +TEST(XlingsInvocationEnv, NeitherPathNorTheHomeOutlivesTheInvocation) { + namespace env = mcpp::platform::env; + env::ScopedEnv keepPath("PATH", env::get("PATH")); + env::ScopedEnv keepHome("XLINGS_HOME", std::string("shell-home")); + const auto pathBefore = env::get("PATH"); + + auto e = xlings_env(""); + for (int i = 0; i < 3; ++i) { + mcpp::xlings::ScopedInvocationEnv scope(e); + (void)mcpp::xlings::build_command_prefix(e); +#if defined(_WIN32) + EXPECT_EQ(env::get("XLINGS_HOME"), std::optional(e.home.string())); + const auto path = env::get("PATH"); + ASSERT_TRUE(path.has_value()); + const auto bin = (e.home / "subos" / "default" / "bin").string(); + EXPECT_EQ(path->substr(0, bin.size()), bin) << *path; +#endif + } + EXPECT_EQ(env::get("PATH"), pathBefore); + EXPECT_EQ(env::get("XLINGS_HOME"), std::optional("shell-home")); + + // Building a command alone changes nothing on any platform. + (void)mcpp::xlings::build_command_prefix(e); + EXPECT_EQ(env::get("PATH"), pathBefore); + EXPECT_EQ(env::get("XLINGS_HOME"), std::optional("shell-home")); +} + +// #726: on Windows too the registry's xlings starts in its home, never in the +// directory mcpp was started from, so a project's `.xlings.json` cannot put it +// in that project's mode. +TEST(XlingsInvocationEnv, TheWindowsPrefixStartsInTheHome) { + mcpp::xlings::Env e; + e.home = "C:\\Users\\a b\\.mcpp\\registry"; + e.binary = "C:\\Users\\a b\\.mcpp\\registry\\bin\\xlings.exe"; + EXPECT_EQ(mcpp::xlings::windows_command_prefix(e), + "cd /d \"C:\\Users\\a b\\.mcpp\\registry\" && " + "\"C:\\Users\\a b\\.mcpp\\registry\\bin\\xlings.exe\""); +} + #if !defined(_WIN32) TEST(XlingsInvocationEnv, ThePosixPrefixRendersTheDecision) { auto global = mcpp::xlings::build_command_prefix(xlings_env(""));