Development - #36
Merged
Merged
Development#36
Conversation
# Conflicts: # .gitignore
`.copy` keeps an item's own name and inner structure and nothing above it, while a structured resource keeps its whole path: GRDB's `Betty.jpeg` ended up at `Sources/GRDBTests/GRDBTests/Betty.jpeg` inside the bundle, and the test asking the bundle for `Betty.jpeg` found nothing. Each directory a copied item sits in becomes a resource group that strips the path above the item. A group and a glob cannot be added together in one attribute, so once there is a group the processed resources become one too.
A `swift_test` never bundles resources: `apple_resource_bundle` is something a bundling rule consumes, so `Bundle.module` had nothing to find and GRDB's tests died on the first resource they asked for. A test target is now a library plus the macOS test bundle over it, at the version the package asks of macOS.
The generated manifest's `path:` dependencies were read back out of it with a regular expression, to find the local packages whose sources are read in place. The plugin that writes that manifest already has them, so they are handed to the loader instead: one function and one failure mode fewer, and a local package can no longer go missing because the manifest could not be read.
A package in the project's own repository has its build tool plugins run while the workspace is generated, and the run said they were not: the report covered every package, and a plugin target was listed as an unsupported kind on top of that. Only a dependency's plugins go unrun, so only those are named. A plugin that was meant to run and could not is named as well, through the notes a run prints rather than the unified log nothing shows: what the target loses is whatever the plugin generates, and the compile error that follows names those files instead of the plugin. A plugin target is never a rule this workspace builds, whatever its capability, so the distinction between the two kinds no longer decides anything and is gone.
Only the Swift files a plugin wrote were taken, so a plugin that generates C sources, a header for them, or a resource wrote into a target that never saw them. SwiftPM splits what a plugin produced by the target's own file rules, into its sources and its resources, and that split is now mirrored: - an extension the target compiles goes into its `srcs`; - a header is neither compiled nor bundled. It is an input of the generated source beside it, which includes it by name — the only thing SwiftPM offers either, since a hand-written source cannot reach a generated header; - everything else is a resource, so a target whose only resources come from a plugin gets a bundle, as it does under SwiftPM.
The corpus's plugins are all linters and TbCodeGenerater's writes Swift, so nothing proved a plugin's other outputs reach the target that asked for it. The fixture's local package now has a build tool plugin whose tool — the package's own executable target — writes a Swift file and a resource for a Swift target, and a C source with its header for the C-family one. Both targets use what it wrote, so the app fails to build if any of the three kinds is dropped. The header is declared rather than included by the hand-written source: SwiftPM puts a generated header on no search path of one, which is what the rules mirror.
The SwiftPM notes now say which of a plugin's files become sources, which is an include of the generated source beside it, and which are bundled, plus that a plugin which could not run is named. Two decisions are recorded rather than left open: `Package.swift` and `Package.resolved` stay because `.build/checkouts` is where a rule's sources live and only those two rebuild it, and every SwiftPM step stays a toolchain command instead of libSwiftPM — measured at 0.6s per manifest, and slower when run concurrently.
A project references Xcode's own settings as freely as its own, and nobody can type them in: iina writes `$(SDK_VERSION)` and `$(XCODE_VERSION_ACTUAL)` into its `Info.plist`, and neither appears anywhere in the project file, so both keys were dropped for carrying a reference that could not be resolved. They are read from the installed toolchain instead, as defaults a project that states one of them itself still wins over: `xcodebuild -showsdks` answers for every platform's SDK name and version at once, and `xcodebuild -version` for the four digit number Xcode spells itself as. `CONFIGURATION` needs no asking — it is the name of the configuration being generated. `xcodebuild -showBuildSettings` knows all of them without a table, but it resolves the package graph and takes some ten seconds per target; these two commands take under a second for a whole run. The platform is the one a target resolves to rather than whatever `SDKROOT` says, because that setting is optional and `auto` names no SDK.
No stage implements `.package(id:)`: nothing in the corpus uses one, and the work is recognising one more kind of dependency rather than changing the shape of the output, so the open question becomes a stated non-goal with what happens until then. The note about substituting `$(SETTING)` in a plist says where that landed instead.
Every file found under a target's plugin output directory was linked into one flat directory by its name alone, and everything found there was taken as the target's own. Neither holds: a plugin has a directory of its own under that root and writes a tree inside it if it likes, and SwiftPM leaves the files a previous build declared in place rather than deleting them — so a run picked up a file no plugin produces any more and bundled it. The output directory of a target is therefore removed before SwiftPM is asked to build it, and what it then contains is linked keeping its path. The fixture's plugin writes its resource into a subdirectory to hold that.
Two modules read the same `.xcodeproj`: `XCode`, which nothing but the plugin loader still imported, and `XCode2`, which everything that generates a workspace is written against. The first one is gone, and the second holds the name — spelled the way Apple spells the product, so the module, its types, its files and the `bazelize xcode` subcommand all read `Xcode`. Removing it takes libSwiftPM with it: `SwiftPMDataModel` was linked for one function in the old parser, and it pulled SwiftDriver, llbuild, swift-crypto and the rest behind it. The package resolves 9 dependencies now instead of 18, and no longer pins a branch to match the toolchain — which is what the SwiftPM notes said the alternative to the `swift` command would cost. `XCodeBazelize` and the `XCodeProj` repo enum keep their spelling: one is the organisation, the other names generated files.
`Cocoapod` was not a target of this package, so nothing built it and nothing could: it read the Xcode parser that is gone. Its tests, the `--skip CocoapodTests` the test target still carried, the three `pod` steps in CI and `Env.pod` — which no code read — go with it. Everything that spelled the product `XCode` now spells it `Xcode`: the `XcodeProj` repo enum and the plugin named after it, the file they live in, and the prose in the docs. `XCodeBazelize` keeps its capitals, being the name of the organisation this repository lives in rather than a word about Xcode.
`swift package resolve` was run in the output directory whatever the project was, and a project with no Swift packages has no manifest written there for it to read: the run ended on `Could not find Package.swift`, after the rules were already written, so the workspace looked generated and was not. stats is such a project.
A file whose path does not start with the project's is dropped to its name alone, and two spellings of one path do not compare equal: `/tmp` is a link to `/private/tmp`, and a file resolved through XcodeProj keeps the link where the root has already lost it. Every such file then globbed as `Sources/<name>/**` — matching nothing — and the package failed to load with `glob pattern didn't match anything`. Both sides are resolved with `realpath` before the comparison, which is what the SwiftPM plugin already does for the same reason.
The corpus is not in this repository — `app/` and `spm/` are ignored — so CI never touched what the notes measure. Two lanes now do, both downloading the `bazelize` artifact the release build already uploads rather than building it again: one clones an app, generates its workspace and builds it with Bazel, the other clones a package whose tests only compile through a source its own build tool plugin generates. Every entry pins the revision it was measured against, so upstream moving is a deliberate bump rather than a red build nobody caused. Each lane keeps a Bazel disk cache of its own, and the matrix does not fail fast: one app failing says nothing about the others.
`swift build` had its standard error discarded, so a plugin that did not run was reported as "swift build failed" and nothing else — which is every reason at once: a toolchain that cannot build the target, a checkout it could not fetch, a source that does not compile. The error lines SwiftPM printed are carried into the note instead. CI stops on that note rather than on what it causes: a target missing the sources a plugin generates fails to compile several steps later, naming those sources and never the plugin.
Two things the runner's Swift 6.3 does that 6.4 does not, both of which end as "cannot find X in scope" several steps after the cause: - a build tool plugin's tool is looked up as a product, so an executable target without a product of the same name fails the whole package with "no product named". The fixture's `local1-tool` product is named `Local1Tool` now. - a package with no `platforms:` builds its macro for the oldest macOS SwiftPM supports, which is older than swift-syntax declares. The fixture states macOS 10.15. 6.3 also does not run a build tool plugin for a C-family target at all — a clean build produces the Swift target's generated sources and nothing for the clang one — so the fixture no longer calls into what the plugin generates for `LocalTarget3`. The rules still compile it wherever the toolchain produces it. iina and MacPass leave the CI matrix commented out until they are green on a runner rather than only here.
iina waits on the dylib-as-source problem the notes already describe, MacPass on a Carthage step nothing has proven on a runner, and TbCodeGenerater on a product rename in its own repository. The three differences between the runner's Swift 6.3 and the 6.4 used here are written down beside them, because each one ends as a missing symbol rather than as anything about a plugin.
Asking SwiftPM to run them meant building the target they are attached to, which fails for reasons that have nothing to do with the plugin — and on Swift 6.3 silently does not run a plugin for a C-family target at all, reporting success and producing nothing. bazelize is the host now: it compiles the plugin against the toolchain's `PackagePlugin`, hands it the package graph and the directory its output belongs in, and runs the commands it asks for. The protocol is SwiftPM's: a length-prefixed JSON message each way over the plugin's standard input and output. Only what a build tool plugin uses is mirrored, and a message that cannot be decoded is reported as the toolchain speaking a protocol this does not, rather than read as an absent command. What this buys, measured on both toolchains: the fixture's C-family target gets its generated sources on Swift 6.3, which `swift build` never produced there. The files are written where they belong rather than linked out of `.build`, so the generated workspace no longer needs the package's build directory to exist. The tool a plugin runs is still built by SwiftPM — it is an executable target with ordinary dependencies — but only that one product, looked up in the manifest rather than guessed from the target's name.
What a build tool plugin writes is the plugin's to decide, so changing the
plugin changes the files a target compiles while nothing else about the project
moves — and regenerating the whole workspace to pick that up is the wrong unit
of work. The generated workspace now carries the command instead:
bazel run //:plugins
It runs `bazelize plugins`, which compiles each plugin, hands it the package
graph and the directory its output belongs in, and writes what comes back into
`Packages/*/Generated/*Plugin` — the rules are not touched.
They do not need touching, because they no longer name a plugin's files: the
rules glob the directory by the kinds of file found in it. A plugin that starts
writing `Generated2.swift` instead of `Generated.swift` needs `bazel run
//:plugins` and nothing else. Only the kinds that are there are globbed, so an
empty directory fails the package loudly rather than compiling without sources.
`sh_binary` comes from rules_shell, which Bazel no longer ships itself.
The guard that stops a lane when a plugin did not run matched the old wording, which no longer exists: a run now names the plugin it could not run rather than saying a target's plugins failed as a group. The package lane is commented out with the reason it cannot pass yet — Swift 6.3 looks a plugin's tool up as a product, and TbCodeGenerater's product is named `tbCodeGenerater` while its plugin names the target `TbCodeGenerater`, so the package does not build on a runner at all.
Each lane kept a Bazel disk cache and nothing else, so every run fetched rules_apple, rules_swift and the rest again, and resolved the app's Swift packages from scratch before a rule was written. Three caches now, written into `~/.bazelrc` so every `bazel` in the job reads them: the disk cache for what a build produced, the repository cache for what it downloaded, and the output directory's `.build` for what SwiftPM resolved. The fixture lane was caching a directory nothing writes, keyed on a `WORKSPACE` file that no longer exists; it caches the same three things now. `swift_deps.bzl` and `swift_deps_index.json` go with them: rspm generated those into the fixture and nothing has read them since it was removed.
bazelize used to generate into the fixture directory itself, and the leftovers were still there: a `BUILD` in every target's source directory, a `Package.swift` and its resolution at the fixture root, a `.bazelrc` importing a `config.bazelrc` that no longer exists, a `.bazelversion` pinning a Bazel the workspace stopped using, and 450MB of `cache/` and `.build/` from builds run out of that layout. The workspace is `App/` now, generated whole by `make bazelize`, and it carries its own `.bazelrc` and `.bazelversion`. The ignore rules followed the old layout too, naming each file it generated into the fixture. One directory is ignored now — and explicitly: `App/` was only being ignored because a rule meant for the corpus directory `app/` matches it on a case-insensitive filesystem.
…time The runner ran four simulator tests at once against a simulator that did not exist yet, so each paid to create and boot its own: the first test took 247s, the second 152s, and the last timed out at 302s having spent all of it waiting. None of that is the tests — locally the same four take four seconds against a simulator that is already there. The lane creates and boots the device up front, outside any test's clock, and runs the tests serially so they share it — rules_apple's runner reuses a simulator of the right type and version rather than making another. The timeout is raised to 900s and failures print their log, so the next one of these says what happened instead of only that it took too long.
Two ways the last run failed, neither of them the code under test: The simulator step named a runtime this repository is developed against — `iOS-27-0` — and the runner ships whatever its own Xcode has, so `simctl` answered `Invalid runtime` and the step took the job down with it. The runner's newest available iOS runtime and an iPhone on it are looked up instead, created only if there is none, and the device and version are passed to `bazel test` rather than written into it. A device name has spaces (`iPhone Air`), so the three fields come back one per line. The app lane restored `<app>/App/.build` before cloning the app, and `git clone` refuses a directory that exists. The cache is restored after the clone.
A plugin was compiled by bazelize and its tool built by SwiftPM, which is the one thing left that could fail for reasons having nothing to do with the plugin: a toolchain that looks a plugin's tool up as a product cannot even load the package the tool lives in, so nothing about that package can be generated with its plugins run. Both are Bazel targets now. A plugin is a `swift_binary` with the toolchain's `PackagePlugin` on its search path — it links nothing else — and the tool already was one. `//:plugins` carries them as `data`, so `bazel run //:plugins` builds them and hands their paths to the host, which speaks the protocol and runs what the plugin asks for. No second Bazel inside the first one's lock, and no SwiftPM in the plugin path at all. The rules glob a target's plugin directory whenever the target declares a plugin, before anything has written there, so `allow_empty` is set: a package that cannot load is a package that cannot run `//:plugins` to fill it. Measured with the plugin directory, `.bazelize` and the package's `.build` deleted: `bazel run //:plugins` puts all four of the fixture's generated files back without compiling a plugin or building a tool itself, and does the same for TbCodeGenerater. The lane for that package is a lane again.
The fixture is an Xcode project, and Xcode's build system does not run a build tool plugin for a C-family target: `LocalTarget3` called a symbol from the source `Local1Gen` generates for it, so opening the fixture in Xcode ended in `Undefined symbol: _local1_plugin_value`. `Local1Macros` and `Local1Tool` go with it — the package is back to the three plain targets it had. What those covered is covered by a package that is not an Xcode project: `spm/TbCodeGenerater`, whose tests only compile through a source its own build tool plugin generates, and whose plugin and tool are built by Bazel. The SwiftPM notes said the fixture stood for macros and generated sources; they name that package now.
`//:plugins` is written into the root `BUILD` for any project that has Swift packages — whether one of those packages has a build tool plugin is not known when that file is written — but the `//Packages:plugins` it names was only written when there was a plugin to put in it. A project with packages and no plugins therefore had a label that does not resolve, and `bazel build //...` failed to analyse before it built anything. Rectangle is such a project. The group and the script are written for any project with packages now, empty when there is nothing to run: a command that does nothing beats a workspace that does not load.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.