fix(packages): read ebuild.lock back, so it pins what it records

.gitattributes

@@ -0,0 +1,8 @@
+# YAML is linted with yamllint's default new-lines rule (LF). The Windows
+# runners check out with core.autocrlf=true, which turned every LF file into
+# CRLF and failed the rule on line 1 of the first file -- except a file that
+# already carried a stray CR, which autocrlf leaves alone. Pin the encoding
+# instead of guessing at the checkout's, so the same bytes are linted on
+# every OS.
+*.yml   text eol=lf
+*.yaml  text eol=lf

.github/workflows/auto-assign.yml

@@ -30,4 +30,4 @@ jobs:
               repo: context.repo.repo,
               issue_number: context.payload.pull_request.number,
               assignees: [context.payload.pull_request.user.login]
-            });
+            });

.github/workflows/scorecard.yml

@@ -21,14 +21,14 @@ jobs:
       security-events: write
       id-token: write
     steps:
-      - uses: actions/checkout@v4
+      - uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262  # v4
         with:
           persist-credentials: false
-      - uses: ossf/scorecard-action@v2.4.0
+      - uses: ossf/scorecard-action@4eaacf0543bb3f2c246792bd56e8cdeffafb205a  # v2.4.3
         with:
           results_file: results.sarif
           results_format: sarif
           publish_results: true
-      - uses: github/codeql-action/upload-sarif@v3
+      - uses: github/codeql-action/upload-sarif@faaca9a8f6edddba5725ffe5adefdab6669a2eca  # v3
         with:
           sarif_file: results.sarif

CHANGELOG.md

@@ -22,6 +22,27 @@
   `cjson` (v1.7.18), `nanopb` (v0.4.9.1), `lvgl` (v9.2.2), `tinyusb` (v0.18.0), and `unity` (v2.6.1).
 
 ### Fixed
+- **`ebuild.lock` is read, not only written.** `_install_packages` created the `Lockfile` after resolution and only ever saved it, and nothing called `Lockfile.load()`, so the lock pinned nothing: an unpinned package resolved to the newest recipe on every machine and every run, and a stale `ebuild.lock` sat beside it claiming otherwise. The lock is now loaded before resolving and handed to `PackageResolver.resolve(requested, lockfile=...)`, which uses it as a pin for every package the request leaves open. Precedence is request, then lock, then newest. A locked version no recipe provides is an error (`ebuild.lock pins 'zlib' at v9.9.9 ...`), not a silent fallback, and a locked entry whose recorded URL or checksum no longer matches the recipe of that version is refused -- the version reproduces the name, the lock exists to reproduce the bytes. An explicit request outranks the lock and the lock is rewritten from the result, as before. `Lockfile.load()` drops malformed entries instead of failing later. A lock that is not valid YAML, or cannot be opened, is reported as a resolution error (`[error] ... ebuild.lock is not valid YAML: ...`) instead of a `yaml.scanner.ScannerError` traceback; the recorded `build` system is compared on read-back like `url` and `checksum` (`Lockfile.CHECKED_FIELDS` is the one list both sides use); and `docs/dependency-management.md` documents the file, its precedence, and how to upgrade. (`ebuild/packages/resolver.py`, `ebuild/packages/lockfile.py`, `ebuild/cli/commands.py`, `docs/dependency-management.md`)
+- **CI on master runs to completion again.** ruff stopped the pipeline at its
+  first step on findings the merges had introduced (an F811 duplicate import,
+  W292, E402). `PackageRecipe.to_dict()` -- defined by #111, deleted by #112's
+  replay of the same file -- is restored; nine `test_index_sync` cases and mypy
+  had failed without it. It now emits `install_args`, which the original never
+  did, and hands back copies of its list fields rather than the live lists;
+  the index sync mapping carries `install_args` through to the cached recipe.
+  The vendored `core/eos/docs/three-way-alignment.md` is reverted to its pin
+  (the correction #109 made there is filed upstream as embeddedos-org/eos#149).
+  The OSSF Scorecard action moved to the ghcr.io-hosted release and is pinned
+  by commit. yamllint on the Windows legs: YAML is pinned to LF in
+  `.gitattributes`, so an existing Windows clone needs its files checked out
+  again once (`git rm --cached -r . && git reset --hard HEAD`, or a re-clone;
+  see CONTRIBUTING.md). On Python 3.10 and 3.11 `ebuild/plugins/__init__.py`
+  now type-checks: the `entry_points()` fallback is spelled out with a cast
+  instead of a `# type: ignore` naming the wrong error code.
+  (`ebuild/packages/recipe.py`, `ebuild/packages/index_sync.py`,
+  `ebuild/plugins/__init__.py`, `core/eos/docs/three-way-alignment.md`,
+  `.github/workflows/scorecard.yml`, `.gitattributes`, `.yamllint.yml`, and
+  the three lint-fixed test files.)
 - **`ebuild test` now finds Windows test binaries.** The Ninja edge for a
   native `type: test` target already carried the platform suffix
   (`_exe_suffix()` names it `<name>.exe` on Windows), but `ebuild test`

CONTRIBUTING.md

@@ -20,6 +20,14 @@ This certifies you have the right to submit the code under the MIT license.
 6. Commit with DCO sign-off: git commit -s
 7. Push and create Pull Request
 
+**Windows contributors:** `.gitattributes` pins `*.yml` and `*.yaml` to LF so
+yamllint sees the same bytes on every platform. The attribute governs future
+checkouts, not files already sitting in a working tree, and `git add
+--renormalize .` rewrites only the index, never the files. After pulling that
+change, from a clean tree run `git rm --cached -r . && git reset --hard HEAD`
+once (or re-clone) so the YAML files are checked out again as LF; otherwise
+yamllint will still see CRLF locally.
+
 ## Coding Standards
 
 ### C (ISO C11)

core/eos/docs/three-way-alignment.md

@@ -8,7 +8,7 @@ This document tracks alignment between all three EoS components to ensure they r
 
 | Dimension | eos | eboot | ebuild | Status |
 |-----------|-----|-------|--------|--------|
-| Board definitions | 84 board YAMLs in `eos/boards/` (upstream) | 83 board dirs / 138 `eboot_add_board()` calls in `eboot/boards/` (upstream, pinned rev) | `TARGET_ARCH` 14 + `MCU_TO_EBOOT_BOARD` 138 in `ebuild/sdk_generator.py` | ⚠️ Unverified: 84 vs 83 vs 14/138 — three inventories describe the same set and nothing cross-checks them; see the resolver drift note in PR #109 |
+| Board definitions | 25 YAML files in `eos/boards/` | 25 board ports in `eboot/boards/` | `MCU_TO_EBOOT_BOARD` + `EOS_BOARD_MAP` in project generator | ✅ Aligned |
 | Product profiles | 41 profiles in `eos/products/*.h` | — | `PRODUCT_MAP` (41 entries) in project generator | ✅ Aligned |
 | Platform enum | — | 24 `eos_platform_t` entries in `eos_hal.h` | MCU_DATABASE (100+ MCUs) in hw analyzer | ✅ Aligned |
 | Peripheral keywords | 33 HAL APIs in `hal.h` + `hal_extended.h` | — | `PERIPHERAL_KEYWORDS` (24 types) + `ComponentDB` (200+ parts) | ✅ Aligned |
@@ -118,7 +118,7 @@ Customer Input
 │     LLMClient (optional) ──► deep analysis                  │
 │                                                              │
 │  2. EosProjectGenerator                                     │
-│     MCU_TO_EBOOT_BOARD (alias of ebuild/sdk_generator.py) ► eboot board dir  │
+│     MCU_TO_EBOOT_BOARD ──────────────────► eboot board dir  │
 │     EOS_BOARD_MAP ──────────────────────► eos board YAML    │
 │     PRODUCT_MAP (41 entries) ──────────► eos product .h     │
 │     MULTICORE_MCUS ─────────────────────► multicore config  │

docs/dependency-management.md

@@ -360,6 +360,31 @@ ebuild includes a curated suite of pre-packaged recipes under `recipes/`:
 | **`unity`** | 2.6.1 | CMake | MIT | Standard embedded C unit testing framework |
 | **`zlib`** | 1.3.1 | CMake | Zlib | General-purpose lossless data compression |
 
+### `ebuild.lock` — Reproducible Package Resolution
+
+Every resolution (`ebuild configure`, `ebuild build`, `ebuild package`) writes
+`ebuild.lock` next to `build.yaml`, recording for each resolved package the
+exact `version`, `url`, `checksum` and `build` system it used — and reads it
+back first, so the next resolution lands on the same artifacts:
+
+- **Precedence:** a version named in `build.yaml` wins; otherwise the version
+  in `ebuild.lock` wins; only a package in neither is resolved to the newest
+  recipe. A lock never overrides an explicit request, and an unpinned package
+  therefore stays put until you move it.
+- **Drift is refused, not silently accepted.** If the lock pins a version whose
+  recipe now has a different `url`, `checksum` or `build`, resolution stops with
+  an error naming the field, because the version alone would reproduce the
+  package's name and not its bytes. Delete `ebuild.lock` or pin the version
+  explicitly if the change is intended.
+- **Commit it** for a reproducible build; CI and colleagues then resolve to the
+  same artifacts you did.
+- **To upgrade** a locked package, name the new version in `build.yaml`, or
+  delete `ebuild.lock` to resolve everything afresh; either way the next
+  resolution rewrites the lock.
+- A lock that is not valid YAML, or cannot be opened, is reported as a
+  resolution error and stops the build. A lock of the wrong shape (an entry
+  that is not a mapping) is ignored entry by entry and rewritten.
+
 ### Offline & Air-Gapped Operation
 
 For isolated CI/CD pipelines and field deployments:

ebuild/cli/commands.py

@@ -40,7 +40,7 @@
 from ebuild.packages.builder import BuildError, PackageBuilder
 from ebuild.packages.cache import PackageCache
 from ebuild.packages.fetcher import FetchError, PackageFetcher
-from ebuild.packages.lockfile import Lockfile
+from ebuild.packages.lockfile import Lockfile, LockfileError
 from ebuild.packages.recipe import RecipeError
 from ebuild.packages.registry import create_registry, find_recipe_dirs
 from ebuild.packages.resolver import PackageResolver, ResolveError
@@ -81,16 +81,26 @@ def _install_packages(
     registry = create_registry(*recipe_dirs)
     log.debug(f"Registry: {registry.package_count} recipes from {[str(p) for p in registry.search_paths]}")
 
+    # The lockfile is read before resolving, so an unpinned package lands on
+    # the version the last resolution recorded rather than on whatever is
+    # newest now. It was only ever written before, which pinned nothing.
+    lock_path = cfg.source_dir / Lockfile.FILENAME
+    lockfile = Lockfile(lock_path)
+    try:
+        lockfile.load()
+    except LockfileError as e:
+        # A lock that cannot be read is a resolution the user has to settle,
+        # reported through the handler every caller of this function has.
+        raise ResolveError(str(e)) from e
+    if lockfile.package_names:
+        log.debug(f"Lockfile read: {lock_path} ({len(lockfile.package_names)} packages)")
+
     resolver = PackageResolver(registry)
     requested = [{"name": p.name, "version": p.version} for p in cfg.packages]
-    resolved = resolver.resolve(requested)
+    resolved = resolver.resolve(requested, lockfile=lockfile)
 
     log.info(f"Packages to install: {', '.join(r.name + ' v' + r.version for r in resolved)}")
 
-    # Lockfile
-    lock_path = cfg.source_dir / Lockfile.FILENAME
-    lockfile = Lockfile(lock_path)
-
     # Cache and fetcher
     pkg_cache_dir = build_dir / "packages"
     cache = PackageCache(pkg_cache_dir)

ebuild/packages/index_sync.py

@@ -343,6 +343,7 @@ def sync(
                 "dependencies": entry.get("dependencies", []),
                 "configure_args": entry.get("configure_args", []),
                 "build_args": entry.get("build_args", []),
+                "install_args": entry.get("install_args", []),
                 "patches": entry.get("patches", []),
             }
 

ebuild/packages/lockfile.py

@@ -17,6 +17,16 @@
 from ebuild.packages.recipe import PackageRecipe
 
 
+class LockfileError(Exception):
+    """``ebuild.lock`` exists but could not be read.
+
+    Distinct from a lock of the wrong *shape*, which :meth:`Lockfile.load`
+    sanitises to an empty lock: a file that is not YAML, or cannot be opened,
+    is a condition the user has to resolve, and the CLI reports it as a
+    resolution error rather than a traceback.
+    """
+
+
 class Lockfile:
     """Manages the ebuild.lock file for reproducible package resolution.
 
@@ -26,6 +36,15 @@ class Lockfile:
 
     FILENAME = "ebuild.lock"
 
+    # Lock field -> PackageRecipe attribute, for every field lock() records
+    # that identifies the artifact. The resolver compares each of these; a
+    # field recorded here and compared nowhere is what this file used to be.
+    CHECKED_FIELDS = (
+        ("url", "url"),
+        ("checksum", "checksum"),
+        ("build", "build_system"),
+    )
+
     def __init__(self, lock_path: str | Path) -> None:
         self.lock_path = Path(lock_path)
         self._entries: Dict[str, Dict[str, str]] = {}
@@ -39,15 +58,37 @@ def load(self) -> None:
         if not self.lock_path.exists():
             return
 
-        with open(self.lock_path, "r", encoding="utf-8") as f:
-            raw = yaml.safe_load(f)
+        try:
+            with open(self.lock_path, "r", encoding="utf-8") as f:
+                raw = yaml.safe_load(f)
+        except (yaml.YAMLError, UnicodeDecodeError) as e:
+            # A lock that is not UTF-8 (a foreign editor, a bad binary merge)
+            # fails in the codec, not the parser, and UnicodeDecodeError is a
+            # ValueError -- neither a YAMLError nor an OSError. Same message:
+            # the file is not something this tool wrote, and the remedy is
+            # the same.
+            raise LockfileError(
+                f"{self.lock_path} is not valid YAML: {e}. "
+                f"Fix it, or delete it to resolve afresh."
+            ) from e
+        except OSError as e:
+            raise LockfileError(
+                f"{self.lock_path} could not be read: {e}. "
+                f"Fix it, or delete it to resolve afresh."
+            ) from e
 
         if not isinstance(raw, dict):
             return
 
         packages = raw.get("packages", {})
         if isinstance(packages, dict):
-            self._entries = packages
+            # Keep only well-formed entries; a hand-edited or truncated file
+            # must not turn into a KeyError deep inside the resolver.
+            self._entries = {
+                str(name): {str(k): str(v) for k, v in entry.items() if v is not None}
+                for name, entry in packages.items()
+                if isinstance(entry, dict)
+            }
 
     def save(self) -> None:
         """Write the lockfile to disk."""

ebuild/packages/recipe.py

@@ -89,6 +89,37 @@ def validate(self) -> None:
                 f"Must be one of {self.VALID_BUILD_SYSTEMS}."
             )
 
+    def to_dict(self) -> Dict[str, Any]:
+        """Convert recipe to dictionary for YAML serialization."""
+        data: Dict[str, Any] = {
+            "package": self.name,
+            "version": self.version,
+        }
+        if self.description:
+            data["description"] = self.description
+        if self.license:
+            data["license"] = self.license
+        data["url"] = self.url
+        if self.checksum:
+            data["checksum"] = self.checksum
+        data["build"] = self.build_system
+        # Copies, not the live lists: a caller that appends to what it got
+        # back must not edit the recipe behind its back. The key order is the
+        # one index_sync's recipe_dict uses, with install_args after
+        # build_args; parse_recipe() reads every key by name, so a dump and a
+        # reload agree field for field regardless of order.
+        if self.dependencies:
+            data["dependencies"] = list(self.dependencies)
+        if self.configure_args:
+            data["configure_args"] = list(self.configure_args)
+        if self.build_args:
+            data["build_args"] = list(self.build_args)
+        if self.install_args:
+            data["install_args"] = list(self.install_args)
+        if self.patches:
+            data["patches"] = list(self.patches)
+        return data
+
 
 def _parse_string_list(
     raw: Dict[str, Any],