Skip to content

emrg: ask whether a release is owed at all - #2958

Open
pm25coder wants to merge 5 commits into
masterfrom
feature/release-owed-guard
Open

pm25coder wants to merge 5 commits into
masterfrom
feature/release-owed-guard

Conversation

@pm25coder

Copy link
Copy Markdown
Collaborator

Closes #2957

scripts/check-release-owed.py (new) plus tests/test_check_release_owed.py (7 cases). No existing guard is changed and no workflow is touched.

What it answers

The five release guards all start from a tag that already exists, so a tree declaring a version nothing has tagged is invisible to every one of them. This one asks that question directly: it reads emrg/__init__.py's anchored __version__ (the same file and shape check-release-tag.py and tests/test_version_sync.py use), reads the repository's tags, and compares them by numeric parts.

On this checkout, unmodified:

tree: C:\Users\Administrator\.emrg\evolution\emrg
declared: 0.3.59 (emrg\__init__.py)
tags: 158 version-shaped, newest v0.3.58
A RELEASE IS OWED: the tree declares 0.3.59 and the newest tag is v0.3.58 -- 9 commit(s)
on HEAD are not in v0.3.58, so that work has merged and reaches no host until it is
published. Cut the release: ...                                                        rc 1

The three facts a release request needs — declared version, newest tag, commits between — are the output, not a summary of it.

Versions compare as numbers

0.3.10 is newer than 0.3.9; string order says the opposite, and a guard that sorted names as strings would pick v0.3.9, then report a release owed for a version already tagged — a false red on the very release it was asked to confirm. One test pins that pair. A tag that is not three dotted numbers (latest) is counted, reported as skipped, and never chosen as the newest.

It is a host-side screen, not a CI gate

On an unreleased master it is red by design. Wired into CI it would make the release cadence a permanently failing check and teach its readers to ignore it, so the acceptance is the command a host runs before cutting a release — the role bump-version.py --check and check-release-tag.py already play.

Two things the tests changed

A defect the suite caught in the guard, before this PR existed. My first version ran git tag --list with cwd=root and nothing else. git walks up to find a repository, so pointed at any plain directory inside any checkout it read that checkout's tags and answered about a repository it was never asked about — measured here as "declares 0.3.59, newest v0.3.58, release owed" for a directory that is not a work tree at all. That is the class of defect this session keeps finding: a reading about a subject it did not open. The guard now requires git rev-parse --show-toplevel to equal the root it was given and reports not measurable otherwise. The test that found it is test_an_unreadable_repository_is_unmeasurable_never_a_pass.

One acceptance item in the issue was wrong, and is corrected here. Item 3 listed "no tags at all" among the exit-2 cases. An empty tag list is a measurement — git tag --list answered, and the answer is that nothing carries the declaration — so it is exit 1, and a fresh fork that has published nothing is told exactly that rather than "not measurable". Exit 2 stays for the questions that could not be put: no readable declaration, no repository, a git that will not run.

Verification

  • 7 tests, 7 passed through a harness that supplies tmp_path and capsys (pytest is not installable on this machine); CI is the first collection of them, so its state here is CI pending, never green.
  • Each case builds its own git init repository under tmp_path, with the identity pinned by flags so a host's own git config cannot decide whether a fixture can commit.
  • check-undefined-names.py OK over 439 files; py_compile on both new files; the client import and python -m emrg --help both clean.
  • Both directions on the real checkout: exit 1 at the work tree root, exit 2 when pointed at scripts/.

What this does not do

It does not cut the release. v0.3.59 is still untagged and this PR does not tag anything — pushing that tag triggers a six-platform build and a real publication, which is a separate decision with its own pre-tag guards already green on master.

@pm25coder

Copy link
Copy Markdown
Collaborator Author

CI caught a real defect in this PR, and it was mine: the new guard was never classified in tests/test_a_tree_reading_guard_names_its_tree.py, the file that requires every scripts/check-*.py to be accounted for before it can pass. Fixed in 32e9bd5a.

What failed, and why it is right

Both legs failed on the same two tests:

FAILED tests/test_a_tree_reading_guard_names_its_tree.py::test_every_guard_in_the_family_is_classified
  AssertionError: these guards are in scripts/ and are not classified above: ['check-release-owed.py']
FAILED tests/test_a_tree_reading_guard_names_its_tree.py::test_every_guard_advertising_a_root_flag_is_accounted_for
  AssertionError: check-release-owed.py advertises `--root` in its own --help and is in neither
  POINTABLE_AT_A_TREE nor ROOT_FLAG_BUT_NOT_POINTABLE

The first version of this PR added a tree-reading guard with a --root flag and did not tell the family's own bookkeeping table. That is precisely the hole the two tests exist to close, and the fix is to classify rather than to exempt.

Where it now sits, and why there rather than in RUN_HERE

POINTABLE_AT_A_TREE           "check-release-owed.py": ("--root",)
NAMES_ITS_TREE_BUT_IS_NOT_RUN_HERE   reason recorded in the entry

The table's membership rule is "<guard> --root <tree> is an invocation that guard accepts", not "it has a root flag". This guard takes no required positional, so that invocation is valid and it belongs in the pointable table — unlike check-release-tag.py, whose required tag argument is why it sits in ROOT_FLAG_BUT_NOT_POINTABLE.

It does not belong in RUN_HERE, whose leg requires rc 0 on the checkout under test. This guard answers rc 1 whenever a release is owed, which is correct and is exactly the interval between a bump merging and its tag being cut. Putting it in that loop would make every such interval a failing suite — the same reason it is a host-side screen and not a CI gate. The bucket's entry says so and names the test that verifies the naming it claims.

The claims the classification leans on are now measurements

The bucket's convention is that a member's naming IS verified rather than asserted, so two tests were added rather than the claim being written down alone:

  • test_the_tree_it_read_is_named_before_the_verdict — the first line is tree: <the tree it was pointed at>.
  • test_a_pointable_guard_refuses_a_tree_it_cannot_read — an empty directory and an absent path both answer rc 2 with a reason, and print no verdict alongside it. The same two shapes the family's empty-tree leg puts to every pointable member.

Verified on this head

Four family legs called directly (pytest is not installable on this host, so the bodies are run through a stub harness):

PASS test_every_guard_advertising_a_root_flag_is_accounted_for
PASS test_every_pointing_row_names_a_guard_that_accepts_what_it_names
PASS test_every_guard_in_the_family_is_classified
FAIL test_a_guard_pointed_at_a_tree_it_read_nothing_from_refuses_to_pass
     check-staged-inverse.py answered rc=0 for an empty directory ...
     tree: C:\Users\Administrator\.emrg\evolution\emrg

The fourth is not this PR, and the mechanism is visible in its own output: it named this checkout for a directory I had created inside it. git walks up to find a repository, so a temp path inside the checkout resolves to the checkout — on CI that leg's directory lives under /tmp, outside any repository, which is why CI passes it and named only my two failures. Measured both ways here: that guard refuses (rc 2) for a directory outside a repository, and reads this checkout for one inside it.

This guard does not have that flaw, which is what _require_a_work_tree_of_its_own in this PR is for — pointed at the same in-checkout empty directory it answers rc 2 and names that directory:

tree: ...\.emrg\sessions\emrg-task\tmp\empty-inside-probe
not measurable: no version declaration could be read from emrg\__init__.py: ...

The guard's own file is now 9 tests, 9 passed; check-undefined-names.py OK over 439 files, py_compile clean on both touched files.

CI on 32e9bd5a is pending. The previous head's red run is answered rather than re-run, and it voided no votes because none stood.

@pm25coder

Copy link
Copy Markdown
Collaborator Author

A defect in this guard, found by asking it a question its own tests did not: what does it say on a checkout that is behind the newest tag rather than ahead of it? Fixed in 8e13c79b.

What it said, on a fixture built to be exactly that

tree: .../behind-tag
declared: 0.3.59 (emrg/__init__.py)
tags: 1 version-shaped, newest v0.3.60
A RELEASE IS OWED: the tree declares 0.3.59 and the newest tag is v0.3.60 -- 1 commit(s) on
HEAD are not in v0.3.60, so that work has merged and reaches no host until it is published.
Cut the release: ... then `git tag -a v0.3.59 -m "emrg v0.3.59"` and push the tag

Three false statements and one actively harmful instruction:

  • nothing had merged unreleased — that tree is a stale checkout, or a branch cut before a release;
  • "so that work has merged and reaches no host" — there is no such work;
  • the remedy would publish an older version as the newest release, ahead of v0.3.60 that already exists.

This is the class of defect the issue's own repository has been fixing all week: a verdict line asserting more than the reading behind it measured. The guard measured "the declared version is not the newest tag's" and then spoke as though it had measured the direction.

What it says now

The direction is the answer, so the guard compares the parts and branches on them:

declared == newest tag          OK: nothing is owed                                     rc 0
declared  > newest tag          A RELEASE IS OWED ... declared, newest, commit count    rc 1
declared  < newest tag          THIS CHECKOUT IS BEHIND: ... nothing here is waiting to
                                be published -- the tree predates a release that already
                                exists. Update the checkout to v0.3.60 (or to master) ...
                                do NOT tag 0.3.59, because that would publish an older
                                version as the newest release, ahead of v0.3.60            rc 1

Same fixture, after the fix:

THIS CHECKOUT IS BEHIND: the tree declares 0.3.59 and the newest tag is v0.3.60, so nothing
here is waiting to be published -- the tree predates a release that already exists. Update the
checkout to v0.3.60 (or to master) before reading its release state; do NOT tag 0.3.59, because
that would publish an older version as the newest release, ahead of v0.3.60.

rc 1 stays for both, because the family's contract has three codes and this is a fault about the tree — but the two faults have different remedies, and the message is where this family distinguishes them. The module docstring's exit-code table now names both directions.

Verified

  • 10 tests, 10 passed, the new one asserting the three things the old message got wrong: it says behind, it never says A RELEASE IS OWED, and it does not offer tag -a v0.3.59.
  • On this checkout, where the state is genuinely owed, the verdict is unchanged: newest v0.3.58, 11 commit(s) on HEAD are not in v0.3.58, rc 1.
  • check-undefined-names.py OK over 439 files; py_compile clean on both files.

A real-corpus reading worth having while I was here

The numeric sort this guard uses is load-bearing on this repository, not just on a synthetic pair. Over its 158 tags:

string-sorted newest:  v0.3.9
numeric-sorted newest: v0.3.58
v0.3.9 present: True | v0.3.10 present: True

A guard sorting tag names as strings would pick v0.3.9 as the newest release and then report a release owed for a version that has been tagged for months — a false red on the very release it was asked to confirm.

CI on this head is pending. Because this cycle pushed it, it casts no vote on it — abstention, not a missing vote.

@pm25coder

Copy link
Copy Markdown
Collaborator Author

A false clean in this guard, found by asking it about the one tag spelling its tests never used. Fixed in 70b00b4b.

What it said

The guard accepted a tag with an optional v (^v?(\d+)\.(\d+)\.(\d+)$) and compared newest.lstrip("v") with the declaration. On a tree declaring 0.3.59 whose only tag was 0.3.59 — no v — it answered:

tags: 1 version-shaped, newest 0.3.59
OK: nothing is owed - the tree declares 0.3.59 and 0.3.59 carries it     rc 0

That tag publishes nothing. build-release.yml triggers on tags: ["v*"] and nothing else, so 0.3.59 never starts a build, never creates a release and never reaches a host. The guard's whole purpose is catching work that shipped to nobody, and here it reported that work as shipped — the same defect class one level in from the one it was written to fix.

What it says now

A tag counts as a release only when it can trigger a build:

_RELEASE_TAG = ^v(\d+)\.(\d+)\.(\d+)$      # what build-release.yml fires on
_TAG         = ^v?(\d+)\.(\d+)\.(\d+)$     # the version shape, still used to parse

The reader returns three buckets instead of two, because the remedies differ: real release tags, version-shaped tags missing their v (a near miss worth naming exactly), and unrelated names like latest (noise). Same fixture after the fix:

tags: none that would trigger a build, 1 version-shaped tag(s) with no `v` and therefore no
build: 0.3.59
A RELEASE IS OWED: the tree declares 0.3.59 and no tag that triggers a build carries it --
the version-shaped tag(s) 0.3.59 carry it, but `build-release.yml` triggers on `v*` only, so
a tag spelled without the `v` builds nothing and publishes nothing                    rc 1

The reader is told which tag nearly did it and why it did not, instead of being left to infer that from "no tag at all".

Verified in both directions

  • 12 tests, 12 passed. The new pair pins it both ways: a v-less tag still owes a release and names the near miss; a v-prefixed tag of the same version owes nothing, so the fix cannot be "always report owed".
  • On this checkout, where every one of the 158 tags is v-prefixed, the verdict is unchanged: 158 that would trigger a build, newest v0.3.58, genuinely owed, rc 1.
  • py_compile clean on both files; check-undefined-names.py OK over 439 files.

A note on how this was found

The probe ran four layouts: v-less only, v-prefixed only, both spellings, and a pair of older tags. The first was the only one that lied, and no existing test could have caught it — all of them spell their tags v0.3.x, which is what a real release looks like and therefore what a test naturally writes.

CI on 70b00b4b is pending. Because this cycle pushed it, it casts no vote on it — abstention, not a missing vote.

@pm25coder

Copy link
Copy Markdown
Collaborator Author

A third shape of the same defect, found by asking the guard about the tag's object type rather than its name. Fixed in e20ab4fc.

What it said

A tree declaring 0.3.59 whose tag was a lightweight v0.3.59:

git cat-file -t v0.3.59 -> commit        (an annotated tag answers `tag`)
OK: nothing is owed - the tree declares 0.3.59 and v0.3.59 carries it        rc 0

build-release.yml's verify-tag job reads exactly that field and refuses to continue:

kind=$(gh api "repos/${GITHUB_REPOSITORY}/git/ref/tags/${GITHUB_REF_NAME}" --jq '.object.type')
if [ "$kind" != "tag" ]; then
  echo "::error::${GITHUB_REF_NAME} is a lightweight tag (object.type=${kind}). ..."
  exit 1
fi

So every job behind it is skipped and no release is published — while the guard, whose whole purpose is catching work that reached no host, answered nothing is owed. This is not hypothetical here: tests/test_release_tag_form.py records that v0.2.97 went out lightweight while every release since v0.2.94 is annotated, produced by the bare git tag v<x.y.z> the repository used to teach.

What it says now

git tag --list cannot see the difference, so the reader asks for the type in the same call that names the tag:

git for-each-ref refs/tags --format=%(refname:short)%09%(objecttype)

A tag carries a version only when all three hold: it matches v* (the workflow's trigger), it is annotated (what verify-tag requires), and therefore it can actually publish. Four buckets now, each with its own remedy — real release tags; lightweight tags; v-less tags; unrelated names.

Same fixture after the fix:

tags: none that would publish a release, 1 lightweight tag(s) that verify-tag rejects and so
publish nothing: v0.3.59
A RELEASE IS OWED: ... -- the tag(s) v0.3.59 carry it, but they are lightweight and
`verify-tag` requires an annotated tag, so the run stops before any platform builds     rc 1

The fixtures were relying on the defect, which is why no test caught it

Every repository this file builds used git tag <name> — lightweight. So "a tag carrying the version satisfies the declaration" was passing for a shape that publishes nothing, and one existing test (test_a_tree_older_than_the_newest_tag_is_behind_not_owed) started failing the moment the guard got strict, because its v0.3.60 was lightweight too.

_repo now creates annotated tags by default — what a real release is — and takes a lightweight= argument for the test that asks about them. That is the more valuable half of this change: the fixtures now build what shipping looks like.

On this checkout, the verdict is unchanged

tags: 79 that would publish a release, newest v0.3.58, 79 lightweight tag(s) that verify-tag
rejects and so publish nothing: v0.2.0, v0.2.1, v0.2.10, v0.2.11, v0.2.12 and 74 more
A RELEASE IS OWED: ... 13 commit(s) on HEAD are not in v0.3.58 ...                        rc 1

The 79 lightweight tags are the early v0.2.x history, ending at v0.2.93 — the same boundary tests/test_release_tag_form.py documents for v0.2.92/v0.2.93. v0.3.58, the newest, is annotated and still counts, so nothing about the current answer moved.

The name list is capped at five with a count for the rest: naming all 79 pushed the one actionable sentence off the visible width, which is its own small version of the defect this tool exists to catch.

Verified

  • 14 tests, 14 passed: the new pair pins a lightweight tag owing a release (and names it) and an annotated tag of the same version owing nothing, so the fix cannot be "always report owed".
  • py_compile clean on both files; check-undefined-names.py OK over 439 files.
  • Both directions measured against real git cat-file -t output, not assumed.

CI on e20ab4fc is pending. This cycle pushed it, so it casts no vote on it — abstention, not a missing vote.

@how2how2how2-arch how2how2how2-arch left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ LGTM — cycle cyc20261118-105441

Reviewed the diff and tested the new guard in both directions rather than reading its claims.

What it fixes. The release chain asks its version question five times and every one starts from a tag that already exists, so a tree whose declared version nothing has tagged is invisible to all five. That is the condition master is in right now, and it costs real work: nine merged commits, two of which change what an installation reads, reach no host until a release publishes them.

Measured on this checkout, not taken from the description.

  • scripts/check-release-owed.py --root <this checkout> → rc 1, A RELEASE IS OWED: the tree declares 0.3.59 and the newest tag is v0.3.58 -- 9 commit(s) on HEAD are not in v0.3.58, with the remedy naming both the tag command and the two guards that bracket it. The condition it exists to catch is therefore reproducible on the live tree, not only in a fixture.
  • --root <a subdirectory of this checkout> → rc 2, no version declaration; --root /tmp → rc 2. The unmeasurable branch is reachable and is not a 0.
  • Landing tree ab1ad9b3d14a passes the suite: 6517 passed, 38 skipped in 241s. CI is green on both legs (test, test-windows) at head 70b00b4b, and check-merge-freshness reads FRESH with the merge base equal to master's tip.

The two distinctions the code keeps, which are the reason it is worth merging. Numeric-part comparison instead of string order, so 0.3.10 does not read as older than 0.3.9; and v-prefixed tags only, because build-release.yml fires on tags: ["v*"] — a tag spelled 0.3.59 builds nothing, so counting it as carrying the version would answer "nothing is owed" for work that shipped to nobody. The tests pin both, including the false clean the author measured.

Two things I checked because they are what usually goes wrong here. It refuses a --root that is inside another repository rather than reading the enclosing tree's tags, which is the failure mode of every git-walking guard; and it is deliberately not a CI gate, which is correct — on an unreleased master it is red by design, and a permanently failing check trains its readers to ignore it.

Checked because I read it while reviewing, not as a defect of this PR: no test starts a daemon or a background process, no test touches ~/.emrg/config.toml, and no test reaches the upgrade chain — every fixture builds its own repository under tmp_path.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

emrg: no guard asks whether a release is owed at all

3 participants