Skip to content

feat(role): judge voters by etcd's own vote and let a holder renew the regrade Lease - #210

Merged
Mearman merged 2 commits into
mainfrom
feat/voter-by-etcd-vote-and-lease-renew
Nov 3, 2026
Merged

Mearman merged 2 commits into
mainfrom
feat/voter-by-etcd-vote-and-lease-renew

Conversation

@Mearman

@Mearman Mearman commented Nov 3, 2026

Copy link
Copy Markdown
Member

Refs #183

PR E1 of the implementation plan in #183: a voter is judged by etcd's own vote, and the holder of the regrade Lease can renew it.

Lease. renew_lease HOLDER in fleet-regrade-core.sh replaces the Lease with a later renewTime and keeps the holder, acquireTime, duration and annotations. It fails and leaves the Lease alone when another holder has it, when it was released or is absent, when it cannot be read, or when it changed while being renewed (the replacement carries the resourceVersion it read).

Voters. scripts/fleet-role.jq reads a voter as a node with k3s's etcd label and the EtcdIsVoter=True condition. A node with the label and no condition is a member but not a voter: it is left out of the voters, so it also starts the membership cooldown when it is promoted, and it is a block ("member but not a voter"). The count the gate checks is $config.expected_voters, the target or the target plus one while a swap holds a new voter; scripts/fleet-role.sh passes the target, since this controller still changes no membership.

Decisions where the plan and the code differ:

  • renew_lease takes the holder as an argument and not NODE_NAME: the controller's holder is its own identity, which is no node. A holder whose Lease has run out may renew it if nobody took it since; the replacement's resourceVersion check is what keeps that safe.
  • fleet-role.sh is touched beyond the plan's file list: without expected_voters in its config the jq would read null and every pass would block on the count. The README's Role controller paragraph names the new voter rule.
  • The test fixtures give a voter node the condition (node(voter=True)) and a learner only the label (node(learner=True)); no existing test was edited.
  • The expected-count tests run fleet-role.jq directly, since nothing in the controller yet raises the count above the target; the swap that does is PR E2.

Tests (each seen failing against the old code first): renew keeps the holder, moves the renew time to now, keeps acquireTime and the duration, keeps annotations, extends past the earlier expiry, is refused for another holder, a released Lease, no Lease, a conflicting write and an unreadable Lease; a learner by label is not a voter and blocks; a node whose condition is False is not a voter; the expected count is the target, or the target plus one while a swap holds a new voter, and a count short of it blocks. Each rule was broken once in the code to see its tests fail.

The core acquires the Lease once for a step sized at a few operation timeouts, so a holder that
works for longer, such as the role controller, had no way to keep it.

renew_lease takes the holder's identity, writes a later renew time and keeps everything else
(acquire time, duration, annotations). It fails, leaving the Lease as it was, when another holder
has it, when it was released or is absent, or when it changed while being renewed.
k3s labels a server as an etcd node when it joins as a learner and never removes the label, so a
learner counted as a voter and a removed member still did.

A voter is now a node with the etcd label and the EtcdIsVoter condition True. A labelled node
without the condition is reported as a block ("member but not a voter") and is no longer among the
voters, so it also starts the membership cooldown when it is promoted.

The decision takes the voter count to expect as config.expected_voters, the target or one more
while a swap holds a new voter. A pass of the controller expects the target.
@Mearman
Mearman marked this pull request as ready for review November 3, 2026 22:10
@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
🔒 Security Review ⚠️ Failed 2026-11-03T22:10:33.714356Z
We couldn’t complete this review. Comment @codex security review to try again.
b14cfcd Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector

Copy link
Copy Markdown

This security review couldn’t start because the paying account has exhausted its Codex allowance and has insufficient available credits. The payer can check their balance and add credits in the Codex usage dashboard, then try again. If you do not manage the paying account, contact this repository's admins.

@Mearman
Mearman merged commit 4d1d640 into main Nov 3, 2026
28 checks passed
@Mearman
Mearman deleted the feat/voter-by-etcd-vote-and-lease-renew branch November 3, 2026 22:10
@github-actions

github-actions Bot commented Nov 3, 2026

Copy link
Copy Markdown

🎉 This PR is included in version 2.33.0 🎉

The release is available on:

Your semantic-release bot 📦🚀

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant