Skip to content

Release 4.2.0: node_admin plugin, validator signing-key fix, HF16 no-contest fee freeze - #181

Open
On1x wants to merge 104 commits into
masterfrom
pm
Open

On1x wants to merge 104 commits into
masterfrom
pm

Conversation

@On1x

@On1x On1x commented Oct 19, 2026 •

Copy link
Copy Markdown
Member

Summary

This branch adds an operator-facing admin surface to the node (node_admin plugin), fixes a
consensus-adjacent block production bug, and adds HF16 (freeze the no-contest fee at
announcement). The admin surface is optional and inert unless configured.

node_admin plugin

A separate HTTP listener (loopback by default, bearer tokens, append-only audit) exposing 31
admin.* JSON-RPC methods plus:

  • Streamable HTTP MCP on the same listener and tokens (initialize, tools/list,
    tools/call), one tool per admin method, for AI agents.
  • Built-in web UI at /ui/ — single static page, no external assets; the token stays in
    sessionStorage. Works behind a reverse-proxy prefix (/admin/ui/) after this branch.
  • P2P control: status (head/LIB, peer lifecycle, lag, bans), add_node, add_trusted,
    clear_banned; actions are serialized onto the P2P thread.
  • Snapshots: status, create, in-process restore, authenticated download.
  • Webserver policy: per-IP counters, fixed-window rate limits (minute/hour/day), whitelist
    and blacklist, hot-reloadable and persisted next to the data dir.
  • config.ini / plugin list edits through two-phase plan/apply with a SHA-256 guard and
    atomic replace, plus admin.restart for a supervised restart.
  • Validator operator actions: enable / disable / set signing key for validators, signed by
    validator_guard (no extra keys stored), with a persisted hold file so a disabled operator
    validator is not silently re-enabled by a restart.
  • Diagnostics: node info, plugin state, log tail with an allow-list, audit tail.

Scopes are per token: read (every method), p2p, snapshot, webserver, config,
plugins, chain, process. A token that has only a mutating scope can call nothing without
read; a missing scope is a separate error from a failed action. A non-loopback endpoint is
refused unless admin-allow-remote = true.

Bug fixed: a block producer signed with its own pending key

validate_block_header() checks the block signature against the validator's signing key from
pre-block (committed) state, but maybe_validate_block() read the session-visible
validator object and _generate_block() asserted against that same object. A pending
validator_update is already applied in the pending session, so rotating the producing
validator's own key made the node sign block N with the new key while the header was validated
against the old one: validate_signee failed, the pending pool was cleared and the update was
lost.

Fix:

  • database.cpp::_generate_block checks the key after _pending_tx_session.reset() and
    captures the committed key under the same write lock (otherwise a concurrent
    push_transaction() can re-apply the pending change before the read).
  • validator.cpp keeps a committed-key snapshot (_committed_signing_keys), seeded at startup
    and refreshed from the contents of applied blocks — applied_block fires after
    push_block() has restored the pending transactions, so the database there is not a
    committed-state source.

Regression: new section 10 in validator-e2e.sh rotates the producing validator's key and
back; restart/disable/enable paths keep their coverage.

HF16 — no-contest fee frozen at announcement (CHAIN_VERSION 4.2.0)

pm_no_contest opened a 12-hour dispute window but computed the penalty from the median
chain_properties_pm at settlement, so a parameter vote inside the window repriced an
announced void retroactively. HF16 records no_contest_fee on the market at announcement
(with a separate no_contest_fee_frozen flag: pre-fork rows keep the legacy median path, a
legitimate frozen zero does not fall back), charges exactly that at settlement, and adds a
monotone no_contest_slashed_total to the oracle in the same modify() as the existing
counter. Snapshot import is guarded with contains() for the new fields.

Activation timestamps are placeholders in this branch; the mainnet date is not announced.

Testing

  • plugins/node_admin/tests/acceptance.sh — one-shot node, full HTTP/MCP/UI/auth surface.
  • plugins/node_admin/tests/validator-e2e.sh — validator guard operator paths on a private
    single-slot chain, including the self-rotation regression (62 checks, live run green).
  • tests/node_admin/ — Boost.Test unit tests for the pure helpers (13 cases).
  • tests/consensus_sim/scenarios/test_pm_no_contest_fee.cpp — three markets around the
    activation block (legacy / frozen nonzero / frozen zero), median moved afterwards.
  • Local build + the two shell suites were run against this branch; CI builds the production
    image from this PR. The plugin includes fc/compat/asio_compat.hpp before its websocketpp
    headers: Boost >= 1.87 dropped io_service and the vendored websocketpp still spells it.

Upgrade notes

  • node_admin is inert unless admin-endpoint and at least one admin-token are configured;
    the plugin refuses to start with an endpoint and no token.
  • Config edits made through admin.config.apply are read at startup and need admin.restart.
  • HF16 adds fields to pm_market_object / pm_oracle_object, so the shared-memory layout
    changes: redeploy testnet from a snapshot (the node already runs with
    --replay-from-snapshot --snapshot-auto-latest).

On1x added 27 commits October 16, 2026 04:09
docker.md and validator-node.md already recommend it; getting-started
(en/ru/zh-CN) and README did not, so a node started from them stays
down after a host reboot.
pm_no_contest today opens a 12-hour dispute window and leaves the penalty to be
computed from the median chain_properties_pm of the settlement moment, so a
parameter vote inside the window silently repriced an announced void. The fork
records the fee on the market at announcement and charges exactly that.

Consensus:
* pm_market_object gains no_contest_fee + no_contest_fee_frozen. The flag is
  separate from the amount on purpose: a row announced before the fork has
  frozen == false and keeps the legacy median computation (no seed pass), while
  a legitimate frozen zero (penalty percent voted to 0) must not fall back to a
  later vote.
* pm_no_contest_evaluator freezes max(0, dispute_fee * pct / 10000) at
  announcement; settle_market(win < 0) charges the frozen value under HF16.
* pm_oracle_object gains no_contest_slashed_total, grown in the same modify()
  as total_insurance_slashed, for exact attribution without a virtual op.
* Registration in hardfork.d/16.hf + database_hardfork.cpp, CHAIN_VERSION 4.2.0
  in both configs, snapshot import guarded by contains() for the new fields.
  Activation timestamps are placeholders pending the owner's mainnet date.

API (no consensus):
* get_oracle / list_oracles: no_contest_pending_count / _stake / _fee_nominal /
  _fee_capped over by_oracle_finalized (open obligations only, per-block cache).
* market_card: settlement_charge — the price of a void before it is charged.
* get_pm_chain_properties: derived pm_no_contest_fee_per_market, additive.

Proof: tests/consensus_sim/scenarios/test_pm_no_contest_fee.cpp drives three
markets around the activation block (legacy, frozen 1u, frozen 0), moves the
median afterwards, and measures the insurance drop and the counter: 9u total
(8 + 1 + 0). pm_lifecycle_suite: 72 tests, no errors.

Docs: operator checklist in English and Russian, API reference updated.
…ly v1)

An administrative JSON-RPC endpoint inside the node process, for operator tooling
and AI agents. It opens its own HTTP listener rather than riding on webserver:
that one is published through nginx, so every request already arrives as
127.0.0.1 and a source-address check there protects nothing.

- Authentication: `label:scope[,scope]:sha256hex[:expires-unixtime]`, only the
  digest is stored, comparison is constant-time, and a token carrying no `read`
  scope gets -32002.
- Guards: loopback-only unless `admin-allow-remote`, no listener without tokens,
  POST only, 64 KB body cap, 20 req/s per token, log tails restricted to an
  explicit `admin-log-file` allow-list.
- Read-only methods: ping, node_info, diagnostics, plugins.list, config.get,
  logs.tail, audit.tail. Numbers are emitted as JSON numbers, not fc's default
  quoted form.
- Every outcome, including failed authentication and rate limiting, appends one
  JSON audit line with a label, never the token or its hash.
- p2p: add get_connections_count_sync() — the peer map belongs to the p2p thread,
  so diagnostics reads it through fc::thread::async().wait().
- Docs in English and Russian; live acceptance in
  plugins/node_admin/tests/acceptance.sh (35 checks against a throwaway node).

Mutating methods, plan/apply and restart are the next stage.
Add the Model Context Protocol (Streamable HTTP) to the same loopback
listener, with the same bearer tokens and audit trail (q#3052=B: no
sidecar). Methods: initialize, ping, tools/list, tools/call;
notifications are answered with 202 and an empty body. Tools map 1:1 to
the read-only admin.* methods and report execution errors via isError,
while protocol errors stay JSON-RPC errors. Sessionless, no SSE.

Extend acceptance with 16 MCP checks (51 total, all passing).
Expose the P2P picture the node already logs (head/LIB, DLT range, lag behind
the best peer, uptime, per-peer lifecycle and ban state) as admin.p2p.status.
Add the three operator actions the owner asked for: admin.p2p.add_node,
admin.p2p.add_trusted (never soft-banned, kept through known-peer GC) and
admin.p2p.clear_banned. Mutating methods need a token with the p2p scope on
top of read; all three are serialized onto the P2P thread.

Serve a minimal static page at GET /ui/ from the same listener: token in
sessionStorage, status cards, peers table, add/trusted/clear actions, five
second refresh. The page is unauthenticated by design and carries no data;
CSP restricts it to inline assets and same-origin fetch.

Expose the same four methods as MCP tools (11 total) and cover everything in
the acceptance suite (80 checks, including scope denial and the UI route).
Add plan/apply editing of config.ini behind the admin API:

- config.plan validates allow-listed keys (p2p, dlt, webserver, validator-guard),
  normalizes booleans and range-checks integers, and returns the exact diff
  without writing anything. plugins.plan validates the plugin list and refuses
  unknown names, duplicates and removal of chain/node_admin.
- Plans are single-use, bound to the token that created them, expire after
  admin-plan-ttl seconds, and are capped at 64 outstanding entries.
  plan.cancel discards one explicitly.
- apply recomputes the file hash and refuses when config.ini changed since the
  plan, then rewrites raw lines (comments, order, spacing and CRLF preserved)
  through a temp file, fsync and rename, after writing a .bak-<unixtime>
  backup. A config_write_mutex serializes concurrent applies.
- config.backups lists the backups under the plain read scope.
- admin.restart (process scope) asks appbase to quit after the reply flushes;
  the supervisor is what brings the node back, and the reply says so.

Acceptance covers scope and token binding, allow-list, expiry, stale hash,
cancel, replay, backups, plugin protection, a real restart and a
supervisor-style start that reads the edited config back. Unit tests cover the
line editors, normalization and plugin guards.
Add admin.validator.list / plan / apply (scope "chain") and the matching MCP
tools. Reads stay under "read"; plan and apply require a token with "chain"
AND a started validator_guard configured with active keys, so the scope alone
never authorises a signature.

validator_guard gains chain_actions_available(), validators_status() and
apply_validator_actions(). Actions are built and signed inside a weak read
lock, then accepted and broadcast after the lock is released (the same rule
disable_on_shutdown follows). Plans carry the observed signing key; apply
re-checks it and reports "chain state changed" instead of signing a stale
intent.

A successful disable writes an operator hold to
<data-dir>/validator-guard-holds.txt. The periodic auto-restore skips held
validators and startup reads the file back, so a redeploy cannot silently
re-enable block production. The hold follows the outcome, never the intent:
a failed disable leaves no latch, and enable/set_key clear it.

Docs (EN and RU) describe the methods, the plan/apply contract, the hold
semantics and the test coverage; acceptance.sh now covers the gate (guard
missing, guard loaded without an active-key validator) and the tool count.
The signing happy path was verified on a scratch chain with the validator
plugin and cli_wallet.
Covers admin.validator.list/plan/apply on a scratch testnet chain:
disable/enable/set_key, hold persistence across restart, stale plan
rejection, a two-action plan (one tx per action), and contract corners.

Staging notes in the script: register/vest/vote the producing validator
before HF9, run operator actions on a non-scheduled validator, and never
rotate the producing validator's own key (the producer reads the signing
key from session state, so the update is lost - tracked separately).

Docs (EN/RU) name the script and both load-bearing decisions.
A block header is validated against the validator signing key committed at the
previous block, but the producer read the key from the session-visible validator
object, where a pending validator_update is already applied. Rotating the
producing validator's own key made it sign the block with the new key:
validate_signee failed, the pending pool was cleared and the update was lost.

- database: check the signing key after _pending_tx_session.reset(), reading the
  committed object under the same write lock (a concurrent push_transaction after
  the lock is released would make the object session-visible again).
- validator plugin: keep a committed-key snapshot. Seed it at startup and refresh
  it from applied block contents - an applied_block callback runs after
  push_block() has re-applied the pending pool, so a DB read there sees
  uncommitted state. effective_signing_key() returns the committed key when the
  session-visible one differs and the snapshot is for the current head.
- validator-e2e: new section 10 rotates the producing validator's own key and
  rotates it back; the chain keeps producing and the key changes on chain
  (62 passed, 0 failed).
The paragraph still described the pre-fix behaviour (the producer read the
session-visible key, so rotating the producing validator's key lost the
update) and even pinned a source line number. That premise no longer holds:
the producer signs with the key committed at head, and section 10 covers
self-rotation. Reword to what cold is actually for and drop the line ref.
The page always called '/' for JSON-RPC and '/snapshot/…' for downloads, so it only
worked when the listener was reached at its root. Operators also publish it through
nginx under a path (e.g. /admin/ui/), where those absolute URLs leave the admin
listener entirely.

Derive the base from location.pathname (strip a trailing '/ui' or '/ui/'), exactly as
public API clients do for their own base. At the root the value is '/', so direct
access is unchanged. Documented in both locales.
Boost >= 1.87 dropped boost::asio::io_service, which the vendored
websocketpp 0.8.2 still spells ~100 times. webserver_plugin.cpp already
includes fc/compat/asio_compat.hpp for this; node_admin did not, so the
production image (Boost 1.91) failed to build the plugin while a local
build against an older Boost succeeded.
ctest found eight tests but the workflow built only the targets listed in
tests/pm and tests/consensus_sim, so tests/node_admin registered its
node_admin_util test and ctest reported it as "Not Run" (exit 8) while
every binary that was built passed. Take the list from every
tests/*/CMakeLists.txt under tests/.
…ve logs

The built-in /ui/ page could not tell a read-only token from an admin one:
Create snapshot, Restore latest and the peer actions were always shown, the
API refused them only after the click. It also knew nothing about emergency
consensus and could not read the node's own log.

Backend:
- admin.auth.me returns the token's label and scopes (never the token/hash).
- admin.snapshot.status takes offset/limit (default 20, max 200) and reports
  snapshot_total and has_more.
- admin.logs.list names the allow-listed appenders with size and existence.
- admin.validators.active returns the scheduled validator names, the schedule
  size and the emergency consensus state.
- admin.node_info gains emergency_consensus_active/start_block,
  emergency_key_configured and recovering.

UI:
- Status / Peers / Snapshots / Logs / Validators tabs.
- actions are hidden when the token lacks the scope they need.
- snapshots: newest 20 with Load more; the download button disables itself and
  shows a radial spinner, so repeated clicks cannot start parallel downloads.
- logs: tail an allow-listed file, paused by default, Play polls every 2s and
  appends at the bottom; auto-scroll yields when the operator scrolls up.
- status shows the emergency consensus state and whether an emergency key is
  configured; snapshot state says what idle/creating/restoring means.

Acceptance: auth.me, logs.list, validators.active, paging fields and the
emergency fields are now checked; tool count 37.
…the tabbed UI

Describe admin.auth.me, admin.logs.list, admin.validators.active and the new
offset/limit fields of admin.snapshot.status. Rewrite the built-in UI section
around the five tabs, scope-driven action hiding, Load more paging, the
disabled download button with its radial spinner, and the paused-by-default
live log tail. Both the English and the Russian document, same content.
…fset

A snapshot written between two pages shifted every offset below it, so the UI
repeated a row on Load more; the 24 h retention sweep shifts the other way and
made offsets skip a row. admin.snapshot.status now takes after=<last name the
client has> and continues after that exact row; a pruned anchor is reported as
after_found=false and offset applies. The UI anchors to the last shown row,
dedupes by name and restarts from page 1 when its anchor was pruned.
The stock file appender rotates hourly and keeps a day, so debug logging can
grow to gigabytes per hour. Add an opt-in capped_file appender that keeps one
file with only the last max_lines lines (default 10000, 0 = unlimited).

- trim walks the file backwards in 64 KiB chunks, so a startup on an oversized
  log costs a bounded tail read rather than a full scan; the rewrite goes to a
  temp file and is renamed into place
- compaction is amortised: it runs only after the file passes max_lines plus
  10% slack, and the appender counts lines as it writes, so no per-line stat
- nodes without a log.capped_file_appender.* section are unaffected; the
  appender is registered with its own type name and stays unknown to older
  binaries
- node_admin already reads any path, so admin.logs.list/tail pick the file up
  with no API change

Tests: tests/logging (8 cases) cover the tail contract and live compaction.
The file appender rotates hourly and keeps a day, which is not a size cap: at
debug level that is still gigabytes per hour. Say so, and point operators at
the optional capped_file appender for a hard line limit.
The file appender is configured to rotate hourly with one-day retention, but the
rotation chain runs on fc's fiber queue and vizd's main thread lives in
boost::asio after startup, so it never runs. Measured 2026-10-20: a testnet
node wrote one 37 MB file in nine hours at info, with no hourly files; the
mainnet node ran four days without a new one.

Correct the logging docs in en/ru/zh and the node-admin page: the file is
bounded by neither time nor size, and capped_file is the only real cap.
…he file

The line counter ran after a failed trim as well, i.e. exactly when the log is
largest (a full disk is a plausible reason the trim itself failed). Reading the
whole file into an ostringstream there would allocate a second copy of a log that
may be gigabytes — the appender would OOM while reporting that it cannot cap the
file. Count in 64 KiB chunks instead; a missing file still counts as zero lines.

Tests: three new cases (25k lines, missing file, a line spanning chunk borders).
Measured on real images: an older binary (4.1.2, no capped appender) starts fine with a
config that contains the new section, ignores it and keeps logging to stderr — so the
section can be deployed before the upgrade. A new node whose path cannot be opened also
starts, prints one 'cannot open log file' line to stderr, and leaves no file.
tests.yml now discovers every binary named by add_test, so it builds all
six pm_* targets (plus node_admin_util and capped_file_appender) with the
same configure flags and runs the whole ctest — a superset of the
`ctest -R '^pm_'` performed here. The only PRs it skips are docs-only
ones, where there is nothing to test.

So the `pm-tests` label no longer adds coverage; the gaps that remain are
`check-boost-range` (the only job building `vizd` itself on Boost 1.83)
and `testnet-config` (the only job running consensus_sim with testnet
constants, where the hardfork registry is validated). A skipped gate is
not a passing one: a PR with zero labels shows these as skipped next to
green checks.
@On1x On1x added check-boost-range Also run the distro Boost 1.83 range-floor build testnet-config Compile the testnet config and run consensus_sim (fork-gated PM regressions) labels Oct 20, 2026
Owner decision (Q#3081=A): the plugins panel shows the existing allow-list
grouped by plugin, plus log settings. Two gaps were in the way.

Sectioned keys. The config editor only touched top-level options, so
logger.* and log.* could not be edited at all. config_spec now names its
ini section, and plan/apply address the line through
set_config_value_in_section(). A missing section is an error rather than an
insert: inventing [logger.foo] would configure a logger the node never
declares and the next start would ignore it silently. CRLF and the spacing
around '=' are preserved exactly as in the top-level editor.

Logging keys. logger.default.level and logger.p2p.level accept
all/debug/info/warn/error/off; log.capped_file_appender.node.max_lines and
.flush bound the file that can fill the disk. Appender filenames stay
operator-only.

API for the UI. admin.config.get gains 'allow' -- key, group, section, type,
current value (null when the file relies on the code default), bounds and
allowed values -- so the panel renders without a second source of truth.
admin.plugins.list gains 'removable' (chain and node_admin render as
non-toggleable rather than merely off) and 'params' per plugin.

Verified by -fsyntax-only on both translation units; tests for the new
editor and normalizer are in node_admin_util_test.cpp.
On1x added 30 commits October 21, 2026 16:52
…rval()

All t / CHAIN_BLOCK_INTERVAL round sites in database.cpp and the
validator plugin now go through round_of(), so rounds stay contiguous
across the 3 s -> 1 s switch. Missed-slot gap and lag-skip use the
current slot length.
update_bandwidth_reserve_candidates and committee request processing
scale their block modulus by 3 s / block_interval(); the commit-reveal
deadline converts blocks to seconds with block_interval().
…rval()

max_virtual_bandwidth scales with blocks per window (3x at 1 s for the
same maximum_block_size); the guard's sync check allows two current
slots instead of two 3 s slots.
database_api overrides the compile-time value with block_interval()
(under the existing weak read lock) and adds CHAIN_BLOCK_INTERVAL_HF16.
e66e571 moved the node to round_of() while the no-contest scenario
still voted with t / CHAIN_BLOCK_INTERVAL, so the first post-HF16 QC
carried a 3 s round and the next block failed 'Parent QC round is not
the parent's slot'. The double-sign invariant keyed slots the same
way and would merge three 1 s slots into one.
Q#3108=A. inflation_recalc_period, distribution_epoch_length,
pm_batch_epoch_blocks and pm_reveal_window_blocks are multiplied by 3
in every validator_object.props and in the schedule median, so their
wall-time meaning survives the 3 s -> 1 s switch. Scaling the median
alone would be undone by the next schedule recompute.
Q#3109=A. dgpo.round_anchor_time/round_anchor_round: round(t) =
anchor_round + (t - anchor_time) / block_interval(). The HF16 block
re-anchors at its actual time with its pre-switch round; any later
hardfork that changes the slot length re-anchors the same way, so
rounds stored in QC/TC are never renumbered.
…fork

Protocol validate() now allows inflation_recalc_period and
distribution_epoch_length up to 3 * CHAIN_BLOCKS_PER_YEAR (one year of
1 s blocks, and what the HF16 x3 scaling can produce). The versioned
props evaluator enforces the previous one-year-of-3-s-blocks bound on
the resulting props until HF16, so pre-fork consensus is unchanged.
Adds vizd:<tag>-testnet21 (Dockerfile-testnet, TESTNET_VALIDATORS=21)
for the HF16 QC tc-netem bench. Gated on head_ref == pm.
check_fork_alignment() accepted empty peers but not the reverse: a node
started from genesis (no snapshot) found no known block in any peer's
hello, so exchange stayed disabled, blocks #1-#2 were never requested
and every later block was deferred as unlinkable. On the 21-validator
QC stand only the 6 nodes connected while #1 was produced got a chain.

Now head == 0 plus a peer whose DLT range starts at #0/#1 is aligned;
blocks are still fully validated from genesis on apply, and pruned DLT
peers still require a snapshot.
read_block_by_num() and block_exists_in_log_or_fork_db() looked only in
the DLT block log and fork_db. A node without DLT mode (chain started
from genesis, full block_log) advertises dlt_earliest_block = 0, so
peers ask it for #1.., but blocks at or below LIB have already left
fork_db: range replies started at the first reversible block, block #1
was never served and new peers deferred everything as unlinkable.
Fall back to block_log when not in DLT mode.
A node that joined at head #1 anchored #1 without data, then dropped the real
#1 from peers as already known; the LIB writer found no data for #1 and
rejected every #2 (21-validator QC stand).
The activation block is the last 3 s slot and nothing below it is certified.
If its QC does not form (votes lost to a network-wide restart, a hiccup at the
switch) a TC cannot skip it either, since a TC needs the head's parent
certified, so the chain stalls for good. Exempt its child, as the first block
after an emergency already is. Found on the 21-validator stand.
On the 21-validator stand the first post-activation block never gets a QC and
the cause is invisible: refusals were dlog only. Log at warn, once per head:
a validator with no vote candidate for head, a parent-QC refusal, and a vote
for head rejected by push_block_vote (LIB, signature/schedule).
Right after activation the anchor's child carries no parent QC by design. If
its round times out first (validators send timeout votes before the block
reaches them), nobody votes for it, and the TC path never fires: it requires a
certified parent. Drop such a head and rebuild on the anchor without a TC; the
replacement is exempt from the parent-QC vote rule and sits in a later round.
Found on the 21-validator stand (TC 15/15 every slot, head stuck at #67).
Timeout votes exist precisely while head is stuck. With the 64-round bound
measured from head's round, every timeout vote is refused about 64 s into a
stall: no TC can form any more and the chain never recovers, even after the
network heals. Found on the 21-validator stand (netem 300 ms: last TC at
head round +63, then all 21 nodes stuck at steemit#378 after delay was cleared).
The Jolteon consecutive-round check used the compile-time CHAIN_BLOCK_INTERVAL.
After HF16 the slot is 1 s, so the check never matched, QC-LIB was always 0
and LIB fell back to post-validation. Found on the 21-validator stand: LIB lag
was exactly 14 blocks in every run instead of ~2.
…ndidate

fork_db keeps the first block at a height as its head, so a TC replacement of
a QC-less head never reached the certificate comparison: it was stored and not
applied, every next producer built one more sibling and the chain stalled for
good (21-validator stand, netem 300 ms: five blocks steemit#194 on one parent).
Offer the pushed block as the candidate when its certificate round is higher;
a forged certificate fails in switch_forks and is reverted.
A late-QC head carries its grandparent's QC because its parent's never formed.
If it also misses its own QC, the TC skip refused it (QC of a non-parent) and
the chain stalled for good (21-validator stand, netem 300 ms: head steemit#184 with
QC of steemit#182). Drop the head and its uncertified parent and extend the certified
block, as Jolteon prescribes; fork choice now also accepts a candidate below
head when it proves a later round. Skip diagnostics are keyed by head and
reason, so the first refusal no longer hides the real one.
high_qc_round of an uncertified head came from block_cert_round(), which also
counts a TC. After a TC rebuild, every timeout vote claimed a QC at the TC
round that never existed, so the TC skip of that head was refused forever
(21-validator stand, netem 300 ms: head steemit#183 with TC 691, all timeouts
reporting high QC 691). Report the round of the QC extension only.
A QC-less head whose carried QC lags it by g rounds waits 2^floor(log2 g)
rounds (1, 2, 4, capped at 8) before validators give up on it. g is read
from the head, so all validators on it agree on the wait and their
timeouts meet in one round. A certified head keeps the one-round wait.

On the 21-validator stand at 300 ms per hop every rebuilt head was given
up on before its votes arrived: #179 was rebuilt 444 times with a TC
every round and the height never moved.
…d_blocks

assert() is compiled out in release, so get_slot_at_time() == 0 made
missed_blocks-- wrap to 2^32-1 and the miss loop ran for hours. Seen on the
21-validator stand: replay of post-HF16 1 s blocks happens before
testnet_plugin forces HF16, so the 3 s interval puts block time before the
first slot. No accepted block can reach this branch (it would have hung
the node), so history is unaffected.
After HF16 competing blocks are settled by QC/TC fork choice and a double
producer is punished by equivocation evidence. The legacy deferral let one
double-signing validator halt the chain: its twin at head+1 made every
producer defer, the counter grows only on a node's own slot, so the
timeout took 21 own slots (~7 min at 1 s) and gap fill then brought the
twin back. Seen on the 21-validator stand with qc-4 doubling viz2+viz3:
head stuck at #81.
…unds

A livelock rebuilds one height over and over; keying the once-only logs
by height showed only the first variant (stand: #86 rebuilt 70 times,
every later refusal silent). Refusals for an older round were not logged
at all, which is the first suspect for the #86 stall. Now each refusal
above LIB+1 is logged once per block id with the last round and whether
it was voted or timed out; the no-candidate line skips the producer's
own block.
_block_qcs held one QC per height and never replaced it. After a TC the
height is rebuilt in a later round, so once any variant of a height was
certified, no rebuild of it could be: votes were cast (15/15) but the QC
was never formed. Seen on the 21-validator stand with a double signer:
QC for one twin of steemit#546 in round 649, fork choice took another twin, then
40+ rebuilds of steemit#546 without a QC; chain stuck for 20+ minutes.

A later round at the same height is exactly what Jolteon expects after a
timeout, so the higher round replaces the held QC.
The extension path still kept the first QC seen for a height, while the
tally path (bd85d8a) already prefers the higher round. A QC for an
equivocating twin arriving first via a block extension could still pin
the height.
CHAIN_MAX_UNDO_HISTORY is counted in blocks; at 1 s slots 10000 blocks is
2.8 h of LIB stall instead of 8.3 h. After HF16 the cap is 3x so the
tolerance stays the same in time (Q#3108=A).
Pins the switch at the fork block: block_interval 3 -> 1, the fork block
keeps its pre-fork round, every later 1 s block advances the round by one
with no missed slot, pre-anchor times keep the 3 s mapping, and the
block-denominated median (inflation_recalc_period) stays x3 after the
schedule recomputes it.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

check-boost-range Also run the distro Boost 1.83 range-floor build testnet-config Compile the testnet config and run consensus_sim (fork-gated PM regressions)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant