draft: proof of concept for plan-before-write validator-key recovery (#407, #572)

[Crackdevs]

Status: This is an unfunded feasibility prototype for a Round 40 grant application, not a production-complete or merge-ready implementation. It demonstrates the recovery-plan model, plan/commit separation, inactive-validator filtering fix, and zero-write safety test. Remaining work includes integration into recover, rebuild, and test-recovery; CLI/API behavior; production writer adapters; broader failure-path tests; documentation; and maintainer-requested revisions.

---

Description

This PR introduces the Safe Recovery Planner & Per-Key Diagnosis architecture for validator key recovery, prototyping a solution for #407 and #572; #440 provides operator-impact context.

Previously, Smartnode persisted validator keys to disk immediately as each individual key was discovered. In mixed-key setups (mnemonic-derived, custom EIP-2335 keystores, megapools), encountering an error on a later key left the operator in a partially-written, inconsistent state. Additionally, inactive/exited validators on the Beacon chain were filtered into filteredPubkeys, but the target map was populated from the unfiltered list, causing recovery to fail on exited validators (#407).

Key Architectural Changes

1. Two-Stage Plan-Before-Write Execution (shared/services/recovery):
   • Plan Phase (PlanValidatorRecovery): Pure, read-only discovery that assesses minipool/megapool keys, checks Beacon statuses, inspects custom keys, tests mnemonic derivation, and checks installed keystores. Zero disk writes.
   • Commit Phase (CommitPlan): Enforces the safety invariant: in default mode, if any in-scope key is unresolved or invalid, recovery halts before touching disk (ErrUnresolvedKeysRefusingCommit). An explicit --allow-partial-recover option allows committing only validated keys while recording skipped keys.
2. Deterministic Per-Key Diagnosis (types.ValidatorPlanEntry):
   • Explicit outcomes: mnemonic_valid, custom_valid, installed_valid, unresolved, invalid_material, and excluded_inactive.
3. Issue [Bug] wallet recover / rebuild should not write keys to disk until all keys have been recovered. #407 Bug Fix:
   • Fixed recoverNodeKeys in rocketpool/api/wallet/recover.go so pubkeyMap correctly iterates over filteredPubkeys.
4. Spy-Writer Test Suite:
   • Comprehensive tests including TestMixedRecoveryWithOneUnresolvedKey_DefaultFlowZeroWrites asserting through a writer-spy test that writerSpy.TotalWrites() == 0 on unresolved key failure under default mode.

Testing Done

• go test -v -count=1 ./shared/services/recovery/... (All 6 test suites passing)
• go test -v -count=1 ./rocketpool/api/wallet/... (All wallet tests passing)
• Passed golangci-lint/v2@v2.13.1 with 0 issues.

---

Note for maintainers: As this is a contribution from a new fork, the GitHub Actions workflows are awaiting approval to run (action_required). Could a maintainer please approve the workflow runs? Thank you!

[Crackdevs]

Could a maintainer please approve the GitHub Actions workflow runs for this PR? Thank you!