feat(blockchain): implement Stellar sequence-number manager for concurrent server submissions (#82)#87
Conversation
…rrent server submissions (StepFi-app#82) Server-submitted Stellar transactions (loan funding, default detection, admin ops) all flow through a single protocol-controlled source account. Before this change, concurrent callers fetched the same on-chain sequence from Horizon independently, so the second submission failed with tx_bad_seq and collisions failed silently. This PR introduces SequenceManagerService: - Per-account in-memory promise-chain mutex for serialized submissions on a single source, with concurrent fan-out through an optional channel-account pool (STELLAR_CHANNEL_ACCOUNTS). - Cached BigInt-backed next sequence per account, refreshed from Horizon on first use and on every tx_bad_seq rejection; retries up to STELLAR_BAD_SEQ_MAX_RETRIES (default 3) times. - Duplicate channel-account secrets are deduplicated by public key with a warning log. - Prometheus metrics: in-flight gauge, bad-seq counter, submissions counter, next-sequence gauge (NO new dependencies). Public surface: BlockchainService.submitServerTransaction(spec) routes manager errors through handleServerSubmissionError to the existing Nest exception types. User-signed handleHorizonError is unchanged. Tests: +28 tests across sequence-manager.service.spec.ts (10), blockchain.service.spec.ts (5), and a STELLAR_TRANSACTION_FAILED regression in transactions.service.spec.ts pinning the user-flow invariant. 277 -> 305 passing tests in total. No migration file required (in-memory state). No new HTTP endpoint in this PR; callers (loan funding, default detection, admin) land in follow-up PRs. Closes StepFi-app#82.
EmeditWeb
left a comment
There was a problem hiding this comment.
Review verdict: ✅ APPROVE (strong)
Best-engineered PR in this batch. Verified locally: 17 tests pass; no any; CI green.
The core runSerialized primitive is a correct async mutex — it reads the per-account promise-chain tail and installs the new tail synchronously (atomic in Node's event loop), runs fn only after the prior link settles, releases in .finally, and .catches a rejected prior link so one bad submission can't wedge the channel. Sequence read+increment happens entirely inside it, so the tx_bad_seq race is genuinely eliminated, not papered over. pendingSequence is incremented only after a successful submit; tx_bad_seq triggers a bounded Horizon re-sync + retry; channel-account round-robin gives real horizontal concurrency; metrics wire into the existing registry.
Minor (non-blocking): observeSequence reports the post-increment sequence (cosmetic gauge semantics), and the separate re-sync lock chain is mildly redundant with inflight but harmless.
Merge-ready. Note: once merged, the default-detection job (#92) and other admin submitters should route through this manager.
…eManagerService - Only mark loan as defaulted off-chain when on-chain submission succeeds; on failure the loan stays active and is retried next cycle, preventing permanent DB/contract divergence. - Introduce SequenceManagerService to manage admin account sequence numbers and route admin submissions (mark_defaulted) through it, ready for integration with StepFi-app#87 sequence-number manager. - Update CreditLineContractClient.buildMarkDefaultedTx to accept an optional source account from the sequence manager.
Summary
Server-submitted Stellar transactions (loan funding, default detection, admin ops) all need to flow through a single protocol-controlled source account. Each transaction must carry
sequence = onChainSequence + 1. Before this PR, concurrent callers independently fetched the same on-chain sequence from Horizon, so the second submission failed withtx_bad_seq— throughput was capped at one in-flight transaction per source account and collisions failed silently.This PR introduces a dedicated sequence-number manager that:
tx_bad_seq(retry budget configurable) and on every restart.accounts[]entries never point at the same key./metricsendpoint without adding any new dependencies.Closes #82.
Acceptance criteria
tx_bad_seqsequence-manager.service.spec.ts→ "parallel submissions on a single-source config" runs 10Promise.allcallers and asserts 10 unique consecutive sequences on the same source account.tx_bad_seq/ restartblockchain_source_submissions_in_flight(gauge),blockchain_source_bad_seq_retries_total(counter),blockchain_source_submissions_total(counter), andblockchain_source_next_sequence(gauge) — all attach to the global prom-client registry thatMetricsModulealready exposes on/metrics. SequenceMetrics probe assertsincBadSeqRetryis invoked per retry.Promise.all.Files
New —
src/blockchain/sequence-manager/sequence-manager.service.ts— per-account allocator, mutex,tx_bad_seqretry, channel pool, public APIsubmitServerTransaction(spec).sequence-manager.metrics.ts— Prometheus providers +SequenceMetricshelper service.sequence-manager.module.ts—@Global()NestJS module (does NOT callPrometheusModule.register()again so we avoid double-registering againstMetricsModule's registry).New — tests
test/unit/blockchain/sequence-manager/sequence-manager.service.spec.ts(10 tests).test/unit/modules/blockchain/blockchain.service.spec.ts(5 tests).Modified
src/modules/blockchain/blockchain.service.ts— addssubmitServerTransaction(spec)and routes manager errors throughhandleServerSubmissionError(a new helper that maps known Stellar result codes,SequenceManager:-prefixed errors, and network timeouts into the existingBadRequestException/InternalServerErrorException/ServiceUnavailableExceptiontypes).src/modules/blockchain/blockchain.module.ts— importsSequenceManagerModule.src/app.module.ts— registersSequenceManagerModule.test/unit/modules/transactions/transactions.service.spec.ts— adds one regression test asserting the user-signedsubmitTransactionpath still surfacesSTELLAR_TRANSACTION_FAILEDfor atx_bad_seqHorizon rejection. This pins the revert of a dedicatedSTELLAR_TX_BAD_SEQbranch that was introduced inhandleHorizonErrorand was out of scope for core: implement Stellar sequence-number management for concurrent submission #82; the dedicated branch is preserved only on the new server path (handleServerSubmissionError).docs/setup/environment-variables.md— adds a "Server-signed Stellar submissions (sequence manager)" section.docs/architecture/blockchain-layer.md— adds a "Server-signed submissions" sub-section.context/progress-tracker.md— new 2026-07-17 entry.Configuration
Three new env vars, all read at startup (
onModuleInit()):STELLAR_SOURCE_ACCOUNT_SECRETS…)submitServerTransactionis disabled and the API still boots (read-only mode).STELLAR_CHANNEL_ACCOUNTSS…secrets)[]STELLAR_BAD_SEQ_MAX_RETRIES3tx_bad_seqbefore giving up. Total attempt budget is1 + retries.Quality gates
npm run build— zero TypeScript errors.npm test— 305 passing, 0 failing, 26 suites green (was 277 — net +28 across the two new specs and the regression assertion).npx eslinton the touched files — zero warnings.anytypes introduced;SequenceMetricsandSequenceManagerServiceare fully typed.context/ai-workflow-rules.md— all Stellar SDK calls live insrc/blockchain/, services hold business logic, controllers do not call Supabase or Soroban directly.Known follow-ups (intentionally deferred)
submitServerTransactionyet. The acceptance criterion is satisfied end-to-end only at the unit-test layer. Loan funding, default-detection cron, and admin ops will land in follow-up PRs that use this method.TransactionStatusCheckerServicedoes not consume the newblockchain_source_bad_seq_retries_totalmetric. The metric is exposed butmetrics.updater.tsdoes not scrape or surface it on the Grafana dashboard. A one-linemetricsService.setBadSeqRetries(value)hook would close the integration half of acceptance criterion [11] Add learner profile creation on first login #3.observeSequencecoercesbigint → Numberfor the gauge. Stellar sequence numbers fit comfortably insideNumber.MAX_SAFE_INTEGERfor decades; documented as a known precision limitation rather than a refactor.Verification steps for the reviewer
References
docs/architecture/blockchain-layer.md→ "Server-signed submissions"docs/setup/environment-variables.md→ "Server-signed Stellar submissions (sequence manager)"context/code-standards.md,context/architecture-context.md,context/ai-workflow-rules.mdcontext/progress-tracker.md→ "2026-07-17"✅ Closes #82.