Read blocks ahead so gateway latency leaves the critical path

[vladbucur1]

Problem

At 600ms block time the chain produces 1.67 blocks/s/shard. The processor reads one block at a time and awaits each read inline (transaction.processor.ts:106 on main), so the gateway sits idle for the entire duration of the consumer callback and every nonce pays a full round-trip.

Measured against mainnet gateway, catching up 25 nonces/shard across 4 shards:

build	callback	rate	drains a 2000-block backlog in
main	no-op	4.16 nonces/s/shard	13 min
main	100ms	1.60 nonces/s/shard	never

With a 100ms onTransactionsReceived, main runs below the production rate. It is not slow, it is structurally under water — a backlog once formed grows forever. That is the "behind a few thousand blocks and keeps getting back" symptom.

Change

A bounded read-ahead cache per shard (src/utils/block-prefetcher.ts). While nonce N is being handed to the consumer, the reads for N+1..N+maxPrefetch are already in flight, so take() almost always resolves an already-settled promise.

build	callback	rate	drains a 2000-block backlog in
this PR	no-op	20.51 nonces/s/shard	2 min
this PR	100ms	2.37 nonces/s/shard	48 min

Delivery order is unchanged. Still one nonce per shard per pass, in order, setLastProcessedNonce only after the callback returns. Only the reads moved off the critical path. This is deliberate: letting shards drift apart would break getFinalizedCrossShardScrTransactions pass 2 (transaction.processor.ts:307), which silently drops a decrement whose increment has not landed yet — the counter would never reach 0 and the transaction would sit in crossShardDictionary until the 600s prune, never notified. Keeping delivery in lockstep means the SCR counters see blocks in exactly the order they do today.

Safety properties:

• The window never extends past the tip observed at the start of the pass, so it cannot request an unproduced block.
• It collapses to a single read when already caught up.
• Windows are cleared on network reset and pruned after a maxLookBehind jump.
• Memory ceiling is shards × maxPrefetch blocks (~2MB at the default).

Bonus: one flaky shard no longer stops every shard

The .catch is attached synchronously inside prime(), so a failed read resolves to undefined — the value the loop already treats as "block not available". The nonce stays uncommitted for a later retry instead of throwing out of start().

Test: shard 1 always fails, 3 healthy shards 10 nonces behind.

main       healthy shards advanced  1/30 nonces   threw
this PR    healthy shards advanced 30/30 nonces   no throw

Also: shared keep-alive agents in HttpService

start() is normally driven by a sub-second cron and rebuilds its HttpService each tick, so the agents are module-level. Per-instance agents would discard the socket pool every tick and pay a fresh TLS handshake per request (~165ms vs ~52ms measured).

Being straight about the value: Node 19+ already defaults globalAgent.keepAlive to true, so this is worth 0x on modern Node and ~3.1x on Node ≤18. It is a guarantee for SDK consumers on older runtimes plus a socket-pool cap during a read-ahead burst — it is not where the speedup above comes from. All of that is the read-ahead window.

API

One new option, backwards compatible:

maxPrefetch?: number;   // default 10; set to 1 for the previous one-block-at-a-time behaviour

Applied to both Shardblock and Hyperblock modes. Hyperblock gains the most, being a single serial stream.

Verification

tsc --noEmit, npm run lint and npm run build all clean. The repo has no test suite (npm test is still exit 1), so the numbers above come from harness scripts driving the real processor against https://gateway.multiversx.com and comparing against a main build. Nonce and callback counts matched exactly across builds (120 = 30 × 4), so nothing is skipped or double-delivered. Happy to land the harness as a benchmark script if useful.

Known remaining ceiling — not addressed here

2.37 is close to a hard ceiling of 2.5. The callback is awaited one shard at a time inside for (const shardId of this.shardIds), so 4 shards × 100ms is a 400ms floor per pass no matter how warm the cache is. Read-ahead cannot touch that.

Breaking it needs callback delivery parallel across shards, turning sum(400ms) into max(100ms) — roughly 2.5 → 10 nonces/s/shard. That should be auto-gated on waitForFinalizedCrossShardSmartContractResults being falsy, for the ordering reason above. Left for a follow-up.

Separately: package.json version is untouched — a minor bump would be appropriate for the new option.

Supersedes #26.

🤖 Generated with Claude Code

https://claude.ai/code/session_01S2BfVKhgjNR2FdJR6V6GXQ