Pool count from gateway

[stefangutica]

Reasoning

• /pool/count downloaded the whole pool from the gateway and returned its length, so it failed whenever the pool response exceeded the 2MB limit the API sets for it.
• In that case /pool returned a generic error, so the FE could not tell it apart from any other failure, and the pool was downloaded again, up to the limit, on every request.

Proposed Changes

• Gateway:
  • getTransactionPool returns null when the response exceeds the size limit. Any other error still throws.
  • New getTransactionPoolCount sums the per-shard counts from transaction/pool/count.
• Pool cache: a pool that is too large is cached as null under txpool (explicit cacheNullable: true), so it is not downloaded again until the value expires.
• Count: /pool/count still counts the pool, so it matches what /pool lists. Whenever the pool cannot be read, too large or for any other reason, it returns the total from the gateway, cached under txpool:count for 1s, whatever the filters.
• REST: /pool and /pool/:txhash have two kinds of errors:
  • 503 with { "statusCode": 503, "code": "transaction_pool_too_large", "message": "The transaction pool is too large to be displayed" } when the pool is too large, documented in Swagger;
  • 500 for any other failure, as before.
• Websocket: every poolUpdate now carries a status: success, tooLarge or internalServerError.
  • pool is null unless the status is success.
  • poolCount is the total from the gateway when the pool cannot be read, or null when the count fails as well.
• Cache warmer: when the pool is too large, it caches it as null and also refreshes the total count from the gateway, with transactionPoolWarmer.ttlInSeconds, as before.
• Price per unit: unchanged. The pool it reads fails when the pool is too large, so this case is handled like any other failure of the pool.

How to test

• Unit tests:
  • pool.spec.ts: counts, null cache, gateway fallback for too large and for other failures, pool with filters failing when too large;
  • pool.too.large.spec.ts: 503/500 responses, websocket statuses, price per unit;
  • cache.warmer.pool.spec.ts: warming;
  • gateway.pool.count.spec.ts: size limit, other errors, count sum.
• Checked against the real ApiService: a 1MB pool is returned, a 3MB pool is detected as too large, and an unreachable gateway still throws.
• Checked over HTTP: with the pool too large, /pool and /pool/:txhash answer 503 and /pool/count answers 200 with the gateway total. With any other failure, /pool and /pool/:txhash answer 500.
• Note: the gateway count covers only regular transactions, not smart contract results or rewards, updates once per block, and takes no filters. transaction/pool/count exists from mx-chain-proxy-go#520 on; mainnet and testnet gateways don't have it yet at the time of writing.

[github-actions]

k6 load testing comparison.
Base Commit Hash: c533f21
Target Commit Hash: f4332d3

Metric	Base				Target				Diff
	Avg	Max	90	95	Avg	Max	90	95	Avg	Max	90	95
Mex	41.08	878.67	54.15	59.69	42.50	937.74	57.23	61.28	+3.46% 🔴	+6.72% 🔴	+5.69% 🔴	+2.66% 🔴
Pool	40.90	612.40	54.11	59.66	42.32	484.83	57.19	61.28	+3.47% 🔴	-20.83% ✅	+5.69% 🔴	+2.72% 🔴
Transactions	73.90	16484.93	61.04	67.45	71.34	12563.78	64.67	76.02	-3.45% ✅	-23.79% ✅	+5.94% 🔴	+12.70% 🔴
Blocks	50.47	890.63	58.40	62.61	54.93	1021.71	61.48	66.50	+8.85% 🔴	+14.72% 🔴	+5.28% 🔴	+6.21% 🔴
Tokens	41.03	722.40	54.27	59.79	42.34	573.76	57.19	61.28	+3.20% 🔴	-20.58% ✅	+5.38% 🔴	+2.49% 🔴
Accounts	47.62	1239.06	58.72	64.52	46.53	948.92	61.18	67.12	-2.30% ✅	-23.42% ✅	+4.19% 🔴	+4.02% 🔴
Nodes	74.81	17399.24	79.28	90.35	68.24	16778.67	61.14	66.61	-8.77% ✅	-3.57% ✅	-22.88% ✅	-26.28% ✅
Test Run Duration	60002.32				60005.83

Legend: Avg - Average Response Time, Max - Maximum Response Time, 90 - 90th Percentile, 95 - 95th Percentile
All times are in milliseconds.