Skip to content

WsClient: keep local order books sorted and verify begin_nonce continuity - #199

Open
Mohith26 wants to merge 1 commit into
elliottech:mainfrom
Mohith26:feat/ws-order-book-integrity
Open

Mohith26 wants to merge 1 commit into
elliottech:mainfrom
Mohith26:feat/ws-order-book-integrity

Conversation

@Mohith26

@Mohith26 Mohith26 commented Nov 8, 2026

Copy link
Copy Markdown

WsClient keeps a local copy of each subscribed order book, but it had three problems:

  • New price levels were appended to the end of asks/bids, so after a level appears inside the book, bids[0] / asks[0] are no longer the best prices.
  • The book's nonce / offset stayed at the snapshot values forever.
  • Nothing checked continuity. The websocket docs say to verify that each update's begin_nonce equals the previous update's nonce (this is also what Question: How to ensure local orderbook consistency after offsets became non-contiguous? #105 asked about), but the client applied deltas regardless, so a dropped update left a silently wrong book.

This PR:

  • keeps both sides sorted (asks ascending, bids descending) and applies deltas with bisect over a parallel price index instead of scanning and copying the list for every delta
  • updates nonce, begin_nonce, offset, last_updated_at on the local book from each applied update
  • checks begin_nonce == previous nonce; on a mismatch it stops applying deltas for that market, sends unsubscribe + subscribe to get a fresh snapshot, and calls a new optional on_order_book_gap(market_id, expected, received) callback. The server's unsubscribed ack for that resync is not reported as an unhandled message. verify_order_book_continuity=False turns the check off.
  • works on both run() and run_async() (the async path awaits the resubscribe sends)
  • leaves update_orders() unchanged for anyone calling it directly

Testing

  • test/test_ws_client_order_book.py: 11 new tests (sorting, removals, metadata, gap -> resubscribe -> fresh snapshot, async path, disabling the check, and a 3,000-update randomized comparison against a reference book). pytest --ignore-glob="*api.py" passes, flake8 with the CI flags is clean.
  • Recorded the public testnet stream for 4 markets: 30 updates, 0 continuity breaks, and for the 3 markets whose fresh snapshot had the same nonce as the last streamed update, the local book matched the snapshot exactly.
  • Live on testnet: simulated a gap on market 4096; the client resubscribed, got a fresh snapshot, and kept applying the following updates with continuity intact.

Benchmark (python test/bench_ws_order_book.py, 3,000 updates of 1-8 level changes, median of 5 runs, Apple Silicon arm64):

levels per side before after speedup
50 0.0375 s 0.0086 s 4.3x
200 0.1508 s 0.0098 s 15.3x
1000 0.9628 s 0.0540 s 17.8x

…uity

- asks/bids stay sorted (asks ascending, bids descending) when new price levels
  arrive; previously new levels were appended to the end of the lists
- level updates use bisect over a parallel price index instead of scanning and
  copying the list per delta (4-18x faster in test/bench_ws_order_book.py)
- nonce/begin_nonce/offset/last_updated_at on the local book track the last
  applied update instead of staying at the snapshot values
- each update is checked with the documented continuity rule (begin_nonce ==
  previous nonce); on a gap the book is dropped, the market is resubscribed for a
  fresh snapshot and on_order_book_gap is called. verify_order_book_continuity=False
  turns the check off
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant