test(media): add real media-plane fan-out capacity test - #78
Conversation
Puppeteer test that ramps real-media viewers against one broadcaster and samples inbound-rtp getStats (bitrate, packet loss, jitter, fps) to find the actual SFU media ceiling — the signaling load test only covers the control plane. - load-test/media-fanout-capacity.js — getStats via RTCPeerConnection hook; per-batch CSV - load-test/results/MEDIA-FANOUT-CAPACITY.md — method, run, interpretation, caveats Authored by Claude (Anthropic) via Claude Code. Not executed against a live backend — the results section is a placeholder, not fabricated. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
|
Important
This repository does not receive automatic reviews because it has fewer than 10 stars. ⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Plus Run ID: Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
|
![cubic-dev-ai[bot]](https://avatars.githubusercontent.com/in/1082092?s=60&v=4){kind=small}
There was a problem hiding this comment.
8 issues found across 2 files
Prompt for AI agents (unresolved issues)
Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.
<file name="load-test/media-fanout-capacity.js">
<violation number="1" location="load-test/media-fanout-capacity.js:140">
P1: Every run times out after sign-in because `window.__csSocket` is never assigned; `frontend/src/socket.ts` keeps the socket module-private. Wait on an actual readiness signal or expose the socket.</violation>
<violation number="2" location="load-test/media-fanout-capacity.js:212">
P2: The script captures jitter and resolution but never emits them in console or CSV results. Carry these fields through the batch summary and output columns.</violation>
<violation number="3" location="load-test/media-fanout-capacity.js:396">
P1: Broadcaster setup always times out because `BroadcasterPage` stores the room ID in React state and never assigns `window.__csRoomId`. Read an actual page signal or add the marker in the frontend.</violation>
<violation number="4" location="load-test/media-fanout-capacity.js:413">
P2: When `__csLiveAt` is absent, this catch proceeds with a room that may have no flowing producer. Fail setup before launching viewers when broadcaster media readiness is not confirmed.</violation>
<violation number="5" location="load-test/media-fanout-capacity.js:443">
P2: The latency clock starts before viewer navigation and form submission, so join and first-frame latency include page load. Set `startTime` immediately before `form.requestSubmit()`.</violation>
<violation number="6" location="load-test/media-fanout-capacity.js:474">
P1: `ViewerPage` never assigns either marker, so every viewer becomes `joined: false` after two 20-second waits and the latency results are empty. Use actual join state and a video-frame event, or add these markers.</violation>
<violation number="7" location="load-test/media-fanout-capacity.js:617">
P2: Non-multiple limits silently under-run: `maxViewers=3,batchSize=5` launches no viewers and `13/5` launches 10. Iterate until `viewers.length` reaches the limit and cap the final batch.</violation>
<violation number="8" location="load-test/media-fanout-capacity.js:673">
P2: A viewer that joins but never produces a first-frame timestamp is not counted because `failed` checks only `joined`. Count missing first-frame latency as a failure too.</violation>
</file>
Reply with feedback, questions, or to request a fix.
Re-trigger cubic
|
|
||
| try { | ||
| await page.waitForFunction( | ||
| () => Boolean(window.__csJoinedAt), |
There was a problem hiding this comment.
P1: ViewerPage never assigns either marker, so every viewer becomes joined: false after two 20-second waits and the latency results are empty. Use actual join state and a video-frame event, or add these markers.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At load-test/media-fanout-capacity.js, line 474:
<comment>`ViewerPage` never assigns either marker, so every viewer becomes `joined: false` after two 20-second waits and the latency results are empty. Use actual join state and a video-frame event, or add these markers.</comment>
<file context>
@@ -0,0 +1,778 @@
+
+ try {
+ await page.waitForFunction(
+ () => Boolean(window.__csJoinedAt),
+ { timeout: WAIT_TIMEOUT_MS }
+ );
</file context>
| */ | ||
|
|
||
| await page.waitForFunction( | ||
| () => Boolean(window.__csRoomId), |
There was a problem hiding this comment.
P1: Broadcaster setup always times out because BroadcasterPage stores the room ID in React state and never assigns window.__csRoomId. Read an actual page signal or add the marker in the frontend.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At load-test/media-fanout-capacity.js, line 396:
<comment>Broadcaster setup always times out because `BroadcasterPage` stores the room ID in React state and never assigns `window.__csRoomId`. Read an actual page signal or add the marker in the frontend.</comment>
<file context>
@@ -0,0 +1,778 @@
+ */
+
+ await page.waitForFunction(
+ () => Boolean(window.__csRoomId),
+ { timeout: WAIT_TIMEOUT_MS }
+ );
</file context>
|
|
||
| await page.waitForFunction( | ||
| () => | ||
| window.__csSocket && |
There was a problem hiding this comment.
P1: Every run times out after sign-in because window.__csSocket is never assigned; frontend/src/socket.ts keeps the socket module-private. Wait on an actual readiness signal or expose the socket.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At load-test/media-fanout-capacity.js, line 140:
<comment>Every run times out after sign-in because `window.__csSocket` is never assigned; `frontend/src/socket.ts` keeps the socket module-private. Wait on an actual readiness signal or expose the socket.</comment>
<file context>
@@ -0,0 +1,778 @@
+
+ await page.waitForFunction(
+ () =>
+ window.__csSocket &&
+ window.__csSocket.connected === true,
+ { timeout: WAIT_TIMEOUT_MS }
</file context>
| bytesReceived: report.bytesReceived || 0, | ||
| packetsReceived: report.packetsReceived || 0, | ||
| packetsLost: report.packetsLost || 0, | ||
| jitter: report.jitter || 0, |
There was a problem hiding this comment.
P2: The script captures jitter and resolution but never emits them in console or CSV results. Carry these fields through the batch summary and output columns.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At load-test/media-fanout-capacity.js, line 212:
<comment>The script captures jitter and resolution but never emits them in console or CSV results. Carry these fields through the batch summary and output columns.</comment>
<file context>
@@ -0,0 +1,778 @@
+ bytesReceived: report.bytesReceived || 0,
+ packetsReceived: report.packetsReceived || 0,
+ packetsLost: report.packetsLost || 0,
+ jitter: report.jitter || 0,
+ framesPerSecond: report.framesPerSecond || 0,
+ frameWidth: report.frameWidth || 0,
</file context>
| const failed = batch.filter( | ||
| (viewer) => !viewer.joined | ||
| ).length; |
There was a problem hiding this comment.
P2: A viewer that joins but never produces a first-frame timestamp is not counted because failed checks only joined. Count missing first-frame latency as a failure too.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At load-test/media-fanout-capacity.js, line 673:
<comment>A viewer that joins but never produces a first-frame timestamp is not counted because `failed` checks only `joined`. Count missing first-frame latency as a failure too.</comment>
<file context>
@@ -0,0 +1,778 @@
+ batch.map((viewer) => viewer.fps)
+ );
+
+ const failed = batch.filter(
+ (viewer) => !viewer.joined
+ ).length;
</file context>
| const failed = batch.filter( | |
| (viewer) => !viewer.joined | |
| ).length; | |
| const failed = batch.filter( | |
| (viewer) => | |
| !viewer.joined || | |
| !Number.isFinite(viewer.firstFrameLatencyMs) | |
| ).length; |
| */ | ||
|
|
||
| for ( | ||
| let target = BATCH_SIZE; |
There was a problem hiding this comment.
P2: Non-multiple limits silently under-run: maxViewers=3,batchSize=5 launches no viewers and 13/5 launches 10. Iterate until viewers.length reaches the limit and cap the final batch.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At load-test/media-fanout-capacity.js, line 617:
<comment>Non-multiple limits silently under-run: `maxViewers=3,batchSize=5` launches no viewers and `13/5` launches 10. Iterate until `viewers.length` reaches the limit and cap the final batch.</comment>
<file context>
@@ -0,0 +1,778 @@
+ */
+
+ for (
+ let target = BATCH_SIZE;
+ target <= MAX_VIEWERS;
+ target += BATCH_SIZE
</file context>
|
|
||
| await login(page); | ||
|
|
||
| const startTime = Date.now(); |
There was a problem hiding this comment.
P2: The latency clock starts before viewer navigation and form submission, so join and first-frame latency include page load. Set startTime immediately before form.requestSubmit().
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At load-test/media-fanout-capacity.js, line 443:
<comment>The latency clock starts before viewer navigation and form submission, so join and first-frame latency include page load. Set `startTime` immediately before `form.requestSubmit()`.</comment>
<file context>
@@ -0,0 +1,778 @@
+
+ await login(page);
+
+ const startTime = Date.now();
+
+ try {
</file context>
| ); | ||
|
|
||
| console.log("Broadcaster is LIVE."); | ||
| } catch { |
There was a problem hiding this comment.
P2: When __csLiveAt is absent, this catch proceeds with a room that may have no flowing producer. Fail setup before launching viewers when broadcaster media readiness is not confirmed.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At load-test/media-fanout-capacity.js, line 413:
<comment>When `__csLiveAt` is absent, this catch proceeds with a room that may have no flowing producer. Fail setup before launching viewers when broadcaster media readiness is not confirmed.</comment>
<file context>
@@ -0,0 +1,778 @@
+ );
+
+ console.log("Broadcaster is LIVE.");
+ } catch {
+ console.warn(
+ "Room exists, but __csLiveAt was not detected."
</file context>
1 participant
Authored by Claude (Anthropic) via Claude Code. Not executed against a live backend — the results section is a placeholder, not fabricated.
Summary by cubic
Adds a real media-plane fan-out capacity test for CrowdStream's SFU. The existing signaling load test only exercises the control plane, so this Puppeteer script ramps real fake-media viewers against one broadcaster and samples inbound video getStats to find the actual media ceiling.
Details
RTCPeerConnectionviaevaluateOnNewDocumentand pushing every PeerConnection intowindow.__csPCs.bytesReceivedacross two samples taken--statsSettleMsapart.Written for commit 22cc341. Summary will update on new commits.