Skip to content

paper: update for the v4 results (real-broker replay, aging isolation, baselines, sensitivity) - #56

Merged
divyanshusingh2903 merged 2 commits into
mainfrom
paper/v4-results
Oct 18, 2026
Merged

divyanshusingh2903 merged 2 commits into
mainfrom
paper/v4-results

Conversation

@divyanshusingh2903

Copy link
Copy Markdown
Owner

What this is

The paper update for the v4 results. Stacked on #55 (base bench/v4-replay-sensitivity), because the paper builds its numbers from the result files that PR adds. When #55 merges, GitHub retargets this one to main.

What the paper now says

  • Aging, not the predictor, limits the gain under backlogs. Predictive routing as built fails its gate in all six Azure and Huawei replay cells; with aging off it passes on Huawei at both loads and cuts the production flow's mean a further 37.4%, matching the RabbitMQ sidecar (the sidecar paragraph is now a measurement, not an explanation). Weights and pausing aging pass at loads 0.8 and 0.95 on both traces (gate B 4 of 6).
  • The predictor is not the only route to the gain. A queue per job type with no prediction has a mean 42.1% below predictive routing's on the production flow (at a worse all-message P99), and on Huawei a static map from the same history matches or beats the live predictor. No predictor component shows a benefit on the production flow.
  • Gate B is robust to every swept parameter; gate A is not, and the aging threshold dominates (simulator only).
  • The simulator matches the broker to about a point on both replayed traces, where it overstated the production flow by about 40%.

Changes

  • Text: abstract, introduction, related-work positioning, method (replay method, deviations, the table of what informed each experiment), design (parameters, aging, weights, opt-in table), results (new sections 5.2 to 5.6 and the sidecar paragraph), limitations (two retried replay cells, a single Azure run, simulated HPC logs, the dropped sensitivity stage 2) and conclusion.
  • Generators: gen_numbers.py and gen_figures.py read the replay, baselines, aging-replay and sensitivity results (470 macros, 14 tables). The trace figure shows the replay for Azure and Huawei and the simulation for the two HPC logs. The Makefile watches the new files. revision/claims-evidence.md is extended.

Please check

  • This commit also carries earlier, previously uncommitted paper work from another session: the restructured design section, the per-phase table, the parameter table, the LLM-serving related work, and the generated figures and tables. It was never committed, and it is intertwined with these edits in the same files, so I could not split it.
  • Wording of the central claim: "the reason is aging" in the sidecar paragraph rests on one aging-off measurement on the production flow and one Huawei replay. I believe it is supported, but it is a strong sentence.
  • Not done: appendix tables of the new pre-registered criteria; the release tag v0.1.1-preprint named in the limitations; the "Draft \today" line in the title block.
  • Build: the local TeX install lacks biblatex; build with make -C paper docker-pdf. It produces 37 pages with no undefined references; the one remaining overfull box is a bibliography URL that was already there.

🤖 Generated with Claude Code

…, baselines, sensitivity)

Rewrites the claims the new experiments change, and builds every new number and table from the committed result files.

Findings the paper now reports: predictive routing as built fails its gate in all six Azure and Huawei replay cells; with aging
switched off it passes on Huawei and cuts the production flow's mean a further 37.4%; weights and pausing aging pass at loads 0.8 and
0.95 on both traces (gate B 4 of 6); a queue per job type with no prediction beats the predictor on mean latency on the production
flow (42.1% lower, P99 worse), and on Huawei a static map from the same history matches or beats it; no predictor component shows a
benefit; gate B is robust to every swept parameter but gate A is not, and the aging threshold dominates; the simulator matches the
broker to about a point on both replayed traces, where it overstated the production flow by about 40%.

Text: abstract, introduction, related-work positioning, method (replay method, deviations, the what-informed-each-experiment table),
design (parameters, aging, weights, opt-in table), results (sections 5.2 to 5.6, the sidecar paragraph), limitations (two retried
replay cells, single Azure run, simulated HPC logs, dropped sensitivity stage 2) and conclusion. gen_numbers.py and gen_figures.py read
the replay, baselines, aging-replay and sensitivity results; the trace figure shows the replay for Azure and Huawei and the simulation
for the two HPC logs. The Makefile watches the new result files; revision/claims-evidence.md is extended.

This commit also carries the earlier, uncommitted revision of the paper (restructured design section, per-phase table, parameter
table, LLM-serving related work, generated figures and tables), which was never committed.
@divyanshusingh2903
divyanshusingh2903 changed the base branch from bench/v4-replay-sensitivity to main October 18, 2026 06:01
@divyanshusingh2903
divyanshusingh2903 merged commit efbc8d9 into main Oct 18, 2026
6 checks passed
@divyanshusingh2903
divyanshusingh2903 deleted the paper/v4-results branch October 18, 2026 06:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant