Skip to content

docs: record Open-Jev PR-by-PR optimization A/B results - #88

Merged
hsliuustc0106 merged 6 commits into
mainfrom
codex-optimization-blog
Oct 5, 2026
Merged

hsliuustc0106 merged 6 commits into
mainfrom
codex-optimization-blog

Conversation

@hsliuustc0106

@hsliuustc0106 hsliuustc0106 commented Oct 5, 2026 •

Copy link
Copy Markdown
Contributor

Purpose

Add a concise Open-Jev optimization post led by headline numbers and comparison figures, linked from the README and MkDocs navigation. The post is about 850 words, with four main charts, a compact mechanism table, and links to detailed figures, controls and raw records.

Cover native/graph ancestry (#19/#52), PR #55's backend gain and isolated RMSNorm/SiLU/graph-cache A/Bs, GDN preparation (#68), and processing/runtime regression checks (#78/#80). Distinguish bundled, isolated, kernel and HTTP results; keep missing ablations and missed gates visible.

Include the reusable write-optimization-blog skill, A/B evidence template and presentation references. The skill favors short explanations and linked detailed records for concise posts. Installed and repository copies match. Preserve six SVG/PNG pairs, CPU-only regeneration and 78 measured passes / 5,040 timing samples with provenance/parity hashes.

Test Plan

System1-Omni Version / Commit: docs head 5d5398147be15ac2e73409f0de694b5481d8e395; GitHub-reported PR base f594d7dfc4c2bef812e23f7ed73573be9625b287; fetched upstream main 99e368982ea3a9d4ede2f96133c1f1be2995008a; branch merge base d6aa3a9cedcc5c87cf0222d08fabce5accf521db. Measurement revisions are separate in the ledger.

Check the diff and claims against frozen source/evidence; audit hashes, raw timings, parity/gates, rendered links and figures; validate both skills and strict MkDocs build. New inference A/B is unnecessary for documentation changes.

Test Result

  • mkdocs build --strict --site-dir profile/blog-writing-20261005/site: passed.
  • Task-local evidence/link audit: passed. Ten source-hash entries, 12 raw GDN kernel records, 296 GDN HTTP responses and 78 exported passes / 5,040 samples match the ledger. Refactor p95/throughput/gates and headline ratios recompute; Markdown/rendered links resolve.
  • Both skill copies pass quick_validate.py and match byte-for-byte.
  • All six figure pairs and measurement inputs are unchanged in this revision; four figures are embedded and two detailed figures linked. Regeneration and visual inspection passed earlier on this branch.
  • git diff --check origin/main...HEAD: passed. Commit is DCO signed.
  • GitHub CI on 5d53981: Docs build, Rust and benchmarks passed; Pages deployment skipped for pull requests.
  • No new GPU measurements or local Rust/GPU tests: documentation-only changes. Full archives/task-local helpers remain outside the PR.

Demo / evidence

Read the post, PR #55 figure, ledger, timing export, and writing skill.

Historical H200/BF16 JevBench results: 362.21 → 48.50 ms (7.47×) versus raw HF, 74 single-candidate requests/pass, concurrency 1, two warm measured passes after excluded feasibility. Separate RMSNorm/SiLU/Graph-64 A/Bs save 2.58% / 2.48% / 2.03%; they cannot be added into a cumulative ladder. GDN's ~13% longer-call gain yields 0.42% HTTP, missing its 2% gate. Refactor checks establish no speedup.

HF/native decisions match on the subset while probabilities differ. LoRA/native/gate-only attribution and GDN + Graph latency remain unmeasured. Figures plot preserved project evidence; timing exports contain IDs/timings/hashes without request bodies or credentials. Full response/protocol/trace archives remain local. Video is N/A for documentation. These are historical results, not measurements of this docs head or current main.

Self-review

Agent-assisted diff and evidence review found no blocking issues. Contributor checklist state is preserved; the PR remains draft.

  • I have reviewed the full diff and addressed the issues I found.
  • I have checked that the change follows the project's architecture and stays focused on the stated purpose.
  • I have run the checks appropriate to this change and reported commands, results, and anything I could not verify above.
  • I have checked that the PR description, documentation, and any accuracy or performance claims match the implementation and available evidence.

Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
@hsliuustc0106 hsliuustc0106 changed the title docs: add optimization blog skill and Open-Jev post docs: record Open-Jev PR-by-PR optimization A/B results Oct 5, 2026
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
@hsliuustc0106
hsliuustc0106 marked this pull request as ready for review October 5, 2026 05:56
@hsliuustc0106
hsliuustc0106 merged commit f4ee212 into main Oct 5, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant