docs: record Open-Jev PR-by-PR optimization A/B results - #88
Merged
Merged
Conversation
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
hsliuustc0106
marked this pull request as ready for review
October 5, 2026 05:56
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Purpose
Add a concise Open-Jev optimization post led by headline numbers and comparison figures, linked from the README and MkDocs navigation. The post is about 850 words, with four main charts, a compact mechanism table, and links to detailed figures, controls and raw records.
Cover native/graph ancestry (#19/#52), PR #55's backend gain and isolated RMSNorm/SiLU/graph-cache A/Bs, GDN preparation (#68), and processing/runtime regression checks (#78/#80). Distinguish bundled, isolated, kernel and HTTP results; keep missing ablations and missed gates visible.
Include the reusable
write-optimization-blogskill, A/B evidence template and presentation references. The skill favors short explanations and linked detailed records for concise posts. Installed and repository copies match. Preserve six SVG/PNG pairs, CPU-only regeneration and 78 measured passes / 5,040 timing samples with provenance/parity hashes.Test Plan
System1-Omni Version / Commit: docs head
5d5398147be15ac2e73409f0de694b5481d8e395; GitHub-reported PR basef594d7dfc4c2bef812e23f7ed73573be9625b287; fetched upstream main99e368982ea3a9d4ede2f96133c1f1be2995008a; branch merge based6aa3a9cedcc5c87cf0222d08fabce5accf521db. Measurement revisions are separate in the ledger.Check the diff and claims against frozen source/evidence; audit hashes, raw timings, parity/gates, rendered links and figures; validate both skills and strict MkDocs build. New inference A/B is unnecessary for documentation changes.
Test Result
mkdocs build --strict --site-dir profile/blog-writing-20261005/site: passed.quick_validate.pyand match byte-for-byte.git diff --check origin/main...HEAD: passed. Commit is DCO signed.5d53981: Docs build, Rust and benchmarks passed; Pages deployment skipped for pull requests.Demo / evidence
Read the post, PR #55 figure, ledger, timing export, and writing skill.
Historical H200/BF16 JevBench results: 362.21 → 48.50 ms (7.47×) versus raw HF, 74 single-candidate requests/pass, concurrency 1, two warm measured passes after excluded feasibility. Separate RMSNorm/SiLU/Graph-64 A/Bs save 2.58% / 2.48% / 2.03%; they cannot be added into a cumulative ladder. GDN's ~13% longer-call gain yields 0.42% HTTP, missing its 2% gate. Refactor checks establish no speedup.
HF/native decisions match on the subset while probabilities differ. LoRA/native/gate-only attribution and GDN + Graph latency remain unmeasured. Figures plot preserved project evidence; timing exports contain IDs/timings/hashes without request bodies or credentials. Full response/protocol/trace archives remain local. Video is N/A for documentation. These are historical results, not measurements of this docs head or current main.
Self-review
Agent-assisted diff and evidence review found no blocking issues. Contributor checklist state is preserved; the PR remains draft.