LT-22674: fix quadratic NFC offset-map build that made opening decomposed texts take 10+ seconds - #1056
Open
johnml1135 wants to merge 5 commits into
Open
LT-22674: fix quadratic NFC offset-map build that made opening decomposed texts take 10+ seconds#1056johnml1135 wants to merge 5 commits into
johnml1135 wants to merge 5 commits into
Conversation
Render comparison artifactsRender snapshot failures were reported in a0e16e5b722a run 31584662718.1, but the latest run 2d00ee01141f run 31784647591.1 is clean. This comment will be replaced if a future run produces render snapshot failures again. |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #1056 +/- ##
==========================================
+ Coverage 37.91% 38.04% +0.13%
==========================================
Files 1499 1499
Lines 350117 350101 -16
Branches 40233 40233
==========================================
+ Hits 132747 133200 +453
+ Misses 188043 187616 -427
+ Partials 29327 29285 -42
🚀 New features to boost your workflow:
|
RealDataTestsBase refused to delete its reusable test project unless the sentinel file was present, but CreateNewLangProj creates that directory seconds before the sentinel can be written. A run interrupted inside project creation therefore left a directory that no later run would delete, failing every render benchmark scenario with "Refusing to delete" until someone wrote the dotfile by hand. Deletion now restores a missing sentinel when the directory holds no project data beyond the fixture's own generated file, and still refuses when it finds project data the fixture did not generate. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Opening a text whose baseline is a single long paragraph with more than one writing system took over ten seconds. The analysis cache built NFC offset maps for every analysed range by normalizing each prefix of the range from scratch, which is quadratic in the range length and repeated per range. The maps exist only to translate between original and normalized offsets, which is needed only when normalization changes the text. Text that normalizes unchanged needs no maps, so the cache now stores only that text and everything else falls back to the existing on-demand translation. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
TestViewCaches.h was never listed as a dependency of Collection.cpp, and that list is the input to test collection, so its nine tests were never run. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The existing scenarios use text that is identical under NFD and NFC, so they cannot reach the offset-map path at all. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Reviewed the .received.png output for single-para-mixed-ws, nfc-composable-diacritics, and multi-line-wrap-single-ws and confirmed correct rendering, so accepting them as the new verified baselines these scenarios were missing.
johnml1135
force-pushed
the
lt-22674-nfc-offset-map-quadratic
branch
from
August 14, 2026 08:37
b9e6e39 to
a666ea0
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes LT-22674.
Opening a text in Texts & Words took 10+ seconds when the baseline was a single long paragraph containing more than one writing system. Reproduced on Maasai-Parser with camus4-D (1 paragraph, 236 segments, 7,571 characters, two writing systems on two different font families).
Cause
BuildNfcOffsetMapsinSrc/views/lib/UniscribeSegment.cppnormalised every prefix of a range from scratch:Quadratic in the range length, and called once per analysed range, so the aggregate cost grows with the square of the paragraph length. For a 7,571-character paragraph that is tens of millions of characters normalised per layout pass.
The maps exist to translate between original and normalised offsets — Uniscribe shapes the normalised text while selections, cursor offsets and hit-testing live in the original text. They are only needed when normalisation actually changes the string. It fires when the text is not already NFC and the layout-pass cache is present (
UniscribeSegment.cpp:3157before this change). Text reaching the Views layer is typically NFD, so for vernacular data that was the normal path, not an edge case.This is a regression.
BuildNfcOffsetMaps,vichOrigToNfcandLayoutPassCacheall arrived together in 502b832 ("perf: Views engine render optimizations", #724, 2026-05-08). Before it there was no eager construction: translation was on demand viaOffsetInNfc/OffsetToOrig, whose comment notes it "is not called with high frequency".Fix
Text that normalisation leaves unchanged needs no maps at all, because the mapping is the identity. So the analysis cache now stores only that text, and anything normalisation would rewrite falls back to the pre-existing on-demand translation. The eager map machinery is removed rather than rewritten.
The fix is net -21 lines.
ShapeRunCache— the genuine ~3x improvement from the same commit — is untouched.An earlier attempt kept caching non-NFC text and made
BuildNfcOffsetMapslinear by splitting at ICU normalisation boundaries. It was measured and rejected: the decomposed proxy still took 3,468 / 2,174 ms against a 298 / 269 ms control, because a linear build still makes roughly one ICU call per character and is still invoked once per range. Fixing the constant factor could not fix the aggregate.Numbers
Cold render, milliseconds, 2 replicates. "cache-OFF control" is the same binary with
FW_PERF_P125_PATH2=0, which is the configuration confirmed to resolve the problem on the real project.After the fix, cache-on matches the cache-off control, i.e. the penalty is gone rather than reduced.
Verification
Invoke-CppTest.ps1already has explicit handling for; it is unrelated to this change.Views.makhas no header-dependency tracking and a staleViews.dllreports success (see below).Why the existing tests did not catch this
Three safety nets were disconnected, all introduced or left broken by 502b832 itself.
The benchmark scenarios could not reach the regressed path. The suite sets
IsGraphiteEnabled = falseand uses ASCII almost throughout, and ASCII is identical under NFD and NFC, so the offset-map branch never executed.multi-wsandrtl-scriptreached it only incidentally, because Arabic harakat are canonically reordered. This PR adds three scenarios built from decomposed text —nfc-composable-diacritics, and decomposed forms ofsingle-para-mixed-wsandmulti-line-wrap-single-ws— which reproduce the defect at seconds scale.The cache unit tests had never run.
Src/views/Test/TestViewCaches.hwas added by the same commit with 9 test methods acrossTestColorStateCache,TestFontHandleCacheandTestShapeRunCache, but was never listed as a dependency ofCollection.cppintestViews.mak— and that list is the input to test collection. This PR registers it; all 9 pass.Header-only native edits silently produce stale binaries.
Views.makviaBld/_rule.makuses{srcdir}.cpp{objdir}.obj:rules with no header-dependency tracking, so editing a header leaves the previousViews.dllin place while the build reports success. Hit twice during this investigation. Not addressed here — it is a broader build-system defect and needs its own issue.Deliberately not included
FW_PERF_P125_PATH2default is unchanged. With the quadratic construction gone the cache is no longer harmful, so flipping a default would only obscure what this PR claims. The environment variable remains available.TextAnalysisCachecapacity and retention changes. Measured no reproducible benefit and were aimed at a cost that turned out not to be the problem.The three new scenarios have no accepted pixel baseline yet, so they report "Missing verified render baseline" until someone reviews and accepts the
.received.pngfiles. Their timings are still recorded.This change is