Repository navigation
Guard the power-mean relaxation factor estimate - #88
Merged
Merged
Conversation
The asymmetric power-mean sweeps overrelax by a factor `ω` estimated from plain sweeps and raised from the rates of later windows. Two defects in that rule made some solves far slower than plain sweeps. An `ω` set on a plateau was never lowered. Far from the fixed point the imbalance can stay nearly constant for several sweeps before it contracts quickly. An `ω` estimated there is near its cap, and above the optimum the relaxed rate is `ω - 1` whatever the plain rate is, so the windows read it as steady progress. The plain estimate now waits while the contraction is still improving from one sweep to the next, and sets `ω` no higher than 1.5, where a relaxed sweep contracts by at least half even above the optimum. Windows raise it from there. A window was judged against the smallest imbalance of all earlier windows. The imbalance is the size of the plain step, which a relaxed iterate overshoots by `ω - 1` times, so its level rises with `ω`, and windows at a raised `ω` could show no progress while the iteration converged. Those windows tripped the stall rule, which halves `ω - 1` when it should rise. Each window is now judged against the window before it. The first window after `ω` changes is still judged against one at the old `ω` and reads as slow, so one window shrinks `2 - ω` by at most a factor of 8. The handoff to Newton after the plain estimate is predicted at the rate of the factor that estimate implies, as before, not at the capped factor the first relaxed sweeps use. On 1200 random matrices of 2 to 8 rows and columns (lognormal entries with σ of 1, 3 or 6, up to half zeroed), with `p` of 1 and 2 and the default and flat starts, 4800 runs in all. With `newton = false`, on the 4663 runs that plain sweeps finish in 50,000: | | before | after | |---|---|---| | sweeps | 552,513 | 254,336 | | runs more than 2 sweeps slower than plain | 213 | 9 | | worst ratio to plain | 85 | 1.73 | | runs where the stall rule fires (of 4800) | 55 | 3 | | runs plain does not finish that the rule does | 87 of 137 | 101 of 137 | The 3 remaining stalls are runs that neither rule finishes in 10^5 sweeps. With `newton = true`: | | before | after | |---|---|---| | runs handed to Newton | 949 | 974 | | passes over the support | 317,998 | 302,284 | | factorizations | 5130 | 5227 | Sweeps with `newton = false` on larger problems, plain / before / after: | problem | plain | before | after | |---|---|---|---| | bidiagonal n=100, flat start, p=1 | 97,194 | 6114 | 6466 | | bidiagonal n=100, flat start, p=2 | >10^5 | 7610 | 7651 | | cyclic bidiagonal n=50, p=1 | 19,630 | 2063 | 739 | | cyclic bidiagonal n=200, p=2 | >10^5 | 27,464 | 27,459 | | cyclic tridiagonal n=100, p=2 | 82,699 | 940 | 940 | | integral equation n=100, p=1 | >10^5 | 16,220 | 2217 | | sparse lognormal 200x200 3%, flat start, p=2 | 3557 | 223 | 241 | Assisted-by: Claude Opus 5.5 <noreply@anthropic.com>
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #88 +/- ##
==========================================
+ Coverage 98.79% 98.84% +0.04%
==========================================
Files 23 23
Lines 6302 6303 +1
==========================================
+ Hits 6226 6230 +4
+ Misses 76 73 -3 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Overrelaxation amplifies roundoff in the imbalance by about `1/(2 - ω)`. On the 100x100 bidiagonal matrix from a flat start with `p = 2`, the factor reaches its cap and the imbalance stays near 1e-12. `tol = 1e-14` is reached only after windows without progress halve `ω - 1`, which is the one place the default solver relies on that rule. Assisted-by: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The asymmetric power-mean sweeps overrelax by a factor
ωestimated from plain sweeps and raised from the rates of later windows. Two defects in that rule made some solves far slower than plain sweeps.An
ωset on a plateau was never lowered. Far from the fixed point the imbalance can stay nearly constant for several sweeps before it contracts quickly. Anωestimated there is near its cap, and above the optimum the relaxed rate isω - 1whatever the plain rate is, so the windows read it as steady progress. The plain estimate now waits while the contraction is still improving from one sweep to the next, and setsωno higher than 1.5, where a relaxed sweep contracts by at least half even above the optimum. Windows raise it from there.A window was judged against the smallest imbalance of all earlier windows. The imbalance is the size of the plain step, which a relaxed iterate overshoots by
ω - 1times, so its level rises withω, and windows at a raisedωcould show no progress while the iteration converged. Those windows tripped the stall rule, which halvesω - 1when it should rise. Each window is now judged against the window before it. The first window afterωchanges is still judged against one at the oldωand reads as slow, so one window shrinks2 - ωby at most a factor of 8.The handoff to Newton after the plain estimate is predicted at the rate of the factor that estimate implies, as before, not at the capped factor the first relaxed sweeps use.
On 1200 random matrices of 2 to 8 rows and columns (lognormal entries with σ of 1, 3 or 6, up to half zeroed), with
pof 1 and 2 and the default and flat starts, 4800 runs in all. Withnewton = false, on the 4663 runs that plain sweeps finish in 50,000:The 3 remaining stalls are runs that neither rule finishes in 10^5 sweeps. With
newton = true:Sweeps with
newton = falseon larger problems, plain / before / after:Assisted-by: Claude Opus 5.5 noreply@anthropic.com