Skip to content

Guard the power-mean relaxation factor estimate - #88

Merged
timholy merged 2 commits into
mainfrom
teh/powermean-relax
Oct 13, 2026
Merged

timholy merged 2 commits into
mainfrom
teh/powermean-relax

Conversation

@timholy

@timholy timholy commented Oct 13, 2026

Copy link
Copy Markdown
Member

The asymmetric power-mean sweeps overrelax by a factor ω estimated from plain sweeps and raised from the rates of later windows. Two defects in that rule made some solves far slower than plain sweeps.

An ω set on a plateau was never lowered. Far from the fixed point the imbalance can stay nearly constant for several sweeps before it contracts quickly. An ω estimated there is near its cap, and above the optimum the relaxed rate is ω - 1 whatever the plain rate is, so the windows read it as steady progress. The plain estimate now waits while the contraction is still improving from one sweep to the next, and sets ω no higher than 1.5, where a relaxed sweep contracts by at least half even above the optimum. Windows raise it from there.

A window was judged against the smallest imbalance of all earlier windows. The imbalance is the size of the plain step, which a relaxed iterate overshoots by ω - 1 times, so its level rises with ω, and windows at a raised ω could show no progress while the iteration converged. Those windows tripped the stall rule, which halves ω - 1 when it should rise. Each window is now judged against the window before it. The first window after ω changes is still judged against one at the old ω and reads as slow, so one window shrinks 2 - ω by at most a factor of 8.

The handoff to Newton after the plain estimate is predicted at the rate of the factor that estimate implies, as before, not at the capped factor the first relaxed sweeps use.

On 1200 random matrices of 2 to 8 rows and columns (lognormal entries with σ of 1, 3 or 6, up to half zeroed), with p of 1 and 2 and the default and flat starts, 4800 runs in all. With newton = false, on the 4663 runs that plain sweeps finish in 50,000:

before after
sweeps 552,513 254,336
runs more than 2 sweeps slower than plain 213 9
runs where the stall rule fires (of 4800) 55 3

The 3 remaining stalls are runs that neither rule finishes in 10^5 sweeps. With newton = true:

before after
runs handed to Newton 949 974
passes over the support 317,998 302,284
factorizations 5130 5227

Sweeps with newton = false on larger problems, plain / before / after:

problem plain before after
bidiagonal n=100, flat start, p=1 97,194 6114 6466

Assisted-by: Claude Opus 5.5 noreply@anthropic.com

The asymmetric power-mean sweeps overrelax by a factor `ω` estimated
from plain sweeps and raised from the rates of later windows. Two
defects in that rule made some solves far slower than plain sweeps.

An `ω` set on a plateau was never lowered. Far from the fixed point the
imbalance can stay nearly constant for several sweeps before it
contracts quickly. An `ω` estimated there is near its cap, and above
the optimum the relaxed rate is `ω - 1` whatever the plain rate is, so
the windows read it as steady progress. The plain estimate now waits
while the contraction is still improving from one sweep to the next,
and sets `ω` no higher than 1.5, where a relaxed sweep contracts by at
least half even above the optimum. Windows raise it from there.

A window was judged against the smallest imbalance of all earlier
windows. The imbalance is the size of the plain step, which a relaxed
iterate overshoots by `ω - 1` times, so its level rises with `ω`, and
windows at a raised `ω` could show no progress while the iteration
converged. Those windows tripped the stall rule, which halves `ω - 1`
when it should rise. Each window is now judged against the window
before it. The first window after `ω` changes is still judged against
one at the old `ω` and reads as slow, so one window shrinks `2 - ω` by
at most a factor of 8.

The handoff to Newton after the plain estimate is predicted at the rate
of the factor that estimate implies, as before, not at the capped
factor the first relaxed sweeps use.

On 1200 random matrices of 2 to 8 rows and columns (lognormal entries
with σ of 1, 3 or 6, up to half zeroed), with `p` of 1 and 2 and the
default and flat starts, 4800 runs in all. With `newton = false`, on
the 4663 runs that plain sweeps finish in 50,000:

| | before | after |
|---|---|---|
| sweeps | 552,513 | 254,336 |
| runs more than 2 sweeps slower than plain | 213 | 9 |
| worst ratio to plain | 85 | 1.73 |
| runs where the stall rule fires (of 4800) | 55 | 3 |
| runs plain does not finish that the rule does | 87 of 137 | 101 of 137 |

The 3 remaining stalls are runs that neither rule finishes in 10^5
sweeps. With `newton = true`:

| | before | after |
|---|---|---|
| runs handed to Newton | 949 | 974 |
| passes over the support | 317,998 | 302,284 |
| factorizations | 5130 | 5227 |

Sweeps with `newton = false` on larger problems, plain / before / after:

| problem | plain | before | after |
|---|---|---|---|
| bidiagonal n=100, flat start, p=1 | 97,194 | 6114 | 6466 |
| bidiagonal n=100, flat start, p=2 | >10^5 | 7610 | 7651 |
| cyclic bidiagonal n=50, p=1 | 19,630 | 2063 | 739 |
| cyclic bidiagonal n=200, p=2 | >10^5 | 27,464 | 27,459 |
| cyclic tridiagonal n=100, p=2 | 82,699 | 940 | 940 |
| integral equation n=100, p=1 | >10^5 | 16,220 | 2217 |
| sparse lognormal 200x200 3%, flat start, p=2 | 3557 | 223 | 241 |

Assisted-by: Claude Opus 5.5 <noreply@anthropic.com>
@codecov

codecov Bot commented Oct 13, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 92.85714% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 98.84%. Comparing base (cb4bc5e) to head (bb6cf5a).

Files with missing lines Patch % Lines
src/powermean.jl 92.85% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main      #88      +/-   ##
==========================================
+ Coverage   98.79%   98.84%   +0.04%     
==========================================
  Files          23       23              
  Lines        6302     6303       +1     
==========================================
+ Hits         6226     6230       +4     
+ Misses         76       73       -3     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Overrelaxation amplifies roundoff in the imbalance by about
`1/(2 - ω)`. On the 100x100 bidiagonal matrix from a flat start with
`p = 2`, the factor reaches its cap and the imbalance stays near 1e-12.
`tol = 1e-14` is reached only after windows without progress halve
`ω - 1`, which is the one place the default solver relies on that rule.

Assisted-by: Claude Opus 5.5 <noreply@anthropic.com>
@timholy
timholy merged commit e1950d5 into main Oct 13, 2026
4 of 5 checks passed
@timholy
timholy deleted the teh/powermean-relax branch October 13, 2026 19:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant