Skip to content

MATH-1678: Avoid Canberra distance overflow - #336

Open
hutiefang76 wants to merge 3 commits into
apache:masterfrom
hutiefang76:MATH-1678-canberra-overflow
Open

hutiefang76 wants to merge 3 commits into
apache:masterfrom
hutiefang76:MATH-1678-canberra-overflow

Conversation

@hutiefang76

Copy link
Copy Markdown

CanberraDistance can return NaN for opposite finite extremes and zero for same-sign large values because intermediate arithmetic overflows. This fixes MATH-1678 by halving the coordinates only when their finite magnitudes overflow the denominator. Ordinary values and the existing NaN/infinity behavior are preserved. Regressions cover both signs, vectors, near-equal large values against an independent BigDecimal reference, subnormals and signed zero.

Validation:

  • mvn -B clean verify: all 14 reactor modules completed successfully; archived Surefire results contain 3,797 tests, with 33 existing skips and no final failures/errors.
  • The first full run failed in the existing randomized SimplexOptimizerTest. That class also failed on an unmodified baseline. One complete rerun passed using the project's unchanged retry configuration; no tests or assertions were disabled.
  • A clean Java 8 build and the affected CanberraDistanceTest passed all 8 tests. The tests-only baseline reproduced 4 failures before the fix.

AI assistance: DeepSeek and Codex helped implement, review and validate this patch. This description was AI-generated.

Large finite coordinates can overflow the denominator and produce
NaN or zero instead of the distance contribution. Scale both
coordinates by one half only when their finite magnitudes overflow,
while preserving ordinary and non-finite input behavior.

Add extreme-value, independent BigDecimal reference, and special-value
regressions.

Generated-by: DeepSeek (deepseek-v4-flash), Codex
@hutiefang76

Copy link
Copy Markdown
Author

The three workflows currently show action_required with no jobs. Could a maintainer approve the CI runs when convenient?

@codecov-commenter

codecov-commenter commented Oct 23, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 83.33333% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 87.19%. Comparing base (f554608) to head (7eaa1d0).
⚠️ Report is 384 commits behind head on master.

Files with missing lines Patch % Lines
...ons/math4/legacy/ml/distance/CanberraDistance.java 83.33% 0 Missing and 1 partial ⚠️
Additional details and impacted files
@@             Coverage Diff              @@
##             master     #336      +/-   ##
============================================
+ Coverage     86.54%   87.19%   +0.64%     
+ Complexity     9787       91    -9696     
============================================
  Files           532      499      -33     
  Lines         35516    33512    -2004     
  Branches       6194     5846     -348     
============================================
- Hits          30738    29221    -1517     
+ Misses         3518     3168     -350     
+ Partials       1260     1123     -137     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@hutiefang76

Copy link
Copy Markdown
Author

The Ubuntu/JDK 17 failure is in CMAESOptimizerTest.testDiagonalRosen; testCircleFitting2 also needed a retry. Both tests and their optimizer implementations are unchanged by this PR.

I ran those same methods on pristine base a0df91d8 with Java 17 on macOS arm64. Both assertions failed there too and recovered under the existing retry configuration (2 methods, 2 flakes). This is separate baseline evidence, not a reproduction of the Ubuntu runner or a green hosted build. I have left the tests, tolerances, seeds and CI configuration unchanged.

AI-generated reply. The baseline results were checked against the actual execution log.

@aherbert aherbert left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you rebase on master it should collect the changes to the github macOS runner.

// Rescale large finite coordinates to avoid intermediate overflow.
sum += JdkMath.abs(a[i] * 0.5 - b[i] * 0.5) / (absA * 0.5 + absB * 0.5);
} else {
sum += num == 0.0 && denom == 0.0 ? 0.0 : num / denom;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think numerator can ever be non-zero if the denominator is zero. This check could be:

denom == 0.0 ? 0.0 : num / denom;

It is then testing for a[i] == b[i] == 0.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changed to denom == 0.0 ? 0.0 : num / denom in 7eaa1d03. A zero sum of the absolute values means both coordinates are zero, so the numerator check was redundant. The 8 Canberra tests pass, including signed zero and subnormal cases. mvn clean verify also passes locally on Java 17/macOS arm64 (3798 tests, 33 skips).

The branch already includes current master a9aabf60 through merge dadaf4cd, including the macos-latest and Zulu/Java 8 runner changes (56a8f696 and a9aabf60). Both are ancestors of this commit. Hosted CI still needs to run on the new head.

AI-generated reply; validation figures are from the local execution logs.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants