Reject interval scaling that breaches a sampled-ancestor ceiling - #134
Merged
Conversation
A fake node's height is pinned to its direct-ancestor leaf's sampling
time, so it cannot move and acts as a fixed ceiling on the subtree
hanging below it. The interval-scaling recursion scaled that subtree
without checking the ceiling, so a large enough scale factor lifted a
node through its own sampled ancestor and left a child above its parent.
Downstream code does not survive that state: MRCAPrior.getCommonAncestor
spins forever walking parent pointers, and TreeLikelihood.traverse blows
the stack. On the SA package's bears.xml example this presented as a
hang, a StackOverflowError or an infinite posterior depending on which
operators were active.
The room between the ceiling and the fixed leaves below is finite, so
such a move has no valid state to land in and must be rejected rather
than fixed up. Node.intervalScale now throws IllegalArgumentException,
which is the rejection signal the Scalable contract already specifies
and which every scale operator already catches. The check is inline on
the existing traversal, so trees with no sampled ancestors pay nothing;
a leaf child is skipped, since a leaf cannot move and a violation there
would predate the move.
The recursion existed in three copies and only Node.intervalScale was
checked, which is why this survived. UpDownOperator and
IntervalScaleOperator now delegate to Node.intervalScale instead of
keeping private copies; IntervalScaleOperator.proposal also gained the
try/catch it was missing, so a rejection no longer kills the chain.
Tested:
* TreeScalableTest goes from 5 to 17 tests and a new
SampledAncestorScalingTest adds 3 more, driving all three operators
over 2000 proposals each and asserting the tree is never left
invalid. Reverting the three main-source files fails 7 of the 15 new
tests (4 in TreeScalableTest, all 3 in SampledAncestorScalingTest);
the rest cover behaviour the fix preserves rather than changes.
* beast-base suite on a clean build: 477 tests, 0 failures, 3 skipped
* sampled-ancestors bears.xml and bears_ranges.xml now run to
completion; both previously hung or crashed
Addresses: #78
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Interval scaling could lift a node through its own sampled ancestor, leaving a tree with a child above its parent. Downstream code does not survive that state:
MRCAPrior.getCommonAncestorspins forever walking parent pointers, andTreeLikelihood.traverseblows the stack.A fake node's height is pinned to its direct-ancestor leaf's sampling time, so it cannot move and acts as a fixed ceiling on the subtree below it. The room between that ceiling and the fixed leaves underneath is finite, so a large enough scale factor has nowhere valid to land.
Node.intervalScalenow throwsIllegalArgumentException— the rejection signal theScalablecontract already specifies, and which every scale operator already catches.The recursion existed in three copies and only
Node.intervalScalewas checked, which is why this survived.UpDownOperatorandIntervalScaleOperatornow delegate to it instead of keeping private copies.IntervalScaleOperator.proposalalso gained the try/catch it was missing, so a rejection no longer kills the chain.Closes #78.
Impact on analyses without sampled ancestors
None measurable. The check sits inside
if (isFake()), a branch these trees never enter, and that call was already there.Tree.scalemicrobenchmark, this branch vs master, best of 3 runs:Re-running with alternating order flipped the sign, so this is noise under 1%.
Results are bit-identical: reproducing the deleted
resampleNodeHeightverbatim and comparing againstNode.intervalScaleover 500 random trees (ultrametric and heterochronous, 4–200 tips, scale factors 0.2–4.2) matches 102,188 node heights ondoubleToLongBits, with identical dof counts.One behavioural delta worth review: the deleted copies used
node.setHeight(), which also marks both children dirty, whereasNode.intervalScalemarks only the node itself. This is safe because all four likelihood implementations (legacy and spec,TreeLikelihoodandBeagleTreeLikelihood) gate recomputation onupdate != Tree.IS_CLEAN || branchTime != m_branchLengths[nodeIndex], so a changed branch length is picked up regardless of the flag.Node.isDirty()has no other consumers in core, andScaleOperator/ScaleTreeOperatoralready went throughintervalScale, so this makesUpDownOperatorconsistent with the path already in production.Hastings ratio
Rejecting is the correct acceptance probability rather than an approximation: the invalid tree is outside the model's support, so π = 0 and α = 0. Two properties make that sound, both covered by tests:
scale(1/s)can never fail, and interval scaling is exactly invertible (leaf and fake-node heights are never touched).dofcounts non-fake internal nodes; scaling changes no topology, moves no fake node, and cannot create a new zero-length tip branch, so the fake-node set is invariant.Rejections feed
logAlpha = -InfinityintocalcDelta, whereMath.exp(Math.min(logAlpha, 0))is 0 — finite, no NaN — giving Robbins-Monro the intended downward nudge on the scale factor.Zero-length branches outside an FBD analysis
isFake()/isDirectAncestor()are decided purely by height equality and never consult the tree prior, so this applies to any analysis. Both cases are now covered by tests:Both are transient in practice: a node-height operator is additive and steps off the state on the first proposal that touches the node, after which nothing returns to it (measure zero under a continuous prior). TreeParser binarizing a polytomy produces zero-length internal edges only, so consensus and constraint starting trees never reach the ceiling path.
Test plan
TreeScalableTest5 → 17 tests; newSampledAncestorScalingTestadds 3, driving all three operators over 2000 proposals each and asserting the tree is never left invalidTreeScalableTest, all 3 inSampledAncestorScalingTest); the remaining 8 cover behaviour this PR preserves rather than changesbears.xmlandbears_ranges.xmlrun 200k states to completion; both previously hung or crashed at sample 0Tree.scalefrom initialisation code without a try/catch would abort rather than reject if handed a tree with sampled ancestors.BEASTLabs.TreeScaleOperatorcatches and is fine;starbeast3.StarBeastStartStatehas three uncaught call sites, though species trees are not SA trees. Both still use the pre-Scale/UpDown operators rely on StateNode.scale() -- replace by Scalable interface #20intreturn signature and do not compile against current beast3 regardless.Follow-ups, not in this PR
sampled-ancestorsneeds its example XMLs repaired and itsbeast.versionbumped once this is released; its examples currently fail against beta5 and beta7 for exactly this bug.Tree.allowSampledAncestors()has had no callers since the Constraints PR that used it was reverted in fc77f3c. BEAST infers "sampled ancestor" from exact height equality with no flag, which is what makes the zero-length-tip case above ambiguous.