Skip to content

Remove covered derived intervals after propagation completes - #1073

Merged
WaylandYang merged 3 commits into
deeplethe:mainfrom
Floating-Y:fix/derived-interval-cleanup
Oct 4, 2026
Merged

WaylandYang merged 3 commits into
deeplethe:mainfrom
Floating-Y:fix/derived-interval-cleanup

Conversation

@Floating-Y

Copy link
Copy Markdown
Contributor

Why

When a narrow derivation arrives before a wider derivation of the same triple, both remain in the final output. A sub-property derivation of A q D [10,20) followed by a transitive derivation over [0,100) reproduces this issue.

This follows up on the residual issue recorded in the #1034 closing comment and the cleanup suggested in the #1040 review.

What changes

After propagation completes, group outputs by (predicate, subject, object) and reuse span_contains to remove intervals covered by another single output. Equal intervals retain one output; incomparable intervals remain separate without merging or union-based coverage. Existing unbounded interval semantics are preserved.

The surviving Derived retains its complete ordered premises, via, and rule. The reproduction now returns only [0,100), supported by premises 102, 103, 104, with via = q and rule = Transitive.

Cleanup preserves #1051's propagation behavior, including narrower but shorter intermediate proofs needed to finish within MAX_DEPTH. Public interfaces, depth limits, and output-cap semantics remain unchanged.

Seven regression tests cover arrival order, input permutations and UUID ordering, interval boundaries, proof preservation, equal-output cleanup, and the narrower-shorter-proof case.

How it was checked

  • The narrow-first regression failed before the fix and passed afterward.
  • cargo test --locked -p utopia-reason: 102 passed.
  • cargo fmt --all --check: passed.
  • cargo clippy --workspace --all-targets -- -D warnings: passed.
  • cargo test --workspace: 1,426 passed, 0 failed, 6 ignored.
  • cargo build --workspace: passed.
  • Under web/, frozen-lockfile installation, 228 tests, and the build passed.

Workspace tests used a dedicated migrated PostgreSQL 16/pgvector database with UTOPIA_TEST_REQUIRE_DB=1 and UTOPIA_TEST_REQUIRE_PDFTOTEXT=1. An initial run encountered Windows proxy interference; the full suite passed after rerunning with loopback addresses in the test process's NO_PROXY.

Ignored tests, unconfigured external-service integration tests, and the opt-in benchmark were not executed.

Before review

  • Every commit is signed off (git commit -s)
  • cargo fmt --all --check, cargo clippy --workspace --all-targets -- -D warnings and cargo test --workspace pass

WaylandYang and others added 3 commits October 3, 2026 16:46
Brings main's merge commit of deeplethe#1069 into dev so dev is up to date with main.

Signed-off-by: Wayland Yang <wayland0916@gmail.com>
Signed-off-by: Floating-Y <118035379+Floating-Y@users.noreply.github.com>

@WaylandYang WaylandYang left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @Floating-Y. This is the cleanup the #1040 review left open, and doing it after propagation rather than during it is the right place: a narrower output with a shorter proof still has to stay in the adjacency lists while chains are being extended, which your last test pins.

Beyond the 102 tests, I compared dev with this branch on 400 deterministic random graphs (transitive, sub-property and inverse predicates, bounded, half-open and open intervals):

dev this branch
Triples derived 19,028 19,028, the same set
Outputs 34,581 25,523
Outputs covered by another output of the same triple 9,058 0
Intervals on dev not covered by an output here 0

So it removes exactly the covered outputs, a quarter of them on these graphs, and loses no interval. The store's derivation tests pass against a migrated database here as well.

One thing to know, not to change: a removed narrow output has already taken a slot of the per-predicate cap, so a predicate can report as capped with fewer than 20,000 outputs left. That errs on the side of saying so. Landing it.

@WaylandYang
WaylandYang merged commit 0197bd8 into deeplethe:main Oct 4, 2026
6 checks passed
@WaylandYang WaylandYang mentioned this pull request Oct 4, 2026
q-qp-p pushed a commit to q-qp-p/deeplethe-utopia that referenced this pull request Oct 4, 2026
Signed-off-by: Wayland Yang <wayland0916@gmail.com>

deeplethe#1073 was opened against main and merged there. This brings it into dev.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants