Skip to content

fix(runtime): terminate fill-missing time spines for unaligned bounds - #9835

Open
eminemead wants to merge 1 commit into
rilldata:mainfrom
eminemead:xiaofei.yin/fix-time-spine-termination
Open

eminemead wants to merge 1 commit into
rilldata:mainfrom
eminemead:xiaofei.yin/fix-time-spine-termination

Conversation

@eminemead

@eminemead eminemead commented Aug 26, 2026

Copy link
Copy Markdown
  • SelectTimeRangeBins in the starrocks, clickhouse, pinot, druid, and snowflake dialects looped forever when the truncated start never equalled an unaligned end (t != end), allocating unbounded SQL until the process was OOM-killed.
  • A shared timeutil.TimeRangeBins now truncates the start, iterates half-open with t.Before(end), and errors above 1500 actual bins. Those dialects only format that slice.
  • DuckDB / BigQuery / Databricks native-SQL paths are unchanged; the ApproximateBins pre-check in ast.go stays as a fast-fail.
  • Live-verified on an unaligned UTC day fillMissing request: HTTP 200 in under a second with flat memory (previously hung until timeout while resident memory grew to tens of GB).

Checklist:

  • Covered by tests
  • Ran it and it works as intended
  • Reviewed the diff before requesting a review
  • Checked for unhandled edge cases
  • Linked the issues it closes
  • Checked if the docs need to be updated. If so, create a separate Linear DOCS issue
  • Intend to cherry-pick into the release branch
  • I'm proud of this work!

Developed in collaboration with Claude Code

Generate a finite half-open bin list in timeutil, hard-capped at 1500, so looping OLAP dialects cannot hang or allocate unbounded SQL when start and end are not grain-aligned.
@nishantmonu51 nishantmonu51 added Type:Bug Something isn't working Area:Time Size:M Medium change: 100-499 lines labels Sep 1, 2026

@nishantmonu51 nishantmonu51 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The shared TimeRangeBins is the right fix: it replaces the only non-terminating loop on main (StarRocks' t != end) with a bounded half-open iteration, and erroring on TimeGrainUnspecified also closes the case where OffsetTime is a no-op and ApproximateBins returns -1, which the ast.go pre-check does not catch.

Comment on lines +206 to +207
if len(bins) >= MaxTimeRangeBins {
return nil, fmt.Errorf("time range has more than %d bins for %q grain, move to a larger grain", MaxTimeRangeBins, TimeGrainToAPI(tg))

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ApproximateBins divides the raw end - start span while this cap counts bins from the truncated start, so the two limits disagree by one at the boundary. With hour grain, start 2020-01-01T00:30Z and end start + 1500h, ApproximateBins returns 1500 and passes the bins > 1500 pre-check in runtime/metricsview/ast.go:1163 (now 1217 on main), and the request then fails here with a second 1500-bin error; on ClickHouse, Druid, Pinot and Snowflake that request previously succeeded with 1501 rows. The two errors also spell the grain differently, "TIME_GRAIN_HOUR" here versus "hour" in ast.go, so users can see two wordings of the same limit. Either have ast.go use MaxTimeRangeBins and drop the ApproximateBins pre-check for these dialects, or count the truncated start in the pre-check, so one limit and one message govern.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Area:Time Size:M Medium change: 100-499 lines Type:Bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants