Conversation
Generate a finite half-open bin list in timeutil, hard-capped at 1500, so looping OLAP dialects cannot hang or allocate unbounded SQL when start and end are not grain-aligned.
nishantmonu51
left a comment
There was a problem hiding this comment.
The shared TimeRangeBins is the right fix: it replaces the only non-terminating loop on main (StarRocks' t != end) with a bounded half-open iteration, and erroring on TimeGrainUnspecified also closes the case where OffsetTime is a no-op and ApproximateBins returns -1, which the ast.go pre-check does not catch.
| if len(bins) >= MaxTimeRangeBins { | ||
| return nil, fmt.Errorf("time range has more than %d bins for %q grain, move to a larger grain", MaxTimeRangeBins, TimeGrainToAPI(tg)) |
There was a problem hiding this comment.
ApproximateBins divides the raw end - start span while this cap counts bins from the truncated start, so the two limits disagree by one at the boundary. With hour grain, start 2020-01-01T00:30Z and end start + 1500h, ApproximateBins returns 1500 and passes the bins > 1500 pre-check in runtime/metricsview/ast.go:1163 (now 1217 on main), and the request then fails here with a second 1500-bin error; on ClickHouse, Druid, Pinot and Snowflake that request previously succeeded with 1501 rows. The two errors also spell the grain differently, "TIME_GRAIN_HOUR" here versus "hour" in ast.go, so users can see two wordings of the same limit. Either have ast.go use MaxTimeRangeBins and drop the ApproximateBins pre-check for these dialects, or count the truncated start in the pre-check, so one limit and one message govern.
SelectTimeRangeBinsin the starrocks, clickhouse, pinot, druid, and snowflake dialects looped forever when the truncated start never equalled an unaligned end (t != end), allocating unbounded SQL until the process was OOM-killed.timeutil.TimeRangeBinsnow truncates the start, iterates half-open witht.Before(end), and errors above 1500 actual bins. Those dialects only format that slice.ApproximateBinspre-check inast.gostays as a fast-fail.fillMissingrequest: HTTP 200 in under a second with flat memory (previously hung until timeout while resident memory grew to tens of GB).Checklist:
Developed in collaboration with Claude Code