Skip to content

Accept constant TIMESTAMP expressions in table-model time filters - #19069

Open
PDGGK wants to merge 1 commit into
apache:masterfrom
PDGGK:fix/time-filter-cast-literal
Open

PDGGK wants to merge 1 commit into
apache:masterfrom
PDGGK:fix/time-filter-cast-literal

Conversation

@PDGGK

@PDGGK PDGGK commented Nov 1, 2026

Copy link
Copy Markdown
Contributor

Description

In the table model, a predicate on the time column fails when the value is a constant TIMESTAMP expression instead of a plain number. On 2.0.11 and on current master:

SELECT temperature FROM sensor WHERE time >= CAST(1704067201000 AS TIMESTAMP)
-- 301: class ...sql.ast.GenericLiteral cannot be cast to class ...sql.ast.LongLiteral

=, BETWEEN and the literal-on-the-left form fail the same way and IN fails with 305, while time >= 1704067201000 works, and so does the CAST form against a TIMESTAMP field.

Cause

The constant is folded by LiteralEncoder into GenericLiteral("TIMESTAMP", "<long>"). GlobalTimePredicateExtractVisitor only treats a LongLiteral as a global time filter, so the predicate stays in the scan, where ConvertPredicateToFilterVisitor hands it to ConvertPredicateToTimeFilterVisitor. Its getLongValue casts the value to LongLiteral (the server stack ends at ConvertPredicateToTimeFilterVisitor.getLongValue, line 320 on 2.0.11), and visitInPredicate checks and casts the same way. The TIMESTAMP-field path already accepts this literal (TypeServices.getTimestampValue).

Change

ConvertPredicateToTimeFilterVisitor.getLongValue and visitInPredicate also accept a GenericLiteral of type TIMESTAMP, reading its value as the long in the current precision, as TypeServices.getTimestampValue does. Other literal types behave as before.

Not changed here: GlobalTimePredicateExtractVisitor.checkIsTimeFilter still only extracts LongLiteral values, so the CAST form is evaluated as a scan filter rather than used for time-partition pruning. The result is correct either way; I left that out to keep this change to the failure.

#15662 (open) also edits getLongValue, for DoubleLiteral; whichever of the two lands second needs a small merge there.

Verification

  • TablePredicateConversionTest#testTimestampConstantAndTimeColumn (new) converts >=, a literal on the left, IN and BETWEEN with TIMESTAMP literals on the time column. All 8 tests in the class pass. With the getLongValue part reverted the new test fails with the same ClassCastException; with only the visitInPredicate part reverted it fails at the IN case with IllegalArgumentException.
  • With the change, the datanode test classes under queryengine/plan/relational and queryengine/plan/planner that cover predicates and time filters (25 classes, 348 tests) pass, and so do 17 table-model ITs with time predicates in WHERE, among them IoTDBInTableIT, IoTDBCastFunctionTableIT and IoTDBFilterBetweenTableIT (334 tests, -PTableSimpleIT). spotless:check and the with-zh-locale compile pass.
  • On standalone servers built from master 8a5e5ca without and with this change, with rows at time 1000, 2000 and 3000:
query master with this change
time >= CAST(2000 AS TIMESTAMP) 301 2000, 3000
time = CAST(2000 AS TIMESTAMP) 301 2000
time IN (CAST(1000 AS TIMESTAMP), CAST(3000 AS TIMESTAMP)) 305 1000, 3000
time BETWEEN CAST(1500 AS TIMESTAMP) AND CAST(2500 AS TIMESTAMP) 301 2000
CAST(2000 AS TIMESTAMP) < time 301 3000
time >= 2000 (control) 2000, 3000 2000, 3000

NOT IN and NOT BETWEEN with CAST constants already returned the right rows and still do.

I found this through the table-model Flink connector on the flink-iotdb-table-connector branch of iotdb-extras, which rendered its time filters in the CAST form (changed there in apache/iotdb-extras#142); a user writing the CAST form by hand hits the same error.


This PR has:

  • been self-reviewed.
  • added unit tests or modified existing tests to cover new code paths, ensuring the threshold for code coverage.

A predicate such as time >= CAST(1000 AS TIMESTAMP) failed with error 301
(GenericLiteral cannot be cast to LongLiteral), and an IN list of such
values with error 305. The constant is folded into a TIMESTAMP
GenericLiteral, which ConvertPredicateToTimeFilterVisitor cast to
LongLiteral. Read its value as a long in the current precision, as the
TIMESTAMP field path in TypeServices already does.

Signed-off-by: Zihan Dai <1436286758@qq.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant