Repository navigation
Conversation
A predicate such as time >= CAST(1000 AS TIMESTAMP) failed with error 301 (GenericLiteral cannot be cast to LongLiteral), and an IN list of such values with error 305. The constant is folded into a TIMESTAMP GenericLiteral, which ConvertPredicateToTimeFilterVisitor cast to LongLiteral. Read its value as a long in the current precision, as the TIMESTAMP field path in TypeServices already does. Signed-off-by: Zihan Dai <1436286758@qq.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
In the table model, a predicate on the time column fails when the value is a constant TIMESTAMP expression instead of a plain number. On 2.0.11 and on current master:
=,BETWEENand the literal-on-the-left form fail the same way andINfails with 305, whiletime >= 1704067201000works, and so does the CAST form against a TIMESTAMP field.Cause
The constant is folded by
LiteralEncoderintoGenericLiteral("TIMESTAMP", "<long>").GlobalTimePredicateExtractVisitoronly treats aLongLiteralas a global time filter, so the predicate stays in the scan, whereConvertPredicateToFilterVisitorhands it toConvertPredicateToTimeFilterVisitor. ItsgetLongValuecasts the value toLongLiteral(the server stack ends atConvertPredicateToTimeFilterVisitor.getLongValue, line 320 on 2.0.11), andvisitInPredicatechecks and casts the same way. The TIMESTAMP-field path already accepts this literal (TypeServices.getTimestampValue).Change
ConvertPredicateToTimeFilterVisitor.getLongValueandvisitInPredicatealso accept aGenericLiteralof typeTIMESTAMP, reading its value as the long in the current precision, asTypeServices.getTimestampValuedoes. Other literal types behave as before.Not changed here:
GlobalTimePredicateExtractVisitor.checkIsTimeFilterstill only extractsLongLiteralvalues, so the CAST form is evaluated as a scan filter rather than used for time-partition pruning. The result is correct either way; I left that out to keep this change to the failure.#15662 (open) also edits
getLongValue, forDoubleLiteral; whichever of the two lands second needs a small merge there.Verification
TablePredicateConversionTest#testTimestampConstantAndTimeColumn(new) converts>=, a literal on the left,INandBETWEENwith TIMESTAMP literals on the time column. All 8 tests in the class pass. With thegetLongValuepart reverted the new test fails with the sameClassCastException; with only thevisitInPredicatepart reverted it fails at theINcase withIllegalArgumentException.queryengine/plan/relationalandqueryengine/plan/plannerthat cover predicates and time filters (25 classes, 348 tests) pass, and so do 17 table-model ITs with time predicates inWHERE, among themIoTDBInTableIT,IoTDBCastFunctionTableITandIoTDBFilterBetweenTableIT(334 tests,-PTableSimpleIT).spotless:checkand thewith-zh-localecompile pass.time >= CAST(2000 AS TIMESTAMP)time = CAST(2000 AS TIMESTAMP)time IN (CAST(1000 AS TIMESTAMP), CAST(3000 AS TIMESTAMP))time BETWEEN CAST(1500 AS TIMESTAMP) AND CAST(2500 AS TIMESTAMP)CAST(2000 AS TIMESTAMP) < timetime >= 2000(control)NOT INandNOT BETWEENwith CAST constants already returned the right rows and still do.I found this through the table-model Flink connector on the
flink-iotdb-table-connectorbranch of iotdb-extras, which rendered its time filters in the CAST form (changed there in apache/iotdb-extras#142); a user writing the CAST form by hand hits the same error.This PR has: