test: enable native path in lower/upper_enabled sql fixtures - #1
test: enable native path in lower/upper_enabled sql fixtures#1cestercian wants to merge 1 commit into
Conversation
Point the _enabled fixtures at spark.comet.caseConversion.enabled so they exercise the native case-conversion path instead of the no-op allowIncompatible setting. Fixes apache#5611 Co-authored-by: Cestercian <yashafaid@gmail.com>
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Team Run ID: 📒 Files selected for processing (2)
Included review availability: Your plan provides up to 1 included review per hour; 0 remain after this review. 📝 WalkthroughWalkthroughThe pull request updates header comments in the ChangesCase conversion test descriptions
Estimated code review effort: 1 (Trivial) | ~2 minutes Merge Risk: ⚪ Minimal · up to This localized change enables the intended native case-conversion path in two test fixtures without changing production behavior; no actionable merge-blocking risk remains after normal checks and review. Suggested reviewers: 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
Full details: Docstring CoverageExplanation No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check. Docstring coverage is scoped to functions touched by this diff. Analyzed 0 functions across 0 files. (2 skipped: 2 unsupported.) ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Which issue does this PR close?
Fixes apache#5611
Rationale for this change
lower_enabled.sqlandupper_enabled.sqlwere meant to cover the native case-conversion path, but they setspark.comet.expression.Lower/Upper.allowIncompatible=true. That config is a no-op:CometCaseConversionBasenever reportsIncompatible, soallowIncompatibleis never consulted. The real switch isspark.comet.caseConversion.enabled.The fixtures therefore exercised the same codegen dispatcher path as
lower.sql/upper.sql. ASCII inputs produce identical results on both paths, so the drift was not caught.What changes are included in this PR?
In both
spark/src/test/resources/sql-tests/expressions/string/lower_enabled.sqlandupper_enabled.sql:spark.comet.caseConversion.enabled=trueallowIncompatibleopt-inASCII inputs are unchanged.
expect_native(...)annotations are left for after apache#5610 lands.How are these changes tested?
Ran against Spark 4.1 (project default) after
make core:Both passed (
sql-file: expressions/string/lower_enabled.sqlandsql-file: expressions/string/upper_enabled.sql; 1 test each, 0 failures).The Config key matches
CometConf.COMET_CASE_CONVERSION_ENABLED. ASCIIlower/upperresults are identical on the native and dispatcher paths, so existingquerycomparisons remain valid.expect_nativeassertions are intentionally not added here; they depend on apache#5610.Summary by CodeRabbit
lower()andupper()to accurately document the native case-conversion path.