Skip to content

to_timestamp with string input returns an array whose timezone does not match its declared return type #24632

Description

@adriangb

Describe the bug

When datafusion.execution.time_zone is set, to_timestamp() on a string input declares a
return type of Timestamp(ns, <tz>) but produces an array of Timestamp(ns, None). The
declared type and the produced array disagree, so any plan that materializes the column
fails at execution. Planning succeeds — EXPLAIN returns a plan for both queries below.

To Reproduce

datafusion-cli 55.0.0:

SET datafusion.execution.time_zone = 'UTC';
SELECT to_timestamp(s) AS v FROM (VALUES ('2026-01-01T00:00:00')) AS t(s);
Error: Arrow error: Invalid argument error: column types must match schema types,
expected Timestamp(ns, "UTC") but found Timestamp(ns) at column index 0

Not specific to UTC:

SET datafusion.execution.time_zone = 'America/New_York';
SELECT to_timestamp(s) AS v FROM (VALUES ('2026-01-01T00:00:00')) AS t(s);
-- expected Timestamp(ns, "America/New_York") but found Timestamp(ns) at column index 0

A second symptom: because the planner trusts the declared type, it inserts no coercion
before comparing against another timezone-aware value, so the comparison reaches the Arrow
kernel with mismatched types:

SET datafusion.execution.time_zone = 'UTC';
SELECT to_timestamp(concat('2026-01-01T00:00:0', (value % 10)::text)) <= now() AS cmp
FROM generate_series(1, 3) AS t(value);
Error: Arrow error: Invalid argument error:
Invalid comparison operation: Timestamp(ns) <= Timestamp(ns, "UTC")

Unaffected, for contrast:

  • Numeric input: to_timestamp(1767225600) returns Timestamp(ns, "UTC"); declared and
    actual agree.
  • Default config (datafusion.execution.time_zone unset): declared is Timestamp(ns, None)
    and matches the array, so nothing fails.

Expected behavior

to_timestamp on a string input should produce an array annotated with the execution
timezone, matching what return_type advertises.

return_type returns Timestamp(Nanosecond, self.timezone.clone()), and the string branch
does receive the configured timezone (it is passed into the parse helper). The gap appears
to be that the resulting array is not annotated with it.

Additional context

Reproduced on datafusion-cli 54.0.0 and 55.0.0.

Related:

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions