Skip to content

Physical-plan serialization drops ProjectionExec metadata #24695

Description

@gene-bordegaray

Physical-plan serialization

Before ProjectionExecNode serialized:

PhysicalPlanNode input = 1;
repeated PhysicalExprNode expr = 2;
repeated string expr_name = 3;

On decode, DataFusion reconstructed the projection which is fine when the complete output schema can be derived from the expression and input but metadata cannot be.

Given:

Before serialization:

Input field:
  value: Int32
  metadata: {}

Projection expression:
  value@0 AS value

ProjectionExec output schema:
  value: Int32
  field metadata: {"field-key": "field-value"}
  schema metadata: {"schema-key": "schema-value"}

---

After serialization and decoding:

ProjectionExec output schema:
  value: Int32
  field metadata: {}
  schema metadata: {}

this was roginally found in #24670 but not fixed there since it was out of optimizer scope / unrelated bug

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions