Skip to content

Creating Iceberg table does not use ORDER BY as sort order #1935

Description

@xieandrew

Describe the bug

Creating a table from ClickHouse using the Iceberg table engine and ORDER BY does not show
those columns as part of sort order in ice. The Iceberg table is created in a database
with DataLakeCatalog pointing to ice-rest-catalog.

To Reproduce
Steps to reproduce the behavior:

  1. Ensure an instance of ice-rest-catalog is running

  2. Create ice database

CREATE DATABASE ice
  ENGINE = DataLakeCatalog('http://ice-rest-catalog:5000')
  SETTINGS catalog_type = 'rest',
    auth_header = 'Authorization: Bearer your-auth-token', 
    warehouse = 's3://your-warehouse';
  1. Create Iceberg table with ORDER BY
SET write_full_path_in_iceberg_metadata = 1;

CREATE TABLE ice.`test.sorting`
(
    `id` Int64,
    `name` String
)
ENGINE = Iceberg('s3://your-warehouse/test/sorting')
ORDER BY (id, name)
  1. Check the schema using ice and see the sort order is missing
ice describe -s test.sorting

---
- kind: Table
  metadata:
    id: test.sorting
  data:
    schemaRaw: |-
      table {
        1: id: required long
        2: name: required string
      }
    partitionSpecRaw: "[]"
    sortOrderRaw: "[]"
    location: s3://your-warehouse/test/sorting

Expected behavior
The sortOrder should use the ORDER BY from the ClickHouse CREATE TABLE statement.

Creating the table using ice works as intended:

ice create-table --schema-from-parquet=test-sorting.parquet --sort='[{"column":"id"},{"column":"name"}]' test.sorting
ice describe -s test.sorting
---
- kind: Table
  metadata:
    id: test.sorting
  data:
    schemaRaw: |-
      table {
        1: id: required long
        2: name: required string
      }
    partitionSpecRaw: "[]"
    sortOrderRaw: |-
      [
        identity(1) ASC NULLS LAST
        identity(2) ASC NULLS LAST
      ]
    location: s3://your-warehouse/test/sorting

Screenshots
N/A

Key information
Provide relevant runtime details.

  • 26.3.10.20001.altinityantalya
  • Altinity Cloud environment
  • S3 Storage
  • ice-rest-catalog

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions