Skip to content

GH-50976: [C++][Compute] Implement casting from ListView to List with zero-copy fast-path - #50976

Open
Jay846 wants to merge 1 commit into
apache:mainfrom
Jay846:feature/list-view-to-list-cast
Open

GH-50976: [C++][Compute] Implement casting from ListView to List with zero-copy fast-path#50976
Jay846 wants to merge 1 commit into
apache:mainfrom
Jay846:feature/list-view-to-list-cast

Conversation

@Jay846

@Jay846 Jay846 commented Aug 24, 2026

Copy link
Copy Markdown

Rationale for this change

This PR implements missing type-casting compute kernels to convert ListViewType and LargeListViewType arrays to standard ListType and LargeListType arrays. This directly addresses legacy cast removal requirements under Arrow's compute framework.

What changes are included in this PR?

To maximize performance and optimize memory layouts, a dual-execution path was implemented in the CastListView execution functor inside scalar_cast_nested.cc:

  • Contiguous Zero-Copy Fast-Path: Triggered when the input list-view elements are contiguous and adjacent (i.e. offsets[i] + sizes[i] == offsets[i+1]). It avoids copying the child values array entirely, allocating the new output offset buffer, shifting offsets relative to the start, and slicing the child array directly to preserve zero-copy pointer semantics.
  • Non-Contiguous Slower Path: Triggered for gapped, overlapping, or out-of-order layouts. It dynamically tracks target offsets, builds integer index mappings using an Int64Builder, and invokes Arrow's internal take compute kernel to reconstruct a new contiguous child values array.
  • Kernel Registration: Formally wired into Arrow's casting system for standard and large variants of both nested types.

Are these changes tested?

Yes, added comprehensive unit test suites in scalar_cast_test.cc passing 109 out of 109 cases in the scalar cast suite. Tests explicitly cover:

  • Contiguous Arrays: Validated zero-copy memory behavior programmatically by ensuring the source and destination child buffers share the exact same physical memory address.
  • Gapped & Overlapping Arrays: Verified correct offset tracking and reconstruction when offsets intersect or contain unused slots.
  • Null Propagation: Checked that null validity maps propagate correctly at both the parent and child levels.
  • Generic/Nested Types: Validated casting on non-primitive data types (e.g. utf8 strings) and recursive child type promotions (e.g., ListView<int16> to List<int32>).
  • Boundary Slices: Verified that taking slices of non-contiguous arrays computes correct offsets and avoids out-of-bound errors.

Are there any user-facing changes?

No public API contracts were broken. This adds declarative casting support natively to the existing internal compute framework.

@Jay846
Jay846 requested a review from pitrou as a code owner August 24, 2026 20:21
@github-actions

Copy link
Copy Markdown

Thanks for opening a pull request!

This pull request has been automatically converted to a draft because its title doesn't match Arrow's required format.

If this is not a minor PR. Could you open an issue for this pull request on GitHub? https://github.com/apache/arrow/issues/new/choose

Opening GitHub issues ahead of time contributes to the Openness of the Apache Arrow project.

Then could you also rename the pull request title in the following format?

GH-${GITHUB_ISSUE_ID}: [${COMPONENT}] ${SUMMARY}

or

MINOR: [${COMPONENT}] ${SUMMARY}

After updating the title, you can mark the pull request as ready for review.

See also:

@github-actions
github-actions Bot marked this pull request as draft August 24, 2026 20:21

@Reranko05 Reranko05 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you use the Arrow PR title template:-

GH-<Issue Number>: [<Component>] <Title>

@Jay846 Jay846 changed the title feat(compute): implement casting from list_view to list [C++][Compute] Implement casting from ListView to List with zero-copy fast-path Aug 24, 2026
@Jay846 Jay846 changed the title [C++][Compute] Implement casting from ListView to List with zero-copy fast-path GH-50976: [C++][Compute] Implement casting from ListView to List with zero-copy fast-path Aug 24, 2026
@github-actions

Copy link
Copy Markdown

⚠️ GitHub issue #50976 has been automatically assigned in GitHub to PR creator.

@Jay846
Jay846 marked this pull request as ready for review August 24, 2026 22:54
@github-actions github-actions Bot added awaiting committer review Awaiting committer review and removed awaiting review Awaiting review labels Aug 25, 2026
@HuaHuaY

HuaHuaY commented Aug 25, 2026

Copy link
Copy Markdown
Contributor

You should use issue id but not the PR's id. If there isn't an existed issue, you can create a new one.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants