Skip to content

GH-50936: [C++][Integration] Replace RapidJSON with simdjson - #50937

Open
Reranko05 wants to merge 2 commits into
apache:mainfrom
Reranko05:gh-35460-integration2
Open

GH-50936: [C++][Integration] Replace RapidJSON with simdjson#50937
Reranko05 wants to merge 2 commits into
apache:mainfrom
Reranko05:gh-35460-integration2

Conversation

@Reranko05

@Reranko05 Reranko05 commented Aug 21, 2026

Copy link
Copy Markdown
Collaborator

Rationale for this change

This PR continues the simdjson migration by replacing the RapidJSON usage in the C++ JSON integration implementation with simdjson's DOM API and Arrow's existing JsonWriter.

Changes

  • Replace RapidJSON DOM parsing with simdjson's DOM API.
  • Update the integration JSON reader and internal parsing helpers to use simdjson::dom types.
  • Replace RapidJSON JSON serialization with Arrow's existing JsonWriter.
  • Handle optional JSON array members without creating an invalid simdjson array.
  • Remove the unused RapidJSON dependencies from the CMake and Meson integration test targets.
  • Update remaining RapidJSON-specific implementation references.

Fixes: #50936

@github-actions github-actions Bot added the awaiting review Awaiting review label Aug 21, 2026
@Reranko05
Reranko05 marked this pull request as ready for review August 21, 2026 08:32
@Reranko05
Reranko05 requested a review from pitrou as a code owner August 21, 2026 08:32
Copilot AI lite review requested due to automatic review settings August 21, 2026 08:32

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@Reranko05
Reranko05 requested review from kou and rok August 21, 2026 08:32
@Reranko05
Reranko05 force-pushed the gh-35460-integration2 branch from e50b949 to 72a256a Compare August 24, 2026 07:50
@Reranko05

Copy link
Copy Markdown
Collaborator Author

@kou, Could you review this when you have time?

Comment thread cpp/src/arrow/integration/json_internal.cc Outdated
@github-actions github-actions Bot added awaiting changes Awaiting changes and removed awaiting review Awaiting review labels Aug 24, 2026
const auto& it = obj.FindMember(key);
RETURN_NOT_STRING(key, it, obj);
return it->value.GetString();
Result<std::string> GetMemberString(const JsonObject& obj, std::string_view key) {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps we can return Result<std::string_view> here?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I kept the helper returning std::string, as several callers require an owning string, but made the conversion from simdjson's std::string_view explicit.

ARROW_ASSIGN_OR_RAISE(
auto timezone_string,
internal::ResolveSimdjsonResult(timezone.get_string(), "field was not a string"));
return timestamp(unit, std::string(timezone_string));

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The explicit std::string cast isn't necessary, is it?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I checked this, but timestamp() currently takes a const std::string&, so the conversion is required here.

Comment thread cpp/src/arrow/integration/json_internal.cc Outdated
Comment thread cpp/src/arrow/integration/json_integration.cc Outdated
Comment thread cpp/src/arrow/integration/json_internal.cc
Copilot AI review requested due to automatic review settings August 24, 2026 09:42

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@github-actions github-actions Bot added awaiting change review Awaiting change review and removed awaiting changes Awaiting changes labels Aug 24, 2026
Copilot AI review requested due to automatic review settings August 24, 2026 09:57
@Reranko05
Reranko05 force-pushed the gh-35460-integration2 branch from 70f43b3 to e81a967 Compare August 24, 2026 09:57

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

Copilot AI review requested due to automatic review settings August 24, 2026 10:00
@Reranko05
Reranko05 force-pushed the gh-35460-integration2 branch from e81a967 to 6c8592c Compare August 24, 2026 10:00

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

Copilot AI review requested due to automatic review settings August 24, 2026 10:58
@Reranko05
Reranko05 force-pushed the gh-35460-integration2 branch from 6c8592c to f4b4750 Compare August 24, 2026 10:58

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@Reranko05
Reranko05 requested review from kou and pitrou August 24, 2026 13:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

awaiting change review Awaiting change review CI: Extra: C++ Run extra C++ CI Component: C++

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[C++][Integration] Replace RapidJSON with simdjson

4 participants