[JingsongLi]

Purpose

Keep read planning and execution within the selected backend. Public read and scan classes retain configuration, validation and dispatch; Native adapters do not initialize Python executors.

Changes

• Make public AsyncStreamingTableScan and StreamTableScan facades select through the same dispatch rule as builders. Either scan.native-plan.enabled or read.native.enabled selects Native planning. Backend selection completes before frames or checkpoints are consumed; planning errors never switch a live stream to Python.
• Preserve public polling, checkpoint, restore, consumer acknowledgement and cancellation behavior. Remove external follow_up_scanner injection as requested; built-in delta/changelog selection remains internal to each backend.
• Make TableScan.file_scanner configuration independent of backend construction. Ordinary scans, statistics and raw file metadata use dispatch. Accessing this property does not instantiate the Python scanner, and custom manifest-source callbacks are not exposed.
• Adapt typed Rust file plans through the existing manifest metadata codec, preserving historical schema/statistics, canonical and external paths, row tracking and Blob metadata. Configure file policies with with_file_filters and call no-argument plan_files(). Raw file plans require unrestricted whole-table permission and ignore projection, LIMIT, row ranges and distribution.
• Keep scanner predicate updates, authorization field selection, incremental L0 visibility and Pure scanner cache invalidation consistent. Trusted Python writer metadata restoration uses its private scanner and retains its write authorization boundary.
• Delegate timestamp windows to Rust Core after authorization, including empty windows and AUTO changelog selection. Python snapshot resolution stays lazy within its selected executor.
• Route backend-specific read validation through dispatch. Remove 16 private Python execution forwarding methods from public TableRead; shared validation/configuration/output adapters stay outside execution.

Planning API naming

Rename the complete split-planning operation to plan_all() across the public Python API, both backends, read/write/update consumers, tests and documentation. No old-name alias is retained. It returns a split Plan, including L0 and all matching data-evolution column files while preserving predicates, LIMIT and unrestricted authorization. plan_files() returns raw manifest entries.

Review follow-up

• Keep non-null explicit partition selectors independent from later row-predicate updates and include their fields in authorization. Forward the effective selector to Native raw file planning.
• Preserve already authorized explicit selectors when the Python executor rewrites masked predicates. Keep complete read predicates available for Native query authorization.
• Refresh a cached Python scanner's authorization state only after raw file planning receives an unrestricted whole-table grant. Previous row filters and mask-based pruning cannot leak into the new metadata scope; rejected scopes leave the cache untouched.
• Align null selector configuration with the public Java ReadBuilder contract: None removes the explicit override and restores partition pruning derived from the complete row predicate. Remove the Python-only disable-pruning state, partition-conjunct stripping, and special Native-to-Python fallback. Previously captured builder selectors cannot reappear after clearing; later row-predicate updates derive fresh pruning.
• Red/green and independent follow-up reviews cover configuration, authorization and fallback behavior.

Dependency

The typed file-planning API requires apache/paimon-rust#1197. This PR is draft until that dependency merges. Native CI continues to build Rust main; no SDK version probes or compatibility fallback were added.

Tests

• Java-semantics follow-up across seven planning, partition-selection and authorization modules: 341 passed, 76 skipped; PyArrow 16 focused regressions: 12 passed. Native integration coverage now includes null selector reset across append, primary-key and data-evolution tables with and without query authorization. Those extension-dependent cases remain skipped locally.
• Follow-up changed-file Flake8, Python 3.6 grammar checks (7 files), and git diff --check passed. Independent Java/API/auth review found no remaining actionable issues.
• Earlier broad verification across all 106 affected test modules, run in two process groups: 2156 passed, 3959 skipped. The 105-module group passed 2150 cases; distributed_input_execution_test.py passed all 6 cases in a separate process because its Daft global runner otherwise retains a Ray actor from an earlier cluster lifecycle. No unrelated Daft/Ray behavior was changed.
• Final focused planning, partition-selector and authorization regressions: 317 passed, 4 skipped. Changed-file Flake8, Python 3.6 grammar and whitespace checks passed.
• All 94 affected Python test modules: 1651 passed, 3910 skipped (92 modules: 1622 passed/3910 skipped; two additional modules: 29 passed).
• PyArrow 16 focused file-planning, streaming, dispatch and explanation suites: 111 passed, 4 skipped.
• Rename follow-up across 23 affected Python modules: 769 passed, 576 skipped; changed-file Flake8 and Python 3.6 grammar checks passed.
• Follow-up API configuration, dispatch, file-scanner and Ray bucket-join regressions: 95 passed, 4 skipped.
• Changed Python files: Flake8 with repository configuration, Python 3.6 grammar checks for 114 files and git diff --check passed.
• Independent reviews of streaming lifecycle, authorization, file metadata, backend boundaries and writer metadata found no remaining actionable issues after fixes.
• Native extension dependent tests are skipped locally. No Native extension was rebuilt or installed; the existing CI runs those cases against Rust main after the dependency merges.

[leaves12138]

LGTM