Skip to content

Bound xImage preprocessing transfers to the reconstruction region - #130

Merged
tpn merged 1 commit into
mainfrom
codex/ximage-04-roi-transfers
Oct 30, 2026
Merged

tpn merged 1 commit into
mainfrom
codex/ximage-04-roi-transfers

Conversation

@tpn

@tpn tpn commented Oct 24, 2026 •

Copy link
Copy Markdown
Collaborator

A small reconstruction region previously uploaded and preprocessed every input pixel. Slice images, variances and good-pixel masks before tensor conversion, retaining the source border required by VALID padding and preserving original bundle coordinates and shape metadata.

This bounds transfer and preprocessing memory to the requested region. GPU CI runs the xImage parity suite and rejects skipped cases. Coverage includes lazy mask slicing, optional masks/scales, asymmetric PSFs, odd/even regions, VALID boundaries, super-resolution and supplied PSFs.

Earlier production measurements cover backend and file-based reconstruction on three systems; those timings were not rerun for this integration.

@tpn
tpn requested a review from melo-gonzo as a code owner October 24, 2026 18:51
@tpn tpn added the ai-review Request a focused CodeRabbit review label Oct 24, 2026
@coderabbitai

coderabbitai Bot commented Oct 24, 2026 •

Copy link
Copy Markdown

Review in Change Stack →

No actionable comments were generated in the recent review.

Recent review info
Run configuration
Commits

Reviewing files that changed from the base of the PR and between 6286e07 and 096c9dd.

Files selected for processing: 3
  • .github/workflows/ci.yml
  • src/cuphoton/ximage/backends/torch_gpu_preprocess.py
  • tests/ximage/test_torch_gpu_preprocess.py

Included review availability: This review used your included allowance. Your plan provides up to 12 included reviews per hour; 10 remain after this review.

Walkthrough

Torch GPU preprocessing now validates PSFs and ROI bounds before transfer, slices spatial arrays to the selected ROI plus required VALID padding support, and retains original bundle shapes in metadata. Tests cover outputs, transfer sizes, bounds failures, and lazy HDF5 inputs. CI runs and uploads xImage GPU test reports.

Changes: GPU ROI preprocessing

Layer / File(s) Summary
ROI selection and bounded transfer
src/cuphoton/ximage/backends/torch_gpu_preprocess.py, tests/ximage/test_torch_gpu_preprocess.py
The preprocessing function validates PSFs and ROI bounds, slices spatial arrays before transfer, and uses coordinates relative to the transferred source for cutout selection and initialization. Metadata uses original bundle shapes. Tests cover expected outputs, transfer sizes, invalid bounds, lazy HDF5 inputs, and full-image expected values.
GPU test execution and report validation
.github/workflows/ci.yml
The GPU workflow installs the xImage extra, runs xImage GPU tests, validates the JUnit report, and uploads it as an artifact.

Possibly related PRs

  • NVIDIA/cuPhoton#127: Adds the xImage reconstruction workflow and Torch preprocessing path modified here.
  • NVIDIA/cuPhoton#125: Defines the XImageBundle, lazy-array, artifact, and ROI contracts used by this preprocessing path.

Priority: Normal

Unblocks: 1 PR

Merge Risk: Minimal · up to 096c9

No identified issue currently blocks merging the bounded-transfer change, subject to normal checks.

Pre-merge checks: Passed 2
Check name Status Explanation
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.

Comment @coderabbitai help to get the list of available commands.

@tpn

tpn commented Oct 24, 2026

Copy link
Copy Markdown
Collaborator Author

@coderabbitai full review

@coderabbitai

coderabbitai Bot commented Oct 24, 2026 •

Copy link
Copy Markdown
✅ Action performed

Full review finished.

@tpn

tpn commented Oct 24, 2026 •

Copy link
Copy Markdown
Collaborator Author

Production measurements for a 256×256 region from 2048×2048 synthetic inputs:

GPU Backend baseline → PR, ms Speedup File-backed scene reconstruction baseline → PR, ms Speedup
NVIDIA RTX PRO 6000 Blackwell Workstation Edition 46.75 → 8.47 5.52× 57.95 → 16.39 3.54×
NVIDIA GB10, system A 49.67 → 16.19 3.07× 92.12 → 19.01 4.84×
NVIDIA GB10, system B 50.56 → 17.14 2.95× 94.18 → 19.72 4.78×

Both versions use the same GPU solver. The paired baseline restores the preprocessor from bb20a525; the candidate uses this PR's production preprocessor at aefc3d0. Inputs are FP64, eight exposures, supplied 7×7 PSFs, L2/SAME/SRF1, and 15 iterations. Results are medians of five measured runs after two warmups, with eight Torch CPU threads, cuDNN autotuning enabled and TF32 disabled (PyTorch 2.13.0, CUDA 13.0).

Backend timing includes host/device transfers and excludes file I/O. File-backed timing covers configuration/input loading through buffered output writes, using the same input files and a warmed page cache, without fsync. It excludes PSF calibration, WCS alignment, detection, photometry, and metric evaluation. These are within-system comparisons; host and storage differences prevent a GPU-only ranking.

Checked images and iteration/loss traces were bitwise identical in each of the three backend cases on every system and all paired file-backed runs. For the 2048→256 case, peak Torch-allocated device memory fell from 1,632 to 43 MiB on all three systems; this excludes allocator reservations and CUDA context memory. The 512×512 full-frame control was effectively unchanged: 13.32 → 13.49 ms, 49.89 → 49.96 ms, 50.64 → 50.57 ms (table order). Smaller regions are the intended benefit.

The workstation used 7958207; GB10 runs used 782011a9. Relevant production source hashes match, and the intervening commit only updates CLI test counts. Measurements exclude first calls, which share caches. Numerical agreement preserves these tested calculations; it does not establish scientific reconstruction accuracy.

Subsequent CI-download and benchmark cache-release changes leave the measured preprocessing, reconstruction, and standalone metric implementations unchanged.

@tpn

tpn commented Oct 24, 2026

Copy link
Copy Markdown
Collaborator Author

@coderabbitai full review

The updated head reuses the three-line, checksum-verified CFITSIO download fix from #129 to unblock native-wheel CI. The xImage implementation and measured performance source are unchanged.

@coderabbitai

coderabbitai Bot commented Oct 24, 2026 •

Copy link
Copy Markdown
✅ Action performed

Full review finished.

@tpn
tpn force-pushed the codex/ximage-03-integration branch from bb20a52 to 47b3a32 Compare October 27, 2026 02:20
@tpn
tpn force-pushed the codex/ximage-04-roi-transfers branch from 6286e07 to d49f913 Compare October 27, 2026 02:20
@tpn
tpn force-pushed the codex/ximage-03-integration branch from 47b3a32 to d3268d6 Compare October 27, 2026 02:45
@tpn
tpn force-pushed the codex/ximage-04-roi-transfers branch 3 times, most recently from 6a4768e to bce3047 Compare October 28, 2026 05:42
@tpn
tpn force-pushed the codex/ximage-03-integration branch from eaeb8ab to aa3cefc Compare October 28, 2026 05:42
@tpn
tpn force-pushed the codex/ximage-04-roi-transfers branch from bce3047 to 104a3a4 Compare October 28, 2026 06:03

@melo-gonzo melo-gonzo left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The ROI slicing keeps the VALID-padding border and the original bundle coordinates, and the region tests in test_torch_gpu_preprocess.py pass on CPU. The CUDA timings in the PR comment were not reproduced. This PR moves the line where f is built, so the PSF fix requested in #127 will need a rebase here.

@tpn
tpn force-pushed the codex/ximage-04-roi-transfers branch from 104a3a4 to f10e34b Compare October 29, 2026 02:44
@tpn
tpn force-pushed the codex/ximage-03-integration branch from 27c1b54 to 922b25b Compare October 29, 2026 02:45

@melo-gonzo melo-gonzo left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The restack keeps the PSF fix, and the region tests in test_torch_gpu_preprocess.py pass on CPU.

@tpn
tpn force-pushed the codex/ximage-03-integration branch from 922b25b to 9d28cc3 Compare October 30, 2026 08:15
Slice image, variance and mask inputs before tensor conversion,
retaining the source border required by VALID padding. Preserve bundle
coordinates and raw-shape metadata while bounding preprocessing memory.

Run xImage CUDA parity tests in the existing GPU CI job.

Signed-off-by: Trent Nelson <trentn@nvidia.com>
@tpn
tpn changed the base branch from codex/ximage-03-integration to main October 30, 2026 08:32
@tpn
tpn force-pushed the codex/ximage-04-roi-transfers branch from f10e34b to 096c9dd Compare October 30, 2026 08:32
@tpn
tpn merged commit 07abcdf into main Oct 30, 2026
71 of 73 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ai-review Request a focused CodeRabbit review

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants