Repository navigation
Conversation
d6434d6 to
e59b5e9
Compare
pctgradientdescent applicationpctgradientdescent application
pctgradientdescent applicationpctgradientdescent application
e59b5e9 to
a8ea902
Compare
a8ea902 to
2f0e09b
Compare
167a324 to
3bb026e
Compare
| backward_start_time = time.time() | ||
| loss_n.backward() |
There was a problem hiding this comment.
Smaller batches enlarge updates
Each batch uses a mean loss, but loss_n.backward() adds that full gradient without dividing by batch_per_subset. With SGD, splitting the same subset into two equal batches doubles the update at the same learning rate. Normalize the accumulated gradient so --max-batch-size controls memory use rather than changing the effective learning rate.
Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!
There was a problem hiding this comment.
This is indeed a problem now that batch sizes can be uneven. I changed the accumulation method of the loss function to sum, then divided the total by the size of a batch in order to normalize.
3bb026e to
e2ec229
Compare
| proton_mask = (es_fm > 1.0).detach() | ||
| loss_n = ( | ||
| loss(pq_fm[proton_mask], pq_m[proton_mask]) | ||
| + args_info.tv * total_variation(rsp) | ||
| ) / proton_mask.sum() |
There was a problem hiding this comment.
Stopped batches break reconstruction
If the current image makes every proton in a batch stop, forward_iter clamps their energies to 1.0 and proton_mask excludes them all. The loss then divides by zero, producing a non-finite loss that can stop reconstruction during backward. Skip batches with no valid protons before dividing, while still counting them as consumed.
There was a problem hiding this comment.
Added a warning if that occurs, even though it probably indicates a poor learning rate choice or a wrong initial estimate of the RSP map.
There was a problem hiding this comment.
The warning is useful, but the stopped-batch case still is not safe. processed_protons is incremented only after computing the valid-proton loss; the continue therefore leaves it unchanged. Once the dataloader is exhausted, the outer loop keeps creating exhausted iterators and repeating forever.
Count each fetched batch as consumed before applying the mask, or increment the processed count in the zero-valid-proton branch. Also handle an entirely empty loss_tot explicitly, since torch.tensor(loss_tot).mean() then produces nan and optimizer.step() has no gradients.
e2ec229 to
8384f6e
Compare
| parser.add_argument( | ||
| "--hull", | ||
| "--quadric", | ||
| help="Quadric representing the hull of the object", | ||
| type=float, | ||
| nargs=10, | ||
| required=False, |
There was a problem hiding this comment.
Calling pct.pctgradientdescent with a list for hull fails before reconstruction starts. parse_kwargs turns the list into one comma-separated token, but --hull requires ten separate tokens. Only options with nargs="+" get comma splitting.
Support fixed-length lists in the parser so Python callers can supply a hull.
8384f6e to
a266714
Compare
| dataloader = torch.utils.data.DataLoader( | ||
| dataset, | ||
| sampler=RandomSampler(len(dataset)), | ||
| batch_size=batch_size, | ||
| num_workers=args_info.num_workers, | ||
| in_order=False, | ||
| persistent_workers=args_info.num_workers > 0, | ||
| pin_memory=True, | ||
| drop_last=False, |
There was a problem hiding this comment.
ProtonDataset builds each path with a length based on that proton’s detector positions, but this DataLoader uses default batching. When valid list-mode inputs contain different detector distances, a batch can contain unequal-length tensors. Stacking them raises an error and stops reconstruction.
Group equal-length paths, or pad them and exclude padded segments from the model.
Left to do:
opengate.contrib.protonct.protonctto introduce the changes needed byGradientDescentReconstruction.py(PR opened here: Improve protonct.py for latest PCT developments OpenGATE/opengate#1132)LocalTimeorGlobalTimefor TOF (or both?) (will be applied in a separate PR once next version of GATE is released)pytest-xdistto run tests in new processesTo be discussed.
The PR needs fixes for unequal-length paths and batch-dependent updates before merging.
Findings
Summary
Adds gradient-descent reconstruction and a stopping-power table generator.
Diagram
%%{init: {'theme': 'neutral'}}%% flowchart LR A[List-mode files and geometry] --> B[Build proton paths] C[Stopping-power table] --> D[Predict time or energy] B --> D E[Current RSP image] --> D D --> F[Compare with measurements] F --> G[Update RSP image] G --> E G --> H[Save images and checkpoint]Reviews (6) · Last reviewed commit: "ENH: Improve loading of inputs of pctgra..." · Reviewed by Greptile