Repository navigation
perf: build the LoKI workflow once per process - #187
Merged
Merged
Conversation
LokiAtLarmorWorkflow() takes about 0.7 s of pure-Python time, holding the interpreter lock, and the LoKI bindings built it once per request (detector, masks, beam centre, I(Q)). Copying a built workflow takes about 6 ms. The bindings now build it once, lazily, and give each request its own copy, so parameters and providers set by one request do not reach the shared workflow or another request. Measured on the LoKI tutorial files, 24 cores (wall time, peak memory): chain (detector, masks, centre, I(Q)), warm: 7.0-7.5 s -> 4.0-4.1 s 4 x I(Q), serial: 15.0-15.3 s -> 12.2 s (8.2-8.4 GB -> 8.0-8.5 GB) 4 x I(Q), workspace threads: 19.1-20.7 s -> 13.6-14.2 s (~26 GB, unchanged) 4 x I(Q), one stage: 7.4 s -> 6.7-6.8 s (~14 GB, unchanged) Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
5 of 7 tasks
The constructor is slow because sciline keeps each generic key as a node of a networkx graph and rebuilds the graph at each assignment. sciline plans to handle generics differently, so the comment says to measure again then. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The LoKI bindings built the esssans workflow once per request.
LokiAtLarmorWorkflow()takes about 0.7 s of pure-Python time and holds the interpreter lock while it runs; a chain of detector, masks, beam centre, and I(Q) built it four times. The bindings now build it once per process, on first use, and each request works on its own copy (Pipeline.copy(), about 6 ms), so what one request sets does not reach another.Found by the measurement for step 2 of #184. LoKI tutorial files, 24 cores, before → after:
Masks and beam centre were almost entirely workflow construction: about 1.0 s → 0.15–0.3 s each.
Four workspace threads are still slower than one for plain requests, so the construction was not the only cause. The default number of threads is a separate change.
No test added: the existing tests already run requests with different parameters against the shared workflow and compare I(Q) with plain esssans.
🤖 Generated with Claude Code