loki: load the detector once and read it by reference

docs/developer/README.md

@@ -142,7 +142,8 @@ client.failure(result)                               # why it failed, or None
 An application learns what an output is from `client.describe`, which gives the sizes, unit, and coordinate names of each array field and reads no array, and then reads the part it shows with `select=`.
 A value that was persisted and is no longer in memory is read from the store to be described.
 The plot of an array of more than two dimensions has a slider for each dimension but the last two, and moving it reads one cut (`ess.apps.panel.Outputs`; which dimensions get a slider is decided in `ess.apps.plotting`, which needs no widget library).
-A workflow package can configure plots for its spec: `App(client, plots={spec_id: {'Centre on image': function}})`, where a function takes an instance of the spec's outputs model and returns a plopp figure.
+A workflow package can configure plots for its spec: `App(client, plots={spec_id: {'Centre on image': Plot(function, inputs=('detector',))}})`, where the function takes an instance of the spec's outputs model, and returns a plopp figure.
+Each of `inputs` names a parameter that references an output, and is passed by name as that value, so that a small output, such as a mask, is drawn on the large value it was computed from, without holding a copy.
 
 A request is checked in two phases:
 

packages/essapps/src/ess/apps/panel/outputs.py

@@ -24,8 +24,9 @@
 and drawn whole. A field of more dimensions has a slider for each dimension but
 the last two, and a cut of it is read with ``select=`` for the positions of the
 sliders, so no more of the array is read than is drawn. A plot configured for a
-spec is a function of the record's outputs, every one of them read: it returns a
-plopp figure, or fails, which the view shows as text.
+spec (:class:`~ess.apps.plotting.Plot`) is a function of the record's outputs, every
+one of them read, and of the values its request referenced that it declares: it
+returns a plopp figure, or fails, which the view shows as text.
 
 Removing an output releases its record (``client.release``), so the record
 stays in the proposal's history and in the graphs of the records that read it.
@@ -40,11 +41,10 @@
 import panel as pn
 import plopp as pp
 from plopp.core.typing import FigureLike as Figure
-from pydantic import BaseModel
 from pydantic.fields import FieldInfo
 
 from ess.apps import ArrayInfo, Client, Listing, NotReadable, Record, SpecId, Status
-from ess.apps.plotting import slider_dims
+from ess.apps.plotting import Plot, read_inputs, slider_dims
 from ess.spec import field_types
 
 from .graph import Graph
@@ -53,9 +53,8 @@
 _PLOT = 'Plot'
 _PROVENANCE = 'Provenance'
 
-Plots = Mapping[SpecId, Mapping[str, Callable[[BaseModel], Figure]]]
-"""Plots by the name of a view, for the records of a spec: each a function of the
-record's outputs, as an instance of ``spec.outputs``."""
+Plots = Mapping[SpecId, Mapping[str, Plot]]
+"""Plots by the name of a view, for the records of a spec."""
 
 
 class Outputs:
@@ -69,7 +68,7 @@ class Outputs:
     while its record is pending or completed, and when its record has failed,
     until removed. ``refresh`` is called once an output is removed; by
     default it lists the outputs again. ``plots`` holds, for the spec of a
-    record, functions by the name of the view that shows their figure.
+    record, the plots by the name of the view that shows their figure.
     """
 
     def __init__(
@@ -218,13 +217,14 @@ def _configured(self, record: Record, view: str) -> Any:
         if self._listing.status[record.id] is not Status.COMPLETED:
             return pn.pane.Markdown('')
         try:
-            make = self._plots[record.spec][view]
+            plot = self._plots[record.spec][view]
             model = self._listing.specs[record.id].outputs
             values = self._client.output(record)
             outputs = model.model_construct(
                 **{n: values.get(n) for n in model.model_fields}
             )
-            return _figure_pane(make(outputs))
+            inputs = read_inputs(plot, self._client, record)
+            return _figure_pane(plot.function(outputs, **inputs))
         except Exception as error:  # a plot is code of the workflow's author
             return pn.pane.Markdown(f'{view} failed: `{type(error).__name__}: {error}`')
 

packages/essapps/src/ess/apps/plotting.py

@@ -2,7 +2,8 @@
 # Copyright (c) 2026 Scipp contributors (https://github.com/scipp)
 """
 What an application decides about plots before it draws any, which needs no
-widget library: which dimensions of an array get a slider.
+widget library: which dimensions of an array get a slider, and which values a plot
+configured for a spec reads.
 
 An application learns what an output is from :meth:`~ess.apps.Client.describe`,
 which reads no array, and reads the part of an array that it draws with
@@ -11,7 +12,62 @@
 
 from __future__ import annotations
 
-from collections.abc import Mapping
+from collections.abc import Callable, Mapping
+from dataclasses import dataclass
+from typing import Any
+
+from ess.spec import OutputRef
+
+from .client import Client
+from .errors import NotReadable
+from .records import Record
+
+
+@dataclass(frozen=True)
+class Plot:
+    """
+    A plot configured for the records of a spec.
+
+    ``function`` takes the record's outputs, as an instance of ``spec.outputs``, and
+    returns a figure. ``inputs`` names params that hold a reference to a whole output,
+    each passed to ``function`` by that name as the value it references. So a workflow
+    with a small output, such as a mask, draws it on the large value that it was
+    computed from, such as a detector, which its record names::
+
+        Plot(masks_on_image, inputs=('detector',))
+
+    """
+
+    function: Callable[..., Any]
+    inputs: tuple[str, ...] = ()
+
+
+def read_inputs(plot: Plot, client: Client, record: Record) -> dict[str, Any]:
+    """
+    The value of each input of ``plot`` that the request of ``record`` references.
+
+    Raises
+    ------
+    ValueError
+        If a param holds anything but a reference to a whole output.
+    NotReadable
+        If the output is no longer readable, which names the param: computing the
+        output again makes a new record, which this record does not reference.
+    """
+    values = {}
+    for name in plot.inputs:
+        ref = record.request.params[name]
+        if not isinstance(ref, OutputRef) or ref.key is not None:
+            raise ValueError(f'{name} does not reference a whole output')
+        (source,) = client.records(ids=[ref.record])
+        try:
+            values[name] = client.output(source, ref.output)
+        except NotReadable as error:
+            raise NotReadable(
+                f'{name} is no longer available. Compute it again, and this '
+                f'record from the result ({error})'
+            ) from error
+    return values
 
 
 def slider_dims(sizes: Mapping[str, int]) -> list[str]:

packages/essapps/tests/panel/outputs_test.py

@@ -24,6 +24,7 @@
 from ess.apps import Client, Listing, Record, SpecId, Status, local
 from ess.apps.panel import App, Outputs, Records
 from ess.apps.panel.records import describe
+from ess.apps.plotting import Plot
 from ess.apps.testing import FakeFiles, RecordingClient, read_file
 from ess.spec import (
     FileRef,
@@ -104,6 +105,10 @@ class ScaleOutputs(BaseModel):
     total: Total
 
 
+class FrameParams(BaseModel):
+    picture: Referenced[Image]
+
+
 CURVE = WorkflowSpec(
     name='curve',
     version=1,
@@ -122,6 +127,16 @@ class ScaleOutputs(BaseModel):
 )
 
 
+FRAME = WorkflowSpec(
+    name='frame',
+    version=1,
+    title='Frame',
+    description='The total of an image.',
+    params=FrameParams,
+    outputs=ScaleOutputs,
+)
+
+
 CUBE = WorkflowSpec(
     name='cube',
     version=1,
@@ -185,9 +200,13 @@ def curve(params: CurveParams) -> CurveOutputs:
     def scale(params: ScaleParams) -> ScaleOutputs:
         return ScaleOutputs(total=Total(value=params.curve.total))
 
+    def frame(params: FrameParams) -> ScaleOutputs:
+        return ScaleOutputs(total=Total(value=params.picture.counts.sum()))
+
     bind = [
         function_binding(CURVE, curve),
         function_binding(SCALE, scale),
+        function_binding(FRAME, frame),
         function_binding(CUBE, cube),
     ]
     with local(proposal='p1', files=run[0], bind=bind) as client:
@@ -607,7 +626,7 @@ def test_plots_configured_for_a_spec_are_views_of_the_outputs_of_its_records(
     scale = client.submit(SCALE, params={'curve': curve.ref('curve')})
     client.wait([curve, scale])
     seen = Seen()
-    plots = {SpecId.of(CURVE): {'Counts': seen.counts}}
+    plots = {SpecId.of(CURVE): {'Counts': Plot(seen.counts)}}
     outputs = Outputs(client, Listing(client), plots=plots)
     assert outputs.views.options == ['Plot', 'Provenance']
 
@@ -634,7 +653,7 @@ def test_a_view_the_record_does_not_offer_falls_back_to_the_plot(
     curve = client.submit(CURVE, params={'run': file})
     scale = client.submit(SCALE, params={'curve': curve.ref('curve')})
     client.wait([curve, scale])
-    plots = {SpecId.of(CURVE): {'Counts': Seen().counts}}
+    plots = {SpecId.of(CURVE): {'Counts': Plot(Seen().counts)}}
     outputs = Outputs(client, Listing(client), plots=plots)
     outputs.select(curve, 'curve')
     outputs.views.value = 'Counts'
@@ -654,7 +673,7 @@ def test_a_configured_plot_that_fails_shows_why_as_text(
     curve = client.submit(CURVE, params={'run': file})
     client.wait(curve)
     outputs = Outputs(
-        client, Listing(client), plots={SpecId.of(CURVE): {'Bad': explode}}
+        client, Listing(client), plots={SpecId.of(CURVE): {'Bad': Plot(explode)}}
     )
     outputs.select(curve, 'curve')
 
@@ -665,8 +684,66 @@ def test_a_configured_plot_that_fails_shows_why_as_text(
     assert 'RuntimeError: no plot today' in text.object
 
 
+def draw_picture(out: ScaleOutputs, *, picture: Image) -> Any:
+    return pp.plot(picture.counts, title=str(picture.counts.sum().value))
+
+
+@pytest.fixture
+def frame_view(client: Client, file: FileRef) -> tuple[Outputs, Record, Record]:
+    """The outputs with a view of a frame that draws the image it referenced."""
+    curve = client.submit(CURVE, params={'run': file})
+    frame = client.submit(FRAME, params={'picture': curve.ref('image')})
+    client.wait([curve, frame])
+    plots = {SpecId.of(FRAME): {'Picture': Plot(draw_picture, inputs=('picture',))}}
+    return Outputs(client, Listing(client), plots=plots), curve, frame
+
+
+def test_a_configured_plot_takes_the_value_its_record_referenced(
+    client: Client, frame_view: tuple[Outputs, Record, Record]
+) -> None:
+    outputs, curve, frame = frame_view
+    outputs.select(frame, 'total')
+
+    outputs.views.value = 'Picture'
+
+    (pane,) = outputs.view.objects
+    assert isinstance(pane, pn.pane.Matplotlib), pane.object
+    total = client.output(curve, 'image').counts.sum().value
+    assert pane.object.axes[0].get_title() == str(total)
+
+
+def test_a_configured_plot_whose_input_is_released_says_what_to_do(
+    client: Client, frame_view: tuple[Outputs, Record, Record]
+) -> None:
+    outputs, curve, frame = frame_view
+    client.release(curve)
+    outputs.select(frame, 'total')
+
+    outputs.views.value = 'Picture'
+
+    (text,) = outputs.view.objects
+    assert 'Picture failed' in text.object
+    assert 'NotReadable: picture is no longer available' in text.object
+    assert 'Compute it again' in text.object
+
+
+def test_a_configured_plot_input_must_reference_an_output(
+    client: Client, file: FileRef
+) -> None:
+    curve = client.submit(CURVE, params={'run': file})
+    client.wait(curve)
+    plots = {SpecId.of(CURVE): {'Counts': Plot(Seen().counts, inputs=('run',))}}
+    outputs = Outputs(client, Listing(client), plots=plots)
+    outputs.select(curve, 'curve')
+
+    outputs.views.value = 'Counts'
+
+    (text,) = outputs.view.objects
+    assert 'run does not reference a whole output' in text.object
+
+
 def test_a_configured_plot_cannot_take_the_name_of_a_view_of_every_record() -> None:
-    plots = {SpecId.of(CURVE): {'Plot': explode}}
+    plots = {SpecId.of(CURVE): {'Plot': Plot(explode)}}
     with pytest.raises(ValueError, match="curve/v1: \\['Plot'\\]"):
         Outputs(None, None, plots=plots)  # type: ignore[arg-type]
 
@@ -675,7 +752,7 @@ def test_the_app_passes_its_plots_to_the_outputs(client: Client, file: FileRef)
     curve = client.submit(CURVE, params={'run': file})
     client.wait(curve)
     seen = Seen()
-    app = App(client, plots={SpecId.of(CURVE): {'Counts': seen.counts}})
+    app = App(client, plots={SpecId.of(CURVE): {'Counts': Plot(seen.counts)}})
     app.poll()
 
     app.outputs.select(curve, 'curve')

packages/essapps/tests/plotting_test.py

@@ -4,8 +4,13 @@
 
 import subprocess
 import sys
+from datetime import UTC, datetime
 
-from ess.apps.plotting import slider_dims
+import pytest
+
+from ess.apps import Record, Request, SpecId
+from ess.apps.plotting import Plot, read_inputs, slider_dims
+from ess.spec import OutputRef
 
 
 def test_every_dimension_but_the_last_two_has_a_slider() -> None:
@@ -19,6 +24,20 @@ def test_every_dimension_but_the_last_two_has_a_slider() -> None:
     assert slider_dims({}) == []
 
 
+def test_a_plot_input_cannot_be_an_element_of_an_output() -> None:
+    element = OutputRef(record='r', output='images', key='first')
+    record = Record(
+        id='s',
+        request=Request(SpecId(name='s', version=1), params={'image': element}),
+        proposal='p',
+        submitter='u',
+        created=datetime.now(UTC),
+        outputs=(),
+    )
+    with pytest.raises(ValueError, match='image does not reference a whole output'):
+        read_inputs(Plot(print, inputs=('image',)), None, record)  # type: ignore[arg-type]
+
+
 def test_what_an_application_decides_about_plots_needs_no_panel() -> None:
     code = (
         'import sys; import ess.apps, ess.apps.plotting, ess.apps.listing;'

packages/essinstruments/docs/loki-app.ipynb

@@ -96,9 +96,11 @@
     "\n",
     "The tab *Workflows* holds the forms, the tab *Outputs* the results.\n",
     "\n",
-    "1. **Pixel masks**: pick the form *Pixel masks*, give run 60339 and the mask file, and submit. In the tab *Outputs*, select the image and pick the view *Masks on image*.\n",
-    "2. **Beam centre**: pick the form *Beam centre*, give run 60339 and the masks as *From a record*, and submit. The view *Centre on image* marks the centre.\n",
-    "3. **I(Q)**: pick the form *I(Q)*, give a sample run and its transmission run, the can, its transmission run, the empty beam, the direct beam, the masks and the beam centre *From a record*, and the bins, and submit. The view *I(Q)* draws the result. The form keeps its values, so for the next sample change only the two sample fields."
+    "1. **Calibrated detector**: pick the form *Calibrated detector*, give run 60339, and submit.\n",
+    "2. **Pixel masks**: pick the form *Pixel masks*, give the detector as *From a record* and the mask file, and submit. In the tab *Outputs*, select the masks and pick the view *Masks on image*.\n",
+    "3. **Beam centre**: pick the form *Beam centre*, give the detector and the masks as *From a record*, and submit. The view *Centre on image* marks the centre.\n",
+    "4. **Create mask**, if needed: pick the form *Create mask*, give the detector, the beam centre, and the masks as *From a record*, add angle ranges, and submit. The view *Masks on image* leaves their pixels out. Use the result as the masks of I(Q).\n",
+    "5. **I(Q)**: pick the form *I(Q)*, give a sample run and its transmission run, the can, its transmission run, the empty beam, the direct beam, the masks and the beam centre *From a record*, and the bins, and submit. The view *I(Q)* draws the result. The form keeps its values, so for the next sample change only the two sample fields."
    ]
   },
   {