Skip to content

feat: a record of another spec stays readable as stored data - #194

Merged
SimonHeybrock merged 3 commits into
mainfrom
184-record-of-another-spec
Nov 13, 2026
Merged

SimonHeybrock merged 3 commits into
mainfrom
184-record-of-another-spec

Conversation

@SimonHeybrock

Copy link
Copy Markdown
Member

Stacked on #193 (base branch 184-spec-digest), which is stacked on #192. Merge #192, then #193, then this, or this PR lands in its base branch and not in main.

Second part of ADR 0009: a workspace's log stores the digest of each spec, and a record whose spec is not the one bound now stays readable as stored data.

The problem. A folder outlives the technique package that wrote it. Before this change, an outputs class changed under the same version was filled from the stored data without an error, and an output of a spec with no binding could not be read at all:

# iofq/v1 wrote Centre(x=0.5, y=0.25); the package now has, still under v1:
class Centre(BaseModel):
    x_mm: float = 0.0   # renamed from x
    y: float

client.output(old_record, 'centre')
# before: Centre(x_mm=0.0, y=0.25), no error
# now:    DataGroup({'x': 0.5, 'y': 0.25})

The rule. Every submission carries the digest of each spec it names that the log does not hold yet, or holds with another digest. A record is of the bound spec if the workspace has a binding with the same name, version, and digest. Otherwise it is of another spec: the version was bumped, the models changed, or no binding is given.

Of a record of another spec What a client gets
its request name, version, and the params as the stored dict
output, describe, select= the stored scipp data: a DataArray, or a DataGroup
readable, records, latest, provenance, the change feed as for any record
a new request that references its output SubmitError at the reference's position
a restart while it is pending and would run it fails with a reason that says its spec is not bound, and so does a pending record that reads an output of it

Store.read(record, output, None) reads a value as stored; its docstring defines that form, so that another store knows what to return. The log format is 3; a format 2 log is refused and left as it was.

For Simon to decide: Client.bound(records) -> list[bool]. A new public call, shaped like readable. I had said I would propose it before adding it, and added it since a fix needed it: Listing raised KeyError for a folder in which a top-level output was renamed under the same version, so the batch web server, the batch Panel view, and the sheet could not open such a folder. Listing now holds a spec only for records of the bound spec. For any other record it shows the spec's name and version, labels outputs by name, offers no configured plot, and does not offer the outputs for reference in a form. The call is in its own commit (the last one) with the views' use of it. An alternative is a digest field on Record, which exposes a detail of the log.

How a batch row shows, until #191 decides what a batch is:

The record's spec Row
another version than the one bound not reduced
the same version with changed models completed; export writes the stored data

A batch does not yet show the second kind as no longer current.

Known wart. For a record of another spec, the Panel outputs view decides from the described fields whether an output is a group, since no class says so. A stored group with one field named like its output would be shown as an array.

Removed during the work. ADR 0009 briefly said that a request made with a spec object whose digest differs from the bound one's is refused. Only client.submit(spec, ...) holds the spec object; Request(spec) and Template(spec) keep only name and version. A rule that holds on one path was dropped, with the sentence.

Review. A fresh reviewer wrote folders with one set of bindings and opened them with another. It found the Listing crash above, a crash of the Panel plot on a group read as stored, and the as-stored form of numbers and lists misstated; all fixed here, with tests. It found no path that still builds a changed class from stored data (freezes, stages, export, restart chains, a digest that changes and changes back). Four mutations, such as treating a changed digest as bound, each fail tests.

🤖 Generated with Claude Code

SimonHeybrock and others added 3 commits November 13, 2026 07:46
A record knew its spec only by name and version. An outputs class changed
under the same version read stored data into the new class without an
error, and a record whose spec version was bumped or unbound could not be
read at all: NotReadable on output, a bare KeyError on select= and on a
reference at submission.

A submission and an opening now log the digest of each spec they name that
the log does not hold yet under that name and version, or holds with another
digest. The digest of a record's spec is the last one logged before it. A
record is of the bound spec if a binding has the same name, version, and
digest, and otherwise of another spec. For a record of another spec:
client.output reads the stored scipp data (Store.read with tp=None: a
DataArray, or a DataGroup by field name), describe and select= work on it,
readable lists none of its outputs, a request that references one is refused
with a SubmitError at its position, and at a restart a pending record of it,
or one reading its output, fails with a reason that says the spec is not
bound. client.submit given a WorkflowSpec whose digest differs from the bound
one's is refused. Listing no longer raises for a record whose spec has no
binding, so the batch views open such a folder.

What the digest covers is part of the log's format, so FORMAT is 3; older
logs are refused.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
…al by spec object

readable means "may be read now", so an old result can still be shown; a
reference to it is still refused at submission. The digest check of a spec
object passed to submit is gone, since Request(spec) and Template(spec) drop it.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
…the views use it

Client.bound(records) says whether each record's spec is bound with the same
name, version, and digest. Listing holds a spec only for those, so a folder
with a record whose top-level output was renamed under the same version opens
in the batch views; the Panel outputs view reads a DataGroup field by key and
offers no configured plot for a record of another spec. Store.read defines the
as-stored form; Opened no longer carries digests, since a freeze's submission
does; a log naming a spec without a digest is refused with ValueError.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
@SimonHeybrock

Copy link
Copy Markdown
Member Author

LGTM

@SimonHeybrock
SimonHeybrock merged commit 3cfb5de into main Nov 13, 2026
2 checks passed
@SimonHeybrock
SimonHeybrock deleted the 184-record-of-another-spec branch November 13, 2026 10:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant