Repository navigation
Conversation
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #1501 +/- ##
==========================================
+ Coverage 84.70% 89.63% +4.92%
==========================================
Files 52 52
Lines 4590 7165 +2575
==========================================
+ Hits 3888 6422 +2534
- Misses 702 743 +41 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
There was a problem hiding this comment.
🟡 Changes recommended
The no-prefix submodel path can silently ignore an argument binding, and the documented context access emits an internal-variable warning.
Get a fresh assessment by requesting another Copilot review.
Pull request overview
Unifies model arguments, conditioned values, and fixed values as role-tagged model bindings, replacing CondFixContext and missing-based latent selection.
Changes:
- Implements binding precedence, partial overrides, deconditioning, and unfixing.
- Updates compiler, prefix, submodel, and
VarNamedTuplehandling. - Migrates extensions, documentation, and regression tests.
File summaries
| File | Description |
|---|---|
HISTORY.md |
Documents binding semantics. |
docs/src/api.md |
Updates prediction and context APIs. |
docs/src/conditionfix.md |
Documents explicit bindings. |
docs/src/evaluation.md |
Explains evaluation data flow. |
docs/src/onboarding.md |
Updates contributor guidance. |
docs/src/tilde.md |
Revises tilde lowering documentation. |
ext/DynamicPPLComponentArraysExt.jl |
Supports component-backed bindings. |
ext/DynamicPPLInputProvenanceExt.jl |
Traces model-bound inputs. |
ext/DynamicPPLMCMCChainsExt.jl |
Handles absent chain values. |
src/DynamicPPL.jl |
Removes obsolete includes. |
src/compiler.jl |
Generates role-aware model evaluation. |
src/contexts.jl |
Updates context examples. |
src/contexts/conditionfix.jl |
Removes CondFixContext. |
src/contexts/init.jl |
Rejects missing parameters. |
src/contexts/prefix.jl |
Adds prefix templates. |
src/debug_utils.jl |
Adapts model diagnostics. |
src/model.jl |
Implements unified model bindings. |
src/submodel.jl |
Merges parent and child bindings. |
src/subsample.jl |
Accepts argument observations. |
src/varname.jl |
Removes obsolete argument helpers. |
src/varnamedtuple.jl |
Adds nested templates. |
src/varnamedtuple/getset.jl |
Handles nested template indexing. |
src/varnamedtuple/partial_array.jl |
Adds indexed-value preparation. |
src/varnamedtuple/vnt.jl |
Refines lookup and documentation. |
test/compiler.jl |
Tests new lowering behavior. |
test/conditionfix.jl |
Expands binding regression coverage. |
test/context_implementations.jl |
Tests evaluation separation. |
test/contexts.jl |
Removes obsolete context tests. |
test/contexts/init.jl |
Tests missing-value rejection. |
test/debug_utils.jl |
Updates diagnostic tests. |
test/ext/DynamicPPLInputProvenanceExt.jl |
Tests bound-input provenance. |
test/model.jl |
Migrates model behavior tests. |
test/runtests.jl |
Removes deleted test suite. |
test/submodels.jl |
Tests nested binding behavior. |
test/subsample.jl |
Updates observation tests. |
test/utils.jl |
Revises transformation tests. |
test/varnamedtuple.jl |
Tests nested templates and equality. |
Review details
- Files reviewed: 37/37 changed files
- Comments generated: 2
- Review effort level: Balanced
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Contributor
BenchmarksPerformance Ratio: gradient time divided by log-density time. Primal times are for main (94e4e4a) and this PR (7a01c51). For very small models the gradient ratios vary across runs and machines, so compare the raw timings instead. The benchmarks are meant for DynamicPPL developers and mainly catch obvious allocation and type-stability regressions; see the benchmark notes for details. EnvironmentJulia Version 1.13.1 Commit 96ca370cf0e (2026-09-25 19:34 UTC) Build Info: Official https://julialang.org release Platform Info: OS: Linux (x86_64-linux-gnu) CPU: 4 × AMD EPYC 7763 64-Core Processor WORD_SIZE: 64 LLVM: libLLVM-20.1.8 (ORCJIT, znver3) GC: Built with stock GC Threads: 1 default, 1 interactive, 1 GC (on 4 virtual cores) |
Contributor
|
DynamicPPL.jl documentation for PR #1501 is available at: |
The left fold preserves variadic `typejoin` order and its empty result. 1000-element component construction, condition / fix medians: Julia 1.13: 349 / 349 us -> 167 / 168 us. Julia 1.10: 31.6 / 31.6 ms -> 174 / 177 us. Unbound and ordinary argument evaluation stay inferred with zero allocations; their evaluation paths do not use this construction kernel. Assisted-by: Codex <codex@openai.com>
Prune sentinels without subset searches, specialize flat fields, and find removed fixed components without removing every retained sibling. After P3, component decondition / unfix medians at length 1000: Julia 1.13: 4.34 / 18.56 ms -> 43.2 / 226 us. Julia 1.10: 4.30 / 16.87 ms -> 48.4 / 230 us. Whole-field decondition is 1.4 ns / 0 allocations. Argument-default restoration keeps whole-field unfix at 202 ns (1.13), 168 ns (1.10), down from 420 / 432 ns before this change. Unbound and 1000-observation `init!!` retain inference and zero allocations (1.13: 39.5 / 497 ns; 1.10: 46.3 / 3488 ns). Assisted-by: Codex <codex@openai.com>
Use bulk extraction for concrete value types. Keep the existing path for heterogeneous values and growable storage to preserve promotion, container types, and masks exactly. 1000-element conditioned / fixed component query medians: Julia 1.13: 18.1 / 12.0 us -> 0.71 / 0.67 us, 2012 / 1013 allocations -> 7 / 7 allocations. Julia 1.10: 15.6 / 10.4 us -> 0.78 / 0.64 us, 2006 / 1007 allocations -> 4 / 4 allocations. Whole-value queries remain allocation-free at about 1.5 ns. Unbound and 1000-observation `init!!` retain inference and zero allocations (1.13: 39.6 / 510 ns; 1.10: 46.0 / 3458 ns). Assisted-by: Codex <codex@openai.com>
Assisted-by: Codex <codex@openai.com>
Assisted-by: Codex <codex@openai.com>
Assisted-by: Codex <codex@openai.com>
Assisted-by: Codex <codex@openai.com>
Keep keyword splats grouped in defaults until evaluator invocation, so restoration can recover their source argument name and values. Assisted-by: Codex <codex@openai.com>
Convert component replacements exactly before merging, including parent bindings of child arguments. AD observation components need an argument template that can hold their tracer type. Assisted-by: Codex <codex@openai.com>
For a partly latent NamedTuple with a 1000-by-1000 covariate matrix and 1000 responses, warmed logdensity calls on Julia 1.13.1 improve from 8,008,688 bytes / 584.07 us to 8,424 bytes / 7.74 us. The latent response buffer remains private to each evaluation. Assisted-by: Codex <codex@openai.com>
Assisted-by: Codex <codex@openai.com>
Assisted-by: Codex <codex@openai.com>
Assisted-by: Codex <codex@openai.com>
Assisted-by: Codex <codex@openai.com>
Assisted-by: Codex <codex@openai.com>
Keep removal separate: `unfix` restores argument observations. Assisted-by: Codex <codex@openai.com>
Assisted-by: Codex <codex@openai.com>
Let internal read and shape-check exceptions propagate unchanged. Assisted-by: Codex <codex@openai.com>
Assisted-by: Codex <codex@openai.com>
`TransformedValue`, `FixedTransform`, `WithTransforms`, `LinkSome`, and `UnlinkSome` defined `isequal` but not `hash`, so equal values (and `VarInfo`s containing them) hashed differently. Assisted-by: Claude Code <noreply@anthropic.com>
Binding every element of `x ~ MvNormal(...)` without an argument or a schema, as in `condition(m, @varname(x[1]) => 1.0, @varname(x[2]) => 2.0)`, threw "both bound and unbound subvariables". Storage without a template records only the bound indices, so that claim is kept for bindings that leave a hole. Otherwise the error says that parts cannot form the value without a template, and gives the remedy for binding all of it and for leaving some elements unbound. Assisted-by: Claude Code <noreply@anthropic.com>
Growable storage has no template that gives linear indexing a shape. Treating `x[2]` as `x[2, 1]`, or the reverse, silently applies a binding at a different address. Lookup and removal must keep these addresses distinct, including below prefixes and nested slices. A tilde with a different index count from its symbol's growable bindings must fail, even when the model itself mixes index counts. Such partial bindings need storage from a binding schema or argument. Empty bindings remain harmless, and templated storage keeps Julia's indexing semantics. Errors name the binding and tilde with their user-visible prefixes and explain the index counts. Schema guidance names the storage address and uses the binding call's local root and namespace path: merged submodel bindings do not retain the scope of the call that supplied them. Assisted-by: Claude Code <noreply@anthropic.com> Assisted-by: Codex <codex@openai.com>
`_binding_storage` drops growable storage whole, including the whole bindings stored in it, so indices below them could not be resolved. Under an index prefix, after `condition(prefix(m, @varname(q[2])), @varname(q[2].x) => ones(3))`, binding or removing `q[2].x[end]` threw "no storage at `q[2].x`" although the whole binding there owns the shape; likewise for a whole binding at `q[2]` or `q`, both forms in one call, and `p.q[2]`. The same held without a prefix, e.g. `x[1][end]` after `x[1] => ones(3)`. The template for an address now descends growable storage, and its slices, at integer indices or ranges to the binding stored there; the growable level still resolves no `end` or `:`. A slice now bounds the index below it with or without storage, so `x[1:2][3] => v` throws instead of being dropped silently, as it was on fresh storage. This replaces a pre-check that bounded such indices only against existing growable storage. Slices that reach beyond growable storage still contain the owners stored within them. Keep those owners available while resolving the slice so bindings and removals use the same bounds and resolve `end` from the selected value. Valid removals through these slices must remove the selected element and restore the original storage extent. Resizing preserves growable storage so removal cannot introduce a shape owner that prevents later bindings. Linear slices that require growing multidimensional storage are rejected for removal as for binding. Assisted-by: Claude Code <noreply@anthropic.com> Assisted-by: Codex <codex@openai.com>
Value arguments do not use the parameter type hint. Computing it for all arguments repeatedly scans the complete parameter collection in prefixed submodels and triggers costly boxing on Julia 1.13. Query the hint only for `Type` and `TypeWrap` conversion. Delegate all arguments to the two-argument `convert_model_argument` method, using `Any` for value arguments so custom conversions still run. Check that values skip the query and custom type arguments still query the hint. For 100 latent Cartesian-indexed children without type arguments, evaluation drops from milliseconds to under a microsecond on Julia 1.13, with allocations falling from 20402 to 2. Julia 1.12 drops from tens of microseconds to under a microsecond, retaining 2 allocations. The log joint is unchanged. Assisted-by: Codex <codex@openai.com>
Type arguments still need a parameter type hint in every child model. The pair-based fold constructs variable names it never uses and boxes its type accumulator on Julia 1.13, making indexed submodels costly. Fold stored values directly, ignoring unfilled slots and combining leaf types with `promote_type`. This applies to `InitFromParams` and the `DefaultContext` hint. Scan each array-like block once: repeated payload scans make abstract-element blocks quadratic. Allocate a separate visited buffer only when encountering a block, preserving the allocation-free fold for ordinary values. Masks can be immutable, so deduplication must not depend on mutating a mask copy. For 100 children with type arguments on Julia 1.13, evaluation drops from about a millisecond to a few microseconds, and from 20002 to 2 allocations. The log joint is unchanged. Cover nested storage, mutable and immutable masks, empty collections, array-like blocks, and generic numeric types. The allocation, counting and immutable-mask regressions fail before their fixes. Assisted-by: Codex <codex@openai.com>
Boolean masks describe positions, so their storage extent is their length and their slice length is the number of true entries. Recognise these masks as multi-indices when no template supplies the shape. Child slices use their own coordinates. Growing them with parent indices confuses offsets with lengths and retains dimensions dropped by scalar indices, preventing bindings that the equivalent scalar address permits. Use the selected shape in both construction and update paths. Bound mask children by the selected count too, so bindings and removals beyond the slice throw instead of silently dropping the edit. Cover masks and dimension-dropping slices with integer-index controls, including binding and removal diagnostics for the original address. Assisted-by: Codex <codex@openai.com>
Empty-index writes reach a BangBang dimension reduction with no identity, crashing both model LHS assignment and partial value storage. Use a nonempty Cartesian index at the shared array write boundary while keeping `x[]` intact in variable addresses and binding tables. Growable binding storage cannot infer which index arity the model uses. Reject mismatched reads so `x[]` and `x[1]` never silently alias without a template. Deferred indices must resolve before counting so Cartesian indices and masks preserve their dimensionality. Cover sampling, binding removal, submodels, named storage, and gradients through local and deconditioned argument arrays. Assisted-by: Codex <codex@openai.com>
Preserve dense numeric array storage and exact scalar types when a sample is reused as a template. Reject storage that `OfNamedTuple` cannot describe faithfully, including growable arrays with no established shape. Assisted-by: Codex <codex@openai.com>
Masked slots have no value contract and may hold unassigned references. Make compact listings respect the mask, as the plain-text tree already does, while preserving their data and indexing behavior. Assisted-by: Codex <codex@openai.com>
Allow sampled layouts to supply binding storage without pruning unused variables or model arguments. Resolve templates under the model prefix and reject the old local spelling with the required absolute name. Keep existing owners in the edited layer and check conflicts only where the input uses template storage. Use binding template terminology in user-facing documentation, errors, and test descriptions. Defer templates across submodel boundaries until arguments are known. Child arguments must retain their own storage and value conversion. Require concrete parent indices until a whole binding owns the shape. Retain only template leaves overlapping the call's bindings so unused entries cannot replace deferred child storage. Merge deferred entries by role and name; repeated edits must not grow the binding table or change its type. Keep conflicts deferred until child arguments are known. Reuse merged values when attaching templates. Splitting merged child bindings into their layers still allocates; avoid adding another storage representation to bypass it. On Julia 1.12, one intermediate child costs 26,736/24,976 bytes for condition/fix versus 26,336/24,672 before. Twenty identical edits allocate 12,928 bytes, the same as one, versus 58,960 before. Runtime return inference remains `Any`. Assisted-by: Codex <codex@openai.com>
Retain template names as model metadata so ignored entries can be checked without keeping unused storage alive. Check names against the model and reached child namespaces, including prefixed and unprefixed children. Untaken child branches may still use an entry, so warnings do not fail model validation. Match full LHS paths so unused fields under valid roots still warn. Merge template-name tuples without transient vector/set allocation. Runtime binding allocates 8,512 bytes versus 8,832 before this change and 8,368 at the parent revision. Assisted-by: Codex <codex@openai.com>
PartialArray views can contain undefined references behind false mask entries. Copying their backing data eagerly makes valid nested removals throw before reaching the stored binding. Consult the mask when copying reference storage and widening partial storage, and densify only stored values. Keep bulk copying for isbits storage so numeric copies retain their fast path. Assign reference values directly after mask-aware promotion so fallback setters cannot copy unassigned slots when the element type is unchanged. Assisted-by: Codex <codex@openai.com>
An in-bounds array slot remains a valid binding address when its value is unassigned. Separate stored-value lookup from bounds validation so adding such a binding and removing it agree. Traverse slices through views so unrelated unassigned elements do not prevent reaching assigned children. Reject child addresses whose storage is unassigned with the same named error for bindings and removals. Scalar selections retain `getindex` semantics because some array types return zero-dimensional containers from scalar views. Subsampling guards validate membership against their backing storage and keep their wrappers when indexed. Generic array bounds checks and views would inspect the hidden full-data shape or discard batch metadata. Keep storage validation and array membership queries out of Mooncake reverse differentiation because `isassigned` can use unsupported exception handling. Payload lookup and copying remain differentiable. Assisted-by: Codex <codex@openai.com>
Empty selections have no indices that require storage. Taking their maximum raises a reduction error before binding bounds can report the invalid child, while an empty unit range's endpoint can imply storage that the selection never uses. Give both zero extent so bounds checks, growth and template sizing agree without changing nonempty indices. Cover binding and removal errors, empty whole-slice bindings, and one- and two-dimensional storage beside the Boolean-mask controls. Assisted-by: Codex <codex@openai.com>
Covariates cannot receive latent draws, so inspecting their writable storage is unnecessary and can recurse forever through cyclic arrays. Restrict that inspection to arguments that become latent on removal. Assisted-by: Codex <codex@openai.com>
A latent tilde can replace argument storage when the draw does not fit its element type, and always replaces a whole field. Another argument reference then reads stale values: a whole-field tilde on `(a=v, b=v)` can score -3.84 instead of -5.84 without an error on any backend. Reject memory reachable twice within latent arguments, across arguments, or through an argument-bound branch. Name the top-level arguments at binding or removal calls, at placeholder construction, or when recursive evaluation reaches the child. Recognize reshaped and wrapped array memory as shared. The check also covers storage the body only reads. Separate condition/fix values remain valid because latent tildes do not write them. Exclude argument branches replaced by explicit bindings. Only latent arguments need repeat tracking; other arguments need a lookup against latent storage, keeping large covariates inexpensive. Recursive checks use inherited replacements. Walk reachable fields and owning array or memory elements so unfamiliar containers cannot hide latent storage. Wrappers reach their backing storage once, avoiding false sharing on partial removals. Distinct empty arrays own no shared memory; repeated array identities still alias. Compare numeric wrappers by identity, without requiring promotion. Arguments with latent parts reject dictionary keys other than isbits values, `Symbol` or `String` when copied. Arbitrary retained keys can expose latent storage, so use supported keys or a separate covariate. Latent branches also reject custom `Number` leaves whose fields reach mutable storage, including immutable wrappers: retaining them can let a draw mutate caller storage. Keep storage outside the number. Bound and covariate numeric leaves remain accepted. Isbits numbers, `BigFloat`, `BigInt`, and extension-declared opaque AD values remain supported, including ForwardDiff duals with non-isbits values. Base numeric wrappers are retained only when their fields recursively satisfy this rule, so wrapping a storage-bearing custom number cannot let a latent draw mutate caller storage. Reject storage reachable from retained values when an ordinary latent path also reaches it: copying that path would preserve caller storage and let a draw mutate it. Use the opaque-value hook for retention and latent traversal so extensions declare opaque values once. Check at the named copy boundary to catch caller mutation after removal. Keep the shared-storage walk specialized and reuse its graph for argument diagnostics. This avoids reconstructing deep paths, recursive inference in diagnostics, and per-element covariate allocations. Document independent storage and reading a draw through its own address. For ReverseDiff tracked arrays, use an allocating operation such as `m .+ 0`, since `copy(m)` retains storage. Body-created aliases remain outside this check. Assisted-by: Claude Code <noreply@anthropic.com> Assisted-by: Codex <codex@openai.com>
Give binding rules one authoritative statement with their check phases. Remove duplicated and contradictory prose about argument storage. Assisted-by: Codex <codex@openai.com>
Preserve placeholder diagnostics and guards during consolidation. Assisted-by: Codex <codex@openai.com>
Preserve `Missing` precedence and the missing-only initialization guard while avoiding separate placeholder walks. Use `Base.any` to retain its specialization for heterogeneous tuples. At n=1000, `Any` array checks run 5.0-7.9x faster and halve allocations across Julia 1.10, 1.12 and 1.13. Julia 1.10 `Union` array checks fall from 494 to 422 ns. Large heterogeneous numeric tuples allocate nothing on Julia 1.12 and 1.13, with about 3 ms of compilation. Assisted-by: Codex <codex@openai.com>
The old hint failed when followed because prefixed models require absolute names in binding templates. Assisted-by: Codex <codex@openai.com>
Give users an actionable migration for each binding restriction. Keep release notes complete without repeating released changes. Assisted-by: Codex <codex@openai.com>
Avoid rewriting an unchanged property while a latent assignment unwinds a nested optic, and report replacement of a `const` field through the argument binding error. Assisted-by: Codex <codex@openai.com> Assisted-by: Claude Code <noreply@anthropic.com>
ReverseDiff must adapt tracked arrays before latent writes. Rebuild mutable owners when their concrete or const fields cannot accept the adapted child, preserving private tape-connected arguments and their caller values. Assisted-by: Codex <codex@openai.com>
ForwardDiff Dual values may contain custom numeric payloads with mutable storage. Inspect the primal and partials before retaining a Dual so latent copies preserve caller arguments while ordinary Duals stay allocation-free. Assisted-by: Codex <codex@openai.com>
Assisted-by: Codex <codex@openai.com>
Symbolic indices in a child's names resolve only against storage in the layer being edited; templates still apply when the child runs. Give integer-index advice for them, document the rule and cover both binding roles. Assisted-by: Codex <codex@openai.com> Assisted-by: Claude Code <noreply@anthropic.com>
Indexed tilde writes into `g(missing)` raised `MethodError: no method matching possible(::typeof(BangBang._setindex!), ::Missing, ...)`. Check the top-level argument once before using the existing optic setter chain. Assisted-by: Codex <codex@openai.com> Assisted-by: Claude Code <noreply@anthropic.com>
Consolidate repeated models, input tables, and binding/removal checks. Reduce test/conditionfix.jl from 6,882 to 6,712 lines. Covered src/ and ext/ lines are unchanged: 3,477 before and after (3,447 src/ and 30 ext/); no covered lines lost. Assisted-by: Codex <codex@openai.com>
Assisted-by: Claude Code <noreply@anthropic.com>
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Conditioning had three sources of truth: model arguments,
condition's context andfix's. They stacked unpredictably (#1012), ignored each other at arguments (#958), and usedmissingto mark latent variables (#1464). This PR stores one table of bindings on theModel. Observations and fixed bindings form two layers; a later binding replaces an earlier one at the same address, and fixed bindings shadow observations.Binding an argument replaces its value before the body runs, so
f(data)andcondition(f(...); x=data)observe the same value, anddeconditionremoves either. Explicit bindings of enclosing models win, outermost first; a parent's argument-supplied observations never reach a child. A submodel's return slot cannot be bound; bind its LHS variables, such asa.y(#1246). Only whole top-levelmissingornothingarguments andmissingelements of supported top-level argument arrays remain latent (removing them, #1464, remains open); an indexed or field tilde into a whole one, such asx[1] ~ Normal(), throws and asks for concrete storage. Other placeholders throw where a tilde reads them, and placeholders inconditionorfixvalues throw when bound. An argument with latent parts is copied, so latent draws do not reach the caller's value. A positional template such as@of(z = of(Array, 3))(from AbstractPPL) supplies storage for partial bindings of local variables;AbstractDictbinding inputs throw.The full rules are under "Binding rules" in
docs/src/conditionfix.md, andHISTORY.mdgives each break's replacement.With 1 of 1,000 entries deconditioned (
missingon main), a Mooncake gradient drops from 302 to 21 µs; a partial binding costs 22 against 17 µs (Julia 1.12).Fixes #1012; fixes #958; fixes #1246.