Skip to content

evaluation: replace the context hierarchy with explicit inputs - #1503

Open
yebai wants to merge 8 commits into
prefixesfrom
contexts
Open

yebai wants to merge 8 commits into
prefixesfrom
contexts

Conversation

@yebai

@yebai yebai commented Sep 16, 2026 •

Copy link
Copy Markdown
Member

This removes model-attached contexts and the parent/leaf context hierarchy. Evaluation receives an explicit Context holding the RNG, initialisation strategy and transform strategy; it reads inputs only from the model and that context, writes evaluation outputs only to the VarInfo, and returns the model's value separately:

ctx = Context(rng, InitFromParams(values, nothing), UnlinkAll())
retval, vi = evaluate!!(m, ctx, VarInfo())  # parameters absent from `values` throw

HISTORY.md and the migration guide give each removed name's replacement, e.g. evaluate!!(contextualize(m, ctx), vi) → evaluate!!(m, ctx, vi).

Turing.jl needs changes. Particle MCMC calls the removed setleafcontext. The SMCContext tilde_assume!! overload becomes an initialisation strategy evaluated with Context(particle.rng, strategy, UnlinkAll()), and its get_param_eltype overload moves to that strategy. Its context-dispatched tilde_observe!! and store_coloneq_value!! overloads are no longer called; the varinfo copy they make moves to ProduceLogLikelihoodAccumulator's accumulate_observe!!. make_evaluate_args_and_kwargs takes ctx.

Gibbs reads model.args, which bindings no longer update; build_values_vnt should merge only arguments without LHS variables beneath conditioned(model) and fixed(model), and keyword splats are now nested in model.defaults. gibbs_conditional.jl:120 reads the removed model.context. Bindings now override argument values and whole missing arguments remain latent (see #1501), which removes the stated reason for check_no_missing_arguments.

In tests, ADTypeCheckContext becomes an accumulator that checks the element type of the values it sees, conditioned(model.context) becomes conditioned(model), Dict bindings become pairs or NamedTuples, and InitFromParams rejects missing values. @vnt and @template are no longer exported, so partial bindings use an @of template, and get_param_eltype(vi, ctx) becomes get_param_eltype(strategy).

Closes #1469.

@yebai
yebai added this pull request to stack #1505 September 16, 2026 21:16
@codecov

codecov Bot commented Sep 16, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 90.20%. Comparing base (5aff9d6) to head (3febd24).

Additional details and impacted files
@@             Coverage Diff              @@
##           prefixes    #1503      +/-   ##
============================================
+ Coverage     89.95%   90.20%   +0.25%     
============================================
  Files            51       49       -2     
  Lines          7106     7058      -48     
============================================
- Hits           6392     6367      -25     
+ Misses          714      691      -23     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

The architecture-wide breaking API migration affects generated evaluators, extensions, threading, and downstream integrations.

Pull request overview

Replaces model-attached context hierarchies with explicit evaluation inputs, implementing issue #1469’s one-way evaluation flow.

Changes:

  • Introduces Context for RNG, initialization, and transforms.
  • Removes legacy context hierarchy and updates compiler/evaluation APIs.
  • Migrates tests, extensions, documentation, and benchmarks.
File summaries
File Description
Project.toml Raises Compat minimum.
HISTORY.md Records context migration.
src/DynamicPPL.jl Updates exports and includes.
src/accumulators.jl Clarifies thread-safe promotion.
src/compiler.jl Passes context into evaluators.
src/contexts.jl Removes context hierarchy.
src/contexts/default.jl Removes DefaultContext.
src/contexts/init.jl Replaces InitContext with Context.
src/debug_utils.jl Updates evaluator inspection.
src/logdensityfunction.jl Updates context terminology.
src/model.jl Removes model-attached contexts.
src/submodel.jl Propagates explicit contexts.
src/test_utils.jl Removes context test utilities.
src/test_utils/contexts.jl Deletes obsolete utilities.
src/varinfo.jl Uses explicit contexts.
ext/DynamicPPLBridgeStanExt.jl Migrates Stan evaluation.
ext/DynamicPPLInputProvenanceExt.jl Updates model reconstruction.
test/compiler.jl Updates compiler expectations.
test/conditionfix.jl Migrates condition/fix evaluation.
test/context_implementations.jl Tests explicit input/output separation.
test/contexts/init.jl Migrates initialization tests.
test/debug_utils.jl Updates debug tests.
test/ext/DynamicPPLBridgeStanExt.jl Adds explicit-context coverage.
test/ext/DynamicPPLForwardDiffExt.jl Adds numeric and AD coverage.
test/model.jl Migrates model evaluation tests.
test/runtests.jl Updates test ordering/imports.
test/submodels.jl Migrates submodel tests.
test/threadsafe.jl Migrates thread-safe tests.
test/utils.jl Updates utility evaluation.
test/varinfo.jl Migrates VarInfo tests.
docs/src/accs/threadsafe.md Updates thread-safe example.
docs/src/accs/values.md Updates accumulator examples.
docs/src/api.md Documents the new API.
docs/src/evaluation.md Explains explicit evaluation flow.
docs/src/migration.md Provides migration guidance.
docs/src/onboarding.md Updates contributor guidance.
docs/src/tilde.md Updates tilde-pipeline documentation.
benchmarks/benchmarks.jl Adds indexed-submodel benchmark.
Review details
  • Files reviewed: 37/38 changed files
  • Comments generated: 0
  • Review effort level: Balanced

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

@github-actions

Copy link
Copy Markdown
Contributor

DynamicPPL.jl documentation for PR #1503 is available at:
https://TuringLang.github.io/DynamicPPL.jl/previews/PR1503/

@github-actions

github-actions Bot commented Sep 16, 2026 •

Copy link
Copy Markdown
Contributor

Benchmarks

Performance Ratio: gradient time divided by log-density time.

Primal times are for main (94e4e4a) and this PR (3febd24). For very small models the gradient ratios vary across runs and machines, so compare the raw timings instead. The benchmarks are meant for DynamicPPL developers and mainly catch obvious allocation and type-stability regressions; see the benchmark notes for details.

================================================================================================================================
                                                                     primal                            gradient                 
                                                             ----------------------  -------------------------------------------
Model                                         dim    linked        main          PR     FwdDiff    RvsDiff    Mooncake    Enzyme
--------------------------------------------------------------------------------------------------------------------------------
Simple assume observe*                          1     false     5.56 ns     5.56 ns        9.09    1362.27       10.43      5.81
Simple assume observe*                          1      true     5.56 ns     5.56 ns        9.54    1377.80       10.44      6.14
Smorgasbord                                   201     false     5.37 μs     5.31 μs       73.55     148.77        5.91      1.40
Smorgasbord                                   201      true      6.6 μs     6.46 μs       77.66     173.33        6.11      1.51
Smorgasbord fixed                             100     false           —      2.6 μs       50.37     182.26        6.94      1.23
Smorgasbord fixed                             100      true           —     3.05 μs       62.35     198.07        7.32      1.45
Smorgasbord partial condition                 201     false           —     5.99 μs       67.73     130.26        7.60       err
Smorgasbord partial condition                 201      true           —     7.18 μs       78.00     145.77        7.29       err
Smorgasbord partial decondition               202     false           —     8.55 μs       72.81      97.76        5.45       err
Smorgasbord partial decondition               202      true           —     9.59 μs       81.83     115.17        5.54       err
NamedTuple-field LHS variable*                  1     false           —     5.57 ns        9.64    1507.29       10.49      6.16
NamedTuple-field LHS variable*                  1      true           —     5.57 ns        9.63    1525.40       10.49      6.19
MvNormal argument-supplied observation*         1     false           —     82.9 ns        2.61     887.32        3.50      2.83
MvNormal argument-supplied observation*         1      true           —     92.0 ns        2.42     696.84        2.98      2.29
Loop univariate 1k                           1000     false     17.0 μs     17.2 μs      902.32     336.70        4.74      0.75
Loop univariate 1k                           1000      true     18.4 μs     18.8 μs     1327.59     312.13        4.58      0.69
Multivariate 1k                              1000     false     1.24 μs     1.08 μs     1284.02     884.98        1.91      4.04
Multivariate 1k                              1000      true     1.24 μs     1.11 μs     1343.69     933.97        1.88      3.12
Loop univariate 10k                         10000     false    163.0 μs    163.0 μs    21135.56     366.65        5.14      0.69
Loop univariate 10k                         10000      true    177.0 μs    177.0 μs    23857.44     345.35        4.89      0.62
Multivariate 10k                            10000     false     10.1 μs     9.85 μs    27101.55    1067.89        1.75      2.99
Multivariate 10k                            10000      true     10.5 μs     10.0 μs    26726.18    1045.60        1.83      2.93
Dynamic                                        15     false           —     1.42 μs         err      44.62       15.86      5.70
Dynamic                                        10      true      1.8 μs     1.93 μs        1.65      57.33       22.91      7.42
Submodel*                                       1     false     5.57 ns     5.57 ns        9.68    1500.93       10.01      6.15
Submodel*                                       1      true     5.57 ns     5.57 ns        9.02    1651.12       10.07      6.21
Indexed submodels 3k                            1     false           —     3.29 μs       10.28    2270.14       25.22      1.18
Indexed submodels 3k                            1      true           —     3.36 μs       10.63    2174.23       22.05      1.24
Indexed submodels conditioned                   1     false           —     3.31 μs       11.84    2135.64       30.61      2.80
Indexed submodels conditioned                   1      true           —     3.31 μs       12.91    2222.51       31.36      2.78
LDA                                             6      true     3.89 μs     3.64 μs        1.59      13.67       28.87     36.97
================================================================================================================================
Environment
Julia Version 1.13.1
Commit 96ca370cf0e (2026-09-25 19:34 UTC)
Build Info:
  Official https://julialang.org release
Platform Info:
  OS: Linux (x86_64-linux-gnu)
  CPU: 4 × AMD EPYC 7763 64-Core Processor
  WORD_SIZE: 64
  LLVM: libLLVM-20.1.8 (ORCJIT, znver3)
  GC: Built with stock GC
Threads: 1 default, 1 interactive, 1 GC (on 4 virtual cores)

yebai added 3 commits October 10, 2026 21:50
Assisted-by: Codex <codex@openai.com>
Assisted-by: Codex <codex@openai.com>
yebai added 5 commits October 10, 2026 22:16
Assisted-by: Claude Code <noreply@anthropic.com>
Assisted-by: Codex <codex@openai.com>
Assisted-by: Codex <codex@openai.com>
Assisted-by: Codex <codex@openai.com>
Assisted-by: Codex <codex@openai.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Separate model-evaluation inputs from outputs

2 participants