Repository navigation
generative_gap, and describe the lifted search as restricted to lifted generators - #31
Merged
Merged
Conversation
added 3 commits
October 22, 2026 16:25
…tates GenerativeGap(measure, lower, upper, model) compares G[S+:S-] (or C[S+:S-]) with the same quantity restricted to generator states. For the exact measure G[S+:S-] <= C_g <= H[G], so gap = upper - lower bounds C_g - G[S+:S-]. Also available as BidirectionalEpsilonMachine.generative_gap(measure=...).
Lifted generators draw consecutive states independently given the
bidirectional edge; a general generator may couple them. For the interleaved
process Z_t = Z_{t-2} xor Bern(0.1), the product of the chains' minimal
generators (state (U_t, U_{t+1})) is a generator of entropy 1.982, essentially
G[S+:S-], yet violates both lifting conditions, so the lifted search cannot
reach it. State this in the docs, docstrings and CHANGELOG: the exact and Wyner
values are upper bounds over all generators, and generative_gap can be positive
when a non-lifted generator attains the unconstrained value. The functional
search is unaffected (for deterministic channels the coupling is forced).
…it optimum Besides S+, S-, (S+, S-), the dit optimum and random channels, start the continuation from the dit optimum blended with random channels (two strengths, two copies each) and from time-sharing mixtures of it with each eps-machine channel (weights 0.5 and 0.8; disjoint state labels keep the mixture separating). On the 3-state process 012301 this finds the 1.488 generator in two of three seeds instead of one.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Follow-up to #29.
generative_gap(missed #29's merge)generative_gap(measure="exact" | "wyner"), alsoBidirectionalEpsilonMachine.generative_gap(), returnsGenerativeGap(lower, upper, gap, model): the unconstrained common information of(S+, S-)against its value over lifted generator states. For the exact measureG[S+:S-] <= C_g <= H[G], sogapboundsC_g - G[S+:S-]from above. On the (2,∞) process: exact 1.406 → 1.5, Wyner 1.129 → 1.311; zero for first-order Markov processes. This commit was pushed to #29's branch after it was merged, so it is cherry-picked here unchanged.Lifted generators are a proper subclass of generators
#29 described the constrained optimizers as searching "generator states". They search lifted generator states: the HMM built by applying
p(g | s+, s-)at both ends of every bidirectional edge drawsG_0andG_1independently given the edge, and a general generator may couple them.Counterexample: for
Z_t = Z_{t-2} xor Bernoulli(0.1)(two interleaved first-order chains), the product of the chains' minimal generators has stateG_t = (U_t, U_{t+1}), sharingU_{t+1}withG_{t+1}.H[G]G[S+:S-]= 1.9821)I[S+ : S- | G]So it is a generator state that the lifted search cannot reach; sofic reports 1.991–2.0 there. This PR corrects the docs, docstrings and CHANGELOG:
C_g: every lifted model is a genuine generator), not minima over all generators;generative_gapcan be positive when a non-lifted generator attains the unconstrained value;No behavior change beyond
generative_gap. Searching coupled kernels (optimizingp(w_0, w_1 | edge)with fixed marginals) would close the gap and is left for a follow-up.More starts for the lifted search
Besides
S+,S-,(S+, S-), the dit optimum and random channels, the continuation now also starts from:On the 3-state process
012301(G = 1.370, ε-machines 1.584) this finds the 1.488 generator in two of three seeds instead of one; on020213in one of three. Remaining seed dependence comes mostly from dit's unconstrained optimum varying between runs. Cost: the lifted test files take about 4.6 min instead of 1.6 min locally.Tests
tests/test_lifted_common_information.pyandtests/test_minimal_generative_model.py: 64 passed locally; ruff clean.