Repository navigation
Channel capacity per edge: memoryless channel layer; transducer capacity with and without feedback - #8
Merged
Conversation
…ity with and without feedback
- infoflow.channels: estimate a discrete memoryless channel p(y_t | x_{t-d}) per edge,
reporting the mutual information at the observed input and the Blahut-Arimoto capacity
(infer_multiplex(channels=True) / infer_channels).
- Machine layer: bracket each edge epsilon-transducer's capacity without and with feedback
via sofic.generators.capacity (transducer_capacity), after dropping transient states.
- Bump sofic to pick up the capacity module; docs, references, and tests.
Co-authored-by: Cursor <cursoragent@cursor.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds channel capacity as a way to quantify edges, in two places.
New channel layer (
infoflow.channels)infer_multiplex(..., channels=True)orinfer_channels(data, network)treats each discovered edges -> twith delaydas a discrete memoryless channel from the source's symbol att - dto the target's symbol att. The channel matrix is estimated from the aligned present symbols, pooled over trials.network.channels.datasetreports, per edge:mi: the mutual information at the observed input distribution, using the network's estimator (Miller–Madow by default).capacity: the capacity of the estimated channel, computed with dit's Blahut–Arimotochannel_capacity.n_inputs,n_outputs,n_samples.The layer also keeps the channel matrices (
channels) and the capacity-achieving input distributions (optimal_inputs). It depends only on dit.Machine layer: stateful capacity with and without feedback
Each edge ε-transducer gets
capacity_low/capacity_high(without feedback) andfeedback_capacity_low/feedback_capacity_high, computed withsofic.generators.capacitythrough the newtransducer_capacity. These are brackets because neither capacity can be computed exactly in general.machines.skipped, because its capacity depends on the initial state.MachineSettings(capacity=False)turns this off.capacity_options/feedback_capacity_optionspass keyword arguments to sofic.MachineLayer.summary()shows both brackets.Other changes
uv.lock: sofic bumped to pick up the capacity module (Capacity bounds for epsilon-transducers; trapdoor and Ising example channels sofic#32).docs/channels.rst, a Capacity section indocs/machines.rst, the API page, and seven references (Blahut 1972, Arimoto 1972, Permuter 2008, Sabag 2017, Huleihel 2021, Vontobel 2008, Grigorescu 2024).Notes
rtol=1e-10. At dit's default tolerance it could stop slightly below the observed plug-in mutual information.Test plan
tests/test_channels.py:mi<capacity;tests/test_machines.py:capacity=False.