Skip to content

Channel capacity per edge: memoryless channel layer; transducer capacity with and without feedback - #8

Merged
Autoplectic merged 1 commit into
mainfrom
channel-capacity
Oct 24, 2026
Merged

Autoplectic merged 1 commit into
mainfrom
channel-capacity

Conversation

@Autoplectic

Copy link
Copy Markdown
Member

Summary

Adds channel capacity as a way to quantify edges, in two places.

New channel layer (infoflow.channels)

infer_multiplex(..., channels=True) or infer_channels(data, network) treats each discovered edge s -> t with delay d as a discrete memoryless channel from the source's symbol at t - d to the target's symbol at t. The channel matrix is estimated from the aligned present symbols, pooled over trials. network.channels.dataset reports, per edge:

  • mi: the mutual information at the observed input distribution, using the network's estimator (Miller–Madow by default).
  • capacity: the capacity of the estimated channel, computed with dit's Blahut–Arimoto channel_capacity.
  • n_inputs, n_outputs, n_samples.

The layer also keeps the channel matrices (channels) and the capacity-achieving input distributions (optimal_inputs). It depends only on dit.

Machine layer: stateful capacity with and without feedback

Each edge ε-transducer gets capacity_low/capacity_high (without feedback) and feedback_capacity_low/feedback_capacity_high, computed with sofic.generators.capacity through the new transducer_capacity. These are brackets because neither capacity can be computed exactly in general.

  • Transient states: they are dropped before bounding, since the bounds need a strongly connected channel. A transducer with several recurrent components is recorded in machines.skipped, because its capacity depends on the initial state.
  • Settings: MachineSettings(capacity=False) turns this off. capacity_options / feedback_capacity_options pass keyword arguments to sofic.
  • Summary: MachineLayer.summary() shows both brackets.

Other changes

Notes

  • The no-feedback bracket solves a dynamic program on a belief grid. It takes about 15 s per edge for a two-state transducer, and well under a second for memoryless edges.
  • Blahut–Arimoto runs at rtol=1e-10. At dit's default tolerance it could stop slightly below the observed plug-in mutual information.
  • The channel layer's capacity is a plug-in estimate and is biased upward when there are few samples per input symbol. The docs say so.

Test plan

  • tests/test_channels.py:
    • binary symmetric channel and Z channel (capacity log2(5/4) at P(X=1) = 2/5);
    • a biased source showing mi < capacity;
    • a hypothesis property test that mi <= capacity <= log2 of the smaller alphabet;
    • pickle round trip.
  • tests/test_machines.py:
    • a memoryless edge, where both transducer capacities match the channel layer's capacity;
    • a trapdoor edge, where the feedback bracket contains log2 φ and the no-feedback bracket lies strictly below it;
    • transient-state trimming;
    • capacity=False.
  • Full suite passes (212 tests); ruff and ty are clean; the Sphinx build adds no new warnings.

…ity with and without feedback

- infoflow.channels: estimate a discrete memoryless channel p(y_t | x_{t-d}) per edge,
  reporting the mutual information at the observed input and the Blahut-Arimoto capacity
  (infer_multiplex(channels=True) / infer_channels).
- Machine layer: bracket each edge epsilon-transducer's capacity without and with feedback
  via sofic.generators.capacity (transducer_capacity), after dropping transient states.
- Bump sofic to pick up the capacity module; docs, references, and tests.

Co-authored-by: Cursor <cursoragent@cursor.com>
@Autoplectic
Autoplectic merged commit 09b0f88 into main Oct 24, 2026
10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant