Add GARI and generic detector layout support to decoder CLIs

README.md

@@ -144,9 +144,78 @@ Using a Detection Event File and Observable Flips:
 ./tesseract --in events.01 --in-format 01 --obs_in obs.01 --obs-in-format 01 --dem surface_code.dem --out decoded.txt
 ```
 
+Tesseract also accepts explicit detector traversal orders in a JSON file. The
+file has the same list-of-lists form as Python's `TesseractConfig.det_orders`,
+and every inner list must be a complete permutation of the DEM detector IDs:
+
+```json
+[[0, 2, 1, 3], [3, 1, 2, 0]]
+```
+
+Pass it with `--detector-orders orders.json`. This replaces the orders normally
+generated by `--num-det-orders`, `--det-order-seed`, and the `--det-order-*`
+method flags.
+
 Tesseract supports reading and writing from all of Stim's standard [output
 formats](https://github.com/quantumlib/Stim/blob/main/doc/result_formats.md).
 
+### Decoding with GARI
+
+Generate a GARI matrix DEM from a source circuit:
+
+```bash
+python src/py/_tesseract_py_util/gari.py \
+    --circuit circuit_file.stim \
+    --prior xor \
+    --out-dir gari_output
+```
+
+This writes `gari_output/circuit_file_gari_xor.dem`. Its physical detector rows preserve the
+source detector IDs, and its added virtual rows form a suffix. GARI-aware detector orders can be
+written in the generic CLI format when reordering is wanted:
+
+```python
+import json
+import stim
+from tesseract_decoder import demutil
+
+circuit = stim.Circuit.from_file("circuit_file.stim")
+gari_dem = stim.DetectorErrorModel.from_file("gari_output/circuit_file_gari_xor.dem")
+orders = demutil.gari.build_detector_orders(circuit, gari_dem, num_det_orders=5)
+with open("gari_orders.json", "w") as f:
+    json.dump(orders, f)
+```
+
+Sample from the source circuit and decode with the generated DEM:
+
+```bash
+./bazel-bin/src/tesseract \
+    --circuit circuit_file.stim \
+    --dem gari_output/circuit_file_gari_xor.dem \
+    --detector-orders gari_orders.json \
+    --sample-num-shots 100 \
+    --sample-seed 1234 \
+    --threads 1 \
+    --pqlimit 1000000 \
+    --beam 5 \
+    --beam-climbing \
+    --no-revisit-dets \
+    --print-stats \
+    --stats-out gari-stats.json
+```
+
+The GARI matrix DEM is a decoding representation and must not be sampled. When a circuit and a
+larger DEM are supplied together, both command-line decoders read or sample the circuit's detector
+prefix and leave the DEM's virtual suffix zero. Their observable counts must agree. When reading a
+source-width event file, pass both `--circuit` and `--dem`; with `--dem` alone the input width is the
+full DEM width.
+
+For matrix analysis, `gari.py --row-order block` writes a `_block.dem` file in the internal
+physical-X, physical-Z, virtual-Z, virtual-X row order. This research form does not accept source
+syndromes as a direct prefix. See
+[GARI transformed matrices](src/py/README.md#gari-transformed-matrices) for supported circuit
+conventions and Python API details.
+
 ### Performance Optimization
 
 Here are some tips for improving performance:

docs/tutorial.ipynb

@@ -935,7 +935,7 @@
       "source": [
         "gari = tesseract_decoder.demutil.gari\n",
         "\n",
-        "gari_dem, gari_layout = gari.circuit_to_gari(\n",
+        "gari_dem = gari.circuit_to_gari(\n",
         "    circuit,\n",
         "    prior_function=gari.tesseract_xor_prior_probabilities,\n",
         ")"
@@ -950,7 +950,8 @@
       "source": [
         "Sample detection events only from the original circuit. The GARI matrix DEM\n",
         "stores the transformed matrices for decoding and is not sampled. Copy the\n",
-        "source syndrome into its physical rows; the added virtual entries stay zero."
+        "source syndrome into the detector prefix; the added virtual entries stay\n",
+        "zero."
       ]
     },
     {
@@ -964,34 +965,26 @@
       "source": [
         "num_shots = 10\n",
         "gari_dets = np.zeros((num_shots, gari_dem.num_detectors), dtype=bool)\n",
-        "gari_dets[:, gari_layout[\"source_to_gari\"]] = dets[:num_shots]"
-      ]
-    },
-    {
-      "cell_type": "markdown",
-      "id": "abc39564",
-      "metadata": {
-        "id": "gari-detector-order"
-      },
-      "source": [
-        "The layout is physical-then-virtual. Setting `num_det_orders=0` selects one\n",
-        "ascending detector order, so Tesseract processes the rows in that order."
+        "gari_dets[:, :dets.shape[1]] = dets[:num_shots]"
       ]
     },
     {
       "cell_type": "code",
       "execution_count": null,
-      "id": "bc0c802c",
+      "id": "3e397ee8",
       "metadata": {
         "id": "gari-decode-example"
       },
       "outputs": [],
       "source": [
-        "short_beam = tesseract_decoder.make_tesseract_sinter_decoders_dict()[\n",
-        "    \"tesseract-short-beam\"\n",
-        "]\n",
-        "short_beam.num_det_orders = 0\n",
-        "gari_decoder = short_beam.compile_decoder_for_dem(dem=gari_dem).decoder\n",
+        "gari_config = tesseract.TesseractConfig(\n",
+        "    dem=gari_dem,\n",
+        "    det_beam=15,\n",
+        "    beam_climbing=True,\n",
+        "    no_revisit_dets=True,\n",
+        "    pqlimit=200_000,\n",
+        ")\n",
+        "gari_decoder = gari_config.compile_decoder()\n",
         "predicted_observables = gari_decoder.decode_batch(gari_dets)\n",
         "logical_failures = np.count_nonzero(\n",
         "    np.any(predicted_observables != obs[:num_shots], axis=1)\n",

docs/tutorial.py

@@ -358,31 +358,31 @@ def run_tesseract_decoder(decoder, dets, obs):
 # %% id="gari-transform-example"
 gari = tesseract_decoder.demutil.gari
 
-gari_dem, gari_layout = gari.circuit_to_gari(
+gari_dem = gari.circuit_to_gari(
     circuit,
     prior_function=gari.tesseract_xor_prior_probabilities,
 )
 
 # %% [markdown] id="gari-syndrome-layout"
 # Sample detection events only from the original circuit. The GARI matrix DEM
 # stores the transformed matrices for decoding and is not sampled. Copy the
-# source syndrome into its physical rows; the added virtual entries stay zero.
+# source syndrome into the detector prefix; the added virtual entries stay
+# zero.
 
 # %% id="gari-sample-example"
 num_shots = 10
 gari_dets = np.zeros((num_shots, gari_dem.num_detectors), dtype=bool)
-gari_dets[:, gari_layout["source_to_gari"]] = dets[:num_shots]
-
-# %% [markdown] id="gari-detector-order"
-# The layout is physical-then-virtual. Setting `num_det_orders=0` selects one
-# ascending detector order, so Tesseract processes the rows in that order.
+gari_dets[:, :dets.shape[1]] = dets[:num_shots]
 
 # %% id="gari-decode-example"
-short_beam = tesseract_decoder.make_tesseract_sinter_decoders_dict()[
-    "tesseract-short-beam"
-]
-short_beam.num_det_orders = 0
-gari_decoder = short_beam.compile_decoder_for_dem(dem=gari_dem).decoder
+gari_config = tesseract.TesseractConfig(
+    dem=gari_dem,
+    det_beam=15,
+    beam_climbing=True,
+    no_revisit_dets=True,
+    pqlimit=200_000,
+)
+gari_decoder = gari_config.compile_decoder()
 predicted_observables = gari_decoder.decode_batch(gari_dets)
 logical_failures = np.count_nonzero(
     np.any(predicted_observables != obs[:num_shots], axis=1)

src/py/README.md

@@ -17,7 +17,7 @@ Explanation of configuration arguments:
 * `verbose` - A boolean flag that, when `True`, enables verbose logging. This is useful for debugging and understanding the decoder's internal behavior, as it will print information about the search process.
 * `merge_errors` - A boolean flag that, when `True`, merges error channels with identical syndrome patterns before decoding. This is enabled by default.
 * `pqlimit` - An integer that sets a limit on the number of nodes in the priority queue. This can be used to constrain the memory usage of the decoder. The default value is `200000`.
-* `det_orders` - A list of lists of integers, where each inner list represents an ordering of the detectors. This is used for "ensemble reordering," an optimization that tries different detector orderings to improve the search's convergence. The default is an empty list, meaning a single, fixed ordering is used.
+* `det_orders` - A list of complete detector-ID permutations in traversal order. This is used for "ensemble reordering," an optimization that tries different detector orderings to improve the search's convergence. The default is an empty list, meaning a single, fixed ordering is used.
 * `det_penalty` - A floating-point value that adds a cost for each residual detection event. This encourages the decoder to prioritize paths that resolve more detection events, steering the search towards more complete solutions. The default value is `0.0`, meaning no penalty is applied.
 * `create_visualization` - A boolean flag that enables decoder visualization output when set to `True`. The default value is `False`.
 * `sparsify_errors` - Enables per-shot sparse error activation. When enabled, all errors up to `sparsify_base_degree` are always active, and selected higher-degree errors are reactivated per shot.
@@ -707,29 +707,34 @@ nice_calibrated_dem = demutil.regeneralize_spatial_dem(
 #### GARI transformed matrices
 
 `demutil.gari.circuit_to_gari` converts a supported correlated CSS Stim
-circuit into a GARI matrix DEM and companion layout for Tesseract. It
-generates a flattened source DEM with `decompose_errors=False`. Detectors must
-follow the repository's fourth-coordinate convention: values `0`–`2` identify
-X detectors and `3`–`5` identify Z detectors.
+circuit into a GARI matrix DEM. It generates a flattened source DEM with
+`decompose_errors=False`. Detectors must follow the repository's
+fourth-coordinate convention: values `0`–`2` identify X detectors and `3`–`5`
+identify Z detectors.
 
 ```python
 import stim
 from tesseract_decoder import demutil
 
 circuit = stim.Circuit.from_file("circuitFile.stim")
-gari_dem, gari_layout = demutil.gari.circuit_to_gari(
+gari_dem = demutil.gari.circuit_to_gari(
     circuit,
     prior_function=demutil.gari.tesseract_xor_prior_probabilities,
 )
 ```
 
-`circuit_to_gari` returns:
+The returned DEM preserves the source detector IDs as a prefix and appends the
+virtual detector rows. For matrix analysis,
+`circuit_to_gari(..., row_order="block")` instead emits the internal
+`[physical X, physical Z, virtual Z, virtual X]` row order. This research form
+does not accept source syndromes as a direct prefix.
 
-* `gari_dem`: the augmented detector and logical matrices stored using Stim
-  DEM syntax.
-* `gari_layout`: a `tesseract.gari_layout.v1` dictionary containing the source
-  and GARI detector counts, the `source_to_gari` detector mapping, and the
-  `physical_then_virtual` detector order.
+`demutil.gari.build_detector_orders(circuit, gari_dem, num_det_orders, ...)`
+uses the source circuit to build BFS, coordinate, or index orders and then
+appends the virtual detector IDs. The resulting list has the same format as
+`TesseractConfig.det_orders` and the Tesseract CLI's `--detector-orders` JSON
+file. It applies to the default source-aligned GARI DEM, not the research-only
+block form.
 
 Related public APIs:
 
@@ -738,13 +743,14 @@ Related public APIs:
 * `demutil.gari.GariTransform` is passed to prior-policy callbacks. It exposes
   the transformed detector and logical matrices, the `U` and `V` projection
   matrices, the source `e_Z`, `e_X`, and `e_Y` column indices, and the source
-  detector mapping.
+  detector mapping into the internal block rows.
 * `paper_prior_probabilities`, `tesseract_xor_prior_probabilities`, and
   `tesseract_lp_max_barred_cost_prior_probabilities` return one probability for
   each transformed GARI column. A user-defined prior can follow the same
   callable interface.
 
 The returned GARI matrix DEM stores transformed matrices for decoding and must
-not be sampled. Sample from the original circuit and use the companion layout
-to place its physical syndrome. See the
+not be sampled. Sample from the original circuit, copy its syndrome into the
+beginning of a zero-filled GARI syndrome, and leave the virtual suffix zero.
+See the
 [GARI tutorial](../../docs/tutorial.ipynb) for a complete decoding example.