Clone-to-run packaging + finish plan for the SL2610 FunctionGemma demo

sl2610-function-calling/.gitignore

@@ -7,6 +7,9 @@ build/
 *.gguf
 *.vmfb
 # Local
+demo.env
+/.toolchain/
+weights/
 *.log
 .DS_Store
 __pycache__/

sl2610-function-calling/README.md

@@ -8,7 +8,25 @@ Goal: the same on-device voice/text → tool-call pipeline — mic → VAD →
 **Moonshine ASR (Torq NPU)** → **FunctionGemma LLM (CPU)** → action — as a single
 **cross-compiled aarch64 binary**, at ≥ the Python app's speed with Kotlin/KMP comfort.
 Heavy models go **DSL → StableHLO → IREE** (Torq for NPU, IREE `llvm-cpu` +NEON for CPU).
-See the full plan in `../../.claude/plans/`.
+Finish plan (clone → run → finetune): **[`docs/FINISH-PLAN.md`](docs/FINISH-PLAN.md)**.
+
+## Quick start (clone → run)
+
+```bash
+./bootstrap.sh                 # creates demo.env, locates the Torq toolchain, checks models
+$EDITOR demo.env               # set BOARD=root@<ip> and GEMMA_GGUF=<your .gguf>
+./gradlew :jvmRun              # host smoke test — no board needed
+
+# board:
+GEMMA_GGUF=... scripts/compile-gemma.sh board   # self-compile FunctionGemma (Python-free)
+BOARD=root@<ip> ./gradlew deployBoard           # cross-compile aarch64, push, run on the SL2610
+```
+
+All machine/board-specific values live in **`demo.env`** (git-ignored; copy of `demo.env.example`).
+Every `scripts/*.sh` sources it, and inline env still wins (`BOARD=root@10.0.0.5 ./gradlew deployBoard`).
+No paths are hardcoded — the repo is meant to be cloned and run. **Teach it your own commands:**
+[`docs/FINETUNING.md`](docs/FINETUNING.md). The one gated dependency is the private Torq compiler wheel —
+`bootstrap.sh` tells you where to place it (see [`docs/TOOLCHAIN-PIN.md`](docs/TOOLCHAIN-PIN.md)).
 
 ## Based on the original Synaptics example
 
@@ -66,14 +84,15 @@ handlers are log-only (no Coral HAT assumed); register your own to drive real ha
 
 ## Dev loop
 ```bash
+./bootstrap.sh                             # one-time: demo.env + toolchain + model checks
 ./gradlew :jvmRun                          # run on host
 ./gradlew :jvmTest                         # common + jvm tests
 ./gradlew :linkReleaseExecutableLinuxArm64 # cross-compile aarch64 binary
 BOARD=root@<board-ip> ./gradlew deployBoard   # build + push + run on the board
 # or directly:
 BOARD=root@<board-ip> sh scripts/deploy.sh --run
 ```
-The board IP changes per boot — set `BOARD` accordingly. Deploy streams the binary over ssh
+Set `BOARD` in `demo.env` (or inline — it changes per boot). Deploy streams the binary over ssh
 (`cat`; the BusyBox board has no rsync/sftp) and adds a `libcrypt.so.1` compat symlink
 (Kotlin/Native links glibc `libcrypt.so.1`; the board ships libxcrypt `libcrypt.so.2`),
 running with `LD_LIBRARY_PATH` so nothing in the system tree is touched.
@@ -84,13 +103,20 @@ surfaces `jvmRun` and `link…LinuxArm64`. Run configs live in `.run/`.
 
 ## Layout
 ```
+bootstrap.sh              one-time setup (demo.env, Torq toolchain, model checks)
+demo.env.example          the single config surface (BOARD, TORQ_PKG, GEMMA_GGUF, board dirs)
 src/commonMain/voicecc/   actions/ (ActionRouter, the 6 tools) + App.kt
 src/jvmMain/              host main + readSystemStatus actual + export/ (DSL→StableHLO:
                           HloBridge, MoonshineEncoderExport, MoonshineWeights)
 src/linuxArm64Main/       board main + Pipeline (ASR→LLM→codec→action) + asr/ (Moonshine on the NPU)
-scripts/                  deploy.sh · iree-compile-torq-docker.sh · gen-config-discover.sh ·
-                          convert_moonshine_weights.py · .docker/ (Torq compiler + gen_config images)
+scripts/                  bootstrap-sourced: deploy.sh · compile-gemma.sh · iree-compile-torq*.sh ·
+                          moonshine_compile_preprocessor.sh · gen-config-discover.sh · .docker/
+docs/                     FINETUNING.md (teach new commands) · TOOLCHAIN-PIN.md (toolchain currency) ·
+                          BOARD-RUNBOOK.md (self-compiled-default + KV-cache board steps) ·
+                          PERF-LOGBOOK.md · GEMMA-KV-BOARD-LOOP.md
 ```
+See **[`docs/FINISH-PLAN.md`](docs/FINISH-PLAN.md)** for what remains to reach a fully self-compiled,
+zero-Python, KV-fast, clone-to-run port.
 
 ## License
 MIT — see [LICENSE](LICENSE) (consistent with SKaiNET). This demo is a complete

sl2610-function-calling/bootstrap.sh

@@ -0,0 +1,85 @@
+#!/usr/bin/env bash
+# bootstrap.sh — one-time setup for the SKaiNET FunctionGemma demo.
+#
+#   ./bootstrap.sh
+#
+# Idempotent. Does NOT touch the board and does NOT download anything gated without asking.
+# It: (1) creates demo.env from the template, (2) locates/links the Torq toolchain into
+# .toolchain/, (3) checks the model files, (4) verifies host prerequisites — then prints the
+# exact next commands. Anything it can't do for you (private Torq wheel, model download) it
+# tells you how to do.
+set -euo pipefail
+
+ROOT="$(cd "$(dirname "$0")" && pwd)"
+say()  { printf '\033[1;36m[bootstrap]\033[0m %s\n' "$*"; }
+warn() { printf '\033[1;33m[bootstrap] WARN\033[0m %s\n' "$*"; }
+ok()   { printf '\033[1;32m[bootstrap]  ok \033[0m %s\n' "$*"; }
+
+# ── 1. demo.env ──────────────────────────────────────────────────────────────────
+if [ -f "$ROOT/demo.env" ]; then
+    ok "demo.env exists — leaving it untouched."
+else
+    cp "$ROOT/demo.env.example" "$ROOT/demo.env"
+    say "created demo.env from demo.env.example — edit BOARD + GEMMA_GGUF for your setup."
+fi
+# shellcheck disable=SC1091
+. "$ROOT/demo.env"
+
+# ── 2. Torq toolchain (.toolchain/torqpkg) ───────────────────────────────────────
+# The g165e12a Torq-fork iree-compile is a PRIVATE, non-PyPI wheel (see docs/TOOLCHAIN-PIN.md).
+# We cannot fetch it for you. Resolution order:
+#   a) TORQ_PKG already points at a valid unpacked package  -> use it (symlink into .toolchain).
+#   b) a wheel at $TORQ_WHEEL (or ./torq_compiler-*.whl)     -> unpack into .toolchain/torqpkg.
+#   c) neither                                               -> explain and continue (host-only still works).
+TOOLCHAIN="$ROOT/.toolchain"; mkdir -p "$TOOLCHAIN"
+resolve_torq() {
+    if [ -x "${TORQ_PKG:-}/iree/compiler/_mlir_libs/iree-compile" ]; then
+        ok "Torq compiler found at TORQ_PKG=$TORQ_PKG"
+        [ -e "$TOOLCHAIN/torqpkg" ] || ln -sfn "$TORQ_PKG" "$TOOLCHAIN/torqpkg"
+        return 0
+    fi
+    local whl="${TORQ_WHEEL:-$(ls "$ROOT"/torq_compiler-*.whl 2>/dev/null | head -n1 || true)}"
+    if [ -n "$whl" ] && [ -f "$whl" ]; then
+        say "unpacking Torq wheel $whl -> .toolchain/torqpkg"
+        rm -rf "$TOOLCHAIN/torqpkg.tmp"; mkdir -p "$TOOLCHAIN/torqpkg.tmp"
+        ( cd "$TOOLCHAIN/torqpkg.tmp" && python3 -m zipfile -e "$whl" . 2>/dev/null || unzip -q "$whl" )
+        rm -rf "$TOOLCHAIN/torqpkg"; mv "$TOOLCHAIN/torqpkg.tmp" "$TOOLCHAIN/torqpkg"
+        ok "Torq compiler unpacked."
+        return 0
+    fi
+    return 1
+}
+if resolve_torq; then :; else
+    warn "Torq toolchain not found. Board (aarch64/NPU) compiles need it."
+    warn "  Get the g165e12a torq_compiler package (Synaptics AI Developer portal / your Coral build),"
+    warn "  then either:  export TORQ_PKG=/path/to/unpacked/torqpkg   (in demo.env), or"
+    warn "                drop torq_compiler-*.whl next to bootstrap.sh and re-run."
+    warn "  Host-only (x64 llvm-cpu) validation via docker still works without it."
+fi
+
+# ── 3. Models ────────────────────────────────────────────────────────────────────
+if [ -f "${GEMMA_GGUF:-}" ]; then
+    ok "FunctionGemma GGUF: $GEMMA_GGUF"
+else
+    warn "FunctionGemma GGUF not found at GEMMA_GGUF='${GEMMA_GGUF:-}'."
+    warn "  Set GEMMA_GGUF in demo.env to your functiongemma-*-Q5_K_M.gguf (or your finetuned one)."
+fi
+[ -d "$ROOT/weights" ] && ok "Moonshine weights/ present." || \
+    warn "Moonshine weights/ absent — see docs (extract from the moonshine-tiny checkpoint) if self-compiling ASR."
+
+# ── 4. Host prerequisites ────────────────────────────────────────────────────────
+for tool in docker ssh; do
+    command -v "$tool" >/dev/null 2>&1 && ok "$tool present" || warn "$tool not on PATH (needed for compile/deploy)."
+done
+[ -x "$ROOT/gradlew" ] && ok "gradlew present" || warn "gradlew missing?"
+
+# ── Next steps ───────────────────────────────────────────────────────────────────
+cat <<EOF
+
+$(say "setup checked. Next:")
+  1) edit demo.env            (BOARD ip, GEMMA_GGUF path)
+  2) ./gradlew :jvmRun        # host smoke test (no board needed)
+  3) GEMMA_GGUF=... scripts/compile-gemma.sh board   # self-compile FunctionGemma for the board
+  4) BOARD=root@<ip> ./gradlew deployBoard           # push + run on the SL2610
+  finetune on your own commands:  docs/FINETUNING.md
+EOF

sl2610-function-calling/demo.env.example

@@ -0,0 +1,31 @@
+# demo.env — local configuration for the SKaiNET FunctionGemma demo.
+#
+# Copy to `demo.env` (git-ignored) and edit for your machine + board:
+#     cp demo.env.example demo.env
+# `bootstrap.sh` creates this for you. Every script under scripts/ sources it if present.
+#
+# All values use `:=` (assign-if-unset), so anything you pass INLINE on the command line
+# still wins, e.g.  `BOARD=root@10.0.0.5 ./gradlew deployBoard`.
+
+# ── Board ────────────────────────────────────────────────────────────────────────
+# ssh/adb target of the SL2610. The board IP usually changes per boot — update this.
+: "${BOARD:=root@192.168.3.26}"
+# Where the board binary is deployed.
+: "${DEST:=/home/root/voicecc-kt}"
+# On-board directories the runtime loads vmfbs from.
+: "${GEMMA_DIR:=/home/root/ireetest}"   # gemma-gen.{vmfb,irpa} (+ KV vmfbs)
+: "${MOON_DIR:=/home/root/moon}"        # moonshine encoder/decoder/preprocessor vmfbs
+
+# ── Host build toolchain ─────────────────────────────────────────────────────────
+# The g165e12a Torq-fork iree-compile package (board-compatible bytecode). See
+# docs/TOOLCHAIN-PIN.md for why this exact snapshot is pinned. `bootstrap.sh` can
+# fetch/unpack it into .toolchain/ ; point this at that dir.
+: "${TORQ_PKG:=${ROOT}/.toolchain/torqpkg}"
+# Stock IREE docker image used for the host llvm-cpu path + onnx import.
+: "${IREE_IMAGE:=iree-cpu-toolchain:3.11.0}"
+: "${IMPORT_IMAGE:=iree-cpu-toolchain:3.11.0}"
+
+# ── Models ───────────────────────────────────────────────────────────────────────
+# FunctionGemma GGUF consumed by scripts/compile-gemma.sh. Provide your own path, or
+# your finetuned GGUF (see docs/FINETUNING.md — the finetune story is "swap this file").
+: "${GEMMA_GGUF:=$HOME/models/functiongemma-physical-ai-v10-Q5_K_M.gguf}"

sl2610-function-calling/docs/BOARD-RUNBOOK.md

@@ -0,0 +1,117 @@
+# Board runbook — the steps that need the SL2610 (and the final packaging)
+
+Everything in `FINISH-PLAN.md` that can't run off-board, plus the last packaging move. Each step is
+independently runnable and has an explicit "done when." Ordered by the plan's sequencing.
+
+Prereqs: `./bootstrap.sh` done, `demo.env` set (BOARD, TORQ_PKG, GEMMA_GGUF), board reachable
+(`ssh $BOARD true`). Toolchain pin rationale + canary gate: `TOOLCHAIN-PIN.md`.
+
+---
+
+## P0 — freeze the working baseline (no board)
+
+Commit the in-flight work so the pre-restructure state is reproducible. Left to you (I don't commit
+without your ask). The dirty trees at 2026-07-11:
+```bash
+# SKaiNET-transformers (feature branch): GemmaKvDecoder.kt, MoonshineKvDecoder.kt, FunctionGemma KV/int8 tests
+# SKaiNET-embedded (functiongemma-selfcompile): compile-gemma.sh, Pipeline.kt, MoonshineRunner.kt,
+#   PERF-LOGBOOK.md, GEMMA-KV-BOARD-LOOP.md, MoonshineKvDecoder.kt, weights/
+git -C <repo> add -A && git -C <repo> commit -m "wip: KV-cache decoders + perf logbook (pre-finish baseline)"
+```
+**Done when:** both repos have clean trees on their feature branches.
+
+---
+
+## P3 — make the board path fully self-compiled + zero Python
+
+### P3.1 Self-compiled encoder + decoder as the default
+Today `MoonshineRunner`/`MoonshineDecoder` default to the vendor prebuilt vmfbs (`modelDir`), with OUR
+vmfbs opt-in via `MOONSHINE_ENCODER_VMFB` / `MOONSHINE_DECODER_VMFB`. Flip the default so ours ship and the
+vendor path is the fallback.
+```bash
+# build ours (see TOOLCHAIN-PIN.md re: the encoder fusion/40-dispatch caveat — this is the open NPU item):
+./gradlew moonshineEncoderMlir && scripts/iree-compile-torq.sh build/mlir/moonshine-encoder.mlir build/mlir/enc.vmfb
+scripts/compile-moonshine-decoder.sh          # our re-decode decoder (CPU) — see the script
+```
+Then in `src/linuxArm64Main/kotlin/voicecc/asr/MoonshineDecoder.kt`, make the `modelDir` default point at
+`$MOON_DIR` with OUR `enc.vmfb`/`decoder_redecode_cpu.vmfb`, and demote the vendor files to an explicit
+`MOONSHINE_VENDOR=1` opt-in.
+**Done when:** `voicecc asr test.wav` → "One, two, three." with no vendor Moonshine vmfb on the board
+(`ls $MOON_DIR` shows only our artifacts), parity with the 3.0 transcripts.
+
+> Open NPU item (tracked): our self-compiled encoder fragments into ~40 dispatches and returns zeros
+> on-device, while the vendor `encoder.vmfb` is one fused dispatch (`docs/synaptics-support/README.md`).
+> Until the vendor fusing compiler / tiling recipe lands, the *self-compiled* encoder default runs on
+> `--device=local-task` (CPU); NPU stays on the vendor vmfb via `MOONSHINE_VENDOR=1`. This is the one
+> place "fully self-compiled" and "on the NPU" still trade off — call it out honestly in the demo README
+> once resolved.
+
+### P3.2 Kill the last runtime Python (VAD in Kotlin)
+`runListen` (`Pipeline.kt`) still `popen`s `scripts/vad_capture.py` (Silero VAD) via a venv. Replace it:
+- **Option A (simplest, ship first):** an energy/zero-crossing gate in Kotlin/Native (ALSA capture via
+  cinterop) — no new model. Good enough for push-to-talk / quiet rooms.
+- **Option B (parity):** author Silero VAD in the DSL (needs an LSTM layer upstream — GRU is the template,
+  guardrail discussion per PLAN Phase 4.1) or run Silero ONNX as a small IREE vmfb via the existing
+  `torq-run-module` binding.
+Then delete the `venvPython`/`helper` popen in `runListen` and `scripts/vad_capture.py`.
+**Done when:** `voicecc listen mic` segments utterances with **no Python process** (verify on-board:
+`ps | grep -i python` empty during a `listen` session), parity vs the Python helper on recorded wavs.
+
+### P3.3 Retire bring-up stubs (no board)
+Delete `src/jvmMain/kotlin/voicecc/export/{HloBridge,SdpaTrace}.kt` (PLAN 2.4) once nothing references them
+(`grep -rn HloBridge\\\|SdpaTrace src/`). Retire build-only Python helpers already superseded by the DSL
+(`add_argmax_perpos.py`, `make_f16.py`); keep one-time weight-extraction helpers but document them as
+offline steps, not runtime.
+**Done when:** `grep -rn 'popen\|\.py' src/` shows no *runtime* Python; README's "interim vendor" note is gone.
+
+---
+
+## P4 — KV-cache performance (board-verify the already-authored graphs)
+
+The graphs + native loop exist; only the first-board confirmation of trace-derived unknowns remains.
+
+### P4.1 FunctionGemma KV — follow `GEMMA-KV-BOARD-LOOP.md` exactly
+```bash
+GEMMA_KV=1 scripts/compile-gemma.sh board          # builds gemma-prefill.vmfb + gemma-with-past.vmfb
+# deploy all three vmfbs + gemma-gen.irpa to $GEMMA_DIR, then:
+GEMMA_KV=1 BOARD=root@<ip> ./gradlew deployBoard
+GEMMA_KV=1 VOICECC_PROFILE=1 voicecc gen "turn the light on"
+```
+Confirm on the first run (both are silent-corruption traps, both caught by the oracle):
+1. **dynamic `?` in `gemma_with_past` compiles** (first real test of the sentinel-relax; if iree rejects
+   the dynamic concat, fall back to fixed-pad+mask — noted in the loop doc).
+2. **per-block K-vs-V output order** — the converter's terminal ordering may emit V,K not K,V. Verify via the
+   prefill-then-1-step logit match (`GEMMA-KV-BOARD-LOOP.md` §CAVEAT); swap if garbage.
+**Done when:** the loop reproduces the oracle `[262146,236769,3255,718,498,1373,262152,106]` and
+`PERF-LOGBOOK.md` gets the ms/token row (expect the O(seq²)→O(seq) collapse vs the ~6 s/token baseline).
+Then make `GEMMA_KV=1` the default in `Pipeline.kt`.
+
+Follow-up worth doing: have the exporter emit a tiny `.manifest` mapping each output slot → role (K/V/token),
+so the K-vs-V order stops being a guess. Small change in `kgemma/FunctionGemmaExport.kt`.
+
+### P4.2 Moonshine KV — same drill
+`MoonshineKvDecoder` is a `⚠️ BOARD-UNVERIFIED DRAFT` with the same two unknowns (per-block K-vs-V,
+vmfb entry-fn names). Verify `MOONSHINE_KV=1 voicecc asr test.wav` transcribes at parity, then default it on.
+**Done when:** KV ASR transcribes correctly; row added to `PERF-LOGBOOK.md`.
+
+---
+
+## Final packaging — extract the standalone `skainet-fc-demo` repo
+
+The demo dir is already clone-ready (bootstrap + demo.env + externalized scripts). Two things remain to make
+it a *standalone* clone that doesn't need its 4 sibling repos:
+
+1. **The `kgemma` FunctionGemma exporter is a Gradle task in `SKaiNET-transformers`**, not a published CLI —
+   `compile-gemma.sh` does `cd ../../SKaiNET-transformers && ./gradlew :llm-runtime:kgemma:exportFunctionGemma`.
+   For a standalone repo, publish that exporter as a runnable artifact (a thin `llm-apps` CLI or a fat jar on
+   local Maven) and have `compile-gemma.sh` invoke the published tool instead of the sibling checkout. Same
+   for `moonshineEncoderMlir`/decoder export tasks.
+2. **`CompactCodec.TOKEN_TO_NAME` is a private hardcoded map upstream** — adding a tool means patching the
+   dependency (see `FINETUNING.md`/`examples/custom-command`). Make the map injectable (constructor param or
+   a `ToolMap` the demo passes in) so custom tools need zero upstream edits. Small, high-leverage upstream tweak.
+
+Then: `git init skainet-fc-demo`, move this dir's contents in, set the Maven deps to published `0.35.0`
+(already the version in `gradle/libs.versions.toml`), drop the composite `includeBuild`s, and re-run the
+Quick start from a fresh clone as the acceptance test.
+**Done when:** a clone into a fresh temp dir → `./bootstrap.sh` → `./gradlew :jvmRun` works with no sibling
+repos present, and `deployBoard` runs on the board.