Skip to content

webui: wire the YuE2 NAR LoRA adapter into the UI - #614

Merged
0xShug0 merged 1 commit into
0xShug0:mainfrom
christopherthompson81:yue2-nar-lora-ui
Sep 22, 2026
Merged

0xShug0 merged 1 commit into
0xShug0:mainfrom
christopherthompson81:yue2-nar-lora-ui

Conversation

@christopherthompson81

@christopherthompson81 christopherthompson81 commented Sep 19, 2026

Copy link
Copy Markdown
Contributor

Wires the YuE2 NAR LoRA adapter into the WebUI, as requested on #499: "I added NAR LoRA support in the core, but haven't wired it into the UI yet. Would appreciate it if anyone who's better at UI could pick this up." It missed 0.8.1, which shipped on 2026-09-17.

+105 / −56 across 4 files, WebUI only. The adapter store and server-side HF download from the earlier revision are gone, along with the LoRA classifier.

Why this is not just a config entry

My previous revision of this PR added yue2.nar_lora and yue2.nar_lora_scale to model_params.json and nothing else. That does not work, and I only found out by running it.

Yue2Panel.svelte is hand-written and renders a curated set of parameters by name — componentParamNames, coreParamNames, abcParamNames, semanticParamNames, plannerParamNames — plus a bespoke block for ar_lora. It replaces the generic parameter renderer for this family, so a catalog entry that no block draws is invisible. Against a running server the options were present in the bundled catalog and nothing rendered them.

So the change is:

  • model_params.json gains the pair. nar_lora_scale carries an info string for the trap in its spec description: it scales the LoRA deltas only, and any full vae2llm/llm2vae projection replacement in the adapter stays at full strength. The two AR entries gain the "reload the model" hint that main_gguf and vae_gguf already had and that all four LoRA options require.
  • Yue2Panel.svelte gains the NAR block mirroring AR — path, strength, picker, its own error line. selectLora takes the branch instead of hardcoding ar_lora, so one uploader serves both and a failed NAR upload does not post an error under the AR field.
  • +page.svelte seeds nar_lora/nar_lora_scale from the loaded model's session options beside the AR pair, for a server without UI management.

One thing beyond the NAR work, flagged rather than buried

It also ungates the AR picker. That is a pre-existing defect, not part of wiring NAR, but I could not leave it: disabled={!server?.ui_management || ...} disables AR LoRA upload in every default build, and a NAR picker copied from it would have been equally dead.

The endpoint it calls, /v1/ui/upload, requires ui_enabled or ui_management and is compiled outside the AUDIOCPP_HAS_NATIVE_MODEL_MANAGER guard. Confirmed against a --ui-only server reporting ui_management: false:

POST /v1/ui/upload -> {"path":"/tmp/audiocpp-ui-.../1-probe.safetensors","bytes":2048}  HTTP 200

The text and strength inputs are ungated for the same reason: typing a server-side path needs no management rights, and the main_gguf/vae_gguf selectors beside them were never gated. Happy to split this into its own PR if you would rather keep this one strictly NAR — it is a three-line change either way.

The generated bundle

dist/index.html is a genuine rebuild this time. My previous revision patched it surgically to avoid churn, which a config-only change allows; a template change does not. The line diff is 84 lines, but it is a whole-bundle replacement — my toolchain's minifier output differs throughout from whatever produced the committed one. Tell me if you would rather regenerate it yourself.

Verification

npm run check passes (277 files, 0 errors), npm run build succeeds. Rebased onto current main.

Exercised in the browser against audiocpp_server --ui on CUDA, serving a YuE2 3B package with both adapters present, and confirmed working. The four controls render, the pickers are usable on a server without UI management, and the options reach the session.

Earlier revisions of this PR carried a "not verified end to end" caveat. That is discharged — this one was driven through the actual UI before being pushed, which is also how the config-only revision was caught doing nothing.

🤖 Generated with Claude Code

https://claude.ai/code/session_01TMmMgd5xNGnjsQgybuUiK3

@christopherthompson81
christopherthompson81 marked this pull request as draft September 19, 2026 22:06
@christopherthompson81 christopherthompson81 changed the title webui: expose the YuE2 NAR LoRA adapter, not just the AR one webui: make YuE2 LoRA adapters usable — NAR control, persistent store, HF download Sep 19, 2026
@christopherthompson81
christopherthompson81 marked this pull request as ready for review September 19, 2026 23:26
@0xShug0

0xShug0 commented Sep 20, 2026

Copy link
Copy Markdown
Owner

@christopherthompson81 My main concern is that the shared server should store, list, and download adapters, not contain logic specific to a model’s LoRA layout. The "loadable" check is misleading. The PR actually marks a public Whisper LoRA as "loadable": true, "branch": "ar". If we add compatibility checks in the server, they’ll quickly become hard to maintain as more models gain LoRA support.

Also maybe just make the confinement claim more accurate: The new file endpoints can escape the models root through a symlink. The guard compares lexical paths, not where they resolve on disk. I created a symlinked model directory inside a temporary root, then called /v1/ui/loras/delete; it returned HTTP 200 and deleted my test file outside that root. The UI/server are intentionally minimal and don’t include a built-in security layer, so this scenario may simply be out of scope.

christopherthompson81 added a commit to christopherthompson81/audio.cpp that referenced this pull request Sep 20, 2026
Review feedback on 0xShug0#614, both points.

The store no longer says whether an adapter suits a model. It said
"loadable": true and "branch": "ar" for a public Whisper LoRA, which is
what the review caught. That was not an edge case: lora_A/lora_B and
self_attn/mlp are PEFT conventions, not YuE2's, so every PEFT adapter
for any transformer passed. Reading the header of the file from the
report confirms it -- 288 tensors, lora_A/lora_B throughout, names like
base_model.model.model.decoder.layers.0.encoder_attn.k_proj.lora_A.weight,
no metadata to contradict them.

What it reports now is the file's shape, not its fit: "unfused_lora",
true when a file has the lora_A/lora_B pairs every LoRA-consuming family
here reads -- yue2 and vibevoice both do -- and false for a ComfyUI
fused layout, which none of them can read. Whether an unfused adapter
matches a particular model needs that model's tensor layout, so the
loader answers it, which is also where a mismatch already surfaces. The
branch sniffing and the yue2_lora_branch and lora_rank_separate metadata
keys are gone; nothing model-specific is left in app/server.

Losing the branch verdict would have left both pickers offering every
adapter, so the slot is recorded instead of inferred: adapters are
filed under <model>/loras/<group>/ with the group named by the caller.
The panel passes "ar" or "nar" when it uploads or downloads, and offers
each slot its own folder. The server treats the group as an opaque
directory name -- it neither invents one nor reads meaning into it -- so
a family with one adapter slot passes none and gets the flat layout,
which is also what a file dropped in by hand gets, offered everywhere.
Relative options already resolve as model_root / path (yue2/session.cpp),
so "loras/ar/<file>" needs nothing from the loader.

The confinement check now compares resolved paths. It compared lexical
ones, so "<root>/link/loras" read as inside the root however link was
defined, and the review's reproduction -- a symlinked model directory,
then /v1/ui/loras/delete -- returned 200 and removed a file outside the
root. weakly_canonical() follows the links and still tolerates a store
that has not been created yet. Upload additionally checks the
destination and its .partial with symlink_status() rather than exists(),
since exists() follows a dangling symlink and the write lands on its
target. Group names are validated as a single component.

Verified against the built server with the model manager enabled --
note that the endpoints are behind AUDIOCPP_BUILD_NATIVE_MODEL_MANAGER,
so a default build compiles none of this:

  upload group=ar    -> 200  loras/ar/good.safetensors
  upload group=nar   -> 200  loras/nar/good.safetensors
  ComfyUI fused file -> 400  rejected: this engine needs the unfused file
  delete via symlinked model dir  -> 400, file outside the root survives
  upload via symlinked model dir  -> 400, nothing planted
  group "../../../escape"         -> 400
  model "<root>/../outside"       -> 400

and the Whisper adapter from the report now returns unfused_lora with no
claim about which model or branch it belongs to.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TMmMgd5xNGnjsQgybuUiK3
@christopherthompson81

Copy link
Copy Markdown
Contributor Author

Both points were right, and both are fixed in f04f295f (rebased onto current main).

The compatibility check is gone

You're right that it wasn't just wrong on that file — it was wrong by construction. lora_A/lora_B and self_attn/mlp are PEFT conventions, not YuE2's, so every PEFT adapter for any transformer passed. I read the header of the file you linked to confirm: 288 tensors, lora_A/lora_B throughout, names like base_model.model.model.decoder.layers.0.encoder_attn.k_proj.lora_A.weight, and no metadata to contradict them. Nothing in the classifier ever checked that the names belonged to YuE2, so "loadable": true, "branch": "ar" was the only possible answer.

The server now reports the file's shape, not its fit:

  • unfused_lora — true when a file has the lora_A/lora_B pairs every LoRA-consuming family here reads (yue2 and vibevoice both do), false for a ComfyUI fused layout, which none of them can read.
  • The branch sniffing is gone, along with the yue2_lora_branch and lora_rank_separate metadata keys. There is no model-specific logic left in app/server.

Whether an unfused adapter suits a given model needs that model's tensor layout, so the loader answers it — which is where a mismatch already surfaced anyway.

Which dropdown, without a compatibility check

Dropping the branch verdict would have left both pickers offering every adapter, so the slot is recorded when the file is stored rather than inferred afterwards. Adapters are filed under <model>/loras/<group>/, with the group named by the caller. The panel passes ar or nar when it uploads or downloads; each slot offers its own folder.

The server treats the group as an opaque directory name — it neither invents one nor reads meaning into it. A family with one adapter slot passes none and gets the flat layout, which is also what a file dropped in by hand gets, and those are offered in every slot. Relative options already resolve as model_root / path (yue2/session.cpp), so loras/ar/<file> needs nothing new from the loader.

That keeps the maintenance cost you were worried about at zero: a new model with LoRA support picks its own folder names and the server learns nothing about it.

Path confinement

Fixed rather than scoped out — it was a few lines, which seemed better than arguing about the threat model. The check compared lexical paths, so <root>/link/loras read as inside the root however link was defined. It compares resolved paths now (weakly_canonical, which still tolerates a store that hasn't been created yet).

Two things beyond your reproduction: upload checks the destination and its .partial with symlink_status() rather than exists(), because exists() follows a dangling symlink and the write lands on its target; and group names are validated as a single path component.

Your exact scenario, against the built server:

delete via symlinked model dir  -> 400, file outside the root survives
upload via symlinked model dir  -> 400, nothing planted
group "../../../escape"         -> 400
model "<root>/../outside"       -> 400

upload group=ar    -> 200  loras/ar/good.safetensors
upload group=nar   -> 200  loras/nar/good.safetensors
ComfyUI fused file -> 400  rejected: this engine needs the unfused file

And the Whisper adapter from your comment now returns unfused_lora with no claim about which model or branch it belongs to.

One note for anyone verifying: these endpoints are behind AUDIOCPP_BUILD_NATIVE_MODEL_MANAGER, so a default build compiles none of this. I only caught that because my first build "succeeded" without compiling lora_store.cpp at all.

@0xShug0

0xShug0 commented Sep 21, 2026

Copy link
Copy Markdown
Owner

@christopherthompson81 I think this PR should focus on the immediate task: adding the NAR LoRA controls to the WebUI.

Persistent uploads are a separate usability feature, and server-side Hugging Face downloads are another substantial feature. Both need more review and validation. This is also a question of ownership and responsibility boundaries. Say if audio.cpp handles LoRA classification, it assumes responsibility for making that classification robust and covering the relevant edge cases. For example the current PR accepts a safetensors file containing only lora_A or only lora_B, even though a valid LoRA requires matching A/B pairs. The model loader already has the tensor layout and remains the actual authority on compatibility. Adding a weaker classifier to the shared server creates two definitions of validity that can disagree: the WebUI may offer and download an adapter as usable, only for the loader to reject it later.

I suggest limiting this PR to exposing yue2.nar_lora and yue2.nar_lora_scale, along with any small UI changes required to pass those options correctly, so we can make this feature available soon.

@christopherthompson81 christopherthompson81 changed the title webui: make YuE2 LoRA adapters usable — NAR control, persistent store, HF download webui: expose the YuE2 NAR LoRA adapter, not just the AR one Sep 21, 2026
@christopherthompson81

Copy link
Copy Markdown
Contributor Author

Narrowed to what you asked for: yue2.nar_lora and yue2.nar_lora_scale, nothing else. +5 / −3 across 2 files — down from +1154 / −102 across 11. No C++, no new endpoints. Rebased onto current main, so the conflict is gone.

Gone: the persistent store, the HF download, the two model_spec/package.cpp commits that only existed to serve the download path, and the classifier.

On the classifier specifically — your A/B-pair point is right, and it is the third wrong answer that thing gave. First the branch verdict, then the Whisper adapter, now accepting a file with only one side of the pair. That is a pattern rather than three bugs, and "two definitions of validity that can disagree" is the accurate description of why. The loader has the tensor layout and is the only thing that can answer the question, so it should be the only thing that tries.

I parked the removed work on yue2-lora-store-parked rather than deleting it, but I am not proposing it — and the path-confinement fixes in there only mattered for endpoints that no longer exist.

One thing worth your attention in what remains: dist/index.html is edited surgically rather than regenerated. A full rebuild in my environment rewrites the whole bundle — 8,683 differing chunks against the committed one, from a different minifier and per-build hashes — so a four-option change would arrive as an unreviewable diff. I built from source and compared the embedded yue2 block: byte-identical to what a real build produces. Happy to push a genuine rebuild instead if you would prefer the artefact to be authentically generated; it will just be large and none of it meaningful.

0xShug0#586 added NAR LoRA to the core and the maintainer asked on 0xShug0#499 for someone
to wire it into the UI: "I added NAR LoRA support in the core, but haven't
wired it into the UI yet." It missed 0.8.1, which shipped on 2026-09-17.

yue2.nar_lora and yue2.nar_lora_scale have been reachable from the CLI and the
server since 0xShug0#586, but not the WebUI. Adding them to model_params.json is not
enough on its own: Yue2Panel.svelte is hand-written and renders a curated set
of parameters by name, with a bespoke block for ar_lora, so an entry in the
catalog that no block draws is invisible. Verified against a running server --
the options were in the bundled catalog and nothing rendered them.

- model_params.json gains the pair, with an info string for the trap in the
  spec description: nar_lora_scale scales the LoRA deltas only, and any full
  vae2llm/llm2vae projection replacement in the adapter stays at full strength.
  The AR entries gain the "reload the model" hint that main_gguf and vae_gguf
  already had and that all four LoRA options require.
- Yue2Panel.svelte gains the NAR block, mirroring AR: path, strength, a picker
  and its own error line. selectLora takes the branch instead of hardcoding
  ar_lora, so one uploader serves both and a failed NAR upload does not post an
  error under the AR field.
- +page.svelte seeds nar_lora/nar_lora_scale from the loaded model's session
  options next to the AR pair, for a server without UI management.

Also ungates the AR picker, which is a pre-existing defect rather than part of
the NAR work, but could not be left alone: gating it on server.ui_management
disabled AR LoRA upload in every default build, and a NAR picker copied from it
would have been equally dead. The endpoint it calls, /v1/ui/upload, requires
ui_enabled OR ui_management and is compiled outside the model-manager guard.
Confirmed against a --ui-only server with ui_management false: HTTP 200, file
written. The text and strength inputs are ungated for the same reason -- typing
a server-side path needs no management rights, and the main_gguf and vae_gguf
selectors beside them were never gated.

dist/index.html is a genuine rebuild, since a template change cannot be applied
to the committed bundle the way a config-only edit could.

Verified: npm run check passes (277 files, 0 errors), npm run build succeeds,
and a server built from this tree serves both pickers. Not verified end to end
-- loading YuE2 with an adapter and hearing the difference is still untested.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TMmMgd5xNGnjsQgybuUiK3
@christopherthompson81 christopherthompson81 changed the title webui: expose the YuE2 NAR LoRA adapter, not just the AR one webui: wire the YuE2 NAR LoRA adapter into the UI Sep 21, 2026
@christopherthompson81

Copy link
Copy Markdown
Contributor Author

Confirmed working. Exercised in the browser against audiocpp_server --ui on CUDA with a YuE2 3B package and both adapters present: the four controls render, the pickers work on a server without UI management, and the options reach the session. The "not verified end to end" caveat from the earlier revisions is discharged, and the PR body is updated to say so.

Worth noting how the config-only revision was caught: it was only running it that showed the two options sitting in the bundled catalog with nothing drawing them. Reviewing the diff would not have found it.

@0xShug0
0xShug0 merged commit 39342d1 into 0xShug0:main Sep 22, 2026
7 of 8 checks passed
@0xShug0

0xShug0 commented Sep 22, 2026

Copy link
Copy Markdown
Owner

@christopherthompson81 PR merged. I'm confused about the discussion of index.html. Does this mean that, in the normal PR workflow, rebuilding index.html and including it in the PR will still cause conflicts? Does that mean #546 will not work unless we manually edit index.html?

@christopherthompson81

Copy link
Copy Markdown
Contributor Author

#546 is working. My earlier explanation needs a correction, though. There can be desync because the build includes model specs and their doc paths. webui/native/src/lib/catalog.ts:29 inlines every spec at build time. Every update to model_specs will impact dist/index.html. So, a rebuilt index.html in a normal PR will frequently conflict.

I think the fix is two parts. 1. Stop inlining the specs. A prebuild step emits one small derived JSON (catalogue families × the four used fields) and catalog.ts imports that instead of globbing. The bundle stops depending on model_specs/ entirely, so adding a model no longer invalidates it unless the model is actually added to the UI catalogue. Keeps the build working without a JS toolchain. 2. Stop committing the bundle, building it on demand in CMake. That ends the conflict class rather than shrinking it. The catch is the fallback at CMakeLists.txt:2477: when the file is absent the build succeeds with a placeholder page, so this has to fail loudly when the UI is requested and node isn't available.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants