webui: wire the YuE2 NAR LoRA adapter into the UI - #614
Conversation
|
@christopherthompson81 My main concern is that the shared server should store, list, and download adapters, not contain logic specific to a model’s LoRA layout. The "loadable" check is misleading. The PR actually marks a public Whisper LoRA as Also maybe just make the confinement claim more accurate: The new file endpoints can escape the models root through a symlink. The guard compares lexical paths, not where they resolve on disk. I created a symlinked model directory inside a temporary root, then called /v1/ui/loras/delete; it returned HTTP 200 and deleted my test file outside that root. The UI/server are intentionally minimal and don’t include a built-in security layer, so this scenario may simply be out of scope. |
Review feedback on 0xShug0#614, both points. The store no longer says whether an adapter suits a model. It said "loadable": true and "branch": "ar" for a public Whisper LoRA, which is what the review caught. That was not an edge case: lora_A/lora_B and self_attn/mlp are PEFT conventions, not YuE2's, so every PEFT adapter for any transformer passed. Reading the header of the file from the report confirms it -- 288 tensors, lora_A/lora_B throughout, names like base_model.model.model.decoder.layers.0.encoder_attn.k_proj.lora_A.weight, no metadata to contradict them. What it reports now is the file's shape, not its fit: "unfused_lora", true when a file has the lora_A/lora_B pairs every LoRA-consuming family here reads -- yue2 and vibevoice both do -- and false for a ComfyUI fused layout, which none of them can read. Whether an unfused adapter matches a particular model needs that model's tensor layout, so the loader answers it, which is also where a mismatch already surfaces. The branch sniffing and the yue2_lora_branch and lora_rank_separate metadata keys are gone; nothing model-specific is left in app/server. Losing the branch verdict would have left both pickers offering every adapter, so the slot is recorded instead of inferred: adapters are filed under <model>/loras/<group>/ with the group named by the caller. The panel passes "ar" or "nar" when it uploads or downloads, and offers each slot its own folder. The server treats the group as an opaque directory name -- it neither invents one nor reads meaning into it -- so a family with one adapter slot passes none and gets the flat layout, which is also what a file dropped in by hand gets, offered everywhere. Relative options already resolve as model_root / path (yue2/session.cpp), so "loras/ar/<file>" needs nothing from the loader. The confinement check now compares resolved paths. It compared lexical ones, so "<root>/link/loras" read as inside the root however link was defined, and the review's reproduction -- a symlinked model directory, then /v1/ui/loras/delete -- returned 200 and removed a file outside the root. weakly_canonical() follows the links and still tolerates a store that has not been created yet. Upload additionally checks the destination and its .partial with symlink_status() rather than exists(), since exists() follows a dangling symlink and the write lands on its target. Group names are validated as a single component. Verified against the built server with the model manager enabled -- note that the endpoints are behind AUDIOCPP_BUILD_NATIVE_MODEL_MANAGER, so a default build compiles none of this: upload group=ar -> 200 loras/ar/good.safetensors upload group=nar -> 200 loras/nar/good.safetensors ComfyUI fused file -> 400 rejected: this engine needs the unfused file delete via symlinked model dir -> 400, file outside the root survives upload via symlinked model dir -> 400, nothing planted group "../../../escape" -> 400 model "<root>/../outside" -> 400 and the Whisper adapter from the report now returns unfused_lora with no claim about which model or branch it belongs to. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TMmMgd5xNGnjsQgybuUiK3
ad36b8c to
f04f295
Compare
|
Both points were right, and both are fixed in The compatibility check is goneYou're right that it wasn't just wrong on that file — it was wrong by construction. The server now reports the file's shape, not its fit:
Whether an unfused adapter suits a given model needs that model's tensor layout, so the loader answers it — which is where a mismatch already surfaced anyway. Which dropdown, without a compatibility checkDropping the branch verdict would have left both pickers offering every adapter, so the slot is recorded when the file is stored rather than inferred afterwards. Adapters are filed under The server treats the group as an opaque directory name — it neither invents one nor reads meaning into it. A family with one adapter slot passes none and gets the flat layout, which is also what a file dropped in by hand gets, and those are offered in every slot. Relative options already resolve as That keeps the maintenance cost you were worried about at zero: a new model with LoRA support picks its own folder names and the server learns nothing about it. Path confinementFixed rather than scoped out — it was a few lines, which seemed better than arguing about the threat model. The check compared lexical paths, so Two things beyond your reproduction: upload checks the destination and its Your exact scenario, against the built server: And the Whisper adapter from your comment now returns One note for anyone verifying: these endpoints are behind |
|
@christopherthompson81 I think this PR should focus on the immediate task: adding the NAR LoRA controls to the WebUI. Persistent uploads are a separate usability feature, and server-side Hugging Face downloads are another substantial feature. Both need more review and validation. This is also a question of ownership and responsibility boundaries. Say if audio.cpp handles LoRA classification, it assumes responsibility for making that classification robust and covering the relevant edge cases. For example the current PR accepts a safetensors file containing only lora_A or only lora_B, even though a valid LoRA requires matching A/B pairs. The model loader already has the tensor layout and remains the actual authority on compatibility. Adding a weaker classifier to the shared server creates two definitions of validity that can disagree: the WebUI may offer and download an adapter as usable, only for the loader to reject it later. I suggest limiting this PR to exposing |
f04f295 to
7a39099
Compare
|
Narrowed to what you asked for: Gone: the persistent store, the HF download, the two On the classifier specifically — your A/B-pair point is right, and it is the third wrong answer that thing gave. First the branch verdict, then the Whisper adapter, now accepting a file with only one side of the pair. That is a pattern rather than three bugs, and "two definitions of validity that can disagree" is the accurate description of why. The loader has the tensor layout and is the only thing that can answer the question, so it should be the only thing that tries. I parked the removed work on One thing worth your attention in what remains: |
0xShug0#586 added NAR LoRA to the core and the maintainer asked on 0xShug0#499 for someone to wire it into the UI: "I added NAR LoRA support in the core, but haven't wired it into the UI yet." It missed 0.8.1, which shipped on 2026-09-17. yue2.nar_lora and yue2.nar_lora_scale have been reachable from the CLI and the server since 0xShug0#586, but not the WebUI. Adding them to model_params.json is not enough on its own: Yue2Panel.svelte is hand-written and renders a curated set of parameters by name, with a bespoke block for ar_lora, so an entry in the catalog that no block draws is invisible. Verified against a running server -- the options were in the bundled catalog and nothing rendered them. - model_params.json gains the pair, with an info string for the trap in the spec description: nar_lora_scale scales the LoRA deltas only, and any full vae2llm/llm2vae projection replacement in the adapter stays at full strength. The AR entries gain the "reload the model" hint that main_gguf and vae_gguf already had and that all four LoRA options require. - Yue2Panel.svelte gains the NAR block, mirroring AR: path, strength, a picker and its own error line. selectLora takes the branch instead of hardcoding ar_lora, so one uploader serves both and a failed NAR upload does not post an error under the AR field. - +page.svelte seeds nar_lora/nar_lora_scale from the loaded model's session options next to the AR pair, for a server without UI management. Also ungates the AR picker, which is a pre-existing defect rather than part of the NAR work, but could not be left alone: gating it on server.ui_management disabled AR LoRA upload in every default build, and a NAR picker copied from it would have been equally dead. The endpoint it calls, /v1/ui/upload, requires ui_enabled OR ui_management and is compiled outside the model-manager guard. Confirmed against a --ui-only server with ui_management false: HTTP 200, file written. The text and strength inputs are ungated for the same reason -- typing a server-side path needs no management rights, and the main_gguf and vae_gguf selectors beside them were never gated. dist/index.html is a genuine rebuild, since a template change cannot be applied to the committed bundle the way a config-only edit could. Verified: npm run check passes (277 files, 0 errors), npm run build succeeds, and a server built from this tree serves both pickers. Not verified end to end -- loading YuE2 with an adapter and hearing the difference is still untested. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TMmMgd5xNGnjsQgybuUiK3
7a39099 to
f7ee116
Compare
|
Confirmed working. Exercised in the browser against Worth noting how the config-only revision was caught: it was only running it that showed the two options sitting in the bundled catalog with nothing drawing them. Reviewing the diff would not have found it. |
|
@christopherthompson81 PR merged. I'm confused about the discussion of index.html. Does this mean that, in the normal PR workflow, rebuilding |
|
#546 is working. My earlier explanation needs a correction, though. There can be desync because the build includes model specs and their doc paths. webui/native/src/lib/catalog.ts:29 inlines every spec at build time. Every update to model_specs will impact dist/index.html. So, a rebuilt index.html in a normal PR will frequently conflict. I think the fix is two parts. 1. Stop inlining the specs. A prebuild step emits one small derived JSON (catalogue families × the four used fields) and catalog.ts imports that instead of globbing. The bundle stops depending on model_specs/ entirely, so adding a model no longer invalidates it unless the model is actually added to the UI catalogue. Keeps the build working without a JS toolchain. 2. Stop committing the bundle, building it on demand in CMake. That ends the conflict class rather than shrinking it. The catch is the fallback at CMakeLists.txt:2477: when the file is absent the build succeeds with a placeholder page, so this has to fail loudly when the UI is requested and node isn't available. |
Wires the YuE2 NAR LoRA adapter into the WebUI, as requested on #499: "I added NAR LoRA support in the core, but haven't wired it into the UI yet. Would appreciate it if anyone who's better at UI could pick this up." It missed 0.8.1, which shipped on 2026-09-17.
+105 / −56 across 4 files, WebUI only. The adapter store and server-side HF download from the earlier revision are gone, along with the LoRA classifier.
Why this is not just a config entry
My previous revision of this PR added
yue2.nar_loraandyue2.nar_lora_scaletomodel_params.jsonand nothing else. That does not work, and I only found out by running it.Yue2Panel.svelteis hand-written and renders a curated set of parameters by name —componentParamNames,coreParamNames,abcParamNames,semanticParamNames,plannerParamNames— plus a bespoke block forar_lora. It replaces the generic parameter renderer for this family, so a catalog entry that no block draws is invisible. Against a running server the options were present in the bundled catalog and nothing rendered them.So the change is:
model_params.jsongains the pair.nar_lora_scalecarries an info string for the trap in its spec description: it scales the LoRA deltas only, and any fullvae2llm/llm2vaeprojection replacement in the adapter stays at full strength. The two AR entries gain the "reload the model" hint thatmain_ggufandvae_ggufalready had and that all four LoRA options require.Yue2Panel.sveltegains the NAR block mirroring AR — path, strength, picker, its own error line.selectLoratakes the branch instead of hardcodingar_lora, so one uploader serves both and a failed NAR upload does not post an error under the AR field.+page.svelteseedsnar_lora/nar_lora_scalefrom the loaded model's session options beside the AR pair, for a server without UI management.One thing beyond the NAR work, flagged rather than buried
It also ungates the AR picker. That is a pre-existing defect, not part of wiring NAR, but I could not leave it:
disabled={!server?.ui_management || ...}disables AR LoRA upload in every default build, and a NAR picker copied from it would have been equally dead.The endpoint it calls,
/v1/ui/upload, requiresui_enabledorui_managementand is compiled outside theAUDIOCPP_HAS_NATIVE_MODEL_MANAGERguard. Confirmed against a--ui-only server reportingui_management: false:The text and strength inputs are ungated for the same reason: typing a server-side path needs no management rights, and the
main_gguf/vae_ggufselectors beside them were never gated. Happy to split this into its own PR if you would rather keep this one strictly NAR — it is a three-line change either way.The generated bundle
dist/index.htmlis a genuine rebuild this time. My previous revision patched it surgically to avoid churn, which a config-only change allows; a template change does not. The line diff is 84 lines, but it is a whole-bundle replacement — my toolchain's minifier output differs throughout from whatever produced the committed one. Tell me if you would rather regenerate it yourself.Verification
npm run checkpasses (277 files, 0 errors),npm run buildsucceeds. Rebased onto currentmain.Exercised in the browser against
audiocpp_server --uion CUDA, serving a YuE2 3B package with both adapters present, and confirmed working. The four controls render, the pickers are usable on a server without UI management, and the options reach the session.Earlier revisions of this PR carried a "not verified end to end" caveat. That is discharged — this one was driven through the actual UI before being pushed, which is also how the config-only revision was caught doing nothing.
🤖 Generated with Claude Code
https://claude.ai/code/session_01TMmMgd5xNGnjsQgybuUiK3