feat: SCRIBERR_ENABLED_MODELS to skip initializing unused models - #474
Open
Swahjak wants to merge 1 commit into
Open
feat: SCRIBERR_ENABLED_MODELS to skip initializing unused models#474Swahjak wants to merge 1 commit into
Swahjak wants to merge 1 commit into
Conversation
InitializeModels prepared every registered adapter on startup, so a cloud-only setup still installed the local Python environments and downloaded weights for WhisperX, Parakeet, Canary, Voxtral, PyAnnote and Sortformer. SCRIBERR_ENABLED_MODELS takes a comma separated list of model IDs and limits startup preparation to those models. Unset or empty keeps the current behaviour of preparing everything. Skipped models stay registered and are prepared on demand the first time a job uses them.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #473
Motivation
On startup
ModelRegistry.InitializeModelscallsPrepareEnvironmenton every registered adapter, regardless of which model the user actually runs. For a setup that only transcribes through an OpenAI-compatible endpoint, first boot still installs the Python environments (uv sync, torch/CUDA) and downloads the weights for WhisperX, Parakeet, Canary, Voxtral, PyAnnote and Sortformer — several GB of disk and a long wait for models that will never be used.What this adds
A new environment variable,
SCRIBERR_ENABLED_MODELS, holding a comma separated list of model IDs:Valid IDs are the registered adapter IDs:
whisperx,parakeet,canary,voxtral,openai_whisper,pyannote,sortformer.Behaviour
InitializeModelsonly callsPrepareEnvironmenton adapters whose ID is in the list. Skipped adapters get nouv sync, no weight download, noPrepareEnvironmentcall at all — just a log line saying they were skipped.GetTranscriptionAdapter/GetDiarizationAdapterand stay listed in the UI and API, so nothing breaks if a model is looked up by capability.EnsureModelReady(ctx, modelID)prepares such a model on demand right before the job runs it (once, guarded by a mutex so concurrent jobs cannot install the same environment twice). For models that were initialized at startup, and whenever the variable is unset, this is a no-op.Implementation
internal/config: parsesSCRIBERR_ENABLED_MODELSintoConfig.EnabledModelsalongside the other env vars.cmd/server/main.go: passes it to the registry inregisterAdapters.internal/transcription/registry:SetEnabledModels, the init-time filter, andEnsureModelReadyfor on-demand preparation.internal/transcription/unified_service.go: callsEnsureModelReadybefore transcription and before separate diarization.Testing
go build ./...andgo vet ./...clean.internal/transcription/registry/registry_test.gocovers the default (all models prepared), the filtered case (disabled model never prepared but still registered), and on-demand preparation happening exactly once.go test ./internal/... ./tests/...passes apart fromTestListTranscriptionJobsDeltaSync, which fails on unmodifiedmainas well.