You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Standalone fix, no dependency on the other PRs in this stack.
MLLMCalibrator unconditionally routes to get_mllm_dataloader whenever the model is multimodal, which KeyErrors on any dataset not in its MLLM_DATASET registry (its own os.path.isfile(dataset) or dataset in MLLM_DATASET guard still indexes MLLM_DATASET with the raw file path). But a local text dataset (file/dir/HF text set) is perfectly valid calibration for a VLM whose non-text modules are not being quantized (quant_nontext_module=False): the full-model forward runs text-only, and the loop's generic-dict branch already just feeds model(**batch) regardless of dataset shape.
Change
When quant_nontext_module is False and the dataset isn't a known MLLM_DATASET entry, use the standard text get_dataloader() instead of falling through to get_mllm_dataloader.
Heads-up on the Performance-Test-AutoRound (Qwen3_FP8_STATIC) check on this PR — it's failed twice on retrigger, but on opposite sides each time:
Run 1: Current = 22.83GB, Baseline = 20.51GB (+11.31%) → FAIL
Run 2: Current = 20.55GB, Baseline = 23.58GB (-12.85%) → FAIL
The baseline number itself swings ~15% between runs of the identical job, and the sign of the "regression" flips. This PR only touches calibration/mllm.py (routing text-only datasets for non-vision calibration), which has no code path anywhere near FP8_STATIC quantization memory usage, so I don't think this reflects a real change from this diff — looks like VRAM measurement noise on the runner (possibly contention with other concurrent jobs) combined with a tolerance that's too tight to absorb it. Flagging in case it's useful for tightening up the perf-check baseline/tolerance.
Rebased onto current main. This PR only touches calibration/mllm.py and is independent of the disk-streaming/resumability stack, so the rebase was clean with no conflicts. Force-pushed to this branch.
Azure Pipelines successfully started running 1 pipeline(s).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Standalone fix, no dependency on the other PRs in this stack.
MLLMCalibratorunconditionally routes toget_mllm_dataloaderwhenever the model is multimodal, whichKeyErrors on any dataset not in itsMLLM_DATASETregistry (its ownos.path.isfile(dataset) or dataset in MLLM_DATASETguard still indexesMLLM_DATASETwith the raw file path). But a local text dataset (file/dir/HF text set) is perfectly valid calibration for a VLM whose non-text modules are not being quantized (quant_nontext_module=False): the full-model forward runs text-only, and the loop's generic-dict branch already just feedsmodel(**batch)regardless of dataset shape.Change
When
quant_nontext_moduleisFalseand the dataset isn't a knownMLLM_DATASETentry, use the standard textget_dataloader()instead of falling through toget_mllm_dataloader.