Repository navigation
[FIX] Assign local YAML filename fallback to seeds so named queries find them - #3772
Open
Chen Yufeiyang (feiiiiii5) wants to merge 1 commit into
Open
Chen Yufeiyang (feiiiiii5) wants to merge 1 commit into
Chen Yufeiyang (feiiiiii5) wants to merge 1 commit into
Conversation
…ind them A local YAML without dataset_name got a filename fallback on the provider/SeedDataset only; seed.dataset_name stayed None, so the memory insert persisted NULL and named queries/listings could not find the dataset. Assign the resolved name to unnamed seeds before returning, matching the per-seed assignment used by the generated and jailbreak loaders. Fixes microsoft#3740
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #3740.
Description
A local YAML dataset that omits
dataset_namegets a filename fallback at the provider/SeedDataset layer, but its seeds keepdataset_name=None. After the dataset is written to memory, named queries cannot find it:get_seeds_async(dataset_name="<stem>")returns 0 rows andget_seed_dataset_names_async()does not list the name, while the dataset itself appears in the provider list. An explicitly named YAML works.Root cause: the fallback name is decided in
LocalDatasetLoader.fetch_dataset_asyncafterSeedDataset._build_seedshas already run, so it never reachesseed.dataset_name, and the memory insert persists the NULL.The loader now assigns the resolved dataset name to seeds that do not have their own name before returning, matching the per-seed assignment already done by
assert_provider.pyandlocal/jailbreak_dataset.py. Explicit dataset-level and per-seed names are unchanged.Tests and Documentation
New regression cases in
tests/unit/datasets/test_local_dataset_loader.py:get_seed_dataset_names_async(); the explicit per-seed name is not overwritten;Evidence:
Also ran 289 related unit tests (seed datasets, local loader, dataset e2e contract, seed dataset provider) and 6 local integration tests;
ruff check,ruff format --check,ty check, andbuild_scripts/check_async_suffix.pypass on the changed files.Not run: the full unit suite, Windows/Python 3.11, and the full pre-commit set. No documentation change: this fixes an internal naming inconsistency, and the documented load/named-query contract is already what the fix makes true.