Skip to content

encoder: fix holdout splitting and compatibility wrapper - #16

Merged
huangyz0918 merged 1 commit into
mainfrom
fix/encoder-holdout-splits
Sep 16, 2026
Merged

huangyz0918 merged 1 commit into
mainfrom
fix/encoder-holdout-splits

Conversation

@huangyz0918

@huangyz0918 huangyz0918 commented Sep 15, 2026

Copy link
Copy Markdown
Member

Summary

  • Preserve callers' benchmark dictionaries when making a new_model holdout split. Previously, removing a target model also removed it from the original input, corrupting later splits made from the same data.
  • Repair the backward-compatible prepare_holdout_split() wrapper, which passed an unsupported keyword and raised TypeError. It now uses the equivalent new_pair setting.

These preserve the new-model/new-pair separation described in the paper's experimental setup: https://arxiv.org/html/2604.23099v1#S3.SS1

Verification

  • Unchanged repository suite: 123 passed, including PyTorch paths.
  • Local checks cover name/index compatibility calls and repeated new-model splits without input mutation.
  • Only the production bug fixes are included. Additional validation code stays local; no credentials or live API calls are involved.

@huangyz0918
huangyz0918 merged commit 529cdfa into main Sep 16, 2026
7 checks passed
@huangyz0918
huangyz0918 deleted the fix/encoder-holdout-splits branch September 16, 2026 00:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant