research(mir): preregister structure feature noninferiority evidence

[seonghobae]

Problem

#1223 withdrew the chroma_cqt → chroma_stft production switch because synthetic/mock timing did not establish structure accuracy on rights-cleared decoded audio. #1225 therefore requires preregistered real-audio noninferiority evidence before STFT can become the production default.

This Draft owns the Signal-MIR scientific evidence/admission boundary for that decision. Synthetic fixtures are regression evidence only. Production acceptance still requires a reviewed rights-cleared corpus, fixed product/scientific thresholds and claim boundary, exact runtime/host identity, complete-track execution, current-head checks/review, and a separate production-switch change.

Exact current identity

• Protected base: develop@314ddeae7b775a4957594b599358c8255617eb2e.
• Branch: research/structure-noninferiority-1225.
• Exact head: e99ccecc20a72d9531d77be5b854ae6090500aff.
• Protected-base ancestry: ahead 200 / behind 0.
• Production structure extraction still defaults to chroma_feature="cqt"; this PR does not switch production to STFT.

Every source movement invalidates predecessor checks/reviews. Only this exact head may satisfy the verification gate.

Frozen scientific semantics

Functional-label ACC is pinned to ismir-mirex/mirex-evaluation@b9fa0b0b32e2145af31f35830f78fc9d09a4301b:music_structure_analysis.eval_script.calculate_accuracy with the 200 ms frame-grid contract. The segmentation adapter uses the content-addressed mir_eval==0.8.2 research overlay and binds detection at 0.5/3.0 s with beta=1.0, trim=True, deviation with trim=True, and pairwise grouping with frame_size=0.1, beta=1.0.

aggregate_structure_noninferiority.py owns macro-track-v1 / paired-track-bootstrap-v1: registered track-pair sampling, equal-track weighting, macro P/R→harmonic F, macro deviation/ACC/latency means, maximum peak RSS, paired resampling with replacement, numpy.random.Generator(PCG64), and 0.025/0.975 percentile bounds using NumPy method="linear". Stored aggregate/CI summaries are admitted only when they exactly match canonical recomputation from the complete stored track receipts and preregistered uncertainty plan.

isolated-single-shot-v1 owns per-track performance evidence: macOS/Windows only, zero same-process warm-ups, 20 fresh-process observations per lane/track, alternating CQT→STFT / STFT→CQT order, time.perf_counter_ns() around the repository segmenter, NumPy linear p50/p95, and process-lifetime peak RSS (ru_maxrss on macOS; PeakWorkingSetSize on Windows).

The canonical admitted-track consumer gives CQT and STFT the same immutable admitted PCM/reference segmentation and emits schema-v1 track receipts only from complete functional-ACC, recognized segmentation, and registered performance evidence. Public result admission recomputes aggregate and paired CIs from those stored track receipts.

Current RED → GREEN repair: exact execution host and one canonical run path

Fresh review found that registration.runtime.host_profile was only validated as non-empty text. Corpus runtime admission verified source commit, lock, Python, librosa, and NumPy, but never proved that the machine executing latency/RSS measurement matched the host described by preregistration. Since p95 latency ratio is a decision criterion, a free-form host label is not sufficient scientific identity.

This head adds a purpose-bound execution contract:

• RED 7a90b922c0f64090f1ac2717e9afc0369503bd58: require preregistered/observed host mismatch to fail before corpus admission and require host identity to exclude hostname/account identifiers.
• GREEN 70b86022323728c6b3f85d5f14468423ebf15478: add run_structure_noninferiority_experiment.py, structure-performance-host-v1, macOS/Windows host capture, admission→track-evidence binding, canonical result construction, validation, and atomic path-free execution envelope.
• RED f1a6971ff7f60af993c05455e40ccc6d57ab593e: make free-form host labels explicitly invalid for canonical scientific execution.
• GREEN a31bebd79d88b940c9f96517d02f7c0364fac485 / fixture alignment 7b07c00d9ee8c7ddbb24f3a65bd62aa1e22c05c4: add validate_execution_registration() and require structure-performance-host-v1:<sha256> before corpus access.
• 7d3c28d55af1b1ba08afc9ed2a6df85274a7d6f7: add TRACEABILITY for host fields, rejected identifiers/alternatives, owner boundaries, claim limitations, and authoritative Apple/Microsoft system-identification sources.
• e99ccecc20a72d9531d77be5b854ae6090500aff: format the new runner to the repository's 100-column Python lint boundary before treating hosted lint as evidence.

The content-addressed host payload contains platform, OS release/version, architecture, hardware model, CPU model, and logical CPU count. It deliberately excludes hostname/computer name, user/account identity, serial number, MAC/IP, and paths. macOS values are dynamically read with fixed sysctl selectors; Windows values come from fixed machine-level registry locations. Unsupported OSes fail closed. This profile does not claim to freeze thermal state, CPU frequency, power policy, scheduler contention, or page-cache state; those limitations remain in the scientific claim boundary.

The new runner composes existing owners rather than reimplementing them: registration/result validator → host gate → Resource Admission → canonical CQT/STFT consumer → exact admission/evidence identity checks → track receipts → canonical aggregation/bootstrap → result admission → atomic execution envelope. Running lower-level corpus admission alone does not authorize latency/RSS evidence.

Earlier integrity repairs retained on this branch

• canonical mir_eval==0.8.2 artifact/runtime identity and recognized segmentation adapters;
• executable 200 ms functional-label ACC contract pinned to MIREX evaluator semantics;
• complete admitted-track CQT/STFT quality + performance evidence;
• isolated-single-shot-v1 latency/RSS measurement contract;
• caller-free schema-v1 track receipt construction;
• deterministic macro-track-v1 aggregate and paired-track-bootstrap-v1 CI recomputation;
• percentile-bootstrap admission without the obsolete requirement that every CI contain the observed point estimate.

Claim boundary and remaining work

This branch does not establish approved numeric production margins/latency threshold, adequacy or representativeness of a concrete rights-cleared corpus, adequacy of a chosen bootstrap count/seed, a selected and reviewed concrete host-profile payload, scientific noninferiority or latency superiority on real audio, MIREX leaderboard equivalence, or a production feature switch.

Before candidate results are inspected, reviewers still must freeze concrete corpus membership/representativeness, quality noninferiority margins, maximum candidate latency ratio, exact bootstrap count/seed, the actual macOS/Windows structure-performance-host-v1 payload/content address, and the population/runtime claim. Corpus size and inference-plan adequacy remain scientific review decisions. Only after those choices are frozen may the complete rights-cleared corpus be executed through the canonical runner.

A future need for independently verifiable producer attestation or crash-resumable partial scientific execution requires a separately versioned evidence-envelope/checkpoint contract; this PR does not claim that numeric JSON alone proves historical execution provenance.

Verification gate

Fresh workflows for exact e99ccecc20a72d9531d77be5b854ae6090500aff supersede all predecessor evidence. The PR remains Draft until this exact head has terminal protected CI/security/SBOM/SAST/native-build evidence and a qualifying independent non-author current-head approval.

Do not self-approve, force-push, destructively rebase, weaken gates, manufacture statuses, blind-rerun non-terminal jobs, or use no-op commits to obtain fresh evidence.

[coderabbitai]

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

• Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[seonghobae]

@opencode-agent review exact head 47d09320957097b5347f81396a57e422dad5813a only. Review the Signal-MIR scientific evidence-admission boundary, especially preregistration/result identity, failed-track fail-closed behavior, aggregate/per-track consistency assumptions, uncertainty claim boundaries, and evidence parsing/security. Review only; do not mutate source.

[opencode-agent]

Queued @opencode-agent for PR #1228 at head 47d09320957097b5347f81396a57e422dad5813a. Central exact-name Actions artifacts are the durable dispatch ledger; existing review workflows remain authoritative for the final verdict and failure evidence.

[seonghobae]

@opencode-agent review exact head d7c65fdfbc40a68b066eb092e76fe5e87a5ad0cd only. Review the Signal-MIR scientific evidence-admission boundary after the closed-world measurement-receipt repair, especially post-hoc field admission, aggregate/per-track consistency assumptions, preregistered aggregation/uncertainty claim boundaries, failed-track fail-closed behavior, and evidence parsing/security. Review only; do not mutate source.

[opencode-agent]

Queued @opencode-agent for PR #1228 at head e99ccecc20a72d9531d77be5b854ae6090500aff. Central exact-name Actions artifacts are the durable dispatch ledger; existing review workflows remain authoritative for the final verdict and failure evidence.

[opencode-agent]

Queued @opencode-agent for PR #1228 at head e99ccecc20a72d9531d77be5b854ae6090500aff. Central exact-name Actions artifacts are the durable dispatch ledger; existing review workflows remain authoritative for the final verdict and failure evidence.

[opencode-agent]

Queued @opencode-agent for PR #1228 at head e99ccecc20a72d9531d77be5b854ae6090500aff. Central exact-name Actions artifacts are the durable dispatch ledger; existing review workflows remain authoritative for the final verdict and failure evidence.