Add a Vally-based evaluation harness for repository-owned Copilot customizations.

.agents/skills/change-tracking/SKILL.md

@@ -10,12 +10,19 @@ Manages entity states and detects changes for `SaveChanges()`.
 
 ## Core Components
 
-- `StateManager` — central engine, identity maps, tracks all entities
-- `InternalEntityEntry` — per-entity state, property flags, snapshots
-- `SnapshotFactoryFactory` subclasses build snapshot factories for change detection
+- `StateManager` — owns tracked entries, identity/reference maps, fixup, cascades, notifications, and changed counts
+- `InternalEntryBase` and derived classes — own per-entry state, flags, values, snapshots, and ordered state transitions
+- `ChangeDetector` — compares current values with baselines and reports changes through entry mutation APIs
+- `SnapshotFactoryFactory` subclasses — create passive original- and relationship-value baselines
+- `IdentityMap` — permits one active entry per key; shared identity pairs a replacement with the prior `Deleted` entry
 - `PropertyAccessorsFactory`, `ClrPropertyGetterFactory` and `ClrPropertySetterFactory` compile property accessors for efficient snapshotting and change detection
   - Ordinals in `indices` parameter specify element at each complex collection depth
 
+## Change Detection
+
+- Snapshot and notification strategies both call `SetPropertyModified()` to keep property flags and entity state consistent.
+- `SetEntityState()` validates values, updates flags and complex entries, changes state, then runs manager bookkeeping hooks.
+
 ## Testing
 
 Unit tests: `test/EFCore.Tests/ChangeTracking/`. Functional tests: `test/EFCore.Specification.Tests/GraphUpdates/`.

.agents/skills/make-custom-agent/SKILL.md

@@ -242,5 +242,4 @@ After creating or modifying an agent, verify:
 - [GitHub Copilot Extensions documentation](https://docs.github.com/en/copilot/building-copilot-extensions/about-building-copilot-extensions)
 - [GitHub Copilot Custom agents configuration](https://docs.github.com/en/copilot/reference/custom-agents-configuration)
 - [Agent Skills Specification](https://agentskills.io/specification)
-- [make-skill](../make-skill/SKILL.md)
-- [make-instructions](../make-instructions/SKILL.md)
+- Related repository skills: `make-skill` and `make-instructions`

.agents/skills/make-instructions/SKILL.md

@@ -1,6 +1,6 @@
 ---
 name: make-instructions
-description: 'Create VS Code file-based instructions (.instructions.md files). Use when asked to create, scaffold, or add file-based instructions for Copilot. Generates .instructions.md with YAML frontmatter and background knowledge content.'
+description: 'Create and evaluate VS Code file-based instructions (.instructions.md files). Use when asked to create, scaffold, or add file-based instructions for Copilot. Generates scoped instructions and a paired Vally harness eval.'
 ---
 
 # Create File-Based Instructions
@@ -24,6 +24,8 @@ Build understanding of the area the instructions should cover. Identify:
 - [ ] Common pitfalls that Copilot should avoid
 - [ ] Non-obvious domain knowledge that isn't discoverable from code alone
 
+Read the repository-wide instruction files that apply to the same paths and make an explicit exclusion list. Do not repeat those rules in the new file, even when they are relevant examples for the scoped area; include only guidance that becomes more specific or materially different at the narrower scope.
+
 If the scope is unclear or overlaps with existing instructions, ask the user for clarification.
 
 ### Step 2: Choose the file location
@@ -73,7 +75,11 @@ Recommended sections (adapt as needed):
 5. **Key Files** — table of important files for orientation (optional)
 6. **Common Pitfalls** — traps to avoid (optional)
 
-### Step 5: Validate
+### Step 5: Author and validate the harness evaluation
+
+Create `eng/harness-evaluation/instructions/<id>/eval.yaml`, where `<id>` is the instruction path relative to `.github/instructions/` with its suffix removed and nested path separators replaced by `--`.
+
+Follow the authoring and validation rules in `eng/harness-evaluation/README.md`. Exercise guidance distinctive to the instruction and ensure the treatment meaningfully outperforms the omitted-instruction control.
 
 After creating the file, verify:
 
@@ -84,6 +90,7 @@ After creating the file, verify:
 - [ ] Content is concise (aim for under 500 lines or 5000 tokens) — long instructions dilute effectiveness
 - [ ] No secrets, tokens, or internal URLs included
 - [ ] Instructions don't duplicate what's already in `.github/copilot-instructions.md` or under `.agents/skills/`
+- [ ] The paired eval demonstrates behavior that the omitted-instruction control does not provide reliably
 
 ## Common Pitfalls
 

.agents/skills/make-skill/SKILL.md

@@ -1,6 +1,6 @@
 ---
 name: make-skill
-description: 'Create new Agent Skills for GitHub Copilot. Use when asked to create, scaffold, or add a skill. Generates SKILL.md with frontmatter, directory structure, and optional resources.'
+description: 'Create and evaluate new Agent Skills for GitHub Copilot. Use when asked to create, scaffold, or add a skill. Generates SKILL.md, optional resources, and a paired Vally harness eval.'
 ---
 
 # Create Skill
@@ -33,8 +33,7 @@ After investigating, verify:
 - [ ] Have search queries for deeper topics
 - [ ] Can determine if the skill should be user-invocable or background knowledge only
 
-If there are any ambiguities, gaps in understanding, or multiple valid approaches, ask the user for clarification before proceeding to skill creation.
-Also, evaluate whether the task might be better handled by a custom agent, agentic workflow, an existing skill or multiple narrower skills, and discuss this with the user if relevant.
+If there are any ambiguities, gaps in understanding, or multiple valid approaches, ask the user for clarification before proceeding to skill creation. Also, evaluate whether the task might be better handled by a custom agent, agentic workflow, an existing skill or multiple narrower skills, and discuss this with the user if relevant.
 
 ### Step 2: Create the skill directory
 
@@ -92,26 +91,22 @@ Include these recommended sections, following this file's structure:
 
 > ❌ **NEVER** count API failures as success. Return "Unknown" and exclude from positive counts.
 
-### Step 7: Validate the skill
-
-Ensure the name:
-- Does not start or end with a hyphen
-- Does not contain consecutive hyphens
-- Is between 1-64 characters
-- YAML frontmatter name matches directory name exactly
-
-After creating a skill, verify:
-- [ ] frontmatter fields are valid
-- [ ] SKILL.md is under 500 lines and 5000 tokens, split into references if needed
-- [ ] File references use relative paths
-- [ ] Instructions are actionable and specific
-- [ ] Instructions don't duplicate what's already in `.github/copilot-instructions.md` or under `.github/instructions/`
-- [ ] Workflow has numbered steps with clear checkpoints
-- [ ] Validation section exists with observable success criteria
-- [ ] No secrets, tokens, or internal URLs included
-- [ ] Common pitfalls are relevant and have solutions
+### Step 7: Author and validate the harness evaluation
+
+Create `eng/harness-evaluation/skills/<skill-name>/eval.yaml` and follow the authoring and validation rules in `eng/harness-evaluation/README.md`. The eval must require exact invocation of `<skill-name>` and meaningfully distinguish the skilled treatment from the unskilled control.
+
+Also verify:
+
+- [ ] The skill name does not start or end with a hyphen, contain consecutive hyphens, or exceed 64 characters
+- [ ] YAML frontmatter name matches the directory name exactly and all frontmatter fields are valid
+- [ ] SKILL.md is under 500 lines and 5000 tokens, splitting stable detail into references when needed
+- [ ] File references are relative and instructions are actionable and specific
+- [ ] Instructions do not duplicate `.github/copilot-instructions.md` or `.github/instructions/`
+- [ ] The workflow has numbered steps and observable success criteria
+- [ ] No secrets, tokens, or internal URLs are included
 - [ ] Optional directories are used appropriately
-- [ ] Scripts handle edge cases gracefully and return structured outputs and helpful error messages when applicable
+- [ ] Scripts handle edge cases, fail closed, and return structured, helpful errors
+- [ ] The paired Vally comparison demonstrates distinctive value over the unskilled control
 
 ### Step 8: Test with Multi-Model Subagents
 
@@ -141,5 +136,4 @@ Follow [references/testing-patterns.md](references/testing-patterns.md):
 ## References
 
 - [Agent Skills Specification](https://agentskills.io/specification)
-- [Copilot Instructions](../../../.github/copilot-instructions.md)
-- [Contributing Guidelines](../../../.github/CONTRIBUTING.md)
+- Repository guidance: `.github/copilot-instructions.md` and `.github/CONTRIBUTING.md`

.agents/skills/make-skill/references/testing-patterns.md

@@ -184,16 +184,16 @@ task agent_type="general-purpose" model="{different-model}" prompt="Review the s
 
 The two approaches complement each other: writer-critic for creation/iteration, multi-model for validation.
 
-## Waza Eval Testing
+## Vally Evaluation
 
-For repeatable, quantitative skill testing, use the **waza-eval** skill. It provides:
+For repeatable, quantitative skill testing, author the repository's paired Vally eval. It provides:
 
-- **Structured eval suites** — define tasks with prompts, expected outputs, and graders
-- **Progression testing** — compare tool efficiency across skill versions from git history
-- **Session capture** — commit result transcripts as golden sessions for regression detection
-- **CI integration** — gate PRs on eval pass rates
+- **Structured eval suites** — define repository-grounded stimuli, expected outputs, and graders
+- **Unskilled controls** — compare identical tasks with and without the target skill
+- **Quality and efficiency comparison** — report judge preference and token, turn, tool-call, time, and error deltas
+- **CI integration** — require the treatment threshold and fail statistically significant regressions
 
-Use waza evals when you need to *measure* whether a skill change improved behavior. Use multi-model review (above) when you need *qualitative* structural feedback.
+Use `node eng/harness-evaluation/src/cli.mjs eval <skill-name> --workers 1 --require-pass` to measure whether a skill improves behavior with its configured trial count. Use multi-model review (above) for qualitative structural feedback.
 
 ### Regression Heuristics
 
@@ -220,6 +220,7 @@ Evals should include trigger tests (does the skill activate correctly?):
 Before shipping a skill change:
 
 - [ ] Description matches trigger tests (USE FOR phrases appear in should-trigger prompts)
+- [ ] `eng/harness-evaluation/skills/<skill-name>/eval.yaml` passes treatment/control comparison
 - [ ] Stop signals are explicit with numeric bounds
 - [ ] Domain examples present (not just tool schemas)
 - [ ] Token budget met (SKILL.md under 4K orchestrating / 15K knowledge)

.agents/skills/migrations/SKILL.md

@@ -16,7 +16,10 @@ user-invocable: false
 
 - Model snapshots use `typeof(Dictionary<string, object>)` (property bag format), not the actual CLR type. When examining the `ClrType` in a snapshot, don't assume it matches the real entity type.
 - `SnapshotModelProcessor.Process()` is used at design-time to fixup older model snapshots for backward compatibility.
+- `MigrationsModelDiffer` uses provider-agnostic structural comparison between relational models to determine what migration operations are necessary.
 
 ## Testing
 
-Migration operation tests: `test/EFCore.Relational.Tests/Migrations/`. Functional tests: `test/EFCore.{Provider}.FunctionalTests/Migrations/`. Model differ tests: `test/EFCore.Relational.Tests/Migrations/Internal/MigrationsModelDifferTest*.cs`.
+Migration operation tests: `test/EFCore.Relational.Tests/Migrations/`. Functional tests: `test/EFCore.{Provider}.FunctionalTests/Migrations/`. Model differ tests: `test/EFCore.Relational.Tests/Migrations/Internal/MigrationsModelDifferTest*.cs`.
+
+To simulate a snapshot model use `ModelBuilder` calls without conventions.

.agents/skills/run-apichief/SKILL.md

@@ -6,7 +6,7 @@ user-invocable: false
 
 # Run ApiChief
 
-Use the [ApiChief tool](../../../eng/Tools/ApiChief/README.md) to inspect or refresh EF Core public API baselines for projects under `src/`.
+Use the ApiChief tool documented at `eng/Tools/ApiChief/README.md` to inspect or refresh EF Core public API baselines for projects under `src/`.
 
 ApiChief can run against either a compiled assembly or a previously emitted baseline JSON file. Prefer the repo-local `.dotnet` SDK and the checked-in build scripts in this repo.
 

.agents/skills/servicing-pr/SKILL.md

@@ -44,7 +44,7 @@ Brief risk assessment ranked from "extremely low" to "high". Note amount of code
 
 ## Quirk (AppContext Switch)
 
-A quirk lets users opt out of the fix at runtime, reducing patch risk. Add for all cases where it makes sense. Skip when the fix is 100% obvious/risk-free, or when the quirk couldn't be used, like in tools or analyzers.
+A quirk lets users opt out of the fix at runtime, reducing patch risk. Skip when the fix is 100% obvious/risk-free, or when the quirk couldn't be used, like in tools or analyzers.
 
 ### Adding a Quirk
 
@@ -58,6 +58,7 @@ private static readonly bool UseOldBehavior37585 =
 - Change `37585` to the relevant issue number
 - Wrap changes with a condition on `!UseOldBehavior37585` so activating the switch bypasses the fix, prefer to minimize the number of times the switch is checked
 - If the PR closes multiple issues, pick the most appropriate one for the switch name
+- Distinguish a recommendation from the staged implementation. Do not say "Quirk added" unless the source change actually includes the switch.
 
 ## Validation
 

.agents/skills/sqlite-adonet/SKILL.md

@@ -11,4 +11,4 @@ Standalone ADO.NET provider in `src/Microsoft.Data.Sqlite.Core/`, independent of
 ## Notable Implementation Details
 
 - Static constructor calls `SQLitePCL.Batteries_V2.Init()` reflectively
-- `CreateFunction()`/`CreateAggregate()` overloads generated from T4 templates (`.tt` files)
+- `CreateFunction()`/`CreateAggregate()` overloads generated from T4 templates (`.tt` files)

.agents/skills/tooling/SKILL.md

@@ -28,4 +28,4 @@ NuGet package `Microsoft.EntityFrameworkCore.Tasks` provides build/publish-time
 ## Validation
 
 - For tool changes, create a test project and manually run affected commands to verify behavior
-- `dotnet ef migrations script` output matches expected DDL
+- `dotnet ef migrations script` output matches expected DDL

.agents/skills/update-pipeline/SKILL.md

@@ -10,16 +10,12 @@ Converts tracked entity changes into database INSERT/UPDATE/DELETE commands duri
 
 ## Flow
 
-`SaveChanges()` → `DetectChanges()` → `IDatabase.SaveChanges()`
-  → `UpdateAdapter` creates `IUpdateEntry` list
-  → `CommandBatchPreparer.BatchCommands()`
+`SaveChanges()` → `DetectChanges()` → `IDatabase.SaveChanges()` → `UpdateAdapter` creates `IUpdateEntry` list → `CommandBatchPreparer.BatchCommands()`
     → `ModificationCommand` per table row, composed of `ColumnModification` per column
-      → `SharedTableEntryMap` is used to track entries mapped to the same row
+    → `SharedTableEntryMap` is used to track entries mapped to the same row
     → Topological sort via Multigraph (FK dependency ordering)
     → Groups into `ModificationCommandBatch` (respects max batch size)
-  → `UpdateSqlGenerator` generates SQL per batch
-  → `BatchExecutor` executes all batches in a transaction
-  → `StateManager.AcceptAllChanges()`
+    → `UpdateSqlGenerator` generates SQL per batch → `BatchExecutor` executes all batches in a transaction → `StateManager.AcceptAllChanges()`
 
 ## Concurrency