Skip to content

docs(tutor-quality): define Stage 2B semantic evaluation - #116

Merged
ttbombadil merged 3 commits into
mainfrom
docs/tutor-quality-stage-2b-ssot
Sep 28, 2026
Merged

ttbombadil merged 3 commits into
mainfrom
docs/tutor-quality-stage-2b-ssot

Conversation

@ttbombadil

Copy link
Copy Markdown
Collaborator

Summary

Adds the normative Stage 2B SSOT for semantic Tutor evaluation and controlled model, prompt, and strategy benchmarking. It defines the semantic rubric, critical-failure handling, versioned Gold Corpus and human references, Judge calibration, blinded absolute/pairwise evaluation, repeated sampling, comparability, reproducibility, and bounded secret-free artifacts.

This PR is documentation and architecture only. It does not implement Stage 2B, change Stage 1 or Stage 2A contracts, alter Tutor runtime behavior, or expand Stage 2C.

Scope and boundaries

  • Keeps technical execution, deterministic contract conformity, semantic Tutor quality, and actual learning effect as separate claims.
  • Requires Stage 2B to consume immutable Stage 2A transcripts and results rather than duplicating provider transport, Tutor invocation, credential handling, or deterministic checks.
  • Defines semantic status, per-Judge calibration status/identity, and per-comparison comparability as separate reporting axes.
  • Excludes LLM-as-Judge implementation, semantic scoring code, production model selection, prompt/strategy optimization, and any change to existing hard gates.

Verification

  • npm run check:docs — passed (45 Markdown files)
  • git diff --check — passed
  • Branch diff against main: one added SSOT file only; no implementation changes.

@ttbombadil
ttbombadil merged commit 68d2703 into main Sep 28, 2026
5 checks passed
@ttbombadil
ttbombadil deleted the docs/tutor-quality-stage-2b-ssot branch September 28, 2026 11:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant