Add initial deterministic trace grading framework - #54678

Merged

pelikhan merged 29 commits into

mainfrom

copilot/implement-deterministic-trace-graders

Aug 22, 2026

Commits on Aug 22, 2026

Plan deterministic trace graders

Copilot
and
pelikhan
authored
ce1a3ae
Add initial deterministic trace graders

Copilot
and
pelikhan
authored
de6658f
docs(adr): add draft ADR-54678 for deterministic trace grading framework

github-actions[bot]
and
claude
authored
4b4de8a
Merge branch 'main' into copilot/implement-deterministic-trace-graders
github-actions[bot]
authored
9eb6028
Harden trace grader runtime isolation and manifest transport

Copilot
and
gh-aw-bot
authored
0d6a86e
Merge branch 'main' into copilot/implement-deterministic-trace-graders
github-actions[bot]
authored
53938a6
Merge remote-tracking branch 'origin/main' into copilot/implement-deterministic-trace-graders
Copilot
authored
1a1b44d
Refresh branch and fix trace graders JS type casts

Copilot
and
gh-aw-bot
authored
ed01ef4
Merge remote-tracking branch 'origin/main' into copilot/implement-deterministic-trace-graders
Copilot
authored
ef94444
Integrate experiment context into graders and evals

Copilot
and
pelikhan
authored
dce434c
Allow experiments metrics to reference grader results

Copilot
and
pelikhan
authored
7a02562
Mark trace graders as experimental

Copilot
and
pelikhan
authored
b0ad3c7
Merge branch 'main' into copilot/implement-deterministic-trace-graders
github-actions[bot]
authored
673bba9
Merge branch 'main' into copilot/implement-deterministic-trace-graders
github-actions[bot]
authored
1781737
fix trace grader sandbox embedding and script-size contract

Copilot
and
gh-aw-bot
authored
2906c4a
test: add graders integration workflow coverage

Copilot
and
pelikhan
authored
803ce40
Merge branch 'main' into copilot/implement-deterministic-trace-graders
github-actions[bot]
authored
ba7aae8
Merge branch 'main' into copilot/implement-deterministic-trace-graders
github-actions[bot]
authored
9baed09
Merge branch 'main' into copilot/implement-deterministic-trace-graders
github-actions[bot]
authored
f377fa2
Harden trace grader log parsing and summary output

Copilot
and
gh-aw-bot
authored
00ca65f
Harden trace grader fallback parsing and summary safety

Copilot
and
gh-aw-bot
authored
9fff955
chore: start triage for unresolved review feedback

Copilot
and
gh-aw-bot
authored
096aa3e
fix grader sandbox clone fallback and parser edge cases

Copilot
and
gh-aw-bot
authored
455639f
fix: rename grader clone helper to explicit clone-or-undefined

Copilot
and
pelikhan
authored
1d3542e
Merge branch 'main' into copilot/implement-deterministic-trace-graders
github-actions[bot]
authored
4e303c5
refactor: rename trace graders wording to graders

Copilot
and
pelikhan
authored
6c61d48
Merge remote-tracking branch 'origin/main' into copilot/implement-deterministic-trace-graders
Copilot
authored
1b3d870
refactor: rename grader implementation id

Copilot
and
gh-aw-bot
authored
5131cd9
docs: add graders specification

Copilot
and
pelikhan
authored
782f56d

Navigation Menu

Uh oh!

Add initial deterministic trace grading framework - #54678

Add initial deterministic trace grading framework#54678
pelikhan merged 29 commits into
mainfrom
copilot/implement-deterministic-trace-graders

Commits on Aug 22, 2026

Plan deterministic trace graders

Add initial deterministic trace graders

docs(adr): add draft ADR-54678 for deterministic trace grading framework

Merge branch 'main' into copilot/implement-deterministic-trace-graders

Harden trace grader runtime isolation and manifest transport

Merge branch 'main' into copilot/implement-deterministic-trace-graders

Merge remote-tracking branch 'origin/main' into copilot/implement-deterministic-trace-graders

Refresh branch and fix trace graders JS type casts

Merge remote-tracking branch 'origin/main' into copilot/implement-deterministic-trace-graders

Integrate experiment context into graders and evals

Allow experiments metrics to reference grader results

Mark trace graders as experimental

Merge branch 'main' into copilot/implement-deterministic-trace-graders

Merge branch 'main' into copilot/implement-deterministic-trace-graders

fix trace grader sandbox embedding and script-size contract

test: add graders integration workflow coverage

Merge branch 'main' into copilot/implement-deterministic-trace-graders

Merge branch 'main' into copilot/implement-deterministic-trace-graders

Merge branch 'main' into copilot/implement-deterministic-trace-graders

Harden trace grader log parsing and summary output

Harden trace grader fallback parsing and summary safety

chore: start triage for unresolved review feedback

fix grader sandbox clone fallback and parser edge cases

fix: rename grader clone helper to explicit clone-or-undefined

Merge branch 'main' into copilot/implement-deterministic-trace-graders

refactor: rename trace graders wording to graders

Merge remote-tracking branch 'origin/main' into copilot/implement-deterministic-trace-graders

refactor: rename grader implementation id

docs: add graders specification