File: .github/skills/curriculum-quantitative-assessment/curriculum_assessment.py (affects scoring for workshop/side-quest-01-02-environment-reference.md and 87 other files)
Overall Score: 4.93 / 10.0 (corpus mean: 6.14)
Flagged Dimensions:
| Dimension |
Score |
Benchmark |
Delta |
| checkpoint_quality |
0.0 |
≥ 4 items → up to 10 |
-10.0 |
| active_learning |
2.1 |
density ≥ 3 → 10 |
-7.9 |
| cognitive_load |
6.3 |
≤ 800 words |
-3.7 |
Root Cause (≤ 2 sentences):
CHECKPOINT_RE = re.compile(r"##\s+✅\s*Checkpoint", ...) only matches a literal ✅ Unicode emoji character, but the workshop's authoring convention (per AGENTS.md) and every actual workshop file use the GitHub Markdown emoji shortcode ## :white_check_mark: Checkpoint. As a result has_checkpoint evaluates to False for 88 of 93 files (including this one, which has a fully-formed 7-item checkpoint at line 155), zeroing the 2.0-weighted checkpoint_quality dimension across almost the entire corpus and systematically deflating every score in the scorecard.
Evidence (quoted from the file):
## :white_check_mark: Checkpoint
(from workshop/side-quest-01-02-environment-reference.md, line 155 — a real checkpoint with 7 checklist items that the rubric currently scores as absent)
Learning Science Rationale:
This is not a content-quality issue but a measurement-validity issue. A rubric is only useful if its automated instrumentation actually detects the artifact it claims to measure — a checkpoint that satisfies Bloom's "evaluate" tier (learners verify their own outcomes) is present but invisible to the scorer. Treating an undetected checkpoint as "missing" produces false-negative findings (missing_checkpoint flagged on files that already have one), which would misdirect authoring effort toward re-adding sections that already exist instead of toward genuine gaps like word count or activity density.
Improvement Prompt (for an agent):
Open .github/skills/curriculum-quantitative-assessment/curriculum_assessment.py and update the CHECKPOINT_RE regex (currently `re.compile(r"##\s+✅\s*Checkpoint", re.IGNORECASE)`) so it also matches the GitHub Markdown emoji shortcode form used throughout workshop/*.md: `## :white_check_mark: Checkpoint`. Change it to something like:
CHECKPOINT_RE = re.compile(r"##\s+(?:✅|:white_check_mark:)\s*Checkpoint", re.IGNORECASE)
Also check the prerequisite-section regex (`##\s+📋\s*Before You Start`) for the same literal-emoji-vs-shortcode mismatch against workshop/*.md, which use `## :clipboard: Before You Start`, and fix it the same way. Re-run the corpus scoring script to confirm has_checkpoint and has_prereq_section now correctly report True for files that contain these sections, and confirm the corpus mean score rises accordingly. Do not change any workshop content files — this is a scoring-tool bug fix only.
Expected Score After Fix: 6.9 / 10.0 (file-level; corpus mean rises to approximately 8.1 / 10.0 once the regex fix is applied to all 88 affected files)
Generated by 🔬 Curriculum Quality Evaluator · copilot · auto · 76.7 AIC · ⌖ 17.4 AIC · ⊞ 9.5K · ◷
File:
.github/skills/curriculum-quantitative-assessment/curriculum_assessment.py(affects scoring forworkshop/side-quest-01-02-environment-reference.mdand 87 other files)Overall Score:
4.93 / 10.0(corpus mean:6.14)Flagged Dimensions:
Root Cause (≤ 2 sentences):
CHECKPOINT_RE = re.compile(r"##\s+✅\s*Checkpoint", ...)only matches a literal✅Unicode emoji character, but the workshop's authoring convention (perAGENTS.md) and every actual workshop file use the GitHub Markdown emoji shortcode## :white_check_mark: Checkpoint. As a resulthas_checkpointevaluates toFalsefor 88 of 93 files (including this one, which has a fully-formed 7-item checkpoint at line 155), zeroing the 2.0-weightedcheckpoint_qualitydimension across almost the entire corpus and systematically deflating every score in the scorecard.Evidence (quoted from the file):
Learning Science Rationale:
This is not a content-quality issue but a measurement-validity issue. A rubric is only useful if its automated instrumentation actually detects the artifact it claims to measure — a checkpoint that satisfies Bloom's "evaluate" tier (learners verify their own outcomes) is present but invisible to the scorer. Treating an undetected checkpoint as "missing" produces false-negative findings (
missing_checkpointflagged on files that already have one), which would misdirect authoring effort toward re-adding sections that already exist instead of toward genuine gaps like word count or activity density.Improvement Prompt (for an agent):
Expected Score After Fix:
6.9 / 10.0(file-level; corpus mean rises to approximately8.1 / 10.0once the regex fix is applied to all 88 affected files)