Skip to content

Verify Morris source reconstruction - #711

Merged
dcellison merged 1 commit into
mainfrom
verify_source_reconstruction
Aug 6, 2026
Merged

Verify Morris source reconstruction#711
dcellison merged 1 commit into
mainfrom
verify_source_reconstruction

Conversation

@dcellison

Copy link
Copy Markdown
Owner

Summary

  • add project/source_reconstruction_manifest.json with the stored Morris witness, morris label, exact chapter boundaries, and stated normalization rule for chapters 1 through 6
  • add an independent checker that compares each ordered citation stream character for character with its selected source passage
  • report the first mismatch directly as missing, duplicated, reordered, or altered source text
  • require every texts/news_from_nowhere/chapter_*.md file to have a manifest entry, so a later chapter cannot bypass the check
  • add 13 regression tests, including deliberate mutations for all four mismatch classes and the Project Gutenberg line-wrap rule
  • run the checker and its tests in CI, document the command in the translation procedure and repository indexes, record D116, complete NTP-02, and mark NTP-03 next

No Morris citation, Phi sentence, gloss, derived English, vocabulary entry, or grammar rule changes in this PR.

Evidence boundary

The checker reuses the established interlinear decoder only to read citation strings. It selects and normalizes the independently stored source itself, then compares the two complete streams without consulting certification counts or digests. The gutenberg-wrapped-prose-v1 rule rejoins hyphenated words broken by source-file line wrapping and collapses layout whitespace; spelling, punctuation, capitalization, and hyphens remain exact.

Chapter Citations Normalized source characters
1 77 5,214
2 235 16,061
3 233 19,028
4 116 6,237
5 273 13,932
6 120 14,771
Total 1,054 75,243

Prose review

The reader-facing additions received the required Humanizer and Phi voice pass. The pass corrected an overbroad attribution in the root README: lexical validation and independent source reconstruction now have separate, accurately named responsibilities instead of one validator appearing to prove both.

Checks

  • python3 scripts/source_reconstruction.py (6 reconstructions, 1,054 citations, 75,243 normalized characters)
  • python3 scripts/test_source_reconstruction.py (13 tests)
  • python3 scripts/validate_examples.py (1,305 entries, 0 errors, 0 warnings)
  • python3 scripts/validate_sentences.py (3,313 examples, 0 errors)
  • python3 scripts/test_sentence_validator.py (25 tests)
  • python3 scripts/test_vocabulary_schema.py (25 tests)
  • python3 scripts/content_vocabulary_decisions.py --check (82 batches, 280 decisions)
  • python3 scripts/test_content_vocabulary_decisions.py (8 tests)
  • python3 scripts/test_name_forms.py (20 tests)
  • python3 scripts/test_translation_layers.py (11 tests)
  • python3 scripts/translation_process_status.py --check (20 translations)
  • python3 scripts/test_translation_process_status.py (9 tests)
  • python3 scripts/audit_phonetic_neighbors.py --output /tmp/phonetic_neighbors_ntp02.txt and baseline comparison (250 pairs, unchanged)
  • python3 scripts/generate_reference.py (generated references unchanged)
  • python3 scripts/build_site.py
  • python3 -m py_compile scripts/source_reconstruction.py scripts/test_source_reconstruction.py
  • git diff --check

@dcellison
dcellison merged commit d861da3 into main Aug 6, 2026
1 check passed
@dcellison
dcellison deleted the verify_source_reconstruction branch August 6, 2026 13:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants