From 7a324d3b2084bdf6823ed33236d7f3d3785c3be8 Mon Sep 17 00:00:00 2001 From: Holly Grimm Date: Sat, 18 Jul 2026 13:51:32 -0600 Subject: [PATCH] docs(readme): reflect raw/derived transcript design Layout comments: transcript.json is the immutable raw diarization (SPEAKER_NN), transcript.txt the derived speaker-labeled view, captions/ may carry salvaged youtube_captions.txt. Provenance section updated: most transcripts are now WhisperX-diarized (was: captions with Whisper fallback); names live in parts[].speakers; private/ unlisted videos are not transcribed. Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_01UoVcFP7bBu92mK6RqM5nPG --- README.md | 20 +++++++++++++------- 1 file changed, 13 insertions(+), 7 deletions(-) diff --git a/README.md b/README.md index 002d555dc..9467d1c4f 100644 --- a/README.md +++ b/README.md @@ -16,10 +16,10 @@ alongside, e.g. `data/video//…` or `data/// ``` data/video/activeinferenceinstitute/// - metadata.json # canonical: series, item, parts[{video_id, url, title, duration, upload_date}] - transcript.txt # clean text (part-tagged when multi-part) - transcript.json # timestamped segments (where available) - captions/ # original-language .srt + metadata.json # canonical: series, item, parts[{video_id, url, title, duration, upload_date, speakers}] + transcript.txt # derived speaker-labeled text (part-tagged when multi-part) + transcript.json # raw diarized segments — immutable, SPEAKER_NN labels (where available) + captions/ # original-language .srt (+ youtube_captions.txt where captions predate diarization) translations/ # translated .srt (per language) assets/ # images, html, prose, appendices, bibliography, … README.md # human nav: titles, links, contents @@ -42,6 +42,12 @@ is represented (uncategorized videos live under `Other/`). ## Provenance Transcripts and metadata are pulled completely and idempotently from the Institute -YouTube channel (captions, or local Whisper where captions are absent) by -[Journal-Utilities](https://github.com/ActiveInferenceInstitute/Journal-Utilities) -(`scripts/refactor_journal.py`, `scripts/download_channel.py`). See [`docs/SCHEMA.md`](docs/SCHEMA.md). +YouTube channel by +[Journal-Utilities](https://github.com/ActiveInferenceInstitute/Journal-Utilities). +Most transcripts are WhisperX-diarized: `transcript.json` holds the immutable raw +segments (`SPEAKER_NN`), human speaker identifications are recorded in +`metadata.json` `parts[].speakers`, and `transcript.txt` is regenerated from the +two (`scripts/apply_speaker_names.py`). Items without diarization yet carry +YouTube-caption text; original captions always remain under `captions/`. +Private/unlisted videos are not transcribed. See [`docs/SCHEMA.md`](docs/SCHEMA.md) +("Transcripts — raw vs derived").