Skip to content

Preserve source accounts when editing prose - #330

Open
ariobarin wants to merge 1 commit into
mainfrom
source-faithful-editing
Open

ariobarin wants to merge 1 commit into
mainfrom
source-faithful-editing

Conversation

@ariobarin

Copy link
Copy Markdown
Owner

Readable edits were preserving project facts while losing the author's objections and qualifications. Ground Unslop in the original account and latest corrections, preserve five primary research sources with their limits, and make substantive editorial changes inspectable.

@ariobarin

Copy link
Copy Markdown
Owner Author

Verification and limits for this change:

git diff --check passed. scripts/test-all.ps1 passed under both PowerShell 7 and Windows PowerShell 5.1, including installing the selected skill and its references into temporary homes and checking the portable allowlist. No live Compass installation was changed.

I also ran a small editorial check with Codex 0.147.0, reporting model gpt-5.6-sol and reasoning effort none, using fresh ephemeral invocations, ignored user configuration, a read-only sandbox, supplied text, and no agent tool calls. Three invented fixtures were presented together: a flattened project story with original notes, an old account superseded by an explicit correction and withdrawn anecdote, and an ordinary README setup list.

The initial comparison used the same compact restatement of the existing pattern guidance in both conditions, adding the proposed source-account paragraphs only to the candidate. The baseline already recovered much of the story and respected the correction. The candidate additionally made substantive editorial choices visible in source notes; both kept the useful README list. This is evidence for inspectability, not a demonstrated general improvement in prose quality.

There were residual failures. Both versions introduced an unsupported somehow into the author's account. A further run with the full skill and an extra warning against adding certainty or doubt still did so, and also expanded a dry-run description beyond the supplied text. I removed the extra warning rather than retain an instruction that had not earned its place. The retained revision is the narrow source-comparison and editorial-note change, not a claim that prompting eliminates drift.

These were single-run, source-present smoke checks, not a blinded human evaluation, repeated-edit study, repository-retrieval test, or automatic skill-triggering test. Do not treat the passing installation suite as a writing-quality benchmark.

Fixtures for repeating the editorial check

Common task: Work only with the supplied text. Do not use tools or access files. Edit each independent piece for its stated purpose. Return revised pieces, with brief editorial notes only where useful.

A. Please tighten this small website story and make it less bland. I included the original notes in case they help.

Original notes: Before this I made a timetable in a spreadsheet, then a second spreadsheet to keep score in a game. At this point I should probably have stopped using spreadsheets. Instead I made the third one into a dashboard. I avoided the chart library because I was already trying to learn Python and didn't want another thing to learn. The refresh trick was ridiculous: one cell produced an error, the next cleared it, and that kicked the data loose. It worked. The second version was technically better but I was less interested in it. Rebuilding the same thing properly wasn't as fun as discovering that the bad version worked.

Current draft: At this point I built a spreadsheet dashboard backed by Python. A custom refresh mechanism kept the data current. A second version improved the implementation. The project taught me to iterate and choose tools that fit the problem.

B. Please edit this project page for clarity.

Old account: I trained a boosted-tree model, served it, and watched the console all night. I was sure we'd beaten the simple baseline.

Latest author correction: Those were experiments, not the shipped model. The deployed predictor was a calibrated random forest. The test set was too small to claim it beat the baseline. I don't want to repeat the console story on this page anymore.

Current draft: I conducted model experiments and built a deployed prediction pipeline with a calibrated random forest. The results demonstrated the power of iterative experimentation. More evaluation is needed.

C. Please clean up these setup instructions for a README.

Draft: It is important to note that there are three simple steps to get started with the powerful exporter. First, install the package. Second, copy sample.toml to settings.toml. Third, run export --dry-run. The dry run does not write files. This seamless workflow is a testament to the tool's intuitive design.

The compact pattern guidance used in the first comparison was: Edit the writing to remove AI patterns. Preserve its meaning and intended tone; do not invent opinions, anecdotes, or personality to make it seem human. Patterns to notice: empty praise in place of facts; vague authority without an identifiable source; inflated language and decorative metaphors; forced contrasts and generic conclusions; filler and redundant presentation; false variety and stacked hedges; canned rapport. These are clues, not a banned-word list. Keep precise technical terms, useful lists, contrasts that explain a choice, and qualifications that change a claim. Leave working sentences alone.

For the candidate, replace its opening with the new opening and insert the new Keep the source in view paragraphs from this PR before the unchanged compact pattern guidance. The full-skill follow-up used the entire skill instead. For a fresh evaluation, test the actual old and new skills under matched conditions rather than assuming this condensed comparison establishes their relative performance.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant