Skip to content

fix: act on the post-0.2.0 exhaustive review findings - #519

Draft
mberrys wants to merge 1 commit into
devfrom
claude/loop-exhaustive-review-73k4t5
Draft

fix: act on the post-0.2.0 exhaustive review findings#519
mberrys wants to merge 1 commit into
devfrom
claude/loop-exhaustive-review-73k4t5

Conversation

@mberrys

@mberrys mberrys commented Sep 2, 2026

Copy link
Copy Markdown
Contributor

Works the September 2026 exhaustive read-only review ("Loop Exhaustive Review After 0.2.0") to the extent it holds up against the current tree. Each finding was re-verified live before being acted on; several did not reproduce and are recorded below rather than "fixed."

Fixed

Finding Change
M-05 PDFLogScrubber scrubs credential material — URL userinfo (a Sentry DSN's shape), HTTP authorization values, and secret-named key/value pairs — ahead of the path/email passes, so a leaked key reads as <CREDENTIAL> rather than being partly eaten by the email pass. Key vocabulary matches isSensitiveKey() in pdfartifactidentity.cpp.
H-02 fetch-images, fetch-text and attachments always record an output.empty-result diagnostic when they produce nothing, and accept a shared --fail-if-empty that raises it to an error and exits 1 findings. A pipeline gating on "figures were produced" can no longer be green-lit by an empty output directory.
M-03 The loop-ocr sidecar reads a staged raster once by descriptor (O_NOFOLLOW where available, size-capped, regular-file checked) and hands the bytes to PIL and easyocr, instead of re-resolving the path for isfile(), Image.open() and readtext() in turn. Language codes are shape-validated so a traversal-shaped value never reaches easyocr's model file names.
L-10 PdfTool sets the staged raster 0600. (The staging directory was already private — QTemporaryDir uses mkdtemp.)
H-03 writeIncremental reports through an optional IncrementalWriteOutcome whether it appended or byte-copied an unchanged document — previously indistinguishable from the success value alone.
M-02 Damaged-document recovery bounds its dense object table by objects actually recovered rather than by the highest object number declared, and carries a real source digest so writeIncremental's "the file changed underneath us" guard is no longer silently disabled for permissively recovered documents.
L-11 PDFNameTreeLoader caps key length and entry count.
M-07 (structure half) Structure-tree parsing bounds recursion depth; cycles were already refused, long acyclic chains were not.
L-01 isPathContained no longer reports a planned output as escaping merely because its target directory does not exist yet, keeping the stricter symlinked-parent rule on the file side.
L-02 makeUniqueFileName probes 128 sequential names, then random discriminators, instead of scanning up to 100k candidates.
L-08 Diagnostics bundles truncate plugin display fields.
M-04 (as it actually exists) The "eng"/"deu" divergence does not exist — "en" is used consistently and no LoopPageMaster/MasterInit.qml is in the tree. What does exist is the OCR defaults (languages, dpi, min-text-chars) being written twice, once in the capability-discovery table and once in the parser; they are now defined once and shared.

Net-new, not in the review

  • PDFNameTreeLoader followed cyclic Kids chains without any cycle or depth guard — a self-referencing name tree recursed until the stack ran out. Named destinations, embedded-file names and multimedia assets all load through it.
  • PDFJBIG2Bitmap::paint skipped the dimension check on its expandY path, the one place a bitmap grows after construction. With region offsets now correctly signed, offsetY reaches MAX_BITMAP_SIZE, so a wide page could ask for a multi-hundred-megabyte resize (and overflow the int pixel count).

Did not reproduce

  • H-01. The premise is stale: embedded-fonts is "coverage": "covered" in docs/preflight-check-catalog-overlay.json, pdfinfofonts already reports is-embedded per font, and the substitution warnings already name both the original and the replacement font.
  • L-04. Already correct and already documented — g_level is std::atomic<int> and the header states the thread-safety contract.
  • L-09 was filed as "no change required" by the review itself.

Deferred, with reasons

  • M-01 (cumulative decode budget at the unbudgeted getDecodedStream call sites) is real — annotation appearance streams, fonts, patterns and actions all take the overload that passes nullptr. Closing it means plumbing PDFProcessingBudget through PDFAnnotationManager and PDFFontCache into the render path, which newly refuses work that previously succeeded. That needs a build and a calibration corpus to size; landing it blind risks rejecting legitimate documents mid-render.
  • M-06 (evidence/annotation ingestion charging) and the XFA half of M-07 are the same class — budget plumbing plus calibration — with XFA additionally spread across a 13k-line generated-style parser.
  • L-03 (two enforcers for one stream-parameter property) and L-07 (Scope::Unknown thread-pool fan-out) are a refactor and a measurement task respectively, neither verifiable here.

Contract changes (flagged deliberately)

Per agent-policy.json, these touch declared contracts and should be reviewed as such:

  1. docs/PDFTOOL_CLI_CONTRACT.md gains the output.empty-result diagnostic code and an "Empty results" section. No new exit code was invented — the empty case maps onto the existing 1 findings ("completed correctly but found … failed checks").
  2. loop-ocr/schemas/ocr-sidecar.schema.json constrains languages items with a pattern matching what engine.py now enforces. Previously-accepted shapes such as "en-US" are now rejected; nothing in the tree emits them.
  3. PDFDocumentWriter (exported) gains a nested enum and a defaulted trailing parameter on both writeIncremental overloads — additive, no existing caller changes. The method has no production callers today; only tests.

Verification

Not built. This environment has no Qt (and no configured build tree), so nothing C++ here has been compiled. That is the main risk in this PR and the reason it is a draft.

What was run:

  • scripts/agent/check-change.py --base origin/dev: changelog, source_integrity, architecture_catalog, policy_adapters all pass; build:* and focused_tests fail on the missing build tree; clang_tidy incomplete.
  • clang-format: every touched file was compared against its own baseline. The gate is already red on dev for several of these files; the five that my edits regressed were re-formatted over the changed line ranges only, and no file is now worse than it was.
  • loop-ocr: python3 -m unittest discover -s tests — 10 tests, all pass, including the four new ones.
  • The new scrubber patterns were validated against a reference implementation of the same pass ordering, which caught two defects before they were committed: a false positive on the English word "token" ("Unexpected token appeared") and a non-idempotent key => value case.

Tests added but not executed: tst_diagnosticstest (4), tst_pdftoolcontract (3), tst_incrementalsavetest (2), tst_budgetexhaustiontest (2), tst_processingbudgettest (1), tst_filenamesanitizertest (1), tst_safefilewritertest (1), tst_jbig2decodertest (1).

🤖 Generated with Claude Code

https://claude.ai/code/session_01FDJQDFR5ctoKDJbp7LG4GS


Generated by Claude Code

Works the net-new findings from the September 2026 read-only review that were
reproducible against the current tree, plus three adjacent defects the review
did not name.

Security / privacy
- PDFLogScrubber scrubs credential material (URL userinfo such as a Sentry DSN,
  HTTP authorization values, secret-named key/value pairs) ahead of the existing
  path and email passes. The bare auth-scheme pass deliberately excludes "Token"
  so parser diagnostics are not redacted as secrets.
- loop-ocr reads a staged raster once by descriptor instead of re-resolving the
  path for isfile(), PIL, and easyocr in turn, closing the TOCTOU window;
  language codes are shape-validated before they reach easyocr's model file
  names; PdfTool stages the raster 0600.
- Diagnostics bundles truncate plugin display fields.

Fail-closed behaviour
- PdfTool extraction commands record output.empty-result and accept a shared
  --fail-if-empty (exit 1, findings) so an empty output directory cannot pass a
  pipeline that gates on produced files.
- writeIncremental reports whether it appended or only byte-copied; damaged
  documents now carry a source digest, so its "file changed underneath us"
  guard is no longer silently disabled for permissively recovered documents.

Bounds on attacker-controlled shapes
- Damaged-document recovery bounds its dense object table by objects recovered,
  not by the highest declared object number.
- PDFNameTreeLoader terminates cyclic Kids chains and caps depth, entry count,
  and key length (net-new: the cycle was unbounded recursion).
- Structure-tree parsing bounds recursion depth on long acyclic chains.
- PDFJBIG2Bitmap::paint validates grown dimensions on its expandY path, the one
  path that escaped the constructor's dimension check (net-new).

Ergonomics
- isPathContained no longer rejects a planned output whose target directory does
  not exist yet, keeping the stricter symlinked-parent rule for the file side.
- makeUniqueFileName probes 128 sequential names, then random ones.
- OCR option defaults are defined once and shared by capability discovery and
  the command-line parser.

Not built or run here: this environment has no Qt, so the C++ changes are
unverified by compilation. The Python sidecar tests pass and the new scrubber
patterns were validated against a reference implementation of the same passes.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FDJQDFR5ctoKDJbp7LG4GS
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants