Skip to content

Wire transparency flattening into standards-convert (#167) - #524

Draft
mberrys wants to merge 14 commits into
devfrom
cc/pensive-cerf-kkvr3f
Draft

Wire transparency flattening into standards-convert (#167)#524
mberrys wants to merge 14 commits into
devfrom
cc/pensive-cerf-kkvr3f

Conversation

@mberrys

@mberrys mberrys commented Sep 2, 2026

Copy link
Copy Markdown
Contributor

Summary

Issue #167 ("Add PDF/X and PDF/A conversion, not only validation") turns out to already have substantial, working, registered Core scaffolding: standards-convert (LoopLibCore/sources/pdfstandardconversion.{h,cpp}) already converts to PDF/X-1a:2001, PDF/X-3:2002, PDF/X-4, and PDF/A-2b, produces a pre-conversion change report, is fail-closed, requires an independent validator before committing, and is already exposed identically through PdfTool's repair command and PageMaster's export job.

One concrete, verifiable gap remained: PDF/X-1a and PDF/X-3 forbid live transparency, and the issue explicitly calls flattening out as "a hard prerequisite for X-1a" — but standards-convert never called the existing PDFTransparencyFlattener operation (#164, already used as a separate PageMaster pipeline stage). Live transparency was therefore an unconditional, unfixable blocker in standards-convert even though Core already had a working flatten path.

This PR wires the two together, mirroring the existing RGB→CMYK integration in the same file exactly:

  • PDFStandardConversionSettings::flattenTransparency (new; default-on for X-1a/X-3, matching the existing normalizeColor default pattern; opt-in for X-4/PDF-A, which permit live transparency).
  • pdfx.transparency.allowed becomes a fixable preflight blocker only when flattening is requested/defaulted-on — fixtures with no live transparency are unaffected, since that rule already reports Passed when transparencyObjects == 0 (verified by reading preflightengine.cpp; the existing unconvertiblePdfxHasNoMarker test uses an empty-page fixture with zero transparency objects, so its blocker comes from elsewhere and is untouched by this change).
  • The flatten runs before the output-intent/page-box rewrite in apply(), and its report is surfaced verbatim under a new transparency_flatten report field — a real, reported content change, never a silent approximation (matching the issue's "not an opaque generative editing system" boundary).
  • New flatten_transparency operation parameter, available identically from PdfTool (repair --operation standards-convert --param flatten_transparency=...) and PageMaster's export job, since both already share the one Core implementation.

Also corrected two stale documentation claims found while investigating:

  • docs/STANDARD_CONVERSION.md said an Editor adapter could land "after the 0.1.1 GUI gate." That gate is already complete; docs/LOOP_SHELL_CONTRACT.md is explicit that product GUI work instead remains gated by the still-closed S21 canvas / S22 Quick admission contracts, so this doc now points there instead of implying a stale timeline.
  • docs/REPO_MAP.md's LoopEditorPlugins/ module does not exist in the current Qt-Quick-based tree (the app moved to LoopEditor/ + LoopLibInteraction/ + LoopLibQuick/). Noted in the new doc text so a future Editor adapter isn't scoped to a directory that no longer exists.

Why not more of #167?

Verification

No Qt/CMake toolchain is available in this sandbox, so I could not build or run UnitTestsStandardOracle / UnitTestsConversionOracle / UnitTestsRepairOperation locally. What I could run and did run, all passing:

  • clang-format --dry-run --Werror on every touched file.
  • python scripts/ci/check_source_integrity.py
  • python scripts/generate-architecture-catalogs.py --check
  • python scripts/agent/generate-adapters.py (adapters already current)
  • python scripts/agent/check-change.py --base origin/dev: changelog, source_integrity, architecture_catalog, policy_adapters, and format:<every file I touched> all report pass. The build:*/focused_tests checks report fail only because no build directory exists in this environment — that's an environment limitation, not a result from this diff. I traced the exact API shapes (PDFTransparencyFlattener::apply/hasLiveTransparency, PDFTransparencyFlattenSettings, the CMake target list) by reading the source directly and mirrored an already-compiling, already-tested sibling code path line-for-line, but this PR's first real compile/test signal will be CI.

I'll watch this PR and fix anything CI turns up.

Test plan

  • CI build (LoopLibCore and dependents) is green.
  • UnitTestsStandardOracle, UnitTestsConversionOracle, UnitTestsRepairOperation pass unchanged (no regression to the existing unconvertible/oracle fixtures).
  • scripts/agent/check-change.py --base origin/dev reports pass in CI where a real build directory exists.

🤖 Generated with Claude Code

https://claude.ai/code/session_01JgoNmy614kqRiTBjiSjRJU


Generated by Claude Code

mberrys and others added 11 commits September 2, 2026 02:23
- LoopLibCore: pass PDFProcessingBudget to PDFTextLayoutGenerator during searchDocumentText to enforce hostile-workload limits
- LoopEditor QuickOutlineModel: expose page role from PDFOutlineItem destination and route via implemented goToPage/goToOutlinePage
- LoopEditor DocumentPane/Host: make searchPanelVisible one-shot via acknowledgeSearchPanel to prevent repeated reveal on presentationChanged
- UnitTests: bump catalog implemented count 16 -> 25 to match shell-implemented find/layout commands added in unstable
<!-- CURSOR_AGENT_PR_BODY_BEGIN -->
Promotes `unstable` into `stable` for the 0.2.1 release line.

**Merge commit:** `2ab848b7253714a735640067031eaf45f29a7d65`

This is the Session 07 package-workflow SHA after PR #490 landed on
`unstable`.
<!-- CURSOR_AGENT_PR_BODY_END -->

<div><a
href="https://cursor.com/agents/bc-add9255c-ab75-4ce7-a6ef-326374bfe342?cursor_ref=pr_footer&cursor_cta=open_in_web"><picture><source
media="(prefers-color-scheme: dark)"
srcset="https://cursor.com/assets/images/open-in-web-dark.png"><source
media="(prefers-color-scheme: light)"
srcset="https://cursor.com/assets/images/open-in-web-light.png"><img
alt="Open in Web" width="114" height="28"
src="https://cursor.com/assets/images/open-in-web-dark.png"></picture></a>&nbsp;<a
href="https://cursor.com/background-agent?bcId=bc-add9255c-ab75-4ce7-a6ef-326374bfe342&cursor_ref=pr_footer&cursor_cta=open_in_cursor"><picture><source
media="(prefers-color-scheme: dark)"
srcset="https://cursor.com/assets/images/open-in-cursor-dark.png"><source
media="(prefers-color-scheme: light)"
srcset="https://cursor.com/assets/images/open-in-cursor-light.png"><img
alt="Open in Cursor" width="131" height="28"
src="https://cursor.com/assets/images/open-in-cursor-dark.png"></picture></a>&nbsp;</div>
AppImage packaging failed because appstreamcli could not reach the
legacy mberrys/Loop-pdf homepage, bugtracker, and help URLs (404).
Update metainfo to the canonical studio-berry/loop repository.

Co-authored-by: michael berry <mberrys@users.noreply.github.com>
Windows MSI packaging failed because sentry-cli uploaded PDBs to
berry-studios/loop-pdf, which does not exist on de.sentry.io. The org
project slug is loupe-pdf (project id 4511866328449104, matching the
compile-time DSN).

Co-authored-by: michael berry <mberrys@users.noreply.github.com>
…tract

sentry-cli accepts project id 4511866328449104 (same as compile-time DSN)
instead of the legacy slug, avoiding the banned product token in tracked
source while still uploading to the correct berry-studios EU project.

Co-authored-by: michael berry <mberrys@users.noreply.github.com>
No Loop-named project slug exists on de.sentry.io; loop-pdf is absent and
the legacy slug is policy-banned. Derive the numeric project id from the
canonical LOOP_SENTRY_DSN in CMakeLists.txt instead of hardcoding slugs in
workflows or tracked defaults.

Co-authored-by: michael berry <mberrys@users.noreply.github.com>
…521)

<!-- CURSOR_AGENT_PR_BODY_BEGIN -->
## Summary

Session 07 packaging fixes on one branch for merge into `stable`.

### Linux AppImage
AppStream metainfo URLs → `studio-berry/loop` (fixes `url-not-reachable`
on legacy `mberrys/Loop-pdf`).

### Windows MSI — Sentry debug upload
`loop-pdf` slug does not exist on `de.sentry.io`. The real project is id
**`4511866328449104`** (matches compile-time DSN in `CMakeLists.txt`).

`sentry-cli debug-files upload` accepts that numeric project id.
Workflows and script default now use `4511866328449104` instead of any
legacy slug, satisfying `check_loop_identity` (banned legacy product
token) while uploading to the correct `berry-studios` EU project.

## Verification (local)
- `python3 scripts/ci/check_loop_identity.py` — pass
- `python3 -m unittest discover -s scripts/ci -p 'test_*.py'` — pass
(223 tests)
- `python3 scripts/ci/check_supply_chain_pins.py` — pass
- `appstreamcli validate --pedantic` on appdata.xml — pass

## Scope
Minimal packaging/config only. Not merged; parent handles merge +
package re-dispatch.
<!-- CURSOR_AGENT_PR_BODY_END -->

<div><a
href="https://cursor.com/agents/bc-9291ca40-dca3-4c3a-9911-540c440e8f37?cursor_ref=pr_footer&cursor_cta=open_in_web"><picture><source
media="(prefers-color-scheme: dark)"
srcset="https://cursor.com/assets/images/open-in-web-dark.png"><source
media="(prefers-color-scheme: light)"
srcset="https://cursor.com/assets/images/open-in-web-light.png"><img
alt="Open in Web" width="114" height="28"
src="https://cursor.com/assets/images/open-in-web-dark.png"></picture></a>&nbsp;<a
href="https://cursor.com/background-agent?bcId=bc-9291ca40-dca3-4c3a-9911-540c440e8f37&cursor_ref=pr_footer&cursor_cta=open_in_cursor"><picture><source
media="(prefers-color-scheme: dark)"
srcset="https://cursor.com/assets/images/open-in-cursor-dark.png"><source
media="(prefers-color-scheme: light)"
srcset="https://cursor.com/assets/images/open-in-cursor-light.png"><img
alt="Open in Cursor" width="131" height="28"
src="https://cursor.com/assets/images/open-in-cursor-dark.png"></picture></a>&nbsp;</div>
PDF/X-1a:2001 and PDF/X-3:2002 forbid live transparency, but
standards-convert never called the existing PDFTransparencyFlattener
operation (#164) -- live transparency was an unconditional, unfixable
blocker even though Core already has a working flatten path that
PageMaster's export pipeline uses as a separate stage.

Wire PDFTransparencyFlattener::apply()/hasLiveTransparency() into
PDFStandardConversion::preview()/apply(), mirroring the existing
RGB-to-CMYK integration exactly:

- New PDFStandardConversionSettings::flattenTransparency (default-on
  for X-1a/X-3, matching normalizeColor's existing default pattern;
  opt-in for X-4/PDF-A, which permit live transparency).
- pdfx.transparency.allowed becomes a fixable preflight blocker only
  when flattening is requested, so unrelated fixtures without live
  transparency are unaffected (transparencyObjects stays 0, the rule
  already reports Passed).
- The flatten runs before the output-intent/page-box rewrite and its
  report is surfaced verbatim under a new transparency_flatten report
  field -- a real, reported content change, never a silent
  approximation.
- New flatten_transparency parameter on the standards-convert
  operation, available identically from PdfTool's repair command and
  PageMaster's export job (the one shared Core implementation).

Also correct docs/STANDARD_CONVERSION.md's stale claim that an Editor
adapter can land "after the 0.1.1 GUI gate": that gate is already
complete per docs/LOOP_SHELL_CONTRACT.md, which gates product GUI work
behind the still-closed S21/S22 admission contracts instead. Note that
docs/REPO_MAP.md's LoopEditorPlugins/ module does not exist in the
current Qt-Quick-based tree, so a future Editor adapter belongs under
LoopLibInteraction/ + LoopEditor/qml/.

No Qt/CMake toolchain is available in this environment, so the build
and UnitTestsStandardOracle/UnitTestsConversionOracle/UnitTestsRepairOperation
targets could not be run locally; clang-format, source-integrity, and
architecture-catalog checks all pass. CI will provide the first real
build/test signal for this change.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JgoNmy614kqRiTBjiSjRJU
CI on #524 failed with three unrelated pre-existing dev-branch bugs
(reproduced identically on origin/dev's own tip, confirmed by diffing
these exact files against origin/dev before this commit):

- .github/workflows/reusable-linux.yml and reusable-windows.yml's
  "Verify processing-budget exhaustion corpus" step used
  working-directory: loupe, a pre-rebrand path that doesn't exist
  (the checkout path is "loop") -- this is what broke agent-fast/build.
- UnitTests/testdata/budget-exhaustion/manifest.json,
  UnitTests/tst_budgetexhaustiontest.cpp, and
  scripts/budget_exhaustion/generate_corpus.py still used the legacy
  "loupe-processing-budget-exhaustion-corpus" schema_kind token, which
  scripts/ci/check_loop_identity.py's fail-closed legacy-token scan
  correctly flags -- this is what broke source_integrity and policy.

Ported the same fix already sitting in open PR #518 (which bundles it
with unrelated 0.2.1 milestone doc changes not relevant here) rather
than duplicating the diagnosis. Merged origin/dev first since this
topic branch was 14 commits behind it and didn't have these files
locally yet.

Verified locally (all pass): scripts/budget_exhaustion/generate_corpus.py
--check, scripts/ci/test_check_loop_identity.py, the full
scripts/ci/test_*.py suite (223 tests), check_source_integrity.py,
check_supply_chain_pins.py, verify-loop-shell-contract.py,
generate-architecture-catalogs.py --check, clang-format.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JgoNmy614kqRiTBjiSjRJU

mberrys commented Sep 2, 2026

Copy link
Copy Markdown
Contributor Author

CI on the first push failed 3 checks (source_integrity, policy, agent-fast / build), none caused by this PR's diff:

  • test_repository_contract_passes failed identically in source_integrity and policy on 5 pre-existing "legacy product token" findings in files this PR never touches. I confirmed these files are byte-identical to origin/dev's own tip, so this already fails on dev.
  • agent-fast / build failed because a workflow step's working-directory: loupe doesn't exist (checkout path is loop) — same pre-rebrand leftover, also already on dev.

A fix for exactly this already exists in open PR #518 (bundled there with unrelated 0.2.1 milestone doc changes), so I ported just the CI-relevant hunks here rather than duplicating the diagnosis or waiting on that PR to merge: .github/workflows/reusable-linux.yml/reusable-windows.yml (working-directory: loupeloop) and the three loupe-processing-budget-exhaustion-corpusloop-processing-budget-exhaustion-corpus schema-kind occurrences (UnitTests/testdata/budget-exhaustion/manifest.json, UnitTests/tst_budgetexhaustiontest.cpp, scripts/budget_exhaustion/generate_corpus.py). This topic branch was also 14 commits behind dev and didn't have these files yet, so I merged origin/dev in first (clean, no conflicts).

Verified locally: generate_corpus.py --check, test_check_loop_identity.py, the full scripts/ci/test_*.py suite (223 tests), check_source_integrity.py, check_supply_chain_pins.py, verify-loop-shell-contract.py, generate-architecture-catalogs.py --check, and clang-format all pass. Pushed as c3d610d; watching for the next CI run.


Generated by Claude Code

…tor.h

Unrelated to this PR's diff (introduced by commit 6c628b4, already on
this topic branch before this PR's work began) but blocking agent-fast/
build's format:LoopLibCore/sources/pdftextlayoutgenerator.h check.
Single blank line removed inside the constructor body; no behavior
change. Confirmed via the same CI run that build:LoopLibCore,
build:UnitTestsStandardOracle, build:UnitTestsConversionOracle, and
focused_tests all pass with this PR's actual code changes.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JgoNmy614kqRiTBjiSjRJU

mberrys commented Sep 2, 2026

Copy link
Copy Markdown
Contributor Author

Good news buried in that failure: agent-fast / build actually got far enough this time to run a real build, and it confirms the core change — build:LoopLibCore, build:UnitTestsConversionOracle, build:UnitTestsStandardOracle, and focused_tests (the ctest run) all reported pass. No regressions to the existing oracle/fixture tests, and the new transparency-flattening code compiles cleanly.

The only failure was one pre-existing clang-format violation in LoopLibCore/sources/pdftextlayoutgenerator.h (a single stray blank line, introduced by commit 6c628b4 already on this branch before this PR's work — not in this PR's diff, and not present on dev). Fixed in dcebaed (mechanical, whitespace-only). Watching for the next CI run.


Generated by Claude Code

mberrys commented Sep 2, 2026

Copy link
Copy Markdown
Contributor Author

All 12 checks are green on dcebaed (CI/agent-fast build, source_integrity, policy, architecture-docs, fuzz, CodeQL ×3, Supply Chain Policy ×2). No merge conflicts, no open review threads. This PR is ready for human review — still in draft since I opened it; happy to mark ready or keep it draft, whichever the maintainer prefers.


Generated by Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants