Accept empty authoritative sync transactions and expose safe failure details

docs/agent-guide/implemented/bugfix/2026-10-19-authoritative-transition-diagnostics.md

@@ -0,0 +1,103 @@
+# Authoritative Transition Diagnostics
+
+## Problem
+
+The installed a0746e97 worker discards begin_authoritative error variants and
+publishes one generic rejection with Storage_busy. The Worker errors publisher
+then replaces every authoritative failure with one generic ledger entry.
+This prevents distinguishing closed databases, sync-version conflicts,
+discontinuous cursors and unsupported transaction formats.
+
+## Decision
+
+Keep the existing overlay variants and worker Error codes. Use one pure
+authoritative transition error mapper in the effect runner, shared by begin and
+apply. All messages and diagnostic category signals must be fixed strings;
+discard raw decode, integrity, persistence and fatal-state text.
+Preserve the category through the existing reducer Diagnostic signal and the
+service's finite allowlist, so Last error and Worker errors both distinguish it.
+Do not change recovery, synchronization policy, protected types, Dune, account
+access, phone installation or graph data.
+
+The runner mapper stays private. Deterministic runtime tests will exercise actual
+cursor and format rejection through Runner and Core, without fabricated producer
+results. Separate propagation tests cover all four typed errors at the reducer
+boundary, and the service's existing pure diagnostic row projection is exported
+through its ordinary (non-spec) mli for direct tests of the actual UI allowlist.
+These propagation tests do not claim producer coverage of close/token races.
+The two deterministic producer races remain explicitly uncovered because the
+existing public runner interface has no such scheduling boundary. No protected
+spec or Dune change is made.
+
+## Alternatives considered
+
+### Only replace the runner message
+
+Rejected because Worker errors would still normalize every failure. Mapping tests
+that inject hand-written worker errors would not verify producer classification.
+Adding a second error enum or moving overlay policy into the contract would add
+unnecessary coupling and potentially require prohibited Dune changes.
+
+## Acceptance criteria
+
+- All four begin-authoritative variants have distinct fixed messages and accurate
+  existing worker Error codes; only genuinely busy states use Storage_busy.
+- Producer mapping and actual downstream diagnostics projection are covered by
+  deterministic tests where publicly reachable, including raw-sensitive-string exclusion.
+- Existing source and public type boundaries are respected; production code
+  changes require no protected-interface extension.
+- A real Web/Simulator reproduction must use the same verified dedicated graph,
+  ordinary UI edits and authenticated production synchronization, not worker injection.
+
+## Risks
+
+- Discarding raw lower-level messages intentionally limits detail to safe categories.
+- Host-restamped Simulator builds retain cross_abi_validated=false.
+- The current 1446 ordinary App has no authenticated session. Its normal Cognito
+  login form requires credentials unavailable in the configured environment.
+  No credentials or tokens may be copied from the Mac browser.
+
+## Questions
+
+- None for the authorized implementation. Explicit parent direction on 2026-10-19
+  keeps the producer mapper private, accepts the honest deterministic coverage gap,
+  and forbids modifying protected interfaces. Simulator authentication and any
+  future producer test-interface extension remain separate pending user actions.
+
+## Consequences
+
+The private shared mapper now emits fixed messages and origin categories using
+existing Error codes. Only actual prepare/commit busy variants use Storage_busy.
+Checksum/integrity/persistence use the broad Ambiguous_sync_state code with their
+exact fixed origin category; a fatal-state commit closes the database and uses
+Closed_session. Neither ownership failure nor disk corruption is inferred.
+Core retains the origin category in its Diagnostic signal. The service's finite
+allowlist retains that category in the existing applyAuthoritative ledger row.
+No recovery, retry, commit, graph, authentication or asset behavior changed.
+
+Three actual runner/database rejection cases failed against the old uniform
+Storage_busy result and passed after the change. Four-category downstream
+propagation and the actual service row allowlist also failed before and passed
+afterward. The producer mapper remains private: database-close and sync-version
+preparation races have no deterministic public hook and are not claimed covered.
+The service projection test is compositional coverage, not mounted UI acceptance.
+
+A clean full @all build and forced full runtest passed: 21 Alcotest suites,
+977 cases, plus standalone programs. The new decision passes validation; the
+whole document check retains four pre-existing invalid historical documents.
+Independent review's two classification findings were corrected and final diff
+review found no further blockers. All temporary evidence is outside the repo.
+
+An external synthetic probe uses real Runner/Database commits and only delays
+consumption of their actual completion. Its control consumes checkpoint 1;
+backgrounding before consumption instead leaves the manager at 0 while SQLite
+is durably at 1. Foreground plus a normal repeated pull then actually rejects
+cursor 1 as authoritativeCursorDiscontinuous. No completion is fabricated and
+no product recovery fix was made. This proves a local client race path, not
+real Web synchronization or the cause of the user's phone screenshot.
+
+The Mac Web test graph is verified online with local/remote tx 872 and no edits
+submitted. Existing Simulator 1446 lacks a logged-in ordinary Journal session;
+its normal Cognito login form requires user input. No credentials were copied,
+no iPhone was used and no graph was deleted. Protected spec and Dune files remain
+unchanged. Simulator packaging, if prepared later, retains experimental ABI limits.

docs/agent-guide/implemented/bugfix/2026-10-20-authoritative-decode-detail-diagnostics.md

@@ -0,0 +1,100 @@
+# Authoritative Decode Detail Diagnostics
+
+## Problem
+
+Build 50 classifies the phone failure as Authoritative_decode_failed but loses
+which preparation, decoding or in-memory application step failed. The existing
+string includes raw Transit, exception and potentially graph-specific content,
+so simply displaying it would leak data and still mislabel application exceptions.
+
+## Decision
+
+Keep the protected Authoritative_decode_failed of string interface and all sync
+behavior. Add a shared, strictly validated bounded diagnostic codec in ordinary
+logseq_db_types sources. Only fixed stages, reasons, type/operation/attribute
+allowlists and nonnegative bounded structural counters may cross this carrier.
+Codec functions keep their current signatures; authoritative decoding emits
+safe records, database adds stage/transaction context, and runner/reducer/service
+propagate only validated records into Last error and Worker errors. Legacy or
+invalid carriers retain the existing generic category, never arbitrary text.
+
+Catch Datascript.with_tx exceptions as application-exception plus a fixed
+exception constructor class. Do not inspect exception strings to guess causes.
+An unresolved lookup ref can be visible only as a structural lookup-reference
+count, not a fabricated precise failure reason or failing operation index.
+Do not repair empty transactions, cursor races, retries, or recovery.
+
+The pure reducer cannot originate wire decoding errors: its public events carry
+already completed effects. Previous public runner/database reproductions reached
+both preparation and application failures. Extend that narrow producer suite;
+service projection tests independently verify its actual sanitizing boundary.
+No private module access, protected spec or Dune changes are required.
+
+## Alternatives considered
+
+### Expose raw lower errors or extend protected variants
+
+Rejected: raw errors can contain user values and paths. Extending protected specs
+requires separate explicit authorization and is unnecessary with a strict safe
+codec over the existing string carrier. Exception-message substring guessing
+would falsely imply a known cause and is excluded.
+
+## Acceptance criteria
+
+- Actual wire decoding failures distinguish preparation, decode and application
+  stages, fixed reasons, transaction/operation position and useful safe types.
+- Last error and Worker errors preserve the same safe detail. Legacy/malicious
+  diagnostic inputs cannot display arbitrary content or malformed counters.
+- Synthetic boundary cases cover empty/invalid transactions, operations/fields,
+  protected values, entity/reference/source-tx/numeric types, missing lookups and
+  application exceptions. Tests assert checkpoint and data remain uncommitted.
+- Full build/tests and independent review pass before building a correct signed
+  iphoneos artifact, backing up Build 50 data and installing an in-place update.
+- No raw payload, plaintext, UUID, account/graph names, token, custom attribute,
+  original exception string, protected spec or Dune change is introduced.
+
+## Risks
+
+- Application exceptions remain intentionally broad because the public API does
+  not identify their failing operation or expose typed cause information.
+- Historical raw strings must fail closed to a generic diagnostic.
+- Host-restamped device builds remain cross_abi_validated=false; USB launch does
+  not prove complete ABI/runtime acceptance or the original incident cause.
+
+## Questions
+
+- None. The user explicitly approved diagnostic detail and iPhone update on
+  2026-10-20. Independent review is requested; no PR, push or merge is authorized.
+
+## Consequences
+
+A shared abstract diagnostic codec restricts the existing string carrier to
+bounded fixed enums and structural counters; unknown or malformed carriers are
+never displayed. Both Last error and Worker errors use its safe formatter.
+Preparation, operation decode and transaction application retain distinct stages.
+Codec failures retain 1-based transaction/operation context and finite type,
+operation and builtin-attribute descriptors. Custom names become custom/other.
+Application exceptions retain only a fixed constructor class and bounded
+best-effort lookup-reference/operation counts, never an operation attribution.
+The original recursive exception fallback remains nonfatal with unknown decode
+classification. Protected spec, Dune and transaction acceptance branches remain
+unchanged; no empty-transaction, cursor-race or retry fix was made.
+
+The actual runner/database producer suite now has 22 deterministic cases,
+including preparation/decode/application failures, second transaction/operation
+position, CAS expected/value, numeric/ref types and bounded statistics. It checks
+Last error, emitted signal, unchanged checkpoint and exclusion of synthetic
+content, paths, UUID and raw application exception text. Service tests cover the
+real row projection, unknown/duplicate/malformed/oversized fields, invalid enums
+and counters, legacy raw input, byte budget and roundtrip. No private module or
+protected interface was opened for tests. The first full build and runtest pass
+21 Alcotest suites / 996 cases plus standalone programs; final verification after
+preserving the recursive catch is recorded externally before packaging.
+
+Independent review found and corrected a test-only cursor fixture regression,
+CAS expected-field attribution, potentially unbounded diagnostic statistics, and
+the original recursive catch scope. The public interfaces cannot deterministically
+induce every unexpected recursive exception or decrypt-count mismatch; these
+are not claimed individually reproduced. The actual phone subtype remains to
+be observed in the next installed diagnostics build, not inferred from fixtures.
+Installation evidence and final hashes remain in this task's external evidence.

docs/agent-guide/implemented/bugfix/2026-10-20-authoritative-empty-transaction-compatibility.md

@@ -0,0 +1,86 @@
+# Authoritative Empty Transaction Compatibility
+
+## Problem
+
+Build 51 shows prepare / empty-transaction / transaction=1 on the phone.
+The first transaction in the current authoritative batch parses as an empty
+Transit array or list; the client rejects it before application. The upstream
+normalized transaction log permits empty operations while advancing its cursor.
+The diagnostic ordinal is not a server cursor.
+
+## Decision
+
+Allow an empty operation array/list only at the private codec authoritative-input
+entry points. Keep strict defaults for durable outgoing-record validation and
+insertion helpers. Preserve the empty transaction as one element of the batch,
+using all existing candidate, origin validation, staging, replay and atomic
+checkpoint/outbox/revision commit paths. No cursor shortcut or dropped batch slot.
+Do not modify protected specs, Dune, sync policy, race recovery, attachments,
+upstream normalization or any other suspected issue. User authorization explicitly
+limits this task to the confirmed empty authoritative transaction gap and iPhone update.
+
+The pure sync reducer stores opaque wire text and cannot originate the decoding
+failure through pure events without fabricating a producer error. The previous
+actual runner/database reproductions confirmed ownership in the overlay codec;
+new regressions use the public Database boundary, the narrowest accessible owner.
+Existing runner diagnostic regressions for empty input must change to success,
+while all other existing failure/privacy cases remain intact.
+
+## Alternatives considered
+
+### Drop empty transactions or accept empty input globally
+
+Rejected: dropping changes cursor/report/origin alignment; a global parser change
+also loosens outgoing validation. Manually moving the checkpoint would bypass
+atomicity and token/generation guards.
+
+## Acceptance criteria
+
+- Old source fails the public Database empty authoritative regressions; the minimal
+  change passes array/list, durable reopen and empty/nonempty/empty batch cases.
+- No-content changes advance the server checkpoint and sync token without an
+  unnecessary logical projection change. Real mixed transactions still apply.
+- Late malformed transaction and storage failure roll back the whole batch;
+  stale token, empty outer batch, malformed input and outgoing-empty validation
+  retain their existing rejection behavior.
+- Final full checks and independent review use the final source. Correct signed
+  iphoneos output is installed in place only after fresh Build 51 backup and
+  rollback preservation; actual device version/startup are read back.
+
+## Risks
+
+- Existing origin validation can still reject an empty transaction that does not
+  match an accepted local submission; this integrity guard must remain unchanged.
+- This fixes client compatibility, not why upstream produced an empty transaction.
+- Experimental host-restamp remains cross_abi_validated=false.
+
+## Questions
+
+- None. The user explicitly approved only this compatibility fix and iPhone
+  update on 2026-10-20, and forbade unrelated speculative fixes.
+
+## Consequences
+
+The private parser/protected-value collector/decoder keep allow_empty=false by
+ default, and only the three authoritative-input calls pass true. Legal empty
+arrays/lists now produce empty operations through the existing candidate,
+origin, storage, replay and atomic commit paths. No production storage, sync
+manager, outbox, recovery, attachment, upstream or unrelated behavior changed.
+The protected spec and Dune remain unchanged.
+
+Eight public Database cases cover empty arrays/lists, mixed batch slots,
+checkpoint/token persistence after reopen and subsequent cursor application,
+late decode and actual SQLite metadata commit failures with disk rollback,
+token guards and malformed neighbour/outer-batch rejection. Seven failed on the
+old producer; all passed after the change. The late failure explicitly checks
+stage=decode/transaction=3. The existing outgoing corruption fixture now also
+checks [] still fails closed. Existing runner empty/second-transaction regressions
+now assert actual success, checkpoint advancement and no Last error; they failed
+on the old source and pass with this change. All other diagnostic failures and
+privacy cases are retained. The complete storage suite (85 cases) also passes.
+Final full build/test results, independent review, source and install evidence
+are recorded externally before phone delivery.
+
+No additional issue was fixed. The user authorization and excluded scope remain
+as stated above; the cause of upstream empty generation and the independent
+attachment error are not investigated or changed in this implementation.