Skip to content

feat(observation): add v3 relation-scoped index evidence - #46

Draft
seonghobae wants to merge 927 commits into
codex/pr6-v3-representationfrom
codex/pr6-v3-index-evidence
Draft

seonghobae wants to merge 927 commits into
codex/pr6-v3-representationfrom
codex/pr6-v3-index-evidence

Conversation

@seonghobae

@seonghobae seonghobae commented Sep 11, 2026

Copy link
Copy Markdown
Contributor

Stacked on #45 (codex/pr6-v3-representation). This remains the Draft Source Observation successor and single writer for the active relation/index-partition source-domain slice. Ordinary-forward commits are adopted; exact-head execution/review evidence never transfers after head movement.

Current authority — 2026-09-17 KST

  • Base: feat(observation): add domain-separated v3 successor schema representation #45 exact 6b2a8f555725dc79f60432afbc492d6005290a4a.
  • Start-of-pass predecessor: cc64bab7d81f2adb691f627e7de852ace6946227.
  • Current exact head: e2284292ed83c0c957bbcdc5745ca15585f56438, 8 ordinary commits ahead / 0 behind the pass predecessor.
  • Lifecycle: OPEN / Draft / mechanically mergeable. Mechanical mergeability is not acceptance evidence.
  • Protected/default ConceptWeave main is f4f440dd58c77d7cd90dff8a1eb2eeb9a9940425.
  • No issued predecessor digest domain is rewritten.

PostgreSQL 18 ordinary EXCLUDE implementation-function security context

Review 5230117505 found a P1 on exact predecessor cc64bab7...: the chain already bound exact conexclop -> pg_operator, raw binary oprkind, self-commutator, exact oprcode -> pg_proc, Boolean oprresult/prorettype, scalar proretset=false, raw proisstrict, raw provolatile, raw proparallel, and raw prokind='f', but it did not preserve raw pg_proc.prosecdef.

PostgreSQL 18.6 stores prosecdef independently. SECURITY INVOKER executes with the caller's privileges and is the default; SECURITY DEFINER executes with the function owner's privileges. ALTER FUNCTION ... SECURITY INVOKER|SECURITY DEFINER can change that execution-privilege boundary without changing the input-argument signature that identifies the function. An otherwise identical oprcode -> pg_proc binding can therefore have materially different security semantics while collapsing to the same governed identity if prosecdef is omitted.

This successor is observational, not prescriptive: no PostgreSQL ordinary-EXCLUDE rule requiring prosecdef=false is asserted. Both raw Boolean states are representable and must remain distinguishable.

Ordinary-forward repair:

  • finding review 5230117505;
  • structural source/compile contract 4127151673d52b647e40e42f3cf310d958ee409e; the public security-context types did not yet exist, so no executed compiler failure is claimed;
  • production successor 27b9215a35151926c8bf90a21f9e941535d358a4;
  • public composition 926019b08c8a96ff0937e9b2876d20516d9f1766;
  • PostgreSQL/APA decision record cc3a71e4d11bb09c41fb0dd5c8852aeff3f050cb;
  • lossless predecessor-baseline archive d3c0f80bfeffae4b38a8dd96184fc15e9b715ff4;
  • product/technical-gap currentization 2d14f0723978a86d3b94305ea0ea783329a01a59;
  • lossless predecessor-CHANGELOG archive 2513cf6862dd0877f51c6a71ef887c2b4b355139;
  • CHANGELOG/current exact head e2284292ed83c0c957bbcdc5745ca15585f56438.

IndexExclusionConstraintOperatorProcedureSecurityDefinerSnapshot derives its exact coordinate/key inventory from IndexExclusionConstraintOperatorProcedureKindSnapshot. Every governed position requires one independently observed raw prosecdef fact bound to the same stable operator and exact oprcode function. Missing/duplicate evidence, operator/function binding drift, zero positions, and unknown receipt coordinates fail closed. false and true are both admitted but are included in the new domain-separated digest, so invoker and definer execution modes cannot collapse.

The focused contract covers invoker provenance, invoker/definer digest separation, operator/function binding drift, missing evidence, duplicate coordinates, zero position, unknown receipt coordinates, and public composition. The test was introduced before production to create a structural source/compile RED; no synthetic runtime RED is claimed.

Retained authority

All prior ordinary-EXCLUDE repairs remain in force: exact backing-index lifecycle and same-v3-source-generation binding; independently observed backing-index pg_class.relnamespace; constraint namespace; name/role/immediacy; access-method exclusion capability; catalog-family shape; exact conkey/conexclop; raw oprkind='b'; independently resolved oprcom; exact oprcode -> pg_proc; independent oprresult/prorettype = pg_catalog.bool; raw proretset=false; raw proisstrict; raw provolatile; raw proparallel; raw prokind='f'; timing/enforcement/validation/raw connoinherit/period; and retained Source Observation/relation-partition contracts. Temporal PRIMARY KEY/UNIQUE WITHOUT OVERLAPS and FOREIGN KEY PERIOD remain with their owner families.

Acceptance boundary

No executed Rust RED/GREEN or hosted Product acceptance is claimed for e2284292.... The current execution host has no Rust toolchain, so repository-pinned Rust 1.98 fmt, strict workspace/all-target Clippy, focused/retained tests, workspace/doc tests, release build, rustdoc, and owned coverage have not been executed here. Exact-head hosted workflow/status inventory must also be re-read and reach terminal acceptance on one unchanged head. Any head movement resets acceptance.

The PostgreSQL 18 bounded live differential must resolve every exact conexclop OID to one pg_operator row and independently read oprkind, oprcom, oprresult, and oprcode; follow oprcode to the exact pg_proc row; independently read prokind, prosecdef, prorettype, proretset, proisstrict, provolatile, and proparallel; and preserve retained operator-family/strategy plus backing-index namespace/lifecycle/access-method/catalog controls in the same v3 source-content generation. prosecdef must come from the exact joined pg_proc row and may not be inferred from owner, function name, language, configuration, result/cardinality, strictness, volatility, parallel safety, or routine kind.

Canonical prerequisite

Canonical .github#2106 is currently exact bff097e9702a094c483a31b52327bd13d7b1c9f1 on protected .github/main@6513c7a9651af887c4cb96efc2d23080da3d4b07, OPEN / Draft / mergeable. Its owner-local RED remains the two exact owner-qualified repository-identity corrections in docs/product-technical-gap-baseline.md; exact-current security/quality workflows are still nonterminal. ConceptWeave does not compete with that owner lane or manufacture wake/no-op evidence.

Product bootstrap #35 must still reacquire current-generation acceptance after the canonical workflow prerequisite; Ready/mergeable alone is not landing authority. #45 and #6 remain dependent parents and must not partially adopt this child.

Required next order

Canonical .github#2106 owner source repair/terminal settlement -> fresh compatible #35 acceptance/normal landing -> one unchanged #46 native+hosted terminal GREEN -> PostgreSQL 18 bounded live differential including independent raw pg_proc.prosecdef plus all retained EXCLUDE catalog integrity -> fresh terminal GREEN -> continue bounded material-catalog review -> complete ordinary/non-force #46 adoption into #45 -> fresh #45 acceptance -> #6 propagation.

No force-push, destructive rebase, self-approval, review dismissal, administrator bypass, synthetic status, copied central workflow, manual/no-op rerun, gate weakening, partial parent adoption, predecessor-evidence transfer, or premature publication/release is authorized.

@coderabbitai

coderabbitai Bot commented Sep 11, 2026

Copy link
Copy Markdown

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

  • Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@seonghobae seonghobae left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 representation invariant: IndexObservation currently stores the key/INCLUDE role twice — by outer collection (key_attributes vs include_attributes) and by IndexAttributeKind — but IndexObservation::new does not require those two representations to agree. The current test even constructs an Include attribute inside key_attributes and a Key attribute inside include_attributes and accepts it as a valid fixture. PostgreSQL 18 pg_index defines indnkeyatts as the key-column count and indkey as indnatts entries with key columns before non-key included columns; therefore one observation must have one unambiguous ordinal boundary. A contradictory value object can otherwise produce a stable v3 digest/receipt for evidence that cannot faithfully correspond to the source catalog. Add a behavioral RED that rejects role/collection disagreement (and preferably a non-contiguous key→INCLUDE ordinal layout), then make construction fail closed before digest/snapshot creation. Preserve expression key attributes; do not infer from search_path. This is independent of the still-open #45 relation-kind coordinate finding.

@seonghobae seonghobae left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Exact-current follow-up: review 5174516003 now has a committed behavioral RED contract at this head. index_layout_contract.rs requires IndexObservation::new to reject (a) collection/IndexAttributeKind disagreement and (b) non-contiguous key→INCLUDE ordinals, while accepting a contiguous key then INCLUDE layout. The production constructor in representation_v3.rs still only sorts the two vectors and therefore does not satisfy either RED. No hosted PR workflow has materialized for this head; combined status exposes CodeRabbit only, so this is intentionally RED/non-GREEN. Next causal repair is the constructor invariant plus correction of the inherited role_swapped test that currently treats contradictory evidence as a valid digest variant. Keep expression key support and keep #45's relation-kind coordinate finding separate.

@seonghobae seonghobae left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Second exact-current representation finding: PostgreSQL 18 does not support expressions as INCLUDE/non-key payload columns, but the current VO permits IndexAttributeObservation::expression(..., IndexAttributeKind::Include, ...) and IndexObservation::new accepts it in include_attributes. Exact head a1eb5b242d7ecb4d2e4af51c7e69c286e204c34f adds that behavioral RED alongside the role/ordinal REDs. Repair the same index_attribute_layout boundary so INCLUDE entries are contiguous non-key column attributes only, while expressions remain valid key attributes (zero indkey source positions represented structurally, not by permitting expression INCLUDE). Do not claim GREEN until the production constructor and inherited contradictory digest fixture are repaired and fresh Rust 1.98/hosted evidence exists.

@seonghobae seonghobae left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 docs-to-code authority finding: docs/product-technical-gap-baseline.md on this exact branch still snapshots 2026-09-10 and names obsolete active coordinates (#35 22709ec..., Source Observation #6 18d882..., central .github/main f578d8d...). Live authority is #35 9bb82f..., #6 287165d..., #45 6b2a8f..., this #46 a1eb5b24..., and central protected main cb0872c9.... The baseline also still describes first-class index representation as a future P0 rather than the current #46 implemented-but-RED state. Refresh the baseline in the same causal repair lineage after the index invariant fix; do not call this PR code-current or representation-complete while the canonical baseline contradicts live source/stack state.

@seonghobae seonghobae left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 — current IndexObservation::validate_attribute_layout correctly repairs role/collection disagreement, non-contiguous ordinals, and expression INCLUDE, but it still admits key_attributes.is_empty(). That allows both an entirely empty index and an INCLUDE-only observation to enter v3 snapshot identity. PostgreSQL 18 CREATE INDEX requires at least one key index_elem in the parenthesized key list; INCLUDE is optional payload after that key list, not a substitute for it. See https://www.postgresql.org/docs/18/sql-createindex.html. Add a behavioral RED for empty and INCLUDE-only layouts, then minimally fail closed at the same index_attribute_layout constructor boundary when there is no key attribute. Preserve valid expression keys and current key→INCLUDE ordering semantics. Exact-head native/Product acceptance must be regenerated after the repair.

@seonghobae seonghobae left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Exact-current follow-up: the PostgreSQL key-cardinality finding is source-addressed by 808faa922c466a586ea3bc8bf9f928f049ec3b95 (behavioral RED contract) -> 6dd59aef9a69e08aae6e7ede54ed1e204a634d9f (minimal constructor/rustdoc repair). 033ae72b95e3ef490c47ec84c191b83632601b10 then currentizes docs/product-technical-gap-baseline.md to the live #35/#6/#45/#46 and central .github/main authority without changing production behavior. Static inspection now has one coherent IndexObservation::new admission invariant: nonempty key set, role/collection agreement, contiguous one-based key→INCLUDE positions, expression keys permitted, INCLUDE columns only. This is source/docs GREEN only. Do not transfer historical Rust/Product evidence: exact 033ae72... still requires repository-pinned Rust 1.98 fmt, strict all-target Clippy, workspace tests, rustdoc/doc tests, release/coverage and applicable hosted Product/security/review checks before Ready/merge/adoption.

@seonghobae seonghobae left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 semantic-identity finding on exact b2806d0b2d02d1a635d5a4850a88082c99c38fda.

The v3 index VO is still not lossless for PostgreSQL key semantics. IndexAttributeObservation currently preserves only position, Key/INCLUDE role, and column-vs-expression source. PostgreSQL 18 carries additional material per-key facts in pg_index: indcollation (key collation), indclass (operator class), and indoption (access-method-specific per-key flag bits). CREATE INDEX also exposes per-key COLLATE, operator class/parameters, ASC|DESC, and NULLS FIRST|LAST; these alter index behavior/use and are not equivalent metadata.

Because the digest currently encodes only the present attribute fields, two otherwise-identical v3 observations that differ solely in key collation/opclass/order/null placement can collapse to the same governed identity unless the optional reconstructed index_definition happens to be populated. A lossy optional text blob cannot substitute for first-class facts when this branch claims representation-complete index evidence.

Required RED before repair: construct two valid index observations over the same key/source that differ in one structured key semantic and prove distinct v3 digests; also prove missing/unresolvable qualified collation/opclass evidence fails closed where applicable. Minimal repair should extend key attributes (not INCLUDE payloads) with explicit canonical evidence for collation, opclass(+ parameters if captured), and access-method option semantics, preserving exact source coordinates without OID-as-identity or search_path inference. Keep server-reconstructed pg_get_indexdef as provenance text, not the sole semantic identity carrier.

Primary authority: PostgreSQL 18 pg_index (indcollation, indclass, indoption) and CREATE INDEX syntax/semantics. Do not attach transport or call the representation complete until this is RED→GREEN and exact-head accepted.

@seonghobae seonghobae left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Exact-current follow-up on 189bfdc642b5ad14230bb32c6002db44d2865e4e: the baseline is now code-current for review 5175066205, but this is documentation/authority repair only. The per-key semantic-identity P1 is still open in production source: current IndexAttributeObservation has no first-class carrier for PostgreSQL 18 indcollation, indclass, or indoption, so do not call this head representation-complete or GREEN. Next source action remains RED first, then minimal structured VO/digest repair, then exact-head Rust/Product acceptance. Keep Draft; no predecessor execution evidence transfers across this docs head movement.

@seonghobae seonghobae left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 refinement on exact 189bfdc642b5ad14230bb32c6002db44d2865e4e: the open per-key semantic gap is not only missing fields. PostgreSQL 18 defines indcollation, indclass, and indoption as arrays of exactly indnkeyatts entries, so they belong only to key positions; INCLUDE positions explicitly do not participate in index semantics. indoption is also interpreted by the index access method. The current v3 model keeps access_method optional and with_access_method accepts arbitrary text, so adding raw option bits without binding them to a nonblank observed access method would still leave governed identity semantically ambiguous.

RED/repair must therefore prove all of the following together: (1) one semantic record exists for every key position and for no INCLUDE position; (2) changing qualified collation, qualified operator class, or exact access-method option bits changes structured v3 identity even when pg_get_indexdef is absent; (3) blank/unqualified operator-class or collation coordinates fail closed; (4) a relation-attached index cannot enter a snapshot without a nonblank observed access method and complete per-key semantic evidence. Do not decode indoption into B-tree-specific ASC/DESC/NULLS flags in the generic domain layer; preserve the exact per-key bit pattern under its access-method binding and let a PostgreSQL adapter or access-method-specific layer interpret it. This keeps OIDs/search_path out of identity and preserves INCLUDE as payload-only.

@seonghobae seonghobae left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Exact-current review on f24821ed91d9069fab3fbb073a5b7b1bb0a7d1e1: the source-level RED introduced at ebac140d... remains intentionally unsatisfied. This successor only currentizes the product/technical baseline; it does not add QualifiedOperatorClassName, IndexKeySemantics, with_key_semantics, relation/snapshot admission completeness, or v3 digest framing for per-key semantics. Fresh current-head evidence is still 0 pull-request workflow runs and CodeRabbit success only, so there is no Rust/Product GREEN to transfer. Keep Draft. Next causal source delta is the minimal production VO/admission/digest repair against the RED, followed by exact-head Rust 1.98 and applicable hosted acceptance.

@seonghobae seonghobae left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 authority drift on exact 173ae2f5a187341421ec5ed94f3fcc7d69c1de35: ordinary forward commit 173ae2f... implements the active per-key RED (QualifiedOperatorClassName, IndexKeySemantics, complete relation admission and digest framing), but both this PR body and docs/product-technical-gap-baseline.md still declare f24821ed.../ebac140d... as the current RED and say those production seams are not implemented. That is now false code authority and can cause a successor/restack writer to reimplement completed work or treat the exact current head as knowingly RED for the wrong reason. Adopt the intervening delta; do not revert it. Currentize the body/baseline to 173ae2f... as source-shaped repair pending fresh exact-head Rust/Product evidence. No predecessor GREEN transfers: current head has no pull-request workflow run and combined status exposes CodeRabbit only, so keep Draft and do not call native/Product GREEN.

@seonghobae seonghobae left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Exact-current follow-up on 503a3c2d273f47a7b73f2eb7284826c5382f5a0a: review 5175722509 is addressed on the authority plane. The ordinary forward production repair 173ae2f... is retained, the PR body now names it as implemented, and this head currentizes docs/product-technical-gap-baseline.md from REPRESENTATION_V3_RED_ACTIVE to source-repaired/pending exact-head GREEN. Static review therefore treats the stale-authority defect as repaired. This is not executable acceptance: fresh 503a3c2d... has zero pull-request workflow runs and combined status exposes CodeRabbit success only. Keep Draft and do not adopt into #45/#6 until repository-pinned Rust 1.98 plus applicable Product/security/dependency/review gates are terminal GREEN on one exact head.

@seonghobae seonghobae left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 semantic-identity/invariant finding on exact 4c3336aa6ed9afa1b33411c4e8b4290e415d18ea.

QualifiedOperatorClassName is currently only (schema_name, operator_class_name), while PostgreSQL operator-class identity also depends on the index access method. PostgreSQL 18 pg_opclass.opcmethod binds every class to pg_am, and CREATE OPERATOR CLASS explicitly permits two operator classes in the same schema to have the same name when they are for different index methods. The current coordinate can therefore collapse two distinct valid catalog objects into one governed identity. Separately, RelationObservation::with_indexes verifies only that the index has a nonblank access_method; it cannot prove that each key's operator class belongs to that same method, so an impossible/misbinding combination can enter snapshot identity and receipts.

Required RED before repair: (1) represent same-schema/same-name operator classes for two different access methods as distinct coordinates; (2) reject a relation-attached index when any per-key operator-class access-method binding differs from the index's observed access method, before snapshot digest/receipt creation. Minimal causal repair should make access method part of the operator-class coordinate (or an equivalent canonical owner object), include it in v3 digest framing, and enforce equality with the index access method during complete index admission. Do not infer this through search_path, OIDs, or pg_get_indexdef; those remain non-canonical join/provenance mechanisms. Keep #46 Draft and reset exact-head evidence after the repair.

@seonghobae seonghobae left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Finding verification/refinement on exact 16e8c06924aa8b2779bad5c2b47e51a61b40f073: the identity-collision portion of review 5176147253 does not survive closer inspection and should not drive a redundant model change. encode_index already frames the parent index access_method before every key's schema/name operator-class coordinate, so a btree/public/x_ops and hash/public/x_ops index do not collapse at the governed snapshot-digest level even though QualifiedOperatorClassName is method-relative rather than globally unique. The RED commit at this head will therefore be superseded rather than forcing a false repair.

A verified adjacent P1 remains: PostgreSQL 18 permits per-key operator-class parameters (opclass (opclass_parameter = value, ...)), and operator-class options are material behavior. Current IndexKeySemantics preserves collation, operator-class name and indoption, but has no first-class carrier for these per-key opclass options; index_definition remains optional provenance text. Two otherwise-identical valid indexes that differ only in opclass parameters can therefore still collapse when reconstructed text is absent. Primary authority: https://www.postgresql.org/docs/18/sql-createindex.html (syntax and note that an operator class with optional parameters may be specified per column) and pg_attribute.attoptions at https://www.postgresql.org/docs/18/catalog-pg-attribute.html. Replace the provisional RED with a behavioral RED proving opclass-option materiality/canonical ordering, then minimally add exact per-key option evidence and digest framing. Preserve exact option strings or a deterministically ordered structured key/value representation; do not infer provider-specific semantics in the generic domain layer.

@seonghobae seonghobae left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Exact-current source review for 3a838d7acac79a69b2616fa74d77e3b66f34e013.

The verified operator-class-parameter RED is now causally repaired in source: d87eb560... establishes material option-value identity/canonical ordering/duplicate-name fail-closed behavior; 1e880fb8... adds the VO, admission and digest framing; the public export/follow-up is present by d25a9d50...; 3a838d7a... makes the canonical baseline code-current. The earlier method-relative collision hypothesis from review 5176147253 is superseded by verification review 5176183141, because the parent index access method is already framed before the per-key operator-class coordinate. No false model expansion was retained.

This is source/docs GREEN only, not native/Product GREEN. The current runtime has no rustc/cargo; exact 3a838d7a... currently has no pull-request workflow run and combined commit status exposes only CodeRabbit success. Keep Draft. Do not adopt into #45/#6, add transport, publish semantics, or transfer predecessor execution evidence until this same exact head obtains pinned Rust 1.98 fmt/strict Clippy/workspace+doc tests/release/owned coverage and applicable protected Product/security/dependency/review terminal evidence.

@seonghobae seonghobae left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 catalog-invariant finding on exact 3a838d7acac79a69b2616fa74d77e3b66f34e013: IndexObservation::new accepts is_unique = false together with nulls_not_distinct = Some(true) and the v3 digest frames that contradictory state as governed evidence. PostgreSQL 18 pg_index.indnullsnotdistinct is only meaningful for unique indexes; indisprimary is likewise documented to imply uniqueness. A non-unique index that claims NULLS NOT DISTINCT = true cannot faithfully represent a PostgreSQL catalog row and must fail closed before relation/snapshot identity. Add a behavioral RED using the existing constructor (so this is a runtime invariant, not a compile-only placeholder), then minimally reject only the impossible !is_unique && Some(true) combination at IndexObservation::new; preserve Some(false) for a directly observed non-unique catalog row and preserve None as unobserved. This is independent of the broader still-unmodeled pg_index state vector (indisprimary, indisexclusion, indimmediate, indisclustered, indcheckxmin, indisreplident), which should be handled as a separate representation gap rather than smuggled into this minimal repair. Primary authority: PostgreSQL 18 pg_index, https://www.postgresql.org/docs/18/catalog-pg-index.html.

@seonghobae seonghobae left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 representation audit result on exact 68abcc67a4318d12d8cb7f129005dc45bbcc1570: the v3 index VO still drops six PostgreSQL 18 pg_index facts that are independent of the already-modeled uniqueness/null-uniqueness and ready/valid/live fields: indisprimary, indisexclusion, indimmediate, indisclustered, indcheckxmin, and indisreplident. PostgreSQL documents these as distinct catalog state: primary-key backing, exclusion-constraint backing, immediate uniqueness enforcement, last-CLUSTER choice, HOT/xmin safety gating, and replica-identity selection. Several are not recoverable from current v3 fields (TableConstraintObservation has no exclusion variant or backing-index link), so two materially different catalog snapshots can currently collapse to the same governed index identity when optional pg_get_indexdef text is absent. Add a behavioral RED for an explicit observed state vector: each flag change must alter v3 digest; absent state must remain distinguishable from an observed all-false vector; and indisprimary=true must fail closed when is_unique=false, matching the documented PostgreSQL invariant. Minimal repair should preserve exact booleans without inventing defaults or interpreting indcheckxmin; keep existing ready/valid/live fields stable. Handle pg_class.reloptions as a separate next audit slice rather than conflating physical storage options with this catalog-flag repair. Primary authority: PostgreSQL 18 pg_index, https://www.postgresql.org/docs/18/catalog-pg-index.html.

@seonghobae seonghobae left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 remaining representation finding on exact 5addab0e82e944a934f88b099c9c92980a1a7380: PostgreSQL 18 stores index access-method-specific storage configuration in the index relation's pg_class.reloptions as keyword=value strings. ALTER INDEX ... SET/RESET changes these index-method-specific parameters, and PostgreSQL documents that changes such as fillfactor can require REINDEX to take full effect. Current v3 has no structured carrier, so two indexes that differ only in explicit storage parameters can collapse to the same governed source identity when optional reconstructed definition text is absent. Add a behavioral RED that (1) makes one option value change alter the v3 digest, (2) canonicalizes option input order, (3) rejects duplicate exact option names, (4) keeps unobserved reloptions distinct from an explicitly observed empty set, and (5) rejects blank names while preserving exact values without decoding access-method semantics. Minimal repair should add an optional exact storage-option vector on IndexObservation and deterministic digest framing; interpretation remains in the access-method/adapter boundary. Primary authority: PostgreSQL 18 pg_class, ALTER INDEX, and REINDEX: https://www.postgresql.org/docs/18/catalog-pg-class.html, https://www.postgresql.org/docs/18/sql-alterindex.html, https://www.postgresql.org/docs/18/sql-reindex.html.

@seonghobae seonghobae left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 — v3는 이제 pg_class.reloptions를 first-class identity로 보존하지만 index pg_class.reltablespace는 아직 구조화된 관찰 사실로 보존하지 않습니다. PostgreSQL 18에서 reltablespace는 index가 저장되는 tablespace를 가리키며 0은 database default tablespace를 의미하고, CREATE INDEX ... TABLESPACE / ALTER INDEX ... SET TABLESPACE로 실제 변경 가능합니다. 현재 IndexObservation에는 tablespace 필드가 없으므로 pg_get_indexdef가 없거나 display/provenance text를 identity carrier로 쓰지 않는 정상 경로에서 동일 definition/storage-options라도 tablespace만 다른 두 index가 같은 v3 structured digest로 수렴할 수 있습니다.

다음 RED는 OID를 저장하지 않고 exact resolved tablespace name을 qualified source-independent coordinate로 보존해야 합니다. 최소 계약: (1) explicit nondefault tablespace A/B가 다르면 v3 digest가 달라짐, (2) observed database-default tablespace와 unobserved state를 구분, (3) blank tablespace name fail closed, (4) tablespace input은 index storage options와 독립적으로 digest-framed, (5) OID나 search_path를 governed identity로 사용하지 않음. PostgreSQL 18 pg_classCREATE INDEX가 authoritative 근거입니다.

@seonghobae seonghobae left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Exact-current follow-up: ordinary-forward pg_index/reloptions repairs were re-read and adopted. The tablespace finding is now executable RED at 1a47d6b16838006e5f7a75407e69464740f368b1, and 630040b71335b36095b1b10d262888fbb5bebd31 only currentizes the canonical baseline/PR authority. This head is therefore RED-active, not source GREEN and not native/Product GREEN. Keep Draft. Next causal code change is the minimal first-class IndexTablespace VO + optional IndexObservation admission/public export + presence/state/name digest framing; then the same exact successor must pass pinned Rust 1.98 and applicable hosted gates before #45/#6 adoption.

@seonghobae seonghobae left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Exact-current review: tablespace RED 1a47d6b... is now causally represented in source. IndexTablespace distinguishes unobserved state from observed database-default and explicit named assignment, rejects blank names, is admitted/exported, and is framed into v3 identity by presence + default-marker state + exact resolved name. The implementation keeps catalog OIDs outside governed identity and leaves adapter resolution for the transport ACL. Static source/RED inspection is consistent. This is not native/Product GREEN: exact eaecb512... has no PR workflow runs, only CodeRabbit success, and this runtime has no Rust toolchain. Keep Draft; next admissible transition is exact-head Rust 1.98 + hosted gate evidence, with causal repair if any real failure appears.

@seonghobae seonghobae left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 representation finding on exact eaecb512aeee37eb53f9e5602b5466ab3e2de40d: IndexObservation preserves pg_index semantics plus selected pg_class index state, but it does not preserve the index relation's own pg_class.relkind. PostgreSQL 18 distinguishes ordinary indexes (relkind = 'i') from partitioned indexes (relkind = 'I'), and partitioned indexes are virtual parents whose leaf indexes hold the actual data. Two otherwise-equal index observations can therefore collapse into one governed v3 identity even though their source object kind and operational semantics differ. Add an executable RED showing ordinary vs partitioned index kind changes snapshot_digest, then minimally admit a first-class index relation-kind value into IndexObservation and encode_index. Keep catalog OIDs and partition-membership topology out of this narrow repair; those are separate coordinates. Primary authority: PostgreSQL 18 pg_class and table-partitioning documentation.

@seonghobae seonghobae left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Superseding review after deeper PostgreSQL 18 source verification: review 5177806003 and RED d425bdfb00652321367dd1fef0c77930d2ade327 over-modeled index relkind as an independent identity dimension. In PostgreSQL 18 DefineIndex, partitioned is derived directly from the owning relation's relkind == RELKIND_PARTITIONED_TABLE; that flag is then passed as INDEX_CREATE_PARTITIONED, and index_create deterministically sets the index relation kind to RELKIND_PARTITIONED_INDEX or RELKIND_INDEX. Therefore the RED's same partitioned owning relation + Ordinary/Partitioned alternatives include an impossible catalog state. Because v3 already frames owning RelationKind, adding a second independently mutable IndexRelationKind would duplicate a derived invariant and permit contradictions. Remove that RED by ordinary-forward commit; do not add redundant index-kind identity. Carry the useful part into the future adapter contract instead: read index pg_class.relkind and fail closed unless it matches the owning relation-derived PostgreSQL invariant before crossing the ACL. Primary PostgreSQL 18 source: src/backend/commands/indexcmds.c (partitioned = rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE, then INDEX_CREATE_PARTITIONED) and src/backend/catalog/index.c (relkind = partitioned ? RELKIND_PARTITIONED_INDEX : RELKIND_INDEX).

@seonghobae seonghobae left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Exact-current follow-up on 9ed1339e7251f74e423b90cd32c9503c821d53b2: the invalid relation-kind RED is gone ordinary-forward, the canonical gap baseline and PR authority now preserve the corrected derived-invariant model, and retained Source Observation source repairs remain present. Treat this as source/docs repaired only. Fresh PR-triggered workflow runs on this exact head are still absent and combined status exposes CodeRabbit success only, so Rust 1.98 fmt/strict Clippy/workspace+doc tests/release/owned coverage and applicable Product/security/dependency/review acceptance remain unproven. Keep Draft; no parent adoption, transport, merge, publication, or release before unchanged-head acceptance.

@seonghobae seonghobae left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 PostgreSQL namespace invariant on exact 9ed1339e7251f74e423b90cd32c9503c821d53b2: v3 canonicalization rejects duplicate owning RelationObservation names and duplicate index names only inside one relation, but PostgreSQL pg_class enforces one unique (relname, relnamespace) namespace across tables, indexes, sequences, views, materialized views, etc. (pg_class_relname_nsp_index). The current model can therefore publish an impossible snapshot with the same index name under two relations in one schema, or with an index name colliding with another relation name in that schema. Add a behavioral RED that uses only existing APIs and requires snapshot construction to fail for both cases. Minimal repair belongs in snapshot canonicalization: validate the shared schema-local pg_class name namespace across relation names plus nested index names, while allowing the same name in different schemas. Do not change coordinate vocabulary or frozen v2.

@seonghobae seonghobae left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Exact-head checkpoint: review 5229099545 P1 is repaired ordinary-forward through a27c69915d1e76af29752ad07b89b3fcdc9af22e. New IndexExclusionConstraintOperatorProcedureVolatilitySnapshot preserves independently observed raw pg_proc.provolatile (i|s|v) per exact predecessor constraint/key/operator/oprcode procedure binding, domain-separates all three legal PostgreSQL states, and fails closed on missing/duplicate/binding-drift/unknown discriminator evidence. Doctoring, lossless predecessor-baseline archive, active gap baseline, CHANGELOG, public composition, and PR authority are current. No executed Rust 1.98 RED/GREEN is claimed: this exact head has zero repository-owned PR workflow runs and the current host lacks the pinned Rust toolchain. The bounded PostgreSQL 18 differential and native+hosted terminal acceptance therefore remain open. #35 is also correctly held: its old CodeQL scan itself produced zero Medium+ findings, while canonical .github dispatch/re-entry later failed on target-status publication/wake capability; that prerequisite belongs to .github#2106, not a local gate bypass.

@seonghobae seonghobae left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 — ordinary EXCLUDE procedure evidence still collapses distinct PostgreSQL pg_proc.proparallel states. Exact head already binds conexclop -> pg_operator.oprcode -> pg_proc and preserves prorettype, proretset, proisstrict, and provolatile, but the repository has no governed proparallel evidence. PostgreSQL 18.6 stores proparallel independently (s safe, r restricted to leader, u unsafe/serial); the planner cannot derive this property from volatility or signature, and mislabeling can cause errors or wrong answers in parallel queries. Preserve the raw catalog discriminator observationally for the exact implementation procedure; do not invent an EXCLUDE-specific PARALLEL SAFE validity rule. Add a new domain-separated successor over the volatility snapshot, exact operator/procedure binding, raw s|r|u validation, complete coordinate inventory, provenance receipt, focused edge contract, PostgreSQL/APA doctoring, and code-current gap/CHANGELOG. Exact-head Rust/hosted GREEN remains separate and must not be inferred from source-shaped repair.

@seonghobae seonghobae left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Exact-head checkpoint on 68ee4b2e852fb2f41750caa184f944d666bdf8a0: the raw pg_proc.proparallel source repair is present with public composition, focused source contract, doctoring, code-current gap baseline, and CHANGELOG. The focused contract/source proves raw s|r|u representability/rejection boundaries and the production unit test domain-separates the three states; full predecessor completeness/binding behavior, retained-suite compatibility, rustdoc/coverage, and PostgreSQL 18 live differential remain execution obligations. Repository-owned PR workflow inventory for this exact head is currently empty and CodeRabbit remains in-progress, so no native/hosted GREEN or release readiness is asserted. Preserve this exact head for acceptance unless a valid intervening repair requires ordinary-forward movement; any movement resets exact-head evidence.

@seonghobae seonghobae left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 — raw pg_proc.prokind is still missing from the ordinary-EXCLUDE operator-procedure evidence chain. The current head binds exact conexclop -> pg_operator.oprcode -> pg_proc and preserves result/cardinality/strictness/volatility/parallel-safety state, but QualifiedProcedureSignature alone does not prove that the resolved catalog row is a normal function. PostgreSQL 18 pg_proc.prokind independently distinguishes f function, p procedure, a aggregate, and w window function, while CREATE OPERATOR explicitly requires its implementation routine to be a function even though the legacy syntax also accepts the keyword PROCEDURE. A malformed/synthetic capture could therefore normalize a non-function routine into the same governed signature. Add an independently observed prokind='f' successor for every exact operator/procedure position, bound to the current parallel-safety predecessor, and fail closed on p/a/w or unknown/missing/duplicate/binding-drift evidence. Preserve existing digest domains; this should be a new successor domain.

@seonghobae seonghobae left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Exact-head checkpoint for cc64bab7d81f2adb691f627e7de852ace6946227: the pg_proc.prokind P1 is source-shaped repaired ordinary-forward from 68ee4b2e... with structural RED 7f9d964d..., production successor c9f4d8e7..., public composition 616ce311..., focused completeness/binding/provenance contract e7d10cdb..., PostgreSQL/APA doctoring 42a8cfa8..., lossless baseline/CHANGELOG archives, and code-current baseline/CHANGELOG. PostgreSQL 18 requires operator implementation routines to be normal functions; raw prokind='f' is now independently required and p/a/w fail closed. No native/hosted GREEN transfers: current exact-head PR workflow inventory is empty and combined status only says CodeRabbit skipped the Draft. Rust 1.98 native suite, hosted gates, and bounded PostgreSQL 18 live differential remain required on one unchanged head.

@seonghobae seonghobae left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 — ordinary EXCLUDE operator implementation identity still collapses security-execution semantics. The current exact head preserves raw pg_proc.prokind after proparallel, but it does not preserve raw pg_proc.prosecdef. PostgreSQL 18 stores prosecdef independently; SECURITY INVOKER executes with caller privileges while SECURITY DEFINER executes with the function owner's privileges, and the attribute can be changed independently with ALTER FUNCTION. Two otherwise-identical pg_operator.oprcode -> pg_proc bindings can therefore have materially different privilege boundaries yet currently converge on the same governed digest.

Required repair: add an observational successor over IndexExclusionConstraintOperatorProcedureKindSnapshot that requires one exact raw prosecdef fact per (constraint coordinate, key_position), rebinds the same operator and exact oprcode routine, preserves both false and true as representable source states, and includes the raw Boolean in a new domain-separated digest. Do not invent a false-only PostgreSQL admission rule. RED/GREEN coverage must include false/true digest separation, predecessor completeness, duplicate/missing coordinates, operator/routine binding drift, zero position, provenance/receipt location, and public composition. Live differential must read prosecdef from the exact joined pg_proc row rather than infer it from name, owner, language, volatility, parallel safety, kind, or configuration.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request priority: medium

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant