# v0.87.15 - Versioned Row Identity V2 Contracts

> **Status:** Planned
> **Scope:** 7 person-weeks
> **User promise:** *"Every row identity is deterministic, exact, and portable across supported PostgreSQL environments."*
> **Blocked by:** [v0.87.14](v0.87.14.md)
> **Proposal:** [Exact, Versioned Row Identity V2](../proposals/PROPOSAL_VERSIONED_ROW_IDENTIFY_ENCODING.md)

## Theme

Freeze the semantic and wire contracts for the V2 row identity before changing
storage or DVM operators. The release replaces display-text hashing as an
identity definition with typed PostgreSQL datum encoding, explicit type and
domain registries, and independently reproducible golden vectors. It delivers
the encoder and validation foundation, but does not yet change existing stream
table storage or consume V2 state in production refresh paths. A checked-in
normative wire-format artifact is a release deliverable, not an implementation
note, and later code must be tested against that artifact.

## Scope allocation

| Work package | Person-weeks |
|---|---:|
| Normative wire format and version/domain registry | 1.50 |
| Typed scalar and structural encoder dispatch | 2.00 |
| Type, collation, key-eligibility, and size validation | 1.25 |
| Golden vectors, cross-platform checks, and documentation | 1.25 |
| Review, failure-path tests, and release integration | 1.00 |
| **Total** | **7.00** |

## Items

### ROWID-1: Freeze the V2 wire format

Write and check in the normative byte-format specification consumed by the
implementation and tests. Assign identity version `2`, probe version `1`, and
stable values for identity domains, field type tags, NULL/value tags,
fixed-width transforms, escaped variable-width payloads, nested framing, tuple
termination, numeric normalization, the XXH3-128 seed, and digest byte order.
The artifact must be linked from the proposal and this roadmap, and a
spec-to-code vector check must fail when the implementation diverges. Once
released, changes to these assignments require a new identity version and a
rebuild rather than silently changing existing bytes.

**Dependencies:** v0.87.14 correctness oracle; proposal sections 3-5 and 8.
**Schema change:** No.

### ROWID-2: Add one extension-wide type and domain registry

Define registry entries by concrete type, typmod, collation, equality operator
family, nested metadata, and PostgreSQL major version. Each entry must declare
equality-canonical status, default non-NULL B-tree ordering, volatility, maximum
encoded size, supported majors, and DDL invalidations. Register the initial
scalar families and publish an initial structural support matrix that names
which arrays, ranges, multiranges, JSONB, composites, and other structural
families ship in V2; every family not marked supported is rejected at
validation. Explicitly reject types, collations, operator classes, and
structural forms whose equality or serialization contract is not proven.

**Dependencies:** ROWID-1; PostgreSQL catalog and operator-family contracts.
**Schema change:** No.

### ROWID-3: Implement typed datum encoding

Add the typed `encode_row_id_v2(domain, record) RETURNS bytea` entry point and a
single dispatch path that deforms PostgreSQL datums from the record descriptor.
Use stable semantic tags rather than runtime OIDs, preserve field order and
NULLs, canonicalize signed zero, NaN, numeric scale, the complete 128-bit
PostgreSQL interval comparison value, network values, and enum labels, and
recurse only through registered structural encoders. Resolve dispatch once per
record typmod with `fn_extra` and write to a reusable scratch buffer; never fall
back to `TEXT`, output functions, or a 64-bit hash.

**Dependencies:** ROWID-1 and ROWID-2.
**Schema change:** No.

### ROWID-4: Validate identity schemas before state creation

Validate every potential identity field at stream-table creation and defining
query change. Check equality semantics, source-key eligibility, collation and
operator class policy, nested support, volatility, maximum encoded size, DDL
invalidation coverage, and the running PostgreSQL major. Source-key validation
must accept only a primary key, or a non-partial, non-expression, immediate
unique index whose columns are all `NOT NULL` or whose constraint is `NULLS NOT
DISTINCT`; it must reject nullable `NULLS DISTINCT`, partial, expression,
deferrable, and other non-immediate unique forms. Return contextual errors
naming the expression, resolved type, and unsupported property; treat
uncertainty as rejection rather than silently selecting a weaker identity.

**Dependencies:** ROWID-2 and ROWID-3.
**Schema change:** No.

### ROWID-5: Publish reproducible identity vectors

Create an independent vector corpus covering every supported scalar family,
NULLs, composite framing, all ordering transforms, type modifiers, domains,
enums, deterministic and non-deterministic collations, nested values, malformed
or over-limit inputs, rejected source-index forms, the complete 128-bit interval
comparison value, and every identity domain. Verify exact bytes, equality
agreement, prefix freedom, and error classification on every supported
PostgreSQL major and CPU endianness. Include a spec-to-code check so the
independent vectors cannot silently drift from the normative artifact.
Keep vectors independent from the encoder implementation so a shared bug cannot
make the oracle agree with itself.

**Dependencies:** ROWID-1 through ROWID-4; v0.87.14 schema oracle.
**Schema change:** No.

## Correctness

| ID | Title | Effort | Priority |
|---|---|---:|:---:|
| CORR-1 | Prove equality and injectivity contracts | M | P0 |
| CORR-2 | Enforce prefix-free tuple framing | S | P0 |
| CORR-3 | Verify canonical special values | M | P0 |

**CORR-1** proves in both directions that PostgreSQL equality for each accepted
identity type agrees with byte equality, and that unequal values cannot silently
share a complete identity. Property tests compare the registry contract with
SQL equality across NULL, typmod, domain, enum, and supported nested cases.

**CORR-2** verifies field count, domain, type tags, variable-value escaping,
nested framing, and tuple termination. The corpus includes adversarial tuples
such as `(ab, c)` versus `(a, bc)` and complete identities where one payload
would otherwise be a prefix of another.

**CORR-3** locks down signed zero, NaN, infinities, numeric scale, interval
comparison normalization, `cidr` host bits, and enum label encoding. Any
semantic ambiguity produces a rejected input or a new explicit wire rule.

## Stability

| ID | Title | Effort | Priority |
|---|---|---:|:---:|
| STAB-1 | Bound encoder resource usage | M | P0 |
| STAB-2 | Classify unsupported inputs explicitly | S | P0 |
| STAB-3 | Preserve backend error safety | S | P1 |

**STAB-1** publishes concrete maximum nesting-depth and total encoded-size
limits, alongside typmod-aware bounds. Resource exhaustion raises a contextual
PostgreSQL error without switching algorithms, corrupting scratch state, or
leaving a partially returned varlena value visible.

**STAB-2** gives unsupported types, non-deterministic collations, unproven
operator classes, unsupported PostgreSQL majors, and invalid identity schemas
distinct errors with actionable remediation. No path uses an unchecked fallback
or accepts a type merely because PostgreSQL exposes an output function.

**STAB-3** exercises encoder errors, catalog lookup errors, malformed records,
and repeated calls in one backend. PostgreSQL error capture and `fn_extra`
rebuilds must leave the backend reusable and must not retain stale type metadata.

## Performance

| ID | Title | Effort | Priority |
|---|---|---:|:---:|
| PERF-1 | Make fixed-width encoding allocation-light | M | P0 |
| PERF-2 | Cache dispatch by record typmod | S | P0 |
| PERF-3 | Benchmark encoder hot paths | M | P1 |

**PERF-1** keeps fixed-width integer, UUID, date/time, and short composite paths
branch-light, writes directly into one scratch buffer, and copies only the
completed result into PostgreSQL-owned memory. The implementation records
allocation and throughput baselines for later storage comparisons.

**PERF-2** caches tuple descriptors and resolved field encoders per call site and
anonymous record typmod. It validates the current typmod and rebuilds the cache
when PostgreSQL invalidates the descriptor, without catalog queries or fmgr
lookups per row on the unchanged path.

**PERF-3** measures one integer, UUID, composite integers, short and long text,
keyless wide rows, arrays or JSONB where supported, and enum-heavy workloads.
The benchmark records throughput, allocations, error rates, and the cost of
syscache enum-label resolution.

## Scalability

| ID | Title | Effort | Priority |
|---|---|---:|:---:|
| SCAL-1 | Keep registry resolution bounded | S | P0 |
| SCAL-2 | Bound structural recursion | S | P0 |
| SCAL-3 | Exercise many identity call sites | M | P1 |

**SCAL-1** ensures registry lookup is keyed by the complete semantic descriptor
and does not grow with the number of rows processed. Invalidation and cache
replacement remain local to the affected type and call site.

**SCAL-2** applies one documented depth and size policy to arrays, ranges,
multiranges, JSONB, and composites. A deeply nested value fails predictably
rather than consuming unbounded backend memory.

**SCAL-3** runs the same encoder through source keys, keyless rows, group keys,
join keys, set-operation keys, window keys, and synthetic identities. The test
matrix checks that domains stay disjoint without adding per-domain algorithms.

## Ease of Use

| ID | Title | Effort | Priority |
|---|---|---:|:---:|
| UX-1 | Document identity support and rejection | S | P0 |
| UX-2 | Explain identity validation failures | S | P0 |
| UX-3 | State the opaque identity contract | XS | P1 |

**UX-1** adds a support matrix for scalar and structural types, collations,
operator classes, PostgreSQL majors, size bounds, volatility, and DDL
invalidations. It clearly distinguishes exact identity support from native
B-tree ordering support.

**UX-2** makes validation errors identify the stream table, expression, type,
collation, or operator class and explain the supported alternative. Preflight
and create-time diagnostics use the same classifications.

**UX-3** documents that `__pgt_row_id` is an opaque implementation value, not a
business identifier or user-generated key. Diagnostics use a short fingerprint
rather than exposing complete reversible identity bytes by default.

## Test Coverage

| ID | Title | Effort | Priority |
|---|---|---:|:---:|
| TEST-1 | Maintain independent golden vectors | M | P0 |
| TEST-2 | Property-test equality agreement | M | P0 |
| TEST-3 | Test catalog and cache invalidation | S | P1 |

**TEST-1** runs the vector corpus in unit and PostgreSQL-backed tiers and checks
byte-for-byte output, ordering, prefix freedom, and error outcomes. Vectors are
reviewable fixtures, not snapshots regenerated by the implementation under test.

**TEST-2** generates supported values and compares encoded-byte equality with
PostgreSQL equality and unequal-value separation. It includes composite and
nested values plus the explicit NULL policy for identity fields.

**TEST-3** warms encoder caches, changes record metadata, renames enum labels,
and retries from a second session. Tests verify syscache visibility and DDL
invalidation without introducing a custom enum generation counter. The same
fixture covers non-deterministic collation rejection and all accepted and
rejected source-index forms.

## Conflicts & Risks

- The wire format is an immutable compatibility contract. Any unresolved tag,
  byte-order, numeric, interval, enum, or structural rule blocks ROWID-3 and
  must be resolved before the next release.
- Stable enum-label encoding means `encode_row_id_v2` is `STABLE`, not
  `IMMUTABLE`; the later index probe function must remain separate and truly
  immutable.
- The release may implement encoder and validation code while leaving V1
  storage untouched. No test may imply that V1 and V2 state can be consumed in
  one graph.
- Structural families not explicitly proven remain rejected. Adding support
  later uses a new type tag and does not rewrite existing V2 identities.
- Encoder code is SQL-reachable and must contain no `unwrap()` or `panic!()`;
  unsafe PostgreSQL access requires precise `// SAFETY:` comments.

## Exit criteria

- [ ] Normative V2 wire-format and probe-version specifications are reviewed.
- [ ] A checked-in normative wire-format artifact is linked from the proposal,
  and independent vectors fail on any spec-to-code drift.
- [ ] Type and identity-domain registries reject unsupported contracts
      explicitly and list all accepted PostgreSQL majors.
- [ ] The initial structural support matrix explicitly names shipped and
  rejected arrays, ranges, multiranges, JSONB, composites, and other
  structural families.
- [ ] Typed encoding produces exact, prefix-free, deterministic bytes without
      `TEXT` conversion or a final hash.
- [ ] Numeric normalization and the complete 128-bit interval comparison value
  are covered by vectors and acceptance tests.
- [ ] Source-key validation covers primary keys, `NULLS NOT DISTINCT`, nullable
  `NULLS DISTINCT`, partial, expression, deferrable, and non-immediate forms.
- [ ] Resource limits, cache invalidation, and backend reuse tests pass.
- [ ] Golden vectors pass independently on every supported PostgreSQL major and
      CPU endianness.
- [ ] Unit, PostgreSQL, property, documentation, and full repository CI gates
      pass.