# v0.77.0 Full Implementation Status

## Theme: Correctness Stop-the-Line & DVM Proof Infrastructure

**Release arc:** Assessment-15 Hardening Arc v0.77.0–v0.80.0
**Status:** ✅ Released

## Implementation Checklist

| Item | Description | Status |
|------|-------------|--------|
| C-1 | TRUNCATE LSN fix (`pg_current_wal_lsn` → `pg_current_wal_insert_lsn`) | ✅ Done |
| T-4 | TRUNCATE marker E2E regression tests (2 tests) | ✅ Done |
| C-3/DVM-1 | q12 CASE/IN-list forced FULL fallback + `classify_case_in_list_aggregate_drift()` | ✅ Done |
| DVM-2/P-1 | q20 correlated subquery forced FULL fallback + `classify_correlated_aggregate_subquery_in_where()` | ✅ Done |
| D-1 | Multi-consumer cleanup advisory lock (`pg_try_advisory_xact_lock`) + overlap E2E tests | ✅ Done |
| D-2 | IMMEDIATE mode SAVEPOINT/rollback E2E tests (3 tests) | ✅ Done |
| C-2 | Source-placeholder coverage assertion in `resolve_lsn_placeholders` | ✅ Done |
| DVM-3 | `pg_trickle.validate_delta_invariants` GUC + validation in `execute_differential_refresh` | ✅ Done |
| T-1 | DVM algebra property generators (12 unit tests for C-3/DVM-1 and DVM-2/P-1 detection) | ✅ Done |
| C-4 | Semgrep CI rule (formalised existing `rust.panic-in-sql-path` as C-4 hard gate) | ✅ Done |

## Files Changed

### Core Implementation

| File | Change |
|------|--------|
| `src/cdc/mod.rs` | C-1: `pg_current_wal_lsn()` → `pg_current_wal_insert_lsn()` in TRUNCATE trigger |
| `src/refresh/mod.rs` | C-3/DVM-1: `classify_case_in_list_aggregate_drift()` · DVM-2/P-1: `classify_correlated_aggregate_subquery_in_where()` |
| `src/refresh/merge/mod.rs` | C-3/DVM-1 and DVM-2/P-1 fallback checks in `execute_differential_refresh` · DVM-3 validation call |
| `src/refresh/codegen.rs` | C-2: placeholder coverage assertion in `resolve_lsn_placeholders` · D-1: advisory lock in `drain_pending_cleanups` |
| `src/config.rs` | DVM-3: `PGS_VALIDATE_DELTA_INVARIANTS` GUC declaration + `pg_trickle_validate_delta_invariants()` accessor + `register_gucs()` call |
| `.semgrep/pg_trickle.yml` | C-4: annotation of existing `rust.panic-in-sql-path` as Assessment-15 hard gate |

### Tests

| File | Change |
|------|--------|
| `tests/e2e_cdc_tests.rs` | T-4: `test_t4_truncate_lsn_insert_ordering_regression` + `test_t4b_truncate_then_insert_separate_transactions` |
| `tests/e2e_v077_tests.rs` | D-1: `test_d1_shared_source_two_consumers_no_row_loss` + `test_d1_multi_cycle_cleanup_preserves_slow_consumer_rows` · D-2: `test_d2_savepoint_rollback_no_ghost_rows` + `test_d2_partial_savepoint_rollback_committed_rows_visible` + `test_d2_nested_savepoints_full_rollback_no_ghost_rows` |
| `src/refresh/tests.rs` | T-1: 12 property unit tests for DVM detection functions |
| `tests/e2e_tpch_tests.rs` | q20 removed from `LARGE_SCALE_DIFFERENTIAL_SKIP` · q12 re-added to sustained churn |
| `tests/property_tests.rs` | Added T-1 documentation comment |

### Supporting Files

| File | Change |
|------|--------|
| `Cargo.toml` | Version bump: `0.76.0` → `0.77.0` |
| `META.json` | Version bump: `0.76.0` → `0.77.0` |
| `sql/pg_trickle--0.76.0--0.77.0.sql` | No-op upgrade migration (no schema changes) |
| `CHANGELOG.md` | v0.77.0 entry |
| `ROADMAP.md` | v0.77.0 status updated to ✅ Released |
| `roadmap/v0.77.0.md` | Plain-language companion |

## Design Notes

### C-3/DVM-1 and DVM-2/P-1: Detection Approach

The detection is text-based (no full AST parse) and is intentionally conservative:
- False positives (benign queries incorrectly flagged) are safe because FULL refresh is always correct.
- False negatives (drift patterns not detected) would cause incorrect DIFFERENTIAL results.

The text heuristics target the canonical patterns:
- `SUM(CASE` or `COUNT(CASE` + `IN ('` → CASE/IN-list drift
- `> (SELECT` or `< (SELECT` etc. + aggregate keywords → correlated subquery

Root-cause fixes (correct delta rules) are planned for v0.78.0.

### D-1: Advisory Lock Analysis

The `pg_try_advisory_xact_lock(bigint)` call is non-blocking (returns `false`
on contention rather than blocking). The lock is held for the duration of the
transaction that owns the cleanup window. If the lock is not acquired, the
cleanup for that source OID is skipped — it will be retried in the next
scheduler tick. This is safe because:

1. Skipping cleanup only means rows stay in the change buffer one tick longer.
2. The consuming stream tables' frontiers are always read-only during their own
   differential refresh (not during cleanup), so there is no read-write race.
3. MVCC ensures that rows visible to a reader are never deleted mid-read.

### DVM-3: Validation GUC

The `validate_delta_invariants` GUC runs after a successful differential MERGE.
It executes `SELECT count(*) FROM (defining_query)` to compare against the
stream table row count. A count mismatch emits a WARNING but does not fail
the refresh — the discrepancy is logged for investigation.

This is intentionally a lightweight check (row count only, not multiset
equality). Full multiset equality checking is expensive; the property-based
tests in `tests/e2e_property_tests.rs` handle that.