# v0.21.0 — Correctness, Safety, and Test Hardening

> **Full technical details:** [v0.21.0.md-full.md](v0.21.0.md-full.md)

**Status: ✅ Released** | **Scope: Large** (~6–8 weeks)

> Closes the last known data-correctness gap in join delta computation,
> enforces a zero-crash guarantee across the codebase, expands unit test
> coverage in previously untested modules, and introduces a shadow/canary
> mode for safely testing query changes before going live.

---

## What problem does this solve?

Systematic analysis of the codebase after v0.20.0 identified several
categories of risk: a remaining join correctness bug (EC-01 phantom rows
in multi-table LEFT/RIGHT JOINs), `.unwrap()` calls that could crash the
PostgreSQL backend, modules with low test coverage, and the risk of
disruption when modifying a production stream table's query.

---

## EC-01 JOIN Delta Phantom Row Fix

The EC-01 bug — phantom rows appearing in stream tables after specific
sequences of DELETE and INSERT on multi-table JOINs — was first identified
in v0.12.0 and partially addressed across several releases. v0.21.0
delivers the complete fix:

- The row identity hash for Part 1b of the join delta algorithm (the
  "right-side unchanged" portion) is now computed correctly, ensuring both
  halves of the join delta emit the same row identifier and cancel each other
  out properly
- Prior-cycle phantom rows are cleaned up by the refresh process
- TPC-H Q07 (which exercises this pattern) is validated to pass
  deterministically across 5,000 randomised property test iterations

*In plain terms:* if your stream table computes a LEFT JOIN and rows are
deleted and re-inserted on either side, the results are now always correct —
never showing rows that should not be there.

---

## Zero `.unwrap()` in Production Code

A `clippy::unwrap_used` lint rule was added that fails the build if any
`.unwrap()` call appears in non-test code. Every `.unwrap()` in the production
code path was converted to proper error handling that returns a descriptive
error to the caller instead of crashing the backend.

The unsafe code surface was reduced by 40% through the introduction of safe
wrapper functions around PostgreSQL C internals.

---

## Non-Deterministic Function Rejection

At stream table creation time, pg_trickle now rejects (or warns about)
queries that use non-deterministic functions like `now()`, `random()`, or
volatile user-defined functions without an explicit `non_deterministic => true`
acknowledgement. This prevents a whole class of subtle drift bugs before they
reach production.

---

## Test Coverage Campaign

Three large, previously under-tested modules received comprehensive unit test
coverage:

- **`api/helpers.rs`** (25+ new tests) — query validation, schema helpers,
  CDC orchestration utilities
- **`api/diagnostics.rs`** (15+ new tests) — `explain_st`, `health_summary`,
  `cache_stats` formatting
- **`dvm/parser/rewrites.rs`** (30+ new tests) — all seven SQL rewrite passes

A **parser fuzz target** was added that runs random SQL through the pg_trickle
query parser for an hour without panics — any panic would indicate a code
path that could crash the backend on unexpected input.

A **crash-recovery test** kills the background worker mid-refresh and
verifies that the database is left in a consistent state: no partially-applied
refreshes, WAL decoder resumed from the correct position.

---

## Shadow / Canary Mode for Safer Query Changes

When you need to change a production stream table's query, doing so directly
is risky — the new query might produce different results or refresh more
slowly.

`alter_stream_table(name, dry_run_shadow => true)` creates a shadow copy
of the stream table (`pgt_shadow_<name>`) that runs the new query on the
same schedule as the live table. Operators can compare the two versions with
`pgtrickle.canary_diff(name)` before committing the change. When satisfied,
`pgtrickle.canary_promote(name)` atomically swaps the shadow into production.

*In plain terms:* test your query change on real production data alongside the
live version, verify the results match your expectations, then flip the switch.

---

## New Operational Helpers

- `pgtrickle.pause_all()` / `resume_all()` — suspend all stream tables at
  once during maintenance
- `pgtrickle.refresh_if_stale(name, max_age)` — only trigger a refresh if
  the stream table is older than a specified age
- `pgtrickle.stream_table_definition(name)` — returns the full definition
  of a stream table for auditing

---

## Prometheus HTTP Endpoint

The background worker now serves Prometheus metrics directly over HTTP
(port configurable via `pg_trickle.metrics_port`), removing the need for
a separate Prometheus exporter process.

---

## Performance Tuning Cookbook

A new `docs/PERFORMANCE_COOKBOOK.md` document consolidates all performance
tuning advice — previously scattered across FAQ, TROUBLESHOOTING, and
SCALING — into a single reference: symptom → likely cause → configuration
to adjust → how to measure improvement.

---

## Scope

v0.21.0 is a comprehensive quality release. The EC-01 fix closes the last
known data-correctness gap. The zero-unwrap guarantee eliminates a class of
potential backend crashes. The test coverage campaign and fuzz target
significantly raise the floor on correctness confidence. Shadow/canary mode
makes production query changes safer.
