# v0.4.0 — Parallel Refresh and Cross-Source Consistency

> **Full technical details:** [v0.4.0.md-full.md](v0.4.0.md-full.md)

**Status: ✅ Released** | **Scope: Medium** (~4 weeks)

> Run multiple stream table refreshes simultaneously, capture changes more
> efficiently with statement-level triggers, and guarantee a consistent
> snapshot when a stream table reads from multiple source tables.

---

## What problem does this solve?

As deployments grew to dozens of stream tables, the serial refresh scheduler
became a bottleneck — each table waited for the previous one to finish.
Meanwhile, for high-insert-rate workloads, row-level triggers on every
individual row were adding measurable overhead. And for stream tables that
join multiple source tables, a refresh that reads Table A at one point in
time and Table B slightly later could produce a snapshot that never existed.

---

## Parallel Refresh Workers

The background scheduler gains the ability to refresh multiple independent
stream tables concurrently. Stream tables that do not depend on each other
(no parent-child relationship in the dependency graph) are dispatched to
separate worker processes and run in parallel.

*In plain terms:* if you have 20 independent stream tables and 4 worker
processes, the scheduler runs 4 refreshes simultaneously instead of
sequentially. Total refresh time drops proportionally.

The `pg_trickle.max_parallel_workers` configuration controls the degree of
parallelism, defaulting to serial mode for backward compatibility.

---

## Statement-Level CDC Triggers

Previously, pg_trickle installed **row-level** triggers that fire once for
every individual row inserted, updated, or deleted. For bulk operations
(`INSERT INTO ... SELECT ...` inserting thousands of rows), this means the
trigger fires thousands of times with significant overhead.

**Statement-level triggers** fire once per SQL statement regardless of how
many rows it affects. pg_trickle now uses transition tables (the PostgreSQL
`NEW TABLE` / `OLD TABLE` feature) to capture the complete set of changed
rows from a statement in a single trigger invocation.

*In plain terms:* bulk INSERT operations become much faster because the
trigger overhead is amortised across all rows in the statement, not charged
once per row.

---

## Cross-Source Snapshot Consistency

A stream table that joins `orders` with `customers` reads from two tables.
If `orders` and `customers` are refreshed at slightly different times, the
snapshot could combine new orders data with old customers data — a
consistency window that never existed in the real database.

v0.4.0 adds **cross-source snapshot consistency**: when a refresh reads from
multiple source tables, it takes a consistent snapshot of all sources at the
same database transaction boundary. The result always reflects a point in
time that actually existed.

---

## Code Coverage Integration

The CI pipeline now measures test code coverage and reports it alongside
each pull request. This surfaces gaps in test coverage early and provides a
baseline for future test campaigns.

---

## Scope

v0.4.0 delivers three independent improvements that each address a different
scaling concern: throughput (parallelism), overhead (statement-level triggers),
and correctness (cross-source consistency). Together they prepare pg_trickle
for larger, more demanding deployments.