# v0.17.0 — Cost-Based Strategy, Incremental DAG Rebuild, and pg_ivm Migration

> **Full technical details:** [v0.17.0.md-full.md](v0.17.0.md-full.md)

**Status: ✅ Released** | **Scope: Large** (~5–6 weeks)

> A fully data-driven refresh strategy decision based on measured performance
> history, incremental updates to the dependency graph (no full recompute on
> every schema change), API modularisation for maintainability, a migration
> guide from pg_ivm, and a Docker Compose playground for local experimentation.

---

## What problem does this solve?

The AUTO mode cost model relied on fixed thresholds and single-sample
measurements. Real-world performance varies, and a more data-driven approach
— based on actual measured history — makes better decisions. The dependency
graph was fully rebuilt from scratch whenever any stream table was added,
modified, or dropped, which was slow for large deployments. And users coming
from pg_ivm (another PostgreSQL IVM extension) needed a clear migration path.

---

## Cost-Based Refresh Strategy

The AUTO mode decision engine was rebuilt on a proper cost model:

- **Historical measurements** from `pgt_refresh_history` are used to compute
  running statistics (mean, standard deviation) for both DIFFERENTIAL and
  FULL refresh times per stream table
- **Regression-based prediction** estimates whether DIFFERENTIAL or FULL will
  be faster for the current change batch size
- **Confidence intervals** prevent switching based on noise — the strategy
  only changes when the evidence is clear

The cost model is visible through `pgtrickle.df_threshold_advice` — a
diagnostic view that shows the current recommendation, confidence level, and
the data behind it.

*In plain terms:* pg_trickle now learns from your actual workload to pick
the fastest refresh strategy, rather than applying a fixed formula.

---

## Incremental DAG Rebuild

The dependency graph tracks which stream tables depend on which source tables
and on each other. Previously, adding, modifying, or dropping any stream
table triggered a full rebuild of the entire graph.

Incremental DAG rebuild updates only the affected portion of the graph on
each DDL operation. For a deployment with 200 stream tables, adding one new
table no longer requires re-analysing all 200 relationships.

---

## api.rs Modularisation

The main API module grew to over 10,000 lines of code as features accumulated.
v0.17.0 splits it into focused sub-modules:

- `api/create.rs` — stream table creation
- `api/alter.rs` — stream table modification
- `api/diagnostics.rs` — diagnostic and inspection functions
- `api/publication.rs` — downstream publication management
- `api/helpers.rs` — shared utilities

This is an internal change with no effect on the public SQL API, but makes
future development significantly safer and faster.

---

## pg_ivm Migration Guide

**pg_ivm** is another PostgreSQL incremental view maintenance extension.
Teams wanting to migrate from pg_ivm to pg_trickle previously had to work
out the mapping manually.

The new migration guide covers:

- Feature equivalences and differences
- SQL syntax translation for common patterns
- A migration checklist
- Known behavioural differences to watch for

---

## Docker Compose Playground

A self-contained Docker Compose environment in `playground/` lets anyone
experiment with pg_trickle locally in minutes:

- Pre-configured PostgreSQL with pg_trickle installed
- A seed dataset with source tables and example stream tables
- The TUI dashboard pre-connected

*In plain terms:* `docker compose up` gives you a working pg_trickle
environment to explore, no installation required.

---

## Scope

v0.17.0 delivers a data-driven cost model (the most significant improvement
to AUTO mode since its introduction), incremental DAG rebuilds for large
deployments, and the pg_ivm migration guide that lowers the barrier for
teams switching from other IVM tools. The playground makes it easy for new
users to evaluate pg_trickle without any setup.