# v0.11.0 — Partitioned Stream Tables, Event-Driven Scheduling, and Safety

> **Full technical details:** [v0.11.0.md-full.md](v0.11.0.md-full.md)

**Status: ✅ Released** | **Scope: Large** (~6 weeks)

> Stream tables that are themselves partitioned by date or category,
> a scheduler that wakes up the moment changes arrive (34× latency
> reduction), differential refresh from stream table to stream table,
> and a circuit breaker that prevents a broken stream table from
> affecting its neighbours.

---

## What problem does this solve?

Power users needed their stream table outputs to be partitioned for query
performance — a "daily sales summary" stream table should be partitioned by
date so queries for a specific date range are fast. The polling scheduler was
adding unnecessary latency even when changes were available immediately.
Stream tables that depended on other stream tables were re-deriving everything
from source tables instead of propagating deltas through the chain. And a
stream table stuck in an error state could hold up the entire refresh pipeline.

---

## Partitioned Stream Tables

Just as source tables can be partitioned (v0.6.0), stream tables themselves
can now be partitioned. A stream table defined with a `PARTITION BY` clause
creates an output table with the same partitioning structure as a regular
PostgreSQL partitioned table.

New partitions are created automatically as new partition keys appear in the
data. The differential engine correctly routes delta rows to the right
partition.

*In plain terms:* a "monthly revenue by region" stream table can now be
physically partitioned by month, so queries for "last quarter" only scan
three partitions instead of all the data.

---

## Event-Driven Scheduler: 34× Latency Reduction

The previous scheduler woke up on a fixed timer and checked for pending
changes. Even if changes arrived 1 millisecond after the last check, the
stream table would not be refreshed until the next timer tick.

The new **event-driven scheduler** uses PostgreSQL's `LISTEN`/`NOTIFY`
mechanism. When a change is captured, the CDC trigger sends an immediate
notification to the scheduler, which wakes up and processes it without
waiting for a timer.

*In plain terms:* if your stream table previously refreshed every 5 seconds
by polling, it now refreshes within milliseconds of changes arriving — a
34× improvement in typical refresh latency with no additional resource cost.

---

## Stream Table to Stream Table Differential Refresh

When a stream table A depends on stream table B (which itself depends on
source tables), the previous approach would re-derive A from the original
source tables on each refresh. The new **ST-to-ST differential** approach
propagates the delta from B directly into A — only the changes need to flow
upward through the dependency chain.

*In plain terms:* if you have a chain of stream tables — raw events → daily
summaries → weekly rollups — updating the weekly rollup now only processes
the changes in the daily summaries, not all raw events.

---

## FUSE Circuit Breaker

If a stream table encounters a persistent error — a bug in the defining query,
a data type mismatch, an out-of-memory condition — it would previously block
the refresh of all downstream stream tables that depend on it.

The **FUSE circuit breaker** marks a stream table as *blown* after a
configurable number of consecutive failures and skips it in the scheduler.
Downstream tables that do not depend on the broken one continue to refresh
normally. An alert is emitted so operators are informed.

---

## VARBIT Bitmask for Change Detection

The change detection mechanism now uses a compact VARBIT bitmask to track
which columns changed in an UPDATE operation. This makes it possible for the
differential engine to skip unchanged columns in complex queries, reducing the
amount of work per refresh cycle.

---

## Scope

v0.11.0 is a large feature release addressing several major production
requirements: partitioned output tables, near-real-time latency, efficient
multi-layer dependency chains, and fault isolation. The event-driven scheduler
alone is a qualitative change in the responsiveness of the system.
