# v0.62.0 — Scheduler Throughput & `pg_aqueduct` Prerequisites (Full Details)

> **Summary:** [v0.62.0.md](v0.62.0.md)

---

## Performance

### PERF-1: Change-Buffer Fan-Out

**Motivation:** Every stream table that depends on source table T must process
T's change buffer on each scheduler tick.  Before this release each dependent
node issues an independent `SELECT` against `pgtrickle_changes.changes_<oid>`,
resulting in O(N) scans for N dependents.  For diamond-topology DAGs this cost
is paid multiple times even when every branch ultimately processes the same
rows.

**Implementation:**

1. The scheduler's per-tick dispatch loop collects all source OIDs whose change
   buffers are non-empty (single `pg_class` + buffer-size check, already
   exists).
2. For each non-empty source OID, a single read of `changes_<oid>` materialises
   the delta rows into a `Vec<ChangeRow>` in the BGW's local memory.
3. The delta is cloned (shallow — rows are `Arc`-wrapped) and dispatched to
   each dependent node's refresh task, replacing the per-node buffer scan.
4. The buffer is compacted once after all dependents have acknowledged
   processing — not once per dependent.

**GUC:** `pg_trickle.enable_change_buffer_fanout` (bool, default `true`).
Setting to `false` restores pre-v0.62.0 per-node scan behaviour for
diagnostic purposes.

**Benchmark gate:**

```
cargo bench --bench refresh_bench -- diamond_fanout
```

The `diamond_fanout` benchmark runs a 6-node diamond DAG (1 source → 2 middle
nodes → 2 leaf nodes → 1 convergence node) with Δ = 10 K inserted rows and
measures total change-buffer scan time.  Gate: ≥ 30 % reduction vs. v0.61.0
baseline.  A regression that fails this gate blocks the release.

**Test coverage:** New E2E test `test_scheduler_fanout_deduplicates_buffer_scan`
asserts that with N dependents on one source, `pg_stat_user_tables.seq_scan`
on the change-buffer table increments by 1 per tick (not N).

---

## New SQL API

### API-1 & API-2: `pause_scheduler` / `resume_scheduler`

**Files:** `src/api/scheduler_control.rs` (new), `src/lib.rs` (registration)

```sql
pgtrickle.pause_scheduler(nodes text[]) RETURNS void
pgtrickle.resume_scheduler(nodes text[]) RETURNS void
```

**Behaviour of `pause_scheduler`:**

1. For each name in `nodes`, set a `paused = true` flag in the shared-memory
   scheduler state for that stream table (new field in `StreamTableState`).
2. Poll `pgtrickle.pgt_stream_tables` for `refresh_status = 'running'` on the
   affected nodes every 100 ms, up to `pg_trickle.scheduler_drain_timeout`
   (default 30 s, GUC-configurable).
3. Return once all in-flight refreshes have completed or the timeout expires.
   On timeout, raise a `WARNING` and return — the caller (`pg_aqueduct`) is
   responsible for deciding whether to abort or proceed.

**Behaviour of `resume_scheduler`:**

Clears the `paused` flag for each listed node.  The scheduler picks up those
nodes on the next tick.  Safe to call multiple times (idempotent).

**Error handling:** Raises `ERROR` if any name in `nodes` does not exist in
`pgtrickle.pgt_stream_tables`.  Does not raise on names that are already
paused/resumed.

**Security:** `SECURITY DEFINER` with `SET search_path = pgtrickle, pg_catalog`.
Requires `USAGE` on the `pgtrickle` schema (same as `create_stream_table`).

**New GUC:** `pg_trickle.scheduler_drain_timeout` (interval, default `'30s'`).

---

### API-3: `stream_table_spec`

**Files:** `src/api/spec.rs` (new), `src/lib.rs` (registration)

```sql
pgtrickle.stream_table_spec(relid oid) RETURNS jsonb
```

Queries `pgtrickle.pgt_stream_tables` and related catalog tables for `relid`
and assembles the canonical JSON projection.  Returns `NULL` if `relid` is not
a managed stream table.

**Field stability contract:**

- All fields in the v0.62.0 response are stable across future minor releases.
- New optional fields may be added; no existing fields will be removed or
  renamed before v2.0.
- The response is always a JSON object (never null) when `relid` is a valid
  stream table OID.

**Convenience overload:**

```sql
pgtrickle.stream_table_spec(qualified_name text) RETURNS jsonb
```

Accepts `'schema.name'` or `'name'` (resolves via `to_regclass`).

**Test coverage:** Unit test in `src/api/spec.rs` verifying field presence and
types.  Integration test asserting round-trip fidelity: `import` a stream table,
call `stream_table_spec`, compare hash against `spec_hash` produced by
`pg_aqueduct import`.

---

## Test Coverage

| Test | Location | What it covers |
|------|----------|----------------|
| `test_scheduler_fanout_deduplicates_buffer_scan` | `tests/e2e_scheduler_tests.rs` | seq_scan count = 1 per tick with 4 dependents |
| `test_pause_resume_scheduler_basic` | `tests/e2e_scheduler_tests.rs` | pause blocks new refreshes; resume restores them |
| `test_pause_scheduler_drain_timeout` | `tests/e2e_scheduler_tests.rs` | timeout returns WARNING, does not hang |
| `test_stream_table_spec_fields` | `src/api/spec.rs` (#[cfg(test)]) | all required fields present, correct types |
| `test_stream_table_spec_unknown_oid` | `src/api/spec.rs` (#[cfg(test)]) | returns NULL for unknown OID |

---

## Upgrade Notes

No breaking changes.  Two new functions (`pause_scheduler`,
`resume_scheduler`, `stream_table_spec`) are additive.  The fan-out
optimisation is transparent; set `pg_trickle.enable_change_buffer_fanout = false`
to revert to the pre-v0.62.0 scan behaviour if needed.