# v0.70.0 Full Details — Scheduler, Validator & Security Hardening

> **Summary:** [v0.70.0.md](v0.70.0.md)
> **Assessment source:** [plans/PLAN_OVERALL_ASSESSMENT_13.md](../plans/PLAN_OVERALL_ASSESSMENT_13.md)

---

## Motivation

After v0.68.0 and v0.69.0 fixed the most acute data-integrity and DuckLake
reliability findings, v0.70.0 addresses the correctness and performance gaps
in the scheduler and DVM validator that affect all pg_trickle deployments:

- COR-002: LATERAL validation bypass — present since v0.63.0 fused refresh.
- PERF-001: per-source SPI fan-out — regressed in the v0.62.0 scheduler
  throughput work when the fan-out elimination was incomplete.
- PERF-002: fused eligibility O(N×M) SPI cost — introduced in v0.63.0.
- PERF-003: GUC wired but ignored — a promise made and never kept.
- PERF-004: uncapped memory in parallel refresh — a latent risk activated when
  planner-aggressive mode is combined with `parallel_refresh_mode = on`.
- SCAL-002: launcher poll cost — acceptable at ten databases, visible at fifty.
- SEC-001: name-parser inconsistency — a subtle correctness/security gap.
- OBS-002: invisible prune failures — a maintenance trap.
- TEST-001/TEST-002: missing test coverage enabling the above.

---

## Detailed Implementation

### COR-002: LATERAL Validator Body Scanning

**Files changed:**
- `src/dvm/parser/validation.rs` — `tree_collect_volatility()` match arm for
  `OpTree::LateralFunction`: parse `func_sql` with `parse_expression_tree()`,
  compute its volatility, and return `worst.combine(lateral_body_volatility)`;
  similarly for `LateralSubquery::subquery_sql`
- `src/dvm/parser/validation.rs` — `check_ivm_support_inner()`: recurse into
  lateral body SQL for both variants
- `tests/e2e_error_tests.rs` — three new tests
- `tests/e2e_lateral_tests.rs` — immutable LATERAL pass test

**Acceptance criteria:**
1. `LATERAL (SELECT random())` with `volatile_function_policy = 'reject'` raises
   `UnsupportedFeature` during stream table creation.
2. Existing LATERAL tests that use immutable SRFs continue to pass.

---

### PERF-001: Batched Buffer Health Checks

**Files changed:**
- `src/monitor/mod.rs` — `check_slot_health_and_alert()`: collect all buffer
  names into a `Vec`, build
  `SELECT table_name, count(*) FROM (SELECT 'name1' AS table_name, count(*) ... UNION ALL ...) sub GROUP BY 1`
  query, execute as a single SPI call, iterate results
- `benches/scheduler_bench.rs` — `bench_buffer_health_1k` and
  `bench_buffer_health_10k` benchmarks

**Acceptance criteria:**
1. Single-SPI execution verified by `EXPLAIN` in test harness.
2. `bench_buffer_health_1k` baseline captured by Criterion CI.

---

### PERF-002: Batched Fused Eligibility

**Files changed:**
- `src/scheduler/mod.rs` — extract `FusedEligibilityContext` struct; preload
  dependency rows in one query keyed on `pgt_id IN (...)` for all candidates;
  preload source OIDs and latest frontier LSNs in one query each
- `src/scheduler/mod.rs` — reuse context across eligibility loop iterations

---

### PERF-003: History Prune GUC + Index

**Files changed:**
- `src/scheduler/scheduler_loop.rs` — replace hard-coded
  `HISTORY_CLEANUP_INTERVAL_MS = 24h` with
  `pg_trickle_history_prune_interval_seconds() * 1000`
- `sql/pg_trickle--0.69.0--0.70.0.sql` — `CREATE INDEX CONCURRENTLY`
- `tests/integration_tests.rs` — `test_history_prune_interval_guc`

---

### PERF-004: Work-Mem Cap Default

**Files changed:**
- `src/config.rs` — `PGS_DELTA_WORK_MEM_CAP_MB` default changed to `256`
- `src/api/preflight.rs` (or `monitor.rs`) — add preflight check for
  zero cap + planner-aggressive combo
- `docs/GUC_CATALOG.md` — update default documentation

---

### SCAL-002: Launcher DB Cache

**Files changed:**
- `src/shmem.rs` — add `LauncherDbCache` shared-memory structure: bitset or
  `PgLwLock<Vec<Oid>>` of databases with pg_trickle installed
- `src/hooks.rs` — populate/invalidate cache on `pg_trickle` CREATE/DROP
  EXTENSION events
- `src/scheduler/scheduler_loop.rs` — read cache; use 60s interval when cache
  is non-empty and no invalidation; fall back to full scan on cache miss

---

### SEC-001: Name Parser Unification

**Files changed:**
- `src/api/publication.rs` — remove private `parse_qualified_name()`;
  replace calls with `crate::api::helpers::parse_qualified_name()`
- `tests/e2e_publication_tests.rs` — add `test_publication_non_public_schema`
  and `test_publication_quoted_identifier`

---

### OBS-002: Prune Error Visibility

**Files changed:**
- `src/scheduler/scheduler_loop.rs` — capture SPI error, emit `warning!`,
  increment `HISTORY_PRUNE_ERRORS` counter
- `src/shmem.rs` — add `pg_trickle_history_prune_errors_total` atomic counter
- `src/monitor/mod.rs` — add `history_prune_status()` pg_extern

---

## Upgrade Notes

`delta_work_mem_cap_mb` now defaults to `256`. If you run with a very small
`work_mem` and many parallel refresh workers, verify that the cap remains
appropriate before upgrading. The configuration page documents the formula
`max(1, parallel_workers) * work_mem * safety_factor` as a starting point.

The LATERAL validation change is a behaviour change, not a schema change. Queries
that previously passed validation with volatile LATERAL bodies will now fail at
stream table creation time with a clear `UnsupportedFeature` error message
pointing to the offending function.