> **See also:** [ROADMAP.md](../ROADMAP.md) ## v1.4.0 — PGlite WASM Extension > **Release Theme** > This release delivers the first working PGlite extension — the moment > pg_trickle's incremental view maintenance runs in the browser. By > wrapping `pg_trickle_core` (extracted in v0.29.0) in a thin C/FFI shim > and compiling to WASM via PGlite's Emscripten toolchain, we ship an npm > package (`@pgtrickle/pglite`) that gives PGlite users the full DVM > operator vocabulary — outer joins, window functions, subqueries, > recursive CTEs — in IMMEDIATE mode. This dramatically exceeds pg_ivm's > PGlite offering (INNER joins + basic aggregates only). The release also > establishes the cross-platform correctness and performance baselines that > all future PGlite work builds on. See [PLAN_PGLITE.md](plans/ecosystem/PLAN_PGLITE.md) §5 Strategy A and §7 Phase 2 for the full architecture. ### PGlite WASM Build (Phase 2) > **In plain terms:** This takes the `pg_trickle_core` crate extracted in > v0.29.0 and wraps it in a thin C shim that PGlite's Emscripten-based > extension build system can compile to WASM. The result is a PGlite > extension package (`@pgtrickle/pglite`) that provides > `create_stream_table()`, `drop_stream_table()`, and `alter_stream_table()` > — all running IMMEDIATE mode inside the WASM PostgreSQL engine with the > full DVM operator set. | Item | Description | Effort | Ref | |------|-------------|--------|-----| | PGL-2-1 | **C shim for PGlite.** Thin C wrapper bridging PGlite's Emscripten environment to `pg_trickle_core` via Rust FFI. Handles `raw_parser` calls through PGlite's built-in PostgreSQL parser. | 1–2wk | [PLAN_PGLITE.md](plans/ecosystem/PLAN_PGLITE.md) §5 Strategy A | | PGL-2-2 | **`DatabaseBackend` for PGlite.** Implement the trait for PGlite's single-connection SPI and built-in parser. Remove advisory lock acquisition (trivial in single-connection). | 3–5d | [PLAN_PGLITE.md](plans/ecosystem/PLAN_PGLITE.md) §5 Strategy A | | PGL-2-3 | **WASM bundle build.** Integrate with PGlite's extension toolchain (`postgres-pglite`). Produce `.tar.gz` WASM bundle. Target bundle size < 2 MB. | 3–5d | [PLAN_PGLITE.md](plans/ecosystem/PLAN_PGLITE.md) §8 | | PGL-2-4 | **TypeScript wrapper.** `@pgtrickle/pglite` npm package with PGlite plugin API. `createStreamTable()`, `dropStreamTable()`, `alterStreamTable()` with full IMMEDIATE mode support. | 2–3d | [PLAN_PGLITE.md](plans/ecosystem/PLAN_PGLITE.md) §7 Phase 2 | | PGL-2-5 | **IMMEDIATE mode E2E tests on PGlite.** Verify inner joins, outer joins, aggregates, DISTINCT, UNION ALL, window functions, subqueries, CTEs (non-recursive + recursive), LATERAL, view inlining, DISTINCT ON, GROUPING SETS. | 1–2wk | [PLAN_PGLITE.md](plans/ecosystem/PLAN_PGLITE.md) §4.1 | | PGL-2-6 | **PG 17 vs PG 18 parse tree compatibility.** PGlite tracks PG 17; pg_trickle targets PG 18. Audit and gate any node struct differences with conditional compilation. | 3–5d | [PLAN_PGLITE.md](plans/ecosystem/PLAN_PGLITE.md) §8 | > **Phase 2 subtotal: ~5–7 weeks** ### Correctness | ID | Title | Effort | Priority | |----|-------|--------|----------| | CORR-1 | PG 17/18 parse tree node divergence audit | M | P0 | | CORR-2 | Delta SQL cross-platform equivalence | M | P0 | | CORR-3 | Advisory lock no-op safety proof | S | P1 | | CORR-4 | IMMEDIATE trigger ordering in single-connection | S | P1 | **CORR-1 — PG 17/18 parse tree node divergence audit** > **In plain terms:** PGlite embeds PostgreSQL 17's parser; pg_trickle's > `OpTree` construction targets PostgreSQL 18 node structs. Any struct > layout difference (added fields, renamed members, changed enum values) > would cause the C shim to misinterpret parse trees, producing silently > wrong delta SQL. Systematically diff the PG 17 and PG 18 parse tree > headers (`nodes/parsenodes.h`, `nodes/primnodes.h`) and catalog every > node type that pg_trickle traverses. Gate incompatible nodes behind > `#[cfg(pg17)]` / `#[cfg(pg18)]` conditional compilation. Verify: a CI job compiles `pg_trickle_core` against both PG 17 and PG 18 parse tree headers. A test generates OpTrees from the same SQL on both versions and asserts structural equality. Dependencies: PGL-2-6. Schema change: No. **CORR-2 — Delta SQL cross-platform equivalence** > **In plain terms:** The same SQL view definition must produce the exact > same delta SQL on native PostgreSQL 18 and PGlite (WASM + PG 17 parser). > Any divergence means one platform gets wrong incremental results. Create > a snapshot test suite that runs all 22 TPC-H stream table definitions > through both the native and WASM `DatabaseBackend` implementations and > asserts byte-for-byte identical delta SQL output. Verify: snapshot comparison test passes for all 22 TPC-H queries on both platforms. Any diff is a hard failure. Dependencies: PGL-2-2, CORR-1. Schema change: No. **CORR-3 — Advisory lock no-op safety proof** > **In plain terms:** The native extension uses `pg_advisory_xact_lock()` > to prevent concurrent refresh of the same stream table. PGlite is > single-connection — the lock acquisition is a no-op. Verify that > removing the lock cannot cause re-entrancy (a trigger firing > `create_stream_table()` from within a refresh) by auditing all SPI > call paths from the PGlite `DatabaseBackend` for re-entrant calls. Verify: code review + integration test that attempts re-entrant refresh from within a trigger. Must error cleanly, not corrupt state. Dependencies: PGL-2-2. Schema change: No. **CORR-4 — IMMEDIATE trigger ordering in single-connection** > **In plain terms:** IMMEDIATE mode relies on AFTER triggers firing in a > specific order when multiple source tables are modified in the same > statement (e.g., a CTE with multiple INSERTs). Verify that PGlite's > trigger execution order matches native PostgreSQL's for the trigger > configurations pg_trickle creates. Verify: integration test with multi-table CTE INSERT on PGlite; assert stream table state matches native. Dependencies: PGL-2-5. Schema change: No. ### Stability | ID | Title | Effort | Priority | |----|-------|--------|----------| | STAB-1 | WASM heap OOM graceful degradation | M | P0 | | STAB-2 | C shim panic/unwind boundary safety | S | P0 | | STAB-3 | Extension load/unload lifecycle correctness | S | P0 | | STAB-4 | Native extension upgrade path (0.24 → 0.25) | S | P0 | | STAB-5 | npm package version synchronization | XS | P1 | **STAB-1 — WASM heap OOM graceful degradation** > **In plain terms:** WASM environments have a finite heap (typically > 256 MB in browsers, configurable in Node). A large stream table with > many operators could exhaust WASM memory during OpTree construction or > delta SQL generation. The extension must detect allocation failures and > return a clear PostgreSQL error rather than crashing the WASM instance > (which would kill all PGlite state). Implement a memory-aware allocator > wrapper or check `emscripten_get_heap_size()` at entry points. Verify: stress test creating stream tables over increasingly complex views until OOM; assert PGlite remains functional and returns an actionable error. Dependencies: PGL-2-1. Schema change: No. **STAB-2 — C shim panic/unwind boundary safety** > **In plain terms:** Rust panics must not cross the FFI boundary into C. > The C shim must catch panics via `std::panic::catch_unwind()` and > convert them to PostgreSQL `ereport(ERROR)` calls. Any uncaught panic in > WASM would abort the entire PGlite instance. Audit every `#[no_mangle] > extern "C"` entry point in the shim for panic safety. Verify: test that triggers a panic path (e.g., invalid SQL) from TypeScript; assert PGlite returns a SQL error, not a WASM trap. Dependencies: PGL-2-1. Schema change: No. **STAB-3 — Extension load/unload lifecycle correctness** > **In plain terms:** PGlite extensions can be loaded and unloaded. The > C shim must free all Rust-allocated memory on unload and not leave > dangling pointers or leaked state. Test the full lifecycle: load > extension → create stream tables → drop stream tables → unload > extension → reload extension → create new stream tables. Verify: lifecycle test with memory profiling shows zero leaked allocations after unload/reload cycle. Dependencies: PGL-2-1, PGL-2-4. Schema change: No. **STAB-4 — Native extension upgrade path (0.27 → 0.28)** > **In plain terms:** v0.30.0 adds PGlite support but makes no SQL-visible > changes to the native extension. The upgrade migration from 0.27.0 to > 0.28.0 must leave existing stream tables intact and refreshable. Verify: upgrade E2E test confirms stream tables survive and refresh correctly after `0.27.0 -> 0.28.0`. **STAB-5 — npm package version synchronization** > **In plain terms:** The `@pgtrickle/pglite` npm package version must > match the extension version (0.28.0). Add a CI check that verifies > `package.json` version matches `pg_trickle.control` version, similar to > the existing `just check-version-sync` target. Verify: `just check-version-sync` also validates npm package version. Dependencies: PGL-2-4. Schema change: No. ### Performance | ID | Title | Effort | Priority | |----|-------|--------|----------| | PERF-1 | WASM vs native refresh latency benchmark | M | P0 | | PERF-2 | WASM bundle size optimization (< 2 MB target) | M | P0 | | PERF-3 | PGlite cold-start extension load time | S | P1 | **PERF-1 — WASM vs native refresh latency benchmark** > **In plain terms:** WASM is expected to be 1.5–3× slower than native > (per PLAN_PGLITE.md §8). Quantify the actual overhead by benchmarking > IMMEDIATE-mode refresh on both platforms using the same schema + data. > The overhead must stay below the threshold where IMMEDIATE mode is still > faster than full re-evaluation — otherwise PGlite users would be better > off just re-running the query. Establish a Criterion-like benchmark suite > for PGlite (potentially using Node.js + `@electric-sql/pglite`). Verify: benchmark report showing WASM refresh latency for 5 representative stream tables (scan, join, aggregate, window, recursive CTE). Document native-to-WASM overhead ratio. Dependencies: PGL-2-5. Schema change: No. **PERF-2 — WASM bundle size optimization (< 2 MB target)** > **In plain terms:** The WASM bundle must be < 2 MB for acceptable > download times in browser environments (PostGIS is 8.2 MB, pgcrypto is > 1.1 MB — pg_trickle should be closer to pgcrypto). Apply `wasm-opt -Oz`, > LTO, `codegen-units = 1`, strip debug info, and feature-gate large > operator modules (e.g., recursive CTE, window functions) behind optional > features if needed to meet the target. Verify: CI job measures WASM bundle size after `wasm-opt` and fails if > 2 MB. Document size breakdown by operator module. Dependencies: PGL-2-3. Schema change: No. **PERF-3 — PGlite cold-start extension load time** > **In plain terms:** The first `CREATE EXTENSION pg_trickle` in a PGlite > session compiles and loads the WASM module. This must complete in < 500 ms > in a browser and < 200 ms in Node.js. Measure and optimize by using > streaming WASM compilation (`WebAssembly.compileStreaming()`) and ensuring > the extension `_PG_init()` function does minimal work. Verify: benchmark measuring time from `CREATE EXTENSION` to first `create_stream_table()` on fresh PGlite instance. Document cold-start time. Dependencies: PGL-2-1, PGL-2-3. Schema change: No. ### Scalability | ID | Title | Effort | Priority | |----|-------|--------|----------| | SCAL-1 | Stream table count ceiling in WASM | S | P1 | | SCAL-2 | Wide-table OpTree memory footprint | S | P1 | | SCAL-3 | Dataset size practical limit for IMMEDIATE mode | S | P2 | **SCAL-1 — Stream table count ceiling in WASM** > **In plain terms:** Each stream table consumes memory for its OpTree, > delta SQL templates, and trigger metadata. In native PostgreSQL with > gigabytes of RAM this is trivial, but in a 256 MB WASM heap it matters. > Determine the practical limit by creating stream tables in a loop until > OOM, then document the ceiling and add a guard that errors at 80% > capacity with an actionable message. Verify: stress test documents the ceiling (e.g., "~200 stream tables with average 3-table join in 256 MB heap"). Guard errors at 80%. Dependencies: STAB-1. Schema change: No. **SCAL-2 — Wide-table OpTree memory footprint** > **In plain terms:** A stream table over a 100-column source table > produces a large OpTree and long delta SQL strings. Profile the memory > consumption of OpTree construction for wide tables and ensure it fits > within the WASM heap budget alongside typical stream table counts. Verify: profile OpTree allocation for 10, 50, 100-column source tables. Document memory per stream table as a function of column count. Dependencies: PGL-2-5. Schema change: No. **SCAL-3 — Dataset size practical limit for IMMEDIATE mode** > **In plain terms:** IMMEDIATE mode fires triggers on every DML, so > overhead scales with write frequency. In a WASM environment with ~2× > slower execution, determine at what dataset size (rows × columns × > writes/second) IMMEDIATE mode becomes impractical. Document the > breakpoint so PGlite users know when their use case has outgrown the > browser and should migrate to native pg_trickle with DIFFERENTIAL mode. Verify: benchmark with increasing write rates; document the throughput ceiling (e.g., "> 10K rows/sec INSERT rate degrades stream table latency past 100 ms"). Dependencies: PERF-1. Schema change: No. ### Ease of Use | ID | Title | Effort | Priority | |----|-------|--------|----------| | UX-1 | TypeScript API ergonomics and type safety | S | P0 | | UX-2 | PGlite getting-started guide | M | P0 | | UX-3 | WASM-context error message quality | S | P1 | | UX-4 | npm package README with runnable examples | S | P1 | **UX-1 — TypeScript API ergonomics and type safety** > **In plain terms:** The `@pgtrickle/pglite` TypeScript API must follow > PGlite plugin conventions (`PGlitePlugin` interface, `init()` lifecycle). > All methods must be fully typed — no `any` types. The API surface must > be minimal: `createStreamTable(sql)`, `dropStreamTable(name)`, > `alterStreamTable(name, sql)`, `listStreamTables()`, and > `refreshStreamTable(name)`. Review against existing PGlite plugins > (`@electric-sql/pglite-repl`, `pglite-vector`) for consistency. Verify: TypeScript strict mode compilation with no errors. API review against PGlite plugin conventions checklist. Dependencies: PGL-2-4. Schema change: No. **UX-2 — PGlite getting-started guide** > **In plain terms:** A `docs/tutorials/PGLITE_QUICKSTART.md` guide > walking a user from `npm install` to a working React app with live > stream tables in < 10 minutes. Include: install, create PGlite instance > with extension, define source table + stream table, insert data, observe > stream table update. Provide a CodeSandbox / StackBlitz link for > zero-install try-it-now experience. Verify: a new developer can follow the guide and see a working stream table in PGlite in a browser within 10 minutes. Dependencies: PGL-2-4, UX-1. Schema change: No. **UX-3 — WASM-context error message quality** > **In plain terms:** Error messages from the Rust/C shim must be > JavaScript-friendly: no raw pg_sys error codes, no memory addresses. > Every error must include the stream table name, the failing SQL > fragment, and a remediation hint. Unsupported features (DIFFERENTIAL > mode, scheduled refresh, parallel workers) must error with > "Not supported in PGlite: . Use IMMEDIATE mode." rather than > cryptic internal errors. Verify: audit all error paths in the C shim + PGlite `DatabaseBackend`. Every error message includes table name + remediation hint. Dependencies: PGL-2-1, PGL-2-2. Schema change: No. **UX-4 — npm package README with runnable examples** > **In plain terms:** The npm package must have a README with: badge for > PGlite compatibility, install command, 3 runnable examples (basic > aggregate, join, window function), API reference, link to the full > PGlite quickstart guide, and a "Limitations vs native pg_trickle" > section clearly stating: no DIFFERENTIAL mode, no scheduled refresh, > no parallel workers, PG 17 parser only. Verify: README renders correctly on npmjs.com; examples are copy-pasteable into a Node.js REPL. Dependencies: PGL-2-4, UX-2. Schema change: No. ### Test Coverage | ID | Title | Effort | Priority | |----|-------|--------|----------| | TEST-1 | Full DVM operator E2E suite on PGlite | L | P0 | | TEST-2 | PG 17/18 parse tree compatibility tests | M | P0 | | TEST-3 | WASM memory stress tests | M | P1 | | TEST-4 | TypeScript integration tests | M | P0 | | TEST-5 | Bundle size regression gate in CI | S | P0 | **TEST-1 — Full DVM operator E2E suite on PGlite** > **In plain terms:** Run every DVM operator (23 operators across inner > join, outer join, full join, semi-join, anti-join, aggregate, distinct, > union/intersect/except, subquery, scalar subquery, CTE scan, recursive > CTE, lateral function, lateral subquery, window function, scan, filter, > project) through IMMEDIATE mode in PGlite. This is the primary > correctness gate for the WASM extension. Use a Node.js test harness > with `@electric-sql/pglite` to run the tests headlessly. Verify: test suite with ≥ 1 test per operator (23+ tests) passes in CI using PGlite Node.js. Test matrix: INSERT, UPDATE, DELETE for each operator. Dependencies: PGL-2-5. Schema change: No. **TEST-2 — PG 17/18 parse tree compatibility tests** > **In plain terms:** For every parse tree node type that pg_trickle > traverses, generate a test query that exercises that node, parse it on > both PG 17 (PGlite) and PG 18 (native), and assert that the resulting > `OpTree` is structurally identical. This catches version-specific > divergences before they reach users. Verify: compatibility test suite covers all node types referenced in `pg_trickle_core`. Any divergence is a hard failure with clear diagnostic. Dependencies: CORR-1. Schema change: No. **TEST-3 — WASM memory stress tests** > **In plain terms:** Create increasing numbers of stream tables with > increasing complexity until OOM. Verify that: (a) the guard from SCAL-1 > fires at 80% capacity, (b) PGlite remains functional after the guard > fires, (c) dropping stream tables actually frees memory. Run under > different heap sizes (64 MB, 128 MB, 256 MB) to validate the guard > thresholds. Verify: stress test with 3 heap sizes completes without WASM trap. Guard fires at documented threshold. Memory reclaimed after DROP. Dependencies: STAB-1, SCAL-1. Schema change: No. **TEST-4 — TypeScript integration tests** > **In plain terms:** Test the `@pgtrickle/pglite` TypeScript API end-to-end > using Jest or Vitest in Node.js. Cover: create/drop/alter stream table, > error handling (invalid SQL, unsupported features), plugin lifecycle > (init/cleanup), and concurrent operations on different stream tables. > Run as part of CI on every PR that touches `pg_trickle_pglite/`. Verify: ≥ 20 TypeScript integration tests pass in CI. Test coverage report for the TypeScript wrapper shows > 90% line coverage. Dependencies: PGL-2-4, UX-1. Schema change: No. **TEST-5 — Bundle size regression gate in CI** > **In plain terms:** Add a CI job that builds the WASM bundle, runs > `wasm-opt`, measures the final `.wasm` file size, and fails if it > exceeds 2 MB. Store the current size as a baseline and alert on any > increase > 10%. This prevents bundle bloat as features are added. Verify: CI job `check-wasm-size` runs on every PR touching `pg_trickle_core/` or `pg_trickle_pglite/`. Fails at > 2 MB. Dependencies: PGL-2-3, PERF-2. Schema change: No. ### Conflicts & Risks 1. **CORR-1 (PG 17/18 parse tree compatibility) is the highest risk.** PGlite embeds PG 17; pg_trickle targets PG 18. If node struct layouts diverged significantly between versions (e.g., `JoinExpr` gained a field, `RangeTblEntry` changed a flag), the C shim must handle both layouts via conditional compilation. In the worst case, some operators may need version-specific code paths. Start this audit early — it blocks PGL-2-1 and PGL-2-2. 2. **PERF-2 (bundle size < 2 MB) may conflict with full operator coverage.** If the 23-operator delta SQL generator compiles to > 2 MB, we may need to feature-gate rarely-used operators (recursive CTE, GROUPING SETS) behind cargo features. This would reduce the "full DVM vocabulary" claim and require documenting which operators are available by default. Measure early with a minimal build to establish baseline. 3. **PGlite's Emscripten toolchain is a moving target.** PGlite's extension build system (`postgres-pglite`) is not yet stable. Breaking changes in the toolchain could block PGL-2-3. Pin the PGlite version and track upstream releases. Have a fallback plan: manual Emscripten compilation without the PGlite toolchain. 4. **STAB-2 (panic boundary) and STAB-1 (OOM handling) interact.** A Rust OOM in WASM triggers a panic, which must not cross the FFI boundary. Both items must be implemented together: the OOM guard (STAB-1) sets a pre-panic threshold, and the catch_unwind wrapper (STAB-2) is the last-resort safety net. 5. **No prior C FFI in the codebase.** The only C code is `scripts/pg_stub.c` (test helper). The C shim (PGL-2-1) introduces a new language and toolchain requirement. Ensure the C code is minimal (< 500 lines), well-documented, and covered by the TypeScript integration tests. 6. **TEST-1 and TEST-4 require a PGlite-based CI runner.** Need Node.js 18+ with `@electric-sql/pglite` in CI. This is a new CI dependency. Add it to the existing CI matrix as a separate job that only runs when `pg_trickle_pglite/` or `pg_trickle_core/` files are modified. > **v1.4.0 total: ~5–7 weeks (WASM build) + ~2–3 weeks (testing + polish)** **Exit criteria:** - [ ] PGL-2-1: C shim compiles and links against PGlite's WASM PostgreSQL headers - [ ] PGL-2-2: PGlite `DatabaseBackend` passes all IMMEDIATE-mode operator tests - [ ] PGL-2-3: WASM bundle size < 2 MB after `wasm-opt` - [ ] PGL-2-4: `@pgtrickle/pglite` npm package published to npmjs.com - [ ] PGL-2-5: All 23 DVM operators pass E2E tests on PGlite - [ ] PGL-2-6: PG 17 parse tree differences documented and handled with `#[cfg]` - [ ] CORR-1: PG 17/18 parse tree audit complete; compatibility tests pass - [ ] CORR-2: Delta SQL cross-platform snapshot tests pass for all 22 TPC-H queries - [ ] CORR-3: Re-entrant refresh test passes on PGlite - [ ] CORR-4: Multi-table CTE trigger ordering matches native - [ ] STAB-1: OOM stress test: PGlite survives with actionable error - [ ] STAB-2: Panic from invalid SQL returns SQL error, not WASM trap - [ ] STAB-3: Load/unload/reload lifecycle test: zero leaked allocations - [ ] STAB-4: Extension upgrade path tested (`1.3.0 → 1.4.0`) - [ ] PERF-1: WASM vs native benchmark report published (≤ 3× overhead) - [ ] PERF-2: WASM bundle ≤ 2 MB (CI gated) - [ ] PERF-3: Cold-start load time < 500 ms browser, < 200 ms Node.js - [ ] TEST-1: ≥ 23 operator E2E tests pass on PGlite in CI - [ ] TEST-2: Parse tree compatibility tests cover all traversed node types - [ ] TEST-3: Memory stress tests pass under 64/128/256 MB heap sizes - [ ] TEST-4: ≥ 20 TypeScript integration tests with > 90% line coverage - [ ] TEST-5: CI `check-wasm-size` job passes on every PR - [ ] UX-1: TypeScript strict mode compilation: zero errors - [ ] UX-2: PGlite getting-started guide published with CodeSandbox link - [ ] UX-4: npm README renders correctly on npmjs.com - [ ] `just check-version-sync` passes (incl. npm package version) ---