> **See also:** [ROADMAP.md](../ROADMAP.md)

## v1.3.0 — Core Extraction (`pg_trickle_core`)

> **Release Theme**
> This release surgically separates pg_trickle's "brain" — the DVM engine,
> operator delta SQL generation, query rewrite passes, and DAG computation —
> into a standalone Rust crate (`pg_trickle_core`) with zero pgrx dependency.
> The extraction touches ~51,000 lines of code across 30+ source files but
> produces zero user-visible behavior change: every existing test must pass
> unchanged. The payoff is threefold: the core crate compiles to WASM
> (enabling the PGlite extension in v0.30.0), pure-logic unit tests run
> without a PostgreSQL instance (10x faster CI), and the main extension
> gains a cleaner internal architecture. Approximately 500 unsafe blocks in
> the parser require an abstraction layer over raw `pg_sys` node traversal,
> making this the most technically demanding refactoring in the project's
> history.

See [PLAN_PGLITE.md](plans/ecosystem/PLAN_PGLITE.md) §5 Strategy A for the
full extraction architecture.

### Core Crate Extraction (Phase 1)

> **In plain terms:** pg_trickle's "brain" — the code that analyses SQL
> queries, builds operator trees, and generates delta SQL — is currently
> tangled with pgrx (the Rust-to-PostgreSQL bridge). This milestone
> surgically separates the pure logic into its own crate so it can be
> compiled independently. The existing extension continues to work
> unchanged; it just imports from `pg_trickle_core` instead of having the
> code inline. A `trait DatabaseBackend` abstracts SPI and parser access
> so the core logic can be tested without a running PostgreSQL instance.

| Item | Description | Effort | Ref |
|------|-------------|--------|-----|
| PGL-1-1 | **Create `pg_trickle_core` crate.** Workspace member with `[lib]` target, no pgrx dependency. Move `OpTree`, `Expr`, `Column`, `AggExpr`, and all shared types. | 1–2d | [PLAN_PGLITE.md](plans/ecosystem/PLAN_PGLITE.md) §5 Strategy A |
| PGL-1-2 | **Extract operator delta SQL generation.** Move all `src/dvm/operators/` logic (~24K lines, 23 files) into the core crate. Each operator's `generate_delta_sql()` becomes a pure function taking abstract types. | 3–5d | [PLAN_PGLITE.md](plans/ecosystem/PLAN_PGLITE.md) §5 Strategy A |
| PGL-1-3 | **Extract auto-rewrite passes.** Move view inlining, DISTINCT ON rewrite, GROUPING SETS expansion, and SubLink extraction into `pg_trickle_core::rewrites`. | 2–3d | [PLAN_PGLITE.md](plans/ecosystem/PLAN_PGLITE.md) §5 Strategy A |
| PGL-1-4 | **Extract DAG computation.** Move dependency graph, topological sort, cycle detection, diamond detection into `pg_trickle_core::dag`. | 1–2d | [PLAN_PGLITE.md](plans/ecosystem/PLAN_PGLITE.md) §5 Strategy A |
| PGL-1-5 | **Define `trait DatabaseBackend`.** Abstract trait for SPI queries and raw_parser access. Implement for pgrx in the main extension crate. | 2–3d | [PLAN_PGLITE.md](plans/ecosystem/PLAN_PGLITE.md) §5 Strategy A |
| PGL-1-6 | **WASM compilation gate.** Verify `pg_trickle_core` compiles to `wasm32-unknown-emscripten` target. CI check for WASM build. | 1–2d | [PLAN_PGLITE.md](plans/ecosystem/PLAN_PGLITE.md) §5 Strategy A |
| PGL-1-7 | **Existing test suite passes.** All unit, integration, and E2E tests pass with the refactored crate structure. Zero behavior change. | 2–3d | — |

> **Phase 1 subtotal: ~3–4 weeks**

### Correctness

| ID | Title | Effort | Priority |
|----|-------|--------|----------|
| CORR-1 | Delta SQL output byte-for-byte equivalence | M | P0 |
| CORR-2 | OpTree serialization round-trip fidelity | S | P0 |
| CORR-3 | Rewrite pass ordering preservation | S | P1 |
| CORR-4 | DAG cycle detection parity after extraction | S | P1 |

**CORR-1 — Delta SQL output byte-for-byte equivalence**

> **In plain terms:** After the extraction, every operator's
> `generate_delta_sql()` must produce the exact same SQL string as it did
> before the refactoring. Any byte-level difference — even whitespace —
> indicates a semantic shift that could change query plans or correctness.
> Capture the SQL output for all 22 TPC-H stream tables before and after
> the extraction and assert bit-for-bit equality.

Verify: snapshot test comparing delta SQL for all TPC-H queries + the full
E2E test suite. Any diff fails the build.
Dependencies: PGL-1-2. Schema change: No.

**CORR-2 — OpTree serialization round-trip fidelity**

> **In plain terms:** The `OpTree` types are moving to a new crate. If any
> field is accidentally dropped or retyped during the move, the delta SQL
> generator will silently produce wrong output. Add a round-trip test:
> serialize an OpTree to JSON, deserialize it back, and assert structural
> equality. This catches missing `#[derive]` attributes and field ordering
> issues.

Verify: proptest generating random OpTrees; serialize-deserialize round-trip
produces identical trees.
Dependencies: PGL-1-1. Schema change: No.

**CORR-3 — Rewrite pass ordering preservation**

> **In plain terms:** The auto-rewrite passes (view inlining, DISTINCT ON,
> GROUPING SETS, SubLink extraction) must execute in the same order after
> extraction. Reordering could change the resulting OpTree and thereby the
> delta SQL. Add an integration test that runs all rewrite passes on a
> complex query (joining 3 tables with DISTINCT ON + GROUPING SETS) and
> asserts the final OpTree matches a golden snapshot.

Verify: golden-snapshot test for rewrite pass output on complex query.
Dependencies: PGL-1-3. Schema change: No.

**CORR-4 — DAG cycle detection parity after extraction**

> **In plain terms:** The cycle detection algorithm in `dag.rs` has subtleties
> around self-referencing views and diamond patterns. After moving to the
> core crate, the algorithm must detect the same cycles. Run the existing
> cycle-detection unit tests and add 3 new edge cases: self-referencing CTE,
> diamond with mixed IMMEDIATE/DIFFERENTIAL, and 4-level cascade.

Verify: all existing DAG unit tests pass + 3 new edge-case tests.
Dependencies: PGL-1-4. Schema change: No.

### Stability

| ID | Title | Effort | Priority |
|----|-------|--------|----------|
| STAB-1 | pg_sys node abstraction layer (~500 unsafe blocks) | L | P0 |
| STAB-2 | Compile-time pgrx dependency leak detection | S | P0 |
| STAB-3 | Cargo workspace configuration correctness | S | P0 |
| STAB-4 | Extension upgrade path (0.19 to 0.20) | S | P0 |
| STAB-5 | Feature-flag isolation for WASM target | S | P1 |

**STAB-1 — pg_sys node abstraction layer (~500 unsafe blocks)**

> **In plain terms:** `rewrites.rs` (118 unsafe blocks, 295 `pg_sys` refs)
> and `sublinks.rs` (367 unsafe blocks, 492 `pg_sys` refs) are the most
> deeply coupled to pgrx. The core crate cannot contain raw `pg_sys` calls.
> Define a `trait NodeVisitor` (or equivalent) that wraps pg_sys node
> traversal behind safe method calls. The pgrx backend implements the trait
> using actual pg_sys pointers; a mock backend can be used for unit tests.
> This is the single highest-effort item in the release.

Verify: zero `pg_sys::` references in `pg_trickle_core/`; `grep -r pg_sys
pg_trickle_core/src/` returns empty.
Dependencies: PGL-1-1, PGL-1-5. Schema change: No.

**STAB-2 — Compile-time pgrx dependency leak detection**

> **In plain terms:** After extraction, any accidental `use pgrx::*` in the
> core crate would break the WASM build. Add a CI job that compiles
> `pg_trickle_core` in isolation (without the pgrx feature) and fails if any
> pgrx symbol is referenced. This catches leaks immediately rather than at
> WASM build time.

Verify: `cargo build -p pg_trickle_core --no-default-features` succeeds in
CI.
Dependencies: PGL-1-1. Schema change: No.

**STAB-3 — Cargo workspace configuration correctness**

> **In plain terms:** Adding a workspace member changes `Cargo.lock`
> resolution, feature unification, and `cargo pgrx` behavior. Verify:
> `cargo pgrx package` still produces a valid `.so`, `cargo test` runs all
> workspace tests, and `cargo pgrx test` works for the extension crate.
> pgrx version must remain pinned at 0.17.x.

Verify: `cargo pgrx package`, `cargo test --workspace`, `cargo pgrx test`
all succeed.
Dependencies: PGL-1-1. Schema change: No.

**STAB-4 — Extension upgrade path (0.19 to 0.20)**

> **In plain terms:** v0.29.0 makes no SQL-visible changes (same functions,
> same catalog schema), but the upgrade migration must still be tested.
> `ALTER EXTENSION pg_trickle UPDATE` from 0.27.0 to 0.28.0 must leave
> existing stream tables intact and refreshable.

Verify: upgrade E2E test confirms stream tables survive and refresh
 correctly after `0.27.0 -> 0.28.0`.

**STAB-5 — Feature-flag isolation for WASM target**

> **In plain terms:** The core crate must compile on both native and WASM.
> Any platform-specific code (e.g., `std::time::Instant` unavailable on
> `wasm32-unknown-emscripten`) must be gated behind `#[cfg]` attributes.
> Add a CI matrix entry for the WASM target that catches platform leaks.

Verify: `cargo build --target wasm32-unknown-emscripten -p pg_trickle_core`
succeeds in CI.
Dependencies: PGL-1-6. Schema change: No.

### Performance

| ID | Title | Effort | Priority |
|----|-------|--------|----------|
| PERF-1 | Zero-overhead abstraction for DatabaseBackend | M | P0 |
| PERF-2 | Benchmark regression gate across extraction | S | P0 |
| PERF-3 | Core-only unit test speedup measurement | S | P1 |

**PERF-1 — Zero-overhead abstraction for DatabaseBackend**

> **In plain terms:** The `trait DatabaseBackend` introduces dynamic
> dispatch (`dyn DatabaseBackend` or generics). For the native extension,
> the abstraction must add zero measurable overhead. Use monomorphization
> (generics, not trait objects) for the hot path — delta SQL generation is
> called on every refresh cycle and must not regress. Measure with Criterion
> before/after on the `diff_operators` benchmark suite.

Verify: Criterion benchmark shows < 1% regression on `diff_operators` suite
after extraction.
Dependencies: PGL-1-5. Schema change: No.

**PERF-2 — Benchmark regression gate across extraction**

> **In plain terms:** The extraction touches 51K lines of code. Even
> without functional changes, module restructuring can alter inlining,
> cache locality, and link-time optimization. Run the full Criterion
> benchmark suite before and after and assert no regression > 5%.

Verify: `scripts/criterion_regression_check.py` passes with 5% threshold on
all existing benchmarks.
Dependencies: PGL-1-7. Schema change: No.

**PERF-3 — Core-only unit test speedup measurement**

> **In plain terms:** One of the key benefits of extraction is that
> `pg_trickle_core` unit tests run without starting PostgreSQL. Measure the
> wall-clock time for `cargo test -p pg_trickle_core` vs the old in-tree
> unit tests. Document the speedup in the CHANGELOG — expect 5-10x faster
> CI for unit-level tests.

Verify: document test execution times before/after in PR description.
Dependencies: PGL-1-7. Schema change: No.

### Scalability

| ID | Title | Effort | Priority |
|----|-------|--------|----------|
| SCAL-1 | Workspace build parallelism verification | S | P1 |
| SCAL-2 | Core crate binary size for WASM budget | S | P1 |
| SCAL-3 | Incremental compilation impact assessment | S | P2 |

**SCAL-1 — Workspace build parallelism verification**

> **In plain terms:** With two crates, `cargo build` can compile
> `pg_trickle_core` and other non-dependent crates in parallel. Verify that
> the workspace DAG allows parallel compilation and measure the
> incremental rebuild time for a change in `pg_trickle_core` only.

Verify: `cargo build --timings` shows parallel compilation of core crate.
Dependencies: PGL-1-1. Schema change: No.

**SCAL-2 — Core crate binary size for WASM budget**

> **In plain terms:** v0.30.0 targets < 2 MB WASM bundle. Measure the
> compiled size of `pg_trickle_core` for the WASM target now so the budget
> is known before Phase 2. If > 5 MB, investigate `wasm-opt` stripping and
> feature-gating large operator modules.

Verify: `wasm32-unknown-emscripten` build of `pg_trickle_core` produces < 5
MB unoptimized. Document size in tracking issue.
Dependencies: PGL-1-6. Schema change: No.

**SCAL-3 — Incremental compilation impact assessment**

> **In plain terms:** Splitting into two crates changes the incremental
> compilation boundary. A change in `pg_trickle_core` now forces a
> recompile of the extension crate. Measure incremental compile time for
> common edit patterns (add a test, modify an operator, change a rewrite
> pass) and ensure developer-experience compile times remain < 30s.

Verify: document incremental compile times for 3 edit patterns.
Dependencies: PGL-1-1. Schema change: No.

### Ease of Use

| ID | Title | Effort | Priority |
|----|-------|--------|----------|
| UX-1 | Workspace-aware justfile targets | S | P0 |
| UX-2 | Developer guide for core crate contributions | S | P1 |
| UX-3 | ARCHITECTURE.md update for two-crate layout | S | P1 |

**UX-1 — Workspace-aware justfile targets**

> **In plain terms:** Existing `just` targets (`just test-unit`, `just
> lint`, `just fmt`) must work seamlessly with the new workspace layout.
> Update the justfile so `just test-unit` runs both `pg_trickle_core` unit
> tests and extension unit tests. Add `just test-core` for core-only tests.

Verify: all existing `just` targets pass; `just test-core` runs core-only
tests in < 5 seconds.
Dependencies: PGL-1-1. Schema change: No.

**UX-2 — Developer guide for core crate contributions**

> **In plain terms:** Contributors need to know the rules: what goes in
> `pg_trickle_core` (pure logic, no pgrx) vs the extension crate (SPI, FFI,
> SQL functions). Add a section to `CONTRIBUTING.md` explaining the crate
> boundary, the `DatabaseBackend` trait contract, and how to add a new
> operator to the core crate.

Verify: CONTRIBUTING.md updated with crate boundary rules.
Dependencies: PGL-1-5. Schema change: No.

**UX-3 — ARCHITECTURE.md update for two-crate layout**

> **In plain terms:** The module layout diagram in `docs/ARCHITECTURE.md`
> and `AGENTS.md` must reflect the new two-crate structure. Update both
> files so new contributors see the correct layout.

Verify: `docs/ARCHITECTURE.md` and `AGENTS.md` module diagrams show
`pg_trickle_core/` and `pg_trickle/` crates.
Dependencies: PGL-1-7. Schema change: No.

### Test Coverage

| ID | Title | Effort | Priority |
|----|-------|--------|----------|
| TEST-1 | Delta SQL snapshot tests for all 22 TPC-H queries | M | P0 |
| TEST-2 | Pure-Rust unit tests for extracted operators | L | P0 |
| TEST-3 | Mock DatabaseBackend for in-memory testing | M | P1 |
| TEST-4 | WASM build smoke test in CI | S | P0 |
| TEST-5 | Cargo deny / audit for new crate | XS | P0 |

**TEST-1 — Delta SQL snapshot tests for all 22 TPC-H queries**

> **In plain terms:** Before extraction, capture the exact delta SQL output
> for each of the 22 TPC-H stream table definitions. After extraction, run
> the same generator and diff. Any change is a hard failure. This is the
> primary correctness gate for the refactoring.

Verify: `cargo test -p pg_trickle_core -- snapshot` passes with zero diffs.
Dependencies: CORR-1. Schema change: No.

**TEST-2 — Pure-Rust unit tests for extracted operators**

> **In plain terms:** The 23 operator files currently have ~1,700 unit tests
> that run inside `cargo pgrx test` (requires PostgreSQL). After extraction,
> all pure-logic tests should run via `cargo test -p pg_trickle_core`
> without a database. Tests that require SPI (e.g., catalog lookups) stay
> in the extension crate. Audit and migrate every test that can run without
> PostgreSQL.

Verify: > 80% of existing operator unit tests run in `pg_trickle_core`
without PostgreSQL.
Dependencies: PGL-1-2, TEST-3. Schema change: No.

**TEST-3 — Mock DatabaseBackend for in-memory testing**

> **In plain terms:** For core crate tests that need to call the parser or
> SPI, provide a `MockBackend` that returns canned parse trees and query
> results. This allows testing the full pipeline (parse -> rewrite ->
> operator tree -> delta SQL) without PostgreSQL.

Verify: `MockBackend` supports at least: `raw_parser()` returning a canned
`OpTree`, and `spi_query()` returning a canned result set. 10+ tests use it.
Dependencies: PGL-1-5. Schema change: No.

**TEST-4 — WASM build smoke test in CI**

> **In plain terms:** Add a CI job that compiles `pg_trickle_core` to
> `wasm32-unknown-emscripten` on every PR. This catches platform-specific
> code leaks before they accumulate. The job does not need to run the WASM
> binary — just compile it.

Verify: CI job `build-wasm` passes on every PR targeting the core crate.
Dependencies: PGL-1-6, STAB-5. Schema change: No.

**TEST-5 — Cargo deny / audit for new crate**

> **In plain terms:** The new `pg_trickle_core` crate may introduce new
> transitive dependencies. Ensure `cargo deny check` and `cargo audit`
> cover the new crate and report no advisories.

Verify: `cargo deny check` and `cargo audit` pass for the full workspace.
Dependencies: PGL-1-1. Schema change: No.

### Conflicts & Risks

1. **STAB-1 is the critical path.** The ~500 unsafe blocks in `rewrites.rs`
   and `sublinks.rs` require a `NodeVisitor` abstraction over raw
   `pg_sys` pointer traversal. This is the highest-effort, highest-risk
   item. If the abstraction proves too leaky (e.g., too many pg_sys node
   types to wrap), consider leaving `rewrites.rs` and `sublinks.rs` in the
   extension crate and extracting only operators + DAG + types to the core
   crate. This reduces v0.29.0 scope but still delivers the WASM-compilable
   operator engine for v0.30.0.

2. **PERF-1 must be validated before merging.** Introducing a
   `trait DatabaseBackend` could add vtable dispatch overhead on the hot
   refresh path. Use monomorphization (generics) rather than `dyn Trait`
   for the extension-side implementation. If Criterion shows > 1%
   regression, investigate `#[inline]` annotations and LTO settings.

3. **No schema changes, but workspace restructuring can break `cargo pgrx`.**
   The `cargo-pgrx` tool makes assumptions about workspace layout (e.g.,
   expecting a single `lib.rs` entry point). Test `cargo pgrx package`,
   `cargo pgrx test`, and `cargo pgrx run` early. If `cargo-pgrx` 0.17.x
   cannot handle the workspace, consider upgrading to a newer pgrx that
   supports workspaces, or use a `[patch]` section in `Cargo.toml`.

4. **TEST-2 depends on TEST-3 (MockBackend).** Pure-Rust operator tests
   need a way to feed canned parse trees. Build the MockBackend early so
   TEST-2 can proceed.

5. **WASM target may not be available in standard CI runners.** The
   `wasm32-unknown-emscripten` target requires Emscripten SDK. Either
   install it in CI (adds ~2 min setup) or use a pre-built Docker image
   with the SDK. Budget for CI setup time.

6. **Extraction is all-or-nothing per module.** Partially extracting a
   module (e.g., moving half of `rewrites.rs`) creates circular
   dependencies. Each module must move completely or stay. Plan the
   extraction order: types -> operators -> DAG -> diff -> rewrites ->
   sublinks.

> **v1.3.0 total: ~3–4 weeks (extraction) + ~1–2 weeks (abstraction layer + testing)**

**Exit criteria:**
- [ ] PGL-1-1: `pg_trickle_core` crate exists as a workspace member with zero pgrx dependencies
- [ ] PGL-1-2: All operator delta SQL generation lives in the core crate
- [ ] PGL-1-3: All auto-rewrite passes live in the core crate
- [ ] PGL-1-4: DAG computation lives in the core crate
- [ ] PGL-1-5: `trait DatabaseBackend` defined; pgrx implementation passes all existing tests
- [ ] PGL-1-6: `cargo build --target wasm32-unknown-emscripten -p pg_trickle_core` succeeds
- [ ] PGL-1-7: `just test-all` passes with zero regressions
- [ ] CORR-1: Delta SQL snapshot tests pass for all 22 TPC-H queries (byte-for-byte match)
- [ ] CORR-2: OpTree serialize-deserialize round-trip passes proptest
- [ ] CORR-3: Rewrite pass ordering golden snapshot matches
- [ ] CORR-4: DAG cycle detection passes with 3 new edge-case tests
- [ ] STAB-1: Zero `pg_sys::` references in `pg_trickle_core/src/`
- [ ] STAB-2: `cargo build -p pg_trickle_core --no-default-features` passes in CI
- [ ] STAB-3: `cargo pgrx package` and `cargo pgrx test` succeed with workspace layout
- [ ] STAB-4: Extension upgrade path tested (`1.2.0 → 1.3.0`)
- [ ] STAB-5: WASM target builds in CI
- [ ] PERF-1: Criterion shows < 1% regression on `diff_operators` benchmark
- [ ] PERF-2: Full benchmark suite passes with < 5% regression threshold
- [ ] TEST-1: TPC-H delta SQL snapshot tests pass
- [ ] TEST-2: > 80% of operator unit tests run without PostgreSQL
- [ ] TEST-3: MockBackend used by 10+ core crate tests
- [ ] TEST-4: CI `build-wasm` job passes on every PR
- [ ] TEST-5: `cargo deny check` and `cargo audit` pass for workspace
- [ ] UX-1: All existing `just` targets pass; `just test-core` added
- [ ] UX-3: ARCHITECTURE.md and AGENTS.md updated with two-crate layout
- [ ] `just check-version-sync` passes

---