> **Plain-language companion:** [v0.64.0.md](v0.64.0.md) ## v0.64.0 — DuckLake Ecosystem Phase 1 **Status: Released.** Derived from [ROADMAP.md](../ROADMAP.md) and [plans/ecosystem/PLAN_DUCKLAKE.md](../plans/ecosystem/PLAN_DUCKLAKE.md). > **Release Theme** > Pure documentation, demo, and community release — no changes to the extension > itself. Launches pg_trickle as a recognised first-class participant in the > DuckLake ecosystem, establishing the narrative that pg_trickle is the > incremental view maintenance engine that every PostgreSQL-backed DuckLake > deployment has been waiting for. --- ## Implementation Status | ID | Item | Type | Status | |----|------|------|--------| | T-1 | Tutorial: "Real-Time Dashboards on Your Data Lake" | Tutorial | ✅ Done | | T-2 | Tutorial: "The Modern Data Stack in One Box" | Tutorial | ✅ Done | | T-3 | Tutorial: "Monitoring Your DuckLake with pg_trickle" | Tutorial | ✅ Done | | B-1 | Blog: "Why pg_trickle + DuckLake Is the Missing Piece for Lakehouse IVM" | Blog | ✅ Done | | B-2 | Blog: "DuckLake's `table_changes()` Meets pg_trickle's DVM Engine" | Blog | ✅ Done | | D-1 | Docs: DuckLake examples in `blog/foreign-table-sources.md` | Docs | ✅ Done | | Demo-A | Demo: "The Five-Second Funnel" (`demos/ducklake-funnel/`) | Demo | ✅ Done | | Demo-D | Demo: "DuckLake Observability in a Box" (`demos/ducklake-observability/`) | Demo | ✅ Done | | C-1 | Community: Named-user outreach (PostHog, Windmill, locals.com, Ascend.io, Sliplane, Media Cluster Norway) | Community | ✅ Done (plan documented) | | C-2 | Community: DuckCon and PGConf talk submissions | Community | ✅ Done (CFP texts written) | | C-3 | Community: DuckLake GitHub Discussion + Discord engagement | Community | ✅ Done (scripts written) | --- ## Deliverables ### Tutorials #### T-1: "Real-Time Dashboards on Your Data Lake" **File:** `blog/ducklake-real-time-dashboards.md` DuckDB writes synthetic events into a DuckLake table at high throughput. pg_trickle stream tables compute per-minute and per-user aggregations over the lake data using the existing foreign-table/bridge-table path. A Grafana dashboard connected to the PostgreSQL stream-table results updates in sub-second latency. The tutorial covers setup, the stream-table SQL, and a comparison of query times before and after. #### T-2: "The Modern Data Stack in One Box" **File:** `blog/ducklake-modern-data-stack.md` A reference architecture that replaces the classic five-system Debezium-Kafka-Flink-Iceberg pipeline with a single PostgreSQL instance plus an S3 bucket. OLTP writes in PostgreSQL, pg_trickle computes real-time aggregations, DuckLake stores the history in Parquet on S3, DuckDB handles ad-hoc analytics. No Kafka, no Airflow, no JVM, no separate clusters. #### T-3: "Monitoring Your DuckLake with pg_trickle" **File:** `blog/ducklake-monitoring.md` DuckLake's ~28 metadata tables live in PostgreSQL and are rich with operational signals. This tutorial builds a set of stream tables that deliver real-time operational visibility: snapshot rate per minute, tables with many small files (compaction candidates), per-tenant write activity for multi-tenant deployments, and storage growth curves. Includes a Grafana dashboard definition. --- ### Blog Posts #### B-1: "Why pg_trickle + DuckLake Is the Missing Piece for Lakehouse IVM" **File:** `blog/ducklake-ivm-missing-piece.md` Thought-leadership post aimed at Hacker News, /r/dataengineering, and the dbt Slack. Frames the lakehouse-IVM gap, shows why existing solutions (Materialize, RisingWave, Flink) require separate infrastructure, and explains how pg_trickle fills the gap from inside the catalog database — the infrastructure DuckLake already requires. Links to DuckLake's own roadmap item as the anchor. #### B-2: "DuckLake's `table_changes()` Meets pg_trickle's DVM Engine" **File:** `blog/ducklake-table-changes-dvm.md` Technical deep-dive for the systems-programming audience. Explains DuckLake's change-feed API (`table_changes()`, `table_insertions()`, `table_deletions()`), maps its output format row-by-row to pg_trickle's internal change-buffer schema, and discusses why DuckLake's stable `rowid` column makes it one of the cleanest CDC sources imaginable for differential IVM. --- ### Documentation #### D-1: DuckLake Examples in `foreign-table-sources.md` **File:** `blog/foreign-table-sources.md` (updated with DuckLake section) Added a dedicated "DuckLake Sources" section to the existing foreign-table- sources reference page. Covers: 1. Bridge-table pattern (works today, sub-millisecond latency via triggers) 2. DuckLake metadata tables as stream table sources (for monitoring) 3. Foreign-table polling path (for large DuckLake tables) 4. Preview of the Phase 2 native `table_changes()` adapter (v0.65.0) --- ### Demos #### Demo A: "The Five-Second Funnel" **Directory:** `demos/ducklake-funnel/` A self-contained `docker compose up` demo. Containers: - PostgreSQL 18 + pg_trickle (stream tables pre-installed) - Python load generator (50 events/second, configurable) - Grafana (dashboard auto-provisioned) - MinIO (S3-compatible object storage, for the DuckLake data path) Stream tables: - `revenue_by_minute` — revenue aggregated by minute and product - `funnel_by_product` — view/cart/purchase funnel with conversion rates Designed to be shared at conference talks and on social media; runs on any developer laptop. #### Demo D: "DuckLake Observability in a Box" **Directory:** `demos/ducklake-observability/` A companion to Tutorial T-3. Connects to an **existing** DuckLake PostgreSQL catalog. Two components: 1. `init_monitoring.sql` — installs five stream tables over DuckLake metadata 2. `docker compose up grafana` — starts a pre-provisioned Grafana instance Five minutes from `git clone` to a production-quality Grafana observability dashboard over any DuckLake PostgreSQL catalog. --- ### Community #### C-1: Named-User Outreach **Document:** `docs/DUCKLAKE_COMMUNITY_OUTREACH.md` Tailored outreach plan for six named DuckLake production users: PostHog, Windmill, locals.com, Ascend.io, Sliplane, and Media Cluster Norway. Includes personalised pitch angles, contact channel recommendations, and a tracking table. #### C-2: Conference Talk Submissions **Document:** `docs/DUCKLAKE_COMMUNITY_OUTREACH.md` (C-2 section) Complete CFP submission texts for: - DuckCon: "pg_trickle: The IVM Engine DuckLake's Roadmap Is Asking For" - PGConf EU: "Sub-second materialized views on your data lake" - PGCon: "Differential Dataflow Inside PostgreSQL" #### C-3: DuckLake GitHub Engagement **Document:** `docs/DUCKLAKE_COMMUNITY_OUTREACH.md` (C-3 section) GitHub Discussion draft for `duckdb/ducklake` repository proposing to co-author a reference architecture page. DuckDB Discord post template. --- ## Extension Changes None. v0.64.0 is a zero-code-change release. The extension binary, SQL schema, GUCs, and catalog tables are identical to v0.63.0. The upgrade script `sql/pg_trickle--0.63.0--0.64.0.sql` contains only documentation comments. --- ## Exit Criteria - [x] All 11 deliverables created and reviewed - [x] Tutorial T-1 includes working SQL code samples - [x] Tutorial T-2 includes the system comparison table - [x] Tutorial T-3 includes all five monitoring stream tables - [x] Blog B-1 references DuckLake's own roadmap IVM item - [x] Blog B-2 includes the change-type → pg_trickle weight mapping table - [x] `foreign-table-sources.md` has DuckLake section with three code paths - [x] Demo A has docker-compose.yml, init.sql, generator, and Grafana config - [x] Demo D has docker-compose.yml, init_monitoring.sql, teardown script, and Grafana config - [x] Community outreach document has per-company pitch angles - [x] CFP submission texts are complete and ready to submit - [x] Version bumped to 0.64.0 in Cargo.toml and META.json - [x] SQL upgrade script `sql/pg_trickle--0.63.0--0.64.0.sql` created - [x] CHANGELOG.md updated with v0.64.0 entry - [x] ROADMAP.md updated to mark v0.64.0 as Released