> **Plain-language companion:** [v0.41.0.md](v0.41.0.md)

## v0.41.0 — Embedding Pipeline Infrastructure & ANN Maintenance

**Status: Planned.** Derived from [plans/ecosystem/PLAN_PGVECTOR.md](plans/ecosystem/PLAN_PGVECTOR.md) §6.2, rescheduled after the assessment-driven hardening arc in [plans/PLAN_OVERALL_ASSESSMENT_8.md](plans/PLAN_OVERALL_ASSESSMENT_8.md).

> **Release Theme**
> v0.41.0 resumes the deferred embedding programme once the roadmap has closed
> the immediate correctness and operational gaps. It adds post-refresh action
> hooks (ANALYZE, REINDEX, drift-based re-clustering), vector-aware monitoring,
> pgai integration material, and a bundled development image for production
> embedding pipelines on PostgreSQL.

> **Hard prerequisite:** v0.38.0-v0.40.0 must complete their hardening exit
> criteria first. The embedding programme is intentionally sequenced after the
> correctness and operator-trust work, not in parallel with it.

---

### Features

| ID | Title | Effort | Priority |
|----|-------|--------|----------|
| VP-1 | `post_refresh_action` ST option (`none` / `analyze` / `reindex` / `reindex_if_drift`) | M | P1 |
| VP-2 | `reindex_drift_threshold` ST option plus drift counter in catalog | M | P1 |
| VP-3 | `pgtrickle.vector_status()` view: embedding lag, ANN age, drift % | S | P1 |
| VP-4 | Cookbook: always-fresh RAG with pg_trickle + pgvector + pgai | S | P2 |
| VP-5 | Docker image variant: `pg_trickle + pgvector + pgai` | S | P2 |
| VP-6 | Citus vector-aggregate cases folded into distributed coverage | M | P2 |

**VP-1 — Post-refresh actions.**
Add a `post_refresh_action` option to vector-heavy stream tables so operators can
run `ANALYZE`, `REINDEX`, or `reindex_if_drift` after a refresh commits. The
work runs in a lower-priority queue after the refresh transaction, so it does
not lengthen the critical refresh window or turn ANN maintenance into hidden
refresh latency.

**VP-2 — Drift detection.**
Catalog state tracks `rows_changed_since_last_reindex` and a
`reindex_drift_threshold`. When a vector table changes enough since its last
index rebuild, the post-refresh action can queue a targeted REINDEX instead of
forcing operators to guess when ANN quality has drifted too far.

**VP-3 — Vector status view.**
Add `pgtrickle.vector_status()` to expose embedding lag, last refresh, last
reindex time, rows changed since reindex, drift percentage, and vector-aggregate
usage. This view plugs into the monitoring foundation built in the hardening
arc and gives RAG operators a first-class place to look for ANN maintenance
pressure.

**VP-4 — pgai cookbook.**
Publish `docs/tutorials/PGVECTOR_RAG_COOKBOOK.md` with copy-paste workflows for
pre-computed embeddings, denormalized corpora, hybrid search, and operational
sizing guidance. The key outcome is that AI engineers can start with a working
pattern instead of reverse-engineering examples from tests.

**VP-5 — Docker image.**
Ship a bundled development image containing `pg_trickle`, `pgvector`, and
`pgai` so the cookbook and demo workflows run with one command. Keep version
pinning explicit to avoid silent breakage from upstream minor releases.

**VP-6 — Citus vector-aggregate cases folded into distributed coverage.**
The v0.39 Citus rig becomes the place where shard-additive vector-aggregate
behavior is verified. This keeps vector work honest in distributed deployments
without treating Citus as an afterthought.

### Test Coverage

| ID | Title | Effort | Priority |
|----|-------|--------|----------|
| T-VP1 | Drift-based reindex integration test on HNSW stream table | M | P1 |
| T-VP2 | `vector_status()` accuracy and reset behavior | S | P1 |
| T-VP3 | Citus shard-additive vector aggregate verification | M | P2 |

**T-VP1.**
Create a vector stream table with `post_refresh_action = 'reindex_if_drift'`,
change more than the configured threshold, and verify that REINDEX is queued and
completed outside the refresh transaction.

**T-VP2.**
Verify that `pgtrickle.vector_status()` reports lag, drift, and last-reindex
metadata correctly before and after a drift-triggered REINDEX.

**T-VP3.**
Run vector-aggregate cases inside the v0.39 Citus rig so shard-local partial
results still converge to the same answer as a global `avg(vector)` or
`sum(vector)` computation.

### Conflicts & Risks

- **VP-1/VP-2** add queueable post-refresh work. Backlog visibility must be
  clear so operators can see if ANN maintenance is lagging behind the refresh
  pipeline.
- **VP-3** is observational, but large fleets still need scrape-friendly
  monitoring rather than expensive ad hoc queries across huge numbers of stream
  tables.
- **VP-5** depends on careful dependency pinning; the image should be treated as
  a reproducible bundle, not a floating latest tag.

### Exit Criteria

- [ ] VP-1: `post_refresh_action` is stored in the catalog and dequeued correctly after refresh commit
- [ ] VP-2: drift threshold logic increments and resets correctly over repeated refresh cycles
- [ ] VP-3: `pgtrickle.vector_status()` reports correct lag and drift for vector stream tables
- [ ] VP-4: RAG cookbook published with multiple worked examples
- [ ] VP-5: bundled image builds and passes extension smoke tests
- [ ] VP-6: vector-aggregate cases pass inside the Citus distributed test rig
- [ ] Extension upgrade path tested (`0.40.0 → 0.41.0`)
- [ ] `just check-version-sync` passes

---
