# Serving-Memory Model Decision (2026-07-16)

Status: **decided — hybrid shared base plus backend-local delta.** This decision record captures why the hybrid model was chosen on
2026-07-16, after reviewing the measurements below.

## The problem being decided

The HNSW engine serves warm traversals from a packed graph generation: an
immutable, contiguous in-memory image of the graph that each backend builds
from the authoritative PostgreSQL index pages and reuses until the metapage
publication identity changes. Packing is what makes single-client latency
Pareto-dominant over pgvector — and it is backend-local, which is the root
cause of every negative result in the July 2026 benchmark suite.

## Measurements forcing the decision

All from clean commits on Apple M4 Pro, PostgreSQL 17.9, 100k×384 synthetic
corpus (archived under `benchmarks/pgvector_comparison/results/`):

| Failure mode | Evidence | Artifact commit |
|---|---|---|
| Memory × connections | 32 clients hold 8.4 GiB of packed generations total (~260 MiB each) vs pgvector's 4.4 GiB shared-buffer footprint | `86330dcd` |
| Throughput inversion | Aggregate QPS falls 758 → 539 from 8 → 32 clients while pgvector scales 1,741 → 2,298 (4.3× pgContext) | `86330dcd` |
| Cold start | First query after restart: 553 ms vs 0.42 ms steady (~1,300×); every backend repacks on first touch | `847e0581` |
| Invalidation repack | First query after a churn round: 557 ms; every insert/update batch invalidates every backend's pack | churn round-1 observation, 2026-07-16 |

## The decision

**Hybrid: a shared read-mostly packed generation plus a small backend-local
delta.**

- The packed base generation lives once in dynamic shared memory under a
  global budget, so resident memory stops scaling with connections and cold
  start packs once per server, not once per backend.
- Incremental changes apply as a bounded backend-local (or shared,
  implementation's choice) delta instead of invalidating the whole pack, so
  churn stops causing full repack storms.
- Delta compaction folds accumulated changes back into a new shared base
  generation on a policy trigger (size ratio or invalidation count).

Rejected alternatives, for the record:

- *Fully shared DSM generations* solves the same three failure modes but
  couples every invalidation to shared-memory lifecycle and locking on the
  hot path; the delta tier exists precisely to keep invalidations cheap.
- *Budgeted per-backend LRU* is the smallest change but retains memory
  duplication and per-backend cold cliffs, which the measurements show are
  the dominant production risks.

## Constraints the implementation must keep

1. PostgreSQL index pages and WAL remain the only authoritative graph state;
   packed bases and deltas are rebuildable caches.
2. Over-budget or unavailable shared memory degrades to page-native
   traversal — never an error, never a silent exact scan.
3. Cache behavior is observable: SQL-visible counters for base packs, delta
   applications, compactions, and fallbacks (shared-registry telemetry).
4. The churn, cold-cache, and concurrency lanes in
   `benchmarks/pgvector_comparison/` are the acceptance tests: gates G3 and
   G4 close on their artifacts, not on design intent.