-- pg_turbovec v1.17.0 -- Phase F-2: persistent ColBERT token index. -- -- This file is a *reference mirror*. The authoritative install -- script is generated by `cargo pgrx schema`. -- -- v1.17.0 makes ColBERT late interaction INDEX-NATIVE: a persistent -- on-disk token index, replacing F-1's per-call backend-cache build. -- -- CREATE INDEX ON docs USING turbovec (tokens vec_colbert_ops); -- -- tokens is a turbovec.vector[] column (per-doc token arrays) -- -- SELECT id, score FROM turbovec.colbert_search( -- 'docs'::regclass, 'id', 'tokens', :q_tokens, 10); -- -- now reads stage-1 from the persistent index (no per-call build; -- -- the F-1 ~28 MB/call leak is eliminated on this path) -- -- New SQL object: the vec_colbert_ops operator class over -- turbovec.vector[] (support function max_sim, NO order-by operator -- -- so the planner can never select it for ORDER BY; reads go through -- turbovec.colbert_search). cargo pgrx schema emits the CREATE -- OPERATOR CLASS into the authoritative 1.16.0--1.17.0 upgrade script; -- this reference mirror does not duplicate it. -- -- WIRE FORMAT: additive bump 4 -> 5, strictly per index kind. -- * A single-vector index (vec_cosine_ops / vec_ip_ops / vec_l2_ops -- / vec_l1_ops over a `vector` column) STILL emits wire version 4 -- with a zeroed `kind` byte (page offset 30, formerly reserved). -- Its relfile is BYTE-IDENTICAL to v1.16.0 (verified by tests). -- * Only an index built `USING turbovec (col vec_colbert_ops)` over -- a `vector[]` column is v5 (kind = KIND_COLBERT) -- a new shape -- (per-token slots, doc TID repeated in the ids chain), IVF -- (cell-contiguous tokens), tombstone-on-VACUUM. -- -- MIGRATION: NO REINDEX. A v4 single-vector index decodes under the -- v5 binary as kind = KIND_SINGLE (the kind byte was a reserved zero -- on v4), so is_legacy_v4() never trips. There is no in-place -- migration of a v4 single-vector index into a v5 ColBERT index; a -- ColBERT index is built fresh. `ALTER EXTENSION pg_turbovec UPDATE -- TO '1.17.0';` registers the new opclass and is sufficient.