# Box for the vector benchmark (pg_pinecone 0.10.0) One `r6id.4xlarge` (16 vCPU, 128 GB, 950 GB NVMe instance store) on Ubuntu 24.04 x86_64 in us-east-1 with PGDG Postgres 17 (18 on demand) and pgvector, the pg_pinecone release tarball and this harness on the same machine. The scripts in this directory provision it (`provision.sh`, `user-data.sh`, `postgresql.bench.conf`), upload the committed harness (`upload.sh`), run steps detached (`step.sh`, `step-run.sh`) and tear it down (`teardown.sh`). The resolved facts of each session (AMI, kernel, versions, settings, checksums) are in `BOX-2026-10-16-halfvec.md`, `BOX-2026-10-22-hvfilt.md` and `BOX-2026-10-22-overram.md` with their `settings-*.json`; the sessions below say which one to follow. ## Run it again Needs the AWS CLI v2 logged in (`aws sso login`) with `AWS_PROFILE` set, `BOX_OWNER` (the Owner tag), and `jq`. The Pinecone-backed phases also need a file `ee.env` holding `PINECONE_API_KEY=...` (never committed, never printed) and spend real money: the harness tallies every request and stops at 6M write units, 1M read units or $40 (`harness/remote.py`). ``` curl -fsSLO https://github.com/pinecone-io/pg_pinecone/releases/download/v0.10.0/pg_pinecone-0.10.0-pg17-linux-x86_64.tar.gz export AWS_PROFILE= BOX_OWNER= BOX_DATE= BOX_JSON=$HOME/.pg_pinecone-bench/box-.json bench/vector/box/provision.sh # key pair, security group (SSH from this IP only), instance EE_ENV=/path/to/ee.env TARBALL=$PWD/pg_pinecone-0.10.0-pg17-linux-x86_64.tar.gz bench/vector/box/upload.sh source bench/vector/box/conn.sh && bssh # a shell on the box ``` On the box (`B=~/pg_pinecone/bench`; every long step detached with `vector/box/step.sh `, log `/data/logs/.log`, state `/data/state/step-.json`): ``` $B/vector/box/install-pg_pinecone.sh # sha256-checked, into the PGDG directories $B/vector/box/setup-box.sh # env file, venv, key file mode, sar $B/vector/box/step.sh data $B/vector/box/phase.sh data # wiki_dpr: 5.3 GB of shards, 3.4 GB npz, sha256 list $B/vector/box/step.sh prep $B/vector/box/phase.sh prep # k-means and ground-truth caches of every config, no Postgres $B/vector/box/step.sh smoke $B/vector/box/phase.sh smoke # 5k-row rehearsal of the Pinecone lane ``` `phase.sh` phases (finished ones leave `/data/state/done-` and are skipped when the chain is started again): | Phase | What | Pinecone | |---|---|---| | `local_1m`, `c1`, `c2`, `ext`, `ext_model` | leg A: `local_1m_0.10.yaml`, `residual-c1-0.10.yaml`, `residual-c2-0.10.yaml`, `residual-ext-0.10.yaml`, `residual-ext-model.yaml` (its mode-L system alone) | no | | `remote_1m_build`, `remote_1m`, `remote_filtered` | `remote_1m.yaml` (build, then measure) and `remote_filtered_keep.yaml` on the same index | one 1M-row index | | `lane_am_build`, `lane_am`, `lane_topk_build`, `lane_topk` | the two 100k lane configs | two 100k-row indexes | | `partition_build`, `partition` | `remote_partition.yaml`, 100 tenants on 500k rows | one index, 100 namespaces | | `cleanup` | `pc_indexes.py delete-all --yes`: every index whose name contains `pg-pinecone-bench-vec` | deletes | A `*_build` phase loads the table, creates the index, waits for the backfill and the gate and stops, leaving the cluster under `/data/cl/pgdata-` and the index in place; the matching measure phase reuses both (`run.py --stage measure`). Backfills take hours at the default worker settings, so the build phases run beside leg A (a backfill is network-bound and uses well under one core); the measure phases run after it. ## Kernel microbench The kernel microbench (Hamming over the native sketches, 1-bit code x 4-bit query planes in scalar, AVX2 and AVX-512, record-major and 64-code blocks, read roofline, core-frequency probe) needs only a Rust toolchain (`phase.sh tools` installs one; the VM has no PMU, so `perf` runs with `-e cpu-clock`): ``` cd bench/vector/kernels && cargo run --release -- --cpu 3 ``` ## Cohere residual cells `residual-cohere-c1.yaml` and `residual-cohere-c2.yaml` on the box recipe above (Cohere wikipedia-22-12 1M, cosine), PostgreSQL 18 (copy the pg18 tarball as in the filtered halfvec session below), no Pinecone, no key file. `data_cohere` downloads the HDF5 and converts it (needs `h5py`, in `requirements.txt`). ``` $S cohere $P pg18_install data_cohere residual_cohere_c1_prep residual_cohere_c1 residual_cohere_c2_prep residual_cohere_c2 ``` (`S=$B/vector/box/step.sh`, `P=$B/vector/box/phase.sh`.) Teardown as below. ## Halfvec session `configs/halfvec_1m.yaml` (pgvector FP16 arms against FP32 hnsw and native) on the box recipe above, no Pinecone, no key file. Its native arms measure **pg_pinecone built from main** on the box, not the release tarball: the tarball is still uploaded and installed only because `upload.sh` and `setup-box.sh` expect one, and `hv_install` replaces it. The pg_pinecone commit is the `origin/main` uploaded below, stamped at run time in `/data/state/pg_pinecone-build.txt` (commit and library sha256); the run itself stamps `pg_pinecone_extversion` 0.10.0. Copy both into the session's facts file, and run `settings.py