# Corpus gate

A standing differential and plan regression for the aggregate and column paths: `cases.sql` holds 380 statements (the shapes of the pushdown families, one `-- @case slug | family | note` marker each) that run on the lab of `lab.sql` (20000 events and the tables around them, deterministic: `setseed(0.42)`).

Every statement runs twice, each time in a **new backend** (`\connect`, so no cache, relcache entry or memory context of an earlier statement is warm):

1. sift off (`sift.enable_*=off`): the native planner;
2. sift on.

It fails when

- the sorted rows differ (`extra_float_digits=-5`), or a statement errors;
- the **plan slug** is not the pinned one in `plans.tsv`: the Custom Scan (Sift...) nodes of `explain (costs off)` and the verdict of `sift.explain_why` per arm (`t` used, `f:SLUG` refused). A statement that stops engaging, starts engaging, or is refused for another reason is a failure. Changing the set is allowed with `scripts/corpus.sh --update` **in the same commit**: read the diff of `plans.tsv`, a statement that left the engaged set is a regression unless the change says so.

A node `~` in `plans.tsv` leaves a statement whose engagement is a cost call that differs across majors unpinned (rows are still compared); `--update` keeps it.

## Run

    scripts/corpus-local.sh                # one command: installs the library, starts a throwaway cluster, runs the corpus, removes the cluster
    scripts/corpus.sh                      # the same on the cluster of scripts/pgtest.sh (SIFT_TMP, SIFT_PORT; installed and started), as the CI job does
    scripts/corpus.sh --only 'agg_*'       # a focused run (the statements whose slug matches; the lab is loaded as ever)
    scripts/corpus.sh --update             # re-record plans.tsv
    CORPUS_ON_SET='set sift.agg_dense=off;' scripts/corpus.sh   # try a setting: what it does to rows and plans

On an installed cluster it takes **about 2 s** (lab load 1 s, statements 1 s, PostgreSQL 18); `corpus-local.sh` adds the build and install (about 40 s the first time). The CI job is `corpus` (every major; the pinned plans are the same on 17, 18 and 19 apart from the `~` rows).

## Add a statement

Append a `-- @case` block to `cases.sql` (the header of the file lists the markers: `@pre` setup of the session, `@txn` a transaction rolled back after, `@K@`/`@K2@` the entity keys), run `scripts/corpus.sh --update`, and commit `cases.sql` with the new line of `plans.tsv`.