# Build Pipeline

Build orchestration validates a checked byte budget before the builder turns
registered PostgreSQL tables into an `Engine`. The builder avoids retaining all
raw edges in Rust memory by using PostgreSQL temporary spool tables and bounded
SPI cursor batches.

## PostgreSQL Source Boundary

Before reading graph registrations, the build takes `SHARE ... NOWAIT` locks on
the graph catalog in a fixed order. It then resolves registered relation OIDs
to partition roots, locks each root, discovers descendants while attachment is
blocked, and locks descendants in OID order. Caller-owned write locks are
rejected in production builds so an uncommitted source change cannot escape
through a filesystem artifact.

After sync-trigger reconciliation, the exclusive writer barrier defines
watermark `W0`. Catalog identity, source schema, caller privileges, and the
maximum sync-log ID are checked again before persistence and installation; a
build is publishable only when the catalog fingerprint is unchanged and
`W1 == W0`.

## Inputs

The build receives:

| Input | Rust type |
|---|---|
| Registered tables | `Vec<RegisteredTable>` |
| Registered edges | `Vec<RegisteredEdge>` |
| Registered filter columns | `Vec<RegisteredFilterColumn>` |

The SQL facade and catalog modules validate registrations before they are
stored. The builder still keeps defensive checks for malformed or drifting
inputs.

## Pipeline

<ProcessTree title="build_graph(tables, edges, filters)">
  <ProcessStep title="parse graph.build_scan_mode" />
  <ProcessStep title="resolve serving, safety, and replacement byte budgets" />
  <ProcessStep title="create pg_temp.graph_build_nodes" />
  <ProcessStep title="for each registered table">
    - open SPI cursor
    - fetch graph.build_batch_size rows
    - assign node_idx
    - append NodeStore row
    - write node lookup batch
    - insert resolution entry
    - collect filter and tenant values
  </ProcessStep>
  <ProcessStep title="index/analyze node lookup spool" />
  <ProcessStep title="register FilterIndex columns" />
  <ProcessStep title="write collected filter values" />
  <ProcessStep title="finalize ResolutionIndex" />
  <ProcessStep title="create pg_temp.graph_build_edges" />
  <ProcessStep title="for each registered edge">
    - open SPI cursor
    - fetch endpoint rows
    - resolve endpoints through node lookup spool
    - write directed edge rows to edge spool
  </ProcessStep>
  <ProcessStep title="stream sorted edge spool into CSR builder" />
  <ProcessStep title="build reverse CSR" />
  <ProcessStep title="mark engine built" />
</ProcessTree>

<TreeDiagram>
  <TreeNode title="Catalog rows">
    <TreeNode title="Memory preflight">
      <TreeNode title="Node cursor reads">
        <TreeBranch>
          <TreeNode title="NodeStore + ResolutionIndex">
            <TreeNode title="Finalize resolution" />
          </TreeNode>
          <TreeNode title="pg_temp.graph_build_nodes">
            <TreeNode title="Edge endpoint resolution">
              <TreeNode title="pg_temp.graph_build_edges">
                <TreeNode title="SortedEdgeStoreBuilder">
                  <TreeNode title="forward CSR">
                    <TreeNode title="reverse CSR">
                      <TreeNode title="Engine" />
                    </TreeNode>
                  </TreeNode>
                </TreeNode>
              </TreeNode>
            </TreeNode>
          </TreeNode>
        </TreeBranch>
      </TreeNode>
    </TreeNode>
  </TreeNode>
</TreeDiagram>

## Governed Build Runs

The persisted-build run layer uses a manual little-endian record format with a
128-byte checksummed header. Every run is bound to one graph ID, build ID,
catalog fingerprint, and data kind: nodes, outbound relationships, inbound
relationships, filters, filter dictionaries, or resolution entries. Readers
verify the header, exact file length, complete payload checksum, record bounds,
and sorted order before returning the first record.

Collectors charge rows, bytes, disk, files, elapsed time, and interrupt checks.
Runs are merged with a fixed fanout and one decoded record per input. Merge
outputs are validated and fsynced before inputs are released. Workspaces use
mode `0700`, run files use `0600`, partial files are removed on failure, and
abandoned workspace cleanup accepts only a well-formed build directory name
while the graph build lock is held.

## Build Scan Mode

`graph.build_scan_mode` supports:

| Value | Current behavior |
|---|---|
| `select` | SPI cursor batches; implemented |
| `copy` | Reserved; current code returns a clear error |

Do not document or implement caller-facing COPY behavior until the code owns a
safe, tested server-side COPY reader path.

## Node Loading

For each registered table:

1. Resolve table OID.
2. Build a primary-key SQL expression.
3. Include registered filter columns and tenant column in the SELECT list.
4. Open an SPI cursor.
5. Fetch up to `graph.build_batch_size` rows per batch.
6. Append each row to `NodeStore`.
7. Buffer `(table_oid, primary_key, node_idx)` rows into
   `pg_temp.graph_build_nodes`.
8. Add resolution entry.
9. Collect filter values for later `FilterIndex` initialization.
10. Add tenant membership if the table is tenanted.

Composite primary keys use:

```text
jsonb_build_array(col1::text, col2::text, ...)::text
```

## Filter Loading

Filter columns are registered after all nodes are loaded because storage choice
depends on `node_count` and populated count. Values collected during node
loading are then encoded and written to `FilterIndex`.

Encoding examples:

| Column type | Build expression or conversion |
|---|---|
| `numeric` | `column::bigint` |
| `boolean` | `column::boolean` |
| `text` | `column::text`, then intern dictionary token |
| `date` | `(column::date - DATE '2000-01-01')::bigint` |
| `timestamptz` | `EXTRACT(EPOCH FROM column::timestamptz) * 1000000` |
| `uuid` | text to canonical `u128` |

## Edge Loading

For each registered edge:

1. Register static label or prepare dynamic `label_column`.
2. Determine endpoint expression style.
3. Read endpoint keys, optional weight, optional dynamic label.
4. Resolve endpoints through `pg_temp.graph_build_nodes`.
5. Add one directed edge, and a reverse directed edge when `bidirectional`.
6. Flush to `pg_temp.graph_build_edges` in bounded batches.

Then `load_edge_store_from_spool()` streams:

```sql
SELECT source, target, type_id, weight
FROM pg_temp.graph_build_edges
ORDER BY source, target, type_id
```

into `SortedEdgeStoreBuilder`.

## Relationship multiplicity

CSR construction retains every registered source relationship row. Two rows
with the same endpoints and label remain separate relationships; applications
that want a set of projected results can request `DISTINCT` in their query.
The projection does not use endpoint/type topology as a substitute for a
source-row identity.

## Build Completion

At the end of `build_graph()`:

| Field | Set to |
|---|---|
| `engine.edge_store` | Forward CSR from edge spool |
| `engine.reverse_edge_store` | Reverse CSR built from forward CSR |
| `engine.built` | `true` |
| `engine.is_read_only` | `false`; an over-budget build is rejected before construction |
| `engine.last_build` | Current PostgreSQL timestamp |

Persistence is orchestrated outside `builder.rs` by `sql_build.rs`.

The build governor remains alive through optional persistence and reload.
Persistent engine growth and transient node, edge, endpoint-resolution, and
serialization work use separate RAII leases. The spool batches retain the
configured row ceiling but flush earlier when their measured strings and array
elements reach the adaptive byte target. Major Rust vectors call
`try_reserve` after the policy lease is accepted so allocator failure is
reported as `PG001` instead of panicking.

## Failure Modes

| Failure | Error |
|---|---|
| Estimated peak bytes exceed the configured memory limit | `GraphError::Oom` before engine construction |
| COPY scan mode selected | `GraphError::Internal` with reserved-mode message |
| SPI cursor/read failure | `GraphError::Internal` |
| Edge endpoint cannot be resolved | Edge row is skipped |
| Edge label count exceeds limit | `GraphError::EdgeTypeLimit` |
| Invalid UUID filter value | `GraphError::InvalidFilter` |

Skipping unresolved edge rows is intentional: source relationships that point to
unregistered or missing nodes cannot become graph edges.