# Persistence Format

The `.pggraph` file is the launch artifact for fast backend startup. Format v6
stores graph-sized topology, filters, dictionaries, relationship identities,
and tenant metadata in explicitly described mmap-friendly sections. The file
is derived state; PostgreSQL source tables remain authoritative.

## File Layout

```text
Header, 512 bytes, explicitly encoded little endian
  magic                    4 bytes   "PGGH"
  version                  u32       current version = 6
  header_size              u32       512
  flags                    u32       weights and unidirectional-edge presence
  node_count               u32
  forward_edge_count       u32
  inbound_edge_count       u32
  section_count            u32       26
  body_length              u64
  body_crc32               u32
  header_crc32             u32
  reserved                 16 bytes  zero
  section_descriptors      26 * { offset: u64, length: u64 }
  reserved                 32 bytes  zero

Section 0    active bits                         ceil(node_count / 8) bytes
Section 1    table OIDs                          node_count * 4
Section 2    primary-key offsets                 (node_count + 1) * 8
Section 3    primary-key UTF-8 bytes
Sections 4-9 forward CSR                         offsets, targets, type IDs,
                                                 schema direction, optional
                                                 weights, relationship IDs
Sections 10-15 inbound CSR                       same parallel arrays
Section 16   resolution index
Section 17   filter catalog                      column descriptors and names
Section 18   filter values                       canonical typed arrays
Section 19   filter text dictionaries            lexical offsets and UTF-8 bytes
Section 20   edge-type registry
Section 21   relationship identity descriptors   mapping ID and key range
Section 22   relationship identity key bytes      UTF-8
Section 23   tenanted table OIDs                  sorted unique u32 values
Section 24   tenant dictionary                    lexical offsets and UTF-8 bytes
Section 25   per-node tenant tokens               node_count * 4
```

Every section begins at a 64-byte boundary and has an explicit offset and
length, including empty sections. Both CSR directions carry a relationship-ID
sidecar. ID `0` is reserved for adjacencies without a source-row identity.

## Write And Publication Path

Persisted build and vacuum do not construct a complete owned `Engine`. The
production path:

1. Reads the fenced PostgreSQL snapshot in bounded SPI batches.
2. Stages nodes, resolution, filters, dictionaries, tenants, both relationship
   orientations, and relationship identities in checksummed external-sort runs.
3. Merges runs with a fixed fanout under memory, disk, file-count, row, work,
   elapsed-time, and record-size limits.
4. Computes all 26 section lengths with checked arithmetic.
5. Reserves the complete candidate size, creates a generation-specific path
   with `create_new`, zero-fills it, and fsyncs the allocation.
6. Streams sequential sections and writes dense filter slots at checked final
   offsets without retaining graph-sized section buffers.
7. Recomputes the body CRC with a fixed 64 KiB buffer, backpatches the header,
   fsyncs the artifact, and atomically writes its sync and projection-mode
   sidecars.
8. Prepares an immutable generation manifest, loads the unpublished candidate
   through the production validator, and repeats the catalog, privilege, schema,
   and source-watermark checks.
9. Publishes the generation manifest and switches `projection-current.json`
   only if the generation observed before the build is still current.

The previous generation remains available for rollback and pinned readers.
Scan, run, merge, quota, write, checksum, fsync, load, and source-validation
failures remove only private staging state. After publication begins, cleanup
first verifies the current pointer: a candidate that is current, or whose state
cannot be read conclusively, is preserved so an ambiguous post-rename fsync
error cannot invalidate the serving generation. The nonpersisted build path
keeps the owned builder for backend-local use and differential testing.

## Load Path

`load_graph_file()`:

```text
open file
copy into anonymous mapping
make snapshot read-only
validate header size
validate magic
validate version
read counts and offsets
validate CRC
validate section layout
validate CSR and primary-key offset content
construct snapshot-backed NodeStore arrays
construct snapshot-backed forward and inbound EdgeStore CSR arrays
keep both CSR relationship-ID arrays snapshot-backed
keep ResolutionIndex section snapshot-backed
construct FilterIndex over mapped typed values and lexical dictionaries
construct relationship identity store over mapped descriptors and key bytes
construct tenant index over mapped lexical dictionary and dense node tokens
decode only bounded column and edge-label metadata into backend heap
set resolution mode to MmapBacked
store snapshot Arc in Engine._mmap
read sync checkpoint
return Engine
```

<Callout type="info">
The node arrays, both CSR directions, both relationship-ID sidecars, resolution
index, filter values and dictionaries, and relationship identity data are views
over one backend-local immutable mapping after load. Mutable filter changes use
sparse backend-local deltas. Relationship identities added after load use an
owned suffix overlay without copying the mapped base.
</Callout>

## Validation Boundaries

Loader validation rejects:

| Check | Failure reason |
|---|---|
| File smaller than header | Cannot read fixed header |
| Magic mismatch | Not a graph file |
| Version mismatch | Incompatible format |
| CRC mismatch | Corrupt payload |
| Section offsets out of order or out of file | Invalid layout |
| Required alignment missing | Unsafe typed slice would be invalid |
| Required section too small | Out-of-bounds typed reads |
| Weights section wrong size | Parallel arrays would diverge |
| `schema_reversed` value outside `0..=1` | Edge provenance flag invalid |
| Resolution section malformed | Lookup index invalid |
| CSR offsets not monotonic | Neighbor slices invalid |
| Final CSR offset not equal to edge count | Missing or extra edges |
| Target node index out of range | Traversal could index invalid node |
| Primary-key offsets not monotonic or out of bytes | Invalid UTF-8 slice bounds |
| First/final primary-key offsets do not span the byte section | Missing or unaddressable key bytes |
| A primary-key offset pair does not select valid UTF-8 | Invalid persisted node identity |
| Edge registry index 0 not empty | Root edge type invariant broken |
| Filter catalog, typed value width, or row count mismatch | Filter reads could cross a column boundary |
| Text dictionary offsets are invalid or values are not lexical | Token lookup would be ambiguous |
| Relationship descriptor or UTF-8 key range is invalid | Source-row identity would be ambiguous |
| Tenant OIDs, dictionary offsets/order, or node tokens are invalid | Tenant visibility would be ambiguous |

The section-layout validator and the crate-private mapped-array constructors
both enforce the value invariants used by typed access over the immutable
snapshot. This deliberate second check keeps malformed fixtures and loader call sites
from constructing a mapped store even if they bypass file parsing.

## Mmap Lifetime

The loader copies the file into an anonymous mapping and makes it read-only
before typed views are created. The private `MappedGraphArtifact` owns that
`Arc<Mmap>` and constructs node and edge stores only after the complete layout
validates. Each mapped store retains an `Arc` clone; `Engine._mmap` retains
another clone for resolution lookup and memory accounting. Safe store access
therefore cannot outlive its mapping, and source-inode writes or truncation
cannot change the snapshot.

```text
Engine
  _mmap shares ownership for resolution and accounting
  node_store owns validated ranges plus Arc<Mmap>
  forward and inbound edge stores own validated ranges plus Arc<Mmap>
  resolution mode references _mmap section
  filter_index owns mapped base ranges plus sparse heap deltas
  relationship identities own mapped base ranges plus an optional heap suffix
  tenant index owns validated mapped dictionary and token ranges
  edge_type_registry owns bounded heap strings decoded from its section
```

## Compatibility Rules

Version 6 extends the explicit v5 layout with persisted tenant-table metadata,
a lexical tenant dictionary, dense per-node tenant tokens, and a durable
unidirectional-edge flag. Version 5 and earlier artifacts are derived state and
are not read by this loader; users should run `SELECT graph.build()` to
regenerate the current artifact after upgrading. Future versions also fail
closed with the same rebuild guidance.

Any future change to fixed section ordering, binary representation, header
semantics, or persisted value interpretation should bump the format version and
provide a clear regeneration path. The current loader rejects non-current
versions with an incompatible-version error instructing users to rebuild.

## Fuzz Surface

The fuzz target `fuzz/fuzz_targets/load_graph_file.rs` exercises the loader.
Keep parser and loader checks total and non-panicking for arbitrary bytes.