# Architecture

This page is the contributor-level map of the `graph` crate. It explains the
crate graph, data flow, ownership model, lifetimes, and the design choices that
shape the implementation.

For SQL behavior, start with the [User Guide](/user_guide). This page assumes
you are changing Rust code or reviewing how the extension works inside a
PostgreSQL backend.

## System Shape

<ArchDiagram>
  <ArchGroup label="PostgreSQL Backend Process">
    <ArchRow>
      <ArchCard
        title="SQL Interface"
        subtitle="pgrx functions in the graph schema"
        tags={['graph.traverse()', 'graph.search()', 'graph.build()', 'graph.auto_discover()']}
      />
      <ArchCard
        title="Background Workers"
        bullets={[
          'asynchronous build worker',
          'asynchronous maintenance worker',
          'database/user metadata passed through worker extra',
        ]}
      />
    </ArchRow>

    <ArchArrow />

    <ArchGroup label="Graph Engine (Rust)" variant="engine">
      <ArchCard
        title="SQL Facade"
        subtitle="argument parsing, ACL/admin checks, SQLSTATE reporting, hydration"
        wide
      />
      <ArchRow>
        <ArchCard title="NodeStore" subtitle="SoA node metadata and active bits" />
        <ArchCard title="EdgeStore" subtitle="forward CSR adjacency" />
        <ArchCard title="Reverse EdgeStore" subtitle="owned inbound CSR" />
      </ArchRow>
      <ArchRow>
        <ArchCard title="ResolutionIndex" subtitle="table+primary key to node_idx" />
        <ArchCard title="FilterIndex" subtitle="typed sparse/dense traversal filters" />
        <ArchCard title="SQL-backed Search" subtitle="source-table predicates through SPI" />
      </ArchRow>
      <ArchCard
        title="Safety Layer"
        variant="safety"
        wide
        bullets={[
          'bounded traversal: depth, visited nodes, and frontier limits',
          'memory guard before build allocations',
          'pgrx/PostgreSQL error boundary maps GraphError to SQLSTATE',
          'owning validated mmap ranges before typed store reads',
        ]}
      />
    </ArchGroup>

    <ArchArrow />

    <ArchCard
      title="Persistence Layer"
      subtitle=".pggraph files under $PGDATA/<graph.data_dir>; atomic temp-write, fsync, rename, mmap load"
      variant="persistence"
      wide
    />
    <ArchCard
      title="Configuration Catalog"
      subtitle="graph._registered_tables, graph._registered_edges, graph._registered_filter_columns"
      variant="config"
      wide
    />
    <ArchCard
      title="Source Tables"
      subtitle="ordinary PostgreSQL tables remain authoritative"
      variant="source"
      wide
    />
  </ArchGroup>
</ArchDiagram>

## Runtime Ownership

Each PostgreSQL backend owns its own `Engine` in thread-local storage:

```rust
thread_local! {
    static ENGINE: RefCell<Engine> = RefCell::new(Engine::new());
}
```

That means there is no shared Rust heap between connections. Sharing happens
only through PostgreSQL storage, the filesystem, and OS page-cache backed mmap
pages.

Each backend also tracks one loaded graph slot beside the engine. The slot is
tagged with the graph id/name and residency that produced the active engine, so
runtime calls can reject or clear a stale engine before serving a different
selected graph. `graph.select_graph()` changes the session selection,
`graph.load_graph()` loads one persisted artifact into the backend-local slot,
and `graph.loaded_graphs()` reports zero or one row for the current backend.
`graph.graph_runtime_status()` combines the slot with named-graph metadata and
artifact checks for operator-facing residency/status inspection.

Compatibility SQL calls such as `graph.add_table()`, `graph.build()`,
`graph.status()`, `graph.traverse()`, and `graph.reset()` resolve the selected
graph, or the default graph when no graph has been selected. The default graph
behavior remains available while named graph identity is explicit in catalogs,
jobs, artifacts, runtime loading, sync replay, and projection metadata.

The default graph identity is intentionally explicit in the Rust policy layer:
the compatibility graph is named `default`, uses the `public` graph namespace,
and has a reserved canonical UUID. Future catalog rows use PostgreSQL `uuid`
values, while Rust passes graph identity as a typed value instead of deriving it
from paths, backend globals, or session state.

| Object | Owner | Lifetime |
|---|---|---|
| `Engine` | One PostgreSQL backend process | Backend lifetime, graph selection change, explicit unload, or `graph.reset()` |
| `ResourceGovernor` | One governed preflight or operation scope | Tracks checked reservations and elapsed budget for that scope |
| Loaded graph slot | One PostgreSQL backend process | Tracks the graph id/name for the active `Engine` |
| `NodeStore` owned mode | `Engine` heap | Mutable build/sync lifetime |
| `NodeStore` mapped mode | Validated ranges with an `Arc` to the immutable snapshot | The store owns its mapping independently of engine field/drop order |
| Forward and inbound `EdgeStore` mapped mode | Validated ranges with an `Arc` to the immutable snapshot | Each store owns its mapping independently of engine field/drop order |
| `FilterIndex` after load | Mapped base plus `Engine` heap deltas | Graph-sized typed values and lexical dictionaries stay mapped; post-load changes are sparse |
| `edge_type_registry` after load | `Engine` heap | Bounded labels are decoded from their v6 section |
| Relationship identities after load | Mapped base plus `Engine` heap suffix | Source identity descriptors and keys stay mapped; later identities extend them without copying the base |
| `ResolutionIndex` mapped mode | Byte range inside the immutable snapshot | The engine retains an `Arc` for resolution lookup and accounting |
| Durable base generations, manifests, segments, and chunks | Files in each graph's artifact root | Shared through the filesystem and retained by graph-scoped active-generation metadata |
| `edge_buffer` and `resolution_delta` | `Engine` heap | Backend-local compatibility sync overlay state |
| `TxGraphDelta` | Current backend transaction | Transaction-local read-your-own-writes state only |

## Multi-Graph Architecture

The system supports multiple isolated named graphs within the same database. This architecture is implemented across all layers:

| Area | Current behavior |
|---|---|
| Registration catalogs | `graph._registered_tables`, `graph._registered_edges`, and `graph._registered_filter_columns` are keyed by `graph_id`; compatibility SQL resolves the selected or default graph before reading or writing them. |
| Runtime | `ENGINE` stores one backend-local graph engine tagged by a loaded graph slot. `ensure_current_graph()` validates that the loaded slot matches the selected graph before query execution. |
| Build and hosted jobs | `graph._build_jobs`, `graph._maintenance_jobs`, `graph._jobs`, `graph._job_runs`, and `graph._sync_policies` are keyed by `graph_id`; workers and hosted schedulers restore database, user, and graph context before running. |
| Persistence | Each graph uses `$PGDATA/<graph.data_dir>/<graph_id>/` as its artifact root. `main.pggraph` is the logical compatibility path; current persisted builds serve the generation-specific base selected by `projection-current.json` and its manifest. |
| Sync | `graph._sync_log` and `graph._sync_buffer` are global source-table streams; replay, status, and query freshness filter `_sync_log` through the selected graph's registered table OIDs. |
| Projection generations | `graph._projection_generations` is keyed by `graph_id` so active-generation heartbeats and GC protection are graph-local. |
| Discovery | `graph.auto_discover()` and `graph.auto_discover_tables()` target a named graph or the selected/default graph. |
| Reset | `graph.reset()` clears the backend-local engine and removes files for the selected graph's artifact root. |

Named-graph changes moved each of these components from implicit global state to explicit graph-scoped inputs while preserving the default-graph SQL workflow for backward compatibility.

## Data Flow: Build

<PipelineDiagram>
  <PipelineStep title="graph catalog" />
  <PipelineStep arrowLabel="read registered tables, edges, filters" title="SPI cursor batches over source tables">
    - bounded node and edge batches
    - filter and resolution records
    - tenant and relationship-identity records
  </PipelineStep>
  <PipelineStep title="checksummed external-sort runs">
    - bounded merge fanout
    - governed memory, disk, work, and elapsed time
  </PipelineStep>
  <PipelineStep title="streamed immutable v6 base candidate" />
  <PipelineStep title="production loader and source revalidation" />
  <PipelineStep title="atomic current-manifest switch" />
  <PipelineStep title="mmap-backed active Engine" />
</PipelineDiagram>

This is the persisted build and vacuum path. It does not construct a complete
owned replacement graph: graph-sized ordering is performed through bounded
runs, and the candidate is streamed directly into its final mmap-ready layout.
The nonpersisted build path retains the owned builder for backend-local use.
Query paths do not allocate or sort graph-sized state.

## Data Flow: Load

`load_graph_file()` validates the file before constructing any mmap-backed
store:

<PipelineDiagram>
  <PipelineStep title="resolve projection-current.json and its manifest" />
  <PipelineStep title="open the selected generation base, or the compatibility fallback" />
  <PipelineStep title="mmap read-only" />
  <PipelineStep title="validate v6 header, both CRCs, 26 descriptors, CSR, PK offsets, filters, identities, and tenants">
    - mmap-backed NodeStore arrays
    - mmap-backed forward and inbound EdgeStore arrays and relationship IDs
    - mmap-backed ResolutionIndex section
    - mmap-backed FilterIndex values and lexical dictionaries
    - mmap-backed relationship identities
    - bounded decoded column and edge-label metadata
  </PipelineStep>
  <PipelineStep title="Engine owns _mmap and all derived stores" />
</PipelineDiagram>

The graph-sized base representation is mmap-backed in both directions. Sparse
filter deltas, relationship identity suffixes, and other mutable overlays remain
backend-local heap allocations.

## Data Flow: Query

<PipelineDiagram>
  <PipelineStep title="SQL call">
    - check graph.enabled
    - auto-load persisted graph if needed
    - validate call options
    - ACL/admin check where required
  </PipelineStep>
  <PipelineStep title="engine call">
    - resolve seed table+PK to node_idx
    - select forward or reverse CSR
    - run bounded traversal/path/search algorithm
    - apply filters, tenants, overlays, pagination
  </PipelineStep>
  <PipelineStep title="graph coordinates" />
  <PipelineStep arrowLabel="optional SPI hydration from source tables" title="SQL rows" />
</PipelineDiagram>

Graph algorithms operate on compact node indexes. SQL-facing functions translate
between PostgreSQL coordinates and those internal indexes.

## Sync And Maintenance Flow

Trigger sync records source-table changes in `graph._sync_log`. Topology query
functions apply pending trigger-sync rows for the selected graph by default up
to a captured high-water mark for that graph's registered table OIDs; operators
can set `graph.query_freshness = 'off'` for compatibility/manual catch-up or
`error_on_pending` to fail instead of reading stale topology.
Persisted `mutable_overlay` graphs publish committed edge changes as durable L0
projection segments and reload the latest projection manifest, so other
backends can observe committed topology changes without a full rebuild.

<TreeDiagram>
  <TreeNode title="source table change">
    <TreeNode title="graph sync trigger">
      <TreeNode title="graph._sync_log">
        <TreeBranch>
          <TreeNode title="graph.apply_sync()">
            <TreeNode title="durable projection ingest or backend-local replay" />
          </TreeNode>
          <TreeNode title="graph.maintenance()">
            <TreeNode title="fresh build from source tables">
              <TreeBranch>
                <TreeNode title="new active Engine" />
                <TreeNode title="new immutable base artifact" />
                <TreeNode title="validated base-only projection generation" />
                <TreeNode title="atomic current-pointer switch" />
              </TreeBranch>
            </TreeNode>
          </TreeNode>
        </TreeBranch>
      </TreeNode>
    </TreeNode>
  </TreeNode>
</TreeDiagram>

Node inserts and tombstones can update backend-local state. For persisted
`mutable_overlay` graphs, committed edge mutations publish durable projection
segments that layered reads merge with the base CSR. Segment metadata records
the dirty source-node range when sync replay knows it, and future compaction can
replace those ranges with copy-on-write base chunks without mutating the mmap'd
CSR file. Non-persisted or compatibility paths may still use backend-local
overlays until maintenance or vacuum rebuilds the base CSR. Transaction-local
deltas remain backend-local and are applied last for read-your-own-writes
behavior.

## Design Decisions

| Decision | Why it exists |
|---|---|
| SQL is the public API | Keeps application integration inside PostgreSQL and avoids a new query language. |
| Source tables stay authoritative | pgGraph is an acceleration layer, not a second source of truth. |
| Backend-local `Engine` | Matches PostgreSQL process isolation and avoids shared mutable Rust state. |
| CSR for topology | Compact adjacency slices make traversal cache-friendly and predictable. |
| Reverse CSR is materialized | Inbound traversal stays O(degree) instead of scanning all forward edges. |
| Read-only anonymous mapping for persisted forward arrays | Each backend gets a stable immutable snapshot that cannot be invalidated by same-inode writes or truncation. |
| Bincode metadata is heap-loaded | Filter and registry structures are variable-size Rust data that are easier to validate and use as owned values. |
| Explicit maintenance | Expensive rebuild work is visible and controllable from SQL. |
| Durable projection generations | Committed mutable-overlay topology changes become cross-backend-visible without turning pgGraph into a second source of truth. |
| Circuit breakers everywhere | The extension runs in PostgreSQL backends and must bound memory and traversal work. |

## Safety Boundaries

Unsafe code exists for performance and PostgreSQL integration, not as a general
escape hatch. The core unsafe boundary is mmap-backed store construction:

| Boundary | Required invariant |
|---|---|
| `MmapNodeArrays` | Active bytes, OID array, PK offsets, and PK byte ranges are present, aligned, and bounded by the mmap. |
| `MmapEdgeArrays` | CSR offsets, targets, type IDs, schema flags, optional weights, and relationship IDs are present, aligned, and bounded by the mmap. |
| `MappedGraphArtifact` | Full-layout validation creates an unforgeable production capability; every mapped store retains shared ownership of the read-only mapping. |
| `Engine._mmap` | An `Arc<Mmap>` owns the backend-local immutable snapshot for resolution lookup and accounting while node and edge stores retain independent clones. |
| `GraphError::report()` | A pgrx `ErrorReport` unwinds Rust frames before PostgreSQL emits the standard SQLSTATE and pgGraph diagnostic. |

Detailed rules live in [Safety And Security](./safety-security). Keep rustdoc
`# Safety` sections and local `// SAFETY:` comments current when touching any
unsafe area.

`release/unsafe-allowlist.json` assigns every production unsafe block to a
named boundary and owner. `scripts/check_unsafe_allowlist.py` rejects a new or
removed block, a count change, or a block without a nearby `SAFETY:`
explanation. Persistence and projection decoding use checked arithmetic and
fallible integer conversions before these typed mapped views are constructed;
pgrx-specific raw values remain inside the SQL, catalog, transaction, and
background-job adapters.

Release-risk ownership is intentionally split without changing the public SQL
surface: `resource.rs` owns byte/work policy and reservations;
`projection/manifest.rs`, `gc.rs`, and `recovery.rs` own generation publication
and retention; `persistence.rs` owns artifact layout and validation; and the
persisted-build scanner/pipeline modules own bounded source-to-artifact work.
This keeps validation and failure tests local to each boundary. A broader crate
or module decomposition is a post-1.0 roadmap decision because it is not needed
to prove the 1.0 compatibility contract.

## Where To Make Changes

| Change | Start here |
|---|---|
| SQL function shape or return columns | `src/sql_facade/*`, then `docs/user_guide/api-reference.mdx` |
| Registration validation | `src/catalog/validate.rs` |
| Build ingestion | `src/builder.rs` |
| Traversal behavior | `src/bfs.rs`, `src/engine.rs`, `src/sql_facade/traversal.rs` |
| Shortest path behavior | `src/path_finder.rs`, `src/engine.rs` |
| Persistence format | `src/persistence.rs`, `docs/contributor_guide/persistence-format.mdx` |
| Mmap-backed stores | `src/node_store.rs`, `src/edge_store.rs`, `src/persistence.rs` |
| Sync behavior | `src/sync.rs`, `src/sql_sync.rs`, `src/sql_facade/admin.rs` |
| SQLSTATE or error semantics | `src/safety.rs`, `docs/user_guide/troubleshooting.mdx` |