# Phase 2 benchmark harness for pg_mentat

**This is a scaled-down Phase 2 run on a developer laptop.** The ROADMAP's
Phase 2 target is 10M+ datoms on dedicated hardware with a hermetic
environment. This harness runs the same workload at 100K / 300K / 1M
datoms on a laptop so we have a defensible curve, a flamegraph, and an
apples-to-apples EAV baseline — but the numbers are **not a production
performance claim**.

## Layout

```
benchmarks/phase2/
├── README.md                  # this file
├── schema.edn                 # Datalog schema (issue tracker)
├── eav_baseline/
│   └── schema.sql             # equivalent plain-EAV schema (Postgres tables)
├── gen_dataset.py             # deterministic dataset generator (seed = fixed)
├── queries/
│   ├── q1_point_lookup.edn    # find user by email
│   ├── q1_point_lookup.sql    # same as plain EAV SQL
│   ├── q2_ref_traversal.edn   # issues assigned to a user
│   ├── q2_ref_traversal.sql   # ditto
│   ├── q3_aggregate.edn       # count of issues per state
│   ├── q3_aggregate.sql       # ditto
│   ├── q4_predicate.edn       # high-priority open issues
│   └── q4_predicate.sql       # ditto
├── run.sh                     # main driver: gen + load + warm + measure + CSV
├── perf_capture.sh            # profile the hot path, produce flamegraph.svg
└── tools/
    ├── stackcollapse-perf.pl  # Brendan Gregg's FlameGraph scripts (BSD)
    └── flamegraph.pl
```

## What we measure

- **Load throughput**: wall time for bulk insert.
- **Query latency**: p50 / p95 / p99 from 30 repetitions per query.
- **Plan**: `EXPLAIN (ANALYZE, BUFFERS)` for every query, captured.
- **pg_stat_statements**: top-10 slowest normalised statements.
- **Index/table sizes**: `pg_total_relation_size` per narrow table.
- **CPU flamegraph**: `perf record` during the query replay at the
  middle scale, then `perf script | stackcollapse | flamegraph`.

## Reproduce

```
CARGO_HOME=$HOME/.cargo_pg_mentat bash benchmarks/phase2/run.sh
```

Results land in `benchmarks/results/phase2-<timestamp>/`.

## Caveats (read these before citing any number)

1. **Laptop, not a benchmark box.** Intel i9-12900H with thermal
   throttling, background browser tabs, and whatever Chrome is doing.
   Run-to-run variance is visible at larger scales.
2. **pgrx-managed Postgres**, not a tuned production instance.
   `shared_buffers`, `work_mem`, `effective_cache_size`, `max_wal_size`
   are all defaults.
3. **Single connection, no concurrency.** Nothing here speaks to how
   pg_mentat behaves under contention.
4. **One workload family.** Issue-tracker schema; YMMV on document-
   heavy or graph-heavy workloads.
5. **The EAV baseline is intentionally dumb.** A plain `(entity,
   attribute, value_long, value_text, ...)` table with EAVT/AEVT
   indexes. It isolates the overhead pg_mentat adds on top of an
   equivalent storage layout; it is **not** an optimised Datalog
   implementation. A query-engineered SQL rewrite against the same
   EAV tables would beat both.

The hermetic-environment, 10M-datom, long-duration version of this
suite is the real Phase 2 and remains open.