# Stack Overflow Benchmark Suite

This directory contains schemas, indexes, and queries for benchmarking ParadeDB on the Stack Overflow dataset, against best practice Postgres indexes.

## Comparison Strategy

Queries are split into three families with different comparison strategies:

- Top-K:
  - Single table - Measures Block-Max WAND optimizations, single-worker behavior, tiebreakers, and range filters across selectivity tiers without external baseline comparisons.
    - Note that we currently do not compare to GIN here.
  - Joins - PostgreSQL can be competitive when supported by appropriate indexes (often finding efficient nested loop join plans or early-termination paths). B-tree and GIN indexes are defined in `indexes/bm25.sql`. Highly uncompetitive PostgreSQL join baselines (such as `join_top_k_score_desc_low_selectivity/postgres.sql`, which took ~9.0 seconds per iteration and required a dedicated 410 MB GIN index on `stackoverflow_posts.body`) were omitted to keep benchmark runtimes fast; see that query directory's `README.md` for a query plan breakdown.
- Aggregates:
  - Not compared against PostgreSQL. Standard PostgreSQL aggregates over GIN full-text indexes require scanning full posting lists and heap tuples, taking tens of seconds to minutes on large datasets. ParadeDB executes these queries faster than PostgreSQL even when using its base scan (`paradedb.enable_aggregate_custom_scan = off`). To keep benchmark execution fast, `basescan.sql` variants are retained only for representative queries (`count_filter`, `aggregate_topk_count`, `join_aggregate_count`, and `cardinality/basescan_distinct.sql`), while the remaining aggregate queries benchmark DataFusion pushdown (`aggregate_scan.sql`).