# Benchmark results

| | |
|---|---|
| PostgreSQL | PostgreSQL 18.4 (Homebrew) on aarch64-apple-darwin24.6.0, compiled by Apple clang version 17.0.0 (clang-1700.6.4.2), 64-bit |
| ICU | 78.3 |
| CPU | Apple M4 Pro |
| Corpus | Wikipedia article titles, dump 20260901, 500000 titles per language |
| Runs | median of 5 after one warm-up run, one backend, no parallel workers, JIT off |
| LC_CTYPE | C.UTF-8 |

## Tokenizing

Time to tokenize every title once, minus the time to scan the table. "vs default" divides by the default parser's time.

| Language | Titles | Avg bytes | Method | µs per title | vs default |
|---|--:|--:|---|--:|--:|
| en | 500,000 | 21.0 | default parser | 0.730 | 1.00× |
| en | 500,000 | 21.0 | icu_parser | 0.631 | 0.86× |
| en | 500,000 | 21.0 | pg_trgm | 1.107 | 1.52× |
| en | 500,000 | 21.0 | pg_bigm | 0.908 | 1.24× |
| zh | 500,000 | 16.9 | default parser | 0.434 | 1.00× |
| zh | 500,000 | 16.9 | icu_parser | 1.438 | 3.31× |
| zh | 500,000 | 16.9 | pg_trgm | 0.762 | 1.76× |
| zh | 500,000 | 16.9 | pg_bigm | 0.484 | 1.11× |
| ja | 500,000 | 22.2 | default parser | 0.569 | 1.00× |
| ja | 500,000 | 22.2 | icu_parser | 1.718 | 3.02× |
| ja | 500,000 | 22.2 | pg_trgm | 1.011 | 1.78× |
| ja | 500,000 | 22.2 | pg_bigm | 0.659 | 1.16× |
| ko | 500,000 | 19.1 | default parser | 0.547 | 1.00× |
| ko | 500,000 | 19.1 | icu_parser | 0.667 | 1.22× |
| ko | 500,000 | 19.1 | pg_trgm | 0.893 | 1.63× |
| ko | 500,000 | 19.1 | pg_bigm | 0.580 | 1.06× |
| th | 366,332 | 45.9 | default parser | 1.083 | 1.00× |
| th | 366,332 | 45.9 | icu_parser | 1.681 | 1.55× |
| th | 366,332 | 45.9 | pg_trgm | 2.184 | 2.02× |
| th | 366,332 | 45.9 | pg_bigm | 1.432 | 1.32× |

## GIN index build

Single-process `CREATE INDEX` with `maintenance_work_mem = 1GB`.

| Language | Method | Build seconds | Index size MB |
|---|---|--:|--:|
| en | default parser | 1.4 | 25.5 |
| en | icu_parser | 1.3 | 24.0 |
| en | pg_trgm | 1.7 | 29.5 |
| en | pg_bigm | 1.7 | 20.4 |
| zh | default parser | 1.2 | 38.2 |
| zh | icu_parser | 1.5 | 14.3 |
| zh | pg_trgm | 2.2 | 72.0 |
| zh | pg_bigm | 2.1 | 41.8 |
| ja | default parser | 1.3 | 39.6 |
| ja | icu_parser | 1.7 | 18.6 |
| ja | pg_trgm | 2.3 | 63.7 |
| ja | pg_bigm | 2.0 | 33.9 |
| ko | default parser | 1.2 | 30.1 |
| ko | icu_parser | 1.3 | 28.8 |
| ko | pg_trgm | 1.8 | 48.9 |
| ko | pg_bigm | 1.5 | 24.3 |
| th | default parser | 1.2 | 37.6 |
| th | icu_parser | 1.3 | 13.7 |
| th | pg_trgm | 1.6 | 23.4 |
| th | pg_bigm | 1.6 | 18.8 |

## Word queries

Each language runs its query terms one at a time as `count(*)`, through each method's index. The tsvector methods use `plainto_tsquery`, and the n-gram methods use `LIKE '%term%'`. Recall and overmatch come from the labeled sample below: recall is the share of real matches the method returned, and overmatch is the share of the method's returned titles that aren't real matches.

| Language | Terms | Method | Queries using the index | Avg ms per query | Hits | Recall | Overmatch |
|---|--:|---|--:|--:|--:|--:|--:|
| en | 200 | default parser | 200 | 0.167 | 123,231 | — | — |
| en | 200 | icu_parser | 200 | 0.163 | 119,253 | — | — |
| en | 200 | pg_trgm | 183 | 2.564 | 659,396 | — | — |
| en | 200 | pg_bigm | 200 | 1.698 | 659,396 | — | — |
| zh | 200 | default parser | 200 | 0.027 | 5,396 | 10.3% | 0.0% |
| zh | 200 | icu_parser | 200 | 0.106 | 68,070 | 92.6% | 15.6% |
| zh | 200 | pg_trgm | 22 | 21.677 | 79,536 | 100.0% | 18.4% |
| zh | 200 | pg_bigm | 200 | 0.105 | 79,536 | 100.0% | 18.4% |
| ja | 200 | default parser | 200 | 0.035 | 7,373 | 7.0% | 0.0% |
| ja | 200 | icu_parser | 200 | 0.128 | 86,443 | 86.1% | 19.5% |
| ja | 200 | pg_trgm | 57 | 18.650 | 175,927 | 100.0% | 54.0% |
| ja | 200 | pg_bigm | 200 | 0.245 | 175,927 | 100.0% | 54.0% |
| ko | 200 | default parser | 200 | 0.064 | 35,053 | 46.7% | 0.0% |
| ko | 200 | icu_parser | 200 | 0.067 | 36,678 | 48.2% | 1.1% |
| ko | 200 | pg_trgm | 91 | 16.965 | 93,655 | 100.0% | 22.0% |
| ko | 200 | pg_bigm | 200 | 0.135 | 93,655 | 100.0% | 22.0% |
| th | 200 | default parser | 200 | 0.037 | 6,997 | 5.3% | 0.0% |
| th | 200 | icu_parser | 200 | 0.279 | 215,162 | 78.9% | 43.8% |
| th | 200 | pg_trgm | 133 | 16.912 | 620,044 | 100.0% | 77.2% |
| th | 200 | pg_bigm | 200 | 0.967 | 620,044 | 100.0% | 77.2% |

## Labeled sample

For each language, `bench/pairs.sql` draws term/title pairs at random from every title in the sample that contains one of the query terms as a substring, and `bench/labels.csv` marks each pair `match` or `over`. A pair is a `match` when the title uses the term as a word: on its own, with grammatical endings (Korean particles, Japanese inflection), or as part of a compound whose meaning includes it (`機場` in `國際機場`). It is `over` when the characters are there but the word isn't: inside a transliterated name (`阿拉` in `阿拉巴马州`), across a word boundary (`京都` in `東京都`), or inside an unrelated word. Every method is scored on the same pairs, so a method that returns every substring scores 100% recall and pays in overmatch.

| Language | Labeled pairs | Real matches |
|---|--:|--:|
| zh | 250 | 81.6% |
| ja | 250 | 46.0% |
| ko | 250 | 78.0% |
| th | 250 | 22.8% |