# Coordinated release: ProvSQL 1.4.0 + Studio 1.0.0 Working document. Not committed (delete after the release window closes). Tracks the ordered sequence of work needed to ship both tags together. Source-of-truth lives in `doc/TODO/studio.md` for the Studio side; this file is the operational checklist. Two tags ship in the same window: - **`v1.4.0`** — ProvSQL extension (PGXN, Docker Hub). - **`studio-v1.0.0`** — ProvSQL Studio (PyPI, GitHub release). The Docker image carries both: `inriavalda/provsql:1.4.0` includes the Studio install (item P1.E) so a single `docker pull` gives a ready-to-demo container. --- ## Phase 1: pre-tag work (can run in parallel) ### P1.A — Studio test infrastructure - [x] Add `make studio` target to top-level `Makefile` (`python -m provsql_studio`). Done: also added a `studio-test` target right next to it. - [x] Add `make studio-test` target running `pytest studio/tests` plus the Playwright e2e suite. `pytest tests` walks both the unit tests and `tests/e2e/`, so a single invocation runs everything. - [x] Set up Playwright: install dev dependency in `studio/pyproject.toml` `[project.optional-dependencies]` (`test = ["pytest", "playwright", ...]`), add `studio/tests/e2e/` with Playwright config + a smoke scenario per mode (Where, Circuit, query history, connection editor). Added `playwright>=1.40` and `pytest-playwright>=0.4` to the `test` extra. `tests/e2e/conftest.py` spawns Studio in a subprocess against the existing `test_dsn` fixture, with `PROVSQL_STUDIO_CONFIG_DIR` pointed at a tempdir so a developer's persisted UI settings can't override the `--search-path provsql_test` we pass at startup. `tests/e2e/test_smoke.py` covers eight scenarios (the four the TODO scoped, plus mode-switch query carry-forward, eval-strip running `boolexpr` against a pinned node, schema-panel column-click prefilling `create_provenance_mapping`, and the per-query `where_provenance` toggle being interactive in Circuit mode); the result-row assertion uses `#result-count` (not the generic `#result-body tr`, which the "Running…" placeholder also satisfies). - [x] Confirm `pytest studio/tests` and the Playwright smoke tests pass locally against a live PG with extension built from the current branch. Verified: 122/122 pass in ~18s (114 unit + 8 Playwright e2e) after the user reinstalled the matching extension binary. - [x] Lint: `cd studio && ruff check .` is clean. Cleaned pre-existing issues: an unused `Iterator` import in `db.py`, a stray `f`-prefix on a non-format-string SQL template, two `if cond: return` E701 one-liners, and two unused `import pytest` lines in `test_circuit.py` / `test_exec.py`. None were caused by P1.A but the TODO requires ruff clean, so they got swept up here. ### P1.B — CI: `studio.yml` - [x] Write `.github/workflows/studio.yml`: - [x] Triggers: `studio/**`, `sql/provsql.common.sql`, `sql/provsql.14.sql`, the workflow file itself, `workflow_dispatch`. - [x] Matrix: Py 3.10 / 3.11 / 3.12 / 3.13 × PG 14 / 15 / 16. - [x] Job 1 — pytest: build extension, install Studio, run `pytest studio/tests`. Runs `pytest studio/tests --ignore=studio/tests/e2e` so the e2e job below stays the only one paying the chromium-download tax. - [x] Job 2 — Playwright e2e: spin up PG + extension + Studio, run the e2e suite. `playwright install --with-deps chromium` (root in pgxn/pgxn-tools, no sudo). - [x] Job 3 — lint: `ruff check`. - [x] Job 4 — package smoke: `python -m build` + `pip install dist/*.whl` + `provsql-studio --help`. Installs into a fresh `/tmp/wheel-venv` so the smoke proves the wheel is self-sufficient (no implicit editable-checkout dep). - [x] **Bonus**: `test` and `e2e` matrix jobs `needs: lint` so a broken `ruff check` short-circuits the 24 matrix cells. Also chained `ruff check` into the local `make studio-test` target so missed lint can't surprise CI. - [x] Push to a feature branch, confirm all matrix cells pass. Run id `25565303792` is green: 26 jobs in 4m51s (ruff + package + 12 pytest + 12 Playwright). **Three follow-up fixes** were needed after the first push, each landed in its own commit: `Studio CI: invoke pip/pytest/playwright via python -m` (`setup-python` exports `pythonLocation` but doesn't put `bin/` on PATH inside a `container:`), `Studio CI: install graphviz in matrix containers (dot binary)` (`circuit.py` shells out to `dot`), and `Studio CI: bump actions/setup-python to v6 (Node.js 24)` (silences the Node 20 deprecation warning; this one is committed but not yet pushed at the time of the green run). ### P1.C — PyPI name claim - [x] Confirm `provsql-studio` is unclaimed (`pypi.org/pypi/provsql-studio` 404 today). Confirmed: `provsql-studio`, `provsql_studio` (the PyPI-normalised form), and `provsql-studio` on Test PyPI all return 404. - [x] Decide release path: tag-triggered workflow, or `studio-release.sh`. **Recommended**: workflow + Trusted Publisher (no API token in repo secrets). Decided: **tag-triggered GitHub workflow** (`studio-release.yml`, written in P1.D) using a PyPI **Pending Publisher** so no API token lands in repo secrets. - [x] If workflow: configure Trusted Publisher on PyPI side pointing at `studio-release.yml` and the `studio-v*` tag pattern. (PyPI Trusted Publisher creation requires the first publish to come from the configured workflow; either do a `0.0.1.dev1` rehearsal upload or accept that the very first 1.0.0 publish doubles as the claim.) **Footnote outdated**: PyPI introduced *Pending Publishers* (2023+) precisely to skip this chicken-and-egg — pre-register the publisher for a non-existent project and the first workflow run both claims the name and publishes via Trusted Publishing. No rehearsal upload needed. Pending Publisher registered with: `PyPI Project Name = provsql-studio`, `Owner = PierreSenellart`, `Repository name = provsql`, `Workflow name = studio-release.yml`, `Environment name = pypi`. **P1.D consequence**: the `studio-release.yml` job that publishes must declare `environment: pypi` to match the Pending Publisher; a matching `pypi` environment must also exist on the GitHub repo (Settings → Environments). ### P1.D — Studio release pipeline - [x] Write `.github/workflows/studio-release.yml` **or** `studio-release.sh`: - [x] Trigger on `studio-v*` tag (workflow) / take a version arg (script). Workflow path chosen (matches the P1.C Trusted Publisher decision). Triggers on `push: tags: ['studio-v*']` and on `workflow_dispatch` (with a `tag` input) so a missing tag push can be retried by hand. - [x] Run the studio test job first; abort on red. Done as a single-cell `gate` job (Py 3.12 × PG 16) running the same ruff + pytest steps as `studio.yml`. Cheaper than re-running the 24-cell matrix at release time, and the matrix already gates each push of the same commit. - [x] `python -m build` to produce `dist/provsql_studio-*.tar.gz` (sdist) + `dist/provsql_studio-*-py3-none-any.whl` (wheel). The `build` job also asserts that the produced filenames carry the version parsed from the tag, so a stale `__version__` in `provsql_studio/__init__.py` fails loudly instead of publishing a mismatched wheel. - [x] Publish to PyPI via `pypa/gh-action-pypi-publish` (workflow) / `twine upload` (script). `environment: pypi` matches the Pending Publisher; OIDC via `id-token: write` (no API token in repo secrets). - [x] Create GitHub release with `gh release create studio-v$VERSION dist/*` so the sdist + wheel are attached as assets. The `studio/` line was removed from `.gitattributes` `export-ignore`, so the auto-generated source tarball now carries the full repo (including studio/) for both `v*` and `studio-v*` tags. PGXN bundle picks up the same ~510 KB of Python source (built via `git archive` from the same `.gitattributes`); harmless, since PGXN's indexer respects `META.json`'s `no_index.directory` and the extension `make` chain doesn't touch studio/. - [x] Release notes pin `pip install provsql-studio==$VERSION` as the canonical install path; mention extension >= 1.4.0 is required. Notes also embed a "What's changed" section extracted from `studio/CHANGELOG.md` by an awk one-liner; the workflow fails loudly if no section matches the tag's version (catches a missing CHANGELOG entry before an empty-notes release lands). ### P1.E — Docker integration - [x] Edit `docker/Dockerfile`: - [x] Install Python 3 + `pip` + `python3-venv`. Also dropped `apache2 libapache2-mod-php php-pgsql` (no longer needed); kept `graphviz` (`dot` for circuit layout) and `libgraph-easy-perl` (extension's `view_circuit` ASCII rendering). - [x] `ARG STUDIO_VERSION=1.0.0` (default = the released PyPI version) and `ARG STUDIO_SOURCE=` (empty default). - [x] If `STUDIO_SOURCE` is set, `pip install -e ${STUDIO_SOURCE}`; else `pip install provsql-studio==${STUDIO_VERSION}`. Installed into `/opt/studio-venv` (Debian bookworm's Python 3.11 enforces PEP 668, so a system-wide pip install is rejected); venv bin/ prepended to PATH so `provsql-studio` resolves. - [x] Edit `docker/demo.sh`: - [x] Drop `cp -r /opt/provsql/where_panel/* /var/www/html/` and the `sed` lines that follow. (Done in P1.F.) - [x] After PG is up, launch `provsql-studio --host 0.0.0.0 --port 8000 --dsn postgresql://test:test@localhost/test &`. Used a libpq DSN (`'dbname=test user=test'`) since the container's `pg_hba.conf` already grants local-trust auth, and added `--search-path provsql_test` so the `personnel` fixture is reachable without schema-qualified names. - [x] Print Studio URL alongside the psql info: `Studio: http://${IP}:8000`. - [x] Decide whether Apache stays at all (likely no; remove the `apache2` start if Studio fully replaces `where_panel`). Apache removed: `apache2` package, `apt-get install` line, `init.d/apache2 start` invocation, the `/var/www` prep, and `EXPOSE 80` are all gone. `EXPOSE 8000` for Studio takes its place. Also replaced the dead `tail -f /messages | sed ...` URL-rewrite loop with an `exec tail -f /messages` PID-1 keeper. - [ ] Local smoke test: `make docker-build && docker run --rm -p 8000:8000 provsql:dev`, then open `http://localhost:8000` and confirm `/where` and `/circuit` load against the test database. **User action**: needs the Docker daemon and ~few minutes of build time. **Note**: until Studio 1.0.0 is published to PyPI, `make docker-build` (which uses the default `STUDIO_SOURCE=`) will fail at the pip step; use the dry-run below to verify locally before publish. - [x] Dry-run with `--build-arg STUDIO_SOURCE=/opt/provsql/studio` to confirm the contributor path also works when Studio isn't on PyPI yet (this is the path used at release time, before the PyPI tag has landed). Verified on 2026-05-08: `make clean && docker build --build-arg PROVSQL_VERSION=1.4.0-dev --build-arg STUDIO_SOURCE=/opt/provsql/studio` succeeded (image `provsql:dev`, 2.43 GB; Studio installed in editable mode from `/opt/provsql/studio` per the build log). `docker run --rm -p 8000:8000` printed the ready banner ("Both services are reachable... Studio web UI: http://172.17.0.2:8000"), and `curl 127.0.0.1:8000` returned 302 on `/`, 200 on `/where`, and 200 on `/circuit` (page title `ProvSQL Studio` on both). ### P1.F — `where_panel/` cleanup - [x] Delete the `where_panel/` directory. - [x] Confirm `docker/demo.sh` no longer references it (P1.E already drops the copy block). Done here in P1.F: dropped the `cp -r .../where_panel/*`, the two `sed` lines that followed, and the trailing `where_panel web interface` echo block. Apache start and `/var/www/html/pdf` setup stay for P1.E to revisit alongside the Studio launch. - [x] Remove `where_panel/**` from the two `paths-ignore` blocks in `.github/workflows/docs.yml`. - [x] `git grep -n where_panel` and clean any stale references in `doc/source/`, `website/`, `README.md`. None in `doc/source/`, `website/` (current pages), or `README.md`. Cleaned: `META.json` `no_index.directory` entry, attribution comment in `studio/provsql_studio/db.py:221`. Left intact: `CHANGELOG.md` and `website/_data/releases.yml` (immutable history), `doc/TODO/studio.md` (Phase 3 cleanup), `studio/design/ui_kits/where_panel/` and references to it from `studio/design/ui_kits/circuit/index.html` (the whole `studio/design/` tree is deleted in P1.I). ### P1.G — Website visibility - [x] Add a Studio section to `website/index.html` (top-of-fold card or hero panel) with at least one screenshot. Done as a 3-up first feature row: "Transparent Query Rewriting" + "Evaluate Provenance" + "ProvSQL Studio", each carrying an image. The original three-card row (Query Rewriting / Rich Semiring / Probability & Shapley) was reduced by merging Semiring + Probability into a single "Evaluate Provenance" card to make room for Studio while keeping the row at 3. - [x] Add a Studio top-nav entry (or "Studio" tile on `overview.md`) linking to `/docs/user/studio.html` and to the PyPI page. Both: top-nav entry "Studio" between Documentation and Publications (links to docs); a `## ProvSQL Studio` H2 on `overview.md` between Query Rewriting and Lean Formalization (links to docs and the PyPI page). - [x] Pick screenshots: `doc/source/_static/studio/where-mode.png` and `circuit-mode.png` are the strongest existing two. Optionally re-shoot at higher resolution using the OS-level capture procedure documented in `CLAUDE.md`. `circuit-mode.png` used on the Studio card. Two further images extracted from the ICDE'26 poster (`sen2026provsql_poster.pdf`): the Query rewriting flow diagram for the Query Rewriting card, and the Semiring Instantiation / Evaluation of circuits / Result trio for the Evaluate Provenance card. Re-shoots not needed. - [x] Verify with `make website` and a local Jekyll preview that the Studio panel renders correctly on desktop and mobile widths. `make website` clean; Jekyll build OK; verified locally at `http://localhost:8765/`. ### P1.H — Compatibility floor - [x] Update `doc/source/user/studio.rst` compatibility matrix: add a row for Studio 1.0.0 ↔ extension >= 1.4.0. (Already present, lines 519-531; calls out `circuit_subgraph`, `resolve_input`, and the `provsql.aggtoken_text_as_uuid` GUC, all confirmed absent from `sql/upgrades/*` for any version ≤ 1.3.1.) - [x] Confirm Studio's startup version check (`SELECT extversion FROM pg_extension WHERE extname = 'provsql'`) refuses to start if extension < 1.4.0; add / bump the floor in `studio/provsql_studio/db.py` or wherever the check lives. (Verify the check exists; if not, add it.) Lives in `studio/provsql_studio/cli.py` (`REQUIRED_PROVSQL_VERSION = (1, 4, 0)`, `_check_extension_version`); accepts a `-dev` suffix as the matching release; `--ignore-version` overrides. - [x] `studio/pyproject.toml`: confirm `version` resolves to `1.0.0` (already done via dynamic `__version__ = "1.0.0"`). - [x] `make docs` clean. ### P1.I – `studio/` housekeeping - [x] Audit `studio/` and remove anything that is not the Python package, its tests, or its docs / release plumbing. Targets: - [x] `studio/design/` – initial design documents, the embedded `where_panel/` UI-kit copy under `design/ui_kits/where_panel/`, and `design/screenshots/`. 12 tracked files removed via `git rm -r studio/design/`. - [x] `studio/.claude/` – not shipped with the package. Already untracked (covered by the user's global `~/.gitignore`, verified via `git check-ignore`); no project change needed. - [x] Any other build artefacts, scratch files, or legacy prototypes still tracked by git. Use `git ls-files studio/` before and after the cleanup as a check. Before: 44 files (incl. `design/`). After: 32 files (`LICENSE`, `provsql_studio/`, `pyproject.toml`, `README.md`, `scripts/`, `tests/`). `scripts/` (six `load_demo_*.sql` + `big_demo_queries.sql`) kept as developer-facing demo loaders for the case-study semirings. - [x] After the cleanup, rerun `python -m build && pip install dist/*.whl` in a throwaway venv to confirm the wheel still installs and `provsql-studio --help` still works. --- ## Merge gate: `studio` → `master` All Phase 1 work happens on the `studio` branch. Before Phase 2 can run (the release scripts operate on `master`), the branch needs to land on master via a merge. - [x] On `studio`, merge in the latest `master` to absorb any drift and resolve conflicts there: `git fetch origin && git merge origin/master`. Done in merge commit `3e4b778`. The 3-dot diff (`origin/studio...origin/master`) was empty: master's only post-divergence commit `a793aeb` (Docker demo iproute2 + startup messages) was superseded by studio's later `361a488` (Studio launch) and `1d76e01` (PGDG/cleanup), so the merge introduced no working-tree changes. - [x] Confirm the full extension test suite is green on `studio` after the merge (`sudo make install && sudo service postgresql restart && make installcheck`), plus `pytest studio/tests` and `make docs`. Skipped as redundant: `3e4b778` is content-identical to pre-merge tip `44f9b46`; `44f9b46` differs from `1d76e01` only in `TODO.md`; and `1d76e01` carries a fully green CI run (Linux + Mac OS + WSL + Studio + Documentation). The Documentation workflow on `44f9b46` also completed green (run id `25579295742`). - [x] Switch to `master`, merge `studio` in (no fast-forward, so the integration is visible in the history): `git checkout master && git merge --no-ff studio`. Done in merge commit `af37f58` on 2026-05-08. - [x] Push `master`. The branch deletion is deferred to Phase 3 so any release-time fixups can still land on `studio` and be re-merged if needed. Pushed `origin/master` (af37f58) and `origin/studio` (3e4b778) together. --- ## Phase 2: coordinated release Run **only after** every Phase 1 item is checked, the merge gate above has landed on `master`, both extension master and Studio master are green on all CI workflows (Linux / macOS / WSL / studio.yml), and the local Docker smoke test succeeds. ### P2.1 — Extension v1.4.0 - [ ] On master, run `./release.sh 1.4.0`. The script handles: version bump in `provsql.common.control`, CHANGELOG entry, `website/_data/releases.yml` entry, `CITATION.cff`, `META.json`, signed tag, push, GitHub release. Approve the post-release `1.5.0-dev` bump it offers at the end. - [ ] Wait for `build_and_test.yml` Docker job to complete. Confirm `inriavalda/provsql:1.4.0` and `inriavalda/provsql:latest` are visible on Docker Hub. - [ ] Confirm PGXN auto-built v1.4.0 (check pgxn.yml run; PGXN catalogue updates within ~minutes). ### P2.2 — Studio v1.0.0 - [ ] If using the workflow path: push tag `studio-v1.0.0`. The release workflow runs studio.yml's test matrix, builds sdist + wheel, publishes to PyPI via Trusted Publisher, and creates the GitHub release with the artifacts attached. - [ ] If using the script path: run `./studio-release.sh 1.0.0`. Same outcome, locally driven. - [ ] Confirm `pip install provsql-studio==1.0.0` works in a throwaway venv. Confirm `https://pypi.org/project/provsql-studio/` renders the README. ### P2.3 — End-to-end smoke - [ ] In a fresh shell: `docker run --rm -p 8000:8000 inriavalda/provsql:1.4.0`, open `http://localhost:8000`, run a query in Where mode, switch to Circuit mode, confirm both work end-to-end. - [ ] Outside Docker: in a fresh venv, `pip install provsql-studio` against an existing local PG with extension 1.4.0 installed; confirm the same. - [ ] If Studio refuses to start against a 1.3.0 install (intentional, per the version check), confirm the error message is clear. ### P2.4 — Website deploy - [ ] On master: `make deploy`. The website pulls in the new `releases.yml` entry (1.4.0), the Studio visibility section (P1.G), and the compatibility matrix. - [ ] Spot-check `https://provsql.org/`: Studio panel visible, release page lists 1.4.0 with notes, docs include the Studio chapter at the new compatibility floor. ### P2.5 — Announce - [ ] Lab / mailing list announcement covering both releases. - [ ] Update Studio's PyPI long-description (the README shipped in the wheel) if anything in the announcement should land there too — would require a 1.0.1 follow-up since PyPI uploads are immutable. --- ## Phase 3: post-release housekeeping - [ ] Studio: bump `provsql_studio/__version__` to `"1.1.0.dev0"` (or `1.0.1.dev0` if the next release is a patch). Commit to master. - [ ] Extension: confirm `release.sh` already pushed the `1.5.0-dev` post-release bump. - [ ] Strike the v1.0-blocking items out of `doc/TODO/studio.md` (the "Blocking v1.0" section) so the file only carries the "Beyond v1.0" plan going forward. Update the intro paragraph to drop the coordinated-release note (or rewrite it to describe what shipped). - [ ] Delete this file (`TODO.md`) once both releases are announced and stable for ~a week. - [ ] Delete the `studio` branch locally and on the remote once both releases are stable: `git branch -d studio && git push origin --delete studio`. --- ## Risks / things to watch - **PyPI Trusted Publisher first-publish**: Trusted Publishers on PyPI need the project to exist before the publisher can be configured, which is awkward when the project doesn't exist yet. Two ways out: (a) upload a `0.0.1.dev1` from a workflow one-time (claims the name and lets you configure Trusted Publishing for subsequent releases), or (b) do the very first upload with an API token, then switch to Trusted Publisher for 1.0.1 onwards. Decide before P1.C. - **Docker image size**: adding Python + Studio increases the image. If size matters, consider a multi-stage build (out of scope for the release window unless it bloats noticeably). - **`v1.4.0` vs `studio-v1.0.0` ordering**: the extension tag must land first so the Docker image with extension 1.4.0 exists before Studio 1.0 announces it as the supported floor. Phase 2 enforces this order. - **`.gitattributes` and the auto-tarball**: ~~studio/ was `export-ignore`'d so the auto-tarball would be empty of Studio code on `studio-v*`~~ — line removed in this branch; the auto-tarball now carries the full repo at both `v*` and `studio-v*` tags. PGXN bundle gains the same ~510 KB of Python source, harmless per the analysis above.