Chapter 22. For Developers

If you want to get into the details of the Veil2 implementation, this section is for you.

22.1. Performing Local Builds

For most users there will never be any need to build Veil2 locally using anything except the pgxn client as described in the Installation section.

If, however, you wish to hack on the Veil2 internals or documentation, or are just curious you can clone Veil2 from the Veil2 github page, or get a zipped copy from Marc's PGXN pages.

22.1.1. Directory Tree

The Veil2 directory structure is pretty flat and very simple:

bin

Home to a small number of helper scripts for the documentation build.

demo

Contains files associated with the demo extensions.

diagrams

Contains diagrams created by dia.

docs

Contains, mostly, xml source files for the documentation system. Is also used for some intermediate files for the documentation build, including the anchors and extracts directories.

html

Created by the docs target from make. Contains generated html documentation. Point your browser to index.html in here to see your latest updated docs.

sql

Contains the SQL sources for Veil2.

src

Contains the C source files for the Veil2 shared library.

test

Scripts for running unit tests on the Veil2 extension.

You may notice makefiles in many sub-directories. These simply change directory to their parent directory before re-running make there. This enables make to be run from any directory in the tree, making builds from tools like emacs much easier.

22.1.2. Operating System Environment

You should be able to install Veil2 on any operating system, though something Unix-like is recommended, as we use gnu make for a lot of tasks.

22.1.3. Software Requirements

Your requirements will depend on what you are trying to do.

22.1.3.1. To Build Veil2 From Sources

You'll need:

  • make;
  • a C build environment (gcc, llvm, or some such);
  • a currently supported version of Postgres.

22.1.3.2. To Build Veil2 Documentation

You'll need:

  • xsltproc;
  • libxslt1.1;
  • libxml2;
  • dia;
  • p2toimg;
  • gawk;
  • a Unix-like shell;
  • make;
  • docbook 4.5 stylesheets.

22.1.4. The Build System

Veil2 uses a pretty standard autotools-based build system, though the GNUmakefile is largely hand-crafted.

22.1.4.1. autoconf

There should be no need to re-run autoconf unless new dependencies for documentation builds, etc need to be discovered. In this case you would

  • Manually update the configure.ac file.
  • Run autoconf.

This would update the configure file, which you would then re-run as shown below.

22.1.4.2. ./configure

The ./configure script should be run any time that you need to bump the Veil2 version. You will update the VERSION file, and then run ./configure. This will update Makefile.global, which is included from GNUmakefile.

You would also run the ./configure script after any updates to Makefile.global.in which you might make if you need to change compilation flags or some such.

22.1.4.3. make

The makefile, GNUmakefile manages all aspects of the build and installation processes. It uses the postgres pgxs build mechanisms to build the shared library and manage the installation of our postgres extensions.

The default target (run make with no parameters) is the C shared library veil2.so. Other targets include:

help

Provides a list of the makefile's major targets.

db

Creates a new clean vpd database into which we can install Veil2.

docs

Creates an html documentation tree including Doxygen documentation.

install

Installs our extensions into the active Postgres environment. This target does not imply a docs build. If you want docs installed you should build them first and then this target will ensure they are installed.

clean

Remove target, intermediate and junk files.

22.2. Running Unit Tests

The Veil2 unit tests can be run from make using the unit target. They require the postgres unit testing extension pgtap.

The tests leave no residue in the test database, so can be run on any clean database. By default the vpd database will be used, but this can be changed by providing make with a value for TESTDB, eg:

marc:veil2$ make unit TESTDB=veil_unit_test_db
Creating database veil_unit_test_db...
CREATE DATABASE
Performing unit tests...
Running Veil2 unit tests...
. . . 
      

This test database can be dropped using:

marc:veil2$ make drop TESTDB=veil_unit_test_db
Dropping database veil_unit_test_db...
Pager usage is off.
DROP DATABASE
marc:veil2$ 
      

22.3. The Documentation Build

The Veil2 documentation is built using docbook. It is a relatively complex beast though, as there is automated extraction of SQL code, automated creation of diagram images and maps, and linkage with Doxygen documentation. This is what you need to know:

22.3.1. General

There is a lot of nasty shell scripting involved. Trying to do this in a non-Unix environment is probably not an option.

22.3.2. Images

Images are converted into .png files from their .dia sources using pstoimg, which on Debian comes from the latex2html package. This is done by a general %.png: %.dia rule in the makefile.

The ERD and Views diagrams in the html documentation provide clickable links. The maps for these are generated via intermediate .xml and .coords targets, again created by rules in the makefile.

The process for generating the map files is best discovered, if you need it, by exploring the code, starting from the makefile.

22.3.3. SQL Extracts

The Appendices A, B and C contain documentation that is largely automatically generated from the main veil SQL script. The extracts are created from the makefile by some fairly ugly shell scripts. Look for targets containing the word extract in the makefile.

The individual extract files are processed into the docbook sources by sql-definition xsl processing instruction directives in the xml sources. These are handled by an xsl template defined in the docs/html_stylesheet.xsl stylesheet.

22.3.4. Linkage To Doxygen docs

These are handled by targets containing the word anchor in the makefile, and by doxygen-ulink xsl processing-instructions handled by the docs/html_stylesheet.xsl stylesheet.

22.4. Releases

Releasing a new version of Veil2 to the world involves a number of steps. The makefile provides various targets to help in this process, primarily the zipfile target which creates a zipfile for release to PGXN once make is satisfied that all necesary steps have been taken.

Specifically, before creating the zipfile, make insists on the following:

  • that the PGXN metadata file, META.json contains correct filename and version information;

    Specifically, that the version numbers match what is defined in the VERSION file.

  • that there are no uncommitted changes;

  • that the current git branch is master;

    This is particularly necessary as other branches have to be used when uploading documentation to github.

  • that the HEAD branch in git has been tagged;

  • that github and other upstream repositories are up to date with respect to the local git repository;

  • that the latest version of documentation has been published to github pages.