=head1 OmniPITR - omnipitr-archive

=head2 USAGE

/some/path/omnipitr/bin/omnipitr-archive [options] "%p"

Options:

=over

=item --data-dir (-D)

Where PostgreSQL datadir is located (path)

=item --force-data-dir (-f)

Forces usage of given (or default) data dir even if it doesn't look like
PostgreSQL data directory.

=item --dst-local (-dl)

Where to copy the wal segment on current server (path) (you can provide many of
these).

You can also specify compression per-destination. Check L<COMPRESSION>
section of the doc.

=item --dst-remote (-dr)

Where to copy the wal segment on remote server. Supported ways to transport
files are rsync and rsync over ssh. Please see L<DESCRIPTION> for more
information (you can provide many of these)

You can also specify compression per-destination. Check L<COMPRESSION>
section of the doc.

=item --dst-backup (-db)

Special destination, that cannot be compressed (existence of compression prefix
will cause I<omnipitr-archive> to raise exception, has to be absolute path
(starts with /), and can be non-existing.

This is special directory that shouldn't usually exist, but if it exists - it
will receive copy of every wal segment rotated.

The sole purpose of this option is to make I<omnipitr-master-backup> working. If
you do not intend to make hot backups on master - you can safely not use it.

=item --dst-pipe (-dp)

Specifies path to program that should receive xlog on stdin.

The program will be called multiple times (for each xlog separately). Name of
the xlog will be given as first, and only, argument to the program.

If the file is to be compressed (using standard compression=/path syntax)
- given file name will contain compression extension.

=item --temp-dir (-t)

Where to create temporary files (defaults to /tmp or I<$TMPDIR> environment
variable location)

=item --log (-l)

Name of logfile (actually template, as it supports %% L<strftime(3)>
markers. Unfortunately due to the %x usage by PostgreSQL, We cannot use %%
macros directly. Instead - any occurence of ^ character in log dir will be first
changed to %, and later on passed to strftime.

Please note that on some systems (Solaris for example) default shell treats ^ as
special character, which requires you to quote the log filename (if it contains
^ character). So you'd better write it as:

    --log '/var/log/omnipitr-^Y-^m-^d.log'

=item --state-dir (-s)

Name of directory to use as state-directory to handle errors when sending wal
segments to many locations.

=item --pid-file

Name of file to use for pidfile. If it is specified, than only one copy of
I<omnipitr-archive> (with this pidfile) can run at the same time.

Trying to run second copy of I<omnipitr-archive> will result in an error.

=item --parallel-jobs (-PJ)

Number of parallel jobs that I<omnipitr-archive> can spawn to deliver archives
to remote destinations.

=item --verbose (-v)

Log verbosely what is happening.

=item --not-nice (-nn)

Do not use nice for compressions.

=item --nice-path (-np)

Full path to nice program - in case you can't set proper PATH environment
variable.

=item --gzip-path (-gp)

Full path to gzip program - in case you can't set proper PATH environment
variable.

=item --bzip2-path (-bp)

Full path to bzip2 program - in case you can't set proper PATH environment
variable.

=item --lzma-path (-lp)

Full path to lzma program - in case you can't set proper PATH environment
variable.

=item --lz4-path (-ll)

Full path to lz4 program - in case you can't set proper PATH environment
variable.

=item --xz-path (-xz)

Full path to xz program - in case you can't set proper PATH environment
variable.

=item --rsync-path (-rp)

Full path to rsync program - in case you can't set proper PATH environment
variable.

=item --version (-V)

Prints version of I<omnipitr-archive>, and exists.

=item --help (-?)

Prints this manual, and exists.

=item --config-file (--config / --cfg)

Loads options from config file.

Format of the file is very simple - each line is treated as argument with
optional value.

Examples:

    --verbose
    --host 127.0.0.1
    -h=127.0.0.1
    --host=127.0.0.1

It is important that you don't need to quote the values - value will always
be up to the end of line (trailing spaces will be removed). So if you'd
want, for example, to have magic-option set to "/mnt/badly named directory",
you'd need to quote it when setting from command line:

    /some/omnipitr/program --magic-option="/mnt/badly named directory"

but not in config:

    --magic-option=/mnt/badly named directory

Empty lines, and comment lines (starting with #) are ignored.

=back

=head2 DESCRIPTION

Call to I<omnipitr-archive> should be in I<archive_command> GUC in
I<postgresql.conf>.

Which options should be given depends only on installation, but generally you
will need at least:

=over

=item * --data-dir

PostgreSQL "%p" passed file path relative to I<DATADIR>, so it is required to
know it. If not provided, it is assumed to be "." ( which is perfectly in line
with PostgreSQL assumptions about archive_command ).

=item * --log

to make sure that information is logged someplace about archiving progress

=item * one of --dst-local or --dst-remote

to specify where to send the WAL segments to

=back

If you'll specify more than 1 destination, you will also need to specify
I<--state-dir>

Of course you can provide many --dst-local or many --dst-remote or many mix of
these.

Generally omnipitr-archive will try to deliver WAL segment to all destinations,
and will fail if B<any> of them will not accept new segment.

Segments will be transferred to destinations in this order:

=over

=item 1. All B<local> destinations, in order provided in command line

=item 2. All B<remote> destinations, in order provided in command line

=back

In case any destination will fail, I<omnipitr-archive> will save state (which
destinations it delivered the file to) and return error to PostgreSQL - which
will cause PostgrerSQL to call I<omnipitr-archive> again for the same WAL
segment after some time.

State directory will be cleared after every successfull file send, so it should
stay small in size (expect 1 file of under 500 bytes).

When constructing command line to put in I<archive_command> PostgreSQL GUC,
please remember that while providing C<"%p" "%f"> will work, I<omnipitr-archive>
requires only "%p"

=head3 Remote destination specification

I<omnipitr-archive> delivers WAL segments to destination using rsync program.
Both direct-rsync and rsync-over-ssh are supported (it's better to use direct
rsync - it uses less resources due to lack of encryption.

Destination url/location should be in a format that is usable by I<rsync>
program.

For example you can use:

=over

=item * rsync://user@remote_host/module/path/

=item * host:/path/

=back

To allow remote delivery you need to have rsync program. In case you're using
rsync over ssh, I<ssh> program has also to be available.

In case your rsync/ssh programs are in custom directories simply set I<$PATH>
environemnt variable before starting PostgreSQL.

=head2 COMPRESSION

Every destination can have specified compression. To use it you should prefix
destination path/url with compression type followed by '=' sign.

Allowed compression types:

=over

=item * gzip

Compresses with gzip program, used file extension is .gz

=item * bzip2

Compresses with bzip2 program, used file extension is .bz2

=item * lzma

Compresses with lzma program, used file extension is .lzma

=item * lz4

Compresses with lz4 program, used file extension is .lz4

=item * xz

Compresses with xz program, used file extension is .xz

=back

All compressions are done I<on B<NICE>> to make the operation as unobtrusive as
possible.

All programs are passed I<--stdout> option to compress without modifying source file.

If you want to pass any extra arguments to compression program, you can either:

=over

=item * make a wrapper

Write a program/script that will be named in the same way your actual
compression program is named, but adding some parameters to call

=item * use environment variables

All of supported compression programs use environment variables:

=over

=item * gzip - GZIP

=item * bzip2 - BZIP2

=item * lzma - XZ_OPT

=back

For details - please consult manual to your choosen compression tool.

=back

=head2 EXAMPLES

=head3 Minimal setup, with copying file to local directory:

    archive_command='/.../omnipitr-archive -D /mnt/data/ -l /var/log/omnipitr/archive.log -dl /mnt/wal_archive/ "%p"'

=head3 Minimal setup, with copying file to remote directory over rsync:

    archive_command='/.../omnipitr-archive -D /mnt/data/ -l /var/log/omnipitr/archive.log -dr rsync://slave/postgres/wal_archive/ "%p"'

=head3 2 remote, compressed destinations, 1 local, with auto rotated logfile:

    archive_command='/.../omnipitr-archive -D /mnt/data/ -l /var/log/omnipitr/archive-^Y-^m-^d.log -dr gzip=rsync://slave/postgres/wal_archive/ -dr bzip2=backups@backupserver:/mnt/backups/wal_archive/ -dl /mnt/wal_archive/ -s /var/lib/postgres/.omnipitr/ "%p"'

=head2 IMPORTANT NOTICES

=over

=item * You need to provide state dir in case you have > 1 destination. This is
crucial, as if you're using "-dr ... -db" - your number of destinations changes
from 1 to 2 (depending on whether -db directory exists) - i.e. not all runs of
I<omnipitr-archive> will catch lack of state-dir as error!

=item * It is generally good to provide 2 destination - one used by slave system
to keep warm standby, and another to be handy in case (for example) backup files
get corrupted. This second set of xlogs should be cleared with cronjob that
removes too old files. As for what is too old - 2x your backup schedule. I.e. if
you're doing daily hot backups - you should keep 2 days worth of backup xlogs.

=back

=head2 COPYRIGHT

The OmniPITR project is Copyright (c) 2009-2013 OmniTI. All rights reserved.