WALMGR(1)

NAME
walmgr - tools for managing WAL-based replication for PostgreSQL.

SYNOPSIS

walmgr.py <config.ini> command

DESCRIPTION

It is both admin and worker script for PostgreSQL PITR replication.

QUICK START

    1. Set up passwordless ssh authentication from master to slave

        master$ test -f ~/.ssh/id_dsa.pub || ssh-keygen -t dsa
        master$ cat ~/.ssh/id_dsa.pub | ssh slave cat \>\> .ssh/authorized_keys

    2. Configure paths

        master$ edit master.ini
        slave$ edit slave.ini

        Make sure that walmgr.py executable has same pathname on slave and master.

    3. Start archival process and create a base backup

        master$ ./walmgr.py master.ini setup
        master$ ./walmgr.py master.ini backup

        Note: starting from PostgreSQL 8.3 the archiving is enabled by setting
        archive_mode GUC to on. However changing this parameter requires the
        server to be restarted.

    4. Prepare postgresql.conf and pg_hba.conf on slave and start replay

        master$ scp $PGDATA/*.conf slave:
        slave$ ./walmgr.py slave.ini restore

        For debian based distributions the standard configuration files are located
        in /etc/postgresql/x.x/main directory. If another scheme is used the postgresql.conf
        and pg_hba.conf should be copied to slave full_backup directory. Make sure to
        disable archive_command in slave config.

        'walmgr.py restore' moves data in place, creates recovery.conf and starts postmaster
        in recovery mode.

    5. In-progress WAL segments can be backup by command:

        master$ ./walmgr.py master.ini sync

    6. If need to stop replay on slave and boot into normal mode, do:

        slave$ ./walmgr.py slave.ini boot

GENERAL OPTIONS

Common options to all walmgr.py commands.

-h, --help
    show this help message and exit

-q, --quiet
    make program silent

-v, --verbose
    make program more verbose

-n, --not-really
    Show what would be done without actually doing anything.

MASTER COMMANDS

  setup

Sets up postgres archiving, creates necessary directory structures on slave.

  sync

Synchronizes in-progress WAL files to slave.

  syncdaemon

Start WAL synchronization in daemon mode. This will start periodically synching the in-progress WAL files to slave.

The following parameters are used to drive the syncdaemon: loop_delay - how long to sleep between the synchs. use_xlog_functions - use record based shipping to synchronize in-progress WAL segments.

  stop

Deconfigures postgres archiving.

  periodic

Runs periodic command, if configured. This enables to execute arbitrary commands on interval, useful for synchronizing scripts, config files, crontabs etc.

  listbackups

List backup sets available on slave node.

  backup

Creates a new base backup from master database. Will purge expired backups and WAL files on slave if keep_backups is specified. During a backup a lock file is created in slave completed_wals directory. This is to prevent simultaneous backups and resulting corruption. If running backup is terminated, the BACKUPLOCK file may have to be removed manually.

  restore <set> <dst>

EXPERIMENTAL. Attempts to restore the backup from slave to master.

SLAVE COMMANDS

  boot

Stop log playback and bring the database up.

  pause

Pauses WAL playback.

  continue

Continues previously paused WAL playback.

  listbackups

Lists available backups.

  backup

EXPERIMENTAL. Creates a new backup from slave data. Log replay is paused, slave data directory is backed up to full_backup directory and log replay resumed. Backups are rotated as needed. The idea is to move the backup load away from production node. Usable from postgres 8.2 and up.

  restore [src][dst]

Restores the specified backup set to target directory. If specified without arguments the latest backup is moved to slave data directory (doesn’t obey retention rules). If src backup is specified the backup is copied (instead of moving). Alternative destination directory can be specified with dst.

CONFIGURATION

  Common settings

job_name

    Optional. Indentifies this script, used in logging. Keep unique if using central logging.

logfile

    Where to log.

use_skylog

    Optional. If nonzero, skylog.ini is used for log configuration.

  Master settings

pidfile

    Pid file location for syncdaemon mode (if running with -d). Otherwise not required.

master_db

    Database to connect to for pg_start_backup() etc. It is not a good idea to use dbname=template if running syncdaemon in record shipping mode.

master_data

    Master data directory location.

master_config

    Master postgresql.conf file location. This is where archive_command gets updated.

master_restart_cmd

    The command to restart master database, this used after changing archive_mode parameter. Leave unset, if you cannot afford to restart the database at setup/stop.

slave

    Slave host and base directory.

slave_config

    Configuration file location for the slave walmgr.

completed_wals

    Slave directory where archived WAL files are copied.

partial_wals

    Slave directory where incomplete WAL files are stored.

full_backup

    Slave directory where full backups are stored.

config_backup

    Slave directory where configuration file backups are stored. Optional.

loop_delay

    The frequency of syncdaemon updates. In record shipping mode only incremental updates are sent, so smaller interval can be used.

use_xlog_functions

    Use pg_xlog functions for record based shipping (available in 8.2 and up).

compression

    If nonzero, a -z flag is added to rsync cmdline. Will reduce network traffic at the cost of extra CPU time.

periodic_command

    Shell script to be executed at specified time interval. Can be used for synchronizing scripts, config files etc.

command_interval

    How ofter to run periodic command script. In seconds, and only evaluated at log switch times.

hot_standby ===

    Boolean. If set to true, walmgr setup will set wal_level to hot_standby (9.0 and newer).

  Sample master.ini

    [wal-master]
    logfile              = master.log
    pidfile              = master.pid
    master_db            = dbname=template1
    master_data          = /var/lib/postgresql/8.0/main
    master_config        = /etc/postgresql/8.0/main/postgresql.conf
    slave                = slave:/var/lib/postgresql/walshipping
    completed_wals       = %(slave)s/logs.complete
    partial_wals         = %(slave)s/logs.partial
    full_backup          = %(slave)s/data.master
    loop_delay           = 10.0
    use_xlog_functions   = 1
    compression          = 1

  Slave settings

slave_data

    Postgres data directory for the slave. This is where the restored backup is copied/moved.

slave_config_dir

    Directory for postgres configuration files. If specified, "walmgr restore" attempts to restore configuration files from config_backup directory.

slave_stop_cmd

    Script to stop postmaster on slave.

slave_start_cmd

    Script to start postmaster on slave.

slave

    Base directory for slave files (logs.complete, data.master etc)

slave_bin

    Specifies the location of postgres binaries (pg_controldata, etc). Needed if they are not already in the PATH.

completed_wals

    Directory where complete WAL files are stored. Also miscellaneous control files are created in this directory (BACKUPLOCK, STOP, PAUSE, etc.).

partial_wals

    Directory where partial WAL files are stored.

full_backup

    Directory where full backups are stored.

keep_backups

    Number of backups to keep. Also all WAL files needed to bring earliest

    backup up to date are kept. The backups are rotated before new backup is started, so at one point there is actually one less backup available.

    It probably doesn’t make sense to specify keep_backups if periodic backups are not performed - the WAL files will pile up quickly.

    Backups will be named data.master, data.master.0, data.master.1 etc.

archive_command

    Script to execute before rotating away the oldest backup. If it fails backups will not be rotated.

slave_pg_xlog

    Set slave_pg_xlog to the directory on the slave where pg_xlog files get written to. On a restore to the slave walmgr.py will create a symbolic link from data/pg_xlog to this location.

backup_datadir

    Set backup_datadir to no to prevent walmgr.py from making a backup of the data directory when restoring to the slave. This defaults to yes

  Sample slave.ini

    [wal-slave]
    logfile              = slave.log
    slave_data           = /var/lib/postgresql/8.0/main
    slave_stop_cmd       = /etc/init.d/postgresql-8.0 stop
    slave_start_cmd      = /etc/init.d/postgresql-8.0 start
    slave                = /var/lib/postgresql/walshipping
    completed_wals       = %(slave)s/logs.complete
    partial_wals         = %(slave)s/logs.partial
    full_backup          = %(slave)s/data.master
    keep_backups         = 5
    backup_datadir       = yes