
= londiste3(1) =

== NAME ==

londiste3 - tool for managing trigger-based replication for PostgreSQL.

== SYNOPSIS ==

  londiste3 <config.ini> command [subcommand] [options]

== DESCRIPTION ==

Londiste allows you to setup and administer the replication, and is run as a
daemon to handle the replication itself. (Londiste is just a complex
PgQ Consumer).

See <<examples,EXAMPLES>> below to start your first trigger-based
replication in a few steps.

The 'command' is one of Node, Replication, Information or Internal
commands listed below.

Londiste introduces the notion of 'takeover'.  It is the action when a
local node takes over the work of another node. The other node can be a
root node or a branch node and it can be dead or alive when the action
is run.

Londiste also allows (among many other features): cascading replication,
partial replication and custom handlers for replication.

== GENERAL OPTIONS ==

  -V, --version::
    Print version info and exit.

  -h, --help::
    Show this help message and exit.

  -q, --quiet::
    Log only errors and warnings.

  -v, --verbose::
    Log verbosely.

  -d, --daemon::
    Run in daemon mode (go background).

== SPECIFIC OPTIONS ==

  --ini::
    Display sample ini file.

  --set="'param=val[,param=value]'"::
    Override config setting.

== DAEMON OPTIONS ==

  -r, --reload::
    Reload config (send SIGHUP).

  -s, --stop::
    Stop program safely (send SIGINT).

  -k, --kill::
    Kill program immediately (send SIGTERM).

== REPLICATION EXTRA ARGUMENTS ==

  --rewind::
    Change queue position according to destination.

  --reset::
    Reset queue position on destination side.

== NODE INITIALIZATION COMMANDS ==

Initialization commands will set up "public connect string" for
current node.  It is a connect string that other nodes will
use to connect to current node.  The local Londiste itself
uses 'db' option from config file to connect to local node,
which can have different user rights than scripts coming
over public connect string.

Connect strings can be set in either command line or config file.
Command line overrides config.  Setting them up in config
might be more comfortable.  See `londiste3 --ini` for details.

=== create-root <node> [<public_connstr_for_node>] ===

Initializes a Master node.

The <node> is the name of the node, it should be unique.

=== create-branch <node> [<public_connstr_for_node>] [--provider=<connstr_to_provider>] ===

Initializes a Slave node which can be used as a reference for other
nodes.

The <node> is the name of the node, it should be unique.
The <public_connstr_for_node> argument is the connection string to the database
on the current node and <connstr_to_provider> is the connection string to the
provider database (it can be a root node or a branch node).

=== create-leaf <node> [<public_connstr_for_node>] [--provider=<connstr_to_provider>] ===

Initializes a slave node which can not be used as a reference for other
nodes.

The <node> is the name of the node, it should be unique.
The <public_connstr_for_node> argument is the connection string to the database
on the current node and <connstr_to_provider> is the connection string to the
provider database (it can be a root node or a branch node).

  --merge='qname'::
    combined queue name

== NODE ADMINISTRATION COMMANDS ==

=== pause ===

Pause the consumer: the replication of the events is stopped and can be
resumed later.

=== resume ===

When the consumer has been paused, let it replay again.

=== change-provider <tonode> ===

Make <tonode> become the new provider for the current node.

TODO: londiste.py need update (param change from --provider)

=== takeover <fromnode> [--target=<tonode>] [--all] [--dead] ===

This command allows to achieve failover and switchover for any of your
providers (root or branch nodes).

  --target='tonode'::
    Target node of the takeover.

  --all::
    In addition to take over the work from the 'fromnode', make other
    nodes change their provider to the current node.

  --dead::
    Don't wait to take the new role and flag the 'fromnode' as dead.

  --dead='deadnode'::
    Assume node is dead.
    TODO : why use this one ?

  --dead-root::
    Old node was root.

  --dead-branch::
    Old node was branch

=== resurrect ===

In case root was down and taken over with `--dead-root`, this command
fixes queue contents on that old root to match the rest of cascade.
Events that did not propagate to rest of the cascade before failure
(lost events) are dumped into file in JSON format and deleted from queue.
Then the node is registered into cascade and it's worker will
be paused.

It requires that there is another active root in cascade and
there is consumer named `NODENAME.gravestone` registered
on same node, it uses that to get position where rest of the
cascade moved on.

It does not touch actual tables, which means there must
be external mechanism to survive unsynced tables.  Options:

* Ignore lost events.  May need trigger/rule on tables to handle conflicts.
* Replay the lost events on new root.
  May need trigger/rule on tables to handle conflicts.
* Roll back table changes.
  May need old version of row stored in events.
  (Achieved with `backup` parameter to `pgq.logutriga`)

=== drop-node <node> ===

Remove the node from the Londiste replication.

Londiste triggers on the node are removed but Londiste or PgQ are not
removed.

=== tag-dead <node> ===

Tag the node as dead, the command can be run from any node in the
replication.

=== tag-alive <node> ===

Tag the node as alive, the command can be run from any node in the
replication.

== INFORMATION COMMANDS ==

=== status ===

Show status of the replication viewed by the current node.

The output is a tree of the members of the replication with their lags,
last tick, status and the number of tables in state: ok/half/ignored
(replicated, initial copy not finnished, table not replicated locally).

=== members ===

Show members of the replication viewed by the current node.

Output the nodes name, status and node location (connection string to
the node).

=== show-consumers [--node] ===

TODO: command is not working

== REPLICATION DAEMON COMMAND ==

=== worker ===

Replay events to subscriber: it is needed to make the replication active
as it will start to replay the events.

== REPLICATION ADMINISTRATION COMMANDS ==

=== add-table <table> [args] ===

Add the table to the replication.

See <<add_args,ADD ARGUMENTS>> below for the list of possible arguments.

=== remove-table <table> ===

Remove the table from the replication.

=== add-seq <seq> [args] ===

Add the sequence to the replication.

See <<add_args,ADD ARGUMENTS>> below for the list of possible arguments.

=== remove-seq <seq> ===

Remove the sequence from the replication.

=== tables ===

Show all tables on provider.

=== seqs ===

Show all sequences on provider.

=== missing ===

List tables subscriber has not yet attached to.

=== resync <table> ===

Do full copy of the table, again.

== ADD ARGUMENTS [[add_args]] ==

  --all::
    Include all possible tables.

  --wait-sync::
    Wait until newly added tables are synced fully.

  --dest-table='table'::
    Redirect changes to different table.

  --force::
    Ignore table differences.

  --expect-sync::
    No copy needed.

  --skip-truncate::
    Keep old data.

  --create::
    Create table/sequence if not exist, with minimal schema.

  --create-full::
    Create table/sequence if not exist, with full schema.

  --trigger-flags='trigger_flags'::
    Trigger creation flags, see below for details.

  --trigger-arg='trigger_arg'::
    Custom trigger arg (can be specified multiply times).

  --no-triggers::
    Dont put triggers on table (makes sense on leaf node).

  --handler='handler'::
    Custom handler for table.

  --handler-arg='handler_arg'::
    Argument to custom handler.

  --copy-node='NODE_NAME'::
    Do initial copy from that node instead from provider.
    Useful if provider does not contain table data locally
    or is simply under load.

  --merge-all::
    Merge tables from all source queues.

  --no-merge::
    Don't merge tables from source queues.

  --max-parallel-copy='max_parallel_copy'::
    Max number of parallel copy processes.

  --skip-non-existing::
    Skip objects that do not exist.

Trigger creation flags (default: AIUDL):

- I - ON INSERT
- U - ON UPDATE
- D - ON DELETE
- Q - use pgq.sqltriga() as trigger function
- L - use pgq.logutriga() as trigger function
- B - BEFORE
- A - AFTER
- S - SKIP

== REPLICATION EXTRA COMMANDS ==

=== check ===

Compare table structure on both sides.

=== fkeys ===

Print out fkey drop/create commands.

=== compare [<table>] ===

Compare table contents on both sides.

  --count-only::
    Just count rows, do not compare data.

=== repair [<table>] [--force] ===

Repair data on subscriber.

  --force::
    Ignore lag.

=== execute [filepath] ===

Execute SQL files on each node of the cascaded queue.  The SQL file is
executed locally in single transaction and inserted into queue in same
transaction.  Thus guaranteeing that is will be replayed in subscriber
databases at correct position.

The filename is stored in `londiste.applied_execute` table, and checked
before execution.  If same filename already exists, the SQL execution
is skipped.

==== SQL meta-data attributes ====

SQL file can contain attributes that limit where the SQL is executed:

  --*--
  --*-- Local-Table: mytable, othertable,
  --*--              thirdtable
  --*-- Local-Sequence: thisseq
  --*--

The magic comments are searched only in file start, before any actual SQL
statement is seen.  Empty lines and lines with regular SQL comments
are ignored.

Supported keys:

  Local-Table::
    Table must be added to local node with `add-table`.

  Local-Sequence::
    Sequence must be added to local node with `add-seq`.

  Local-Destination::
    Table must be added to local node and actual destination table must exists.
    This is for cases where table is added to some nodes with handler
    that does not need actual table to exist.

  Need-Table::
    Physical table must exist in database.  It does not matter if it is
    replicated or not.

  Need-Sequence::
    Sequence must exist in database.

  Need-Function::
    Database function must exists.  The function name is in form `function_name(nargs)`.
    If the `(nargs)` portion is missed then nargs is taken as 0.

  Need-View::
    A view must exist in database.

  Need-Schema::
    Schema mist exist in database.

Londiste supports table renaming, where table is attached to queue with one name
but events are applied to local table with different name.  To make this
work with EXECUTE, the Local-Toble and Local-Destination support tag replacement,
where queue's table name that is mentioned in attribute is replaced
with actual table name in local database:

  --*-- Local-Table: mytable
  ALTER TABLE @mytable@ ...;


=== show-handlers ['handler'] ===

Show info about all or a specific handler.

=== wait-sync ===

Wait until all added tables are copied over.

=== wait-provider ===

Wait until local node passes latest queue position on provider.

=== wait-root ===

Wait until local node passes latest queue position on root.


== INTERNAL COMMAND ==

=== copy ===

Copy table logic.

== EXIT STATUS ==

  0::
    Successful program execution.

== ENVIRONMENT ==

PostgreSQL environment variables can be used.

== EXAMPLES [[examples]] ==

Londiste is provided with HowTos to help you make your fisrt steps:

- How to set up simple replication.
- How to set up cascaded replication.
- How to set up table partitioning (handlers).
