=encoding utf8

=head1 NAME

HADaemon::Control - Create init scripts for Perl high-available (HA) daemons

=head1 DESCRIPTION

HADaemon::Control provides a library for creating init scripts for HA daemons in perl.
It allows you to run one or more main processes accompanied by a set of standby processes.
Standby processes constantly check presence of main ones and if later exits or dies
promote themselves and replace gone main processes. By doing so, HADaemon::Control
achieves high-availability and fault tolerance for a service provided by the deamon.
Your perl script just needs to set the accessors for what and how you
want something to run and the library takes care of the rest.

The library takes idea and interface from L<Daemon::Control> and combine them
with facilities of L<IPC::ConcurrencyLimit::WithStandby>. L<IPC::ConcurrencyLimit::WithStandby>
implements a mechanism to limit the number of concurrent processes in a cooperative
multiprocessing environment. For more information refer to the documentation
of L<IPC::ConcurrencyLimit> and L<IPC::ConcurrencyLimit::WithStandby>

=head1 SYNOPSIS

    #!/usr/bin/env perl

    use strict;
    use warnings;
    use HADaemon::Control;

    my $dc = HADaemon::Control->new({
        name => 'test.pl',
        user => 'nobody',
        pid_dir => '/tmp/test',
        log_file => '/tmp/test.log',
        program => sub { sleep 10; },
    });

    exit $dc->run();

You can then call the program:

    /usr/bin/my_program_launcher.pl start

By default C<run> will use @ARGV for the action, and exit with an LSB compatible
exit code. For finer control, you can use C<run_command>, which will return
the exit code, and accepts the action as an argument.  This enables more programatic
control, as well as running multiple instances of L<HADaemon::Control> from one script.

    my $dc = HADaemon::Control->new({
        ...
    });

    my $exit = $daemon->run_command(“start”);

=head1 CONSTRUCTOR

The constructor takes the following arguments.

=head2 name

The name of the program the daemon is controlling. This will be used in status messages.
See also C<process_name_change>.

=head2 program

This should be a coderef of actual programm to run.

    $daemon->program( sub { ... } );

=head2 program_args

This is an array ref of the arguments for the program. Args will be given to the program
coderef as @_, the HADaemon::Control instance that called the coderef will be passed
as the first arguments.  Your arguments start at $_[1].

    $daemon->program_args( [ 'foo', 'bar' ] );

=head2 pid_dir

This option defines directory where all pidfile will be created

    $daemon->pid_dir('/var/run/my_program_launcher');

=head2 ipc_cl_options

This option gives ability to tune settings of underlying L<IPC::ConcurrencyLimit::WithStandby> object.
By default HADaemon::Control sets following settings:

    ipc_cl_options => {
        type              => 'Flock',                             # the only supported type
        max_procs         => 1,                                   # one main process
        standby_max_procs => 1,                                   # one standby process
        interval          => 1,                                   # stanby tries to acquire main lock every second
        retries           => sub { 1 },                           # keep retrying forever
        path              => $daemon->pid_dir . '/lock/',         # path for main locks
        standby_path      => $daemon->pid_dir . '/lock-standby/', # path for standby locks
    },

=head2 main_stop_file

This option provides an alternative way of stopping main processes apart of sending a signal (ex. TERM). If specified,
HADaemon::Control touch this file and wait L<stop_file_kill_timeout> or L<kill_timeout> seconds hoping that main processes will respect the file
and exit. If not, normal termination loop is entered (i.e. sending sequence of signals TERM TERM INT KILL).
The filename can include %p which is replaced by PID of a process. Default value is undef.

=head2 standby_stop_file

The path to stop file for standby process. See C<do_start>, C<do_stop>, C<do_restart> for details. By default is set to:

    $daemon->standby_stop_file($daemon->pid_dir . '/standby-stop-file');

=head2 stop_signals

An array ref of signals that should be tried (in order) when stopping the daemon. Default signals are C<TERM>, C<TERM>, C<INT> and C<KILL> (yes, C<TERM> is tried twice).

=head2 log_file

HADaemon::Control uses C<log_file> for two purposes:

=over 4

=item * HADaemon::Control redirects STDOUT and STDERR for forked processes to given file

=item * HADaemon::Control prints its own log to given file

=back

If you don't want to mix logs of the application and init script consider using C<stdout_file> and C<stderr_file>.
Verbosity of logs of HADaemon::Control can be controled by C<HADC_TRACE> environment variable.

=head2 process_name_change

If set, HADaemon::Control will set name of the process to C<name>. Also, it adds process_name_change option into C<ipc_cl_options>.
As result, C<process_name_change> makes nice names for both main and standby processes. For example:

    my $dc = HADaemon::Control->new({
        name => 'My test daemon',
        pid_dir => '/tmp/test',
        log_file => '/tmp/test.log',
        program => sub { sleep 10; },
        process_name_change => 1,
    });

leads to:

    My test daemon              # name of main process
    My test daemon - standby    # name of standby process

=head2 user

When set, the username supplied to this accessor will be used to set
the UID attribute. When this is used, C<uid> will be changed from
its initial settings if you set it (which you shouldn't, since you're
using usernames instead of UIDs). See L</uid> for setting numerical
user ids.

    $daemon->user('www-data');

=head2 group

When set, the groupname supplied to this accessor will be used to set
the GID attribute. When this is used, C<gid> will be changed from
its initial settings if you set it (which you shouldn't, since you're
using groupnames instead of GIDs). See L</gid> for setting numerical
group ids.

    $daemon->group('www-data');

=head2 uid

If provided, the UID that the program will drop to when forked. This will
only work if you are running as root. Accepts numeric UID. For usernames
please see L</user>.

    $daemon->uid( 1001 );

=head2 gid

If provided, the GID that the program will drop to when forked. This will
only work if you are running as root. Accepts numeric GID, for groupnames
please see L</group>.

    $daemon->gid( 1001 );

=head2 umask

If provided, the umask of the daemon will be set to the umask provided,
note that the umask must be in oct. By default the umask will not be
changed.

    $daemon->umask( 022 );

    Or:

    $daemon->umask( oct("022") );

=head2 directory

If provided, chdir to this directory before execution.

=head2 stdout_file

If provided stdout of main process will be redirected to the given file.

    $daemon->stdout_file( "/tmp/mydaemon.stdout" );

=head2 stderr_file

If provided stderr of main process will be redirected to the given file.

    $daemon->stderr_file( "/tmp/mydaemon.stderr" );

=head2 kill_timeout

This provides an amount of time in seconds between trying different means
of terminating the daemon. This value should be increased if your daemon has
a longer shutdown period. By default 1 second is used.

    $daemon->kill_timeout( 7 );

This value is used both for stop files and signals.

=head2 stop_file_kill_timeout

This is a more specific variant of L<kill_timeout>. It provides the amount of seconds
we allow the daemon to terminate itself once a stop file has been created.

If provided, this value has priority over L<kill_timeout>.

    $daemon->stop_file_kill_timeout( 42 );

=head2 signal_kill_timeout

This is a more specific variant of L<kill_timeout>. It provides the amount of seconds
between firing different signals to terminate the daemon.

If provided, this value has priority over L<kill_timeout>.

    $daemon->signal_kill_timeout( 42 );

=head2 quiet

If this boolean flag is set to a true value all output from the init script
(NOT your daemon) to STDOUT will be suppressed.

    $daemon->quiet( 1 );

=head2 close_fds_on_start

By default HADC closes all file descriptors apart of STDIN, STDOUT, STDERR
and lock fd (see HADC_lock_fd) when it starts main process.
This is done to make sure that main and standby processes are really
independent of each other.

If this behaivor is not desirable, one can set close_fds_on_start to 0.
But it should be understood that if parent process open any file descriptor
(i.e. establish any connection, open file, create pipe, etc) those FDs will
be available in *both* main and standby processes. Such case can be dangerous. Consider this:
- parent processes (i.e. processes which run HADC) connect to DB
- HADC starts main processes which uses that DB connection
- HADC starts standby process
- after a while main processes crashes and leave connectiong to DB in unpredicted state
- standby processes promotes to main one and try to use *same* connection to DB.
  Since connection is in unpredicted state, it failes to use connection and crashes too.

=head1 INIT FILE CONSTRUCTOR OPTIONS

The constructor also takes the following arguments to generate init file. See L</do_get_init_file>.

=head2 path

The path of the script you are using HADaemon::Control in. This will be used in
the LSB file generation to point it to the location of the script. If this is
not provided, the absolute path of $0 will be used.

=head2 init_config

The name of the init config file to load. When provided your init script will
source this file to include the environment variables. This is useful for setting
a C<PERL5LIB> and such things.

    $daemon->init_config( "/etc/default/my_program" );

    If you are using perlbrew, you probably want to set your init_config to
    C<$ENV{PERLBREW_ROOT} . '/etc/bashrc'>.

=head2 init_code

When given, whatever text is in this field will be dumped directly into
the generated init file.

    $daemon->init_code( "Arbitrary code goes here." )

=head2 lsb_start

The value of this string is used for the 'Required-Start' value of
the generated LSB init script. See L<http://wiki.debian.org/LSBInitScripts>
for more information.

    $daemon->lsb_start( '$remote_fs $syslog' );

=head2 lsb_stop

The value of this string is used for the 'Required-Stop' value of
the generated LSB init script. See L<http://wiki.debian.org/LSBInitScripts>
for more information.

    $daemon->lsb_stop( '$remote_fs $syslog' );

=head2 lsb_sdesc

The value of this string is used for the 'Short-Description' value of
the generated LSB init script. See L<http://wiki.debian.org/LSBInitScripts>
for more information.

    $daemon->lsb_sdesc( 'My program...' );

=head2 lsb_desc

The value of this string is used for the 'Description' value of
the generated LSB init script. See L<http://wiki.debian.org/LSBInitScripts>
for more information.

    $daemon->lsb_desc( 'My program controls a thing that does a thing.' );

=head1 METHODS

=head2 run_command

This function will process an action on the HADaemon::Control instance.
Valid arguments are those which a C<do_> method exists for, such as 
B<start>, B<stop>, B<restart>. Returns the LSB exit code for the
action processed.

=head2 run

This will make your program act as an init file, accepting input from
the command line. Run will exit with 0 for success and uses LSB exit
codes. As such no code should be used after ->run is called. Any code
in your file should be before this. This is a shortcut for 

    exit HADaemon::Control->new(...)->run_command( @ARGV );

=head2 do_start

Is called when start is given as an argument. Starts the forking and
exits. The forking includes starting C<ipc_cl_options->{max_procs}> main and
C<ipc_cl_options->{standby_max_procs}> standby processes. Exit with success
only if all processes were spawned. Called by:

    /usr/bin/my_program_launcher.pl start

=head2 do_stop

Is called when stop is given as an argument. Stops the all running proceses
which belongs to the daemon if it can. Stopping is done via:

=over 4

=item * touching C<standby_stop_file> file to stop standby processes and prevent
new proceses to be started via C<do_fork> command.

=item * if C<main_stop_file> specified, touch it to stop main processes. See L<main_stop_file>.

=item * send "TERM TERM INT KILL" sequence of signals to kill main processes

=back

Called by:

    /usr/bin/my_program_launcher.pl stop

=head2 do_restart

Is called when restart is given as an argument. This command triggers restart cycle which
includes several steps:

=over 4

=item * stop all standby daemons by touching C<standby_stop_file>

=item * start new instances of standby processes

=item * kill main processes one by one. Once a main processes is dead, running standby immediately
become main one hence minimize downtime to C<ipc_cl_options->{interval}> seconds (or miliseconds).

=item * again start standby processes to compensate the lost of standby processes

=back

Called by:

    /usr/bin/my_program_launcher.pl restart

=head2 do_hard_restart

Is called when hard_restart is given as an argument. Calls C<do_stop> and C<do_start>.
Called by:

    /usr/bin/my_program_launcher.pl hard_restart

=head2 do_fork

Is called when fork is given as an argument. This command is almost equal to L<do_start>,
but is design for periodical run in a cronjob. Called by:

    /usr/bin/my_program_launcher.pl fork

=head2 do_reload

Is called when reload is given as an argument. Sends a HUP signal to the
main processes.

    /usr/bin/my_program_launcher.pl reload

=head2 do_status

Is called when status is given as an argument. Displays the statuses of the
program (i.e. all running processes), basic on the PID files. Called by:

    /usr/bin/my_program_launcher.pl status

=head2 do_foreground

Is called when B<foreground> is given as an argument. Starts the
program or code reference and stays in the foreground -- no forking
and locking is done, regardless of the compile-time arguments.
Additionally, turns C<quiet> on to avoid showing L<HADaemon::Control> output.

    /usr/bin/my_program_launcher.pl foreground

=head2 do_get_init_file

Is called when get_init_file is given as an argument. Dumps an LSB
compatible init file, for use in /etc/init.d/. Called by:

    /usr/bin/my_program_launcher.pl get_init_file

=head2 pretty_print

This is used to display status to the user. It accepts a message and a color.
It will default to green text, if no color is explicitly given. Only supports
red and green. If C<HADC_NO_COLORS> environment variable is set no colors are used.

    $daemon->pretty_print( "My Status", "red" );

=head1 KNOWN ISSUES

HADaemon::Control uses C<flock> based locks. This type of locks have property of getting
inherited accross C<fork> system call. This behavior is not desirable and actually
destructible for HADaemon::Control. Once the locked is inherited, two processes
(parent and child) will own the same lock. Only releasing the lock from both processes
allows another one to acuire the lock. To prevent such behivour HADaemon::Control exposes
lock's file descriptor via HADC_lock_fd environment variable.

If an application forks, a child process should close lock's file descriptor right after
exiting from C<fork> syscal. One of the possible ways is to run:

    $ENV{HADC_lock_fd} and POSIX::close($ENV{HADC_lock_fd});

Another source of troubles could be the fact that HADC closes all file descriptors apart
of STDIN, STDOUT, STDERR and lock fd upon starting main processes (tunable via C<close_fds_on_start>).
This's done for a reason. See C<close_fds_on_start> for details.

=head1 AUTHOR

Ivan Kruglov, C<ivan.kruglov@yahoo.com>

=head1 CONTRIBUTORS

Alexey Surikov C<alexey.surikov@booking.com>

=head1 ACKNOWLEDGMENT

This module was inspired by module L<Daemon::Control|https://github.com/symkat/Daemon-Control>.

This module was originally developed for Booking.com.
With approval from Booking.com, this module was generalized
and put on CPAN, for which the authors would like to express
their gratitude.

=head1 COPYRIGHT AND LICENSE

(C) 2013, 2014 Ivan Kruglov. All rights reserved.

This code is available under the same license as Perl version
5.8.1 or higher.

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

=head2 AVAILABILITY

The most current version of HADaemon::Control can be found at L<https://github.com/ikruglov/HADaemon-Control>
