=pod

=head1 NAME

Net::Appliance::Session::Cookbook::Recipe06 - Storing Device Configurations in
Files

=head1 NOTE

This Cookbook was contributed to the L<Net::Appliance::Session> project by
Nigel Bowden. Source code from the Cookbook is shipped in the C<examples>
folder of this module's distribution.

=head1 PROBLEM

You want to telnet or SSH to all of your Cisco IOS devices and pull off their
configurations so that you can build up a historic record of configuration
files across your network.

=head1 SOLUTION

Well, if this is a cookbook, then this particular section is a 6 course
banquet. This is a huge script compared to the previous examples we have
looked at.

It brings together all of the areas we have looked at previously, but I have
pulled in the services of a number of other Perl modules to try to provide a
usable utility that could perhaps be used on your network to gather your Cisco
device configurations.

All of the credentials that are required to access your devices are now stored
in a CSV file, so devices can be added or removed as required. As different
devices may have differing access requirements (e.g. transports,
username/passwords required etc.) each device is configured with its own set
of access credentials. CSV files (comma seperated values) are easy to update
using many spreadsheet applications, making it easy to maintain the device
credential data for this script.

Also, the script uses an external intialization file (.ini) to configure the
operation of the script (e.g. where configuration files will be dumped),
rather than having to hard-code values in the body of the script itself.

The script comprises two files:

=over 4

=item Recipe_06.pl

The file below, in this Cookbook.

=item setup.ini

The initialization file that must be in the same directory as C<Recipe_06.pl>.

=item device_credentials.csv

The CSV file whose location is configured in C<setup.ini>.

=back

The main body of the script should be relatively easy to follow, as it is very
similar to previous examples. Where new functions are introduced (e.g. reading
and parsing a CSV file), then those functions have (in the main) been
separated off in to additional subroutines.

Here is the full solution:

 use strict;
 use warnings;
 use Carp;
 
 use Net::Appliance::Session;
 use Config::INI::Simple;
 use Text::CSV_XS;
 use File::Basename;
 
 # create Config::INI::Simple object and read in our ini file data
 my $ini_filename = dirname($0) . "/setup.ini"; # where we can find ini file
 my %ini_data = parse_ini_file($ini_filename);
 
 # get the device credentials for our devices
 my @device_data = parse_data_file($ini_data{device_csv_file});
 
 # step through each device and try to get and store our configs 
 DEVICE:
 for my $device_ref (@device_data) {
     
     my $device_name = $device_ref->{device_name} || $device_ref->{device_ip};
     
     # set up some logging
     my $debug_log = "$ini_data{debug_dir}/$device_name.debug.log";
     my $error_log = "$ini_data{error_dir}/$device_name.error.log";
     
     # create our config file name
     my $file_timestamp = file_timestamp();
     my $running_config_file = "$ini_data{repository_dir}/$device_name.$file_timestamp.conf";
     
     # create our Net::Appliance::Session with the transport for this device
     my $session_obj = Net::Appliance::Session->new(
         Host      => $device_ref->{device_ip},
         Transport => $device_ref->{transport},
     );
     
     # send the debug for this session to a device-specific file
     $session_obj->input_log($debug_log);
     my @running_config;
     
     # generate the required fields for the priv_array subroutine
     my @priv_array = priv_array($device_ref);
     
     # tell our session object we don't need enable password if none supplied
     unless ($priv_array[0]) {
         $session_obj->do_privileged_mode(0);
     }
     
     # do our interactive (Telnet/SSH) stuff...
     eval {
     
         # try to login to the ios device, ignoring host check
         $session_obj->connect( connect_hash($device_ref), SHKC => 0 );
           
         if ( $priv_array[0] ) {
     
             # if we need to use some enable credentials, supply them
             $session_obj->begin_privileged( @priv_array );
         }
         
         # get our running config
         @running_config =  $session_obj->cmd('show running');
     };
     
     # did we get an error ?
     if ($@) {
         
         # log error to file and move on to next device
         log_error( error_report($@, $device_name), $error_log );
         next DEVICE;
     }
     
     # chop out the extra info top and bottom of the config
     @running_config = @running_config[ 2 .. (@running_config -1)];
     
     # dump the config to a file
     open(CONFIG , " > $running_config_file")
        or warn("Unable to open config file for : $device_name : $!");
     print CONFIG @running_config;
     close CONFIG;
     
     # close down our session
     $session_obj->close;
 }   
 
 #####################################
 # Subroutines
 #####################################
 sub parse_ini_file {
 
     # parse our ini file to get the parameters we need in to
     # some convenient variables
     
     my $ini_filename = shift or croak("No ini file name passed");
     
     my $config_obj = Config::INI::Simple->new();
 
     $config_obj->read($ini_filename) or die("Cannot open ini file : $ini_filename (reason: $!)");
     
     my %ini_data; # variable to use as data hash to hold all ini file data
     
     # set up some variables for later use
     $ini_data{error_dir}        = $config_obj->{Logs}->{error_dir};
     $ini_data{debug_dir}        = $config_obj->{Logs}->{debug_dir};
     $ini_data{device_csv_file}  = $config_obj->{CSV}->{device_csv_file};
     $ini_data{repository_dir}   = $config_obj->{Repository}->{repository_dir};
     $ini_data{timestamp_format} = $config_obj->{Repository}->{timestamp_format};
 
     return %ini_data;
 }
 
 sub parse_data_file {
     
     # parse the CSV data file we are using to hold our  device
     # credential data
     
     my $device_csv_file = shift or croak("No csv file named passed");
 
     # create our csv object ready to parse in the data from our csv file
     my $csv_obj = Text::CSV_XS->new();
     
     #read in our csv file
     open my $csv_fh, "< $device_csv_file"
        or croak("Cannot open device csv file : $device_csv_file (reason: $!)");
     
     # take off the top row that has the field names
     my $top_row = $csv_obj->getline($csv_fh); 
     
     my @device_data;
     
     # take each entry in the CSV file and massage it into a complex
     # data structure
     while (my $data_row = $csv_obj->getline($csv_fh)) {
         my $hash_ref;
         map { ($hash_ref->{$_} = shift @$data_row) } @$top_row;
         
         push(@device_data, $hash_ref);
     }
     
     close $csv_fh;
     return @device_data;
 }
 
 sub connect_hash {
     
     # depending on the combination of credentials supplied, determine
     # the combination of login username/password to use
     
     my $device_ref = shift or croak("No device credentials ref passed !");
 
     # decide which set of credentials we have
     if ( exists($device_ref->{username}) && exists($device_ref->{password}) ) {
         
         # username & password supplied
         return ( Name => $device_ref->{username}, Password => $device_ref->{password} );
     }
     elsif ( exists($device_ref->{password}) ) {
         
         # password only supplied
         return ( Password => $device_ref->{password} );
     }
     else {
         croak("Invalid or missing credentials to log in to this device : "
                    . $device_ref->{device_name} );
     }
 }
 
 sub priv_array {
     
     # depending on the combination of credentials supplied, determine
     # the combination of enable username/password to use
     
     my $device_ref = shift or croak("No device credentials ref passed !");
 
     # decide which set of priv credentials we have
     if ( $device_ref->{enable_username} && $device_ref->{enable_password} ) {
         
         # username & password supplied
         return( $device_ref->{enable_username}, $device_ref->{enable_password} );
     }
     elsif ( $device_ref->{enable_password} ) {
         
         # password only supplied
         return( $device_ref->{enable_password} );
     }    
     elsif ( $device_ref->{enable_username} ) {
         
         # username only supplied - error !
         croak( "Invalid enable login credentials provided (only username provided !" );
     }    
     else {
         
         # no enble pwd required (assume drop straight in to enable mode)
         return 0;
     }
 }
 
 sub error_report {
     
     # standard subroutine used to extract failure info when
     # interactive session fails
     
     my $err         = shift or croak("No err !");
     my $device_name = shift or croak("No device name !");
     
     my $report; # holder for report message to return to caller
     
     if ( UNIVERSAL::isa($err, 'Net::Appliance::Session::Exception') ) {
             
             # fault description from Net::Appliance::Session
             $report  =  "We had an error during our Telnet/SSH session to device  : $device_name \n"; 
             $report .= $err->message . " \n";
                 
             # message from Net::Telnet
             $report .= "Net::Telnet message : " . $err->errmsg . "\n"; 
                 
             # last line of output from your appliance  
             $report .=  "Last line of output from device : " . $err->lastline . "\n\n";
 
         }
         elsif (UNIVERSAL::isa($err, 'Net::Appliance::Session::Error') ) {
             
             # fault description from Net::Appliance::Session
             $report  = "We had an issue during program execution to device : $device_name \n";
             $report .=  $err->message . " \n";
     
         }
         else {
             
             # we had some other error that wasn't a deliberately created exception
             $report  = "We had an issue when accessing the device : $device_name \n";
             $report .= "The reported error was : $err \n";
         }
         
         return $report;
 }
 
 sub log_error {
     
     # log an error message to a file
     
     my $error_message = shift;
     my $file_name     = shift;
     
     open(ERR , " > $file_name") or carp("Unable to error file : $file_name : $!");
     print ERR $error_message;
     close ERR;
 }
 
 sub file_timestamp {
 
     # create a timestamp to add to the conf files created
 
     my ($sec,$min,$hour,$mday,$mon,$year,$wday,$yday,$isdst) = localtime(time);
     
     if ($ini_data{timestamp_format} eq 'uk') {
     
         # UK format
         return sprintf( "%02d-%02d-%4d-%02d%02d", $mday, ($mon + 1), ($year + 1900), $hour, $min );
     }
     else {
         
         # US format
         return sprintf( "%02d-%02d-%4d-%02d%02d", ($mon + 1), $mday, ($year + 1900), $hour, $min );   
     }
 }

=head1 DISCUSSION

As discussed above, the main body of this script pulls together most of the
principles that are discussed in previous examples. So, I'm not going to
discuss the whole script in detail (my fingers can't take the typing!). Instead
I'll hightlight the major differences to previous examples and focus more on
the features provided by the various subroutines that have been added.

One thing to mention before we go on are the couple of additional modules that
we've pulled in to the script to provide us with some valuable features for
our solution:

=over 4

=item C<Config::INI::Simple>

Allows us to read in a standard Windows-format ini file in to a Perl data structure.

=item C<Text::CSV_XS>

Allows us to easily read in the contents of a CSV file.

=back

Looking through the main body of the code, after the intial declarations to
use the new modules listed above, we call a couple of new subroutines to read
in the device credential data in our CSV file and the C<.ini> file that
configures the operation of the script:

 use strict;
 use warnings;
 use Carp;
 
 use Net::Appliance::Session;
 use Config::INI::Simple;
 use Text::CSV_XS;
 use File::Basename;
 
 # create Config::INI::Simple object and read in our ini file data
 my $ini_filename = dirname($0) . "/setup.ini"; # where we can find ini file
 my %ini_data = parse_ini_file($ini_filename);
 
 # get the device credentials for our devices
 my @device_data = parse_data_file($ini_data{device_csv_file});

The call to the C<parse_ini_file> subroutine will read in the C<.ini> file that
is passed as a parameter and make its data available as a hash.

The call to the C<parse_data_file> subroutine will read in the credentials CSV
file that is passed as a parameter and make its data available as an array.

Next, we step through each of the devices whose credentials we retrieved from
the CSV file, and create some file names for the various files we may create
for this particular device:

 # step through each device and try to get and store our configs 
 DEVICE:
 for my $device_ref (@device_data) {
     
     my $device_name = $device_ref->{device_name} || $device_ref->{device_ip};
     
     # set up some logging
     my $debug_log = "$ini_data{debug_dir}/$device_name.debug.log";
     my $error_log = "$ini_data{error_dir}/$device_name.error.log";
     
     # create our config file name
     my $file_timestamp = file_timestamp();
     my $running_config_file = "$ini_data{repository_dir}/$device_name.$file_timestamp.conf";
     
     # create our Net::Appliance::Session with the transport for this device
     my $session_obj = Net::Appliance::Session->new(
         Host      => $device_ref->{device_ip},
         Transport => $device_ref->{transport},
     );
     
     # send the debug for this session to a device-specific file
     $session_obj->input_log($debug_log);

We will create a debug file for each device, that will be placed in the
C<debug_dir> that is defined in our C<.ini> file.

Also, if we experience an error accessing a device, we will also dump an error
message in to a device-specific error file in the C<error_dir> that is defined
in our C<.ini> file.

One other new item is the C<file_timestamp> subroutine that is called to give
us a timestamp to add to our device configuration file. Each time a device
configuration is succesfully retrieved, a file based on the device name and
the date/time will be created. This allows us to keep a historical view of
configurations retrieved from our network devices.

Once we get into our actual interactive session, we have to make a few
decisions about how we are going to access each device based on its
credentials. For instance, some devices will require a username/login
combination for access, others may only require a password.

To cater for these varying access requirements, a couple of new subroutines
have been added to provide the correct combination of credentials, dependant
upon the credentials provided in our CSV file:

     # generate the required fields for the priv_array subroutine
     my @priv_array = priv_array($device_ref);
     
     # tell our session object we don't need enable password if none supplied
     unless ($priv_array[0]) {
         $session_obj->do_privileged_mode(0);
     }
     
     # do our interactive (Telnet/SSH) stuff...
     eval {
     
         # try to login to the ios device, ignoring host check
         $session_obj->connect( connect_hash($device_ref), SHKC => 0 );
           
         if ( $priv_array[0] ) {
     
             # if we need to use some enable credentials, supply them
             $session_obj->begin_privileged( @priv_array );
         }
         
         # get our running config
         @running_config =  $session_obj->cmd('show running');
     };

The C<connect_hash> subroutine will look at the C<username> and C<password>
fields of the credentials CSV file and supply the correct format array to pass
to the C<connect> method of our session object.

To get in to enable mode with the C<begin_privileged> method, we need to ensure
that we supply the correct credentials for each device. These are provided by
the enable_username and enable_password fields of the CSV file. The
C<priv_array> subroutine will format an array to ensure that we can access
enable mode (if required). If no enable mode credentials are provided, it is
assumed that we are already in enable mode, and the session object is advised
using the C<do_privileged_mode> method.

Well, that's pretty much it in terms of discussing this script. As I say, much
of it has been covered in previous examples.

One footnote I would like to add is that although this is a fully functioning
script (apart from the bugs in it I haven't found yet!), it could still use
quite a lot of improvements for a production environment. For instance, there
is no checking of the type or presence of device credential data in the CSV
file. If there is no IP address for a device, the whole thing falls in a heap.
But, I had to draw the line somewhere when developing this script to
demonstrate how it might be used in the real world, so please bear these
limitations in mind.

=head1 INSTALLATION

As stated previously, the script comprises three files.

In addition to this script, there is a CSV file that contains the device
credential data, and an initialization file that contains various parameters
to control the operation of the script.

In case you don't have the two accompanying files, below are examples of the
files that you require. If both of the files are placed in to the same
directory as this script, then everything should work just fine.

To run the script (once you have all three files configured and together),
just run it from the command line:

 $ perl Recipe_06.pl

If all runs well, you can maybe run it from your system scheduler to automate
the process. However, don't forget to check for error logs periodically to
make sure things are still running OK!

Configuration files that are retrieved should be available in the directory
configured by the C<repository_dir> parameter of the C<setup.ini> file.

=head2 C<device_credentials.csv>

 device_name,device_ip,username,password,enable_username,enable_password,transport
 rtr_1,10.250.249.215,cisco,cisco,,,Telnet
 rtr_2,10.250.249.216,cisco,cisco,,,Telnet

=head2 C<setup.ini>

 [Logs]
 # directory for error logs
 error_dir=/home/nigel/perl/logs
 
 # directory for session debugging logs 
 debug_dir=/home/nigel/perl/logs
 
 [CSV]
 # CSV file with containing device credential info
 device_csv_file=./device_credentials.csv
 
 [Repository]
 # directory where configs are dumped
 repository_dir=/home/nigel/perl/configs
 
 # format of config file timestamp: 'uk' or 'us'
 timestamp_format=uk

=head1 SEE ALSO

=over 4

=item L<Net::Appliance::Session>

=back

=head1 AUTHOR

Nigel Bowden, with POD formatting by Oliver Gorwits.

=head1 COPYRIGHT & LICENSE

Copyright (c) Nigel Bowden 2007. All Rights Reserved.

You may distribute and/or modify this documentation under the same terms as
Perl itself.

=cut
