CedarBackup2.config

Package CedarBackup2 :: Module config |
[hide private]
[frames] | no frames]

# Module config

source code

Provides configuration-related objects.

# Summary

Cedar Backup stores all of its configuration in an XML document typically called `cback.conf`. The standard location for this document is in `/etc`, but users can specify a different location if they want to.

The `Config` class is a Python object representation of a Cedar Backup XML configuration file. The representation is two-way: XML data can be used to create a `Config` object, and then changes to the object can be propogated back to disk. A `Config` object can even be used to create a configuration file from scratch programmatically.

The `Config` class is intended to be the only Python-language interface to Cedar Backup configuration on disk. Cedar Backup will use the class as its internal representation of configuration, and applications external to Cedar Backup itself (such as a hypothetical third-party configuration tool written in Python or a third party extension module) should also use the class when they need to read and write configuration files.

# Backwards Compatibility

The configuration file format has changed between Cedar Backup 1.x and Cedar Backup 2.x. Any Cedar Backup 1.x configuration file is also a valid Cedar Backup 2.x configuration file. However, it doesn't work to go the other direction, as the 2.x configuration files contains additional configuration is not accepted by older versions of the software.

# XML Configuration Structure

A `Config` object can either be created "empty", or can be created based on XML input (either in the form of a string or read in from a file on disk). Generally speaking, the XML input must result in a `Config` object which passes the validations laid out below in the Validation section.

An XML configuration file is composed of seven sections:

- reference: specifies reference information about the file (author, revision, etc)
- extensions: specifies mappings to Cedar Backup extensions (external code)
- options: specifies global configuration options
- peers: specifies the set of peers in a master's backup pool
- collect: specifies configuration related to the collect action
- stage: specifies configuration related to the stage action
- store: specifies configuration related to the store action
- purge: specifies configuration related to the purge action

Each section is represented by an class in this module, and then the overall `Config` class is a composition of the various other classes.

Any configuration section that is missing in the XML document (or has not been filled into an "empty" document) will just be set to `None` in the object representation. The same goes for individual fields within each configuration section. Keep in mind that the document might not be completely valid if some sections or fields aren't filled in - but that won't matter until validation takes place (see the Validation section below).

# Unicode vs. String Data

By default, all string data that comes out of XML documents in Python is unicode data (i.e. `u"whatever"`). This is fine for many things, but when it comes to filesystem paths, it can cause us some problems. We really want strings to be encoded in the filesystem encoding rather than being unicode. So, most elements in configuration which represent filesystem paths are coverted to plain strings using util.encodePath. The main exception is the various `absoluteExcludePath` and `relativeExcludePath` lists. These are not converted, because they are generally only used for filtering, not for filesystem operations.

# Validation

There are two main levels of validation in the `Config` class and its children. The first is field-level validation. Field-level validation comes into play when a given field in an object is assigned to or updated. We use Python's `property` functionality to enforce specific validations on field values, and in some places we even use customized list classes to enforce validations on list members. You should expect to catch a `ValueError` exception when making assignments to configuration class fields.

The second level of validation is post-completion validation. Certain validations don't make sense until a document is fully "complete". We don't want these validations to apply all of the time, because it would make building up a document from scratch a real pain. For instance, we might have to do things in the right order to keep from throwing exceptions, etc.

All of these post-completion validations are encapsulated in the Config.validate method. This method can be called at any time by a client, and will always be called immediately after creating a `Config` object from XML data and before exporting a `Config` object to XML. This way, we get decent ease-of-use but we also don't accept or emit invalid configuration files.

The Config.validate implementation actually takes two passes to completely validate a configuration document. The first pass at validation is to ensure that the proper sections are filled into the document. There are default requirements, but the caller has the opportunity to override these defaults.

The second pass at validation ensures that any filled-in section contains valid data. Any section which is not set to `None` is validated according to the rules for that section (see below).

Reference Validations

No validations.

Extensions Validations

The list of actions may be either `None` or an empty list `[]` if desired. Each extended action must include a name, a module and a function. Then, an extended action must include either an index or dependency information. Which one is required depends on which order mode is configured.

Options Validations

All fields must be filled in except the rsh command. The rcp and rsh commands are used as default values for all remote peers. Remote peers can also rely on the backup user as the default remote user name if they choose.

Peers Validations

Local peers must be completely filled in, including both name and collect directory. Remote peers must also fill in the name and collect directory, but can leave the remote user and rcp command unset. In this case, the remote user is assumed to match the backup user from the options section and rcp command is taken directly from the options section.

Collect Validations

The target directory must be filled in. The collect mode, archive mode and ignore file are all optional. The list of absolute paths to exclude and patterns to exclude may be either `None` or an empty list `[]` if desired.

Each collect directory entry must contain an absolute path to collect, and then must either be able to take collect mode, archive mode and ignore file configuration from the parent `CollectConfig` object, or must set each value on its own. The list of absolute paths to exclude, relative paths to exclude and patterns to exclude may be either `None` or an empty list `[]` if desired. Any list of absolute paths to exclude or patterns to exclude will be combined with the same list in the `CollectConfig` object to make the complete list for a given directory.

Stage Validations

The target directory must be filled in. There must be at least one peer (remote or local) between the two lists of peers. A list with no entries can be either `None` or an empty list `[]` if desired.

If a set of peers is provided, this configuration completely overrides configuration in the peers configuration section, and the same validations apply.

Store Validations

The device type and drive speed are optional, and all other values are required (missing booleans will be set to defaults, which is OK).

The image writer functionality in the `writer` module is supposed to be able to handle a device speed of `None`. Any caller which needs a "real" (non-`None`) value for the device type can use `DEFAULT_DEVICE_TYPE`, which is guaranteed to be sensible.

Purge Validations

The list of purge directories may be either `None` or an empty list `[]` if desired. All purge directories must contain a path and a retain days value.

Author: Kenneth J. Pronovici <pronovic@ieee.org>

Classes | [hide private]
| ActionDependencies Class representing dependencies associated with an extended action.
| ActionHook Class representing a hook associated with an action.
| PreActionHook Class representing a pre-action hook associated with an action.
| PostActionHook Class representing a pre-action hook associated with an action.
| ExtendedAction Class representing an extended action.
| CommandOverride Class representing a piece of Cedar Backup command override configuration.
| CollectFile Class representing a Cedar Backup collect file.
| CollectDir Class representing a Cedar Backup collect directory.
| PurgeDir Class representing a Cedar Backup purge directory.
| LocalPeer Class representing a Cedar Backup peer.
| RemotePeer Class representing a Cedar Backup peer.
| ReferenceConfig Class representing a Cedar Backup reference configuration.
| ExtensionsConfig Class representing Cedar Backup extensions configuration.
| OptionsConfig Class representing a Cedar Backup global options configuration.
| PeersConfig Class representing Cedar Backup global peer configuration.
| CollectConfig Class representing a Cedar Backup collect configuration.
| StageConfig Class representing a Cedar Backup stage configuration.
| StoreConfig Class representing a Cedar Backup store configuration.
| PurgeConfig Class representing a Cedar Backup purge configuration.
| Config Class representing a Cedar Backup XML configuration document.
| ByteQuantity Class representing a byte quantity.
| BlankBehavior Class representing optimized store-action media blanking behavior.

Functions | [hide private]
readByteQuantity(parent, name) Read a byte size value from an XML document. | source code
addByteQuantityNode(xmlDom, parentNode, nodeName, byteQuantity) Adds a text node as the next child of a parent, to contain a byte size. | source code

Variables | [hide private]
| DEFAULT_DEVICE_TYPE = ``'``cdwriter``'`` The default device type.
| DEFAULT_MEDIA_TYPE = ``'``cdrw-74``'`` The default media type.
| VALID_DEVICE_TYPES = ``[``'``cdwriter``'``,` `'``dvdwriter``'``]`` List of valid device types.
| VALID_MEDIA_TYPES = ``[``'``cdr-74``'``,` `'``cdrw-74``'``,` `'``cdr-80``'``,` `'``cdrw-80``'``,``...`` List of valid media types.
| VALID_COLLECT_MODES = ``[``'``daily``'``,` `'``weekly``'``,` `'``incr``'``]`` List of valid collect modes.
| VALID_ARCHIVE_MODES = ``[``'``tar``'``,` `'``targz``'``,` `'``tarbz2``'``]`` List of valid archive modes.
| VALID_ORDER_MODES = ``[``'``index``'``,` `'``dependency``'``]`` List of valid extension order modes.
| logger = `logging.getLogger("CedarBackup2.log.config")`
| VALID_CD_MEDIA_TYPES = ``[``'``cdr-74``'``,` `'``cdrw-74``'``,` `'``cdr-80``'``,` `'``cdrw-80``'``]``
| VALID_DVD_MEDIA_TYPES = ``[``'``dvd+r``'``,` `'``dvd+rw``'``]``
| VALID_COMPRESS_MODES = ``[``'``none``'``,` `'``gzip``'``,` `'``bzip2``'``]`` List of valid compress modes.
| VALID_BLANK_MODES = ``[``'``daily``'``,` `'``weekly``'``]``
| VALID_BYTE_UNITS = ``[`0`,` 1`,` 2`,` 4`]``
| VALID_FAILURE_MODES = ``[``'``none``'``,` `'``all``'``,` `'``daily``'``,` `'``weekly``'``]``
| REWRITABLE_MEDIA_TYPES = ``[``'``cdrw-74``'``,` `'``cdrw-80``'``,` `'``dvd+rw``'``]``
| ACTION_NAME_REGEX = ``'``^[a-z0-9]*$``'``
| __package__ = ``'``CedarBackup2``'``

Function Details | [hide private]

### readByteQuantity(parent, name) | source code
Read a byte size value from an XML document. A byte size value is an interpreted string value. If the string value ends with "MB" or "GB", then the string before that is interpreted as megabytes or gigabytes. Otherwise, it is intepreted as bytes. Parameters: `parent` - Parent node to search beneath. `name` - Name of node to search for. Returns: ByteQuantity parsed from XML document

### addByteQuantityNode(xmlDom, parentNode, nodeName, byteQuantity) | source code
Adds a text node as the next child of a parent, to contain a byte size. If the `byteQuantity` is None, then the node will be created, but will be empty (i.e. will contain no text node child). The size in bytes will be normalized. If it is larger than 1.0 GB, it will be shown in GB ("1.0 GB"). If it is larger than 1.0 MB ("1.0 MB"), it will be shown in MB. Otherwise, it will be shown in bytes ("423413"). Parameters: `xmlDom` - DOM tree as from `impl.createDocument()`. `parentNode` - Parent node to create child for. `nodeName` - Name of the new container node. `byteQuantity` - ByteQuantity object to put into the XML document Returns: Reference to the newly-created node.

Variables Details | [hide private]

  ### VALID_MEDIA_TYPES List of valid media types. Value:
      ['cdr-74', 'cdrw-74', 'cdr-80', 'cdrw-80', 'dvd+r', 'dvd+rw']