setup-storage(8)

NAME
setup-storage - automatically prepare storage devices

SYNOPSIS
setup-storage [-X] [-f filename] [-d] [-h]

DESCRIPTION
Using FAI disk_config files, setup-storage computes effective partition and volume sizes and executes the necessary commands to configure storage devices. It manages disk drives as well as RAID and LVM volumes. It handles all file systems supported by parted(8) as well as ntfs but is flexible enough to be extended to further types as well. Once the storage devices are prepared, an appropriate fstab(5) file is generated. Without the -X parameter setup-storage runs in test-only mode and does not execute commands other than writing disk labels to a blank disk. The exit code of setup-storage is 0 if all operations were performed successfully and non-zero if an error occurs.

USAGE
To enable the use of the new partioning tool setup-storage you have to set USE_SETUP_STORAGE=1 in the according class file in configspace (like /srv/fai/config/class/FAIBASE.var for example).

OPTIONS

-X  Really write the configuration to disk. Otherwise setup-storage runs in test-only mode.

-f filename
    Normally setup-storage selects an appropriate configuration from $FAI/disk_config/ by picking the first class from classes that has an existing file. If however -f is given the configuration in filename is used.

-d  Enable debugging output.

-h  Display the synopsis and version info and exit.

ENVIRONMENT
setup-storage will use the following environment variables:

disklist
    The disklist variable must contain a newline separated list of disk drives available in the system. Their order matters as they may be referred to as disk1, etc. in disk_config.

debug
    If debug is set to a non-zero value all actions and details to track the operation of setup-storage are printed to stderr.

FAI The location of the config space to find the disk_config directory.

classes
    The list of FAI classes to determine the appropriate configuration to choose.

LOGDIR
    setup-storage generates disk_var.sh and fstab (see below) in this directory.

flag_initial
    This variable determines if partitions should be preserved when they are tagged as to be preserved. Normally set by the list of FAI flags (FAI_FLAGS).

FILES
If setup-storage executes successfully, an fstab(5) file matching the specified configuration is generated as $LOGDIR/fstab. Furthermore the file $LOGDIR/disk_var.sh is generated. This file defines the following variables, if not yet set: SWAPLIST, ROOT_PARTITION, BOOT_PARTITION (which is only set in case this resides on a disk drive), and BOOT_DEVICE. The latter two describe the partition and disk/RAID/LVM device hosting the mount point for /boot. If /boot has no extra mount point, / is used instead. You may source $LOGDIR/disk_var.sh to get the variables set.

SYNTAX
This section describes the syntax of disk_config files file ::= <lines> EOF
lines ::= EOL
          /* empty lines or whitespace only */
          | <comment> EOL
          | <config> EOL
comment ::= #.*
config ::= disk_config lvm( <lvmoption>)*
           | disk_config raid( <raidoption>)*
           | disk_config cryptsetup( <cryptsetupoption>)*
           | disk_config tmpfs
           | disk_config end
           | disk_config disk[[:digit:]]+( <option>)*
           | disk_config [^[:space:]]+( <option>)*
           /* fully qualified device-path or short form, like hda, whereby full
            * path is assumed to be /dev/hda; may contain shell globbing such
            * as /dev/disk/by-id/scsi-* */
           | <volume>
lvmoption ::= /* empty */
           | preserve_always:[^/,\s\-]+-[^/,\s\-]+(,[^/,\s\-]+-[^/,\s\-]+)*
           /* preserve volumes -- always */
           | preserve_reinstall:[^/,\s\-]+-[^/,\s\-]+(,[^/,\s\-]+-[^/,\s\-]+)*
           /* preserve volumes -- unless the system is installed for the
           first time */
           | preserve_lazy:[^/,\s\-]+-[^/,\s\-]+(,[^/,\s\-]+-[^/,\s\-]+)*
           /* preserve volumes -- unless these don't exist yet */
           | always_format:[^/,\s\-]+-[^/,\s\-]+(,[^/,\s\-]+-[^/,\s\-]+)*
           /* run mkfs on the volumes, even if marked as preserve */
           | resize:[^/,\s\-]+-[^/,\s\-]+(,[^/,\s\-]+-[^/,\s\-]+)*
           /* attempt to resize partitions */
           | fstabkey:(device|label|uuid)
           /* when creating the fstab, the key used for defining the device
           may be the device (/dev/xxx), a label given using -L, or the uuid
           */
raidoption ::= /* empty */
           | preserve_always:[[:digit:]]+(,[[:digit:]]+)*
           /* preserve volumes -- always */
           | preserve_reinstall:[[:digit:]]+(,[[:digit:]]+)*
           /* preserve volumes -- unless the system is installed for the
           first time */
           | preserve_lazy:[[:digit:]]+(,[[:digit:]]+)*
           /* preserve volumes -- unless these don't exist yet */
           | always_format:[[:digit:]]+(,[[:digit:]]+)*
           /* run mkfs on the volumes, even if marked as preserve */
           | fstabkey:(device|label|uuid)
           /* when creating the fstab the key used for defining the device
           may be the device (/dev/xxx), a label given using -L, or the uuid
           */
cryptsetupoption ::= /* empty */
           | randinit
           /* initialise all encrypted partitions with random data */
option ::= /* empty */
           | preserve_always:[[:digit:]]+(,[[:digit:]]+)*
           /* preserve partitions -- always */
           | preserve_reinstall:[[:digit:]]+(,[[:digit:]]+)*
           /* preserve partitions -- unless the system is installed for the
           first time */
           | preserve_lazy:[[:digit:]]+(,[[:digit:]]+)*
           /* preserve partitions -- unless these don't exist yet */
           | always_format:[[:digit:]]+(,[[:digit:]]+)*
           /* run mkfs on the partitions, even if marked as preserve */
           | resize:[[:digit:]]+(,[[:digit:]]+)*
           /* attempt to resize partitions */
           | disklabel:(msdos|gpt|gpt-bios)
           /* write a disklabel - default is msdos */
           | bootable:[[:digit:]]+
           /* mark a partition bootable, default is / */
           | virtual
           /* do not assume the disk to be a physical device, use with xen */
           | fstabkey:(device|label|uuid)
           /* when creating the fstab the key used for defining the device
           may be the device (/dev/xxx), a label given using -L, or the uuid
           */
           | sameas:(disk[[:digit:]]+|[^[:space:]]+)
           /* Indicate that this disk will use the same scheme
           as the given device. The referenced device must be
           defined before the device using this option. Use only
           with identical hardware.
           */
           | align-at:([[:digit:]]+[kKMGTPiB]*)
           /* Align partitions at multiples of the given block size (unit
           defaults to MiB, if omitted). Such an alignment, e.g., 4K, might be
           important for proper performance of RAID arrays which use a logical
           block size other than the sector size of the underlying disks. It
           must, however, always be a multiple of this sector size.
           */
volume ::= <type> <mountpoint> <size> <filesystem> <mount_options> <fs_options>
           | vg <name> <size> <fs_options>
           /* lvm vg */
           | tmpfs <mountpoint> <tmpfs_size> <mount_options>
           /* tmpfs volume */
type ::= primary
         /* for physical disks only */
         | logical
         /* for physical disks only */
         | raid[0156]
         /* raid level */
         | luks
         /* encrypted partition using LUKS */
         | tmp
         /* encrypted partition for /tmp usage, will be
            recreated with a random key at each boot and
            reformatted as ext2 */
         | swap
         /* encrypted partition for swap space usage, will
            be recreated with a random key at each boot and
            reformatted as swap space */
         | [^/[:space:]]+-[^/[:space:]]+
         /* lvm logical volume: vg name and lv name*/
mountpoint ::= (-|swap|/[^[:space:]]*)
               /* do not mount, mount as swap, or mount at fully qualified path */
name ::= [^/[:space:]]+
         /* lvm volume group name */
sizespec ::= RAM:[[:digit:]]+%|[[:digit:]]+[kKMGTP%iB]*
         /* size in kilo (KiB), mega (default, MiB), giga (GiB), tera (TiB),
          * petabytes (PiB) or percentage of disk size or RAM size; integers
          * only, no decimal numbers.
          * In future releases KB, MB, GB, ... will be treated as 1000 instead
          * of 1024 (KiB, MiB, GiB, ...) multipliers */ size ::= <sizespec>(-(<sizespec>)?)?(:resize)?
         /* size, possibly given as a range; physical partitions or lvm logical
          * volumes only */
         | -<sizespec>(:resize)?
         /* size given as upper limit; physical partitions or lvm logical
          * volumes only */
         | [^,:[:space:]]+(:(spare|missing))*(,[^,:[:space:]]+(:(spare|missing))*)*
         /* devices and options for a raid or lvm vg */
tmpfs_size ::= <sizespec>
         /* tmpfs size */
mount_options ::= [^[:space:]]+
filesystem ::= -
               | swap
               | [^[:space:]]
               /* mkfs.xxx must exist */
fs_options ::= (createopts=".*"|tuneopts=".*"|(pv|vg|lv|md)createopts=".*")*
               /* options to append to mkfs.xxx and to the filesystem-specific
                * tuning tool, pvcreate, vgcreate, lvcreate or mdadm */

MIGRATION FROM SETUP_HARDDISKS
The major differences to the prior configuration syntax are:

•   The disk_config ... line allows for the keywords lvm and raid

•   Options may need to be appended to the disk_config line

•   The ";" is not used anymore, the options that were given there have now been split up

•   The filesystem is now an explicit parameter; note that the order of filesystem/mount-options is the same /etc/fstab as opposed to the previous format of disk_config.

•   Any options to mkfs.xxx may be given using createopts="".

•   The "preserveX" and "boot" options are one of the options now given on the disk_config line, using preserve_reinstall, preserve_always, or preserve_lazy, and bootable. preserve_always is equivalent to the previous preserveX option, whereas preserve_reinstall preserves the partition unless "initial" is given as one of the FAI_FLAGS. preserve_lazy allows to preserve partitions only if these exist already. Otherwise they are created.

•   The "always_format" option overrides preserving filesystems (via one of the "preserveX" options), like the "format" option in setup_harddisks.

•   Support for LVM and RAID is completely new

•   Resizing partitions and filesystems is supported

EXAMPLES
Simple configuration of /dev/hda

disk_config hda preserve_always:6,7 disklabel:msdos bootable:3
primary	/boot	20-100	ext3	rw
primary	swap	1000	swap	sw
primary	/	12000	ext3	rw	createopts="-b 2048"
logical	/tmp	1000	ext3	rw,nosuid
logical	/usr	5000	ext3	rw
logical	/var	10%-	ext3	rw
logical	/nobackup	0-	xfs	rw

•   Preserve the 6th and the 7th partition. The disklabel is msdos which is the default for x86. Furthermore the 3rd partition is made bootable.

•   Create a primary partition /dev/hda1 with a size between 20 and 100 MiB and mount it read-write as /boot; it is formatted using ext3 filesystem.

•   /dev/hda2 will be a swap space of 1000 MiB

•   /dev/hda3 should be formatted using ext3 filesystem; when calling mkfs.ext3 the option "-b 2048" is appended.

•   Create the logical partition /dev/hda5

•   Make /dev/hda7 at least 10% of the disk size

•   Use mkfs.xfs to format the partition 8

Create a softRAID

    disk_config raid
    raid1	/	sda1,sdd1	ext2	rw,errors=remount-ro
    raid0	-	disk2.2,sdc1,sde1:spare:missing	ext2	default

•   Create a RAID-1 on /dev/sda1 and /dev/sdd1, format using mkfs.ext2 and mount it as /

•   Create a RAID-0 on the second partition of the second disk, /dev/sdc1, and /dev/sde1 as a spare partition. The latter may be missing.

Simple LVM example

    disk_config sda bootable:1
    primary	/boot	500	ext3	rw
    primary	-	4096-	-	-
    disk_config lvm
    vg	my_pv	sda2
    my_pv-_swap	swap	2048	swap	sw
    my_pv-_root	/	2048	ext3	rw

Crypt example

    disk_config /dev/sdb
    primary	/	21750	ext3	defaults,errors=remount-ro
    primary	/boot	250	ext3	defaults
    logical	-	4000	-	-
    logical	-	2000	-	-
    logical	-	10-	-	-
    disk_config cryptsetup
    swap	swap	/dev/sdb5	swap	defaults
    tmp	/tmp	/dev/sdb6	ext2	defaults
    luks	/local00	/dev/sdb7	ext3	defaults,errors=remount-ro  createopts="-m	0"

tmpfs example

    disk_config tmpfs
    tmpfs	/tmp	RAM:20%	defaults
    tmpfs	/scratch	3GiB	defaults

•   Mount a tmpfs on /tmp with a maximum size equal to 20% of the total amount of RAM in the machine. This is equivalent to using size=20% in the tmpfs mount options.

•   Mount a tmpfs on /scratch with a maximum size of 3 GiB.

External log device example

    disk_config /dev/sda fstabkey:uuid bootable:2
    primary	/	20GiB	ext3		defaults
    primary	/boot	250	ext2		defaults
    primary	swap	4GiB	swap		defaults
    logical	-	256	ext3_journal	-
    logical	-	256	ext4_journal	-
    logical	-	256	xfs_journal	-
    disk_config /dev/sdb fstabkey:uuid
    primary	/mnt/ext3	33%	ext3:journal=/dev/sda5		defaults
    primary	/mnt/ext4	33%	ext4:journal=/dev/sda6		defaults
    primary	/mnt/xfs	33%	xfs:journal=/dev/sda7		defaults

•   Mount an ext3 filesystem on /dev/sdb1 with an external journal on /dev/sda5

•   Mount an ext4 filesystem on /dev/sdb2 with an external journal on /dev/sda6

•   Mount an XFS filesystem on /dev/sdb3 using /dev/sda7 as the log device

CAVEATS

•   Partition UUID cannot be obtained: In case a partition was previously used as part of a software RAID volume and now is intended as swap space, udev fails when asked for a UUID. This happens because mkswap does not overwrite the previous RAID superblock. You can remove it using mdadm --zero-superblock <device>.

•   Machine does not boot because not partition is marked as bootable: If the bootable option is not specified, not partition will be marked as such. Modern BIOSes don't seem to require such markers anymore, but for some systems it may still be necessary. Previous versions of setup-storage by default marked the partition mounting / as bootable, but this is not a sane default for all cases. If you want to be sure not boot failures happen because of a missing bootable marker, explicitly set the bootable option. Of course, there are lots of other reasons why a system may fail to boot.

•   Crypto support requires some site-specific changes: If you use cryptsetup stanza, a crypttab(5) file and key files for all luks volumes will be created (unless you used the passphrase option). The key files are left in /tmp/fai; you will want to copy these to some removable media. To make encrypted root devices actually usable, you need to add busybox (and initramfs-tools) to your package config.

•   For backwards compatibility or other system-specific reasons an alignment to cylinder boundaries may be necessary. Yet other systems will have other alignment constraints. setup-storage sets the alignment as follows: If align-at is set, align accordingly. Otherwise, if any partition on the particular disk is to be preserved, default to cylinder alignment. Else use sector alignment.

SEE ALSO
This program is part of FAI (Fully Automatic Installation). The FAI homepage is http://fai-project.org. Further documentation, including coding related information, is maintained in a wiki page at http://wiki.fai-project.org/index.php/Setup-storage.

AUTHOR
FAI is courtesy of Thomas Lange <lange@informatik.uni-koeln.de>. Michael Tautschnig <mt@debian.org> contributed the initial version of setup-storage to replace the previous setup-harddisks, with the help of Christian Kern.