Backup & Recovery: Inexpensive Backup Solutions for Open Systems

2017-07-07 02:10:07

3.4. Backing Up with the dump Utility

For many environments, dump may be all you need to ensure good-quality backups. There's a lot of controversy surrounding dump, though, stemming from the fact that it doesn't access the data through the filesystem the way most other backup utilities do. dump accesses the filesystem device directly. This is why it can back up files without changing their access times. However, it's also why the manpages for dump have always said to unmount filesystems prior to backing them up. Of course, no one ever does that, hence the controversy.

dump on Mac OS and Linux

Linux administrators should be aware that dump is not considered a good way to back up a Linux system, and dump doesn't support the HFS+ filesystem in Mac OS. RedHat officially deprecated dump in RedHat 9, and the following quote from Linux Torvalds sums up the Linux community's attitude towards dump:

"dump simply won't work reliably at all even in 2.4.x: the buffer cache and the page cache (where all the actual data is) are not coherent. This is only going to get even worse in 2.5.x, when the directories are moved into the page cache as well. So anybody who depends on dump getting backups right [on a Linux system] is already playing Russian roulette with their backups. It's not at all guaranteed to get the right resultsyou may end up having stale data in the buffer cache that ends up being "backed up".... dump may work fine for you a thousand times. But it will fail under the right circumstances. And there is nothing you can do about it."

You will have to make up your own mind on whether or not dump is right for you, but apparently dump isn't the best way to back up a Linux system.

dump and restore are available on Mac OS, but they work only with the UFS filesystem. There is no hfsdump for the HFS+ filesystem, and I know of no plans to create such a tool.

To use dump and restore for regular system backups, you need to understand the following:

How to use dump to back up a filesystem (with the appropriate options)

How the backup ends up on the volume

How to get the table of contents of a dump volume

How to manipulate the volume and restore from a backup created by dump

The limitations of dump and restore

What you should be doing if you are using dump on a regular basis

The first thing to understand is what your dump command is and what its options are. See Table 3-1 for a listing of dump commands on various Unix versions. The following section is essentially a unified manpage for these dump-like commands on specific operating systems.

Although there is a dump command on Mac OS, it does not support the HFS+ filesystem, which is the most common filesystem type on Mac OS.

Table 3-1. dump-like commands on different Unix versions
Unix version	Command
HP-UX 9.x/HP-UX 10/SunOS/IRIX	(r)dump
Solaris	ufsdump
SCO	xdump
Network Appliance	dump
AIX	`backup` and `rdump`
Linux	dump
SGI	`dump` and `xfsdump`
Tru64 Unix	`dump` and `vdump`
Linux/Mac OS	See the sidebar "dump on Mac OS and Linux"

3.4.1. Syntax of the dump Command

Let's start with the basic dump command:

# dump levelunbdsf blkg-factor density size device-name file_system

The following are examples of running this command:

To create a full backup of /home to a local tape drive called /dev/rmt/0cbn:

# dump 0unbdsf 126 141000 11500 /dev/rmt/0cbn /home

To create a full backup of /home to an optical or CD device called /backup/home.dump:

# dump 0unbdsf 126 141000 11500 /backup/home.dump /home

To create a full backup of /home to the remote tape drive /dev/rmt/0cbn on elvis:

# (r)dump 0unbdsf 126 141000 11500 elvis:/dev/rmt/0cbn /home

The preceding commands use three options (0, u, and n) that do not require arguments and four options (b, d, s, and f) that require a "companion" argument.

The dump command accepts as its first argument a list of options, then each option's argument is placed on the command line in the same order in which the options are listed. Figure 3-2 illustrates how the dump command options relate to their companion arguments.

Figure 3-2. Sample dump command

3.4.2. The Options to the dump Command

The dump utility has seven main options that are available on most platforms:

0-9: Specifies the level of backup that dump should perform.
b: Specifies the blocking factor that dump should use.
u: Tells dump to update the dumpdates file.
n: Tells dump to notify the members of the Operator group when a dump is completed.
d and s: Tells dump how large the backup volume is. dump uses these numbers to estimate how much "tape" is available.
f: Tells dump what device to use.
W, w: Tells dump to perform a dry run that tells you what filesystems need to be backed up (these are seldom used).

If you are using dump for regular system backups, you should be using most of the preceding options. It is important to note that many of these options have default values, eliminating the need to specify that option and its argument in the dump command. For example, the default backup level is usually 9. The problem with the default values is that they vary between operating systems and may also vary even on the same operating system, depending on factors such as media type. It is better to specify each of these options the same way on all your dump backups to simplify making restores at a later date.

dump and restore Save the Day

It had been a long, hard week, and we were trying to finish up a few things so we could go home. That's when we got the call. That's always when you get the call. A very important directory, which contained a seldom used but essential utility, was missing from the system. "No problem," I said, "we've got it on tape." Or so I thought. When I went to recover the files, I realized that this directory had been missing for a while. In fact, it had been missing for so long that it had not been backed up by the commercial utility we were using. You can imagine the feeling that was in my stomach.

I looked over at the old filing cabinet where we kept a pile of poorly organized, inadequately labeled, and almost forgotten ufsdump tapes. At that moment, they were the most important tapes in the world, because they had been made before we started using the commercial utility. I put those tapes in the drive, one by one, using the table of contents option of ufsrestore, in hopes that one of them would be the right one. The stack was getting shorter and shorter. Finally, one of the tapes looked like it could be the one. I switched modes, using the interactive option, and there it was. I selected the directory and extracted it. The directory was saved, and the customer never even knew that we almost weren't able to restore the data. That was one day I was really glad that I knew dump and restore. (I also learned how important it is to archive monthly full backups.)

3.4.2.1. Specifying a complete or incremental backup (09)

The first argument that you can specify is the dump level; you can use any number from 0 to 9. (See Chapter 2 for an explanation of backup levels.) Incremental dumps refer to the dumpdates file for the date of the last lower-level backup. (This file is discussed in the section "Updating the dumpdates file (u)" later in this chapter.) For example, if you are performing a level 5 backup, dump backs up all files that have changed since the last backup that was level 4 or lower. It gets the date of this backup from dumpdates (usually /etc/dumpdates). Since the dumpdates file is needed for incremental backups, you must use the u option to update it.

3.4.2.2. Specifying a blocking factor (b)

The b option specifies the number of blocks to write in a single output operation. This refers to the number of physical blocks. The size of the entire block that dump writes depends on the size of the physical block multiplied by the blocking factor. For most versions of Unix, the physical block size for dump is 1024 bytes. So, if you specify a blocking factor of 10, the size of the actual block that dump writes is 10,240, or 10 K. This option is not available on SCO.

At least one flavor of Unix allows you to change the blocking factor for dump but not for restore. This means that you can create dump volumes that you can't read! Make sure that your flavor of restore allows you to change the blocking factor.

3.4.2.3. Updating the dumpdates file (u)

The u option causes dump to update the dumpdates file for the filesystem that you backed up. (The dumpdates file is usually /etc/dumpdates, but is /var/adm/dumpdates on HP-UX 10.x.) This is a plain-text file that lists each filesystem's raw device and the date that the last backup of each level was taken on that device. Here is an example /etc/dumpdates file taken from a Solaris box:

/dev/rdsk/c0t1d0s0 0 Sun Apr 30 23:07:22 2006 /dev/rdsk/c0t1d0s0 1 Wed May 3 02:49:51 2006 /dev/rdsk/c0t3d0s0 0 Sat May 20 00:31:49 2006 /dev/rdsk/c0t3d0s0 1 Mon May 29 01:33:33 2006 /dev/rdsk/c0t3d0s0 5 Wed May 31 00:28:14 2006

You can see that device c0t1d0s0 had a level 0 backup on April 30, and a level 1 backup on May 3, 2006. Device c0t3d0s0 had a level 0 backup on May 20, a level 1 on May 29, and a level 5 on May 31.

There are a few important things to note about the dumpdates file. The first time you run dump on a system, you must first create an empty dumpdates file, and it must be owned by root. If it is not there or is not owned by root, dump does not create it. Your dump continues, but it will complain. Note that dumpdates is updated only if the entire dump completes successfully. If any errors cause dump to abort, dumpdates is not updated. This means that it is a good file to use for an automated script that checks to see if your dumps worked. The following list shows the various names and locations of the dumpdates file:

HP-UX 9.x, SunOS, Solaris, AIX, Linux, IRIX: /etc/dumpdates

HP-UX 10.0: /var/adm/dumpdates

SCO: /etc/ddate

You might not want to use the u option when making a special "one-time" backup volume, because doing so changes the behavior of other backups. For example, if you are making a one-time level 0 backup for someone and use the u option, your automated level 1 backups will reference that level 0 backup that has been given to someone else and is not a part of your normal backup pool.

The dumpdates file, whatever it may be called, can be viewed or modified with a standard text editor. You might want to do this, for example, if you know that this week's level 0 backup has been eaten by a hungry tape drive. You don't have time to rerun a full level again, but you want some sort of backup. However, if you run a level 1, it references this week's level 0 backup, which you know is no good. You can edit the level 0 line for the appropriate filesystems, changing the date to the date of last week's level that has not been eaten. Your level 1 then references last week's level 0 rather than this week's level 0, which was destroyed. This can allow you to sleep a little better after that level is destroyed, without having to rerun a complete level 0.

3.4.2.4. Notifying your backup operators (n)

The n option causes dump to notify everyone in the operator group, as specified in the /etc/group file, if a dump backup requires attention. This notification looks similar to a wall message. (This option is not available on SCO.) A dump backup may require attention when any of the following occurs:

A dump backup reaches the end of a tape, or your CD fills up.

A backup drive malfunctions, causing write errors.

There are difficulties reading from the disk drive.

3.4.2.5. Specifying density and size (d and s)

The density (d) and size (s) options do not affect how data is written to the backup media. The dump command uses them only to determine how much data can fit on a given volume and to determine when it has reached the logical-end-of-tape (LEOT, or the point at which dump thinks the volume is full) before it reaches the physical-end-of-tape (PEOT). dump then prompts the operator to switch volumes. The logic behind this is to keep the volume from hitting PEOT, because older versions of dump do not handle this well. Here is a quick explanation of these two flags:

d (density): By specifying a density, you are telling dump how much data fits on one inch of tape. (This value is really a throwback to the nine-track tape days, but dump uses it in combination with the s option to figure out how large the backup volume is.) If you want to make sure that dump uses the entire volume, use a large value such as 80,000.
s ("tape" size in feet): This option tells dump how long the tape is. It then calculates how much data fits on the tape using the values provided for size and density. If you want to make sure that dump uses the entire volume, use a large value like 500,000. Using 80,000 as the density and 500,000 as the size effectively tells dump that your volume is capable of storing 480 GB! (Yes, this and the d option both seem silly if you're backing up to disk or CD, but they are important. See the following section "Do I have to use the s and d options?" for more information.)

In actual practice, these options are very difficult to use and yield very little value. Most people fake out dump using values that make dump think it will never run out of tape. This causes dump to use the entire volume and lets it discover the PEOT if or when it gets that far. There are many reasons for this:

The dump command can now detect and handle PEOT (dump used to abort upon reaching PEOT). In Solaris, they even have an option that causes the tape to eject, and if you are using an autochanger, it then inserts the next tape. On Solaris, therefore, dump could then continue without intervention.

The calculations work only if it's the only backup that dump has put on the volume. (For example, each time you use dump, you tell it the tape is 10,000 feet long. If you have already put at least one backup on the volume, it's no longer 10,000 feet long).

If you were to use "real" values, you would probably have a small density value with a very large size value. Many Unix versions tell you that doing this can cause problems. (I'm serious. You have to make them up!)

If you want dump to actually stop before PEOT, you need to underestimate the values, which results in using less space than the volume actually has. (Some budgets necessitate using every inch of every volume that you paid for.)

Adding compression into the calculation really complicates the process, since compression is one area in which the phrase "your mileage may vary" really applies.

Avoid Creating a dump Backup Across Multiple Volumes

By "across multiple volumes," I mean that this is a single dump backup that starts on one volume, runs until it hits LEOT or PEOT, and then continues on another volume. For example, if you have a 4 GB DDS tape drive and are backing up a 2 GB filesystem and a 3 GB filesystem, the first dump backup would fit on the tape. The second one would fill up the rest of the tape, requiring you to insert a second tape to allow dump to finish (see Figure 3-3).

In my opinion, creating a backup in this manner is asking for trouble. If you have no choice, then you must do it, but it raises some questions and adds difficulty to your restore. For example, you have to load tape 1 and start reading it before you can load tape 2. It's already hard enough to do a restore in the first place! Also, I start wondering about how safe the files are that are stored near the end of the first tape. Are you sure they're safe? The dump command can be funny sometimes.

Figure 3-3. Example of a multiple-volume dump backup

3.4.2.6. Do I have to use the s and d options?

A few newer versions of dump have done away with these options and provided a new size in kilobytes option you can use to specify the size of the volume in kilobytes. Even so, I personally use the s and d options with every dump command I run so that I don't have to remember how different versions work. You will find this is a common theme throughout this book: the more things you can do the same everywhere, the fewer things you have to worry about. The more per-host and per-OS customization you do, the more trouble you can get into. (For example, the size in kilobytes option uses a different letter on each version of Unix that supports it!) In this case, using the archaic size and density options actually makes writing shell scripts much easier, because you can use the same options on most versions of Unix.

What happens, then, if you don't use the s, d, or size in kilobytes options? On some Unix flavors, dump uses the default values for size and density (except for AIX, which has apparently done away with these options altogether). Unfortunately, the default values are usually set to work with a nine-track tape. (Solaris has changed its default values to be slightly more sensible.) If this happens, dump will think it needs several volumes. The output of dump looks something like the following:

DUMP: Estimated 5860 blocks (3006KB) on 39.00 tapes.

Notice that it thinks it's going to need 39 tapes. This is what can happen if you do not use the size and density options to specify the capacity of the volume. As mentioned before, you can easily disable this feature by setting these values to some ridiculously high figure so that dump never thinks that it has run out of tape. (I personally use numbers like 1,000,000 for both.)

3.4.2.7. Specifying a backup device file (f)

The f option specifies the name of the backup device to which you are sending the data. (This "device," of course, could be either an actual tape device or a file sitting on a disk, optical platter, or CD.) If you are expecting to use the hardware compression feature of your tape drive, make sure that you choose a device that supports compression. If you want to send the data to a drive on another system, use the format remote_system_name:device. Most versions of Unix support using remote devices in dump, as long as you're alright with using rsh as an authentication mechanism.

The use of rsh and /.rhosts files is a major security hole, and many sites no longer allow their use! Don't go creating /.rhosts files everywhere and blame it on me. Make sure you investigate whether you are allowed to use rsh at your site before you start using it. If you are not allowed to use rsh, you might want to look at implementing ssh as a drop-in replacement for rsh. See the section "Using ssh or rsh as a Conduit Between Systems" near the end of this chapter for more information.

Remote devices require that the host with the remote device trust this host via the /.rhosts file. If you try to use a remote device from a nontrusted system, you might get the dreaded message:

Permission Denied

To test if you are a trusted host, try issuing the following command as root:

# rsh remote_system uname -a

If it does not work, you need to put a line with this system's name in the remote system's ~root/.rhosts file.

Unfortunately, in today's mixed environments, you don't always know what other systems think a particular system's name is. The remote system might be using DNS, NIS, or a local hosts file. When you rsh to a system, it initially sees you as an IP address. It then does a gethostbyaddr( ) and tries to resolve that address into a name. Depending on how your particular system is set up, it may consult DNS, NIS, or the local /etc/hosts file; the order in which it consults these sources also varies with your setup. If it uses the local hosts file or NIS for address resolution, it may or may not appear with a fully qualified domain name such as apollo.domain.com. If it uses DNS, it appears with the fully qualified domain name. It is important to know this because this is the name you must put into the /.rhosts file. Suppose your system is called apollo, and the remote system is elvis. If you want to rsh from apollo to elvis, you should try the easy step first. On elvis, enter this command:

$ echo apollo >>/.rhosts

If that doesn't work, apollo appears as something else to elvis (e.g., apollo.domain.com). To find out for sure, you can telnet to elvis from apollo, then use commands such as last, who, tty, or netstat to look at the field that lists the system from which you came. If it turns out to be apollo.domain.com, put that into the /.rhosts file on elvis. (For example, at one client site, it appears as apollo.DOMAIN.COM.) Once you have put the correct name in /.rhosts, rsh should work.

3.4.2.8. Displaying which filesystems need to be backed up (W and w)

The W and w dump options are available on most Unix systems and display information about which filesystems need to be backed up. Usually, the w option displays information on all filesystems, while the W option lists only those filesystems that need to be backed up, based on the backup level you have chosen. These options have slight variations between Unix flavors, so read the appropriate manpage.

3.4.2.9. Interesting options for Solaris's ufsdump utility

Solaris's ufsdump has a few options not found in other versions of Unix. It supports the l (autoloader), o (offline), a (archive file), and v (verify) options:

l: The autoloader option ejects the tape if it reaches PEOT before dump is done. It then waits up to two minutes for the next tape to be inserted. This works well with sequential autoloaders.
o: The offline option merely ejects the tape at the end of the backup, protecting the tape from being overwritten by another process.
a: The archive file option writes dump's table of contents to archive_ file (as well as writing it to the volume, as all dump commands do). This file can then be used by ufsrestore to see if a file is on a given volume without having to mount that media.
v: The verify option compares the backup to the actual filesystem. While this may sound good in theory, it requires the filesystem to be unmounted, which is not practical in many applications.

3.4.3. What a dump Backup Looks Like

This section explains one primary difference between dump and its cousins, tar and cpio. dump writes a table of contents at the beginning of each volume while tar and cpio do not.

3.4.3.1. dump records an index on the volume

The index is read during an interactive restore, allowing you to run commands such as cd and ls on this table of contents, viewing and selecting files that you want for the restore. (The restore utility is discussed later in this chapter.) This interactive restore feature is one of restore's biggest advantages over tar and cpio. Note one important thing about this index: it is made at the beginning of the backup, before it has tried to actually back up anything. The presence of the index makes the interactive restore efficient because you don't have to read the whole volume before you can see what's on it. However, the fact that it's created before the backup data is written, and possibly minutes or hours before the data is written to tape, means that files made during the backup are not included, and files deleted during the backup are listed on the index but are not actually on the volume.

3.4.3.2. Using the index to create a table of contents

You can create a table of contents of a dump volume by physically reading the contents of the index that dump creates and seeing what dump intended to write to the volume. Also, it is important to mention that this reading of the volume in no way guarantees the integrity of the actual file on the volume any more than an ls -l on a file in a directory verifies its integrity. You may be wondering why this discussion is included here, in the section about dump; it is because making this table of contents should be a part of every dump backup that you take. Having said that, how do you create a table of contents of a dump file? First, what does "dump file" really mean? Perhaps an illustration would help; see Figure 3-4.

Figure 3-4. The format of a dump tape

A volume created by dump may have multiple dump files, sometimes called partitions, on it. Each file ends in an end-of-file (EOF) mark, symbolized in Figure 3-4 by shaded areas.

You have two options if you want to obtain a table of contents for dump file 3 in Figure 3-4:

You can tell restore to read the third file on the tape using the s option; this causes restore to skip files 1 and 2 and read file 3. (This option does not apply to disk-based dump backups.)

You can manually position the tape (using mt or tpctl) so that it is sitting at the beginning of that file, then tell restore to read it as if it were the first file on the tape.

You must know the blocking factor in which the volume was written. If you are not sure, try the default by not specifying a blocking factor. If that doesn't work, see the section "How Do I Read This Volume?" in Chapter 23.

The first method is the easiest, because it involves only one step. The syntax of the command is as follows:

$ restore tsbfy file blocking-factor device

To read the third dump file on the tape with a blocking factor of 32, use the following command:

$ restore tsbfy 3 32 /dev/rmt/0cbn

Here's a list of the options used and what they do:

The t option tells restore to read the volume index and provide a table of contents.

The s option, and its accompanying argument 3, tells restore to read the third dump file on a tape.

The b option, and its accompanying argument 32, tells restore that you used a blocking factor of 32 when you wrote this dump file.

The f option, and its accompanying argument dev, specifies that the dump file is on that device.

The y option tells restore to continue in the case of errors, instead of asking you if you want to continue.

If you do choose to manually manipulate the tape, as in the second option, you need to be familiar with your Unix version's magnetic tape command. This is usually mt. It has five optionsstatus, rewind, offline, fsf, and fsrfour of which you might use when manipulating dump tapes. The format of the command is:

$ mt -t device argument

If you are planning to position the tape, make sure you are using a nonrewinding device, such as /dev/rmt/0n. Otherwise, it rewinds as soon as you finish positioning it!

Some versions of mt use a -f instead of a -t. The device argument is the no-rewind tape device that you are using, such as /dev/rmt/0n. Now specify one of the following for argument:

status: This gives you the ioctl status of the tape device. It does not require an accompanying argument.
rewind: This rewinds the tape to the beginning. This option is spelled rew on some versions of Unix. It does not require an accompanying argument.
offline: This ejects the tape from the tape drive. This option is spelled offl on some versions of Unix. It does not require an accompanying argument.
fsf x: This is short for "forward space file." It positions the tape forward x file marks, where x is a number greater than 0. (If you do not specify a value for x, it defaults to 1.) If you are at the beginning of the tape, you are at file 1, so if you want to be at file 3, you need to go forward two files. This requires an fsf 2, as in mt -t device fsf 2.
fsr x: This is short for "forward space record," and is not needed when manipulating dump tapes. (If you do not specify a value for x, it defaults to 1.)

The following are examples of how to use the mt command. To rewind the tape /dev/rmt/0cbn, issue the command:

# mt -t /dev/rmt/0cbn rewind

To fast-forward the tape /dev/rmt/0cbn to the second file on the tape, issue the command:

# mt -t /dev/rmt/0cbn fsf 1

To eject the tape /dev/rmt/0cbn, issue the command:

# mt -t /dev/rmt/0cbn offline

To get the status of the tape /dev/rmt/0cbn, issue the command:

# mt -t /dev/rmt/0cbn status

Once you have positioned the tape to the proper file, simply use the same restore command as before, leaving off the s option and its argument:

$ restore tbfy 32 /dev/rmt/0cbn

Whichever method you use, the table of contents is sent to standard output, which you should redirect into a file. One important thing to note about this output is that the name of the filesystem dumped to this volume is not in the output. This table of contents is relative to that filesystem, whatever its name was. For example, if you backed up /var, and you were looking for /var/adm/messages, the output would look something like this:

345353 ./adm/messages

I recommend that you create a table of contents for each dump volume when you make it and store this output in a file that matches the name of the volume. Obviously, you should use a unique name, like:

./dump.system.filesystem.level0.May19.2006

Saving tables of contents in this way is very handy when you're searching for a file and you can't seem to find it on any volume. A quick grep of all the dump files shows you which volume you need.

A Day Late and a Dollar Short

I was once told that data needed to be recovered from a machine that had been decommissioned 10 years earlier. I was told the name of the machine and about where the tapes were stored, so I started digging.

When I found the tapes, once I scrounged a tape drive with low enough density to be able to read them, I discovered that they were in a dump format that was no longer supported! I found the source code for the original restore program (the BSD 4.1 one in this case), downloaded it to my machine (SunOS 4.0.1 in this case, a BSD 4.3-like system), and started working on porting the old program. No good. I soon realized it would take me weeks to do it; the filesystem and dump formats had changed that much.

There had to be a different way, so I searched the data vaults for more tapes. Luckily, I found another stack of tapes, marked as being in tar format. I had lucked out! Most of these tapes were still readable, and the data came off the first try.

Moral of the story: when you decommission a machine, make an archival copy of the data in every format you can, on every type of media you can. Some, like dump, are very efficient but might not be supported someday, while others, like tar and cpio, have stayed around year in and year out. Times change, media changes, formats change, so make as many variations as you can so your data will be retrievable for as long as possible.

This made me a big fan of using tar for archival purposes, but that makes excellent sense. Its name stands for Tape ARchiver, after all.

Doug Freyburger