FreeBSD 6 Unleashed

Premake world Tasks

So, you've chosen to rebuild your system. Whether you're going to synchronize to interim sources along one of the development branches or you want to rebuild to upgrade to the next major release, there are a number of things you need to do before embarking on the rather convoluted rebuilding process. A lot of things can go wrong, so it pays to follow the game plan:

1. Make a backup of your system. Nothing is more heartbreaking than performing a simple, routine upgrade, only to find that your filesystem is destroyed and all your users' data irretrievably lost. Backup solutions might include tape drives, optical disks (CD-R, DVD-RAM, DVD-R, and so on), a second hard drive that you mount only when you need to mirror your first drive's contents onto it, or a second machine that you keep synchronized via NFS or CVSup (you learn about backup methods in Chapter 21, "FreeBSD Survival Guide").

   Caution

   Having a proper backup solution is one of the most crucial, and yet most onerous, parts of system administration. Many readers will undoubtedly ignore this warning and proceed anywayit happens all the timebut if you can possibly manage the cost and hassle of setting up a backup solution, the benefits are absolutely worth the investment.

2. Check the timing of your upgrade. Pay attention to the mailing list for your chosen branch, if you're tracking an active development codebase. Make sure your upgrade won't occur during an announced bug fix or testing phase. If the time isn't right for your upgrade and you desperately need just a certain patch or bug fix, see if there's a way to rebuild just that one component.

3. Synchronize your sources. Before you can really do anything further to prepare yourself for the build, you need to synchronize your sources, as explained in the section that follows. You only need to do the setup for this once; after that, it's a process that can (and should) be automated and run regularly.

Synchronizing Your Source Tree with the STABLE or CURRENT Tree, an Errata Fix Branch, or a Release

You have a number of ways available to you to synchronize your sources. The methods suggested by the FreeBSD developers include CTM, Anonymous CVS, and CVSupand each one has its own particular advantages. You can read more about each of these at the FreeBSD website under the "Synchronizing Your Source" heading of the online handbook. This chapter covers using CVSup because it's the most advanced and efficient (not to mention coolest) option available.

Configuring and Running CVSup

We discussed how to set up CVSup in "Keeping Ports Up to Date with CVSup" in Chapter 16. If you haven't set up a supfile (CVSup configuration file) yet, as described in that chapter, refer to it for background information on installing CVSup and writing an appropriate supfile. If you did set up a supfile (for example, the /etc/ports-supfile shown in Listing 16.7), you can modify it to synchronize the entire source tree instead of just the ports by just adding a couple of lines as shown in Listing 19.1, and give it a more general name (for instance, /etc/6stable-supfile).

Listing 19.1. A General CVSup Config File for Synchronizing a STABLE Branch

The file is interpreted sequentially, and later *default lines override previous ones, applying only to the commands that appear below them. Notice that the first set of *default lines, which define the settings for where CVSup should put the files it downloads and what source repository to use, set the branch tag to RELENG_6. As you've seen, this is the proper tag for the 6-STABLE branch. However, these settings only apply to the src-all source collection, the FreeBSD system sources. After that line, another *default line resets the branch tag to a period (.), which is a synonym for the HEAD of the source tree. This is the appropriate tag for the ports collection (ports-all) and the documentation tree (docall), which each only have a single development branchthey aren't broken out into branches like the system sources, and the most current state of each tree is always considered the best version at any given time.

After you've set up your supfile, run a CVSup update with the following command:

# cvsup -L 2 /etc/6stable-supfile Parsing supfile "/etc/6stable-supfile" Connecting to cvsup12.FreeBSD.org Connected to cvsup12.FreeBSD.org Server software version: SNAP_16_1h Negotiating file attribute support Exchanging collection information Establishing multiplexed-mode data connection Running Updating collection src-all/cvs Edit src/UPDATING Add delta 1.416.2.3.2.6 2005.12.19.10.58.56 delphij Edit src/sys/conf/newvers.sh Add delta 1.69.2.8.2.2 2005.12.19.10.58.57 delphij Edit src/sys/nfsclient/nfs_vnops.c Add delta 1.258.4.1 2005.12.19.10.58.57 delphij

Watch the output carefully. You should see a series of filenames whose names are relative to the base (/usr), each one with numbered and signed revisions being applied to itthat's CVSup synchronizing your source tree with the one on the server, downloading and applying only the portions that differ.

Tip

This CVSup command is an excellent candidate for a shell alias (as discussed in Chapter 9, "Customizing the Shell"):

alias sync6 cvsup -L 2 /etc/6stable-supfile

Each line should begin with either Edit or Checkout, or the occasional Delete, indicating a file that's been removed because of redundancy or moved to another location. However, if you see nothing but Delete lines, you should press Ctrl+C to stop the processit means you're synchronizing to a source tree that doesn't exist. Double-check your supfile to make sure your branch tag is correct and points to a valid source branch as listed at http://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/cvs-tags.html.

When you're satisfied that CVSup is synchronizing your sources properly, you probably should add a task for running the CVSup update to your daily periodic file, as described in "Keeping Ports Up to Date with CVSup," in Chapter 16. Every day afterward, you'll have sources that are always no more than a day old.

Tip

At the end of each CVSup process run from the periodic subsystem, a status message that contains all of the process's console output is generated and mailed to root. Pay attention to the contents of this message every time! When you're tracking STABLE or CURRENT (or an errata fix branch), these messages tell you each day what has changed. This way, you'll know exactly when a crucial bug fix has been applied to a certain component you're watching. The output of the CVSup process can be lengthy and unwieldy, but after a few days you'll get a feel for what parts of it are important and what can be ignored. Particularly in the case of an errata fix branch, you'll only see changed files when a new interim release point has been created; those are the times when you'll need to check the UPDATING file (discussed shortly) to see exactly what has changed and whether you need to act on it.

Using CVSup to Synchronize to an Errata Fix Branch or Release

Using CVSup is an excellent way to get the sources for a major release version or interim errata fix release as well as to synchronize to actively developed sources. To do this, you simply need to use the branch tag that specifies the release you want (as discussed earlier in the chapter).

It's tempting, especially if you're performing an upgrade from one major version series to another, to sync to a major release version (such as 6.0-RELEASE) and build a system from that; however, you should be aware that the original RELEASE codebase for a particular release branch is usually insecureit's missing whatever critical fixes have been merged into it since the time the release occurred. Generally, RELEASE codebases are relevant only for a few weeks after the initial release on that branch appears, or until the code begins to change in response to security bulletins.

The best policy for most FreeBSD administrators is to use an errata fix branch, which gives you the key updates you need for responding to serious security issues, but doesn't make you perform large and invasive upgrades or subject you to unstable or irrelevant code changes. To specify a particular errata fix branch, edit the supfile (refer back to Listing 19.1) and change the branch tag that's relevant to the src-all collection. For example, the errata fix branch sources for FreeBSD 6.1 can be had with the following line:

*default tag=RELENG_6_1

If you really want to synchronize to a RELEASE codebase (such as the initial release of FreeBSD 6.1), knowing that it's less secure than the current state of its corresponding errata fix branch, use a line like the following:

*default tag=RELENG_6_1_0_RELEASE

Now, when you run the CVSup update from this file, you receive the "frozen" sources for the release you've chosen. Also, unlike with the constantly changing STABLE and CURRENT branches, if you update again a week later, nothing will have changed.

You can now use make world from these sources to build a clean system from the initial release sources; but you should then change the release tag in your supfile to indicate an appropriate errata fix branch and sync your sources again to obtain the latest security patches for your chosen release. Then, you can resume daily synchronizations so you can have all the latest source patches and updates at your disposal. This upgrade method works best if you have a critical server to keep updated.

Troubleshooting Questions

Quite a number of things can go wrong with CVSup, and because it's a type of task that has no analogue on most other operating systems, it's hard to tell when the process has gone as planned and when it has abjectly failed. Here are suggested fixes for some of the most commonly reported problems:

• CVSup won't connect to the server I've selected (Connection refused).

  Try picking a different CVSup server. The servers are named cvsup1 tHRough cvsup18; the higher the number (for example, cvsup15.freebsd.org), the less traffic it is likely to be sustaining. Also, try to run CVSup at a time of day when traffic will be minimal (the middle of the night, for example).

• CVSup connects but nothing ever happens.

  This might occur if you're behind a firewall or a NAT router that masks your IP address. CVSup is known to work with NAT-enabled network configurations, but a misconfigured one might cause problems. Also, make sure your firewall will allow CVSup traffic. The server port is 5999. Try using the telnet command to connect to the server at that port to make sure you can connect (telnet cvsup12.freebsd.org 5999); if you get no response or a "Connection refused" message, you might need to make a firewall exception for port 5999.

• CVSup deleted my entire /usr/src directory!

  This occurs if you've specified an invalid branch tag. If you give CVSup a tag that the server doesn't recognize, it responds by giving you the contents of the CVS tree at the branch you've specifiedwhich is nothing. Make sure you've got the right branch tag, and you should be all right; just run CVSup again and a fresh set of sources will be downloaded and merged with whatever's left from before the previous run.

The UPDATING Text File

Now that your sources are updated, you have a current UPDATING file in /usr/src. This file is a "late-breaking news" bulletin that has important information on upgrading your system, almost guaranteed to be more current than anything you read in print or online (this book is no exception). UPDATING contains a reverse chronological listing (each with a date stamp) of notable changes to the build process that you should know about. Read all of them between the top of the file and the date of your last make world.

In an errata fix branch, the entries in UPDATING specifically indicate the nature of each security patch or critical bug fix that was checked in, and each is labeled with the appropriate p number:

20051219: p1 FreeBSD-EN-05:04.nfs Correct a locking issue in nfs_lookup() where a call to vrele() might be made while holding the vnode mutex, which resulted in kernel panics under certain load patterns. 20051101: FreeBSD 6.0-RELEASE

UPDATING gives you special instructions for coping with major changes in the system (moved files, added or deleted features, and so on). Something to remember is that the files in /etc are not altered automatically by the make world process. Any configuration files you have in there will not be overwritten. This ensures that your custom system setup won't be lost; it also means that whenever the default system configuration requires something new to be added to or deleted from /etc, you'll have to merge it in manually. The UPDATING file usually details cases like this. Fortunately, a neat tool called mergemaster allows for an easy merge of new files into /etc. You learn more about mergemaster in "Using mergemaster to Check for Changed Configuration Files," later in this chapter.

Merging /etc/group and /etc/passwd

Most of the /etc files don't affect your ability to build the system. However, occasionally new services are added that have to be installed with ownership matching a certain user and group. Remember that make world is not an installer application with built-in upgrade tasks; it simply recompiles and installs a new version of the operating system on top of the old. This means that make world most likely won't create the users and groups it needs if they don't already exist. You can help avoid any collisions arising from this by merging any new entries into your /etc/group file.

The new version of /etc/group is in /usr/src/etc/group. It probably won't be any longer than 20 lines or so. Open /etc/group in another terminal window. Now, check to see if there are any entries in /usr/src/etc/group that are not in /etc/group. If you find any, simply copy them over. New entries are likely to have GIDs below 100.

One easy way to compare the two files is to use diff, as shown in Listing 19.2.

Listing 19.2. Comparing /etc/group and /usr/src/etc/group Using diff

Groups in your existing /etc/group are shown with a minus sign (-), and new groups in /usr/src/etc/group have a plus sign (+). In this example, you need to copy the network group into your /etc/group file.

It's important to preserve the GIDs suggested in the new group file. In the unlikely event that any of the GIDs don't match, you need to fix your existing /etc/group to match the new file. This fix might cause some existing files in your system (that were owned by that group) to lose their permissions. You can search the system for these files with the find command:

# find / -group GID -print

You can then fix the permissions on these files with chmod, as you saw in Chapter 13, "Users, Groups, and Permissions."

It's even less likely that any mismatches will occur in the user database. The sources don't include a passwd file, just a /usr/src/etc/master.passwd file that's even shorter than the new group file. Quickly scan it and your /etc/master.passwd file for mismatches and then use adduser to insert any new users you find.

Merging /etc/make.conf

The make.conf files are the global configuration files that control all make operations, including make world. You don't have an /etc/make.conf file in a new installation of FreeBSD. However, in the same way that /etc/rc.conf and /etc/defaults/rc.conf work, there's a file named /etc/defaults/make.conf that specifies all the likely default settings. You can leave things as they are, but you can speed things up a bit with the judicious use of a few options.

The FreeBSD Handbook suggests enabling the following lines by copying them out of /usr/share/examples/etc/make.conf and into /etc/make.conf and then uncommenting them:

CFLAGS= -O -pipe NO_PROFILE= true # Avoid compiling profiled libraries

These modifications help to ensure that the make world process will be smooth, without trying to add potentially destabilizing optimizations or unnecessary compilations into the process. You can set some other options if you want to experiment, but that's beyond the scope of both this chapter and this book.