IBM Rational ClearCase, Ant, and CruiseControl: The Java Developers Guide to Accelerating and Automating the Build Process

Chapter 1 introduced the concepts of Software Build and Release Management and broke its definition into a number of key concepts. I also discussed a number of key challenges involved in building and releasing software. This section revisits those concepts and challenges and relates them to what has been discussed throughout the book.

The Last Word on Software Build Management

A large part of this book has concentrated on the build definition phase or the phases directly related to it. Defining a set of optimal build scripts is fundamental to successfully carrying out both Integration and Release Builds. The use of Apache Ant to achieve this has been pervasive throughout this book. Although there is much more to Ant than has been described here, by now you should have a taste for the level of control and automation that is possible. Another way of looking at the targets and tasks defined in your Ant build scripts is that they constitute your build knowledge base, as illustrated in Figure 12.1. This knowledge base details where all the inputs to your build process are located, what is required to transform (or build) them, and what the outputs of this process should be.

With Ant, I believe this knowledge base is more accessible, controlled, and predictable. Compare this to a build process defined entirely within an IDE. In most cases the build process is easier to change, but there is not necessarily the same level of understanding of the impact of any change, because the IDE masks a lot of the complexity. For example, a check in a checkbox could quite easily remove an essential build component or change a CLASSPATH setting that could cause short-term but high-impact problems. Although IDE-based builds are often a fundamental part of a productive Private Build process, the decoupling of the Integration and Release Build process from the IDE with Apache Ant is something I think should be actively implemented.

Over time, you will undoubtedly find that your Ant build scripts become significant and large development assets in their own right. It is therefore important that they are well-maintained and documented just like any other "code" asset. What is perhaps more important, however, is that the interface to these scripts is kept clean and useable by all members of the development team. Obviously, one of the ways to achieve this is to use the build execution phase to automate their invocation. This book has described how CruiseControl can be used to achieve this. When Ant and CruiseControl are used in combination, from a development point of view Integration Builds just "seem to happen," and they are a natural part of the development process. Once they are set up correctly, developers or integrators do not have to remember the exact invocation, relationship, or sequence of build scripts that need to be executed.

Most people create reports on what is happening during the build and release process; however, often they are unsure what types of reports to produce and what they can be used for. When they do know, the creation of reports is often a very manual, laborious, and error-prone process. In the build reporting phase, I described how to automate the execution of a number of report types using a combination of tools such as ClearCase, ClearQuest, and CruiseControl. Each of these tools reports on a different aspect of your build process: ClearCase on the versions of your code and its baselines, ClearQuest on the changes and releases happening in your project, and CruiseControl on the results of the build.

The exact format of your reports and the information that is contained in them is ultimately an organizational or project preference. However, I believe that if you use the tools and types of reports described in Chapters 8 and 9, there really is no excuse for not automating a large part of your reporting activities.

The Last Word on Software Release Management

One of the main themes of this book has been highlighting the differences between Integration and Release Builds. These differences can be summarized as follows:

• Integration Builds are created for internal use, and their outputs are typically transient. Most of the time the only important Integration Build is the last one.

• Release Builds are created for external use (usually customers). Subsequently, their content needs to be identified and controlled, and their outputs typically are persistedfor future reference, maintenance, and fixing.

Essentially this means that your final Release Builds should be carried out in a more controlled and isolated environment, where there is usually a need for some level of approval before the Release Build is deployed or sent to manufacturing. To support this, Chapter 10 described how to create a controlled and isolated environment using ClearCase streams and branches. Chapter 11 described how the build and deployment tracking capability can be used to automate much of the Software Release Management process. This level of control and approval is often essential when you are moving a release over departmental boundaries, as in the example I gave of the IT Development and IT Operations teams. How much time you spend on Software Release Management depends on your environment, but Figure 12.2 shows a typical relationship to the amount of time spent conducting Integration Builds.

Figure 12.2 indicates that most of the time you will be executing Integration Builds. You will also execute Release Builds, but proportionately less. You also will execute release engineering activities, such as packaging and deploying the Release Build to different environments.

For any release, the release packaging and release deployment phases often go hand in hand. After all, you can't deploy something before it has been packaged. A number of techniques can be used for these phases that depend on the development technologies being used and the deployment endpoint. Although the exact form that your packages will take and the technology that is used to deploy them can vary, Chapter 11 tried to be consistent by integrating the chosen approach into the Ant build scripts. This should ensure that you have a single point of entry for any build or release operation.

Finally, I also discussed the importance of defining your deployment unitthe complete set of artifacts that are part of or directly related to your release. It is easy to forget about this supplementary information when you are busy deciding how to automate build or release tasks. Nevertheless, constructing release notes, installation documents, bill-of-materials documents, and so on is an important part of the overall release process. Although you can automate these processes to some extent (for example, defining and managing the deployment record using ClearQuest), some of the supporting documentation must be created manually.

Overcoming the Challenges

Chapter 1 identified a number of common challenges in building and releasing software. Figure 12.3 illustrates these challenges and aligns them with some fundamental concepts or best practices you can implement to help you overcome them.

The remainder of this section discusses each of these best practices, looking at how and why adopting them will help you face the challenges.

Automation

Throughout the book, I have described how to automate all phases of the build and release process. One of the reasons for this is that with automation, traceability can truly start to be realized. For example, Chapters 3, 7, and 10 discussed in detail some best practices of how and when to apply ClearCase baselines or labels. Chapter 9 described how to produce automated reports on these baselines using the ClearCase API. If automated correctly, this should give you enough information on what is going into each build at the file level. If used in combination with ClearQuest, this will also give you information on these baselines at the functional level.

If you automate and manage a deployment process using the build and deployment tracking capability (as described in Chapter 11), you should also be able to produce information that tells you where these baselines have been deployed. This is a level of capability that organizations that are faced with compliance or audit issues often need. A key point here is that I believe traceability can be sustained only through automation, and for automation you need tools. You need a tool for version control and baselining (at a minimum) and a tool for managing and reporting on change requests and deployment. This obviously implies a correct and fundamental use of both ClearCase and ClearQuest.

Responsibility

To ensure completeness you need to enforce responsibility and treat your build scripts as firstclass citizens. What I mean by this is ensuring that you version-control all your build scripts and that you manage and assess the impact of changes to them just as you would any other development asset (such as your code). These scripts should also be kept well documented and modularized, with each "component" of your system having a standard set of scripts (preferably making use of a common library of routines). This can be achieved by using some of the Ant best practices described in Chapter 5.

Responsibility and treating your build scripts as first-class citizens also help with repeatability, enabling you to be sure that your build or release process will be executed the same way every time. It also comes through automating the invocation of the scripts whenever they are frequently and repetitively carried out, such as during active development. This is where CruiseControl and its capability to execute Integration Builds really comes into play. For repeatability it is also important that baselines be created regularly and at the correct point, because through baselines you can set up an environment to build previous or concurrent releases. To support this, the promotion of "good" baselines (using UCM promotion levels or Base ClearCase attributes) is a best practice that I have discussed throughout the book.

For ensuring repeatability of release deployment, you should also ensure that responsibility flows through to the creation of your release packages so that they are constructed and tested as part of your everyday build process and not at the last minute. You also need to ensure that your deployment process is well documented and frequently tested. One way of ensuring this is by documenting these processes as targets in your Ant build scripts, as discussed in Chapter 11. You can also take advantage of the Build and Deployment Tracking capability, because it lets you document and enforce much of your deployment process in ClearQuest itself. Finally, where possible you should also attempt to bridge the gap and try to get the IT Operations Team using the same deployment process and mechanism as the IT Development Team. Again, the Build and Deployment Tracking capability can help significantly with this.

Optimization

It is very rare for someone to get paid specifically for writing the fastest set of build scripts possible. However, it is true that to ensure the greatest possible speed of development, your build should be able to be executed as quickly and efficiently as possible. You should refactor and optimize your build scripts to achieve this whenever possible. This is critical when the build process has the potential to be executed many times a day. To complement this, the build process should also be agile. In other words, the build could be executed many times a day if necessary and potentially could be executed in different ways. This might mean identifying when and how you should do a full or partial build and subsequently how build outputs could be reused to support this.

The correct use of CruiseControl can be a significant advantage in making your build process faster and more agile. Essentially, with CruiseControl you try to automate the project integration rolemaybe even to the degree of putting yourself out of a job! The correct use of reporting is also important. You should optimize the communication flow so that if something goes wrong with a build you can pinpoint where, why, and who was responsible as quickly as possible. Again, through its use of e-mail and Web publishing capabilities, which I discussed in Chapters 6 and 7, CruiseControl can help you achieve this.

Pragmatism

One final point is that every build process needs a degree of pragmatism. You should adopt and implement a process that is suitable for the environment in which you are working, whether it be an iterative, agile, or traditional development model. For example, even if your build process can be executed every 20 minutes, if the complexity of your code means that your developers can deliver only a few times a week, maybe the use of an automated nightly build is more applicable. You can find the most appropriate approach by identifying your "project rhythm," as discussed in the next section.