Expert SQL Server 2005 Integration Services (Programmer to Programmer)

Overview

As a fitting conclusion to this book, considering how to scale SSIS will help tie several concepts together. The topic is not new, as ways to achieve scalability have been presented throughout the book (such as how to efficiently work with transactions and snapshots, and how to scale your dimension processing). This chapter hones the discussion of scalability to several key areas, including data flow tuning, destination adapter optimization, SSIS execution architectures, and effective use of SQL.

This chapter is grouped into three sections:

• The first section, “SSIS Scalability Foundations,” looks at the low-hanging fruit. In other words, you may be reading this chapter because your packages are not performing to the level you would like. In this section, you’ll find out what you should consider when trying to identify and improve performance. Additionally, one of the main areas of concern when architecting a package is when to use database scripts versus leveraging the data flow. Therefore, this section will provide guidelines to identifying when to use each of these valuable tools.

• The second section, “Data Flow Optimization,” deals with scalability of the data flow. When it comes to tuning SSIS performance, a majority of the resource impact (directly from SSIS) will come from the data flow itself, since the control flow simply is coordinating the execution of other services and programs. Among other topics, this optimization review of the data flow will examine destination optimizations.

• The final section, “Package Execution Principles,” discusses package architecture and execution location. Identifying where packages should run impacts scalability. Included in this section is a discussion of distributed package execution.