Monitoring and Managing Microsoft Exchange 2000 Server (HP Technologies)

2.1 Transferring responsibility to the operations team

I have worked with many corporate IT organizations in a variety of industries. No two IT organizations have been structured in the same manner. However, almost all IT organizations, regardless of industry, corporate size , or geographic dispersion, share one common feature. All tend to isolate their operations team from the groups that design and implement new technologies. Unfortunately, only the more enlightened Exchange deployment and migration project teams include representation from the people who will be tasked with managing the Exchange environment once it becomes operational.

It is not unusual for Exchange deployment and migration project teams to simply not consider how the Exchange environment will be managed until the migration project nears completion. The migration project team is doing daily battle with issues that demand immediate attention. Management issues can too easily be postponed while the migration team is busy with more immediate issues. As a result, many companies design and implement complete Windows 2000 domain model and Exchange organizational infrastructures with very little concern for operational staff or procedures. The operations team and the help desk can provide valuable insight since they routinely deal with the reality of how users interact with software products. These key people often have better insight for how a particular implementation or design will affect the users.

Concern, panic, and lengthy meetings often result after the deployment and migration team transfers management responsibility to the operations team. Too often, this transition takes the form of a migration project team member passing a stack of design and migration project documents over the cube wall as the project team heads out the door to their project completion celebration . At least that is how it seems to the operations team.

There are several ways to improve the project-teamtooperations-team transition. First and most important, the operations team should be represented on the Exchange deployment and migration team. Their representation on the team should be on an equal basis with other project team members . Their participation is no less important than that of the technologists designing the Exchange infrastructure, the network designers, and the training coordinators. It is also important to remember that the operations team consists of a variety of different functions, including individuals who will manage and monitor the Exchange environment, the help desk personnel, and the operators responsible for backups . Every decision made during the Exchange deployment project needs to factor in the future operational considerations.

The operations group should begin managing the Exchange environment early in the deployment project while the environment is still relatively small. Operational errors are more easily forgiven when only 100 pilot users are impacted than when 100,000 users find themselves without e-mail.

Finally, the deployment and migration teamincluding the operations team representativesshould fully document all staffing and management requirements for the Exchange environment. This should not be a random selection of migration project documents. Instead, it needs to be a well organized set of documents that will be used far beyond the end of the deployment project. This should be a living set of documents that evolve as the environment and available tools change. Note that documents does not necessarily restrict you to something printed on paper. Web pages are an acceptableperhaps preferredformat for this documentation. Regardless of the chosen media, the documentation should minimally address the following areas:

• Exchange deployment mission statement . What were the original goals for deploying Exchange in the organization? How have these evolved over time?

• Architectural design . What is the Exchange organizational topology? What implementation decisions were made regarding the Exchange architecture? This should include the naming conventions that were selected.

• Network infrastructure . What requirement does Exchange have regarding the underlying network infrastructure? What is the topology of the underlying network? I am constantly amazedand disappointedby the number of otherwise well-managed organizations that do not have a current network map and do not know the available bandwidth between their various locations.

• Monitoring baseline . A monitor baseline should be taken once the Exchange environment is working correctly and consistently. This can be used for comparison in the future if the environment begins to exhibit unwanted characteristics.

• Server build documentation . The Exchange servers deployed during the initial migration project are unlikely to be the last. Additional servers will be required as the organization grows. The team that deployed the original set of Exchange servers may no longer be available when it is time to install new servers. How should these additional servers be configured?

• Server connection documentation . If new connector servers need to be added, how should they be configured? How should servers in different routing groups be connected? What is the corporate-approved method for connecting the Exchange e-mail environment to other corporate e-mail environments such as Microsoft Mail and Lotus Notes? How should the Exchange environment be connected to external environments such as the Internet?

• Client build documentation . New versions of Outlook will be developed. How was Outlook tailored for the corporate rollout? What were the critical factors considered when building the Outlook client kit?

• Application installation instructions . New desktop devices will be added. What are the steps for installing Outlook? How should the desktop operating system and network services be configured to support Outlook?

• Operational procedures . What procedures need to be followed to keep the Exchange environment healthy and happy? This should include a timetable showing which procedures should be performed daily, weekly, monthly, etc.

• Backup and recovery procedures . What needs to be done to prevent loss of data when disaster strikes? What are the correct procedures for backing up an Exchange server? How can these backups be used in various situations, such as recovering from loss of a server, recovering from loss of a disk drive, recovering individual mailboxes, and so on?

• Administrative procedures . How should common administrative functions be performed, such as adding and removing users, creating system distribution lists, and so on? Who should perform these administrative functions?

• Service level agreements . What are the expectations of the departments who rely on the Exchange environment? How muchif any scheduled downtime is acceptable? What documented agreements such as service level agreements (SLAs) have been made with these groups of users?

• Escalation procedures . How does a user report a problem? If the problem cannot be corrected in a timely manner, howand to whomshould the problem be escalated? How are reported problems logged and tracked? What type of problem/resolution knowledge database will be maintained to help solve problems?

• Frequently asked questions . What were the most frequently asked questions during the Exchange rollout? What were the most common problems reported to the help desk during the migration?

• Management model . Will a centralized operations team manage the entire corporate Exchange environment or will some tasks be delegated to regional personnel? Will regional administrators be allowed or required to add and remove users, or will these functions be centralized? Will regional operators perform Exchange backups? Will they also be responsible for restoring data from the backup tapes?

• Roles and responsibilities . Who is involved in the ongoing management of the Exchange environment? What skills and expertise are required to perform each of these roles?

• Training recommendations . What training is recommended for each of the essential operational roles?

These documents should act as the basis for the ongoing implementation of the Exchange environment. Changes over time to the standard configurations or any part of the Exchange environment need to go through a standard change control process, which ensures that the documented procedures always reflect the actual operational environment.

Including the operations team in the Exchange deployment and migration project and providing this level of documentation should result in a much smoother transition from rollout to full operation and will help reduce future user satisfaction issues as the operations staff comes up to speed.

2.1.1 What needs to be managed?

What is involved in managing Exchange, and how large an operations staff is needed? In small companies, a small number of people may have responsibility for multiple areas, while in large corporations, many people may have roles with narrow sets of responsibilities. Often, these people are from separate organizations.

Regardless of the number of people used to manage Exchange or the size of the organization, specific types of management activities must be performed. Examining a typical Exchange implementation helps identify the areas where management must be performed.

• Exchange organization . An Exchange organization is a hierarchical collection of Exchange routing groups, administrative groups, and servers. The complete collection of Exchange components is known as the organization . The Exchange organization consists of one or more routing groups and one or more administrative groups . An Exchange routing group contains one or more Exchange servers . The servers within an Exchange routing group are connected by a high bandwidth permanent network connection. (You can use slower network connections between routing groups in your Exchange organization, but not within a routing group.) The number of routing groups, administrative groups, and servers required in an enterprise depends upon several factors, including the number of users, number of corporate locations, network bandwidth between locations, availability of local IT support staff, and corporate politics. Two enterprises with exactly the same number of users may have an entirely different Exchange organization topology due to differences in network bandwidth, number of locations, or IT group organization.

  Someone must monitor and manage the Exchange Organization. This includes monitoring message transfer between Exchange routing groups, between Exchange and other internal e-mail environments, and between Exchange and external e-mail environments such as the Internet. As Exchange servers are added to or removed from the organization someone must ensure the integrity of the overall e-mail environment.

• Exchange servers, services, and queues . The individual Exchange servers are the most obvious component in the Exchange environment and the Exchange server software must be functioning properly for the e-mail environment to be in good working order. The Exchange 2000 server is not a single, monolithic program. It consists of a cooperating set of Windows 2000 services and queues. A message sent using a MAPI client is first delivered to the Exchange information store. The information store searches the directory to determine where the message should be delivered. The message is then passed to the message transfer agent (MTA), which delivers the message. Anomalies with these services or queues are an immediate indication of a problem with the e-mail environment. These services and queues must be closely monitored .

• Information store . The Exchange 2000 information store consists of a public information store and a private information store . The private information store contains all messages in users server-based e-mail folders. The public information store contains all objects in the public folders. The key management responsibility regarding the information store is to ensure that the data will not be lost if a hardware error causes the loss of a disk drive. This is typically done via regularly scheduled backups and planned recovery exercises.

• Client software and users . Users are the ones who will ultimately decide whether the e-mail environment is functioning properly. Users do not see the Exchange software. Instead, the users view of the e-mail environment is through client software such as Outlook. When a client application sends a message, the Exchange server is responsible for routing the message to its intended recipients. The client application also allows users to access the messages in their mailbox. An end-useroriented help desk is a key operational component of the Exchange e-mail system.

2.1.2 Exchange dependencies

Exchange does not exist in isolation. There are other applications in the networked environment, and other processes coexist on Exchange servers even on dedicated Exchange servers. We must understand the interrelationships between these processes in order to manage a reliable Exchange infrastructure and achieve the service levels that departments and business units demand.

We need a clear understanding of the services and components of the Exchange-based e-mail environment. In particular, we need to understand the services and components on which Exchange relies. Proper functioning of an enterprise-wide Exchange environment relies upon proper functioning of many components, including the following:

• Windows 2000 server software . The Exchange server runs on Windows 2000. Windows 2000 failures will have an immediate impact on the Exchange server software. One of the key Windows 2000 components is the Active Directory. In previous versions, Exchange had its own directory. However, Exchange 2000 uses the Active Directory services that come with Windows 2000. The Windows 2000 Active Directory contains information about Windows 2000 objects, including all Exchange 2000 objects. This includes complete information about Exchange routing groups, administrative groups, servers, connectors, recipients, public folders, users, mailboxes, and distribution lists. Windows 2000 replicates this directory information to other Windows 2000 servers throughout the organization.

  It is important to remember that Exchange 2000 cannot be administered independently of Windows 2000. Because both Windows 2000 and Exchange use the Active Directory, the way you choose to organize your Windows 2000 topology may dictate how you administer Exchange.

• Server hardware . Exchange server software cannot run if the underlying hardware fails. You can use RAID arrays, clusters, and other fault-tolerant mechanisms to improve the availability of Exchange servers.

• DNS . This network service enables processes to locate other systems and processes in the network.

• Windows 2000 domain environment . Exchange depends on Windows 2000 to validate user credentials, and depends on the Windows 2000 Active Directory to provide directory services.

• TCP/IP and the physical network . E-mail systems are networked applications and cannot survive if the underlying network protocols or physical connections fail.

• Client/server connections . The network connection between each users desktop and the Exchange server must be working properly in order to have a properly functioning e-mail environment.

Correct functioning of an Exchange-based e-mail environment requires all of these components to be in good working order. Upon thorough investigation, many Exchange problems are found to really be problems with the components on which the e-mail system relies.

If users expect you to provide a reliable Exchange-based messaging infrastructure with a high level of service, you must first ensure that the other components provide similar or higher levels of service. There is a direct negative impact on the level of service provided by Exchange if any of these components fail to deliver the necessary service levels. Unfortunately, users typically do not understand these dependencies. Users only see the application they are trying to run. If the physical network fails when a user happens to be using Outlook, that user will consider it an e-mail failure. For example, one Exchange performance problem was investigated for many weeks before the cause of the problem was finally identified as a patch cable connecting the Exchange server to a switch. The cable ran too close to the data centers air conditioning unit and uninterruptible power supply, causing interference that forced network retries. Moving the cable fixed the Exchange problem, but not before Exchange and Outlook had received a serious black eye in the user community.

It is unlikely that the group managing the Exchange 2000 environment will also have management responsibility for all of the component areas on which Exchange depends. Instead, the Exchange management group should have an agreement with each of the departments responsible for managing these other components. These SLAs should provide a commitment to deliver an agreed-upon service level that will support the Exchange service level requirements.