Maximizing Performance and Scalability with IBM WebSphere

The SPARC platform is considered to be one of the driving forces in the expansion of the Internet in the late 1990s. You'll now take a closer look at the SPARC platform.

Platform Overview

The SPARC platform has been available for many years now. Since its inception, myriad CPU architectures have come on the market, including the more recent Sun UltraSPARC Is, IIs, and IIIs and the TurboSPARC, MicroSPARC, and SuperSPARC architectures.

For the purposes of this book, you'll focus on the UltraSPARC II and III and other members of their lines. Further, you'll focus on the SPARC processors from Sun Microsystems. Although there are a number of other vendors still producing SPARC-based processors, from a WebSphere implementation perspective, the Sun SPARC processors are the most common (95 percent of the market share).

The range of systems you can find the UltraSPARC II and III in are endless. Sun has produced a vast range of platform choices in the past 10 years, all of which have been high-quality systems.

This list includes server models such as the Ultra series (5, 10, 20, and so on) workstations to Enterprise series servers including E240R, E250, E450, E480R, E3500, E4500, E5500, E6500, and the ominous E10000. Previous models of the Enterprise series also included the E3000, E4000, E5000, and E6000 systems with a range of submodels in-between.

More recently, Sun's UltraSPARC III processor has been found in a vast range of systems, many of which I'll discuss in this chapter. They, however, include volume workgroup or departmental servers such as the V480, V880, and V1280 systems and the SunFire range of the F3800, F4800 (and a short-lived F4810 rack-based version of the F4800), a F6800, F12000 (F12K), and an F15000 (F15K).

Sun's more recent SunFire Enterprise class servers (F3800 upward) provide split system domain capabilities and a vast array of redundancy and high-availability features. I'll discuss these at length in upcoming sections.

You'll now look at Sun's SPARC platform in more detail.

Platform Architecture

As noted previously, the SPARC platform is somewhat vast, but it essentially revolves around the SPARC Compliance Definition, created by the International SPARC Organization. Sun Microsystems originally invented the SPARC architecture and codeveloped it with Fujitsu in 1986.

In 1989, Sun transferred ownership of the SPARC design to the International SPARC Organization, a nonprofit organization that provides an open specification of the SPARC architecture to any organization that wants to use it to build microprocessors. Nowadays, you can find SPARC processors in everything from digital cameras all the way to some of the world's largest Unix superclusters.

As hinted at earlier, this chapter focuses on the UltraSPARC II and III CPUs that are applicable to V and F series SunFire-based servers, such as the V880 and F4800, and E-based enterprise servers, such as the E480R.

Out of the UltraSPARC CPUs available, the most commonly found models are the following:

• UltraSPARC IIe

• UltraSPARC IIi

• UltraSPARC II

• UltraSPARC III

• UltraSPARC IIIi

Each of these CPU models offer differing capabilities, specifically in the areas of CPU clock speeds, bus interface and bus interconnect speeds, and level 1 and 2 cache sizes (and in some cases, level 3 cache).

What's important to note is that both the UltraSPARC II and UltraSPARC III processors are based on the SPARC V9 RISC architecture, as set out by the International SPARC Organization. Both models support 64-bit computing, both in slightly different forms, which I'll discuss shortly. It's important to understand the differences between these processors at this level in order to understand the differences in areas such as addressable memory, performance (not just clock speed), and scalability features.

Unlike the Complex Instruction Set Computing (CISC)-based processors from AMD and Intel, the RISC-based UltraSPARC processors provide high-processing throughputs, with lower clock rates. In many cases, the bus transfer rates of the UltraSPARC processors (even those running at one-third the clock rate of the Intel Pentium 4 processors) can sustain bus transfer speeds (I/O, memory, and so on) far in excess of the x86 counterparts.

I've found that people who are less familiar with or who haven't had as much exposure to high-end processors such as the offerings from Sun, HP (formerly Compaq, which was formerly Digital), and IBM often scoff at the costs of the RISC-based processors. Generally , the costs for comparable lower-end RISC processors are comparable to the CISC or x86 processors.

The flaw to many people's argument over x86 versus RISC is that it's all about clock speed. This couldn't be any further from the truth. The clock speed is more of a marketable term these days, and although it does have some bearing on the end result of the performance of the processor, it's no longer the absolute.

In the old days of 386s and 486s, there wasn't a great deal of complex features in those processors. The legacy of "faster clock speed means faster processing" can be attributed somewhat to those days where the difference between a 386DX33 and a 386DX-40 was a whopping 7MHz, and that 7MHz made a world of difference! However, nowadays, RISC and CISC processors are both complex technologies.

Furthermore, the power consumption of the UltraSPARC processors is far less than that of the x86 equivalents. The UltraSPARC II consumes 19 watts of power, and the Intel Pentium 4 2.8GHz processor consumes approximately 68 watts. Things start to heat up considerably when you have a few processors in a server operating at high wattage levels.

One final point to make about UltraSPARC processors is that all UltraSPARC processors are superscalar, essentially allowing several executions to be active on the CPU at any time. Although this is slightly different to outright pipelining, because of the nature of the simplified instructions on a RISC-based CPU, the outcome is that there's more chance that the CPU can accommodate a superscalar design.

The problem with superscalar design is that if two instructions are being executed at any one time, and instruction A requires the result of instruction B, there will be latency in the CPU core . For this reason, simplified instruction sets work best, and therefore RISC is the outright winner here.

The more recent x86-based CPUs from Intel and AMD both support basic superscalar design but aren't as effective, given the more complex instruction sets.

Given the subtle differences between the UltraSPARC II and the UltraSPARC III family of processors, you'll now take a closer look at four processors in more detail.

Sun UltraSPARC II

The UltraSPARC II was a major step for Sun when it was released. Ensuring Sun was a firm competitor in the RISC-Unix market, the UltraSPARC II is probably what gave Sun its success in the late 1990s as Internet usage became more wide-spread.

The UltraSPARC II is a 64-bit based CPU that, like more recent Sun SPARC CPUs, includes features such as on-CPU I/O and memory controllers. It's interesting to note, as you saw in the x86 section, that the x86-based processors are only just now starting to incorporate these on-CPU features. You'll explore the advantages of having these features onboard the CPU shortly.

The UltraSPARC II comes in four speeds ”360MHz, 400MHz, 450MHz, and 480MHz. The performance of the UltraSPARC II, despite its lower clock speeds, is quite good. It performs at a rate equivalent to other forms of processors with double its clock speed, and it's able to hold up in the system bus transfer ratings.

The processor comes with a level 2 cache, like most processors, that Sun coined as the e-cache . The size for this cache varies from processor to processor but is essentially available from 2MB “8MB. The 2MB version was the de facto standard in larger systems such as E450s and higher.

Bus transfer speeds were reasonable and for CPU to memory transfers, the processor could sustain 960MB per second when the Ultra Port Architecture (UPA)-connected bus was operating at 120MHz.

At the upper end of the processor models, the CPU to level 2 cache transfer rate could peak at 1.92GB per second when fitted with a 480MHz processor.

The UltraSPARC II also employed pipelining capabilities and had a good instruction per clock cycle ratio. The processor essentially supports four-way pipelining, and basic instructions are processed in one clock cycle. More complex instructions such as square-root calculations aren't pipelined and take between 12 and 22 clock cycles depending on single or double precision.

Note these numbers for the future CPU models.

The CPU also provided the ability, through the solid architecture baseline that it sits on, to interconnect with systems of up to 1024 processors. Of course, proprietary hardware implementations could extend this to an almost limitless number.

Sun UltraSPARC IIi

The UltraSPARC IIi was an upgraded version of the UltraSPARC II processor that became available in two models ”the 440MHz and the 650MHz versions. Like the UltraSPARC II processor, the IIi supported 4GB of memory per CPU. Additional CPUs provided additional address space to reference (or associate) additional memory.

Although the UltraSPARC-IIi processor's clock rate is higher than that of the UltraSPARC II processor, the e-cache (level 2 cache) is somewhat smaller than the UltraSPARC-II. Level 2 cache available on these CPUs was a de facto 512KB, which, although providing good throughput and performance, does force the recommendation of these CPUs to more mid-level production systems. Both 1MB and 2MB versions were available in design but not common in implementation.

One of the reasons (if not the only reason) for this reduction in cache size is that the level 2 cache in the UltraSPARC IIi was more tightly integrated onto the processor itself, reducing latency and increasing performance. Given that real estate on the processor die is limited, the trade-off is that the cache memory has to be reduced in size in order to fit.

At the time the UltraSPARC IIi became available, the UltraSPARC III processor family was being released. Sun marketed the UltraSPARC III to servers while the IIi was marketed to more commodity-based or workgroup-based systems and servers. The CPU provided the ability to interface with more commodity-based memory technologies such as PC100/133 Synchronous Dynamic Random Access Memory (SDRAM) commonly found in x86-based systems. The CPU found its way into products such as Sun's rack-based servers and high-end workstations such as V100s and V120s.

The IIi has a CPU to memory transfer rate of 800MB per second, as compared to 960MB per second of the UltraSPARC II. Overall, the performance of the UltraSPARC IIi, even at 650MHz, isn't as high as the UltraSPARC II 480MHz processor but is more cost-effective for lower-end systems.

Sun UltraSPARC IIe

The IIe is a model of Sun SPARC CPUs that are targeted to the low-end server or workstation market. They're more comparable to the x86-based CPUs that you'd find from Intel and AMD and cost a great deal less than the II, IIi, or III processors.

The IIe only supports 2GB of memory per CPU and comes with 256KB of level 2 cache. Memory to CPU transfer speeds are the same as that of the IIi, and therefore, for most low-end applications, it'll provide a solid base for either development, low end, or possibly highly horizontally scaled WebSphere implementations.

I don't recommend servers operating with these CPUs for medium-to large- sized WebSphere implementations because of their low memory capability and small level 2 cache size.

Sun UltraSPARC III

The UltraSPARC III is Sun's answer to the increasing pressure from IBM's PowerPC platform and the Intel/AMD push into the 64-bit market.

The III is, like the II family, a 64-bit SPARC v9-compliant processor supporting greater memory sizes (up to 16GB per CPU) as well as a greatly increased clock speed. Advanced features such as memory-level prefetching (providing memory to core data parallelism) and a redesigned level 2 cache that's accessed via what's known as associative referencing rather than via direct-mapped cache are also included. This associative referencing introduces a level of abstraction that increases the amount of cache that can be easily referenced by the CPU.

Another key feature that helps to increase the performance of the CPU without the concept of simply increasing clock speeds is a new on-CPU component called the Instruction Issue Unit (IIU). The IIU manages where each instruction is routed to within the CPU. That is, the IIU takes instructions and routes them to the integer execution unit, floating-point execution unit, or the load/store execution unit.

Unlike previous UltraSPARCs, the III works within a switching-based architecture. All components within the realm of the processor are interconnected to one another via a Fireplane bus.

You can break the key components of the UltraSPARC III down to four main groups:

• Instruction component such as IIU

• Pipeline processing units such as Floating Point Unit (FPU) and IPU Instruction Processing Unit (IPU)

• Level 1 cache management such as prefetch and data caches

• Auxiliary unit such as an interface to level 2 cache and memory controller

It's probably worth familiarizing yourself with this processor architecture (albeit at a high level) if you don't already understand how processor architectures work. The UltraSPARC III uses a surprisingly straightforward (and logical) high-level architecture for the major components within the CPU itself.

The pipeline services within the III are basically the same as that of the UltraSPARC II in terms of number of pipelines. The number of instructions per clock cycle is the same for the basic operations with division and square-root type instructions being slightly higher (40 to 70 cycles depending on precision).

A key feature of the UltraSPARC III is that it provides a data prefetching capability. This technology essentially allows for increased memory parallelism, which helps to mask the issue of latency when an instruction misses the cache. Without it, the processor would simply wait until the entire instruction to complete before executing any other instruction.

Another key point of the III is that it supports up to 16GB of memory per CPU, as opposed to the 4GB per CPU maximum of the UltraSPARC II. The memory is controlled by an on-CPU memory controller unit that interfaces with another key CPU component known as the System Interface Unit (SIU). This SIU is responsible for all communications between other components off the CPU (such as other CPUs, the main system memory, and the system bus communications).

As discussed earlier, pipelining is an important topic for CPUs. Pipelines essentially allow more things to be happening (or executing) on the CPU at any given time.

Things are no different for the UltraSPARC III. It has a six-stage pipeline, which in my opinion is a good size. As you saw, pipelines that are too big can create latency when a cache miss occurs, and pipelines that are too small will only be marginally better than a CPU with no pipeline. Therefore, the UltraSPARC III with its sic-stage pipeline provides a good result for performance.

In larger systems, the SIU ensures coherency between multiprocessor system caches. Because the UltraSPARC III was designed for both single and multi-CPU systems, it supports a variety of operating modes. However, each UltraSPARC III is essentially the same, with the difference being on the main and logic boards on which the CPU sits. That is, the logic (or uniboard) that the UltraSPARC III CPU sits on is different depending on the system family in which you install it.

The V880, for example, supports up to eight UltraSPARC III CPUs, and although an 8MB e-cache UltraSPARC 950MHz CPU would have the same processor as one installed in an F15K, the interface board that the CPU sits on is different. I'll go into this in some more detail ”it's an important part of Sun's system architecture.

In a dual-CPU configuration, such as in an E280R, UltraSPARC III CPUs are connected via a 128-bit, 150MHz Dual CPU Data Switch (DCDS). Given that there's a limitation on memory per CPU (16GB per CPU), the DCDS is connected to the banks of SDRAM (two banks in a dual-CPU configuration) via a 512-bit bus operating at 75MHz. The DCDS in turn interfaces with the rest of the system over what Sun terms the level 1 data switch via a Fireplane data interconnect at 256 bits.

In four-processor systems such as V480s, the system operates essentially as two dual-CPU systems. Each pair of CPUs is connected, like the dual-CPU configuration, via a DCDS. The two pairs of DCDS are then in turn interconnected via a level 1 data switch. You can probably see where this is going.

Much larger systems start to involve a level 2 data switch as the systems go beyond six processor inboards, and after this limit is reached, a level 3 layer is introduced, known as the Fireplane crossbar , which support systems with more than six inboards.

At the end of the day, however, the same physical CPU remains, but a series of integrated switches are used to extend the size of the system to an almost limitless number. Essentially the design works like a lattice framework ”common components are repeatedly interconnected to build larger and larger systems.

Finally, the Fireplane is a fairly integral part of Sun's larger systems. It's a two-part component, with one part consisting of the point-to-point data interconnection service and the second part being a hierarchical bus for control messages and component addresses.

Sun UltraSPARC IIIi

The UltraSPARC IIIi is a similar incarnation of the UltraSPARC III as the UltraSPARC IIi is to the UltraSPARC II. You'll find that the UltraSPARC IIIi is an update to the UltraSPARC IIi processor, and yet again, it's different from the standard III chip.

Designed for smaller systems of up to four CPUs, it incorporates a smaller level 2 cache (up to 1MB) and 1GHz clock speed. Overall, it's a solid processor for the lower-end systems and is definitely worth looking at if your per-server configurations aren't going to exceed four CPUs or 64GB of memory.

Comparison Chart: Sun SPARC v9 Processors

Of the five UltraSPARC CPUs I've covered, you can categorize each of them in a best-use model. Therefore, the following sections detail where each CPU best fits.

Sun UltraSPARC II

The UltraSPARC II is slowly being phased out but is still available for existing customer systems. Therefore, it's an interesting model of CPU. The Sun E10K is still available and operates with UltraSPARC II CPUs, as do smaller systems such as E450s and so forth.

Overall, the UltraSPARC II is a well-performing CPU but should be avoided for your WebSphere implementation operating on a Solaris environment.

Sun UltraSPARC IIi

The UltraSPARC IIi CPU is readily available on lower-end systems. This includes a variety of models such as V100 and V120 systems. These servers are typically rack based and operate with one to two CPUs per system.

They're best suited for lower-end production systems or if you're able to scale your application horizontally rather than vertically. Given that the V100 and V120 support 2GB and 4GB of memory, respectively, there's not a great deal of memory "head room" for larger or vertically scaled applications.

However, I've seen environments where massive distribution of load is possible through some smart Java Naming and Directory Interface (JNDI) or Remote Method Invocation (RMI), and in these cases, V100s and V120s aren't a bad option. Further, it should be noted that the V120 is supplied with SCSI-2 drives that have 10,000 revolutions per minute (rpm).

For midrange or high-range systems where memory and scalability are issues, then the systems housing UltraSPARC IIi CPUs should generally be avoided. If you're operating a small WebSphere implementation with, let's say, just JSPs and servlets, these servers with this CPU will provide you with a solid base.

Sun UltraSPARC IIe

The UltraSPARC IIe is an entry-level CPU designed for workstations. For production-based WebSphere platform demands, I recommend you avoid these systems. However, for development workstations operating WebSphere, systems based on this CPU (such as Sun Blade 100s) are good performers.

Sun UltraSPARC III

The UltraSPARC III is the workhorse of the current UltraSPARC processor family. As you saw in earlier sections, it's a fast, powerful, and scalable CPU technology.

Given this CPU comes in servers of all sizes, from a single CPU up to the Sun E15K with 106 CPUs, it can be used in any WebSphere environment ”production or development.

Sun UltraSPARC IIIi

After the UltraSPARC III comes the IIIi, which is a good-performing entry-level CPU that can be used for smaller-sized to medium-sized production environments. With the ability to support 16GB of memory per CPU and operating at 1GHz, the CPU can offer a well-balanced performance-to-cost ratio for smaller production environments. Limited to four CPUs per server, you can find the CPU in Sun servers such as V210 and V220 servers.

If you don't believe your production environment (or development for that matter) will exceed four CPUs and you don't have a processing-complex application (in other words, a cryptography or data mapping application), then the IIIi-based servers are a good choice. Given the CPU is based on the III, it also supports up to 64GB of memory on a four-CPU server such as a V220 ”for small to medium environments, sometimes this is more than required.

Again, if you can architect your WebSphere-based J2EE application to be horizontally scalable (processor-wise) and take advantage of the large amount of available memory, then this processor will suit you well.

Selecting Your Sun SPARC-Based Server Platform

You've looked at the various CPUs available in the Sun UltraSPARC family, so you'll now look at some of the more common Sun-based servers and try to apply them to differing types of WebSphere implementations. As you've seen, there are four primary CPU choices to select from, all of which are fairly aligned with increasingly more powerful and high-performing systems.

Table 4-3 gives an overview of the Sun server model in each you can find each of the four main CPUs.

Table 4-3 focuses on the newer range of servers from Sun, which are mostly based on the UltraSPARC III and IIIi processors.

So, Which CPU?

How long is a piece of string? Seriously, "So, which CPU" is one of those questions. I think the question should be more like "So, which server?" You need to make several decisions to answer this question:

• What are your scalability requirements?

• What are your availability requirements?

• What are your performance requirements?

• What are your disaster recovery and dual-site requirements?

Once you've made these decisions, then you'll know what size and type of system you need. However, as a guide, the following sections contain some pointers for the servers that work well for certain WebSphere implementations.

Small Production Environments

In this size of environment, let's assume your small production environment is based on a single WebSphere channel and the application is operating mostly JSPs and servlets but with a small handful of EJBs and a JDBC connection to a backend database. From a user size (in other words, concurrent users and sessions), the number of concurrent users would be from 50 to 250, depending on the characteristics of your transactions. That is, you could have 10 transactions per second that are small and lightweight (for example, a basic select column_name from a table query) or you could have 10 transactions per second that are heavyweight (for example, the same select statement but with joins and unions).

If you're looking for a solution that just works and works with reasonable performance, then the decision process is somewhat easier. However, if you're looking for something that's highly available (in a single chassis), then the costs increase quickly (as you'd probably guess).

The Sun V series of servers are best suited to smaller, not necessarily mission-critical, systems. Typically they're used in WebSphere environments where more than one WebSphere application server is being used, which helps overcome the single server point of failure issue.

The V880 server is probably the "sweet spot" in the V series range. It's an eight-CPU system with the ability to scale to 64GB of memory. It also comes with an internal 12-bay Fibre Channel Arbitrated Loop (FCAL) disk cabinet, allowing for fast FCAL 10,000rpm drives to be installed (six by default).

As I'll discuss in a moment, the V series doesn't have internal redundancy other than the basic components such as power supplies , multiple disks, and possible multiple interface cards. For this reason, if you're budgeting on the V series of servers and availability is a concern, look to purchase two or more application servers.

On the database side of things, the V series is also a solid choice. Obviously, if your database requirements exceed eight CPUs and 64GB of memory, then these may not be what you need. In that case, consider the 12-way V1280 server.

Never discount looking at database clusters or hot-standby servers either. It's good to have a dual redundant pair of WebSphere application servers; however, if you're operating a WebSphere 4 platform or your application requirements are fairly database intensive and then your database goes down, you could have all the application servers in the world, but your environment would stop working without an active database.

Medium Production Environments

A medium-sized environment is one that hosts between 100 and 500 concurrent user sessions. Furthermore, a midsize production WebSphere environment will carry high-availability requirements.

Based on this, there are two schools of thought you could apply. First, you could look at a horizontally scaled system (to get physical redundancy) and V-class systems with plenty of memory, such as a V880 (64GB of memory) or a V1280 (96GB of memory). You could apply this model to both the WebSphere and database tiers.

On the other hand, it's possible to look at the enterprise or SunFire range of Sun systems such as the F4800 or F6800 servers. As I'll discuss shortly, the SunFire class of Sun servers come with myriad redundant and high-availability features.

With the SunFire family, just about every component is hot-swappable and or has a redundant part. This includes the following:

• Redundant CPUs (and hot-swappable)

• Redundant memory (and hot-swappable)

• Redundant CPU and memory boards

• I/O assemblies

• I/O adaptors

• System controllers

• System clock

• Fireplane interconnect switches

• Power supplies

• Redundant transfer switches (for power supplies)

With the SunFire range of servers, it's also possible to use domains (explained in the following sections in more detail). Domains allow you to split a physical frame into logical domains. These domains act as separate servers, sharing some common components, but are essentially self-contained.

For this reason, it's possible to purchase something such as an F4800 that supports two domains and operate your redundant pair of WebSphere application servers within each of the domains. The same applies for database servers.

You should note that WebSphere has no hard limitation on the number of nodes in a WebSphere cluster. For this reason, it's possible to gain a high degree of what I term as soft availability from massively horizontally scaled environments using smaller systems.

Large Production Environments

A large production environment is one operating with between 500 and 5,000 concurrent users. In this environment, availability and performance are critical. The WebSphere application could be for e-commerce, or it could be for a back-office task, such as a telemarketing system interfacing to something such as a PeopleSoft Customer Relationship Management (CRM) environment.

There are several options for this type of environment. Although a Sun model such as the V1280 is a well-performing system, compared with the similar-sized (CPU and memory-wise) F4800, the F4800 outperforms the V1280. It's another misnomer that a 12-CPU V1280 provides the same performance as a 12-CPU F4800.

Mainly for availability and redundancy, I recommend SunFire-class servers for larger environments. Consider single or multiple F4800s or even F6800s if your processing demands require it.

The F4800, as I've hinted at, is a 12-way CPU with the capability of 96GB of memory. It has two domains, allowing it to operate as a single server or with two active domains (servers) using Sun domain technologies.

The F6800 is a 24-way CPU with the capability of 192GB of memory and supports up to four domains per chassis.

If site or chassis redundancy is a requirement for your WebSphere environment, then a pair of F4800s or F6800s (depending on budget) can be split over sites easily with WebSphere clustering and system mirroring.

The F6800 provides the ability to fully segment the chassis down the "spine" and operate different sides of the spine on different power grids.

If 24 CPUs in a single server isn't enough, alternatives to the F6800s include the E10K with 64 UltraSPARC II CPUs and the E12K and E15K that support up to 52 and 106 CPUs, respectively.

Furthermore, memory on these systems, or the availability of it, isn't an issue. The E12K supports 288GB of memory, but the E15K supports more than half a terabyte. The E15K also supports up to 18 domains inside the chassis, effectively providing the ability for 18 servers to be hosted in a single chassis.

Platform Summary

In the previous sections, you've looked at what the Sun UltraSPARC processors have to offer. WebSphere and Solaris mate well together, and as you'll see through the upcoming chapters, other than IBM's AIX operating system, Solaris is probably the next best operating system for your WebSphere implementation.

Sun offers a solution for any WebSphere environment size and complexity. From the baby V100s to the flagship SunFire E15K, WebSphere operates well on all available models ”the decision will come down to purely a performance and capacity requirement and a scalability and redundancy requirement.