Oracle Real Application Clusters

Cache fusion is a new technology that uses a high-speed interprocess communication interconnect to provide cache-to-cache transfer of data blocks between instances in a cluster. This technology for transferring data across nodes through the interconnect became a viable option as the bandwidth for interconnects increased and the transport mechanism improved. Cache fusion architecture is revolutionary in an industry sense because it treats the entire physical distinct RAM for each cluster node logically as one large database SGA, with the interconnect providing the physical transport among them.

Prior to Oracle 9i RAC, transferring a data block from one node to another involved writing the block from the database buffer cache of the holding node to the shared disk storage. The requesting node read the data block from disk into its own cache.

Cache fusion, a natural evolution of the OPS architecture, implements cache synchronization using a write-back model. The GCS and GES processes on each node manage the synchronization by using the cluster interconnects for data block movement between nodes. Cache fusion addresses transaction concurrency between instances. This section will provide a brief introduction to the different scenarios of intercluster sharing of blocks and explore how they work.

• Concurrent reads on multiple nodes: This occurs when two or more instances participating in the clustered configuration need to read the same block of information. The block of requested information is shared between the instances via the cluster interconnect. The first instance that reads the block would be the owning block and the subsequent instances that require access to the same block will request for the block via the cluster interconnect.

• Concurrent reads and writes on different nodes: This is the common type of concurrency that would be noticed, a mixture of read/write operations against the same block. In this scenario the architecture is similar to that of a single instance except that they happen across the cluster interconnect through a different set of background processes. A block can be read from the current version or from the read-consistent previous version of the block.

• Concurrent writes on different nodes: This kind of operation could be classified, or incorporated, into what was just discussed. This is a situation where multiple instances request modification of the same data block frequently. Again, the outcome, or the solution, to complete these requests is via the cluster interconnect.

Figure 4.4: Cache fusion technology.

In all of these transfer of block requests between instances using the interconnect, the GCS process plays a significant role as the master/keeper of all requests between instances. GCS tracks the location and status of data blocks as well as the access privileges of various instances. Oracle uses the GCS for cache coherency when the current version of a data block is on one instance's buffer cache and another instance requests that block for modification.

Figure 4.4 is a simple illustration of the cache fusion technology. In this example, instance RAC3 has read block 500 from the database and is currently holding the block. Now instance RAC2 requires the same block and makes a request to retrieve it from the database. Instance RAC2 during this process communicates with the GCS process, which could be resident (depending on the resource master) on any of the nodes. For this example, we will place it on instance RAC4. Instance RAC4 understands that instance RAC3 is currently the holder of the block and requests instance RAC3 to transfer the block via the cluster interconnect to instance RAC2.

The resource master for a specific data file is obtained by querying against the GV$GCSHVMASTER_INFO.

SQL>SELECT INST_ID, HV_ID, CURRENT_MASTER, PREVIOUS_MASTER, REMASTER_CNT FROM GV$GCSHVMASTER_INFO WHERE REMASTER_CNT > 0;

INST_ID HV_ID CURRENT_MASTER PREVIOUS_MASTER REMASTER_CNT ------- ----- -------------- --------------- ------------- 1 625 0 32767 1 1 626 0 32767 1 1 627 0 32767 1 1 628 0 32767 1 1 629 0 32767 1 1 630 0 32767 1 1 631 0 32767 1 1 632 0 32767 1 1 633 0 32767 1 2 598 0 1 1 2 599 0 1 1

As illustrated above, while most of the sharing of data blocks between instances happens via the cluster interconnect, Oracle continues to maintain cache coherency between instances. Examples are:

• The cache-to-cache data transfer is done through the high-speed IPC interconnect. This virtually eliminates any disk I/O to achieve cache coherency.

• The GCS tracks one or more past images (PI) for a block in addition to the traditional GCS resource roles and modes. (A past image is the copy of a block retained after the holding instance has shipped the block to the requesting node.) The amount of work that the GCS is required to perform is proportional to the number of instances participating in the clustered configuration.

When multiple instances require access to a block, and a different node masters the block, it is the GCS resources that track the movement of blocks through the cluster. As a result of block transfers between instances, multiple copies of the same block could be on different nodes. These blocks in different instances have different resource characteristics. These characteristics are identified by the following factors:

1. Resource mode

2. Resource role

Resource mode  Resource mode is determined by various factors, such as who the original holder of the block is, what operation the block was acquired for, what operation the requesting holder intending to perform is, what the outcome of the operation is, etc.

Table 4.1 illustrates the different modes and their descriptions.

Resource role  While the resource modes are being maintained between the instances they could be held by the local instance, where its requirement is of a local nature, or could be utilized by more than one instance, where the requirements would be of a global nature.

Table 4.2 illustrates the different roles and their descriptions.