Network Analysis, Architecture and Design, Second Edition (The Morgan Kaufmann Series in Networking)

4.10 Example Application of Flow Analysis

We will now bring the concepts of flow analysis together for an example network, in this case a network to support computing and storage management. For this network project, the computing and storage devices already exist in multiple buildings on a campus. The buildings and devices that will be using this network for computing and storage management are shown in Figure 4.40.

Figure 4.40: Building and device locations for example.

From the requirements analysis process, we have been able to determine that this network has four types of flows:

Type 1: Flows between high-performance computing devices. There are compute servers that are the high-performance devices for this network. The first type of flow consists of traffic flows between these devices. Flows are sometimes (approximately 10% of the time) synchronized between pairs of devices. At other times, these devices may draw data from the local data store of another computing device, from the storage server at the Main Engineering building, or request a computing or data management task from a device. Most computing tasks are controlled from Main Engineering.

Type 2: Data migration from computing to storage at Main Engineering. These flows may be from each compute server as its task is completed, from the local data store at each compute server at a specific time, or from the compute server at Main Engineering at the completion of a combined (multiserver) task.

Type 3: Data migration to external server. Final data sets, or extracts of final data sets, are pushed from the Main Engineering storage server to a storage server at a building where external (Internet) access is managed. Data at this external access server are used to feed other campuses and for remote (off-site) data archival.

Type 4: Data archival and Internet access. These are flows from the external access server to users on the Internet, as well as flows to an off-site archival server.

These flow types are added to our map and are shown in Figure 4.41.

Figure 4.41: Map with flow types added.

From discussions with various users of these applications, we learned that flows usually run in this order: type 1, type 2, type 3, and type 4. For flow type 1, a Main Engineering compute server may act as a server for the compute servers in buildings A through C, and getting data from the storage servers in the Main Engineering building. This follows a hierarchical client-server flow model. Compute servers from buildings A through C and Main Engineering may also act as synchronized peers, using a distributed-computing flow model.

From discussions with engineering users, we found that the computing application runs in either batch or interactive mode, from about 10 minutes to several hours, generating files of sizes ranging from 1 to 2 MB (synchronization, or sync, files), 10 to 100 MB (interactive updates), and 100 MB to more than 1 GB for final data sets.

Interactivity for the computing application is the ability to steer or direct the computation. This requires synchronization on the order of human response time (HRT) (100 ms) and updates on the order of 1 second. Users expect to have up to two tasks running concurrently.

For flow type 2, data are stored at the storage servers in Main Engineering. Data can be from interactive updates, final data sets, and extracts (selected subsets of the final data set). Data also migrate from local stores at each computer server, usually every few hours.

For flow type 3, full data sets and extracts of the data sets are migrated to the external access server. Extracts are approximately 80% of the size of a full data set. Data sets are migrated hourly.

For flow type 4, users from other campuses, via the Internet, access data. Data sets are archived at an off-site facility. The system is expected to support the download of a full final data set within a few minutes.

Performance Envelope from Requirements Analysis

Flows of type 1 involve the frequent passing of 1- to 2-MB sync files with delays on the order of HRT, 10- to 100-MB update files on the order of 1 second, and final data sets of 500 MB to 1 GB on the order of minutes to hours, with up to two tasks running concurrently. From this information, we can estimate a range for capacity performance for these flows. Each of these flows is multiplied by 2 for concurrency.

Type 2 flows involve migrating (pushing) updates, final data sets, and extracts. The delay characteristics of these flows are much less strict than those for the computing function, with delays ranging from 10 to 10⁴ seconds.

Update files: (10 to 100 MB)(8 b/B)/(10 to 10⁴ s) 8 Kb/s to 80 Mb/s Final data sets: same as for flow type 1

The performance envelope for final data sets, updates, and synchronization files is shown in Figure 4.42.

Figure 4.42: Performance envelope for example.

Flow Models

For flow type 1 between computer servers and the Main Engineering storage servers, flows can follow distributed-computing and hierarchical client-server computing flow models, as shown in Figure 4.43.

Figure 4.43: Flow models for flow type 1.

In the distributed-computing model, each device can act as a data source and as a data sink and data transfer is synchronized between devices at about 100 ms. In the hierarchical client-server model, data sets can flow from the storage server to the compute servers in Main Engineering, which then pass down to compute servers in Buildings A through C. There is no synchronization for this model.

Flow type 2 consists of data pushes from each compute server to the storage server in Main Engineering. Each compute server is a data source, and the Main Engineering storage server is a data sink (Figure 4.44).

Figure 4.44: Flow model for flow type 2.

For flow type 3, the storage servers in Main Engineering are data sources and the external access server is a data sink (Figure 4.45).

Figure 4.45: Flow model for flow type 3.

For flow type 4, a client-server flow model exists between external users of the data, including off-site archival, and the external access server (Figure 4.46).

Figure 4.46: Flow model for flow type 4.

Flow Map

Figure 4.47 is an example of a flow map that describes flows between buildings. Note that all devices have been removed from this map. This is often done for larger networks because the number of devices on a map can become unwieldy. Also, a flow aggregation point has been added between Buildings A through C and Main Engineering. This is done to show the aggregated performance requirements at Main Engineering (flow F4).

Figure 4.47: Flow map for example.

For the flow map shown in Figure 4.47, flows F1, F2, and F3 have the same performance requirements, consisting of flow types 1 and 2. Flow F4 is an aggregate of flows F1, F2, and F3 (flow types 1 and 2). Flow F5 consists of flow type 3, and flows F6 and F7 are flows of flow type 4.

Next, we combine the performance requirements from each of the flow types and apply them to flows F1 through F7, as shown in Figure 4.48.

When these performance requirements are added to the flow map from Figure 4.47, we get Figure 4.49. Two performance profiles (P1 and P2) were generated for multiple flows: P1 for flows F1, F2, and F3; and P2 for flows F5 and F7.

Figure 4.49: Performance requirements added to flow map.

A two-part flowspec for each flow that has a performance profile of P1 (F1, F2, and F3) would look like Figure 4.50.

Figure 4.50: Two-part flowspec for each flow with performance profile P1.