Business Information Warehouse for SAP (Prima Techs SAP Book Series)

Before loading transaction data in SAP BW, you need to load the reference data. Reference data is master data, texts, hierarchies, and other configuration-specific data that is used across all analytical applications in SAP BW. Moreover, having master data in SAP BW first speeds up the InfoCube population process. This is because SID tables for master data are created during master data loading, and these SIDs are used in InfoCube dimensions as pointers for master data.

SAP BW version 1.2B does not support incremental data load services for reference data. This means that you need to load full reference data in SAP BW and then execute change run procedures to reconcile data that is already populated in InfoCube structures.

To load master data in SAP BW, you do not need to. run any statistical update procedures. However, the order in which you load reference data in SAP BW is very important, as listed here:

1. Master Data

2. Texts

3. Hierarchies

The actual disk space used by reference data tables in SAP BW will be almost the same as in SAP R/3. If you are planning to support multi-language SAP BW implementation, the number of master data text records may be several times that of the actual master data record, just as in SAP R/3. For example, assume you have one million material master records and support three languages. You will have master data text in three languages, resulting in three million material master text records. Make sure that you have sufficient disk space in SAP BW to accommodate fivefold material master text records.

Quality of master data for SAP BW is important. Make sure you load only the data that you really need from SAP R/3 in SAP BW. When SAP BW requests master data from SAP R/3, it pulls all master data records, including those that are marked as delete or obsolete. This may cause problems when loading master records.

For example, say a user created a master material record in SAP R/3 with the Material ID of 16-6667-AR#. After updating this record, the user realized that the Material ID has a # character, which is an error. It is supposed to be a 3. The user deleted this record and re-created it with a correct Material ID of 16-6667-AR3. Remember that SAP does not physically delete the master data; rather it flags it to be deleted. No one can use this for new transactions. However, when SAP BW requests material master data from SAP R/3, the deleted flag master records are also shipped to SAP BW.

You now have two problems with this data. First, it contains an invalid character, and the loading job will fail when it encounters this record. Second, this record is not needed because it is an invalid record in SAP R/3 OLTP.

Luckily, SAP BW provides a mechanism to identify permitted characters beyond typical text and numbers, as shown in Figure 9-4. There you identify characters that will be acceptable in SAP BW. This is a quick fix to acknowledge invalid data, but you need to simply exclude such data sets at the data source level, SAP R/3.

Figure 9-4: Identifying Permitted Characters in SAP BW Beyond Standard Text and Numeric Characters in SAP BW 1.2B. A Similar Method Exists in SAP BW 2.0A.

This is a typical data quality issue in data warehousing. SAP BW data quality is no exception, even when data is coming from an SAP R/3 instance. Receiving data from SAP R/3 does not mean that you have quality data. For this reason, you need to establish data quality guidelines to qualify data before loading it in SAP BW.

For efficient data loading in SAP BW, collecting and grouping information about data sets in SAP R/3 is very important for reference and transaction data.

Analyze your master data and identify ways to group master data in small buckets. Estimate the number of records in each group. Suppose you have four million material master records in SAP R/3. You need to find a way to group four million records in a good sizable bucket that is right from your SAP R/3, network, and SAP BW resources. You may group material master records by Product Family or by range of Material IDs. Schedule 400K data records per request. To load all four million records, you will schedule 10 jobs. If 400K records are processed within a reasonable time period, say 10 minutes, you may increase the number of records and determine what number of records per data request is best for your environment.

For a full initial SAP BW data baseline, analyzing transaction data is much harder than master data. For the Sales and Distribution (SD) subject data warehouse, first you need to analyze base SAP R/3 SD transaction tables to run a statistical update in groups, and then segment statistical data to load in SAP BW.

For example, one cannot assume that all orders always have the same number or line items or number of shipments or invoices. For this reason, you need to do extensive analysis of existing order data and identify a collection of orders, invoices, and delivery records for an acceptable statistical update and data sets for SAP BW loads.

After running statistical updates, one way to get a count of records per order by field combination is to execute a few SQL queries against SAP BW LIS tables in SAP R/3. Use these statistics to process groups of orders at a time.

For transaction data loads in SAP BW, you can use TRFC and store data in ODS before loading it in an InfoCube. This way, large data sets are first sent to ODS. Then in SAP BW, you can make additional data quality or data selections to populate InfoCubes without having to pull data from SAP R/3.

Preparing Statistical Data in SAP R/3 for SAP Standard SD InfoCubes

Preparing and loading initial statistical data for SD InfoCubes consists of the following for primary LIS structures.

• Set up an LIS environment using report RMCSBIWC. SAP provides several programs in SAP R/3 that set up other objects for LIS environment in SAP R/3. To set up LIS structures for SAP BW, use report RMCVNEUA for S260, S263, and S262; RMCVNEUL for S261; and RMCVNEUA for S264 (new in SAP BW 2.0A).

• Deactivate LIS structures using transaction OM01 to disable LIS updating. Make sure to turn on updating after initial loads in SAP BW.

• Execute statistical update program. Use transaction OLI7 for S260, OLI8 for S261, OLI9 for S262, and OLI7/OLI8 for S263. Based on the transaction data volume in your OLTP instance, you may need to run several statistical updates to fully populate one LIS table, say S260. Here you will use information that you have collected in the previous section, grouping transaction data for statistical updates. For example, you may decide to run one statistical update session that updates only orders with Document numbers range from 1 through 3000, as shown in Figure 9-5. You may also have an option to restrict the transaction data set by company code or sales organization or other parameters as shown in Figure 9-5. Remember that you can run one or more such groups of statistical updates over several days based on your OLTP system availability. This technique reduces the downtime needed to complete full statistical updates.

  Figure 9-5: Setting Statistical Update for SAP BW LIS Structures in SAP R/3 OLTP Data Source.

• Load data in SAP BW. This is done from SAP BW by scheduling InfoPackage for a specific InfoSource. You learn more about loading in SAP BW in the next section.

• Activate LIS structures updating (OM01, OM02).

For delta updates, several programs fill LIS structures for BW. As shown in Figure 9-3, these programs first determine the status of which table to fill (xxxBIW1 or xxxBIW2) by looking in the TMCBIW table. For example, program RMCSS260 fills S260 LIS table, and RMCSS261 updates S261 table.

Table RSSGTPDIR contains the program names of SAP BW load programs. The function module, SAPLMCSBW, is used to fetch data from these tables.