Business Intelligence Roadmap: The Complete Project Lifecycle for Decision-Support Applications

In most organizations, business analysis issues usually revolve around unmet information needs from current heterogeneous data sources and poor quality of the source data.

Information Needs

With the help of business analysts, formulate the business issues that need to be resolved by each BI application objective. Determine what results you want to obtain from the business analysis, for example, answers to such questions as, "Why are we losing 50 percent market share to ABC Company in New England?" Then define the information requirements for the business issues at hand. Determine the subject areas, timing, level of detail, granularity of data, and even what external data you need to answer the business questions. Identify the associated business roles (e.g., senior business management, business analyst, and so on) that would be active in the various decision-support functions.

Identify possible data sources where the required information could reside. Data sources can be internal as well as external, and business insights often lie buried in the relationships among the multiple data sources.

Types of Data Sources

One of the challenges in building a BI decision-support environment is to merge data from different types of data sources. There are three major types of data sources: operational, private, and external (Figure 1.2).

Operational Data

Online transaction processing (OLTP) and batch systems provide internal operational data about subject areas, such as the following:

• Financial

• Logistics

• Sales

• Order entry

• Personnel

• Billing

• Research and engineering

Private Data

This internal departmental data usually comes from the desktops and workstations of business analysts, knowledge workers, statisticians, and managers. Examples include the following:

• Product analysis spreadsheets

• Regional product usage spreadsheets

• Prospective customer databases

External Data

Organizations often purchase external data from vendors that specialize in collecting industry-specific information available in the public domain, such as the following:

• Health care statistics

• Customer profile information

• Customer catalog-ordering habits

• Customer credit reports

External data is usually clustered around the following categories:

• Sales and marketing data: lists of prospective customers

• Credit data: individual credit ratings, business viability assessments

• Competitive data: products, services, prices, sales promotions, mergers, takeovers

• Industry data: technology trends, marketing trends, management science, trade information

• Economic data: currency fluctuations, political indicators, interest rate movements, stock and bond prices

• Econometric data: income groups, consumer behavior

• Demographic data: age profiles, population density

• Commodity data: raw material prices

• Psychometric data: consumer profiling

• Meteorological data: weather conditions, rainfall, temperature ( especially for agricultural and travel industries)

Source Data Quality

Merging and standardizing data is usually a requirement of every BI application but one that is not so easy to accomplish. One of the difficulties in merging and standardizing data from different types of data sources is that the data is stored in different file structures on different platforms. What makes the process even more difficult is that the keys for the same objects on different data sources usually do not match, the definitions for the same apparent data are often inconsistent, and the values are often missing or conflicting. In addition, different people in the organization have authority to determine business rules and policies for data from different types of data sources, and resolving data conflicts among them or getting clarification is often all but impossible .

Standardizing data from internal operational data sources is difficult enough, but standardizing data from private and external data sources is a major challenge and could be costly. This cost should be calculated and included in the cost-benefit analysis.