Showing posts with label Data Warehousing. Show all posts
Showing posts with label Data Warehousing. Show all posts

SAP BW Third Party Solutions

SAP BW is based on common industry-accepted standards that enable third-party vendors to integrate their products with SAP BW. Since 1998, several third-party vendors have certified their products to load data to and access it from SAP BW. Data loading tools use Staging BAPI, and data access tools use ODBO.

Most data warehouse and business intelligence vendors provide a collection of tools that support a specific function needed to construct a data warehouse such as data extraction, data transformation, and end-user data access tools. Often, tools provided by vendors, even from one vendor, do not support a common development environment or user interface. This situation makes it difficult for organizations to build dedicated teams for specific tasks of a data warehouse construction process.

Though the number of vendors providing data warehousing solutions and tools is increasing, the following section compares only a few tools available at the time of this writing that interface with SAP R/3 and SAP BW.

ActaWorks for SAP from Acta Technology is a data extraction, transformation, and data-loading product. Though ActaWorks provides some initial SAP R/3 data extraction schemes and generates ABAP code to populate data marts, you must develop change identification and delta change capture techniques for transaction data in SAP R/3. ActaWorks is a good product, but from an overall global information delivery perspective, it is not an information delivery environment. To fill this gap, Acta is partnering with Cognos and other vendors to provide a frontend environment to complete its data warehouse solution.

inSight from arcplan is among the first few ODBO-certified products for SAP BW. inSight is a very mature product. You can access EIS data from SAP R/3 and SAP BW queries. arcplan has gone one step further when it comes to delivering SAP R/3 and SAP BW data analysis and reporting environments to the Internet by providing ActiveX controls for pure Web clients. arcplan has introduced another product, DynaSite, for pure Java-based clients.

Evolutionary Technology Inc. (ETI) is best known for its data extraction technologies. Its products are powerful and can extract data from just about any data source, including SAP R/3. Its products are also certified to load data in SAP BW using Staging BAPI. In comparison to SAP BW, ETI covers only data functionality in BW.

Information Builders, Inc. (IBI) is one of the pioneers in enterprise reporting and data warehousing. IBI provides several tools that cover similar functionality as SAP BW. However, such tools still use old technologies and are not integrated. They do, however, provide a robust Enterprise Data Access (EDA) server that supports distributed queries across homogeneous and heterogeneous data sources and platforms.


Informatica has been a leading data mart construction tool provider for more than a decade. Informatica products can extract data from flat files and relational databases to process and load data in target data marts. Informatica was late to enter the ERP data extraction race. Recently, Informatica has filled the ERP data extraction gap by adding new data extraction components for SAP R/3 and PeopleSoft. Informatica's PowerCenter was among the first wave of products certified for Staging BAPI to load data in SAP BW.

Oracle is in direct competition with SAP BW in the data warehouse market. Oracle provides many tools to build data warehouses and deliver information. Oracle has a good potential to provide a unified data warehouse environment but still lacks product integration when compared with SAP BW. Oracle also provides Oracle Tool Kit for SAP R/3.

Seagate is a midrange data warehouse construction and information delivery tool provider. Seagate is acquiring technologies from several vendors and offers an integrated data warehouse solution. Seagate Holos OLAP is the OLAP engine. The Reporting tools come from Crystal that can access data from most relational databases. Seagate also announced that it will build data extraction tools for SAP R/3.

Related Posts

Data ware house building statagies

Components of data ware house

Construction of data ware house

How to get better performace with data ware house?

SAP reporting

Limitations of SAP reporting

ABAP TOPIC WISE COMPLETE COURSE

BDC OOPS ABAP ALE IDOC'S BADI BAPI Syntax Check
Interview Questions ALV Reports with sample code ABAP complete course
ABAP Dictionary SAP Scripts Script Controls Smart Forms
Work Flow Work Flow MM Work Flow SD Communication Interface


Sample SAP BW Report

The first step to access SAP BW reports is to start the BEX Browser. You will be prompted for your user name and password assigned by your SAP BW administrator.

Once SAP BW accepts your user name and password, the BEX Browser displays all reports and analytical applications for which you are authorized. On the left are the channels. Each channel contains one or more reports.

Figure below shows that you are in the General Purpose Channel; the title also shows the channel name. This channel has two clusters. Clusters are used to logically group a set of reports that address a similar subject area. This example uses Customer and Employee channels.

To launch an analytical application from the BEX Browser, simply right-click the report text, Sales Distribution by Employee, as shown in figure below. This action gives you several options, such as Execute, Preview, and Change. Select Execute to open the selected report.

Here you can drill down by product hierarchy as shown below or some other navigational attributes. You can also normalize data in several ways to meet your analytical needs.

Related Posts

Data ware house building statagies

Components of data ware house

Construction of data ware house

How to get better performace with data ware house?

SAP reporting

Limitations of SAP reporting

ABAP TOPIC WISE COMPLETE COURSE

BDC OOPS ABAP ALE IDOC'S BADI BAPI Syntax Check
Interview Questions ALV Reports with sample code ABAP complete course
ABAP Dictionary SAP Scripts Script Controls Smart Forms
Work Flow Work Flow MM Work Flow SD Communication Interface

Data Flow in SAP BW

When the user launches a report from BEX Explorer, it starts a BEX Analyzer session and opens an Excel workbook that contains an embedded BW query.

BEX Analyzer sends a request to the BW application server for relevant data. If this is a new query for a user, no data is found at the application server level. The query is then pushed further down to the database server to fetch a result set from an InfoCube or an aggregate associated with an InfoCube.

The database server fetches data from an InfoCube and hands it over to the application server, where it restructures the table-oriented data to a multidimensional structure, called a query cube, as shown below.

A query cube is the subset of an InfoCube, which is specified by the query. Some characteristics are restricted to specific values, whereas some are not chosen in the query. For this reason, the size of the query cube is much smaller than the actual InfoCube.

A query cube is unique to a user request and, hence, is not shared by other users. If several users start the same query, all will have their own unique query cubes built at the application server.

This is because the result set may vary based on selected dimensions or special filters. Also, each will have full control on how to navigate through the query cube (slicing and dicing) without interfering with analytical work done by others.

As the number of query cubes increases, the application server may run out of memory very quickly. For this reason, the application server for SAP BW requires more memory as compared to traditional SAP R/3 for OLTP instance. Moreover, the SAP BW application server needs more computer power due to data aggregations and computation requirements to satisfy end-user data analysis needs.

Once a query cube completes, a portion of data from the query cube is pushed up to load data in Microsoft Excel worksheets based on the current end user's data view.

Thenthe end user performs typical slice-and-dice and drill-down data analysis using BEX Analyzer functions. If the user needs additional drill-down data that is not presented at a given time, the request is sent to the query cube to send data in the spreadsheet for drill-down analysis.

If that detailed data is not in the query cube, the OLAP processor will refresh the query cube from the InfoCube. This navigational and data refresh process continues until the data requirements are met at each level.

A query cube always relates to only one InfoCube. In other words, you cannot build a query cube that gets data from more than one InfoCube.

Related Posts

Data ware house building statagies

Components of data ware house

Construction of data ware house

How to get better performace with data ware house?

SAP reporting

Limitations of SAP reporting

ABAP TOPIC WISE COMPLETE COURSE

BDC OOPS ABAP ALE IDOC'S BADI BAPI Syntax Check
Interview Questions ALV Reports with sample code ABAP complete course
ABAP Dictionary SAP Scripts Script Controls Smart Forms
Work Flow Work Flow MM Work Flow SD Communication Interface

SAP Data Warehouse Building Strategies

SAP customers have taken several approaches to building a data warehouse using R/3 technologies.

Database-Centric Data Warehouse:

In the database centric approach we have to make a copy of an R/3 database for users to report against. Conceptually, the idea is simple, but it is an operations keeping R/3 OLTP and R/3 reporting in synch is very difficult.

The benefit of this model is that you will have no performance degradation on your R/3 OLTP environment due to reporting and data analysis activities. We can define special user authorization and profiles for reporting users . This R/3 can also become the data source for intra-application data instead of OLTP R/3.

The problem in this approach is that we you have access to only the data that exists in the OLTP instance that is we have no historical data to report on. Moreover, we will face the same limitations and complexities of R/3 information systems and reporting tools that are not Web enabled. This database-centric approach will be an expensive proposition due to additional hardware and intense operations tasks to keep both instances operational at all times.

ALE-Centric Data Warehouse

In this model we do not copy a full database but rather use ALE to propagate specific changes into the reporting environment-another R/3 instance.

Based on the customer's business model, ALE packages transaction and/or master data in the form of Intermediate Documents (IDOCs). IDOCs are data containers similar to EDI standard documents. The figure below shows an example of how a MNC used this approach to implement an ALE-based reporting environment .

Here four individual R/3 instances were connected via ALE:

  1. R/3 instance to manage master data.
  2. R/3 instance for financial operations.
  3. Instance for logistics operations.
  4. Instance for reporting.

Under this model, special SIS structures and special ledger structures are transferred to reporting instances using ALE, copy management, and some homegrown techniques to synchronize objects across all instances. ALE copied new changes in the reporting instance at 10-minute intervals.

Additional SIS structures were updated within the reporting instance using copy management techniques to build additional tables containing several levels of aggregated data for reporting. This enabled end users to access near real-time data without impacting performance on the R/3 OLTP operations. For corporate central data warehouses, new data was extracted from the reporting instance instead of the R/3 OLTP instance.

The ALE centric reporting environment advantages:

  1. Contains only that data required for reporting and analysis. It is not an identical copy of OLTP R/3 instances.

  2. Can accumulate historical data.

  3. Can import non-R/3 data for reporting.

The disadvantages of this environment are :

  1. Has the same inflexible R/3 reporting tools.

  2. Requires additional application servers to manage ALE traffic for all four instances.

  3. ALE technology is not rich and mature enough to handle large data volumes and user-specific data objects.

  4. Physical database implementation is still R/3-transaction-centric. Has no true multidimensional star data structures.

  5. Is heavy on hardware/software/network resources.

Third-Party-Tool-Centric Data Warehouse:

Third-party data extraction tools work to a point when transaction and associated data volume is small and business rules/configuration behind such data sets is simple. Some data extraction tools extract data at the database level, leaving critical data behind in pool and cluster tables

Often, the ABAP code generated by these tools is not optimized enough to fetch and transport data in the most efficient manner. One of the major problems in implementing a third-party solution is that the OLTP and data warehouse/OLAP teams do not share the same infrastructure.

Related Posts

Components of data ware house

Construction of data ware house

How to get better performace with data ware house?

SAP reporting

Limitations of SAP reporting

ABAP TOPIC WISE COMPLETE COURSE

BDC OOPS ABAP ALE IDOC'S BADI BAPI Syntax Check
Interview Questions ALV Reports with sample code ABAP complete course
ABAP Dictionary SAP Scripts Script Controls Smart Forms
Work Flow Work Flow MM Work Flow SD Communication Interface

SAP Reporting Limitations

The following are the shortfalls of SAP reporting:

1.Information on existing reports is either missing or unclear. Searching a report among thousands of available reports in R/3 is a big problem.

2.Available reports are designed to meet operational and transactional information needs. Most reports are predefined, list oriented, and provide very limited OLAP functionality.

3.Several reporting modules and associated reporting tools make it hard to select a specific tool. Reporting tools are inconsistent, and designing reports is a complex process. Maintenance of thousands of reports for software upgrades is a huge challenge.

4.Fragmented reporting menu access requires extensive end-user training to navigate through several multi-level menus to display a few reports.

5.Performance impact on R/3 OLTP operations due to reporting is another major issue. The R/3 systems are configured to provide high OLTP transaction rates. Building a robust reporting and OLAP environment under an OLTP environment requires different configuration parameters that will degrade OLTP operations.

To overcome OLTP and reporting co-existence problems,Application architects at SAP planned to break down the traditional SAP R/3 very tightly integrated application modules into several loosely coupled applications; these applications would still be connected to each other by use of Application Link Enabling (ALE) technology .

The intent of Business Framework is to make individual application components stand-alone manageable entities that still have a robust business level integration among all. ALE technology became the "glue" that holds all application modules together.

The previous post of the blog deals with SAP reporting environment.

ABAP TOPIC WISE COMPLETE COURSE

BDC OOPS ABAP ALE IDOC'S BADI BAPI Syntax Check
Interview Questions ALV Reports with sample code ABAP complete course
ABAP Dictionary SAP Scripts Script Controls Smart Forms
Work Flow Work Flow MM Work Flow SD Communication Interface


SAP Reporting Environment

Several information systems exist in R/3 that provide a reporting and data analysis environment limited to specific application modules. Information systems in R/3 are similar to data marts where subject-specific data is collected and stored in database tables dedicated for reporting. R/3 information systems provide you with information that enhances the capabilities of more than 2,000 standard R/3 reports.

Open Information Warehouse and Information Systems in SAP :

Information systems in SAP R/3 are components of SAP's Open Information Warehouse (OIW) framework.

Individual information systems-LIS, FIS, HIS, and so on-in SAP R/3 are limited to a specific application area and associated data objects, except the EIS. These information systems provide an effective method of retrieving summarized data. Following are the major information systems in R/3:

  1. Executive Information System (EIS)

  2. Logistics Information System (LIS)

  3. Financial Information System (FIS)

  4. Human Resources Information System (HIS)

Granularity of data available in information systems depends on the type of reporting environment selected. At the EIS level, you access summarized information that can go horizontally across one or more application modules. In other information systems, you are limited to only one application area at a time.


Logistics Information System (LIS)

The LIS is widely used by R/3 customers and plays a major role in preparing information for SAP BW. It consists of the following sub-information systems (the list may vary based on the SAP R/3 OLTP release):

  1. Sales Information System (SIS)

  2. Purchasing Information System (PIS)

  3. Inventory Controlling (INVCO)

  4. Shop Floor Information System (SFIS)

  5. Plant Maintenance Information System (PMIS)

  6. Quality Management Information System (QMIS)

  7. Retail Information System (RIS)

  8. Warehouse Management Information System (WMIS)

  9. Transportation Information System (TIS)

R/3 transaction tables and LIS information structures are updated in two modes: synchronous and asynchronous. When the synchronous mode is selected, both the transaction tables and information structures are updated simultaneously. However, when the asynchronous mode is selected, transaction tables are updated first, and information structures later. In this case, you will not find up-to-the-second information in LIS info structures for reporting. The reason is that the R/3 system commits transaction data quickly and notifies the end user that the transaction is complete. This enables the end user to start another transaction.

But in the background, SAP R/3 delivers the LIS table update job to another dialog work process to update LIS information structures. This delay might range from a few seconds to a few minutes, depending on your system's available resources. Transaction OMO1 is used to define the synchronization scheme for a specific LIS information structure. LIS information structures are simple database tables named S001 through S499. These tables are used for reporting and analysis.

You can define your own custom information structures to capture specific data for reporting (name range S500 through S999). SAP BW pulls data from all LIS information structures defined in SAP R/3; however, the update logic for SAP BW-specific information structures varies from traditional update rules used for LIS reporting.

You can browse here about the previous post better performance achievement with data ware house


Get better Performance of Data Warehouse

A data warehouse has several characteristics-dimensions-responsible for its performance. Often only one dimension contributes to poor data warehouse performance. It may be , narrow network bandwidth causes delays in moving data from transaction systems as well as delivering information to end users while data transformation and end-user query execution are not performance issues. To end users, poor network bandwidth is perceived as poor data warehouse performance.

The following figure shows characteristics of a high-performance data warehouse. It shows that this data warehouse supports a large number of users. Most users issue less complex queries without any performance issues and have a low data warehouse operations/maintenance time requirement, making this data warehouse highly available. Under this environment, everyone is satisfied with the data warehouse performance. In reality, this seldom happens.

The number of users, is not a good indication of data warehouse performance. One hundred users may be issuing simple queries and not have a problem until one analyst starts a complex analytical job that virtually joins a large number of tables to do complex sales trend analysis. This exercise will probably block the data warehouse server for hours. Data warehouse architects must design governing features to control such runaway resource-consuming processes during peak hours.

The consequence of not having complete order operation information in the data warehouse is that the planning, finance, sales, and marketing organizations will not have a full view of corporate operations, such as product inventories and what to stock to fulfill consumers' demands.

The performance issue here is that extracting complete data sets from OLTP and loading that information in a data warehouse all in one step can consume significant OLTP and data warehouse resources, such as the locking up of source and target data objects, network bandwidth, and CPU/memory usage.

Solution is to keep the data extraction process out of the daily OLTP maintenance operation and break down large data extraction processes into multiple tasks. Each task is scheduled several times during regular OLTP business operations to extract and move new data in the data warehouse in an operational data store. Then refresh the data warehouse once or twice a day by combining all incremental data sets from the operational data store without touching the OLTP systems. (1.4)

ABAP TOPIC WISE COMPLETE COURSE

BDC OOPS ABAP ALE IDOC'S BADI BAPI Syntax Check
Interview Questions ALV Reports with sample code ABAP complete course
ABAP Dictionary SAP Scripts Script Controls Smart Forms
Work Flow Work Flow MM Work Flow SD Communication Interface

How to Construct Data Warehouse

The previous post of the blog deals with the basic components of Data ware housing. Here we are going to see the technical issues that have to be taken cared while designing the data warehouse.

Global Information Delivery

The Internet and intranets are the primary vehicles for information delivery. There must be robustness (scalability, security, reliability, and cost) of Web services needed to deliver information to end users. It is critical that such Web services provide seamless integration and provide similar development/management environments with the rest of the data objects in all data warehouse layers.

Global Catalog

Global catalog goes hand in hand with the information delivery services. It is hard to find what you need because catalogs are based on an individual data warehouse. End users spend a lot of time finding what they need. The support of one global catalog is a key component of a global information delivery system.

Metadata Management

Metadata is information about the data such as data source type, data types, content description and usage, transformations/derivations rules, security, and audit control attributes. Access to metadata is not limited to the data warehouse administrator but must also be made available to end users. For example, users want to know how revenue figures are calculated. Metadata defines rules to qualify data before storing it in the database. The end result is a data warehouse that contains complete and clean data.

Manageability

At present scenario data objects are distributed across the world and also reside on end-user workstations (laptops), it is a difficult to manage such environments. As part of its Customer Relationship Management (CRM) initiative, SAP is planning a Mobile Sales Automation (MSA) server that integrates and manages data between the SAP Business Information Warehouse and the data sets on a salesperson's laptop.

Adaptation of New Technologies

Implementation of an enterprise data warehouse is usually a multi-year project. echnically, as long as you have built your data warehouse based on an architecture using accepted industry standard APIs, you should be able to incorporate emerging technologies without extensive reengineering.

If your existing data warehouse environment uses open APIs, you can easily join information from multidimensional and relational data sources using ODBC. ODBO, for example, integrates multidimensional data from SAP BW and Microsoft's OLAP server. It shall be made sure that the data warehouse can adopt new emerging technologies with the least amount of work.

Security

A data warehouse environment must support a very robust security administration by using roles and profiles that are information object behavior-centric rather than pure database-centric. For example, a role such as cost center auditor defined in a data warehouse allows one to view cost center information for a specific business unit for a given quarter but not to print or download cost center information to the workstation.

Reliability and Availability

New data warehouse construction products provide methods to make systems highly available during data refresh. Products like SAP BW refresh a copy of an existing data object for incoming new data updates while end users keep using the existing information objects. However, when a new information object is completely refreshed, all new requests are automatically pointed to the newly populated information object. Such technologies must be an integral component of enterprise data warehouses.

Upgradability and Continual Improvements

If any component of a data warehouse (database management system, hardware, network, or software) needs upgrades, it must not lock out users, preventing them from doing their regular tasks. Moreover, any time a new functionality is added to the environment, it must not disrupt end-user activities. One can apply certain software patches or expand hardware components (storage) while users are using the data warehouse environment. During such upgrades, end users may notice some delays in retrieving information, but they are not locked out of the system.

Scalability and Data Distribution

Due to large data volume movement requirements, data warehouses consume enormous network resources (four- to five-fold more than a typical OLTP transaction environment). In an OLTP environment, one can predict network bandwidth requirements because data content associated with each transaction is somewhat fixed. In data warehousing, it is very hard to estimate network bandwidth to meet end-user needs because the data volume may change for each request based on data selection criterion. Large data sets are distributed to remote locations across the world to build local data marts. The network must be scalable to accommodate large data movement requests.

To build dependent data marts, you need to extract large data sets from a data warehouse and copy them to a remote server. A data warehouse must support very robust and scalable services to meet data distribution and/or replication demands. These services also provide encryption and compression methods to optimize usage of network resources in a secured fashion.

This can be demonstrated as shown below.


ABAP TOPIC WISE COMPLETE COURSE

BDC OOPS ABAP ALE IDOC'S BADI BAPI Syntax Check
Interview Questions ALV Reports with sample code ABAP complete course
ABAP Dictionary SAP Scripts Script Controls Smart Forms
Work Flow Work Flow MM Work Flow SD Communication Interface