performance tuning for sap business information warehouse · 2019-11-12 · performance tuning for...
TRANSCRIPT
SAP TechEd ‘03 Basel
© 2003 SAP AG BW 354, Uwe Heinz; Alexander Peter 1
Performance Tuning for SAP Business Information WarehouseAlexander PeterProduct Manager BI, SAP AGUwe HeinzDeveloper BI, SAP AG
SAP AG 2003, TechED EMEA, BW354, Alexander Peter, Uwe Heinz
Learning Objectives
As a result of this workshop, you will be able to:
Differentiate and Analyse Performance IssuesExplain effects of modeling on performanceSuggest means to improve query and data load performanceTrace Web Queries
SAP TechEd ‘03 Basel
© 2003 SAP AG BW 354, Uwe Heinz; Alexander Peter 2
SAP AG 2003, TechED EMEA, BW354, Alexander Peter, Uwe Heinz
SAP BW Architecture
SAP AG 2003, TechED EMEA, BW354, Alexander Peter, Uwe Heinz
Agenda
Modeling
Query Performance
General Settings and Checklist
Extraction / Data Load Performance
Analysis Methods
SAP TechEd ‘03 Basel
© 2003 SAP AG BW 354, Uwe Heinz; Alexander Peter 3
SAP AG 2003, TechED EMEA, BW354, Alexander Peter, Uwe Heinz
Agenda
Modeling
Query Performance
General Settings and Checklist
Extraction / Data Load Performance
Analysis Methods
SAP AG 2003, TechED EMEA, BW354, Alexander Peter, Uwe Heinz
Performance Tuning in BW
Database
Database
Performance Tuning
Application
Application
Performance Tuning
OLTP SystemsApplication Development and performance tuning separatedPerformance tuning by basis experts
Performance Tuning
SAP BWPerformance Tuning as holistic process over application design and database configuration
SAP BW
OLTP
SAP TechEd ‘03 Basel
© 2003 SAP AG BW 354, Uwe Heinz; Alexander Peter 4
SAP AG 2003, TechED EMEA, BW354, Alexander Peter, Uwe Heinz
General Performance Rules
Basic Rules for Performance Optimization:
Parallel processes are fully scalable
Do it parallel: Deploy parallelism on all Do it parallel: Deploy parallelism on all available levelsavailable levels
Be economic: Eliminate all unnecessary Be economic: Eliminate all unnecessary processesprocesses
Keep it small: Reduce the data volume to Keep it small: Reduce the data volume to be processedbe processed
SAP AG 2003, TechED EMEA, BW354, Alexander Peter, Uwe Heinz
Checklist
Sound Data ModelThe Data Model is one of the most important issues
BW Reporting PerformanceAggregatesPre-calculated Web TemplatesOLAP Cache
Extraction / Data Load PerformanceRecommendations in SAP notes 130253 and 417307
SAP TechEd ‘03 Basel
© 2003 SAP AG BW 354, Uwe Heinz; Alexander Peter 5
SAP AG 2003, TechED EMEA, BW354, Alexander Peter, Uwe Heinz
Performance Guidelines – General Aspects – 1 –
Database (or Physical) PartitioningDatabase tables are cut into smaller chunks (partitions)One logical database tableTransparent for userAvailable for the following database management systems
Range Partitioning: ORACLE, Informix, IBM DB2/390Hash Partitioning: IBM DB2/UDB
Year / Month Customer Sales2003/01 4711 102003/01 4712 152003/01 4713 202003/01 4714 232003/02 4711 442003/02 4713 322003/02 4714 17
DatabaseTable(logical)
DatabasePartition
DatabasePartition
SAP AG 2003, TechED EMEA, BW354, Alexander Peter, Uwe Heinz
Performance Guidelines – General Aspects – 2 –
Database (or Physical) PartitioningBenefits
Parallel accesses to partitionsRead smaller sets of dataFast Deletion of partitions (DROP PARTITION instead DELETE FROM WHERE)
Automatically Partitioned Database Tables (for Range Partitioning)InfoCube F-Fact table: partitioned by requestPSA table: partitioned by requestODS Change Log: similar to PSA table
User Defined Partitioning Criteria (for Range Partitioning)InfoCube E-Fact tablePartition Criteria: Time characteristics like month or fiscal periodNote: Both fact tables are extended by the SID of the chosen time characteristic
SAP TechEd ‘03 Basel
© 2003 SAP AG BW 354, Uwe Heinz; Alexander Peter 6
SAP AG 2003, TechED EMEA, BW354, Alexander Peter, Uwe Heinz
Performance Guidelines – General Aspects – 3 –
Database (or Physical) PartitioningQuery Access (Partition Pruning)
Jan - MarApr - JunJul - SepOct - Dec
Package 2Package 5Package 7
month
Master Data
Time
Package
Dimension
Fact Table
„F“
„E“
Example:
Month = August 2003
Apply Restrictions to Dimensions Apply Restriction
to fact table
Month Customer SalesAug 03 4711 15Aug 03 4712 66
Discard irrelevant partitions
F table has to be read, too
SAP AG 2003, TechED EMEA, BW354, Alexander Peter, Uwe Heinz
Agenda
Modeling
Query Performance
General Settings and Checklist
Extraction / Data Load Performance
Analysis Methods
SAP TechEd ‘03 Basel
© 2003 SAP AG BW 354, Uwe Heinz; Alexander Peter 7
SAP AG 2003, TechED EMEA, BW354, Alexander Peter, Uwe Heinz
Checklist
Design of Operational Store, Data Warehouse Layer and Multidimensional Model
Dimensions of InfoCubes
Logical (MultiProvider) Partitioning
Non-Cumulative Key Figures
SAP AG 2003, TechED EMEA, BW354, Alexander Peter, Uwe Heinz
Operational Data Store and Data Warehouse Layer
Data WarehouseNon volatileGranularIntegratedHistorical foundationBuilt with ODS Objects
Operational Data StoreOperational Reporting Near Real-Time / VolatileGranularBuilt with ODS Objects
Multidimensional ModelsMultidimensional analysisAggregated viewIntegratedBuilt with InfoCubes
ODS Object ODS Object
SAP TechEd ‘03 Basel
© 2003 SAP AG BW 354, Uwe Heinz; Alexander Peter 8
SAP AG 2003, TechED EMEA, BW354, Alexander Peter, Uwe Heinz
InfoCubes SAP-Extended Star Schema
ONE Normalization of DimensionsMaster Data is shared, not part of the InfoCube
Fact TableFact Table
DimensionDimension
DimensionDimensionDimensionDimension
DimensionDimension
Dimension ID
MasterData
MasterData
MasterData
MasterData
MasterData
MasterData
MasterData
MasterData
Surrogate Key
SAP AG 2003, TechED EMEA, BW354, Alexander Peter, Uwe Heinz
InfoCubes Master Data
HierarchiesAttributesTexts (Multi-Language Support)
Fact TableFact Table
DimensionDimension
MasterData
MasterDataSID TableSID Table
Attributes
Hierarchies
TextsPlant UKPlant GermanyPlant Australia
Customer Street Account
SAP TechEd ‘03 Basel
© 2003 SAP AG BW 354, Uwe Heinz; Alexander Peter 9
SAP AG 2003, TechED EMEA, BW354, Alexander Peter, Uwe Heinz
Data Modeling – Line Item Dimensions – 1 –
Line Item DimensionsDirect Link from InfoCube Fact Table to Master Data SID Table – without dimension tableOnly one characteristic is possible in a line item dimensionRecommended for large dimensions (e.g. invoice number, order number, but also reasonable for large material or customer dimensions)
SID Dimension Fact Table Dimension SID SID Fact Table Dimension SID
Without lineitemdimension
With lineitemdimension
SAP AG 2003, TechED EMEA, BW354, Alexander Peter, Uwe Heinz
Data Modeling – Line Item Dimensions – 2 –
Line Item DimensionsBenefits
Saves one table join at query runtimeSaves determination of dimension ID’s at data load time
DisadvantagesF4 – Help not possible on dimension values (only on master data values)
SAP TechEd ‘03 Basel
© 2003 SAP AG BW 354, Uwe Heinz; Alexander Peter 10
SAP AG 2003, TechED EMEA, BW354, Alexander Peter, Uwe Heinz
Data Modeling – MultiProviders – 1 –
MultiProvider (or Logical) PartitioningPossible partitioning criteria: year, plan/actual, regions, business areaParallel sub-queries are started automatically to basic InfoCubesUse MultiProvider Partitioning to cut large amounts of data in chunks
MultiProvider
Basic InfoCubes Europe Asia USA
Consolidated view on all data
SAP AG 2003, TechED EMEA, BW354, Alexander Peter, Uwe Heinz
Data Modeling – MultiProviders – 2 –
MultiProviders (SAP BW 3.x)Definition
Combination of InfoCubes, ODS objects, InfoObjects and InfoSets
With
Dat
a
Master Data
Basic InfoCube
ODS Object
No
Dat
a
InfoSet
Virtual InfoCubeMulti
ProviderMulti
Provider
SAP TechEd ‘03 Basel
© 2003 SAP AG BW 354, Uwe Heinz; Alexander Peter 11
SAP AG 2003, TechED EMEA, BW354, Alexander Peter, Uwe Heinz
Data Modeling – MultiProviders – 3 –
MultiProvidersBenefits
No additional data storageSingle InfoProviders smaller, less complex and less sparsely filled than one big InfoProvider(Parallel) Data load into individual InfoProvidersQueries are split automatically and distributed to InfoProviders (parallel execution)Transparent usage for ReportingLocal queries on each InfoProvider possibleArchiving of single basic InfoProvider is very easy
DisadvantagesAdminstration (with aggregates)Additional I/O
SAP AG 2003, TechED EMEA, BW354, Alexander Peter, Uwe Heinz
Data Modeling – Non-Cumulative KFs – 1 –
Non-cumulative key figuresNon-cumulative key figures cannot be cumulated meaningfully over time, e.g. inventory, number of employeesStorage: Historical movements and reference point (in E fact table)Reference point is updated when InfoCube is compressed; if all requests are compressed, the reference point represents e.g. the current stockExample:
Month Material Plant Material flow Reference pointJan 02 4711 1000 10 0
10Feb 02 4711 1000 20 10Mar 02 4711 1000 5 10
35
Compressing
Compressing
Note: the reference points are only stored in the compressed E-table; they have the time value “infinity”, e.g. for day 31.12.9999
SAP TechEd ‘03 Basel
© 2003 SAP AG BW 354, Uwe Heinz; Alexander Peter 12
SAP AG 2003, TechED EMEA, BW354, Alexander Peter, Uwe Heinz
Data Modeling – Non-Cumulative KFs – 2 –
Performance Tips & Tricks:Compress InfoCubes with non-cumulative key figures as soon as possibleVery granular characteristic value combinations generate lots ofreference points in the E table
Aggregate rebuild is significantly influencedDelete unused values (e.g. old seasonal material) from the InfoCube by selective deletion without timely restriction)
LAST- and FIRST- key figures can use aggregates; AVG-key figures cannot use them. Try to split these key figures in separate queries, if semantically possibleAt query time use tight restrictions on time characteristics; ideally request only current valuesSuppress sum lines if not neededDo not use partial time characteristics (e.g. FISCPER3) if not needed
SAP AG 2003, TechED EMEA, BW354, Alexander Peter, Uwe Heinz
Data Modeling – Non-Cumulative KFs – 3 –
Validity ObjectsValidity Objects are characteristics for which a validity period can be definedThe validity is stored within the validity tableExample: If plant 1 is closed at the end of March, it should not show up any value (even no zero) in April and May
Every entry in the validity table generates an individual SELECT at query runtime.Use as few validity objects as possibleDo not misuse validity objects for termination (e.g. insurance contract)
Jan Feb Mar Apr May Reference pointPlant 1 +10 +20 +30 100Plant 2 +20 +10 +10 +20 +30 150
SAP TechEd ‘03 Basel
© 2003 SAP AG BW 354, Uwe Heinz; Alexander Peter 13
SAP AG 2003, TechED EMEA, BW354, Alexander Peter, Uwe Heinz
Checklist – Query Design
Multi-dimensional Query Design
Inclusion / Exclusion
MultiProvider Query
Cell Calculation
Customer Exits
Query Read Mode
SAP AG 2003, TechED EMEA, BW354, Alexander Peter, Uwe Heinz
Agenda
Modeling
Query Performance
General Settings and Checklist
Extraction / Data Load Performance
Analysis Methods
SAP TechEd ‘03 Basel
© 2003 SAP AG BW 354, Uwe Heinz; Alexander Peter 14
SAP AG 2003, TechED EMEA, BW354, Alexander Peter, Uwe Heinz
Checklist – Query and Web Performance –Overview
ST03 / Technical ContentST03 / Technical Content
DatabaseDatabase OLAPOLAP FrontendFrontend
1. Data Model
2. Query Definition
3. Aggregates
4. OLAP Cache
5. Pre-Calculated Web Templates
6. Compressing
7. Indices
8. DB Statistics
9. DB and basis (Buffer) Parameter
1. Data Model
2. Query Definition
3. Aggregates
4. OLAP Cache
5. Pre-Calculated Web Templates
6. Compressing
7. Indices
8. DB Statistics
9. DB and basis (Buffer) Parameter
1. Data Model
2. Query Definition (including OLAP features)
3. Aggregates
4. OLAP Cache
5. Virtual Key Figures / Characteristics
6. Authorizations
1. Data Model
2. Query Definition (including OLAP features)
3. Aggregates
4. OLAP Cache
5. Virtual Key Figures / Characteristics
6. Authorizations
1. Query Definition
2. Network
3. WAN and BEx
4. Client Hardware
5. VBA / Java
6. Documents
7. Formatting
8. ODBO / 3rd party
1. Query Definition
2. Network
3. WAN and BEx
4. Client Hardware
5. VBA / Java
6. Documents
7. Formatting
8. ODBO / 3rd party
SQL Trace (ST05)
RSRV
RSRT, RSRTRACE
SQL Trace (ST05)
RSRV
RSRT, RSRTRACE
RSRT, RSRTRACE
SQL Trace (ST05)
ABAP Trace (SE30)
RSRT, RSRTRACE
SQL Trace (ST05)
ABAP Trace (SE30)IEMON
RSRT, RSRTRACE
IEMON
RSRT, RSRTRACE
Which component contributes most?
Tools
Check these points
SAP AG 2003, TechED EMEA, BW354, Alexander Peter, Uwe Heinz
Performance Layers
ReuseReuse
PerformancePerformance
Offline Analysis
Pre-Calculation
OLAP Cache
Aggregates
InfoCubes
Performance LayersFor specific Scenarios
SAP TechEd ‘03 Basel
© 2003 SAP AG BW 354, Uwe Heinz; Alexander Peter 15
SAP AG 2003, TechED EMEA, BW354, Alexander Peter, Uwe Heinz
Aggregates – 1 –
Aggregate DefinitionMaterialization of aggregated subsets of InfoCube fact table dataIndependent structures where summary data is stored within separate, transparent InfoCubesTransparency: Users do not notice if aggregate is hit or notImproved query performance by reducing the amount of data to be read from DB
Aggregates can be createdOnly on top of basic InfoCubesFor dimension characteristicsFor navigational attributesOn hierarchy levelsUsing time-dependent navigational attributes (as of BW 3.x)Using hierarchy levels where the structure is time-dependent (as of BW 3.x)
Note: Aggregates cannot be built on MultiProviders, SAP Remote Cubes, Remote Cubes or ODS ObjectsAggregates can improve query performance considerably, but keep in mind that they also impact the load performance. For more information, see chapter on data load performance.
SAP AG 2003, TechED EMEA, BW354, Alexander Peter, Uwe Heinz
Aggregates – 2 – Example
Example for flow with and without aggregates
Flow without aggregate
InfoCube Number of records read on the database Records
transferred to BW instance after
being summarized on the database
Month Material RevenueJuly Hammer 10July Nail 20August Hammer 10August Nail 20
Aggregate
Month RevenueJuly 30August 30
Month Material RevenueJuly Hammer 10July Nail 20August Hammer 10August Nail 20
Month RevenueJuly 30August 30
Month RevenueJuly 30August 30
Flow with aggregate
Database Selection OLAP Engine
SAP TechEd ‘03 Basel
© 2003 SAP AG BW 354, Uwe Heinz; Alexander Peter 16
SAP AG 2003, TechED EMEA, BW354, Alexander Peter, Uwe Heinz
Aggregates – 3 – Tips for Creating Aggregates
Do the following stepsCall query or InfoCube overview in technical content or ST03Sort by mean overall time to find queries/InfoCubes with highest runtimesCalculate the KPI ‘aggregation ratio’ = number of records read from DB / number of records transferred Check quota of database time to total runtimeAs a rule of thumb, an aggregate is reasonable and may be created, if:
Aggregation ratio > 10Aggregation ratio > 10, I.e. 10 times more records are read than are
displayed, ANDANDPercentage of DB time > 30%Percentage of DB time > 30%, I.e. the time spent on database is a substantial part of the whole query runtime
SAP AG 2003, TechED EMEA, BW354, Alexander Peter, Uwe Heinz
OLAP Cache – 1 –
Caching in SAP BW 3.x OLAP CacheGlobal cache which is accessible from all sessionsOLAP Cache is part of application buffer (Imp/Exp SHM) and can also be stored in a DB table or a fileThe Cache stores query results and navigation statuses as highly compressed cluster dataQuery Cache is used for equal queries or subsets of cached queriesBenefits: OLAP Cache reduces workload on database and application serverInvalidations:
Data Load into underlying DataTargetQuery Generation
SAP TechEd ‘03 Basel
© 2003 SAP AG BW 354, Uwe Heinz; Alexander Peter 17
SAP AG 2003, TechED EMEA, BW354, Alexander Peter, Uwe Heinz
Pre-Calculated Web Templates – 1 –
Pre-Calculated Web TemplatesPre-calculation is a set of techniques where you can distribute the workload of running the report to off-peak hours, and have the report result set ready for very fast access to the dataData Pre-Calculation or HTML Pre-Calculation
HTML Pre-Calculation can be used for Offline AnalysisBenefits
Fast response timeSystem workload shifted to off-peak hoursRe-use data that goes to many queriesReports also available offline
SAP AG 2003, TechED EMEA, BW354, Alexander Peter, Uwe Heinz
Pre-Calculated Web Templates – 2 –
ScenarioSupports “Newspaper” scenario
Quick access to many reportsReports are requested by many usersStatic reporting, usually little navigationrequirementsActive Reporting Authorisations
RestrictionsNo invalidation when new data is loadedRestricted navigation: only filtering possibles
SAP TechEd ‘03 Basel
© 2003 SAP AG BW 354, Uwe Heinz; Alexander Peter 18
SAP AG 2003, TechED EMEA, BW354, Alexander Peter, Uwe Heinz
Reporting Performance – Indices
IndicesIf you report on ODS objects with a restrictive selection key, check if indices are defined for this keyCheck if the InfoCube Indices exist
In ORACLE, you can choose between bitmap index and B-tree indexUse B-tree index, if dimension size exceeds 10% of the fact table size
If you select on navigational attributes, be sure that an appropriate index is available
SAP AG 2003, TechED EMEA, BW354, Alexander Peter, Uwe Heinz
Reporting Performance – ODS Objects
Indexing ODS ObjectsIndices can be defined in the
3.x ODS object maintenance; in BW 2.x indices must be defined via SE11
Use indices on characteristics which you are accessing regularly (for reporting or DataMart interface)
Doc.No. Customer Sales4711 Miller 104712 Smith 204713 Miller 254714 Cassano 354715 Birch 124716 Smith 44
ODS Object
Index
Doc.No. Customer Sales4711 Miller 10 Miller4712 Smith 20 Smith4713 Miller 25 Cassano4714 Cassano 35 Birch4715 Birch 124716 Smith 44
ODS ObjectExample: If you look for sales figures with customer Birch, a full table scan is necessary when there is no index (see picture above); if the index has been created, only two DB accesses are necessary (see picture on the left).
Full
table
scan
SAP TechEd ‘03 Basel
© 2003 SAP AG BW 354, Uwe Heinz; Alexander Peter 19
SAP AG 2003, TechED EMEA, BW354, Alexander Peter, Uwe Heinz
Reporting Performance – Indices
Index types (selection)Bitmap Index
Well-suited for few characteristic valuesSeveral bitmap indexes can be used for one table (bitmap merge join)
B-Tree IndexGood for range queries and for high cardinality
Miller
Leask Osbourne Smith
BrownLeask Miller Moloney
OsbournePageSmith Trapp
rowidMiller 00110000111Smith 10000011100Trapp 01001100000
SAP AG 2003, TechED EMEA, BW354, Alexander Peter, Uwe Heinz
Indexing in SAP BW
Fact TableNon-Unique Index on every dimension ID
ORACLE: Bitmap (except for “high cardinality” dimensions)ORACLE: B-tree on F-fact table for transactional InfoCubes(guarantees parallel read&write)ORACLE: additional bitmap index on time characteristic SID for partitioned InfoCubesDB2/AS400: EVIOther DBMSs: B-TreeDB2/OS390: B-Tree not on package dimension ID
Non-unique B-Tree on all dimension IDs (Primary Key)Used for compressionORACLE/Informix: only on E-fact table
SAP TechEd ‘03 Basel
© 2003 SAP AG BW 354, Uwe Heinz; Alexander Peter 20
SAP AG 2003, TechED EMEA, BW354, Alexander Peter, Uwe Heinz
Indexing in SAP BW
DimensionsDimension ID (primary key)
Unique B-TreeMaster Data SIDs
Non-Unique B-TreeORACLE / DB2/AS400: not on first SID (this is included in the index on all SIDs)
On all master data SIDsNon-Unique B-Tree
Master Data X- and Y- tables (SIDs for all navigational attributes)
Unique B-Tree on <SID, Objectversion> (primary key)Additional indices could be defined for individual SID-columns to avoid full table scans (for selective accesses)
S-table (SID)Unique B-Tree on master data key (primary key)Unique B-Tree on SID
SAP AG 2003, TechED EMEA, BW354, Alexander Peter, Uwe Heinz
Agenda
Modeling
Query Performance
General Settings and Checklist
Extraction / Data Load Performance
Analysis Methods
SAP TechEd ‘03 Basel
© 2003 SAP AG BW 354, Uwe Heinz; Alexander Peter 21
SAP AG 2003, TechED EMEA, BW354, Alexander Peter, Uwe Heinz
Checklist – Data Load Performance – Overview 1
Technical Content, Data Load MonitorTechnical Content, Data Load Monitor
ExtractionExtraction TransferTransfer Load Into PSALoad Into PSA
1. Customer Exits
2. Resource Utilization
3. Load Balancing
4. Data Package Size
5. Indices on tables
6. Flat File format
7. Content vs. generic extractor
1. Customer Exits
2. Resource Utilization
3. Load Balancing
4. Data Package Size
5. Indices on tables
6. Flat File format
7. Content vs. generic extractor
1. Resource Contraint
2. CPU / Memory Bottleneck
3. Network
4. Application Buffer Synchronization
1. Resource Contraint
2. CPU / Memory Bottleneck
3. Network
4. Application Buffer Synchronization
1. I/O Contention1. I/O Contention
Extractor Checker (RSA3),
ABAP Trace (SE30),
SQL Trace (ST05)
Extractor Checker (RSA3),
ABAP Trace (SE30),
SQL Trace (ST05)
SM50
SQL Trace (ST05)
OS Monitor (ST06)
SM50
SQL Trace (ST05)
OS Monitor (ST06)OS Monitor (ST06)
DB Monitor (ST04)
OS Monitor (ST06)
DB Monitor (ST04)
Which component contributes most?
Tools
Check these points
SAP AG 2003, TechED EMEA, BW354, Alexander Peter, Uwe Heinz
Checklist – Data Load Performance – Overview 2
1. Transformation Rules / ABAP Coding
2. Transformation Library Formulas
1. Transformation Rules / ABAP Coding
2. Transformation Library Formulas
1. Roll-up
2. Change Run
3. Compression
4. Indices
5. Load Master Data before Transaction Data
6. Buffering Number Ranges
1. Roll-up
2. Change Run
3. Compression
4. Indices
5. Load Master Data before Transaction Data
6. Buffering Number Ranges
Debugger within Monitor
ABAP Trace (SE30),
SQL Trace (ST05)
Debugger within Monitor
ABAP Trace (SE30),
SQL Trace (ST05) SQL Trace (ST05)SQL Trace (ST05)
Which component contributes most?
Tools
Check these points
Transfer RulesTransfer Rules
Update RulesUpdate RulesLoad Into Data TargetsLoad Into Data Targets
InfoCubesInfoCubes ODS ObjectsODS Objects
1. Parallel ODS activation
2. Unique Data Records
3. Flag BExReporting
4. Indices
1. Parallel ODS activation
2. Unique Data Records
3. Flag BExReporting
4. Indices
Technical Content, Data Load MonitorTechnical Content, Data Load Monitor
Master DataMaster Data
1. Buffering Number Ranges
2. Change Run
1. Buffering Number Ranges
2. Change Run
SAP TechEd ‘03 Basel
© 2003 SAP AG BW 354, Uwe Heinz; Alexander Peter 22
SAP AG 2003, TechED EMEA, BW354, Alexander Peter, Uwe Heinz
Loading Master Data – 1 –
Number RangesSID number range can be buffered instead of accessing the DB for each SIDIf you encounter massive accesses to DB table NRIV via SQL trace (ST05), increase the number range buffer in transaction SNROIf possible, reset the value to its original state after the load (to avoid unnecessary memory allocation)
SID Material Number1 47112 08153 4712
SID Buffer45678
New material: 1125Draw new SID from Buffer
SID
SAP AG 2003, TechED EMEA, BW354, Alexander Peter, Uwe Heinz
Roll-Up Performance – 1 –
Roll-upThe roll-up process populates all aggregates of an InfoCubewith the newly loaded delta loadBasic Rule: InfoCube and all depending aggregates must be in-sync, i.e. you must see the same data no matter if it has been derived from the InfoCube or the aggregateNewly loaded data is not available for reporting until it has been rolled up into the aggregates
Aggregate HierarchyAggregates can be built out of other aggregates to reduce the amount of data to be read and, hence, to improve the roll-up performanceAggregate hierarchy is determined automaticallyGeneral Guideline: define few basis (large) aggregates and many small aggregates that can be built from the hierarchy level before
Display aggregate hierarchy
SAP TechEd ‘03 Basel
© 2003 SAP AG BW 354, Uwe Heinz; Alexander Peter 23
SAP AG 2003, TechED EMEA, BW354, Alexander Peter, Uwe Heinz
Roll-Up Performance – 1 – Aggregate Hierarchy
Example: Optimized Aggregate Hierarchy
Basic InfoCube
{Material, Material Group, Customer, Day}
Example
Few large basicaggregates
{Material Group, Customer, Day}
Many smallaggregates
{Material Group, Month}
SAP AG 2003, TechED EMEA, BW354, Alexander Peter, Uwe Heinz
Change Run Performance – 1 –
Change RunAggregates can contain
Dimension CharacteristicsNavigational AttributesHierarchy Levels
When master data changes, the changes of the navigational attributes/hierarchies must be applied to the depending aggregates; this process is called change runNewly loaded master data is not active until the change run has been applied the changes to all aggregatesThreshold for delta and new build-up in customizingThe Change Run can be parallelized across InfoCubes; see SAP note 534630 for more detailsCheck aggregate hierarchy (see Roll-Up for more details)Try to build basis aggregates that are not affected by the change run, i.e. no navigational attributes nor hierarchy levelsThe following slides show details on the process itself …
SAP TechEd ‘03 Basel
© 2003 SAP AG BW 354, Uwe Heinz; Alexander Peter 24
SAP AG 2003, TechED EMEA, BW354, Alexander Peter, Uwe Heinz
Change Run – 2 – Before Master Data Activation
Country Customer Sales
USAGermanyUSAAustriaAustriaGermanyUSA
Winsoft Inc.InternetworksFunny Duds Inc.InternetworksThor IndustriesFunny Duds Inc.Winsoft Inc.
10155
10102025
Fact Table: Sales DataFact Table: Sales Data Aggregate Tables: Sales DataAggregate Tables: Sales Data
Country SpaceIndustry *
Country SpaceIndustry *
Industry Sales
602510
TechnologyConsumer ProductsChemical
TechnologyConsumer ProductsTechnologyChemicalConsumer Products
IndustryCustomer
Winsoft Inc.Funny Duds Inc.InternetworksThor IndustriesInternetworks
Navigational Attribute for Characteristic Customer
Navigational Attribute for Characteristic Customer
Changed master data not available for reporting
Old:
New:
SAP AG 2003, TechED EMEA, BW354, Alexander Peter, Uwe Heinz
Change Run – 3 – After Master Data Activation
Country Customer Sales
USAGermanyUSAAustriaAustriaGermanyUSA
Winsoft Inc.InternetworksFunny Duds Inc.InternetworksThor IndustriesFunny Duds Inc.Winsoft Inc.
10155
10102025
Fact Table: Sales DataFact Table: Sales Data Aggregate Tables: Sales DataAggregate Tables: Sales Data
Country SpaceIndustry *
Country Country SpaceSpaceIndustry Industry **
Industry Sales
355010
TechnologyConsumer ProductsChemical
TechnologyConsumer ProductsTechnologyChemicalConsumer Products
IndustryCustomer
Winsoft Inc.Funny Duds Inc.InternetworksThor IndustriesInternetworks
Navigational Attribute for Characteristic Customer
Navigational Attribute for Characteristic Customer
Old:
New:
Changed master data now available for reporting
SAP TechEd ‘03 Basel
© 2003 SAP AG BW 354, Uwe Heinz; Alexander Peter 25
SAP AG 2003, TechED EMEA, BW354, Alexander Peter, Uwe Heinz
Agenda
Modeling
Query Performance
General Settings and Checklist
Extraction / Data Load Performance
Analysis Methods
SAP AG 2003, TechED EMEA, BW354, Alexander Peter, Uwe Heinz
Query Monitor (RSRT) – 1 –
Analyze specific queries
Debug Option
SAP TechEd ‘03 Basel
© 2003 SAP AG BW 354, Uwe Heinz; Alexander Peter 26
SAP AG 2003, TechED EMEA, BW354, Alexander Peter, Uwe Heinz
Query Monitor (RSRT) – 2 –
Analyze specific queriesCheck if the right aggregate(s) is(are) used by the query
Number of Database Access
Structureto be read
SuggestedAggregates
AvailableAggregates
* disaggregatedspace aggregated
F fixed valueH hierarchy level aggregation
SAP AG 2003, TechED EMEA, BW354, Alexander Peter, Uwe Heinz
Performance Analysis: SQL Trace (ST05) – 1 –
Analyze high database runtimeTrace specific process with SQL Trace
Activate SQL Trace Flag
Activate Trace
Execute process
Deactivate Trace
Analyze Trace
SAP TechEd ‘03 Basel
© 2003 SAP AG BW 354, Uwe Heinz; Alexander Peter 27
SAP AG 2003, TechED EMEA, BW354, Alexander Peter, Uwe Heinz
Performance Analysis: SQL Trace (ST05) – 2 –
Total Num
berof „statement-identical“ statem
ents
Percentageof „value-identical“ statem
ents
Total ExecutionTim
e in microsec
(Prep, Open,
Fetch, Close)
Total numberof row
s
Table name
orDB
procedure
Explain
SAP AG 2003, TechED EMEA, BW354, Alexander Peter, Uwe Heinz
Performance Analysis: SQL Trace – Explain
1. Find all bitmap streams for package restriction
3. Calculate “AND” of bitmap streams
2. Find all bitmap streams for fiscal periods
4. Access fact table partition by partition through local index
5. Join other requested dimensions to the result via hash join
SAP TechEd ‘03 Basel
© 2003 SAP AG BW 354, Uwe Heinz; Alexander Peter 28
SAP AG 2003, TechED EMEA, BW354, Alexander Peter, Uwe Heinz
SQL Explain
Basic Help to Read the Execution PlanRead from inside out of the hierarchy
Read a set of operations at the same level from top to bottomImportant node names (ORACLE)
Hash JoinSmaller Table is used to build up hash table on the join key (optimal: in memory)Then scanning the larger table with hash table
Nested LoopFor each relevant row in the first table, find all matching rows in the other table
Bitmap Merge (=Or) / Bitmap AndPartition Range Iterator
SAP AG 2003, TechED EMEA, BW354, Alexander Peter, Uwe Heinz
Web Query Analysis – Overview
2 different measuring points:
Web Browser
WAS / BW / DB
HTTPrequest
HTMLresponse
ICMOLAPINIT
DB
OLAP
Rendering
OLAP Cache
Monitoring of WAS / BW / DB internal activities
HTTP M
onitor
Frontend monitoringNetwork traffic monitoring
IEMonitor
„frontend“
„backend“
SAP TechEd ‘03 Basel
© 2003 SAP AG BW 354, Uwe Heinz; Alexander Peter 29
SAP AG 2003, TechED EMEA, BW354, Alexander Peter, Uwe Heinz
Web Query Analysis – Frontend Monitoring
Key Performance Indicators for Frontend Monitoring:Query response time
Time to load HTML pageFrontend rendering time
Transferred data volumeNumber of round-trips between frontend and server
IEMon.exeSimple Web browser replacement with protocol logging facilitiesUsed to analyze load and frontend rendering time of an HTML pageEasy deployment
httpmon.exeSet up as a proxy serverRequires changes of IE settingsRequires environment variables to be set for configuration
SAP AG 2003, TechED EMEA, BW354, Alexander Peter, Uwe Heinz
IEMon.exe – Log File Analysis in Excel
Output produced by
analyze_iemon
Graphical representation
with a few clicks
SAP TechEd ‘03 Basel
© 2003 SAP AG BW 354, Uwe Heinz; Alexander Peter 30
SAP AG 2003, TechED EMEA, BW354, Alexander Peter, Uwe Heinz
httpmon.exe – Setup
Set up as a proxy server
Configuration by environment variables:HTTP_PROXY: proxy_server:proxy_port(e.g. pwdf0754:1080)NO_PROXY: list of web server domainswhich do not need proxy (e.g. *.sap.corp;*.sap-ag.de)
httpmon listens on port 8000 by defaultchange Internet Explorer settings
Web Browser Proxy server
proxy_port
httpmon
port 8000
otherWeb servers
(no prxy required)
SAP AG 2003, TechED EMEA, BW354, Alexander Peter, Uwe Heinz
Httpmon.exe – Use Case
First execution: download of MIME
objects
Second execution: MIME objects were read from
browser cache
SAP TechEd ‘03 Basel
© 2003 SAP AG BW 354, Uwe Heinz; Alexander Peter 31
SAP AG 2003, TechED EMEA, BW354, Alexander Peter, Uwe Heinz
Summary
Now you are able to:Differentiate and Analyse Performance IssuesExplain effects of modeling on performanceSuggest means to improve query and data load performanceTrace Web Queries
SAP AG 2003, TechED EMEA, BW354, Alexander Peter, Uwe Heinz
Further Information
Public Web:www.sap.com Solutions Business IntelligenceSAP Customer Services Network: www.sap.com/services/
SAP BW Online Documentationhttp://help.sap.com
SAP Service Marketplacehttp://service.sap.com/bw Performance
Related SAP Training CoursesBW360: Performance & Administration
Collective Performance NotesNote 567745: DB-specific SettingsNote 567746: Query & Web PerformanceNote 567747: Extraction & Loading
SAP TechEd ‘03 Basel
© 2003 SAP AG BW 354, Uwe Heinz; Alexander Peter 32
SAP AG 2003, TechED EMEA, BW354, Alexander Peter, Uwe Heinz
Q&A
Questions?
SAP AG 2003, TechED EMEA, BW354, Alexander Peter, Uwe Heinz
Please complete your session evaluation anddrop it in the box on your way out.
Feedback
Thank You !
The SAP TechEd ’03 Basel Team
SAP TechEd ‘03 Basel
© 2003 SAP AG BW 354, Uwe Heinz; Alexander Peter 33
SAP AG 2003, TechED EMEA, BW354, Alexander Peter, Uwe Heinz
No part of this publication may be reproduced or transmitted in any form or for any purpose without the express permission of SAP AG. The information contained herein may be changed without prior notice.
Some software products marketed by SAP AG and its distributors contain proprietary software components of other software vendors.
Microsoft®, WINDOWS®, NT®, EXCEL®, Word®, PowerPoint® and SQL Server® are registered trademarks of Microsoft Corporation.
IBM®, DB2®, DB2 Universal Database, OS/2®, Parallel Sysplex®, MVS/ESA, AIX®, S/390®, AS/400®, OS/390®, OS/400®, iSeries, pSeries, xSeries, zSeries, z/OS, AFP, Intelligent Miner, WebSphere®, Netfinity®, Tivoli®, Informix and Informix® Dynamic ServerTM are trademarks of IBM Corporation in USA and/or other countries.
ORACLE® is a registered trademark of ORACLE Corporation.
UNIX®, X/Open®, OSF/1®, and Motif® are registered trademarks of the Open Group.
Citrix®, the Citrix logo, ICA®, Program Neighborhood®, MetaFrame®, WinFrame®, VideoFrame®, MultiWin® and other Citrix product names referenced herein are trademarks of Citrix Systems, Inc.
HTML, DHTML, XML, XHTML are trademarks or registered trademarks of W3C®, World Wide Web Consortium, Massachusetts Institute of Technology.
JAVA® is a registered trademark of Sun Microsystems, Inc.
JAVASCRIPT® is a registered trademark of Sun Microsystems, Inc., used under license for technology invented and implemented by Netscape.
MarketSet and Enterprise Buyer are jointly owned trademarks of SAP AG and Commerce One.
SAP, R/3, mySAP, mySAP.com, xApps, xApp, SAP NetWeaver and other SAP products and services mentioned herein as well as their respective logos are trademarks or registered trademarks of SAP AG in Germany and in several other countries all over the world. All other product and service names mentioned are the trademarks of their respective companies.
Copyright 2003 SAP AG. All Rights Reserved