efficient transaction processing in sap hana
TRANSCRIPT
EFFICIENT TRANSACTION PROCESSING IN SAP HANA Presented by Vijay Soppadandi
21363273
CONTENTS
• What is SAP HANA?
• Goal
• Agenda
• Framework
• Summary & discussion
WHAT IS SAP HANA?
• High-Performance Analytic Appliance
• In-memory computing
GOAL
• The overall goal of the SAP HANA database is to provide a generic but powerful system for different query scenarios, both transactional and analytical, on the same data representation within a highly scalable execution environment
AGENDA
OLTP (Online Transaction Proccessing)
o large number of concurrent users and transactions
o high update load
o very selective point queries
o Row-stores
OLAP (Online Analytical Processing)
o aggregation queries over a huge volume of data
o compute statistical models from the data
oColumn-stores
AGENDA
• Zoo of different systems with different capabilities for different application scenarios
• However, workloads usually contain both
o transactional database needs statistical information to make on-the-fly business decisions
o data-warehouses are required to capture transactions feeds for real-time analytics
FRAMEWORK
• Layered Architecture of the SAP HANA
• Lifecycle Management of Database Records
• Types of Merging and optimization
• Summary & discussion
LAYERED ARCHITECTURE OF THE SAP HANA
Reference 3
FRAMEWORK
• Layered Architecture of the SAP HANA
• Lifecycle Management of Database Records
• Types of Merging and optimization
• Summary & discussion
LIFECYCLE MANAGEMENT OF DATABASE RECORDS
Reference 1
LIFECYCLE MANAGEMENT OF DATABASE RECORDS
• It has three main stages
- L1-delta
-L2-delta
-Main
• SAP HANA theoretically escalate records through different stages of a physical representation
L1-DELTA
• Accepts all incoming data requests
• No data compressions
• Stores them in write optimized manner
- Preserves in logical-row format
-Fast insert & deleat, field update and record projections
• Holds 10,000 to 100,000 rows per single-node
L2-DELTA
• The next stage of the Unified table structure
• Organized in column format
• Stores upto 10 millions of row
• It emplyoys dictionary encoding
- However, for performance reasons, the dictionary is unsorted requiring secondary index structures to optimally support point query access patterns
MAIN STORE
• Represents core data
• Final data format
• Organized in column format
• Highest compression rate
-Positions in a sorted dictionary
-Stored in a bit-packet manner
-Dictionary is always comprised
UNIFIED TABLE ACCESS
• A common abstract interface to access different stores •
• Records are propagated asynchronously
– without interfering with running operations
• Two transformations (or merge steps)
– L1-deta to L2-delta
– L2-delta to main
L1-DETA TO L2-DELTA MERGE
• Rows format is converted into column format
- column-by-column inserted into the L2-delta
Steps for merging:
• Step 1 ( parallel)
- new entries are added (advance) to the dictionary
• Step 2 (parallel)
-using dictionary encoding technique new column values are added
• Step 3
- added values are removed from the L1-delta
L2-DELTA TO MAIN MERGE
• New main structure created from L2-delta
• Resourse intensive
-Is carefully schedulled and highly optimized
• Current L2-delta is closed when its done
• Retries the merge on failure
PERSISTENCE MAPPING
Reference 1
CONT...
• Its fully supports ACID gurentees
• Uses REDO logs instead of UNDO mechanisms and saving point
• It has some complications in making merging process
FRAMWORK
• Layered Architecture of the SAP HANA
• Lifecycle Management of Database Records
• Types of Merging and optimization
• Summary & discussion
MERGE OPTIMIZATION
• Basic idea to provide transperent record propagation from write to read optimization.
• The classic merge
• Re-sorting merge
• Partial merge
THE CLASSIC MERGE
• It has two phases
• In first phase generates the new sorted dictionary
• In secound phase, new main index generates on the new dictionary
THE CLASSIC MERGE
Reference 1
RE-SORTING MERGE
• Provides higher compression potential
• reorganizes the content of the full table to yield a data layout which provides higher compression potential
• not easy because all records should have the same order in all columns
• uses a scheme discussed in another paper
PARTIAL MERGE
• It aims to reduce overall merge overhead
• Split the main into two independent main structures
- passive : Not part of the merge process
-Active : Takes part in the merge process with L2-delta
PARTIAL MERGE
Reference 1
CHARACTERISTICS OF RECORD STAGES
Reference 1
REFERENCES
1.Efficient Transaction Processing in SAP HANA Database – The End of a Column Store Myth
-Vishal Sikka SAP 3412 Hillview Ave Palo Alto, CA 94304, USA [email protected]
-Franz Färber SAP Dietmar-Hopp-Allee 16 69190, Walldorf, Germany [email protected]
-Wolfgang Lehner SAP Dietmar-Hopp-Allee 16 69190, Walldorf, Germany [email protected]
-Sang Kyun Cha SAP 63-7 Banpo 4-dong, Seochoku 137-804, Seoul, Korea [email protected]
-Thomas Peh SAP Dietmar-Hopp-Allee 16 69190, Walldorf, Germany [email protected]
-Christof Bornhövd SAP 3412 Hillview Ave Palo Alto, CA 94304, USA [email protected]
2.Efficient Transaction Processing in SAP HANA Database
Presented by Henggang Cui 15-799b Talk
3. The SAP HANA Database–An Architecture Overview
-Franz F¨arber Norman
END
ANY QUESTIONS ?