sap data warehouse cloud a scalable, open, and analytic
Post on 07-Dec-2021
4 Views
Preview:
TRANSCRIPT
WELCOME
Roger Loe – Data Specialist2021
SAP HANA Memory Management StrategiesHigh level view of capabilities
The information in this presentation is confidential and proprietary to SAP and may not be disclosed without the permission of SAP. Except for your obligation to protect confidential information, this presentation is not subject to your license agreement or any other service or subscription agreement with SAP. SAP has no obligation to pursue any course of business outlined in this presentation or any related document, or to develop or release any functionality mentioned therein.
This presentation, or any related document and SAP's strategy and possible future developments, products and or platforms directions and functionality are all subject to change and may be changed by SAP at any time for any reason without notice. The information in this presentation is not a commitment, promise or legal obligation to deliver any material, code or functionality. This presentation is provided without a warranty of any kind, either express or implied, including but not limited to, the implied warranties of merchantability, fitness for a particular purpose, or non-infringement. This presentation is for informational purposes and may not be incorporated into a contract. SAP assumes no responsibility for errors or omissions in this presentation, except if such damages were caused by SAP’s intentional or gross negligence.
All forward-looking statements are subject to various risks and uncertainties that could cause actual results to differ materially from expectations. Readers are cautioned not to place undue reliance on these forward-looking statements, which speak only as of their dates, and they should not be relied upon in making purchasing decisions.
Disclaimer
© 2020 SAP SE or an SAP affiliate company. All rights reserved.
HANA Execution
Problem Statement
HANA Data
SAP HANAIn-Memory
How to manage execution load on a SAP HANA production instance?
How to manage data growth for SAP HANA systems?
Costs
Experience
© 2020 SAP SE or an SAP affiliate company. All rights reserved.
SAP HANA : Memory UsageMemory is a critical SAP HANA resource – How much do I actually have?
Application/s
SAP HANA
• Hardware
• Row Store
• Operating System
• SAP HANA Binaries
• MDC
• Column Store
• Work Area Store
50%
50%
HANA RAM?
• System Objects
HANA Data
HANA Execution
HANA Data
50 GB*
© 2020 SAP SE or an SAP affiliate company. All rights reserved.
Topics
• Operational Best Practices (Execution)
• SAP HANA Data Management (Data)
• Archiving
• Data Temperature Management
• Licence / Hardware Cost Considerations – Data Warehousing
© 2020 SAP SE or an SAP affiliate company. All rights reserved.
SAP HANA : Operational Best Practices (Execution)Memory is a critical SAP HANA resource
Good SQL / Coding• HANA is high performance in-memory columnar database – but
there is still no excuse for bad SQL• E.g. “No select * statements” in custom ABAP or native SQL!
• Ensure performant SQL before it hits production • Avoid untested “free hand” SQL directly in production! • SAP HANA Troubleshooting and Performance Analysis Guide &
OpenSAP course - https://open.sap.com/courses/hanasql1
Good Monitoring • Understanding high load times…and mitigating strategies• Identifying / rectifying problematic SQL• Consider Capture / Replay functionality?
Good Housekeeping• Clear out unnecessary data…
• SAP Note: 2388483 - How-To: Data Management for Technical Tables https://launchpad.support.sap.com/#/notes/2388483
• Consider using URP (Unused Retention Period)• But - Usage can be arbitrary & is column based • Should not be relied on for a space management
strategy• SolMan - DVM -https://support.sap.com/en/release-upgrade-
maintenance/value-support/quick-values-data-volume-management.html
Good Education • SAP Note 1999997 - FAQ: SAP HANA Memory
• https://launchpad.support.sap.com/#/notes/0001999997• 9. Which options exist to reduce the risk of SAP HANA
memory issues? - 35 areas…
Workload Management • Low level granularity –or- MDC (Multitenant Database Container)
based • Use of Active / Active (Read-only)(Licence option)
• 2732012 - Using Active/Active read enabled feature of SAP HANA in SAP S/4HANA https://launchpad.support.sap.com/#/notes/2732012
© 2020 SAP SE or an SAP affiliate company. All rights reserved.
Topics
• SAP HANA Data Management (Data)
© 2020 SAP SE or an SAP affiliate company. All rights reserved.
Multi-Temperature Data Management (Data)Classify data according to its “temperature” – Multi Tier Storage
Time
Data Value
Hot DataFrequent access, high-value, high query performance
Warm DataLess frequent access, less-value, reasonable query performance
Cold DataRarely accessed, low-value, low query performance
Data Value declines over
time
© 2020 SAP SE or an SAP affiliate company. All rights reserved.
Near-Line Storage, SAP IQ
HDFS, K8s, Cloud Storage
ExternalStore
Cold Data
Extension Node
Native Storage Extension
Extended Store
In-Memory Hot Data
Warm Data
What Data Management Should I Use ?
HANADatabase
Suite on HANAS/4HANA
Data Aging(via NSE)
ILM Store w/ IQ
Optane
ILM/ Archiving
Native HANA
DWF/DLM with Spark Controller
Extension Node
Dynamic Tiering
Optane
NSE
BW on HANABW/4HANA
BW NLS,BW/4 DTO w/ IQ
BW NLS,BW/4 DTO
Optane
Extension Node
Data Aging(via NSE)
Time
Data Value
Data Value declines over time
© 2020 SAP SE or an SAP affiliate company. All rights reserved.
Archiving
Key concepts of Data Management
Data Tiering
Data Management
• Needs to be formally archived (regulatory / audit) / unarchived
• Not going to be modified (in most cases)
• No longer needed but would like to keep just in case
• Side-effect: Performance
• Reducing operating costs• Optimizing performance • Maintain SLA’s
© 2020 SAP SE or an SAP affiliate company. All rights reserved.
Topics
• Archiving
© 2020 SAP SE or an SAP affiliate company. All rights reserved.
SAP HANA : Archiving Keep the core clean of old data
• Archiving with SAP software• “ERP” - Standard Archiving (SARA)
• “ERP” - SAP ILM• Licence cost – free SAP IQ • Adaptors
• File system, SAP IQ, HANA or Hadoop• https://help.sap.com/doc/c7ec00060b1946ada9e6898100250c77/7.0/en-
US/HadoopConnectorConfigurationGuideSP13.pdf
• Interfaces• Azure BLOB as DMLT package
• “ERP” – OpenText Archiving • On-Premise or Cloud • Uses ILM concept• Licence cost implication
• BW.x - NLS (Near Line Storage)• Can use SAP IQ or Hadoop• No licence cost • Some very generous hardware limits on SAP IQ
• HANA Full use – N/A• DLM? Not really archiving
Data Archiving,Retention
Management, Decommissioning
Open Text Archive
(On-Premise)
ILM
CAS / NAS
Cloud
Open Text Archive(SaaS)
Storage
Cloud*
S/4 HANA / ERP
Data Archiving
NLS
SAP IQ
SAP BW/4
Hadoop
Data Archiving,Retention
Management, Decommissioning
ILM
Storage
File System
SAP IQ
Hadoop
SAP HANA
Azure BLOB
Archive Link
Data Archiving
Storage
File System
Archive Link
© 2020 SAP SE or an SAP affiliate company. All rights reserved.
Topics
• Data Tiering
© 2020 SAP SE or an SAP affiliate company. All rights reserved.
Performance
µs
ms
sec
min
Volume
< 100 TB
> 100 TB
PB
EB
Price
baseline
~ 5 x cheaper
~ 25 x cheaper
~ 50 x cheaper
SAP HANA : Data Temperature Management The right data in the right place
HotFrequently changed data(In Memory) - HANA
DRAM/PMEMOptane
WarmLess-frequent changed data(Disk…SSD (flash) - HANA
Native Storage Extensions
Extension Nodes
Dynamic Tiering
FrozenRead-only data(non-SAP)
Hadoop / HDFS
Raw Storage / S3 / Swift
CoolRarely changed data(External to HANA)
SAP IQ
SAP DATA Lake
3rd Party Data Lake
© 2020 SAP SE or an SAP affiliate company. All rights reserved.
• Highest value data
• Highest performance level
• Highest cost
• Must handle data modifications
• Managed by SAP HANA
• Intel Optane Technology - PMEM• Benefits
• Fast start-up – NB with larger SAP HANA systems (+-12.5x)
• More RAM in a single chassis ( >4 TB per CPU)
• Lower cost of RAM?
• Requires
• Certain chip level - 7th / 8th Gen Intel Core processor
• SAP HANA ver. 2.00.035 and higher
• DRAM:PMEM Ratios
• Used for SAP HANA column store only
• No performance impact*
• 2700084 - FAQ: SAP HANA Persistent Memory
• https://launchpad.support.sap.com/#/notes/2700084
SAP HANA : Data Temperature Management Hot Area: Memory
Typical “server”
3 TB
7.5 TB
Intel
Intel
© 2020 SAP SE or an SAP affiliate company. All rights reserved.
• Medium value data, performance level *, cost
• Should handle data modifications
• Managed by SAP HANA
• NSE (Native Storage Extensions) (old Paged Attributes)• SAP S/4 HANA, HANA Full Use, SAP BW/4 • Version specific: SAP HANA 2 SP04• 1:4 data ratio to SAP HANA data (max 10TB*)• Buffer RAM needed (per MDC) (Warm Data / 8)• No cost• Table, Partition and Column usage • Same software stack / instance / backup as SAP HANA• Data type / function compatibility with SAP HANA • Manual configuration • Good performance seen• Understand limitations – its NOT archiving!• No HANA native DLM (Data Lifecycle Manager) integration • 2799997 - FAQ: SAP HANA Native Storage Extension (NSE)
• https://launchpad.support.sap.com/#/notes/2799997• 2869647 - Guidance for use of Data Aging in SAP S/4HANA
• https://launchpad.support.sap.com/#/notes/2869647• 2973243 - Guidance for use of HANA Native Storage Extension in SAP S/4HANA and SAP
Business Suite powered by SAP HANA• https://launchpad.support.sap.com/#/notes/2973243
SAP HANA : Data Temperature Management Warm Area
© 2020 SAP SE or an SAP affiliate company. All rights reserved.
• Medium value data
• Medium performance level *
• Medium cost
• Should handle data modifications
• Managed by SAP HANA
• SAP Extension Nodes• “Generally” only for SAP BW.x HANA usage • Requires “HANA hardware” – becomes TDI system
• SAP HANA Scale-out configuration• Only pay for Hot SAP HANA RAM used• 200% over deployment (4 x data)
• Normal: 1 TB Node = 512 GB Data (50/50)• Extension Node: 1 TB = 2 TB Data
• Same software stack / backup as SAP HANA• SAP HANA Data type / function compatibility• Active / Active & HA & MDC supported• Flexible configuration – no max size *• SAP HANA Extension Nodes - FAQ• https://www.sap.com/documents/2018/05/9878c71f-037d-
0010-87a3-c30de2ffd8ff.html
SAP HANA : Data Temperature Management Warm Area
Slave Node
Skylake
2 TB DRAM
Extension
Node
Skylake
2TB DRAM
Master
Node
Skylake
2TB DRAM
SAP HANA scale-out
Symmetric
Slave Node
Skylake
2TB DRAM
Standby
Node
(Opt)
Slave Node
Skylake
2 TB DRAM
Extension
Node
Broadwell
2TB DRAM
Master
Node
Skylake
2TB DRAM
SAP HANA scale-out
Slave Node
Skylake
2TB DRAM
Standby
Node
(Opt)
Asymmetric CPU
Slave Node
Skylake
2 TB DRAM
Extension
Node
Broadwell
4TB DRAM
Master
Node
Skylake
2TB DRAM
SAP HANA scale-out
Slave Node
Skylake
2TB DRAM
Standby
Node
(Opt)
Asymmetric CPU + Memory-Size
Same CPUSame RAM
Diff CPUSame RAM
Diff CPUDiff RAM
© 2020 SAP SE or an SAP affiliate company. All rights reserved.
• Medium value data
• Medium performance level *
• Medium cost
• Should handle data modifications
• Not really manged by SAP HANA
• SAP Dynamic Tiering • HANA Full Use• < 2.5 TB RAM = 4 x RAM• > 2.5 TB RAM = 8 x RAM• Separate Licence (by GB)• Requires HANA Scale-out configuration• Separate software stack to HANA (SAP IQ)• Backup / HA / DR considerations – SAP Note: 2375865• Some Data type / function in-compatibility with HANA• Can mix with Extension nodes in Native use case• MDC supported – each tenant needs separate DT host• Active / Active not supported – SAP Note: 2356851• Data Lifecycle Manager (DLM) support• Should not consider using (replaced by NSE in most cases)• SAP Note: 2636634 - SAP HANA Dynamic Tiering 2.0 SP 04 Release Note
• https://launchpad.support.sap.com/#/notes/2636634
SAP HANA : Data Temperature Management Warm Area
© 2020 SAP SE or an SAP affiliate company. All rights reserved.
Logical Data Area
• Lowest value data *
• Low to Medium performance level *
• Low to Medium cost *
• Should not do data modifications?
• Not Managed by SAP HANA
• Options• Not under SAP HANA control• SAP IQ
• High Performance disk based columnar store• Smart Data Access (SDA)
• SAP HANA Data Lake• Cloud “Version” of SAP IQ
• “Big Data” Integration options• Cloud Storage / Lake• Hadoop / HDFS / Hive / Impala / Spark• Smart Data Integration (SDI) https://support.sap.com/content/dam/launchpad/en_us/pam/pam-essentials/TIP/PAM_HANA_SDI_2_0.pdf
• Smart Data Access (SDA) https://help.sap.com/viewer/6b94445c94ae495c83a19646e7c3fd56/2.0.05/en-US/a07c7ff25997460bbcb73099fb59007d.html
• Data Lifecycle Manager support* (SAP IQ, Dynamic Tiering, Extension node & Spark) – XSC versus XSA• SAP Active / Active read enabled (Licence option)
• 2732012 - Using Active/Active read enabled feature of SAP HANA in SAP S/4HANA Collective Note• https://launchpad.support.sap.com/#/notes/2732012
SAP HANA : Data Temperature Management Cold Area
SAP HANA
Not SAP HANA
Could be Cloud or
On-Premise
User
© 2020 SAP SE or an SAP affiliate company. All rights reserved.
Near-Line Storage, SAP IQ
HDFS, K8s, Cloud Storage
ExternalStore
Cold Data
Extension Node
Native Storage Extension
Extended Store
In-Memory Hot Data
Warm Data
What Data Management Should I Use ?
HANADatabase
Suite on HANAS/4HANA
Data Aging(via NSE)
ILM Store w/ IQ
Optane
ILM/ Archiving
Native HANA
DWF/DLM with Spark Controller
Extension Node
Dynamic Tiering
Optane
NSE
BW on HANABW/4HANA
BW NLS,BW/4 DTO w/ IQ
BW NLS,BW/4 DTO
Optane
Extension Node
Data Aging(via NSE)
Time
Data Value
Data Value declines over time
© 2020 SAP SE or an SAP affiliate company. All rights reserved.
• SAP HANA Full Use / Runtime Editions (licences)• SAP IQ / SAP HANA Data Lake
• Options• SAP HANA Runtime - SAP BW.x
• SAP “BW on HANA” – “Free” - but limited innovation
• SAP “BW/4 HANA” – Runtime licence (GB) – lowest “HANA cost”
• SAP IQ for NLS usage
• SAP HANA Full Use (Enterprise Edition)
• HANA Tiered pricing
• New: SAP IQ allowed for DLM – 1 core per 256GB SAP HANA RAM *
• Data Warehousing: Expose data via SAP HANA Calculation views
• Mixed license mode systems > MDC (Multitenant Database Containers)
• Hardware Considerations• Scale Up <> Scale Out
• PMEM (Optane)
• Active-Active (Read Enabled) – License option
SAP HANA : Data Temperature Management Licence Cost Considerations for Data Warehousing
Q & A?
Thank you.SAP Analytics & Data Warehouse Cloud Team
top related