oracle autonomous data warehouse cloud...–oracle autonomous data warehouse cloud with 16 ocpus and...
TRANSCRIPT
Copyright © 2017 Oracle and/or its affiliates. All rights reserved. |
Oracle Autonomous Data Warehouse Cloud
Date: March 22, 2018Time: 8:30 AM ~ 09:30 AMLocation: Spencer Fox Eccles Business Building
1655 Campus Center Dr., Salt Lake City, UT 84112
Room: 0110, Lower Level
Daniel LiuPrincipal Solution Architect
For Oracle Employees and authorized partners only
UTOUG Training Days 2018
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Safe Harbor Statement
The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle.
Confidential – Oracle Internal/Restricted/Highly Restricted
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Oracle Database 9i, 10g
• Automatic Storage Management (ASM)
• Automatic Memory Management
• Automatic DB Diagnostic Monitor (ADDM)
• Automatic Workload Repository (AWR)
• Automatic Undo tablespaces
• Automatic Segment Space Management
• Automatic Statistics Gathering
• Automatic Standby Management (Broker)
• Automatic Query Rewrite
Oracle Database 11g, 12c
• Automatic SQL Tuning
• Automatic Workload Replay
• Automatic Capture of SQL Monitor
• Automatic Data Optimization
• Automatic Storage Indexes
• Automatic Columnar Cache
• Automatic Diagnostic Framework
• Automatic Refresh of Database Cloning
• Autonomous Health Framework
4
Journey to Autonomous Data Warehouse• Oracle has been developing sophisticated database automation for decades
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Journey to Autonomous Data Warehouse
• With Engineered Systems, Oracle automated database infrastructure
5
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | 6
Next Generation of Cloud…
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Why You Need An Autonomous Cloud
Less time on Administration
• Less time on infrastructure
• Less time on patching, upgrades
• Less time on ensuring availability
• Less time on tuning
7
More time on Innovation
• More time on database design
• More time on developing new apps
• More time on data analytics
• More time on enriching data sets
Challenge: There are more data management tasks than humans to do the work
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Why You Need An Autonomous Cloud
Less time on Administration
• Less time on infrastructure
• Less time on patching, upgrades
• Less time on ensuring availability
• Less time on tuning
8
More time on Innovation
• More time on database design
• More time on developing new apps
• More time on data analytics
• More time on securing data
Challenge: There are more data management tasks than humans to do the work
Payoff is huge - eliminate generic tasks, reduce labor, reduce costs, reduce errors, while
increasing security and availability
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | 9
AutonomousData Warehouse
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Introducing: Autonomous Data Warehouse Cloud
• Easy–Automated management
–Automated tuning: Simply load data and run
• Fast–Based on Oracle’s unique data warehouse technology
• Elastic– Instant scaling of compute or storage with no downtime
10
Expected CY 2017
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | 11
EasySimple provisioning
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Getting Started with Autonomous Data Warehouse Cloud
• Provisioning requires only 5 simple questions:
– Database name?
–Which data center?
– How many CPU’s?
– How many TB’s?
– Admin password?
• New service created in <30 seconds (regardless of size)– Ready to connect via sqlnet
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Demo: Provisioning a new ADWC
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
What You Don’t See Is..
• No decisions for:– Details of the database software
– Configuration of hardware
– Characteristics of the database
– Architecture for backups and availability
• All of this (and more) is automatically configured
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | 20
EasyAutomated Management
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Automated management
• Oracle automates end-to-end management of data warehouse
– Provisioning new database instances
– Growing/shrinking storage and/or compute
– Patching and upgrades
– Backup and recovery
• Full lifecycle managed using ADWC Service Console
21
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Demo: ADWC Management Console
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | 25
EasyAutomated Tuning
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Automated Tuning
• “Load and go”
– Define tables, load data, run queries• No tuning
• No special database expertise required
– Good performance out of the box
• Query using any business analytics tool or cloud service – Built-in SQL notebook also included
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Demo: Monitoring ADWC
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Introducing: Autonomous Data Warehouse Cloud
• Easy–Automated management
–Automated tuning: Simply load data and run
• Fast–Based on Oracle’s unique data warehouse technology
• Elastic– Instant scaling of compute or storage with no downtime
30
Expected CY 2017
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | 31
FastFast straight out of the box
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
• Evaluation of real customer workloads and standard benchmarks
– Performance As-Is versus ADWC
– Run on identical hardware
• ADWC out-of-the-box–Optimized, pre-configured DB setup
• Schema as provided by customer
– Indexes, Partitioning, manual DOP
Performance Proof Points for ADWC
WorkloadOracle ADWC
Customer Schema
NetResult
Customer #1 39 34 …as Good
Customer #2 52 53 Faster
Customer #3 1032 911 …as Good
Customer #4 2138 3051 Faster
Customer #5 132 196 Faster
Customer #6 1478 1831 Faster
Customer #7 2551 5137 Faster
MSTR 1163 1504 Faster
SSB 232 209 …as Good
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Benchmarks - Overview of Schema
• Real world customer workloads
• Workloads consists of medium to complex database schemas
– 20 to 100++ tables –
• Sample queries with varying complexities, representing the customer’s most common data analysis.
– Sample queries were run sequentially on each platform and the total runtime was measured.
Confidential
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Overview of Cost Comparisons
• Pricing is based on published, publicly available price lists
• Cost the price/performance was calculated based on the processing time and the platform
– All costs are based on 24*7 utilization
• Resources allocated to all platforms are comparable in terms of computational power and storage (as close as possible)
– Resource utilization of queries decided by the system ..
• System configurations:
–Oracle Autonomous Data Warehouse Cloud with 16 OCPUs and the minimal amount of storage necessary for the various workloads, in increments of 1TB.
– Redshift Dense Compute systems, 8 nodes of ds2.xlarge
Confidential
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Confidential
Scenario: ADWCS 8OCPUs / 16OCPUs vs. Amazon Redshift
Same workload on ADWC takes ½ the time, with 2x the resources, cost
remains the roughly the same
because more hardware=higher
cost!
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Introducing: Autonomous Data Warehouse Cloud
• Easy–Automated management
–Automated tuning: Simply load data and run
• Fast–Based on Oracle’s unique data warehouse technology
• Elastic– Instant scaling of compute or storage with no downtime
38
Expected CY 2017
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | 39
ElasticOn-Demand, Instant Scaling
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Scaling Out is not Resizing!
Select bigger sized building-block Create required metadata
Put cluster in Read-Only Mode Rebuild
There has to be a better, simpler, more professional way to scale out
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Instant Elasticity: Pay for Exactly What you Use
• Size the DW to the exact number of OCPU’s and TB’s required
–Not constrained by fixed building blocks
• Scale the DW on demand
– Independently scale compute or storage
– Resizing occurs instantly, fully online
• Shut off idle compute save money
– Restart instantly
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Instant Elasticity
• Customer specifies number of database cores (in OCPU) and database size (in TB)
– CPU’s and storage are independent
– Databases can grow and shrink in either dimension
– Pricing based upon CPU ($/CPU/hour) and Storage ($/TB/Month)
• Examples:
– Adding new storage or compute capacity is just a click on the cloud console
– CPU capacity can be added on-demand for end-of-quarter processing, then reduced after the processing is completed
– CPU capacity can be set to zero, while storage remains in place (for example, to reduce costs for databases which are not used during weekends / nights)
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Demo: Scaling ADWC
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Enterprise Scaling OutOracle Autonomous Data Warehouse Cloud
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Enterprise Scaling OutOracle Autonomous Data Warehouse Cloud
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | 46
Safe and SecureData fully encrypted
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Data Security is ON By-Default
• Data is automatically encrypted, by default
– Data is encrypted by default in the cloud, as well as in transit and at rest.
• Oracle staff locked out - well defined “break glass” procedure, with customer approval, for Oracle Operations to access system
• Automated protection from external attacks
– Automatically applies latest security updates online• Quarterly, or off-cycle for high impact security vulnerability
– Native encryption prevents data access from outside the database• Allows keys to be managed by customer’s existing Security Module (HSM)
• Will interface with Database Aware Key Management Cloud Service
47
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | 48
ROBUSTEnterprise Reliability
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Autonomous Data Warehouse Cloud: Reliability
• Enterprise-grade availability – 99.95%– Fault-tolerant, highly-available Exadata infrastructure• Triple-mirrored disks for disk failures
• Real Application Clusters for compute node failures
• And more …
– Automatic backups• Point in time recovery to anytime in last 30 days
• On the roadmap: Mission Critical – 99.995%– Standby database
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Automatically Backed Up
• Backups are scheduled on nightly basis
• To Database Backup Cloud Service
• Or Zero Data Loss Recovery Cloud Service in the future for
– Lowest overhead
– Fastest restore
– Lowest data loss
• Retention Time for backups is configurable
50
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | 51
SAME SKILLSInvestment protection – use your existing DI+BI tools
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Supported by a rich Data Warehouse Ecosystem
• Existing tools running on-premise or in Oracle Cloud
– Oracle BI and data-integration tools
– 3rd party BI tools
– 3rd party data-integration tools
• Connectivity via SQL*Net
Oracle Cloud Services
–Oracle Analytics Cloud
–Oracle Data Integration Platform Cloud
–and others…
Oracle Autonomous Data Warehouse Cloud supports:
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Built-in SQL Notebooks with Oracle ML
• Collaborative UI for business users and data scientists
– Runs in most browsers
– Easy access to shared notebooks, templates
–Management UI for permissions, scheduler, etc.
• Based on open source Apache Zeppelin
53
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Demo: Using SQL Notebooks
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | 55
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | 56
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | 57
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | 58
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Creating Analytical Dashboards
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | 60
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Autonomous Data Warehouse Cloud:Support From Strategic Partners
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Demo – Connecting 3rd party applications
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Connecting to the Database
• Securely connect using credential wallets via SQL*Net, JDBC, ODBC
• Wallet can be downloaded from the service console or using REST APIs
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Location of the wallet
Wallet password
Database service name
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | 66
Data LoadingNew faster, simpler ways to load data into your ADWC
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Data Loading Options
• Data loading via SQL*Net
– Suitable for small volumes of data• SQL*Loader from local filesystem
• ETL scripts that use DML to insert/update data
• Data loading from Oracle Object Storage
– Preferred technique for large volumes of data• Additionally enables data-sharing with other cloud services
– Stage data in Oracle Object Storage, then load into the database using new PL/SQL APIs
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Querying Data on the Oracle Object Store
• Define your credentials for the object store
–Oracle Cloud Infrastructure Object Store username and Swift password required
• Credential stored in the database schema once and used for accessing the object store for all queries
• Call the dbms_cloud API for creating an external table on top of the source files
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Loading Data from Object Stores
• Source data from files on object stores for data loading
–Oracle Object Store or AWS S3
– Any supported Oracle-loader file format
– Roadmap: any Hadoop file format
ADWC
Object Storage
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Demo: Using the Object Store
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
New Cloud API to Access Object Stores, DBMS_CLOUD
• New PL/SQL package for accessing files in object stores
• No need to manually define external tables for loading files
–Makes it easier to specify the format of the source data
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Loading Data from the Oracle Object Store
• Define your credentials for the object store
–Oracle Cloud Infrastructure Object Store username and Swift password required
• Credential stored in the database schema once and used for accessing the object store for all loads
begin
dbms_cloud.create_credential(
credential_name => 'OBJ_STORE_CRED',
username => 'tenant1',
password => ’password'
);
end;
/
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Loading Data from the Oracle Object Store
• Load data directly into the target table without any intermediate steps
• Data format in the source file easily specified as JSON
begin
dbms_cloud.copy_data(
table_name =>'CHANNELS',
credential_name =>'OBJ_STORE_CRED',
file_uri_list =>'https://swiftobjectstorage.us-ashburn-
1.oraclecloud.com/v1/dwcsdemo/DEMO_DATA/chan_v3.dat',
format => json_object('ignoremissingcolumns' value 'true',
'removequotes' value 'true')
);
end;
/
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Querying Data on the Oracle Object Storebegin
dbms_cloud.create_external_table(
table_name =>'CHANNELS_EXT',
credential_name =>'OBJ_STORE_CRED',
file_uri_list => 'https://swiftobjectstorage.us-ashburn-
1.oraclecloud.com/v1/dwcsdemo/DEMO_DATA/chan_v3.dat',
format => json_object('ignoremissingcolumns' value 'true', 'removequotes' value
'true'),
column_list => 'CHANNEL_ID NUMBER,
CHANNEL_DESC VARCHAR2(20),
CHANNEL_CLASS VARCHAR2(20),
CHANNEL_CLASS_ID NUMBER,
CHANNEL_TOTAL VARCHAR2(13),
CHANNEL_TOTAL_ID NUMBER'
);
end;
/
select count(*) from channels_ext;
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Roadmap: Queries across Cloud Storage and more!
• External tables can be created on data in object stores
–Oracle Object Store or AWS S3
– Any supported Oracle-loader file format• Or any Hadoop file format (e.g. Parquet) – coming soon
– Big Data SQL integration – coming soon
• Seamless queries over object store
• Seamless queries across object store and database
Elastic Data Warehouse Compute
Data Storage
ADWCStorage
Object Storage
Kafka HDFS
Data Storage
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | 76
MigrationsMoving your DW to ADWC
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Easy Migration to Autonomous Data Warehouse Cloud
Migration of existing Oracle systems
• Data pump import using ADWC-aware settings
– Ensures compatible data import
– Violating data structures will be ignored or converted
Migration of Redshift systems
• End-to-end migration using migration workbench
– Convert and create data warehouse schema
– Unload Redshift data into S3
– Load data into ADWC
Migration of other systems
• Migration workbench assistance or manual
77
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | 78
ArchitectureOverview ADWC Ecosystem
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Architecture for Autonomous Data Warehouse
Oracle Exadata Cloud Service
Oracle Database Cloud Service
Express Cloud Service
Data Warehouse Services(EDWs, DW, departmental marts and sandboxes)
Autonomous Data Warehouse Cloud
Service Console
Built-in Access Tools
Oracle ML
Service Management
Data Warehouse
SQL Developer
Developer Tools
Data IntegrationServices
3rd Party BI on-premises
Oracle Data Integration Cloud Service
3rd Party DI on Oracle Cloud Compute
3rd Party DI On-premises
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | 80
SummaryAutonomous Data Warehouse Cloud
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | 81
Nearly 4 decades of enterprise innovations now available in the cloud
Industry Leading Data Management Expertise
37 YearsOf Data Management Experience
10 YearsOf Engineered Systems Innovation
Thousands Of GlobalEnterprise Customers
20 Cloud Data Centers World Wide
✓
✓
✓
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Highly ElasticQuick and easily scale up and down compute and storage without having to over pay for fixed blocks of resources
Built-in Web-Based NotebookApache Zeppelin based notebooks ready to run from your browser
Database migration utilityDedicated cloud-ready migration tools for easy migration from AmazonAWS Redshift, SQL Server and other databases
Enterprise Grade SecurityData is encrypted by default in the cloud, as well as in transit and at rest.
High-Performance Queriesand Concurrent WorkloadsOptimized query performance with preconfigured resource profiles for different types of users
Oracle SQLADWC is compatible with all business analytics tools that support Oracle Database
Self DrivingFully automated database for self-tuning patching, upgrades itself while the system is running
Cloud-Based Data LoadingFast, scalable data-loading from Oracle Object Store, AWS S3, or on-premises
Oracle Autonomous Data Warehouse Cloud Key Features
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Autonomous Data Warehouse Cloud
• Easy
– Fully-managed, pre-configured and optimized for DW workloads
– Simply load data and run
• No need to define indexes, create partitions, etc.
• Fast– Based on Exadata technology
• Elastic
– Instant scaling of compute or storage with no downtime