oracle - engineered for innovation · big data buzz “why big data is a big deal” infoworld –...
TRANSCRIPT
![Page 1: Oracle - Engineered for Innovation · Big Data Buzz “Why big data is a big deal” InfoWorld – 9/1/11 “The challenge– and opportunity– of big data” McKinsey Quarterly—5/11](https://reader033.vdocuments.us/reader033/viewer/2022042420/5f37561980184517c01c8c52/html5/thumbnails/1.jpg)
Oracle - Engineered for Innovation
Thomas Kyte
http://asktom.oracle.com
![Page 2: Oracle - Engineered for Innovation · Big Data Buzz “Why big data is a big deal” InfoWorld – 9/1/11 “The challenge– and opportunity– of big data” McKinsey Quarterly—5/11](https://reader033.vdocuments.us/reader033/viewer/2022042420/5f37561980184517c01c8c52/html5/thumbnails/2.jpg)
The Beginning...
• Data Model with Structure
• Data Independent of Code
• Set-oriented
• 1977 the work begins• 1977 the work begins
![Page 3: Oracle - Engineered for Innovation · Big Data Buzz “Why big data is a big deal” InfoWorld – 9/1/11 “The challenge– and opportunity– of big data” McKinsey Quarterly—5/11](https://reader033.vdocuments.us/reader033/viewer/2022042420/5f37561980184517c01c8c52/html5/thumbnails/3.jpg)
GPS 1978
![Page 4: Oracle - Engineered for Innovation · Big Data Buzz “Why big data is a big deal” InfoWorld – 9/1/11 “The challenge– and opportunity– of big data” McKinsey Quarterly—5/11](https://reader033.vdocuments.us/reader033/viewer/2022042420/5f37561980184517c01c8c52/html5/thumbnails/4.jpg)
First RDBMS: Version 2 June 1979
• FIRST Commercial SQL RDBMS
• Impressive First SQL• Joins, Subqueries
• Outer Joins, Connect By• Outer Joins, Connect By
• A Simple Server• No transactions, ‘Limited’ Reliability
• Portability from the Start• Written in Fortran
• But multi-platform – PDP11, Dec VAX
![Page 5: Oracle - Engineered for Innovation · Big Data Buzz “Why big data is a big deal” InfoWorld – 9/1/11 “The challenge– and opportunity– of big data” McKinsey Quarterly—5/11](https://reader033.vdocuments.us/reader033/viewer/2022042420/5f37561980184517c01c8c52/html5/thumbnails/5.jpg)
IBM PC – 1981
IBM model number number
5150, introduced on August 12, 1981.
![Page 6: Oracle - Engineered for Innovation · Big Data Buzz “Why big data is a big deal” InfoWorld – 9/1/11 “The challenge– and opportunity– of big data” McKinsey Quarterly—5/11](https://reader033.vdocuments.us/reader033/viewer/2022042420/5f37561980184517c01c8c52/html5/thumbnails/6.jpg)
Internet (as we know it) –1983
The first TCP/IP-based wide-area network was operational by January 1, 1983 when all hosts 1, 1983 when all hosts on the ARPANET were switched over from the older NCP protocols.
![Page 7: Oracle - Engineered for Innovation · Big Data Buzz “Why big data is a big deal” InfoWorld – 9/1/11 “The challenge– and opportunity– of big data” McKinsey Quarterly—5/11](https://reader033.vdocuments.us/reader033/viewer/2022042420/5f37561980184517c01c8c52/html5/thumbnails/7.jpg)
Portability: Version 3 March 1983
• New Implementation Designed for Portability
• Written in ‘C’
• Single Source• Single Source
• Architectural Changes
• Transactions, multi-versioning, no read consistency
• AI/BI files
• Oracle Corporation – name established
![Page 8: Oracle - Engineered for Innovation · Big Data Buzz “Why big data is a big deal” InfoWorld – 9/1/11 “The challenge– and opportunity– of big data” McKinsey Quarterly—5/11](https://reader033.vdocuments.us/reader033/viewer/2022042420/5f37561980184517c01c8c52/html5/thumbnails/8.jpg)
![Page 9: Oracle - Engineered for Innovation · Big Data Buzz “Why big data is a big deal” InfoWorld – 9/1/11 “The challenge– and opportunity– of big data” McKinsey Quarterly—5/11](https://reader033.vdocuments.us/reader033/viewer/2022042420/5f37561980184517c01c8c52/html5/thumbnails/9.jpg)
![Page 10: Oracle - Engineered for Innovation · Big Data Buzz “Why big data is a big deal” InfoWorld – 9/1/11 “The challenge– and opportunity– of big data” McKinsey Quarterly—5/11](https://reader033.vdocuments.us/reader033/viewer/2022042420/5f37561980184517c01c8c52/html5/thumbnails/10.jpg)
Cooperative Server: Version 5 April 1985
• My First Oracle Experience
• 1st Client/Server
• Cooperative Server
• Distributed Processing• Distributed Processing
• Parallel Server
• Portability• V5 was first to go beyond 640K memory on PCs
• Single-user for Macintosh o/s
• SQL_TRACE
• select trace('sql',1),1 from dual;
![Page 11: Oracle - Engineered for Innovation · Big Data Buzz “Why big data is a big deal” InfoWorld – 9/1/11 “The challenge– and opportunity– of big data” McKinsey Quarterly—5/11](https://reader033.vdocuments.us/reader033/viewer/2022042420/5f37561980184517c01c8c52/html5/thumbnails/11.jpg)
Transaction Processing: Version 6 July 1988
• New Architecture • Performance (first SMP)
• Availability
• TPO• TPO
• PL/SQL
• V6 Lays Architectural Groundwork for the Future
• This was a rewrite of the entire database fundamentally
![Page 12: Oracle - Engineered for Innovation · Big Data Buzz “Why big data is a big deal” InfoWorld – 9/1/11 “The challenge– and opportunity– of big data” McKinsey Quarterly—5/11](https://reader033.vdocuments.us/reader033/viewer/2022042420/5f37561980184517c01c8c52/html5/thumbnails/12.jpg)
World Wide Web – 1990’ish
The World Wide Web was created in 1989 by British scientist Tim Berners-Lee, working at the European working at the European Organization for Nuclear
Research (CERN) in Geneva, Switzerland, and
released in 1992.
![Page 13: Oracle - Engineered for Innovation · Big Data Buzz “Why big data is a big deal” InfoWorld – 9/1/11 “The challenge– and opportunity– of big data” McKinsey Quarterly—5/11](https://reader033.vdocuments.us/reader033/viewer/2022042420/5f37561980184517c01c8c52/html5/thumbnails/13.jpg)
Oracle7.3 February 1996
• Partitioned Views
• Bitmapped Indexes
• Asynchronous read ahead for table scans
• Standby Database
• Deferred transaction recovery on
� Spatial Data Option
� Tablespaces changes - Coalesce, Temporary Permanent,
� Trigger compilation, debug
� Unlimited extents on STORAGE clause.
� Some init.ora parameters modifiable -• Deferred transaction recovery on instance startup
• Updatable Join View
• SQLDBA no longer shipped.
• Index rebuilds
• DBV introduced
• Context Option
• PL/SQL - UTL_FILE
� Some init.ora parameters modifiable -TIMED_STATISTICS
� HASH Joins, Antijoins
� Histograms
� Oracle Trace
� Advanced Replication Object Groups
![Page 14: Oracle - Engineered for Innovation · Big Data Buzz “Why big data is a big deal” InfoWorld – 9/1/11 “The challenge– and opportunity– of big data” McKinsey Quarterly—5/11](https://reader033.vdocuments.us/reader033/viewer/2022042420/5f37561980184517c01c8c52/html5/thumbnails/14.jpg)
![Page 15: Oracle - Engineered for Innovation · Big Data Buzz “Why big data is a big deal” InfoWorld – 9/1/11 “The challenge– and opportunity– of big data” McKinsey Quarterly—5/11](https://reader033.vdocuments.us/reader033/viewer/2022042420/5f37561980184517c01c8c52/html5/thumbnails/15.jpg)
![Page 16: Oracle - Engineered for Innovation · Big Data Buzz “Why big data is a big deal” InfoWorld – 9/1/11 “The challenge– and opportunity– of big data” McKinsey Quarterly—5/11](https://reader033.vdocuments.us/reader033/viewer/2022042420/5f37561980184517c01c8c52/html5/thumbnails/16.jpg)
Data Warehouses Growing RapidlyTripling In Size Every Two Years
800
1000
Te
rab
yte
s o
f D
ata
Size of the Largest Data Warehouses
Source: Winter TopTen Survey, Winter Corporation, Waltham MA, 2008.
200
400
600
1998 2000 2002 2004 2006 2008 2010 2012
Te
rab
yte
s o
f D
ata
![Page 17: Oracle - Engineered for Innovation · Big Data Buzz “Why big data is a big deal” InfoWorld – 9/1/11 “The challenge– and opportunity– of big data” McKinsey Quarterly—5/11](https://reader033.vdocuments.us/reader033/viewer/2022042420/5f37561980184517c01c8c52/html5/thumbnails/17.jpg)
Enabling the Private Database CloudYears of continuous Oracle innovation
Oracle Database 11g
Oracle Exadata
Quality of Service ManagementInfiniBand support
Smart ScansSmart Flash Cache
Hybrid Columnar Compression
Oracle Database 10g
Oracle9i DatabaseReal Application Clusters
Database ServicesCPU Resource Management
Automatic Storage ManagementDynamic Database Services
Instance cagingI/O resource management
Server PoolsQuality of Service Management
© 2011 Oracle Corporation
![Page 18: Oracle - Engineered for Innovation · Big Data Buzz “Why big data is a big deal” InfoWorld – 9/1/11 “The challenge– and opportunity– of big data” McKinsey Quarterly—5/11](https://reader033.vdocuments.us/reader033/viewer/2022042420/5f37561980184517c01c8c52/html5/thumbnails/18.jpg)
Three Major Database Focus AreasEngineered for Innovation
SOCIAL
BLOG 101100101001001001101010
101011100101010100100101
CLOUD COMPUTINGPRIVATE AND PUBLIC
ENGINEERED SYSTEMS BIG DATA
SMART
METER
010100100101
![Page 19: Oracle - Engineered for Innovation · Big Data Buzz “Why big data is a big deal” InfoWorld – 9/1/11 “The challenge– and opportunity– of big data” McKinsey Quarterly—5/11](https://reader033.vdocuments.us/reader033/viewer/2022042420/5f37561980184517c01c8c52/html5/thumbnails/19.jpg)
Building Managed Server and Storage Pools Real Application Clusters
Automatic Storage Management
EnterpriseManager
In-Memory Database Cache
Real Application Clusters
![Page 20: Oracle - Engineered for Innovation · Big Data Buzz “Why big data is a big deal” InfoWorld – 9/1/11 “The challenge– and opportunity– of big data” McKinsey Quarterly—5/11](https://reader033.vdocuments.us/reader033/viewer/2022042420/5f37561980184517c01c8c52/html5/thumbnails/20.jpg)
Service Level ManagementResource Manager and Instance Caging
Resource Manager allocates CPU and
Memory
Instance caging allocates Instance caging allocates
cores per instance
Capacity-on-demand for
elastic cloud computing
![Page 21: Oracle - Engineered for Innovation · Big Data Buzz “Why big data is a big deal” InfoWorld – 9/1/11 “The challenge– and opportunity– of big data” McKinsey Quarterly—5/11](https://reader033.vdocuments.us/reader033/viewer/2022042420/5f37561980184517c01c8c52/html5/thumbnails/21.jpg)
Complete Data SecurityFirewall, Encryption, Separation of Duty and Monitoring
Network SQL Monitoring and Blocking
Encrypted Database
DataMasking
Data
Activity Audit
Applications
Multi-factor authorization
UnauthorizedDBA Activity
Compliance Scan
Vulnerability Scan
Discovery
Patch Automation
![Page 22: Oracle - Engineered for Innovation · Big Data Buzz “Why big data is a big deal” InfoWorld – 9/1/11 “The challenge– and opportunity– of big data” McKinsey Quarterly—5/11](https://reader033.vdocuments.us/reader033/viewer/2022042420/5f37561980184517c01c8c52/html5/thumbnails/22.jpg)
Provisioning Software to the CloudLower complexity via Reference Configurations
Gold image reference
configurations
Standardized deployments via profilesCreate Reference Stage as Gold Provision via profiles
Rapidly provision databases
to the Cloud
Monitor change centrally to
ensure compliance
Create Reference Configuration
Stage as Gold Image
Provision Database on
Cloud
Manage Centrally
![Page 23: Oracle - Engineered for Innovation · Big Data Buzz “Why big data is a big deal” InfoWorld – 9/1/11 “The challenge– and opportunity– of big data” McKinsey Quarterly—5/11](https://reader033.vdocuments.us/reader033/viewer/2022042420/5f37561980184517c01c8c52/html5/thumbnails/23.jpg)
Metering and ChargebackEnterprise Manager 12c
Resource usage
metering
Historical usage trends
Cost allocation and
charge plan evaluation
Reporting for cloud self-
service application
Discover & Plan Track Usage Charge User
![Page 24: Oracle - Engineered for Innovation · Big Data Buzz “Why big data is a big deal” InfoWorld – 9/1/11 “The challenge– and opportunity– of big data” McKinsey Quarterly—5/11](https://reader033.vdocuments.us/reader033/viewer/2022042420/5f37561980184517c01c8c52/html5/thumbnails/24.jpg)
Optimized, Pre-Integrated Cloud PlatformOracle Exadata Database Machine
Database Server Pool– Oracle Database 11g Release 2– Oracle Real Application Clusters– Automatic Storage Management
Storage Server Pool– Up to 336 TB disk– 5 TB flash storage– Oracle Exadata Storage Software
InfiniBand Network– 40 Gb/sec redundant switches
![Page 25: Oracle - Engineered for Innovation · Big Data Buzz “Why big data is a big deal” InfoWorld – 9/1/11 “The challenge– and opportunity– of big data” McKinsey Quarterly—5/11](https://reader033.vdocuments.us/reader033/viewer/2022042420/5f37561980184517c01c8c52/html5/thumbnails/25.jpg)
Improve Data Warehouse performance 10x Exadata Smart Scans
Data intensive processing runs in
Exadata Storage
Servers
Select sum(sales)where salesdate=‘21-Sep-2010’…
What Were Yesterday’s
Sales?
Rows and columns
filtered as data streams
from disks
Sum
Filter Sales for Sep 21
2010
![Page 26: Oracle - Engineered for Innovation · Big Data Buzz “Why big data is a big deal” InfoWorld – 9/1/11 “The challenge– and opportunity– of big data” McKinsey Quarterly—5/11](https://reader033.vdocuments.us/reader033/viewer/2022042420/5f37561980184517c01c8c52/html5/thumbnails/26.jpg)
Storage OptimizationNext Generation Compression Technology
Increase compression as data ages
Improve query performance for table scans
4XUp to
OLTPCompression
50XUp to
Improve cache density for OLTP performance
No changes to existing applications
Cascade storage savings
throughout data center
50XHybrid ColumnarCompression (Exadata, Pillar Axion, Sun ZFS Storage Appliance)
![Page 27: Oracle - Engineered for Innovation · Big Data Buzz “Why big data is a big deal” InfoWorld – 9/1/11 “The challenge– and opportunity– of big data” McKinsey Quarterly—5/11](https://reader033.vdocuments.us/reader033/viewer/2022042420/5f37561980184517c01c8c52/html5/thumbnails/27.jpg)
Extreme Performance for OLTP Exadata Smart Flash Cache
Full rack has 5 TB of
Smart Flash Cache
Can process over 1 million
IOs per second
![Page 28: Oracle - Engineered for Innovation · Big Data Buzz “Why big data is a big deal” InfoWorld – 9/1/11 “The challenge– and opportunity– of big data” McKinsey Quarterly—5/11](https://reader033.vdocuments.us/reader033/viewer/2022042420/5f37561980184517c01c8c52/html5/thumbnails/28.jpg)
Public Cloud Oracle Public Cloud (cloud.oracle.com)
Public Cloud- Self Service- Monthly Subscription- Simple Pricing
ServicesServices- Application- Platform
Platform Services- Java Cloud Service- Database Cloud Service
![Page 29: Oracle - Engineered for Innovation · Big Data Buzz “Why big data is a big deal” InfoWorld – 9/1/11 “The challenge– and opportunity– of big data” McKinsey Quarterly—5/11](https://reader033.vdocuments.us/reader033/viewer/2022042420/5f37561980184517c01c8c52/html5/thumbnails/29.jpg)
Database Applications in the Public Cloud Database Cloud Access and Applications
RESTfulWeb
Oracle Application Express Applications
Web Services
Oracle Java Cloud
SQL Developer
![Page 30: Oracle - Engineered for Innovation · Big Data Buzz “Why big data is a big deal” InfoWorld – 9/1/11 “The challenge– and opportunity– of big data” McKinsey Quarterly—5/11](https://reader033.vdocuments.us/reader033/viewer/2022042420/5f37561980184517c01c8c52/html5/thumbnails/30.jpg)
Big Data Buzz
“Why big data is a big deal”InfoWorld – 9/1/11
“The challenge–and opportunity–of big data”McKinsey Quarterly—5/11
“Ten reasons why Big Data will change the travel industry”Tnooz -8/15/11
“Keeping Afloat in a Sea of 'Big Data”ITBusinessEdge – 9/6/11
“Getting a Handle on Big Data with Hadoop”Businessweek-9/7/11
“The promise of Big Data”Intelligent Utility-8/28/11
![Page 31: Oracle - Engineered for Innovation · Big Data Buzz “Why big data is a big deal” InfoWorld – 9/1/11 “The challenge– and opportunity– of big data” McKinsey Quarterly—5/11](https://reader033.vdocuments.us/reader033/viewer/2022042420/5f37561980184517c01c8c52/html5/thumbnails/31.jpg)
Big Data Use Cases
Today’s Challenge New Data What’s Possible
Healthcare
Expensive office visitsRemote patient monitoring
Preventive care, reduced
hospitalization
Manufacturing
In-person supportProduct sensors Automated diagnosis, support
In-person support
Location-Based Services
Based on home zip codeReal time location data
Geo-advertising, traffic, local
search
Public Sector
Standardized servicesCitizen surveys
Tailored services,
cost reductions
Retail
One size fits all marketingSocial media
Sentiment analysis
segmentation
![Page 32: Oracle - Engineered for Innovation · Big Data Buzz “Why big data is a big deal” InfoWorld – 9/1/11 “The challenge– and opportunity– of big data” McKinsey Quarterly—5/11](https://reader033.vdocuments.us/reader033/viewer/2022042420/5f37561980184517c01c8c52/html5/thumbnails/32.jpg)
What Makes it Big Data?
SOCIAL
BLOG
101100101001
001001101010
VOLUME VELOCITY VARIETY VALUE
SMART
METER
001001101010
101011100101
010100100101
![Page 33: Oracle - Engineered for Innovation · Big Data Buzz “Why big data is a big deal” InfoWorld – 9/1/11 “The challenge– and opportunity– of big data” McKinsey Quarterly—5/11](https://reader033.vdocuments.us/reader033/viewer/2022042420/5f37561980184517c01c8c52/html5/thumbnails/33.jpg)
![Page 34: Oracle - Engineered for Innovation · Big Data Buzz “Why big data is a big deal” InfoWorld – 9/1/11 “The challenge– and opportunity– of big data” McKinsey Quarterly—5/11](https://reader033.vdocuments.us/reader033/viewer/2022042420/5f37561980184517c01c8c52/html5/thumbnails/34.jpg)