the cloud data platform for insights-driven...
TRANSCRIPT
![Page 1: The Cloud Data Platform for Insights-Driven Enterprisesgo.qubole.com/rs/510-QPZ-296/images/Qubole_Oracle_WR_05... · 2020-04-23 · • Hadoop/Spark puts compute and storage togetherwithin](https://reader034.vdocuments.us/reader034/viewer/2022042220/5ec5e92258b9785858481ec7/html5/thumbnails/1.jpg)
TheCloudDataPlatformforInsights-DrivenEnterprises
![Page 2: The Cloud Data Platform for Insights-Driven Enterprisesgo.qubole.com/rs/510-QPZ-296/images/Qubole_Oracle_WR_05... · 2020-04-23 · • Hadoop/Spark puts compute and storage togetherwithin](https://reader034.vdocuments.us/reader034/viewer/2022042220/5ec5e92258b9785858481ec7/html5/thumbnails/2.jpg)
Today’sSpeakers
CraigCarl XingQuanDirectorofSolutionsArchitecture SeniorDirectorofProductManagement
![Page 3: The Cloud Data Platform for Insights-Driven Enterprisesgo.qubole.com/rs/510-QPZ-296/images/Qubole_Oracle_WR_05... · 2020-04-23 · • Hadoop/Spark puts compute and storage togetherwithin](https://reader034.vdocuments.us/reader034/viewer/2022042220/5ec5e92258b9785858481ec7/html5/thumbnails/3.jpg)
BigDataDisruptsMarkets
WhatdotheyhaveinCommon?
DesignproductsthatfitcustomersaccordingtotheirDNA
Programrecommendationsandcommissioningnew
content
Accurateestimatedtimeofarrival
Pricesuggestionsforhosts
Newstoresinverycloseproximity
Searchforsimilarimages
![Page 4: The Cloud Data Platform for Insights-Driven Enterprisesgo.qubole.com/rs/510-QPZ-296/images/Qubole_Oracle_WR_05... · 2020-04-23 · • Hadoop/Spark puts compute and storage togetherwithin](https://reader034.vdocuments.us/reader034/viewer/2022042220/5ec5e92258b9785858481ec7/html5/thumbnails/4.jpg)
ChallengesImplementingBigData
• Variety(40%)andVolume(14%)arethemaindriversforbigdataexplosion– Manydisjointedsources
• Datasilosonlyprovidepartialanswers
• Deployingbigdataon-premises:– Iscomplextomaintainandoperate– Isexpensive– Requiresexpertise– Unabletoscale
Collectmultipledatasources
Makethemusable
Makeitavailabletothebusiness
BigData
![Page 5: The Cloud Data Platform for Insights-Driven Enterprisesgo.qubole.com/rs/510-QPZ-296/images/Qubole_Oracle_WR_05... · 2020-04-23 · • Hadoop/Spark puts compute and storage togetherwithin](https://reader034.vdocuments.us/reader034/viewer/2022042220/5ec5e92258b9785858481ec7/html5/thumbnails/5.jpg)
WhySpark?
SparkStreamingreal-time
SparkSQLStructuredad-hoc
MLlibMachineLearning
GraphXGraphProcessing
SparkCoreScala,Python
• Sparkdoesprocessinginmemory,whichisfasterthantraditionalHDDs• Ithasafully-featuredecosystemofproductsandusecases;inparticular,itis
tailoredtowardaDataScientistandalgorithm/machinelearningdevelopment• IthasaverysimpleAPI• It’sopensourceandhelpsyouavoidvendorandtechnologylock-in
![Page 6: The Cloud Data Platform for Insights-Driven Enterprisesgo.qubole.com/rs/510-QPZ-296/images/Qubole_Oracle_WR_05... · 2020-04-23 · • Hadoop/Spark puts compute and storage togetherwithin](https://reader034.vdocuments.us/reader034/viewer/2022042220/5ec5e92258b9785858481ec7/html5/thumbnails/6.jpg)
HadoopandSparkModel&Issues
• Hadoop/Sparkputscomputeandstoragetogether withinacomputenode
• Forcescomputeandstoragetoscaletogether,whichisnotideal
• Theclustermustbepersistentlyonorelsethedataisinaccessible
C+S
C+S
C+S
C+S
C+S
C+S
C+S
C+S
C+S
C+S C+S C+S
![Page 7: The Cloud Data Platform for Insights-Driven Enterprisesgo.qubole.com/rs/510-QPZ-296/images/Qubole_Oracle_WR_05... · 2020-04-23 · • Hadoop/Spark puts compute and storage togetherwithin](https://reader034.vdocuments.us/reader034/viewer/2022042220/5ec5e92258b9785858481ec7/html5/thumbnails/7.jpg)
AModernDataPlatform
• Leveragethecloud– On-demandandelasticcompute– Scaleoutobjectstorage
• Expandandcontractbasedonworkloads
• Turnkeyservice,ratherthanamanagedsoftwareorhardware– Increasetimetovalue
• Highdegreeofautomation,orchestrationandself-serviceenablement– Reducecostsandcomplexities
BigData
Ephemeral
Automation
Self-service
Orchestration
![Page 8: The Cloud Data Platform for Insights-Driven Enterprisesgo.qubole.com/rs/510-QPZ-296/images/Qubole_Oracle_WR_05... · 2020-04-23 · • Hadoop/Spark puts compute and storage togetherwithin](https://reader034.vdocuments.us/reader034/viewer/2022042220/5ec5e92258b9785858481ec7/html5/thumbnails/8.jpg)
8
OracleBareMetalCloudServices
CraigCarlDirectorofSolutionsArchitecture,BareMetalCloud
![Page 9: The Cloud Data Platform for Insights-Driven Enterprisesgo.qubole.com/rs/510-QPZ-296/images/Qubole_Oracle_WR_05... · 2020-04-23 · • Hadoop/Spark puts compute and storage togetherwithin](https://reader034.vdocuments.us/reader034/viewer/2022042220/5ec5e92258b9785858481ec7/html5/thumbnails/9.jpg)
• Over600peopleinSeattleandNorthernCalifornia• Hundredsofexpertsatdeliveringhigh-scaleproductioncloudproducts
– AWS,Azure,Google,Joyent,F5,Salesforce• Toaonewe’repassionateaboutsolvinglargescaledistributedcompute
problems,passionatepeoplebuildamazingproduct• CombinedwithOracle’sdecadesofsuccessintheenterprisemarket
9
Deepcloudengineeringexperience
OracleBareMetalCloudServices
![Page 10: The Cloud Data Platform for Insights-Driven Enterprisesgo.qubole.com/rs/510-QPZ-296/images/Qubole_Oracle_WR_05... · 2020-04-23 · • Hadoop/Spark puts compute and storage togetherwithin](https://reader034.vdocuments.us/reader034/viewer/2022042220/5ec5e92258b9785858481ec7/html5/thumbnails/10.jpg)
10
Industry’sfirstBareMetalCloudService(withVirtualMachines,ofcourse!)
FullyDedicated
Industry’sfirstfullydedicatedinstances–nohypervisor,agents,noisyneighborsorsharedresources
BuiltforEnterpriseApps
Builttosupportdemandingenterprise
applications
Performance-First
Performance-firstapproachwith
significantlyhigherperformancethan
existingcloudoptions
Pay-as-you-goPricing
Paybythehourforeverything:compute,IPaddressandblockstorage– burstupor
downquickly
AutomatedandAPIDriven
RESTfulAPIs,SDKs,orchestration,CLIs,completeandpublic
documentation
FastProvisioning
Spin-upbaremetalinstancesinlessthan5
minutes,virtualinstancesin90
seconds
MixBareMetalandvirtualinstances
IdenticaluserexperiencebetweenBareMetalandVirtual
instances
![Page 11: The Cloud Data Platform for Insights-Driven Enterprisesgo.qubole.com/rs/510-QPZ-296/images/Qubole_Oracle_WR_05... · 2020-04-23 · • Hadoop/Spark puts compute and storage togetherwithin](https://reader034.vdocuments.us/reader034/viewer/2022042220/5ec5e92258b9785858481ec7/html5/thumbnails/11.jpg)
11
OBMCSFundamentals:AvailabilityDomainsRegionalModelSub-millisecondlatencybetweenADs10Gb/secbetweeneachinstance,interandintraAD
![Page 12: The Cloud Data Platform for Insights-Driven Enterprisesgo.qubole.com/rs/510-QPZ-296/images/Qubole_Oracle_WR_05... · 2020-04-23 · • Hadoop/Spark puts compute and storage togetherwithin](https://reader034.vdocuments.us/reader034/viewer/2022042220/5ec5e92258b9785858481ec7/html5/thumbnails/12.jpg)
12
• Multipleinstancetypes– Standard– 256GBRAM– HighI/O– 12.8TBNVMeSSD,512GBRAM– DenseI/O– 28.8TB NVMeSSD,512GBRAM– 1,2,4,8,16coreVMs(7GBmem/core)
• BareMetalinstanceshapes– 36cores2.3GHzIntel®Xeon®processorE5-2600v3– 10Gbnetwork
• Images– OracleLinux,CentOS,Ubuntu,Windows– SupportforcustomimagesandcustomOSes
Compute
![Page 13: The Cloud Data Platform for Insights-Driven Enterprisesgo.qubole.com/rs/510-QPZ-296/images/Qubole_Oracle_WR_05... · 2020-04-23 · • Hadoop/Spark puts compute and storage togetherwithin](https://reader034.vdocuments.us/reader034/viewer/2022042220/5ec5e92258b9785858481ec7/html5/thumbnails/13.jpg)
13
• SinglenodeOracledatabase– HighandDenseinstances
• 2nodeOracleRAC• Exadata
– Quarter– Half– Fullrack
DBSystems
![Page 14: The Cloud Data Platform for Insights-Driven Enterprisesgo.qubole.com/rs/510-QPZ-296/images/Qubole_Oracle_WR_05... · 2020-04-23 · • Hadoop/Spark puts compute and storage togetherwithin](https://reader034.vdocuments.us/reader034/viewer/2022042220/5ec5e92258b9785858481ec7/html5/thumbnails/14.jpg)
14
Services Oracle BMCSvsAWS
HighPerformanceCompute(DenseIO compared toAWSI2.8xlarge)
8coreVirtual Machine(ComparetoAWSM4.2xlarge)
OutboardDataTransfer $86%Lower
$38%Lower
2.25xCores
$21%Lower2x
RAM11.5xIOPS
4.5xStorage
SimilarRAM
SameCores
1Pricingdimension
vs.4
Freeinter-AD
10xFreeEgress
![Page 15: The Cloud Data Platform for Insights-Driven Enterprisesgo.qubole.com/rs/510-QPZ-296/images/Qubole_Oracle_WR_05... · 2020-04-23 · • Hadoop/Spark puts compute and storage togetherwithin](https://reader034.vdocuments.us/reader034/viewer/2022042220/5ec5e92258b9785858481ec7/html5/thumbnails/15.jpg)
BareMetalcompute
10Gbnetwork
NooversubscriptionLowlatencynetwork
NVMeSSDs
Nonoisyneighbors
Objectstore OracleRDMS
![Page 16: The Cloud Data Platform for Insights-Driven Enterprisesgo.qubole.com/rs/510-QPZ-296/images/Qubole_Oracle_WR_05... · 2020-04-23 · • Hadoop/Spark puts compute and storage togetherwithin](https://reader034.vdocuments.us/reader034/viewer/2022042220/5ec5e92258b9785858481ec7/html5/thumbnails/16.jpg)
Simple• Acompletedataplatformsolution• Noneedtomanageinfrastructure• Self-servicedataaccessacrosstheenterprise
AgileandFast• SparkandHadoopclustersinminutes• BuildsonOracleBareMetalCloudperformanceadvantages
• Getbusinessinsightsfaster
Cost• StandupyourSparkorHadoopinfrastructureatafractionofthecost
• Reduceoperationandmanagementcost
QuboleisaTurnkeyBigDataServiceonOracleBareMetalCloud
![Page 17: The Cloud Data Platform for Insights-Driven Enterprisesgo.qubole.com/rs/510-QPZ-296/images/Qubole_Oracle_WR_05... · 2020-04-23 · • Hadoop/Spark puts compute and storage togetherwithin](https://reader034.vdocuments.us/reader034/viewer/2022042220/5ec5e92258b9785858481ec7/html5/thumbnails/17.jpg)
BuiltforAnyonewhoUsesDataAnalystslDataScientistslDataEngineerslDataAdmins
BigDataYourWay.
Quboleautomates,controlsandorchestratesyourbigdataworkloadssothatyoucanoptimizeperformance,costandscale.
ASinglePlatformforAnyUseCaseETL&ReportinglAdHocQuerieslMachineLearninglStreaminglVerticalApps
OpenSourceEngines,OptimizedfortheCloud
NativeIntegrationwithOracleBareMetalCloudServiceLeveragestheOracleCloudPlatform’sspeedandperformance
![Page 18: The Cloud Data Platform for Insights-Driven Enterprisesgo.qubole.com/rs/510-QPZ-296/images/Qubole_Oracle_WR_05... · 2020-04-23 · • Hadoop/Spark puts compute and storage togetherwithin](https://reader034.vdocuments.us/reader034/viewer/2022042220/5ec5e92258b9785858481ec7/html5/thumbnails/18.jpg)
Spinupreal-timestreamingdataprocessingon-demand
115%Fasterthanon-premises
QUBOLEDATASERVICE(QDS)SPARKSQLONORACLECLOUDPLATFORMINFRASTRUCTURE
• 115%fasteronreportingqueriesand50%fasteronanalyticsqueriesthanClouderaImpalaon-premises*
![Page 19: The Cloud Data Platform for Insights-Driven Enterprisesgo.qubole.com/rs/510-QPZ-296/images/Qubole_Oracle_WR_05... · 2020-04-23 · • Hadoop/Spark puts compute and storage togetherwithin](https://reader034.vdocuments.us/reader034/viewer/2022042220/5ec5e92258b9785858481ec7/html5/thumbnails/19.jpg)
Whatmakesusdifferent
19Qubole Confidential
UserProductivity
• Self-servicedataaccess• SimpleInterfaces• IncreasedPersonasonOracleBMC
AmplifytheCloud
• ObjectStoreasdatalake• LeverageNetworkPerformance• Supportforallshapes
Automation
• AutomaticuseofOracleBMCAPIs• Clusterlifecyclemanagement• Auto-scaling• SoftwareUpgrades
Elasticity
• Scale34xonaverage• ReduceTCOby33%• DrivesscaletoOracleBMC
![Page 20: The Cloud Data Platform for Insights-Driven Enterprisesgo.qubole.com/rs/510-QPZ-296/images/Qubole_Oracle_WR_05... · 2020-04-23 · • Hadoop/Spark puts compute and storage togetherwithin](https://reader034.vdocuments.us/reader034/viewer/2022042220/5ec5e92258b9785858481ec7/html5/thumbnails/20.jpg)
TheMostScalablePlatform
500PB
DataProcessedintheCloudMonthly
500Nodes
LargestSparkClusterintheCloud
2000
ClustersStartedpermonth
6PB 80PB 150PB 500PB
![Page 21: The Cloud Data Platform for Insights-Driven Enterprisesgo.qubole.com/rs/510-QPZ-296/images/Qubole_Oracle_WR_05... · 2020-04-23 · • Hadoop/Spark puts compute and storage togetherwithin](https://reader034.vdocuments.us/reader034/viewer/2022042220/5ec5e92258b9785858481ec7/html5/thumbnails/21.jpg)
DataDrivenCompaniesUseQubole
![Page 22: The Cloud Data Platform for Insights-Driven Enterprisesgo.qubole.com/rs/510-QPZ-296/images/Qubole_Oracle_WR_05... · 2020-04-23 · • Hadoop/Spark puts compute and storage togetherwithin](https://reader034.vdocuments.us/reader034/viewer/2022042220/5ec5e92258b9785858481ec7/html5/thumbnails/22.jpg)
Maximizeproductivityandreducecomplexitywithautomatedlifecycleclustermanagement
Controlcosts– payonlyforwhatyouusewithAuto-scaling
Controlmixedworkloads,multipleclustersanddifferentengineswithasinglecontrolpanelorRESTAPI
DataEngineersandDataAdmins
![Page 23: The Cloud Data Platform for Insights-Driven Enterprisesgo.qubole.com/rs/510-QPZ-296/images/Qubole_Oracle_WR_05... · 2020-04-23 · • Hadoop/Spark puts compute and storage togetherwithin](https://reader034.vdocuments.us/reader034/viewer/2022042220/5ec5e92258b9785858481ec7/html5/thumbnails/23.jpg)
Fasterexploration&iterationwithanagileinfrastructure
Builttoadoptexisting,new&futuretechnologies– novendorlock-in
Improveproductivitywithacollaborativeplatform
DataAnalystsandDataScientists
![Page 24: The Cloud Data Platform for Insights-Driven Enterprisesgo.qubole.com/rs/510-QPZ-296/images/Qubole_Oracle_WR_05... · 2020-04-23 · • Hadoop/Spark puts compute and storage togetherwithin](https://reader034.vdocuments.us/reader034/viewer/2022042220/5ec5e92258b9785858481ec7/html5/thumbnails/24.jpg)
Quboleauto-scalingadvantage12.5
10.0
7.5
5.0
Ten Node Cluster (fixed)
Five Node Cluster (fixed)
7 8 9 10 11 12 13 14 15 16 17 10%cheaper,but90%slower
Commands per Hour Auto-scale –Nodes per Hour
Workloadfluctuation60%ofthetime
13%faster,but32%moreexpensive
![Page 25: The Cloud Data Platform for Insights-Driven Enterprisesgo.qubole.com/rs/510-QPZ-296/images/Qubole_Oracle_WR_05... · 2020-04-23 · • Hadoop/Spark puts compute and storage togetherwithin](https://reader034.vdocuments.us/reader034/viewer/2022042220/5ec5e92258b9785858481ec7/html5/thumbnails/25.jpg)
DataflowDiagramUserAccess
QuboleUIviaBrowser
SDK
ODBC/JDBC
QuboleSaaSTier
WebServersandControlLogic
DatabaseAccountandUserSettingsDefaultHiveMetastore
Customer’sBareMetalCloudTenancy
RESTAPI
OracleBareMetalCompute
EphemeralClusters
Oracle Cloud Platform Object
Store
OracleCloudVCNCompartment
OracleUser
DB DB
OracleBareMetalCompute
OracleBareMetalCompute
OracleBareMetalCompute
OracleBareMetalComputePersistentStorage
![Page 27: The Cloud Data Platform for Insights-Driven Enterprisesgo.qubole.com/rs/510-QPZ-296/images/Qubole_Oracle_WR_05... · 2020-04-23 · • Hadoop/Spark puts compute and storage togetherwithin](https://reader034.vdocuments.us/reader034/viewer/2022042220/5ec5e92258b9785858481ec7/html5/thumbnails/27.jpg)
Thank You
GetFreeTrialGETBOOK REGISTERFORAWEBINARREGISTERFORCONFERENCE
http://bit.ly/DataOpsBook https://www.dataplatforms.com/ https://www.qubole.com/event/