happy aloha friday! · cloud customer engineer [email protected] daniel liu cloud customer...
TRANSCRIPT
![Page 1: Happy Aloha Friday! · Cloud customer Engineer robgrace@google.com Daniel Liu Cloud Customer Engineer danieltliu@google.com. ... 04 Big Data & Hadoop 05 Analyze Streaming Data in](https://reader030.vdocuments.us/reader030/viewer/2022041119/5f30d56df8de523002263829/html5/thumbnails/1.jpg)
April 24, 2020
Workshop #1Data Analytics & Visualization
Daniel Liu, Google
hacc.hawaii.gov
Happy Aloha Friday!
![Page 2: Happy Aloha Friday! · Cloud customer Engineer robgrace@google.com Daniel Liu Cloud Customer Engineer danieltliu@google.com. ... 04 Big Data & Hadoop 05 Analyze Streaming Data in](https://reader030.vdocuments.us/reader030/viewer/2022041119/5f30d56df8de523002263829/html5/thumbnails/2.jpg)
April 24, 2020
Welcome from ETS
Marc MasunoCyber Security Manager
hacc.hawaii.gov
Happy Aloha Friday!
![Page 3: Happy Aloha Friday! · Cloud customer Engineer robgrace@google.com Daniel Liu Cloud Customer Engineer danieltliu@google.com. ... 04 Big Data & Hadoop 05 Analyze Streaming Data in](https://reader030.vdocuments.us/reader030/viewer/2022041119/5f30d56df8de523002263829/html5/thumbnails/3.jpg)
Confidential & Proprietary
Logistic 01 Welcome from ETS
02 1:00 PM to 2:30 PM
03 About Google Meet
04 Introduce to the Google Team
05 Introduce to the HACC Committee
06 Workshop
06 Q & A
![Page 4: Happy Aloha Friday! · Cloud customer Engineer robgrace@google.com Daniel Liu Cloud Customer Engineer danieltliu@google.com. ... 04 Big Data & Hadoop 05 Analyze Streaming Data in](https://reader030.vdocuments.us/reader030/viewer/2022041119/5f30d56df8de523002263829/html5/thumbnails/4.jpg)
Confidential & Proprietary
Google Meet Option 1:
Join Hangouts Meet
Meeting IDmeet.google.com/odb-krud-dvu
Option 2:Phone Numbers ( US ) 475-329-7374 PIN: 438 611 547#
![Page 5: Happy Aloha Friday! · Cloud customer Engineer robgrace@google.com Daniel Liu Cloud Customer Engineer danieltliu@google.com. ... 04 Big Data & Hadoop 05 Analyze Streaming Data in](https://reader030.vdocuments.us/reader030/viewer/2022041119/5f30d56df8de523002263829/html5/thumbnails/5.jpg)
Confidential & Proprietary
Google Meet
![Page 6: Happy Aloha Friday! · Cloud customer Engineer robgrace@google.com Daniel Liu Cloud Customer Engineer danieltliu@google.com. ... 04 Big Data & Hadoop 05 Analyze Streaming Data in](https://reader030.vdocuments.us/reader030/viewer/2022041119/5f30d56df8de523002263829/html5/thumbnails/6.jpg)
The Google Team
Amanda StangeAccount Executive
Rob Grace Cloud customer Engineer
Daniel LiuCloud Customer Engineer
![Page 7: Happy Aloha Friday! · Cloud customer Engineer robgrace@google.com Daniel Liu Cloud Customer Engineer danieltliu@google.com. ... 04 Big Data & Hadoop 05 Analyze Streaming Data in](https://reader030.vdocuments.us/reader030/viewer/2022041119/5f30d56df8de523002263829/html5/thumbnails/7.jpg)
The HACC Committee
![Page 8: Happy Aloha Friday! · Cloud customer Engineer robgrace@google.com Daniel Liu Cloud Customer Engineer danieltliu@google.com. ... 04 Big Data & Hadoop 05 Analyze Streaming Data in](https://reader030.vdocuments.us/reader030/viewer/2022041119/5f30d56df8de523002263829/html5/thumbnails/8.jpg)
April 24, 202001:00 PM ~ 02:30 PMhttps://hacc.hawaii.gov/Daniel Liu, [email protected] Engineer
Google Data Analytics and Visualization Solutions Overview
![Page 9: Happy Aloha Friday! · Cloud customer Engineer robgrace@google.com Daniel Liu Cloud Customer Engineer danieltliu@google.com. ... 04 Big Data & Hadoop 05 Analyze Streaming Data in](https://reader030.vdocuments.us/reader030/viewer/2022041119/5f30d56df8de523002263829/html5/thumbnails/9.jpg)
Confidential & Proprietary
Agenda 01 Data Challenges
02 Our Approach to Data Analytics
03 Modernize Your Data Warehouse
04 Big Data & Hadoop
05 Analyze Streaming Data in Real Time
06 Data Visualization Tools
07 Predictive Analytics & Machine Learning
08 How to Get Start with GCP
![Page 10: Happy Aloha Friday! · Cloud customer Engineer robgrace@google.com Daniel Liu Cloud Customer Engineer danieltliu@google.com. ... 04 Big Data & Hadoop 05 Analyze Streaming Data in](https://reader030.vdocuments.us/reader030/viewer/2022041119/5f30d56df8de523002263829/html5/thumbnails/10.jpg)
Google Mission Statement
Organize the world’s information and make it universally accessible
and useful
![Page 11: Happy Aloha Friday! · Cloud customer Engineer robgrace@google.com Daniel Liu Cloud Customer Engineer danieltliu@google.com. ... 04 Big Data & Hadoop 05 Analyze Streaming Data in](https://reader030.vdocuments.us/reader030/viewer/2022041119/5f30d56df8de523002263829/html5/thumbnails/11.jpg)
Google Mission Statement
Organize the world’s information and make it universally accessible
and useful
![Page 12: Happy Aloha Friday! · Cloud customer Engineer robgrace@google.com Daniel Liu Cloud Customer Engineer danieltliu@google.com. ... 04 Big Data & Hadoop 05 Analyze Streaming Data in](https://reader030.vdocuments.us/reader030/viewer/2022041119/5f30d56df8de523002263829/html5/thumbnails/12.jpg)
Data Volume GrowthDigital Information Measurement Unit
![Page 13: Happy Aloha Friday! · Cloud customer Engineer robgrace@google.com Daniel Liu Cloud Customer Engineer danieltliu@google.com. ... 04 Big Data & Hadoop 05 Analyze Streaming Data in](https://reader030.vdocuments.us/reader030/viewer/2022041119/5f30d56df8de523002263829/html5/thumbnails/13.jpg)
Data Volume GrowthSurvey in 2009
● 2K - A typewritten page● 5M –The complete works of Shakespeare● 10M – One minute of high fidelity sound● 2T – Information generated on YouTube in one day● 10T – 530,000,000 miles of bookshelves at Library of congress● 20P – All hard disk drives in 1995● 700P – Data of 700,000 companies with Revenues less than $200M● 1E – Combined Fortune 1000 company database (1P each)● 1E – Next 9000 world company databases (average 100T each)● 1Z – 1000E (Zettabyte–Grains of sand on beaches)● 100Y –Yottabytes – Addressable memory 128 -bit
![Page 14: Happy Aloha Friday! · Cloud customer Engineer robgrace@google.com Daniel Liu Cloud Customer Engineer danieltliu@google.com. ... 04 Big Data & Hadoop 05 Analyze Streaming Data in](https://reader030.vdocuments.us/reader030/viewer/2022041119/5f30d56df8de523002263829/html5/thumbnails/14.jpg)
Global DatasphereSurvey by IDC
● IDC defines the "global datasphere" as "the quantification of the amount of data created, captured, and replicated across the world."
●
![Page 15: Happy Aloha Friday! · Cloud customer Engineer robgrace@google.com Daniel Liu Cloud Customer Engineer danieltliu@google.com. ... 04 Big Data & Hadoop 05 Analyze Streaming Data in](https://reader030.vdocuments.us/reader030/viewer/2022041119/5f30d56df8de523002263829/html5/thumbnails/15.jpg)
Google Mission Statement
Organize the world’s information and make it universally accessible
and useful
![Page 16: Happy Aloha Friday! · Cloud customer Engineer robgrace@google.com Daniel Liu Cloud Customer Engineer danieltliu@google.com. ... 04 Big Data & Hadoop 05 Analyze Streaming Data in](https://reader030.vdocuments.us/reader030/viewer/2022041119/5f30d56df8de523002263829/html5/thumbnails/16.jpg)
![Page 17: Happy Aloha Friday! · Cloud customer Engineer robgrace@google.com Daniel Liu Cloud Customer Engineer danieltliu@google.com. ... 04 Big Data & Hadoop 05 Analyze Streaming Data in](https://reader030.vdocuments.us/reader030/viewer/2022041119/5f30d56df8de523002263829/html5/thumbnails/17.jpg)
Core tenets
If users can’t spell, it’s our problem.
If they don’t know how to form the query, it’s our problem.
If they don’t know what words to use, it’s our problem.
If they can’t speak the language, it’s our problem.
1 2 3 4 5 6
If there is not enough content on the web, it’s our problem.
If the web is too slow, it’s our problem.
![Page 18: Happy Aloha Friday! · Cloud customer Engineer robgrace@google.com Daniel Liu Cloud Customer Engineer danieltliu@google.com. ... 04 Big Data & Hadoop 05 Analyze Streaming Data in](https://reader030.vdocuments.us/reader030/viewer/2022041119/5f30d56df8de523002263829/html5/thumbnails/18.jpg)
Confidential + Proprietary
Machine Learning is the new ground for gaining competitive edge & creating business value
*Source: MIT Survey 2017; n=375Bain Consulting Study
Competitive advantage ranked as top goal of machine-learning projects for 46% of IT
leaders & 50% of adopters can quantify ROI
2X more data-driven decisions
5X faster decisions
than others
3X faster execution
![Page 19: Happy Aloha Friday! · Cloud customer Engineer robgrace@google.com Daniel Liu Cloud Customer Engineer danieltliu@google.com. ... 04 Big Data & Hadoop 05 Analyze Streaming Data in](https://reader030.vdocuments.us/reader030/viewer/2022041119/5f30d56df8de523002263829/html5/thumbnails/19.jpg)
Confidential + Proprietary
First Step in This Journey Begins with Data
“Every Company will be a Data Company”
*Source: Wired, Bloomberg, Fortune, McKinsey
Proprietary + Confidential
![Page 20: Happy Aloha Friday! · Cloud customer Engineer robgrace@google.com Daniel Liu Cloud Customer Engineer danieltliu@google.com. ... 04 Big Data & Hadoop 05 Analyze Streaming Data in](https://reader030.vdocuments.us/reader030/viewer/2022041119/5f30d56df8de523002263829/html5/thumbnails/20.jpg)
2001Data Challenges
![Page 21: Happy Aloha Friday! · Cloud customer Engineer robgrace@google.com Daniel Liu Cloud Customer Engineer danieltliu@google.com. ... 04 Big Data & Hadoop 05 Analyze Streaming Data in](https://reader030.vdocuments.us/reader030/viewer/2022041119/5f30d56df8de523002263829/html5/thumbnails/21.jpg)
Confidential & Proprietary
Data is EverythingCompanies win or lose based on how do they use it
Governments make the right and wrong decisions based on the data they processed
You make your personal decision based on the data you collected
![Page 22: Happy Aloha Friday! · Cloud customer Engineer robgrace@google.com Daniel Liu Cloud Customer Engineer danieltliu@google.com. ... 04 Big Data & Hadoop 05 Analyze Streaming Data in](https://reader030.vdocuments.us/reader030/viewer/2022041119/5f30d56df8de523002263829/html5/thumbnails/22.jpg)
22
< %1 50
Data analytics is still too hard
* Harvard Business Review magazine; May-June 2017
< %
UnstructuredData
StructuredData
![Page 23: Happy Aloha Friday! · Cloud customer Engineer robgrace@google.com Daniel Liu Cloud Customer Engineer danieltliu@google.com. ... 04 Big Data & Hadoop 05 Analyze Streaming Data in](https://reader030.vdocuments.us/reader030/viewer/2022041119/5f30d56df8de523002263829/html5/thumbnails/23.jpg)
23
Data complexitiesUnstructured data accounts for 90% of enterprise data
*Source: IDC, Wired
1101010101001011011101
011110010001101
Legacy applications
Regulatory environment
Changing view on value of data
Limited skills, hard to recruit
Data silos everywhere
![Page 24: Happy Aloha Friday! · Cloud customer Engineer robgrace@google.com Daniel Liu Cloud Customer Engineer danieltliu@google.com. ... 04 Big Data & Hadoop 05 Analyze Streaming Data in](https://reader030.vdocuments.us/reader030/viewer/2022041119/5f30d56df8de523002263829/html5/thumbnails/24.jpg)
Confidential & Proprietary
Keep system reliable/running
Collaboration within or across organizations
Challenges with Big Data Projects
1
2
3
4
5
6
7
8
9
Keep your data secure
Finding value in existing data very easily
Reducing the time from data collection to action
Continuously accommodating greater data volumes and new data sources
Capture and store all data for all business functions
Complexity of building and maintaining a Big Data system with consistent ease of use
Hurdles to innovate and iterate with Big Data
![Page 25: Happy Aloha Friday! · Cloud customer Engineer robgrace@google.com Daniel Liu Cloud Customer Engineer danieltliu@google.com. ... 04 Big Data & Hadoop 05 Analyze Streaming Data in](https://reader030.vdocuments.us/reader030/viewer/2022041119/5f30d56df8de523002263829/html5/thumbnails/25.jpg)
Confidential & Proprietary
Keep system reliable/running
Collaboration within or across organizations
Challenges with Big Data Projects
1
2
3
4
5
6
7
8
9
Keep your data secure
Finding value in existing data very easily
Reducing the time from data collection to action
Continuously accommodating greater data volumes and new data sources
Capture and store all data for all business functions
Complexity of building and maintaining a Big Data system with consistent ease of use
Hurdles to innovate and iterate with Big Data
![Page 26: Happy Aloha Friday! · Cloud customer Engineer robgrace@google.com Daniel Liu Cloud Customer Engineer danieltliu@google.com. ... 04 Big Data & Hadoop 05 Analyze Streaming Data in](https://reader030.vdocuments.us/reader030/viewer/2022041119/5f30d56df8de523002263829/html5/thumbnails/26.jpg)
Confidential & Proprietary
If you want to unlock the power of your data, you need a customer data platform, not just new tools.
![Page 27: Happy Aloha Friday! · Cloud customer Engineer robgrace@google.com Daniel Liu Cloud Customer Engineer danieltliu@google.com. ... 04 Big Data & Hadoop 05 Analyze Streaming Data in](https://reader030.vdocuments.us/reader030/viewer/2022041119/5f30d56df8de523002263829/html5/thumbnails/27.jpg)
If Your Organization Isn’t Good at
Analytics, It’s Not Ready for AI
“”
Proprietary + Confidential
*Source: Harvard Business Review
![Page 28: Happy Aloha Friday! · Cloud customer Engineer robgrace@google.com Daniel Liu Cloud Customer Engineer danieltliu@google.com. ... 04 Big Data & Hadoop 05 Analyze Streaming Data in](https://reader030.vdocuments.us/reader030/viewer/2022041119/5f30d56df8de523002263829/html5/thumbnails/28.jpg)
28
02
Our Approach to Data Analytics
![Page 29: Happy Aloha Friday! · Cloud customer Engineer robgrace@google.com Daniel Liu Cloud Customer Engineer danieltliu@google.com. ... 04 Big Data & Hadoop 05 Analyze Streaming Data in](https://reader030.vdocuments.us/reader030/viewer/2022041119/5f30d56df8de523002263829/html5/thumbnails/29.jpg)
29
15+ Years of Tackling Big Data Problems
Google Papers
20082002 2004 2006 2010 2012 2014 2015
GFS MapReduce
OpenSource
2005
GoogleCloudProducts
BigTable Spanner
2016
Millwheel TensorflowDataflowFlume JavaDremel
![Page 30: Happy Aloha Friday! · Cloud customer Engineer robgrace@google.com Daniel Liu Cloud Customer Engineer danieltliu@google.com. ... 04 Big Data & Hadoop 05 Analyze Streaming Data in](https://reader030.vdocuments.us/reader030/viewer/2022041119/5f30d56df8de523002263829/html5/thumbnails/30.jpg)
30
15 Years of Tackling Big Data Problems
Google Papers
20082002 2004 2006 2010 2012 2014 2015
GFS MapReduce
OpenSource
2005
GoogleCloudProducts
BigTable Millwheel TensorflowSpanner
2016
DataflowFlume JavaDremel
![Page 31: Happy Aloha Friday! · Cloud customer Engineer robgrace@google.com Daniel Liu Cloud Customer Engineer danieltliu@google.com. ... 04 Big Data & Hadoop 05 Analyze Streaming Data in](https://reader030.vdocuments.us/reader030/viewer/2022041119/5f30d56df8de523002263829/html5/thumbnails/31.jpg)
31
15 Years of Tackling Big Data Problems
Google Papers
20082002 2004 2006 2010 2012 2014 2015
GFS MapReduce Flume Java
OpenSource
2005
GoogleCloudProducts BigQuery Pub/Sub Dataflow Bigtable
BigTable Dremel Spanner
ML
2016
Millwheel TensorflowDataflow
![Page 32: Happy Aloha Friday! · Cloud customer Engineer robgrace@google.com Daniel Liu Cloud Customer Engineer danieltliu@google.com. ... 04 Big Data & Hadoop 05 Analyze Streaming Data in](https://reader030.vdocuments.us/reader030/viewer/2022041119/5f30d56df8de523002263829/html5/thumbnails/32.jpg)
32
Serverless data analyticsFrom infrastructure to platform for insights
Performance tuning
Monitoring
ReliabilityDeployment & configuration
Utilization improvements
Analysis and insights
Resource provisioning
Handling growing scale
Analysis and insights
![Page 33: Happy Aloha Friday! · Cloud customer Engineer robgrace@google.com Daniel Liu Cloud Customer Engineer danieltliu@google.com. ... 04 Big Data & Hadoop 05 Analyze Streaming Data in](https://reader030.vdocuments.us/reader030/viewer/2022041119/5f30d56df8de523002263829/html5/thumbnails/33.jpg)
Data Silos and Legacy
Systems
Limits decision-making and is time consuming
Lacks How-To Predict Business
Outcomes
Depends on guts for predicting the unknown
Enterprise Challenges in Data to ML Journey
Missing Out on Real-Time
Insights
Rear-view approach causes business anxiety
Proprietary + Confidential
![Page 34: Happy Aloha Friday! · Cloud customer Engineer robgrace@google.com Daniel Liu Cloud Customer Engineer danieltliu@google.com. ... 04 Big Data & Hadoop 05 Analyze Streaming Data in](https://reader030.vdocuments.us/reader030/viewer/2022041119/5f30d56df8de523002263829/html5/thumbnails/34.jpg)
Data Silos and Legacy
system limitations
Predicting unknown business outcomes
Key Solutions Powered by
Predictive Analytics / ML
Anticipate customer needs and automate delivery with Machine
Intelligence
Missing out on real-time
insights because of rear-view
approach
Streaming Data Analytics
Process Streaming Data along with batch
data to generate real-time insights
Cloud Data Warehouse
Modern Data Warehousing which
builds foundation for AI
Proprietary + Confidential
![Page 35: Happy Aloha Friday! · Cloud customer Engineer robgrace@google.com Daniel Liu Cloud Customer Engineer danieltliu@google.com. ... 04 Big Data & Hadoop 05 Analyze Streaming Data in](https://reader030.vdocuments.us/reader030/viewer/2022041119/5f30d56df8de523002263829/html5/thumbnails/35.jpg)
35
Complete foundation for data lifecycle
Data ingestionat any scale
Reliable streaming data pipeline
Advanced analyticsData warehousing and data lake
SheetsApache Beam
Cloud Pub/Sub Cloud Dataflow Cloud Dataproc(Hadoop, Spark) BigQuery Cloud Storage Cloud ML Engine Google Data StudioData Transfer Service
Tensorflow
Cloud Composer(Apache Airflow)
Cloud IoT Core Cloud Dataprep(Trifacta)
![Page 36: Happy Aloha Friday! · Cloud customer Engineer robgrace@google.com Daniel Liu Cloud Customer Engineer danieltliu@google.com. ... 04 Big Data & Hadoop 05 Analyze Streaming Data in](https://reader030.vdocuments.us/reader030/viewer/2022041119/5f30d56df8de523002263829/html5/thumbnails/36.jpg)
3603Modernize Your Data Warehouse Get all your business data in one place for faster and comprehensive analysis
![Page 37: Happy Aloha Friday! · Cloud customer Engineer robgrace@google.com Daniel Liu Cloud Customer Engineer danieltliu@google.com. ... 04 Big Data & Hadoop 05 Analyze Streaming Data in](https://reader030.vdocuments.us/reader030/viewer/2022041119/5f30d56df8de523002263829/html5/thumbnails/37.jpg)
37
Data warehouses
From 1st-gen EDWs, increased data collection and analysis has helped build more data-driven
businesses.
90’s 00’s
BI foundations
Data warehousing formed the foundation of reporting and business intelligence.
Cloud data warehousingBigQuery represents
a fundamentally different approach to cloud data
warehousing.
Now
AI foundations
We’re working to make BigQuery the foundation for organizations that will
leverage machine intelligence in their
businesses.
Next
Data warehousing for AI-driven business
![Page 38: Happy Aloha Friday! · Cloud customer Engineer robgrace@google.com Daniel Liu Cloud Customer Engineer danieltliu@google.com. ... 04 Big Data & Hadoop 05 Analyze Streaming Data in](https://reader030.vdocuments.us/reader030/viewer/2022041119/5f30d56df8de523002263829/html5/thumbnails/38.jpg)
BigQuery
ETL
Storage
Analyze
Cloud Storage
Cloud Dataflow
Relational Data
Google Cloud Data Warehouse: Four Typical Flows
Data
Proprietary + Confidential
![Page 39: Happy Aloha Friday! · Cloud customer Engineer robgrace@google.com Daniel Liu Cloud Customer Engineer danieltliu@google.com. ... 04 Big Data & Hadoop 05 Analyze Streaming Data in](https://reader030.vdocuments.us/reader030/viewer/2022041119/5f30d56df8de523002263829/html5/thumbnails/39.jpg)
39
What is BigQuery?
Convenience of standard SQL
Fully managed and serverless
Google Cloud Platform’s enterprise data warehouse for analytics
Encrypted, durable and highly available
Petabyte-scale storage and queries
Real-time analytics on streaming data
![Page 40: Happy Aloha Friday! · Cloud customer Engineer robgrace@google.com Daniel Liu Cloud Customer Engineer danieltliu@google.com. ... 04 Big Data & Hadoop 05 Analyze Streaming Data in](https://reader030.vdocuments.us/reader030/viewer/2022041119/5f30d56df8de523002263829/html5/thumbnails/40.jpg)
40
BigQuery: architectureServerless. Decoupled storage and compute for maximum flexibility.
SQL:2011Compliant
Petabit network
BigQuery High-available cluster compute
(Dremel)Streaming ingest
Free bulk loading
Replicated, distributed storage
(99.9999999999% durability) REST API
Client libraries
In 7 languages
Web UI, CLIDistributed memory
shuffle tier
![Page 41: Happy Aloha Friday! · Cloud customer Engineer robgrace@google.com Daniel Liu Cloud Customer Engineer danieltliu@google.com. ... 04 Big Data & Hadoop 05 Analyze Streaming Data in](https://reader030.vdocuments.us/reader030/viewer/2022041119/5f30d56df8de523002263829/html5/thumbnails/41.jpg)
41
Introducing BigQuery MLMaking machine learning accessible
![Page 42: Happy Aloha Friday! · Cloud customer Engineer robgrace@google.com Daniel Liu Cloud Customer Engineer danieltliu@google.com. ... 04 Big Data & Hadoop 05 Analyze Streaming Data in](https://reader030.vdocuments.us/reader030/viewer/2022041119/5f30d56df8de523002263829/html5/thumbnails/42.jpg)
42
BigQuery MLempowers data
analysts anddata scientists
Execute ML initiatives without moving data from BigQuery
Iterate on models in SQL in BigQuery to increase development speed
Automate model selection, and hypertuning
![Page 43: Happy Aloha Friday! · Cloud customer Engineer robgrace@google.com Daniel Liu Cloud Customer Engineer danieltliu@google.com. ... 04 Big Data & Hadoop 05 Analyze Streaming Data in](https://reader030.vdocuments.us/reader030/viewer/2022041119/5f30d56df8de523002263829/html5/thumbnails/43.jpg)
43
![Page 44: Happy Aloha Friday! · Cloud customer Engineer robgrace@google.com Daniel Liu Cloud Customer Engineer danieltliu@google.com. ... 04 Big Data & Hadoop 05 Analyze Streaming Data in](https://reader030.vdocuments.us/reader030/viewer/2022041119/5f30d56df8de523002263829/html5/thumbnails/44.jpg)
44
Accurate spatial analyses with Geography data type over GeoJSON and WKT formats
Support for core GIS functions – measurements, transforms, constructors, etc... – using familiar SQL
Analyze GIS data in BigQuery with familiar SQL
![Page 45: Happy Aloha Friday! · Cloud customer Engineer robgrace@google.com Daniel Liu Cloud Customer Engineer danieltliu@google.com. ... 04 Big Data & Hadoop 05 Analyze Streaming Data in](https://reader030.vdocuments.us/reader030/viewer/2022041119/5f30d56df8de523002263829/html5/thumbnails/45.jpg)
45
Unlock big data for all users with BigQuery & Sheets
“For analysts spread across the globe, this is a blessing. They can now collaborate easily with a streamlined flow for sharing their insights.” -- Nikhil Mishra @ Yahoo
gsuite.google.com/bq-sheets
![Page 46: Happy Aloha Friday! · Cloud customer Engineer robgrace@google.com Daniel Liu Cloud Customer Engineer danieltliu@google.com. ... 04 Big Data & Hadoop 05 Analyze Streaming Data in](https://reader030.vdocuments.us/reader030/viewer/2022041119/5f30d56df8de523002263829/html5/thumbnails/46.jpg)
46
See your BigQuery data in one click with Data Studio Explorer Tight integration in the BigQuery UI brings visual exploration of your query results in one simple click.
![Page 47: Happy Aloha Friday! · Cloud customer Engineer robgrace@google.com Daniel Liu Cloud Customer Engineer danieltliu@google.com. ... 04 Big Data & Hadoop 05 Analyze Streaming Data in](https://reader030.vdocuments.us/reader030/viewer/2022041119/5f30d56df8de523002263829/html5/thumbnails/47.jpg)
47
Firebase Export
GA360 Export
Google BQ-Data Transfer Service
Partners
You can use BigQuery to build a modern marketing data warehouse
Salesforce, Marketo, Facebook, Twitter,
CRM data etc...
ML
BigQuery
Dataprep Dataflow
DataStudio
DataLab
Extract Transform
Load
Visualization
![Page 48: Happy Aloha Friday! · Cloud customer Engineer robgrace@google.com Daniel Liu Cloud Customer Engineer danieltliu@google.com. ... 04 Big Data & Hadoop 05 Analyze Streaming Data in](https://reader030.vdocuments.us/reader030/viewer/2022041119/5f30d56df8de523002263829/html5/thumbnails/48.jpg)
4804Big Data & Hadoop
![Page 49: Happy Aloha Friday! · Cloud customer Engineer robgrace@google.com Daniel Liu Cloud Customer Engineer danieltliu@google.com. ... 04 Big Data & Hadoop 05 Analyze Streaming Data in](https://reader030.vdocuments.us/reader030/viewer/2022041119/5f30d56df8de523002263829/html5/thumbnails/49.jpg)
Production implementations of hadoop, spark, and other components (like hive) are growing steadily over time
Hadoop will likely be a $50b market by 2020
Hadoop is used by 80% of the Fortune 500
Hadoop Ecosystem is a Popular Choice
Proprietary + Confidential
Source: Lorem ipsum, 2017
![Page 50: Happy Aloha Friday! · Cloud customer Engineer robgrace@google.com Daniel Liu Cloud Customer Engineer danieltliu@google.com. ... 04 Big Data & Hadoop 05 Analyze Streaming Data in](https://reader030.vdocuments.us/reader030/viewer/2022041119/5f30d56df8de523002263829/html5/thumbnails/50.jpg)
Managing Hadoop can be … Complex
Proprietary + Confidential
…and growing
BATCH PROCESSING
(MR2)
SEARCHENGINE
(Soir)
MACHINELEARNING
(Spark)
STREAMPROCESSING
(Spark)
3rd PARTYAPPS
WORKLOAD MANAGEMENT (YARN)
DATAM
ANAG
EMEN
T
ANALYTICSQL
(Impala)
Hadoop engineers are difficult to find and expensive to employ
Analytic use cases require more RAM on Hadoop nodes
(128GB+=hw Refresh)
Balancing custer resources requires constant work/waiting
Lack of elasticity creates challenges adding customers, new use cases and
exploring new ideas
New services pushed by Hadoop vendors compete for resources and add complexity
![Page 51: Happy Aloha Friday! · Cloud customer Engineer robgrace@google.com Daniel Liu Cloud Customer Engineer danieltliu@google.com. ... 04 Big Data & Hadoop 05 Analyze Streaming Data in](https://reader030.vdocuments.us/reader030/viewer/2022041119/5f30d56df8de523002263829/html5/thumbnails/51.jpg)
How can you simplify Hadoop management?
Proprietary + Confidential
![Page 52: Happy Aloha Friday! · Cloud customer Engineer robgrace@google.com Daniel Liu Cloud Customer Engineer danieltliu@google.com. ... 04 Big Data & Hadoop 05 Analyze Streaming Data in](https://reader030.vdocuments.us/reader030/viewer/2022041119/5f30d56df8de523002263829/html5/thumbnails/52.jpg)
Confidential & Proprietary
Bigtable (NoSQL Database -
HBase, Cassandra)
Hadoop to Cloud: options
Complex Simple
BigQuery (Dremel - Cloud DW)
Dataflow (Apache Beam
- ETL, Streaming)
Dataproc (Hadoop, Spark - BigData)
PubSub (Apache Kafka
- Global-scale Messaging)
Compute Engine
Storage
managed by you managed by Google
![Page 53: Happy Aloha Friday! · Cloud customer Engineer robgrace@google.com Daniel Liu Cloud Customer Engineer danieltliu@google.com. ... 04 Big Data & Hadoop 05 Analyze Streaming Data in](https://reader030.vdocuments.us/reader030/viewer/2022041119/5f30d56df8de523002263829/html5/thumbnails/53.jpg)
53
Faster and easier Spark & Hadoop jobs with Cloud Dataproc
![Page 54: Happy Aloha Friday! · Cloud customer Engineer robgrace@google.com Daniel Liu Cloud Customer Engineer danieltliu@google.com. ... 04 Big Data & Hadoop 05 Analyze Streaming Data in](https://reader030.vdocuments.us/reader030/viewer/2022041119/5f30d56df8de523002263829/html5/thumbnails/54.jpg)
54
Cloud DataprocIt is the simpler, more cost-efficient way to make your Apache Spark & Hadoop (MapReduce) deployments a success
It’s flexibleCreate and resize managed Hadoop and Spark clusters in less than 90 seconds
It’s easyLift and shift existing projects or ETL pipelines, no redevelopment necessary
It’s cost effectiveEasily process large datasets at low cost, pay only for the resources you use (by the minute)
It’s openLeverage tools, libraries, and documentation from the Spark and Hadoop ecosystem
![Page 55: Happy Aloha Friday! · Cloud customer Engineer robgrace@google.com Daniel Liu Cloud Customer Engineer danieltliu@google.com. ... 04 Big Data & Hadoop 05 Analyze Streaming Data in](https://reader030.vdocuments.us/reader030/viewer/2022041119/5f30d56df8de523002263829/html5/thumbnails/55.jpg)
55
Integrates easily with other Cloud Platform services
Share data across the platformConnectors available to read/write from BigQuery, Cloud Bigtable, and Cloud Storage
Match processing with dataBring the right processing engine to the workload (and at right cost) on the same storage
Monitor & alert with StackdriverIntegration with Stackdriver, GCP’s logging & monitoring framework, lets you identify/diagnose issues
Clients
ClustersCloud Dataproc
Logging & monitoringStackdriver
Spark
PySpark
Spark SQL
Hadoop
Hive
Pig
Jobs APIGoogle Cloud Storage
Google Cloud Bigtable
Google BigQuery
Data Sources/Sinks
![Page 56: Happy Aloha Friday! · Cloud customer Engineer robgrace@google.com Daniel Liu Cloud Customer Engineer danieltliu@google.com. ... 04 Big Data & Hadoop 05 Analyze Streaming Data in](https://reader030.vdocuments.us/reader030/viewer/2022041119/5f30d56df8de523002263829/html5/thumbnails/56.jpg)
5605Analyze Streaming Data in Real Time
Gain real-time business insights andmake your business more responsive
![Page 57: Happy Aloha Friday! · Cloud customer Engineer robgrace@google.com Daniel Liu Cloud Customer Engineer danieltliu@google.com. ... 04 Big Data & Hadoop 05 Analyze Streaming Data in](https://reader030.vdocuments.us/reader030/viewer/2022041119/5f30d56df8de523002263829/html5/thumbnails/57.jpg)
57
Stream data analytics on Google Cloud Platform
Machine learning & data warehouse
Ingest and distributedata reliably
Fast, correct computations quickly and simply
![Page 58: Happy Aloha Friday! · Cloud customer Engineer robgrace@google.com Daniel Liu Cloud Customer Engineer danieltliu@google.com. ... 04 Big Data & Hadoop 05 Analyze Streaming Data in](https://reader030.vdocuments.us/reader030/viewer/2022041119/5f30d56df8de523002263829/html5/thumbnails/58.jpg)
58
Unified streaming and batch data processing with Cloud Dataflow
![Page 59: Happy Aloha Friday! · Cloud customer Engineer robgrace@google.com Daniel Liu Cloud Customer Engineer danieltliu@google.com. ... 04 Big Data & Hadoop 05 Analyze Streaming Data in](https://reader030.vdocuments.us/reader030/viewer/2022041119/5f30d56df8de523002263829/html5/thumbnails/59.jpg)
59
Cloud DataflowThe fully-managed data processing service that simplifies development and management of stream and batch pipelines
Accelerate development for streaming & batchFast, simplified data pipeline development via expressive Java and Python APIs in the Apache Beam SDK
Simplified management and operationsRemove operational overhead by letting Cloud Dataflow auto-manage performance, scaling, availability, security and compliance.
Build on a foundation for machine learningAdd TensorFlow-based Cloud Machine Learning models and APIs to your data processing pipelines for real-time predictions
![Page 60: Happy Aloha Friday! · Cloud customer Engineer robgrace@google.com Daniel Liu Cloud Customer Engineer danieltliu@google.com. ... 04 Big Data & Hadoop 05 Analyze Streaming Data in](https://reader030.vdocuments.us/reader030/viewer/2022041119/5f30d56df8de523002263829/html5/thumbnails/60.jpg)
60
Fully-managed and serverless
Team-based data wrangling Share & copy flows to collaborate in real-time, reuse custom samples in a recipe, and audit individual user wrangling steps
Data analyst productivityUse shortcuts for popular transforms (pivots, joins, unions), simplify date formatting, and parameterize scheduled flows with dynamic data
Comprehensive design refreshEnsure smooth onboarding, see activity organized by recency and navigate easily between different stages of the workflow
Visually prepare data for analysis or ML projectswith Cloud Dataprep
![Page 61: Happy Aloha Friday! · Cloud customer Engineer robgrace@google.com Daniel Liu Cloud Customer Engineer danieltliu@google.com. ... 04 Big Data & Hadoop 05 Analyze Streaming Data in](https://reader030.vdocuments.us/reader030/viewer/2022041119/5f30d56df8de523002263829/html5/thumbnails/61.jpg)
61
Collaborative Business Intelligence with Data StudioOne-click visualization
Visualize BigQuery data in a click with Data Studio Explorer
Data Studio data blending
Join across data sources with a simple right-click
Data Studio template gallery
Get started fast using Google-built or community-built templates.
Data Studio customVisualizations DEVELOPER PREVIEW
Build custom visualizations using the popular D3.js framework.
![Page 62: Happy Aloha Friday! · Cloud customer Engineer robgrace@google.com Daniel Liu Cloud Customer Engineer danieltliu@google.com. ... 04 Big Data & Hadoop 05 Analyze Streaming Data in](https://reader030.vdocuments.us/reader030/viewer/2022041119/5f30d56df8de523002263829/html5/thumbnails/62.jpg)
62
More from the tools you already usePrep / processing Services partnersData ingest Data integration BI/visualizationData management
![Page 63: Happy Aloha Friday! · Cloud customer Engineer robgrace@google.com Daniel Liu Cloud Customer Engineer danieltliu@google.com. ... 04 Big Data & Hadoop 05 Analyze Streaming Data in](https://reader030.vdocuments.us/reader030/viewer/2022041119/5f30d56df8de523002263829/html5/thumbnails/63.jpg)
63
Serverless platform & auto-optimized usage across the entire data lifecycle
Cloud Pub/Sub
Cloud Storage(files)
DataScientists
BusinessAnalysts
Cloud Dataproc
Stackdriver
Firebase
Storage Transfer Service
Cloud Dataflow
ML Engine Cloud Datalab
Cloud Bigtable(NoSQL)
SQL
GA 360
Adwords
DoubleClick
YouTube
Cloud Dataflow
Use
BigQuery analyticsBigQuery storage(tables)
AnalyzeStoreCapture Process
Cloud Dataprep
Cloud Composer (orchestration)
Cloud Dataproc
![Page 64: Happy Aloha Friday! · Cloud customer Engineer robgrace@google.com Daniel Liu Cloud Customer Engineer danieltliu@google.com. ... 04 Big Data & Hadoop 05 Analyze Streaming Data in](https://reader030.vdocuments.us/reader030/viewer/2022041119/5f30d56df8de523002263829/html5/thumbnails/64.jpg)
64
The Forrester Wave™: Insight Platforms-As-A-Service, Q3 2017.. The Forrester Wave™ is copyrighted by Forrester Research, Inc. Forrester and Forrester Wave™ are trademarks of Forrester Research, Inc. The Forrester Wave™ is a graphical representation of Forrester's call on a market and is plotted using a detailed spreadsheet with exposed scores, weightings, and comments. Forrester does not endorse any vendor, product, or service depicted in the Forrester Wave. Information is based on best available resources. Opinions reflect judgment at the time and are subject to change.
“Our evaluation identified one vendor as a Leader based on the strength of its PaaS strategy, advancedtools for batch and real-time solutions, and machine learning and AI offerings.”
— The Forrester Report
● Google has the highest scores in the Current Offering and Strategy categories.
● Noted as the only vendor in the evaluation to offer insight execution features like full machine learning automation with hyperparameter tuning, container management, and API management.
● Receives recognition for advanced platform features like autoscaling for most of its services, efforts at integrating leading Hadoop cloud services and its data flow service works on both batch and streaming data.
Google: a leader in insight platforms-as-a-service
![Page 65: Happy Aloha Friday! · Cloud customer Engineer robgrace@google.com Daniel Liu Cloud Customer Engineer danieltliu@google.com. ... 04 Big Data & Hadoop 05 Analyze Streaming Data in](https://reader030.vdocuments.us/reader030/viewer/2022041119/5f30d56df8de523002263829/html5/thumbnails/65.jpg)
6506Data Visualization
![Page 66: Happy Aloha Friday! · Cloud customer Engineer robgrace@google.com Daniel Liu Cloud Customer Engineer danieltliu@google.com. ... 04 Big Data & Hadoop 05 Analyze Streaming Data in](https://reader030.vdocuments.us/reader030/viewer/2022041119/5f30d56df8de523002263829/html5/thumbnails/66.jpg)
1 in 2customers integrate insights/experiences
beyond Looker
2000+Organizations
5000+Global
Developers
900+Looker
Specialists
Santa Cruz
San Francisco New YorkChicago
Boulder Tokyo
Dublin London
Empower people with the smarter use of data
![Page 67: Happy Aloha Friday! · Cloud customer Engineer robgrace@google.com Daniel Liu Cloud Customer Engineer danieltliu@google.com. ... 04 Big Data & Hadoop 05 Analyze Streaming Data in](https://reader030.vdocuments.us/reader030/viewer/2022041119/5f30d56df8de523002263829/html5/thumbnails/67.jpg)
The Platform forData Experiences
Google Cloud’s cloud-native Enterprise BI Platform enabling secure access to near real-time data when and where you need it.
![Page 68: Happy Aloha Friday! · Cloud customer Engineer robgrace@google.com Daniel Liu Cloud Customer Engineer danieltliu@google.com. ... 04 Big Data & Hadoop 05 Analyze Streaming Data in](https://reader030.vdocuments.us/reader030/viewer/2022041119/5f30d56df8de523002263829/html5/thumbnails/68.jpg)
Looker Data PlatformIn-database architecture | Semantic modeling layer | API-first & cloud native
Data-drivenWorkflows
CustomApplications
IntegratedInsights
Modern BI & Analytics
Serve up real-time, relevant reports and
dashboards that act as starting points for
more in-depth analysis
Super-charge operational workflows
with complete, near-real time data
Build purpose-specific tools to deliver data in an experience tailored
to the job
Infuse relevant information into the tools and products people already use
![Page 69: Happy Aloha Friday! · Cloud customer Engineer robgrace@google.com Daniel Liu Cloud Customer Engineer danieltliu@google.com. ... 04 Big Data & Hadoop 05 Analyze Streaming Data in](https://reader030.vdocuments.us/reader030/viewer/2022041119/5f30d56df8de523002263829/html5/thumbnails/69.jpg)
6907Predictive Analytics & Machine Learning
![Page 70: Happy Aloha Friday! · Cloud customer Engineer robgrace@google.com Daniel Liu Cloud Customer Engineer danieltliu@google.com. ... 04 Big Data & Hadoop 05 Analyze Streaming Data in](https://reader030.vdocuments.us/reader030/viewer/2022041119/5f30d56df8de523002263829/html5/thumbnails/70.jpg)
Remove Gut from Business Decision by using Machine Learning
“Faster” → “Confident” → “Real-time” Predictive Business Outcomes
Proprietary + Confidential
![Page 71: Happy Aloha Friday! · Cloud customer Engineer robgrace@google.com Daniel Liu Cloud Customer Engineer danieltliu@google.com. ... 04 Big Data & Hadoop 05 Analyze Streaming Data in](https://reader030.vdocuments.us/reader030/viewer/2022041119/5f30d56df8de523002263829/html5/thumbnails/71.jpg)
What is Machine Learning?
![Page 72: Happy Aloha Friday! · Cloud customer Engineer robgrace@google.com Daniel Liu Cloud Customer Engineer danieltliu@google.com. ... 04 Big Data & Hadoop 05 Analyze Streaming Data in](https://reader030.vdocuments.us/reader030/viewer/2022041119/5f30d56df8de523002263829/html5/thumbnails/72.jpg)
ML ExtendedPROPRIETARY + CONFIDENTIAL
![Page 73: Happy Aloha Friday! · Cloud customer Engineer robgrace@google.com Daniel Liu Cloud Customer Engineer danieltliu@google.com. ... 04 Big Data & Hadoop 05 Analyze Streaming Data in](https://reader030.vdocuments.us/reader030/viewer/2022041119/5f30d56df8de523002263829/html5/thumbnails/73.jpg)
ML ExtendedPROPRIETARY + CONFIDENTIAL
![Page 74: Happy Aloha Friday! · Cloud customer Engineer robgrace@google.com Daniel Liu Cloud Customer Engineer danieltliu@google.com. ... 04 Big Data & Hadoop 05 Analyze Streaming Data in](https://reader030.vdocuments.us/reader030/viewer/2022041119/5f30d56df8de523002263829/html5/thumbnails/74.jpg)
ML ExtendedPROPRIETARY + CONFIDENTIAL
![Page 75: Happy Aloha Friday! · Cloud customer Engineer robgrace@google.com Daniel Liu Cloud Customer Engineer danieltliu@google.com. ... 04 Big Data & Hadoop 05 Analyze Streaming Data in](https://reader030.vdocuments.us/reader030/viewer/2022041119/5f30d56df8de523002263829/html5/thumbnails/75.jpg)
Machine Learning systems might:
● Label or classify data
● Predict numerical values
● Cluster similar pieces of data together
● Infer association patterns in data
● Create complex outputs
![Page 76: Happy Aloha Friday! · Cloud customer Engineer robgrace@google.com Daniel Liu Cloud Customer Engineer danieltliu@google.com. ... 04 Big Data & Hadoop 05 Analyze Streaming Data in](https://reader030.vdocuments.us/reader030/viewer/2022041119/5f30d56df8de523002263829/html5/thumbnails/76.jpg)
Machine Learning could be used for early dementia
diagnosis
Automating drone-based wildlife surveys saves time and money
Examples of Machine Learning
Read a couple of news articles involving applications of ML.
1. Would a traditional programming solution be more efficient?
2. Could a human perform the same task in less time?
3. What are the benefits of a Machine Learning model in these instances?
![Page 77: Happy Aloha Friday! · Cloud customer Engineer robgrace@google.com Daniel Liu Cloud Customer Engineer danieltliu@google.com. ... 04 Big Data & Hadoop 05 Analyze Streaming Data in](https://reader030.vdocuments.us/reader030/viewer/2022041119/5f30d56df8de523002263829/html5/thumbnails/77.jpg)
Examples of Machine LearningPROPRIETARY + CONFIDENTIAL
Machine Learning & Bias
Discussion points:
- Initial thoughts? - How can we be mindful of
this moving forward?
![Page 78: Happy Aloha Friday! · Cloud customer Engineer robgrace@google.com Daniel Liu Cloud Customer Engineer danieltliu@google.com. ... 04 Big Data & Hadoop 05 Analyze Streaming Data in](https://reader030.vdocuments.us/reader030/viewer/2022041119/5f30d56df8de523002263829/html5/thumbnails/78.jpg)
Machine Learning Allows You to Solve a Problem Without Codifying the Solution
Proprietary + Confidential
✓ Recognizes patterns in data
✓ Predictive analytics at scale
✓ Builds ML models seamlessly
✓ Fully managed service
✓ Deep Learning capabilitiesGoogle Cloud AI
![Page 79: Happy Aloha Friday! · Cloud customer Engineer robgrace@google.com Daniel Liu Cloud Customer Engineer danieltliu@google.com. ... 04 Big Data & Hadoop 05 Analyze Streaming Data in](https://reader030.vdocuments.us/reader030/viewer/2022041119/5f30d56df8de523002263829/html5/thumbnails/79.jpg)
Confidential + Proprietary
Industry Use-casesIn-loop inferencing for trained models
Cloud AI productsPre-trained ML APIs to
Building custom ML models
ML FrameworkIndustry-standard & widely adopted
InfrastructureBest-in class processors for ML/DL
Proprietary + Confidential
Google Cloud End-to End AI PlatformAccelerate Business Outcomes with Enterprise-Ready Machine Learning Pipeline
CPU GPU TPU
Risk Analysis
Customer Segmentation
Predictive Inventory
Management
Fraud Detection
Demand Forecast
RecommendationEngine
Targeted marketing
Predictive Analytics
![Page 80: Happy Aloha Friday! · Cloud customer Engineer robgrace@google.com Daniel Liu Cloud Customer Engineer danieltliu@google.com. ... 04 Big Data & Hadoop 05 Analyze Streaming Data in](https://reader030.vdocuments.us/reader030/viewer/2022041119/5f30d56df8de523002263829/html5/thumbnails/80.jpg)
Google Cloud TPU
Proprietary + Confidential
![Page 81: Happy Aloha Friday! · Cloud customer Engineer robgrace@google.com Daniel Liu Cloud Customer Engineer danieltliu@google.com. ... 04 Big Data & Hadoop 05 Analyze Streaming Data in](https://reader030.vdocuments.us/reader030/viewer/2022041119/5f30d56df8de523002263829/html5/thumbnails/81.jpg)
App DeveloperData Scientist
Build custom modelsUse/extend OSS SDK Use pre-built models
ML researcher
Cloud MLE ML Perception services
End to End: Google Cloud AI Spectrum
Proprietary + Confidential
![Page 82: Happy Aloha Friday! · Cloud customer Engineer robgrace@google.com Daniel Liu Cloud Customer Engineer danieltliu@google.com. ... 04 Big Data & Hadoop 05 Analyze Streaming Data in](https://reader030.vdocuments.us/reader030/viewer/2022041119/5f30d56df8de523002263829/html5/thumbnails/82.jpg)
Proprietary + Confidential
Cloud MachineLearning Engine
Fully Managed ML Infrastructure
Train Your Own Models
![Page 83: Happy Aloha Friday! · Cloud customer Engineer robgrace@google.com Daniel Liu Cloud Customer Engineer danieltliu@google.com. ... 04 Big Data & Hadoop 05 Analyze Streaming Data in](https://reader030.vdocuments.us/reader030/viewer/2022041119/5f30d56df8de523002263829/html5/thumbnails/83.jpg)
© 2017 Google Inc. All rights reserved.
Flow to build a custom ML model
Identify business problem
Develop hypothesis
Acquire + explore data
Build amodel
Train the model
Apply andscale
1 2 3 4 5 6
![Page 84: Happy Aloha Friday! · Cloud customer Engineer robgrace@google.com Daniel Liu Cloud Customer Engineer danieltliu@google.com. ... 04 Big Data & Hadoop 05 Analyze Streaming Data in](https://reader030.vdocuments.us/reader030/viewer/2022041119/5f30d56df8de523002263829/html5/thumbnails/84.jpg)
Proprietary + Confidential
ML Perception Services
Cloud Speech API
Cloud Vision API
Cloud Translation API
Cloud Natural Language API
Cloud Video Intelligence
![Page 85: Happy Aloha Friday! · Cloud customer Engineer robgrace@google.com Daniel Liu Cloud Customer Engineer danieltliu@google.com. ... 04 Big Data & Hadoop 05 Analyze Streaming Data in](https://reader030.vdocuments.us/reader030/viewer/2022041119/5f30d56df8de523002263829/html5/thumbnails/85.jpg)
Proprietary + Confidential
SearchSearch RankingSpeech Recognition
AndroidKeyboard and Speech Input
PlayApp Recommendations Game Developer Experience
GmailSmart ReplySpam Classification
DriveIntelligence in Apps
ChromeSearch by Image
PhotosPhotos Search
YouTubeVideo Recommendations Better Thumbnails
MapsStreet View ImageParsing Local Search
TranslateText, Graphic and Speech Translations
CardboardSmart Stitching
AdsRicher Text AdsAutomated Bidding
Self Driving Car1.5MM miles driven
Data Center Reduced cooling energy usahe
Alpha GoFirst AI to beat a world Go champion (2016)
Google is a world leader in applying Machine Learning to real-world situations, inside and outside of Google.
![Page 86: Happy Aloha Friday! · Cloud customer Engineer robgrace@google.com Daniel Liu Cloud Customer Engineer danieltliu@google.com. ... 04 Big Data & Hadoop 05 Analyze Streaming Data in](https://reader030.vdocuments.us/reader030/viewer/2022041119/5f30d56df8de523002263829/html5/thumbnails/86.jpg)
● Predictive maintenance or condition monitoring
● Warranty reserve estimation● Propensity to buy● Demand forecasting● Process optimization● Telematics
Manufacturing
● Predictive inventory planning● Recommendation engines● Upsell and cross-channel marketing● Market segmentation and targeting● Customer ROI and lifetime value
Retail
● Alerts and diagnostics from real-time patient data
● Disease identification and risk satisfaction● Patient triage optimization● Proactive health management● Healthcare provider sentiment analysis
Healthcare and Life Sciences
● Aircraft scheduling● Dynamic pricing● Social media – consumer feedback and
interaction analysis● Customer complaint resolution● Traffic patterns and
congestion management
Travel and Hospitality
● Risk analytics and regulation● Customer Segmentation● Cross-selling and up-selling● Sales and marketing
campaign management● Credit worthiness evaluation
Financial Services
● Power usage analytics● Seismic data processing● Carbon emissions and trading● Customer-specific pricing● Smart grid management● Energy demand and supply optimization
Energy, Feedstock and Utilities
Cloud Machine Learning Use Cases
Proprietary + Confidential
![Page 87: Happy Aloha Friday! · Cloud customer Engineer robgrace@google.com Daniel Liu Cloud Customer Engineer danieltliu@google.com. ... 04 Big Data & Hadoop 05 Analyze Streaming Data in](https://reader030.vdocuments.us/reader030/viewer/2022041119/5f30d56df8de523002263829/html5/thumbnails/87.jpg)
Advanced Solutions Lab
![Page 88: Happy Aloha Friday! · Cloud customer Engineer robgrace@google.com Daniel Liu Cloud Customer Engineer danieltliu@google.com. ... 04 Big Data & Hadoop 05 Analyze Streaming Data in](https://reader030.vdocuments.us/reader030/viewer/2022041119/5f30d56df8de523002263829/html5/thumbnails/88.jpg)
© 2017 Google Inc. All rights reserved.
Machine Learning Advanced Solutions Lab
● Intensive ML training led by our best experts● Long-term engagement with Google ML engineers ● World-class, customer facilities on Google campuses
Solving the biggest machine learning challenges, alongside our customers
● Faster time to value● Acquisition of best practices● Competitive advantage through
bleeding edge solution from Google
Better customer value
![Page 89: Happy Aloha Friday! · Cloud customer Engineer robgrace@google.com Daniel Liu Cloud Customer Engineer danieltliu@google.com. ... 04 Big Data & Hadoop 05 Analyze Streaming Data in](https://reader030.vdocuments.us/reader030/viewer/2022041119/5f30d56df8de523002263829/html5/thumbnails/89.jpg)
Proprietary + Confidential
Get your arms around
Big Data
Invest time in understanding
Machine Learning
Work with us. Best practices,
partners to help you
Next Steps for Gaining Competitive Advantage With Machine Learning
![Page 90: Happy Aloha Friday! · Cloud customer Engineer robgrace@google.com Daniel Liu Cloud Customer Engineer danieltliu@google.com. ... 04 Big Data & Hadoop 05 Analyze Streaming Data in](https://reader030.vdocuments.us/reader030/viewer/2022041119/5f30d56df8de523002263829/html5/thumbnails/90.jpg)
9008How to get start with GCP
![Page 91: Happy Aloha Friday! · Cloud customer Engineer robgrace@google.com Daniel Liu Cloud Customer Engineer danieltliu@google.com. ... 04 Big Data & Hadoop 05 Analyze Streaming Data in](https://reader030.vdocuments.us/reader030/viewer/2022041119/5f30d56df8de523002263829/html5/thumbnails/91.jpg)
91
https://cloud.google.com/
![Page 92: Happy Aloha Friday! · Cloud customer Engineer robgrace@google.com Daniel Liu Cloud Customer Engineer danieltliu@google.com. ... 04 Big Data & Hadoop 05 Analyze Streaming Data in](https://reader030.vdocuments.us/reader030/viewer/2022041119/5f30d56df8de523002263829/html5/thumbnails/92.jpg)
92
https://cloud.google.com/
![Page 93: Happy Aloha Friday! · Cloud customer Engineer robgrace@google.com Daniel Liu Cloud Customer Engineer danieltliu@google.com. ... 04 Big Data & Hadoop 05 Analyze Streaming Data in](https://reader030.vdocuments.us/reader030/viewer/2022041119/5f30d56df8de523002263829/html5/thumbnails/93.jpg)
93
https://cloud.google.com/
![Page 94: Happy Aloha Friday! · Cloud customer Engineer robgrace@google.com Daniel Liu Cloud Customer Engineer danieltliu@google.com. ... 04 Big Data & Hadoop 05 Analyze Streaming Data in](https://reader030.vdocuments.us/reader030/viewer/2022041119/5f30d56df8de523002263829/html5/thumbnails/94.jpg)
94
https://cloud.google.com/
![Page 95: Happy Aloha Friday! · Cloud customer Engineer robgrace@google.com Daniel Liu Cloud Customer Engineer danieltliu@google.com. ... 04 Big Data & Hadoop 05 Analyze Streaming Data in](https://reader030.vdocuments.us/reader030/viewer/2022041119/5f30d56df8de523002263829/html5/thumbnails/95.jpg)
95
GCP Console (Demo)
![Page 96: Happy Aloha Friday! · Cloud customer Engineer robgrace@google.com Daniel Liu Cloud Customer Engineer danieltliu@google.com. ... 04 Big Data & Hadoop 05 Analyze Streaming Data in](https://reader030.vdocuments.us/reader030/viewer/2022041119/5f30d56df8de523002263829/html5/thumbnails/96.jpg)
96
Public Dataset
![Page 97: Happy Aloha Friday! · Cloud customer Engineer robgrace@google.com Daniel Liu Cloud Customer Engineer danieltliu@google.com. ... 04 Big Data & Hadoop 05 Analyze Streaming Data in](https://reader030.vdocuments.us/reader030/viewer/2022041119/5f30d56df8de523002263829/html5/thumbnails/97.jpg)
97
Q & A
![Page 98: Happy Aloha Friday! · Cloud customer Engineer robgrace@google.com Daniel Liu Cloud Customer Engineer danieltliu@google.com. ... 04 Big Data & Hadoop 05 Analyze Streaming Data in](https://reader030.vdocuments.us/reader030/viewer/2022041119/5f30d56df8de523002263829/html5/thumbnails/98.jpg)
98
That’s a wrap.HACC Kick Off
Saturday, October 24 at the Hawaii State Capitol