talking to tulips - goto conference · how we built google tulip using serverless technology and...
TRANSCRIPT
![Page 1: Talking to Tulips - GOTO Conference · How we built Google Tulip using Serverless technology and Machine Learning Ma Feigal Christiaan Hees Lee Boonstra Google Tulip Engineering Team](https://reader034.vdocuments.us/reader034/viewer/2022042306/5ed232eab2660731f5654561/html5/thumbnails/1.jpg)
Talking to TulipsHow we built Google Tulip using Serverless technology and Machine Learning
Matt FeigalChristiaan Hees
Lee BoonstraGoogle Tulip Engineering Team
![Page 3: Talking to Tulips - GOTO Conference · How we built Google Tulip using Serverless technology and Machine Learning Ma Feigal Christiaan Hees Lee Boonstra Google Tulip Engineering Team](https://reader034.vdocuments.us/reader034/viewer/2022042306/5ed232eab2660731f5654561/html5/thumbnails/3.jpg)
Matt FeigalFounder & CTO Google Tulip
![Page 4: Talking to Tulips - GOTO Conference · How we built Google Tulip using Serverless technology and Machine Learning Ma Feigal Christiaan Hees Lee Boonstra Google Tulip Engineering Team](https://reader034.vdocuments.us/reader034/viewer/2022042306/5ed232eab2660731f5654561/html5/thumbnails/4.jpg)
One Giant Leap for Plant-kind
![Page 6: Talking to Tulips - GOTO Conference · How we built Google Tulip using Serverless technology and Machine Learning Ma Feigal Christiaan Hees Lee Boonstra Google Tulip Engineering Team](https://reader034.vdocuments.us/reader034/viewer/2022042306/5ed232eab2660731f5654561/html5/thumbnails/6.jpg)
Architecture
ConversationalDialogflow
Mobile frontendHTML5/JS ClientCloud Run
BackendNode.jsCloud Run
Detect ImageAutoML Vision Base64 Image
![Page 7: Talking to Tulips - GOTO Conference · How we built Google Tulip using Serverless technology and Machine Learning Ma Feigal Christiaan Hees Lee Boonstra Google Tulip Engineering Team](https://reader034.vdocuments.us/reader034/viewer/2022042306/5ed232eab2660731f5654561/html5/thumbnails/7.jpg)
your data + our models
AutoML
our data + our models
Vision API
Data Loss Prevention API
Dialogflow
Ease of UseCustomisationTrain on state-of-the-art modelsBuild your own models Call pretrained models
Cloud TPUs
your data + your model
TensorFlow
![Page 8: Talking to Tulips - GOTO Conference · How we built Google Tulip using Serverless technology and Machine Learning Ma Feigal Christiaan Hees Lee Boonstra Google Tulip Engineering Team](https://reader034.vdocuments.us/reader034/viewer/2022042306/5ed232eab2660731f5654561/html5/thumbnails/8.jpg)
AutoML Demo
![Page 9: Talking to Tulips - GOTO Conference · How we built Google Tulip using Serverless technology and Machine Learning Ma Feigal Christiaan Hees Lee Boonstra Google Tulip Engineering Team](https://reader034.vdocuments.us/reader034/viewer/2022042306/5ed232eab2660731f5654561/html5/thumbnails/9.jpg)
And when we get sensor data again...
![Page 10: Talking to Tulips - GOTO Conference · How we built Google Tulip using Serverless technology and Machine Learning Ma Feigal Christiaan Hees Lee Boonstra Google Tulip Engineering Team](https://reader034.vdocuments.us/reader034/viewer/2022042306/5ed232eab2660731f5654561/html5/thumbnails/10.jpg)
Christiaan HeesVP of Infrastructure, Google Tulip
![Page 11: Talking to Tulips - GOTO Conference · How we built Google Tulip using Serverless technology and Machine Learning Ma Feigal Christiaan Hees Lee Boonstra Google Tulip Engineering Team](https://reader034.vdocuments.us/reader034/viewer/2022042306/5ed232eab2660731f5654561/html5/thumbnails/11.jpg)
Architecture
ConversationalDialogflow
Mobile frontendHTML5/JS ClientCloud Run
BackendNode.jsCloud Run
Detect ImageAutoML Vision
![Page 12: Talking to Tulips - GOTO Conference · How we built Google Tulip using Serverless technology and Machine Learning Ma Feigal Christiaan Hees Lee Boonstra Google Tulip Engineering Team](https://reader034.vdocuments.us/reader034/viewer/2022042306/5ed232eab2660731f5654561/html5/thumbnails/12.jpg)
As a developer I want to
● Write an app in a language I’m comfortable with
● Deploy● Don’t want to worry about server config,
maintenance and scaling
![Page 13: Talking to Tulips - GOTO Conference · How we built Google Tulip using Serverless technology and Machine Learning Ma Feigal Christiaan Hees Lee Boonstra Google Tulip Engineering Team](https://reader034.vdocuments.us/reader034/viewer/2022042306/5ed232eab2660731f5654561/html5/thumbnails/13.jpg)
Cloud Run● Serverless platform● Years of experience from GAE,
GAE Flex and Cloud Functions● Auto scales (also down to 0)● Not just functions● Any web app you can put in a
container
● Add a Dockerfile and deploy● Google handles the rest
![Page 14: Talking to Tulips - GOTO Conference · How we built Google Tulip using Serverless technology and Machine Learning Ma Feigal Christiaan Hees Lee Boonstra Google Tulip Engineering Team](https://reader034.vdocuments.us/reader034/viewer/2022042306/5ed232eab2660731f5654561/html5/thumbnails/14.jpg)
Example Dockerfile
![Page 15: Talking to Tulips - GOTO Conference · How we built Google Tulip using Serverless technology and Machine Learning Ma Feigal Christiaan Hees Lee Boonstra Google Tulip Engineering Team](https://reader034.vdocuments.us/reader034/viewer/2022042306/5ed232eab2660731f5654561/html5/thumbnails/15.jpg)
Build & Deploy Containers
docker build -t gcr.io/<my-project>/tulip .
docker push gcr.io/<my-project>/tulip
gcloud beta run deploy tulip \ --image gcr.io/<my-project>/tulip
![Page 16: Talking to Tulips - GOTO Conference · How we built Google Tulip using Serverless technology and Machine Learning Ma Feigal Christiaan Hees Lee Boonstra Google Tulip Engineering Team](https://reader034.vdocuments.us/reader034/viewer/2022042306/5ed232eab2660731f5654561/html5/thumbnails/16.jpg)
![Page 17: Talking to Tulips - GOTO Conference · How we built Google Tulip using Serverless technology and Machine Learning Ma Feigal Christiaan Hees Lee Boonstra Google Tulip Engineering Team](https://reader034.vdocuments.us/reader034/viewer/2022042306/5ed232eab2660731f5654561/html5/thumbnails/17.jpg)
Customer Case
● Heavily regulated● Personally Plant Identifying Information (PII)● General Plant Protection Regulation (GPPR)● Use existing Kubernetes infra
![Page 18: Talking to Tulips - GOTO Conference · How we built Google Tulip using Serverless technology and Machine Learning Ma Feigal Christiaan Hees Lee Boonstra Google Tulip Engineering Team](https://reader034.vdocuments.us/reader034/viewer/2022042306/5ed232eab2660731f5654561/html5/thumbnails/18.jpg)
Knativeopen source building blocks
for serverless on Kubernetes
![Page 19: Talking to Tulips - GOTO Conference · How we built Google Tulip using Serverless technology and Machine Learning Ma Feigal Christiaan Hees Lee Boonstra Google Tulip Engineering Team](https://reader034.vdocuments.us/reader034/viewer/2022042306/5ed232eab2660731f5654561/html5/thumbnails/19.jpg)
Cloud Run
Fully managed, deploy your workloads and don’t see the cluster.
Cloud Run on GKE
Deploy into your GKE cluster, run serverless side-by-side with your existing workloads.
Cloud Run Everywhere Else
Use the same APIs and tooling anywhere you run Kubernetes with Knative.
Serverless on Kubernetes Everywhere
![Page 20: Talking to Tulips - GOTO Conference · How we built Google Tulip using Serverless technology and Machine Learning Ma Feigal Christiaan Hees Lee Boonstra Google Tulip Engineering Team](https://reader034.vdocuments.us/reader034/viewer/2022042306/5ed232eab2660731f5654561/html5/thumbnails/20.jpg)
![Page 21: Talking to Tulips - GOTO Conference · How we built Google Tulip using Serverless technology and Machine Learning Ma Feigal Christiaan Hees Lee Boonstra Google Tulip Engineering Team](https://reader034.vdocuments.us/reader034/viewer/2022042306/5ed232eab2660731f5654561/html5/thumbnails/21.jpg)
Lee BoonstraVP of Engineering, Google Tulip
![Page 22: Talking to Tulips - GOTO Conference · How we built Google Tulip using Serverless technology and Machine Learning Ma Feigal Christiaan Hees Lee Boonstra Google Tulip Engineering Team](https://reader034.vdocuments.us/reader034/viewer/2022042306/5ed232eab2660731f5654561/html5/thumbnails/22.jpg)
How does this work?
![Page 23: Talking to Tulips - GOTO Conference · How we built Google Tulip using Serverless technology and Machine Learning Ma Feigal Christiaan Hees Lee Boonstra Google Tulip Engineering Team](https://reader034.vdocuments.us/reader034/viewer/2022042306/5ed232eab2660731f5654561/html5/thumbnails/23.jpg)
How does this work?
Point your mobile camera to a flower
![Page 24: Talking to Tulips - GOTO Conference · How we built Google Tulip using Serverless technology and Machine Learning Ma Feigal Christiaan Hees Lee Boonstra Google Tulip Engineering Team](https://reader034.vdocuments.us/reader034/viewer/2022042306/5ed232eab2660731f5654561/html5/thumbnails/24.jpg)
How does this work?
AutoML Vision detects the type of
flower
![Page 25: Talking to Tulips - GOTO Conference · How we built Google Tulip using Serverless technology and Machine Learning Ma Feigal Christiaan Hees Lee Boonstra Google Tulip Engineering Team](https://reader034.vdocuments.us/reader034/viewer/2022042306/5ed232eab2660731f5654561/html5/thumbnails/25.jpg)
How does this work?
Talk to the flower via your mobile microphone
![Page 26: Talking to Tulips - GOTO Conference · How we built Google Tulip using Serverless technology and Machine Learning Ma Feigal Christiaan Hees Lee Boonstra Google Tulip Engineering Team](https://reader034.vdocuments.us/reader034/viewer/2022042306/5ed232eab2660731f5654561/html5/thumbnails/26.jpg)
How does this work?
Spoken audio streams into Dialogflow, which
detects Intent and plays flower voice
![Page 27: Talking to Tulips - GOTO Conference · How we built Google Tulip using Serverless technology and Machine Learning Ma Feigal Christiaan Hees Lee Boonstra Google Tulip Engineering Team](https://reader034.vdocuments.us/reader034/viewer/2022042306/5ed232eab2660731f5654561/html5/thumbnails/27.jpg)
![Page 28: Talking to Tulips - GOTO Conference · How we built Google Tulip using Serverless technology and Machine Learning Ma Feigal Christiaan Hees Lee Boonstra Google Tulip Engineering Team](https://reader034.vdocuments.us/reader034/viewer/2022042306/5ed232eab2660731f5654561/html5/thumbnails/28.jpg)
![Page 29: Talking to Tulips - GOTO Conference · How we built Google Tulip using Serverless technology and Machine Learning Ma Feigal Christiaan Hees Lee Boonstra Google Tulip Engineering Team](https://reader034.vdocuments.us/reader034/viewer/2022042306/5ed232eab2660731f5654561/html5/thumbnails/29.jpg)
Google Tulip Demo
![Page 30: Talking to Tulips - GOTO Conference · How we built Google Tulip using Serverless technology and Machine Learning Ma Feigal Christiaan Hees Lee Boonstra Google Tulip Engineering Team](https://reader034.vdocuments.us/reader034/viewer/2022042306/5ed232eab2660731f5654561/html5/thumbnails/30.jpg)
Architecture
ConversationalDialogflow
Mobile frontendHTML5/JS ClientCloud Run
BackendNode.jsCloud Run
Detect ImageAutoML Vision
Websockets
Browser Microphone
![Page 31: Talking to Tulips - GOTO Conference · How we built Google Tulip using Serverless technology and Machine Learning Ma Feigal Christiaan Hees Lee Boonstra Google Tulip Engineering Team](https://reader034.vdocuments.us/reader034/viewer/2022042306/5ed232eab2660731f5654561/html5/thumbnails/31.jpg)
const request = {
session: this.sessionPath,
queryInput: {
audioConfig: {
sampleRateHertz: 48000,
audioEncoding: 'OUTPUT_AUDIO_ENCODING_LINEAR_16',
languageCode: 'en_US',
},
},
inputAudio: inputAudio,
outputAudioConfig: {
audioEncoding: 'OUTPUT_AUDIO_ENCODING_LINEAR_16',
sampleRateHertz: 48000,
synthesizeSpeechConfig: {
voice: { ssmlGender: 'SSML_VOICE_GENDER_FEMALE' }
}}};
const [response] = await this.sessionClient.detectIntent(request);
Dialog ow SDK
![Page 32: Talking to Tulips - GOTO Conference · How we built Google Tulip using Serverless technology and Machine Learning Ma Feigal Christiaan Hees Lee Boonstra Google Tulip Engineering Team](https://reader034.vdocuments.us/reader034/viewer/2022042306/5ed232eab2660731f5654561/html5/thumbnails/32.jpg)
Conclusion
![Page 33: Talking to Tulips - GOTO Conference · How we built Google Tulip using Serverless technology and Machine Learning Ma Feigal Christiaan Hees Lee Boonstra Google Tulip Engineering Team](https://reader034.vdocuments.us/reader034/viewer/2022042306/5ed232eab2660731f5654561/html5/thumbnails/33.jpg)
In case you wondered. The Google Tulip Project is an April Fools joke.
However, with Google Cloud technology, like AutoML Vision, Cloud Run and Dialogflow STT, TTSwe were close, in getting there.
![Page 34: Talking to Tulips - GOTO Conference · How we built Google Tulip using Serverless technology and Machine Learning Ma Feigal Christiaan Hees Lee Boonstra Google Tulip Engineering Team](https://reader034.vdocuments.us/reader034/viewer/2022042306/5ed232eab2660731f5654561/html5/thumbnails/34.jpg)
Thank you!More Info & Source Code
https://tulip.dev
https://github.com/GoogleCloudPlatform/tulip
Video
https://www.youtube.com/watch?v=nsPQvZm_rgM
Technical Blog post
https://medium.com/google-cloud/google-tulip-the
-technical-details-719c317bf2df