debugging & troubleshooting microservices it runs in ... · @saturnism @jbaruch @jfrog...
TRANSCRIPT
@saturnism @jbaruch @jfrog @gcpcloud #devoxxua jfrog.com/shownotes
Debugging & Troubleshooting Microservicesw/ Kubernetes
It runs in production, or is it?
@saturnism @jbaruch @jfrog @gcpcloud #devoxxua jfrog.com/shownotes
Ray TsangDeveloper AdvocateGoogle Cloud Platform
Java Champion
Spring Cloud GCPcloud.spring.io/spring-cloud-gcp/
JHipster, JDeferred
@saturnism
@saturnism @jbaruch @jfrog @gcpcloud #devoxxua jfrog.com/shownotes
BaruchSadogurskyDeveloper AdvocateJFrog
@jbaruch
@saturnism @jbaruch @jfrog @gcpcloud #devoxxua jfrog.com/shownotes
Shownotes!
● jfrog.com/shownotes○ Slides○ Video (tomorrow)○ Links○ Feedback○ Raffle
@saturnism @jbaruch @jfrog @gcpcloud #devoxxua jfrog.com/shownotes
@saturnism @jbaruch @jfrog @gcpcloud #devoxxua jfrog.com/shownotes
Hello World Service - Greet
Guestbook Service - Create
Guestbook Service - Retrieve
@saturnism @jbaruch @jfrog @gcpcloud #devoxxua jfrog.com/shownotes
GuestbookUI
Hello World Service
Redis
session replication
greeting
MySQL
GuestbookService
CRUD
@saturnism @jbaruch @jfrog @gcpcloud #devoxxua jfrog.com/shownotes
Let's see it!
@saturnism @jbaruch @jfrog @gcpcloud #devoxxua jfrog.com/shownotes
2015 - FAIL! 14:01
@saturnism @jbaruch @jfrog @gcpcloud #devoxxua jfrog.com/shownotes
2016 - FAIL! 1:49:50
@saturnism @jbaruch @jfrog @gcpcloud #devoxxua jfrog.com/shownotes
2017 - FAIL! 31:00
@saturnism @jbaruch @jfrog @gcpcloud #devoxxua jfrog.com/shownotes
2018 - FAIL! 2:28:47
@saturnism @jbaruch @jfrog @gcpcloud #devoxxua jfrog.com/shownotes
Visibility & Insight is Key
@saturnism @jbaruch @jfrog @gcpcloud #devoxxua jfrog.com/shownotes
Repository metadata - key insight into artifacts and their relationships
@saturnism @jbaruch @jfrog @gcpcloud #devoxxua jfrog.com/shownotes
Production vs Non-Production :(
@saturnism @jbaruch @jfrog @gcpcloud #devoxxua jfrog.com/shownotes
Distributed Systems are hard!
@saturnism @jbaruch @jfrog @gcpcloud #devoxxua jfrog.com/shownotes
We've been there, done that
@saturnism @jbaruch @jfrog @gcpcloud #devoxxua jfrog.com/shownotes
Information - Logs!
kubectl logs -f pod_id
@saturnism @jbaruch @jfrog @gcpcloud #devoxxua jfrog.com/shownotes
Inspecting the process / container
kubectl exec -ti pod_id /bin/bash
@saturnism @jbaruch @jfrog @gcpcloud #devoxxua jfrog.com/shownotes
Port forward to container
kubectl port-forward pod_id local_port:remote_port
@saturnism @jbaruch @jfrog @gcpcloud #devoxxua jfrog.com/shownotes
Isolate problematic instance w/ Labels
kubectl label pod pod_id --overewrite serving=false
@saturnism @jbaruch @jfrog @gcpcloud #devoxxua jfrog.com/shownotes
Pod
frontend
Pod
serving = true
version = 1.0
Pod
serving = true
version = 1.0
Service
Label selectors: version = 1.0 serving = true
Pod
serving = true
version = 1.0
@saturnism @jbaruch @jfrog @gcpcloud #devoxxua jfrog.com/shownotes
Pod
frontend
Pod
serving = true
version = 1.0
Pod
serving = true
version = 1.0
Service
Label selectors: version = 1.0 serving = true
Pod
serving = false
version = 1.0
@saturnism @jbaruch @jfrog @gcpcloud #devoxxua jfrog.com/shownotes
Log aggregation, at scale
@saturnism @jbaruch @jfrog @gcpcloud #devoxxua jfrog.com/shownotes
Stackdriver Logginghttps://cloud.google.com/logging/
@saturnism @jbaruch @jfrog @gcpcloud #devoxxua jfrog.com/shownotes
@saturnism @jbaruch @jfrog @gcpcloud #devoxxua jfrog.com/shownotes
Metrics from Logs → Alerting
Near-realtime insight
@saturnism @jbaruch @jfrog @gcpcloud #devoxxua jfrog.com/shownotes
Search and Query Your Logs
@saturnism @jbaruch @jfrog @gcpcloud #devoxxua jfrog.com/shownotes
Export to BigQuery, Storage, Pub/Sub
Near-realtime insight
@saturnism @jbaruch @jfrog @gcpcloud #devoxxua jfrog.com/shownotes
@saturnism @jbaruch @jfrog @gcpcloud #devoxxua jfrog.com/shownotes
@saturnism @jbaruch @jfrog @gcpcloud #devoxxua jfrog.com/shownotes
Tracing at Google - Dapperhttps://research.google.com/pubs/pub36356.html
@saturnism @jbaruch @jfrog @gcpcloud #devoxxua jfrog.com/shownotes
Zipkin
@saturnism @jbaruch @jfrog @gcpcloud #devoxxua jfrog.com/shownotes
Stackdriver Tracehttps://cloud.google.com/trace/
@saturnism @jbaruch @jfrog @gcpcloud #devoxxua jfrog.com/shownotes
Zipkin → Stackdriver Tracehttps://cloud.google.com/trace/docs/zipkin
@saturnism @jbaruch @jfrog @gcpcloud #devoxxua jfrog.com/shownotes
@saturnism @jbaruch @jfrog @gcpcloud #devoxxua jfrog.com/shownotes
@saturnism @jbaruch @jfrog @gcpcloud #devoxxua jfrog.com/shownotes
@saturnism @jbaruch @jfrog @gcpcloud #devoxxua jfrog.com/shownotes
@saturnism @jbaruch @jfrog @gcpcloud #devoxxua jfrog.com/shownotes
Automatic reports
Detect performance regression
@saturnism @jbaruch @jfrog @gcpcloud #devoxxua jfrog.com/shownotes
Oops… I forgot a log message. Darn!
@saturnism @jbaruch @jfrog @gcpcloud #devoxxua jfrog.com/shownotes
Don't you wish you can use a debugger?
@saturnism @jbaruch @jfrog @gcpcloud #devoxxua jfrog.com/shownotes
Stackdriver Debugger - Production Debugger
https://cloud.google.com/debugger/
@saturnism @jbaruch @jfrog @gcpcloud #devoxxua jfrog.com/shownotes
Add debugger agent
java -agentpath:/opt/cdbg/cdbg_java_agent.so ... -jar PATH_TO_JAR_FILE
@saturnism @jbaruch @jfrog @gcpcloud #devoxxua jfrog.com/shownotes
Logpoint - dynamically add log messages!
No redeployment, Magic.
gcloud debug logpoints create HelloworldService.java:35 \"Received endpoint: {endpoint}/{name}" --target helloworld-ui-1.0-SNAPSHOT
@saturnism @jbaruch @jfrog @gcpcloud #devoxxua jfrog.com/shownotes
Snapshots
Inspect call stack and variables
@saturnism @jbaruch @jfrog @gcpcloud #devoxxua jfrog.com/shownotes
@saturnism @jbaruch @jfrog @gcpcloud #devoxxua jfrog.com/shownotes
@saturnism @jbaruch @jfrog @gcpcloud #devoxxua jfrog.com/shownotes
Trace
Log
Debug
@saturnism @jbaruch @jfrog @gcpcloud #devoxxua jfrog.com/shownotes
Trace → Stackdriver Trace
Logs → Stackdriver Logging
Debug → Stackdriver Debuggerhttps://cloud.google.com/stackdriver/
@saturnism @jbaruch @jfrog @gcpcloud #devoxxua jfrog.com/shownotes
Thanks!Source: https://github.com/saturnism/spring-boot-dockerShownotes: https://jfrog.com/shownotesStackdriver: https://cloud.google.com/stackdriver/
Ray Tsang@saturnismGoogle Cloud Platform
Baruch Sadogursky@jbaruchJFrog
@saturnism @jbaruch @jfrog @gcpcloud #devoxxua jfrog.com/shownotes
@saturnism @jbaruch @jfrog @googlecloud #devoxxFR jfrog.com/shownotes