implementing saas on kubernetes - linux foundation events...public implementing saas on kubernetes...
TRANSCRIPT
![Page 1: Implementing SaaS on Kubernetes - Linux Foundation Events...Public Implementing SaaS on Kubernetes Multi-Tenancy and Tenant Isolation on Kubernetes Michael Knapp Senior Software Engineer](https://reader034.vdocuments.us/reader034/viewer/2022042916/5f5446e34c281125871000df/html5/thumbnails/1.jpg)
Public
Implementing SaaS on KubernetesMulti-Tenancy and Tenant Isolation on Kubernetes
Michael KnappSenior Software EngineerOctober 11, 2018Certified Kubernetes Administrator
Public
Andrew GaoSoftware EngineerOctober 11, 2018
![Page 2: Implementing SaaS on Kubernetes - Linux Foundation Events...Public Implementing SaaS on Kubernetes Multi-Tenancy and Tenant Isolation on Kubernetes Michael Knapp Senior Software Engineer](https://reader034.vdocuments.us/reader034/viewer/2022042916/5f5446e34c281125871000df/html5/thumbnails/2.jpg)
Public
• Understand how “Software as a Service” products can be architected on Kubernetes.
Goals
![Page 3: Implementing SaaS on Kubernetes - Linux Foundation Events...Public Implementing SaaS on Kubernetes Multi-Tenancy and Tenant Isolation on Kubernetes Michael Knapp Senior Software Engineer](https://reader034.vdocuments.us/reader034/viewer/2022042916/5f5446e34c281125871000df/html5/thumbnails/3.jpg)
Public
• Have a basic understanding of restful web APIs.• Preferred: basic knowledge of Kubernetes:
• Namespaces• Pods• Deployments• Services• Volumes• Config Maps• Ingress
Pre-Requisites
![Page 4: Implementing SaaS on Kubernetes - Linux Foundation Events...Public Implementing SaaS on Kubernetes Multi-Tenancy and Tenant Isolation on Kubernetes Michael Knapp Senior Software Engineer](https://reader034.vdocuments.us/reader034/viewer/2022042916/5f5446e34c281125871000df/html5/thumbnails/4.jpg)
Public
• Kubernetes Review• Kubernetes Tools for Isolation• Tools for distributed applications in Kubernetes• Architecture of SaaS in Kubernetes
Agenda
![Page 5: Implementing SaaS on Kubernetes - Linux Foundation Events...Public Implementing SaaS on Kubernetes Multi-Tenancy and Tenant Isolation on Kubernetes Michael Knapp Senior Software Engineer](https://reader034.vdocuments.us/reader034/viewer/2022042916/5f5446e34c281125871000df/html5/thumbnails/5.jpg)
Public
• Assumption: Your team is running a Kubernetes cluster• Problem: External teams or people must collaborate with your team
to run their software on your platform. • Examples:
• Add a Flink application to a Flink cluster• Provision apache NiFi instances on demand• Create a new Flink cluster• Create a custom database• Score events with a machine learning model
Problem
![Page 6: Implementing SaaS on Kubernetes - Linux Foundation Events...Public Implementing SaaS on Kubernetes Multi-Tenancy and Tenant Isolation on Kubernetes Michael Knapp Senior Software Engineer](https://reader034.vdocuments.us/reader034/viewer/2022042916/5f5446e34c281125871000df/html5/thumbnails/6.jpg)
Public
• Software as a Service• At a user’s request, we deploy a software application and make it
available to them.• Examples:
• RDS• DynamoDB• Elasticache• SQS• SNS
SaaS?
![Page 7: Implementing SaaS on Kubernetes - Linux Foundation Events...Public Implementing SaaS on Kubernetes Multi-Tenancy and Tenant Isolation on Kubernetes Michael Knapp Senior Software Engineer](https://reader034.vdocuments.us/reader034/viewer/2022042916/5f5446e34c281125871000df/html5/thumbnails/7.jpg)
Public
Match these up:
Challenge
Infrastructure as a Service (IaaS)
Platform as a Service (PaaS)
Software as a Service (SaaS)
Amazon’s Elastic Container Service for Kubernetes (EKS)
Amazon’s Elastic Compute Cloud (EC2)
Amazon’s ElasticCache
![Page 8: Implementing SaaS on Kubernetes - Linux Foundation Events...Public Implementing SaaS on Kubernetes Multi-Tenancy and Tenant Isolation on Kubernetes Michael Knapp Senior Software Engineer](https://reader034.vdocuments.us/reader034/viewer/2022042916/5f5446e34c281125871000df/html5/thumbnails/8.jpg)
Public
Match these up:
Challenge
Infrastructure as a Service (IaaS)
Platform as a Service (PaaS)
Software as a Service (SaaS)
Amazon’s Elastic Container Service for Kubernetes (EKS)
Amazon’s Elastic Compute Cloud (EC2)
Amazon’s ElasticCache
![Page 9: Implementing SaaS on Kubernetes - Linux Foundation Events...Public Implementing SaaS on Kubernetes Multi-Tenancy and Tenant Isolation on Kubernetes Michael Knapp Senior Software Engineer](https://reader034.vdocuments.us/reader034/viewer/2022042916/5f5446e34c281125871000df/html5/thumbnails/9.jpg)
Public
Brief Kubernetes Review
![Page 10: Implementing SaaS on Kubernetes - Linux Foundation Events...Public Implementing SaaS on Kubernetes Multi-Tenancy and Tenant Isolation on Kubernetes Michael Knapp Senior Software Engineer](https://reader034.vdocuments.us/reader034/viewer/2022042916/5f5446e34c281125871000df/html5/thumbnails/10.jpg)
Public
Kubernetes Architecture
https://kubernetes.io/docs/concepts/architecture/cloud-controller/
![Page 11: Implementing SaaS on Kubernetes - Linux Foundation Events...Public Implementing SaaS on Kubernetes Multi-Tenancy and Tenant Isolation on Kubernetes Michael Knapp Senior Software Engineer](https://reader034.vdocuments.us/reader034/viewer/2022042916/5f5446e34c281125871000df/html5/thumbnails/11.jpg)
Public
![Page 12: Implementing SaaS on Kubernetes - Linux Foundation Events...Public Implementing SaaS on Kubernetes Multi-Tenancy and Tenant Isolation on Kubernetes Michael Knapp Senior Software Engineer](https://reader034.vdocuments.us/reader034/viewer/2022042916/5f5446e34c281125871000df/html5/thumbnails/12.jpg)
Public
Deployment
![Page 13: Implementing SaaS on Kubernetes - Linux Foundation Events...Public Implementing SaaS on Kubernetes Multi-Tenancy and Tenant Isolation on Kubernetes Michael Knapp Senior Software Engineer](https://reader034.vdocuments.us/reader034/viewer/2022042916/5f5446e34c281125871000df/html5/thumbnails/13.jpg)
Public
Kubernetes Tools for Isolation
Public
![Page 14: Implementing SaaS on Kubernetes - Linux Foundation Events...Public Implementing SaaS on Kubernetes Multi-Tenancy and Tenant Isolation on Kubernetes Michael Knapp Senior Software Engineer](https://reader034.vdocuments.us/reader034/viewer/2022042916/5f5446e34c281125871000df/html5/thumbnails/14.jpg)
Public
NetworkPolicy
![Page 15: Implementing SaaS on Kubernetes - Linux Foundation Events...Public Implementing SaaS on Kubernetes Multi-Tenancy and Tenant Isolation on Kubernetes Michael Knapp Senior Software Engineer](https://reader034.vdocuments.us/reader034/viewer/2022042916/5f5446e34c281125871000df/html5/thumbnails/15.jpg)
Public
Constraining Resources
![Page 16: Implementing SaaS on Kubernetes - Linux Foundation Events...Public Implementing SaaS on Kubernetes Multi-Tenancy and Tenant Isolation on Kubernetes Michael Knapp Senior Software Engineer](https://reader034.vdocuments.us/reader034/viewer/2022042916/5f5446e34c281125871000df/html5/thumbnails/16.jpg)
Public
Role Based Access Control
You can assign ServiceAccountsto pods!
![Page 17: Implementing SaaS on Kubernetes - Linux Foundation Events...Public Implementing SaaS on Kubernetes Multi-Tenancy and Tenant Isolation on Kubernetes Michael Knapp Senior Software Engineer](https://reader034.vdocuments.us/reader034/viewer/2022042916/5f5446e34c281125871000df/html5/thumbnails/17.jpg)
Public
Role Based Access Control
![Page 18: Implementing SaaS on Kubernetes - Linux Foundation Events...Public Implementing SaaS on Kubernetes Multi-Tenancy and Tenant Isolation on Kubernetes Michael Knapp Senior Software Engineer](https://reader034.vdocuments.us/reader034/viewer/2022042916/5f5446e34c281125871000df/html5/thumbnails/18.jpg)
Public
What can we leverage to prevent tenants from hogging all the RAM in our cluster?a) Roles, RoleBinding, RBACb) NetworkPolicyc) ResourceQuotad) LimitRange
Pop Quiz
![Page 19: Implementing SaaS on Kubernetes - Linux Foundation Events...Public Implementing SaaS on Kubernetes Multi-Tenancy and Tenant Isolation on Kubernetes Michael Knapp Senior Software Engineer](https://reader034.vdocuments.us/reader034/viewer/2022042916/5f5446e34c281125871000df/html5/thumbnails/19.jpg)
Public
What can we leverage to prevent tenants from hogging all the RAM in our cluster?a) Roles, RoleBinding, RBACb) NetworkPolicyc) ResourceQuotad) LimitRange
Pop Quiz
A LimitRange may constrain the RAM usage of a single pod, but it cannot limit the total number of pods. A ResourceQuota can.
![Page 20: Implementing SaaS on Kubernetes - Linux Foundation Events...Public Implementing SaaS on Kubernetes Multi-Tenancy and Tenant Isolation on Kubernetes Michael Knapp Senior Software Engineer](https://reader034.vdocuments.us/reader034/viewer/2022042916/5f5446e34c281125871000df/html5/thumbnails/20.jpg)
Public
Kubernetes Tools for Distributed Applications
Public
![Page 21: Implementing SaaS on Kubernetes - Linux Foundation Events...Public Implementing SaaS on Kubernetes Multi-Tenancy and Tenant Isolation on Kubernetes Michael Knapp Senior Software Engineer](https://reader034.vdocuments.us/reader034/viewer/2022042916/5f5446e34c281125871000df/html5/thumbnails/21.jpg)
Public
• Overview• Open-source key value store• Built for clusters• Backbone of K8s
• Advantages• Automated restore from backup upon cluster node failure• Use etcd revision watchers for ordered/reliable/atomic event
streams• Out-of-the-box leader election
ETCD for your clustered deployment
![Page 22: Implementing SaaS on Kubernetes - Linux Foundation Events...Public Implementing SaaS on Kubernetes Multi-Tenancy and Tenant Isolation on Kubernetes Michael Knapp Senior Software Engineer](https://reader034.vdocuments.us/reader034/viewer/2022042916/5f5446e34c281125871000df/html5/thumbnails/22.jpg)
Public
• Overview• Service mesh• Load Balancing• Metrics
• Advantages• Discovery• Rate Limiting • Canary Releases• A/B testing
Making a Mesh with Istio
![Page 23: Implementing SaaS on Kubernetes - Linux Foundation Events...Public Implementing SaaS on Kubernetes Multi-Tenancy and Tenant Isolation on Kubernetes Michael Knapp Senior Software Engineer](https://reader034.vdocuments.us/reader034/viewer/2022042916/5f5446e34c281125871000df/html5/thumbnails/23.jpg)
Public
A/B Testing
![Page 24: Implementing SaaS on Kubernetes - Linux Foundation Events...Public Implementing SaaS on Kubernetes Multi-Tenancy and Tenant Isolation on Kubernetes Michael Knapp Senior Software Engineer](https://reader034.vdocuments.us/reader034/viewer/2022042916/5f5446e34c281125871000df/html5/thumbnails/24.jpg)
Public
Kubernetes Software as a Service
Public
![Page 25: Implementing SaaS on Kubernetes - Linux Foundation Events...Public Implementing SaaS on Kubernetes Multi-Tenancy and Tenant Isolation on Kubernetes Michael Knapp Senior Software Engineer](https://reader034.vdocuments.us/reader034/viewer/2022042916/5f5446e34c281125871000df/html5/thumbnails/25.jpg)
Public
![Page 26: Implementing SaaS on Kubernetes - Linux Foundation Events...Public Implementing SaaS on Kubernetes Multi-Tenancy and Tenant Isolation on Kubernetes Michael Knapp Senior Software Engineer](https://reader034.vdocuments.us/reader034/viewer/2022042916/5f5446e34c281125871000df/html5/thumbnails/26.jpg)
Public
• Defines a nomenclature for an object.• Does NOT define fields that it has!• The controller-manager dictates what fields it
has.
apiVersion: apiextensions.k8s.io/v1beta1kind: CustomResourceDefinitionmetadata:name: tenants.example.com
spec:group: example.comversion: v1scope: Clusternames:plural: tenantssingular: tenantkind: TenantshortNames:- tnt
Custom Resource Definition (CRD)
![Page 27: Implementing SaaS on Kubernetes - Linux Foundation Events...Public Implementing SaaS on Kubernetes Multi-Tenancy and Tenant Isolation on Kubernetes Michael Knapp Senior Software Engineer](https://reader034.vdocuments.us/reader034/viewer/2022042916/5f5446e34c281125871000df/html5/thumbnails/27.jpg)
PublicFollow this example: https://github.com/kubernetes/sample-controller
![Page 28: Implementing SaaS on Kubernetes - Linux Foundation Events...Public Implementing SaaS on Kubernetes Multi-Tenancy and Tenant Isolation on Kubernetes Michael Knapp Senior Software Engineer](https://reader034.vdocuments.us/reader034/viewer/2022042916/5f5446e34c281125871000df/html5/thumbnails/28.jpg)
Public
![Page 29: Implementing SaaS on Kubernetes - Linux Foundation Events...Public Implementing SaaS on Kubernetes Multi-Tenancy and Tenant Isolation on Kubernetes Michael Knapp Senior Software Engineer](https://reader034.vdocuments.us/reader034/viewer/2022042916/5f5446e34c281125871000df/html5/thumbnails/29.jpg)
Public
![Page 30: Implementing SaaS on Kubernetes - Linux Foundation Events...Public Implementing SaaS on Kubernetes Multi-Tenancy and Tenant Isolation on Kubernetes Michael Knapp Senior Software Engineer](https://reader034.vdocuments.us/reader034/viewer/2022042916/5f5446e34c281125871000df/html5/thumbnails/30.jpg)
Public
![Page 31: Implementing SaaS on Kubernetes - Linux Foundation Events...Public Implementing SaaS on Kubernetes Multi-Tenancy and Tenant Isolation on Kubernetes Michael Knapp Senior Software Engineer](https://reader034.vdocuments.us/reader034/viewer/2022042916/5f5446e34c281125871000df/html5/thumbnails/31.jpg)
Public
1. Why are the custom API server and the custom controller manager separate?
2. Why have a separate custom API server? Why not just use the Kube-APIServer?
Questions
![Page 32: Implementing SaaS on Kubernetes - Linux Foundation Events...Public Implementing SaaS on Kubernetes Multi-Tenancy and Tenant Isolation on Kubernetes Michael Knapp Senior Software Engineer](https://reader034.vdocuments.us/reader034/viewer/2022042916/5f5446e34c281125871000df/html5/thumbnails/32.jpg)
Public
![Page 33: Implementing SaaS on Kubernetes - Linux Foundation Events...Public Implementing SaaS on Kubernetes Multi-Tenancy and Tenant Isolation on Kubernetes Michael Knapp Senior Software Engineer](https://reader034.vdocuments.us/reader034/viewer/2022042916/5f5446e34c281125871000df/html5/thumbnails/33.jpg)
Public
1. Why are the custom API server and the custom controller manager separate?
• Update them separately• Can post CRDs directly to the Kube-APIServer.
Answer
![Page 34: Implementing SaaS on Kubernetes - Linux Foundation Events...Public Implementing SaaS on Kubernetes Multi-Tenancy and Tenant Isolation on Kubernetes Michael Knapp Senior Software Engineer](https://reader034.vdocuments.us/reader034/viewer/2022042916/5f5446e34c281125871000df/html5/thumbnails/34.jpg)
Public
2. Why have a separate custom API server? Why not just use the Kube-APIServer?
• Tenants don’t need to learn Kubernetes.• Don’t want tenants to even know Kubernetes is hosting their
software.• Limit tenants to only deploying our approved software.
Answer
![Page 35: Implementing SaaS on Kubernetes - Linux Foundation Events...Public Implementing SaaS on Kubernetes Multi-Tenancy and Tenant Isolation on Kubernetes Michael Knapp Senior Software Engineer](https://reader034.vdocuments.us/reader034/viewer/2022042916/5f5446e34c281125871000df/html5/thumbnails/35.jpg)
Public
Any Questions?
Public
![Page 36: Implementing SaaS on Kubernetes - Linux Foundation Events...Public Implementing SaaS on Kubernetes Multi-Tenancy and Tenant Isolation on Kubernetes Michael Knapp Senior Software Engineer](https://reader034.vdocuments.us/reader034/viewer/2022042916/5f5446e34c281125871000df/html5/thumbnails/36.jpg)
Public
Supplemental Material
Public
![Page 37: Implementing SaaS on Kubernetes - Linux Foundation Events...Public Implementing SaaS on Kubernetes Multi-Tenancy and Tenant Isolation on Kubernetes Michael Knapp Senior Software Engineer](https://reader034.vdocuments.us/reader034/viewer/2022042916/5f5446e34c281125871000df/html5/thumbnails/37.jpg)
Public
• Label tenant resources:• Tenant Name (i.e. red-team)
• Creator (i.e. bob)
• Software Application Name (i.e. redis)
• Software Instance Name (i.e. bobs-redis)
• Makes it much easier to discover who is causing problems, and to manage their resources.• For instance, you can bounce all their pods, or delete their software instance all together with one
command.
Tip: Use Kubernetes Labels
![Page 38: Implementing SaaS on Kubernetes - Linux Foundation Events...Public Implementing SaaS on Kubernetes Multi-Tenancy and Tenant Isolation on Kubernetes Michael Knapp Senior Software Engineer](https://reader034.vdocuments.us/reader034/viewer/2022042916/5f5446e34c281125871000df/html5/thumbnails/38.jpg)
Public
• CRUD – create, read, update, delete.• List/Query all instances of CRD for tenant. Usually has some method of filtering.• Describe – provides thorough information about the resource and its status. • For your CRD and also tenant instances.
Common API Endpoints
![Page 39: Implementing SaaS on Kubernetes - Linux Foundation Events...Public Implementing SaaS on Kubernetes Multi-Tenancy and Tenant Isolation on Kubernetes Michael Knapp Senior Software Engineer](https://reader034.vdocuments.us/reader034/viewer/2022042916/5f5446e34c281125871000df/html5/thumbnails/39.jpg)
Public
• Nonce – prevents repeat attacks• Login/API key check
• Authentication
• Signature check – hashes the nonce and other request parameters to confirm the user made the request.
• Authorization• To use this API
• To act on behalf of this tenant
• Admin? Or tenant member?
• Read-only vs Write access
• To view/alter the specific resource
Middleware aka Web Filters
![Page 40: Implementing SaaS on Kubernetes - Linux Foundation Events...Public Implementing SaaS on Kubernetes Multi-Tenancy and Tenant Isolation on Kubernetes Michael Knapp Senior Software Engineer](https://reader034.vdocuments.us/reader034/viewer/2022042916/5f5446e34c281125871000df/html5/thumbnails/40.jpg)
Public
• https://kubernetes.io/docs/tasks/access-kubernetes-api/custom-resources/custom-resource-definitions/
• https://github.com/kubernetes/sample-controller
More Information