kubecon eu 2016: scaling open edx with kubernetes
TRANSCRIPT
![Page 1: KubeCon EU 2016: Scaling Open edX with Kubernetes](https://reader030.vdocuments.us/reader030/viewer/2022020108/587081461a28ab57368b66a1/html5/thumbnails/1.jpg)
Scaling Open edX with KubernetesKubeCon London2016.03.10Morgan Robertson
![Page 2: KubeCon EU 2016: Scaling Open edX with Kubernetes](https://reader030.vdocuments.us/reader030/viewer/2022020108/587081461a28ab57368b66a1/html5/thumbnails/2.jpg)
Hi, I’m Morgan
![Page 3: KubeCon EU 2016: Scaling Open edX with Kubernetes](https://reader030.vdocuments.us/reader030/viewer/2022020108/587081461a28ab57368b66a1/html5/thumbnails/3.jpg)
What we’ll cover
● Introducing Open edX
● The problem: Walled gardens
● The solution: Open edX SaaS
● Open edX on Kubernetes
● Lessons learned
![Page 4: KubeCon EU 2016: Scaling Open edX with Kubernetes](https://reader030.vdocuments.us/reader030/viewer/2022020108/587081461a28ab57368b66a1/html5/thumbnails/4.jpg)
Introducing Open edX
![Page 5: KubeCon EU 2016: Scaling Open edX with Kubernetes](https://reader030.vdocuments.us/reader030/viewer/2022020108/587081461a28ab57368b66a1/html5/thumbnails/5.jpg)
Technical components
LMS/CMS (Django/Python)
Forum (Sinatra/Ruby)
User DB (MySQL)
Course DB (Mongo)
Tasks (Celery/RabbitMQ)
Caching (Memcache)
Proxy (Nginx)
Search (ElasticSearch)
Mapreduce (Hadoop)
![Page 6: KubeCon EU 2016: Scaling Open edX with Kubernetes](https://reader030.vdocuments.us/reader030/viewer/2022020108/587081461a28ab57368b66a1/html5/thumbnails/6.jpg)
![Page 7: KubeCon EU 2016: Scaling Open edX with Kubernetes](https://reader030.vdocuments.us/reader030/viewer/2022020108/587081461a28ab57368b66a1/html5/thumbnails/7.jpg)
The walled garden
![Page 8: KubeCon EU 2016: Scaling Open edX with Kubernetes](https://reader030.vdocuments.us/reader030/viewer/2022020108/587081461a28ab57368b66a1/html5/thumbnails/8.jpg)
Open edX Testdrive
![Page 9: KubeCon EU 2016: Scaling Open edX with Kubernetes](https://reader030.vdocuments.us/reader030/viewer/2022020108/587081461a28ab57368b66a1/html5/thumbnails/9.jpg)
Individual deployments don’t scale!
● Inefficient use of resources
● Can’t demo advanced features
● Painful upgrade process
![Page 10: KubeCon EU 2016: Scaling Open edX with Kubernetes](https://reader030.vdocuments.us/reader030/viewer/2022020108/587081461a28ab57368b66a1/html5/thumbnails/10.jpg)
Open edX SaaS
AMC
![Page 11: KubeCon EU 2016: Scaling Open edX with Kubernetes](https://reader030.vdocuments.us/reader030/viewer/2022020108/587081461a28ab57368b66a1/html5/thumbnails/11.jpg)
Kubernetes vs. Docker Swarm & friends
Kubernetes Swarm Compose
Scheduling ✔ ✔
Service discovery ✔ ✅ ✅
Container scaling ✔ ✔
Health checking ✔
Secret management ✔
Rolling updates ✔
![Page 12: KubeCon EU 2016: Scaling Open edX with Kubernetes](https://reader030.vdocuments.us/reader030/viewer/2022020108/587081461a28ab57368b66a1/html5/thumbnails/12.jpg)
Kubernetes vs. Docker Swarm & friends
Kubernetes Swarm Compose Consul
Scheduling ✔ ✔
Service discovery ✔ ✅ ✅ ✔
Container scaling ✔ ✔
Health checking ✔ ✔
Secret management ✔
Rolling updates ✔
![Page 13: KubeCon EU 2016: Scaling Open edX with Kubernetes](https://reader030.vdocuments.us/reader030/viewer/2022020108/587081461a28ab57368b66a1/html5/thumbnails/13.jpg)
![Page 14: KubeCon EU 2016: Scaling Open edX with Kubernetes](https://reader030.vdocuments.us/reader030/viewer/2022020108/587081461a28ab57368b66a1/html5/thumbnails/14.jpg)
Containerizing Open edX
edxapp
forum
![Page 15: KubeCon EU 2016: Scaling Open edX with Kubernetes](https://reader030.vdocuments.us/reader030/viewer/2022020108/587081461a28ab57368b66a1/html5/thumbnails/15.jpg)
Building images
FROM phusion/baseimage:0.9.15
# Clone the configuration repoRUN git clone https://github.com/appsembler/configuration.git \ /tmp/configuration
# Run AnsibleWORKDIR /tmp/configuration/playbooksRUN ansible-playbook -vvv -i 127.0.0.1, -c local forum.yml
CMD ["/sbin/my_init"]
![Page 16: KubeCon EU 2016: Scaling Open edX with Kubernetes](https://reader030.vdocuments.us/reader030/viewer/2022020108/587081461a28ab57368b66a1/html5/thumbnails/16.jpg)
Open edX on Kubernetes
![Page 17: KubeCon EU 2016: Scaling Open edX with Kubernetes](https://reader030.vdocuments.us/reader030/viewer/2022020108/587081461a28ab57368b66a1/html5/thumbnails/17.jpg)
Persistence: Mongo and MySQL
● Mongo replica set on VMs
● Google Cloud SQL
● Kubernetes services can abstract non-Pod backends
![Page 18: KubeCon EU 2016: Scaling Open edX with Kubernetes](https://reader030.vdocuments.us/reader030/viewer/2022020108/587081461a28ab57368b66a1/html5/thumbnails/18.jpg)
Using Endpoints for non-Pod backends# mongodb-endpoint.ymlapiVersion: v1kind: Endpointsmetadata: name: mongodbsubsets: - addresses: - ip: 10.240.0.3 - ip: 10.240.0.4 - ip: 10.240.0.5 ports: - name: mongodb port: 27017
# mongodb-service.ymlapiVersion: v1kind: Servicemetadata: name: mongodb labels: name: mongodbspec: clusterIP: None ports: - name: mongodb port: 27017
![Page 19: KubeCon EU 2016: Scaling Open edX with Kubernetes](https://reader030.vdocuments.us/reader030/viewer/2022020108/587081461a28ab57368b66a1/html5/thumbnails/19.jpg)
Initialization tasks using Jobs
# Create users and DBs in MySQLapiVersion: extensions/v1beta1kind: Jobmetadata: name: mysql-initspec: template: metadata: labels: name: mysql-init spec: containers: - name: mysql-init image: gcr.io/appsemblerx/mysql-init restartPolicy: OnFailure
● One-time tasks
● Batch tasks
● Run until success
![Page 20: KubeCon EU 2016: Scaling Open edX with Kubernetes](https://reader030.vdocuments.us/reader030/viewer/2022020108/587081461a28ab57368b66a1/html5/thumbnails/20.jpg)
Secrets
$ base64 secret.json
ewogICAgInNlY3JldCI6ICJteSBzZWNyZXQiCn0K
Kubernetes API
Pod 1secret. json
Pod 2secret. json
![Page 21: KubeCon EU 2016: Scaling Open edX with Kubernetes](https://reader030.vdocuments.us/reader030/viewer/2022020108/587081461a28ab57368b66a1/html5/thumbnails/21.jpg)
Google Cloud Monitoring
![Page 22: KubeCon EU 2016: Scaling Open edX with Kubernetes](https://reader030.vdocuments.us/reader030/viewer/2022020108/587081461a28ab57368b66a1/html5/thumbnails/22.jpg)
Lessons learned
● Containerizing an existing application can be difficult● Once containerized, deploying on Kubernetes is
generally straightforward● Not everything must be containerized to use
Kubernetes
![Page 23: KubeCon EU 2016: Scaling Open edX with Kubernetes](https://reader030.vdocuments.us/reader030/viewer/2022020108/587081461a28ab57368b66a1/html5/thumbnails/23.jpg)
Thank you for your time!