containers, docker, and security: state of the union · if you don't trust upstream, you have...
TRANSCRIPT
![Page 1: Containers, Docker, and Security: State of the Union · If you don't trust upstream, you have to ... stop using apt-get and yum with public repos rebuild everything from source verify](https://reader033.vdocuments.us/reader033/viewer/2022060501/5f1b9a05a8f62f4c0d2b3737/html5/thumbnails/1.jpg)
Containers, Docker, and Security:
State of the Union
1 / 38
![Page 2: Containers, Docker, and Security: State of the Union · If you don't trust upstream, you have to ... stop using apt-get and yum with public repos rebuild everything from source verify](https://reader033.vdocuments.us/reader033/viewer/2022060501/5f1b9a05a8f62f4c0d2b3737/html5/thumbnails/2.jpg)
Who am I?Jérôme Petazzoni (@jpetazzo)
French software engineer living in California
Joined Docker (dotCloud) more than 4 years ago (I was at Docker before it was cool!)
I built and scaled the dotCloud PaaS (millions of containers, no known security issues)
I learned a few things about running containers (in production)
2 / 38
![Page 3: Containers, Docker, and Security: State of the Union · If you don't trust upstream, you have to ... stop using apt-get and yum with public repos rebuild everything from source verify](https://reader033.vdocuments.us/reader033/viewer/2022060501/5f1b9a05a8f62f4c0d2b3737/html5/thumbnails/3.jpg)
OutlineYesterday
Today
Tomorrow
3 / 38
![Page 4: Containers, Docker, and Security: State of the Union · If you don't trust upstream, you have to ... stop using apt-get and yum with public repos rebuild everything from source verify](https://reader033.vdocuments.us/reader033/viewer/2022060501/5f1b9a05a8f62f4c0d2b3737/html5/thumbnails/4.jpg)
Yesterday
4 / 38
![Page 5: Containers, Docker, and Security: State of the Union · If you don't trust upstream, you have to ... stop using apt-get and yum with public repos rebuild everything from source verify](https://reader033.vdocuments.us/reader033/viewer/2022060501/5f1b9a05a8f62f4c0d2b3737/html5/thumbnails/5.jpg)
Containers and Security yesterdayLast year, at LinuxCon...
"Is it safe to run applications in containers?"
5 / 38
![Page 6: Containers, Docker, and Security: State of the Union · If you don't trust upstream, you have to ... stop using apt-get and yum with public repos rebuild everything from source verify](https://reader033.vdocuments.us/reader033/viewer/2022060501/5f1b9a05a8f62f4c0d2b3737/html5/thumbnails/6.jpg)
Containers and Security yesterdayLast year, at LinuxCon...
"Is it safe to run applications in containers?"
really meant
"Can one container break out, and into another?"
6 / 38
![Page 7: Containers, Docker, and Security: State of the Union · If you don't trust upstream, you have to ... stop using apt-get and yum with public repos rebuild everything from source verify](https://reader033.vdocuments.us/reader033/viewer/2022060501/5f1b9a05a8f62f4c0d2b3737/html5/thumbnails/7.jpg)
Containers and Security yesterdayLast year, at LinuxCon...
"Is it safe to run applications in containers?"
really meant
"Can one container break out, and into another?"
Main concern: isolation
7 / 38
![Page 8: Containers, Docker, and Security: State of the Union · If you don't trust upstream, you have to ... stop using apt-get and yum with public repos rebuild everything from source verify](https://reader033.vdocuments.us/reader033/viewer/2022060501/5f1b9a05a8f62f4c0d2b3737/html5/thumbnails/8.jpg)
What was the answer?
8 / 38
![Page 9: Containers, Docker, and Security: State of the Union · If you don't trust upstream, you have to ... stop using apt-get and yum with public repos rebuild everything from source verify](https://reader033.vdocuments.us/reader033/viewer/2022060501/5f1b9a05a8f62f4c0d2b3737/html5/thumbnails/9.jpg)
What was the answer?
9 / 38
![Page 10: Containers, Docker, and Security: State of the Union · If you don't trust upstream, you have to ... stop using apt-get and yum with public repos rebuild everything from source verify](https://reader033.vdocuments.us/reader033/viewer/2022060501/5f1b9a05a8f62f4c0d2b3737/html5/thumbnails/10.jpg)
What was the answer?"It's complicated"
Long list of recommendations
some were easy (and automatically enforced by Docker)
some were not obvious (and had to be enabled manually)
some were hard to deploy (or required missing kernel features)
Video and Slides
10 / 38
![Page 11: Containers, Docker, and Security: State of the Union · If you don't trust upstream, you have to ... stop using apt-get and yum with public repos rebuild everything from source verify](https://reader033.vdocuments.us/reader033/viewer/2022060501/5f1b9a05a8f62f4c0d2b3737/html5/thumbnails/11.jpg)
How is this different today?People still ask about container isolation
Much more frequently, they ask about image security and provenance
They want to know:
if docker pull debian is what it claims to be
if jpetazzo/dind has vulnerabilities
if a given image has been vetted by their sec team
11 / 38
![Page 12: Containers, Docker, and Security: State of the Union · If you don't trust upstream, you have to ... stop using apt-get and yum with public repos rebuild everything from source verify](https://reader033.vdocuments.us/reader033/viewer/2022060501/5f1b9a05a8f62f4c0d2b3737/html5/thumbnails/12.jpg)
Why has it changed?Who cares about container isolation?
hosting providers (more density = more $$$)PAAS (for rapid deployment; on-demand activation)
→ early adopters
Who doesn't care about container isolation?
people who use VMs only because autoscalingpeople who would put multiple components permachine anyway
→ second wave of users
12 / 38
![Page 13: Containers, Docker, and Security: State of the Union · If you don't trust upstream, you have to ... stop using apt-get and yum with public repos rebuild everything from source verify](https://reader033.vdocuments.us/reader033/viewer/2022060501/5f1b9a05a8f62f4c0d2b3737/html5/thumbnails/13.jpg)
Today
13 / 38
![Page 14: Containers, Docker, and Security: State of the Union · If you don't trust upstream, you have to ... stop using apt-get and yum with public repos rebuild everything from source verify](https://reader033.vdocuments.us/reader033/viewer/2022060501/5f1b9a05a8f62f4c0d2b3737/html5/thumbnails/14.jpg)
Docker and Containers Security TodayImproving what we had yesterday (fine-grained permissions, immutable containers)
Addressing new challenges (provenance, content verification, notary)
Defense in depth (containers + VM)
The infosec mindset (better upgrades, security benchmarks, policies)
14 / 38
![Page 15: Containers, Docker, and Security: State of the Union · If you don't trust upstream, you have to ... stop using apt-get and yum with public repos rebuild everything from source verify](https://reader033.vdocuments.us/reader033/viewer/2022060501/5f1b9a05a8f62f4c0d2b3737/html5/thumbnails/15.jpg)
Finer-grained permissionsPer-container ulimit
Capability reduction --cap-drop / --cap-add e.g.: --cap-add net_admin
Device access restrictions --device (better than --privileged!)
Improved handling of LSM (SELinux, AppArmor)
15 / 38
![Page 16: Containers, Docker, and Security: State of the Union · If you don't trust upstream, you have to ... stop using apt-get and yum with public repos rebuild everything from source verify](https://reader033.vdocuments.us/reader033/viewer/2022060501/5f1b9a05a8f62f4c0d2b3737/html5/thumbnails/16.jpg)
Smaller attack surfaceHardware management done on the host (no kernel, drivers, device handling... in containers)
Package management is optional (once a container is built, it doesn't need to be changed)
Minimal distros can be used (e.g. buildroot, Alpine Linux...)
Less software = less risk
16 / 38
![Page 17: Containers, Docker, and Security: State of the Union · If you don't trust upstream, you have to ... stop using apt-get and yum with public repos rebuild everything from source verify](https://reader033.vdocuments.us/reader033/viewer/2022060501/5f1b9a05a8f62f4c0d2b3737/html5/thumbnails/17.jpg)
Immutable containersdocker run --read-only (makes it impossible to entrench in a container)
Helps with vulnerability detection (audit can be performed on offline images)
Even without --read-only flag:
copy-on-write prevents changes from being permanent
break a container when hacking it → it gets recycled
docker diff allows easy audit of changes
17 / 38
![Page 18: Containers, Docker, and Security: State of the Union · If you don't trust upstream, you have to ... stop using apt-get and yum with public repos rebuild everything from source verify](https://reader033.vdocuments.us/reader033/viewer/2022060501/5f1b9a05a8f62f4c0d2b3737/html5/thumbnails/18.jpg)
Image provenanceHow can I trust docker pull debian?
I must trust upstream (i.e. Debian and whoever maintains the image)
I must trust Docker Inc. (operator of the Hub)
I must trust the transport (between the Hub and my Docker host)
18 / 38
![Page 19: Containers, Docker, and Security: State of the Union · If you don't trust upstream, you have to ... stop using apt-get and yum with public repos rebuild everything from source verify](https://reader033.vdocuments.us/reader033/viewer/2022060501/5f1b9a05a8f62f4c0d2b3737/html5/thumbnails/19.jpg)
Image provenanceHow can I trust docker pull debian?
I must trust upstream (i.e. Debian and whoever maintains the image)
I must trust Docker Inc. (operator of the Hub)
I must trust the transport (between the Hub and my Docker host)
That's a lot of trust
19 / 38
![Page 20: Containers, Docker, and Security: State of the Union · If you don't trust upstream, you have to ... stop using apt-get and yum with public repos rebuild everything from source verify](https://reader033.vdocuments.us/reader033/viewer/2022060501/5f1b9a05a8f62f4c0d2b3737/html5/thumbnails/20.jpg)
"I don't want to trust anybody!"If you don't trust upstream, you have to ...
stop using apt-get and yum with public repos
rebuild everything from source
verify source integrity (full audit + review all changes)
If you don't trust Docker Inc., you probably should ...
audit the whole Docker Engine code
audit every single patch that goes into Docker (if you can do that ... we're looking for reviewers)
20 / 38
![Page 21: Containers, Docker, and Security: State of the Union · If you don't trust upstream, you have to ... stop using apt-get and yum with public repos rebuild everything from source verify](https://reader033.vdocuments.us/reader033/viewer/2022060501/5f1b9a05a8f62f4c0d2b3737/html5/thumbnails/21.jpg)
Security reminderIt's OK to be paranoid, but beware of:
Bumps in the carpet (moving a problem rather than solving it)
Usability (if security makes it hard/impossible to work, people will work around it!)
Tinfoil hats
21 / 38
![Page 22: Containers, Docker, and Security: State of the Union · If you don't trust upstream, you have to ... stop using apt-get and yum with public repos rebuild everything from source verify](https://reader033.vdocuments.us/reader033/viewer/2022060501/5f1b9a05a8f62f4c0d2b3737/html5/thumbnails/22.jpg)
Can we trust the transport?Registry v1 protocol had serious issues:
arbitrary layer IDs
no integrity check (other than TLS transport integrity)
Registry v2 protocol has:
content-based layer IDs
signed image manifests
Is that enough?
22 / 38
![Page 23: Containers, Docker, and Security: State of the Union · If you don't trust upstream, you have to ... stop using apt-get and yum with public repos rebuild everything from source verify](https://reader033.vdocuments.us/reader033/viewer/2022060501/5f1b9a05a8f62f4c0d2b3737/html5/thumbnails/23.jpg)
Notary: a better trust
framework
23 / 38
![Page 24: Containers, Docker, and Security: State of the Union · If you don't trust upstream, you have to ... stop using apt-get and yum with public repos rebuild everything from source verify](https://reader033.vdocuments.us/reader033/viewer/2022060501/5f1b9a05a8f62f4c0d2b3737/html5/thumbnails/24.jpg)
What are we trying to address?Distributed content should be signed
Stealing a key should be hard
Stealing a key shouldn't have dire consequences
Replay attacks should be hard (=can't serve you yesterday's vulnerable version)
Should use known models and research
Existing distribution infrastructure should be used (=HTTP, HTTPS, FTP… are good)
Trusting Docker Hub should not be mandatory
24 / 38
![Page 25: Containers, Docker, and Security: State of the Union · If you don't trust upstream, you have to ... stop using apt-get and yum with public repos rebuild everything from source verify](https://reader033.vdocuments.us/reader033/viewer/2022060501/5f1b9a05a8f62f4c0d2b3737/html5/thumbnails/25.jpg)
NotaryBased on TUF (The Update Framework)
Sign content with offline keys
Trust thresholds (require K out of N keys)
(Stealing a key reduces signing requirements, but doesn't break the whole model)
Guarantee freshness
Distribute signed content on (potentially insecure) servers (leverage existing (insecure) transport and mirrors)
Enabled in Docker 1.8 by setting DOCKER_CONTENT_TRUST
25 / 38
![Page 26: Containers, Docker, and Security: State of the Union · If you don't trust upstream, you have to ... stop using apt-get and yum with public repos rebuild everything from source verify](https://reader033.vdocuments.us/reader033/viewer/2022060501/5f1b9a05a8f62f4c0d2b3737/html5/thumbnails/26.jpg)
Defense in depthSo, VM or containers?
26 / 38
![Page 27: Containers, Docker, and Security: State of the Union · If you don't trust upstream, you have to ... stop using apt-get and yum with public repos rebuild everything from source verify](https://reader033.vdocuments.us/reader033/viewer/2022060501/5f1b9a05a8f62f4c0d2b3737/html5/thumbnails/27.jpg)
Defense in depthSo, VM or containers?
VM and containers!
27 / 38
![Page 28: Containers, Docker, and Security: State of the Union · If you don't trust upstream, you have to ... stop using apt-get and yum with public repos rebuild everything from source verify](https://reader033.vdocuments.us/reader033/viewer/2022060501/5f1b9a05a8f62f4c0d2b3737/html5/thumbnails/28.jpg)
Defense in depthSo, VM or containers?
VM and containers!
Reduce number of VMs (when security perimeter allows it)
Colocated containers are safer than colocated processes
Malicious code has to escape both layers
Docker provides an extra layer of isolation
Applications are safer with containers than without
28 / 38
![Page 29: Containers, Docker, and Security: State of the Union · If you don't trust upstream, you have to ... stop using apt-get and yum with public repos rebuild everything from source verify](https://reader033.vdocuments.us/reader033/viewer/2022060501/5f1b9a05a8f62f4c0d2b3737/html5/thumbnails/29.jpg)
The infosec mindsetBetter upgrades
Accurate, actionable security benchmarks
Clear, sensible security policies
29 / 38
![Page 30: Containers, Docker, and Security: State of the Union · If you don't trust upstream, you have to ... stop using apt-get and yum with public repos rebuild everything from source verify](https://reader033.vdocuments.us/reader033/viewer/2022060501/5f1b9a05a8f62f4c0d2b3737/html5/thumbnails/30.jpg)
Better upgradesDockerfile = easy, fast, reliable builds and rebuilds
"But now I have 1000s of container images to upgrade!"
Yes, but that's way better than the 100s of server imagesthat you had before
The organizational risk is lower (because if something goes wrong, you have reliable rollbacks)
30 / 38
![Page 31: Containers, Docker, and Security: State of the Union · If you don't trust upstream, you have to ... stop using apt-get and yum with public repos rebuild everything from source verify](https://reader033.vdocuments.us/reader033/viewer/2022060501/5f1b9a05a8f62f4c0d2b3737/html5/thumbnails/31.jpg)
Security benchmarksCIS (Center of Internet Security) Docker Benchmark
Docker Bench (https://dockerbench.com) : automated assessment tool to check compliance
31 / 38
![Page 32: Containers, Docker, and Security: State of the Union · If you don't trust upstream, you have to ... stop using apt-get and yum with public repos rebuild everything from source verify](https://reader033.vdocuments.us/reader033/viewer/2022060501/5f1b9a05a8f62f4c0d2b3737/html5/thumbnails/32.jpg)
PoliciesDocker Inc. (the company) and the Docker Project (opensource) have clear security guidelines
Mandatory code reviews (see CONTRIBUTING.md) toensure quality of code base
Quarterly security audits and pen tests of ourinfrastructure
We support responsible disclosure
32 / 38
![Page 33: Containers, Docker, and Security: State of the Union · If you don't trust upstream, you have to ... stop using apt-get and yum with public repos rebuild everything from source verify](https://reader033.vdocuments.us/reader033/viewer/2022060501/5f1b9a05a8f62f4c0d2b3737/html5/thumbnails/33.jpg)
Tomorrow
33 / 38
![Page 34: Containers, Docker, and Security: State of the Union · If you don't trust upstream, you have to ... stop using apt-get and yum with public repos rebuild everything from source verify](https://reader033.vdocuments.us/reader033/viewer/2022060501/5f1b9a05a8f62f4c0d2b3737/html5/thumbnails/34.jpg)
Container security in the futurePersonal predictions - not Docker Inc.'s roadmap!
34 / 38
![Page 35: Containers, Docker, and Security: State of the Union · If you don't trust upstream, you have to ... stop using apt-get and yum with public repos rebuild everything from source verify](https://reader033.vdocuments.us/reader033/viewer/2022060501/5f1b9a05a8f62f4c0d2b3737/html5/thumbnails/35.jpg)
Container security in the futurePersonal predictions - not Docker Inc.'s roadmap!
Offline image audit
Hardening of immutable containers (noexec, nosuid)
Better GRSEC, PAX, LSM integration
User namespaces (eventually!)
Better default seccomp profiles
35 / 38
![Page 36: Containers, Docker, and Security: State of the Union · If you don't trust upstream, you have to ... stop using apt-get and yum with public repos rebuild everything from source verify](https://reader033.vdocuments.us/reader033/viewer/2022060501/5f1b9a05a8f62f4c0d2b3737/html5/thumbnails/36.jpg)
Last wordsDavid Mortman at DEFCON this year:
“ A year ago, [Docker and security] was pretty horrible,six months ago it wasn't so bad, and now it's pretty usable.
36 / 38
![Page 37: Containers, Docker, and Security: State of the Union · If you don't trust upstream, you have to ... stop using apt-get and yum with public repos rebuild everything from source verify](https://reader033.vdocuments.us/reader033/viewer/2022060501/5f1b9a05a8f62f4c0d2b3737/html5/thumbnails/37.jpg)
ResourcesDocker security page
Docker security presentation at DockerCon 2015 SF
Docker Security CheatSheet
Notary on GitHub
Docker Bench for Security
37 / 38
![Page 38: Containers, Docker, and Security: State of the Union · If you don't trust upstream, you have to ... stop using apt-get and yum with public repos rebuild everything from source verify](https://reader033.vdocuments.us/reader033/viewer/2022060501/5f1b9a05a8f62f4c0d2b3737/html5/thumbnails/38.jpg)
Thanks! Questions?
@jpetazzo @docker
38 / 38