securing the pipeline
TRANSCRIPT
To m D u c k e r i n g & P a t D o w n e y
SECURING THE PIPELINEIdeas, practices and food for thought to improve the security surrounding regular delivery of software to
production.
WHO ARE WE AND WHAT DO WE KNOW?
2
Dev OpsTom & Pat
WHO ARE WE AND WHAT DO WE KNOW?
3
Dev Ops
Sec
Tom & Pat
WHAT HAVE WE SEEN?
4
Insecure & Fast
“Over secure” & Slow
WHAT HAPPENS IN HERE?
5
User accounts Secure coding
Algorithm choice
Penetrationtesting
What about the pipeline!?
YOUR BUILD SYSTEM IS PRODUCTION!
6
SECURING THE PIPELINEFrom head to tail
7
PIPELINE
8
WorkstationCode Repo
CI Server
Build Agent
Deploy Agent
Pkg Repo
Local Cache
Prod.
Staging
QA
3rd party code
Devs
PIPELINE
9
WorkstationCode Repo
CI Server
Build Agent
Deploy Agent
Pkg Repo
Local Cache
Prod.
Staging
QA
3rd party code
Devs
SHARED ACCOUNTS
10
Pa$$w0rd2015
SECURE WORKSTATIONS
11
WHO COMMITED?
12
commit 4698b247268f053299230843dd1ae68e4d15a7e3 Author: You can put anything here <[email protected]> Date: Mon Jul 6 16:23:06 2015 +0100
#837: Send logs via syslog Lorem ipsum dolor sit amet, consetetur sadipscing elitr,
sed diam nonumy eirmod tempor invidunt ut labore et dolore magna aliquyam erat, sed diam voluptua. At vero eos et accusam et justo duo dolores et ea rebum. Stet clita kasd gubergren, no sea takimata sanctus est.
KEYS OR PASSWORDS?
13
USE HTTPS OR SSH
14
There’s simply no good reason not to.
CENTRALISED CONTROL
15
Code Repo User Directory
PIPELINE
16
WorkstationCode Repo
CI Server
Build Agent
Deploy Agent
Pkg Repo
Local Cache
Prod.
Staging
QA
3rd party code
Devs
HOW MUCH IS *YOUR* CODE?
17
WHERE TO START?
18
Use modelling and threat analysis to prioritise the susceptible
Discover what you depend on
Assess the origin of that code for maturity of security practices
PIPELINE
19
WorkstationCode Repo
CI Server
Build Agent
Deploy Agent
Pkg Repo
Local Cache
Prod.
Staging
QA
3rd party code
Devs
CI SERVER & IT’S AGENTS
20
It’s a remote execution problem
Separate agents to avoid compromises
Isolate builds using chroots and containers
PIPELINE
21
WorkstationCode Repo
CI Server
Build Agent
Deploy Agent
Pkg Repo
Local Cache
Prod.
Staging
QA
3rd party code
Devs
PACKAGING
22
Use package system facilities to verify and sign code
But lots of them need “root” :(
Containers and unikernels offer a possible approach
But they’re immature in other ways :(
PIPELINE
23
WorkstationCode Repo
CI Server
Build Agent
Deploy Agent
Pkg Repo
Local Cache
Prod.
Staging
QA
3rd party code
Devs
DEPLOYMENT EXECUTION
24
Deploy Agent
Web Server
Service A
Data Store
Service B
Service C
Push deployments with:
automated key based ssh!
and rights to install as root!
to all machines!
Limit the commands (e.g. via sudo and ssh)
Consider a notification and pull based approach
KEEPING SECRETS
25
KEY, CERT & SECRET MANAGEMENT
26
Secrets required for credentials
Try to use PKI where you can
If it has to be a password then encrypt them per environment.
Try not to move private keys
Plan for rotation
There’s a chaining problem. It’s hard.
CONTROL VS. AUDIT
27
CONTROL VS. AUDIT
28
Stop bad thing from being possible Know when a bad thing happened
Impact of the threat is greater than impact on
productivity
Productivity impacted too much to stop it
completely
Need to know immediately
Acceptable to know afterwards
THE “NSA” WAY
29
Log all the things
Alert on bad things
Look for patterns
Tell everyone that you’re doing it (unlike the NSA)
COMPLIANCE
30
SEGREGATION OF DUTIES
31
Not always explicitly mandated so RTFM
Good principle: “no single person…”
Bring it forward in the pipeline with pairing, PRs and code reviews
HOW TO GET THERE?
32
HOW TO GET THERE?
33
Dev Sec Ops
Collaborative
Goal
HOW TO GET THERE?
34
Structured & Objective
HOW TO GET THERE?
35
No Silver Bullet. Hard things still hard
QUESTIONS?
36
