switzerland has bunkers, we have vault · 16 $ vault operator unseal unsealing_key_1 key value---...
TRANSCRIPT
![Page 1: Switzerland has bunkers, we have Vault · 16 $ vault operator unseal unsealing_key_1 Key Value--- ----- Seal Type shamir ... 18. 19 Authentication & authorization Client Authn method](https://reader035.vdocuments.us/reader035/viewer/2022071000/5fbcf448c899a830de532b62/html5/thumbnails/1.jpg)
Switzerland has bunkers, we have Vault
Christophe Tafani-Dereeper09.11.2018
![Page 2: Switzerland has bunkers, we have Vault · 16 $ vault operator unseal unsealing_key_1 Key Value--- ----- Seal Type shamir ... 18. 19 Authentication & authorization Client Authn method](https://reader035.vdocuments.us/reader035/viewer/2022071000/5fbcf448c899a830de532b62/html5/thumbnails/2.jpg)
~$ whoami
➢ Security engineer @ Hacknowledge
➢ Threat hunting, SOC analyst, Infra, Dev(Sec)Ops
2
Christophe Tafani-Dereeper (christophetd)
![Page 3: Switzerland has bunkers, we have Vault · 16 $ vault operator unseal unsealing_key_1 Key Value--- ----- Seal Type shamir ... 18. 19 Authentication & authorization Client Authn method](https://reader035.vdocuments.us/reader035/viewer/2022071000/5fbcf448c899a830de532b62/html5/thumbnails/3.jpg)
Goal of the talk
● Present the concepts and features of Hashicorp Vault
● Demonstrate how Vault can be used in the real-world scenarios
3
![Page 4: Switzerland has bunkers, we have Vault · 16 $ vault operator unseal unsealing_key_1 Key Value--- ----- Seal Type shamir ... 18. 19 Authentication & authorization Client Authn method](https://reader035.vdocuments.us/reader035/viewer/2022071000/5fbcf448c899a830de532b62/html5/thumbnails/4.jpg)
Challenges of secret management
4
● What is a secret?
● Secrets sprawled everywhere
● Hard to know where secrets are, who has access to them
● Hard to log accesses to secrets
![Page 5: Switzerland has bunkers, we have Vault · 16 $ vault operator unseal unsealing_key_1 Key Value--- ----- Seal Type shamir ... 18. 19 Authentication & authorization Client Authn method](https://reader035.vdocuments.us/reader035/viewer/2022071000/5fbcf448c899a830de532b62/html5/thumbnails/5.jpg)
5
● What secrets were accessed? ● When? ● By whom?● How to revoke them?
BREACH
![Page 6: Switzerland has bunkers, we have Vault · 16 $ vault operator unseal unsealing_key_1 Key Value--- ----- Seal Type shamir ... 18. 19 Authentication & authorization Client Authn method](https://reader035.vdocuments.us/reader035/viewer/2022071000/5fbcf448c899a830de532b62/html5/thumbnails/6.jpg)
The Vault way
6
● Secrets are centralized in Vault
● Secrets are short-lived and revokable
● Role-based ACLs for granular access control
● Audit trail for strong accountability and non-repudiation
![Page 7: Switzerland has bunkers, we have Vault · 16 $ vault operator unseal unsealing_key_1 Key Value--- ----- Seal Type shamir ... 18. 19 Authentication & authorization Client Authn method](https://reader035.vdocuments.us/reader035/viewer/2022071000/5fbcf448c899a830de532b62/html5/thumbnails/7.jpg)
Vault 101
7
![Page 8: Switzerland has bunkers, we have Vault · 16 $ vault operator unseal unsealing_key_1 Key Value--- ----- Seal Type shamir ... 18. 19 Authentication & authorization Client Authn method](https://reader035.vdocuments.us/reader035/viewer/2022071000/5fbcf448c899a830de532b62/html5/thumbnails/8.jpg)
Hashicorp Vault
8
● First version in April 2015, 1.0 released on October 23rd 2018
● Free, with paid advanced features (not discussed in this talk)
● REST API, CLI
![Page 9: Switzerland has bunkers, we have Vault · 16 $ vault operator unseal unsealing_key_1 Key Value--- ----- Seal Type shamir ... 18. 19 Authentication & authorization Client Authn method](https://reader035.vdocuments.us/reader035/viewer/2022071000/5fbcf448c899a830de532b62/html5/thumbnails/9.jpg)
9
Secrets storage Secrets engines
Authenticationmethods
Auditing
![Page 10: Switzerland has bunkers, we have Vault · 16 $ vault operator unseal unsealing_key_1 Key Value--- ----- Seal Type shamir ... 18. 19 Authentication & authorization Client Authn method](https://reader035.vdocuments.us/reader035/viewer/2022071000/5fbcf448c899a830de532b62/html5/thumbnails/10.jpg)
10
Secrets storage Secrets engines
Authenticationmethods
Auditing
![Page 11: Switzerland has bunkers, we have Vault · 16 $ vault operator unseal unsealing_key_1 Key Value--- ----- Seal Type shamir ... 18. 19 Authentication & authorization Client Authn method](https://reader035.vdocuments.us/reader035/viewer/2022071000/5fbcf448c899a830de532b62/html5/thumbnails/11.jpg)
11
Secrets storage
● Secrets are stored encrypted in a storage backend of your choice○ Filesystem, MySQL, S3, etcd, Consul…
● Storage backends are untrusted○ Compromising the storage doesn’t allow to compromise the secrets stored in Vault○ Authenticated encryption (AES GCM)
● How does Vault know how to decrypt its storage?⇒ Unsealing process
![Page 12: Switzerland has bunkers, we have Vault · 16 $ vault operator unseal unsealing_key_1 Key Value--- ----- Seal Type shamir ... 18. 19 Authentication & authorization Client Authn method](https://reader035.vdocuments.us/reader035/viewer/2022071000/5fbcf448c899a830de532b62/html5/thumbnails/12.jpg)
12
Unsealing
Encrypted Vault data
??
Unsealing
Encrypted Vault data
decryption/encryption with master encryption key
![Page 13: Switzerland has bunkers, we have Vault · 16 $ vault operator unseal unsealing_key_1 Key Value--- ----- Seal Type shamir ... 18. 19 Authentication & authorization Client Authn method](https://reader035.vdocuments.us/reader035/viewer/2022071000/5fbcf448c899a830de532b62/html5/thumbnails/13.jpg)
Master key splitting
● The master encryption key is split using Shamir's Secret Sharing algorithm
● The different parts are distributed to several trusted individuals
● The master key can be reconstructed with a certain number of key shares○ num_shares = 3, threshold = 2
⇒ Any combination of 2 administrators can unseal Vault
13
![Page 14: Switzerland has bunkers, we have Vault · 16 $ vault operator unseal unsealing_key_1 Key Value--- ----- Seal Type shamir ... 18. 19 Authentication & authorization Client Authn method](https://reader035.vdocuments.us/reader035/viewer/2022071000/5fbcf448c899a830de532b62/html5/thumbnails/14.jpg)
14
$ vault operator init -key-shares=3 -key-threshold=2
Unseal Key 1: uCLmRwheyiBjI38so2ayYtearJyENppycC6XU//oRcHpUnseal Key 2: 7VrbOoxN6y2X/ieTKhAz4BILTnenFMOYj2IzvVISd4gaUnseal Key 3: ZkNnWwYnj20VGF+Ib9brR7oeHY+3dfkWdtaw2HgGwAv5
![Page 15: Switzerland has bunkers, we have Vault · 16 $ vault operator unseal unsealing_key_1 Key Value--- ----- Seal Type shamir ... 18. 19 Authentication & authorization Client Authn method](https://reader035.vdocuments.us/reader035/viewer/2022071000/5fbcf448c899a830de532b62/html5/thumbnails/15.jpg)
15
$ vault operator init \ -key-shares=3 \ -key-threshold=2 \ -pgp-keys=keybase:christophetd,keybase:milkmix,keybase:lbarman
Unseal Key 1: (encrypted unseal key 1)Unseal Key 2: (encrypted unseal key 2)Unseal Key 3: (encrypted unseal key 3)
![Page 16: Switzerland has bunkers, we have Vault · 16 $ vault operator unseal unsealing_key_1 Key Value--- ----- Seal Type shamir ... 18. 19 Authentication & authorization Client Authn method](https://reader035.vdocuments.us/reader035/viewer/2022071000/5fbcf448c899a830de532b62/html5/thumbnails/16.jpg)
Unsealing process
16
$ vault operator unseal unsealing_key_1
Key Value--- -----Seal Type shamirSealed trueTotal Shares 3Threshold 2Unseal Progress 1/2
$ vault operator unseal unsealing_key_2
Key Value--- -----Seal Type shamirSealed false
Admin 1 Admin 2
![Page 17: Switzerland has bunkers, we have Vault · 16 $ vault operator unseal unsealing_key_1 Key Value--- ----- Seal Type shamir ... 18. 19 Authentication & authorization Client Authn method](https://reader035.vdocuments.us/reader035/viewer/2022071000/5fbcf448c899a830de532b62/html5/thumbnails/17.jpg)
17
Secrets storage Secrets engines
Authenticationmethods
Auditing
![Page 18: Switzerland has bunkers, we have Vault · 16 $ vault operator unseal unsealing_key_1 Key Value--- ----- Seal Type shamir ... 18. 19 Authentication & authorization Client Authn method](https://reader035.vdocuments.us/reader035/viewer/2022071000/5fbcf448c899a830de532b62/html5/thumbnails/18.jpg)
Authentication & authorization
● Clients authenticate to Vault using an authentication method○ For humans: LDAP, RADIUS, Github, username/password○ For applications: AppRole, Kubernetes RBAC, AWS instance role
● Authorization: ○ Operators define ACLs on secret paths○ Authentication engine configured to map authentication-method-specific user groups to Vault ACLs
e.g. Users of the “engineering” team on Github map to the ACL “engineer” in Vault
18
![Page 19: Switzerland has bunkers, we have Vault · 16 $ vault operator unseal unsealing_key_1 Key Value--- ----- Seal Type shamir ... 18. 19 Authentication & authorization Client Authn method](https://reader035.vdocuments.us/reader035/viewer/2022071000/5fbcf448c899a830de532b62/html5/thumbnails/19.jpg)
19
Authentication & authorization
Client
Authn methodauthenticates
token(bound to ACLs)
Vault APIsecrets engines,
configuration, etc.token
![Page 20: Switzerland has bunkers, we have Vault · 16 $ vault operator unseal unsealing_key_1 Key Value--- ----- Seal Type shamir ... 18. 19 Authentication & authorization Client Authn method](https://reader035.vdocuments.us/reader035/viewer/2022071000/5fbcf448c899a830de532b62/html5/thumbnails/20.jpg)
20
Example: Github authentication
![Page 21: Switzerland has bunkers, we have Vault · 16 $ vault operator unseal unsealing_key_1 Key Value--- ----- Seal Type shamir ... 18. 19 Authentication & authorization Client Authn method](https://reader035.vdocuments.us/reader035/viewer/2022071000/5fbcf448c899a830de532b62/html5/thumbnails/21.jpg)
21
Initial setup
1. Create an ACL (“policy”) for the engineering team2. Enable the Github authentication method3. Map Github users from the engineering team to the engineering team ACL
Example: Github authentication
Usage
1. User authenticates to Vault using a Github access token2. Vault returns a token (bound to the ACL for the engineering team)3. User can use this token to interact with Vault
![Page 22: Switzerland has bunkers, we have Vault · 16 $ vault operator unseal unsealing_key_1 Key Value--- ----- Seal Type shamir ... 18. 19 Authentication & authorization Client Authn method](https://reader035.vdocuments.us/reader035/viewer/2022071000/5fbcf448c899a830de532b62/html5/thumbnails/22.jpg)
22
$ vault policy write engineers-policy - <<POLICY
path "static/engineering/*" { capabilities = ["create", "read", "update", "list"]}
POLICY
Initial setup (Operator): Create an ACL (“policy”) for the engineering team:
Example: Github authentication
![Page 23: Switzerland has bunkers, we have Vault · 16 $ vault operator unseal unsealing_key_1 Key Value--- ----- Seal Type shamir ... 18. 19 Authentication & authorization Client Authn method](https://reader035.vdocuments.us/reader035/viewer/2022071000/5fbcf448c899a830de532b62/html5/thumbnails/23.jpg)
$ vault auth enable github
Initial setup (Operator): Set up the Github authentication method
Example: Github authentication
$ vault write auth/github/config organization=Hacknowledge
$ vault write auth/github/map/teams/engineers value=engineers-policy
23
![Page 24: Switzerland has bunkers, we have Vault · 16 $ vault operator unseal unsealing_key_1 Key Value--- ----- Seal Type shamir ... 18. 19 Authentication & authorization Client Authn method](https://reader035.vdocuments.us/reader035/viewer/2022071000/5fbcf448c899a830de532b62/html5/thumbnails/24.jpg)
24
$ vault login -method=github
GitHub Personal Access Token: ***************
Usage (normal user):
Example: Github authentication
Success! You are now authenticated.
Key Value--- -----token 24GKildxW2aOy0cBFpNOEmsY[...]token_policies [default engineers-policy]
![Page 25: Switzerland has bunkers, we have Vault · 16 $ vault operator unseal unsealing_key_1 Key Value--- ----- Seal Type shamir ... 18. 19 Authentication & authorization Client Authn method](https://reader035.vdocuments.us/reader035/viewer/2022071000/5fbcf448c899a830de532b62/html5/thumbnails/25.jpg)
25
$ vault read static/engineering/secret=== Data ===Key Value--- -----foo bar
Example: Github authentication
Usage (normal user):
$ vault read static/ops/secretError reading static/ops/secret: Error making API request.permission denied
![Page 26: Switzerland has bunkers, we have Vault · 16 $ vault operator unseal unsealing_key_1 Key Value--- ----- Seal Type shamir ... 18. 19 Authentication & authorization Client Authn method](https://reader035.vdocuments.us/reader035/viewer/2022071000/5fbcf448c899a830de532b62/html5/thumbnails/26.jpg)
26
Authentication and authorization wrap-up
● Policies define the permissions each client has
● Authentication methods allow to map external identities to a set of policies
![Page 27: Switzerland has bunkers, we have Vault · 16 $ vault operator unseal unsealing_key_1 Key Value--- ----- Seal Type shamir ... 18. 19 Authentication & authorization Client Authn method](https://reader035.vdocuments.us/reader035/viewer/2022071000/5fbcf448c899a830de532b62/html5/thumbnails/27.jpg)
27
Secrets storage Secrets engines
Authenticationmethods
Auditing
![Page 28: Switzerland has bunkers, we have Vault · 16 $ vault operator unseal unsealing_key_1 Key Value--- ----- Seal Type shamir ... 18. 19 Authentication & authorization Client Authn method](https://reader035.vdocuments.us/reader035/viewer/2022071000/5fbcf448c899a830de532b62/html5/thumbnails/28.jpg)
Secrets engines● Secrets engines are at the core of Vault
○ allow to store, generate, and manage all kind of secrets
● Lots of different secrets engines○ Key-value (example on previous slide)○ MySQL, PostgreSQL○ AWS, Azure○ SSH○ PKI
28
![Page 29: Switzerland has bunkers, we have Vault · 16 $ vault operator unseal unsealing_key_1 Key Value--- ----- Seal Type shamir ... 18. 19 Authentication & authorization Client Authn method](https://reader035.vdocuments.us/reader035/viewer/2022071000/5fbcf448c899a830de532b62/html5/thumbnails/29.jpg)
Secrets engines● Everything is a path: secrets engines can be mounted (enabled) and
unmounted (disabled) in Vault
● Each secret has a path within the engine they belong to(e.g. static/banking/credit-card)
29
$ vault secrets enable -path=static kvSuccess! Enabled the kv secrets engine at: static/
![Page 30: Switzerland has bunkers, we have Vault · 16 $ vault operator unseal unsealing_key_1 Key Value--- ----- Seal Type shamir ... 18. 19 Authentication & authorization Client Authn method](https://reader035.vdocuments.us/reader035/viewer/2022071000/5fbcf448c899a830de532b62/html5/thumbnails/30.jpg)
Static secrets engine: Key Value● Most basic secret engine - can store arbitrary key-value pairs
30
$ vault write static/banking/credit-card number=123456 exp=01/2021 Success! Data written to: static/banking/credit-card
$ vault read static/banking/credit-cardKey Value--- -----exp 01/2021number 123456
![Page 31: Switzerland has bunkers, we have Vault · 16 $ vault operator unseal unsealing_key_1 Key Value--- ----- Seal Type shamir ... 18. 19 Authentication & authorization Client Authn method](https://reader035.vdocuments.us/reader035/viewer/2022071000/5fbcf448c899a830de532b62/html5/thumbnails/31.jpg)
Dynamic secrets engines
● A dynamic secret engine generates secrets on-the-fly○ MySQL/PostgreSQL: create user account ○ AWS: generate IAM credentials○ PKI: sign certificate
● Dynamic secrets are supposed to be short-lived and revokable
31
![Page 32: Switzerland has bunkers, we have Vault · 16 $ vault operator unseal unsealing_key_1 Key Value--- ----- Seal Type shamir ... 18. 19 Authentication & authorization Client Authn method](https://reader035.vdocuments.us/reader035/viewer/2022071000/5fbcf448c899a830de532b62/html5/thumbnails/32.jpg)
Dynamic secret engine example: MySQL
● Vault holds root MySQL credentials
● Vault dynamically generates MySQL credentials with specific rights
● Credentials are limited in time and can be revoked
32
![Page 33: Switzerland has bunkers, we have Vault · 16 $ vault operator unseal unsealing_key_1 Key Value--- ----- Seal Type shamir ... 18. 19 Authentication & authorization Client Authn method](https://reader035.vdocuments.us/reader035/viewer/2022071000/5fbcf448c899a830de532b62/html5/thumbnails/33.jpg)
Dynamic secret engine example: MySQL
33
User / application
1. Get temporary read-only credentials
2. Dynamically create user account with read-only rights
3. Return credentials
![Page 34: Switzerland has bunkers, we have Vault · 16 $ vault operator unseal unsealing_key_1 Key Value--- ----- Seal Type shamir ... 18. 19 Authentication & authorization Client Authn method](https://reader035.vdocuments.us/reader035/viewer/2022071000/5fbcf448c899a830de532b62/html5/thumbnails/34.jpg)
Dynamic secret engine example: MySQL
34
Setup (operator):
$ vault secrets enable -path=db database
$ vault write db/config/mysql-prod \ plugin_name="mysql-database-plugin" \ connection_url="{{username}}:{{password}}@tcp(127.0.0.1:3306)/" \ username="root" \ password="my-secret-pw" \ allowed_roles="mysql-prod-readonly"
$ vault write db/roles/mysql-prod-readonly \ db_name=mysql-prod \ creation_statements="CREATE USER '{{name}}'@'%' IDENTIFIED BY '{{password}}';\ GRANT SELECT ON *.* TO '{{name}}'@'%';" \ default_ttl="10m"
![Page 35: Switzerland has bunkers, we have Vault · 16 $ vault operator unseal unsealing_key_1 Key Value--- ----- Seal Type shamir ... 18. 19 Authentication & authorization Client Authn method](https://reader035.vdocuments.us/reader035/viewer/2022071000/5fbcf448c899a830de532b62/html5/thumbnails/35.jpg)
Dynamic secret engine example: MySQL
35
$ vault read db/creds/mysql-prod-readonly
Key Value--- -----lease_id db/creds/mysql-prod-readonly/4bFeHHV4fzJSs6T9xLQrFhdHlease_duration 10mlease_renewable truepassword A1a-HpsqK2m547gSP5I0username v-root-mydb-reado-6akkZS1xkOsm2P
Usage (user or application):
Access control with an ACL!
![Page 36: Switzerland has bunkers, we have Vault · 16 $ vault operator unseal unsealing_key_1 Key Value--- ----- Seal Type shamir ... 18. 19 Authentication & authorization Client Authn method](https://reader035.vdocuments.us/reader035/viewer/2022071000/5fbcf448c899a830de532b62/html5/thumbnails/36.jpg)
Dynamic secrets: leases
36
● Most dynamic secrets have a lease
lease = { id, time_to_live, is_renewable }
● In our previous MySQL example, we had:
$ vault read db/creds/mydb-readonly
Key Value--- -----lease_id db/creds/mysql-prod-readonly/4bFeHHV4fzJSs6T9xLQrFhdHlease_duration 10mlease_renewable truepassword A1a-HpsqK2m547gSP5I0username v-root-mydb-reado-6akkZS1xkOsm2P
![Page 37: Switzerland has bunkers, we have Vault · 16 $ vault operator unseal unsealing_key_1 Key Value--- ----- Seal Type shamir ... 18. 19 Authentication & authorization Client Authn method](https://reader035.vdocuments.us/reader035/viewer/2022071000/5fbcf448c899a830de532b62/html5/thumbnails/37.jpg)
Dynamic secrets: leases
37
● Leases can be renewed
(makes the previously obtained credentials valid for 10 more minutes)
$ vault lease renew db/creds/mysql-prod-readonly/4bFeHHV4fzJSs6T9xLQrFhdH
Key Value--- -----lease_id db/creds/mysql-prod-readonly/4bFeHHV4fzJSs6T9xLQrFhdHlease_duration 10mlease_renewable true
![Page 38: Switzerland has bunkers, we have Vault · 16 $ vault operator unseal unsealing_key_1 Key Value--- ----- Seal Type shamir ... 18. 19 Authentication & authorization Client Authn method](https://reader035.vdocuments.us/reader035/viewer/2022071000/5fbcf448c899a830de532b62/html5/thumbnails/38.jpg)
Dynamic secrets: leases
38
● Leases can be revoked by operators, individually or by prefix
$ vault lease revoke db/creds/mysql-prod-readonly/4bFeHHV4fzJSs6T9xLQrFhdH
Success! Revoked lease: db/creds/mysql-prod-readonly/4bFeHHV4fzJSs6T9xLQrFhdH
$ vault lease revoke -prefix db/creds/mysql-prod-readonly
Success! Revoked any leases with prefix: db/creds/mysql-prod-readonly
![Page 39: Switzerland has bunkers, we have Vault · 16 $ vault operator unseal unsealing_key_1 Key Value--- ----- Seal Type shamir ... 18. 19 Authentication & authorization Client Authn method](https://reader035.vdocuments.us/reader035/viewer/2022071000/5fbcf448c899a830de532b62/html5/thumbnails/39.jpg)
Additional concept: Response Wrapping● Building block that can be used in more complex workflows
● When party A needs to communicate a secret to party B over an insecure channel
39
Party A
read secret, wrap response
single-use wrapping token
Party B
single-use wrapping token
unwrap
secret data
![Page 40: Switzerland has bunkers, we have Vault · 16 $ vault operator unseal unsealing_key_1 Key Value--- ----- Seal Type shamir ... 18. 19 Authentication & authorization Client Authn method](https://reader035.vdocuments.us/reader035/viewer/2022071000/5fbcf448c899a830de532b62/html5/thumbnails/40.jpg)
Additional concept: Response Wrapping
● Coverage: the transmitted information is only a reference to the actual secret
● Malfeasance detection: party B detects if the communication has been intercepted
○ Vault will tell it that the wrapping token is not valid○ Party B can then raise an alert
● Limits exposure lifetime○ wrapping token typically expires very quickly○ its lifetime is independant than the TTL of the secret it wraps
● (Does not provide confidentiality)40
![Page 41: Switzerland has bunkers, we have Vault · 16 $ vault operator unseal unsealing_key_1 Key Value--- ----- Seal Type shamir ... 18. 19 Authentication & authorization Client Authn method](https://reader035.vdocuments.us/reader035/viewer/2022071000/5fbcf448c899a830de532b62/html5/thumbnails/41.jpg)
41
Secrets storage Secrets engines
Authenticationmethods
Auditing
![Page 42: Switzerland has bunkers, we have Vault · 16 $ vault operator unseal unsealing_key_1 Key Value--- ----- Seal Type shamir ... 18. 19 Authentication & authorization Client Authn method](https://reader035.vdocuments.us/reader035/viewer/2022071000/5fbcf448c899a830de532b62/html5/thumbnails/42.jpg)
Audit log
42
● Vault has an audit log for every request / response
● Can be shipped to syslog, or local file
SIEMaudit log
![Page 43: Switzerland has bunkers, we have Vault · 16 $ vault operator unseal unsealing_key_1 Key Value--- ----- Seal Type shamir ... 18. 19 Authentication & authorization Client Authn method](https://reader035.vdocuments.us/reader035/viewer/2022071000/5fbcf448c899a830de532b62/html5/thumbnails/43.jpg)
Audit log
43
{ "time": "2018-02-31T13:37:37.123Z", "type": "request", "auth": {
"display_name": "github-christophetd","policies": [
"default", "engineers-policy"
],"metadata": {
"org": "Hacknowledge", "username": "christophetd"
}, }, "request": {
"id": "97166a54-6b7b-f577-749a-96f191c9a10c","operation": "read","path": "secret/supersecret","remote_address": "10.0.1.47",
}, "error": "1 error occurred:\n\n* permission denied"}
![Page 44: Switzerland has bunkers, we have Vault · 16 $ vault operator unseal unsealing_key_1 Key Value--- ----- Seal Type shamir ... 18. 19 Authentication & authorization Client Authn method](https://reader035.vdocuments.us/reader035/viewer/2022071000/5fbcf448c899a830de532b62/html5/thumbnails/44.jpg)
Audit log use-cases
44
● Anomaly detection○ Access denied errors○ Failed authentications
● Logs correlation
● “Honey secrets”○ Give an application access to secret/honey○ Allow the application to read the policy attached to its token (sys/policy/app-policy)○ Raise alert if secret/honey is accessed - can indicate an attacker enumerating its privileges
![Page 45: Switzerland has bunkers, we have Vault · 16 $ vault operator unseal unsealing_key_1 Key Value--- ----- Seal Type shamir ... 18. 19 Authentication & authorization Client Authn method](https://reader035.vdocuments.us/reader035/viewer/2022071000/5fbcf448c899a830de532b62/html5/thumbnails/45.jpg)
45
Hands-on with Vault
![Page 46: Switzerland has bunkers, we have Vault · 16 $ vault operator unseal unsealing_key_1 Key Value--- ----- Seal Type shamir ... 18. 19 Authentication & authorization Client Authn method](https://reader035.vdocuments.us/reader035/viewer/2022071000/5fbcf448c899a830de532b62/html5/thumbnails/46.jpg)
46
Scenario #1: SSH access management
● Context:○ You have a fleet of Linux servers○ You want to provide SSH accesses to your team in a scalable way
● Approaches○ 1 Linux user per employee per machine○ 1 user on all machines, employees’ public keys in the authorized_keys on each machine○ PAM○ Vault’s SSH secret backend
![Page 47: Switzerland has bunkers, we have Vault · 16 $ vault operator unseal unsealing_key_1 Key Value--- ----- Seal Type shamir ... 18. 19 Authentication & authorization Client Authn method](https://reader035.vdocuments.us/reader035/viewer/2022071000/5fbcf448c899a830de532b62/html5/thumbnails/47.jpg)
Scenario #1: SSH access management
● Vault holds a SSH CA key, signs employees’ public keys
● Linux servers trust Vault’s CA certificate
● Built-in OpenSSH feature!○ 0 additional software to install○ 0 communication needed between Linux servers and Vault
47
![Page 48: Switzerland has bunkers, we have Vault · 16 $ vault operator unseal unsealing_key_1 Key Value--- ----- Seal Type shamir ... 18. 19 Authentication & authorization Client Authn method](https://reader035.vdocuments.us/reader035/viewer/2022071000/5fbcf448c899a830de532b62/html5/thumbnails/48.jpg)
Scenario #1: SSH access management
48
Server running OpenSSH
CA cert
CA key
trusts
signs
authenticatesSSH pubkey signed by Vault
SSH pubkey
![Page 49: Switzerland has bunkers, we have Vault · 16 $ vault operator unseal unsealing_key_1 Key Value--- ----- Seal Type shamir ... 18. 19 Authentication & authorization Client Authn method](https://reader035.vdocuments.us/reader035/viewer/2022071000/5fbcf448c899a830de532b62/html5/thumbnails/49.jpg)
Scenario #1: Initial setup phase
● Enable Vault’s SSH secret backend
● Generate a SSH CA certificate and key (only stored in Vault)
49
$ vault secrets enable sshSuccess! Enabled the ssh secrets engine at: ssh/
$ vault write ssh/config/ca generate_signing_key=true
Key Value--- -----public_key ssh-rsa AAAAB3NzaC…..
![Page 50: Switzerland has bunkers, we have Vault · 16 $ vault operator unseal unsealing_key_1 Key Value--- ----- Seal Type shamir ... 18. 19 Authentication & authorization Client Authn method](https://reader035.vdocuments.us/reader035/viewer/2022071000/5fbcf448c899a830de532b62/html5/thumbnails/50.jpg)
Scenario #1: Initial setup phase
● Deploy Vault’s SSH CA certificate as a trusted SSH CA on Linux machines
50
TrustedUserCAKeys /etc/ssh/vault-ssh-ca.crt
/etc/ssh/sshd_config
![Page 51: Switzerland has bunkers, we have Vault · 16 $ vault operator unseal unsealing_key_1 Key Value--- ----- Seal Type shamir ... 18. 19 Authentication & authorization Client Authn method](https://reader035.vdocuments.us/reader035/viewer/2022071000/5fbcf448c899a830de532b62/html5/thumbnails/51.jpg)
● Create a role in the SSH secrets engine, specifying…○ A TTL: for how much time should Vault sign users’ public keys?○ A remote user to allow connection as○ (optionally) A CIDR list from which access should be allowed○ (optionally) SSH features to allow (PTY, port forwarding, etc)
51
Scenario #1: Initial setup phase
$ vault write -f ssh/roles/developer - <<EOF { "ttl": "10m", "allowed_users": "developer,tomcat", "default_user": "developer", "default_critical_options": { "source-address": "10.0.0.0/24" }, "default_extensions": { "permit-pty": "", "permit-port-forwarding": "" }, "allow_user_certificates": true, "key_type": "ca"}EOF
![Page 52: Switzerland has bunkers, we have Vault · 16 $ vault operator unseal unsealing_key_1 Key Value--- ----- Seal Type shamir ... 18. 19 Authentication & authorization Client Authn method](https://reader035.vdocuments.us/reader035/viewer/2022071000/5fbcf448c899a830de532b62/html5/thumbnails/52.jpg)
● Ask Vault to sign our SSH public key
52
Scenario #1: Usage
$ vault write ssh/sign/developer \ [email protected]/id_rsa.pub \ valid_principals=developer
Key Value--- -----serial_number 458e609f5eed0a8asigned_key [email protected] AAAA...
$ ssh -i signed_key.pub [email protected]
Welcome to Ubuntu 16.04.4 LTS (GNU/Linux 4.13.0-36-generic x86_64)developer@server:~$
● Connect to a Linux server trusting Vault’s SSH CA
![Page 53: Switzerland has bunkers, we have Vault · 16 $ vault operator unseal unsealing_key_1 Key Value--- ----- Seal Type shamir ... 18. 19 Authentication & authorization Client Authn method](https://reader035.vdocuments.us/reader035/viewer/2022071000/5fbcf448c899a830de532b62/html5/thumbnails/53.jpg)
● vault ssh wrapper can do both in a single command
53
Scenario #1: Usage (wrapper)
$ vault ssh -mode=ca -role=developer [email protected]
![Page 54: Switzerland has bunkers, we have Vault · 16 $ vault operator unseal unsealing_key_1 Key Value--- ----- Seal Type shamir ... 18. 19 Authentication & authorization Client Authn method](https://reader035.vdocuments.us/reader035/viewer/2022071000/5fbcf448c899a830de532b62/html5/thumbnails/54.jpg)
● Once the TTL is over, the signed key is not valid anymore
● What TTL to use? ○ Tradeoff between performance / availability and easy revokation
54
Scenario #1: TTL
$ ssh -i .ssh/id_rsa -i signed_key.pub [email protected]
[email protected]: Permission denied (publickey).
![Page 55: Switzerland has bunkers, we have Vault · 16 $ vault operator unseal unsealing_key_1 Key Value--- ----- Seal Type shamir ... 18. 19 Authentication & authorization Client Authn method](https://reader035.vdocuments.us/reader035/viewer/2022071000/5fbcf448c899a830de532b62/html5/thumbnails/55.jpg)
55
● Authentication easy for humans, harder for applications
● Our requirements:○ Applications should be deployable automatically (e.g. via a CI/CD pipeline)○ Each application should have a dedicated policy only allowing it to retrieve its
own secrets
● Most of the time, AppRole authentication method is the way to go○ but it only provides a building block
Scenario #2: Authenticating applications
![Page 56: Switzerland has bunkers, we have Vault · 16 $ vault operator unseal unsealing_key_1 Key Value--- ----- Seal Type shamir ... 18. 19 Authentication & authorization Client Authn method](https://reader035.vdocuments.us/reader035/viewer/2022071000/5fbcf448c899a830de532b62/html5/thumbnails/56.jpg)
56
Scenario #2: Authenticating applications with AppRole
App
role-id, secret-id
token
How does the application know it?
● Tentative 1: Hardcode the secret-id on the VM/container where the app runs⇒ But how do you initially get the secret-id?
● Tentative 2: Have the CI/CD inject the secret-id in the VM/container at deployment time⇒ But how can the CI/CD authenticate to Vault to have access to the secret-id?
![Page 57: Switzerland has bunkers, we have Vault · 16 $ vault operator unseal unsealing_key_1 Key Value--- ----- Seal Type shamir ... 18. 19 Authentication & authorization Client Authn method](https://reader035.vdocuments.us/reader035/viewer/2022071000/5fbcf448c899a830de532b62/html5/thumbnails/57.jpg)
57
● The platform assigns a cryptographic and verifiable identity to each application instance○ AWS: IAM EC2/ECS instance role○ Kubernetes: Pod service account
● At runtime, the platform gives an easy way to the application to prove its identity○ AWS: Metadata service running on 169.254.169.254 (only accessible from the instance)○ Kubernetes: Injected in a volume /var/run/secrets/
● Vault has several authentication engines to allow application authentication with their platform-specific identity
○ AWS, Azure, AliCloud, Google Cloud, Kubernetes secrets engines
Option 1: Platform integration
![Page 58: Switzerland has bunkers, we have Vault · 16 $ vault operator unseal unsealing_key_1 Key Value--- ----- Seal Type shamir ... 18. 19 Authentication & authorization Client Authn method](https://reader035.vdocuments.us/reader035/viewer/2022071000/5fbcf448c899a830de532b62/html5/thumbnails/58.jpg)
Platform 58
App
Platform control plane(AWS IAM API, Kubernetes API, …)
1. Retrieve platform specific, verifiable, identity
2. Use it to authenticate to Vault
3. Verify identity
4. Map platform identity to Vault policies
Vault token!
Option 1: Platform integration
![Page 59: Switzerland has bunkers, we have Vault · 16 $ vault operator unseal unsealing_key_1 Key Value--- ----- Seal Type shamir ... 18. 19 Authentication & authorization Client Authn method](https://reader035.vdocuments.us/reader035/viewer/2022071000/5fbcf448c899a830de532b62/html5/thumbnails/59.jpg)
59
● e.g. your applications run in VMs on an on-prem ESXi cluster
● How do you pass the authentication secret (secret-id) to your applications?
● Challenging problem - no silver bullet○ highly dependent on the environment and technologies in use○ hard to have a solution as secure as with platform integration
Option 2: No platform integration
![Page 60: Switzerland has bunkers, we have Vault · 16 $ vault operator unseal unsealing_key_1 Key Value--- ----- Seal Type shamir ... 18. 19 Authentication & authorization Client Authn method](https://reader035.vdocuments.us/reader035/viewer/2022071000/5fbcf448c899a830de532b62/html5/thumbnails/60.jpg)
60
● Trusted orchestrator: We extends our trust to an additional componente.g. Jenkins, Gitlab CI
● Orchestrator is authenticated to Vault
● Orchestrator passes the AppRole secret-id to application it deploys
Option 2: Trusted orchestrator
![Page 61: Switzerland has bunkers, we have Vault · 16 $ vault operator unseal unsealing_key_1 Key Value--- ----- Seal Type shamir ... 18. 19 Authentication & authorization Client Authn method](https://reader035.vdocuments.us/reader035/viewer/2022071000/5fbcf448c899a830de532b62/html5/thumbnails/61.jpg)
61
● Orchestrator:○ Can only retrieve the application’s AppRole secret-id (cannot read application secrets)
○ Is in a different network than the applications it deploys
● Applications: ○ authenticate using a dedicated AppRole○ AppRole is configured to only allow authentications from the apps network○ can only read their own secrets
path "auth/approle/role/my-app/secret-id" { capabilities = ["create", "update"]}
Option 2: Trusted orchestrator
![Page 62: Switzerland has bunkers, we have Vault · 16 $ vault operator unseal unsealing_key_1 Key Value--- ----- Seal Type shamir ... 18. 19 Authentication & authorization Client Authn method](https://reader035.vdocuments.us/reader035/viewer/2022071000/5fbcf448c899a830de532b62/html5/thumbnails/62.jpg)
Network zone
Network zone
62
Trusted orchestrator scenario (with response wrapping)
Orchestrator
single-use wrapping token
Application
Deploy + pass wrapping token unwrap
secret-id
read auth/approle/role/my-role/secret-id, wrap response
AppRole authentication with
{role-id, secret-id}
Vault token
![Page 63: Switzerland has bunkers, we have Vault · 16 $ vault operator unseal unsealing_key_1 Key Value--- ----- Seal Type shamir ... 18. 19 Authentication & authorization Client Authn method](https://reader035.vdocuments.us/reader035/viewer/2022071000/5fbcf448c899a830de532b62/html5/thumbnails/63.jpg)
Network zone
Network zone
63
Trusted orchestrator scenario
Orchestrator
secret-id
Application
Deploy + pass secret-id
read auth/approle/role/my-role/secret-id
AppRole authentication with
{role-id, secret-id}
Vault token
![Page 64: Switzerland has bunkers, we have Vault · 16 $ vault operator unseal unsealing_key_1 Key Value--- ----- Seal Type shamir ... 18. 19 Authentication & authorization Client Authn method](https://reader035.vdocuments.us/reader035/viewer/2022071000/5fbcf448c899a830de532b62/html5/thumbnails/64.jpg)
64
Trusted orchestrator scenario: result #1Good:
● Applications can only access secrets as defined by their AppRole policy● Orchestrator cannot access applications’ secrets
○ It cannot authenticate using the application’s secret-id (CIDR restrictions)
Bad:
● A compromised orchestrator can be used to deploy a backdoored application that leaks secrets
Can we do better?
![Page 65: Switzerland has bunkers, we have Vault · 16 $ vault operator unseal unsealing_key_1 Key Value--- ----- Seal Type shamir ... 18. 19 Authentication & authorization Client Authn method](https://reader035.vdocuments.us/reader035/viewer/2022071000/5fbcf448c899a830de532b62/html5/thumbnails/65.jpg)
65
● Problem: Orchestrator has total control over the nodes where the apps run
● Consequence: Compromised orchestrator ⇒ compromised apps secrets
Application node
App
https://schd.ws/hosted_files/appsecusa2015/a5/Turtles.pdf
Orchestrator clientOrchestrator server
secret-id
Artifacts repository
Pull app artifactAuthenticate using {role-id,
secret-id}
token
● Potential solution:
![Page 66: Switzerland has bunkers, we have Vault · 16 $ vault operator unseal unsealing_key_1 Key Value--- ----- Seal Type shamir ... 18. 19 Authentication & authorization Client Authn method](https://reader035.vdocuments.us/reader035/viewer/2022071000/5fbcf448c899a830de532b62/html5/thumbnails/66.jpg)
66
Trusted orchestrator scenario: result #2
● Orchestrator cannot deploy backdoored applications anymore
● It must still authenticate to Vault by some way (e.g. hardcoded token)○ … but compromising the orchestrator becomes much less interesting!
● Potential improvement: response wrapping
![Page 67: Switzerland has bunkers, we have Vault · 16 $ vault operator unseal unsealing_key_1 Key Value--- ----- Seal Type shamir ... 18. 19 Authentication & authorization Client Authn method](https://reader035.vdocuments.us/reader035/viewer/2022071000/5fbcf448c899a830de532b62/html5/thumbnails/67.jpg)
● Transit secret backend: Encryption As a Service
● PKI secrets backend
● High-Availability mode
● Web UI
Wrapping up - Other Vault capabilities
67
![Page 68: Switzerland has bunkers, we have Vault · 16 $ vault operator unseal unsealing_key_1 Key Value--- ----- Seal Type shamir ... 18. 19 Authentication & authorization Client Authn method](https://reader035.vdocuments.us/reader035/viewer/2022071000/5fbcf448c899a830de532b62/html5/thumbnails/68.jpg)
● Unsealing process hard to automate
● Can easily become a single point of failure
● Not all secrets can be dynamic
● Added complexity
Wrapping up - Vault limitations
68
![Page 69: Switzerland has bunkers, we have Vault · 16 $ vault operator unseal unsealing_key_1 Key Value--- ----- Seal Type shamir ... 18. 19 Authentication & authorization Client Authn method](https://reader035.vdocuments.us/reader035/viewer/2022071000/5fbcf448c899a830de532b62/html5/thumbnails/69.jpg)
● Provider-dependent solutions:○ AWS KMS○ Google Cloud KMS○ Azure Key Vault
● Hardware Security Modules
● Software Solutions○ Square’s KeyWhiz○ Pinterest’s Knox
Wrapping up - Vault alternatives
69
![Page 70: Switzerland has bunkers, we have Vault · 16 $ vault operator unseal unsealing_key_1 Key Value--- ----- Seal Type shamir ... 18. 19 Authentication & authorization Client Authn method](https://reader035.vdocuments.us/reader035/viewer/2022071000/5fbcf448c899a830de532b62/html5/thumbnails/70.jpg)
Readings and resources● Hashicorp Learning center
https://learn.hashicorp.com/vault/
● “Secrets at Scale: Automated Bootstrapping of Secrets & Identity in the Cloud” (Netflix)https://www.youtube.com/watch?v=15H5uCj1hlE
● “The Secure Introduction Problem: Getting Secrets Into Containers”https://slideshare.net/DynamicInfraDays/containerdays-nyc-2016-the-secure-introduction-problem-getting-secrets-into-containers-jeff-mitchell
● “Secret Security Turtles”https://blog.alanthatcher.io/vault-security-turtles
70