Puppet CA: Certificates explainedThomas Gelf - PuppetCamp Düsseldorf 2014
Thomas Gelf, nice to meet you!
joined NETWAYS in 2010
formerly more than ten years of...
web (application) development
routing/switching: bank/ISP backbones
ISP: Mail, Hosting, SIP-Carrier, IPv6...
Origins
nationality: Italian
mother tongue: German
kind of.
SOUTH TYROLEAN!!!
Me and Puppet
first Puppet steps with 0.24
talks, articles, blog posts
trainer, consultant
over-certified
Me @ PuppetConf 2014
Had a great time, the conference was awesome!
PuppetConf 2015 will be in Portland - see you there :)
NETWAYS
Netways and Puppet
German Puppet Labs Training Partner
Trainings
Consulting
Workshops
Puppet Trainings
http://www.netways.de/training
What this talk is all about
certificates
puppet certificates
REST API
distributed environments
security issues and their consequences
certificate lifecyle
WHY SHOULD I CARE?
Running Puppet Enterprise?
CERTIFICATES
Public Key Infrastructure - PKI
everybody has it's own private key
signs or encrypts a message
verification/decryption uses public key
algorithms: RSA, DSA...
PKI - Wikipedia
X.509
describes how our Puppet PKI works
https:// - you use it every day
ITU-T standard
defines a strict hierarchy
a tree instead of a "web of trust"
X509v3: allows extensions
Certificate structure
(distinguished) name
serial number
algorythm
issuer
validity: FROM - TO
...
The distinguished name: DN
just a string
often a DNS name
could also be "CA: puppet master"
something you should care about!
The revocation list
allows to invalidate certificates
does so based on serial numbers
important if you "loose" certificates
Filename extensions
.csr: certificate signing request, Base64
-----BEGIN CERTIFICATE REQUEST-----...-----END CERTIFICATE REQUEST-----
.pem: a certificate, Base64
-----BEGIN CERTIFICATE-----
Puppet uses .pem also for private keys:
-----BEGIN RSA PRIVATE KEY-----
PUPPET CERTIFICATES
Puppet certificates: archeology
Want to see a fresh new Puppet CA? Try it out!
mkdir /tmp/ssltestpuppet master --no-daemonize --verbose \--ssldir /tmp/ssltest \--certname test.example.com
Puppet certificates: archeology
A fresh new Puppet CA!
Puppet certificates: archeology
ls -l /tmp/ssltest
Same thing for the agent
puppet agent --test \--ssldir /tmp/sslagent \--certname test.example.com
We all know the basics
puppet cert list
puppet cert list --all
puppet cert sign test.example.com
puppet cert revoke test.example.com
puppet cert clean test.example.com
find ./ -name 'test.example.com*' --delete
SSL directories
puppet master --configprint ssldirpuppet agent --configprint ssldir
manual configuration makes sense
think about user permissions
~/.puppet, /var/lib/puppet
master and agent on the same host
passenger VS debug (--no-daemonize)
Let's dump a certificate
openssl x509 -in testexample.com.pem -noout -text
puppet cert print test.example.com
Custom data in your certificates
https://docs.puppetlabs.com/puppet/latest/reference /ssl_attributes_extensions.html
/etc/puppet/csr_attributes.yaml
custom attributes in your CSR
MCollective
Study security guidelines!
Study security guidelines!
Study security guidelines!
STUDY SECURITY GUIDELINES!
puppetlabs.com/mcollective/security-overview
Get inspired by existing modules
make sure you understood them
or write your own ones
re-use Puppet certificates
read about trust
and STUDY THE SECURITY GUIDELINES!
THE REST API
It's a web application!
<VirtualHost *:8140>
SSLEngine onSSLProtocol ALL -SSLv2 -SSLv3SSLCipherSuite EDH+CAMELLIA:EDH+aRSA:EECDH+aRSA+...SSLHonorCipherOrder on
SSLCertificateFile $ssldir/certs/$fqdn.pemSSLCertificateKeyFile $ssldir/private_keys/$fqdn.pemSSLCertificateChainFile $ssldir/ca/ca_crt.pemSSLCACertificateFile $ssldir/ca/ca_crt.pemSSLCARevocationFile $ssldir/ca/ca_crl.pemSSLVerifyClient optionalSSLVerifyDepth 1SSLOptions +StdEnvVars +ExportCertData
The Rest API
# http://docs.puppetlabs.com/guides/rest_api.html
https://master:8140/{environment}/{resource}/{key}
available on puppet master
and on VERY ancient agents (listen=true)
Puppet REST API URI examples
GET /{environment}/catalog/{node certificate name}
GET /{environment}/file_bucket_file/md5/{checksum}
GET /{environment}/facts/{node certname}
Permissions
http://docs.puppetlabs.com/guides/rest_auth_conf.html
# auth.conf
# allow all nodes to store their own reportspath ~ ^/report/([^/]+)$method saveallow $1
SSL-enabled curl example
Use your certificates and discover the API:
curl \--cert /var/lib/puppet/ssl/certs/host.pem \--key /var/lib/puppet/ssl/private_keys/host.pem \--cacert /var/lib/puppet/ssl/ca/ca_crt.pem \-k -H "Accept: yaml" \https://master:8140/production/facts/somehostname
DISTRIBUTED SETUP
Puppet Advanced* Training
Configuration for such a setup
One CA is more than enough:
[master]ca = false
[agent]ca_server = ca.example.com
Optionally, still experimental: DNS SRV records
Chain of trust
Since 3.2.1 you can use intermediate CAs to delegate trust
# http://docs.puppetlabs.com/puppet/3/reference\# /config_ssl_external_ca.html[agent]ssl_client_ca_auth = $certdir/issuer.pem
Tell Apache about your chain:
SSLCertificateChainFile "/path/to/ca_bundle.pem"
It could look like this
+------------------------+ | Root self-signed CA | +------+----------+------+ | | +----------+ +------------+ | | v v+-----------------+ +----------------+| Master CA | | Agent CA |+--------+--------+ +--------+-------+ | | v v+-----------------+ +----------------+| Master SSL Cert | | Agent SSL Cert |+-----------------+ +----------------+
SSL Professional?
integrate it in your existing hierarchy
use your own toolchain
ship signed certificates (carefully)
SECURITY
Puppet and security issues
Read Security Disclosures!
https://puppetlabs.com/security/
Thank Heartbleed!
docs.puppetlabs.com/trouble_remediate _heartbleed_overview.html
docs.puppetlabs.com/latest/reference /ssl_regenerate_certificates.html
A specific security problem
Very interesting and worth to read: CVE-2011-3872
"In versions prior to 2.6.12 and 2.7.6, the Puppet CA willimproperly insert any certdnsnames values into agentcertificates as well as master certificates. This bug wasintroduced in Puppet 0.24.0."
puppet master --configprint certdnsnamespuppet, puppet.example.com
Study it!
http://links.puppetlabs.com/cve20113872_remediation
Have a look at the remediation toolkit
And to be on the safe side, check your agent certs:
openssl x509 -in test.example.com.pem -noout -text | \grep 'Subject Alt' -A 1
X509v3 Subject Alternative Name: DNS:test.example.com, DNS:puppet, DNS:puppet.example.com
WARNING
"upgrading" doesn't fix a mess like this
old certificates would remain valid
you have to switch to a new CA...
...and this leads us to the next topic
CA LIFECYCLE MANAGEMENT
Bad news
Puppet should allow for automatic resigning of SSL certs
http://projects.puppetlabs.com/issues/7272
There is no such thing in Puppet
"...will be available with Puppet Sites"
YOU ARE ON YOUR OWN
One way of replacing a CA
stop all agents
throw away their certificates
create a new CA with a new name
start your agents
sign their new CSRs
CA...
master: rm -rf $(puppet master --configprint ssldir)agents: rm -rf $(puppet agent --configprint ssldir)
# default ca_name: "Puppet CA: <master certname>"CERTNAME=$(puppet master --configprint certname)TS=$(date +%Y-%m-%d)puppet cert --generate \--ca_name "Puppet CA: $CERTNAME <$TS>" $CERTNAME \--dns_alt_names puppet,puppet.example.com
puppet cert --allow-dns-alt-names sign $CERTNAME
You could also
get inspired by the remediation kits
write your own SSH loop
fix it with MCollective (carefully!)
open new feature requests
Don't like trouble?
Before generating your CA:
[master]ca_ttl = 20y
Leave your company in time
NB: expiration > 2038-01-19 == bad idea
BTW: WE ARE HIRING ;-)
Thank you for your attention!
Questions?
class puppetcamp {
package { 'questions': ensure => answered }
}