encryption at scale on aws - international association for ... · encryption at scale on aws matt...
TRANSCRIPT
Agenda
• Describe the AWS Key Management Service
• Client Side Encryption• AWS Encryption SDK
• Server Side Encryption• S3 Object Encryption
Amazon Simple Storage Service (S3)
• Every object (file) is stored in a bucket (a container of objects).• Upload and download objects
(PutObject/GetObject)• 10s of trillions of objects today.• 60 terabytes/seconds in some regions.• Can upload parts in parallel to increase
throughput• Upload 5TB in 1m 12s, and read in 42s.
• Designed for 11 9s of durability.• One of over 100+ services
Amazon S3
Goals for encryption at scale• Customers want• Root keys stored in FIPS certified hardware, • Control over these keys through fine grain usage policies,• Integrate into services to make encryption easy, • Provide detail audit records for all use of their keys, and• Us to manage basic key usage issues – algorithm selection, IV
generation, key rotation, etc., • Not feasible to run all data encryption through a single service.• Provide data keys to encrypt data
• Keys durability must exceed that of the data it protects.
AWS KMS
CreateKey(policy)EKT
ct = Encrypt(keyID, AAD, pt)CMK = D(DK, EKT)
OID ß $OK = KDF(CMK, OID)ct = E(OK, AAD, pt)
DK ß $CMK ß $
EKT = E(DK, CMK)
EKT, AAD, pt
ct
policy_____________________EKT
HSM
AWS Key Management Service
Last year Shay Gueron presented the KMS derived-key mode for GCM to encrypt at scale
up to 240 customer master keys, and
each master key can be used to perform 250 encryptions,
While ensuring the probability of a (key, iv) collision and an adversary‘s PRP-PRF distinguishability advantage remains below 2-32.
AWS Key Management Service
AWS Key Management Service
Fleet of HSMsHSM HSM HSM HSM HSM HSM
KMS KMS KMS KMS KMS KMSAvailability zone 1 Availability zone 2 Availability zone 3
EKT EKT EKT EKT EKT EKT
Region
Alexa for Business Amazon Glacier Amazon WorkMail AWS Snowball
Amazon Athena Amazon Kinesis Data Streams Amazon WorkSpaces AWS Snowmobile
Amazon Aurora Amazon Kinesis Firehose AWS Certificate Manager AWS Snowball Edge
Amazon CloudWatch Logs Amazon Kinesis Video Streams AWS Cloud9 AWS Storage Gateway
Amazon Comprehend Amazon Lex AWS CloudTrail AWS X-Ray
Amazon Managed Streaming for Kafka (MSK) Amazon Lightsail AWS CodeBuild AWS Lambda
AWS Database Migration Service Amazon Elastic Transcoder AWS CodeCommit Amazon S3
Amazon DynamoDB Accelerator (DAX) Amazon Elasticsearch Service AWS CodeDeploy Amazon EMR
Amazon Simple Email Service (SES) Amazon Neptune AWS CodePipeline Amazon EBS
Amazon Simple Queue Service (SQS) Amazon Redshift Amazon DynamoDB Amazon EFS
Amazon Relational Database Service (RDS) Amazon SageMaker AWS Secrets Manager AWS Systems Manager
Amazon FSx for Windows File Server Amazon Translate Amazon Connect AWS Glue
AWS KMS Integrated Services
Envelope encryption with AWS KMS
3) For plaintext pt, encrypt, ct = E(dk, pt).
4) Send/store the pair (edk, ct).
1) Alice calls KMS to generate a data key.
2) Receive an encrypted data key edk, and a data key dk. edk, dk = GenerateDataKey(keyID)
AWS KMS
Authorization and key policy{
"Sid": ”Alice","Effect": "Allow","Principal": { "AWS": "arn:aws:iam::012345678901:user/alice" },"Action": [ "kms:GenerateDataKey", "kms:Encrypt"],"Resource": "*",
},{
"Sid": ”Recipients","Effect": "Allow","Principal": { "AWS": [
"arn:aws:iam:: 012345678901:user/bob", "arn:aws:iam:: 012345678901:user/charlie"] },
"Action": "kms:Decrypt","Resource": "*",
}
Alice can encrypt
Bob and Charlie can decrypt
4) Encrypt alternate message ct’ = E(dk, pt’)
5) Send (edk, ct’) of to Charlie
1) Bob starts with valid ciphertext (edk, ct)2) Call KMS to decrypt an encrypted data key
3) Receive the data key dk.dk = Decrypt(edk)
Envelope encryption with AWS KMS
AWS KMS
• Make it easy for AWS KMS customer to encrypt securely
• Use envelope encryption to protect large plaintext values
• Enable the policy on the customer master key to extend to the envelope encrypted data.
• Encrypt/decrypt many plaintexts under a single call to AWS KMS.
AWS Encryption SDK
plaintext1
sig1
edkQ
h_ta
g 1m
sg_i
d 1
dk, edk = GenerateDataKey(keyId, AAD), where Q is contained in AADdß $, Q public key
k = KDF(dk, msg_id1)
sig1 = sign(d, *)
AWS Encryption SDK
(ivi++, cparti, tagi) = E(k, parti)
AWS KMS
msg_id1 ß $
h_ta
g 2m
sg_i
d 2
k2 = KDF(dk, msg_id2)msg_id2 ß $
sig2
sig2 = sign(d, *)
(ivi++, cparti, tagi) = E(k2, parti)
part1 part2 part3 partF
cpart1iv1
tag 1 cpart2iv2
tag 2 cpart3iv3
tag 3 cpartFivF
tag F
plaintext2part1 part2 part3 partF
cach
e
• Consider a ki,j collision for a set of N cached data keys dkj
• Probability of ki,j collisions for (Q*N) many random 256-bit values is < (Q*N)2/2257 = 2-129, negligible, when N and Q < 232.
• Probability of two equal dkj’s in N is < N2/2257 < 2-192 when N < 232, negligible.• For any fixed dkj, the probability of ki,j collision < 2-64, for Q < 232
• For N-many such dk’s, we have a probability < 1 – (1 – 2-64)N ≈N*2-64.
• Prob ki,j collision for N cached data keys dkj doing at most Q encryptions < N*Q2/2257 < 2-33, when N, Q < 232.
AWS Encryption SDK
Single-user multi-cached-key for multi-key encryption scenariolarge keys (256 bits) + deterministic IV’s
make the probabilities for “bad event” sufficiently small
Multi-user scenario matters less, as SDK contexts are separate application spaces
• Supports Advanced Encryption Standard with 256-bit keys in Galois Counter Mode (AES-256-GCM) for authenticated encryption.• Supports Elliptic Curve Digital Signature Algorithm (ECDSA) with SHA384 on the
curve NIST P-384 (secp384r1).• Open source solution that supports• Supports messages of size ~263 bytes.• Supports framing data for streaming.• Supports caching AWS KMS data key across multiple messages, based on a
security context, time, number of derived keys, or total bytes encrypted.• Extensible to additional key management infrastructures.
AWS Encryption SDK
AWS Kinesis
producer consumer
DynamoDB
AWS KMS
VPC
Amazon Simple Storage Service (S3)
Amazon S3
TLS
Client Side Encryption Server Side Encryption
meta-
data
edkAmazon S3 SSE-KMS
Adds 12-byte IV and 16-byte MAC per
chunk.
Ob
ject
ct
iv
tag
chunk
chunk
chunk
chunk
chunk
edk = KMS.GenerateDataKey(keyId)
Store edk in object meta-data
(iv, ct, tag) = AES_256_GCM(K, chunk)O
bje
ct
ct
iv
tag
ct
iv
tag
ct
iv
tag
ct
iv
tag
E( )
E( )
E( )
E( )
E( )
Amazon S3
PutObject using S3 SSE
meta-dataedkAmazon S3 SSE-KMS
GetObject with range [x, y]
Decrypt and verify the necessary chunks
Assemble and return the range
ct
iv
tag
ct
iv
tag
ct
iv
tag
ct
iv
tag
ct
iv
tag
Amazon S3
ct
iv
tag
ct
iv
tag
chunk
chunk
range
D( )
D( )
dk = KMS.Decrypt(edk)
Fetch necessary chunks
range
• 5 terabyte limit for S3 objects ~243 bytes
• Larger chunk – less storage overhead, increase size limit, increase ‘Get’ latency• Smaller chunk – more storage overhead, lower size limit, decrease ‘Get’ latency
Amazon S3 SSE
Chunk Size Number of Chunks Probability of IV collision
Max size while collision probability < 2-32
2048 = 211 ~232 - 230 < 2-32 8 terabytes32768 = 215 ~ 228 < 2-41 140 terabytes
262144 = 218 ~227 < 2-47 1.125 petabytes
Summary
• AWS is making encryption of customer data easy with secure default configurations.
• Tension between one-size fits all and every application requires a custom solution.
• Want to minimize design patterns and tools we need to maintain and reason about.
• Managing state across distributed systems is performance inhibiting for encryption.
• IV-collisions probabilities are exasperated by data volume across distributive systems.
• An efficient and standardized wide block encryption primitive would ease these tension.
Thank you!
https://aws.amazon.com/kms/https://d0.awsstatic.com/whitepapers/KMS-Cryptographic-Details.pdf
https://github.com/aws/aws-encryption-sdk-javahttps://github.com/aws/aws-encryption-sdk-pythonhttps://github.com/aws/aws-encryption-sdk-cli
https://docs.aws.amazon.com/AmazonS3/latest/dev/serv-side-encryption.htmlhttps://docs.aws.amazon.com/AmazonS3/latest/dev/UsingClientSideEncryption.html