safebricks: shielding network functions in the cloud · nf tls ipsec tls safebricks also supports...
TRANSCRIPT
SafeBricks: Shielding Network Functions in the Cloud
Rishabh Poddar, Chang Lan, Raluca Ada Popa, Sylvia Ratnasamy
UC Berkeley
Network Functions (NFs) in the cloud
�2
DestinationEnterprise
Clients
Network Functions (NFs) in the cloud
�3
DestinationEnterprise
Clients
Network Functions (NFs) in the cloud
�4
NF providersNF providersNF providersNF providers
Enterprise
Clients
Destination
Problem: Security
�5
NF providersNF providersNF providersNF providers
Enterprise
Clients
Destination
�6
NF providersNF providersNF providersNF providers
Enterprise
Clients
Destination
Need to protect traffic from the cloud provider1
Hackers /curious employees
Problem: Security
Problem: Security
�7
NF providersNF providersNF providersNF providers
Enterprise
Clients
Destination
Need to protect traffic from the NF providers
Exfiltration
2
Problem: Security
�8
NF providersNF providersNF providersNF providers
Enterprise
Clients
Destination
Need to protect NF code and rulesets from client enterprise and cloud3
Cryptographic solutions do not suffice
�9
NF providersNF providersNF providersNF providers
Enterprise
Clients
Destination
Cryptographic solutions do not suffice
�10
NF providersNF providersNF providersNF providers
Enterprise
Clients
Destination
Standard encryption: e.g. end-to-end TLS1
Cryptographic solutions do not suffice
�11
NF providersNF providersNF providersNF providers
Enterprise
Clients
Destination
Standard encryption: e.g. end-to-end TLS • Functionality: Doesn’t allow any computation on encrypted payload
?
1
Cryptographic solutions do not suffice
�12
NF providersNF providersNF providersNF providers
Enterprise
Clients
Destination
Standard encryption: e.g. end-to-end TLS • Functionality: Doesn’t allow any computation on encrypted payload • Security: Unencrypted fields (e.g. IP headers) still leak information
?
1
Cryptographic solutions do not suffice
�13
Specialized encryption: e.g. BlindBox, Embark [Sherry et al. (SIGCOMM’15)]
[Lan et al. (NSDI’16)]
2
Cryptographic solutions do not suffice
�14
Specialized encryption: e.g. BlindBox, Embark • Too limited in functionality!
2
• Header-based comparisons
• Keyword matching
• Regular expressions
• Cross-flow analysis
• Statistical computations
How to achieve full functionality and
our security goals simultaneously?
�15
SafeBricks
�16
2
3
Protects traffic from the NF providers
Protects NF source code and rulesets from client enterprise and cloud
Protects traffic from the cloud provider1
SafeBricks
�17
2
3
Protects traffic from the NF providers
Protects traffic from the cloud provider1
Hardware enclaves + language-based isolation
Protects NF source code and rulesets from client enterprise and cloud
• Secure region of memory (enclaves) protected by hardware
Background: Hardware enclaves (e.g. Intel SGX)
�18
Operating System (untrusted)
Application (untrusted)
Enclave (trusted)
• Secure region of memory (enclaves) protected by hardware
�19
Operating System (untrusted)
Application (untrusted)
Enclave (trusted)
Secretdata
Trustedcode
Background: Hardware enclaves (e.g. Intel SGX)
• Secure region of memory (enclaves) protected by hardware
�20
Operating System (untrusted)
Application (untrusted)
Enclave (trusted)
Secretdata
Trustedcode
Client
• Remote attestation by clients
Background: Hardware enclaves (e.g. Intel SGX)
• Secure region of memory (enclaves) protected by hardware
�21
Operating System (untrusted)
Application (untrusted)
Enclave (trusted)
Secretdata
Trustedcode
Client
• Remote attestation by clients • Remotely verify enclave contents
Background: Hardware enclaves (e.g. Intel SGX)
• Secure region of memory (enclaves) protected by hardware
�22
Operating System (untrusted)
Application (untrusted)
Enclave (trusted)
Secretdata
Trustedcode
Client
• Remote attestation by clients • Remotely verify enclave contents • Establish a secure channel with enclave
Background: Hardware enclaves (e.g. Intel SGX)
Background: NetBricks
�23
[Panda et al. (OSDI’16)]
I/O interface
Poll for I/O
Programming abstractions
State abstractions
Scheduler
DPDK
NICs
NetBricks• Framework for developing arbitrary NFs
�24
[Panda et al. (OSDI’16)]
• Framework for developing arbitrary NFs • MapReduce like programming
abstractions (operators) for packet processing
I/O interface
Poll for I/O
Programming abstractions
State abstractions
Scheduler
DPDK
NICs
NetBricks
Background: NetBricks
�25
[Panda et al. (OSDI’16)]
• Framework for developing arbitrary NFs • MapReduce like programming
abstractions (operators) for packet processing
• NFs represented as a directed graph with operators as nodes
I/O interface
Poll for I/O
Programming abstractions
State abstractions
Scheduler
DPDK
NICs
NetBricks
Background: NetBricks
�26
• Written in Rust
• Fast, safe, zero-copy semantics
• Isolates NFs deployed in a chain while running them in the same address space
NICs
NetBricks
NF 1 NF 1
Background: NetBricks
SafeBricks
�27
Protects traffic from the NF providers
Protects traffic from the cloud provider
2
3
1
Protects NF source code and rulesets from client enterprise and cloud
SafeBricks
�28
Protects traffic from the NF providers
Protects traffic from the cloud provider
2
3
1
Protects NF source code and rulesets from client enterprise and cloud
Outsourcing NFs using hardware enclaves
�29 Enterprise
Clients
Gateway
Enclave
OS (untrusted)
Cloud (untrusted)
Destination
Outsourcing NFs using hardware enclaves
�30 Enterprise
Clients
Gateway
Enclave
OS (untrusted)
Cloud (untrusted)
Destination
NF
Outsourcing NFs using hardware enclaves
�31 Enterprise
Clients
Gateway
Enclave
OS (untrusted)
Cloud (untrusted)
Destination
NF
Outsourcing NFs using hardware enclaves
�32 Enterprise
Clients
Gateway
Enclave
OS (untrusted)
Cloud (untrusted)
Destination
NF
IPSec
IPSec
Outsourcing NFs using hardware enclaves
�33 Enterprise
Clients
Gateway
Enclave
OS (untrusted)
Cloud (untrusted)
Destination
NF
IPSec
IPSec
TLS
Interception proxy
Outsourcing NFs using hardware enclaves
�34 Enterprise
Clients
Gateway
Enclave
OS (untrusted)
Cloud (untrusted)
Destination
NF
TLS
IPSec
TLS
IPSec
Outsourcing NFs using hardware enclaves
�35 Enterprise
Clients
Gateway
Enclave
OS (untrusted)
Cloud (untrusted)
Destination
NF
TLSTLSTLSTLS
IPSecIPSec
TLS
IPSec
Outsourcing NFs using hardware enclaves
�36 Enterprise
Clients
Gateway
Enclave
OS (untrusted)
Cloud (untrusted)
Destination
NF
TLSTLSTLSTLS
IPSecIPSec
TLS
IPSec
Packet headers also encrypted
Outsourcing NFs using hardware enclaves
�37 Enterprise
Clients
Gateway
Enclave
OS (untrusted)
Cloud (untrusted)
Destination
NF
TLSTLSTLSTLS
IPSec
TLS
IPSec
TLS
IPSec
Outsourcing NFs using hardware enclaves
�38 Enterprise
Clients
Gateway
Enclave
OS (untrusted)
Cloud (untrusted)
Destination
NF
TLSTLSTLSTLS
IPSec
TLS
SafeBricks also supports “direct” delivery of traffic
IPSec
Outsourcing NFs using hardware enclaves
�39 Enterprise
Clients
Gateway
Enclave
OS (untrusted)
Cloud (untrusted)
Destination
NF
TLSTLSTLSTLS
IPSec
TLS
Can use general purpose frameworks,
e.g. Haven, Scone
IPSec
TLS
IPSec
Challenges
�40
Small trusted computing base (TCB) — enclave should contain minimal amount of code
1
Challenges
�41
High performance — Transitioning into / out of enclaves is expensive!
Small trusted computing base (TCB) — enclave should contain minimal amount of code
2
1
Challenges
�42
High performance — Transitioning into / out of enclaves is expensive!
Illegal enclave instructions — SGX does not support system calls or instructions that may lead to a VMEXIT
Small trusted computing base (TCB) — enclave should contain minimal amount of code
2
3
1
Challenges
�43
2
3
High performance — Transitioning into / out of enclaves is expensive!
Illegal enclave instructions — SGX does not support system calls or instructions that lead to a VMEXIT
Small trusted computing base (TCB) — enclave should contain minimal amount of code
1
�44
I/O interface
Poll for I/O
Programming abstractions
State abstractions
Scheduler
DPDK
NICs
NetBricks1
�45
I/O interface
Poll for I/O
Programming abstractions
State abstractions
Scheduler
DPDK
NICs
Enclave1
• Maximal TCB: NetBricks stack entirely within enclave
�46
I/O interface
Poll for I/O
Programming abstractions
State abstractions
Scheduler
DPDK
NICs
Enclave1
• Minimal TCB: Only security-critical components within enclave
• One enclave transition per node per packet batch
• Intermediate TCB • One enclave transition per
packet batch
�47
I/O interface
Poll for I/O
Programming abstractions
State abstractions
Scheduler
DPDK
NICs
Enclave1
�48
I/O interface
Poll for I/O
Programming abstractions
State abstractions
Scheduler
DPDK
NICs
Glue code (trusted)
Glue code (untrusted)
SafeBricksenclave(trusted)
SafeBrickshost
(untrusted)
1
• Partitioned NetBricks framework; glue code connects trusted and untrusted code
�49
I/O interface
Poll for I/O
Programming abstractions
State abstractions
Scheduler
DPDK
NICs
Glue code (trusted)
Glue code (untrusted)
SafeBricksenclave(trusted)
SafeBrickshost
(untrusted)
1
• Partitioned NetBricks framework; glue code connects trusted and untrusted code
Two new operators for packet transfer to/from enclave:
and toHosttoEnclave
Challenges
�50
2
3
High performance — Transitioning into / out of enclaves is expensive!
Small trusted computing base (TCB) — enclave should contain minimal amount of code
1
Illegal enclave instructions — SGX does not support system calls or instructions that lead to a VMEXIT
SafeBricks host
SafeBricks enclave
�51
NICs
2
toHosttoEnclave
NF
• One enclave transition per packet batch
SafeBricks host
SafeBricks enclave
�52 NICs
2
toHosttoEnclave
NF
• Shared queues in non-enclave heap
• Separate enclave and host threads
• Access queues without exiting enclave — zero enclave transitions
send recv
Enclave I/O
Host I/O
Challenges
�53
2
3
High performance — Transitioning into / out of enclaves is expensive!
Small trusted computing base (TCB) — enclave should contain minimal amount of code
1
Illegal enclave instructions — SGX does not support system calls or instructions that lead to a VMEXIT
�54
3
Observation: NFs in general do not require support for system calls / instructions that lead to VMEXITs
�55
3
Observation: NFs in general do not require support for system calls / instructions that lead to VMEXITs, except:
• Logging
• Timestamps (using rdtsc)
�56
Observation: NFs in general do not require support for system calls / instructions that lead to VMEXITs, except:
• Logging
• Timestamps (using rdtsc)
3
SafeBricks designs custom solutions for these operations without enclave transitions
�57
2
3
1
SafeBricks
Protects traffic from the NF providers
Protects traffic from the cloud provider
Protects NF source code and rulesets from client enterprise and cloud
Problem: Malicious NFs within enclaves
�58
Malicious NFs inside the enclave can exfiltrate or tamper with packets!!
Problem: Malicious NFs within enclaves
�59
Observation: NFs typically need access only to specific packet fields
• E.g. Firewall needs read-only access to TCP/IP headers
• E.g. NAT needs both read-write access to headers but not to packet payload
Malicious NFs inside the enclave can exfiltrate or tamper with packets!!
Problem: Malicious NFs within enclaves
�60
Observation: NFs typically need access only to specific packet fields
• E.g. Firewall needs read-only access to TCP/IP headers
• E.g. NAT needs both read-write access to headers but not to packet payload
Malicious NFs inside the enclave can exfiltrate or tamper with packets!!
IP addresses; TCP ports; HTTP
payload
Problem: Malicious NFs within enclaves
�61
Observation: NFs typically need access only to specific packet fields
• E.g. Firewall needs read-only access to TCP/IP headers
• E.g. NAT needs both read-write access to headers but not to packet payload
Malicious NFs inside the enclave can exfiltrate or tamper with packets!!
SafeBricks enforces least privilege across NFs within the enclave
toEnclave toHost
Host
�62
Run NFs within the same enclave
Least privilege enforcement
Firewall NAT
�63
Run NFs within the same enclaveFirewall
toEnclave
NAT
toHost
Host
Least privilege enforcement
�64
Run NFs within the same enclave
• Stitch NFs together interspersed with an operator ( wList ) that embeds a vector of permissions in packets — two bits per packet field
Firewall
toEnclave
NAT
toHost
Host
wList wList
wList
Least privilege enforcement
�65
Enforce permissions by mediating access to
packets using Rust’s ownership model
SafeBricks Controller
Least privilege enforcement
�66
Enforce permissions by mediating access to
packets using Rust’s ownership model
• Controller module holds ownership of packet buffers
SafeBricks Controller
Packet buffer
Least privilege enforcement
�67
Enforce permissions by mediating access to
packets using Rust’s ownership model
• Controller module holds ownership of packet buffers
• NFs borrow references to packet fields from the Controller, which checks permissions vector in packet
SafeBricks Controller
NAT Firewall
Packet buffer
Least privilege enforcement
�68
Enforce permissions by mediating access to
packets using Rust’s ownership model
• Controller module holds ownership of packet buffers
• NFs borrow references to packet fields from the Controller, which checks permissions vector in packet
SafeBricks Controller
Packet buffer
NAT Firewall
Returns an immutable reference for read-only access, and a mutable reference for
write access
Least privilege enforcement
Assumption: Trusted compilation of NFs
�69
Least privilege guarantees only hold if NFs are built using a compiler that prohibits unsafe operations!
Assumption: Trusted compilation of NFs
�70
Least privilege guarantees only hold if NFs are built using a compiler that prohibits unsafe operations!
E.g. Check array bounds, no pointer arithmetic, no
unsafe type casts
Assumption: Trusted compilation of NFs
�71
Least privilege guarantees only hold if NFs are built using a compiler that prohibits unsafe operations!
• Possible solution: Client obtains NF source codes from providers and assembles them locally
NF providersNF providersNF providersNF providers
Client Enterprise
NF code + rulesets
Assumption: Trusted compilation of NFs
�72
Least privilege guarantees only hold if NFs are built using a compiler that prohibits unsafe operations!
• Possible solution: Client obtains NF source codes from providers and assembles them locally
• Problem: This violates the confidentiality of NF source code!
NF providersNF providersNF providersNF providers
Enterprise
NF code + rulesets
�73
2
3
1
SafeBricks
Protects traffic from the NF providers
Protects traffic from the cloud provider
Protects NF source code and rulesets from client enterprise and cloud
Assembling NFs
�74
• Key idea: Build NFs within a special “meta”-enclave in the cloud using an agreed upon compiler
Assembling NFs
�75
• Key idea: Build NFs within a special “meta”-enclave in the cloud using an agreed upon compiler
• Both client and NF providers can verify the agreed upon compiler using remote attestation
Assembling NFs
�76
Enterprise
NF providersNF providersNF providersNF providers
Loader Compiler
Assembly enclave
Assembling NFs
�77
Enterprise
NF providersNF providersNF providersNF providers
Loader Compiler
Assembly enclave
Remote attestation
Assembling NFs
�78
Enterprise
NF providersNF providersNF providersNF providers
Loader Compiler
Assembly enclave
Remote attestation
Assembling NFs
�79
Enterprise
NF providersNF providersNF providersNF providers
Loader Compiler
Assembly enclave
Remote attestation
Assembling NFs
�80
Enterprise
NF providersNF providersNF providersNF providers
Loader Compiler
Assembly enclave
NF code + rulesets
Assembling NFs
�81
Enterprise
NF providersNF providersNF providersNF providers
Loader Compiler
Assembly enclave
NF code + rulesets
Config
Assembling NFs
�82
Enterprise
NF providersNF providersNF providersNF providers
Loader Compiler
Assembly enclave
NF code + rulesets
Config
Placement of NFs, least privilege policies per NF
Assembling NFs
�83
Enterprise
NF providersNF providersNF providersNF providers
Loader Compiler
Assembly enclave
NF code + rulesets
Config
Assembling NFs
�84
Enterprise
NF providersNF providersNF providersNF providers
Loader Compiler
Assembly enclave
Deployment enclave
SafeBricks
�85
2
3
Protects traffic from the NF providers
Protects NF source code and rulesets from client enterprise and cloud
Protects traffic from the cloud provider1
Performance
�86
Throughput decline across NFs
�87~0–15% overhead across applications for different packet sizes
Thro
ughp
ut d
eclin
e
0%
10%
20%
30%
40%
Packet size (bytes)64 256 512 1024
Firewall NATLoad Balancer DPI
DPI performance with increasing no. of rules
�88 Overhead spikes when NF working set exceeds enclave memory
Thro
ughp
ut d
eclin
e
0%
25%
50%
75%
100%
Number of rules
0 500010000
1500020000
25000
94MB64MB8MB
DPI performance with increasing no. of rules
�89 Overhead spikes when NF working set exceeds enclave memory
Thro
ughp
ut d
eclin
e
0%
25%
50%
75%
100%
Number of rules
0 500010000
1500020000
25000
94MB64MB8MB
Not a fundamental limitation
SafeBricks uses a combination of hardware enclaves
and language-based isolation to:
• Protect client traffic from the cloud provider
• Enforce least privilege across NFs
• Protect the confidentiality of NF code and rulesets
�90
Summary
Modest overhead across a range of applications