![Page 1: Secure Tera-scale Data Crunching with a Small TCBbvavala/pub/dsn2017slides.pdf · -If Supervisor is trusted, invalid data => no resume (e.g.: TrustVisor) -If Supervisor is untrusted](https://reader033.vdocuments.us/reader033/viewer/2022053007/5f0b1fb27e708231d42ef727/html5/thumbnails/1.jpg)
Secure Tera-scale Data Crunching
with a Small TCBBruno Vavala Nuno Neves Peter Steenkiste
UL / CMU UL CMU
47th IEEE/IFIP International Conference on Dependable Systems and Networks (DSN’17)
![Page 2: Secure Tera-scale Data Crunching with a Small TCBbvavala/pub/dsn2017slides.pdf · -If Supervisor is trusted, invalid data => no resume (e.g.: TrustVisor) -If Supervisor is untrusted](https://reader033.vdocuments.us/reader033/viewer/2022053007/5f0b1fb27e708231d42ef727/html5/thumbnails/2.jpg)
delivering security guarantees for generic and large-scale data processing on untrusted hosts
delivering security guarantees for large-scale data processing
on untrusted hosts with a small TCB
Goal
2
![Page 3: Secure Tera-scale Data Crunching with a Small TCBbvavala/pub/dsn2017slides.pdf · -If Supervisor is trusted, invalid data => no resume (e.g.: TrustVisor) -If Supervisor is untrusted](https://reader033.vdocuments.us/reader033/viewer/2022053007/5f0b1fb27e708231d42ef727/html5/thumbnails/3.jpg)
3
delivering security guarantees for large-scale data processing
on untrusted hosts with a small TCBtrusted HW based data
integrity
![Page 4: Secure Tera-scale Data Crunching with a Small TCBbvavala/pub/dsn2017slides.pdf · -If Supervisor is trusted, invalid data => no resume (e.g.: TrustVisor) -If Supervisor is untrusted](https://reader033.vdocuments.us/reader033/viewer/2022053007/5f0b1fb27e708231d42ef727/html5/thumbnails/4.jpg)
4
delivering security guarantees for large-scale data processing
on untrusted hosts with a small TCB
1 TB
![Page 5: Secure Tera-scale Data Crunching with a Small TCBbvavala/pub/dsn2017slides.pdf · -If Supervisor is trusted, invalid data => no resume (e.g.: TrustVisor) -If Supervisor is untrusted](https://reader033.vdocuments.us/reader033/viewer/2022053007/5f0b1fb27e708231d42ef727/html5/thumbnails/5.jpg)
5
delivering security guarantees for large-scale data processing
on untrusted hosts with a small TCB
small code
small interface
No HW devices
![Page 6: Secure Tera-scale Data Crunching with a Small TCBbvavala/pub/dsn2017slides.pdf · -If Supervisor is trusted, invalid data => no resume (e.g.: TrustVisor) -If Supervisor is untrusted](https://reader033.vdocuments.us/reader033/viewer/2022053007/5f0b1fb27e708231d42ef727/html5/thumbnails/6.jpg)
Some use cases
6
public cloud service provider
![Page 7: Secure Tera-scale Data Crunching with a Small TCBbvavala/pub/dsn2017slides.pdf · -If Supervisor is trusted, invalid data => no resume (e.g.: TrustVisor) -If Supervisor is untrusted](https://reader033.vdocuments.us/reader033/viewer/2022053007/5f0b1fb27e708231d42ef727/html5/thumbnails/7.jpg)
Some use cases
computational genomics
7
public cloud service provider
0.3TB per genome
![Page 8: Secure Tera-scale Data Crunching with a Small TCBbvavala/pub/dsn2017slides.pdf · -If Supervisor is trusted, invalid data => no resume (e.g.: TrustVisor) -If Supervisor is untrusted](https://reader033.vdocuments.us/reader033/viewer/2022053007/5f0b1fb27e708231d42ef727/html5/thumbnails/8.jpg)
…more generally…
![Page 9: Secure Tera-scale Data Crunching with a Small TCBbvavala/pub/dsn2017slides.pdf · -If Supervisor is trusted, invalid data => no resume (e.g.: TrustVisor) -If Supervisor is untrusted](https://reader033.vdocuments.us/reader033/viewer/2022053007/5f0b1fb27e708231d42ef727/html5/thumbnails/9.jpg)
Model trusted hardware module
SV
P
9
![Page 10: Secure Tera-scale Data Crunching with a Small TCBbvavala/pub/dsn2017slides.pdf · -If Supervisor is trusted, invalid data => no resume (e.g.: TrustVisor) -If Supervisor is untrusted](https://reader033.vdocuments.us/reader033/viewer/2022053007/5f0b1fb27e708231d42ef727/html5/thumbnails/10.jpg)
Model
1. provide state authentication data
2. outsource large state
SV
P
10
![Page 11: Secure Tera-scale Data Crunching with a Small TCBbvavala/pub/dsn2017slides.pdf · -If Supervisor is trusted, invalid data => no resume (e.g.: TrustVisor) -If Supervisor is untrusted](https://reader033.vdocuments.us/reader033/viewer/2022053007/5f0b1fb27e708231d42ef727/html5/thumbnails/11.jpg)
Model3. send request
SV
P
1. provide state authentication data
2. outsource large state
11
![Page 12: Secure Tera-scale Data Crunching with a Small TCBbvavala/pub/dsn2017slides.pdf · -If Supervisor is trusted, invalid data => no resume (e.g.: TrustVisor) -If Supervisor is untrusted](https://reader033.vdocuments.us/reader033/viewer/2022053007/5f0b1fb27e708231d42ef727/html5/thumbnails/12.jpg)
Model3. send request
4. execute command
SV
P
1. provide state authentication data
2. outsource large state
12
![Page 13: Secure Tera-scale Data Crunching with a Small TCBbvavala/pub/dsn2017slides.pdf · -If Supervisor is trusted, invalid data => no resume (e.g.: TrustVisor) -If Supervisor is untrusted](https://reader033.vdocuments.us/reader033/viewer/2022053007/5f0b1fb27e708231d42ef727/html5/thumbnails/13.jpg)
Model3. send request
5. receive authenticated reply
SV
P
1. provide state authentication data
2. outsource large state
4. execute command
13
![Page 14: Secure Tera-scale Data Crunching with a Small TCBbvavala/pub/dsn2017slides.pdf · -If Supervisor is trusted, invalid data => no resume (e.g.: TrustVisor) -If Supervisor is untrusted](https://reader033.vdocuments.us/reader033/viewer/2022053007/5f0b1fb27e708231d42ef727/html5/thumbnails/14.jpg)
Outline
• Goal • Previous Work • Our solution: key ideas and
overview • Evaluation
![Page 15: Secure Tera-scale Data Crunching with a Small TCBbvavala/pub/dsn2017slides.pdf · -If Supervisor is trusted, invalid data => no resume (e.g.: TrustVisor) -If Supervisor is untrusted](https://reader033.vdocuments.us/reader033/viewer/2022053007/5f0b1fb27e708231d42ef727/html5/thumbnails/15.jpg)
Outline
• Goal • Previous Work • Our solution: key ideas and
overview • Evalution
![Page 16: Secure Tera-scale Data Crunching with a Small TCBbvavala/pub/dsn2017slides.pdf · -If Supervisor is trusted, invalid data => no resume (e.g.: TrustVisor) -If Supervisor is untrusted](https://reader033.vdocuments.us/reader033/viewer/2022053007/5f0b1fb27e708231d42ef727/html5/thumbnails/16.jpg)
Haven (OSDI’14)
picoprocess
host OS
enclave
service
libOS
interface
— designed for Intel SGX — large TCB (due to libOS) — 10s of new interface calls + works with unmodified applications
VHD
16
![Page 17: Secure Tera-scale Data Crunching with a Small TCBbvavala/pub/dsn2017slides.pdf · -If Supervisor is trusted, invalid data => no resume (e.g.: TrustVisor) -If Supervisor is untrusted](https://reader033.vdocuments.us/reader033/viewer/2022053007/5f0b1fb27e708231d42ef727/html5/thumbnails/17.jpg)
VC3 (IEEE S&P’15)
process
host OS
enclave
map & reduce functions
job execution protocol
narrow interface
— designed for Intel SGX — specific for Hadoop + small TCB + data confidentiality + can run unmodified Hadoop applications
17
![Page 18: Secure Tera-scale Data Crunching with a Small TCBbvavala/pub/dsn2017slides.pdf · -If Supervisor is trusted, invalid data => no resume (e.g.: TrustVisor) -If Supervisor is untrusted](https://reader033.vdocuments.us/reader033/viewer/2022053007/5f0b1fb27e708231d42ef727/html5/thumbnails/18.jpg)
small TCB
Large State
Interface calls
App Specific
Trusted Computing
arch.
Haven (OSDI’14) No Yes tens No SGX
VC3 (S&P’15) Yes Yes
MapReduce workloads
R,W Yes SGX
XMHF-TrustVisor
(S&P’13,’10)Yes No none
(but Minibox has tens)
No TPM / TXT
LaStGT Yes Yes zero! No TV&SGX
A Niche in the State of the Art
18
![Page 19: Secure Tera-scale Data Crunching with a Small TCBbvavala/pub/dsn2017slides.pdf · -If Supervisor is trusted, invalid data => no resume (e.g.: TrustVisor) -If Supervisor is untrusted](https://reader033.vdocuments.us/reader033/viewer/2022053007/5f0b1fb27e708231d42ef727/html5/thumbnails/19.jpg)
Outline
• Goal • Previous Work • Our solution: key ideas and
overview • Evalution
![Page 20: Secure Tera-scale Data Crunching with a Small TCBbvavala/pub/dsn2017slides.pdf · -If Supervisor is trusted, invalid data => no resume (e.g.: TrustVisor) -If Supervisor is untrusted](https://reader033.vdocuments.us/reader033/viewer/2022053007/5f0b1fb27e708231d42ef727/html5/thumbnails/20.jpg)
untrusted env.
trusted env.
20
Scenario: two execution environments
![Page 21: Secure Tera-scale Data Crunching with a Small TCBbvavala/pub/dsn2017slides.pdf · -If Supervisor is trusted, invalid data => no resume (e.g.: TrustVisor) -If Supervisor is untrusted](https://reader033.vdocuments.us/reader033/viewer/2022053007/5f0b1fb27e708231d42ef727/html5/thumbnails/21.jpg)
untrusted env.
trusted env.
app’s execution flow
21
the service code is running
![Page 22: Secure Tera-scale Data Crunching with a Small TCBbvavala/pub/dsn2017slides.pdf · -If Supervisor is trusted, invalid data => no resume (e.g.: TrustVisor) -If Supervisor is untrusted](https://reader033.vdocuments.us/reader033/viewer/2022053007/5f0b1fb27e708231d42ef727/html5/thumbnails/22.jpg)
untrusted env.
trusted env.
access data in block bi
is bi in memory?
app’s execution flow
22
the service code accesses data in memory
![Page 23: Secure Tera-scale Data Crunching with a Small TCBbvavala/pub/dsn2017slides.pdf · -If Supervisor is trusted, invalid data => no resume (e.g.: TrustVisor) -If Supervisor is untrusted](https://reader033.vdocuments.us/reader033/viewer/2022053007/5f0b1fb27e708231d42ef727/html5/thumbnails/23.jpg)
untrusted env.
trusted env.
access data in block bi
is bi in memory?
app’s execution flow
keep goingyes
23
when data is available, there are no interruptions
![Page 24: Secure Tera-scale Data Crunching with a Small TCBbvavala/pub/dsn2017slides.pdf · -If Supervisor is trusted, invalid data => no resume (e.g.: TrustVisor) -If Supervisor is untrusted](https://reader033.vdocuments.us/reader033/viewer/2022053007/5f0b1fb27e708231d42ef727/html5/thumbnails/24.jpg)
untrusted env.
trusted env.
access data in block bi
is bi in memory?
app’s execution flow
handle page fault
load data
keep goingyes
no
24
otherwise, the service is interrupted anddata memory pages are loaded
![Page 25: Secure Tera-scale Data Crunching with a Small TCBbvavala/pub/dsn2017slides.pdf · -If Supervisor is trusted, invalid data => no resume (e.g.: TrustVisor) -If Supervisor is untrusted](https://reader033.vdocuments.us/reader033/viewer/2022053007/5f0b1fb27e708231d42ef727/html5/thumbnails/25.jpg)
untrusted env.
trusted env.
access data in block bi
is bi in memory?
app’s execution flow
handle page fault
load data
validate data
keep goingyes
no
25
data is validated inside trusted environment,independently from service execution
![Page 26: Secure Tera-scale Data Crunching with a Small TCBbvavala/pub/dsn2017slides.pdf · -If Supervisor is trusted, invalid data => no resume (e.g.: TrustVisor) -If Supervisor is untrusted](https://reader033.vdocuments.us/reader033/viewer/2022053007/5f0b1fb27e708231d42ef727/html5/thumbnails/26.jpg)
untrusted env.
trusted env.
access data in block bi
is bi in memory?
app’s execution flow
handle page fault
load data
validate data
keep goingyes
no
resume
26
service is resumed andonly if data is valid, service can make progress
![Page 27: Secure Tera-scale Data Crunching with a Small TCBbvavala/pub/dsn2017slides.pdf · -If Supervisor is trusted, invalid data => no resume (e.g.: TrustVisor) -If Supervisor is untrusted](https://reader033.vdocuments.us/reader033/viewer/2022053007/5f0b1fb27e708231d42ef727/html5/thumbnails/27.jpg)
…in practice…
![Page 28: Secure Tera-scale Data Crunching with a Small TCBbvavala/pub/dsn2017slides.pdf · -If Supervisor is trusted, invalid data => no resume (e.g.: TrustVisor) -If Supervisor is untrusted](https://reader033.vdocuments.us/reader033/viewer/2022053007/5f0b1fb27e708231d42ef727/html5/thumbnails/28.jpg)
trusted
untrustedUntrusted address
space
HardwareSGX/TPM
Architecture
other untrusted services
Trusted address space
state handler
service code
SMM(State map manager)
OSSupervisor
28
![Page 29: Secure Tera-scale Data Crunching with a Small TCBbvavala/pub/dsn2017slides.pdf · -If Supervisor is trusted, invalid data => no resume (e.g.: TrustVisor) -If Supervisor is untrusted](https://reader033.vdocuments.us/reader033/viewer/2022053007/5f0b1fb27e708231d42ef727/html5/thumbnails/29.jpg)
untrusted
untrustedUntrusted address
space
HardwareSGX/TPM
Architecture
other untrusted services
Trusted address space
state handler
service code
SMM(State map manager)
OSSupervisor
29
on TrustVisor, Supervisor is trusted on SGX, Supervisor is untrusted
![Page 30: Secure Tera-scale Data Crunching with a Small TCBbvavala/pub/dsn2017slides.pdf · -If Supervisor is trusted, invalid data => no resume (e.g.: TrustVisor) -If Supervisor is untrusted](https://reader033.vdocuments.us/reader033/viewer/2022053007/5f0b1fb27e708231d42ef727/html5/thumbnails/30.jpg)
LaStGT in 5 steps
•Offline data protection at the source •State registration •Data processing •Lazy loading from memory & disk •Execution verification
4
![Page 31: Secure Tera-scale Data Crunching with a Small TCBbvavala/pub/dsn2017slides.pdf · -If Supervisor is trusted, invalid data => no resume (e.g.: TrustVisor) -If Supervisor is untrusted](https://reader033.vdocuments.us/reader033/viewer/2022053007/5f0b1fb27e708231d42ef727/html5/thumbnails/31.jpg)
•Offline data protection at the source
•State registration •Data processing •Lazy loading from memory & disk •Verification
1
![Page 32: Secure Tera-scale Data Crunching with a Small TCBbvavala/pub/dsn2017slides.pdf · -If Supervisor is trusted, invalid data => no resume (e.g.: TrustVisor) -If Supervisor is untrusted](https://reader033.vdocuments.us/reader033/viewer/2022053007/5f0b1fb27e708231d42ef727/html5/thumbnails/32.jpg)
0 1 0 0 0 1 1 1 1 0 1 0 0 0
Data protectionHierarchical
- Incremental as data is createdMade for:
- Incremental validation as data is loaded
- Fast verification - Single hash tree is
unsuitable210
220
230
240
220 225 230 235 240 245 250
Tree
siz
e (b
ytes
)
State size (bytes)
bytes/block 210
215
220
225
230
0 1 0 0 0 1 1 1 1 0 1 0 0 0
32
![Page 33: Secure Tera-scale Data Crunching with a Small TCBbvavala/pub/dsn2017slides.pdf · -If Supervisor is trusted, invalid data => no resume (e.g.: TrustVisor) -If Supervisor is untrusted](https://reader033.vdocuments.us/reader033/viewer/2022053007/5f0b1fb27e708231d42ef727/html5/thumbnails/33.jpg)
State Hierarchy
files
blocks
chunk chunkchunks
masterchunkmaster chunks
directorydirectory
state rootroot
33
![Page 34: Secure Tera-scale Data Crunching with a Small TCBbvavala/pub/dsn2017slides.pdf · -If Supervisor is trusted, invalid data => no resume (e.g.: TrustVisor) -If Supervisor is untrusted](https://reader033.vdocuments.us/reader033/viewer/2022053007/5f0b1fb27e708231d42ef727/html5/thumbnails/34.jpg)
State Hierarchy
34
state root
directory
masterchunk
chunk chunkmasterchunk
chunk chunk
- components are loaded separately
- unneeded components not loaded in memory
- state root (1 hash) allows state validation
![Page 35: Secure Tera-scale Data Crunching with a Small TCBbvavala/pub/dsn2017slides.pdf · -If Supervisor is trusted, invalid data => no resume (e.g.: TrustVisor) -If Supervisor is untrusted](https://reader033.vdocuments.us/reader033/viewer/2022053007/5f0b1fb27e708231d42ef727/html5/thumbnails/35.jpg)
•Offline data protection at the source •State registration •Data processing •Lazy loading from memory & disk •Verification
2
![Page 36: Secure Tera-scale Data Crunching with a Small TCBbvavala/pub/dsn2017slides.pdf · -If Supervisor is trusted, invalid data => no resume (e.g.: TrustVisor) -If Supervisor is untrusted](https://reader033.vdocuments.us/reader033/viewer/2022053007/5f0b1fb27e708231d42ef727/html5/thumbnails/36.jpg)
Untrusted address space
Trusted address space
SMM(State map manager)
OSSupervisor
state handler
service code
When the trusted execution environment is created, only the code is available inside
36
![Page 37: Secure Tera-scale Data Crunching with a Small TCBbvavala/pub/dsn2017slides.pdf · -If Supervisor is trusted, invalid data => no resume (e.g.: TrustVisor) -If Supervisor is untrusted](https://reader033.vdocuments.us/reader033/viewer/2022053007/5f0b1fb27e708231d42ef727/html5/thumbnails/37.jpg)
Untrusted address space
Trusted address space
SMM(State map manager)
OSSupervisor
state handler
service code
grab root from disk
37
![Page 38: Secure Tera-scale Data Crunching with a Small TCBbvavala/pub/dsn2017slides.pdf · -If Supervisor is trusted, invalid data => no resume (e.g.: TrustVisor) -If Supervisor is untrusted](https://reader033.vdocuments.us/reader033/viewer/2022053007/5f0b1fb27e708231d42ef727/html5/thumbnails/38.jpg)
Untrusted address space
Trusted address space
SMM(State map manager)
OSSupervisor
state handler
service code
grab root from diskregister state
- registration is the first execution
- state handler installs root
- root is trusted
38
![Page 39: Secure Tera-scale Data Crunching with a Small TCBbvavala/pub/dsn2017slides.pdf · -If Supervisor is trusted, invalid data => no resume (e.g.: TrustVisor) -If Supervisor is untrusted](https://reader033.vdocuments.us/reader033/viewer/2022053007/5f0b1fb27e708231d42ef727/html5/thumbnails/39.jpg)
Untrusted address space
Trusted address space
SMM(State map manager)
OSSupervisor
state handler
service code
state root
39
- state root is available before service code runs
![Page 40: Secure Tera-scale Data Crunching with a Small TCBbvavala/pub/dsn2017slides.pdf · -If Supervisor is trusted, invalid data => no resume (e.g.: TrustVisor) -If Supervisor is untrusted](https://reader033.vdocuments.us/reader033/viewer/2022053007/5f0b1fb27e708231d42ef727/html5/thumbnails/40.jpg)
•Offline data protection at the source •State registration •Data processing •Lazy loading from memory & disk •Verification
3
![Page 41: Secure Tera-scale Data Crunching with a Small TCBbvavala/pub/dsn2017slides.pdf · -If Supervisor is trusted, invalid data => no resume (e.g.: TrustVisor) -If Supervisor is untrusted](https://reader033.vdocuments.us/reader033/viewer/2022053007/5f0b1fb27e708231d42ef727/html5/thumbnails/41.jpg)
- service code has view of entire state - state not readily available: inefficient loading it upfront
pages NOT availablepages available
OSSupervisor
Untrusted address space
Trusted address space
SMM
41
state handler
service code
state root data
![Page 42: Secure Tera-scale Data Crunching with a Small TCBbvavala/pub/dsn2017slides.pdf · -If Supervisor is trusted, invalid data => no resume (e.g.: TrustVisor) -If Supervisor is untrusted](https://reader033.vdocuments.us/reader033/viewer/2022053007/5f0b1fb27e708231d42ef727/html5/thumbnails/42.jpg)
page hit on access
- Service code execution begins - Service accesses data in memory - Data retrieval is fast if data is already available
OSSupervisor
Untrusted address space
Trusted address space
SMM
42
state handler
service code
state root data
![Page 43: Secure Tera-scale Data Crunching with a Small TCBbvavala/pub/dsn2017slides.pdf · -If Supervisor is trusted, invalid data => no resume (e.g.: TrustVisor) -If Supervisor is untrusted](https://reader033.vdocuments.us/reader033/viewer/2022053007/5f0b1fb27e708231d42ef727/html5/thumbnails/43.jpg)
page miss on access
OSSupervisor
- Service code may access data on missing pages
Untrusted address space
Trusted address space
SMM
43
state handler
service code
state root data
![Page 44: Secure Tera-scale Data Crunching with a Small TCBbvavala/pub/dsn2017slides.pdf · -If Supervisor is trusted, invalid data => no resume (e.g.: TrustVisor) -If Supervisor is untrusted](https://reader033.vdocuments.us/reader033/viewer/2022053007/5f0b1fb27e708231d42ef727/html5/thumbnails/44.jpg)
- A page fault is triggered - Execution is interrupted,
seamlessly waiting to continue
OSSupervisor
page fault!
Untrusted address space
Trusted address space
SMM
44
state handler
service code
state root data
![Page 45: Secure Tera-scale Data Crunching with a Small TCBbvavala/pub/dsn2017slides.pdf · -If Supervisor is trusted, invalid data => no resume (e.g.: TrustVisor) -If Supervisor is untrusted](https://reader033.vdocuments.us/reader033/viewer/2022053007/5f0b1fb27e708231d42ef727/html5/thumbnails/45.jpg)
•Offline data protection at the source •State registration •Data processing •Lazy loading from memory & disk •Verification
4
![Page 46: Secure Tera-scale Data Crunching with a Small TCBbvavala/pub/dsn2017slides.pdf · -If Supervisor is trusted, invalid data => no resume (e.g.: TrustVisor) -If Supervisor is untrusted](https://reader033.vdocuments.us/reader033/viewer/2022053007/5f0b1fb27e708231d42ef727/html5/thumbnails/46.jpg)
Untrusted address space
Trusted address space
SMM
OSSupervisor
page fault!
46
state handler
service code
state root data
![Page 47: Secure Tera-scale Data Crunching with a Small TCBbvavala/pub/dsn2017slides.pdf · -If Supervisor is trusted, invalid data => no resume (e.g.: TrustVisor) -If Supervisor is untrusted](https://reader033.vdocuments.us/reader033/viewer/2022053007/5f0b1fb27e708231d42ef727/html5/thumbnails/47.jpg)
Untrusted address space
Trusted address space
OSSupervisor
page fault!
page address
grab state component from disk- Let SMM handle missing data
- SMM loads data from disk
SMM
47
state handler
service code
state root data
![Page 48: Secure Tera-scale Data Crunching with a Small TCBbvavala/pub/dsn2017slides.pdf · -If Supervisor is trusted, invalid data => no resume (e.g.: TrustVisor) -If Supervisor is untrusted](https://reader033.vdocuments.us/reader033/viewer/2022053007/5f0b1fb27e708231d42ef727/html5/thumbnails/48.jpg)
Untrusted address space
Trusted address space
OSSupervisor
page fault!
SMM data
validate data
- in TrustVisor, validate in place - in SGX, copy, validate, copy
48
state handler
service code
state root data
![Page 49: Secure Tera-scale Data Crunching with a Small TCBbvavala/pub/dsn2017slides.pdf · -If Supervisor is trusted, invalid data => no resume (e.g.: TrustVisor) -If Supervisor is untrusted](https://reader033.vdocuments.us/reader033/viewer/2022053007/5f0b1fb27e708231d42ef727/html5/thumbnails/49.jpg)
Untrusted address space
Trusted address space
OSSupervisor
page fault!
SMM data
data is valid
- If Supervisor is trusted,invalid data => no resume (e.g.: TrustVisor)
- If Supervisor is untrustedinvalid data => no accept,so no access (e.g.: SGX)
49
state handler
service code
state root data
![Page 50: Secure Tera-scale Data Crunching with a Small TCBbvavala/pub/dsn2017slides.pdf · -If Supervisor is trusted, invalid data => no resume (e.g.: TrustVisor) -If Supervisor is untrusted](https://reader033.vdocuments.us/reader033/viewer/2022053007/5f0b1fb27e708231d42ef727/html5/thumbnails/50.jpg)
Untrusted address space
Trusted address space
OSSupervisor
SMM data
page hit on access
resume
fault solved, data accessible on resume, continue…
50
state handler
service code
state root data
![Page 51: Secure Tera-scale Data Crunching with a Small TCBbvavala/pub/dsn2017slides.pdf · -If Supervisor is trusted, invalid data => no resume (e.g.: TrustVisor) -If Supervisor is untrusted](https://reader033.vdocuments.us/reader033/viewer/2022053007/5f0b1fb27e708231d42ef727/html5/thumbnails/51.jpg)
•Offline data protection at the source •State registration •Data processing •Lazy loading from memory & disk •Execution verification
5- HW-based attestation of code identity,
including input request, state root, output reply, nonce
- Client checks validity of attestationand intended identities/hashes
![Page 52: Secure Tera-scale Data Crunching with a Small TCBbvavala/pub/dsn2017slides.pdf · -If Supervisor is trusted, invalid data => no resume (e.g.: TrustVisor) -If Supervisor is untrusted](https://reader033.vdocuments.us/reader033/viewer/2022053007/5f0b1fb27e708231d42ef727/html5/thumbnails/52.jpg)
Outline
• Goal • Previous Work • Our solution: key ideas and
overview • Implemention(s) • Evaluation
![Page 53: Secure Tera-scale Data Crunching with a Small TCBbvavala/pub/dsn2017slides.pdf · -If Supervisor is trusted, invalid data => no resume (e.g.: TrustVisor) -If Supervisor is untrusted](https://reader033.vdocuments.us/reader033/viewer/2022053007/5f0b1fb27e708231d42ef727/html5/thumbnails/53.jpg)
TCB size
VC3 HavenLaStGT
hypervisor library SQLite (example)
KSLoC (lines of
code x 1000)9.2 O(103) 17 7.7 92.6
library is small compared to real service
SGX-basedTPM/TXT
based
53
![Page 54: Secure Tera-scale Data Crunching with a Small TCBbvavala/pub/dsn2017slides.pdf · -If Supervisor is trusted, invalid data => no resume (e.g.: TrustVisor) -If Supervisor is untrusted](https://reader033.vdocuments.us/reader033/viewer/2022053007/5f0b1fb27e708231d42ef727/html5/thumbnails/54.jpg)
load&hash data upfront
LaStGT entry/exit & block validation
ComparisonXMHF-TrustVisor vs. LaStGT
024681012
0MB
128MB
256MB
384MB
512MB
seconds
TrustVisorLaSt-GT
LaStGT is Incremental, Faster & Scalable
LaStGT chunk loading
54
![Page 55: Secure Tera-scale Data Crunching with a Small TCBbvavala/pub/dsn2017slides.pdf · -If Supervisor is trusted, invalid data => no resume (e.g.: TrustVisor) -If Supervisor is untrusted](https://reader033.vdocuments.us/reader033/viewer/2022053007/5f0b1fb27e708231d42ef727/html5/thumbnails/55.jpg)
SQLite on LaStGT
0.0
0.2
0.4
128MB
256MB
512MB
1GB
2GB
... 0.25TB
3.2
3.4
3.6seconds
0
5
10
15
20
1MB
2MB
4MB
8MB
16MB
32MB
64MB
128MB
256MB
512MB
1GB
2GB
... 0.25TB
seconds
- First large-scale experiment on hypervisor
- Data I/O can be optimized through state hierarchy
- SGX expected to improve substantially
55
![Page 56: Secure Tera-scale Data Crunching with a Small TCBbvavala/pub/dsn2017slides.pdf · -If Supervisor is trusted, invalid data => no resume (e.g.: TrustVisor) -If Supervisor is untrusted](https://reader033.vdocuments.us/reader033/viewer/2022053007/5f0b1fb27e708231d42ef727/html5/thumbnails/56.jpg)
Conclusions• Security for large-scale data processing can
be guaranteed with a small TCB
• Virtual memory-based data handling => zero interface
• No change to source code => easy integration
• One design can fit diverse HW & SW
![Page 57: Secure Tera-scale Data Crunching with a Small TCBbvavala/pub/dsn2017slides.pdf · -If Supervisor is trusted, invalid data => no resume (e.g.: TrustVisor) -If Supervisor is untrusted](https://reader033.vdocuments.us/reader033/viewer/2022053007/5f0b1fb27e708231d42ef727/html5/thumbnails/57.jpg)
Secure Tera-scale Data Crunching with a Small TCBBruno Vavala1,2, Nuno Neves1, Peter Steenkiste2
1LaSIGE, Faculdade de Ciencias, Universidade de Lisboa, Portugal 2CSD, Carnegie Mellon University, U.S.
Abstract—Outsourcing services to third-party providers comeswith a high security cost—to fully trust the providers. Us-ing trusted hardware can help, but current trusted executionenvironments do not adequately support services that processvery large scale datasets. We present LASTGT, a system thatbridges this gap by supporting the execution of self-containedservices over a large state, with a small and generic trustedcomputing base (TCB). LASTGT uses widely deployed trustedhardware to guarantee integrity and verifiability of the executionon a remote platform, and it securely supplies data to theservice through simple techniques based on virtual memory. Asa result, LASTGT is general and applicable to many scenariossuch as computational genomics and databases, as we showin our experimental evaluation based on an implementation ofLASTGT on a secure hypervisor. We also describe a possibleimplementation on Intel SGX.
1. INTRODUCTIONOutsourced applications such as cloud services (databases,
storage, etc.) are widely deployed but strong security guaran-tees are taken for granted. The de facto security model assumesthat the service provider is fully trusted. In the real world, how-ever, one third of the top threats listed by the Cloud SecurityAlliance [33] concern an attacker tampering with the integrityof computation or data, namely: (i) service hijacking [38], (ii)malicious insiders [1], (iii) system vulnerabilities [39], and (iv)shared technology issues [2]. This can raise suspicions on thetrustworthiness of the results produced by a service.
The above threats stem from at least three issues:• the lack of strong execution isolation, whereby a sub-
verted OS, or hypervisor, or service application, can makethreats affect other running software.
• a large TCB, which makes systems hard to verify; also,when it includes the OS—containing millions of lines ofcode [35]—a bug in the kernel [34] endangers security ofall the applications and data.
• a complex OS interface—hundreds of system calls—which is difficult to secure [30] and whose maliciousalteration can subvert an application [3].
Unfortunately, service owners and end-users have little or nomeans to distinguish between correct and compromised servicecode or input data by just looking at the results received fromthe cloud.
Trusted Computing (TC) technology is making progresstowards allowing clients to verify results. The technology (e.g.,Trusted Platform Modules (TPMs) [4] and Intel SGX [29]) isavailable in commodity platforms, and it is tied to a hardwareroot of trust certified by the manufacturer. This can be usedby a service provider to isolate the service execution and toattest the identity of the executed code for remote verification.
Software support for such trusted hardware however is not(or just partially) suitable for many applications that processa huge amount of data (e.g., clinical decision support [5],predictive risk assessment for diseases [6], malware detection[7], workloads for sensitive financial records outsourced onpublic clouds [8], and genome analytics [9]). Previous systems
support the execution of either small pieces of code and data[10], or large code bases [11], or specific software like databaseengines [12] or MapReduce applications [13]. Recent work[14] has shown how to support unmodified services. However,since ”the interface between modern applications and operatingsystems is so complex” [30], it relies on a considerable TCBthat includes a library OS. In addition, the above systems arespecific for TPMs [10], [15], secure coprocessors [12], or IntelSGX [13]. Hence, porting them to alternative architectures(e.g., the upcoming AMD Secure Memory Encryption andSecure Encrypted Virtualization [36], [37]) requires significanteffort. Clearly, it is desirable to design a generic system “notrelying on idiosyncratic features of the hardware” [16].
We present LASTGT, a system that can handle a LArgeSTate on a Generic Trusted component with a small TCB.LASTGT supports a wide range of applications and hardwarebecause its design only relies on commonly available hardwarefeatures—mainly paged virtual memory. LASTGT uses mem-ory maps that allow the application to manage the placementof data in memory, and authenticated data structures forefficiently validating the data before it is processed. As mostof the LASTGT’s mechanisms (e.g., data validation and mem-ory management) are implemented at the application level,they can be optimized for different application requirements.LASTGT ultimately delivers the following guarantee: if theclient can verify the results attested by the trusted componenton the service provider platform, then the client request wasprocessed by the intended code on the intended input state, sothe received response can be trusted.
We provide the following contributions.• We describe LASTGT’s design, and show how it can
protect large-scale data in memory efficiently and how itenables a client to verify the correctness of service code,data and results.
• We detail how LASTGT has been implemented on XMHF-TrustVisor [10] using a commodity platform equippedwith a TPM. Also, we discuss a possible implementationusing the Intel SGX instruction set. In addition, we high-light important differences between the two architecturesand how LASTGT deals with them.
• We evaluate our XMHF-TrustVisor-based implementationfor datasets up to one terabyte. We show that LASTGT hasa small TCB compared to state-of-the-art prototypes, andgood performance. We also discuss expected improve-ments with an SGX-based implementation.
2. RELATED WORKWe describe related work on trusted execution, trusted
execution targeting large-scale data, and other solutions forensuring the integrity of computation on large data.Trusted Execution Environments. TrustVisor [10], Mini-box [15] and Haven [14] all support secure execution. Thefirst two focus on keeping the TCB small by removing theOS from the trust boundaries, thus supporting self-containedapplications (i.e., with statically linked libraries and no OS
ad maiora.
IEEE/IFIP DSN’17 Bruno Vavala, UL / CMU, [email protected]
![Page 58: Secure Tera-scale Data Crunching with a Small TCBbvavala/pub/dsn2017slides.pdf · -If Supervisor is trusted, invalid data => no resume (e.g.: TrustVisor) -If Supervisor is untrusted](https://reader033.vdocuments.us/reader033/viewer/2022053007/5f0b1fb27e708231d42ef727/html5/thumbnails/58.jpg)
(blank)
58
![Page 59: Secure Tera-scale Data Crunching with a Small TCBbvavala/pub/dsn2017slides.pdf · -If Supervisor is trusted, invalid data => no resume (e.g.: TrustVisor) -If Supervisor is untrusted](https://reader033.vdocuments.us/reader033/viewer/2022053007/5f0b1fb27e708231d42ef727/html5/thumbnails/59.jpg)
Steve Wozniak
“Never trust a computer you can’t throw out the window.”
59
![Page 60: Secure Tera-scale Data Crunching with a Small TCBbvavala/pub/dsn2017slides.pdf · -If Supervisor is trusted, invalid data => no resume (e.g.: TrustVisor) -If Supervisor is untrusted](https://reader033.vdocuments.us/reader033/viewer/2022053007/5f0b1fb27e708231d42ef727/html5/thumbnails/60.jpg)
(excerpt from) Intel’s Legal Desclaimer
“No computer system can be absolutely secure.”
60