1

Runtime VM Protection By Intelreg Multi-Key Total Memory Encryption

(MKTME)Kai Huang Intel Corporation

LINUXCON + CONTAINERCON + CLOUDOPENBeijing China 2018

2

Legal Disclaimer

No license (express or implied by estoppel or otherwise) to any intellectual property rights is granted by this document

Intel disclaims all express and implied warranties including without limitation the implied warranties of merchantability fitness for a particular purpose and non-infringement as well as any warranty arising from course of performance course of dealing or usage in trade

This document contains information on products services andor processes in development All information provided here is subject to change without notice Contact your Intel representative to obtain the latest forecast schedule specifications and roadmaps

The products and services described may contain defects or errors known as errata which may cause deviations from published specifications Current characterized errata are available on request

Copies of documents which have an order number and are referenced in this document may be obtained by calling 1-800-548-4725 or by visiting wwwintelcomdesignliteraturehtm

Intel and the Intel logo are trademarks of Intel Corporation or its subsidiaries in the US andor other countries

Other names and brands may be claimed as the property of others

copy Intel Corporation

3

Agenda

bull MKTME Introductionbull MKTME Use Casesbull MKTME Enabling

4

Background Trusted VM in Cloud

VM protection by using encryptionbull VM encrypted lsquoat-restrsquo lsquoin-transitrsquo and lsquoruntimersquobull There has been existing technologies for lsquoat-restrsquo

and lsquoin-transitrsquo encryptionbull Qemu TLS support for live migrationbull Qemu encrypted image support

bull VM runtime encryption requires hardware memory encryption support

bull AMDreg SMESEVbull Intelreg MKTME

Launch VM on lsquoTrustiness Verifiedrsquo Hostbull Trusted hardwarelocation etcbull Trusted Cloud SW stack

Cloud Orchestrator

Cloud Agent

VM Image Repo

Compute NodeIntelreg Arch

VM Launch

Cloud Agent

Compute NodeIntelreg Arch

VM LaunchLive Migration

❶ At-rest

❷ Runtime❸ In-transit

Typical VM Lifecycle in Cloud

5

Trusted VM w OpenCIT -- OpenStack as ExampleEnterprise Data Center Cloud Service Provider

Key Broker

OpenStack Barbican

Trust Director

OpenStack Nova

OpenStack Glance

OpenStack Horizon

AppOS

AppOS

AppOS

TPM Intel Arch w TXT

Compute Node

R e p o r t i n g

P o l i c y E n f o r c e m e n t

V e r i f i e r

Trust Agent

In t e l O p e n S t a c k

Attestation Server

KB Proxy

M e a s u r e m e n t

Attestation hub

tbootxm

Intelreg Open CIT helps on Host trustiness verification

6

TME amp MKTME Introduction

bull New AES-XTS engine in data path to external memory busbull Data encrypteddecrypted on-the-fly when enteringleaving

memory

bull AES-XTS uses physical address as ldquotweakrdquobull Same plaintext different physical address -gt different ciphertext

bull TME (Total Memory Encryption)bull Full memory encryption by TME key (CPU generated)

bull EnabledDisabled by BIOS

bull Transparent to OS amp user apps

bull MKTME (Multi-key Total Memory Encryption)bull Memory encryption by using multiple keys

bull Use upper bits of physical address as keyID (see next)

7

MKTME KeyIDs

bull Repurpose upper bits of physical address as KeyID as shown belowbull Reduces useable physical address bits

bull Creates ldquoaliasesrdquo of physical memory locations different keyIDs can refer to the same page

bull Cache-coherency is not guaranteed for the same page that accessed by different keyIDsbull CPU caches are unaware of keyID(still treat keyIDas part of PA)

bull Architecturally upto 2^15-1 keyIDs (15 keyID bits)bull Reported by MSR Configured by BIOSbull KeyID 0 is reserved as TMErsquos key (not useable by MKTME)

bull New PCONFIG instruction to program keyID w associated key (see next)

Platform Addressable MemoryKeyID

MAXPHYADDR - 1

0

Reserved63

MK_TME_MAX_KEYID_BITS

8

MKTME KeyID Programming Overview

New Ring-0 instruction PCONFIG to program the KEYID and associated keybull Package scoped

bull Supports programming keyID to 4 modes

bull Using CPU generated random ephemeral key (invisible to SW)

bull Using SW provided key (tenantrsquos key)

bull No encryption ndash plaintext domain

bull Clearing a key (using TMErsquos key effectively)

bull Allows SW to specify crypto algorithms

bull Only AES-XTS-128 for initial server intercept

8

9

VM Protection amp Isolation With MKTME

bull Protectionbull Use keyID to encrypt VM memory at runtime

bull Isolationbull Use different keyIDs for different VMs

bull Software Enablingbull For CPU access SW sets keyID at PTEsbull IA page table (host)bull EPT (KVM)

bull For Device access (DMA)bull w IOMMU Set keyIDto IOMMU page tablebull Physical DMA Apply keyIDto PA directly

10

Highlights of MKTME

Guests continue to run ldquowithout modificationsrdquo in MKTME domainsbull Encrypted with 1) CPU-generated ephemeral key or 2) the one provided by API (ldquotenant-

controlled keysrdquo)

bull Virtio including optimization (direct access to guest memory by kernel) continues to work

bull Direct IO (including accelerators FPGA) assignment (including SR-IOV VFs) is available

bull Live migration can be supported (among platforms that support MKTME)

bull vNVDIMM can be supported w limitation (because of physical address ldquotweakrdquo)

bull Host DIMM configuration cannot be changed cross reboots

bull Qemu DIMM amp vNVDIMM configuration cannot be changed cross VM reboots

11

Agenda

bull MKTME Introductionbull MKTME Use Casesbull MKTME Enabling

12

MKTME Enabled Use Cases

1 Launch Tenant VMs with in-use protection (CPU generated keys)bull Let CSP handle the keys

bull VM image provided by CSP

2 Launch Tenant VMs with at-rest and in-use protection with full tenant-controlbull VM image encrypted rest with tenant-specific keys

bull Keys in tenant controlbull VM memory isolation with tenant-specific keys

bull Trustiness verified hostbull Additional integrity verification of VM image

Use-case Extension KeyID Sharing for all VMs launched by single tenant with the same tenant-key (or CPU generated key)

13

Cloud

Agent

HARDWARE

Intelreg TXT TPM

Intelreg MKTME

HYPERVISOR (QemuKVM)

VM

Policy

Agent

MKTME ephemeral key

DRAM

Keyid 3

Keyid 3

Keyid 0

Keyid 0

Keyid 3

Cloud

Manager

1

2

3

Image

Repo

VM Launch w CPU Generated Keys

TME key

VM Launch

Policy Enforcement

VM Launchw KeyID

CSP Controlled

VM Launch wbull CPU generated keybull CSP provided VM image

Security Propertiesbull w VM runtime protectionbull w or wo at-rest and in-transit protectionbull No Host Trustiness Verification

14

Cloud Agent

Attestation Service

HYPERVISOR (QemuKVM)

VM

DRAM

KeyID 3

KeyID 3

KeyID 1

KeyID 1

KeyID 3

Cloud Manager

2

3

4

ImageRepo

VM Launch w Tenant Controlled Keys

1TrustinessVerification

VM Launch

Policy Enforcement

Key Server

Upload VM image encryptedw tenant key

0

Request Tenant-keyReturn wrapped key

Wrapped tenant key

Tenant key

MKTME ephemeral key

VM Launchw KeyID

VM Launch wbull Tenant provided keybull Tenant provided encrypted VM imagebull Tenant controlled key serverbull Trustiness verified hostbull VM image integrity verifiedbull Use TPM to wrapunwrap tenant-key

CSP ControlledTenant Controlled

Security Propertiesbull w VM runtime protectionbull w VM at-rest protectionbull w or wo in-transit protectionbull w Host trustiness verificationbull w VM image integrity verification

PolicyAgent

TME key

HARDWARE Intelreg TXT TPM

Intelreg MKTME

15

KeyID Sharing Among VMs

Compute Node

Cloud SW

mKey API KVM

MktmePolicy

tenant_id ltUUIDgtkey_type ldquoephemeralrdquo | ldquopersistentrdquo

key_server httpsallow_to_share ldquoyesrdquo | ldquonordquo

Qemu

KeyID P olicy KeyID V M s

Policy1 lttenant1 ldquoephem eralrdquogt keyID 1 VM 1 VM 2

Policy2 lttenant2 ldquopersistentrdquo xxxxxxgt keyID 2 VM 3

N ew VM Launch

w M ktm ePolicy

Exam ple KeyID sharing is based on KeyID Policy lttenant_id key_type tenant_keygt

Cloud SW bull M aintains lsquoKeyID Policy-to-KeyID rsquo tablebull M akes keyID sharing decision according to the tablebull U pdates the table on VM launch and teardow n

m Key API M KTM E key m anagem ent API

Apply keyID to

VM memory

Launch VM

w keyID

Cloud SW makes decision whether to share or not

Launch VM

2

Legal Disclaimer

No license (express or implied by estoppel or otherwise) to any intellectual property rights is granted by this document

Intel disclaims all express and implied warranties including without limitation the implied warranties of merchantability fitness for a particular purpose and non-infringement as well as any warranty arising from course of performance course of dealing or usage in trade

This document contains information on products services andor processes in development All information provided here is subject to change without notice Contact your Intel representative to obtain the latest forecast schedule specifications and roadmaps

The products and services described may contain defects or errors known as errata which may cause deviations from published specifications Current characterized errata are available on request

Copies of documents which have an order number and are referenced in this document may be obtained by calling 1-800-548-4725 or by visiting wwwintelcomdesignliteraturehtm

Intel and the Intel logo are trademarks of Intel Corporation or its subsidiaries in the US andor other countries

Other names and brands may be claimed as the property of others

copy Intel Corporation

3

Agenda

bull MKTME Introductionbull MKTME Use Casesbull MKTME Enabling

4

Background Trusted VM in Cloud

VM protection by using encryptionbull VM encrypted lsquoat-restrsquo lsquoin-transitrsquo and lsquoruntimersquobull There has been existing technologies for lsquoat-restrsquo

and lsquoin-transitrsquo encryptionbull Qemu TLS support for live migrationbull Qemu encrypted image support

bull VM runtime encryption requires hardware memory encryption support

bull AMDreg SMESEVbull Intelreg MKTME

Launch VM on lsquoTrustiness Verifiedrsquo Hostbull Trusted hardwarelocation etcbull Trusted Cloud SW stack

Cloud Orchestrator

Cloud Agent

VM Image Repo

Compute NodeIntelreg Arch

VM Launch

Cloud Agent

Compute NodeIntelreg Arch

VM LaunchLive Migration

❶ At-rest

❷ Runtime❸ In-transit

Typical VM Lifecycle in Cloud

5

Trusted VM w OpenCIT -- OpenStack as ExampleEnterprise Data Center Cloud Service Provider

Key Broker

OpenStack Barbican

Trust Director

OpenStack Nova

OpenStack Glance

OpenStack Horizon

AppOS

AppOS

AppOS

TPM Intel Arch w TXT

Compute Node

R e p o r t i n g

P o l i c y E n f o r c e m e n t

V e r i f i e r

Trust Agent

In t e l O p e n S t a c k

Attestation Server

KB Proxy

M e a s u r e m e n t

Attestation hub

tbootxm

Intelreg Open CIT helps on Host trustiness verification

6

TME amp MKTME Introduction

bull New AES-XTS engine in data path to external memory busbull Data encrypteddecrypted on-the-fly when enteringleaving

memory

bull AES-XTS uses physical address as ldquotweakrdquobull Same plaintext different physical address -gt different ciphertext

bull TME (Total Memory Encryption)bull Full memory encryption by TME key (CPU generated)

bull EnabledDisabled by BIOS

bull Transparent to OS amp user apps

bull MKTME (Multi-key Total Memory Encryption)bull Memory encryption by using multiple keys

bull Use upper bits of physical address as keyID (see next)

7

MKTME KeyIDs

bull Repurpose upper bits of physical address as KeyID as shown belowbull Reduces useable physical address bits

bull Creates ldquoaliasesrdquo of physical memory locations different keyIDs can refer to the same page

bull Cache-coherency is not guaranteed for the same page that accessed by different keyIDsbull CPU caches are unaware of keyID(still treat keyIDas part of PA)

bull Architecturally upto 2^15-1 keyIDs (15 keyID bits)bull Reported by MSR Configured by BIOSbull KeyID 0 is reserved as TMErsquos key (not useable by MKTME)

bull New PCONFIG instruction to program keyID w associated key (see next)

Platform Addressable MemoryKeyID

MAXPHYADDR - 1

0

Reserved63

MK_TME_MAX_KEYID_BITS

8

MKTME KeyID Programming Overview

New Ring-0 instruction PCONFIG to program the KEYID and associated keybull Package scoped

bull Supports programming keyID to 4 modes

bull Using CPU generated random ephemeral key (invisible to SW)

bull Using SW provided key (tenantrsquos key)

bull No encryption ndash plaintext domain

bull Clearing a key (using TMErsquos key effectively)

bull Allows SW to specify crypto algorithms

bull Only AES-XTS-128 for initial server intercept

8

9

VM Protection amp Isolation With MKTME

bull Protectionbull Use keyID to encrypt VM memory at runtime

bull Isolationbull Use different keyIDs for different VMs

bull Software Enablingbull For CPU access SW sets keyID at PTEsbull IA page table (host)bull EPT (KVM)

bull For Device access (DMA)bull w IOMMU Set keyIDto IOMMU page tablebull Physical DMA Apply keyIDto PA directly

10

Highlights of MKTME

Guests continue to run ldquowithout modificationsrdquo in MKTME domainsbull Encrypted with 1) CPU-generated ephemeral key or 2) the one provided by API (ldquotenant-

controlled keysrdquo)

bull Virtio including optimization (direct access to guest memory by kernel) continues to work

bull Direct IO (including accelerators FPGA) assignment (including SR-IOV VFs) is available

bull Live migration can be supported (among platforms that support MKTME)

bull vNVDIMM can be supported w limitation (because of physical address ldquotweakrdquo)

bull Host DIMM configuration cannot be changed cross reboots

bull Qemu DIMM amp vNVDIMM configuration cannot be changed cross VM reboots

11

Agenda

bull MKTME Introductionbull MKTME Use Casesbull MKTME Enabling

12

MKTME Enabled Use Cases

1 Launch Tenant VMs with in-use protection (CPU generated keys)bull Let CSP handle the keys

bull VM image provided by CSP

2 Launch Tenant VMs with at-rest and in-use protection with full tenant-controlbull VM image encrypted rest with tenant-specific keys

bull Keys in tenant controlbull VM memory isolation with tenant-specific keys

bull Trustiness verified hostbull Additional integrity verification of VM image

Use-case Extension KeyID Sharing for all VMs launched by single tenant with the same tenant-key (or CPU generated key)

13

Cloud

Agent

HARDWARE

Intelreg TXT TPM

Intelreg MKTME

HYPERVISOR (QemuKVM)

VM

Policy

Agent

MKTME ephemeral key

DRAM

Keyid 3

Keyid 3

Keyid 0

Keyid 0

Keyid 3

Cloud

Manager

1

2

3

Image

Repo

VM Launch w CPU Generated Keys

TME key

VM Launch

Policy Enforcement

VM Launchw KeyID

CSP Controlled

VM Launch wbull CPU generated keybull CSP provided VM image

Security Propertiesbull w VM runtime protectionbull w or wo at-rest and in-transit protectionbull No Host Trustiness Verification

14

Cloud Agent

Attestation Service

HYPERVISOR (QemuKVM)

VM

DRAM

KeyID 3

KeyID 3

KeyID 1

KeyID 1

KeyID 3

Cloud Manager

2

3

4

ImageRepo

VM Launch w Tenant Controlled Keys

1TrustinessVerification

VM Launch

Policy Enforcement

Key Server

Upload VM image encryptedw tenant key

0

Request Tenant-keyReturn wrapped key

Wrapped tenant key

Tenant key

MKTME ephemeral key

VM Launchw KeyID

VM Launch wbull Tenant provided keybull Tenant provided encrypted VM imagebull Tenant controlled key serverbull Trustiness verified hostbull VM image integrity verifiedbull Use TPM to wrapunwrap tenant-key

CSP ControlledTenant Controlled

Security Propertiesbull w VM runtime protectionbull w VM at-rest protectionbull w or wo in-transit protectionbull w Host trustiness verificationbull w VM image integrity verification

PolicyAgent

TME key

HARDWARE Intelreg TXT TPM

Intelreg MKTME

15

KeyID Sharing Among VMs

Compute Node

Cloud SW

mKey API KVM

MktmePolicy

tenant_id ltUUIDgtkey_type ldquoephemeralrdquo | ldquopersistentrdquo

key_server httpsallow_to_share ldquoyesrdquo | ldquonordquo

Qemu

KeyID P olicy KeyID V M s

Policy1 lttenant1 ldquoephem eralrdquogt keyID 1 VM 1 VM 2

Policy2 lttenant2 ldquopersistentrdquo xxxxxxgt keyID 2 VM 3

N ew VM Launch

w M ktm ePolicy

Exam ple KeyID sharing is based on KeyID Policy lttenant_id key_type tenant_keygt

Cloud SW bull M aintains lsquoKeyID Policy-to-KeyID rsquo tablebull M akes keyID sharing decision according to the tablebull U pdates the table on VM launch and teardow n

m Key API M KTM E key m anagem ent API

Apply keyID to

VM memory

Launch VM

w keyID

Cloud SW makes decision whether to share or not

Launch VM

3

Agenda

bull MKTME Introductionbull MKTME Use Casesbull MKTME Enabling

4

Background Trusted VM in Cloud

VM protection by using encryptionbull VM encrypted lsquoat-restrsquo lsquoin-transitrsquo and lsquoruntimersquobull There has been existing technologies for lsquoat-restrsquo

and lsquoin-transitrsquo encryptionbull Qemu TLS support for live migrationbull Qemu encrypted image support

bull VM runtime encryption requires hardware memory encryption support

bull AMDreg SMESEVbull Intelreg MKTME

Launch VM on lsquoTrustiness Verifiedrsquo Hostbull Trusted hardwarelocation etcbull Trusted Cloud SW stack

Cloud Orchestrator

Cloud Agent

VM Image Repo

Compute NodeIntelreg Arch

VM Launch

Cloud Agent

Compute NodeIntelreg Arch

VM LaunchLive Migration

❶ At-rest

❷ Runtime❸ In-transit

Typical VM Lifecycle in Cloud

5

Trusted VM w OpenCIT -- OpenStack as ExampleEnterprise Data Center Cloud Service Provider

Key Broker

OpenStack Barbican

Trust Director

OpenStack Nova

OpenStack Glance

OpenStack Horizon

AppOS

AppOS

AppOS

TPM Intel Arch w TXT

Compute Node

R e p o r t i n g

P o l i c y E n f o r c e m e n t

V e r i f i e r

Trust Agent

In t e l O p e n S t a c k

Attestation Server

KB Proxy

M e a s u r e m e n t

Attestation hub

tbootxm

Intelreg Open CIT helps on Host trustiness verification

6

TME amp MKTME Introduction

bull New AES-XTS engine in data path to external memory busbull Data encrypteddecrypted on-the-fly when enteringleaving

memory

bull AES-XTS uses physical address as ldquotweakrdquobull Same plaintext different physical address -gt different ciphertext

bull TME (Total Memory Encryption)bull Full memory encryption by TME key (CPU generated)

bull EnabledDisabled by BIOS

bull Transparent to OS amp user apps

bull MKTME (Multi-key Total Memory Encryption)bull Memory encryption by using multiple keys

bull Use upper bits of physical address as keyID (see next)

7

MKTME KeyIDs

bull Repurpose upper bits of physical address as KeyID as shown belowbull Reduces useable physical address bits

bull Creates ldquoaliasesrdquo of physical memory locations different keyIDs can refer to the same page

bull Cache-coherency is not guaranteed for the same page that accessed by different keyIDsbull CPU caches are unaware of keyID(still treat keyIDas part of PA)

bull Architecturally upto 2^15-1 keyIDs (15 keyID bits)bull Reported by MSR Configured by BIOSbull KeyID 0 is reserved as TMErsquos key (not useable by MKTME)

bull New PCONFIG instruction to program keyID w associated key (see next)

Platform Addressable MemoryKeyID

MAXPHYADDR - 1

0

Reserved63

MK_TME_MAX_KEYID_BITS

8

MKTME KeyID Programming Overview

New Ring-0 instruction PCONFIG to program the KEYID and associated keybull Package scoped

bull Supports programming keyID to 4 modes

bull Using CPU generated random ephemeral key (invisible to SW)

bull Using SW provided key (tenantrsquos key)

bull No encryption ndash plaintext domain

bull Clearing a key (using TMErsquos key effectively)

bull Allows SW to specify crypto algorithms

bull Only AES-XTS-128 for initial server intercept

8

9

VM Protection amp Isolation With MKTME

bull Protectionbull Use keyID to encrypt VM memory at runtime

bull Isolationbull Use different keyIDs for different VMs

bull Software Enablingbull For CPU access SW sets keyID at PTEsbull IA page table (host)bull EPT (KVM)

bull For Device access (DMA)bull w IOMMU Set keyIDto IOMMU page tablebull Physical DMA Apply keyIDto PA directly

10

Highlights of MKTME

Guests continue to run ldquowithout modificationsrdquo in MKTME domainsbull Encrypted with 1) CPU-generated ephemeral key or 2) the one provided by API (ldquotenant-

controlled keysrdquo)

bull Virtio including optimization (direct access to guest memory by kernel) continues to work

bull Direct IO (including accelerators FPGA) assignment (including SR-IOV VFs) is available

bull Live migration can be supported (among platforms that support MKTME)

bull vNVDIMM can be supported w limitation (because of physical address ldquotweakrdquo)

bull Host DIMM configuration cannot be changed cross reboots

bull Qemu DIMM amp vNVDIMM configuration cannot be changed cross VM reboots

11

Agenda

bull MKTME Introductionbull MKTME Use Casesbull MKTME Enabling

12

MKTME Enabled Use Cases

1 Launch Tenant VMs with in-use protection (CPU generated keys)bull Let CSP handle the keys

bull VM image provided by CSP

2 Launch Tenant VMs with at-rest and in-use protection with full tenant-controlbull VM image encrypted rest with tenant-specific keys

bull Keys in tenant controlbull VM memory isolation with tenant-specific keys

bull Trustiness verified hostbull Additional integrity verification of VM image

Use-case Extension KeyID Sharing for all VMs launched by single tenant with the same tenant-key (or CPU generated key)

13

Cloud

Agent

HARDWARE

Intelreg TXT TPM

Intelreg MKTME

HYPERVISOR (QemuKVM)

VM

Policy

Agent

MKTME ephemeral key

DRAM

Keyid 3

Keyid 3

Keyid 0

Keyid 0

Keyid 3

Cloud

Manager

1

2

3

Image

Repo

VM Launch w CPU Generated Keys

TME key

VM Launch

Policy Enforcement

VM Launchw KeyID

CSP Controlled

VM Launch wbull CPU generated keybull CSP provided VM image

Security Propertiesbull w VM runtime protectionbull w or wo at-rest and in-transit protectionbull No Host Trustiness Verification

14

Cloud Agent

Attestation Service

HYPERVISOR (QemuKVM)

VM

DRAM

KeyID 3

KeyID 3

KeyID 1

KeyID 1

KeyID 3

Cloud Manager

2

3

4

ImageRepo

VM Launch w Tenant Controlled Keys

1TrustinessVerification

VM Launch

Policy Enforcement

Key Server

Upload VM image encryptedw tenant key

0

Request Tenant-keyReturn wrapped key

Wrapped tenant key

Tenant key

MKTME ephemeral key

VM Launchw KeyID

VM Launch wbull Tenant provided keybull Tenant provided encrypted VM imagebull Tenant controlled key serverbull Trustiness verified hostbull VM image integrity verifiedbull Use TPM to wrapunwrap tenant-key

CSP ControlledTenant Controlled

Security Propertiesbull w VM runtime protectionbull w VM at-rest protectionbull w or wo in-transit protectionbull w Host trustiness verificationbull w VM image integrity verification

PolicyAgent

TME key

HARDWARE Intelreg TXT TPM

Intelreg MKTME

15

KeyID Sharing Among VMs

Compute Node

Cloud SW

mKey API KVM

MktmePolicy

tenant_id ltUUIDgtkey_type ldquoephemeralrdquo | ldquopersistentrdquo

key_server httpsallow_to_share ldquoyesrdquo | ldquonordquo

Qemu

KeyID P olicy KeyID V M s

Policy1 lttenant1 ldquoephem eralrdquogt keyID 1 VM 1 VM 2

Policy2 lttenant2 ldquopersistentrdquo xxxxxxgt keyID 2 VM 3

N ew VM Launch

w M ktm ePolicy

Exam ple KeyID sharing is based on KeyID Policy lttenant_id key_type tenant_keygt

Cloud SW bull M aintains lsquoKeyID Policy-to-KeyID rsquo tablebull M akes keyID sharing decision according to the tablebull U pdates the table on VM launch and teardow n

m Key API M KTM E key m anagem ent API

Apply keyID to

VM memory

Launch VM

w keyID

Cloud SW makes decision whether to share or not

Launch VM

4

Background Trusted VM in Cloud

VM protection by using encryptionbull VM encrypted lsquoat-restrsquo lsquoin-transitrsquo and lsquoruntimersquobull There has been existing technologies for lsquoat-restrsquo

and lsquoin-transitrsquo encryptionbull Qemu TLS support for live migrationbull Qemu encrypted image support

bull VM runtime encryption requires hardware memory encryption support

bull AMDreg SMESEVbull Intelreg MKTME

Launch VM on lsquoTrustiness Verifiedrsquo Hostbull Trusted hardwarelocation etcbull Trusted Cloud SW stack

Cloud Orchestrator

Cloud Agent

VM Image Repo

Compute NodeIntelreg Arch

VM Launch

Cloud Agent

Compute NodeIntelreg Arch

VM LaunchLive Migration

❶ At-rest

❷ Runtime❸ In-transit

Typical VM Lifecycle in Cloud

5

Trusted VM w OpenCIT -- OpenStack as ExampleEnterprise Data Center Cloud Service Provider

Key Broker

OpenStack Barbican

Trust Director

OpenStack Nova

OpenStack Glance

OpenStack Horizon

AppOS

AppOS

AppOS

TPM Intel Arch w TXT

Compute Node

R e p o r t i n g

P o l i c y E n f o r c e m e n t

V e r i f i e r

Trust Agent

In t e l O p e n S t a c k

Attestation Server

KB Proxy

M e a s u r e m e n t

Attestation hub

tbootxm

Intelreg Open CIT helps on Host trustiness verification

6

TME amp MKTME Introduction

bull New AES-XTS engine in data path to external memory busbull Data encrypteddecrypted on-the-fly when enteringleaving

memory

bull AES-XTS uses physical address as ldquotweakrdquobull Same plaintext different physical address -gt different ciphertext

bull TME (Total Memory Encryption)bull Full memory encryption by TME key (CPU generated)

bull EnabledDisabled by BIOS

bull Transparent to OS amp user apps

bull MKTME (Multi-key Total Memory Encryption)bull Memory encryption by using multiple keys

bull Use upper bits of physical address as keyID (see next)

7

MKTME KeyIDs

bull Repurpose upper bits of physical address as KeyID as shown belowbull Reduces useable physical address bits

bull Creates ldquoaliasesrdquo of physical memory locations different keyIDs can refer to the same page

bull Cache-coherency is not guaranteed for the same page that accessed by different keyIDsbull CPU caches are unaware of keyID(still treat keyIDas part of PA)

bull Architecturally upto 2^15-1 keyIDs (15 keyID bits)bull Reported by MSR Configured by BIOSbull KeyID 0 is reserved as TMErsquos key (not useable by MKTME)

bull New PCONFIG instruction to program keyID w associated key (see next)

Platform Addressable MemoryKeyID

MAXPHYADDR - 1

0

Reserved63

MK_TME_MAX_KEYID_BITS

8

MKTME KeyID Programming Overview

New Ring-0 instruction PCONFIG to program the KEYID and associated keybull Package scoped

bull Supports programming keyID to 4 modes

bull Using CPU generated random ephemeral key (invisible to SW)

bull Using SW provided key (tenantrsquos key)

bull No encryption ndash plaintext domain

bull Clearing a key (using TMErsquos key effectively)

bull Allows SW to specify crypto algorithms

bull Only AES-XTS-128 for initial server intercept

8

9

VM Protection amp Isolation With MKTME

bull Protectionbull Use keyID to encrypt VM memory at runtime

bull Isolationbull Use different keyIDs for different VMs

bull Software Enablingbull For CPU access SW sets keyID at PTEsbull IA page table (host)bull EPT (KVM)

bull For Device access (DMA)bull w IOMMU Set keyIDto IOMMU page tablebull Physical DMA Apply keyIDto PA directly

10

Highlights of MKTME

Guests continue to run ldquowithout modificationsrdquo in MKTME domainsbull Encrypted with 1) CPU-generated ephemeral key or 2) the one provided by API (ldquotenant-

controlled keysrdquo)

bull Virtio including optimization (direct access to guest memory by kernel) continues to work

bull Direct IO (including accelerators FPGA) assignment (including SR-IOV VFs) is available

bull Live migration can be supported (among platforms that support MKTME)

bull vNVDIMM can be supported w limitation (because of physical address ldquotweakrdquo)

bull Host DIMM configuration cannot be changed cross reboots

bull Qemu DIMM amp vNVDIMM configuration cannot be changed cross VM reboots

11

Agenda

bull MKTME Introductionbull MKTME Use Casesbull MKTME Enabling

12

MKTME Enabled Use Cases

1 Launch Tenant VMs with in-use protection (CPU generated keys)bull Let CSP handle the keys

bull VM image provided by CSP

2 Launch Tenant VMs with at-rest and in-use protection with full tenant-controlbull VM image encrypted rest with tenant-specific keys

bull Keys in tenant controlbull VM memory isolation with tenant-specific keys

bull Trustiness verified hostbull Additional integrity verification of VM image

Use-case Extension KeyID Sharing for all VMs launched by single tenant with the same tenant-key (or CPU generated key)

13

Cloud

Agent

HARDWARE

Intelreg TXT TPM

Intelreg MKTME

HYPERVISOR (QemuKVM)

VM

Policy

Agent

MKTME ephemeral key

DRAM

Keyid 3

Keyid 3

Keyid 0

Keyid 0

Keyid 3

Cloud

Manager

1

2

3

Image

Repo

VM Launch w CPU Generated Keys

TME key

VM Launch

Policy Enforcement

VM Launchw KeyID

CSP Controlled

VM Launch wbull CPU generated keybull CSP provided VM image

Security Propertiesbull w VM runtime protectionbull w or wo at-rest and in-transit protectionbull No Host Trustiness Verification

14

Cloud Agent

Attestation Service

HYPERVISOR (QemuKVM)

VM

DRAM

KeyID 3

KeyID 3

KeyID 1

KeyID 1

KeyID 3

Cloud Manager

2

3

4

ImageRepo

VM Launch w Tenant Controlled Keys

1TrustinessVerification

VM Launch

Policy Enforcement

Key Server

Upload VM image encryptedw tenant key

0

Request Tenant-keyReturn wrapped key

Wrapped tenant key

Tenant key

MKTME ephemeral key

VM Launchw KeyID

VM Launch wbull Tenant provided keybull Tenant provided encrypted VM imagebull Tenant controlled key serverbull Trustiness verified hostbull VM image integrity verifiedbull Use TPM to wrapunwrap tenant-key

CSP ControlledTenant Controlled

Security Propertiesbull w VM runtime protectionbull w VM at-rest protectionbull w or wo in-transit protectionbull w Host trustiness verificationbull w VM image integrity verification

PolicyAgent

TME key

HARDWARE Intelreg TXT TPM

Intelreg MKTME

15

KeyID Sharing Among VMs

Compute Node

Cloud SW

mKey API KVM

MktmePolicy

tenant_id ltUUIDgtkey_type ldquoephemeralrdquo | ldquopersistentrdquo

key_server httpsallow_to_share ldquoyesrdquo | ldquonordquo

Qemu

KeyID P olicy KeyID V M s

Policy1 lttenant1 ldquoephem eralrdquogt keyID 1 VM 1 VM 2

Policy2 lttenant2 ldquopersistentrdquo xxxxxxgt keyID 2 VM 3

N ew VM Launch

w M ktm ePolicy

Exam ple KeyID sharing is based on KeyID Policy lttenant_id key_type tenant_keygt

Cloud SW bull M aintains lsquoKeyID Policy-to-KeyID rsquo tablebull M akes keyID sharing decision according to the tablebull U pdates the table on VM launch and teardow n

m Key API M KTM E key m anagem ent API

Apply keyID to

VM memory

Launch VM

w keyID

Cloud SW makes decision whether to share or not

Launch VM

5

Trusted VM w OpenCIT -- OpenStack as ExampleEnterprise Data Center Cloud Service Provider

Key Broker

OpenStack Barbican

Trust Director

OpenStack Nova

OpenStack Glance

OpenStack Horizon

AppOS

AppOS

AppOS

TPM Intel Arch w TXT

Compute Node

R e p o r t i n g

P o l i c y E n f o r c e m e n t

V e r i f i e r

Trust Agent

In t e l O p e n S t a c k

Attestation Server

KB Proxy

M e a s u r e m e n t

Attestation hub

tbootxm

Intelreg Open CIT helps on Host trustiness verification

6

TME amp MKTME Introduction

bull New AES-XTS engine in data path to external memory busbull Data encrypteddecrypted on-the-fly when enteringleaving

memory

bull AES-XTS uses physical address as ldquotweakrdquobull Same plaintext different physical address -gt different ciphertext

bull TME (Total Memory Encryption)bull Full memory encryption by TME key (CPU generated)

bull EnabledDisabled by BIOS

bull Transparent to OS amp user apps

bull MKTME (Multi-key Total Memory Encryption)bull Memory encryption by using multiple keys

bull Use upper bits of physical address as keyID (see next)

7

MKTME KeyIDs

bull Repurpose upper bits of physical address as KeyID as shown belowbull Reduces useable physical address bits

bull Creates ldquoaliasesrdquo of physical memory locations different keyIDs can refer to the same page

bull Cache-coherency is not guaranteed for the same page that accessed by different keyIDsbull CPU caches are unaware of keyID(still treat keyIDas part of PA)

bull Architecturally upto 2^15-1 keyIDs (15 keyID bits)bull Reported by MSR Configured by BIOSbull KeyID 0 is reserved as TMErsquos key (not useable by MKTME)

bull New PCONFIG instruction to program keyID w associated key (see next)

Platform Addressable MemoryKeyID

MAXPHYADDR - 1

0

Reserved63

MK_TME_MAX_KEYID_BITS

8

MKTME KeyID Programming Overview

New Ring-0 instruction PCONFIG to program the KEYID and associated keybull Package scoped

bull Supports programming keyID to 4 modes

bull Using CPU generated random ephemeral key (invisible to SW)

bull Using SW provided key (tenantrsquos key)

bull No encryption ndash plaintext domain

bull Clearing a key (using TMErsquos key effectively)

bull Allows SW to specify crypto algorithms

bull Only AES-XTS-128 for initial server intercept

8

9

VM Protection amp Isolation With MKTME

bull Protectionbull Use keyID to encrypt VM memory at runtime

bull Isolationbull Use different keyIDs for different VMs

bull Software Enablingbull For CPU access SW sets keyID at PTEsbull IA page table (host)bull EPT (KVM)

bull For Device access (DMA)bull w IOMMU Set keyIDto IOMMU page tablebull Physical DMA Apply keyIDto PA directly

10

Highlights of MKTME

Guests continue to run ldquowithout modificationsrdquo in MKTME domainsbull Encrypted with 1) CPU-generated ephemeral key or 2) the one provided by API (ldquotenant-

controlled keysrdquo)

bull Virtio including optimization (direct access to guest memory by kernel) continues to work

bull Direct IO (including accelerators FPGA) assignment (including SR-IOV VFs) is available

bull Live migration can be supported (among platforms that support MKTME)

bull vNVDIMM can be supported w limitation (because of physical address ldquotweakrdquo)

bull Host DIMM configuration cannot be changed cross reboots

bull Qemu DIMM amp vNVDIMM configuration cannot be changed cross VM reboots

11

Agenda

bull MKTME Introductionbull MKTME Use Casesbull MKTME Enabling

12

MKTME Enabled Use Cases

1 Launch Tenant VMs with in-use protection (CPU generated keys)bull Let CSP handle the keys

bull VM image provided by CSP

2 Launch Tenant VMs with at-rest and in-use protection with full tenant-controlbull VM image encrypted rest with tenant-specific keys

bull Keys in tenant controlbull VM memory isolation with tenant-specific keys

bull Trustiness verified hostbull Additional integrity verification of VM image

Use-case Extension KeyID Sharing for all VMs launched by single tenant with the same tenant-key (or CPU generated key)

13

Cloud

Agent

HARDWARE

Intelreg TXT TPM

Intelreg MKTME

HYPERVISOR (QemuKVM)

VM

Policy

Agent

MKTME ephemeral key

DRAM

Keyid 3

Keyid 3

Keyid 0

Keyid 0

Keyid 3

Cloud

Manager

1

2

3

Image

Repo

VM Launch w CPU Generated Keys

TME key

VM Launch

Policy Enforcement

VM Launchw KeyID

CSP Controlled

VM Launch wbull CPU generated keybull CSP provided VM image

Security Propertiesbull w VM runtime protectionbull w or wo at-rest and in-transit protectionbull No Host Trustiness Verification

14

Cloud Agent

Attestation Service

HYPERVISOR (QemuKVM)

VM

DRAM

KeyID 3

KeyID 3

KeyID 1

KeyID 1

KeyID 3

Cloud Manager

2

3

4

ImageRepo

VM Launch w Tenant Controlled Keys

1TrustinessVerification

VM Launch

Policy Enforcement

Key Server

Upload VM image encryptedw tenant key

0

Request Tenant-keyReturn wrapped key

Wrapped tenant key

Tenant key

MKTME ephemeral key

VM Launchw KeyID

VM Launch wbull Tenant provided keybull Tenant provided encrypted VM imagebull Tenant controlled key serverbull Trustiness verified hostbull VM image integrity verifiedbull Use TPM to wrapunwrap tenant-key

CSP ControlledTenant Controlled

Security Propertiesbull w VM runtime protectionbull w VM at-rest protectionbull w or wo in-transit protectionbull w Host trustiness verificationbull w VM image integrity verification

PolicyAgent

TME key

HARDWARE Intelreg TXT TPM

Intelreg MKTME

15

KeyID Sharing Among VMs

Compute Node

Cloud SW

mKey API KVM

MktmePolicy

tenant_id ltUUIDgtkey_type ldquoephemeralrdquo | ldquopersistentrdquo

key_server httpsallow_to_share ldquoyesrdquo | ldquonordquo

Qemu

KeyID P olicy KeyID V M s

Policy1 lttenant1 ldquoephem eralrdquogt keyID 1 VM 1 VM 2

Policy2 lttenant2 ldquopersistentrdquo xxxxxxgt keyID 2 VM 3

N ew VM Launch

w M ktm ePolicy

Exam ple KeyID sharing is based on KeyID Policy lttenant_id key_type tenant_keygt

Cloud SW bull M aintains lsquoKeyID Policy-to-KeyID rsquo tablebull M akes keyID sharing decision according to the tablebull U pdates the table on VM launch and teardow n

m Key API M KTM E key m anagem ent API

Apply keyID to

VM memory

Launch VM

w keyID

Cloud SW makes decision whether to share or not

Launch VM

6

TME amp MKTME Introduction

bull New AES-XTS engine in data path to external memory busbull Data encrypteddecrypted on-the-fly when enteringleaving

memory

bull AES-XTS uses physical address as ldquotweakrdquobull Same plaintext different physical address -gt different ciphertext

bull TME (Total Memory Encryption)bull Full memory encryption by TME key (CPU generated)

bull EnabledDisabled by BIOS

bull Transparent to OS amp user apps

bull MKTME (Multi-key Total Memory Encryption)bull Memory encryption by using multiple keys

bull Use upper bits of physical address as keyID (see next)

7

MKTME KeyIDs

bull Repurpose upper bits of physical address as KeyID as shown belowbull Reduces useable physical address bits

bull Creates ldquoaliasesrdquo of physical memory locations different keyIDs can refer to the same page

bull Cache-coherency is not guaranteed for the same page that accessed by different keyIDsbull CPU caches are unaware of keyID(still treat keyIDas part of PA)

bull Architecturally upto 2^15-1 keyIDs (15 keyID bits)bull Reported by MSR Configured by BIOSbull KeyID 0 is reserved as TMErsquos key (not useable by MKTME)

bull New PCONFIG instruction to program keyID w associated key (see next)

Platform Addressable MemoryKeyID

MAXPHYADDR - 1

0

Reserved63

MK_TME_MAX_KEYID_BITS

8

MKTME KeyID Programming Overview

New Ring-0 instruction PCONFIG to program the KEYID and associated keybull Package scoped

bull Supports programming keyID to 4 modes

bull Using CPU generated random ephemeral key (invisible to SW)

bull Using SW provided key (tenantrsquos key)

bull No encryption ndash plaintext domain

bull Clearing a key (using TMErsquos key effectively)

bull Allows SW to specify crypto algorithms

bull Only AES-XTS-128 for initial server intercept

8

9

VM Protection amp Isolation With MKTME

bull Protectionbull Use keyID to encrypt VM memory at runtime

bull Isolationbull Use different keyIDs for different VMs

bull Software Enablingbull For CPU access SW sets keyID at PTEsbull IA page table (host)bull EPT (KVM)

bull For Device access (DMA)bull w IOMMU Set keyIDto IOMMU page tablebull Physical DMA Apply keyIDto PA directly

10

Highlights of MKTME

Guests continue to run ldquowithout modificationsrdquo in MKTME domainsbull Encrypted with 1) CPU-generated ephemeral key or 2) the one provided by API (ldquotenant-

controlled keysrdquo)

bull Virtio including optimization (direct access to guest memory by kernel) continues to work

bull Direct IO (including accelerators FPGA) assignment (including SR-IOV VFs) is available

bull Live migration can be supported (among platforms that support MKTME)

bull vNVDIMM can be supported w limitation (because of physical address ldquotweakrdquo)

bull Host DIMM configuration cannot be changed cross reboots

bull Qemu DIMM amp vNVDIMM configuration cannot be changed cross VM reboots

11

Agenda

bull MKTME Introductionbull MKTME Use Casesbull MKTME Enabling

12

MKTME Enabled Use Cases

1 Launch Tenant VMs with in-use protection (CPU generated keys)bull Let CSP handle the keys

bull VM image provided by CSP

2 Launch Tenant VMs with at-rest and in-use protection with full tenant-controlbull VM image encrypted rest with tenant-specific keys

bull Keys in tenant controlbull VM memory isolation with tenant-specific keys

bull Trustiness verified hostbull Additional integrity verification of VM image

Use-case Extension KeyID Sharing for all VMs launched by single tenant with the same tenant-key (or CPU generated key)

13

Cloud

Agent

HARDWARE

Intelreg TXT TPM

Intelreg MKTME

HYPERVISOR (QemuKVM)

VM

Policy

Agent

MKTME ephemeral key

DRAM

Keyid 3

Keyid 3

Keyid 0

Keyid 0

Keyid 3

Cloud

Manager

1

2

3

Image

Repo

VM Launch w CPU Generated Keys

TME key

VM Launch

Policy Enforcement

VM Launchw KeyID

CSP Controlled

VM Launch wbull CPU generated keybull CSP provided VM image

Security Propertiesbull w VM runtime protectionbull w or wo at-rest and in-transit protectionbull No Host Trustiness Verification

14

Cloud Agent

Attestation Service

HYPERVISOR (QemuKVM)

VM

DRAM

KeyID 3

KeyID 3

KeyID 1

KeyID 1

KeyID 3

Cloud Manager

2

3

4

ImageRepo

VM Launch w Tenant Controlled Keys

1TrustinessVerification

VM Launch

Policy Enforcement

Key Server

Upload VM image encryptedw tenant key

0

Request Tenant-keyReturn wrapped key

Wrapped tenant key

Tenant key

MKTME ephemeral key

VM Launchw KeyID

VM Launch wbull Tenant provided keybull Tenant provided encrypted VM imagebull Tenant controlled key serverbull Trustiness verified hostbull VM image integrity verifiedbull Use TPM to wrapunwrap tenant-key

CSP ControlledTenant Controlled

Security Propertiesbull w VM runtime protectionbull w VM at-rest protectionbull w or wo in-transit protectionbull w Host trustiness verificationbull w VM image integrity verification

PolicyAgent

TME key

HARDWARE Intelreg TXT TPM

Intelreg MKTME

15

KeyID Sharing Among VMs

Compute Node

Cloud SW

mKey API KVM

MktmePolicy

tenant_id ltUUIDgtkey_type ldquoephemeralrdquo | ldquopersistentrdquo

key_server httpsallow_to_share ldquoyesrdquo | ldquonordquo

Qemu

KeyID P olicy KeyID V M s

Policy1 lttenant1 ldquoephem eralrdquogt keyID 1 VM 1 VM 2

Policy2 lttenant2 ldquopersistentrdquo xxxxxxgt keyID 2 VM 3

N ew VM Launch

w M ktm ePolicy

Exam ple KeyID sharing is based on KeyID Policy lttenant_id key_type tenant_keygt

Cloud SW bull M aintains lsquoKeyID Policy-to-KeyID rsquo tablebull M akes keyID sharing decision according to the tablebull U pdates the table on VM launch and teardow n

m Key API M KTM E key m anagem ent API

Apply keyID to

VM memory

Launch VM

w keyID

Cloud SW makes decision whether to share or not

Launch VM

7

MKTME KeyIDs

bull Repurpose upper bits of physical address as KeyID as shown belowbull Reduces useable physical address bits

bull Creates ldquoaliasesrdquo of physical memory locations different keyIDs can refer to the same page

bull Cache-coherency is not guaranteed for the same page that accessed by different keyIDsbull CPU caches are unaware of keyID(still treat keyIDas part of PA)

bull Architecturally upto 2^15-1 keyIDs (15 keyID bits)bull Reported by MSR Configured by BIOSbull KeyID 0 is reserved as TMErsquos key (not useable by MKTME)

bull New PCONFIG instruction to program keyID w associated key (see next)

Platform Addressable MemoryKeyID

MAXPHYADDR - 1

0

Reserved63

MK_TME_MAX_KEYID_BITS

8

MKTME KeyID Programming Overview

New Ring-0 instruction PCONFIG to program the KEYID and associated keybull Package scoped

bull Supports programming keyID to 4 modes

bull Using CPU generated random ephemeral key (invisible to SW)

bull Using SW provided key (tenantrsquos key)

bull No encryption ndash plaintext domain

bull Clearing a key (using TMErsquos key effectively)

bull Allows SW to specify crypto algorithms

bull Only AES-XTS-128 for initial server intercept

8

9

VM Protection amp Isolation With MKTME

bull Protectionbull Use keyID to encrypt VM memory at runtime

bull Isolationbull Use different keyIDs for different VMs

bull Software Enablingbull For CPU access SW sets keyID at PTEsbull IA page table (host)bull EPT (KVM)

bull For Device access (DMA)bull w IOMMU Set keyIDto IOMMU page tablebull Physical DMA Apply keyIDto PA directly

10

Highlights of MKTME

Guests continue to run ldquowithout modificationsrdquo in MKTME domainsbull Encrypted with 1) CPU-generated ephemeral key or 2) the one provided by API (ldquotenant-

controlled keysrdquo)

bull Virtio including optimization (direct access to guest memory by kernel) continues to work

bull Direct IO (including accelerators FPGA) assignment (including SR-IOV VFs) is available

bull Live migration can be supported (among platforms that support MKTME)

bull vNVDIMM can be supported w limitation (because of physical address ldquotweakrdquo)

bull Host DIMM configuration cannot be changed cross reboots

bull Qemu DIMM amp vNVDIMM configuration cannot be changed cross VM reboots

11

Agenda

bull MKTME Introductionbull MKTME Use Casesbull MKTME Enabling

12

MKTME Enabled Use Cases

1 Launch Tenant VMs with in-use protection (CPU generated keys)bull Let CSP handle the keys

bull VM image provided by CSP

2 Launch Tenant VMs with at-rest and in-use protection with full tenant-controlbull VM image encrypted rest with tenant-specific keys

bull Keys in tenant controlbull VM memory isolation with tenant-specific keys

bull Trustiness verified hostbull Additional integrity verification of VM image

Use-case Extension KeyID Sharing for all VMs launched by single tenant with the same tenant-key (or CPU generated key)

13

Cloud

Agent

HARDWARE

Intelreg TXT TPM

Intelreg MKTME

HYPERVISOR (QemuKVM)

VM

Policy

Agent

MKTME ephemeral key

DRAM

Keyid 3

Keyid 3

Keyid 0

Keyid 0

Keyid 3

Cloud

Manager

1

2

3

Image

Repo

VM Launch w CPU Generated Keys

TME key

VM Launch

Policy Enforcement

VM Launchw KeyID

CSP Controlled

VM Launch wbull CPU generated keybull CSP provided VM image

Security Propertiesbull w VM runtime protectionbull w or wo at-rest and in-transit protectionbull No Host Trustiness Verification

14

Cloud Agent

Attestation Service

HYPERVISOR (QemuKVM)

VM

DRAM

KeyID 3

KeyID 3

KeyID 1

KeyID 1

KeyID 3

Cloud Manager

2

3

4

ImageRepo

VM Launch w Tenant Controlled Keys

1TrustinessVerification

VM Launch

Policy Enforcement

Key Server

Upload VM image encryptedw tenant key

0

Request Tenant-keyReturn wrapped key

Wrapped tenant key

Tenant key

MKTME ephemeral key

VM Launchw KeyID

VM Launch wbull Tenant provided keybull Tenant provided encrypted VM imagebull Tenant controlled key serverbull Trustiness verified hostbull VM image integrity verifiedbull Use TPM to wrapunwrap tenant-key

CSP ControlledTenant Controlled

Security Propertiesbull w VM runtime protectionbull w VM at-rest protectionbull w or wo in-transit protectionbull w Host trustiness verificationbull w VM image integrity verification

PolicyAgent

TME key

HARDWARE Intelreg TXT TPM

Intelreg MKTME

15

KeyID Sharing Among VMs

Compute Node

Cloud SW

mKey API KVM

MktmePolicy

tenant_id ltUUIDgtkey_type ldquoephemeralrdquo | ldquopersistentrdquo

key_server httpsallow_to_share ldquoyesrdquo | ldquonordquo

Qemu

KeyID P olicy KeyID V M s

Policy1 lttenant1 ldquoephem eralrdquogt keyID 1 VM 1 VM 2

Policy2 lttenant2 ldquopersistentrdquo xxxxxxgt keyID 2 VM 3

N ew VM Launch

w M ktm ePolicy

Exam ple KeyID sharing is based on KeyID Policy lttenant_id key_type tenant_keygt

Cloud SW bull M aintains lsquoKeyID Policy-to-KeyID rsquo tablebull M akes keyID sharing decision according to the tablebull U pdates the table on VM launch and teardow n

m Key API M KTM E key m anagem ent API

Apply keyID to

VM memory

Launch VM

w keyID

Cloud SW makes decision whether to share or not

Launch VM

8

MKTME KeyID Programming Overview

New Ring-0 instruction PCONFIG to program the KEYID and associated keybull Package scoped

bull Supports programming keyID to 4 modes

bull Using CPU generated random ephemeral key (invisible to SW)

bull Using SW provided key (tenantrsquos key)

bull No encryption ndash plaintext domain

bull Clearing a key (using TMErsquos key effectively)

bull Allows SW to specify crypto algorithms

bull Only AES-XTS-128 for initial server intercept

8

9

VM Protection amp Isolation With MKTME

bull Protectionbull Use keyID to encrypt VM memory at runtime

bull Isolationbull Use different keyIDs for different VMs

bull Software Enablingbull For CPU access SW sets keyID at PTEsbull IA page table (host)bull EPT (KVM)

bull For Device access (DMA)bull w IOMMU Set keyIDto IOMMU page tablebull Physical DMA Apply keyIDto PA directly

10

Highlights of MKTME

Guests continue to run ldquowithout modificationsrdquo in MKTME domainsbull Encrypted with 1) CPU-generated ephemeral key or 2) the one provided by API (ldquotenant-

controlled keysrdquo)

bull Virtio including optimization (direct access to guest memory by kernel) continues to work

bull Direct IO (including accelerators FPGA) assignment (including SR-IOV VFs) is available

bull Live migration can be supported (among platforms that support MKTME)

bull vNVDIMM can be supported w limitation (because of physical address ldquotweakrdquo)

bull Host DIMM configuration cannot be changed cross reboots

bull Qemu DIMM amp vNVDIMM configuration cannot be changed cross VM reboots

11

Agenda

bull MKTME Introductionbull MKTME Use Casesbull MKTME Enabling

12

MKTME Enabled Use Cases

1 Launch Tenant VMs with in-use protection (CPU generated keys)bull Let CSP handle the keys

bull VM image provided by CSP

2 Launch Tenant VMs with at-rest and in-use protection with full tenant-controlbull VM image encrypted rest with tenant-specific keys

bull Keys in tenant controlbull VM memory isolation with tenant-specific keys

bull Trustiness verified hostbull Additional integrity verification of VM image

Use-case Extension KeyID Sharing for all VMs launched by single tenant with the same tenant-key (or CPU generated key)

13

Cloud

Agent

HARDWARE

Intelreg TXT TPM

Intelreg MKTME

HYPERVISOR (QemuKVM)

VM

Policy

Agent

MKTME ephemeral key

DRAM

Keyid 3

Keyid 3

Keyid 0

Keyid 0

Keyid 3

Cloud

Manager

1

2

3

Image

Repo

VM Launch w CPU Generated Keys

TME key

VM Launch

Policy Enforcement

VM Launchw KeyID

CSP Controlled

VM Launch wbull CPU generated keybull CSP provided VM image

Security Propertiesbull w VM runtime protectionbull w or wo at-rest and in-transit protectionbull No Host Trustiness Verification

14

Cloud Agent

Attestation Service

HYPERVISOR (QemuKVM)

VM

DRAM

KeyID 3

KeyID 3

KeyID 1

KeyID 1

KeyID 3

Cloud Manager

2

3

4

ImageRepo

VM Launch w Tenant Controlled Keys

1TrustinessVerification

VM Launch

Policy Enforcement

Key Server

Upload VM image encryptedw tenant key

0

Request Tenant-keyReturn wrapped key

Wrapped tenant key

Tenant key

MKTME ephemeral key

VM Launchw KeyID

VM Launch wbull Tenant provided keybull Tenant provided encrypted VM imagebull Tenant controlled key serverbull Trustiness verified hostbull VM image integrity verifiedbull Use TPM to wrapunwrap tenant-key

CSP ControlledTenant Controlled

Security Propertiesbull w VM runtime protectionbull w VM at-rest protectionbull w or wo in-transit protectionbull w Host trustiness verificationbull w VM image integrity verification

PolicyAgent

TME key

HARDWARE Intelreg TXT TPM

Intelreg MKTME

15

KeyID Sharing Among VMs

Compute Node

Cloud SW

mKey API KVM

MktmePolicy

tenant_id ltUUIDgtkey_type ldquoephemeralrdquo | ldquopersistentrdquo

key_server httpsallow_to_share ldquoyesrdquo | ldquonordquo

Qemu

KeyID P olicy KeyID V M s

Policy1 lttenant1 ldquoephem eralrdquogt keyID 1 VM 1 VM 2

Policy2 lttenant2 ldquopersistentrdquo xxxxxxgt keyID 2 VM 3

N ew VM Launch

w M ktm ePolicy

Exam ple KeyID sharing is based on KeyID Policy lttenant_id key_type tenant_keygt

Cloud SW bull M aintains lsquoKeyID Policy-to-KeyID rsquo tablebull M akes keyID sharing decision according to the tablebull U pdates the table on VM launch and teardow n

m Key API M KTM E key m anagem ent API

Apply keyID to

VM memory

Launch VM

w keyID

Cloud SW makes decision whether to share or not

Launch VM

9

VM Protection amp Isolation With MKTME

bull Protectionbull Use keyID to encrypt VM memory at runtime

bull Isolationbull Use different keyIDs for different VMs

bull Software Enablingbull For CPU access SW sets keyID at PTEsbull IA page table (host)bull EPT (KVM)

bull For Device access (DMA)bull w IOMMU Set keyIDto IOMMU page tablebull Physical DMA Apply keyIDto PA directly

10

Highlights of MKTME

Guests continue to run ldquowithout modificationsrdquo in MKTME domainsbull Encrypted with 1) CPU-generated ephemeral key or 2) the one provided by API (ldquotenant-

controlled keysrdquo)

bull Virtio including optimization (direct access to guest memory by kernel) continues to work

bull Direct IO (including accelerators FPGA) assignment (including SR-IOV VFs) is available

bull Live migration can be supported (among platforms that support MKTME)

bull vNVDIMM can be supported w limitation (because of physical address ldquotweakrdquo)

bull Host DIMM configuration cannot be changed cross reboots

bull Qemu DIMM amp vNVDIMM configuration cannot be changed cross VM reboots

11

Agenda

bull MKTME Introductionbull MKTME Use Casesbull MKTME Enabling

12

MKTME Enabled Use Cases

1 Launch Tenant VMs with in-use protection (CPU generated keys)bull Let CSP handle the keys

bull VM image provided by CSP

2 Launch Tenant VMs with at-rest and in-use protection with full tenant-controlbull VM image encrypted rest with tenant-specific keys

bull Keys in tenant controlbull VM memory isolation with tenant-specific keys

bull Trustiness verified hostbull Additional integrity verification of VM image

Use-case Extension KeyID Sharing for all VMs launched by single tenant with the same tenant-key (or CPU generated key)

13

Cloud

Agent

HARDWARE

Intelreg TXT TPM

Intelreg MKTME

HYPERVISOR (QemuKVM)

VM

Policy

Agent

MKTME ephemeral key

DRAM

Keyid 3

Keyid 3

Keyid 0

Keyid 0

Keyid 3

Cloud

Manager

1

2

3

Image

Repo

VM Launch w CPU Generated Keys

TME key

VM Launch

Policy Enforcement

VM Launchw KeyID

CSP Controlled

VM Launch wbull CPU generated keybull CSP provided VM image

Security Propertiesbull w VM runtime protectionbull w or wo at-rest and in-transit protectionbull No Host Trustiness Verification

14

Cloud Agent

Attestation Service

HYPERVISOR (QemuKVM)

VM

DRAM

KeyID 3

KeyID 3

KeyID 1

KeyID 1

KeyID 3

Cloud Manager

2

3

4

ImageRepo

VM Launch w Tenant Controlled Keys

1TrustinessVerification

VM Launch

Policy Enforcement

Key Server

Upload VM image encryptedw tenant key

0

Request Tenant-keyReturn wrapped key

Wrapped tenant key

Tenant key

MKTME ephemeral key

VM Launchw KeyID

VM Launch wbull Tenant provided keybull Tenant provided encrypted VM imagebull Tenant controlled key serverbull Trustiness verified hostbull VM image integrity verifiedbull Use TPM to wrapunwrap tenant-key

CSP ControlledTenant Controlled

Security Propertiesbull w VM runtime protectionbull w VM at-rest protectionbull w or wo in-transit protectionbull w Host trustiness verificationbull w VM image integrity verification

PolicyAgent

TME key

HARDWARE Intelreg TXT TPM

Intelreg MKTME

15

KeyID Sharing Among VMs

Compute Node

Cloud SW

mKey API KVM

MktmePolicy

tenant_id ltUUIDgtkey_type ldquoephemeralrdquo | ldquopersistentrdquo

key_server httpsallow_to_share ldquoyesrdquo | ldquonordquo

Qemu

KeyID P olicy KeyID V M s

Policy1 lttenant1 ldquoephem eralrdquogt keyID 1 VM 1 VM 2

Policy2 lttenant2 ldquopersistentrdquo xxxxxxgt keyID 2 VM 3

N ew VM Launch

w M ktm ePolicy

Exam ple KeyID sharing is based on KeyID Policy lttenant_id key_type tenant_keygt

Cloud SW bull M aintains lsquoKeyID Policy-to-KeyID rsquo tablebull M akes keyID sharing decision according to the tablebull U pdates the table on VM launch and teardow n

m Key API M KTM E key m anagem ent API

Apply keyID to

VM memory

Launch VM

w keyID

Cloud SW makes decision whether to share or not

Launch VM

10

Highlights of MKTME

Guests continue to run ldquowithout modificationsrdquo in MKTME domainsbull Encrypted with 1) CPU-generated ephemeral key or 2) the one provided by API (ldquotenant-

controlled keysrdquo)

bull Virtio including optimization (direct access to guest memory by kernel) continues to work

bull Direct IO (including accelerators FPGA) assignment (including SR-IOV VFs) is available

bull Live migration can be supported (among platforms that support MKTME)

bull vNVDIMM can be supported w limitation (because of physical address ldquotweakrdquo)

bull Host DIMM configuration cannot be changed cross reboots

bull Qemu DIMM amp vNVDIMM configuration cannot be changed cross VM reboots

11

Agenda

bull MKTME Introductionbull MKTME Use Casesbull MKTME Enabling

12

MKTME Enabled Use Cases

1 Launch Tenant VMs with in-use protection (CPU generated keys)bull Let CSP handle the keys

bull VM image provided by CSP

2 Launch Tenant VMs with at-rest and in-use protection with full tenant-controlbull VM image encrypted rest with tenant-specific keys

bull Keys in tenant controlbull VM memory isolation with tenant-specific keys

bull Trustiness verified hostbull Additional integrity verification of VM image

Use-case Extension KeyID Sharing for all VMs launched by single tenant with the same tenant-key (or CPU generated key)

13

Cloud

Agent

HARDWARE

Intelreg TXT TPM

Intelreg MKTME

HYPERVISOR (QemuKVM)

VM

Policy

Agent

MKTME ephemeral key

DRAM

Keyid 3

Keyid 3

Keyid 0

Keyid 0

Keyid 3

Cloud

Manager

1

2

3

Image

Repo

VM Launch w CPU Generated Keys

TME key

VM Launch

Policy Enforcement

VM Launchw KeyID

CSP Controlled

VM Launch wbull CPU generated keybull CSP provided VM image

Security Propertiesbull w VM runtime protectionbull w or wo at-rest and in-transit protectionbull No Host Trustiness Verification

14

Cloud Agent

Attestation Service

HYPERVISOR (QemuKVM)

VM

DRAM

KeyID 3

KeyID 3

KeyID 1

KeyID 1

KeyID 3

Cloud Manager

2

3

4

ImageRepo

VM Launch w Tenant Controlled Keys

1TrustinessVerification

VM Launch

Policy Enforcement

Key Server

Upload VM image encryptedw tenant key

0

Request Tenant-keyReturn wrapped key

Wrapped tenant key

Tenant key

MKTME ephemeral key

VM Launchw KeyID

VM Launch wbull Tenant provided keybull Tenant provided encrypted VM imagebull Tenant controlled key serverbull Trustiness verified hostbull VM image integrity verifiedbull Use TPM to wrapunwrap tenant-key

CSP ControlledTenant Controlled

Security Propertiesbull w VM runtime protectionbull w VM at-rest protectionbull w or wo in-transit protectionbull w Host trustiness verificationbull w VM image integrity verification

PolicyAgent

TME key

HARDWARE Intelreg TXT TPM

Intelreg MKTME

15

KeyID Sharing Among VMs

Compute Node

Cloud SW

mKey API KVM

MktmePolicy

tenant_id ltUUIDgtkey_type ldquoephemeralrdquo | ldquopersistentrdquo

key_server httpsallow_to_share ldquoyesrdquo | ldquonordquo

Qemu

KeyID P olicy KeyID V M s

Policy1 lttenant1 ldquoephem eralrdquogt keyID 1 VM 1 VM 2

Policy2 lttenant2 ldquopersistentrdquo xxxxxxgt keyID 2 VM 3

N ew VM Launch

w M ktm ePolicy

Exam ple KeyID sharing is based on KeyID Policy lttenant_id key_type tenant_keygt

Cloud SW bull M aintains lsquoKeyID Policy-to-KeyID rsquo tablebull M akes keyID sharing decision according to the tablebull U pdates the table on VM launch and teardow n

m Key API M KTM E key m anagem ent API

Apply keyID to

VM memory

Launch VM

w keyID

Cloud SW makes decision whether to share or not

Launch VM

11

Agenda

bull MKTME Introductionbull MKTME Use Casesbull MKTME Enabling

12

MKTME Enabled Use Cases

1 Launch Tenant VMs with in-use protection (CPU generated keys)bull Let CSP handle the keys

bull VM image provided by CSP

2 Launch Tenant VMs with at-rest and in-use protection with full tenant-controlbull VM image encrypted rest with tenant-specific keys

bull Keys in tenant controlbull VM memory isolation with tenant-specific keys

bull Trustiness verified hostbull Additional integrity verification of VM image

Use-case Extension KeyID Sharing for all VMs launched by single tenant with the same tenant-key (or CPU generated key)

13

Cloud

Agent

HARDWARE

Intelreg TXT TPM

Intelreg MKTME

HYPERVISOR (QemuKVM)

VM

Policy

Agent

MKTME ephemeral key

DRAM

Keyid 3

Keyid 3

Keyid 0

Keyid 0

Keyid 3

Cloud

Manager

1

2

3

Image

Repo

VM Launch w CPU Generated Keys

TME key

VM Launch

Policy Enforcement

VM Launchw KeyID

CSP Controlled

VM Launch wbull CPU generated keybull CSP provided VM image

Security Propertiesbull w VM runtime protectionbull w or wo at-rest and in-transit protectionbull No Host Trustiness Verification

14

Cloud Agent

Attestation Service

HYPERVISOR (QemuKVM)

VM

DRAM

KeyID 3

KeyID 3

KeyID 1

KeyID 1

KeyID 3

Cloud Manager

2

3

4

ImageRepo

VM Launch w Tenant Controlled Keys

1TrustinessVerification

VM Launch

Policy Enforcement

Key Server

Upload VM image encryptedw tenant key

0

Request Tenant-keyReturn wrapped key

Wrapped tenant key

Tenant key

MKTME ephemeral key

VM Launchw KeyID

VM Launch wbull Tenant provided keybull Tenant provided encrypted VM imagebull Tenant controlled key serverbull Trustiness verified hostbull VM image integrity verifiedbull Use TPM to wrapunwrap tenant-key

CSP ControlledTenant Controlled

Security Propertiesbull w VM runtime protectionbull w VM at-rest protectionbull w or wo in-transit protectionbull w Host trustiness verificationbull w VM image integrity verification

PolicyAgent

TME key

HARDWARE Intelreg TXT TPM

Intelreg MKTME

15

KeyID Sharing Among VMs

Compute Node

Cloud SW

mKey API KVM

MktmePolicy

tenant_id ltUUIDgtkey_type ldquoephemeralrdquo | ldquopersistentrdquo

key_server httpsallow_to_share ldquoyesrdquo | ldquonordquo

Qemu

KeyID P olicy KeyID V M s

Policy1 lttenant1 ldquoephem eralrdquogt keyID 1 VM 1 VM 2

Policy2 lttenant2 ldquopersistentrdquo xxxxxxgt keyID 2 VM 3

N ew VM Launch

w M ktm ePolicy

Exam ple KeyID sharing is based on KeyID Policy lttenant_id key_type tenant_keygt

Cloud SW bull M aintains lsquoKeyID Policy-to-KeyID rsquo tablebull M akes keyID sharing decision according to the tablebull U pdates the table on VM launch and teardow n

m Key API M KTM E key m anagem ent API

Apply keyID to

VM memory

Launch VM

w keyID

Cloud SW makes decision whether to share or not

Launch VM

12

MKTME Enabled Use Cases

1 Launch Tenant VMs with in-use protection (CPU generated keys)bull Let CSP handle the keys

bull VM image provided by CSP

2 Launch Tenant VMs with at-rest and in-use protection with full tenant-controlbull VM image encrypted rest with tenant-specific keys

bull Keys in tenant controlbull VM memory isolation with tenant-specific keys

bull Trustiness verified hostbull Additional integrity verification of VM image

Use-case Extension KeyID Sharing for all VMs launched by single tenant with the same tenant-key (or CPU generated key)

13

Cloud

Agent

HARDWARE

Intelreg TXT TPM

Intelreg MKTME

HYPERVISOR (QemuKVM)

VM

Policy

Agent

MKTME ephemeral key

DRAM

Keyid 3

Keyid 3

Keyid 0

Keyid 0

Keyid 3

Cloud

Manager

1

2

3

Image

Repo

VM Launch w CPU Generated Keys

TME key

VM Launch

Policy Enforcement

VM Launchw KeyID

CSP Controlled

VM Launch wbull CPU generated keybull CSP provided VM image

Security Propertiesbull w VM runtime protectionbull w or wo at-rest and in-transit protectionbull No Host Trustiness Verification

14

Cloud Agent

Attestation Service

HYPERVISOR (QemuKVM)

VM

DRAM

KeyID 3

KeyID 3

KeyID 1

KeyID 1

KeyID 3

Cloud Manager

2

3

4

ImageRepo

VM Launch w Tenant Controlled Keys

1TrustinessVerification

VM Launch

Policy Enforcement

Key Server

Upload VM image encryptedw tenant key

0

Request Tenant-keyReturn wrapped key

Wrapped tenant key

Tenant key

MKTME ephemeral key

VM Launchw KeyID

VM Launch wbull Tenant provided keybull Tenant provided encrypted VM imagebull Tenant controlled key serverbull Trustiness verified hostbull VM image integrity verifiedbull Use TPM to wrapunwrap tenant-key

CSP ControlledTenant Controlled

Security Propertiesbull w VM runtime protectionbull w VM at-rest protectionbull w or wo in-transit protectionbull w Host trustiness verificationbull w VM image integrity verification

PolicyAgent

TME key

HARDWARE Intelreg TXT TPM

Intelreg MKTME

15

KeyID Sharing Among VMs

Compute Node

Cloud SW

mKey API KVM

MktmePolicy

tenant_id ltUUIDgtkey_type ldquoephemeralrdquo | ldquopersistentrdquo

key_server httpsallow_to_share ldquoyesrdquo | ldquonordquo

Qemu

KeyID P olicy KeyID V M s

Policy1 lttenant1 ldquoephem eralrdquogt keyID 1 VM 1 VM 2

Policy2 lttenant2 ldquopersistentrdquo xxxxxxgt keyID 2 VM 3

N ew VM Launch

w M ktm ePolicy

Exam ple KeyID sharing is based on KeyID Policy lttenant_id key_type tenant_keygt

Cloud SW bull M aintains lsquoKeyID Policy-to-KeyID rsquo tablebull M akes keyID sharing decision according to the tablebull U pdates the table on VM launch and teardow n

m Key API M KTM E key m anagem ent API

Apply keyID to

VM memory

Launch VM

w keyID

Cloud SW makes decision whether to share or not

Launch VM

13

Cloud

Agent

HARDWARE

Intelreg TXT TPM

Intelreg MKTME

HYPERVISOR (QemuKVM)

VM

Policy

Agent

MKTME ephemeral key

DRAM

Keyid 3

Keyid 3

Keyid 0

Keyid 0

Keyid 3

Cloud

Manager

1

2

3

Image

Repo

VM Launch w CPU Generated Keys

TME key

VM Launch

Policy Enforcement

VM Launchw KeyID

CSP Controlled

VM Launch wbull CPU generated keybull CSP provided VM image

Security Propertiesbull w VM runtime protectionbull w or wo at-rest and in-transit protectionbull No Host Trustiness Verification

14

Cloud Agent

Attestation Service

HYPERVISOR (QemuKVM)

VM

DRAM

KeyID 3

KeyID 3

KeyID 1

KeyID 1

KeyID 3

Cloud Manager

2

3

4

ImageRepo

VM Launch w Tenant Controlled Keys

1TrustinessVerification

VM Launch

Policy Enforcement

Key Server

Upload VM image encryptedw tenant key

0

Request Tenant-keyReturn wrapped key

Wrapped tenant key

Tenant key

MKTME ephemeral key

VM Launchw KeyID

VM Launch wbull Tenant provided keybull Tenant provided encrypted VM imagebull Tenant controlled key serverbull Trustiness verified hostbull VM image integrity verifiedbull Use TPM to wrapunwrap tenant-key

CSP ControlledTenant Controlled

Security Propertiesbull w VM runtime protectionbull w VM at-rest protectionbull w or wo in-transit protectionbull w Host trustiness verificationbull w VM image integrity verification

PolicyAgent

TME key

HARDWARE Intelreg TXT TPM

Intelreg MKTME

15

KeyID Sharing Among VMs

Compute Node

Cloud SW

mKey API KVM

MktmePolicy

tenant_id ltUUIDgtkey_type ldquoephemeralrdquo | ldquopersistentrdquo

key_server httpsallow_to_share ldquoyesrdquo | ldquonordquo

Qemu

KeyID P olicy KeyID V M s

Policy1 lttenant1 ldquoephem eralrdquogt keyID 1 VM 1 VM 2

Policy2 lttenant2 ldquopersistentrdquo xxxxxxgt keyID 2 VM 3

N ew VM Launch

w M ktm ePolicy

Exam ple KeyID sharing is based on KeyID Policy lttenant_id key_type tenant_keygt

Cloud SW bull M aintains lsquoKeyID Policy-to-KeyID rsquo tablebull M akes keyID sharing decision according to the tablebull U pdates the table on VM launch and teardow n

m Key API M KTM E key m anagem ent API

Apply keyID to

VM memory

Launch VM

w keyID

Cloud SW makes decision whether to share or not

Launch VM

14

Cloud Agent

Attestation Service

HYPERVISOR (QemuKVM)

VM

DRAM

KeyID 3

KeyID 3

KeyID 1

KeyID 1

KeyID 3

Cloud Manager

2

3

4

ImageRepo

VM Launch w Tenant Controlled Keys

1TrustinessVerification

VM Launch

Policy Enforcement

Key Server

Upload VM image encryptedw tenant key

0

Request Tenant-keyReturn wrapped key

Wrapped tenant key

Tenant key

MKTME ephemeral key

VM Launchw KeyID

VM Launch wbull Tenant provided keybull Tenant provided encrypted VM imagebull Tenant controlled key serverbull Trustiness verified hostbull VM image integrity verifiedbull Use TPM to wrapunwrap tenant-key

CSP ControlledTenant Controlled

Security Propertiesbull w VM runtime protectionbull w VM at-rest protectionbull w or wo in-transit protectionbull w Host trustiness verificationbull w VM image integrity verification

PolicyAgent

TME key

HARDWARE Intelreg TXT TPM

Intelreg MKTME

15

KeyID Sharing Among VMs

Compute Node

Cloud SW

mKey API KVM

MktmePolicy

tenant_id ltUUIDgtkey_type ldquoephemeralrdquo | ldquopersistentrdquo

key_server httpsallow_to_share ldquoyesrdquo | ldquonordquo

Qemu

KeyID P olicy KeyID V M s

Policy1 lttenant1 ldquoephem eralrdquogt keyID 1 VM 1 VM 2

Policy2 lttenant2 ldquopersistentrdquo xxxxxxgt keyID 2 VM 3

N ew VM Launch

w M ktm ePolicy

Exam ple KeyID sharing is based on KeyID Policy lttenant_id key_type tenant_keygt

Cloud SW bull M aintains lsquoKeyID Policy-to-KeyID rsquo tablebull M akes keyID sharing decision according to the tablebull U pdates the table on VM launch and teardow n

m Key API M KTM E key m anagem ent API

Apply keyID to

VM memory

Launch VM

w keyID

Cloud SW makes decision whether to share or not

Launch VM

15

KeyID Sharing Among VMs

Compute Node

Cloud SW

mKey API KVM

MktmePolicy

tenant_id ltUUIDgtkey_type ldquoephemeralrdquo | ldquopersistentrdquo

key_server httpsallow_to_share ldquoyesrdquo | ldquonordquo

Qemu

KeyID P olicy KeyID V M s

Policy1 lttenant1 ldquoephem eralrdquogt keyID 1 VM 1 VM 2

Policy2 lttenant2 ldquopersistentrdquo xxxxxxgt keyID 2 VM 3

N ew VM Launch

w M ktm ePolicy

Exam ple KeyID sharing is based on KeyID Policy lttenant_id key_type tenant_keygt

Cloud SW bull M aintains lsquoKeyID Policy-to-KeyID rsquo tablebull M akes keyID sharing decision according to the tablebull U pdates the table on VM launch and teardow n

m Key API M KTM E key m anagem ent API

Apply keyID to

VM memory

Launch VM

w keyID

Cloud SW makes decision whether to share or not

Launch VM

16

Agenda

bull MKTME Introductionbull MKTME Use Casesbull MKTME Enabling

17

MKTME EnablingHigh Level

bull Host kernelbull mkey APIs

bull KeyKeyID Managementbull Core-MM KeyID supportbull VFIOIOMMU KeyID supportbull DMA KeyID support

bull KVMbull KeyID setup in EPT

bull Qemu

bull Receive KeyID from Cloud SWbull Apply KeyID to guest memory

IA -PTKeyID sfor H ost

EPT

VT-d

KeyID sfor guest

KeyID sfor D M A

KVM

KeyKeyIDM anagem ent

M KTM E Engine

C ore-M M code w ith KeyID

Setting KeyID sin EPT

VMG uest M em ory

Q EM U

D evice(N IC SC SI etc)

m key APIs

G uest m em ory w ith KeyID

VFIO IO M M Uw ith KeyID

KeyID s

Encrypted M em ory w ith

KeyID

PCO N FIG

N ew C ode

Vhost-kernel

D irect IO V irtio Vhost Live M igration

D M A w ith KeyID

M M U N otifier

Cloud SW Launch VMw KeyID

KeyID sfor D M A

18

MKTME Enabling Current Status

bull Specification has been published [1]bull Core kernel enabling statusbull Some preliminary patches have been upstreamedbull Feature emulation (CPUID MSR) PCONFIG

bull Proposal of some components have been sent to community for feedbackbull Key management API Using kernel key retention service

bull Other components WIP internallybull Core-MM keyIDsupport IOMMU keyIDsupport DMA keyIDsupport hellip

bull KVMQemu enabling statusbull PoC has been done to prove MKTME actually works

bull Depending on core kernel parts ready for formal patches

[1] httpssoftwareintelcomsitesdefaultfilesmanageda516Multi-Key-Total-Memory-Encryption-Specpdf

THANKS

17

MKTME EnablingHigh Level

bull Host kernelbull mkey APIs

bull KeyKeyID Managementbull Core-MM KeyID supportbull VFIOIOMMU KeyID supportbull DMA KeyID support

bull KVMbull KeyID setup in EPT

bull Qemu

bull Receive KeyID from Cloud SWbull Apply KeyID to guest memory

IA -PTKeyID sfor H ost

EPT

VT-d

KeyID sfor guest

KeyID sfor D M A

KVM

KeyKeyIDM anagem ent

M KTM E Engine

C ore-M M code w ith KeyID

Setting KeyID sin EPT

VMG uest M em ory

Q EM U

D evice(N IC SC SI etc)

m key APIs

G uest m em ory w ith KeyID

VFIO IO M M Uw ith KeyID

KeyID s

Encrypted M em ory w ith

KeyID

PCO N FIG

N ew C ode

Vhost-kernel

D irect IO V irtio Vhost Live M igration

D M A w ith KeyID

M M U N otifier

Cloud SW Launch VMw KeyID

KeyID sfor D M A

18

MKTME Enabling Current Status

bull Specification has been published [1]bull Core kernel enabling statusbull Some preliminary patches have been upstreamedbull Feature emulation (CPUID MSR) PCONFIG

bull Proposal of some components have been sent to community for feedbackbull Key management API Using kernel key retention service

bull Other components WIP internallybull Core-MM keyIDsupport IOMMU keyIDsupport DMA keyIDsupport hellip

bull KVMQemu enabling statusbull PoC has been done to prove MKTME actually works

bull Depending on core kernel parts ready for formal patches

[1] httpssoftwareintelcomsitesdefaultfilesmanageda516Multi-Key-Total-Memory-Encryption-Specpdf

THANKS

18

MKTME Enabling Current Status

bull Specification has been published [1]bull Core kernel enabling statusbull Some preliminary patches have been upstreamedbull Feature emulation (CPUID MSR) PCONFIG

bull Proposal of some components have been sent to community for feedbackbull Key management API Using kernel key retention service

bull Other components WIP internallybull Core-MM keyIDsupport IOMMU keyIDsupport DMA keyIDsupport hellip

bull KVMQemu enabling statusbull PoC has been done to prove MKTME actually works

bull Depending on core kernel parts ready for formal patches

[1] httpssoftwareintelcomsitesdefaultfilesmanageda516Multi-Key-Total-Memory-Encryption-Specpdf

THANKS

THANKS