intel new rdt features and implementation...
TRANSCRIPT
Software & Services Group
Intel New RDT Features and
Implementation Introduction
Yi Sun
Jun. 10th, 2017
1
Software & Services Group
Agenda
• Shared Resource Contention
• Solution: Intel Resource Director Technology (RDT)
• Performance Improvement Proofs
• New Shared Resource Allocation Features
• Shared Resource Allocation Features Key Mechanisms
• Refactor Shared Resource Allocation Codes
2
Software & Services Group
Shared Resource Contention
3
• Resource contention
causes up to 4X
slowdown in
performance
• Need ability to
monitor and enforce
cache/memory
resource usage
Software & Services Group
Shared Resource Contention
It is the Root Cause of performance degradation:
• Physical CPU:
– Real time scheduler.
– CPU pinning/isolation for high priority VM.
• Memory:
– Cache aware memory allocation.
– Dedicated memory region for high priority VM.
• Shared Device:
– Dedicated device assignment.
• Platform Shared Resource (L3/L2 Cache, Memory Bandwidth)
– SW can do nothing!!!
4
Software & Services Group
Solution: Intel Resource Director Technology (RDT)
5
Software & Services Group
Performance Improvement Proofs
6
High-Priority thread=6MB
isolated, 3 Low-Priority
threads share 2MB
Software & Services Group
Core
app
Core
app
New Shared Resource Allocation Features
7
app app
L2Cache
DRAM
L2 CATManage L2 cache allocation, based on
• Class of Service(COS);
• QOS bitmask MSRs.
L2Cache
Last LevelCache
Software & Services Group
High-speed Interconnect
New Shared Resource Allocation Features
8
Core
app
Core
appapp app
L2 Cache
Memory Bandwidth Allocation (MBA) • With MBA, data transfers with lower priority can
be throttled through the programmable request
rate controller between the cores and the high-
speed interconnect;
• Manages memory bandwidth allocation based
on
• Class of Service(COS);
• QOS throttling MSR.
L2 Cache
Last LevelCache
Programmable Request Rate
Controller
Programmable Request Rate
Controller
MemoryController
Software & Services Group
Shared Resource Allocation Features Key Mechanisms
9
Socket0 (Physical Package)
Core-0
L2 Cache
Logical Processor Logical Processor
L3 Cache
Rsv
IA32_PQR_ASSOC
COS-ID Rsv RMID
IA32_L3_QOS_MASK_0~n
COS-0 COS-1 ... COS-n
IA32_L2_QOS_MASK_0~m IA32_L2_QoS_Ext_BW_Thrtl_0~z
COS-0 COS-1 ... COS-m COS-0 COS-1 ... COS-z
VM VM
Rsv
IA32_PQR_ASSOC
COS-ID Rsv RMID
...Core-n
L2 Cache
Logical Processor Logical Processor
Rsv
IA32_PQR_ASSOC
COS-ID Rsv RMID Rsv
IA32_PQR_ASSOC
COS-ID Rsv RMID
Software & Services Group
Shared Resource Allocation Codes
10
struct psr_cat_socket_info {unsigned int cbm_len;unsigned int cos_max;struct psr_cat_cbm *cos_to_cbm;spinlock_t cbm_lock;
};
int psr_get_cat_l3_info(unsigned int socket, uint32_t *cbm_len,uint32_t *cos_max, uint32_t *flags);
int psr_get_l3_cbm(struct domain *d, unsigned int socket,uint64_t *cbm, enum cbm_type type);
int psr_set_l3_cbm(struct domain *d, unsigned int socket,uint64_t cbm, enum cbm_type type);
struct psr_socket_info {bool feat_init;struct feat_node *features[PSR_SOCKET_FEAT_NUM];spinlock_t ref_lock;unsigned int cos_ref[MAX_COS_REG_CNT];DECLARE_BITMAP(dom_set, DOMID_IDLE + 1);
};
int psr_get_info(unsigned int socket, enum psr_val_type type,uint32_t data[], unsigned int array_len);
int psr_get_val(struct domain *d, unsigned int socket,uint32_t *val, enum psr_val_type type);
int psr_set_val(struct domain *d, unsigned int socket,uint64_t new_val, enum psr_val_type type);
Current codes: Codes after refactor:
Software & Services Group
Next Step
• Hypervisor codes refactoring is done in L2 CAT enabling patch set.
• Next, refactor tools layer.
11
Software & Services Group
Q&A
12
Software & Services Group
Terms
ASSOC: Association
CAT: Cache Allocation Technology
CBM: Capacity Bitmasks
CMT: Cache Monitoring Technology
COS/CLOS: Class of Service
MBA: Memory Bandwidth Allocation
MBM: Memory Bandwidth Monitoring
MSR: Model Specific Register
QOS: Quality of Service
RDT: Resource Director Technology
13
Software & Services Group
14