dhtm: durable hardware transactional memory · arpit joshi, vijay nagarajan, marcelo cintra,...
Post on 14-Sep-2020
1 Views
Preview:
TRANSCRIPT
Arpit Joshi, Vijay Nagarajan, Marcelo Cintra, Stratis Viglas
DHTM: Durable Hardware Transactional Memory
ISCA 2018
Persistent Memory is here…
!2
Persistent Memory is here…
!2
Persistent Memory SystemsL1
LLC
Persistent Memory
L1
!3
Persistent Memory SystemsL1
LLC
Persistent Memory
L1• Persistent Memory
- Non-volatility over the memory bus- Load/Store interface to persistent data
!3
Persistent Memory SystemsL1
LLC
Persistent Memory
L1• Persistent Memory
- Non-volatility over the memory bus- Load/Store interface to persistent data
!3
System Crashes
Persistent Memory SystemsL1
LLC
Persistent Memory
L1• Persistent Memory
- Non-volatility over the memory bus- Load/Store interface to persistent data
• Crash Consistency- Is the persistent state consistent?- Programming Model: ACID Transactions
!3
System Crashes
Persistent Memory SystemsL1
LLC
Persistent Memory
L1• Persistent Memory
- Non-volatility over the memory bus- Load/Store interface to persistent data
• Crash Consistency- Is the persistent state consistent?- Programming Model: ACID Transactions
!3
System Crashes
“Ensuring failure atomicity for all this computation without failure-atomic transactions is practically infeasible, if not impossible.”
Marathe et al. [HotStorage’17]
Persistent Memory SystemsL1
LLC
Persistent Memory
L1• Persistent Memory
- Non-volatility over the memory bus- Load/Store interface to persistent data
• Crash Consistency- Is the persistent state consistent?- Programming Model: ACID Transactions
!3
System Crashes
“Ensuring failure atomicity for all this computation without failure-atomic transactions is practically infeasible, if not impossible.”
Marathe et al. [HotStorage’17]
How fast can we support ACID?
ACID TransactionsL1
LLC
Persistent Memory
L1
!4
ACID TransactionsL1
LLC
Persistent Memory
L1
!4
Atomic Visibility
ACID TransactionsL1
LLC
Persistent Memory
L1
!4
Atomic Visibility
Atomic Durability
ACID TransactionsL1
LLC
Persistent Memory
L1
!4
Atomic Visibility
Atomic Durability
Locks HTMSTM
ACID TransactionsL1
LLC
Persistent Memory
L1
!4
Atomic Visibility
Atomic Durability
Locks HTMSTM
Check-pointing
H/W Logging
S/WLogging
ACID TransactionsL1
LLC
Persistent Memory
L1
!4
Atomic Visibility
Atomic Durability
Locks HTMSTM
Check-pointing
H/W Logging
S/WLogging
Atomic Visibility: HTM
!5
Atomic Visibility: HTM
• Commercial HTMs [Intel, IBM]
!5
L1 Cache
Cache Line
A = 15
R
B = 20
W
11
Atomic Visibility: HTM
• Commercial HTMs [Intel, IBM]- Version Management: read/write sets in
L1 cache
!5
L1 Cache
Cache Line
A = 15
R
B = 20
W
11
Atomic Visibility: HTM
• Commercial HTMs [Intel, IBM]- Version Management: read/write sets in
L1 cache- Conflict Detection: piggy back on the
coherence protocol
!5
L1 Cache
Cache Line
A = 15
R
B = 20
W
11
Atomic Visibility: HTM
• Commercial HTMs [Intel, IBM]- Version Management: read/write sets in
L1 cache- Conflict Detection: piggy back on the
coherence protocol- Commit: make updates non-speculative
!5
L1 Cache
Cache Line
A = 15
R
B = 20
W
Atomic Visibility: HTM
• Commercial HTMs [Intel, IBM]- Version Management: read/write sets in
L1 cache- Conflict Detection: piggy back on the
coherence protocol- Commit: make updates non-speculative- Abort: invalidate write set
!5
L1 Cache
Cache Line R
B = 20
W
Atomic Visibility: HTM
• Commercial HTMs [Intel, IBM]- Version Management: read/write sets in
L1 cache- Conflict Detection: piggy back on the
coherence protocol- Commit: make updates non-speculative- Abort: invalidate write set
!5
L1 Cache
Cache Line R
B = 20
W
Write-sets in commercial HTMs limited by the size of the L1 cache.
Atomic Durability: Logging
!6
Atomic Durability: Logging
• Logging for durability [Doshi’16, Joshi’17, Shin’17, Ogleari’18]
!6
Persistent Memory
In-place Values
A = 10B = 20C = 30
Atomic Durability: Logging
• Logging for durability [Doshi’16, Joshi’17, Shin’17, Ogleari’18]- Write a log entry for every update
!6
Persistent Memory
In-place Values
A = 10B = 20C = 30
Transaction Log
A = 15B = 25
Atomic Durability: Logging
• Logging for durability [Doshi’16, Joshi’17, Shin’17, Ogleari’18]- Write a log entry for every update- Commit: Update the values in-place
!6
Persistent Memory
In-place Values
A = 15B = 25C = 30
Transaction Log
Atomic Durability: Logging
• Logging for durability [Doshi’16, Joshi’17, Shin’17, Ogleari’18]- Write a log entry for every update- Commit: Update the values in-place- Abort: Undo any in-place updates
!6
Persistent Memory
In-place Values
A = 15B = 25C = 30
Transaction Log
A = 10B = 20
Atomic Durability: Logging
• Logging for durability [Doshi’16, Joshi’17, Shin’17, Ogleari’18]- Write a log entry for every update- Commit: Update the values in-place- Abort: Undo any in-place updates
!6
Persistent Memory
In-place Values
A = 15B = 25C = 30
Transaction Log
A = 10B = 20
In-place updates in the critical path of commit High memory write bandwidth requirement
ACID = HTM + Logging
Goals:- Support fast commits - Minimise memory bandwidth consumption- Extend the supported transaction size- Maintain the simplicity of commercial HTMs
!7
DHTM: Durable Hardware Transactional Memory
L1
LLC
Persistent Memory
L1
!8
Log Writes
Commercial HTM + Hardware Redo Log
DHTM: Durable Hardware Transactional Memory
L1
LLC
Persistent Memory
L1
!8
Log Writes
Commercial HTM + Hardware Redo Log- H/W Redo Log + Log Buffer
Reduced memory bandwidth Fast commits
DHTM: Durable Hardware Transactional Memory
L1
LLC
Persistent Memory
L1
!8
Log Writes
Commercial HTM + Hardware Redo Log- H/W Redo Log + Log Buffer
Reduced memory bandwidth Fast commits
- H/W Log + Sticky State Extended transaction size to the LLC Simplicity of commercial HTM
DHTM: Durable Hardware Transactional Memory
L1
LLC
Persistent Memory
L1
!8
Log Writes
!9
L1
LLC
Persistent Memory
L1
Log Writes
DHTM: Log Buffer
!9
L1
LLC
Persistent Memory
L1• Redo Log Bandwidth Problem
Log Writes
DHTM: Log Buffer
!9
L1
LLC
Persistent Memory
L1• Redo Log Bandwidth Problem
- write a log entry for every storeLog Writes
DHTM: Log Buffer
!9
L1
LLC
Persistent Memory
L1• Redo Log Bandwidth Problem
- write a log entry for every store- multiple stores create multiple log entriesLog Writes
DHTM: Log Buffer
!9
L1
LLC
Persistent Memory
L1• Redo Log Bandwidth Problem
- write a log entry for every store- multiple stores create multiple log entries
• Solution: Log Buffer
Log Writes
DHTM: Log Buffer
!9
L1
LLC
Persistent Memory
L1• Redo Log Bandwidth Problem
- write a log entry for every store- multiple stores create multiple log entries
• Solution: Log Buffer - track cache lines being modified
Log Writes
DHTM: Log Buffer
!9
L1
LLC
Persistent Memory
L1• Redo Log Bandwidth Problem
- write a log entry for every store- multiple stores create multiple log entries
• Solution: Log Buffer - track cache lines being modified- multiple writes coalesced in a log entry
Log Writes
DHTM: Log Buffer
!9
L1
LLC
Persistent Memory
L1• Redo Log Bandwidth Problem
- write a log entry for every store- multiple stores create multiple log entries
• Solution: Log Buffer - track cache lines being modified- multiple writes coalesced in a log entry - log entry written to persistent memory on eviction
from log buffer
Log Writes
DHTM: Log Buffer
DHTM: Transaction States
!10
DHTM: Transaction States
!10
Active
Begin Transaction
DHTM: Transaction States
!10
Active Commit
Begin Transaction
End Transaction&
Log RecordsPersisted
DHTM: Transaction States
!10
Active Commit CommitComplete
Begin Transaction
End Transaction&
Log RecordsPersisted
In-place DataPersisted
DHTM: Transaction States
!10
Active Commit CommitComplete
Abort
Begin Transaction
End Transaction&
Log RecordsPersisted
In-place DataPersisted
Conflict
DHTM: Commit ExampleL1 Cache
Cache Line R W
Persistent Memory
In-place Values
A = 15B = 25A = 10B = 20C = 30
Transaction Log
A = 10B = 20
State
Log Buffer Begin_Transaction
Write (A=15)
Read (B)
Write (B=25)
End_Transaction
!11
Active
DHTM: Commit ExampleL1 Cache
Cache Line R W
Persistent Memory
In-place Values
A = 15B = 25A = 10B = 20C = 30
Transaction Log
A = 10B = 20
State
Log Buffer Begin_Transaction
Write (A=15)
Read (B)
Write (B=25)
End_Transaction
!11
ActiveActive
DHTM: Commit ExampleL1 Cache
Cache Line R W
Persistent Memory
In-place Values
A = 15B = 25A = 10B = 20C = 30
Transaction Log
A = 10B = 20
State
Log Buffer Begin_Transaction
Write (A=15)
Read (B)
Write (B=25)
End_Transaction
!11
A = 15 1A
ActiveActive
DHTM: Commit ExampleL1 Cache
Cache Line R W
Persistent Memory
In-place Values
A = 15B = 25A = 10B = 20C = 30
Transaction Log
A = 10B = 20
State
Log Buffer Begin_Transaction
Write (A=15)
Read (B)
Write (B=25)
End_Transaction
!11
A = 15 1A
A = 15 1AB = 20 1
ActiveActive
DHTM: Commit ExampleL1 Cache
Cache Line R W
Persistent Memory
In-place Values
A = 15B = 25A = 10B = 20C = 30
Transaction Log
A = 10B = 20
State
Log Buffer Begin_Transaction
Write (A=15)
Read (B)
Write (B=25)
End_Transaction
!11
A = 15 1A
A = 15 1AB = 20B = 25 11 1
A = 15
B
ActiveActiveCommit
DHTM: Commit ExampleL1 Cache
Cache Line R W
Persistent Memory
In-place Values
A = 15B = 25A = 10B = 20C = 30
Transaction Log
A = 10B = 20
State
Log Buffer Begin_Transaction
Write (A=15)
Read (B)
Write (B=25)
End_Transaction
!11
A = 15 1A
A = 15 1AB = 20B = 25 11 1
A = 15
B
B = 25
B = 25
A = 15
Commit
1
ActiveActiveCommit
DHTM: Commit ExampleL1 Cache
Cache Line R W
Persistent Memory
In-place Values
A = 15B = 25A = 10B = 20C = 30
Transaction Log
A = 10B = 20
State
Log Buffer Begin_Transaction
Write (A=15)
Read (B)
Write (B=25)
End_Transaction
!11
A = 15 1A
A = 15 1AB = 20B = 25 11 1
A = 15
B
B = 25
B = 25
A = 15
CommitA = 15B = 25
Complete
CommitComplete
CommitB = 25
1
DHTM: Supporting Overflow
!12
DHTM: Supporting Overflow
• Problems with Overflow:
!12
DHTM: Supporting Overflow
• Problems with Overflow:- Version Management:
- global operation on write-set on a commit/abort- overhead infeasible in larger caches (beyond L1)
!12
DHTM: Supporting Overflow
• Problems with Overflow:- Version Management:
- global operation on write-set on a commit/abort- overhead infeasible in larger caches (beyond L1)
- Conflict Detection: - additional metadata to detect conflicts- increased complexity due to NACK based protocols
!12
DHTM: Supporting Overflow
!13
DHTM: Supporting Overflow
!13
• Solution
DHTM: Supporting Overflow
!13
LLC
Persistent Memory
• Solution- Version Management:
- Overflow List
DHTM: Supporting Overflow
!13
LLC
Persistent Memory
Overflow List
C
AB
• Solution- Version Management:
- Overflow List
DHTM: Supporting Overflow
!13
LLC
Persistent Memory
Overflow List
C
AB
• Solution- Version Management:
- Overflow List
DHTM: Supporting Overflow
!13
LLC
Persistent Memory
Overflow List
C
AB
• Solution- Version Management:
- Overflow List- Conflict Detection:
- maintain sticky state on overflow (similar to LogTM)
- avoid NACK by restricting overflow to LLC
DHTM: Supporting Overflow
!13
LLC
Persistent Memory
Overflow List
C
AB
• Solution- Version Management:
- Overflow List- Conflict Detection:
- maintain sticky state on overflow (similar to LogTM)
- avoid NACK by restricting overflow to LLC
Further details on supporting overflows are in the paper.
Evaluation
• System Configuration- We evaluate an 8-core machine with a 2-level cache hierarchy- HTM’s implement (first) writer wins conflict resolution policy
!14
Atomic Visibility Atomic Durability
ATOM Locks Hardware Undo Log
LogTM+ATOM HTM (LogTM) Hardware Undo Log
DHTM HTM Hardware Redo Log (Log Buffer)
Evaluation
!15
Evaluation
!15
1
1.25
1.5
1.75
2
queue hash sdg sps btree rbtree gmean
ATOM LogTM+ATOM DHTM
Evaluation
!15
1
1.25
1.5
1.75
2
queue hash sdg sps btree rbtree gmean
ATOM LogTM+ATOM DHTM
Evaluation
!15
1
1.25
1.5
1.75
2
queue hash sdg sps btree rbtree gmean
ATOM LogTM+ATOM DHTM
Evaluation
!15
1
1.25
1.5
1.75
2
queue hash sdg sps btree rbtree gmean
ATOM LogTM+ATOM DHTM
26%
Evaluation
!15
1
1.25
1.5
1.75
2
queue hash sdg sps btree rbtree gmean
ATOM LogTM+ATOM DHTM
17%
Conclusion
• Persistent memory systems require crash consistency• ACID Transactions: widely understood crash consistency mechanism
• DHTM: ACID transactions in hardware- Atomic Visibility: commercial HTM- Atomic Durability: bandwidth optimized hardware redo log- Leverage hardware logging to extend transaction size unto LLC
!16
Arpit Joshi, Vijay Nagarajan, Marcelo Cintra, Stratis Viglas
DHTM: Durable Hardware Transactional Memory
ISCA 2018
top related