tape storage and crc protection
TRANSCRIPT
![Page 1: Tape Storage and CRC Protection](https://reader038.vdocuments.us/reader038/viewer/2022102723/55c53200bb61ebbf328b45b2/html5/thumbnails/1.jpg)
Logical Block Protection Previous results Multithreading Benchmarks Conclusion
Tape Storage and CRC ProtectionSummer Student Project 2014
Karel Ha
IT-DSS-TABCERN
1 / 17
![Page 2: Tape Storage and CRC Protection](https://reader038.vdocuments.us/reader038/viewer/2022102723/55c53200bb61ebbf328b45b2/html5/thumbnails/2.jpg)
Logical Block Protection Previous results Multithreading Benchmarks Conclusion
Outline
1 Logical Block Protection
2 Previous results
3 Multithreading
4 Benchmarks
5 Conclusion
2 / 17
![Page 3: Tape Storage and CRC Protection](https://reader038.vdocuments.us/reader038/viewer/2022102723/55c53200bb61ebbf328b45b2/html5/thumbnails/3.jpg)
Logical Block Protection Previous results Multithreading Benchmarks Conclusion
Outline
1 Logical Block Protection
2 Previous results
3 Multithreading
4 Benchmarks
5 Conclusion
2 / 17
![Page 4: Tape Storage and CRC Protection](https://reader038.vdocuments.us/reader038/viewer/2022102723/55c53200bb61ebbf328b45b2/html5/thumbnails/4.jpg)
Logical Block Protection Previous results Multithreading Benchmarks Conclusion
Outline
1 Logical Block Protection
2 Previous results
3 Multithreading
4 Benchmarks
5 Conclusion
2 / 17
![Page 5: Tape Storage and CRC Protection](https://reader038.vdocuments.us/reader038/viewer/2022102723/55c53200bb61ebbf328b45b2/html5/thumbnails/5.jpg)
Logical Block Protection Previous results Multithreading Benchmarks Conclusion
Outline
1 Logical Block Protection
2 Previous results
3 Multithreading
4 Benchmarks
5 Conclusion
2 / 17
![Page 6: Tape Storage and CRC Protection](https://reader038.vdocuments.us/reader038/viewer/2022102723/55c53200bb61ebbf328b45b2/html5/thumbnails/6.jpg)
Logical Block Protection Previous results Multithreading Benchmarks Conclusion
Outline
1 Logical Block Protection
2 Previous results
3 Multithreading
4 Benchmarks
5 Conclusion
2 / 17
![Page 7: Tape Storage and CRC Protection](https://reader038.vdocuments.us/reader038/viewer/2022102723/55c53200bb61ebbf328b45b2/html5/thumbnails/7.jpg)
Logical Block Protection Previous results Multithreading Benchmarks Conclusion
Basic picture
3 / 17
![Page 8: Tape Storage and CRC Protection](https://reader038.vdocuments.us/reader038/viewer/2022102723/55c53200bb61ebbf328b45b2/html5/thumbnails/8.jpg)
Logical Block Protection Previous results Multithreading Benchmarks Conclusion
Used CRC algorithms
CRC algorithm supported on tapes:
Reed-Solomon
CRC32-C
4 / 17
![Page 9: Tape Storage and CRC Protection](https://reader038.vdocuments.us/reader038/viewer/2022102723/55c53200bb61ebbf328b45b2/html5/thumbnails/9.jpg)
Logical Block Protection Previous results Multithreading Benchmarks Conclusion
Used CRC algorithms
CRC algorithm supported on tapes:
Reed-Solomon
CRC32-C
4 / 17
![Page 10: Tape Storage and CRC Protection](https://reader038.vdocuments.us/reader038/viewer/2022102723/55c53200bb61ebbf328b45b2/html5/thumbnails/10.jpg)
Logical Block Protection Previous results Multithreading Benchmarks Conclusion
Used CRC algorithms
CRC algorithm supported on tapes:
Reed-Solomon
CRC32-C
4 / 17
![Page 11: Tape Storage and CRC Protection](https://reader038.vdocuments.us/reader038/viewer/2022102723/55c53200bb61ebbf328b45b2/html5/thumbnails/11.jpg)
Logical Block Protection Previous results Multithreading Benchmarks Conclusion
File verification of protection information
5 / 17
![Page 12: Tape Storage and CRC Protection](https://reader038.vdocuments.us/reader038/viewer/2022102723/55c53200bb61ebbf328b45b2/html5/thumbnails/12.jpg)
Logical Block Protection Previous results Multithreading Benchmarks Conclusion
Previous benchmarks
tape-write (no CRC protection)
cca. 250 MB/s
tape-write with Reed-Solomon CRC
cca. 130 MB/s
Reed-Solomon CRC with output to /dev/null
cca. 200 MB/s
6 / 17
![Page 13: Tape Storage and CRC Protection](https://reader038.vdocuments.us/reader038/viewer/2022102723/55c53200bb61ebbf328b45b2/html5/thumbnails/13.jpg)
Logical Block Protection Previous results Multithreading Benchmarks Conclusion
Previous benchmarks
tape-write (no CRC protection)
cca. 250 MB/s
tape-write with Reed-Solomon CRC
cca. 130 MB/s
Reed-Solomon CRC with output to /dev/null
cca. 200 MB/s
6 / 17
![Page 14: Tape Storage and CRC Protection](https://reader038.vdocuments.us/reader038/viewer/2022102723/55c53200bb61ebbf328b45b2/html5/thumbnails/14.jpg)
Logical Block Protection Previous results Multithreading Benchmarks Conclusion
Previous benchmarks
tape-write (no CRC protection)
cca. 250 MB/s
tape-write with Reed-Solomon CRC
cca. 130 MB/s
Reed-Solomon CRC with output to /dev/null
cca. 200 MB/s
6 / 17
![Page 15: Tape Storage and CRC Protection](https://reader038.vdocuments.us/reader038/viewer/2022102723/55c53200bb61ebbf328b45b2/html5/thumbnails/15.jpg)
Logical Block Protection Previous results Multithreading Benchmarks Conclusion
Previous benchmarks
tape-write (no CRC protection)
cca. 250 MB/s
tape-write with Reed-Solomon CRC
cca. 130 MB/s
Reed-Solomon CRC with output to /dev/null
cca. 200 MB/s
6 / 17
![Page 16: Tape Storage and CRC Protection](https://reader038.vdocuments.us/reader038/viewer/2022102723/55c53200bb61ebbf328b45b2/html5/thumbnails/16.jpg)
Logical Block Protection Previous results Multithreading Benchmarks Conclusion
CRC concurrently
CRC computations for every block are independent of one another.
Speed can be increased using parallel computing.This technique is applicable for any CRC algorithm!
0
5
10
15
20
25
30
50 100 150 200 250 300 350 400 450 500
# t
hre
ad
s n
ee
de
d
speed of the CRC algorithm (MB/s)
# threads needed to reach 250MB/s drive speed
7 / 17
![Page 17: Tape Storage and CRC Protection](https://reader038.vdocuments.us/reader038/viewer/2022102723/55c53200bb61ebbf328b45b2/html5/thumbnails/17.jpg)
Logical Block Protection Previous results Multithreading Benchmarks Conclusion
CRC concurrently
CRC computations for every block are independent of one another.Speed can be increased using parallel computing.
This technique is applicable for any CRC algorithm!
0
5
10
15
20
25
30
50 100 150 200 250 300 350 400 450 500
# t
hre
ad
s n
ee
de
d
speed of the CRC algorithm (MB/s)
# threads needed to reach 250MB/s drive speed
7 / 17
![Page 18: Tape Storage and CRC Protection](https://reader038.vdocuments.us/reader038/viewer/2022102723/55c53200bb61ebbf328b45b2/html5/thumbnails/18.jpg)
Logical Block Protection Previous results Multithreading Benchmarks Conclusion
CRC concurrently
CRC computations for every block are independent of one another.Speed can be increased using parallel computing.This technique is applicable for any CRC algorithm!
0
5
10
15
20
25
30
50 100 150 200 250 300 350 400 450 500
# t
hre
ad
s n
ee
de
d
speed of the CRC algorithm (MB/s)
# threads needed to reach 250MB/s drive speed
7 / 17
![Page 19: Tape Storage and CRC Protection](https://reader038.vdocuments.us/reader038/viewer/2022102723/55c53200bb61ebbf328b45b2/html5/thumbnails/19.jpg)
Logical Block Protection Previous results Multithreading Benchmarks Conclusion
CRC concurrently
CRC computations for every block are independent of one another.Speed can be increased using parallel computing.This technique is applicable for any CRC algorithm!
0
5
10
15
20
25
30
50 100 150 200 250 300 350 400 450 500
# t
hre
ad
s n
ee
de
d
speed of the CRC algorithm (MB/s)
# threads needed to reach 250MB/s drive speed
7 / 17
![Page 20: Tape Storage and CRC Protection](https://reader038.vdocuments.us/reader038/viewer/2022102723/55c53200bb61ebbf328b45b2/html5/thumbnails/20.jpg)
Logical Block Protection Previous results Multithreading Benchmarks Conclusion
CRC concurrentlyOverview
CRCs
Compute
Threads
Input
Ouput
8 / 17
![Page 21: Tape Storage and CRC Protection](https://reader038.vdocuments.us/reader038/viewer/2022102723/55c53200bb61ebbf328b45b2/html5/thumbnails/21.jpg)
Logical Block Protection Previous results Multithreading Benchmarks Conclusion
Architecturetpsrv212
Host: tpsrv212.cern.ch
Indexes: physical
Date: mer. 17 sept. 2014 11:30:15 CEST
Machine (16GB)
Socket P#0
L3 (8192KB)
L2 (256KB) L2 (256KB) L2 (256KB) L2 (256KB)
L1d (32KB) L1d (32KB) L1d (32KB) L1d (32KB)
L1i (32KB) L1i (32KB) L1i (32KB) L1i (32KB)
Core P#0 Core P#1 Core P#2 Core P#3
PU P#0
PU P#4
PU P#1
PU P#5
PU P#2
PU P#6
PU P#3
PU P#7
9 / 17
![Page 22: Tape Storage and CRC Protection](https://reader038.vdocuments.us/reader038/viewer/2022102723/55c53200bb61ebbf328b45b2/html5/thumbnails/22.jpg)
Logical Block Protection Previous results Multithreading Benchmarks Conclusion
Architecturetpsrv100
Host: tpsrv100.cern.ch
Indexes: physical
Date: mer. 17 sept. 2014 11:31:05 CEST
Machine (63GB)
Socket P#0
L3 (15MB)
L2 (256KB) L2 (256KB) L2 (256KB) L2 (256KB) L2 (256KB) L2 (256KB)
L1d (32KB) L1d (32KB) L1d (32KB) L1d (32KB) L1d (32KB) L1d (32KB)
L1i (32KB) L1i (32KB) L1i (32KB) L1i (32KB) L1i (32KB) L1i (32KB)
Core P#0 Core P#1 Core P#2 Core P#3 Core P#4 Core P#5
PU P#0
PU P#6
PU P#1
PU P#7
PU P#2
PU P#8
PU P#3
PU P#9
PU P#4
PU P#10
PU P#5
PU P#11
10 / 17
![Page 23: Tape Storage and CRC Protection](https://reader038.vdocuments.us/reader038/viewer/2022102723/55c53200bb61ebbf328b45b2/html5/thumbnails/23.jpg)
Logical Block Protection Previous results Multithreading Benchmarks Conclusion
0
100
200
300
400
500
600
700
800
900
0 2 4 6 8 10 12 14 16
MB
/s
n threads
Reed-Solomon on tpsrv212, output to /dev/null
11 / 17
![Page 24: Tape Storage and CRC Protection](https://reader038.vdocuments.us/reader038/viewer/2022102723/55c53200bb61ebbf328b45b2/html5/thumbnails/24.jpg)
Logical Block Protection Previous results Multithreading Benchmarks Conclusion
0
200
400
600
800
1000
1200
1400
0 5 10 15 20
MB
/s
n threads
Reed-Solomon on tpsrv100, output to /dev/null
12 / 17
![Page 25: Tape Storage and CRC Protection](https://reader038.vdocuments.us/reader038/viewer/2022102723/55c53200bb61ebbf328b45b2/html5/thumbnails/25.jpg)
Logical Block Protection Previous results Multithreading Benchmarks Conclusion
0
500
1000
1500
2000
2500
0 2 4 6 8 10 12 14 16
MB
/s
n threads
CRC32-C on tpsrv212, output to /dev/null
13 / 17
![Page 26: Tape Storage and CRC Protection](https://reader038.vdocuments.us/reader038/viewer/2022102723/55c53200bb61ebbf328b45b2/html5/thumbnails/26.jpg)
Logical Block Protection Previous results Multithreading Benchmarks Conclusion
0
500
1000
1500
2000
2500
3000
3500
0 5 10 15 20 25 30
MB
/s
n threads
CRC32-C on tpsrv100, output to /dev/null
14 / 17
![Page 27: Tape Storage and CRC Protection](https://reader038.vdocuments.us/reader038/viewer/2022102723/55c53200bb61ebbf328b45b2/html5/thumbnails/27.jpg)
Logical Block Protection Previous results Multithreading Benchmarks Conclusion
0
50
100
150
200
250
300
0 2 4 6 8 10 12 14 16
MB
/s
n threads
Reed-Solomon on tpsrv212, output to tape
15 / 17
![Page 28: Tape Storage and CRC Protection](https://reader038.vdocuments.us/reader038/viewer/2022102723/55c53200bb61ebbf328b45b2/html5/thumbnails/28.jpg)
Logical Block Protection Previous results Multithreading Benchmarks Conclusion
Conclusion
Using only 1-2 additional threads, we are already able to reachmaximum tape speed.
And CRC32-C algorithm promises even greater speeds!
16 / 17
![Page 29: Tape Storage and CRC Protection](https://reader038.vdocuments.us/reader038/viewer/2022102723/55c53200bb61ebbf328b45b2/html5/thumbnails/29.jpg)
Logical Block Protection Previous results Multithreading Benchmarks Conclusion
Conclusion
Using only 1-2 additional threads, we are already able to reachmaximum tape speed.And CRC32-C algorithm promises even greater speeds!
16 / 17
![Page 30: Tape Storage and CRC Protection](https://reader038.vdocuments.us/reader038/viewer/2022102723/55c53200bb61ebbf328b45b2/html5/thumbnails/30.jpg)
Logical Block Protection Previous results Multithreading Benchmarks Conclusion
Thank you!Merci!
17 / 17