technological pivot points lightning talks - red hat · technological pivot points lightning talks...
TRANSCRIPT
![Page 1: TECHNOLOGICAL PIVOT POINTS LIGHTNING TALKS - Red Hat · TECHNOLOGICAL PIVOT POINTS LIGHTNING TALKS Jon Masters Steven Carter Veer Muchandi & Tom Corcoran Tom Coughlan ... Filesystem](https://reader030.vdocuments.us/reader030/viewer/2022040609/5ecd02e736a47132e852a3ed/html5/thumbnails/1.jpg)
TECHNOLOGICAL PIVOT POINTS LIGHTNING TALKS
Jon MastersSteven CarterVeer Muchandi & Tom CorcoranTom Coughlan
May 2, 2017
![Page 2: TECHNOLOGICAL PIVOT POINTS LIGHTNING TALKS - Red Hat · TECHNOLOGICAL PIVOT POINTS LIGHTNING TALKS Jon Masters Steven Carter Veer Muchandi & Tom Corcoran Tom Coughlan ... Filesystem](https://reader030.vdocuments.us/reader030/viewer/2022040609/5ecd02e736a47132e852a3ed/html5/thumbnails/2.jpg)
Modernizing HPC for Cloud
Steven CarterChief Cloud Architect, Public Sector2 May 2017
![Page 3: TECHNOLOGICAL PIVOT POINTS LIGHTNING TALKS - Red Hat · TECHNOLOGICAL PIVOT POINTS LIGHTNING TALKS Jon Masters Steven Carter Veer Muchandi & Tom Corcoran Tom Coughlan ... Filesystem](https://reader030.vdocuments.us/reader030/viewer/2022040609/5ecd02e736a47132e852a3ed/html5/thumbnails/3.jpg)
ResultsWall Clock
VS
![Page 4: TECHNOLOGICAL PIVOT POINTS LIGHTNING TALKS - Red Hat · TECHNOLOGICAL PIVOT POINTS LIGHTNING TALKS Jon Masters Steven Carter Veer Muchandi & Tom Corcoran Tom Coughlan ... Filesystem](https://reader030.vdocuments.us/reader030/viewer/2022040609/5ecd02e736a47132e852a3ed/html5/thumbnails/4.jpg)
Sequencers
Web Apps
Storage
Computation
Researchers
Practitioners
Participants
Science is Multidisciplinary
![Page 5: TECHNOLOGICAL PIVOT POINTS LIGHTNING TALKS - Red Hat · TECHNOLOGICAL PIVOT POINTS LIGHTNING TALKS Jon Masters Steven Carter Veer Muchandi & Tom Corcoran Tom Coughlan ... Filesystem](https://reader030.vdocuments.us/reader030/viewer/2022040609/5ecd02e736a47132e852a3ed/html5/thumbnails/5.jpg)
CloudSupercomputer
VS
![Page 6: TECHNOLOGICAL PIVOT POINTS LIGHTNING TALKS - Red Hat · TECHNOLOGICAL PIVOT POINTS LIGHTNING TALKS Jon Masters Steven Carter Veer Muchandi & Tom Corcoran Tom Coughlan ... Filesystem](https://reader030.vdocuments.us/reader030/viewer/2022040609/5ecd02e736a47132e852a3ed/html5/thumbnails/6.jpg)
But, not just a Supercomputer in the cloud...
![Page 7: TECHNOLOGICAL PIVOT POINTS LIGHTNING TALKS - Red Hat · TECHNOLOGICAL PIVOT POINTS LIGHTNING TALKS Jon Masters Steven Carter Veer Muchandi & Tom Corcoran Tom Coughlan ... Filesystem](https://reader030.vdocuments.us/reader030/viewer/2022040609/5ecd02e736a47132e852a3ed/html5/thumbnails/7.jpg)
Object StoragePosix Filesystem
VS
![Page 8: TECHNOLOGICAL PIVOT POINTS LIGHTNING TALKS - Red Hat · TECHNOLOGICAL PIVOT POINTS LIGHTNING TALKS Jon Masters Steven Carter Veer Muchandi & Tom Corcoran Tom Coughlan ... Filesystem](https://reader030.vdocuments.us/reader030/viewer/2022040609/5ecd02e736a47132e852a3ed/html5/thumbnails/8.jpg)
ContainersCode
VS
![Page 9: TECHNOLOGICAL PIVOT POINTS LIGHTNING TALKS - Red Hat · TECHNOLOGICAL PIVOT POINTS LIGHTNING TALKS Jon Masters Steven Carter Veer Muchandi & Tom Corcoran Tom Coughlan ... Filesystem](https://reader030.vdocuments.us/reader030/viewer/2022040609/5ecd02e736a47132e852a3ed/html5/thumbnails/9.jpg)
MicroServicesMonolithic
VS
![Page 10: TECHNOLOGICAL PIVOT POINTS LIGHTNING TALKS - Red Hat · TECHNOLOGICAL PIVOT POINTS LIGHTNING TALKS Jon Masters Steven Carter Veer Muchandi & Tom Corcoran Tom Coughlan ... Filesystem](https://reader030.vdocuments.us/reader030/viewer/2022040609/5ecd02e736a47132e852a3ed/html5/thumbnails/10.jpg)
Web Apps
Researchers
Practitioners
Participants
Post Processing
Sequencers
Alignment
Variant Calling
FaaS
On-Prem Hosted
IaaS Abstraction
AP
I Ab
stra
ctio
n
Geo
-Rep
licat
ed
Obj
ect S
tora
ge
![Page 11: TECHNOLOGICAL PIVOT POINTS LIGHTNING TALKS - Red Hat · TECHNOLOGICAL PIVOT POINTS LIGHTNING TALKS Jon Masters Steven Carter Veer Muchandi & Tom Corcoran Tom Coughlan ... Filesystem](https://reader030.vdocuments.us/reader030/viewer/2022040609/5ecd02e736a47132e852a3ed/html5/thumbnails/11.jpg)
Circuit Breaker using 3Scale
Dealing with Microservice failures
Tom Corcoran, Sr. Solution Architect, Red Hat
Veer Muchandi, Principal Architect, Red Hat
05/02/2017
![Page 12: TECHNOLOGICAL PIVOT POINTS LIGHTNING TALKS - Red Hat · TECHNOLOGICAL PIVOT POINTS LIGHTNING TALKS Jon Masters Steven Carter Veer Muchandi & Tom Corcoran Tom Coughlan ... Filesystem](https://reader030.vdocuments.us/reader030/viewer/2022040609/5ecd02e736a47132e852a3ed/html5/thumbnails/12.jpg)
Problem
![Page 13: TECHNOLOGICAL PIVOT POINTS LIGHTNING TALKS - Red Hat · TECHNOLOGICAL PIVOT POINTS LIGHTNING TALKS Jon Masters Steven Carter Veer Muchandi & Tom Corcoran Tom Coughlan ... Filesystem](https://reader030.vdocuments.us/reader030/viewer/2022040609/5ecd02e736a47132e852a3ed/html5/thumbnails/13.jpg)
Client calling service
![Page 14: TECHNOLOGICAL PIVOT POINTS LIGHTNING TALKS - Red Hat · TECHNOLOGICAL PIVOT POINTS LIGHTNING TALKS Jon Masters Steven Carter Veer Muchandi & Tom Corcoran Tom Coughlan ... Filesystem](https://reader030.vdocuments.us/reader030/viewer/2022040609/5ecd02e736a47132e852a3ed/html5/thumbnails/14.jpg)
Service Failure
![Page 15: TECHNOLOGICAL PIVOT POINTS LIGHTNING TALKS - Red Hat · TECHNOLOGICAL PIVOT POINTS LIGHTNING TALKS Jon Masters Steven Carter Veer Muchandi & Tom Corcoran Tom Coughlan ... Filesystem](https://reader030.vdocuments.us/reader030/viewer/2022040609/5ecd02e736a47132e852a3ed/html5/thumbnails/15.jpg)
Service failure effect multiplied
![Page 16: TECHNOLOGICAL PIVOT POINTS LIGHTNING TALKS - Red Hat · TECHNOLOGICAL PIVOT POINTS LIGHTNING TALKS Jon Masters Steven Carter Veer Muchandi & Tom Corcoran Tom Coughlan ... Filesystem](https://reader030.vdocuments.us/reader030/viewer/2022040609/5ecd02e736a47132e852a3ed/html5/thumbnails/16.jpg)
Circuit Breaker Implementation
using 3Scale
![Page 17: TECHNOLOGICAL PIVOT POINTS LIGHTNING TALKS - Red Hat · TECHNOLOGICAL PIVOT POINTS LIGHTNING TALKS Jon Masters Steven Carter Veer Muchandi & Tom Corcoran Tom Coughlan ... Filesystem](https://reader030.vdocuments.us/reader030/viewer/2022040609/5ecd02e736a47132e852a3ed/html5/thumbnails/17.jpg)
Normal Operation - 3 Scale forwards requests to the main service
![Page 18: TECHNOLOGICAL PIVOT POINTS LIGHTNING TALKS - Red Hat · TECHNOLOGICAL PIVOT POINTS LIGHTNING TALKS Jon Masters Steven Carter Veer Muchandi & Tom Corcoran Tom Coughlan ... Filesystem](https://reader030.vdocuments.us/reader030/viewer/2022040609/5ecd02e736a47132e852a3ed/html5/thumbnails/18.jpg)
Fail over - Circuit broken to the main service and requests forwarded to the alternate service
Acts as circuit breaker object
![Page 19: TECHNOLOGICAL PIVOT POINTS LIGHTNING TALKS - Red Hat · TECHNOLOGICAL PIVOT POINTS LIGHTNING TALKS Jon Masters Steven Carter Veer Muchandi & Tom Corcoran Tom Coughlan ... Filesystem](https://reader030.vdocuments.us/reader030/viewer/2022040609/5ecd02e736a47132e852a3ed/html5/thumbnails/19.jpg)
Graceful handling on Alternative service failure with cached response
Acts as circuit breaker object
![Page 20: TECHNOLOGICAL PIVOT POINTS LIGHTNING TALKS - Red Hat · TECHNOLOGICAL PIVOT POINTS LIGHTNING TALKS Jon Masters Steven Carter Veer Muchandi & Tom Corcoran Tom Coughlan ... Filesystem](https://reader030.vdocuments.us/reader030/viewer/2022040609/5ecd02e736a47132e852a3ed/html5/thumbnails/20.jpg)
Graceful handling of Alternative service failure with Error response
Acts as circuit breaker object
![Page 21: TECHNOLOGICAL PIVOT POINTS LIGHTNING TALKS - Red Hat · TECHNOLOGICAL PIVOT POINTS LIGHTNING TALKS Jon Masters Steven Carter Veer Muchandi & Tom Corcoran Tom Coughlan ... Filesystem](https://reader030.vdocuments.us/reader030/viewer/2022040609/5ecd02e736a47132e852a3ed/html5/thumbnails/21.jpg)
Advantages of using 3Scale
Circuit breaker becomes an infrastructure service
- Works with simple configuration changes in the nginx.conf
- No code intrusion, No annotations in business logic
Underlying technology does not matter
Language Neutral
![Page 22: TECHNOLOGICAL PIVOT POINTS LIGHTNING TALKS - Red Hat · TECHNOLOGICAL PIVOT POINTS LIGHTNING TALKS Jon Masters Steven Carter Veer Muchandi & Tom Corcoran Tom Coughlan ... Filesystem](https://reader030.vdocuments.us/reader030/viewer/2022040609/5ecd02e736a47132e852a3ed/html5/thumbnails/22.jpg)
Demo
![Page 23: TECHNOLOGICAL PIVOT POINTS LIGHTNING TALKS - Red Hat · TECHNOLOGICAL PIVOT POINTS LIGHTNING TALKS Jon Masters Steven Carter Veer Muchandi & Tom Corcoran Tom Coughlan ... Filesystem](https://reader030.vdocuments.us/reader030/viewer/2022040609/5ecd02e736a47132e852a3ed/html5/thumbnails/23.jpg)
SUPPORT – SLIDE - Circuit Breaker – on NginxNginx uses● Upstreams – for API Back Ends
● Locations – to capture incoming API requests and proxy pass them
![Page 24: TECHNOLOGICAL PIVOT POINTS LIGHTNING TALKS - Red Hat · TECHNOLOGICAL PIVOT POINTS LIGHTNING TALKS Jon Masters Steven Carter Veer Muchandi & Tom Corcoran Tom Coughlan ... Filesystem](https://reader030.vdocuments.us/reader030/viewer/2022040609/5ecd02e736a47132e852a3ed/html5/thumbnails/24.jpg)
SUPPORT – SLIDE - Circuit Breaker – on Nginx
Override default selection of Next Upstream.
Back end failover – fine grained control
![Page 25: TECHNOLOGICAL PIVOT POINTS LIGHTNING TALKS - Red Hat · TECHNOLOGICAL PIVOT POINTS LIGHTNING TALKS Jon Masters Steven Carter Veer Muchandi & Tom Corcoran Tom Coughlan ... Filesystem](https://reader030.vdocuments.us/reader030/viewer/2022040609/5ecd02e736a47132e852a3ed/html5/thumbnails/25.jpg)
SUPPORT – SLIDE - Circuit Breaker – on Nginx
Slow Start
Rate Limiting
Failure Mitigation
![Page 26: TECHNOLOGICAL PIVOT POINTS LIGHTNING TALKS - Red Hat · TECHNOLOGICAL PIVOT POINTS LIGHTNING TALKS Jon Masters Steven Carter Veer Muchandi & Tom Corcoran Tom Coughlan ... Filesystem](https://reader030.vdocuments.us/reader030/viewer/2022040609/5ecd02e736a47132e852a3ed/html5/thumbnails/26.jpg)
SUPPORT – SLIDE - Circuit Breaker – on Nginx
Response caching.
Error Page.
Last Resort Options – no back end available
![Page 27: TECHNOLOGICAL PIVOT POINTS LIGHTNING TALKS - Red Hat · TECHNOLOGICAL PIVOT POINTS LIGHTNING TALKS Jon Masters Steven Carter Veer Muchandi & Tom Corcoran Tom Coughlan ... Filesystem](https://reader030.vdocuments.us/reader030/viewer/2022040609/5ecd02e736a47132e852a3ed/html5/thumbnails/27.jpg)
Fast as lightning : NVMe and NVDIMM's Impact on your data center
Tom Coughlan Sr. Engineering ManagerFilesystem and Storage Team, Red Hat May 2, 2017
![Page 28: TECHNOLOGICAL PIVOT POINTS LIGHTNING TALKS - Red Hat · TECHNOLOGICAL PIVOT POINTS LIGHTNING TALKS Jon Masters Steven Carter Veer Muchandi & Tom Corcoran Tom Coughlan ... Filesystem](https://reader030.vdocuments.us/reader030/viewer/2022040609/5ecd02e736a47132e852a3ed/html5/thumbnails/28.jpg)
10 ns 1ms 1s1 µs
CPUI/O
Ctrlr.HBA
PCILocalHDD
SAS/SATA
Shared
Array
FC, Ethernet
Little change over 40+ years...
I/O Latency
![Page 29: TECHNOLOGICAL PIVOT POINTS LIGHTNING TALKS - Red Hat · TECHNOLOGICAL PIVOT POINTS LIGHTNING TALKS Jon Masters Steven Carter Veer Muchandi & Tom Corcoran Tom Coughlan ... Filesystem](https://reader030.vdocuments.us/reader030/viewer/2022040609/5ecd02e736a47132e852a3ed/html5/thumbnails/29.jpg)
10 ns 1ms 1s1 µs
CPUI/O
Ctrlr.
HBA
PCI
LocalHDD
SAS/SATA
Shared
Array
FC, EthernetAn order of magnitude improvement
I/O LatencyFlash and NVMe arrive.
NVMe Flash
FlashSSD
![Page 30: TECHNOLOGICAL PIVOT POINTS LIGHTNING TALKS - Red Hat · TECHNOLOGICAL PIVOT POINTS LIGHTNING TALKS Jon Masters Steven Carter Veer Muchandi & Tom Corcoran Tom Coughlan ... Filesystem](https://reader030.vdocuments.us/reader030/viewer/2022040609/5ecd02e736a47132e852a3ed/html5/thumbnails/30.jpg)
10 ns 1ms 1s1 µs
CPUI/O
Ctrlr.
HBA
PCI
LocalHDD
SAS/SATA
Shared
Array
FC, EthernetAnother order of magnitude improvement
I/O LatencyPersistent Memory Arrives
NVMe Flash
FlashSSD
pmem
![Page 31: TECHNOLOGICAL PIVOT POINTS LIGHTNING TALKS - Red Hat · TECHNOLOGICAL PIVOT POINTS LIGHTNING TALKS Jon Masters Steven Carter Veer Muchandi & Tom Corcoran Tom Coughlan ... Filesystem](https://reader030.vdocuments.us/reader030/viewer/2022040609/5ecd02e736a47132e852a3ed/html5/thumbnails/31.jpg)
10 ns 1ms 1s1 µs
CPUI/O
Ctrlr.
HBA
PCI
LocalHDD
SAS/SATA
Shared
Array
FC, Ethernet, Infiniband
Lower latency for shared storage.
I/O LatencyNVMe over Fabric
NVMe Flash
FlashSSD
pmem
![Page 32: TECHNOLOGICAL PIVOT POINTS LIGHTNING TALKS - Red Hat · TECHNOLOGICAL PIVOT POINTS LIGHTNING TALKS Jon Masters Steven Carter Veer Muchandi & Tom Corcoran Tom Coughlan ... Filesystem](https://reader030.vdocuments.us/reader030/viewer/2022040609/5ecd02e736a47132e852a3ed/html5/thumbnails/32.jpg)
10 ns 1ms 1s1 µs
CPUI/O
Ctrlr.
HBA
LocalHDD
SAS/SATA
Shared
Array
FC, Ethernet, InfinibandEven lower-latency for shared storage...
I/O LatencyPCI Switch
SharedNVMe Flash Flash
SSD
pmem
CPUI/O
Ctrlr.
PCI
SW
![Page 33: TECHNOLOGICAL PIVOT POINTS LIGHTNING TALKS - Red Hat · TECHNOLOGICAL PIVOT POINTS LIGHTNING TALKS Jon Masters Steven Carter Veer Muchandi & Tom Corcoran Tom Coughlan ... Filesystem](https://reader030.vdocuments.us/reader030/viewer/2022040609/5ecd02e736a47132e852a3ed/html5/thumbnails/33.jpg)
RHEL Status● Persistent Memory
○ Block-mode access supported in 7.3
○ Byte-mode direct access (DAX) is Tech. Preview in 7.3 (and 7.4)
■ Uses mmap on XFS or ext4
○ HPE results, using NVDIMMs for…
■ Oracle OLTP redologs
● https://www.hpe.com/h20195/v2/GetPDF.aspx/4AA6-6008ENW.pdf
■ PostgreSQL Write-Ahead Logging
● http://h20195.www2.hpe.com/V2/GetDocument.aspx?docname=4AA6-8881ENW
● NVMeF - try it in RHEL 7.4 Beta
![Page 34: TECHNOLOGICAL PIVOT POINTS LIGHTNING TALKS - Red Hat · TECHNOLOGICAL PIVOT POINTS LIGHTNING TALKS Jon Masters Steven Carter Veer Muchandi & Tom Corcoran Tom Coughlan ... Filesystem](https://reader030.vdocuments.us/reader030/viewer/2022040609/5ecd02e736a47132e852a3ed/html5/thumbnails/34.jpg)
Additional sessions of interest...● “On the path to persistent memory”
BOF B113761 Tuesday, 10:15 to 11 am (oops...) Room 158Linda Knippers (HPE) and Jeff Moyer (Red Hat)
● “Bring your performance and scale problems to the experts”Session 101558 Tuesday 5:30 PM - 7:00 PM Room 205C
● “Utilizing Persistent Memory to Improve DB Performance and Reduce Costs” Session S111008Wednesday at 4:30 pm Room 105Karen Dorhamer (HPE)
![Page 35: TECHNOLOGICAL PIVOT POINTS LIGHTNING TALKS - Red Hat · TECHNOLOGICAL PIVOT POINTS LIGHTNING TALKS Jon Masters Steven Carter Veer Muchandi & Tom Corcoran Tom Coughlan ... Filesystem](https://reader030.vdocuments.us/reader030/viewer/2022040609/5ecd02e736a47132e852a3ed/html5/thumbnails/35.jpg)
THANK YOUplus.google.com/+RedHat
linkedin.com/company/red-hat
youtube.com/user/RedHatVideos
facebook.com/redhatinc
twitter.com/RedHatNews
![Page 36: TECHNOLOGICAL PIVOT POINTS LIGHTNING TALKS - Red Hat · TECHNOLOGICAL PIVOT POINTS LIGHTNING TALKS Jon Masters Steven Carter Veer Muchandi & Tom Corcoran Tom Coughlan ... Filesystem](https://reader030.vdocuments.us/reader030/viewer/2022040609/5ecd02e736a47132e852a3ed/html5/thumbnails/36.jpg)