distributed systemspxk/417/notes/content/01-intro.pdf · more time for protocol processing – ecc,...
TRANSCRIPT
![Page 1: Distributed Systemspxk/417/notes/content/01-intro.pdf · more time for protocol processing – ECC, TCP checksums, parsing – Image, audio compression feasible Faster network ®](https://reader033.vdocuments.us/reader033/viewer/2022043008/5f986da3e02f990a610657e0/html5/thumbnails/1.jpg)
Distributed Systems01. Introduction
Paul Krzyzanowski
Rutgers University
Spring 2016
1September 12, 2016 © 2014-2016 Paul Krzyzanowski
![Page 2: Distributed Systemspxk/417/notes/content/01-intro.pdf · more time for protocol processing – ECC, TCP checksums, parsing – Image, audio compression feasible Faster network ®](https://reader033.vdocuments.us/reader033/viewer/2022043008/5f986da3e02f990a610657e0/html5/thumbnails/2.jpg)
What can we do nowthat we could not do before?~30 years ago
1986: The Internet is 17 years old
2
![Page 3: Distributed Systemspxk/417/notes/content/01-intro.pdf · more time for protocol processing – ECC, TCP checksums, parsing – Image, audio compression feasible Faster network ®](https://reader033.vdocuments.us/reader033/viewer/2022043008/5f986da3e02f990a610657e0/html5/thumbnails/3.jpg)
Technology advances
ProcessorsMemory
Networking
StorageProtocols
3
![Page 4: Distributed Systemspxk/417/notes/content/01-intro.pdf · more time for protocol processing – ECC, TCP checksums, parsing – Image, audio compression feasible Faster network ®](https://reader033.vdocuments.us/reader033/viewer/2022043008/5f986da3e02f990a610657e0/html5/thumbnails/4.jpg)
Networking: Ethernet – 1973, 1976
June 1976: Robert Metcalfe presents the concept of Ethernet at the National Computer Conference
1980: Ethernet introduced as de facto standard (DEC, Intel, Xerox)
4
![Page 5: Distributed Systemspxk/417/notes/content/01-intro.pdf · more time for protocol processing – ECC, TCP checksums, parsing – Image, audio compression feasible Faster network ®](https://reader033.vdocuments.us/reader033/viewer/2022043008/5f986da3e02f990a610657e0/html5/thumbnails/5.jpg)
Network architectureLAN speeds– Original Ethernet: 2.94 Mbps
– 1985: thick Ethernet: 10 Mbps – 1 Mbps with twisted pair networking– 1991: 10BaseT - twisted pair: 10 Mbps – Switched networking: scalable bandwidth– 1995: 100 Mbps Ethernet
– 1998: 1 Gbps (Gigabit) Ethernet
– 2001: 10 Gbps introduced
– 2005-now: 40/100 Gbps
+ Wireless LAN1999: 802.11b (wireless Ethernet) standardized
2014: 802.11ac = 8×866.7 Mbps = 7 Gbps
+ Personal Area Networks: Bluetooth, ZigBee, Z-Wave
100 – >10,000xfaster
5
![Page 6: Distributed Systemspxk/417/notes/content/01-intro.pdf · more time for protocol processing – ECC, TCP checksums, parsing – Image, audio compression feasible Faster network ®](https://reader033.vdocuments.us/reader033/viewer/2022043008/5f986da3e02f990a610657e0/html5/thumbnails/6.jpg)
Network ConnectivityThen:
– Large companies and universities on Internet
– Gateways between other networks– Consumers used dial-up
bulletin boards– 1985: 1,961 hosts on the Internet
Now:– One Internet (mostly)– Over a billion hosts– Widespread connectivity– High-speed WAN connectivity: >50 Mbps … 1 Gbps– Switched LANs– Wireless networking
6
https://www.isc.org/network/survey/
![Page 7: Distributed Systemspxk/417/notes/content/01-intro.pdf · more time for protocol processing – ECC, TCP checksums, parsing – Image, audio compression feasible Faster network ®](https://reader033.vdocuments.us/reader033/viewer/2022043008/5f986da3e02f990a610657e0/html5/thumbnails/7.jpg)
Metcalfe’s Law
The value of a telecommunications network is proportional to the square of the number of connected users of the system.
This makes networking interesting to us!
7
![Page 8: Distributed Systemspxk/417/notes/content/01-intro.pdf · more time for protocol processing – ECC, TCP checksums, parsing – Image, audio compression feasible Faster network ®](https://reader033.vdocuments.us/reader033/viewer/2022043008/5f986da3e02f990a610657e0/html5/thumbnails/8.jpg)
Computing Power
Computers got…–Smaller–Cheaper–Power efficient–Faster
Microprocessors became technology leaders
8
![Page 9: Distributed Systemspxk/417/notes/content/01-intro.pdf · more time for protocol processing – ECC, TCP checksums, parsing – Image, audio compression feasible Faster network ®](https://reader033.vdocuments.us/reader033/viewer/2022043008/5f986da3e02f990a610657e0/html5/thumbnails/9.jpg)
1985-now: – 714x smaller transistors– >7000x more transistors– >120x faster clock
Computing Power (Intel Processors)
9
80802 MHz6K transistors @ 10µm
386DX33 MHz275K transistors @ 1.5µm
1977 1985 2005
Pentium D2.6 – 3.7 GHz2 cores169M transistors @ 90nm
Pentium Pro200 MHz5.5M transistors @ 500nm
1995 2015
I7-6700K Skylake4.0 GHz4 cores, 8 MB shared cache~1.3M transistors @ 14nm
Xeon Haswell-E52.3 GHz18 cores, 2.5 MB cache/core5.6M transistors @ 22nm
We can no longer make CPUs much faster.How do we get increased performance? More cores.→Parallel system on a chip
GPUs scaled too: 2016 – Quadro P6000: 12 billion transistors, 3,840 CUDA cores
![Page 10: Distributed Systemspxk/417/notes/content/01-intro.pdf · more time for protocol processing – ECC, TCP checksums, parsing – Image, audio compression feasible Faster network ®](https://reader033.vdocuments.us/reader033/viewer/2022043008/5f986da3e02f990a610657e0/html5/thumbnails/10.jpg)
Network Content: MusicExample: 9,839 songs
– 49 GB– Average song size: 5.2 MB
Today– Streaming (Pandora/Spotify): 96-320 kbps– Download time per song @100 Mbps: ~ 0.4 seconds– Storage cost for the collection: ~ $1.60 ($120 for a 4 TB drive)
~30 years ago (1985)– Streaming not practical– Download time per song, V90 modem @44 Kbps: 15 minutes– Storage cost: $511,640 (40 MB at $400 – over 1,279 drives!)
10
![Page 11: Distributed Systemspxk/417/notes/content/01-intro.pdf · more time for protocol processing – ECC, TCP checksums, parsing – Image, audio compression feasible Faster network ®](https://reader033.vdocuments.us/reader033/viewer/2022043008/5f986da3e02f990a610657e0/html5/thumbnails/11.jpg)
Network Content: Video
Today– Netflix streaming 4K video @ 15.6 Mbps (HEVC/h.265 codec)– YouTube: stores ~76 PB (76×1015) per year
~30 years ago (1985)– Video streaming not feasible
11
![Page 12: Distributed Systemspxk/417/notes/content/01-intro.pdf · more time for protocol processing – ECC, TCP checksums, parsing – Image, audio compression feasible Faster network ®](https://reader033.vdocuments.us/reader033/viewer/2022043008/5f986da3e02f990a610657e0/html5/thumbnails/12.jpg)
Protocols
Many have been devloped ®These are the APIs for network interaction
Faster CPU ®more time for protocol processing
– ECC, TCP checksums, parsing– Image, audio compression feasible
Faster network ®® support bigger (and bloated) protocols– e.g., SOAP/XML, JSON – human-readable, explicit typing
12
![Page 13: Distributed Systemspxk/417/notes/content/01-intro.pdf · more time for protocol processing – ECC, TCP checksums, parsing – Image, audio compression feasible Faster network ®](https://reader033.vdocuments.us/reader033/viewer/2022043008/5f986da3e02f990a610657e0/html5/thumbnails/13.jpg)
Building and classifyingparallel and distributed systems
13
![Page 14: Distributed Systemspxk/417/notes/content/01-intro.pdf · more time for protocol processing – ECC, TCP checksums, parsing – Image, audio compression feasible Faster network ®](https://reader033.vdocuments.us/reader033/viewer/2022043008/5f986da3e02f990a610657e0/html5/thumbnails/14.jpg)
Flynn’s Taxonomy (1966)
SISD– traditional uniprocessor system
SIMD– array (vector) processor– Examples:
• GPUs – Graphical Processing Units for video• AVX: Intel’s Advanced Vector Extensions• GPGPU (General Purpose GPU): AMD/ATI, NVIDIA
MISD– Generally not used and doesn’t make sense– Sometimes (rarely!) applied to classifying redundant systems
MIMD– multiple computers, each with:
• program counter, program (instructions), data– parallel and distributed systems
Number of instruction streams and number of data streams
14
![Page 15: Distributed Systemspxk/417/notes/content/01-intro.pdf · more time for protocol processing – ECC, TCP checksums, parsing – Image, audio compression feasible Faster network ®](https://reader033.vdocuments.us/reader033/viewer/2022043008/5f986da3e02f990a610657e0/html5/thumbnails/15.jpg)
Subclassifying MIMD
memory– shared memory systems: multiprocessors– no shared memory: networks of computers, multicomputers
interconnect– bus– switch
delay/bandwidth– tightly coupled systems– loosely coupled systems
15
![Page 16: Distributed Systemspxk/417/notes/content/01-intro.pdf · more time for protocol processing – ECC, TCP checksums, parsing – Image, audio compression feasible Faster network ®](https://reader033.vdocuments.us/reader033/viewer/2022043008/5f986da3e02f990a610657e0/html5/thumbnails/16.jpg)
Parallel Systems: Multiprocessors
• Shared memory
• Shared clock
• All-or-nothing failure
16
![Page 17: Distributed Systemspxk/417/notes/content/01-intro.pdf · more time for protocol processing – ECC, TCP checksums, parsing – Image, audio compression feasible Faster network ®](https://reader033.vdocuments.us/reader033/viewer/2022043008/5f986da3e02f990a610657e0/html5/thumbnails/17.jpg)
Bus
Bus-based multiprocessors
CPU A
SMP: Symmetric Multi-ProcessingAll CPUs connected to one bus (backplane)
Memory and peripherals are accessed via shared bus. System looks the same from any processor.
CPU Bmemory Device
I/O
The bus becomes a point of congestion … limits performance
17
![Page 18: Distributed Systemspxk/417/notes/content/01-intro.pdf · more time for protocol processing – ECC, TCP checksums, parsing – Image, audio compression feasible Faster network ®](https://reader033.vdocuments.us/reader033/viewer/2022043008/5f986da3e02f990a610657e0/html5/thumbnails/18.jpg)
Bus-based multiprocessors• The cache: great idea to deal with bus overload & memory contention
– Memory that is local to a processor
• CPU performs I/O to cache memory– Access main memory only on cache miss
Bus
memory DeviceI/O
CPU A
cache
CPU B
cache
18
![Page 19: Distributed Systemspxk/417/notes/content/01-intro.pdf · more time for protocol processing – ECC, TCP checksums, parsing – Image, audio compression feasible Faster network ®](https://reader033.vdocuments.us/reader033/viewer/2022043008/5f986da3e02f990a610657e0/html5/thumbnails/19.jpg)
Working with a cacheCPU A reads location 12345 from memory
12345:7 DeviceI/O
CPU A
12345: 7
CPU B
Bus
19
![Page 20: Distributed Systemspxk/417/notes/content/01-intro.pdf · more time for protocol processing – ECC, TCP checksums, parsing – Image, audio compression feasible Faster network ®](https://reader033.vdocuments.us/reader033/viewer/2022043008/5f986da3e02f990a610657e0/html5/thumbnails/20.jpg)
Working with a cacheCPU A modifies location 12345
Bus
12345:7 DeviceI/O
CPU A
12345: 7
CPU B
12345: 3
20
![Page 21: Distributed Systemspxk/417/notes/content/01-intro.pdf · more time for protocol processing – ECC, TCP checksums, parsing – Image, audio compression feasible Faster network ®](https://reader033.vdocuments.us/reader033/viewer/2022043008/5f986da3e02f990a610657e0/html5/thumbnails/21.jpg)
Working with a cacheCPU B reads location 12345 from memory
12345:7 DeviceI/O
CPU A
12345: 3
CPU B
12345: 7
Gets old value
Memory not coherent!
Bus
21
![Page 22: Distributed Systemspxk/417/notes/content/01-intro.pdf · more time for protocol processing – ECC, TCP checksums, parsing – Image, audio compression feasible Faster network ®](https://reader033.vdocuments.us/reader033/viewer/2022043008/5f986da3e02f990a610657e0/html5/thumbnails/22.jpg)
Write-through cacheFix coherency problem by writing all values through bus to main memory
12345:7 DeviceI/O
CPU A
12345: 7
CPU B
CPU A modifies location 12345 – write-throughmain memory is now coherent
12345: 312345:3
Bus
22
![Page 23: Distributed Systemspxk/417/notes/content/01-intro.pdf · more time for protocol processing – ECC, TCP checksums, parsing – Image, audio compression feasible Faster network ®](https://reader033.vdocuments.us/reader033/viewer/2022043008/5f986da3e02f990a610657e0/html5/thumbnails/23.jpg)
Write-through cache … continuedCPU B reads location 12345 from memory
- loads into cache
12345:3 DeviceI/O
CPU A
12345: 3
CPU B
12345: 3
Bus
23
![Page 24: Distributed Systemspxk/417/notes/content/01-intro.pdf · more time for protocol processing – ECC, TCP checksums, parsing – Image, audio compression feasible Faster network ®](https://reader033.vdocuments.us/reader033/viewer/2022043008/5f986da3e02f990a610657e0/html5/thumbnails/24.jpg)
Write-through cacheCPU A modifies location 12345
- write-through
12345:3 DeviceI/O
CPU A
12345: 3
CPU B
12345: 3
Cache on CPU B not updatedMemory not coherent!
12345:012345: 0
Bus
24
![Page 25: Distributed Systemspxk/417/notes/content/01-intro.pdf · more time for protocol processing – ECC, TCP checksums, parsing – Image, audio compression feasible Faster network ®](https://reader033.vdocuments.us/reader033/viewer/2022043008/5f986da3e02f990a610657e0/html5/thumbnails/25.jpg)
Snoopy cacheAdd logic to each cache controller:
monitor the bus for writes to memory
12345: 3 DeviceI/O
CPU A
12345: 3
CPU B
12345: 3
write [12345]¬ 0
12345: 3
Virtually all bus-based architectures use a snoopy cache
Bus
12345: 012345: 0
12345: 0
25
![Page 26: Distributed Systemspxk/417/notes/content/01-intro.pdf · more time for protocol processing – ECC, TCP checksums, parsing – Image, audio compression feasible Faster network ®](https://reader033.vdocuments.us/reader033/viewer/2022043008/5f986da3e02f990a610657e0/html5/thumbnails/26.jpg)
Switched multiprocessors
• Bus-based architecture does not scale linearly to large number of CPUs (e.g., beyond 8)
26
![Page 27: Distributed Systemspxk/417/notes/content/01-intro.pdf · more time for protocol processing – ECC, TCP checksums, parsing – Image, audio compression feasible Faster network ®](https://reader033.vdocuments.us/reader033/viewer/2022043008/5f986da3e02f990a610657e0/html5/thumbnails/27.jpg)
Switched multiprocessorsDivide memory into groups and connect chunks of memory to the processors with a crossbar switch
n2 crosspoint switches – expensive switching fabricWe still want to cache at each CPU – but we cannot snoop!
CPU
CPU
CPU
CPU
mem mem mem mem
27
![Page 28: Distributed Systemspxk/417/notes/content/01-intro.pdf · more time for protocol processing – ECC, TCP checksums, parsing – Image, audio compression feasible Faster network ®](https://reader033.vdocuments.us/reader033/viewer/2022043008/5f986da3e02f990a610657e0/html5/thumbnails/28.jpg)
NUMA
• Hierarchical Memory System
• All CPUs see the same address space
• Each CPU has local connectivity to a region of memory– fast access
• Access to other regions of memory – slower
• Placement of code and data becomes challenging– Operating system has to be aware of memory allocation and CPU
scheduling
28
![Page 29: Distributed Systemspxk/417/notes/content/01-intro.pdf · more time for protocol processing – ECC, TCP checksums, parsing – Image, audio compression feasible Faster network ®](https://reader033.vdocuments.us/reader033/viewer/2022043008/5f986da3e02f990a610657e0/html5/thumbnails/29.jpg)
NUMA• SGI Origin’s ccNUMA
• AMD64 Opteron– Each CPU gets a bank of DDR memory– Inter-processor communications are sent over a HyperTransport link
• Intel– Integrated Memory Controller (IMC): fast channel to local memory– QuickPath Interconnect: point-to-point interconnect among processors
• Linux ≥2.5 kernel, Windows ≥7– Multiple run queues– Structures for determining layout of memory and processors
29
![Page 30: Distributed Systemspxk/417/notes/content/01-intro.pdf · more time for protocol processing – ECC, TCP checksums, parsing – Image, audio compression feasible Faster network ®](https://reader033.vdocuments.us/reader033/viewer/2022043008/5f986da3e02f990a610657e0/html5/thumbnails/30.jpg)
NUMA Cache Coherence NUMA: Intel Example
• Home Snoop: Home-based consistency protocol– Each CPU is responsible for a region of memory– It is the “home agent” for that memory
• Each home agent maintains a directory (table) that keeps track of who has the latest version
30
CPU 1
CPU 2
CPU 3
CPU 4
MemoryInterface
MemoryInterface
MemoryInterface
MemoryInterface
![Page 31: Distributed Systemspxk/417/notes/content/01-intro.pdf · more time for protocol processing – ECC, TCP checksums, parsing – Image, audio compression feasible Faster network ®](https://reader033.vdocuments.us/reader033/viewer/2022043008/5f986da3e02f990a610657e0/html5/thumbnails/31.jpg)
NUMA Cache Coherence NUMA: Intel Example
1. CPU sends request to home agent
2. Home agent requests status fromthe CPU that may have a cachedcopy (caching agent)
31
CPU 1
CPU 2
CPU 3
CPU 4
Home AgentCaching Agent
1
2
![Page 32: Distributed Systemspxk/417/notes/content/01-intro.pdf · more time for protocol processing – ECC, TCP checksums, parsing – Image, audio compression feasible Faster network ®](https://reader033.vdocuments.us/reader033/viewer/2022043008/5f986da3e02f990a610657e0/html5/thumbnails/32.jpg)
NUMA Cache Coherence NUMA: Intel Example
3. (a) Caching agent sends data update tonew caching agent
(b) Caching agent sends status update tohome agent
4. Home agent resolves any conflicts &completes transaction
32
CPU 1
CPU 2
CPU 3
CPU 4
Home AgentCaching Agent
3a
3b
New Caching Agent
4
![Page 33: Distributed Systemspxk/417/notes/content/01-intro.pdf · more time for protocol processing – ECC, TCP checksums, parsing – Image, audio compression feasible Faster network ®](https://reader033.vdocuments.us/reader033/viewer/2022043008/5f986da3e02f990a610657e0/html5/thumbnails/33.jpg)
Networks of computers
• Eventually, other bottlenecks occur– Network, disk
• We want to scale beyond multiprocessors– Multicomputers
• No shared memory, no shared clock
• Communication mechanism needed– Traffic much lower than memory access– Network
33
![Page 34: Distributed Systemspxk/417/notes/content/01-intro.pdf · more time for protocol processing – ECC, TCP checksums, parsing – Image, audio compression feasible Faster network ®](https://reader033.vdocuments.us/reader033/viewer/2022043008/5f986da3e02f990a610657e0/html5/thumbnails/34.jpg)
Bus-based multicomputers
Collection of workstations on a LAN
Interconnect
CPU
memoryLANconnector
CPU
memoryLANconnector
CPU
memoryLANconnector
CPU
memoryLANconnector
34
A shared bus-based interconnect gives us the option of snooping on network traffic
![Page 35: Distributed Systemspxk/417/notes/content/01-intro.pdf · more time for protocol processing – ECC, TCP checksums, parsing – Image, audio compression feasible Faster network ®](https://reader033.vdocuments.us/reader033/viewer/2022043008/5f986da3e02f990a610657e0/html5/thumbnails/35.jpg)
Switched multicomputers
Collection of workstations on a LAN
CPU
memoryLANconnector
LANswitch
35
A switched interconnect does not allow snooping
CPU
memoryLANconnector
CPU
memoryLANconnector
CPU
memoryLANconnector
![Page 36: Distributed Systemspxk/417/notes/content/01-intro.pdf · more time for protocol processing – ECC, TCP checksums, parsing – Image, audio compression feasible Faster network ®](https://reader033.vdocuments.us/reader033/viewer/2022043008/5f986da3e02f990a610657e0/html5/thumbnails/36.jpg)
Wide Area Distribution
CPU
memoryLANconnector
LANswitch
36
CPU
memoryLANconnector
CPU
memoryLANconnector
CPU
memoryLANconnector
LANswitch
Router
Router
![Page 37: Distributed Systemspxk/417/notes/content/01-intro.pdf · more time for protocol processing – ECC, TCP checksums, parsing – Image, audio compression feasible Faster network ®](https://reader033.vdocuments.us/reader033/viewer/2022043008/5f986da3e02f990a610657e0/html5/thumbnails/37.jpg)
What is a Distributed System?
A collection of independent, autonomous hosts connected through a communication network.
– No shared memory (must use the network)– No shared clock
37
![Page 38: Distributed Systemspxk/417/notes/content/01-intro.pdf · more time for protocol processing – ECC, TCP checksums, parsing – Image, audio compression feasible Faster network ®](https://reader033.vdocuments.us/reader033/viewer/2022043008/5f986da3e02f990a610657e0/html5/thumbnails/38.jpg)
Single System Image
Collection of independent computers that appears as a single system to the user(s)
– Independent = autonomous– Single system: user not aware of distribution
38
![Page 39: Distributed Systemspxk/417/notes/content/01-intro.pdf · more time for protocol processing – ECC, TCP checksums, parsing – Image, audio compression feasible Faster network ®](https://reader033.vdocuments.us/reader033/viewer/2022043008/5f986da3e02f990a610657e0/html5/thumbnails/39.jpg)
You know you have a distributed system when the crash of a computer you’ve never heard of stops you from getting any work done.
– Leslie Lamport
39
![Page 40: Distributed Systemspxk/417/notes/content/01-intro.pdf · more time for protocol processing – ECC, TCP checksums, parsing – Image, audio compression feasible Faster network ®](https://reader033.vdocuments.us/reader033/viewer/2022043008/5f986da3e02f990a610657e0/html5/thumbnails/40.jpg)
Why build distributed systems?
40
![Page 41: Distributed Systemspxk/417/notes/content/01-intro.pdf · more time for protocol processing – ECC, TCP checksums, parsing – Image, audio compression feasible Faster network ®](https://reader033.vdocuments.us/reader033/viewer/2022043008/5f986da3e02f990a610657e0/html5/thumbnails/41.jpg)
How can you get massive performance?• Multiprocessor systems don’t scale
• Example: movie rendering– Disney’s Cars 2 required 11.5 hours to render each frame
(average) – some took 90 hours to render!• 12,500 cores on Dell render blades
– Monsters University required an average of 29 hours per frame• Total time: over 100 million CPU hours• 3,000 to over 5,000 AMD processors; 10 Gbps and 1 Gbps networks
• Google– Over 40,000 search queries per second on average– Index >50 billion web pages– Uses hundreds of thousands of servers to do this
41
![Page 42: Distributed Systemspxk/417/notes/content/01-intro.pdf · more time for protocol processing – ECC, TCP checksums, parsing – Image, audio compression feasible Faster network ®](https://reader033.vdocuments.us/reader033/viewer/2022043008/5f986da3e02f990a610657e0/html5/thumbnails/42.jpg)
Google: Curious facts• In 1999, it took Google one month to crawl and build an index of
about 50 million pages. In 2012, the same task was accomplished in less than one minute.
• 16% to 20% of queries that get asked every day have never been asked before.
• Every query has to travel on average 1,500 miles to a data center and back to return the answer to the user.
• A single Google query uses 1,000 computers in 0.2 seconds to retrieve an answer
42
Source: http://www.internetlivestats.com/google-search-statistics/
![Page 43: Distributed Systemspxk/417/notes/content/01-intro.pdf · more time for protocol processing – ECC, TCP checksums, parsing – Image, audio compression feasible Faster network ®](https://reader033.vdocuments.us/reader033/viewer/2022043008/5f986da3e02f990a610657e0/html5/thumbnails/43.jpg)
Why build distributed systems?• Performance ratio
– Scaling multiprocessors may not be possible or cost effective
• Distributing applications may make sense– ATMs, graphics, remote monitoring
• Interactive communication & entertainment– Work, play, keep in touch:
messaging, photo/video sharing, gaming, telephony
• Remote content– Web browsing, music & video downloads, IPTV, file servers
• Mobility
• Increased reliability
• Incremental growth
43
![Page 44: Distributed Systemspxk/417/notes/content/01-intro.pdf · more time for protocol processing – ECC, TCP checksums, parsing – Image, audio compression feasible Faster network ®](https://reader033.vdocuments.us/reader033/viewer/2022043008/5f986da3e02f990a610657e0/html5/thumbnails/44.jpg)
Design goals: TransparencyHigh level: hide distribution from usersLow level: hide distribution from software
– Location transparencyUsers don’t care where resources are
– Migration transparencyResources move at will
– Replication transparencyUsers cannot tell whether there are copies of resources
– Concurrency transparencyUsers share resources transparently
– Parallelism transparencyOperations take place in parallel without user’s knowledge
44
![Page 45: Distributed Systemspxk/417/notes/content/01-intro.pdf · more time for protocol processing – ECC, TCP checksums, parsing – Image, audio compression feasible Faster network ®](https://reader033.vdocuments.us/reader033/viewer/2022043008/5f986da3e02f990a610657e0/html5/thumbnails/45.jpg)
Design challengesReliability– Availability: fraction of time system
is usable• Achieve with redundancy• But consistency is an issue!
– Reliability: data must not get lost• Includes security
Scalability– Distributable vs. centralized
algorithms– Can we take advantage of having
lots of computers?
Performance– Network latency, replication,
consensus
Programming– Languages & APIs
Network– Disconnect, latency, loss of data
Security– Important but we want convenient
access as well
45
![Page 46: Distributed Systemspxk/417/notes/content/01-intro.pdf · more time for protocol processing – ECC, TCP checksums, parsing – Image, audio compression feasible Faster network ®](https://reader033.vdocuments.us/reader033/viewer/2022043008/5f986da3e02f990a610657e0/html5/thumbnails/46.jpg)
Main themes in distributed systems
• Scalability– Things are easy on a small scale– But on a large scale
• Geographic latency (multiple data centers), administration, dealing with many thousands of systems
• Latency & asynchronous processes– Processes run asynchronously: concurrency– Some messages may take longer to arrive than others
• Availability & fault tolerance– Fraction of time that the system is functioning– Dead systems, dead processes, dead communication links, lost messages
• Security– Authentication, authorization, encryption
46
![Page 47: Distributed Systemspxk/417/notes/content/01-intro.pdf · more time for protocol processing – ECC, TCP checksums, parsing – Image, audio compression feasible Faster network ®](https://reader033.vdocuments.us/reader033/viewer/2022043008/5f986da3e02f990a610657e0/html5/thumbnails/47.jpg)
Key approaches in distributed systems
• Divide & conquer– Break up data sets and have each system work on a small part– Merging results is usually efficient
• Replication– For high availability, caching, and sharing data– Challenge: keep replicas consistent even if systems go down and come up
• Quorum/consensus– Enable a group to reach agreement
47
![Page 48: Distributed Systemspxk/417/notes/content/01-intro.pdf · more time for protocol processing – ECC, TCP checksums, parsing – Image, audio compression feasible Faster network ®](https://reader033.vdocuments.us/reader033/viewer/2022043008/5f986da3e02f990a610657e0/html5/thumbnails/48.jpg)
Service Models (Application Architectures)
48
![Page 49: Distributed Systemspxk/417/notes/content/01-intro.pdf · more time for protocol processing – ECC, TCP checksums, parsing – Image, audio compression feasible Faster network ®](https://reader033.vdocuments.us/reader033/viewer/2022043008/5f986da3e02f990a610657e0/html5/thumbnails/49.jpg)
Centralized model
• No networking
• Traditional time-sharing system
• Single workstation/PC or direct connection of multiple terminals to a computer
• One or several CPUs
• Not easily scalable
• Limiting factor: number of CPUs in system– Contention for same resources (memory, network, devices)
49
![Page 50: Distributed Systemspxk/417/notes/content/01-intro.pdf · more time for protocol processing – ECC, TCP checksums, parsing – Image, audio compression feasible Faster network ®](https://reader033.vdocuments.us/reader033/viewer/2022043008/5f986da3e02f990a610657e0/html5/thumbnails/50.jpg)
Client-Server model
• Clients send requests to servers
• A server is a system that runs a service
• The server is always on and processes requests from clients
• Clients do not communicate with other clients• Examples
– FTP, web, email
50
![Page 51: Distributed Systemspxk/417/notes/content/01-intro.pdf · more time for protocol processing – ECC, TCP checksums, parsing – Image, audio compression feasible Faster network ®](https://reader033.vdocuments.us/reader033/viewer/2022043008/5f986da3e02f990a610657e0/html5/thumbnails/51.jpg)
Layered architectures
• Break functionality into multiple layers
• Each layer handles a specific abstraction– Hides implementation details and specifics of hardware, OS,
network abstractions, data encoding, …
51
Hardware
Operating System
Middleware
Applications
Includes layering forfile systems, networking, devices, memory
Includes naming, security, persistence,notifications, agreement, remote procedures,data encoding, …
![Page 52: Distributed Systemspxk/417/notes/content/01-intro.pdf · more time for protocol processing – ECC, TCP checksums, parsing – Image, audio compression feasible Faster network ®](https://reader033.vdocuments.us/reader033/viewer/2022043008/5f986da3e02f990a610657e0/html5/thumbnails/52.jpg)
Tiered architectures
• Tiered (multi-tier) architectures – distributed systems analogy to a layered architecture
• Each tier (layer)– Runs as a network service– Is accessed by surrounding layers
• The “classic” client-server architecture is a two-tier model– Clients: typically responsible for user interaction– Servers: responsible for back-end services (data access, printing, …)
52
![Page 53: Distributed Systemspxk/417/notes/content/01-intro.pdf · more time for protocol processing – ECC, TCP checksums, parsing – Image, audio compression feasible Faster network ®](https://reader033.vdocuments.us/reader033/viewer/2022043008/5f986da3e02f990a610657e0/html5/thumbnails/53.jpg)
Multi-tier example
53
client middle tier
User interfaceData presentation& validation
• Queuing requests• Coordinating a
transaction among multiple servers
• Managing connections• Formatting/converting
data
back end
• Database system• Legacy software
![Page 54: Distributed Systemspxk/417/notes/content/01-intro.pdf · more time for protocol processing – ECC, TCP checksums, parsing – Image, audio compression feasible Faster network ®](https://reader033.vdocuments.us/reader033/viewer/2022043008/5f986da3e02f990a610657e0/html5/thumbnails/54.jpg)
Multi-tier example
54
client web server
object storeapplication
server
database
![Page 55: Distributed Systemspxk/417/notes/content/01-intro.pdf · more time for protocol processing – ECC, TCP checksums, parsing – Image, audio compression feasible Faster network ®](https://reader033.vdocuments.us/reader033/viewer/2022043008/5f986da3e02f990a610657e0/html5/thumbnails/55.jpg)
Multi-tier example
55
clie
nt
web
ser
ver object
store
appl
icat
ion
serv
er
database
Some tiers may be transparent to the applicationfir
ewal
l
firew
all
load
ba
lanc
er
cach
e
![Page 56: Distributed Systemspxk/417/notes/content/01-intro.pdf · more time for protocol processing – ECC, TCP checksums, parsing – Image, audio compression feasible Faster network ®](https://reader033.vdocuments.us/reader033/viewer/2022043008/5f986da3e02f990a610657e0/html5/thumbnails/56.jpg)
Peer-to-Peer (P2P) Model• No reliance on servers
• Machines (peers) communicate with each other
• Goals– Robustness
• Expect that some systems may be down
– Self-scalability: the system can handle greater workloads as more peers are added
• Examples– BitTorrent, Skype
56
clients servers
peers
![Page 57: Distributed Systemspxk/417/notes/content/01-intro.pdf · more time for protocol processing – ECC, TCP checksums, parsing – Image, audio compression feasible Faster network ®](https://reader033.vdocuments.us/reader033/viewer/2022043008/5f986da3e02f990a610657e0/html5/thumbnails/57.jpg)
Hybrid model
• Many peer-to-peer architectures still rely on a server– Look up, track users– Track content– Coordinate access
• But traffic-intensive workloads are delegated to peers
57
![Page 58: Distributed Systemspxk/417/notes/content/01-intro.pdf · more time for protocol processing – ECC, TCP checksums, parsing – Image, audio compression feasible Faster network ®](https://reader033.vdocuments.us/reader033/viewer/2022043008/5f986da3e02f990a610657e0/html5/thumbnails/58.jpg)
Processor pool model• Collection of CPUs that can be assigned processes on demand
• Render farms
58
![Page 59: Distributed Systemspxk/417/notes/content/01-intro.pdf · more time for protocol processing – ECC, TCP checksums, parsing – Image, audio compression feasible Faster network ®](https://reader033.vdocuments.us/reader033/viewer/2022043008/5f986da3e02f990a610657e0/html5/thumbnails/59.jpg)
Cloud Computing
Resources are provided as a network (Internet) service
– Software as a Service (SaaS)Remotely hosted software• Salesforce.com, Google Apps, Microsoft Office 365
– Infrastructure as a Service (IaaS)Compute + storage + networking• Microsoft Azure, Google Compute Engine, Amazon Web Services
– Platform as a Service (PaaS)Deploy & run web applications without setting up the infrastructure• Google App Engine, AWS Elastic Beanstalk
– StorageRemote file storage• Dropbox, Box, Google Drive, OneDrive, …
59
![Page 60: Distributed Systemspxk/417/notes/content/01-intro.pdf · more time for protocol processing – ECC, TCP checksums, parsing – Image, audio compression feasible Faster network ®](https://reader033.vdocuments.us/reader033/viewer/2022043008/5f986da3e02f990a610657e0/html5/thumbnails/60.jpg)
The end
60September 12, 2016 © 2014-2016 Paul Krzyzanowski