University of Puerto Rico at MayaguezElectrical and Computer Engineering Department

Lecture 1Introduction

Prof. Wilson Rivera

Parallel and Distributed Computing Laboratory @ UPRM

Outline

Definition of distributed systems

Goals of distributed systems

Transparency

Scalability and Scaling techniques

System architecturesMultiprocessor

Multicomputer» Cluster configurations» Cloud architecture

Client-server model

Multitier model

Parallel and Distributed Computing Laboratory @ UPRM

Definition of a Distributed System (1)

A distributed system is:

A collection of independent computers that appears to its users as a single coherent system.

Parallel and Distributed Computing Laboratory @ UPRM

Definition of a Distributed System (2)

A distributed system organized as middleware.Note that the middleware layer extends over multiple machines.

Parallel and Distributed Computing Laboratory @ UPRM

Goals of Distributed Systems

Connecting users and resourcesSecurity

TransparencyHigh degree of transparency vs. performance of the system

OpennessAn open distributed system offers services according to standardrules that describes syntax and semantics of those services

Complete and neutral semantic specifications

Interoperability and portability

ScalabilitySize, geographically, management

Parallel and Distributed Computing Laboratory @ UPRM

Transparency in a Distributed System

Different forms of transparency in a distributed system.

Hide whether a (software) resource is in memory or on diskPersistence

Hide the failure and recovery of a resourceFailure

Hide that a resource may be shared by several competitive usersConcurrency

Hide that a resource may have multiple copies of data Replication

Hide that a resource may be moved to another location while in useRelocation

Hide that a resource may move to another locationMigration

Hide where a resource is locatedLocation

Hide differences in data representation and how a resource is accessedAccess

DescriptionTransparency

Parallel and Distributed Computing Laboratory @ UPRM

Scalability Problems

Examples of scalability limitations.

Doing routing based on complete informationCentralized algorithms

A single on-line telephone bookCentralized data

A single server for all usersCentralized services

ExampleConcept

Parallel and Distributed Computing Laboratory @ UPRM

Scaling Techniques (1)

Hiding communication latencies Asynchronous communication (batch processing)

Shipping code (interactive applications)

DistributionDistributed algorithms

Domain Name System

Replication and CachingReplication Resource

Caching Client

Parallel and Distributed Computing Laboratory @ UPRM

Scaling Techniques (2)

The difference between letting:

a) a server or

b) a client check forms as they are being filled

Parallel and Distributed Computing Laboratory @ UPRM

Scaling Techniques (3)

Distributed algorithms No machine has complete info about the system

Machines make decisions based only on local info

Failure of one machine should not affect the others

There is not implicit assumption that a global clock exists

Parallel and Distributed Computing Laboratory @ UPRM

Scaling Techniques (4)

1.5

An example of dividing the DNS name space into zones.

Parallel and Distributed Computing Laboratory @ UPRM

Scaling Techniques (5)

Scalability

Replication

Consistency

Parallel and Distributed Computing Laboratory @ UPRM

System Architectures (1)

Parallel and Distributed Computing Laboratory @ UPRM

System Architectures (2)

Multiprocessors (Shared Memory)SMP (Symmetric Multi Processor)

NUMA (Non Uniform Memory Access)

Multicomputers (Distributed Memory)Homogeneous multicomputers

Heterogeneous multicomputers

Parallel and Distributed Computing Laboratory @ UPRM

Multiprocessors (1)

•Bus overload (scalability)

•Cache coherence

Parallel and Distributed Computing Laboratory @ UPRM

Multiprocessors (2)

a) A crossbar switchb) An omega switching network

Parallel and Distributed Computing Laboratory @ UPRM

Cluster Configuration

Head Node

Node 1

Node 2

Node n

Switch Local StorageExternal Storage

Parallel and Distributed Computing Laboratory @ UPRM

Cloud LayersInfrastructure as a Service (IaaS)

» Flexible in terms of the applications to be hosted » Hard to provide automatic scalability and failover because the

semantics associated to replication and other state management issues are highly application dependent

» e.g. Amazon EC2, RackSpace, Nimbus, EucalyptusPlatform as a Service (PaaS)

» Application domain-specific platforms» e.g. Google App Engine for example is targeted exclusively at

request-reply web based applications» MS Azure, Heroku

Software as a Service (SaaS)» Service domain-specific» e.g. Salesforce, NetSuite

Parallel and Distributed Computing Laboratory @ UPRM

Cloud Architecture

Servers

Physical Resources

Storage Network Sensors

Virtual Servers

Virtual Resources

Virtual Storage Virtual Network

CoordinationInformation Sharing

Infrastructure Services

Infrastructure Management Virtualization

SharedFile Systems

Application Services

Distributed Processing

ScalableDatabases

Resource Provisioning

Security Mechanisms

Parallel and Distributed Computing Laboratory @ UPRM

Clients-Server model

Parallel and Distributed Computing Laboratory @ UPRM

Processing Level

The general organization of an Internet search engine into three different layers

Parallel and Distributed Computing Laboratory @ UPRM

Multi-tiered Architectures (1)

Alternative client-server organizations (a) – (e).

Parallel and Distributed Computing Laboratory @ UPRM

Multi-tiered Architectures (2)

An example of a server acting as a client.