distributed system - upmlaurel.datsi.fi.upm.es/~jmpena/insa/ds-3-communications.pdf– parameter...

Fernando PérezJosé María Peña

Víctor RoblesFrancisco Rosales

Sistemas Operativos Distribuidos

Distributed SystemCommunications

Distributed SystemCommunications



Operating System Support2

Distributed Communication

1. Distributed Communication:• Interaction Models• Communication Primitives • Addressing• Failure Management

2. Distributed Development:• Message Passing• RPCs Approaches• Distributed Objects




Distributed System Communication

Native communication model depends on hardware architecture:– SMPs use shared memory.– Clusters or distributed systems use message passing.

High-level (programming level) communication mechanisms are:– Shared memory areas.– Point-to-point multicast messages– Remote procedure calls or distributed object frameworks.




Shared Memory Communication

Requires:– Uniform or Non-uniform memory access (Hardware).– Operating system functionalities (Software):

• Process management and shared memory allocation.• Thread programming.

– Atomic operations (Hardware/Software):• Mutex and semaphore instructions.

– No special failure management control.




Communication Layers

TCP/UDP

Protocol andRepresentation

RMI/RPCApp./Services

3) Remote procedure callsand distributedobjects.

ATM/Ethernet

Interfaceand

CommunicationLogic

App./Services

2) Low-levelcommunicationfacilities. DistributedOperating Systems.

TCP/UDPAPI (sockets)

Applications&

Services

1) Pure messagepassing. Networkapplications.




Message Passing Communication

Requires:– Network devices (Hardware).– Operating system funcionalities (Software):

• Specific I/O control (network messages).– Network Protocols(Hardware/Software):

• Media access, transport, ...– Special failure management control:

• Partial failures.• Security issues.




Communication Primitives

Elemental communication functions.– send(target,message).– receive(source,message).– connect(target).– close().– ...

Implementation:–An API function. –Network messages.–Status information.

Characteristics:– Blocking vs. Non-blocking. – Synchronous vs. Asynchronous.– Reliable vs. No-reliable.– State vs. stateless




Blocking vs. Non-blocking

The blocking characteristics are:– Blocking primitives: The operation interrupts caller execution until

the operation is “completed”. – Non-blocking primitives: The operation does not stop the

execution. No finalization status information returned.

Their meaning depends on the primitive function:– Non-blocking send: The sender copies the message into kernel

space and continues its execution.– Non-blocking receive: If there is any data available the receiver

reads it, otherwise the execution continues.




Synchronous vs. Asynchronous

These characteristics represents the interaction of two primitives (send/receive) in a communication:

– Synchronous communication: Send and receive must be executed at the same instant..

– Asynchronous communication: Send primitive does not require the receiver to be waiting.

Synchronous communication uses an intermedia buffer either at kernel level or user level..This function can be used to force independent processes to synchronise.




Reliability

A reliable communication guaranties that a message sent is always received. Otherwise the sender is informed.Reliability uses special acknowledgement messages (ACKs).

Reliability is provided either by:– The communication protocol (TCP yes but UDP no).– The communication logic of the sender and receiver.



Sistemas Operativos Distribuidos11

State and Stateless Services

• State vs. Stateless serves:– Indicates whether the server maintains state information about the clients.

• Sate service benefits:– Shortert request messages.– Better performance (information is retained in memory).– Allows optimization strategies:

• Predictive strategies: Analyzing client’s access patterns. • Stateless service benefits:

– Fault tolerance (server restarting).• Selfcontained requests.

– Less number of messages: There is no session start/finish.– Less resource-demanding on server-side (no memory usage)

• Inherently state-based services (distributed locks).• State management using stateless services (HTTP+cookies).




Communication Primitives

SENDERSENDER

SenderKernel

REDRECEIVERRECEIVER

ReceiverKernel

12

7

3

64

8 5

msgACK

• Non-blocking send: [1:8] Sender copies the message into the kernel.

• Blocking send: [1:2:7:8] Sender waits for the kernel to send the message through the network.

• Reliable blocking send: [1:2:3:6:7:8]: Sender waits for the opposite kernel to receive the message.

• Explicit blocking send: [1:2:3:4:5:6:7:8]: Idem, but the application (not the kernel) send the acknowledgement.

• Request-reply: [1:2:3:4:<service>:5:6:7:8]: Sender waits for the receiver to complete the operation requested.




Addressing

Information necessary to identify communication elements. How a message receptor is described.

Mechanisms:– Location-dependent address:

• E.g.: IP address + port number.• Provides no transparency.

– Location-independent address (logical address):• Allows transparency.• Requires a logical address translation mechanism:

– Using broadcast messages.– Using a localisation server (which records all element addresses)..

• Address caches improves performance.




Multicast Communication

Multiple message receivers for a single message.– Requires:

• Group membership control.• Efficient message propagation (Multicast routing).

– Applications:• Fault tolerance due to replicated servers.• Discovery of services in a distributed environment.• Better performance in replicated systems.• Software actualisation.• Collective operations in parallel processing.




Message Passing

A prototype of message passing communication is:

CLIENTmsg

Send(msg)Send(msg)msg

SERVER msg

Receive(msg)

Message msg,reply;msg=<data to be sent>send(msg);receive(reply);if(isOK(reply))<operation is OK>

else<operation error>

...

Message op,ack;receive(op);if(validOp(op))ack=OK

elseack=ERROR

send(ack);...




Representation Formats

Binary-type communication in heterogeneous domain should tackle the following problems:

– Numeric data size and format.– Byte ordering.– Text characters: ASCII vs. EBCDIC.

little-endianArchitecture

Original data: 5

0 0 0 5Value: 0x224+0x216+0x28+5

big-endianArchitecture

Data received: 83.886.080

0 0 0 5Value: 5x224+0x216+0x28+0

0 0 0 5

0123

3210




Remote Procedure Call

...send(msg)...receive(rpy)

...receive(msg)...send(rpy)

Message passing (low-level interaction)

...

...x=do_it(756)...

int do_it(int cod){......return val;

}

Remote procedure calls (higher level) Programming oriented

Clie

ntServer

Clie

ntServer

¿?

msg

rpy

¿?




Remote Procedure Call

RPCs evolution:– Proposed by Birrel & Nelson in 1985.– Sun RPC is the foundation of many common nowaday’s services (NFS

or NIS).– A complete RPC development framework was designed in 1990: DCE

(Distributed Computing Environment) by OSF.– Object oriented perspective since CORBA and JavaRMI. Remore

method invocation.




Overall RPC Execution

Client:– The program calls the procedure.– This call packs the arguments of the invocation in a message and

sends them to the other process.– The process waits the response

Server:– A message with packed argument is received.– These arguments are used to call an internal procedure in the

server.– The result of the procedure is packed and sent back to the client.

Objective: To make remote procedure invocation look like conventional procedure invocation.




Necessary Elements

• Client code.• Server code.• Representation format (XDR).• Interface definition.• Server location.• Failure semantics.

...

...x=do_it(756)...

int do_it(int cod){......return val;

}

Clie

nt

Server




Client Code/Server Code

The abstraction layer elements in a RPC are called stubs.

CLIENT SYSTEM

APPLICATION CODE

PREPAREINPUT

ADAPTOUTPUT

CALLSTART

CALLEND

RPCEXECUTION

LIBRARY

SENDINPUT

RECEIVEOUTPUT

CLIENTSTUB 1

28

9

5

3

PROCEDURES

REMOTEPROCEDUREEXECUTION

SERVER SYSTEM

RECEIVE

SERVERSTUB

PREPAREOUTPUT

SEND BACKOUTPUT

ADAPTINPUT

RPC.EXECUTION

LIBRARY

4

6

7




Stubs

It is automaticly generated by the RPC software tools:– They are independent of client and server implementation.Only

depend on the interface.Tasks performed by the stubs:

– Server look-up.– Parameter packing and message construction.– Send messages.– Wait for message reception and unpacking the results.

The common issues are implemented in a RPC library.




Representation Format

One of the functions performed by the stubs is packing the arguments: marshalling.

Representation problems:– Client and server might be executed on different hardware

architectures..– XDR (external data representation) an estándar to represent different

data types.– Types of parameters (in/out):

• Pointer parameters: An address has a meaning only inside a specific address space.




Interface Definition

IDL (Interface Definition Language) is alangage to represent anddesribe remote interfaces:

– Many IDL variants:• Integrated with the program language (Cedar, Argus).• Specific for interface defintion (Sun’s RPC and RPC of DCE).

– Define procedures and argument types (Not implementation).– It is the foundation of automatic stub generation.

Server TicketServer{

procedure void reset();procedure int getTicket(in string ident);procedure bool isValid(in int ticket);

}




Communication Errors

• The server could be down.• The client and server uses different service versions.• How a client is notified of an error:

– Returning an error code (-1)• It is not transparent

– E.g.: add(a,b)– Throwing an exception

• Requires a language that handles exceptions.




Error Semantics

Message from client to server lost

Time-out: If there is no response then replay.

Reliable protocol solves this.

Message from server to client lost

Time-out (from client): How long is this time out? DANGEROUS

Some operations are re-done.

Request number and reply bit.

Client fails

Request submitted but the client dies before reply.

Computation time wasted (server).

Client re-staring could be hazarous.

Server fails

Server dies before or after execute the operations (CLIENT DOESN’T KNOW).

Semantics: At leas once (1+), At most once (0-1), Exactly once (IDEAL).




Middleware Environments: CORBA

Part I




Middleware Environments

1. From RPCs to RMIs:• Remote methods• Distributed objects

2. Distributed Objects Frameworks:• OMG’s CORBA• Microsoft’s COM+ / .NET• Java Distributed Computing

3. CORBA:• CORBA architecture/elements• Objects adaptor• Communication protocols




From RPCs to RMIs

The extension of remote procedure calls to an object-oriented framework develops what is called remote method invocation (RMI).

RMIs provide:– Remote methods are assigned to the concept of distributed objects.– More appropriate to object oriented development.– Allow event-driven development.

New issues:– Remote object reference.– Volatile and persistent objects.




Distributed Objects

Characteristics:– Middleware: Abstraction layer. It provides communication for

distributed objects. Abstraction form:• Object location.• Communication protocols.• Hardware.• Operating system.

– Distributed object model: Represents the elements of the object-oriented paradigm accepted by the technology: Inheritance, Interfaces, Exceptions, Polymorphism, ...

– Garbage Collection: Selects unused objects (following object references) and releases their resources.




Distributed Objects Technologies

– ANSA (1989-1991) first project to propose an object-oriented formalisms to represent distributed systems.

– Microsoft’s DCOM, COM+ and .NET.– OMG’s CORBA.– Java Technologies from Sun Microsytems:

• Remote Method Invocation (RMI).• Enterprise Java Beans (EJB).• Jini.

– Other proprietary technologies.




CORBA

Standard developed by the Object Management Group (OMG)

OMG: A group of companies that work together on the definition of standards for heterogeneous environment interoperability.

– OMG was funded in 1989– At the present moment there are 800+ companies as members of

OMG.




OMA

(Object Management Architecture)

Abstract architecture used as reference to define distributed applications on a heterogeneous domain. CORBA is the technology related to this generic architecture..

CORBA specification is divided in several formal models:– Object Model– Interaction Model– ...




ORB

Services Common Facilities

Domain Interfaces

Applications

OMA

An application in OMA is a set of distributed collaborative objects. These objects are divided into different groups:




Servervoid withdraw(long){

....

.... }

Clientx->withdraw(30)

ORB

ORB

The communication between any pair of objects is based on the ORB operations. An elemental operation scenario is:




CORBA IDL

(Interface Definition Language)This language is used to describe the methods a distributed object offers to other elements of the system.

interface Account{

void deposit(in long amount);void withdraw(in long amount);long balance();

};




CORBA IDL

Language Mappings:– Translate an IDL description into another programming language:

• C++, • Ada, • COBOL, • SmallTalk, • Java,• ...

– This translation generates two code pieces called Stub and Skeleton. These code modules represent client and server communicationissues.




CORBA IDL

The client code (stub) inludesmarshalling operations.

Marshalling: Translation of the methodparameters into a commonrepresentation forORBcommunication.

interface Account

{

void withdraw(in long amount);

}

Account.idl

IDL Compiler

Account_Skel.c++

class Account_Stub : ...

{

void withdraw(CORBA::Long &amount)

{ <MARSHALLING> }

}

Account_Stub.c++




CORBA IDL

The IDL-generated server code(skeleton) performs the oppositeoperation (de-marshalling).

De-marshalling: Unpaking callerparameters from the message andbuilding back the procedure call.

IDL Compiler

Account_Stub.c++

class Account_Skel : ...

{

virtual

void withdraw(CORBA::Long &account)=0;

void __withdraw()

{

<DE-MARSHALLING>

withdraw(c);

}

}

Account_Skel.c++




CORBA IDL

Object Implementation:– The actual object implementation is summoned by the Skeleton

methods. It is used to be a derived class of this skeleton.

class Account_Impl: public Account_Skel{

void withdraw(CORBA::Long &amount){

money += amount;}

};




ORB ORB

DII Stub ORBInterface

ORBInterface

Skel. DSI

Object Adapter (OA)

Client Server Object

ORB Components

The complete communication architecture in CORBA has the following elements:




ORB Components

Stub:– Client code that represents the remote object to be called. It emulates

the remote code methods invoked by the client. Stubs perform communication functions instead of the actual method calls.

Skeleton:– Server code of the object. It has the opposite role of the stub. It

emulates client actions for remote method calls. It performs local method calls to the actual object implementation.




ORB Components

DII:– (Dynamic Invocation Interface)– It is an alternative to static stub usage that allows clients to request

server operations that were unknown during compilation phase.

DSI:– (Dynamic Skeleton Interface)– It is a dynamic alternative to static skeletons when they are defined

during compilation phase. This feature is useful for server thatincludes new services during execution phase.




ORB Components

ORB and ORB Interface:– This component deals with interconnection issues between client and

server spaces (& many other functions). This element hides different PS and hardware differences.

– There are the same features provided for both client and server sides.The two main ORB tasks are:

– Object location: The client does not know where remote object computer is.

– Communication between client and server: Independently of protocol and implementation issues (language, operating system, ...)




ORB Component

Object Adapter:– Remote object use this element to be registered in, This element

keep all the object references inside the same service space (samenode). The object adapter is able to redirect remote methodinvocations from clients to the appropriate skeletons.

There are two object adapters specified by OMG:– BOA: (Basic Object Adapter).– POA: (Portable Object Adapter).




ORB Components

The main object adapter tasks are:– Two-fold multiplexing (per object and per method). – To keep remote object implementation information (stored into an

Implementation Repository). This element is in charge of the object activation (if it isn’t active).

– To provide different activation modes:• Persistent: Object status is kept between two executions.• Shared: All the clients share the same object.• Un-shared: Each client has its own object implementation.• Per-method: Each method call activates a new object.

– To generate object references inside the environment. This object reference is unique.




ORB ORB


ORBInterface

Skel. DSI

Object Adapter (OA)


CORBA Communication

Communication mechanism:1- Client calls stub method. Client stub marshals the parameters. 2- Client stub uses ORB sending to addressed the request to the remote object.3- Server ORB receives the request and forwards it to assigned Object Adapter (it

could be more than one).

1

2 3

4

5

6

7




CORBA Communication4- The Object Adapter solves which object is referred to and which of its method is

requested.5- Server skeleton de-marshals request arguments and calls the actual object.6- Object implementation performs the operation and the result (and any out/in-out

parameter) are sent back to the skeleton.7- Result reply is analogous.

ORB ORB


ORBInterface

Skel. DSI

Object Adapter (OA)


1

2 3

4

5

6

7




ORB Implementation

ORB is a logical bus that glues objects and clients. The implementation options for this element are:

– Inside client/server: Additional code that both client and server link.– System daemon: A system service that centralised the operations.– Inside OS: As a part of the OS kernel.– Library: When client and server shares the same memory space

(same computer and process).

distributed system - upmlaurel.datsi.fi.upm.es/~jmpena/insa/ds-3-communications.pdf– parameter...

Documents