distributed systems - tiedekunnat · distributed systems principles and paradigms chapter 10...

Distributed SystemsPrinciples and Paradigms

Chapter 10(version April 7, 2008)

Maarten van Steen

Vrije Universiteit Amsterdam, Faculty of ScienceDept. Mathematics and Computer Science

Room R4.20. Tel: (020) 598 7784E-mail:[email protected], URL: www.cs.vu.nl/∼steen/

01 Introduction02 Architectures03 Processes04 Communication05 Naming06 Synchronization07 Consistency and Replication08 Fault Tolerance09 Security10 Distributed Object-Based Systems11 Distributed File Systems12 Distributed Web-Based Systems13 Distributed Coordination-Based Systems

00 – 1 /

Remote Distributed Objects (1/2)

• Data and operations encapsulated in an object• Operations are implemented as methods , and

are accessible through interfaces• Object offers only its interface to clients• Object server is responsible for a collection of

objects• Client stub (proxy) implements interface• Server skeleton handles (un)marshaling and ob-

ject invocation

Server machine

Object

Client machine

Proxy

Sameinterfaceas object

Interface

State

MethodClientinvokesa method

Network

Skeletoninvokessame methodat object

Marshalled invocationis passed across network

Client OS Server OS

Server

Skeleton

Client

10 – 1 Distributed Object-Based Systems/10.1 Architecture

Remote Distributed Objects (2/2)

Compile-time objects: Language-level objects, fromwhich proxy and skeletons are automatically gener-ated.

Runtime objects: Can be implemented in any lan-guage, but require use of an object adapter that makesthe implementation appear as an object.

Transient objects: live only by virtue of a server: ifthe server exits, so will the object.

Persistent objects: live independently from a server:if a server exits, the object’s state and code remain(passively) on disk.


Example: Enterprise Java Beans

What is it: Java object hosted by special server thatallows for different means of calling the object by re-mote clients.

Local OS

Network

Server kernelJM

S

JND

I

JDB

C

RM

IEJBEJBEJB

Container

Server

Services


Types of EJBs

• Stateless session bean: Transient object, calledonce, does its work and is done. Example: exe-cute an SQL query and return result to caller.

• Stateful session bean: Transient object, but main-tains client-related state until the end of a session.Example: shopping cart.

• Entity bean: Persistent, stateful object, can beinvoked during different sessions. Example: ob-ject maintaining client info on last number of ses-sions.

• Message-driven bean: Reactive objects, oftentriggered by message types. Used to implementpublish/subscribe forms of communication.


Globe Distributed Objects (1/2)

Observation: Most distributed objects are not dis-tributed at all: state is kept at a single node. Alterna-tive: Globe objects, which are physically distributedacross multiple machines.

Local object

Distributed shared object

Process

Interface

Network


Globe Distributed Objects (2/2)

Note: To make DSOs generic, we need to separatefunction from distribution support:

Controlsubobject

Replicationsubobject

Semanticssubobject

Communicationsubobject

Communicationwith other local

objects

Same interface as implementedby semantics subobject

Note: replication subobject essentially decides howand when the local semantics subobject will be in-voked.10 – 6 Distributed Object-Based Systems/10.1 Architecture

Processes: Object Servers (1/2)

Servant: The actual implementation of an object, some-times containing only method implementations:

• Collection of C or COBOL functions, that act onstructs, records, database tables, etc.

• Java or C++ classes

Skeleton: Server-side stub for handling network I/O:

• Unmarshalls incoming requests, and calls the ap-propriate servant code

• Marshalls results and sends reply message• Generated from interface specifications

Object adapter: The “manager” of a set of objects:

• Inspects (as first) incoming requests• Ensures referenced object is activated (requires

identification of servant)• Passes request to appropriate skeleton, following

specific activation policy• Responsible for generating object references

10 – 7 Distributed Object-Based Systems/10.2 Processes

Processes: Object Servers (2/2)

Local OS

Requestdemultiplexer

Object adapter

Object's stub(skeleton)

Server with three objectsServer machine

Object adapter

Observation: Object servers determine how their ob-jects are constructed


Example: Ice

Important: Ice aims at simplicity , and achieves thispartly by putting policies into the middleware.

main(int argc, char* argv[]) {Ice::Communicator ic;Ice::ObjectAdapter adapter;Ice::Object object;ic = Ice::initialize(argc, argv);

adapter = ic->createObjectAdapterWithEndPoints( "MyAdapter","tcp -p 10000");

object = new MyObject;

adapter->add(object, objectID);adapter->activate();

ic->waitForShutdown();}

Note: Activation policies can be changed by modify-ing the properties attribute of an adapter.


Client-to-Object Binding (1/2)

Object reference: Having an object reference allowsa client to bind to an object:

• Reference denotes server, object, and communi-cation protocol

• Client loads associated stub code• Stub is instantiated and initialized for specific ob-

ject

Two ways of binding:

• Implicit: Invoke methods directly on the refer-enced object

• Explicit: Client must first explicitly bind to objectbefore invoking it

10 – 10 Distributed Object-Based Systems/10.3 Communication

Client-to-Object Binding (2/2)

Distr_object* obj_ref;obj_ref = ...;obj_ref->do_something();

Implicit

Distr_object* obj_ref;Local_object* obj_ptr;obj_ref = ...;obj_ptr = bind(obj_ref);obj_ptr->do_something();

Explicit

Some remarks:

• Reference may contain a URL pointing to an im-plementation file

• (Server,object) pair is enough to locate target ob-ject

• We need only a standard protocol for loading andinstantiating code

Observation: Remote-object references allow us topass references as parameters. This was difficult withordinary RPCs.


Remote Method Invocation

Basics: (Assume client stub and server skeleton arein place)

• Client invokes method at stub• Stub marshals request and sends it to server• Server ensures referenced object is active:

– Create separate process to hold object– Load the object into server process– ...

• Request is unmarshaled by object’s skeleton, andreferenced method is invoked

• If request contained an object reference, invoca-tion is applied recursively (i.e., server acts as client)

• Result is marshaled and passed back to client• Client stub unmarshals reply and passes result to

client application


RMI: Parameter Passing (1/2)

Object reference: Much easier than in the case ofRPC:

• Server can simply bind to referenced object, andinvoke methods

• Unbind when referenced object is no longer needed

Object-by-value: A client may also pass a completeobject as parameter value:

• An object has to be marshaled:– Marshall its state– Marshall its methods, or give a reference to

where an implementation can be found• Server unmarshals object. Note that we have now

created a copy of the original object.• Object-by-value passing tends to introduce nasty

problems


RMI: Parameter Passing (2/2)

Local objectO1

Copy of O1

Remote objectO2

Localreference L1

New localreference

Remotereference R1

Remoteinvocation withL1 and R1 asparameters

Copy of R1 to O2

Machine A Machine B

Machine C

Client code withRMI to server at C(proxy)

Server code(method implementation)

Note: Systemwide object reference generally containsserver address, port to which adapter listens, and lo-cal object ID.

Extra’s: Information on protocol between client andserver (TCP, UDP, SOAP, etc.)

Question: What’s an alternative implementation for aremote-object reference?


Object-Based Messaging

Messaging facilities: reliable asynchronous and per-sistent method invocations:

Clientproxy

Callbackinterface

ClientRTS

Client application

2. Request to server

4. Call by the RTS

1. Call by theapplication

3. Response from server

Clientproxy

Pollinginterface

ClientRTS

Client application

2. Request to server


3. Response from server



Object References (1/2)

Observation: In order to invoke remote objects, weneed a means to uniquely refer to them. Example:CORBA object references:

Repositoryidentifier

IIOPversion Host Port Object key Components

ProfileID

Tagged Profile

Objectidentifier

Adapteridentifier

Other server-specific information

Profile

Interoperable Object Reference (IOR)

10 – 16 Distributed Object-Based Systems/10.4 Naming

Object References (2/2)

Observation: It is not important how object refer-ences are implemented per object-based system, aslong as there is a standard to exchange them betweensystems:

Object system A Object system B

Object server Interoperable references

(Half) gateway

Solution: Object references passed from one RTS toanother are transformed by the bridge through whichthey pass (different transformation schemes can beimplemented)

Observation: Passing an object reference refA fromRTS A to RTS B circumventing the A-to-B bridge maybe useless if RTS B doesn’t understand refA


Globe Object References

Important: Globe references are location indepen-dent , in contrast to CORBA’s IORs.

Stacked address: Stack of addresses representingthe protocol to speak:

Field DescriptionProtocol ID Constant representing a (known) protocolProtocol addr. Protocol-specific addressImpl. handle Reference to a file in a repository

Instance address: Contains all that is needed to talkin a propritary way to an object:

Field DescriptionImpl. handle Reference to a file in a repositoryInitialization string Used to initialize an implementation


Synchronization

Issue: Client stubs hide many of the interactions withobjects, which may not always be a good idea:

Object Object

Process ProcessLock

Unlock

(a) (b)

Other problem: Should we have server-side or client-side synchronization:

• Server: Problems when client crashes• Clients: Problems when clients at different ma-

chines need to synchronize

Java: Clients at same machine are automatically syn-chronized; those from different machines are not.10 – 19 Distributed Object-Based Systems/10.5 Synchronization

Consistency and Replication

Observation: Objects form a natural means for real-izing entry consistency :

• Data are grouped into units, and protected by asynchronization variable (i.e., lock )

• Synchronization variables adhere to sequentialconsistency (i.e., values are set atomically)

• Operations of grouped data can be nicely grouped:object

Problem: What happens when objects are replicated?One way or the other we need to ensure that opera-tions on replicated objects are properly ordered.

10 – 20 Distributed Object-Based Systems/10.6 Consistency and Replication

Replicated Objects

Problem: We need to make sure that requests areordered correctly at the servers and that threads aredeterministically sheduled :

Middleware

Local OS

Threads

Unordered requests

Totally ordered requests

Middleware

Local OS

ThreadsThread

scheduler

Deterministic thread scheduling

Unordered requests

Object

Computer 1 Computer 2

T11 T1

2 T21 T2

2

Observation: We are dealing with nasty issues here.Simplicity may dictate completely serialized (i.e., single-threaded) executions at the server.


Replicated Invocations (1/2)

Active replication: Updates are forwarded to multi-ple replicas, where they are carried out. There aresome problems to deal with in the face of replicatedinvocations :

Client replicatesinvocation request

All replicas seethe same invocation

Object receivesthe same invocationthree times

Replicated object

A

B1

B2

B3

C


Replicated Invocations (2/2)

Solution: Assign a coordinator on each side (clientand server), which ensures that only one invocation,and one reply is sent:

Coordinatorof object B

Result

Coordinatorof object C

(a) (b)

Client replicatesinvocation request

B1 B1

B2 B2

B3 B3

C1 C1

C2 C2

A A

Result


Fault Tolerance

Observation: Nothing really new to report – objectsystems handle fault tolerance as you would expectthem to do.

Example: CORBA uses special interoperable objectgroup references to allow a client to access replicas:

Repositoryidentifier

IIOPver.

IIOPver.Host-1 Host-NPort-1 Port-N

Objectkey-1

Objectkey-NComponents Components

ProfileID

ProfileID

TAGPRIMARY

TAGBACKUP

Other group-specific information

Other group-specific information

Profile-1 Profile-N

Interoperable Object Group Reference (IOGR)

10 – 24 Distributed Object-Based Systems/10.7 Fault Tolerance

CORBA FT: Architecture

Replicationmanager

Object groupmanager

Propertymanager

Middleware/RTS

Object

Interceptor

Replication

Reliable multicasting

Logging &Recovery

Log

To other replicas

Client orobject server

10 – 25 Distributed Object-Based Systems/10.7 Fault Tolerance

Security

When objects can (also) be moved/copied around:We need to consisder three different security aspects:

• Secure binding : is the binding between clientand object authorized?

• Secure invocation : is the bounded client autho-rized to invoke a specific method?

• Platform security : is the object protected againstthe underlying platform and vice versa?

Example: For Globe we have the following public/privatekey pairs:

• Object key : Owner of the private key can set ac-cess rights for users and replica servers.

• Replica key : Used to make sure that a replicaserver is hosting an object.

• User key : Associated with every user.

10 – 26 Distributed Object-Based Systems/10.8 Security

Globe security

sig(O, {R,U,D, })

User certificate

U:

K+Alice

K+Alicesig(O, {U, })

Replica certificate

K+Repl K+

Adm

K+Replsig(O, {R, })

K+Adm

00 00 01 111 0

U:01 00 11 111 1

D:1

11 00 00 111 0R: 11 11 00 111 0R:

Administrative certificate

(c)(b)(a)

• User certificate : U[i] = 1 iff user is allowed toinvoke method Mi.

• Replica certificate : R[i] = 1 iff replica is allowedto execute Mi.

• Administrative certificate : U[i] = 1 iff a usercertificate for Mi may be issued. Same for R[i].


Secure Method Invocation

Check user permissions

Request secure channel

Find suitable replica

Establish secure channelEncrypt/sign

network packet

Decrypt & authenticate

request

Authorize request

Execute operation

3

2

4

8 9

11

12

13

107

1Request invocation

5

6

Marshall request

Pass request to channel

Client-side stub Server-side replica

Security object

Security object

Control object

Control object

Repl. object

Repl. object

Comm. object

Comm. object

Semant. object

Unmarshall

Pass request

1: Application issues an invocation request.

2: Control subobject checks user permissions; security objectshould have valid user certificate.

3: Request is marshaled and passed on.

4: Replication subobject requests middleware to set up securechannel to suitable replica.

5: Security object initiates a replica lookup.







network packet


request

Authorize request

Execute operation

3

2

4

8 9

11

12

13

107

1Request invocation

5

6

Marshall request



Security object

Security object

Control object

Control object

Repl. object

Repl. object

Comm. object

Comm. object

Semant. object

Unmarshall

Pass request

6: Security subobject sets up secure channel with its peer.The replica must prove it is allowed to carry out the re-quested invocation.

7: Request is passed on to comm. subobject.

8: Comm. subobject encrypts and signs the request.

9: Request is decrypted and authenticated.

10: Request is passed to server-side repl. subobject.







network packet


request

Authorize request

Execute operation

3

2

4

8 9

11

12

13

107

1Request invocation

5

6

Marshall request



Security object

Security object

Control object

Control object

Repl. object

Repl. object

Comm. object

Comm. object

Semant. object

Unmarshall

Pass request

11: Authorization takes place; user certificate from client-sidestub has been passed to replica to verify that request canbe carried out.

12: Request is unmarshaled.

13: Operation can be executed.


distributed systems - tiedekunnat · distributed systems principles and paradigms chapter 10...

Documents