a group selection pattern applied to grid

8/3/2019 A Group Selection Pattern Applied to Grid

1/33

A GROUP SELECTION PATTERN APPLIED TO GRID RESOURCE

MANAGEMENT

T.Arun Prakash and M.Abdul Rahman

Kalasalingam University

Anand Nagar

Krishnankoil 626190

[email protected]

ABSTRACT

A key challenge in Grid computing is the achievement of efficient and self-

organized resource management. Grids are often large scale, heterogeneous,

and unpredictable systems. Introducing group structures can help to distribute

coordination efforts, but higher levels of adaptation and learning in thecoordination protocols are still required in order to cope with system complexity.

We provide a solution based on a self organized and emergent mechanism

evolving congregations of resource management agents through a Group

Selection process which maximizes utility outcomes for system-wide

performance. We provide a formalization of this process into a Group Selection

pattern, and we propose several instantiations optimizing Grid resource

management scenarios such as adaptive job scheduling, market-based resource

management, and policy coordination in Virtual Organizations (VOs). We further

evaluate by simulation the performance of the mechanism in those scenarios.

The results support the conclusion that Group Selection optimizes coordination

by evolving small and dynamic groups.


2/33

INTRODUCTION

The popularity of Grids has been growing very rapidly, driven by the promise that

they will enable knowledge and computing resources to be delivered to and used

by citizens and organizations as traditional utilities or in novel forms. Contrarily to

other distributed systems, Grids have many independent resource providers with

varying resource characteristics and availability. In addition to large sizes, the

dynamicity of resources leads to a very complex coordination task that cannot be

handled manually by users. Automatic and adaptive resource management isproposed as a solution to these challenges. To that aim, the potential synergies

between Grids and multi-agent systems (MAS) have been outlined. Among

others, MAS have exploited group structure to partition the population in

interaction groups, heavily impacting the coordination.

Group Selection refers to a process of natural selection that favors

characteristics in individuals that increase the fitness of the group the individuals

belong relative to other groups. Group Selection implies that every member of

the group depends on a group characteristic that is not isolated in a single

individual. Such groups form isolated niches where the sub-populations are

allowed to evolve behaviors independently of the rest of the populations. The

existence of niches maintains a large diversity in an evolving population since the

evolutionary paths in separated niches may develop in entirely different ways.

This evolutionary process can promote the evolution of individual agents

characteristics benefiting the group the agent belongs to. In several coordination

scenarios, inter-agents miss-coordination (due to ignorance, high complexity, or

other reasons) might be leading to very suboptimal social welfare outcomes.

Group Selection guides the co-evolution of group-structured systems to

optimized configurations.

2


3/33

In this paper, we provide a decentralized coordination mechanism for Grid

resource management based on Group Selection. We distribute the agents in

groups. Further agents group migrations through a Group Selection process

improve the overall population performance. We propose formalization into an

engineering pattern and we show by simulation how it performs optimizing

several resource management tasks in Grids: learning agents for adaptive job

scheduling, bargaining agents in a market-based resource allocation, and policy

coordination in VOs.

STATE OF THE ART IN MAS BASED GRID RESOURCE

MANAGEMENT

We detail in this section the state of the art in MAS based Grid resource

management mechanisms, leaving for the next section the state of the art in

group formation mechanism in MAS and a comparison to the Group Selection

mechanism.

Conventional solutions in Grid resource management apply manual management

or, at most, centralized mechanisms ensuring a predictable outcome. This

normally leads to important scalability limitations due to increased manual and

computational costs, as well as limited tolerance to changes or failure (human

operators and centralized management offer a single failure point in the system).

In Grid scheduling, conventional parallel computer schedulers, such as PBS and

LSF, address the scheduling problem by implementing a synchronous

schedule/enactment process. The scheduling algorithm has full knowledge of the

resource properties. This solution is not appropriate in a federated environment

where no central entity has sole authority over local resource states. The Condor

high-throughput system avoids this problem by providing almost no delivery

guarantees; a centralized matchmaker makes simple greedy decisions to place

3


4/33

jobs and simply retries when it encounters resource states in conflict with its

internal view of the environment. The Pegasus Grid workflow mapping system

includes a random job placement behavior to spread small parts of an overall

workflow into Grid resources.

Coordination is an important topic in large scale Grid systems management. In,

different coordination mechanisms for Grids are evaluated. From lower to higher

level, the following mechanisms are proposed: scripting languages, shared-data

spaces and middleware agents, the later being identified as the more flexible.

However none of these solutions incorporate the notion of self-organization

capturing systems dynamics as our mechanism does. A comprehensive survey

on self-organizing agent-based and autonomic computing applications in Grid

Computing is presented in. Market based resource allocation has received a

great deal of attention in the last years. The GridBus Project is a reference in

Grid Economy and utility based computing, and has proposed a great variety of

market models and tools for the trading of Grid Resources. However, its strong

emphasis on computational intensive Grids and the hierarchical nature of some

of the proposed components, like the Grid Market Directory, diverges from the

fully decentralized resource allocation mechanisms used here. Other centralizedapproaches exist such as, but scalability issues both in size and computational

requirements further complicate its applicability to large size Grids. Tycoon is a

market-based system for managing compute resources in distributed clusters or

Grids. It uses distributed auctions with users having a limited amount of credits.

Users who provide resources can, in turn, spend their earnings to use resources

later. A fully decentralized approach is the one adopted with the catallactic

agents. In this approach, bilateral negotiations are established between learning

agents, and the spontaneous price coordination arises from both the bargaining

and co-evolutionary learning processes. However, the bootstrapping and

evolution of these bargaining agents in markets is an open issue.

4


5/33

Grids environments are being organized in VOs, associating heterogeneous

users and resource providers to coordinate resource sharing; a set of individuals

and/or institutions defined by their sharing rules form a Virtual Organization.

There are many projects using VOs conceptually, but very few projects are

addressing the management of VOs themselves in Grids. While the notion of a

VO seems to be intuitive and natural, we still lack well-defined procedures for

deciding when a new VO should be formed, who should be in that VO, what they

should do, when the VO should be changed, and when the VO should ultimately

be disbanded. In Conoise-G project an agent system supporting robust and

resilient VOs formation and operation is presented. Another project focusing on

Trust issues is Trustcom aiming to provide a trust and contract management

framework enabling the definition and secure enactment of collaborative

business processes within VOs that are formed on-demand, self managing and

evolve dynamically. In both Conoise-G and Trustcom approaches to VO

management, components for helping automated VO management are

developed.

STATE OF THE ART IN GROUP FORMATION

MECHANISMS FOR MAS

Exploiting group structure in MAS has been proposed in previous research. It is

acknowledged that, in general, there is no single type of organizational paradigm

that is suitable for all situations. Some claim that there is not even perfect

organization in any situation, due to the inevitable tradeoffs that must be made

and the uncertainty, lack of global coherence and dynamics present in any

realistic population. In hierarchies the data produced by lower level agents

travels upwards to provide a broader view, while control flows downwards as the

higher level agents provide direction to those below. Holarchies follow a very

similar pattern, but typically include homogeneous, self-similar elements at each

hierarchical level, and the constituents of each group are partially-autonomous.

5


6/33


7/33

supposing complete system knowledge to the central coalition-maker, with few

exceptions. This panorama contrast with realistic present day large scale

distributed systems, where small, decentralized components need to deal

autonomously with coordinated decision making.

Group Selection is a fully decentralized mechanism which focuses on the

dynamic view of the groups, iteratively ruling its evolution towards more optimal

configurations. It has been shown that Group Selection can lead to the spread of

group beneficial characteristics in many different grouped settings of agents

populations. This enables the application of Group Selection processes in any

group-like structured agent population. Applications of Group Selection have

appeared in biology and sociology and also in economic theory. In engineering,

novel socially-inspired mechanism have been developed building to some extent

on Group Selection processes. These mechanisms use a tag (or social label) to

identify groups. Agents interactions are biased by Tags (i.e. within the groups)

and inter-group migrations are ruled by Group Selection processes. This has

been demonstrated most notably in the application to free-riding prevention in

Peer-to-Peer networks by Hales. Also other applications build on Tag

mechanisms such as query routing and processing for Peer-to-Peer web search.Table I compares Group Selection with the rest of group formation mechanism

reviewed. It can be seen that Group Selection potentially addresses several

important issues in existent group formation mechanisms.

TABLE I. GROUP FORMATION MECHANISMS COMPARISON

MECHANISM BENEFITS DRAWBACKS APP TO

GRIDSHierarchies Maps many

domains;

predictable

Centralized, tends

to static;

bottlenecks

Globus MDS

Holarchies Autonomy Lack of

predictability in

Holonic VO

management

7


8/33

holons organization

Coalitions Goal-oriented,

potentially optimal

Formation can be

costly; prevents

applications in very

dynamic settings

A few research

papers

Congregations Long lived, utility

directed; modular;

can be plugged

with others (e.g.

markets)

Groups may be

restrictive;

restricted dynamics

To electronic

markets

Group Selection Dynamic view,

evolves system

towards optimal

outcomes; scalable;

very generic

process,

complements

others

Potentially ever-

evolving suboptimal

allocations;

unpredictability;

needs to be

engineered as an

emergent

coordination

mechanism

-

GROUP SELECTION PATTERN

A. SOFTWARE PATTERNS

A software engineering design pattern is a general repeatable solution to a

commonly occurring problem in software design. This is a description or template

for how to solve a problem that can be used in many different situations.Software patterns have their roots in Christopher Alexanders work in

architecture. He proposed a design pattern as a three-part rule that expresses a

relation between a certain context, problem and solution. A good pattern provides

more than just the details of these sections; it should also be generative. Patterns

are not solutions; rather patterns generate solutions.

8


9/33

Software patterns became popular after the publication of the classic book by

Gamma et al. The documentation for a design pattern should contain enough

information about the problem that the pattern addresses, the context in which it

is used, and the suggested solution. Patterns allow developers to communicate

using well-known, well understood names for software interactions. Common

design patterns can be improved over time, making them more robust than ad-

hoc designs.

Software engineering design patterns have also been introduced in MAS,

including blackboard, meeting, master and slave, market-maker and others.

Pattern languages can help a developer to build entire MAS. For example, an

individual pattern can help also in designing a specific aspect of an agent, such

as how it models its beliefs, but a pattern language can help to use those beliefs

to build agents that plan and learn by putting individual patterns into context.

However, an important drawback of current software design patterns is that their

application to large and decentralized systems suffers from manageability and

scalability issues. Blackboards, master and slave, market-maker and othercommon patterns imply centralization of agents activities. This poses strong

limitations to system scalability. As a proposed solution, engineers have started

to build on bottom up approaches to develop emergent and self-organized design

patterns.

Patterns in computer science have also been used from other disciplines. A

relevant case is for bio-inspired computing .They state in order to motivate the

proposal of a family of design patterns coming from biology: The motivation of

the present work is that large-scale and dynamic distributed systems have strong

similarities to some of the biological environments. This makes it possible to

abstract away design patterns from biological systems and to apply them in

distributed systems. In other words, we do not wish to extract design patterns

9


10/33

from software engineering practice as it is normally done. Instead, we wish to

extract design patterns from biology, and we argue that they can be applied

fruitfully in distributed systems. Another field which has inspired several software

design patterns is sociology.

Since Grids are naturally composed in VOs, a basic group unit already exists.

The Group Selection process operates through natural selection in several

group-structured systems in nature: biological systems (evolving group-

advantageous behaviors); humans societies (promoting high levels of

cooperation); and economies (promoting the emergence of leading firms). We

want to build on these good properties of the mechanism to port the Group

Selection process to an engineering pattern usable in large scale distributed

systems amenable to group structure, such as Grids.

B. GROUP SELECTION PATTERN

Building on the experience gained by Babaoglu et al., we describe the following

attributes for our Group Selection pattern: name, context, problem, forces,

solution, example, and finally, design rationale. The meaning of these attributesshould be self-explanatory, except perhaps in the case of context. The context is

defined by the system model: the participants, their capabilities and the

constraints on the way they can interact. Our system model is a Grid composed

of N agents, structured as a variable number of groups or VOs. Each agent

maintains an action, strategy, or policy (depending on the scenario considered)

to take resource management decisions. This property, together with group

membership, can be evolved continuously by the Group Selection process.

Name: Group Selection pattern in MAS

10


11/33

Context: The basic units of the model are agents. The population of agents is

structured into functional groups. These groups compose a very dynamic

environment in which agents may enter and leave continuously.

Problem: I want to engineer a system composed of N agents. The structure in

which these entities can be arranged in the system is a free design parameter,

and the movement of entities from one structure to another should not be

constraint. In open MAS, the definition of an optimal structure can be ruined at

any time by the entrance of new agents in the system. As a result of this, self-

organized emergence of the properties is a good-to-have characteristic.

Forces: Entities in the system should keep the highest autonomy level possible.

Coordination optimization should be as controllable from the designer point of

view as possible. These two forces compose a classical dilemma when

engineering self-organizing systems.

Solution: By organizing the agent interaction in subgroups, which are

dynamically created, desired values for the macroscopic variables (cooperation,

coordination, stability)can emerge from agents co-evolution in the groups,modeled as interaction and migration phases. The group structure determines

the interaction scope of the agents. The dynamic view considers the migration of

agents from group to group, potentially implying the change of agent properties

by evolutionary learning, i.e., fitness based learning. This process self-organizes

into group level selection where groups performing well survive and groups with

agents implementing poor coordination strategies die out.

Examples: In a Grid environment the agents become Grid users or Grid service

providers. An example of cooperative setting is job scheduling in a pool of Grid

resources by a set of decentralized co-schedulers. Another example is a set of

agents coordinating their resource management policies in the VOs they form.

11


12/33

Design rationale: We provide an algorithmic approach to the pattern which can

be instantiated in different flavours by simple variation of Interaction and

Migration rules.

The implementation of the Group Selection pattern is depicted in Fig. 1. In the left

side, we see the flowchart for an individual agent. First, the agent performs in the

scope of its group an interaction phase (dependent on the domain of application),

which can be for example a job submission, an economic resource allocation, or

a resource management policy selection. After this, a migration phase is applied

which reallocate the agents in a new group. A migration condition decides

whether the change in group happens or not. A migration might encompass an

agent property change or nor also dependent on the domain of application. For

example, in an economic-based resource allocation domain, this property might

be the negotiation strategy.

In the right side of Fig. 1, the algorithmic representation of the full MAS operation

is depicted. The proposed synchronous algorithmic realization does not prevent

the application use in a realistic, asynchronous environment, since there is not

any synchronization step required to update agents strategies and groupmembership. In this algorithm, first the agents population is bootstrapped

(randomly or any pre-configured arrangement) into a number of groups. The two

phases (interaction and migration) mentioned earlier are then executed:

Interaction phase: Interaction rule is applied inside the group, payoffs are

collected and utility derived following a rule. This rule is domain dependent. The

interaction can be performed bilaterally, for each pair of agents and depending or

their strategies (as in a game theoretic setting with one player and its opponent).

Alternatively, we can consider a collective interaction inside the group, with the

payoff shared equally between all the agents composing the group (collective

payoffs). More elaborated payoff sharing schemes (might be a combination of the

other two) can be also applied.

12


13/33

Migration phase: Provided the condition for migration is met (see Fig. 1, left), a

migration rule is applied. This can be implemented in several ways: for example,

agents can compare with agents from other groups and migrate to groups

hosting outperforming agents. Alternatively, agents can inspect internally their

own performance in the last interactions and then decide if exploring randomly

new groups, or staying in their current group. The migration phase is

complemented by a group mutation, modeled simply as the creation of a brand

new group by an agent, abandoning its current group. Other agents can migrate

to this seed group in the future, growing the group size as a result. Migration

phase and group mutations each apply with a different probability (with mutation

probability being generally one order of magnitude lower than migration

probability). These two values are important free parameters, ruling the dynamics

of the system, and should be explored and tuned by the system designer in order

to achieve a desired emergent system behavior.

13


14/33

GROUP SELECTION PATTERN APPLIED TO GRID COMPUTING

In Grid resource management, good examples of domains subject to

improvement through agent based coordination are decision making in resource

allocation and job scheduling, as well as policy coordination in Vos. Given a

general Group Selection pattern for structured MAS, various applications to Grid

Computing scenarios are possible; regardless of how utilities are derived in

interaction phase or how migration phase is performed, the important thing to

keep inside the pattern is that both phases must be present. By varying

interaction and migration rules we get different instantiations of the Group

Selection pattern, producing different coordination mechanisms. We instantiate

the pattern in three different flavors accounting for applications in three Grid

resource management scenarios: adaptive job scheduling, economic-based

resource allocation based on bargaining agents and coordination of resource

management policies in VOs.

The pattern is instantiated basically in two classes or modalities, depending on

the type of property to be affected by the Group Selection process. The first class

consists in putting together compatible agents, without the need of modifyingtheir properties (actions, strategies, etc). In this case the property being evolved

is nothing but the presence in a group. The disposition of agents in the

convenient group is what gives the agent competitive advantage compared to

agents in other groups. We show this modality in the application to adaptive

learning schedulers, which get their learning spaces changed as a result of new

group memberships, but no strategy is changed due to group selection. The

second class of instantiation also transmits group beneficial characteristics. The

evolution of specific agents characteristics is managed upon migration phase by

the copy of characteristics of agents from other groups which might be

outperforming current agent utilities. We show this modality in the Grid markets

segmentation application, when negotiation types are modified when changing

groups, with Group Selection promoting the copy of outperforming agents type in

14


15/33

the new group. A similar case happens for policy coordination in VOs, with

successful resource sharing policies being imitated by agents joining new

groups.

For the three scenarios, we provide a baseline which does not use inter-group

migrations based on utilities. These baselines are extracted from state of the art

Grid resource management. In Table II, we show how the Group Selection

optimization compares to each baseline mechanism, and what the optimization

consists of.

All the experiments in the next subsections are conducted in an open source,

generic agent-based Grid simulator specifically built for developing agent

coordination mechanism on top of Grids. The simulator is implemented in java,

on top of a MAS discrete event simulator, Repast. The agent based Grid

simulator leverages the excellent analysis tools of Repast and its core, and adds

a Grid model and an Agent Framework, focusing on the development of agents

for the coordination of Grid activities. The models explained in this paper are

included as scenarios for the Grid simulator and its source code can be

inspected. The experiments are fully repeatable by downloading the simulator inthe provided URL. We base our results here in single simulations, without

averaging over multiple runs. We show here only sample executions of the

simulations, but we have obtained the same results on multiple runs, confirming

its validity. We refer also to prior work on Group Selection mechanisms with

averaged results over multiple runs.

SCENARIO BASELINE (state

of the art)

GROUP

SELECTION

OPTIMIZATION

Adaptive job

scheduling

Flat population of

adaptive job

schedulers

Evolves groups of

adaptive Job

schedulers. The

mechanism evolve

Schedulers

learning spaces

are shorter and

converge Quicker.

15


16/33

group

memberships

System load

decreases

Market-based

resource allocation

Flat decentralized

market of

bargaining agents

Evolves market

segments from the

flat market. The

mechanism evolve

group

memberships &

negotiation types

Increases social

welfare derived

from allocation

utility. Outperforms

alternatives in job

scheduling

Coordination of

resource

management

policies in VOs

Static VO policies

with rule-based

policy

management

Dynamic VOs

evolved through

Group Selection.

The mechanism

evolve VO agents

memberships &

agents policies

Increased

coordination

trough adaptive

policy

management

GROUP SELECTION FOR ADAPTIVE DISTRIBUTED GRID

SCHEDULING

The adaptive Grid scheduling model builds on Reinforcement learning (RL). RL is

learning from interaction with the environment, from the consequences of action,

rather than from explicit teaching. The general goal of RL is to find a policy

mapping observations (which might be states) to actions which maximizes

expected reward over multiple time steps. RL has been widely studied in agent

theory, mostly in settings with one learner in a static environment. Porting these

successful early results to MAS settings has proven a very difficult and

controversial as pointed out by researchers in the domain.

Galstyans model of adaptive distributed Grid scheduling is our first baseline. It

applies RL to a flat population of independent job co-schedulers. An important

assumption is that the allocation strategies employed by users and brokers in a

Grid have limited real-time environment knowledge at their disposal. This

16


17/33

suggests that feasible allocation mechanisms should not depend strongly on the

availability of current global knowledge of the system. The RL approach relies on

minimal monitoring capabilities to compare resources, only requiring that the

agent obtains status signals for job requests issued by the same agent. The

model considers a simplified representation of the resources and local

schedulers. They assume that each resource is characterized by its processing

power P, which is the CPU time needed to complete a job of a unit length. The

resources are dedicated, with just a single job running at the system at a given

time. This makes the load of the resources to be a function of the size of the

queue of incoming jobs in the resource. All the local schedulers prioritize the jobs

by their arrival time. As for the learning, the agents maintain a table of values (Q-

values), with a performance assigned to each resource. These Q-values are

updated each time a new job gets completed and the agent gets the result back.

This only requires private information of the agents of its past performances.

Being constantly updated upon job completions, this leads to responsiveness to

environmental changes. The metric used to measure the performance of the

model is the average load of the resources in the Grid, weighted by the capacity

of each resource. In Galstyan study, a comparison between random, least loaded

with global information (subject to probabilistic fails in the information update)and RL selection mechanism is performed. The results suggest that, under a

small level of failure in resource information update, the RL algorithm performs

well compared with least loaded mechanism, and outperforms clearly random

selection mechanism. RL can also cope with stochastic reward information, no

matter whether the noise in the reward information is due to variance in actual

resource behavior or in reporting.

The model described in the previous paragraph is our baseline. Now consider an

optimization using the Group Selection Pattern (depicted in Fig. 2). Each

scheduler agent can only select the next resource from the resources belonging

to its same group. This means also that agents only learn about resources on

their current group. The idea behind this segmentation is to boost Q-tables

17


18/33

convergence through a reduced learning space, implying reduced Q-tables

updates. I.e., we partition in groups the learning space. Agent migration to

another group and group mutation to create a new group encompasses also the

migration to that group of the resource hosting the agent. In this pattern

instantiation, the RL Interaction and Migration rules take the following form:

RL Interaction rule: apply learning upon any delayed reward available (update Q-

tables) and apply e-greedy selection over the set of resources in the group to

decide the resource the next job will be submitted to.

RL Migration rule: Agents (with their resources) move between groups depending

on the relative fitness. Agents will migrate to groups with agents outperforming

themselves. That is, compare my own performance to the performance of

another agent in a different group. If the other agent is outperforming me, then

move to its group. This operation is applied with a migration probability and some

variability is added trough a mutation probability, implying the creation of a brand

new group on its own by the agent.

The experimental setup is as follows: we fix an agent population of 100

resources processing jobs and 100 agents submitting jobs. The resources are of

variable capacities (as typical in Grid settings) in the interval (0,100), measured

in abstract processing power units. A new job submission is performed in each

18


19/33

agent at a fixed rate on each simulation tick. The higher this rate becomes, the

higher the task load on the resources. Jobs can be have both fixed size (50) and

variable sizes (10,100), measured in abstract processing power required units.

For too high values of the job submission rate, the load in the resources will grow

infinitely and the users wait time will follow that growth. For each experiment, we

follow the evolution of the system over 2000 ticks simulation period. Jobs

submission (following the job submission rate probability) and job processing at

resources advance at each simulation tick in the discrete-event simulator.

In our baseline implementation in the simulator, we fix a job submission rate of

0.4 and replicate results close to the simulations by Galstyan, using Q-learning

algorithm for Qvalue updates. The Q-learning algorithm (see Equation 1)

operates with a learning rate indicating how much the new experience is taken

into account.

The e-greedy resource selection for next job submission is performed following

the Q-table with high probability, (1-e), which means selecting the fittest resource

so far for the next job scheduling, and exploring randomly a new resource with a

low probability e. We take to be = 0.1 and e to be e=0.01.

We measure the system centric metric average resource load as the total job

queue length divided by the resources capacity. The final metric is the average of

this value over the set of resources in the Grid. The goal of the adaptive

schedulers (the agents) is achieving load balancing, minimizing averageresource load metric. We provide a complementary user centric metric: the wait

time for each job to be completed in the resource, weighted by the job size.

Averaging this over the set of jobs submitted into the Grid by an agent we get the

average weighted response time (AWRT) metric typically used to measure user-

centric performance in Grid settings.

19


20/33

Lets try scaling up the baseline results for RL schedulers. Increasing the job

arrival rate to higher values (from 0.4 to 0.5), we want to see how does the RL

behave and what benefits can we obtain using utility-based dynamic groups on

the agents populations. We can see from Fig. 3 that RL in a flat population is not

able to scale to a higher job submission rate of 0.5. The performance degrades in

this case almost in an attenuated linear increase. If we introduce the Group

Selection mechanism with small and dynamic groups, the RL agents are able to

cope with the increased workload pretty well and stabilize from the very

beginning. This means that schedulers became more adaptive just by applying

segmentation in groups and a simple utility-based agents migration process. We

try now to vary the Group Selection mechanism parameters to see which values

are leading to a better performance. Two parameters rule the dynamics of the

mechanism: first, the migration probability. This is the probability at which the

agents apply migration rule, that is to compare with other schedulers in the

system and move to their group in the case the scheduler outperforms

themselves. This basically rules the rate at which the agent is testing the

existence of better congregations of schedulers and resources. The other

parameter, the mutation probability, rules the extent to what the agent decides toexplore a brand new group, starting a group on its own and waiting for others to

join. It is important to notice the importance of this parameter, since setting this to

0 would provoke a quick convergence to one single group (the original flat

setting). Group mutation needs to be enabled in order to introduce variability; in

the same sense exploration is often required in RL algorithms.

We see from the results in Fig. 4 that, for a fixed mutation probability of 0.1, a

comparable migration probability leads to too many groups generated by

mutations, around 80, which mean that most of the agents are isolated, and the

system becomes useless for grouping agents and balancing load in the system.

Increasing the migration probability to 0.3 and then to 0.5 leads to improvements

on the load balancing Further increasing the migration rate upper than 0.5, the

20


21/33

improvements are negligible. The high variation in groups is provoked by the full

variation in a mechanism parameter, namely the migration probability in each

single simulation. This high variation in the number of groups shows that a high

migration probability provokes a higher variation in the number of groups.

The number of dynamic groups in this migration to mutation ratio is around 40

groups. This means having a large number of very small groups, which partition

learning spaces in very small (but dynamic) sets. Agents learning in those sets

converge quickly and adapt easier to the moves of the rest of independent

schedulers in the system.

SEGMENTATION IN MARKET BASED RESOURCE

MANAGEMENT

21


22/33

An alternative resource management mechanism uses a set f bargaining agents

in a decentralized Grid market. Decentralized Grid markets based on catallactic

agents have been proposed as suitable coordination mechanisms for Grids and

Service Oriented Architectures. The market here is nothing more than a

communication bus it is not a central entity and does not participate in

matching participants requirements using optimization mechanisms as in typical

centralized markets (auctions).

The Catallaxy mechanism was originally proposed by von Hayek. The Catallaxy

approach is a coordination mechanism for systems consisting of autonomous

decentralized agents that makes use of a free-market self organization

approach. It enables prices within the market to be adjusted based on constant

negotiation and price signaling between agents. Catallactic agents for Grid

resource allocation are explored in the CATNETS project.

It is intuitively clear that decentralized bidding in a huge flat market is not the

most efficient setting for any given number of buyers and sellers in a Grid market.

The higher the heterogeneity and dynamicity of the system are, the bigger is theprobability of un-coordinated outcomes leading to low system-wide performance.

We propose to extend standard catallactic Grid markets, enabling the

bootstrapping of the agents in submarkets and the automatic evolution of these

groups towards optimized market segments. Hayek itself supported the idea of

Group Selection as a transmitter of free market norms and institutions between

societies. The idea behind market segmentation is identifying groups of similar

customers and potential customers, to prioritize the groups to address and to

respond with appropriate strategies that satisfy the different preferences of each

segment. Markets segments can be set up manually, but this clearly does not

scale nor adapt to open Grid systems. Group Selection self-organizes market

segments evolution in a scalable manner which works with large scale open

systems.

22


23/33

In Fig. 5 (left) we see an example of a typical flat catallactic market. Two brokers

(Client 1 and 2) try to contact service providers in order to engage into bilateral

negotiations with them. In this example, the discovery mechanism finds 4 Grid

services providers (Services 1 to 4). Each service provider maintains a different

negotiation type, defined by its strategy, contractual implications, and legal

issues and so on. An important issue is that negotiation type information must be

kept private in order to avoid cheating agents with incentives to free-ride on other

agents. In order to simplify, we reduce in the example the negotiation type space

to two different types, S1 and S2. In this case, we can see that Client 1 with

strategy S1 will end up negotiating with Service 2 and Service 4, which are more

compatible with Clients using S2. This is a case of miss coordination leading to

spurious negotiations which should be avoided since they are time and resource

consuming.

We build on automatic market segmentation in order to optimize an agents

discovery process. Instead of performing decentralized search and negotiation

over the whole set of services (in a single market), as it is done in a typical

catallactic market, we propose a segmentation of the markets for services inspecialized VOs, each identified by a tag. Once the negotiation partner is chosen

with the enhanced discovery, the economic negotiation proceeds in exactly the

same way as it would do in a normal catallactic market. A minimal segmentation

of the Grid market is depicted in Fig. 5 (right).

23


24/33

After one market iteration, and after utilities from the services executions are

derived, the users send back their perceived utility as feedback. This can be a

simple metric such as the service provision time, up to more elaborated metric

comprising QoS. Then, Clients and Services must decide how to evolve on the

different market segments. They can compare fitness with agents in other VOs,

migrating to VOs where they find outperforming agents. In this scenario, it is this

negotiation types closeness characteristic being automatically selected at the

group level. In this manner Client 1 and Client 2 will interact preferentially with

Services belonging to the same group, which tend to be Services closer to them

in negotiation abilities and goals, hence increasing the probability of successful

allocations.

In this pattern instantiation, the Market-based Interaction and Migration rules take

the form:

Market-based Interaction rule: Negotiate with agent/agents from my market

segment. Calculate allocation utility and perform job submission following market

allocations.

24


25/33


26/33

to model realistic scenarios with communication costs, we limit the scope of the

CFP to 5 Services to be reached on each groupcast, selected at random either

from the whole population (in the flat scenario) or from agents belonging to the

same group (in the Group Selection scenario).We introduce five different

negotiation types. Utility derived by clients is related to the closeness between

negotiation types. The closer the types are, the higher the utility derived. In the

baseline with a single flat decentralized market, agents try to learn negotiation

types of other agents by adopting the negotiation types of agents outperforming

them. In the Group Selection scenario, the same learning process is structured:

agents migrate with a high probability to other groups whenever they find other

agents achieving higher payoffs. This migration implies the copy of the

outperforming agent negotiation type.

We measure the utility extracted from allocations as directly proportional to the

negotiation type closeness.

Given M different negotiation types, negotiation types are represented by

integers from 1 to M. We use (2) for utility calculation, with 1 nt being the

negotiation type of buyer agent 1 and 2 ntbeing the negotiation type of seller

agent 2. The exception is that when 1 2 nt ntwe consider the maximum utility 1

achieved. The rest of cases oscillate between 0 and 1.

26


27/33

If we compare a flat market with a grouped setting evolving through Group

Selection, we see how the latter coordinate easier the negotiation types, leading

to better accumulated allocation utilities (Fig. 6, left). Prices vary smoothly

influenced by offer/demand in the bounds 80 to 70 in the Group Selection

scenario). The price stability around the initial selling prices (75) renders the

market fair (Fig. 6, right). The price evolution for the baseline flat market is

similar; hence we do not show the graph. We can conclude that market

segmentation is able to increase resource allocation utilities of the traders without

compromising price stability. A simple decentralized economic algorithm can be

optimized in allocation utilities by applying a Group Selection process, boosting

the convergence in the same groups of traders to compatible negotiation types. It

is easier for agent segmented in groups to find compatible trade partners than for

agents in the flat decentralized market.

We try now to vary the Group Selection mechanism migration probability and

mutation probability, to see which values are leading to a better performance.Intuitively, since migration tends to homogenize groups while mutation creates

brand new sub-markets, the lower the relation migration/to mutation, the larger

number of groups. We fix a mutation probability of 0.01. This is one order of

magnitude smaller than in the RL scenario. For the Market based resource

allocation scenario, too much new sub-markets would affect traders which are

27


28/33

not able to reach agreements. We need for a slower paced learning in this

scenario. Having a fixed mutation probability, we conduct a parametric study of

the migration probability. From the results in Fig. 7 (left) we see that, for a

migration probability of 0.1, the utility is comparable to the case when migration

probability is 0.3. Increasing the migration probability to 0.5, and then to 0.7,

decreases performance. The best configuration corresponds to a Grid

segmented in about 15 to 20 sub-markets, each of 5 or so agents, as shown in

Figure 7(right). This supports the same conclusion as for the RL scenario: small

groups converge quicker to compatible negotiation types.

ALIGNMENT IN POLICY BASED RESOURCE MANAGEMENT

VOs have several key properties that distinguish them from traditional IT

architectures: autonomy of its members, which behave independently,

constrained only by their contracts; heterogeneity of its members, which are

independently designed and constructed, constrained only by the applicable

interface descriptions; dynamism of the VO members which can join and leave

with minimal constraints, affecting the configuration of a VO at runtime; structure,

with VOs having complex internal structures, reflected in the relationships among

their members. Importantly, even in cases where the above properties are not

required (such as within an enterprise where the members are controlled by one

party), it is appropriate to architect a VO as if it had the above properties.

28


29/33

In our VO model, each agent (representing an organization) has a policy A = A

(p), from a set of M policies. A VO consists of a set of agents. VO = {A1 (p1), A2

(p2), A3 (p3).}. The VO defines the scope of agent operation. Policy based

resource sharing utility gets maximized by coordinating the policies of the M

agents forming each VO. The objective is achieving policy coordination in each of

the VOs in the system, forming C clusters out of the N agents in the population,

with most agents in each cluster using compatible policies. The compatibly

depends on the specific scenario and is measured differently depending on it.

Fig. 8 (left) shows a flat Grid without VOs where users of any policy can access

providers of any policy. Fig. 8 (right) shows a Grid organized in VOs whose

members policies are compatible, thus maximizing members utility.

Clearly, for policy based resource management, the scenario with group

structure outperforms the flat arrangement. The challenge in this case is how to

manage policy coordination in dynamic VOs. We aim to achieve automatic policy

coordination by means of the Group Selection pattern. The two rules of the

pattern are instantiated as follows.

29


30/33

Policy Coordination Interaction rule: We use collective interaction inside the

group, with a payoff shared equally between the agents composing the group.

We have implemented the simplest of the collective interactions, corresponding

to a VO policy alignment scenario. The payoff is calculated on the alignment level

over the whole VO.

Policy Coordination Migration rule: The agents compare their performance

against their own past performance (internal learning). Migration to a group

implies the copying of the policy of one random agent in this target group. This

maps VOs configurations of large pools of resources optimizing their

performance by acting together with a similar policy. The goal is to minimize

diversity (entropy) within each VO, achieving the highest policy alignment

possible.

Equation 3 implements our metric for this scenario. i p stands for one of the

policies in a set of size M. Minimizing the index is equivalent to maximizing

homogeneity in VOs. We reverse this measure to have a performance scale from

30


31/33

0 to 1: This gives 1 when all policies in the group are aligned (minimal entropy)

and 0 when all present policy in the group is represented equally (maximal

entropy).

The total number of agents is N=100. The set of different policies is M=10.

Mutation probability is fixed to a small value of 0.01 like in the markets

segmentation scenario. We see from Fig. 9 (left) that reaching a high alignment

(low entropy) up 0.8 is possible before 1000 rounds and maintained afterwards

for a migration probability of 0.3.Fig. 9 (right) shows that the number of VOs

oscillates in this case between 10 and 20. If we use a higher migration probability

of 0.7, this implies a higher ration migration/mutation and consequently less

number of groups in the system in average. As we can see from Fig. 9, larger

number of small groups generates better coordination, which recovers the same

conclusion as in precedent scenarios.

CONCLUSION

In large scale Grids, system dynamicity and uncertainty are high; automatic,

decentralized and self-organized control becomes a requirement. Our proposal is

that a simple, rather powerful coordination mechanism based on Group Selection

can be used to self-organize a set of agents in VOs to operate more effectively

on a Grid environment. The Group Selection mechanism complements rather

than compete with much of the existent coordination mechanisms for MAS. We

have formalized a distributed systems engineering pattern Group Selection

pattern, amenable to be instantiated in several Grid resource management

scenarios. We have shown, by simulations in a general purpose agent-based

Grid simulator, how splitting the Grid participants (agents and resources) in

groups and further evolving those groups based on utility can lead to optimization

in several Grid resources management tasks.

31


32/33

First, we have shown how to apply the pattern to evolve congregations of

independent RL schedulers, leading to optimized operation/learning in groups,

compared with a baseline with a flat population of schedulers. We have also

analyzed which values for the parameters of the mechanism (the group migration

to group mutation rate) achieve better performance, concluding that generating

trough this rate a population of very dynamic and small groups achieves the best

optimization in this scenario. Evolving smaller Q-tables allows for the RL agents

to converge quicker to coordinated resource usage, and changes or problems in

resources can be quickly addressed by migration to new groups or creation of

new groups trough mutation. In general RL schedulers learn quicker and adapt

quicker to changes in dynamic, small groups.

In the case of plugging the mechanism into a decentralized Grid market, each

group represents a sub-market of the market segmentation, and migration of

agents from one sub-market to another is ruled by the Group Selection process.

The results extracted by simulations show higher profits for the society of agents

trading in decentralized Grid markets which structure the population in sub-

markets and incorporate Group Selection, compared to the flat decentralized

markets. The performance results show that agent-based automatic fair tradingof resources at stable prices can be achieved using the decentralized market

mechanism coupled with the Group Selection process. Studying the impact of

varying migration to mutation rate draws a similar conclusion as in the previous

scenario: agents trading in small groups are able to converge quicker to more

compatible negotiation types.

As for plugging the mechanism into policy-based VO management scenario, the

migration of agents from one VO to another and its policy adaptation by aligning

policies with the agents in the target groups is ruled by the Group Selection

process, achieving optimization of policy-based resource sharing utilities. The

results extracted by simulations show how small groups of agents can align

32


33/33

quicker and to a higher degree (following Shannon entropy index metric) their

resource sharing policies.

Clustering in small groups is a tendency largely observed in all kind of human

organizations. In cooperation building scenarios, it has been shown that smaller

group sizes ease cooperation in both social-networks based cooperation and

Group Selection based evolution of cooperation. Our results for the evaluation of

Group Selection in three different Grid coordination scenarios suggest that the

same conclusion applies in fully cooperative domains: small and dynamic

groups of agents evolved trough Group Selection optimize better fully

cooperative coordination scenarios. In our Group Selection pattern, the group

migration to mutation rate determines the average number of dynamic groups in

the systems. A rate tuned to evolve dynamic and small groups achieves the best

optimization in all three scenarios. In the adaptive scheduling scenario, just group

membership is evolved in order to segment learning Q-tables of agents. In the

other two scenarios (market segmentation and VO policy based resource

sharing) an agent characteristic (negotiation type and policy respectively) its also

evolved via imitation upon migration to new groups. For the population of 100

agents evaluated in the simulator, the specific migration to mutation rate (this isscenario-dependent) dynamically generating small groups of around 5 agents

achieved the best performance in all three cases.

Future work includes considering Grid users and services belonging to many

VOs simultaneously, mapping more realistic scenarios. Deployment of the

pattern into a Grid prototype can add more insights into practical feasibility of the

approach.

a group selection pattern applied to grid

Documents