Download - A Group Selection Pattern Applied to Grid
-
8/3/2019 A Group Selection Pattern Applied to Grid
1/33
A GROUP SELECTION PATTERN APPLIED TO GRID RESOURCE
MANAGEMENT
T.Arun Prakash and M.Abdul Rahman
Kalasalingam University
Anand Nagar
Krishnankoil 626190
ABSTRACT
A key challenge in Grid computing is the achievement of efficient and self-
organized resource management. Grids are often large scale, heterogeneous,
and unpredictable systems. Introducing group structures can help to distribute
coordination efforts, but higher levels of adaptation and learning in thecoordination protocols are still required in order to cope with system complexity.
We provide a solution based on a self organized and emergent mechanism
evolving congregations of resource management agents through a Group
Selection process which maximizes utility outcomes for system-wide
performance. We provide a formalization of this process into a Group Selection
pattern, and we propose several instantiations optimizing Grid resource
management scenarios such as adaptive job scheduling, market-based resource
management, and policy coordination in Virtual Organizations (VOs). We further
evaluate by simulation the performance of the mechanism in those scenarios.
The results support the conclusion that Group Selection optimizes coordination
by evolving small and dynamic groups.
-
8/3/2019 A Group Selection Pattern Applied to Grid
2/33
INTRODUCTION
The popularity of Grids has been growing very rapidly, driven by the promise that
they will enable knowledge and computing resources to be delivered to and used
by citizens and organizations as traditional utilities or in novel forms. Contrarily to
other distributed systems, Grids have many independent resource providers with
varying resource characteristics and availability. In addition to large sizes, the
dynamicity of resources leads to a very complex coordination task that cannot be
handled manually by users. Automatic and adaptive resource management isproposed as a solution to these challenges. To that aim, the potential synergies
between Grids and multi-agent systems (MAS) have been outlined. Among
others, MAS have exploited group structure to partition the population in
interaction groups, heavily impacting the coordination.
Group Selection refers to a process of natural selection that favors
characteristics in individuals that increase the fitness of the group the individuals
belong relative to other groups. Group Selection implies that every member of
the group depends on a group characteristic that is not isolated in a single
individual. Such groups form isolated niches where the sub-populations are
allowed to evolve behaviors independently of the rest of the populations. The
existence of niches maintains a large diversity in an evolving population since the
evolutionary paths in separated niches may develop in entirely different ways.
This evolutionary process can promote the evolution of individual agents
characteristics benefiting the group the agent belongs to. In several coordination
scenarios, inter-agents miss-coordination (due to ignorance, high complexity, or
other reasons) might be leading to very suboptimal social welfare outcomes.
Group Selection guides the co-evolution of group-structured systems to
optimized configurations.
2
-
8/3/2019 A Group Selection Pattern Applied to Grid
3/33
In this paper, we provide a decentralized coordination mechanism for Grid
resource management based on Group Selection. We distribute the agents in
groups. Further agents group migrations through a Group Selection process
improve the overall population performance. We propose formalization into an
engineering pattern and we show by simulation how it performs optimizing
several resource management tasks in Grids: learning agents for adaptive job
scheduling, bargaining agents in a market-based resource allocation, and policy
coordination in VOs.
STATE OF THE ART IN MAS BASED GRID RESOURCE
MANAGEMENT
We detail in this section the state of the art in MAS based Grid resource
management mechanisms, leaving for the next section the state of the art in
group formation mechanism in MAS and a comparison to the Group Selection
mechanism.
Conventional solutions in Grid resource management apply manual management
or, at most, centralized mechanisms ensuring a predictable outcome. This
normally leads to important scalability limitations due to increased manual and
computational costs, as well as limited tolerance to changes or failure (human
operators and centralized management offer a single failure point in the system).
In Grid scheduling, conventional parallel computer schedulers, such as PBS and
LSF, address the scheduling problem by implementing a synchronous
schedule/enactment process. The scheduling algorithm has full knowledge of the
resource properties. This solution is not appropriate in a federated environment
where no central entity has sole authority over local resource states. The Condor
high-throughput system avoids this problem by providing almost no delivery
guarantees; a centralized matchmaker makes simple greedy decisions to place
3
-
8/3/2019 A Group Selection Pattern Applied to Grid
4/33
jobs and simply retries when it encounters resource states in conflict with its
internal view of the environment. The Pegasus Grid workflow mapping system
includes a random job placement behavior to spread small parts of an overall
workflow into Grid resources.
Coordination is an important topic in large scale Grid systems management. In,
different coordination mechanisms for Grids are evaluated. From lower to higher
level, the following mechanisms are proposed: scripting languages, shared-data
spaces and middleware agents, the later being identified as the more flexible.
However none of these solutions incorporate the notion of self-organization
capturing systems dynamics as our mechanism does. A comprehensive survey
on self-organizing agent-based and autonomic computing applications in Grid
Computing is presented in. Market based resource allocation has received a
great deal of attention in the last years. The GridBus Project is a reference in
Grid Economy and utility based computing, and has proposed a great variety of
market models and tools for the trading of Grid Resources. However, its strong
emphasis on computational intensive Grids and the hierarchical nature of some
of the proposed components, like the Grid Market Directory, diverges from the
fully decentralized resource allocation mechanisms used here. Other centralizedapproaches exist such as, but scalability issues both in size and computational
requirements further complicate its applicability to large size Grids. Tycoon is a
market-based system for managing compute resources in distributed clusters or
Grids. It uses distributed auctions with users having a limited amount of credits.
Users who provide resources can, in turn, spend their earnings to use resources
later. A fully decentralized approach is the one adopted with the catallactic
agents. In this approach, bilateral negotiations are established between learning
agents, and the spontaneous price coordination arises from both the bargaining
and co-evolutionary learning processes. However, the bootstrapping and
evolution of these bargaining agents in markets is an open issue.
4
-
8/3/2019 A Group Selection Pattern Applied to Grid
5/33
Grids environments are being organized in VOs, associating heterogeneous
users and resource providers to coordinate resource sharing; a set of individuals
and/or institutions defined by their sharing rules form a Virtual Organization.
There are many projects using VOs conceptually, but very few projects are
addressing the management of VOs themselves in Grids. While the notion of a
VO seems to be intuitive and natural, we still lack well-defined procedures for
deciding when a new VO should be formed, who should be in that VO, what they
should do, when the VO should be changed, and when the VO should ultimately
be disbanded. In Conoise-G project an agent system supporting robust and
resilient VOs formation and operation is presented. Another project focusing on
Trust issues is Trustcom aiming to provide a trust and contract management
framework enabling the definition and secure enactment of collaborative
business processes within VOs that are formed on-demand, self managing and
evolve dynamically. In both Conoise-G and Trustcom approaches to VO
management, components for helping automated VO management are
developed.
STATE OF THE ART IN GROUP FORMATION
MECHANISMS FOR MAS
Exploiting group structure in MAS has been proposed in previous research. It is
acknowledged that, in general, there is no single type of organizational paradigm
that is suitable for all situations. Some claim that there is not even perfect
organization in any situation, due to the inevitable tradeoffs that must be made
and the uncertainty, lack of global coherence and dynamics present in any
realistic population. In hierarchies the data produced by lower level agents
travels upwards to provide a broader view, while control flows downwards as the
higher level agents provide direction to those below. Holarchies follow a very
similar pattern, but typically include homogeneous, self-similar elements at each
hierarchical level, and the constituents of each group are partially-autonomous.
5
-
8/3/2019 A Group Selection Pattern Applied to Grid
6/33
-
8/3/2019 A Group Selection Pattern Applied to Grid
7/33
supposing complete system knowledge to the central coalition-maker, with few
exceptions. This panorama contrast with realistic present day large scale
distributed systems, where small, decentralized components need to deal
autonomously with coordinated decision making.
Group Selection is a fully decentralized mechanism which focuses on the
dynamic view of the groups, iteratively ruling its evolution towards more optimal
configurations. It has been shown that Group Selection can lead to the spread of
group beneficial characteristics in many different grouped settings of agents
populations. This enables the application of Group Selection processes in any
group-like structured agent population. Applications of Group Selection have
appeared in biology and sociology and also in economic theory. In engineering,
novel socially-inspired mechanism have been developed building to some extent
on Group Selection processes. These mechanisms use a tag (or social label) to
identify groups. Agents interactions are biased by Tags (i.e. within the groups)
and inter-group migrations are ruled by Group Selection processes. This has
been demonstrated most notably in the application to free-riding prevention in
Peer-to-Peer networks by Hales. Also other applications build on Tag
mechanisms such as query routing and processing for Peer-to-Peer web search.Table I compares Group Selection with the rest of group formation mechanism
reviewed. It can be seen that Group Selection potentially addresses several
important issues in existent group formation mechanisms.
TABLE I. GROUP FORMATION MECHANISMS COMPARISON
MECHANISM BENEFITS DRAWBACKS APP TO
GRIDSHierarchies Maps many
domains;
predictable
Centralized, tends
to static;
bottlenecks
Globus MDS
Holarchies Autonomy Lack of
predictability in
Holonic VO
management
7
-
8/3/2019 A Group Selection Pattern Applied to Grid
8/33
holons organization
Coalitions Goal-oriented,
potentially optimal
Formation can be
costly; prevents
applications in very
dynamic settings
A few research
papers
Congregations Long lived, utility
directed; modular;
can be plugged
with others (e.g.
markets)
Groups may be
restrictive;
restricted dynamics
To electronic
markets
Group Selection Dynamic view,
evolves system
towards optimal
outcomes; scalable;
very generic
process,
complements
others
Potentially ever-
evolving suboptimal
allocations;
unpredictability;
needs to be
engineered as an
emergent
coordination
mechanism
-
GROUP SELECTION PATTERN
A. SOFTWARE PATTERNS
A software engineering design pattern is a general repeatable solution to a
commonly occurring problem in software design. This is a description or template
for how to solve a problem that can be used in many different situations.Software patterns have their roots in Christopher Alexanders work in
architecture. He proposed a design pattern as a three-part rule that expresses a
relation between a certain context, problem and solution. A good pattern provides
more than just the details of these sections; it should also be generative. Patterns
are not solutions; rather patterns generate solutions.
8
-
8/3/2019 A Group Selection Pattern Applied to Grid
9/33
Software patterns became popular after the publication of the classic book by
Gamma et al. The documentation for a design pattern should contain enough
information about the problem that the pattern addresses, the context in which it
is used, and the suggested solution. Patterns allow developers to communicate
using well-known, well understood names for software interactions. Common
design patterns can be improved over time, making them more robust than ad-
hoc designs.
Software engineering design patterns have also been introduced in MAS,
including blackboard, meeting, master and slave, market-maker and others.
Pattern languages can help a developer to build entire MAS. For example, an
individual pattern can help also in designing a specific aspect of an agent, such
as how it models its beliefs, but a pattern language can help to use those beliefs
to build agents that plan and learn by putting individual patterns into context.
However, an important drawback of current software design patterns is that their
application to large and decentralized systems suffers from manageability and
scalability issues. Blackboards, master and slave, market-maker and othercommon patterns imply centralization of agents activities. This poses strong
limitations to system scalability. As a proposed solution, engineers have started
to build on bottom up approaches to develop emergent and self-organized design
patterns.
Patterns in computer science have also been used from other disciplines. A
relevant case is for bio-inspired computing .They state in order to motivate the
proposal of a family of design patterns coming from biology: The motivation of
the present work is that large-scale and dynamic distributed systems have strong
similarities to some of the biological environments. This makes it possible to
abstract away design patterns from biological systems and to apply them in
distributed systems. In other words, we do not wish to extract design patterns
9
-
8/3/2019 A Group Selection Pattern Applied to Grid
10/33
from software engineering practice as it is normally done. Instead, we wish to
extract design patterns from biology, and we argue that they can be applied
fruitfully in distributed systems. Another field which has inspired several software
design patterns is sociology.
Since Grids are naturally composed in VOs, a basic group unit already exists.
The Group Selection process operates through natural selection in several
group-structured systems in nature: biological systems (evolving group-
advantageous behaviors); humans societies (promoting high levels of
cooperation); and economies (promoting the emergence of leading firms). We
want to build on these good properties of the mechanism to port the Group
Selection process to an engineering pattern usable in large scale distributed
systems amenable to group structure, such as Grids.
B. GROUP SELECTION PATTERN
Building on the experience gained by Babaoglu et al., we describe the following
attributes for our Group Selection pattern: name, context, problem, forces,
solution, example, and finally, design rationale. The meaning of these attributesshould be self-explanatory, except perhaps in the case of context. The context is
defined by the system model: the participants, their capabilities and the
constraints on the way they can interact. Our system model is a Grid composed
of N agents, structured as a variable number of groups or VOs. Each agent
maintains an action, strategy, or policy (depending on the scenario considered)
to take resource management decisions. This property, together with group
membership, can be evolved continuously by the Group Selection process.
Name: Group Selection pattern in MAS
10
-
8/3/2019 A Group Selection Pattern Applied to Grid
11/33
Context: The basic units of the model are agents. The population of agents is
structured into functional groups. These groups compose a very dynamic
environment in which agents may enter and leave continuously.
Problem: I want to engineer a system composed of N agents. The structure in
which these entities can be arranged in the system is a free design parameter,
and the movement of entities from one structure to another should not be
constraint. In open MAS, the definition of an optimal structure can be ruined at
any time by the entrance of new agents in the system. As a result of this, self-
organized emergence of the properties is a good-to-have characteristic.
Forces: Entities in the system should keep the highest autonomy level possible.
Coordination optimization should be as controllable from the designer point of
view as possible. These two forces compose a classical dilemma when
engineering self-organizing systems.
Solution: By organizing the agent interaction in subgroups, which are
dynamically created, desired values for the macroscopic variables (cooperation,
coordination, stability)can emerge from agents co-evolution in the groups,modeled as interaction and migration phases. The group structure determines
the interaction scope of the agents. The dynamic view considers the migration of
agents from group to group, potentially implying the change of agent properties
by evolutionary learning, i.e., fitness based learning. This process self-organizes
into group level selection where groups performing well survive and groups with
agents implementing poor coordination strategies die out.
Examples: In a Grid environment the agents become Grid users or Grid service
providers. An example of cooperative setting is job scheduling in a pool of Grid
resources by a set of decentralized co-schedulers. Another example is a set of
agents coordinating their resource management policies in the VOs they form.
11
-
8/3/2019 A Group Selection Pattern Applied to Grid
12/33
Design rationale: We provide an algorithmic approach to the pattern which can
be instantiated in different flavours by simple variation of Interaction and
Migration rules.
The implementation of the Group Selection pattern is depicted in Fig. 1. In the left
side, we see the flowchart for an individual agent. First, the agent performs in the
scope of its group an interaction phase (dependent on the domain of application),
which can be for example a job submission, an economic resource allocation, or
a resource management policy selection. After this, a migration phase is applied
which reallocate the agents in a new group. A migration condition decides
whether the change in group happens or not. A migration might encompass an
agent property change or nor also dependent on the domain of application. For
example, in an economic-based resource allocation domain, this property might
be the negotiation strategy.
In the right side of Fig. 1, the algorithmic representation of the full MAS operation
is depicted. The proposed synchronous algorithmic realization does not prevent
the application use in a realistic, asynchronous environment, since there is not
any synchronization step required to update agents strategies and groupmembership. In this algorithm, first the agents population is bootstrapped
(randomly or any pre-configured arrangement) into a number of groups. The two
phases (interaction and migration) mentioned earlier are then executed:
Interaction phase: Interaction rule is applied inside the group, payoffs are
collected and utility derived following a rule. This rule is domain dependent. The
interaction can be performed bilaterally, for each pair of agents and depending or
their strategies (as in a game theoretic setting with one player and its opponent).
Alternatively, we can consider a collective interaction inside the group, with the
payoff shared equally between all the agents composing the group (collective
payoffs). More elaborated payoff sharing schemes (might be a combination of the
other two) can be also applied.
12
-
8/3/2019 A Group Selection Pattern Applied to Grid
13/33
Migration phase: Provided the condition for migration is met (see Fig. 1, left), a
migration rule is applied. This can be implemented in several ways: for example,
agents can compare with agents from other groups and migrate to groups
hosting outperforming agents. Alternatively, agents can inspect internally their
own performance in the last interactions and then decide if exploring randomly
new groups, or staying in their current group. The migration phase is
complemented by a group mutation, modeled simply as the creation of a brand
new group by an agent, abandoning its current group. Other agents can migrate
to this seed group in the future, growing the group size as a result. Migration
phase and group mutations each apply with a different probability (with mutation
probability being generally one order of magnitude lower than migration
probability). These two values are important free parameters, ruling the dynamics
of the system, and should be explored and tuned by the system designer in order
to achieve a desired emergent system behavior.
13
-
8/3/2019 A Group Selection Pattern Applied to Grid
14/33
GROUP SELECTION PATTERN APPLIED TO GRID COMPUTING
In Grid resource management, good examples of domains subject to
improvement through agent based coordination are decision making in resource
allocation and job scheduling, as well as policy coordination in Vos. Given a
general Group Selection pattern for structured MAS, various applications to Grid
Computing scenarios are possible; regardless of how utilities are derived in
interaction phase or how migration phase is performed, the important thing to
keep inside the pattern is that both phases must be present. By varying
interaction and migration rules we get different instantiations of the Group
Selection pattern, producing different coordination mechanisms. We instantiate
the pattern in three different flavors accounting for applications in three Grid
resource management scenarios: adaptive job scheduling, economic-based
resource allocation based on bargaining agents and coordination of resource
management policies in VOs.
The pattern is instantiated basically in two classes or modalities, depending on
the type of property to be affected by the Group Selection process. The first class
consists in putting together compatible agents, without the need of modifyingtheir properties (actions, strategies, etc). In this case the property being evolved
is nothing but the presence in a group. The disposition of agents in the
convenient group is what gives the agent competitive advantage compared to
agents in other groups. We show this modality in the application to adaptive
learning schedulers, which get their learning spaces changed as a result of new
group memberships, but no strategy is changed due to group selection. The
second class of instantiation also transmits group beneficial characteristics. The
evolution of specific agents characteristics is managed upon migration phase by
the copy of characteristics of agents from other groups which might be
outperforming current agent utilities. We show this modality in the Grid markets
segmentation application, when negotiation types are modified when changing
groups, with Group Selection promoting the copy of outperforming agents type in
14
-
8/3/2019 A Group Selection Pattern Applied to Grid
15/33
the new group. A similar case happens for policy coordination in VOs, with
successful resource sharing policies being imitated by agents joining new
groups.
For the three scenarios, we provide a baseline which does not use inter-group
migrations based on utilities. These baselines are extracted from state of the art
Grid resource management. In Table II, we show how the Group Selection
optimization compares to each baseline mechanism, and what the optimization
consists of.
All the experiments in the next subsections are conducted in an open source,
generic agent-based Grid simulator specifically built for developing agent
coordination mechanism on top of Grids. The simulator is implemented in java,
on top of a MAS discrete event simulator, Repast. The agent based Grid
simulator leverages the excellent analysis tools of Repast and its core, and adds
a Grid model and an Agent Framework, focusing on the development of agents
for the coordination of Grid activities. The models explained in this paper are
included as scenarios for the Grid simulator and its source code can be
inspected. The experiments are fully repeatable by downloading the simulator inthe provided URL. We base our results here in single simulations, without
averaging over multiple runs. We show here only sample executions of the
simulations, but we have obtained the same results on multiple runs, confirming
its validity. We refer also to prior work on Group Selection mechanisms with
averaged results over multiple runs.
SCENARIO BASELINE (state
of the art)
GROUP
SELECTION
OPTIMIZATION
Adaptive job
scheduling
Flat population of
adaptive job
schedulers
Evolves groups of
adaptive Job
schedulers. The
mechanism evolve
Schedulers
learning spaces
are shorter and
converge Quicker.
15
-
8/3/2019 A Group Selection Pattern Applied to Grid
16/33
group
memberships
System load
decreases
Market-based
resource allocation
Flat decentralized
market of
bargaining agents
Evolves market
segments from the
flat market. The
mechanism evolve
group
memberships &
negotiation types
Increases social
welfare derived
from allocation
utility. Outperforms
alternatives in job
scheduling
Coordination of
resource
management
policies in VOs
Static VO policies
with rule-based
policy
management
Dynamic VOs
evolved through
Group Selection.
The mechanism
evolve VO agents
memberships &
agents policies
Increased
coordination
trough adaptive
policy
management
GROUP SELECTION FOR ADAPTIVE DISTRIBUTED GRID
SCHEDULING
The adaptive Grid scheduling model builds on Reinforcement learning (RL). RL is
learning from interaction with the environment, from the consequences of action,
rather than from explicit teaching. The general goal of RL is to find a policy
mapping observations (which might be states) to actions which maximizes
expected reward over multiple time steps. RL has been widely studied in agent
theory, mostly in settings with one learner in a static environment. Porting these
successful early results to MAS settings has proven a very difficult and
controversial as pointed out by researchers in the domain.
Galstyans model of adaptive distributed Grid scheduling is our first baseline. It
applies RL to a flat population of independent job co-schedulers. An important
assumption is that the allocation strategies employed by users and brokers in a
Grid have limited real-time environment knowledge at their disposal. This
16
-
8/3/2019 A Group Selection Pattern Applied to Grid
17/33
suggests that feasible allocation mechanisms should not depend strongly on the
availability of current global knowledge of the system. The RL approach relies on
minimal monitoring capabilities to compare resources, only requiring that the
agent obtains status signals for job requests issued by the same agent. The
model considers a simplified representation of the resources and local
schedulers. They assume that each resource is characterized by its processing
power P, which is the CPU time needed to complete a job of a unit length. The
resources are dedicated, with just a single job running at the system at a given
time. This makes the load of the resources to be a function of the size of the
queue of incoming jobs in the resource. All the local schedulers prioritize the jobs
by their arrival time. As for the learning, the agents maintain a table of values (Q-
values), with a performance assigned to each resource. These Q-values are
updated each time a new job gets completed and the agent gets the result back.
This only requires private information of the agents of its past performances.
Being constantly updated upon job completions, this leads to responsiveness to
environmental changes. The metric used to measure the performance of the
model is the average load of the resources in the Grid, weighted by the capacity
of each resource. In Galstyan study, a comparison between random, least loaded
with global information (subject to probabilistic fails in the information update)and RL selection mechanism is performed. The results suggest that, under a
small level of failure in resource information update, the RL algorithm performs
well compared with least loaded mechanism, and outperforms clearly random
selection mechanism. RL can also cope with stochastic reward information, no
matter whether the noise in the reward information is due to variance in actual
resource behavior or in reporting.
The model described in the previous paragraph is our baseline. Now consider an
optimization using the Group Selection Pattern (depicted in Fig. 2). Each
scheduler agent can only select the next resource from the resources belonging
to its same group. This means also that agents only learn about resources on
their current group. The idea behind this segmentation is to boost Q-tables
17
-
8/3/2019 A Group Selection Pattern Applied to Grid
18/33
convergence through a reduced learning space, implying reduced Q-tables
updates. I.e., we partition in groups the learning space. Agent migration to
another group and group mutation to create a new group encompasses also the
migration to that group of the resource hosting the agent. In this pattern
instantiation, the RL Interaction and Migration rules take the following form:
RL Interaction rule: apply learning upon any delayed reward available (update Q-
tables) and apply e-greedy selection over the set of resources in the group to
decide the resource the next job will be submitted to.
RL Migration rule: Agents (with their resources) move between groups depending
on the relative fitness. Agents will migrate to groups with agents outperforming
themselves. That is, compare my own performance to the performance of
another agent in a different group. If the other agent is outperforming me, then
move to its group. This operation is applied with a migration probability and some
variability is added trough a mutation probability, implying the creation of a brand
new group on its own by the agent.
The experimental setup is as follows: we fix an agent population of 100
resources processing jobs and 100 agents submitting jobs. The resources are of
variable capacities (as typical in Grid settings) in the interval (0,100), measured
in abstract processing power units. A new job submission is performed in each
18
-
8/3/2019 A Group Selection Pattern Applied to Grid
19/33
agent at a fixed rate on each simulation tick. The higher this rate becomes, the
higher the task load on the resources. Jobs can be have both fixed size (50) and
variable sizes (10,100), measured in abstract processing power required units.
For too high values of the job submission rate, the load in the resources will grow
infinitely and the users wait time will follow that growth. For each experiment, we
follow the evolution of the system over 2000 ticks simulation period. Jobs
submission (following the job submission rate probability) and job processing at
resources advance at each simulation tick in the discrete-event simulator.
In our baseline implementation in the simulator, we fix a job submission rate of
0.4 and replicate results close to the simulations by Galstyan, using Q-learning
algorithm for Qvalue updates. The Q-learning algorithm (see Equation 1)
operates with a learning rate indicating how much the new experience is taken
into account.
The e-greedy resource selection for next job submission is performed following
the Q-table with high probability, (1-e), which means selecting the fittest resource
so far for the next job scheduling, and exploring randomly a new resource with a
low probability e. We take to be = 0.1 and e to be e=0.01.
We measure the system centric metric average resource load as the total job
queue length divided by the resources capacity. The final metric is the average of
this value over the set of resources in the Grid. The goal of the adaptive
schedulers (the agents) is achieving load balancing, minimizing averageresource load metric. We provide a complementary user centric metric: the wait
time for each job to be completed in the resource, weighted by the job size.
Averaging this over the set of jobs submitted into the Grid by an agent we get the
average weighted response time (AWRT) metric typically used to measure user-
centric performance in Grid settings.
19
-
8/3/2019 A Group Selection Pattern Applied to Grid
20/33
Lets try scaling up the baseline results for RL schedulers. Increasing the job
arrival rate to higher values (from 0.4 to 0.5), we want to see how does the RL
behave and what benefits can we obtain using utility-based dynamic groups on
the agents populations. We can see from Fig. 3 that RL in a flat population is not
able to scale to a higher job submission rate of 0.5. The performance degrades in
this case almost in an attenuated linear increase. If we introduce the Group
Selection mechanism with small and dynamic groups, the RL agents are able to
cope with the increased workload pretty well and stabilize from the very
beginning. This means that schedulers became more adaptive just by applying
segmentation in groups and a simple utility-based agents migration process. We
try now to vary the Group Selection mechanism parameters to see which values
are leading to a better performance. Two parameters rule the dynamics of the
mechanism: first, the migration probability. This is the probability at which the
agents apply migration rule, that is to compare with other schedulers in the
system and move to their group in the case the scheduler outperforms
themselves. This basically rules the rate at which the agent is testing the
existence of better congregations of schedulers and resources. The other
parameter, the mutation probability, rules the extent to what the agent decides toexplore a brand new group, starting a group on its own and waiting for others to
join. It is important to notice the importance of this parameter, since setting this to
0 would provoke a quick convergence to one single group (the original flat
setting). Group mutation needs to be enabled in order to introduce variability; in
the same sense exploration is often required in RL algorithms.
We see from the results in Fig. 4 that, for a fixed mutation probability of 0.1, a
comparable migration probability leads to too many groups generated by
mutations, around 80, which mean that most of the agents are isolated, and the
system becomes useless for grouping agents and balancing load in the system.
Increasing the migration probability to 0.3 and then to 0.5 leads to improvements
on the load balancing Further increasing the migration rate upper than 0.5, the
20
-
8/3/2019 A Group Selection Pattern Applied to Grid
21/33
improvements are negligible. The high variation in groups is provoked by the full
variation in a mechanism parameter, namely the migration probability in each
single simulation. This high variation in the number of groups shows that a high
migration probability provokes a higher variation in the number of groups.
The number of dynamic groups in this migration to mutation ratio is around 40
groups. This means having a large number of very small groups, which partition
learning spaces in very small (but dynamic) sets. Agents learning in those sets
converge quickly and adapt easier to the moves of the rest of independent
schedulers in the system.
SEGMENTATION IN MARKET BASED RESOURCE
MANAGEMENT
21
-
8/3/2019 A Group Selection Pattern Applied to Grid
22/33
An alternative resource management mechanism uses a set f bargaining agents
in a decentralized Grid market. Decentralized Grid markets based on catallactic
agents have been proposed as suitable coordination mechanisms for Grids and
Service Oriented Architectures. The market here is nothing more than a
communication bus it is not a central entity and does not participate in
matching participants requirements using optimization mechanisms as in typical
centralized markets (auctions).
The Catallaxy mechanism was originally proposed by von Hayek. The Catallaxy
approach is a coordination mechanism for systems consisting of autonomous
decentralized agents that makes use of a free-market self organization
approach. It enables prices within the market to be adjusted based on constant
negotiation and price signaling between agents. Catallactic agents for Grid
resource allocation are explored in the CATNETS project.
It is intuitively clear that decentralized bidding in a huge flat market is not the
most efficient setting for any given number of buyers and sellers in a Grid market.
The higher the heterogeneity and dynamicity of the system are, the bigger is theprobability of un-coordinated outcomes leading to low system-wide performance.
We propose to extend standard catallactic Grid markets, enabling the
bootstrapping of the agents in submarkets and the automatic evolution of these
groups towards optimized market segments. Hayek itself supported the idea of
Group Selection as a transmitter of free market norms and institutions between
societies. The idea behind market segmentation is identifying groups of similar
customers and potential customers, to prioritize the groups to address and to
respond with appropriate strategies that satisfy the different preferences of each
segment. Markets segments can be set up manually, but this clearly does not
scale nor adapt to open Grid systems. Group Selection self-organizes market
segments evolution in a scalable manner which works with large scale open
systems.
22
-
8/3/2019 A Group Selection Pattern Applied to Grid
23/33
In Fig. 5 (left) we see an example of a typical flat catallactic market. Two brokers
(Client 1 and 2) try to contact service providers in order to engage into bilateral
negotiations with them. In this example, the discovery mechanism finds 4 Grid
services providers (Services 1 to 4). Each service provider maintains a different
negotiation type, defined by its strategy, contractual implications, and legal
issues and so on. An important issue is that negotiation type information must be
kept private in order to avoid cheating agents with incentives to free-ride on other
agents. In order to simplify, we reduce in the example the negotiation type space
to two different types, S1 and S2. In this case, we can see that Client 1 with
strategy S1 will end up negotiating with Service 2 and Service 4, which are more
compatible with Clients using S2. This is a case of miss coordination leading to
spurious negotiations which should be avoided since they are time and resource
consuming.
We build on automatic market segmentation in order to optimize an agents
discovery process. Instead of performing decentralized search and negotiation
over the whole set of services (in a single market), as it is done in a typical
catallactic market, we propose a segmentation of the markets for services inspecialized VOs, each identified by a tag. Once the negotiation partner is chosen
with the enhanced discovery, the economic negotiation proceeds in exactly the
same way as it would do in a normal catallactic market. A minimal segmentation
of the Grid market is depicted in Fig. 5 (right).
23
-
8/3/2019 A Group Selection Pattern Applied to Grid
24/33
After one market iteration, and after utilities from the services executions are
derived, the users send back their perceived utility as feedback. This can be a
simple metric such as the service provision time, up to more elaborated metric
comprising QoS. Then, Clients and Services must decide how to evolve on the
different market segments. They can compare fitness with agents in other VOs,
migrating to VOs where they find outperforming agents. In this scenario, it is this
negotiation types closeness characteristic being automatically selected at the
group level. In this manner Client 1 and Client 2 will interact preferentially with
Services belonging to the same group, which tend to be Services closer to them
in negotiation abilities and goals, hence increasing the probability of successful
allocations.
In this pattern instantiation, the Market-based Interaction and Migration rules take
the form:
Market-based Interaction rule: Negotiate with agent/agents from my market
segment. Calculate allocation utility and perform job submission following market
allocations.
24
-
8/3/2019 A Group Selection Pattern Applied to Grid
25/33
-
8/3/2019 A Group Selection Pattern Applied to Grid
26/33
to model realistic scenarios with communication costs, we limit the scope of the
CFP to 5 Services to be reached on each groupcast, selected at random either
from the whole population (in the flat scenario) or from agents belonging to the
same group (in the Group Selection scenario).We introduce five different
negotiation types. Utility derived by clients is related to the closeness between
negotiation types. The closer the types are, the higher the utility derived. In the
baseline with a single flat decentralized market, agents try to learn negotiation
types of other agents by adopting the negotiation types of agents outperforming
them. In the Group Selection scenario, the same learning process is structured:
agents migrate with a high probability to other groups whenever they find other
agents achieving higher payoffs. This migration implies the copy of the
outperforming agent negotiation type.
We measure the utility extracted from allocations as directly proportional to the
negotiation type closeness.
Given M different negotiation types, negotiation types are represented by
integers from 1 to M. We use (2) for utility calculation, with 1 nt being the
negotiation type of buyer agent 1 and 2 ntbeing the negotiation type of seller
agent 2. The exception is that when 1 2 nt ntwe consider the maximum utility 1
achieved. The rest of cases oscillate between 0 and 1.
26
-
8/3/2019 A Group Selection Pattern Applied to Grid
27/33
If we compare a flat market with a grouped setting evolving through Group
Selection, we see how the latter coordinate easier the negotiation types, leading
to better accumulated allocation utilities (Fig. 6, left). Prices vary smoothly
influenced by offer/demand in the bounds 80 to 70 in the Group Selection
scenario). The price stability around the initial selling prices (75) renders the
market fair (Fig. 6, right). The price evolution for the baseline flat market is
similar; hence we do not show the graph. We can conclude that market
segmentation is able to increase resource allocation utilities of the traders without
compromising price stability. A simple decentralized economic algorithm can be
optimized in allocation utilities by applying a Group Selection process, boosting
the convergence in the same groups of traders to compatible negotiation types. It
is easier for agent segmented in groups to find compatible trade partners than for
agents in the flat decentralized market.
We try now to vary the Group Selection mechanism migration probability and
mutation probability, to see which values are leading to a better performance.Intuitively, since migration tends to homogenize groups while mutation creates
brand new sub-markets, the lower the relation migration/to mutation, the larger
number of groups. We fix a mutation probability of 0.01. This is one order of
magnitude smaller than in the RL scenario. For the Market based resource
allocation scenario, too much new sub-markets would affect traders which are
27
-
8/3/2019 A Group Selection Pattern Applied to Grid
28/33
not able to reach agreements. We need for a slower paced learning in this
scenario. Having a fixed mutation probability, we conduct a parametric study of
the migration probability. From the results in Fig. 7 (left) we see that, for a
migration probability of 0.1, the utility is comparable to the case when migration
probability is 0.3. Increasing the migration probability to 0.5, and then to 0.7,
decreases performance. The best configuration corresponds to a Grid
segmented in about 15 to 20 sub-markets, each of 5 or so agents, as shown in
Figure 7(right). This supports the same conclusion as for the RL scenario: small
groups converge quicker to compatible negotiation types.
ALIGNMENT IN POLICY BASED RESOURCE MANAGEMENT
VOs have several key properties that distinguish them from traditional IT
architectures: autonomy of its members, which behave independently,
constrained only by their contracts; heterogeneity of its members, which are
independently designed and constructed, constrained only by the applicable
interface descriptions; dynamism of the VO members which can join and leave
with minimal constraints, affecting the configuration of a VO at runtime; structure,
with VOs having complex internal structures, reflected in the relationships among
their members. Importantly, even in cases where the above properties are not
required (such as within an enterprise where the members are controlled by one
party), it is appropriate to architect a VO as if it had the above properties.
28
-
8/3/2019 A Group Selection Pattern Applied to Grid
29/33
In our VO model, each agent (representing an organization) has a policy A = A
(p), from a set of M policies. A VO consists of a set of agents. VO = {A1 (p1), A2
(p2), A3 (p3).}. The VO defines the scope of agent operation. Policy based
resource sharing utility gets maximized by coordinating the policies of the M
agents forming each VO. The objective is achieving policy coordination in each of
the VOs in the system, forming C clusters out of the N agents in the population,
with most agents in each cluster using compatible policies. The compatibly
depends on the specific scenario and is measured differently depending on it.
Fig. 8 (left) shows a flat Grid without VOs where users of any policy can access
providers of any policy. Fig. 8 (right) shows a Grid organized in VOs whose
members policies are compatible, thus maximizing members utility.
Clearly, for policy based resource management, the scenario with group
structure outperforms the flat arrangement. The challenge in this case is how to
manage policy coordination in dynamic VOs. We aim to achieve automatic policy
coordination by means of the Group Selection pattern. The two rules of the
pattern are instantiated as follows.
29
-
8/3/2019 A Group Selection Pattern Applied to Grid
30/33
Policy Coordination Interaction rule: We use collective interaction inside the
group, with a payoff shared equally between the agents composing the group.
We have implemented the simplest of the collective interactions, corresponding
to a VO policy alignment scenario. The payoff is calculated on the alignment level
over the whole VO.
Policy Coordination Migration rule: The agents compare their performance
against their own past performance (internal learning). Migration to a group
implies the copying of the policy of one random agent in this target group. This
maps VOs configurations of large pools of resources optimizing their
performance by acting together with a similar policy. The goal is to minimize
diversity (entropy) within each VO, achieving the highest policy alignment
possible.
Equation 3 implements our metric for this scenario. i p stands for one of the
policies in a set of size M. Minimizing the index is equivalent to maximizing
homogeneity in VOs. We reverse this measure to have a performance scale from
30
-
8/3/2019 A Group Selection Pattern Applied to Grid
31/33
0 to 1: This gives 1 when all policies in the group are aligned (minimal entropy)
and 0 when all present policy in the group is represented equally (maximal
entropy).
The total number of agents is N=100. The set of different policies is M=10.
Mutation probability is fixed to a small value of 0.01 like in the markets
segmentation scenario. We see from Fig. 9 (left) that reaching a high alignment
(low entropy) up 0.8 is possible before 1000 rounds and maintained afterwards
for a migration probability of 0.3.Fig. 9 (right) shows that the number of VOs
oscillates in this case between 10 and 20. If we use a higher migration probability
of 0.7, this implies a higher ration migration/mutation and consequently less
number of groups in the system in average. As we can see from Fig. 9, larger
number of small groups generates better coordination, which recovers the same
conclusion as in precedent scenarios.
CONCLUSION
In large scale Grids, system dynamicity and uncertainty are high; automatic,
decentralized and self-organized control becomes a requirement. Our proposal is
that a simple, rather powerful coordination mechanism based on Group Selection
can be used to self-organize a set of agents in VOs to operate more effectively
on a Grid environment. The Group Selection mechanism complements rather
than compete with much of the existent coordination mechanisms for MAS. We
have formalized a distributed systems engineering pattern Group Selection
pattern, amenable to be instantiated in several Grid resource management
scenarios. We have shown, by simulations in a general purpose agent-based
Grid simulator, how splitting the Grid participants (agents and resources) in
groups and further evolving those groups based on utility can lead to optimization
in several Grid resources management tasks.
31
-
8/3/2019 A Group Selection Pattern Applied to Grid
32/33
First, we have shown how to apply the pattern to evolve congregations of
independent RL schedulers, leading to optimized operation/learning in groups,
compared with a baseline with a flat population of schedulers. We have also
analyzed which values for the parameters of the mechanism (the group migration
to group mutation rate) achieve better performance, concluding that generating
trough this rate a population of very dynamic and small groups achieves the best
optimization in this scenario. Evolving smaller Q-tables allows for the RL agents
to converge quicker to coordinated resource usage, and changes or problems in
resources can be quickly addressed by migration to new groups or creation of
new groups trough mutation. In general RL schedulers learn quicker and adapt
quicker to changes in dynamic, small groups.
In the case of plugging the mechanism into a decentralized Grid market, each
group represents a sub-market of the market segmentation, and migration of
agents from one sub-market to another is ruled by the Group Selection process.
The results extracted by simulations show higher profits for the society of agents
trading in decentralized Grid markets which structure the population in sub-
markets and incorporate Group Selection, compared to the flat decentralized
markets. The performance results show that agent-based automatic fair tradingof resources at stable prices can be achieved using the decentralized market
mechanism coupled with the Group Selection process. Studying the impact of
varying migration to mutation rate draws a similar conclusion as in the previous
scenario: agents trading in small groups are able to converge quicker to more
compatible negotiation types.
As for plugging the mechanism into policy-based VO management scenario, the
migration of agents from one VO to another and its policy adaptation by aligning
policies with the agents in the target groups is ruled by the Group Selection
process, achieving optimization of policy-based resource sharing utilities. The
results extracted by simulations show how small groups of agents can align
32
-
8/3/2019 A Group Selection Pattern Applied to Grid
33/33
quicker and to a higher degree (following Shannon entropy index metric) their
resource sharing policies.
Clustering in small groups is a tendency largely observed in all kind of human
organizations. In cooperation building scenarios, it has been shown that smaller
group sizes ease cooperation in both social-networks based cooperation and
Group Selection based evolution of cooperation. Our results for the evaluation of
Group Selection in three different Grid coordination scenarios suggest that the
same conclusion applies in fully cooperative domains: small and dynamic
groups of agents evolved trough Group Selection optimize better fully
cooperative coordination scenarios. In our Group Selection pattern, the group
migration to mutation rate determines the average number of dynamic groups in
the systems. A rate tuned to evolve dynamic and small groups achieves the best
optimization in all three scenarios. In the adaptive scheduling scenario, just group
membership is evolved in order to segment learning Q-tables of agents. In the
other two scenarios (market segmentation and VO policy based resource
sharing) an agent characteristic (negotiation type and policy respectively) its also
evolved via imitation upon migration to new groups. For the population of 100
agents evaluated in the simulator, the specific migration to mutation rate (this isscenario-dependent) dynamically generating small groups of around 5 agents
achieved the best performance in all three cases.
Future work includes considering Grid users and services belonging to many
VOs simultaneously, mapping more realistic scenarios. Deployment of the
pattern into a Grid prototype can add more insights into practical feasibility of the
approach.