Distributed Intelligent Systems – W6Machine-Learning and Multi-Machine Learning and Multi

Level Modeling Methods Applied t Di t ib t d R b ti S tto Distributed Robotic Systems

(Part I and II)( )

Outline – Part I: Machine-Learning• Challenges in multi-robot scenarios

– Credit assignment problemsg p– Single-vehicle adaptation in a world

shared by multiple vehicles

• Co-adaptation strategies• Co-shaping obstacle avoidance

– PSO vs. GA– Off-board centralized vs. on-board

distributed implementationdistributed implementation– Effect of physical constraints

• Co-shaping coordinated motionCo shaping coordinated motion

F Si l t From Single to Multi Unit Systems:Multi-Unit Systems:Co-Adaptation in a Co-Adaptation in a

Shared WorldShared World

Adaptation in Multi-Robot ScenariosAdaptation in Multi Robot Scenarios

• Collective: fitnessCollective: fitness become noisy due to partial perception, independent parallelindependent parallel actions

Credit Assignment ProblemCredit Assignment ProblemWith limited communication, no communication at all, or partial perception:

Design and Optimization of a Design and Optimization of a Sensory System for an Sensory System for an

Intelligent Vehicle

A Reasonable Traffic SimulatorA Reasonable Traffic Simulator

2001 2003 W b 2 ki i i l diff i l2001-2003: Webots 2, kinematic simulator; differential wheel vehicles; more later in the course

Evolving Sensor Configurationg g

• Number nNumber n • Type

R– Range ρ– Cone of view δ– Cost f(ρ, δ)

• Placement– Position φ– Orientation θ

An Expensive Optimization ProblemAn Expensive Optimization ProblemThe design and optimization of the sensory system is an expensive optimization problem because:

1. Large evaluation time of candidate solutions

an expensive optimization problem because:

Recipe: test candidate solutions on more abstract models

2. The performance evaluation is noisy (sensor and actuator noise; local perception in a loosely coordinated system solely based on traffic rules and road template) Recipe: noise resistant algorithms with aggregationRecipe: noise-resistant algorithms with aggregation

3. Search space high dimensionality (actually unbound)Recipe: formulate the problem as an engineering trade-offRecipe: formulate the problem as an engineering trade-off so that solution remains in a reasonable dimension space

Reducing the Evaluation TimeReducing the Evaluation Time

1. Exploit underlying computationally expensive level to generate a

b bili i di ib iprobabilistic distribution2. Randomly allocate vehicles in

detection region using this distributiondistribution

3. Evaluate a candidate solution based on a certain number of randomly placed vehiclesrandomly placed vehicles

V hi l O P b biliVehicle Occurrence Probability

• 1-Dimensional (angle) • 2-Dimensional

Probabilistic Evaluation Tests

Vehicles to be

• 1-dimensional • 2-dimensional

detected

T t hi lTest vehicle Detection region

Relative Time Cost

Evaluation Tests Time Cost Relative Cost

Static 4.6 s 11D Full Coverage 9.7 s 2.12D Full Coverage 10 3 min 1342D Full Coverage 10.3 min 1341D Probabilistic 15.6 s 3.42D Probabilistic 8 9 min 1162D Probabilistic 8.9 min 116Embodied (kinematic) 10.4 hours 8100

Example: 2500 evaluations conducted on computers with 1 5 GHz AMD CPUExample: 2500 evaluations conducted on computers with 1.5 GHz AMD CPU

Generalized Aggregation Functionsgg gAggregation function operators Ps [Scott and

Antonsson 1998]Antonsson 1998]

• μxx: fuzzy preference on factor xx• ωxx: weight on factor xx• s: degree of compensation (express the engineering trade• s: degree of compensation (express the engineering trade-

off strategy; higher values means higher willingness to find compromise among preferences)

• For design trade-offs (some compensation):• Perfect weighted sum (neutral): s = 1

l f f /d i f b• Note: Example for 2 factors/design preferences; can be generalized to arbitrary multiple factors

Properties of GeneralizedProperties of Generalized Aggregation Functions

[Scott and Antonsson 1998]

Fitness Function Formulated as E i i T d OffEngineering Trade-Off

1ss

coveragescost μωμ ⎟

⎞⎜⎛ ⋅+ μ : fuzzy preference on factor xx

1),( coveragecostsFitness

ω

ωμμ

ω ⎟⎟⎠

⎜⎜⎝ +

= μxx: fuzzy preference on factor xx(0: totally unacceptable;1: completely acceptable)

ω : weight on factor xx

Vcost

coverage

ωω

ω =ωxx: weight on factor xxs: degree of compensationV: actual number of vehicles

been in the detection region ),PDF(

1n

iiii rkCoverage α=∑

=

gduring an evaluation span

ki: 1 if the ith car at distance riand approaching angle αi

)(1

n

ii

fcost

costCost

δρ

=∑=

detected; 0 if not detectedn: number of sensors usedρi: ith sensor’s rangeδ ith ’ f i),( iii fcost δρ= δi: ith sensor’s cone of view

Preference Functions• Preference function given by the designer and/or the customer on all

relevant criteria of a candidate solution• Mapping of the actual cost in a normalized cost taking into account the pp g g

preferences• Expressed here with fuzzy sets (0: totally unacceptable; 1: completely

acceptable)

0.8

0.9

1

0.8

0.9

1

μcost

acceptab e)

μcoverage

0.5

0.6

0.7

0.8

over

age

0.5

0.6

0.7

0.8

μ cost

μcost

0.2

0.3

0.4μ cov

0.2

0.3

0.4

μ c

0 0.2 0.4 0.6 0.8 10

0.1

Coverage0 5 10 15 20 25

0

0.1

Total cost

Sample Results• Fitness evolution process and the final best design• Maximal compensation among coverage and cost• Higher importance of cost vs. coverage

s = 0, ω = 3/17

Coverage = 53%, Total_cost = 4.6

Sample Results• Fitness evolution process and the final best design• No compensation between coverage and cost

s = -∞, ω arbitrary

Coverage = 82%, Total_cost = 7.7

Sample Results• Fitness evolution process and the final best design• Maximal compensation among coverage and cost

s = 0, ω = 4• Higher importance of coverage vs. cost

Coverage = 98%, Total_cost = 11.9

Sample Results• Evolved approximate Pareto frontier for the design trade-offs

present in the case studyE h d t i t t th b t lt f ti l• Each data point represents the best result of one particular evolutionary experiment under a given combination of ω and s

[Zhang et al., Research in Engineering Design, 2008 ]

Co-Adaptation in a pCollaborative Framework

Co-Shaping Collaborative Behaviorp g

Three orthogonal axes to consider (extremities and b l d l i ibl )balanced solutions are possible):

1. Performance evaluation: individual vs. group fitness or reinforcement

l i h i2. Solution sharing: private vs. public policies

3 T di i3. Team diversity: homogeneous vs. heterogeneous learning

Search Algorithms for Multi-RobotSearch Algorithms for Multi Robot Systems

Policy Performance Sharing Diversity

i-pr-he individual private heterogeneous

i-pr-ho individual private homogeneous

i-pu-he individual public heterogeneouspi-pu-ho individual public homogeneous

g-pr-he group private heterogeneousg pr he g p p g

g-pr-ho group private homogeneous

g-pu-he group public heterogeneousg pu he group public heterogeneous

g-pu-ho group public homogeneous

Search Algorithms for MR Systems

Do not make sense (inconsistent)

Interesting (consistent)

Possible but not scalable

Policy Performance Sharing Diversity

i pr he individual private heterogeneous

g ( )

i-pr-he individual private heterogeneous

i-pr-ho individual private homogeneous

i pu he individual public heterogeneousi-pu-he individual public heterogeneous

i-pu-ho individual public homogeneous

h i t h tg-pr-he group private heterogeneous

g-pr-ho group private homogeneous

g-pu-he group public heterogeneous

g-pu-ho group public homogeneous

Population-Based Search Algorithmsfor Multi Robot Systems

Example of collaborative

for Multi-Robot Systems

Example of collaborative co-learning with binary encoding of 100 candidate solutionscandidate solutions and 2 robots

Co-Shaping Competitive BehaviorCo Shaping Competitive Behavior

fi f ≠ fi ffitness f1 ≠ fitness f2

Three Case Studies• Obstacle avoidance (today)

C di t d ti (t d )• Coordinated motion (today)• Collaborative stick pulling (next week)

Notes: 1. each case study investigates different strategies for y g g

distributing, sharing, and evaluating candidate solutions using a multi-robot system

2. each case study investigates additional peculiar methods (e.g., impact of on-board distributedness, leveraging different modeling abstractionsleveraging different modeling abstractions, measuring diversity and specialization)

Co-Shaping Obstacle Co Shaping Obstacle Avoidance

• Effects of robotic group sizeEffects of robotic group size• Effects of communication constraints

Population-Based Search Algorithms for Multi-Robot Systems

Distributed Robotic Adaptation• Standard robotic adaptation approach: evaluate

population in serialpopulation in serial• New distributed adaptation approach: robots

l t diff t ti l i ll levaluate different particles in parallel • Potentially much faster adaptation rate• Why PSO:

– appears interesting since this metaheuristic works pp gwell with small pools of candidate solutions: candidate pool size ≈ robot team size

– limited particle neighborhood sizes → scalable, on-board operation

Varying the Robotic Group SizeVarying the Robotic Group Size

Varying the Robotic Group SizeVarying the Robotic Group Size

Varying the Robotic Group SizeVarying the Robotic Group Size

Var ing the Robotic Gro p Si eVarying the Robotic Group Size

Varying the RoboticG Si GA PSO

• Same control architecture as [Floreano & Mondada, 1996] (ANN 22 i h )

Group Size – GA vs. PSO

1996] (ANN, 22 weights to tune)• Same fitness function as [Floreano & Mondada, 1996]• Similar Webots world as [Pugh et al., 2005] but 2x2 m• Robot group size: 1, 2, 5, 10, 20• GA and PSO parameters:

Varying the RoboticGroup Size – GA vs. PSO

Performance of best controllers after evolution

[Pugh and Martinoli, AAMAS 2006]

Varying the Robotic Group Size –E l i T ti E i tEvolving vs. Testing Environment

• PSO only; numberPSO only; number of particles = 10

• Gradually increase ynumber of robots on team

• Up to 10x faster learning with little performance loss

• Arena 3x3 m

[Pugh and Martinoli, Swarm Intelligence J., 2009]

Communication-Based Neighborhoods

• Default neighborhood ring topology 2• Default neighborhood - ring topology, 2 neighbors for each robot

• Problem for real robots: neighbor could be• Problem for real robots: neighbor could be very far away

• Possible solutions – use two closest robots in• Possible solutions – use two closest robots in the arena (capacity limitation), use all robots within some radius r (range limitation); w t so e ad us ( a ge tat o );reality is affected often by both

• How will this affect the performance?ow w s ec e pe o ce?

A Possible Robot DistributionA Possible Robot Distribution

Index-Based Neighborhoodg

Ring Topology - Standard

Communication-Based NeighborhoodsNeighborhoods

2-closest – Model A

Communication-Based N i hb h dNeighborhoods

Within range r – Model B

Communication-Based NeighborhoodsCommunication Based Neighborhoods• Realistic simulation results

(Webots)(Webots)• 10 Khepera III robots (one

particle per robot)• Standard neighborhood is 2• Standard neighborhood is 2

fixed particles• New particle

neighborhoods defined byneighborhoods defined by communication limitations:– Model A: particles from

closest two robots– Model B: particles from all

robots within range r = 120 cm

• No drawback to• No drawback to communication-based neighborhoods[Pugh and Martinoli, Swarm Intelligence J., 2009]

V i th C i ti RVarying the Communication RangeArena: 3x3 m, 10 robots total,

Varying the Communication Range• Realistic simulation results (Webots)• Low communication ranges → learning too slowly

High comm nication ranges fitness platea s too earl• High communication ranges → fitness plateaus too early• Limited communication range gives best performance →

scalable solution

[Pugh and Martinoli, Swarm Intelligence J., 2009]

Distributed Adaptation with Real RobotsEnabling Technology

• See Week 4 slides• On board relative positioning system

- Enabling Technology

• On-board relative positioning system• Principle:

– belt of IR emitters (LED) and receivers ( h di d )(photodiode)

– IR LED used as antennas; modulated light (carrier 10.7 MHz)RF hi b hi d d RSSI– RF chip behind, measured RSSI

– Measure range & bearing of the next robot (relative positioning) can be coupled with RF channel (e g 802 11) for headingRF channel (e.g. 802.11) for heading assessment

– Can also be used for 20 kbit/s com channel

• [Pugh et al., IEEE Trans. on Mechatronics, 2009]

Distributed Adaptation with Real Robots- Sample Results

• All operations on-board using relative positioning system• Effective learning of obstacle avoidance behavior• Effective learning of obstacle avoidance behavior• Similar results for communication-based neighborhoods• Arena 3 x 3 m

[Pugh and Martinoli, Swarm Intelligence J., 2009]

Distributed Adaptation withR l R b tReal Robots

Before adaptation (5x speed-up)

Distributed Adaptation withR l R b tReal Robots

After adaptation (5x speed-up)

Evolving Coordinated MotionEvolving Coordinated Motion• Physically connected robots• Group performance• Group performance

The SWARM-BOTS project (2001-2005)http://www.swarm-bots.org

We call “swarm-bot” an artifact composed of a number of simpler robots, called “s-bots”, capable of self assembling and self organizingcapable of self-assembling and self-organizing to adapt to its environmentS bots can connect to and disconnect from eachS-bots can connect to and disconnect from eachother to self-assemble and form structures whenneeded, and disband at will,

The coordinated motion task

• Four s-bots are connected in a swarm-bot formation

• Their chassis are randomly oriented

• The s-bots should be able to – collectively choose a direction of

motionmotion – move as far as possible

• Simple perceptrons are evolved asSimple perceptrons are evolved as controllers

Coordinated motion:h iThe traction sensor

• Connected s-bots apply turretturret

pp ypulling/pushing forces to each other when moving

• Each s-bot can measure a traction force acting on its turret/chassis

i tractiontractionconnection• The traction force indicates the

i t h b t

traction sensortraction sensor

mismatch between – the average direction of motion of the

groupgroup– the desired direction of motion of the

single s-bot

Coordinated motion:The evolutionary algorithmi d d• Binary encoded genotype

– 8 bits per real valued parameter of the neural controllers• Generational evolutionary algorithmGenerational evolutionary algorithm

– 100 individual evolved for 100 generations– 20 best individual are allowed to reproduce in each

generationgeneration– Mutation (3% per bit) is applied to the offspring

• The perceptron is cloned and downloaded to each s-boteach s-bot

• Fitness is evaluated looking at the swarm-botperformance– Each individual is evaluated with equal starting

conditions

Population-Based Search Algorithms for Multi Robot Systemsfor Multi-Robot Systems

Coordinated motion:Fit l tiFitness evaluation

• The fitness F of a genotype is given by the g yp g ydistance covered by the group:

where X(t) is the coordinate vector of the center of mass at time t, and D is the maximum ,distance that can be covered in 150 simulation cycles

• Fitness is evaluated 5 times (fixed number per• Fitness is evaluated 5 times (fixed number per candidate solution!), starting from different random initializationsTh lti i i d t th t• The resulting average is assigned to the genotype

Coordinated motion: Results Post-evaluationPost-evaluation

Replication Performance1 0.878882 0.83959

Average fitnessAverage fitness

2 0.839593 0.883384 0.715675 0 795735 0.795736 0.752097 0.834258 0.858489 0.87222

10 0.76111

Coordinated motion:Coordinated motion: Real s-bots

flexibilityflexibilitydefault (used for evolution)default (used for evolution)

Coordinated motion:Coordinated motion: Scalability

flexibility and scalabilityflexibility and scalabilityscalabilityscalability flexibility and scalabilityflexibility and scalabilityscalabilityscalability

ConclusionConclusion

Take Home Messages• Machine-learning techniques can be used not only for design and

optimization of controllers but also of further hardware features pwith the help of appropriate simulation tools

• The cost of an optimization problem is heavily influenced by the amount of noise in the evaluation function the time needed foramount of noise in the evaluation function, the time needed for evaluating a candidate solution, and the dimension of the parameter space

• Collaborative co adaptation strategies can be differentiate along• Collaborative co-adaptation strategies can be differentiate along three axes: public/private solutions; homogeneous/heterogeneous system, individual/group performance

• Multi-robot platforms can be exploited for testing in parallel multiple candidate solutions

• PSO appears to be well suited for fully distributed on-board pp yoperation and strategies based on local, geographical (com-based) neighborhoods have been successfully tested

O li P II M l i L l M d liOutline – Part II: Multi-Level Modeling

• Multi-Level Modeling Methodology– Rationale– Assumptions

• Methodological frameworkg– Overview– Some theoretical background

• An incremental bottom-up modeling recipe

Modeling Rationale and Modeling Rationale and ChoicesChoices

Motivation for Modeling

• Understanding the interplay of the various elements of the system (e g robot features robot numbersof the system (e.g., robot features, robot numbers, environment, noise level)

• Having additional tools for designing and optimizingHaving additional tools for designing and optimizing the distributed robotic system

• Delivering performance predictions for the ensembleDelivering performance predictions for the ensemble in shorter time or before doing actual experiments

• Investigating experimental conditions difficult or• Investigating experimental conditions difficult or impossible to reproduce in reality

• Formally analyzing system properties• Formally analyzing system properties

Modeling Choices• Gray-box approach: to easily incorporate a priori

information (e.g., # of agents, technological and environmental features)

• Probabilistic: to capture noisy interactions, noisy robotic components, stochastic control policies, and enable

ti h t d b t tiaggregation schemes towards abstraction

l i l l li i l diff d i• Multi-level: to represent explicitly different design choices, trade off computational speed and faithfulness to reality bridge mathematically tractable models and realityreality, bridge mathematically tractable models and reality in an incremental way

Originality and Differences with th M d l B d A hother Model-Based Approaches

• Specifically targeted to (miniature) swarm-robotic systems: exploit robust control design techniques for resource constrained individualrobust control design techniques for resource-constrained individual nodes (e.g. BB, ANN); incorporate submicroscopic constraints (e.g., technology, environment) from start, bottom-up modeling and multi-level system-centered design approachlevel system centered design approach

• Different from traditional model-based techniques in robotics: top-down approaches start from idealized node model and relaxdown approaches, start from idealized node model and relax assumptions; compensate with potential expensive/sophisticated technology and control algorithms for meeting the requirements; (multi-level) control-centered design approach( ) g pp

• Different from traditional modeling in biology (and other natural sciences): as simple as possible macroscopic models targeting asciences): as simple as possible macroscopic models targeting a given scientific question; free parameters + fitting based on macroscopic measurements since often submicroscopic/microscopic information not available/accurate

Experimental Invariant pFeatures and Modeling

A tiAssumptions

Invariant Experimental FeaturesInvariant Experimental Features• Short-range (typically 1 robot diameter), crude (noisy,

a few discrimination levels) proximity sensinga few discrimination levels) proximity sensing• Full mobility but limited navigation (no planning, no

absolute localization)absolute localization)• Limited use of long-range communication channels

available on the platforms (only as a teammate sensor)v b e o e p o s (o y s e e se so )• Reactive, behavior-based control, with a few internal

states• No overcrowded arenas• Multiple runs (typically 5+) for the same experimentalMultiple runs (typically 5 ) for the same experimental

parameters; randomized robot poses at the beginning

Modeling Assumptions g p1. State-based description for environment and multi-

robot systemrobot system2. Semi-Markovian properties: the system future state

is a function of the current state (and possibly of the ( p yamount of time spent in it)

3. Nonspatial metrics for swarm performance4. Well-mixed system: because of simple navigation,

obstacle avoidance, and convex environment the mean spatial distribution of agents over multiplemean spatial distribution of agents over multiple runs can be assumed to be homogeneous

5. Linear superposition of object/robot detection areasp p j(sparse object/robot distribution; no overcrowding)

Assumptions 1 and 2Assumptions 1 and 2• We work with statesWe work with states

S

pin poutTx

Sx

Sx: state xTx: duration of state xpin, pout: probabilities to entry and leave state x

• States can characterize both robots and the environment

Assumptions 3,4,5Assumptions 3,4,5• Trajectories and object location do not count -> 1D

Montecarlo simulationMontecarlo simulation• N objects of type i -> N x (prob. to encounter i)

2D physical space 1D probability space2D physical space 1D probability space

O1

O1O1

O2

O2

O2OFree space

O2O2

Experimental Validation of A i 3 4 5Assumption 3,4,5

PositionN b di d b lNonembodied obstacles = detection surfaces

Shape

Size Square Rect. Round All shapes Geometry

Numerical example (mean ± std dev, 3 locations, 100 h simulated time):

q p y

robot 0.31 ± 0.04 0.3 ± 0.03 0.32 ± 0.02 0.31 ± 0.03 0.31

Experimental Validation of A ti 3 4 5Assumptions 3,4,5

Symmetry DefaultSymmetryof Stick Distribution

Default

# sticks

Methodological Framework: gOverview and Theoretical

B k dBackground

Multi-Level Modeling Methodology

n

g gy

∑ ∑′ ′

′−′′=n n

nnn tNtnnWtNtnnW

dttdN )(),|()(),|()(

S S rics

trac

tionSs Sa

Ss SaS SMicroscopic : multi-agent models,

Macroscopic: rate equations, mean field approach, whole swarm

mon

met

r

Abs

t

me

Ss SaSs Sa only relevant robot features captured, 1 agent = 1 robot

Com

m

ntal

timSubmicroscopic: intra-robot (e.g.,

S&A, transceiver) and environment (e.g., physics) details reproduced f ithf ll

peri

men

Target system (physical reality):

faithfully

Exp

info on controller, S&A, communication, morphology and environmental features

Microscopic LevelMicroscopic Levelp(n,t) = probability of an agent to be in the state n at time tIf Markov properties fulfilled:

−Δ+=Δ tnpttnptnp ),(),(),(

If Markov properties fulfilled:

∑ ∑′ ′

Δ+′−′Δ+=n n

tnptnttnptnptnttnp ),(),|,(),'(),|,(

i fl tflinflow outflow

Probability the agent was in a Transition probabilitygiven state n’

tnttnp ′Δ+ )|(

Sum over all possible states n’ the agent can be in

ttnttnptnnW

t ΔΔ+

=′→Δ

),|,(lim);|(

0Transition rate

Macroscopic Level – Time-ContinuouspLeft and right side of the equation: averaging over the total number of agents, dividing by Δt, limit Δt → 0; neglect di ib i f h h i i bl ( fi ld h)

tdN )( R t E ti

distributions of the stochastic variables (mean field approach):

∑ ∑′ ′

′ ′−′=n n

nnn tNtnnWtNtnnW

dttdN )(),|()(),|()( Rate Equation

(time-continuous)

i fl tflinflow outflown, n’ = states of the agents (all possible states at each instant)N f ti ( b ) f t i t t t ti tNn = average fraction (or mean number) of agents in state n at time tW = transition rates

Macroscopic Level – Time-DiscreteRate Equation (time-discrete):

∑ ∑′ ′−′+=+ nnnn kTNkTnnTWkTNkTnnTWkTNTkN )(),|()(),|()())1((

p

inflow outflow

∑ ∑′ ′n n

k = iteration indexT = time step, sampling intervalTW = transition probability per time stepTW = transition probability per time step

Notation often simplified to:

∑ ∑′ ′

′ ′−′+=+n n

nnnn kNknnPkNknnPkNkN )(),|()(),|()()1(

T i ifi d i th t t f ll P i l l t d f T*W thT is specified in the text once of all, P is calculated from T*W or other calibration methods

Time Discretization: Th E i i R iThe Engineering Receipt

Time-discrete vs. time-continuous models:1. Assess what’s the time resolution needed for your system

performance metrics

Time discrete vs. time continuous models:

2. Consider unit-to-unit interaction essence: probabilistic/deterministic, asynchronicity role, …

3. Choose whenever possible the most computationally efficient model:3. Choose whenever possible the most computationally efficient model: time-discrete less computationally expensive than emulation of continuity (e.g. Runge-Kutta, etc.); in our systems/metrics there is no e idence of decreased prediction acc racevidence of decreased prediction accuracy

4. Advantage of time-discrete models: a single common sampling rate can be defined among different abstraction levels

Methodological Framework: gAn Incremental Bottom-Up

R iRecipe

1 Target System & Task(s)1. Target System & Task(s)Perform basic design choices for the experimental set-up:

• Hardware and software for the robotic platform• Hardware and software for the robotic platform• Environment in which robots operate• Task(s) robots must accomplishTask(s) robots must accomplish

2 Metric(s) and State Space2. Metric(s) and State Space• Define system performance metric(s)• Define state space (number of states, granularity) • Performance metric(s) and state definitions well aligned!• Exploit controller blueprint (if available) as additional source ofExploit controller blueprint (if available) as additional source of

information for defining the state space

Obstacle detected

Search AvoidanceObstacle avoided

C(k) = pg2Ns(k)Ng(k)

Grip

( ) pg2 s( ) g( )

3 Submicroscopic Model3. Submicroscopic ModelImplement faithfully your design choices in a submicroscopic model (in principle even running the samesubmicroscopic model (in principle even running the same control code; libraries and APIs are usually provided in standard commercial or open-source simulators)standard commercial or open source simulators)

4. Microscopic Model• Aggregate local interactions and reduce intra-robot details• Maintain state space’s structure as defined at step 2• Maintain individual representation (and exact discrete quantities) for• Maintain individual representation (and exact discrete quantities) for

each robotic node and environmental object of interest

Ss Sa R11 Se Sd

SiRn1

…Distributed robotic system

Ss Sa

Ss Sa

R12

R1l

…

C t

Si…

Se Sd

Si

Rnm

Caste 1

Coupling (e.g., manipulation)

Caste n

Environment…

O11 O1pOq1 Oqr

… …Sa Sb Sa Sb

5. Macroscopic Model

Distributed robotic system

• Aggregate individual nodes into one or multiple representations (castes) at

Distributed robotic system

Ss Sa

collective level• Maintain state space’s structure as

defined at step 2

Se Sd

Caste1

C

defined at step 2• Solve exactly (stochastically) or

using the ODE approximation if the system is defined as a set of ODEs

Coupling

Si Caste nsystem is defined as a set of ODEs• Exploit conservation laws (e.g. # of

robots in an enclosed arena) to

Type 1

T

Sa Sb

simplify the representation of the dynamical system

Environment

Type q

6. Parametric Calibration• Number of parameters is decreasing with the abstraction level• Calibrate a given level based on the underlying one (e.g.,

b i i i h h i l i i i h

6. Parametric Calibration

submicroscopic with physical system; microscopic with submicroscopic, macroscopic with microscopic)

• Parametric (e.g., mean only, mean and variance) or non t i ( t l di t ib ti d d t th l l l)parametric (actual distribution recorded at the lower level)

assumptions• Various methods available

Ad h i ( i i i f f i )– Ad hoc experiments (e.g., interaction time, sensor transfer functions)– System identification techniques (constrained parameter fitting)– Statistical verification techniques (e.g., trajectory analysis)

• Parameter e ample for micro and macroscopic models:• Parameter example for micro- and macroscopic models: – State durations– State-to-state transition probabilities

pin poutTstate

Conclusion

Take Home Messages• Models help understanding and generalizing properties of

real-time distributed intelligent systems• Three main levels of models: submicro, micro and macro• Macroscopic models using the mean field approach result

i ODEin ODE• Multi-level modeling allows for different approximations,

/ t ti t d ffaccuracy/computation trade-offs• A methodology with precise inter-level mapping has been

developed for a given class of experimentsdeveloped for a given class of experiments• If carefully designed, models allow also for system

optimization and closing the loop between analysis andoptimization and closing the loop between analysis and synthesis

Additional Literature – Week 6Books• T. Balch and L. E. Parker (Eds.), Robot teams: From diversity to polymorphism.

Natick, MA: A K Peters.,• Ross S. M., Introduction to Probability Models, Academic Press, San Diego,

CA, USA, 1997.

PPapers• Pugh J. and Martinoli A., “Multi-Robot Learning with Particle Swarm

Optimization”. Proc. of the Fifth ACM Int. Joint Conf. on Autonomous Agents and Multi-Agent Systems, May 2006, Hakodate, Japan, pp. 441–448.g y y p pp

• Murciano A. and Millán J. del R., "Specialization in Multi-Agent Systems Through Learning". Biological Cybernetics, 76: 375-382, 1997.

• Dorigo M., Trianni V., Sahin E., Groß R., Labella T., Nolfi S., Baldassare G., Deneubourg J L Mondada F Floreano D and Gambardella L “EvolvingDeneubourg J.-L., Mondada F., Floreano D., and Gambardella L.. Evolving Self-organising Behaviours for a Swarm-bot”. Autonomous Robots, 17:223–245, 2004

• Mataric, M. J. “Learning in behavior-based multi-robot systems: Policies, d l d th t ” S i l I M lti di i li t di f ltimodels, and other agents”. Special Issue on Multi-disciplinary studies of multi-

agent learning, Ron Sun, editor, Cognitive Systems Research, 2(1):81-93, 2001.