15 June 2010 ABM Workshop -Leeds

Salem Adra and Phil McMinn

Automated Discovery of Emergent Misbehaviour

in Agent-Based Models

15 June 2010 ABM Workshop -Leeds

Outline

Background:

•Agent-Based modelling

•Agent-Based modelling Applications

Testing Agent-Based Models:

•Motivations and Challenges

•Automated Discovery of Emergent Misbehaviour: Our Vision

•Testing Strategies

15 June 2010 ABM Workshop -Leeds

Background

What is Agent-Based computational modelling?

•Increasingly popular modelling paradigm

•Main Idea:

Simulates the interactions of autonomous entities (agents) with each

other and their local environment to predict higher level emergent

patterns

•Bottom-up procedure

The focus goes into the individual interacting units

•Benefits:

Deeper understanding of the interacting agents and their emergent

behaviour

15 June 2010 ABM Workshop -Leeds

Background

Example of some Agent-Based Modelling Frameworks:

FLAME (C Based)

MASON (Java Based)

RePast (Java Based)

Swarm (Objective C)

15 June 2010 ABM Workshop -Leeds

Background

Agent-Based Models Examples

The Epitheliome Project:(Part of the Physiome Project, a worldwide public domain effort to provide a computational framework for understanding human physiology)

Goal: Modelling the Emergent Behaviour of Epithelial Cells

Benefits:

•Better Understanding of Cell Proliferation and self-organisation

•Capability to explore the roles of a single parameter in a complex biological system

•Helps in the development of reconstruction methods for artificial human skin

15 June 2010 ABM Workshop -Leeds

Background

Prediction and validation about Epithelial scratch wound healing process

Agent-Based Models Examples

15 June 2010 ABM Workshop -Leeds

Background

Modelling of cardiac cells and tissues

Agent-Based Models Examples

15 June 2010 ABM Workshop -Leeds

Background

Macroeconomic Policy Modelling

Agent-Based Models Examples

15 June 2010 ABM Workshop -Leeds

Background

Social Insects: Pharaohs Ant

Agent-Based Models Examples

15 June 2010 ABM Workshop -Leeds

Testing Agent-Based Models

•Motivation:

•ABM are increasingly used to make predictions that can affect organisations and even society

•Hidden errors lurking in such models can be very costly

•E.g. In the banking sector, losses made by NatWest, Barclays and Deutsche Morgan Grenfell totalling tens of millions of pounds were blamed on decisions that involved economic model errors

•The correctness of such models and their thorough testing is vital

•No Formal Testing Technique for ABM has yet been established

15 June 2010 ABM Workshop -Leeds

Agents vs. Objects

“Object-oriented programmers often fail to see anything novel or new in the idea of agents. … While there are obvious similarities, there are also significant differences between agents and objects.”

N. R. Jennings, K. Sycara, and M. Wooldridge, “A roadmapof agent research and development,” Journal of AutonomousAgents and Multi-Agent Systems, vol. 1, no. 1, pp. 7–38, 1998.

Testing Agent-Based Models

15 June 2010 ABM Workshop -Leeds

Agent vs. Objects:

Sources:

(1) James J. Odell: "Objects and Agents Compared", in Journal of Object Technology, vol. 1, no. 1, May-June 2002, pp. 41-53.

(2) Parunak, H. Van Dyke, "Go to the Ant": Engineering Principles from Natural Agent Systems, Annals of

Operations Research, 75, 1997, pp. 69-101.

Evolution of Programming Approaches

Testing Agent-Based Models

15 June 2010 ABM Workshop -Leeds

Agents vs. Objects:

Main Differences: AUTONOMY and FLEXIBILITY

Objects Agents

Behaviour controlled by external entities Self-governed (goals and rules)

Always Say “Yes” - “No” is an error Allowed to Say “No”

Predictive behaviour (Static Functionality – which facilitates inheritance)

Unpredictable - Behave differently in different scenarios (can learn from experience and evolve)

Some examples of differences between Objects and Agents

Testing Agent-Based Models

15 June 2010 ABM Workshop -Leeds

Testing Agent-Based Models

•Challenges:

•ABM are hard to test

•Models often built to predict previously unseen emergent behaviour

•Modellers themselves are not sure what to expect

•Main purpose for modelling in the 1st place

•Model Complexity

•Simulating interactions of thousands and millions of agents is common

•Tracing back certain model behaviour to certain agent or event is very hard

15 June 2010 ABM Workshop -Leeds

Errors and Artefacts in Agent-Based Modelling

Galan et al. (2009), Journal of Artificial Societies and Social Simulation vol. 12, no. 1

Testing Agent-Based Models

15 June 2010 ABM Workshop -Leeds

Without visual inspection this bug would have not been caught

Most modelling work involve batch processing tasks (i.e. no visualisation)

Testing Agent-Based Models

15 June 2010 ABM Workshop -Leeds

Some Suggested Informal Validation Solutions:

1. Application of an ABM to extreme scenarios

2. Re-implementing an ABM using:

• different programming languages,

• programming paradigms

• agent-based modelling frameworks

Testing Agent-Based Models

15 June 2010 ABM Workshop -Leeds

The Misbehaviour Project:

Project’s Aim:

Automated Discovery of Emergent Misbehaviour (ADEM)

Develop testing techniques and frameworks for agent-based software that can lead to the detection of systems’ Emergent Misbehaviour

Towards Formalising ABM testing

Testing Agent-Based Models

15 June 2010 ABM Workshop -Leeds

Approach: Automated Testing Framework composed of 2 main steps

Step 1- Learn and Build up an abstract model of the ABM to be tested

Formulate fitness functions (simpler for task oriented ABMs) and constraints that define the model’s normal behaviour)Tools: Daikon (invariant detection), statistical and visual analysis methods

Step 2- Search-Based Testing Process (using a black box approach)

Search for scenarios, parameters or settings that might force the ABM to uncover “Misbehaviour” or bring it close to an extreme behaviour (i.e. violating a fitness functions and/or constraint)Tools: Metaheuristic techniques (e.g. GAs, Memetic Algorithms, … etc)

Towards Formalising ABM testing

Testing Agent-Based Models

15 June 2010 ABM Workshop -Leeds

ADEM Procedure

Testing Agent-Based Models

Learn Model(DAIKON)

Misbehaviour Detected?

Found Bug

No

Yes

Produce new Behaviour

(GAs)

Max Iterations Reached?

Start

End

No

No Bug Found

Yes

15 June 2010 ABM Workshop -Leeds

•Case Studies: Cancer Research and BT

•1 - Learn real model (i.e. invariants, agent’s properties and normal behaviour)

•2 - Produce buggy models out of the real models

•How?

Mutation Testing: Introduction of artificial bugs

•3 – Apply ADEM Procedure:

Repeat until Misbehaviour Found or maximum iterations reached

IF any Misbehaviour already detected or any Invariant violated

Task accomplished

ELSE

Deploy metaheuristics to search for Misbehaviour

END

15 June 2010 ABM Workshop -Leeds

TheEnd.

ThankYou.