Testing Software Agents & MAS

Cu D. Nguyen, PhD.Software Engineering (SE) unit,

Fondazione Bruno Kessler (FBK)http://selab.fbk.eu/dnguyen/

1

Testing is criticalSoftware agents & multi-agent systems are enabling technologies to build today’s complex systems, thanks to:

¨ Adaptivity and autonomy properties¨ Open and dynamic nature

As they are increasingly applied, testing to build confidence in their operations is extremely crucial !

FACT

49FACT

49

NASA’s agents are designed to achieve the goals andintentions of the designers, not merely to respond topredefined events, so that they can react to unimaginedevents and still ensure that the spacecraft does not waste fuel while keeping to its mission.

NASA satellites use autonomous agents to balancemultiple demands, such as staying on course, keeping experiments running, and dealing with the unexpected, thereby avoiding waste.

2

Software agents and MASSoftware agents are programs that are situated and have their own control and goals.

Properties:

React iv i ty Proact iv i ty

Social abi l i ty

perceive env i ronment

response to changes

Goal -or iented

Autonomous

Del ibera t ive

Col laborat ive Compet i t ive

Multi-agent systems (MAS) are composed of• Autonomous agents and their

interactions• Environment where the agents

operate• Rules, norms, constraints that restrict

the behaviors of the agents

Envi ronment 1

Envi ronment N

Agent A Agent B

Agent Z

MAS

Host N

Host 1

D is t r ibuted network( In te rne t )

3

Challenges in testing agents & MASTraditional software• deterministic• observable

inputs outputs

Agent• non-deterministic, due to self-* and the instant changes of the environment

inputs

sensors

outputs

self-*

MAS• distributed, asynchronous• message passing• cooperative, emergent behaviours

state α

4

Testing phasesn Acceptance: ensure the system meets the

stakeholder goalsn System: test for the macroscopic properties and

qualities of the system n Integration: check the collective behaviors and

the interaction of agents with the environmentn Agent: check for the integration of agent

components (goal, plan, beliefs, etc.) and agent’s goal fulfillment.

n Unit: testing agent units: blocks of code, agent components (plan, goal, etc.)

5

Testing BDI Agents

inputssensors

outputs

self-*

6

Some factsn Many BDI agent dev. languages exist:

¨ JADEX, Jason, JACK Intelligent Agents, AgentSpeak(RT)

n No “popular” de-factor language yetn Often built on top of Javan There are IDEs (integrated development

environment)¨ With testing facility

n We will use JADEX as a reference language

7

BDI Architecture (recap)n Beliefs: represent the informational state of the

agentn Desires: represent the motivational state of the

agent¨ Operationalized as Goals + [contextual

conditions]n Intentions: represent the deliberative state of

the agent – what the agent has chosen to do¨ Operationalized as Plans

n Events: internal/external triggers that an agent receives/perceives and will react to.

8

Testing agent beliefsn Belief state is program state in traditional testing

sense:n Example Agent: { belief: Bank-Account-Balance

goal: Buy-A-Car }

state 1: Bank-Account-Balance = $1,000,0000state 2: Bank-Account-Balance = $100

n What to test:¨ Belief update (read/write)

n direct: injection, change the agent belief brutallyn indirect: perform belief update via plan

execution

9

Testing agent goalsn Goals are driven by contextual conditions

¨ conditions to activate¨ conditions to hibernate/drop¨ target/satisfactions condition

n What to test:¨ goal triggering¨ goal achievement¨ goal interaction

n one goal might trigger or inhibit other goalsn goal reasoning to solve conflicts or to archive higher

level goals

10

Testing agent plansn Plans are triggered by goals, a goal

activated will trigger a plan executionn Plan execution results in:

¨ interacting with external world¨ changing the external world¨ changing agent belief¨ triggering other goals

n What to test:¨ plan instantiation¨ plan execution results

11

Testing eventsn Events are primarily test inputs in agent

testingn Can be:

¨ Messages ¨ Observing state of the environment

n What to test:¨ Event filtering, what are the events an agent

should receive¨ Event handling, to trigger goals or update beliefs

12

Example: testing a cleaning agentn Environment:

¨ wastebins¨ charging stations¨ obstacles¨ waste

n This agent has to keep the floor clean

13

Example (contd.)n Example of Beliefs:

…………...<!-- The current cleaner location. -->

<belief name="my_location" class="Location" exported="true"> <fact>new Location(0.5, 0.5)</fact> </belief> <!-- Last visited location --> <belief name="last_location" class="Location"> <fact>new Location(0.5, 0.5)</fact> </belief> <!-- target location, moving to this location --> <belief name="target_location" class="Location"> <fact>new Location(0.5, 0.5)</fact> </belief>

…………...

n Test concerns:n is my_location updated after every move?n is the next target_location determined? how does it differ

from current_location?n …..

14

Example (contd.)n Example of goal

<!-- Observe the battery state. --> <maintaingoal name="maintainbatteryloaded" retry="true" recur="true" retrydelay="0"> <deliberation cardinality="-1"> <inhibits ref="performlookforwaste" inhibit="when_in_process"/> <inhibits ref="achievecleanup" inhibit="when_in_process"/> <inhibits ref="achievepickupwaste" inhibit="when_in_process"/> <inhibits ref="achievedropwaste" inhibit="when_in_process"/> </deliberation> <!-- Engage in actions when the state is below MINIMUM_BATTERY_CHARGE. --> <maintaincondition> $beliefbase.my_chargestate > MyConstants.MINIMUM_BATTERY_CHARGE </maintaincondition> <!-- The goal is satisfied when the charge state is 1.0. --> <targetcondition> $beliefbase.my_chargestate >= 1.0 </targetcondition> </maintaingoal>

n Test concerns:n are the conditions correctly specified? n is the goal activated when the maintaincondition satisfied?n …...

15

Oraclesn Different agent types demand for different types

of oracle¨ Reactive agents: oracles can be pre-determined at

test design ¨ Proactive (autonomous, evolving) agents: “evolving

& flexsible” oracles are needed n It’s hard to say if a behavior is correct because the agent

has evolved, learned overtimen Some exist type of oracles:

¨ Constraint/contract based ¨ Ontology based¨ Stakeholder soft-goal based

16

Constraint-based oraclesn Agents’ behaviours can change, but they must

respect designed constracts/constraints (if any)¨ low-level constraints: pre, post conditions, invariants¨ high-level constraints: norms, regulations

n Constraint violations are faults

CHAPTER 4. TESTING TECHNIQUES 4.1. AGENT EVALUATION

execute correctly. However, software agents are distributed programs that

run at geographically di�erent hosts, handling OCL constraint violations

need to take this into account. Our monitoring agent network, introduced

in Section 4.2, can deal with this issue.

From OCL constraints, monitoring guards (to check constraint viola-

tions) can be generated automatically, using a tool called OCL4Java1 and

its user-defined handler. We specialize this type of violation handler to

notify a local monitoring agent during testing whenever a constraint is

violated. Local monitoring agent is an agent that runs in the same place

with the agents under test. It is in charge of monitoring not only constraint

violations but also many more types of events, such as communications,

exceptions, belief changes, and so on. Details about the monitoring agent

will be introduced shortly.Following is an example of pre-/post-condition specified in OCL, which

requires the order attribute to be not null and ensures that after updatingthe proposed price must be between 0 and 2000:

public class ExecuteOrderPlan extends Plan {

....

@Constraint("pre: self.order->notEmpty\n" +

"post: price > 0 and price < 2000")

public void body() {

....

}

....

}

The following code is generated by OCL4Java from the constraint above.

The implementation of the method handleConstraintFailed is specialized

to inform the local monitoring agent whenever the constraint is violated.

public class ExecuteOrderPlan extends Plan {

....

1http://www.ocl4java.org

59

Example:

17

Ontology-based oraclesn Interaction ontology defines the semantics of agent

interactionsn Mismatching ontology specifications is faulty

Thing Concept

AgentAct ion Propose

+book: Book

+price: float

Book

+title: String

+author: String

Fig. 2. The book-trading interaction ontology, specified as UML class diagram

<owl:Restriction><owl:onProperty rdf:resource="#price"/>

<owl:hasValue ...>min 0 and max 2000</owl:hasValue></owl:Restriction>

In the course of negotiation, a buyer initiates the interaction by sending acall for proposals for a given book (an instance of Book) to all the sellers thatit knows of. Upon reception of a call, sellers check whether the book is availableand make price proposals (instances of Propose). The initiating buyer, then,selects the seller with the minimum price and sends an accept for the proposalto the selected seller, or it changes the proposal and re-sends it to the seller,according to the FIPA Contract Net protocol.

Testing the Buyer and Seller agents accounts for generating instances ofthe messages that each agent is supposed to be able to process and let theTester Agent send them to the proper agents. The Tester Agent continues theinteraction in accordance with the selected protocol and generates new messageswhenever needed. The Monitoring Agents observe and record any deviation fromthe expected behavior of the agents under test. Hence, the problem for the TesterAgent is how to generate meaningful messages in the course of the interaction.We take advantage of the interaction ontology for this purpose.

4.1 Domain ontology and ontology alignment

By domain ontology we mean ontologies that exist in a specific domain ofinterest, not necessarily being interaction ontologies. For instance, there areontologies that describe books and all related information such as title, au-thor, category, year of publication and the like. A number of domain ontologiesare available on the Internet (a useful ontology search service is available at:http://swoogle.umbc.edu).

Ontology alignment [8, 10] refers to techniques aimed at finding relationshipsbetween elements of two ontologies. It can be used to map classes, properties,rules etc. of one ontology onto another one, and eventually to compare or transferdata from one to the other. Several tools are available that support ontologymapping (e.g., Prompt, available at http://protege.stanford.edu/plugins/prompt/prompt.html).

In order to generate meaningful testing data, i.e., data that represent realinstances of ontology classes in the domain of interest, we use ontology alignment,

Rule example

+ title: string+ price: double

18

Requirement-based (stakeholder soft-goals based)n Soft-goals capture quality requirements, e.g. performance,

safetyn Soft-goals can be represented as quality functions (metrics)n In turn, quality functions are used to assess the agents

under test

Efficient Robust

Good looking

ε

time

d > ε

time

19

Input space in testing agentsn Test inputs for an agents:

¨ Passive: n Messages from other agents n Control signals from users or controller agents

¨ Active:n Information obtained from monitoring/sensing the environmentn Information obtained from querying third party services

n Agents often operate in an open & dynamic environment¨ other agents and objects can be intelligent, leading to

nondeterministic behaviors¨ instant changes, e.g. contextual information

20

Example of dynamic environmentn Environment:

¨ wastebins¨ charging stations¨ obstacles¨ waste

n Obstacles can moven The locations of these

objects changes n New objects might

come in

21

Mock Agentsn Mock Agents are sample implementation of

an agent used for testing ¨ Mock Agents simulate a few functionality of the

real agentn An Agent under test can interact with mock

agents, instead of real agents, during test execution

SaleAgent

PaymentAgent

Mock Payment

Agent

Example: during testing the Sale Agent, we use a Mock Payment Agent instead of the real one to avoid real payment

22

Tester Agentn Tester Agent is a special agent that plays

the role of a human tester ¨ interact with the Agent under test, use the same

language is the agent under test¨ manipulate test inputs, generate test inputs¨ monitoring the behavior of the agent under test¨ evaluating the agent under test, according to the

human tester’s requirementsn Tester Agent stays on the different side, against the

agent under test!!!!n Used in continuous testing (next part)

23

Continuous Testing of Autonomous Agents

24

Why?n Autonomous agents evolve over timen One single test execution is not enough

because the next execution of the same test case, test result can be different¨ because of learning¨ because of self-programing (e.g. genetic

programming)

25

Continuous testingn Consists of input generation, execution

and monitoring, and output evaluationn Test cases are evolved and executed

continuously and automatically

Test execution& Monitoring

Evaluation

Generation & Evolution

final results

Agent

initial test cases(random, or existing)

outputs

self-*inputs

26

Test input generation• Manual

• Random‣ messages: randomly-selected interaction protocol + random

message content‣ environment settings: random values of artefacts’ attributes

• Ontology‣ rules and concept definitions can be used to generate

messages

• Evolutionary‣ quality of test case is measured by a fitness function f(TC)‣ use f to guide the meta-heuristic search to generate better

test cases‣ example: quality-function-based fitness

27

Random generation• Messages:

‣ select randomly a standard interaction protocol, combine it with randomly-generated data or domain specific data

• Environmental settings:‣ identify the attributes of the

entities in the environment

‣ generate randomly values for these attributes

Example:

28

Ontology-based generation• Information available inside interaction ontology:

‣ concepts, relations, data types of properties. E.g. action Propose is an AgentAction, two properties: book: Book , price: Double

‣ instances of concepts, user-defined or obtained from ontology alignment

• Use these data to generate messages automatically

tester agent agent under test

Book(title:”A”)

Propose(book:”A”, price:10)

Propose(book:”A”, price:9)

Example:

29

Quality-function-based evolutionary generation• Build the fitness of test cases based on quality functions

• Use this fitness measure to guide the evolution, using a genetic algorithm, e.g GA, NSGAII etc.

• For example:‣ soft-goal: safety

‣ quality function: the closest distance of the agent to obstacles must be greater than 0.5cm

‣ fitness f = d - 0.5, search for test cases that give f < 0

0.5

distance

time

0.5

distance

time

generation i generation i + K

30

Example: evolutionary testing of the cleaner agent

n Test case (environment) encoding¨ coordinates (x,y) of

wastebins, charging stations, obstacles, wastes

¨ TCi = <x1,y1,x2,y2,….,xN,yN>n Fitness functions:

¨ fpower = 1/Total power consumption n search for environments where the agent consumes

more power ¨ fobs = 1/Number of obstacles encountered

n search for environments where the agent encounters more obstacles

31

Example (contd)

n Genetic algorithm, driven by fpower and fobs, will search for test cases in which the agent will have higher chance to run out of battery, and hit obstacle. ¨ that violates the user’s requirements

n Results:n Evolutionary test generation

technique founds test cases where:1) wastes are far away from wastebin -> more power2) obstacles on the way to wastes -> easy to hit

Black circles: obstaclered dots: wastes - squares: charging stations, red circles: wastebins

32

Example (contd)

n More about the evolutionary of the environment:¨ http://www.youtube.com/watch?v=xx3QG5OuBz0

n The search converts to the test cases where the 2 fitness functions are optimized.

33

Conclusionsn Testing software agents is important, yet still

immature.n Concerns in testing BDI agent:

¨ BDI components: beliefs, goals, plans, events¨ Their integrations

n Oracles:¨ Reactive agents: can be specified at design time¨ Proactive agents: new types of oracles are needed,

e.g. use quality function derived from soft-goalsn Many approaches to generate test inputs

¨ Evolutionary proved to be effective

34

Additional resourcesn CD Nguyen (2009) Testing Techniques for Software Agents. PhD thesis, University of Trento,

Fondazione Bruno Kessler. http://eprints-phd.biblio.unitn.it/68/n CD Nguyen, Anna Perini, Paolo Tonella, Simon Miles, Mark Harman, and Michael Luck. 2009.

Evolutionary testing of autonomous software agents. In Proceedings of The 8th International Conference on Autonomous Agents and Multiagent Systems - Volume 1 (AAMAS '09), Vol. 1. International Foundation for Autonomous Agents and Multiagent Systems, Richland, SC, 521-528.

n CD Nguyen, A Perini, C Bernon, J Pavón, J Thangarajah, Testing in multi-agent systems, Agent-Oriented Software Engineering X, 180-190.

n Zhiyong Zhang, Automated unit testing of agent systems, PhD Thesis, Computer Science and Information Technology, RMIT University.

n Roberta de Souza Coelho, Uirá Kulesza, Arndt von Staa, Carlos José Pereira de Lucena, Unit Testing in Multi-agent Systems using Mock Agents and Aspects, In Proceedings of the 2006 international workshop on Software engineering for large-scale multi-agent systems (SELMAS '06). ACM, New York, NY, USA, 83-90. DOI=10.1145/1138063.1138079 http://doi.acm.org/10.1145/1138063.1138079

35