september 7, 20051 control agents making robotics a reality dr. reuven granot faculty of science and...

September 7, 2005 1

Control Agents making Robotics a reality

Dr. Reuven GranotFaculty of Science and Scientific Education

University of Haifa, Israel

Visiting scholar at UBC/Mining Eng. Dept.

[email protected]

September 7, 2005 2

The need for unmanned systems.

Why tele-robotics?

A new layer of software should be developed in order to support application users in fields of robotics and other semi or autonomous systems to develop more effectively their specific applications.

follow upcoming standards like JAUS and CORBA.

follow the Distributed and Object Oriented paradigm, but be more than an object or an expert system by being reactive, autonomous and proactive.

transparently take care of inter agent communication and other basic tasks a control agent needs.

- a control agent is performing some control task while communicating with other agents or humans as needed.

We, at University of Haifa, Israel, are suggesting to collaboratively developing the needed infrastructure for Human Supervised Autonomous Control Agents.

A relevant application of semi-autonomous bulldozer.

Outline

September 7, 2005 3

Illustration from NASA and ORNL publication

Robot colony on a site preparation task for a PV tent on Mars.

The Need of Unmanned Systems

• DDD

– Dull

– Dirty

– Dangerous

• Distant – at different scale

– Macro: space,

– Micro: telesurgery, micro and nano devices

Is well recognized to perform tasks that are:

September 7, 2005 4

Remote Controlled vehicles in combat environment

RC is still preferred by designers

o Simple, but not practical for combat or other very demanding environment because the human operator:

is very much dependent upon the controlled process needs long readjustment time to switch between the controlled and the local environment.

The state of the art of the current technology has not yet solved the problem of controlling complex tasks autonomously in unexpected contingent environments.

o dealing with unexpected contingent events remains to be a major problem of robotics.

Consequence: A human operator should be able to interfere: remains at least in the supervisory loop.

September 7, 2005 5

All these applications require an effective interface between the machine and a human in charge of operating/ commanding the machine.

We are suggesting to perform Human Supervised Autonomous Control, which is known as tele-robotics.

That be done using the software agent technology

The needed control metaphor:

Human Supervised Autonomous

September 7, 2005 6

The Tele-robotics paradigm

Telerobotics is a form of Supervised Autonomous Control.

A machine can be distantly operated by:

• continuous control: the HO is responsible to continuously supply the robot all the needed control commands.

• a coherent cooperation between man and machine, which is known to be a hard task.

Supervision and intervention by a human would provide the advantages of on-line fault correction and debugging, and would relax the amount of structure needed in the environment, since a human supervisor could anticipate and account for many unexpected situations.

September 7, 2005 7

The spectrum of control modes.

Solid line= major loops are closed through computer, minor loops through human.

• traded control: control is or at operator or at the autonomous sub-system.

• shared control: the instructions given by HO and by the robot are combined.

• strict supervisory control: the HO instructs the robot, then observes its autonomous actions.

A telerobot can use:

September 7, 2005 8

Human Robot Interaction

• In supervised autonomously controlled equipment, a human operator generates tasks, and a computer

autonomously closes some of the controlled loops. • Control bandwidth

– Robot SW: high

– Human response: slow

• Human Operator is expected to Control several machines/ equipment/ systems be capable to deal with other duties (like a combat environment requests) in somehow relaxed mode of operation.

Make the machine an agent in human operator’s service.

September 7, 2005 9

Software Infrastructure

• Architecture– Distributed– Objects or some more sophisticated entities?– Communication

• Development Environment– Language– Tools– Reusable units

September 7, 2005 10

What is a Robot Architecture?

• There are many different ways in which a robot control program can be put together.

In order to program a robot in a structured and principled fashion, we use an appropriate robot control architecture.


Robot Architecture

• A control architecture provides a set of principles for organizing a control system. – It provides structure and constraints which aid the

designer in producing a well-behaved controller.

• To be successful a system designer has to decide how (in what order? with what priority?) does he put together multiple feedback controllers in a principled fashion and how to scale up control to more complex robots, which generally have to deal with many behaviors at once.

– How would you put multiple feedback controllers together?

– How would you decide which one to use when and for how long and in what priority relative to the others?


Misconceptions.1. Programming languages are implementation tools and not

architectures.

2. The issue of fundamental power or expressiveness of a robot

control architecture: claims have been made about one control architecture being able to compute fundamentally more than another. This cannot be true if we understand that all are grounded in Turing-complete programming languages.

However, the above is not to say that all architectures are the same. On the contrary, architectures impose strong constraints on how robot programs are structured, and the resulting control software ends up looking very different.


Robot Architecture Major Classes/Categories

Intuitively, this means that there are infinitely many ways to structure a robot program, but they all fall into one of major classes/categories of control: • Deliberative Control : Think hard, act later.

─ SPA, serial, complete each step first – then proceed

• Reactive Control : Don’t think, (re)act.

─ Direct connection between perception to action, no memory, no planning.• Hybrid Control : Think and act independently, in parallel.

─ Deliberative and Reactive modules run independently at different time scales

• Behavior-Based Control : Think the way you act. – Distributed by behavioral task decomposition– Each behavior has its restricted planning and execution capabilities


The Choice of the Control Architecture

• When it comes to more complex robots, i.e., robots that have to deal with complex environments and complex tasks, the control architecture becomes very important.

• The different properties of an environment that will impact the robot's controller (and therefore the choice of control architecture): – noisy,

– speed/response time of sensors and effectors

– total/partial hidden state/ observable

– discrete v. continuous state ; static v. dynamic ...

• Similarly, the properties of the robot's task impact the choice of the control architecture. The task requirements can constrain the architecture choice.


Parallel Processing Paradigm.

• As robot control is engaged to deal with more complex problems, centralized supervisory architectures encounter barriers to real time performance caused by computational complexity coupled with insufficient computing power and sensor resources. – Despite startling advances in hardware and software

technology and similarly surprising cost reductions, these fundamental barriers remain unchanged.

• The parallel-processing paradigm may be the only technology to challenge this fact.


Asynchronous and Synchronous processes

• The other leading architectural trend is typified by a mixture of asynchronous and synchronous control and data flow. – Asynchronous processes are characterized as loosely

coupled and event-driven without strict execution deadlines.

– Synchronous processes, in contrast, are tightly coupled, utilize a common clock and demand hard real-time execution.


Time Scale.

• Reactive systems respond to the real-time requirements of the environment,

• while deliberative system look ahead (plan) and thus work on a longer time-scale.

• Hybrid systems must combine the two time-scales in an effective way, usually requiring a middle layer; consequently they are often called three-layer architectures.

• Finally, behavior-based systems attempt to bring the different time-scales closer together by distributing slower computation over concurrent behavior modules.

Time-scale is an important way of distinguishing control architectures.


Representation

• Another key distinguishing feature between architectures is representation of the world/environment, also called world modeling.

• Some tasks and architectures involve storing information about the environment internally, in the form of an internal representation of the environment.

• For example, while exploring a maze, a robot may want to remember a sequence of moves it has made (e.g., "left, left, right, straight, right, left"), so it can back-track and find its way.

Thus, the robot is constructing a representation of its path through the maze.

The robot can also build a map of the maze, by drawing it using exact lengths of corridors and distances between walls, etc. .

This is also a representation of its environment, a model of the world.

• If two robots are working together, and one is much slower than the other, if the fast robot remembers/learns that the other is always slower, that is also a type of a model of the world, in this case, a model of the other robot.


Different World Models.

• There are numerous aspects of the world that a robot can represent/model, and numerous ways in which it can do it, including:

* spatial metric or topological: maps, navigable spaces, structures

* objects instances of detectable things in the world

* actions outcomes of specific actions on the self and environment

* self/ego stored proprioception: sensing internal state, self- limitations, etc.

* intentional goals, intended actions, plans

* symbolic abstract encoding of state/information


• Regarding the architecture of robotic systems, we discussed so far two key issues distinguishing architectures, as had to do with – time-scale (reactive) and

– looking ahead (deliberative).

• A third key issue we need to consider is modularity, i.e., the way in which the architecture decomposes into components.


What is a behavior?

• An individual behavior is a stimulus/ response pair for a given environmental setting that is modulated by attention and determined by intention.

• Behaviors serve as the basic building blocks for robotic actions.

Attention: prioritizes tasks and focuses sensory resources and is determined by the current environmental context.

Intention: determines which set of behaviors should be active based on the robotic agent’s internal goals and objectives.

Apparent or emergent behavior: the global behavior of the robot as a consequence of the interaction of the active individual behaviors.


What are Behaviors?

• typically has the following properties:

– are feedback controllers (closed-loop, extended in time )

– achieve specific tasks/goals

– are typically executed in parallel/concurrently

– can store state and be used to construct world models/representation

– can directly connect sensors and effectors

– are typically higher-level than actions

– can also take inputs from other behaviors and send outputs to other behaviors

• when assembled into distributed representations, behaviors can be used to look

ahead but at a time-scale comparable with the rest of the behavior-based system.


Behaviors and Modularity

Behavior-based systems are not limited in the ways that reactive systems are. As a result, behavior-based systems have the following key properties:

1) the ability to react in real-time.

2) the ability to use representations to generate efficient (not only reactive) behavior.

3) the ability to use a uniform structure and representation throughout the system (so no intermediate layer).


Assembling Behaviors.

• Systems are constructed from multiple behaviors.

• Emergent behavior implies a holistic (attention to the “whole”) capability where the sum is considerably greater than its parts.

• Emergence is “the appearance of novel properties in whole systems”.

• Intelligence emerges from the interaction of the components of the system.

• Coordination functions are algorithms used to assemble behaviors.– Conflict can result when two or more behaviors are active, each

with its own independent response.


The Agent

• An agent is a computer system capable of autonomous action in some environments.

• A general way in which the term agent is used is to denote a hardware or software-based computer system that enjoys the following properties: – autonomy: agents operate without the direct intervention of

humans or others, and have some kind of control over their actions and internal state;

– social ability: agents interact with other agents (and possibly humans) via some kind of agent-communication language;

– reactivity: agents perceive their environment, (which may be the physical world, a user via a graphical user interface, or a collection of other agents), and respond in a timely fashion to changes that occur in it;

– pro-activeness: agents do not simply act in response to their environment; they are able to exhibit goal-directed behavior by taking the initiative.


Agents and Behaviors

• Behavior is defined as the way how we/people observe the system/robot acts/behaves.

• The robot system is NOT aware of what we know about it.– What makes the system act as we observe is its

software.

• Behaviors are implemented by agents.


Interface Agent

• A software entity, which is capable to represent the human in the computer SW environment.

• It acts on behalf of the human• Follows rules and has a well defined expected

attitude/ action.• May be instructed on the fly and may receive

during mission updated commands from the human operator.

We need to build agents in order to carry out the tasks, without the need to tell the agents how to perform these tasks.


Agents are not Objects

• Differ from Objects– autonomous, reactive and pro-active

– encapsulate some state,

– are more than expert systems

– are situated in their environment and take action instead of just advising to do so.

• Agents may act inside the robot software to implement behaviors:

Feedback controllers Control subassemblies Perform Local Goals/ tasks


Agent control loop

• agent starts in some initial internal state i0 .

• observes its environment state e, and generates a percept see(e).

• internal state of the agent is then updated via next function, becoming next_(i0, see(e)).

• the action selected by agent is action (next(i0, see(e))))

This action is then performed.

• Goto (2).


Human Operator• Monitors the activities and the performance of the assembly of

agents.• Responsible for the completion of the major task (global goal)

– may interfere by sending change orders. • emergent (executed immediately, without considering any

possibility to return to achieve the goal/in the shortest possible way)

• “as is ordered” or• normal

– checked by the interface agent – which negotiates execution with other agents in order to

optimize execution performance

– Conflict resolution algorithm• defined as default, or• defined by the human operator in its change order or • suggested to the operator by a simplified decision support

algorithm.


D9 Bulldozer

• The operator has very limited information about his surroundings or machine performance.

• A good starting project: – earthmoving tasks are loosely coupled with

locomotion tasks.– earthmoving tasks are not really simple and – locomotion tasks are not really complicated.


RCSEmbeds a hierarchy of agents within a hierarchy of organizational units: Intelligent Nodes or RCS_Nodes.

SquadCommander

SquadCommander

Squad Commander

PlatoonCommander

VehicleCommander

VehicleCommander

VehicleCommander

VehicleCommander

SquadCommander

JAUS

From M. W. Torrie

A hierarchy of Commanders

different resolution in space and time


RCS_Node

Value Judgment

Sensory Processing

World Modeling

Behavior Generation

Knowledge Database

Update Plan

StatePredicted Input

Observed Input

Perceived Objects &

Events

Commanded Actions

(Subgoals)

Commanded Task (Goal)Plan

EvaluationPla

n R

esul

tsS

ituation E

valuation


Agents in Behavior Generation hierarchy

• Tasks are decomposed and assigned in a command chain.

• Actions are coordinated

• Resources are allocated as plan approved.

• Tasks achievements are monitored (VJ)

• Execution in parallel


Small Size Scaled Model

• The implementation differs by mechanical, perceptual and control elements from the full scale application.

• It still may help to identify unusual situations which the software agent must be capable to deal with.

• Full scale machines may be tested only at field ranges, which are time consuming and very expensive.

• A small scale model may be tested in office environment, enabling the software developers to shorten test cycles by orders of magnitude.


Expected situations

• The bulldozer moves forward placing the blade too low – The human decides: the blade should be placed higher

Command issued: “lift the blade”.

• experiencing too much power to enable earth moving forward – the human operator would prefer to withdraw and

attack the soil from a new position behind– the human operator is distant– the bulldozer is “close” to the ditch;

> a better practice would be to first complete the maneuver.

Bulldozer using Fuzzy Control decides to perform the better practice and withdraws only after the maneuver is completed.


The Model


autonomous-bulldozer\robot.WMV

autonomous-bulldozer\robot.mpg

4 min

3 min