Active

Active

Active, a platform for building intelligent software

Dr. Charles Baur (EPFL)

Adam Cheyer (SRI International)

Didier Guzzoni (EPFL)

Active

Presentation Plan• Introduction

– Problem space– Active proposition

• Active Framework– Active Ontologies– Implementation– Methodologies

• Applications

• ConclusionIntroduction

Active

Motivation

• Our information environment is rich and complex– Ubiquitous access to a wealth of data and services– Software and hardware industry constant innovations– UIs have not changed: Simple click-and-do approach not

enough

• Need for computer assistants– Interact naturally with humans– Can be delegated complex tasks– Observe, understand, anticipate and act

Introduction

Active

Act

Understand

Observe

Intelligent Systems

Sense

Interpret

Listen

Resolve Anticipate

Plan action

Effectuate Communicate

Intelligent Systems : Naturally collaborates with human users to deliver services and contents through adaptable, efficient, multimodal user interfaces.

Vision

Introduction

Active

Difficult Task• Requires numerous AI techniques

– Language processing– Plan execution– Dynamic service brokering (MAS)

• Implementation is difficult– HCI components (speech recognition, vision, robotics)– Large teams of specialists– Different programming languages and platforms– Testing, debugging and maintenance is difficult– Performance is likely to be affected

Introduction

Lack of an integrated tool and methodology to easily and effectively build intelligent systems

Active

Goal of Active

Active Framework

• Provide programmers with an integrated framework and a methodology to build complex AI-based systems

• Capable of encapsulating AI techniques– Language processing, plan execution and agent type techniques

• Programmer friendly– Small teams– Based on popular programming languages (Java/Javascript)– Offers an IDE (code, test, debug and deploy)

• Open and standard compliant– SOA-based (SOAP, REST, RMI)– Deployment (Java, J2EE)

Active

Basic Concept : Active Ontologies

• Ontology : A data structure– Formal representation for domain knowledge– Classes, attributes, relations

movie

genre actor ratingP

• Active Ontology : A processing environment– Processing elements arranged according to ontology notions– Communication channels

PP

P

Active Framework

Active

Active Ontology : Processing

movie

• Production Rule System– Rules Sets

• Rules (Conditions, actions)

– Data store (facts)• Current state of the system

– Evaluation Engine• Evaluation passes

• Active Innovations– Organizing rules around Ontologies helps

design and debug– Developer friendly rule language. Enhanced

Java/Javascript with unification

P

rule set

rulecondition

action

rulecondition

action

rulecondition

action

Active Framework

Active

Active Application Design

• One or more Active Ontologies– Hosted on Active runtime– Typically :

• Language processing

• Plan execution

• Dynamic service brokering

• Service oriented (SOA)– Loosely coupled– Sensors (user interfaces, speech

recognition, vision)– Actuators (robots, user interfaces)– Reusable– Dynamically swapped

Active runtime

services

Active Framework

Active

Implementation

• Active Server– Hosts Active Ontologies– Maintains a fact store– Runs evaluation engine– Extensions

• Active Editor– IDE – Code, deploy, test– Pluggins

• Active Console– Manages Active Server

Implementation

Active Server

Active Editor

Active Ontology

Active Console

deploy

Facts storedebug

mon

itor

Active OntologyActive Ontology

Active Ontology

Evaluation

Engine

Active

Active Server

• Runs and hosts Active Ontologies– Evaluation Engine– Fact Store

• Implementation– Java application/J2EE webapp– SOAP / RMI interface– Rule language is Java/JavaScript

enhanced by unification

• Extensions– Encapsulate pre-compiled complex

operations

Active ServerActive Ontologies

Fact store

SOAP/RMI interface

EvaluationEngine

Extensions

Implementation

services

Active

Active Editor• IDE

– Graphical editing of ontologies– Specialized concept and rule

editors– Active ontology definition files

saved locally (XML)

• Active Server Connection– Deploy/undeploy edited ontologies– Integrated test/debug

• Plugins– Automatically creates concepts and

rules based on interactive wizards

Implementation

Active

Active Console

• Management tool– Monitor and configure

deployed Active Ontologies – SOAP/RMI interface

• Query (read) panel– Construct complex queries

to Active Server– Tabular result sets

• Store (write) panel– Stimulates Active Ontologies

by sending events to the server

Implementation

Active

Active Methodologies

• Language Processing– Chart Parsing– Event based

• Agent techniques– Delegated computing– Dynamic service selection

• Plan execution– Process Execution Engine– Reactive Planning

Methodologies

Active

Language Processing : Grammar-Based

• Grammar-based– Grammar based– Chart parsing

• Advantages– Formal parsing

(Mathematical expressions)– Deterministic

• Disadvantages– Not flexible– Not robust to missing words– Not well suited for non-

reliable input modalities (Speech recognition)

Methodologies

Active

Language Processing : Domain-Based• Implementation

– Bottom Up– Leaves : Word set, regex

– Nodes : Gather, Select

• Context– Kept among utterances– Errors, Suggestions

• Advantages– Robust to syntax– Ports well to different

languages

• Wizards– Easy modifications

Methodologies

“find action movies in San Francisco”“nearby Chinese restaurants”

Active

Activity/Dialog Modeling• Modeling

– Dialogs, Activities, Behaviors

• Full-featured workflow management– State / Transitions

– Flow instances, Instance space

• Basic flow constructs– Start, End

– Wait, Switch (branching)

– Parallel, sequences

• Active Implementation– Set of plugins (Editor Wizards)

– Extension (Java/JavaScript) to access flow variables

Methodologies

Active

Dynamic Service Brokering

• Delegated Computing– What instead of how or who

• Service Registry– Service categories

cross-ontology references– Service instances (providers)

• Broker– Parallel, Sequential, Broadcast– Third-party meta-agents

• Active Implementation– Specialized Active Ontology– Server Extension, IDE Wizards

Methodologies

Active

Prototypes / Demonstration

• Information Retrieval Assistant

• Meeting Organizer Assistant

• Operating Room Assistant

Prototypes

ActivePrototypes

Information Retrieval Assistant

City Information

Google Movies

Yahoo! Local

Opentable

Language Processing

Plan Execution

Delegation

Active Server

SOAP

Extensions

Email

gmail SMTP/POP

server

ActivePrototypes

Meeting Organizer Assistant

Language Processing

Plan Execution

Delegation

Active Server

IM SOAP

Extensions

gmail calendar

gmail SMTP/POP server

Email

Active

Speech Recognizer

Gesture Recognizer

Text to Speech

Patient vital signs probes

Mouse 3D

Hand/Head

Tracker

Context History

User Interface

Powered Endoscope

Language Processing

Plan Execution

Actions through Delegation

Prototypes

Operating Room Assistant

Active

Active Advantages• One platform to learn for programmers

– Can build all three tiers of applications in Active – One language and tool to learn– Easier to debug, test and deploy

• AI Encapsulation-Lowering the bar– Methodologies encapsulated as Active Wizards and

Extensions• NLP, process execution, service brokering

– Componentized reusable sensors and actuators• TTS, Speech, Gesture recognizers, vision systems

• Open and Standard-based– SOA design – Ease of integration through SOAP and REST– Reuse of existing components in multiple applications

Conclusion

Active

Ongoing Work

Conclusion

• Research Topics– Combine activity recognition with process execution– Implement and evaluate BDI-like behaviors with Active (goal

and intention stack)

• Active implementation and features– Scalability– Performance optimizations– Lightweight embedded Active Server

• User Evaluations– Put some of our systems on-line– Measure feedback from surgeons (Intelligent Operating Room)

Active

Thank You !

Conclusion

• Questions ?• Suggestions ?• Remarks ?