briefing(virtual(actors:((...
TRANSCRIPT
Briefing Virtual Actors: a First Report on the PRESTO Project
Paolo Buse=a, Delta InformaCca (Trento, IT) Chiara Ghidini, FBK (Trento, IT)
Ma=eo PedroK, AOS (Cambridge, UK) Antonella De Angeli, DISI (Uni. Trento, IT) Zeno Menestrina, DISI (Uni. Trento, IT)
AI & Games, 1st April 2014
Outline
• ObjecCves of PRESTO • NPCs’ cogniCve architecture: DICE • AbstracCng from the environment and VR engine: PRESTO core
• Demo of current core progress • Empowering the “director” • Demo of EUD progress • Next steps
2 01 Apr 2014 Briefing Virtual Actors: a First Report on the PRESTO Project
ObjecCves of PRESTO • 3 year, funded by Provincia Autonoma di Trento (Italy), started in 2013.
Lead by Delta, partners include AOS (UK), e-‐semble (NL), research centers in Trento, EMT experts
• Looking at applying a cogniCve architecture (AOS’ CoJACK) in the world of “serial” (serious) game development, which means: 1. Defining a proper methodology for behavioural specifica6on captured
from domain experts • Including psychological aspects
2. Being able to easily build, share and compose behavioural models 3. Easily adding, controlling, analyzing emo6ons and other non-‐ra6onal
influences when creaCng scenarios (game levels) and during play 4. Enabling trainers / game writers -‐ rather than AI developers -‐ to write their
behaviours for their specific scenarios / levels based on the exisCng models Note the disCncCon between (generic, reusable) behaviour specificaCon and modeling (1, 2) and scenario-‐specific behaviours (3, 4)
• Pilot (and iniCal commercial exploitaCon) domain: emergency management in health insCtuCons (hospitals, hospices, …)
01 Apr 2014 Briefing Virtual Actors: a First Report on the PRESTO Project 3
NPCs’ cogniCve architecture: DICE Delta’s Infrastructure for CogniCon and EmoCon • Inspired by CoJACK (2008)
– Describes “raConal” behaviour (trees of goals and plans, BDI) as well as sub-‐raConal influences that affect decision-‐making at criCcal points as well as general cogniCve performance (speed, Cmespan, accuracy)
– Very sophisCcated but complex to use and understand – CogniCve parameters definiCon and tuning require very inCmate knowledge of models – Integrated with VBS2, Unity and others – Sohware engineering aspects underdeveloped
• A criCcal limitaCon for PRESTO’s objecCves
• DICE adds concepts and funcConality, mostly oriented to sohware engineering, such as: – disCncCon between high-‐level decision-‐making (further divided in “planned” and “reacCve”) and low-‐level
concurrent subsystem controls (“executors”) – model packaging as a capability (“behavioural model”, BM) saCsfying a “role”
• A role is a collecCon of goals • A BM is a collecCon of plans and subgoals achieving a role’s goals plus a number of hooks e.g. for conCnuous percepCon and
reacCve behaviour • A BM can invoke goals of other roles, thus enabling hierarchical decomposi6on and func6onal abstrac6on
– agent as a composi6on of models (character “public” profile as set of roles, actual profile as a set of BMs) – dynamic behaviour selec6on (moderator-‐driven dynamic switching of alternaCve BMs for same role), – high-‐level scrip6ng (DICE Parts), which may also be packaged as a BM itself for custom roles
• Further, supports – full introspecCon and meta-‐level control – mind-‐reading (of an agent by another agent) – roles’ goal invocaCon from outside world (GUIs, session script)
01 Apr 2014 Briefing Virtual Actors: a First Report on the PRESTO Project 4
DICE features • BDI approach: domain expert-‐friendly decomposiCon in goals and procedures
– DICE improves introspecCon & and introduces mind-‐reading (for aKtude recogniCon, coordinaCon) thanks to ontologies
– DICE introduces a framework on top of “pure” BDI, i.e. capabiliCes, plans, goals and beliefs are wri=en according to specific standards, certain types of beliefs (percepCon) are predefined, background knowledge is pre-‐loaded, criCcal regions and prioriCzaCon in decision-‐making are predefined, and so on
• CoJACK’s (conceptual) inheritance: moderators, cogniCve parameters, Cme-‐sorted short term memory – but: – DICE avoids (invasive) uClity funcCons, very fine grained acCon and intenCon scheduling – Effects of cogniCve state on a=enCon and performance in DICE (field of view, speed of percepCon, speed of
execuCon) either influence sensors / actuators (outside scope of CoJACK) or are directly hardwired into models
– Conversely, all cogniCve features can be affected by behavioural models (the development of “psychological” BMs is actually a project objecCve)
• JACK’s features: concurrency of goals, waiCng for condiCons, mulC-‐agent communicaCon, integraCon with Java – DICE’s SDK adds full bidirecConal conversion with C# / CLR, fundamental for use within Unity / Mono
• DICE’s own: – strong engineering framework, including debugging / tracing / monitoring faciliCes, – full integraCon with VR abstrac6on (PRESTO core – more later), – ontological tagging of aMtudes (goals, plans, i.e. moCvaCons and intenCons at run-‐Cme) – composable, dynamically selected behavioural models via roles / moderators / override by GUI or script
• CriCcal to enable a marketplace for behavioural models
01 Apr 2014 Briefing Virtual Actors: a First Report on the PRESTO Project 5
DICE at a glance
01 Apr 2014 Briefing Virtual Actors: a First Report on the PRESTO Project 6
PRESTO: A (Sohware) Summary
01 Apr 2014 Briefing Virtual Actors: a First Report on the PRESTO Project 7
SimulaCon / Gaming
Environment
PRESTO Core
Item / LOI / Actor Control GUIs
Timeline Level Script Editor
Director Assistant Control GUI
DICE Editors / Builders
Ontology Editors / Extractors
DICE Agents
Behavioural Models
Agent / BM / Roles
Meta data
Level Data
PRESTO Ontologies
Game Specific Ontologies
DICE Parts
NavigaCon
2D visualizer, simulator
DICE Framework
SituaCon Awareness
AcCons
AbstracCng from the environment and VR engine
• Base knowledge representaCon technique: ontologies – Upper level derived from DOLCE – Looking at process extensions to tag plans, goals for introspecCon / mind
reading • AbstracCon layers: the PRESTO Core
– PercepCon (“situaCon awareness”) – AcCons (animaCons, translaCons, expressions, etc.) – NavigaCon (path finding, path planning, collision avoidance, etc.) – Built-‐in 2D rendering and simulaCon engine to provide alternaCve views or
alternaCve to full-‐blown VRs / game engines, especially when debugging • Engine specific capabiliCes, if required by DICE models, accessed via
engine-‐specific ontologies and wrapped as BMs – E.g. needed for navigaCon subsystem (“movement” role)
• Main limitaCon: abstracCons cannot be any be=er (provide more data or be=er affect the environment) as the underlying game engine / VR – E.g. colliders, physics, navigaCons meshes, animaCons and body
representaCons, audio support, HCI …
01 Apr 2014 Briefing Virtual Actors: a First Report on the PRESTO Project 8
• Video: An example of training session with NPCs controlled by PRESTO • The ward of a Trento hospital has been reconstructed for our pilot project, but behavioural models
are sCll being developed (from actual specs of emergency procedures, looking at developing a proper methodology) – anyway, PRESTO is not about hospitals
• InteracCve demos available, please ask if interested • The game in use is E-‐semble’s XVR™
– Well known for Emergency Management and Training (EMT) – Based on Unity; PRESTO core built against XVR’ SDK, occasional calls to Unity
Showing PRESTO Core and DICE in acCon (progress to date)
01 Apr 2014 Briefing Virtual Actors: a First Report on the PRESTO Project 9
Typical seKng with the trainer’s assistant at XVR’s console
Empowering the “director” • PRESTO doesn’t build games! • Three end-‐users: gamer, director, modeler • The modeler builds plausible, believable, scenario-‐independent behaviours • The director creates its own game level / training session or equivalent
with minimal effort and competences w.r.t. what is needed when directly using engines (both AI and graphics) – the director only “briefs” virtual actors, doesn’t fully program them – A virtual actor is a composiCon of DICE behavioural models
• Gamers get an experience as good as the available models and the director’s ability to create scenarios (and the VR in use, of course)
• PRESTO is really about enabling directors and modelers to meet mid-‐way – Modelers get an SDK, a methodology, and a marketplace – Directors pick and mix models from the marketplace to build their own
characters, write session scripts and individual parts for those characters, semanCcally annotate scenes, and final run training sessions or package everything as a (mulC-‐user) game
01 Apr 2014 Briefing Virtual Actors: a First Report on the PRESTO Project 10
EUD tools for the Director
Demos: 1. Mock-‐up for the DICE Part editor (goal-‐ and
event-‐driven scripCng for the individual actor) – GeneraCng XML for the DICE Part interpreter – AlternaCve editors / planners can be supported
2. Prototype of the Timeline (level editor and control GUI) – An independent agent (not DICE) acts as the director’s assistant during run-‐Cme
01 Apr 2014 Briefing Virtual Actors: a First Report on the
PRESTO Project 11
Next 18 months plan
• Complete developments of PRESTO core and DICE
• Complete EUD tools – IniCal user evaluaCon studies on mock-‐ups already undertaken, second round when actual systems completed and fully integrated in PRESTO
• Focus on models and methodology – Psychological as well as behavioural – Hospital domain is first, but others may come, driven mostly by opportunity (research or commercial)
01 Apr 2014 Briefing Virtual Actors: a First Report on the PRESTO Project 12
Thanks for your a=enCon!