dissemination and evaluation

AtGentive General Meeting, Amsterdam 22-23 Oct 2007

Dissemination and Evaluation

Claudia Roda, Damien Clauzel, Joona Laukkanen

http://en.wikipedia.org/wiki/Image:European_flag.svg


Dissemination: Atgentive book

• Latest version has been distributed– Need feedback on proposal (5 pages to

read!)– Participation:

• I am listing as committed: Cantoche, OBU, Ontdeknet, and AUP

• Informed but not committed: INSEAD, UTA (I have no description of contribution)

• I had no feedback from: CELN, STC • Could you please check your bios



Dissemination: Atgentive Book

• Timeline:– First version of each chapter: 15 February

2008– Feedback from review: 15 March 2008– Final formatted version of each chapter 15

April 2008– Publication: ??



Evaluation

• Objective: enable the assessment of the value of the conceptual framework and reasoning module in contexts that are not represented in the pilots

• Should this go under strategic evaluation?



Evaluation

• Conceptual Framework - Demonstrate that the conceptual framework may hold in the context of very different implementations and applications.

• Reasoning module - New version of the test centre: – Visualize the reasoning of the reasoning module in

applications such as Ontdeknet– Show how the reasoning module can be used also in less

structured, non-task-based applications, and in the context of several (rather than one) applications.



Evaluation: Conceptual Framework

• We look at the context restoration issue. • Virtual multi-desktop interface as an implementation of

context-restoration facility • We hope to demonstrate that context restore will

enhance both performance and user satisfaction• Hypotheses (Virtual multi-desktop interface vs Control):

– Increased perception of productivity and pleasure– Tasks are completed faster and better– Tasks are resumed faster and more easily– Fewer interface management actions and fewer mistakes on

interface management actions



Evaluation: Conceptual Framework

• Experiment:– Subjects work for 20 minutes on three fully described tasks

(spreadsheet based, translation, video). – They are interrupted regularly by requests to work at a

different task, they are required to satisfy these requests– Control group (standard window interface), experimental

group (virtual multi-desktop)• Measures:

– Task completion level– Task quality level (# of errors)– Task resumption times– Reported evaluation



Task 1: Spreadsheet



Task 2: Translation



Task 3: Video



Interruption



Resume



Evaluation: Reasoning Module

• Reasoning module - New version of the test centre: – Visualize the reasoning of the reasoning

module in applications such as Ontdeknet– Show how the reasoning module can be

used also in less structured, non-task-based applications, and in the context of several (rather than one) applications.



The test center

• Tool for experimenting with the RM• The test center was not very intuitive• Desire to experiment with the future

vision of the RM (still working on)



Desktop



Desktop

• Minimal applications on a desktop– application provides task

model, sends events– interact with the application

instead of creating events



Desktop

• Interventions



User and task properties editable



Reasoning module activity



Utilities for creating environment events: New information available, idle input



EXAMPLE



user has finished intro and is notified about next task (learning goals)..



user starts learning goals (interventions disappear)..



user starts the wrong task, gets interventions suggesting correct subtask. Math task is also suggested as possible alternative task ..



The applications

• Pretty much like real applications– Started from desktop icons..

• Known task model• For now just one application

(AtGentSchool)• Displays interventions allowing..

– to start suggested task– In the future to provide feedback on the

suggestions



The future vision

• Several applications– Some applications provide their task model– Applications may be very different from

what we have seen so far• Task management side bar



Future test center


dissemination and evaluation

Technology

task model applications

task management

different task

math task

wrong task

nontaskbased applications

real applications

task learning goals