not petrelli patch2013-presentation

Curators in the Loop: a Quality Control Process

for Personalization for Tangible Interaction in Cultural Heritage

Not Elena

Fondazione Bruno Kessler

Daniela Petrelli

Sheffield Hallam University

6th International Workshop on Personalized Access to Cultural Heritage PATCH2013 @ UMAP2013 – Roma, 14th June 2013

6th International Workshop on Personalized Access to Cultural Heritage (PATCH 2013)

Summary

• Tangible interaction as a key to unlock the synergy between digital and physical

• The meSch scenario

• Personalization challenges

• A pivotal role for authors

• The meSch research agenda


Challenges for personalization deployed in cultural sites

• Portable devices and displays are an opportunity, but divert attention

• Personal devices interfere with the natural social flow

• Difficult trade-off between deep personalization and effort of content preparation

• Investment on hardware and software architectures


Objectives of the meSch project

<< To bridge digital content and the materiality of cultural heritage through tangible and

embedded interaction that create personally meaningful, sensorily rich, and socially

expanded visitor experiences. >>


The physical back at the center

<< A cultural space with smart objects, each with their own digital content

embedded therein, which will be revealed if and when

conditions are right, for example, when visitors have reached

the proper time in the storyline, or a group of them is acting in a certain

way, or another smart object is close by. >>


Creating personalized experiences for a cultural space


Shift the focus from visitors to curators

• Address the overall cycle of personalization: from creation, to delivery onsite and online, to feedback to creation

– Exploit tangible interaction to bridge the gap between digital, material and social

– Put curators in control of authoring and adaptivity, both in content and in context

– Reuse experience of other visitors and authors to improve the system


Interaction through smart objects

• Direct manipulation of augmented exhibits

Replicas supporting interaction

[Petrelli et al., DIS 2012]

[Petrelli et al., CHI 2010]

– Miniaturized multi-sensory integration platforms

– Consumer electronics

– Simple tags


Interaction through smart objects (2)

• Facilitating objects – To activate

functionalities for small, big, fragile exhibits

– To provide interaction context

– To guide

Replicas supporting interaction


Interaction through smart objects (3)

• Ecology of smart objects

– Distributed stories

– Context-aware network

• Opportunities for social interaction

– Sharing of presentations

– Co-action


Adaptivity of content and interaction

• The complexity of creating an adaptive TI experience: – Conception of the experience – Selection of (alternative) narrative threads and

content – Selection of unlocking (inter-)actions – Micro-decisions during contextual onsite delivery – Adaptation of follow-up online exploration

• Tackling complexity: The curator and the system to share the task – The benefits of human judgment – The strengths of automatic mechanisms


Where personalization technology helps

Contextual delivery of

adaptive content and interaction

Recommendation of relevant material, exploiting onsite

interaction

Templates and instantiation rules to compose adaptive

content + interaction

Adaptive lenses for finding the multimedia material for the authoring task (content filtering and recommendation)

What other visitors/ authors have selected is used to refine the system’s behaviour


A necessary decoupling

• To manage complexity

• To distribute authoring tasks according to expertise

• To foster flexibility and reusability

Personalization of content which content is more interesting for which

people

Personalization in context which interaction

mechanisms are more engaging and for whom



– multiple stories for the same interaction

Treasure hunt in a historic museum Treasure hunt in a science museum



– same story for multiple interactions

Looking for companions holding the map of the opposing army and joining the pieces Simply unfolding a map


Supporting personalization in content

LAYER 1 Digital Content search/ recommendation

Recommended content for online

interaction

Retrieved content for authoring task

AD

AP

TIV

ITY

OF

CO

NTE

NT

Digital archives

Data Integration and Access Component

• Content- and collaborative-based recommendation

similarity/novelty with respect to what already selected for presentation

query relaxations to suggest alternative content

similarity with respect to other filled-in templates

similarity with respect to media/format/size


Supporting personalization in content (2)

LAYER 2 Support for Narration design

Adaptive narrative structure



interaction


AD

AP

TIV

ITY

OF

CO

NTE

NT

Digital archives



Layer 2 – support for narration design

• Rules and templates to guide content composition

– choose the style/genre of the story (e.g. emotional, anecdotal, factual, …)

– Micro-activation networks to:

• orchestrate the presence of alternative narrative threads in the same object

• how a story can split over different objects


Layer 2 – support for narration design (2)

• Co-design with curators pre-packaged schemas, e.g.:

– alternative object interpretations (e.g., historical vs. artistic description; functional vs. fabrication info)

– skeletons for narratives based on a temporal sequence (e.g., the life stages of a historical character)

– reflecting a certain topic organization (e.g., comparison of different making techniques)


Supporting personalization in context


LAYER 2 Support for Narration design


interaction


Adaptive narrative structure

LAYER 4 Context-aware adaptive instantiation

Personalized experience delivery through smart exhibits

Digital archives

LAYER 3 Support for Experience design

Adaptive experience structure

AD

AP

TIV

ITY

IN C

ON

TEX

T


AD

AP

TIV

ITY

OF

CO

NTE

NT


Layer 3 – support for experience design

• Rules and templates to guide interaction composition

– Dictionary of available (inter)actions to release content

• Interaction by object manipulation

• Interaction by movement

• Interaction by co-presence

• Interaction by co-activities

– Micro-activation networks to:

• orchestrate the presence of alternative interactive behaviour in the same object

• How different objects contribute to the same interaction


Layer 3 – support for experience design (2)

• Co-design with exhibition designers/artists pre-packaged schemas: – the visitor goes to the object vs. the object goes

with the visitor

– extensive use of collaborative multi-user actions (such as people marching in line or joining pieces)

– object manipulation (e.g., with content disclosed by varied or prolonged manipulations)

– objects search (e.g. in a treasure hunt or to find your enemy to unlock the full story of the battle)


Layer 4 – context-aware adaptive instantiation

• Low-level adaptivity decisions to be taken autonomously by the augmented objects according to the specific interaction context

• Mediation strategies

– Conflicting content decisions (e.g., several visitors following different narrative threads are close to the object at the same time)

– Different valid (inter)actions (e.g., pick up/put in the basket; one person moving / several persons moving)


meSch research plan

concept generation and sketching-in-hardware what a personalized visit can be

analyzing the technical requirements of the multilayer personalization architecture

• unpack the process curators and artists go through when creating a new exhibition

• define and test various classes of content and interaction rules and pre-packaged schemas

• co-design the authoring tools with heritage professionals

• implement the multipurpose personalization services


Conclusion

• Tangible interaction favors the synergy between digital and material – Preserves the centrality of the visit experience – Encourages social interaction – Supports forms of personalization in content and

interaction

• Pivotal role of curators – Technology helps coping with complexity – Human-supervision assures quality

• Flexible personalization component – serve different personalization tasks – portable to different content and (inter)actions

vocabularies – reusable in different physical sites.

The project (2013-2016) receives funding from the European Community’s Seventh Framework Programme ‘ICT for access to cultural resources’ (ICT Call 9: FP7-ICT-2011-9) under the Grant Agreement 600851.

Not Elena Fondazione Bruno Kessler

[email protected]

Daniela Petrelli Sheffield Hallam University

[email protected]