The Papier-Mâché Toolkit

Document, Image and Voice Analysis Research Group (DIVA)

Department of Informatics (DIUF), Faculty of Science

University of Fribourg, Switzerland

Pedro de Almeida, Dominique Guinard, Martin Eric Ritz

“Automation need not require getting rid of paper strips. We suggestkeeping the existing paper flight strips as physical objects.”(Wendy Mackay, Paris-Sud University)

Introduction (I)

• paperless office: owing to the technological progress paper will disappear from the desks and offices

• Statistics: paper does not decrease, but increase• Result: paperless office is a mythpaperless office is a myth• BUT: paper-saturated office is not a failing of

technology; it is a validation of our expertise with the physical world

• Question: how to better integrate the physical and electronic worlds by building physical interfaces?

The paper is dead, long lives the paper!The paper is dead, long lives the paper!

Introduction (II)

• Answer: Tangible user interfaces (TUIs) augment the physical world by combining everyday physical objects with digital information

• BUT: Currently only some experts can build TUIs because of the difficult acquisition and abstraction of physical input.

• Papier-MâchéPapier-Mâché, was developed to simplify the developing of tangible interfaces.

“Papier-Mâché” (I)

The expression has two meanings:

1. Technique for creating forms by mixing wet paper pulp with glue or paste. The crafted object becomes solid when the paste dries. Papier-Mâché was originated by the Chinese.

“Papier-Mâché” (II)

The expression has two meanings:

2. Papier-Mâché stands for a toolkit for building tangible interfaces using computer vision, electronic tags and barcodes.

“Papier-Mâché” (III)

But, what does tangible user interfaces has

to do with a handicraft technology ?

“Papier-Mâché” (IV)

• probably it points on the characteristic of the Toolkit to be able to join several individual elements simply to a whole one.

• During their practical attempts the developers used scraps of paper with different symbols which served as control units by the optical recognition.

InstallationBefore installing the Papier-Mâché Toolkit:

• Install Java 2 Platform Standard Edition 5.0 , the Java Media Framework and Java Advanced Imaging.

• Install a CVS client: Netbeans (integrated CVS support) • if necessary: Install Phidgets SDK for RFID / Java TWAIN for use

a TWAIN source

Make the Papier-Mâché Source-Forge CVS repository accessible from Eclipse

Set the project in Eclipse to be J2SE 5.0 compliant.

To use a camera first run the JMRegistry application.

To use RFID add the lib folder of Papier-Mâché to your path.

Functions (I)

• The Papier-Mâché toolkit aims at providing toolkit level support for physical input.

• Advantages:

– enables developers to build TUIs quite fast

– permits to adapt the underlying sensing technologies with a small expenditure of time.

Therefore a developer has two main tasks:

– declaring the input that he want to process

– and associate it to application behaviour.

Functions (II)

• Papier-Mâché supports computer vision, electronic tags (e.g. RFID tags), and barcode (includes 2D variants) input

• Vision is the most flexible and powerful of these technologies: it supports any camera with a standard driver, from simple webcams to high-quality 1394 cameras.

• Papier-Mâché represents a physical object as a PHOB.

Functions (III)

… a PHOB?!

X

Functions (IV)

• Phobs (Physical Objects) contain an array of data elements (such as an RFID tag) and an array of properties (e.g. location).

• The toolkit provides a monitoring window which displays the current input objects, image input and processing, and behaviors being created or invoked with the association map.

• programmer only responsible for selecting input types. => no need to discover the attached input or to establish a connection to them.

• Once he has selected an input device, Papier-Mâché generates events representing all state changes of the corresponding sensor.

Functions (IV)

• Event types are the same for all different technologies. (facilitates technology portability)

• Events can be filtered using EventFilters. Currently there are three implemented:

– MeanColorClassifier: filters objects whose colour is within distance ε of a given colour.

– ROIClassifier: filters objects in a particular region of interest of the camera view.

– SizeClassifier: filters objects whose size is within a Euclidean distance ε of an ideal size.

Functions (V)

While all technologies have the same events, each technology provides different types of information about the physical objects it senses:

– RFID: tag ID and the reader ID

– Vision: provides the size, location, orientation, bounding box, and mean colour of an object

– Barcodes: provide the ID, the type (EAN, PDF417, or CyberCode), and a reference to the barcode image, which allows vision information such as location and orientation.

Possibilities and borders

• Papier-Mâché enables programmers to program an application with knowing not more than Java.

• …but within Papier-Mâché, the processing is bound by the image processing computations:

On an ordinary computer, Papier-Mâché runs at interactive rates. The developers of the toolkit report that during their tests a dual Pentium 4 running Windows XP was much sufficient for topological and spatial applications with discrete events. The performance numbers indicated by the developers should be considered as lower bounds, as the image processing code is entirely unoptimized.

Integration into larger systems

• Papier-Mâché is designed so that it can be used to-gether with other toolkits

• Developers give 24 examples of existing tangible user interfaces employing paper and other “everyday” objects. In all cases the Papier-Mâché toolkit could be useful.

• This shows clearly, that Papier-Mâché can be used for a multitude of applications even it supports actually only computer vision, electronic tags and barcode input exclusively.

3 Project: Service Counter System

• The Service Counter System is an environment with the capability of providing counter management functionalities through an tangible interface.

• This environment is based on user identification and both optical and radio-frequency objects recognition. Objects motions will enable the user to interact with the counter and, therefore, execute actions.

3 Project: Service Counter System

Digital Camera

Interactive Board

RFID Sensor

3.1 Scenario

• The Library Assistant

INOUT

1. Detect and identify user

1

2. Identify book

3. Choose an action

2

33

3.2 Required Components

• Hardware– An interactive board– A digital Camera– RFID Sensor– RFID Transponders– A PC station

• Software– Netbeans 5.0 (or another IDE)

3.3 Installation procedure

• Create a new project under NetBeans.

• Import the Service Counter System project (located into the CD-ROM).

• Connect the digital camera and the RFID sensor

• Run file « ServiceCounterSystem.java » as an JAVA application under NetBeans.