datatiles: a modula platform for mixed physical and digtial interactions (rekimoto, chi2001)

Copyright© 2001 Sony Computer Science Laboratories, Inc.

DataTilesDataTiles ：：A Modular Platform for Mixed Physical A Modular Platform for Mixed Physical

and Graphical Interactionsand Graphical Interactions

April 4, 2001April 4, 2001

Jun RekimotoJun Rekimoto** Brygg Ullmer Brygg Ullmer†† Haruo Oba Haruo Oba**

* * Sony CSL Interaction LaboratorySony CSL Interaction Laboratory† † MIT Media LaboratoryMIT Media Laboratory

2Copyright© 2001 Sony Computer Science Laboratories, Inc.

MotivationMotivation

Flexible interface platform for controlling Flexible interface platform for controlling digital informationdigital information

Merging two UI paradigms Merging two UI paradigms Graphicsl UIs & Physical/Tangible UI Graphicsl UIs & Physical/Tangible UI

Visual+Physical “construction kit” for end Visual+Physical “construction kit” for end usersusers

Some historical Some historical ancestors…ancestors…

“Denshi (electronic) Block” (‘70s)Japan Railload “MARS” terminal (‘70s)

“Triangles” (Gorbet/Orth/Ishii, 1998)

“AlgoBlock” (Suzuki/Kato, 1995)

“Intelligent Beermats” (Frazer, 1982)


DataTiles


Transparent objects (tiles) as information modules → tight coupling of input & output

Integration of Physical manipulation & graphical information Used as Home-Computer interfaces, Collaboration supports, Internet-”Toys”

DataTiles


Time Point Information


Tile ClassesTile ClassesApplication Tiles

Portal (Remote) Tiles

Parameter Tiles

Container Tiles

Stores information

Specific Services, Applications

Connects to real-world objects(places, people, devices)

Visualizes & Controls other connected tiles


video


DataTiles DataTiles basic interaction style basic interaction style

Pen Sensitive LCD Display (“Tray”)ID-Reader

tagged transparent objectRFIDRFID

GroovesPrinted Information

Input devices (Pen, etc.)InputGraphical Output


Interacting with DataTilesInteracting with DataTiles Interaction TechniquesInteraction Techniques

Tile placement, reconfiguration, removalTile placement, reconfiguration, removal Pen manipulations on each tile (as GUI)Pen manipulations on each tile (as GUI) Physical manipulators (e.g., dial)Physical manipulators (e.g., dial) Inter-tile pen gesturesInter-tile pen gestures

Tile adjacency Tile adjacency communications between tiles communications between tiles Parameter flow: Query Parameters, Time pointsParameter flow: Query Parameters, Time points Data-flow: Image, movie, soundData-flow: Image, movie, sound


Printed & Grooved TilesPrinted & Grooved Tiles

=+

+ =

printed information:high resolution,

static

displayed information:low resolution,

dynamic

grooved widget:passive haptic

feedback

combined:


Tiles with manipulatorsTiles with manipulators


Container PaintPortal

WebCam

print

image

real printer Application Tile

Container Tile

Portal Tile

Container

Movie Movie Movie

Examples of Tile CombinationsExamples of Tile Combinations

ParametersBaseballTimeWheelScribble

time point

parameter tile parameter tile

query parameters


Visual Feedback of tile connection Visual Feedback of tile connection typestypes

Inter-tile pen gesturesInter-tile pen gestures

Movie Wall Screen ATimeMachineWeather

Simple connection

PeopleContainer

Menu for choosingConnection types

Mail

Fax

Invokes one-shot transmissionPhotoAlbumContainer Remote Portal

Suspends streamed transmission

Animation indicating streamed media transmission

System ConfigurationSystem Configuration

RFID tags (TIRIS)

RFID reader(TIRIS)

RFID multiplexer+

PC interface(PIC16F873)

RS232C

Switches 12 antenna coils（ 100ms / ID)

RFID reader coils

PC (Sony VAIO LX)Windows 2000

Pentium III (866MHz)Java JDK1.3

LCD Display with built-in pen tablet


LCD display

Back lightPen digitizer

Wire to the RF reader

Acrylic tile cover glass

6mm

RF tag

RFID reader coils

Wire to the RFID reader

RFID tag

Reader coil

Sensor ConfigurationSensor Configuration


Implementation (software)Implementation (software)

The System dynamically loads and activates a corresponding Java Class for each tile ID.

Each Java program also dynamically checks nearby tile classes and interfaces.

public class WebCamTile extends Tile implements ImageCreateInterface,

TimeMachineInterface, … {…………

}

……Tile t = getLeftTile();if (t instanceof TimeMachineInterface) {

…..}….


User ExperiencesUser Experiences Data manipulation by physical layoutData manipulation by physical layout

Many users were willing to explore various types of Many users were willing to explore various types of tile combinations. Some of them were beyond our tile combinations. Some of them were beyond our expectation.expectation.

Visual feedback was essentialVisual feedback was essential

Effects of tiles’ graspable capabilityEffects of tiles’ graspable capability Users could always take control Users could always take control

(e.g., remove a music tile to stop it) (e.g., remove a music tile to stop it) Suitable for multi-user activitiesSuitable for multi-user activities


ApplicationsApplications Interfaces to Networked InformationInterfaces to Networked Information

Portal to Information servicesPortal to Information services Information Appliances ControlInformation Appliances Control Remote CollaborationRemote Collaboration

Media Construction EnvironmentsMedia Construction Environments Movie/Music Editing, Video JockeyMovie/Music Editing, Video Jockey (Networked) Game Environment(Networked) Game Environment Education Tools, ToysEducation Tools, Toys

Support of Collaborative Work, Decision Support of Collaborative Work, Decision makingmaking Simulation, database visualizationSimulation, database visualization


SummarySummary

DataTiles:DataTiles: 　　 A platform environment with mixed Physical-A platform environment with mixed Physical-

Graphical InteractionsGraphical Interactions Future WorkFuture Work

Study of Physical Language “Grammar”Study of Physical Language “Grammar” Other configuration method (e.g, stacking)Other configuration method (e.g, stacking) Active (CPU embedded) tiles with sensorsActive (CPU embedded) tiles with sensors

datatiles: a modula platform for mixed physical and digtial interactions (rekimoto, chi2001)

Technology