AuraPlayFluid User Interface Device build over Raspberry Pi

Prakhar Jawre, Sarthak Garg, Shashank Bhushan, Avishek Nath

Indian Institute of Technology Kanpur, Uttar Pradesh, 208016

Abstract—Augmented reality Technology is revolutionizing theway we interact with computers. Today digital information is nolonger adhered to computer screens. In this paper, we introduce -Pi Lamp, an Augmented reality desktop lamp running onRaspberry Pi. It consists of Camera-Projector system and anarticulated lamp structure, the combination of these enables theuser to manipulate as well as reposition projected informationwith simple gestural interface. Pi lamp uses computer visiontechniques in collaboration with micro-controller basedarticulation system to provide a very intuitive way of interaction.The paper also explains detailed hardware and softwareimplementation of the prototype system.

Index Terms—Raspberry Pi, Arduino, table lamp, fluidinterface.

I. INTRODUCTION

Raspberry Pi is a credit-card sized single board computerdeveloped with the intention to promote teaching of basiccomputer science in schools. It has also been a greatprototyping platform for hobbyists and engineers alike. Alsomany research have been carried out in reality augmentation ontable surfaces. One such research [] involved use of table lampsas a way to project digital information onto the physical world.Inspired from this we thought to build a similar version usingRaspberry Pi.

So we came up with AuraPlay- Playing with light. Itconsists of a camera-projector system mounted on anarticulated robotic arm. AuraPlay is an intuitive approach tointeract with technology. The whole system looks like a tablelamp equipped with a computer. It projects desktop screen ofRaspberry Pi and user can interact with it using a simple LEDpen. User can interact with elements on the screen by simplytaking the pen to that point and lighting the LED. We have alsomade an experimental app for the device, it provides a sketchboard like interface and user can scribble on it just like anormal paper.

The articulated arm is capable of moving the screenanywhere on the desk. The screen can be dragged anywhereacross the desk by simply moving the pen to the desiredlocation.

II. MOTIVATION BEHIND THE PROJECT

We wanted to use Raspberry Pi in a different way. LuminAR-an MIT Media Labs project provided us a platform to use ouridea. So we decided to make a prototype of LuminAR.The

purpose was to create a GUI which is more intuitive to userand we can play with it as if it’s is not a screen but just aimagination. We make wanted to make a new system for userto make it put on a table top and make things more handy atuse

III. BACKGROUND THEORY

When we first thought of auraplay it was inspired by the verybasic component on a study table . A TABLE LAMP , the ideawas not to make just basic system to use as a computer thepurpose was to optimize it for a better use when we have noplace for any “pixel screen” . a screen so intuitive and fluid thatworking with it just not feels like it is computer . Making it tobe comfortable assistive and same tive eualy good as acomputer .the processor for such design should be small open-source and easy to work on . Raspberry pi was answer toalmost all needs , it is cheap and easily available , small as acredit card .light weight and adequately enough to haled allprocessing , plus the use of raspberry pi made easier to ude pi-cam tha is designed for it specifically .

IV. HARDWARE MPLIMENTATION

A. Mechanical Design

To cover the whole desk area, we thought to use acylindrical robotic arm structure for the lamp. The arm ishaving 2 degrees of freedom for the structure and 1 degree offreedom is there on the camera-projector system. Combinationof these constrains make the motion of projected screen veryintuitive.

We have used Autodesk Inventor and Google sketch up todesign the arm.

Figure1: Conceptual design of articulated arm.

The arm is mainly made of aluminum. We have used waterjet cutting facility at IIT Kanpur, to cut out desired partsaccording to our need.

At the base, arm is constrained to move in tangentialdirection. We have used thrust bearing needle washers to makeit rotate freely and also to uniformly distribute the downwardforce. A 1cm wide ring is used as a mount to hold bearing inplace. We have made the base out of two parallel circular discwhich provide a stable platform for rest of the structure. Aservo is firmly mounted between the plate. It providesnecessary angular motion. Servomotor is coupled with 2parallel vertical supports.

Diameter of Discs: 25 cm. ID of Thrust bearing washer: 12 cm. Servo Type: Restricted; Torque: 15.6 Kg/cm. Length of vertical supports: 60 cm.

figure2a: Base; Figure2b: Thrust bearing washer

Simple Rack and Pinion mechanism is used for linearmotion of the arm. There are two pinions to support rack. Oneof them is free to rotate and the other one is coupled with acontinuous rotation servomotor.

Figure3: Rack and Pinion mechanism for linear actuation.

Rack is support from the beneath by two rollers mounted onthe vertical support.

B. Electronic Circuitary

Electronics can be mainly divided into 2 parts, Camera-Projector system and Micro-controller - Servo motorsarticulation system. Projector projects the Raspbianenvironment of Raspberry Pi on the table surface where asCamera takes input of the position of laser pen. Real timechanges are projected back by the projector.

We have used open source Computer Vision librariesto track position of pen. On the first stage of detection we trackthe contour of projected display to set our coordinates andcalibrate our code accordingly. After that mouse tracking is

done. Position of pen is then used to move the cursor to thatlocation. For virtual emulation of mouse we have used“Xdotool” library .

The basic algorithm for supervising the Screen was it todivided into 8 small blocks. When the pen is taken to one of theblock, servo adjusts their position until the pen comes to thecentral block. There are two modes involved in this first beingthe screen dragging and lather is mouse controlling the swap ismade by just making a 5 second pause in the center of thescreen . Using this feedback mechanism screen moveswherever the pen goes. For actuation of arm we have used 3Servo motors which are operated using micro-controller. Wehave used Arduino UNO development board with ATmega328micro-controller for the same. UART communication is set upbetween Arduino and Raspberry Pi using the Wiring Pi library.We assign a character to each of the grids. On pointing the laseron the grids the character corresponding to that grid is sent toarduino using the function SerialPutchar. On receiving thecharacter the arduino moves the servo motors accordingly andthe screen is dragged on the table with the laser.

C. Software Implimentation

Open Computer Vision libraries has been used in the codesof mouse tracking to detect the boundary of theprojected screen using Contour Detection and tocalibrate so that we can get the exact position of themouse pointer For that we firstly detect the contour ofour projected screen of the projector. In contourdetection the coordinates of one of the diagonallyapposite vertexes are marked and of the remainingtwo are calculated . These coordinates are in respectof the captured frame of the pi-cam .then as a laser ispointed inside this frame is coordinated are markedfor this threshold ,it is then send to XDOTOOLS forcalibrating the mouse pointer to the laser pointer.

Thescreen drag software handles all the

functions allocated to be performed on whenwe place pointer on any particular area onscreen. We divided the screen in grid systemand a particular motion is performed onpointing a particular grid. When we point thelaser in a grib a character is send to serialcommunication port of arduino , like this itwas identified and code saved on arduinoruns and perform assigned to that specificcharacter. Like when we point on right mostupper corner on screen we made a character‘A’ send to arduino and code was made tomake a motion such that the it rotatesclockwise and arms moves out , like this allgrid had their own assigned motion.

V. LIMITATIONS

The main limitation to our project is the processing speedof Raspberry Pi. The Pi camera board gives a mere 2fps. ThusOpenCV does image processing slowly and thus we get anotable time lag. we have only provided 2DOF so a lessintuitive for a user as their can’t be zoom in or zoom outoption. Projector does not provide a good resolution andbrightness screen as compared to the real pixel screen .

VI. FUTURE SCOPE

We can make the user interface more friendly byimplementing touch. We can do that by finger tracking. We canalso add zoom in and zoom out mode to our screen bymodifying the hardware. In that case we have to add anotherdegree of freedom to move the arm in vertical direction. A highresolution projector need to be installed.