Illuminating Gesture Interfaces withInteractive Light Feedback

Euan FreemanGlasgow Interactive Systems GroupUniversity of GlasgowGlasgow, Scotlandeuan.freeman@glasgow.ac.uk

Stephen BrewsterGlasgow Interactive Systems GroupUniversity of GlasgowGlasgow, Scotlandstephen.brewster@glasgow.ac.uk

Vuokko LantzNokia TechnologiesEspoo, Finlandvuokko.lantz@nokia.com

Beyond the Switch: Explicit and Implicit Interaction with Light workshop atNordiCHI ’14. October 26, 2014, Helsinki, Finland.

AbstractInteractive lighting offers us new ways to interact with lightsin our environment; however, it also gives new ways for ourenvironments to interact with us. We propose using interac-tive lighting for feedback during gesture interaction, illumi-nating the space around devices for feedback. We discussexamples of interactive lighting in gesture interfaces andpresent the Gesture Thermostat, a gesture interface whichuses interactive lighting to give around-device feedback.

Author KeywordsGestures; Interactive Light; Visual Feedback

ACM Classification KeywordsH.5.2 [User Interfaces]: Interaction Styles.

IntroductionInteractive lighting lets us interact with light sources in ourenvironments in new ways, for example adjusting lightingwith mobile phone applications [5] and hand gestures [1].We think interactive lighting also gives interfaces in our en-vironments a new way to interact with us, using light to givefeedback about interaction. LEDs embedded in devices andconnected light sources can be highly expressive, allowinguser interfaces to give us rich and attractive feedback whilewe interact with them.

In our research we focus on gesture interfaces using mid-airhand movements, for example gestures over mobile phonesor near household interfaces. Gesture interfaces especiallycan benefit from interactive light feedback as their usersare often interacting from a distance, where other types ofvisual feedback may be less noticeable. Interactive lightcan illuminate the space around these devices, increasingthe effective display size and giving easily noticed visualcues. Light is also less obtrusive than audio feedback and,unlike tactile feedback, does not require additional tech-nology. Our paper looks at some examples of light beingused in gesture interfaces and we show how we can build onthese interfaces to use interactive light as a useful outputmodality. We also introduce our Gesture Thermostat as anexample of interactive light feedback.

Related WorkLight has been used to extend the displays of small devicesby illuminating surrounding surfaces. Qin et al. [8] usedlights around a mobile phone to show off-screen contentand to give visual hints about how to gesture around thedevice. During incoming phone calls, users could accept orreject calls by touching the table surface beside the phone.Green and red light showed users which side to touch toaccept and reject calls. ambiPad [4] used light around atablet to enhance on-screen content.

We think around-device light could also be used to giveusers feedback while they gesture. Users gesturing fromacross the room (e.g. with Surround-See [9]) would benefitfrom more noticeable feedback in a larger area and usersgesturing closer to the display (e.g. with HoverFlow [3])could receive visual cues without already limited screenspace being used for gesture feedback. We build on Dy-namic Ambient Lighting [8] by using interactive light feed-back for dynamic gesture feedback rather than static cues.

Interactive light feedback was given implicitly from Grace [1],an interactive lamp. As users gestured to adjust light givenfrom the lamp (e.g. decreasing brightness), they were givenfeedback in real time as ambient light changed in responseto gestures. Connected lamps could also be used to givefeedback for other gesture interfaces, particularly those withno physical representation in a room (e.g. whole-home ges-ture sensing with WiSee [7]). Gesture interfaces couldtake advantage of interactive lighting connectivity by an-nexing [6] light sources for giving users feedback.

Figure 1: Rainbowfish [2] glowing green to show gestureacceptance as users gesture over the surface.

Interactive surfaces have also used light to give feedbackabout gestures above the surface. Rainbowfish [2] usedLEDs to give visual feedback as hands hovered over it.Lights created a “shadow” which followed users’ hands andlight was used to show gesture acceptance (Figure 1) or re-jection. In our work we focus on giving similar visual cuesin the space surrounding devices. Light feedback arounddevices extends the display space and keeps the display freefor interactive content.

Gesture ThermostatIn our paper we have shown that interactive lighting couldbe used to give feedback during gesture interaction by illu-minating the space around devices. We think interactivelight feedback can benefit gesture interfaces by extend-ing the display space, making feedback more noticeableas users gesture from a distance. Interactive light feed-back also means devices with small displays do not haveto take screen space away from on-screen content. As anexample of interactive light feedback we present the Ges-ture Thermostat (Figure 2). The Gesture Thermostat isthe first gesture interface which uses light to give dynamicaround-device feedback during interaction.

Figure 2: Gesture Thermostat giving interactive lightfeedback. The blue light shows the current thermostat setting.

As the name suggests, the Gesture Thermostat is a ther-mostat which users can interact with using hand gesturesfrom across the room. Thermostats are typically small withlimited display capabilities so interactive light feedback isideal for communicating with users. As users gesture at ourthermostat (by rotating their hand to the left or right, as

though turning a dial), LEDs embedded around the edge il-luminate the surrounding wall to give users feedback abouttheir gestures.

Interactive Light Feedback DesignAs people are sensed by the thermostat, its lights turn on ata low brightness using white light. This is intended to revealthe interface to potential users without being obtrusive anddistracting. If people show no intention to interact with thethermostat, all lights fade out.

Figure 3: Gesture Thermostat giving interactive lightfeedback. The red light shows the current thermostat setting.

If users raise their hand to gesture, however, all lights beginto pulse gently. This light behaviour is intended to showusers that the thermostat is aware of their hand movements.A single point of light shows the current thermostat setting,using blue light if on the left side of the thermostat dial(Figure 2) and red light if on the right side of the thermostatdial (Figure 3). Blue and red light pulses continuously alongwith the white light to show continued responsiveness.

We also use light to give feedforward, hinting to users howto use the thermostat. If users do not gesture within a fewseconds of activating the interface, light starts to movearound the thermostat dial. Starting at the twelve-o’clockposition, lights turn on in sequence moving anti-clockwise(in blue, suggesting cooler temperatures) until six-o’clock,before starting again in a clockwise direction (in red, sug-gesting warmer temperatures) until six-o’clock. These an-imations show the effect of turning the hand to the left(decrease temperature) and right (increase temperature).

Our interactive light feedback uses several characteristics oflight to help users gesture. Spatial location of light aroundthe thermostat dial is used to show the current thermostatsetting. We use iconic colour (blue and red, for cold andhot) to reinforce the thermostat metaphor and provide ad-ditional cues. Animation shows continued responsivenessto gestures (gently pulsing white light) and also gives hintsabout gesturing. Initial usability evaluations found that in-teractive light feedback is aesthetically pleasing and helpsusers gesture effectively, by showing system attention andresponsiveness to gestures.

ConclusionInteractive lighting can be used to give users feedback dur-ing gesture interaction. Gesture interfaces can illuminatetheir surroundings using embedded lights or control lightfrom connected light sources, giving users easy to noticevisual cues. Small devices in particular can benefit frominteractive light feedback as ambient light increases thedisplay size, is noticeable from across the room and keepsthe main display free for other content. We discussed ex-amples of gesture interfaces which use light for feedbackand illustrated interactive light feedback with a gesture in-terface of our own, the Gesture Thermostat. We encourageinteractive light designers to consider how their lights may

be used by other interfaces for output. We would also liketo see others use interactive light to illuminate their inter-faces with light feedback, giving users useful and attractivevisual feedback from a distance.

AcknowledgementsThis research was part-funded by Nokia Research Center,Finland. We would like to thank Tobias Große-Puppendahlfor permission to use Figure 1.

References[1] T. Djajadiningrat, L. Geurts, J. D. Bont, and P.-y. Chao.

Grace: A gesture-controlled wake-up light. In Proc. DeS-ForM ’12, 130–136, 2012.

[2] T. Grosse-Puppendahl, S. Beck, D. Wilbers, S. Zeiß, J. vonWilmsdorff, and A. Kuijper. Ambient Gesture-RecognizingSurfaces with Visual Feedback. In Proc. DAPI ’14, 97–108.Springer, 2014.

[3] S. Kratz and M. Rohs. HoverFlow: Expanding the De-sign Space of Around-Device Interaction. In Proc. Mobile-HCI ’09, Article 4. ACM, 2009.

[4] M. Lochtefeld, N. Lautemann, S. Gehring, and A. Kruger.ambiPad – Enriching Mobile Digital Media With AmbientFeedback. In Proc. Mobile HCI ’14, to appear. ACM, 2014.

[5] A. Lucero., T. Lashina, and J. Terken. Reducing Complexityof Interaction with Advanced Bathroom Lighting at Home.i-com, 5(1):34–40, 2006.

[6] J.S. Pierce, and H. Mahaney. Opportunistic Annexing forHandheld Devices: Opportunities and Challenges. In Proc.HCIC ’04. 2004.

[7] Q. Pu, S. Gupta, S. Gollakota, and S. Patel. Whole-HomeGesture Recognition Using Wireless Signals. In Proc. Mobi-Com ’13, 27–38. ACM, 2013.

[8] Q. Qin, M. Rohs, and S. Kratz. Dynamic Ambient Lightingfor Mobile Devices. In UIST ’11 Adjunct, 51–52. 2011.

[9] X.-d. Yang, K. Hasan, N. Bruce, and P. Irani. Surround-See:Enabling Peripheral Vision on Smartphones during ActiveUse. In Proc. UIST ’13, 291–300. ACM, 2013.