Demonstration of PocketThumb: aWearable Dual-Sided Touch Interfacefor Cursor-based Control ofSmart-Eyewear

David DobbelsteinUlm University, Ulm, Germanydavid.dobbelstein@uni-ulm.de

Christian WinklerUlm University, Ulm, Germanychristian.winkler@uni-ulm.de

Gabriel HaasUlm University, Ulm, Germanygabriel.haas@uni-ulm.de

Enrico RukzioUlm University, Ulm, Germanyenrico.rukzio@uni-ulm.de

Figure 1: PocketThumb is a dual-sided touch interface embeddedinto the fabric of the trouser’s front pocket. The user controls anabsolute cursor with the thumb by sliding along the touch surfacefrom within the pocket and can tap to select from outside.

Permission to make digital or hard copies of all or part of this work for personal orclassroom use is granted without fee provided that copies are not made or distributedfor profit or commercial advantage and that copies bear this notice and the full citationon the first page. Copyrights for components of this work owned by others than theauthor(s) must be honored. Abstracting with credit is permitted. To copy otherwise, orrepublish, to post on servers or to redistribute to lists, requires prior specificpermission and/or a fee. Request permissions from permissions@acm.org.UbiComp/ISWC’17 Adjunct , September 11–15, 2017, Maui, HI, USA.© 2017 Copyright is held by the owner/author(s). Publication rights licensed to ACM.ACM 978-1-4503-5190-4/17/09...$15.00.https://doi.org/10.1145/3123024.3123185

AbstractWe demonstrate PocketThumb, a wearable touch interfacefor smart-eyewear that is embedded into the fabrics of thefront trouser pocket. The interface is reachable from outsideand inside of the pocket to allow for a combined dual-sidedtouch input. The user can control an absolute cursor withtheir thumb sliding along the fabric from the inside, whileat the same time tapping or swiping with fingers from theoutside to perform joint gestures. This allows for resting thehand in a comfortable and quickly accessible position, whileperforming interaction with a high expressiveness that isfeasible in mobile scenarios.

Author KeywordsWearable input; dual-sided touch; smart-eyewear

ACM Classification KeywordsH.5.m [Information Interfaces and Presentation (e.g., HCI)]:User Interfaces. - Input Devices and Strategies.

IntroductionSmart-eyewear allows for information access and retrievalthat is potentially always available and quickly accessiblewhen the device is worn. This is envisioned to serve as anaugmentation to the user’s memory [7] and to enable shortbursts of interaction that minimize interruption from the taskat hand [1]. With current technology such as Google Glass,

however, interaction is yet a problem. Much like other wear-able devices, input capabilities are negatively affected bythe user’s mobility, by sensing capabilities as well as by alimited input space at the device due to a desired miniatur-ization for wearability.

Figure 2: The PocketThumbinterface is integrated into acommon pair of trousers. Thedual-sided touch sensor can bereached with the thumb from withinthe pocket and with the rest of thehand from the outside.

Wearable interfaces allow to quickly interact while beingmobile. However due to a lack of available input space anddifficulties providing hand stablization in mobile conditions,most wearable touch interaction systems provide only verylimited basic gestures, such as dimensional swiping, thedetection of a general finger tap or individual fixed buttons.While this can be sufficient for very narrow use cases thatdo not rely on many different options, such as accepting ordeclining a phone call, or pausing music, it does not allowfor complex interfaces with many options as familiar fromother mobile devices that allow to directly point at iconsusing a finger or indirectly using a cursor.

In this work, we show the applicability of cursor-basedpointing and selection in wearable contexts. We proposeto use a combined dual-sided touch interaction at the frontpocket of the user’s trousers. By sliding the thumb into thepocket, the hand is stabilized into position where a capac-itive multi-touch sensor is embedded into the fabric (seeFig. 1). The thumb is always in contact with the interfacethrough the fabric and serves as a pointer that is renderedinto the virtual image of the wearable display. The cursorpositioning is absolute, so that the whole display can bereached by sliding the thumb along the interface. Thus, itdoesn’t need to be lifted from the interface during interac-tion, which enhances comfort and hand stabilization at thepocket. The other fingers can access the dual-sided touchsensor from outside the pocket to tap for selection and tofurthermore perform swiping gestures while jointly pointingwith the thumb. We show that this can be used to increase

the input expressiveness of wearable touch interaction.

On-Body Interaction Around the PocketThe pocket and upper thigh area has already been of in-terest in the literature for wearable touch interaction [2].Thomas et al. [8] investigated the placement of a body-attached touchpad mouse for wearable displays in termsof body position and body posture and concluded the frontof the thigh to be the most appropriate position when sit-ting, kneeling and standing. Holleis et al. [4] built capacitivetouch buttons into various garments. People most oftenmentioned the thigh area for where to potentially acceptwearable touch controls.

Saponas et al. [6] showed that capacitive sensing throughfabric is feasible. They re-calibrated a capacitive sensinggrid to enable touch interaction through pockets and inves-tigated signal strength for various fabric materials. It wasshown that stroke-based gestures could be performed fromoutside with most fabrics. We built on top of this finding, byembedding a thin capacitive layer in-between trouser andpocket fabric to allow for sensing not only from the outsidebut also from the inside of the pocket for combined dual-sided interaction.

Dual-Sided Touch on a Single Capacitive LayerPocketThumb [3] is the first dual-sided touch interface uti-lizing a single capacitive touch layer for sensing on bothsides. The capacity intensity of finger touches is similaron both sides. Thus, the sensing grid cannot distinguishand assign its measured signal to a respective side whichgenerates ambiguity. However, its measured intensity isadditive, so that a pinch-through gesture has a high inten-sity that cannot be reached by only touching from one side(see Fig. 4). By this, no separating and shielding layer isrequired, enabling the interface to be thinner. When slid into

the pocket, the thumb’s surface is in contact with the touchinterface, rendering a large blob into the sensor image. Weuse a weighted average of the bottom of the blob as thecursor position representing the tip of the thumb. As long asin pocket, the thumb remains leaning against the interfaceeven during movement, so that it’s absolute position is al-ways rendered as an absolute cursor into the display. Thisway, the thumb does not need to be lifted from the interface(as required by relative touch interfaces) and can remainleaning against the fabric, which enhances hand stabiliza-tion.

Figure 3: The thumb is leaningagainst the rigid touch interface(gray) from within the pocket andserves as a cursor. By tapping withthe index finger from the other side,the user can perform a selection.

a) b) c)

Figure 4: The thumb is sliding along the interface and by thatmoving the cursor at its tip (a&b). As soon as the index fingertouches the interface, the pinch generates a higher capacitiveintensity and can thus be detected (c).

Dual-Sided Touch GesturesThe thumb can not only be used as a cursor, but also as aspatial point of reference for the remaining hand. Hence, wewant to explore the capabilities of using the thumb for point-ing and the remaining fingers for jointly performed gestures.For single-sided touch interaction, the capabilities for point-ing and joint gestures are very limited due to the pointingfinger reducing the degrees of simultaneous movement ofthe remaining hand. The only finger that can independentlybe moved over its saddle joint is the thumb. This is utilized

in current touch systems for pinch-to-zoom, where thumband index finger are moving with a high degree of freedom.However, when other fingers are concurrently used, theyare very dependent on each other and bound to move to-gether, such as when swiping with multiple fingers into thesame direction (e.g. for scrolling). This limitation in handmotion leads to users either pointing at a target with a fin-ger or performing a complex gesture, but not doing both atthe same time with the same hand. By using the thumb asa pointer in dual-sided touch interaction, the high degreeof freedom of the thumb’s saddle joint enables indepen-dent movement of the remaining hand and by that concur-rently performed gestures. Since the thumb is opposing theother fingers, it is not obstructing their movement and caninstead serve as a point of reference in the user interface.We demonstrate four gestures that utilize this interaction(see Fig. 5):

Spatial TappingUsers can use their thumb as a proprioceptive referencefor tapping with their fingers. This is used for the introducedpinching gesture, where users aim for the thumb selection.It is however also possible to aim beside the thumb. Userscan willingly tap left or right of the pointing cursors, whichcan be used as an analogy to left and right clicking of amouse to increase the expressiveness of a touch selection.

Grab-and-dragDragging is a basic operation in many touch-based inter-faces to move a target along the display that is pinned to apointing finger. For the dual-sided PocketThumb interface,a target can be grabbed from both sides and then draggedalong the display. This corresponds to physical interaction,where the thumb is opposing the rest of the hand and pro-viding force to grab and move an object.

Pinch-and-circleWhen the dominant characteristic of a human grip is pre-cision, the gripped object is pinched between index fingerand the opposing thumb [5]. This allows to flex and axiallyrotate both fingers and by that precise manipulations. Weutilize this for a pinch-and-circle gesture where the usercan pinch their thumb and then circle the opposing indexfinger around it for fast and precise interaction. In an userinterface, this allows to rotate a virtual knob or to quicklynavigate through a list by continuous circling without havingto lift the finger.

Point-and-swipe

Figure 5: While the thumb is usedfor cusor-based pointing, theopposing fingers can be used forconcurrently performed gestures:a) to commit a ’right-click’, b) todrag items c) to circle throughcontent and to d) navigate byswiping.

Swiping is commonly used for touch-based interaction tonavigate through content such as when scrolling a page orswitching through displayed interfaces. For PocketThumbusers can use their fingers for swiping while pointing withthe thumb to quickly navigate through complex menu struc-tures. This can be used to switch the current application(left and right swipe) or to invoke or close menu interfacesrelated to a pointed target (up and down swipe).

ConclusionPocketThumb [3] is a wearable touch interface embeddedinto the trousers’ front pocket for combined dual-sided in-teraction utilizing a single capacitive touch layer for richinteraction. The thumb stabilizes the hand from inside thepocket and allows for cursor-based interaction. Further-more, the input expressiveness can be increased by usingthe thumb as a spatial point of reference for finger gesturesperformed on the front of the interface.

REFERENCES1. Daniel Lee Ashbrook. 2010. Enabling mobile

microinteractions. (2010).

2. David Dobbelstein, Philipp Hock, and Enrico Rukzio.2015. Belt: An unobtrusive touch input device forhead-worn displays. In Proceedings of the 33rd AnnualACM Conference on Human Factors in ComputingSystems. ACM, 2135–2138.

3. David Dobbelstein, Christian Winkler, Gabriel Haas,and Enrico Rukzio. 2017. PocketThumb: a WearableDual-Sided Touch Interface for Cursor-based Control ofSmart-Eyewear. Proceedings of the ACM onInteractive, Mobile, Wearable and UbiquitousTechnologies 1, 2 (2017), 9.

4. Paul Holleis, Albrecht Schmidt, Susanna Paasovaara,Arto Puikkonen, and Jonna Häkkilä. 2008. Evaluatingcapacitive touch input on clothes. In Proceedings of the10th international conference on Human computerinteraction with mobile devices and services. ACM,81–90.

5. John R Napier. 1956. The prehensile movements of thehuman hand. Bone & Joint Journal 38, 4 (1956),902–913.

6. T Scott Saponas, Chris Harrison, and Hrvoje Benko.2011. PocketTouch: through-fabric capacitive touchinput. In Proceedings of the 24th annual ACMsymposium on User interface software and technology.ACM, 303–308.

7. Thad E Starner. 2002. Attention, memory, andwearable interfaces. IEEE pervasive computing 1, 4(2002), 88–91.

8. B Thomas, Karen Grimmer, J Zucco, and SteveMilanese. 2002. Where does the mouse go? Aninvestigation into the placement of a body-attachedtouchpad mouse for wearable computers. Personal andUbiquitous computing 6, 2 (2002), 97–112.