Department of Science and Technology Institutionen för teknik och naturvetenskap Linköping University Linköpings universitet

gnipökrroN 47 106 nedewS ,gnipökrroN 47 106-ES

LiU-ITN-TEK-A--17/052--SE

Towards replacing the remotecontrol with commoditysmart-phones through

evaluation of interactiontechniques enabling

television service navigationEmma Forsling Parborg

2017-10-05

LiU-ITN-TEK-A--17/052--SE

Towards replacing the remotecontrol with commoditysmart-phones through

evaluation of interactiontechniques enabling

television service navigationExamensarbete utfört i Medieteknik

vid Tekniska högskolan vidLinköpings universitet

Emma Forsling Parborg

Handledare Karljohan Lundin PalmeriusExaminator Camilla Forsell

Norrköping 2017-10-05

Upphovsrätt

Detta dokument hålls tillgängligt på Internet – eller dess framtida ersättare –under en längre tid från publiceringsdatum under förutsättning att inga extra-ordinära omständigheter uppstår.

Tillgång till dokumentet innebär tillstånd för var och en att läsa, ladda ner,skriva ut enstaka kopior för enskilt bruk och att använda det oförändrat förickekommersiell forskning och för undervisning. Överföring av upphovsrättenvid en senare tidpunkt kan inte upphäva detta tillstånd. All annan användning avdokumentet kräver upphovsmannens medgivande. För att garantera äktheten,säkerheten och tillgängligheten finns det lösningar av teknisk och administrativart.

Upphovsmannens ideella rätt innefattar rätt att bli nämnd som upphovsman iden omfattning som god sed kräver vid användning av dokumentet på ovanbeskrivna sätt samt skydd mot att dokumentet ändras eller presenteras i sådanform eller i sådant sammanhang som är kränkande för upphovsmannens litteräraeller konstnärliga anseende eller egenart.

För ytterligare information om Linköping University Electronic Press seförlagets hemsida http://www.ep.liu.se/

Copyright

The publishers will keep this document online on the Internet - or its possiblereplacement - for a considerable time from the date of publication barringexceptional circumstances.

The online availability of the document implies a permanent permission foranyone to read, to download, to print out single copies for your own use and touse it unchanged for any non-commercial research and educational purpose.Subsequent transfers of copyright cannot revoke this permission. All other usesof the document are conditional on the consent of the copyright owner. Thepublisher has taken technical and administrative measures to assure authenticity,security and accessibility.

According to intellectual property law the author has the right to bementioned when his/her work is accessed as described above and to be protectedagainst infringement.

For additional information about the Linköping University Electronic Pressand its procedures for publication and for assurance of document integrity,please refer to its WWW home page: http://www.ep.liu.se/

© Emma Forsling Parborg

Abstract

This master thesis has been carried out at Zenterio Services in Linköping and the department of science andtechnology at Linköping University. Zenterio Services had a wish to explore different interaction techniquesthat could be performed on a smartphone in order to control an application that is displayed on anotherscreen by a set-top-box, or any other browser based application. The aim for this project was thus to developan application that would be compatible with set-top boxes, or other browser based applications, and re-search what interaction techniques that could be considered a viable substitute for a the traditional remotecontroller without requiring the visual attention of the viewer.

During the development, the application was tested by employees at Zenterio Services. A usability test wasconducted in order to determine if the interaction techniques could be used without forcing the user to lookdown on the phone-screen, and also to determine if the user would use it instead of a remote control. Theuser test was also performed in the interest of broadly evaluating the different interaction techniques usedin the application, and how the UI itself, including nonvisual feedback from both the sender and receiverside is perceived.

The results illustrate that the participants in the user test do mostly prefer a touch based solution, with aGUI that clearly provides information about which item is currently selected and where the user can nav-igate to. The interaction method should enable the user to navigate over multiple items (i.e. scrolling, orskipping without consecutively marking each item between the current and the desired one).

For all the interaction methods tested, potential improvements have been identified that could further re-fine the application and tune the sensitivity for the interaction methods. There are also improvements sug-gested that are primarily related to the graphical UI itself, one example from this category is improving thedistinction between when an item is selected or not. Another is that sounds or tactile feedback in the formof vibration from the phone could be beneficial.

There are also suggestions for next steps that could add value by going beyond the parameters for thisproject, like creating a secure pairing connection between the phone and the receiving device, as well asenhancing the feedback given from both the receiving device and the sending device.

Sammanfattning

Detta examensarbete genomfördes på Zenterio Services i Linköping och Institutionen för Teknik och Natur-vetenskap på Linköpings Universitet. Zenterio Services ville undersöka olika interaktionstekniker som kananvändas för att styra ett användargränssnitt som visas på en annan skärm, exempelvis att styra en TV-enhetmed en smart-phone.

Målet med examensarbetet är att utveckla en webbaserad applikation, som är kompatibel med set-top-boxar eller andra webbläsarbaserade applikationer, samt undersöka vilka interaktionstekniker som är ettbra alternativ till en fjärrkontroll, och som kan användas utan att behöva lämna huvudgränssnittet medblicken.

Under projektets utformande så testades applikationen på plats av anställda på Zenterio Services. Ett an-vändartest utfördes för att testa användarbarheten och effektiviteten hos de olika interaktionsteknikerna,samt för att undersöka huruvida användaren kan nyttja interaktionsmetoderna utan att titta ner på mobi-len. Dessutom undersöktes huruvida användaren skulle vilja använda den istället för en fjärrkontroll.

Resultatet ifrån användartestet visade att användarna gillade en touch-baserad lösning bäst, med ett grafisktanvändargränssnitt som tydligt ger information om vilka komponenter som är valda och vart användarenkan navigera. Interaktionsmetoden ska även göra det möjligt för användaren att kunna navigera över flerakomponenter, exempelvis genom scrollning.

För alla interaktionsmetoder som testades så upptäcktes många potentiella förbättringar som kan imple-menteras. Utöver det så uppmärksammades även att förbättringar bör ske i användargränssnittet, exempel-vis att markeringen av valda komponenter bör bli tydligare. Önskningar om att få återkoppling från mobilentogs upp, där ljud eller vibrationer skulle kunna vara en lösning. Dessa skulle även kunna vara användbaraom fördröjningar sker mellan mottagaren och sändaren, för att visa för användaren att dess aktivitet harblivit registrerad och inte behöver repeteras.

För att kunna vidareutveckla denna produkt, så krävs ytterligare arbete som ligger utanför detta examens-arbetes omfattning. Två exempel är att en säker koppling mellan mobilen och mottagaren bör skapas, samtatt utöka och förbättra den återkoppling som ges från båda enheterna.

Acknowledgements

I would like to extend a special thank you to Zenterio AB for giving me the opportunity to do my masterthesis at their company. I would especially like to thank my supervisors at Zenterio Services, Johan Jernlåsand Martin Henriksson, for their help and guidance throughout this master thesis project. I would also liketo express special thanks to my examiner Camilla Forsell and my supervisor Karljohan Lundin Palmerius.

Nomenclature

List of abbreviations used in this thesis.

Abbreviations

UI User InterfaceUX User ExperienceGUI Graphical User InterfaceSTB Set-top boxIPTV Internet Protocol televisionOTT Over-the-topOTT STB Over-the-top set-top boxRCU Remote Control Unit

Contents

1 Introduction 1

1.1 Purpose . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1

1.2 Problem analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2

1.3 Problem statements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2

1.4 Delimitations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2

1.5 Disposition of the Report . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2

2 Background 4

3 Related work and Theories 5

3.1 Previous work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

3.2 Set-top box . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

3.3 User . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

3.4 Usability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

3.4.1 ISO . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

3.4.2 REAL . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

3.4.3 LEMES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

3.5 User testing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

3.6 User-centered Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

3.7 System architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

3.8 MV* patterns . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10

3.8.1 MVC . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10

4 Approach 11

4.1 Strategy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

4.2 Interaction techniques . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

4.2.1 Method A . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

4.2.2 Method B . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12

4.2.3 Method C . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14

4.3 System architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14

5 Implementation 16

5.1 Tools and Technologies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16

5.1.1 Apache Cordova . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16

5.1.2 Backbone . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16

5.1.3 Mayo . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17

5.1.4 Git . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17

5.1.5 MATLAB . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17

5.2 Graphical design decisions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17

5.3 Prototypes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18

5.4 Implementation of interaction techniques . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19

5.4.1 Method A . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19

5.4.2 Method B . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20

5.4.3 Method C . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20

5.5 Application overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21

5.6 Application development process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22

6 Tests 25

6.1 User test . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25

6.2 The tasks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26

6.2.1 Application used in the user tests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27

7 Results 28

7.1 The users . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28

7.2 User tests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28

7.2.1 Usability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28

7.2.2 Utility . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31

7.2.3 The overall results of the interaction methods . . . . . . . . . . . . . . . . . . . . . . . . . 36

7.3 Application . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37

8 Discussion 40

8.1 Previous experience and expectations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40

8.2 Impact of visual design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40

8.3 Impact of movement styles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41

8.4 Haptic feedback on the client side . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41

8.5 Errors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42

8.6 Test instructor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43

8.7 What is the best method? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43

9 Lessons learned 44

9.1 Abstraction Boundaries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44

9.2 Build process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44

9.3 Development process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45

9.4 STB specific DOM issues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45

9.5 Documentation for javascript libraries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46

9.6 Git . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46

10 Conclusion 47

11 Future work 48

11.1 Sensitivity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48

11.2 Communication and pairing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48

11.3 Feedback . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49

11.4 Future tests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49

References 50

Appendix A Results 52

A.1 ANOVA and Post-hoc test - MATLAB . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53

A.1.1 ANOVA1 and post-hoc calculated for Chapter 7.2 . . . . . . . . . . . . . . . . . . . . . . . 53

A.1.2 ANOVA1 and post-hoc calculated for Chapter 7.2.1.1 . . . . . . . . . . . . . . . . . . . . . 53

A.1.3 ANOVA1 and post-hoc calculated for Chapter 7.2.1.2 . . . . . . . . . . . . . . . . . . . . . 53

A.1.4 ANOVA1 and post-hoc calculated for Chapter 7.2.1.3 . . . . . . . . . . . . . . . . . . . . . 55

A.1.5 ANOVA1 and post-hoc calculated for Chapter 7.2.1.3 . . . . . . . . . . . . . . . . . . . . . 55

A.1.6 ANOVA1 and post-hoc calculated for Chapter 7.2.1.4 . . . . . . . . . . . . . . . . . . . . . 55

A.1.7 ANOVA1 and post-hoc calculated for Chapter 7.2.2.2, Table 5. . . . . . . . . . . . . . . . . 56

A.1.8 ANOVA1 and post-hoc calculated for Chapter 7.2.2.2, Table 6. . . . . . . . . . . . . . . . . 56

A.1.9 ANOVA1 and post-hoc calculated for Chapter 7.2.2.2, Table 7. . . . . . . . . . . . . . . . . 57

A.1.10 ANOVA1 and post-hoc calculated for Chapter 7.2.3, Table 8. . . . . . . . . . . . . . . . . . 57

A.1.11 ANOVA1 and post-hoc calculated for Chapter 7.2.3, Figure 24 and Table A2. . . . . . . . 57

Appendix B Product description (in Swedish) 59

Appendix C User Test Protocol 62

C.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62

C.2 Background information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63

C.3 Questions - Remote control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64

C.4 Questions - Touch swipe and tap . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65

C.5 Questions - Touch with cursor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66

C.6 Questions - Orientation and tap . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67

C.7 Final questions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68

List of Figures

1 JetUI [Zenterio 2016] . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

2 Illustrates how the STB is connected to a TV set and how the user generally interacts with itusing two remote controls: one controlling the TV and the other for controlling the STB. . . . . 6

3 User centered evaluation. Figure adapted from [Saffer, D. 2009]. . . . . . . . . . . . . . . . . . . 9

4 Interaction with the receiving device (a) is performed by tapping (b) and swiping (c) on thesending device, in order to select and highlight items respectively. . . . . . . . . . . . . . . . . . 12

5 Interaction with the receiving device (a) is performed by tapping (b) and dragging (c), in orderto select items and move the cursor, respectively. Dragging two fingers (d) or placing the cursorat the bottom of the page, will trigger a scrolling of the page. . . . . . . . . . . . . . . . . . . . . 13

6 Interaction with the receiving device is performed by tapping and gestures in order to selectitems and move the cursor, respectively. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14

7 Zenterio logo, Figure retrieved from [Zenterio Marketing 2016a] . . . . . . . . . . . . . . . . . . 17

8 Figure illustrating colors taken from the Zenterio logo and from the JetUI interface. . . . . . . . 18

9 Figure illustrating the icons used for the pause, play and back buttons. . . . . . . . . . . . . . . 18

10 Figure illustrating the initial prototype. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18

11 Second prototype. Movie poster images from [Blender Foundation 2008] . . . . . . . . . . . . . 19

12 The limited intervals for the angles α and β . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21

13 Figure illustrating the different models and views created for the application. . . . . . . . . . . 22

14 The different views for the browser and the phone. . . . . . . . . . . . . . . . . . . . . . . . . . . 23

15 The overall application development process. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24

16 Remote control used in the user tests, with the buttons used lying within the red area. . . . . . 26

17 Figure illustrating the UI for the devices during the user tests. . . . . . . . . . . . . . . . . . . . . 27

18 Average time for completing the tasks using the different interaction techniques. . . . . . . . . 29

19 Method C differs from the rest of the interaction methods. . . . . . . . . . . . . . . . . . . . . . . 33

20 Multiple comparison of means. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34

21 The means of Method B and Method C are significantly different. . . . . . . . . . . . . . . . . . 35

22 Illustration of how the user held the phone during the tests and controlled it with the thumb(a), and how the user would have to use the other hand too use the two finger scroll function-ality (b) Images retrieved from [Dreamstime 2017]. . . . . . . . . . . . . . . . . . . . . . . . . . . 35

23 Figures displaying the post-hoc results for the grades given by the users. . . . . . . . . . . . . . 36

24 Figures displaying the responses on which of the methods that the users would like to useinstead of a remote control. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37

25 The final views for the smart-phone and the receiving device. . . . . . . . . . . . . . . . . . . . . 39

26 Fine tuned movements (a) and elbow and shoulder movements (b). Images from [Medlej 2014] 42

27 How the application currently sends information from the smart-phone. . . . . . . . . . . . . . 44

1 | Introduction

Television is one of the central entertainment devices in homes and most people have at one time or an-other used a TV, and indeed, a TV remote control. Not much has changed since the introduction of the TVremote, however many previous attempts have been made to enhance the TV experience by creating alter-native remote controls in order to control the TV and/or other devices in the home.

This master thesis is conducted at Media Technology, the Department of Science and Technology at LinköpingUniversity, and carried out at Zenterio AB in Linköping. Previous works have showed that applications onsmart-phones that tries to replace a smart-phone with buttons, have the disadvantage of requiring the vi-sual attention of the viewer when using it, since no tactile feedback is given from the smart-phone on wherethe buttons are located, in contrast to a real remote control.

The thesis main focus has been interaction methods not requiring the user to look at the phone, also re-ferred to as the sender device, when controlling the receiver device, e.g. a browser, or a TV connected toa set-top-box, STB. For this, three major interaction modes where identified and implemented, and thenevaluated throughout the thesis project.

1.1 Purpose

Zenterio AB is one of the leading developers of STB UIs, and has an interest in providing the best methods ofinteraction for their UIs. The goal, as stated by Zenterio, of this thesis is twofold, firstly to evaluate differentmodes of interaction that could be considered a viable substitute for a remote controller while not requir-ing the visual attention of the viewer and secondly to provide reference implementation for future use, thatwould be compatible with set-top boxes and browsers.

Previous studies have showed that the three interaction techniques touch, gesture and voice are the mostsuccessful, when trying to incorporate it into similar applications [Bernhaupt et al. 2012]. Previous studieshave also showed that some users do prefer a combination of gestures and touch.

For this thesis, different touch-based interaction approaches performed on a smart-phone, in order to con-trol the receiving device, will be investigated, implemented and evaluated. For this an application will beimplemented and tested in order to see if a solution that does not require the visual attention of the usercould be viable, and also to investigate what this kind of application needs in order to become an efficientsubstitute for a remote control.

1

1.2 Problem analysis

The goal presented by Zenterio was deemed to general and unspecific. This is a problem both because thereis not enough guidance concerning what to investigate, but also because there are no delimitations. In thepreliminary phases of this project, the project goals were therefore analyzed and contrasted with previouswork by other researchers as well as general knowledge in the UI/UX and interaction design fields. See Chap-ter 3 for more information about this analysis.

This resulted in four concrete questions, and accompanying delimitations, that were used to focus the rest ofthe process in a more clear way towards the intended goal. These questions were presented to and approvedby Zenterio.

1.3 Problem statements

In order to achieve the purpose stated in section 1.1, the following problems have to be solved:

1. How can a touch based application implemented on a smartphone become an effective substitute

for a remote control, whilst enabling the user to use the application without looking down on the

phone screen?

2. Can other interaction techniques, such as voice and gesture facilitate the use of the application?

3. What feedback should be given to the user to facilitate the use of the application?

4. How should the graphical interface on the smartphone be designed in order to ensure that users

know how to use it whilst not attracting to much attention to itself when being used.

1.4 Delimitations

The application is built with the aim to work on multiple devices, but some limitations were imposed onboth the sender and receiver side. For the sender, only android and ios were considered, and only set-topboxes (in this case running the Netfront browser) and general purpose browsers were targeted for the re-ceiver.

Voice interaction was initially considered, but the feasibility study indicated that such applications requiredmuch more processing power than that which is available on the set-top box, and this technique was there-fore excluded from the scope of this thesis.

1.5 Disposition of the Report

The report is divided into ten chapters: Introduction, Background, Related work, Theories, Implementation,Testing, Results, Discussion, Lessons learned, Conclusion and Future Work.

2

Chapter 1 - Introduction, describes the purpose of the thesis and the problems that it aims to answer.

Chapter 2 - Background, contains information about Zenterio Services and which preconditions that weregiven for this project. Chapter 3 - Related work and Theories, presents previous work and theories that arewithin relevant fields and necessary to understand for this thesis.

Chapter 4 - Approach, describes the approach and ideas that are investigated in this thesis.

Chapter 5 - Implementation, describes the methods used in developing and testing the application.

Chapter 6 - Tests, contains a description of what tests were conducted during this thesis project.

Chapter 7 - Results, presents the results from the user tests and the resulting final application.

Chapter 8 - Discussion, provides an analysis of the methods chosen and the derived results.

Chapter 9 - Lessons learned, present reflections made during the development

Chapter 10 - Conclusion, contains a summary of the thesis and the final comments.

Chapter 11 - Future work, describes how this work could be built upon and developed further.

3

2 | Background

"JetUI is a HTML5 user interface solution consisting of a set of highly customizable applications

that enable the creation of a consistent multiscreen TV experience. Its common code base and

modular architecture minimizes development effort across devices and simplifies backend

integration."

— [Zenterio Marketing 2016b]

This project is being conducted at the company Zenterio AB in Linköping, Sweden. Zenterio AB is a com-pany that develops products related to TV and video delivery. This thesis is focused on one of Zenterio’sproducts, JetUI, which is an HTML5-based UI framework and multiscreen user interface, that allows theuser to watch TV-programs from smartphones, tablets, computers, smart TVs, or set-top boxes for TV [Zen-terio Marketing 2016b, Zenterio Marketing 2016a], see Figure 1. This product allows the user to get all TV-channels supplied from their supplier collected in one application, whilst also providing the opportunityrecord TV-programs or rent movies.

.

Figure 1: JetUI [Zenterio 2016]

Services for such devices are generally controlled using traditional remote controls, but Zenterio wants toexplore and develop a new and better modes of interaction using mobile devices, to replace the traditionalremote control.

Zenterio has previously done preliminary work on developing novel interaction techniques for the tradi-tional remote control, but now want to explore the possibility to utilize a smart-phone, and thus leveragehardware that the user already owns (TV operators are sensitive to per unit prices as it is multiplied by alarge number of users).

4

3 | Related work and Theories

In this chapter, previous work in areas related to this thesis and concepts and theories relevant to under-standing this thesis, will be presented.

3.1 Previous work

When controlling a TV and utilizing interaction techniques, different alternatives exists. For interactive tele-vision, different attempts to build altered remote controls have been performed. In 1998 MacKenzie andEnns [Enns & MacKenzie 1998], introduced a touch-based remote control that besides the original function-ality of a remote control has the possibility to control with touch and gesture-based interaction. [Bernhauptet al. 2012]. More recently, some companies have also produced remote controls that also exhibit differentinteraction techniques besides the original functionalities. One example of this is the Apple TV Remote, thatutilizes SIRI for voice interaction and a glass surface for touch interaction [Apple 2016].

In a media study performed by Bernhaupt et al. in 2012 , the ethnographic orientated method playful prob-ing was utilized in order to establish recommendations on how to design a smart phone application thatenables the user to control IPTV-systems [Bernhaupt et al. 2012]. The media study focused on the threeinteraction techniques touch, gesture, and voice. This research illustrated that the interaction techniquespreferred by users were a hybrid version, that exhibits flexible touch interaction and a limited set of speechcommands and point gestures to detect the device it should control. In this research, from the user’s per-spective, the ideal smart phone application should provide a simplified set of touch-enabled buttons thatcan be used to control and navigate the TV UI.

Since TVs are normally controlled by IR signals, controlling it directly with a smart-phone is problematicsince most phones are not IR enabled. Also most IR-based protocols have a rather low bandwidth andthereby limits the rates of transmitted information. To make matters even worse, there must be a directline of sight between the transmitter and receiver, in order for proper transmission. This can become prob-lematic if gestures are to be performed since the user will be required to turn the remote control away fromthe receiver. The low bandwidth can also become problematic with some interaction patterns, especially ifone wants to send large amount of data, e.g. continually sending coordinates to control the cursor.

The article by Bernhaupt et al. does in addition present two disadvantages that this sort of application canhave. The first being that an application that controls the TV only becomes another remote control and thatthe users would have liked to have an application that can control all other devices that one can have in aliving room, e.g. DVD-player. Solutions for this problem exists, the simplest being a universal remote andmore recently the Anymote IR-blaster. The second disadvantage presented by Bernhaupts et al. being thata touch-based application with buttons cannot be used ’blindly’, meaning the user have to look down onthe phone-screen in order to perform tasks, which creates a usability problem that is non-existent on a realremote control that gives tactile feedback when touching its buttons.

One solution to this problem is to have an exact copy of the TV-screen on the smart-phone that way the

5

user can switch his/her attention directly to the screen and not lose any information. A version of this isimplemented by Youtube, where you can fully interact with the Youtube services on the phone while thecontent shows on the TV-screen.

For this thesis project interaction techniques based on touch and gesture for a smartphone, that doesn’thave ir-sensors, will be explored and evaluated. The main goal is to create a solution that would work onmultiple platforms for both the receiver and sender side, this is achieved by basing the implementation onHTML5 and JavaScript, which can easily be run on any device with a browser.

3.2 Set-top box

A set-top box, STB, also known as set-top unit, STU, or cable box, that traditionally is used in cable andsatellite systems and more recently in Internet television [Dhir 2004][Kangovi 2016]. The STB is a sort ofcomputer that translates digital signals or Internet data into a format that can be displayed on TV screensor other display devices, see Figure 2.

User

STB

TV

.

Figure 2: Illustrates how the STB is connected to a TV set and how the user generally interacts with it using

two remote controls: one controlling the TV and the other for controlling the STB.

STBs can be divided into several categories, ranging from simple boxes that receive and translates incom-ing audio and video signals, to more complex units such as Internet Television STB or units that deliversmultiple services such as video teleconferencing, home networking, IP telephony, Video-on-demand andInternet services. Internet Television STB, also referred to as over-the-top OTT STB, is a more recent variantof STBs that gets its content directly from the Internet, e.g. Apple TV [Percival 2013].

3.3 User

During development and evaluation of a product, the most important people are the users. The users, andthe knowledge of who they are, are crucial when developing a product, since only when the user is knowndoes shaping the user interface based on what that user needs in order to achieve a task and reach thedesired goal become possible. [Lazar 2005].

Besides knowing who the users are, the goals of the users are also important [Dumas & Redish 1993]. Whenthe goals are known, the amount of time spent on reaching the goal will decide if the product is easy to useor learn, and after a while if its perceived to be an efficient usable system.

6

3.4 Usability

"Usability is a quality attribute that assesses how easy user interfaces are to use."

— [Nielsen, J. 2012]

When creating a web application, its usability is of great importance, since it is a necessary condition forsurvival [Nielsen, J. 2012]. Users will not utilize an application that is too difficult to use, hard to properlynavigate or that poorly presents information.

3.4.1 ISO

There are many ways of defining what usability is, and one way is by using ISO’s, International Organisa-

tion for Standardisation, description. ISO, describes that the objective for designing and evaluating usableproducts is to enable the user to achieve goals and meet the needs of the user, [ISO9241-11 1998]. Further,ISO states that usability can be defined in terms of user performance and satisfaction. These factors can bemeasured by the resources that are consumed in order to achieve the intended goals (using the system), ifthe goals are achieved, and to which extent the user finds the use of the product acceptable [Forsell, C. &Cooper, M.D. 2010, ISO9241-11 1998].

3.4.2 REAL

Another definition of what defines usability is REAL, which stands for relevance, efficiency, attitude andlearnability [Löwgren 1993]. To determine if the usability in a system is satisfying, usability tests or usabil-ity evaluations are conducted while the system is developed in order to have opportunities to continuallyimprove the system.

3.4.3 LEMES

A third definition of usability that is similar to REAL is presented by Nielsen, [Nielsen, J. 2012]. Nielsen statesthat the five quality components that define usability are: Learnability, Efficiency, Memorability, Errors andSatisfaction. Learnability represents how easily the user can complete basic tasks when first being intro-duced to the application. The second component, efficiency, denotes how quickly the users can performtasks after the design has become familiar. Memorability is the component that represents how easily theuser can reestablish proficiency after being reintroduced to the system after a period of time. The fourthcomponent, errors, denote the quantity of errors the users make whilst using the application, the severenessof them and how effortlessly the users can recover from them. The last component, satisfaction, representshow pleasant the design of the application is.

Besides these components, Nielsen mentions that other important quality attributes exists and an essentialone is utiliy. Utility refers to the functionality of the design and how well it does what the user desires.In order to determine whether an application is useful or not is according to Nielsen determined by both

7

usability and utility. To study if a system is usable, many different methods exists, and the most commonone is performing user testing.

There are many ways of achieving, testing for, and even defining these properties, in this project the focushas been on two of them, user testing, with usability and utility in mind, and user design.

3.5 User testing

To determine if the usability in a system is useful, user testing should be conducted while the system isdeveloped in order to in order have opportunities to continually improve the system [Nielsen, J. 2012, Löw-gren 1993]. By performing these test on potential users, feedback of the product design and the UI can beevaluated and used for future improvements.

User testing is, according to Nielsen, performed in 3 parts [Nielsen, J. 2012]. The first being to get repre-sentative users. These representative users can e.g. be customers that shall use the system that is beingdeveloped. The second component is to ask the participants to perform representative assignments withthe design. These assignments, also known as tasks, can be constructed by determining which goals theusers may have for the intended system, and what the user would have to do to accomplish them. The lastpart is to observe the user perform the tasks without providing any help, since it may contaminate the testresults. In this part, the observer can see where the user succeeds and what difficulties exist.

When performing test interviews, questionnaires can be used as a complement to observing the partici-pants activities [Benyon et al. 2005]. Questionnaires can be used to gather the reactions of the participantsand background data about the user characteristics. When creating questionnaires items, it is importantto be as specific as possible. Bipolar ratings are recommended to use, in order to prompt the respondentto balance two opposing attributes and determine the relative proportion of these attributes. Constructingquestion sentences like: The screen layout was rather than The screen layout was easy to use (followed byanswer alternatives) can be beneficial, since they avoid prompting users to answer with only one specificattribute. These questions may take more time to construct, but are helpful when probing for specific reac-tions. To get more insight into the answers of the questionnaire questions, short interviews can be used. Byperforming interviews, the user reactions can be clarified and more feedback can be collected.

In order to make sure that each participant in a user test gets the same information, and are treated equally,written instructions can be constructed and given to the users to read before the test. A written protocolon what the test instructor should say to the participants during the test can be also be used [Forsell, C. &Cooper, M.D. 2010]. By producing these documents, more test instructors can be used, if needed, in orderto handle more participants, without introducing a bias to the tests.

When performing user tests, 5 users is typically enough to identify the most important usability problems,[Benyon et al. 2005, Nielsen, J. 2012]. However, some practitioners and researchers say it may be to few ifthere is a large number of representative groups of users. In that case, 5 users per group should be recom-mended.

8

Evaluate and improve Web site

Collect user requirements

Implement and marketWeb site

Stage 5

Stage 4

Stage 3

Stage 2Stage 1

Stage 6Stage 7

Define the missionand target user population

Create and modify physical design

Perform usability testing

Life cycle starts again when Web site redesign occurs

Create and modify

conceptual design

Design loop continues until Web site is ready to go online

Life cycle starts here

Iterative Deisgn

.

Figure 3: User centered evaluation. Figure adapted from [Saffer, D. 2009].

3.6 User-centered Design

User-centered design, UCD, is a design approach in which the system is designed to fit the users needs andgoals. During the development of a system, of any kind, this process can be utilized. The UCD developmentlife can be viewed in Figure 3.

To ensure that a design is user centered, the users, tasks and environment needs to be known and the designshould be based on this knowledge, since it should address the whole user experience. With this approach,the users should be involved in designing and developing the product. When the user requirements aredefined, an iterative design phase begins in which the design is refined, modified and tested on the users.These iterations are then performed until the product is released on the market.

3.7 System architecture

"A system architecture or systems architecture is the conceptual model that defines the

structure, behavior, and more views of a system."

— [Jaakkola, H. & Thalheim, B. 2011]

9

When constructing a system, the system architecture shall work as a blueprint for how the system shall beconstructed. When working in a team, the system architecture is there to work as a common plan that allowseach member to build their piece and know that it fits in the bigger system. The system architecture usuallydefines the modules the system shall contain and how the communication between them shall work.

When a system architecture is to be developed, design patterns can be utilized to help the process. Designpatterns can be traced back to early works of the architect Christopher Alexander, who created reusablearchitectural solutions for use with buildings and towns, and named them design patterns [Osmani 2012].A pattern in software development is a reusable solution that can be used to solve commonly occurringproblems in software design.

Patterns have three main benefits; patterns are proven solutions, patterns are easily reused and patterns can

be expressive [Osmani 2012]. The first benefit states that patterns provides well tested approaches usingproven techniques, that mirrors the experience and insights of previous developers that assisted in definingthe pattern, in order to solve issues in software development. The second benefit expresses that patternsgenerally reflects an out of the box solution, meaning that the developers do not have to redo all the designwork, instead they can derive their solution from something that worked before. The last benefit states thatwhen components of a system can be expressed with clarity and intuitiveness it increases the understandingfor their use and purpose in the system. Simply put, sometimes using a familiar name aids understandingconsiderably.

3.8 MV* patterns

While there are many widespread architectural design patterns, there is one group that has a singular im-pact in the work presented in this thesis. This group is called the MV* pattern, which is a generic name forseveral patterns built upon the same idea, with its origin from Model-View-Controller (MVC), Model-View-Presenter (MVP) and Model-View-ViewModel (MVVM). The common factor for these patterns is that theyall have a Model and a View, and something else (*) to connect everything together. Details vary accordingto specific problems in various programming languages and libraries.

3.8.1 MVC

The design pattern Model-View-Controller consists of three subsystems; Model, View and Controller. TheModel is responsible for managing all data within the MVC-system while the View draws the componentsthe system consist of. It is with the View that the user interacts with, and when actions are performed orchanges are made in the View or to the data in the Model, the Controller communicates these changesbetween the other subsystems.

The Gang of Four, in the book Design Patterns [Gamma et. al. 1994], however, describes that the MVC in-stead of being a design pattern itself rather is a variation of other design patterns. The authors describe thatthe main relationships for MVC is given by the three classical design patterns: the Observer, Strategy andComposite patterns and can in larger application also consist of Factory Method and Decorators. [Osmani2012]. This could possibly explain why there are many similar patterns used to solve this class of problem.

10

4 | Approach

In this chapter the strategy chosen for the thesis project will firstly be described, followed by a descriptionof the interaction techniques researched in this thesis project. Lastly the system architecture chosen for thisthesis project will be presented.

4.1 Strategy

The first step to solving the problems was a clear strategy for arriving at a solution. Previous work showedproblems, where controlling the receiving device required the user to look down on the screen. In order toavoid that, a minimalistic design on the smart-phone could be a solution, in which a non-graphical interac-tion area would be displayed on the phone and all feedback would come from the receiving device (TV).

Based on this realization a number of different ideas and possibilities were explored. Some of the generatedcandidate solutions were touch, gesture and voice interaction. A brief feasibility study showed that no viablevoice interaction framework could be considered for the platforms in question (i.e. STB), at the time, withoutsignificant integration work.

In order to evaluate if a touch-based application or a hybrid version could become an efficient substitute fora remote controller, user tests were performed in which the user could compare the usability of a traditionalremote control to the interaction modes implemented for the application. The user tests would also be away to evaluate which feedback that should be given to the user to facilitate the use of the application.

4.2 Interaction techniques

As discussed in Chapter 3.1 the use of touch based applications have been successful, however most imple-mentations have been limited by the fact that the user can’t use it freely without having to look down on thephone screen [Bernhaupt et al. 2012].

In an attempt to improve on these results, this project aimed to find and implement modes of interactionthat does not require seeing the phone’s display. Based on this, three modes were implemented. The firsttwo utilizing different touch movements and the latter a combination of touch and orientation, and theywill be referred to as Methods A-C.

A Method D was also considered, in which a combination of Method A and B were to be implemented,however, due to time constraints this one was omitted.

4.2.1 Method A

Traditionally, most remote controls have been using arrow buttons to navigate the UI. For the first method,the theory of replacing these actions with different touch event was constructed. In this method, named

11

Method A, swipe and tapping would be used to replace the arrow keys <

∧

∨ > and the OK button on theremote control, respectively.

Using this approach in the application would enable the user to change which element that should be high-lighted by swiping, and tap to select an item, see Figure 4

a b(a) Receiving device

(b) Tap (c) Swipe

Figure 4: Interaction with the receiving device (a) is performed by tapping (b) and swiping (c) on the sending

device, in order to select and highlight items respectively.

This concept is illustrated in Figure 4, where the dark blue rectangle represents a highlighted item. If theuser swipes to the right, the selection shifts to the currently selected item’s immediate right, and that itemwould be highlighted instead. If the user taps on the phone, the dark blue rectangle would be selected anda new page loaded.

4.2.2 Method B

When introducing a touch surface to replace the remote control, some new opportunities arise. Therefore, adifferent class of interaction, that operate by means of a cursor more similar to a computer touchpad, havealso been considered, and the UI logic expanded to handle such interaction. Since the set-top box browserdoesn’t provide a cursor by default, one was created manually for methods B and C. For this cursor basedmethod, the touch movements dragging and tapping were utilized, creating an experience very similar to atouch-pad, see Figure 5.

In Figure 5, the pink circle represents the cursor displayed on the receiving device, and dragging the onefinger in any direction on the phone would move the cursor in the same direction. To implement this, thedistance moved on the phone is normalized and sent to the other screen. The normalization process is donein three steps. Step one handles the irregular shape of the screen and selects the shortest dimension of the

12

(a) Receiving device.

(b) Tap to select.(c) Dragging fingers in any direc-

tion.(d) Two fingers to scroll.

Figure 5: Interaction with the receiving device (a) is performed by tapping (b) and dragging (c), in order to

select items and move the cursor, respectively. Dragging two fingers (d) or placing the cursor at the bottom of

the page, will trigger a scrolling of the page.

phone screen, s fsend , and the largest dimension on the receiving device, s fr ecei ve , see Equation 1. This isdone to ensure that moving along the entire short dimension on the sending device would be equivalents tothe longest dimension on the receiving device. The second step normalizes both the moved x and y distance(di stx and di sty ) by dividing with the sending factor in the previous step (s fsend ), see Equation 2. Thenormalized coordinates from the sender device (d x and d y) are then sent to the receiving device, wherethey are de-normalized using the target device scaling factor (s fr ecei ve ), from step 1, see Equation 3.

s fsend = mi n(wsend ,hsend )

s fr ecei ve = max(wr ecei ve ,hr ecei ve ) (1)

d x = di stx /s fsend

d y = di sty /s fsend (2)

x = xcur sor +d x ∗ s fr ecei ve

y = ycur sor +d y ∗ s fr ecei ve (3)

13

For this method the possibility to enable the user to scroll on the screen was implemented, and can beperformed by dragging two fingers up or down on the phone screen, similar to how scrolling on a laptopcan be performed when using the trackpad. However, if the user holds the phone in the hand similar towhen holding a remote control, the two fingers would mean changing the position of the hand. In order tothen give the user another opportunity to scroll, without having to move the hand, a decision was made toenable the receiving screen to scroll if the position of the cursor is at the top or bottom of the page.

4.2.3 Method C

When implementing gestures with the smartphone, the gyroscope and accelerator can be utilized in orderto retrieve information about the orientation, motion and acceleration of the device.

As a combination of gesture and touch-based interaction, the orientation information was used as wellas taps on the screen. By using the same setup as Method B, in which the user controls a cursor on thereceiving device, but changing the input to the cursor control to be the act of pointing the phone in differentdirections.

The orientation values contains alpha, beta and gamma values, representing rotation around the z-axis, x-axis and y-axis, respectively, see Figure 6. These values can then be used and transformed into the cursorposition on the receiving device.

y

x

z

Figure 6: Interaction with the receiving device is performed by tapping and gestures in order to select items

and move the cursor, respectively.

4.3 System architecture

The single most important thing that Backbone can help you with is keeping your business

logic separate from your user interface. When the two are entangled, change is hard; when logic

doesn’t depend on UI, your interface becomes easier to work with.

— [Backbone 2016]

When creating a web application consisting mainly of Javascript, just like any other language, the structureof the code is of great importance [Osmani 2012]. The MV* patterns have in the past been used a lot but

14

only in the recent years been applied to JavaScript in different versions. One such use is seen in Backbone.js,a front-end library and architectural framework developed by Jeremy Ashkenas, [Osmani 2012, Backbone2016], that can be included in a web-application. Backbone.js is used throughout this project. Backbonedid once develop its own Controller, aiming for the MVC paradigm, however since its functionality weren’tused in the way as a controller should, it was later renamed to Router. The router does handle a part of thecontroller’s responsibility and events for the model can be bound within the router as well as views triggeredto respond to DOM events and rendering. Addy Osmani describes, [Osmani 2012], that Backbone is neithera truly MVC, MVP nor MVVM framework, and that it its better to consider it as a member of the MV* family,and thereby meaning that it approaches the architecture on its own way.

For this thesis project, the decision to utilize a MV* framework was an easy one. Specifically, Backbone.jswas selected as the base of the application architecture, due to the fact that JetUI utilizes this framework. Anintegration of this project into JetUI could then more easily be achieved.

15

5 | Implementation

This chapter presents all the work and implementation throughout this thesis project. Firstly the tools andtechnologies used for the implementation will be mentioned. Secondly the graphical design decisions and,prototypes made for the application will be described, followed by presenting the implementation of thedifferent interaction techniques. Lastly the implemented system architecture will be described as well as anoverview of the development process, in order to give a complete view of the implementation.

5.1 Tools and Technologies

The programming languages for this thesis project were CSS3, HTML5 and JavaScript, so that it in the fu-ture could be incorporated into JetUI and be further developed and extended. In this section, the differentframeworks, libraries and other technologies used in this thesis project are presented.

No external library was deemed necessary for detecting touch events, instead the standard JavaScript TouchAPI is used, as it is supported as is on all major mobile platforms [Mozilla Developer Network 2016].

5.1.1 Apache Cordova

In order to develop an application that would be able to run on different mobile devices and browsers theframework Apache Cordova has been used. Apache Cordova, formerly known as PhoneGap, is an opensource mobile application development framework that allows software programmers to develop mobileapplications using CSS3, HTML5, and JavaScript. When developing an application using Apache Cordova,native plug-ins can be used to extend its functionality. These plug-ins can for instance allow access to adevice’s accelerometer, camera or microphone. Plug-ins used in this project are:

• cordova-plugin-device - Allows the application to retrieve which device is used, and which platformand model it has, e.g. platform: browser, model: Chrome or platform: iOS, model: iPhone7.2.

• cordova-plugin-statusbar - Enables customization of the status bar in iOS and Android.

5.1.2 Backbone

As mentioned in section 4.3, the front-end library Backbone.js was used in order to create a structure forthe application. The only hard dependencies of this framework are Underscore.js and jQuery, and thereforethese libraries have been included in the project.

16

5.1.3 Mayo

To create a connection between the phone and the other device, the library Mayo was utilized, which isa internal library by Zenterio Services. The library is based on node.js and works similar to socket.io. Thelibraries API mimics socket.io but is specifically developed to work well on set-top boxes.

5.1.4 Git

Git was used as the version control system, to track changes made during development and simultaneouslykeep all versions saved. That way any changes are tracked individually, and can be rolled back individuallyif needed.

5.1.5 MATLAB

MATLAB was used to conduct a one-way ANOVA (Analysis of variance) and a post hoc test on the resultsfrom the user tests, in order to determine if any of the interaction methods are significantly different fromanother. For this, the methods anova1(...) and multcompare(...) were utilized. For the post-hoctests the Tukey-Kramer method was employed.

5.2 Graphical design decisions

The application is designed to use the same colors as the Zenterio logo see Figure 7, and the product JetUI.The three colors purple, red and pink, see Figure 8 are retrieved from the Zenterio logo, whilst the colors

Figure 7: Zenterio logo, Figure retrieved from [Zenterio Marketing 2016a]

black and white are from the background colors in JetUI. In order to create a similar interface as JetUI,the color black is to be used as the background color, whilst the other colors are intended to be used insmall sections of the application, e.g. font colors for menu items or displaying colors when objects are beingselected.

The icons used for the back, play and pause buttons, see Figure 9, use the font Entypo by Daniel Bruce, andcan be retrieved from the website Fontello, [Fontello 2017]. This font is chosen since it is the font currentlyused in the JetUI interface.

17

RGB 188-20-28CMYK 0-89-85-26#BC141C

RGB 08-29-127CMYK 23-69-0-50#62277F

RGB 196-0-122CMYK 0-100-38-23#C4007A

RGB 0-0-0CMYK 0-0-0-100#000000

RGB 255-255-255CMYK 0-0-0-0#FFFFFF

Figure 8: Figure illustrating colors taken from the Zenterio logo and from the JetUI interface.

Figure 9: Figure illustrating the icons used for the pause, play and back buttons.

5.3 Prototypes

After the initial functionality for the touch and orientation had been developed, a formal product descrip-tion was produced by Zenterio, see Appendix B, and a prototype of the product was created, see Figure 10.

21 3 4 65 7

BA C D FE G

III III IV VIV VII

ZENSEZENSE

Figure 10: Figure illustrating the initial prototype.

After this prototype was produced, the decision to implement it not as part of JetUI was made, but insteadas a standalone proof of concept. A further refined prototype was created in order for the application to getsimilar qualities as the JetUI interface. This prototype consists of a main menu and a video player on thereceiver device, as well as a main menu and an interaction component on the phone, see Figure 11.

In the second prototype for the receiving device, the colored rectangles from the previous prototype arereplaced with proper cover images for different movies and the rows of items represent different movie

18

ZENSE ZENSE

Phone main menu

Swipe

Cursor

Orientation

Interaction component

Movie genre

Movie genre

Movie genre

Movie title

ZENSE

Browser main menu

Video player

Figure 11: Second prototype. Movie poster images from [Blender Foundation 2008]

.

genres.

The video player view is where the user can watch the selected movie. Two buttons are included in this viewin order for the user to perform the basic interactions play/pause and returning to the main menu.

The main menu of the phone presents three buttons with which the user can select the interaction tech-nique that shall be used. When an interaction technique is selected, the interaction component is displayedto the user. A small return button is displayed at the top left corner in order to enable to user to go back tothe main menu.

5.4 Implementation of interaction techniques

The implementation of the different interaction methods called Method A-C, previously mentioned in Chap-ter 4.2, is described in this section.

5.4.1 Method A

For Method A, the touch events: swipe and tap, were utilized in order to move and select the highlighteditem, respectively.

To implement this method a grid was created for each view presented to the user on the receiving screen,

19

containing all items that can be selected. When a swipe movement is detected on the phone, the directionis sent to the application. The receiver checks which item is currently highlighted in the grid, and adds/sub-tracts the row/column as appropriate to determine what item should be highlighted next.

5.4.2 Method B

For Method B, the touch events: tap and dragging, were used to select items and move the cursor on thereceiving screen, respectively.

To implement this method, all items that the user shall be able to interact with and their position in thereceiving view have to be known. By storing the minimum and maximum x- and y-coordinates (i.e. thebounding rectangle) for all items in a map, the current position of the cursor can be compared with these inorder to determine whether or not an item shall be highlighted or not.

Similar to the Method A, tapping on the phone would select an item that is highlighted. In order to triggerclick events when the user taps on the phone, the application checks whether an element is being hovered,and if it is, it then activates it.

5.4.3 Method C

For Method C, the touch event tap was used to select items and the orientation information was extractedfrom the phone in order to use gestures as a way of moving the cursor on the receiving screen.

For this approach, the α and β angles were extracted in order to determine a 2D-position of the mousepointer on the other screen. This position where calculated using Equation 4.

(

x ′

y ′

)

=

(

1.0−tanα2.0 0

01.0−tanβ

2.0

)

(

x

y

)

(4)

Here (x, y) and (x ′, y ′) represent the current position and the new position. In order to restrict the angles thatshould be sent to the other screen, the angles α and β are capped around 45° centered around the middle ofthe screen, (0°), see Figure 12. This ensures that the largest angle that the phone needs to be turned in orderto reach any point with the cursor doesn’t exceed 45°.

During the implementation of orientation, it was noted that every small movement made by just holding thephone in the hand was registered and sent to the other receiving screen. This resulted in shaky cursor move-ment, possibly due to inexactness or jitter in the sensor data. In order to address this problem, a thresholdvalue was introduced that only allows bigger changes to be sent to the other screen.

With this method, the functionality to select and highlight items is the same as in Method C, meaning thathighlighting is done by comparing the cursor position with item positions stored in a map, and tappingchecks if an item is hovered and chooses it.

20

360°0°

(a) Illustration of α

0°180°-180°

90°

-90°

(b) Illustration of β

Figure 12: The limited intervals for the angles α and β

.

5.5 Application overview

The structure for the application created for this thesis, heavily influenced by Backbone, can be seen in Fig-ure 13 and the full name for the different views and models can be found in Table 1. In Figure 14, screenshotsfrom the receiving device and the phone application are displayed in order to further illustrate where theviews from Table 1 are located in the real application.

The Server Connection Model, SCM, is responsible for establishing a connection between the phone and thereceiving device, by utilizing the external library Mayo. After the connection is established, the TV awaitsinstructions from the phone on which interaction technique that is active and which changes that shallbe made in the receiving device. The router is responsible for switching between the different views thatshall be displayed on the different devices. When changes have been made to any of the models, events aretriggered and the views are updated.

21

SCM

SCM

TSVTCVOV

PMMVCV

VPV

BMMV

BHVMLV1

...

BMMM

Data

MPV MPV...

Views

Model

ViewsModels

SenderReceiver

Router Router

MPV MPV...

Figure 13: Figure illustrating the different models and views created for the application.

Table 1: Tables displaying the full names for the different models and views, displayed in Figure 13, for the a).

Phone and b). Receiver.

a). Phone

SCM Server Connection Model

PMMV Phone Main Menu View

TSV Touch Swipe View

TCV Touch Cursor View

OV Orientation View

b). Browser

SCM Server Connection Model

BMMM Browser Main Menu Model

VPV Video Player View

CV Cursor View

BMMV Browser Main Menu View

BHV Browser Header View

MLV* Movie List View (1,2,3 ...)

MPV Movie Poster View

5.6 Application development process

To summarize, the overall application development process for this thesis project can be viewed in Figure15. The first part was a pre-study, in which necessary information was collected and the thesis project de-scription was formed. Following this was producing Prototype 1, and the UCD process, in which an iterativeprocedure of producing and testing was performed, resulting in the first version of the application. Afterthis, a new prototype was derived and its UCD process started, resulting in version 2, that was then used inthe user test performed on participants not working at Zenterio. Lastly a final version was derived, in whichsome minor adjustments where performed.

22

PMMV

TSV,TCV,OV

(a) Views for the phone.

BHV

MLV* MPV

BMMV

VPV

CV

(b) Views for the receiver device. In the top image CV is not activated.

Figure 14: The different views for the browser and the phone.

23

Pre-study

Iteratively improve the product through input from users.

Inital prototype

Version 1Derived prototype

Version 2

UCD

User testing

Version 3

Iteratively improve the product through input from users.

Final adjustments

UCD

Figure 15: The overall application development process.

24

6 | Tests

During development, the application was tested by readily available users with varying degrees of expertisein software development and set-top box UIs. The UI development method was based on user centereddesign, in which users tested and gave input continuously, in order to iteratively improve the product.

In order to also retrieve input from users without expertise in software engineering, a user test was createdand conducted to estimate how well the user could use the different interaction techniques without lookingdown on the smartphone when controlling the other screen. Another goal of the user test was to evaluatewhether these interaction techniques would be a suitable substitution for a remote control.

When searching for participants, out of caution a few more users than the five recommended, mentionedin Chapter 3.5, were used in the user tests performed.

6.1 User test

In order to determine the systems usability and utility a test was conducted. In this test, an interview with aspecific set of questions was performed, and the time to complete tasks was measured and errors encoun-tered were registered. In doing so, the components Learnability, Efficiency, Errors and Satisfaction wereexamined for determining the usability, leaving out Memorability as this would require repeatedly testingover time.

The test was performed with participants one at a time and each test took between 50 and 70 minutes. Allparticipants completed all tasks of the test.

The tests started with giving the participants an introduction about project and then the test itself was givento the user, see Appendix C.1. After the user had read the introduction, background information about theuser was collected, in which the user had to answer a few questions, see Appendix C.2.

The test was then divided into four parts, in which the user tried to control the application using the differentinteraction techniques: Reference (remote control), Method A (touch with swipe and tap), Method B (touchwith a cursor) and Method C (orientation with tap to choose).

Before performing the tasks, for each part of the test, an introduction was given to the user on the differentdevices whilst the application was displayed on the TV. For the remote control, the user was told whichbuttons that would be used, see Figure 16. For Method A and B, the user was told which area to interactwith, which is furthered described in Chapter 6.2.1. The users were also told that swipe and tap were tobe used for Method A, and that Method B should be used as a touch-pad. For Method C the participantswere told that the cursor movement from Method B had been replaced with moving the phone, and thatselecting an item worked as in Method B. The users were also told that if they felt that the calibration of thestart position had gone wrong, they could recalibrate it by pressing the button at the top-left.

For the user test, the ability to scroll utilizing two fingers for Method B was not mentioned to the users. Thepurpose of this was to see whether the user would notice the two finger scroll option but also to see whether

25

Figure 16: Remote control used in the user tests, with the buttons used lying within the red area.

the automatic scrolling, when the cursor is at the top or bottom of the page was sufficient.

6.2 The tasks

After the instructions had been given, the tasks to perform for each of the interaction methods consisted ofthe following steps:

1. Go to a movie item.

2. Select the movie item.

3. Start playing the movie.

4. Pause the movie.

5. Go back to the start menu.

The participant was told before performing the tasks that the time required to complete each task wouldbe measured. The users were also told that the test manager would not be able to answer any questions,in order to not influence the participant, and would only interfere if problems with the application itselfappeared. For each interaction technique the test manager also observed how many errors that occurredwhen the user performed each task and whether the user looked down on the sending device.

After the user had tested the interaction technique, an interview with the user was performed in whichquestions were asked and notes were taken down by the test manager, see Appendices C.3, C.4, C.5 and C.6.A semantic scale were used for some of the questions, in which the user had to describe on a scale 1-5, howhard or easy, different tasks were to perform.

After these four parts of the test were performed, a few final questions were asked of the user, see AppendixC.7. The users where firstly asked to grade the different interaction techniques compared to each other ona scale 1-5, where 1 and 5 represented hard and easy respectively. During this part, the user was encour-aged to express what made the the different interaction techniques hard or easy. The user was then finallyasked whether the user would consider using any of the different interaction techniques instead of a remotecontrol.

26

6.2.1 Application used in the user tests

The material used for the test was a STB, a remote control, a screen to view the application on, and aniPhone6s.

When the tests were performed, the prototype for the smart-phone displayed previously in Figure 11, wasnot yet completed. The application used for the user test can be viewed in Figure 17a for the phone andFigure 17b for the receiving device.

(a) Smart-phone, consisting of three red buttons and a grey

touch area, registering touch events.(b) Receiving device.

Figure 17: Figure illustrating the UI for the devices during the user tests.

The participants were instructed to ignore the red buttons displayed on the top of the smart-phone screen,since they were only there for the test instructor to switch between the different interaction techniques thatwere to be utilized. The participants were also informed that the touch area, displayed on the phone, isintended to be displayed on the majority of the phone screen.

After the user tests were performed, the results were analyzed, and resulted in some final changes to theapplication, particularly on the phone. Those changes brought the UI to the final version that is presentedin the following chapter.

27

7 | Results

In this chapter the results from the user tests that were performed are presented and the final product isdisplayed.

7.1 The users

Analysis of user responses showed that the participants were aged between 23 and 34 years and the medianage was 26. The test was performed on nine participants, and all were owners of a smart-phone. 88.9% ofthe participants answered that they watch streaming television online and 11.1 % answered that they don’tusually watch any television services at all. The television habits for the remaining 88.9% of participantswere graded into three categories based on watching frequency: 37.5% daily, 50% a few times a week, and12.5% a few times a month.

7.2 User tests

For the user tests, the users were instructed to complete a few tasks, see Chapter 6.2, and the time it tookto complete them was measured. An interview was conducted, see Appendix C, in which the test instructorwrote down feedback from the participants.

After completing each task for each method, the user was asked whether he/she felt they had looked downon the sender device, whilst performing the tasks. The results were as follows, 11% for the remote controland Method A, 0% for Method B, and 22% for Method C. These results corresponded with the observationsmade by the test instructor on which participants that looked down on the screen. The one-way ANOVAand post-hoc test, conducted on the results, see Appendix A.1.1, showed that none of the methods hadsignificantly different means than the others, with a p-value of 0.5525.

7.2.1 Usability

The results for measuring the Learnability, Efficiency, Errors and Satisfaction, in order to determine usability,are presented here.

7.2.1.1 Learnability

To determine the learnability, the time to complete the first task was measured. In table 2, the average timefor all participants to complete the first task for each method is displayed.

28

Table 2: Average time for completing the first task for each methods.

Remote Control [s] Method A [s] Method B [s] Method C [s]

9.9 8.4 8.1 8.8

The results show that the average time for the three methods A, B and C was faster than for the remotecontrol. Method was A 1.5 seconds faster than the remote, Method B 1.8 seconds, and Method C 1.1 seconds.(On an average 1.5 seconds faster). The one-way ANOVA and post-hoc test, conducted on the results, seeAppendix A.1.2, showed that none of the methods had significantly different means than the others, with ap-value of 0.7324.

7.2.1.2 Efficiency

To evaluate the efficiency, the average time for the completed tasks, not including the first task, are displayedFigure 18, and the exact values can be found in Appendix A, Table A1. In the figure, the average time for allparticipants to complete the tasks for each method are displayed as lines, whilst the average time for eachuser is displayed as columns. The lines and columns are colored after which method used.

00,51

1,52

2,53

3,54

4,55

5,56

6,57

7,58

8,59

9,510

10,511

11,512

12,513

1 2 3 4 5 6 7 8 9

Tim

e[s]

User

ChartTitle

RemoteControl MethodA MethodB MethodC

AverageRemoteControl AverageMethodA AverageMethodB AverageMethodC

Figure 18: Average time for completing the tasks using the different interaction techniques.

29

The results show that the average time of utilizing the remote control, Method A and Method B are quitesimilar, where Method A where approximately 0.1s faster. The results show that on an average utilizingMethod C was approximately 1 second slower than the other methods. The one-way ANOVA and post-hoctest, conducted on the results, see Appendix A.1.3, showed that none of the methods had significantly dif-ferent means than the others, with a p-value of 0.3702.

7.2.1.3 Errors

When completing the task, the number of times the user performed a incorrect action (performed anotheraction than the intended, i.e. clicked the wrong button) was measured. The number of errors performed foreach method is displayed in Table 3.

Table 3: Errors performed by the participants for each method

Remote Control Method A Method B Method C

1 11 1 3

The one-way ANOVA test showed that significance exists, with a p-value of 0.0371. The Tukey-Kramer post-hoc test conducted thereafter found no significant difference between the groups, see Appendix A.1.4. Sincethe ANOVA showed that there was in fact a difference, another post-hoc test was performed, utilizing Fisher’sLSD (least significant difference) method, which showed that Method A has means significantly differentfrom the other interaction methods, see Appendix A.1.5.

The participants described for Method A that sometimes when they tried to swipe, a tap was registeredinstead. One of the participants mentioned that this error occurred when trying to swipe the opposite di-rection to which hand that was used, meaning that if the right hand was used swiping left resulted easierwith errors, and the opposite.

Some participants mentioned that in the video-view, when playing/pausing the feedback given from the TVwas sometimes not direct, and an extra action was performed due to the delay, and some sort of feedbackfrom the phone was expressed. Some participants mentioned that vibrations or sounds from the phonewould be a good feature that could help prevent extra actions being performed, if the feedback from the TVhas been delayed.

The results show that for the remote control, Method B and Method C generated the least amount of errors,and Method A the most.

7.2.1.4 Satisfaction

When asked how hard/easy it was to see which item that was highlighted, the participants answered:

30

Table 4: How hard/easy the participants perceived it to see which item that was highlighted.

1 (Hard) 2 3 4 5 (Easy)

Remote Control 0% 11.1% 33.3% 11.1% 44.4%Method A 0% 0% 44.4% 11.1% 44.4%Method B 0% 0% 11.1% 33.3% 55.6%Method C 0% 0% 11.1% 33.3% 55.6%

The average grade, for how easy/hard the participants perceived it to see which item that was highlighted,for the remote control and Method A-C was calculated to 3.8, 4.0, 4.4 and 4.4, respectively. The one-wayANOVA and post-hoc test, conducted, see Appendix A.1.6, showed that none of the methods had signifi-cantly different means than the others, with a p-value of 0.4521.

For all the methods, participants mentioned that it wasn’t at first clear which item that was selected, and thatafter the user had started to interact it became clearer. Some participants described that the the glow aroundthe highlighted was to small and could be sharper. Examples mentioned, on how this can improved were:using a bright color instead of white, enhancing the spread of the glow, using contrasting colored bordersaround the movie items.

Some participants mentioned that the cursor for Method B could sometimes be hard to find. The partici-pants mentioned that this often occurred at the start, when a new page was loaded, and when the cursor waslocated on items that had a color that correspond with the color of the cursor. Example of improvementsmentioned, on how to improve the cursor, was to enlarge it, have both a white and a black border in orderto still stand out if the background is black or white.

When utilizing Method C, many of the participants mentioned that the cursor was easier to find when com-pared to Method B, due to the fact that the hand position corresponded with where on the screen the cursorshould be located. However, the cursor and the buttons on the receiving device was still to small, and shouldbe enlarged.

Some of the participants described that the titles displayed for each movie item was a good feature sincesome of the movie posters don’t clearly display a title.

7.2.2 Utility

When determining utility, the functionality of the design and how well it does what the user desires were tobe determined. In this section, firstly the overall thoughts given by the participants in the user test on howthe functionality was, will be described. Thereafter a more in depth description will be given on how theuser described the navigation and the scrolling functionalities.

7.2.2.1 Overall functionality

For the remote control, participants described that the functionality worked as expected, hard to make mis-takes and thereby easy to use. Some participants mentioned that the buttons were a little hard to press down,

31

but if it would be used over a longer time it would become a habit. But overall the interaction is familiar andthereby its simplicity.

For Method A, some opposite opinions were mentioned. Some participants mentioned that this method re-quired more movements than when using the remote control, and that swipe didn’t feel as a natural move-ment, whilst other participants mentioned that this interaction felt easier than with the remote control sinceless movements were required, since one didn’t need to move the finger to a specific button.

For Method B, some participants liked that the functionality was similar to a cursor controlling a computerscreen, and that it felt easy to interact and fast to choose items. Other participants mentioned that thismethod felt harder to use (compared to the remote control and Method A), since it required moving thecursor to the right position, which could compose of several finger movements (especially in the video view).Some participants mentioned that it required the user to concentrate more in order to keep track of thecursor position, and thereby taking away the ’relaxation’ part that one wishes to have when viewing TV.

For Method C, the overall impression of the participants was that this method needs improvements. Par-ticipants did mention that this method, due to physical feedback given from the hand position, did helpin finding out were the cursor was. However, the majority of the participants thought that the method feltineffective and demanded a lot of movements, that similar to Method B, removed the relaxation part.

7.2.2.2 Navigation

In this section the results concerning the navigation in the main menu, displaying all movie items that areavailable, will firstly be described. Thereafter the results concerning the navigation in the video view (play/-pause, return) will be described as well as results on how the controlling of the cursor was perceived by theuser.

When asked to grade how hard/easy it was to go to a movie item, on a scale 1-5 (hard-easy), the users re-sponded as shown in Table 5.

Table 5: How hard/easy the participants perceived it to go to a movie item

1 (Hard) 2 3 4 5 (Easy)

Remote Control 0% 0% 0% 33.3% 66.7%Method A 0% 0% 11.1% 33.3% 55.6%Method B 0% 0% 11.1% 33.3% 55.6%Method C 0% 33.3% 33.3% 22.2% 11.1%

The average grade on how easy the participants perceived it to got to a movie item for the remote control,Method A-C was calculated to 4.7, 4.4, 4.4 and 3.1, respectively. The one-way ANOVA and post-hoc test,conducted on the results,see Appendix A.1.7, showed that significantly different means exists, with a p-valueof 0.0006. The results illustrated Method C have means significantly different from the interaction methods,see Figure 19.

The results show that the majority of the participants perceived the remote control to be easy. The partici-

32

Figure 19: Method C differs from the rest of the interaction methods.

pants described that it performed as expected.

Method A was described by the users as easy to use, but the possibility to move over more items wouldbe a good feature. Some participants mentioned that once they understood which way a swipe movementaffected the other screen, it became easier to proceed (Since a swipe to the right can on some devices meango right and on others go left).

Method B got the same results, as Method A. Participants described that the method enabled them to moveover more items, which was appreciated, however sometimes the cursor was hard to find, as previouslymentioned in this chapter, see section 7.2.1.4.

For Method C, the participants described that the items located at the center of the screen was easier tonavigate to, than the items at the top or down corners, and that the angle demanded (in the hand) to reachthis positions felt impractical.

When asked how playing/pausing the movie and returning to the menu on a scale 1-5 (hard-easy), the par-ticipants responded:

Table 6: How hard/easy the participants perceived it to play,pause and return to the menu.

1 (Hard) 2 3 4 5 (Easy)

Remote Control 0% 0% 0% 33.3% 66.7%Method A 0% 0% 0% 11.1% 88.9%Method B 0% 22.2% 22.2% 22.2% 33.3%Method C 0% 0% 55.6% 33.3% 11.1%

The average grade on how hard/easy the participants perceived it to got to play/pause and return for theremote control, Method A-C was calculated to 4.7, 4.8, 3.7 and 3.6, respectively. The one-way ANOVA andpost-hoc test, conducted, see Appendix A.1.8, showed that significantly different means exists, with a p-value of 0.0017. The results illustrated that Method B and C had means significantly different from the Re-mote Control, see Figure 20a. It also showed that Method C had a significantly different means than Method

33

A, see Figure 20b.

(a) Method B-C differs from the Remote Control. (b) Method C differs from Method A

Figure 20: Multiple comparison of means.

For Method A the majority of particpants described it as very easy, since there were few objects to interactwith and thereby few actions had to be performed in order to achieve the tasks.

For Method B the users mentioned that it was harder, due to the fact that more movements needed to beperformed in order to place the cursor on the two buttons that one should interact with. Some participantsmentioned that they liked that the play buttons was by highlighted by default, when entering the videoview, and thereby the play/pause actions could easily be performed. However, since the mouse cursor wasnot located by default at the play button, the functionality wasn’t obvious at first, and the distance neededto move to the return button was still a problem. Participants mentioned that if the cursor would be locatedat play/pause by default, and if the distance to the return button was smaller, the actions needed to performto move the cursor would decrease and the method would be much more appreciated.

For Method C some participants mentioned that it was easier to perform these actions, since the two buttonsto interact with was located in the middle of the receiving screen and didn’t demand the user to move thehand as much as in the previous view.

For Method B and Method C, in which a cursor was added, the participants were asked how hard/easy itwas to move the cursor, the responses are shown in Table 7.

Table 7: How hard/easy the participants perceived it to move the cursor on the other screen.

1 (Hard) 2 3 4 5 (Easy)

Method B 0% 0% 22.2% 11.1% 66.7%Method C 0% 33.3% 22.2% 44.4% 0%

The average grade for how easy/hard the participants perceived it was to move the cursor for Method B andMethod C was calculated to 4,4 and 3.1, respectively. The one-way ANOVA and post-hoc test, conducted onthe results, see Appendix A.1.9, showed that significantly different means exists, with a p-value of 0.0065.The results illustrated that Method B and Method C have means significantly different from another, seeFigure 21.

34

Figure 21: The means of Method B and Method C are significantly different.

As previously mentioned, many users stated that it was easier to find the cursor using Method C than whenusing Method B, due the hand position matching where on the screen the cursor would be. However, theparticipants noted that when using Method C, only the items centered on the screen felt easy to choose,whilst others, especially at the bottom of the page, felt harder to reach. This due to the fact that when holdingthe phone, the angle of the wrist needed in order to reach this position was to large.

For Method C, some participants also mentioned that when holding the phone fairly still, the cursor some-times stuttered, and thereby didn’t move as smoothly from one position to another, when compared toMethod B.

7.2.2.3 Scrolling

When the participants used the smart-phone, an observation on how the users held the sender device, wasmade by the test instructor. All the participants held the phone in on hand and controlled the touch eventsit with the thumb, see Figure 22a.

(a) Holding the phone in one hand, and control-

ling it with the same thumb.

(b) Holding the phone in one hand and control-

ling it with the other hand.

Figure 22: Illustration of how the user held the phone during the tests and controlled it with the thumb (a),

and how the user would have to use the other hand too use the two finger scroll functionality (b) Images

retrieved from [Dreamstime 2017].

The automatic scrolling was appreciated by the users, and few noticed that scroll could be enabled by drag-

35

ging two fingers. Those participants that did find this functionality however described that an alternateversion demanding only one finger would be a better solution, since the user would have to use the otherhand to control scrolling functionality, see Figure 22b. One example of this would be to have a dedicatedscroll area to the right of the phone screen, and thereby the user could use the thumb and not need to usethe other hand.

Since Method A did not provide the ability to skip over multiple items, this functionality was desired by all ofthe participants. The possibility to do this for Method B and C, with the automatic scrolling, was appreciated.For Method C, some users described that the sensation of physically pointing the phone enabled them toaccurately move over more items quicker than the previous methods.

7.2.3 The overall results of the interaction methods

After all methods were tested, the user was asked to grade all of the interaction techniques they tested, theresults are as follows:

Table 8: Table displaying the grades given for the different interaction techniques tested in the user test.

1 (Bad) 2 3 4 5 (Good)

Remote Control 0% 0% 22.2% 66.7% 11.1%Method A 0% 0% 22.2% 44.4% 33.3%Method B 0% 11.1% 44.4% 11.1% 33.3%Method C 0% 55.6% 22.2% 11.1% 11.1%

The average grade for the methods were calculated as follows: 3.9, 4.1, 3.7 and 2.8 for the Remote Control andMethod A-C, respectively. The one-way ANOVA and post-hoc test, conducted, see Appendix A.1.10 showedthat significantly different means exists between Method A and Method C, with a p-value of 0.0239. Theresults also show that none of the methods have significantly different means from the Remote Control andMethod B, see Figure 23.

(a) Means of Method A and C are

significantly different.

(b) No groups have means signifi-

cantly different from Remote Con-

trol

(c) No groups have means signifi-

cantly different from Method B.

Figure 23: Figures displaying the post-hoc results for the grades given by the users.

36

The users were also asked if they would use any of the Methods A-C instead of a remote control, the resultsare displayed in Figure 24, and Table A2 in Appendix A.

MethodA

Yes

Yes,ifimproved

No

(a) Method A

MethodB

Yes

Yes,ifimproved

No

(b) Method B

MethodC

Yes

Yes,ifimproved

No

(c) Method C

Figure 24: Figures displaying the responses on which of the methods that the users would like to use instead

of a remote control.

The one-way ANOVA and post-hoc test, conducted on these results, see Appendix A.1.11, showed that noneof the methods had significantly different means than the others, with a p-value of 0.2944.

Some improvements mentioned by the participants, needed for Method A were to have the option to swipeover multiple items, improve the sensitivity to detect swipe and tap, enhance the UI on the receiving devicein order to more clearly visualize which item that is chosen.

Improvements mentioned by the participants for Method B were to enlarge the buttons on the receivingdevice, have a dedicated scroll area on the smart-phone, changing the phone UI by having different areason the phone representing different actions, and make the cursor larger and more visible.

One of the participants mentioned that if a combination of the first two interaction technique was devel-oped, in which the touch movements made for Method B was used to move the highlighted item instead ofa cursor, would be more beneficial, and would be something the user would like to use.

For Method C, some participants mentioned that if a number of improvements were performed, mostlyconcerning the angle sensitivity, this method would be much nicer to use. However some participants ex-pressed that this method would not be wanted, even if improvements were made.

An overall improvement mentioned by the participants were to enhance the feedback given from the phone.Some participants mentioned that if a sound or a small vibration were to be given from the phone, whenperforming an action, would improve the interaction methods. In this way, the the user would get confir-mation that an action has been performed and make of for possible visual delays on the receiving screen.

7.3 Application

The result of this thesis was a web application that allows the user to control another screen with a smart-phone. The final application can be seen in Figure 25.

The application works and has been successfully tested on the platforms iOS and Android for the phone.

37

For the receiving device, the application has been tested on the STB and the browsers Chrome and Safari.By utilizing the framework Apache Cordova, this HTML5 and JavaScript application should be able to workon more mobile platforms as well as browsers, with only minor adjustments, if any.

38

(a) Final views on the smart-phone.

(b) Final views on the receiving device.

Figure 25: The final views for the smart-phone and the receiving device.

39

8 | Discussion

In this chapter the results from the user tests are analysed and discussed. All the tested models of interactionhave proven usable to varying degrees. Of the three methods implemented, Method A and B was preferredthe most by the users. However, more participants said that they would use this Method A as a substituteover a remote control. The users explained that the method itself was simple to understand and fairly easyto use.

8.1 Previous experience and expectations

In some instances the results were clearly colored by the users’ previous experience and expectations. Asmentioned in Chapter 7.2.2.2, some participants mentioned that once they knew which way to performthe swipe movement it was easier to proceed. Since a swipe to the right can on some devices mean goright, and on others go left, there was a learning curve for some participants. As a contrast to many similarsituations, some of the participants that were familiar with similar interaction patterns were already trained,but trained to swipe the wrong way, thus actually performing worse than untrained participants.

The overwhelming majority of the problems that occurred, did so when utilizing Method A, as seen by thenumber of errors that occurred. This could possibly be motivated by the "faulty training", that some partic-ipants suffered from, but also the inability by the application to distinguish between clicks and swipes. Thephenomenon is further discussed in a later section, 8.5.

8.2 Impact of visual design

For the UI on the receiving device, many of the participants remarked that the item that is highlighted couldbe more clearly indicated, see Chapter 7.2.1.4. Currently the white highlight has a good contrast versus thebackground, but not always against the posters themselves. This could possibly be solved by a number ofdesign changes, such as using a sharp border with a contrasting color around the item. This way there isalways a guaranteed contrast towards the highlight effect.

One technique that was employed to mark the currently selected movie was adding a white banner acrossthe top of the movie. This banner contained the movie name in purple, and allowed the user to clearly readthe movie name. Or at least that was the intention, the user tests showed that most users did not notice thisfeature until the last few interaction methods tested, if at all. Those users that did notice it, appriciated it, soimproving this aspect of the UI might be warranted. One way of doing this could be working with additionalcontrasting colors (analogous to the previous section), or just adding the banner below the poster, thuschanging the size and shape of the perceived active element and making it stick out more.

The participants mentioned that it was easier to navigate in the video-player view than in the other viewsusing Method A, since the options were fewer (play/pause, back). It was also more straight forward to un-derstand what swipe directions were available since all the options were neatly aligned in a single row.

40

Using Method B, some users remarked that interaction would be greatly simplified if the buttons were madelarger, and thereby easier to "hit". Some participants also mentioned that, especially in the video selec-tion view, the highlighted individual items helped to locate the cursor when they were highlighted (hoveredover). This effect, however, was greatly reduced in the video-player view, since the vast majority of this viewis inert and therefore does not have highlights on hovered regions other than the two buttons where inter-action is possible.

This indicates that there might be a dependency on the density of interactive components in the view whenanalyzing the usability. In this aspect it seems that Method A does well in a low density setting (the playbackview) and not as well in a high density setting (the video selection view), and Method B has the reversedcharacteristics.

8.3 Impact of movement styles

Even though method C was only preferred by few users, compared to Method B, many users mentioned thatit was easier to find the cursor using Method C, since there is a more direct link between the cursor positionand the physical orientation of the hand. This could possibly indicate that it is more intuitive and requiresless conscious effort to find the cursor when basing the cursor on feedback from the hand position wherethe cursor would be on the other screen.

Method C also had a drawback however, all participants reported a need of improvements when movingthe cursor vertically. As mentioned previously, many users preferred to use small wrist movements, and thevertical movements required by the user to cover the desired parts of the screen were simply uncomfortable.

Some participants solved this by moving the whole arm rather than just the hand. This hints at two sub-groups in the users, where one group is doing fine tuned movements, see Figure 26a using only the wrist,and others are using a larger portion of their body, turning also at the elbow and maybe even shoulder, seeFigure 26b.

8.4 Haptic feedback on the client side

During the user testing, some users expressed the desire to get some haptic feedback from the smart-phonewhen performing an action, previously mentioned in Chapter 7.2.1.3. This feature was especially desired forMethod A, in order to more distinguish that an action have been performed. For instance haptic feedbackcould be used to indicate that a tap or swipe was detected, e.g. by utilizing small vibrations from the smart-phone. Some users also mentioned that a sound or blink could be used for the same purpose.

Incorporating haptic feedback could also potentially make the navigation easier to use in a high latencysituation where connection is bad. Enabling the user to get confirmation that the action was recognizedand does not need to be repeated even if it does not immediately generate feedback on the TV screen.

41

(a) Wrist movements.

(b) Elbow and shoulder movements.

Figure 26: Fine tuned movements (a) and elbow and shoulder movements (b). Images from [Medlej 2014]

8.5 Errors

The errors that were measured during the test reflected if the participant performed another action thenexpected, e.g. pressed pause when trying to press return, or choosing another movie than asked for.

The results show that for Method A, in which swipe and tap movements were used, the most errors occurred,and significant difference between this method and the others was found. Even though this method waspreferred by the most of the participants, and the fastest to use, one way these errors can be explained is dueto the sensitivity of the tap action. When trying to swipe to an item, the application sometimes registered atap movement instead of a swipe. With an application better able to separate a tap from swipe movement,and thereby minimize the risk that a tap is registered instead of a swipe, is likely to make the average timeeven less for using Method A.

Another explanation, previously mentioned in Chapter 7.2.1.3, was that when swiping the opposite direc-tion of the hand holding the phone, could have impacted the number of errors that occurred. Since the

42

number of required right and left swipes are equal, this should not introduce a bias, depending on usersbeing reight or left handed.

The severity of the registered errors were minor (selecting the wrong movie and pressing the play/pausebutton to many times), and the user could easily recover from them (navigate to the back button and tryagain, or press play/pause again). This is supported by the results, where the average time to completionhas no strong correlation to the number of errors. In particular, Method A was still the fastest even thoughit was connected to the wast majority of the errors.

8.6 Test instructor

As previously mentioned in Chapter 6.1, the test instructor measured the time to complete tasks but alsoobserved how many errors that were made and if the user looked down on the sender device. Since there wasonly one test instructor that did all of these things, some bias may exist. In hindsight a better solution wouldhave been to have more test instructors, each having their one task to concentrate on. Another solutioncould be to record the user in order to get an accurately check if the user looked down on the sender device.

When retrieving background information about the users, the gender and education was not collected,which in hindsight might have been interesting to have in the final results.

8.7 What is the best method?

The results from the user test illustrate that all methods could be used as a substitute for a remote control,in some cases predicated on improvements being done. However, the user test also showed that the touchinteraction based Method A and B were generally preferred over the gesture based Method C.

Looking then at Methods A and B, the best choice is dictated by the design of the interface in question.Method A is simpler, but might require many swipes to get where the user wants to go, whereas MethodB is more effective since it allows the user to move directly to the desired item (one "move" can fly acrossseveral items), but also requires more work on the user’s part keeping track of the cursor and coordinatingwith finger movements.

Developing a combination of the first two interaction techniques, as mentioned Chapter in 7.2.3, could bea good solution. By doing so, the concentration needed to locate the cursor would be greatly reduced, bothsince the possible positions would be reduced to only valid items, but also since the highlighted item is moreclearly marked. This approach was not tested, but probably has it’s very own problems and limitations thatwould need to be further explored.

Based on all these findings, Method A seems most suited to replace the remote control as it is, with someminor adjustments, but all candidate methods can be viable substitutes if improved on.

43

9 | Lessons learned

As in many software development projects, the project description was subjected to updates during theproject. The project description in Appendix C is an example of this. In this case it matches the originaldescription rather well, however this might not always be the case, and a lot of software methodology isdesigned to handle these changes, e.g. agile development.

In this chapter some insights that were discovered during this thesis project, are described.

9.1 Abstraction Boundaries

The application architecture is based on evaluating and developing different interaction models, that mightnot be desirable in a live deployment. Instead, a more general solution where the abstraction layers betweenthe sender and receiver are more clearly enforced, might more be beneficial in a future product.

In the current implementation, the sender device sends the orientation angles to the receiver, which thencalculates the new positions for the cursor, see Figure 27. The possibility to change this to be done on thesmart-phone itself, could be an improvement for the application. By doing this, the same function could becalled for both the Method B and Method C, when sending the new distance, and thereby the set-top boxwouldn’t need to know any other information about the phone except the distance moved.

Touch

Orientation

(dx,dy)

(d┢, d┣, wphone, hphone) Calculate (dx,dy)

Move (x,y) (x’,y’)

Figure 27: How the application currently sends information from the smart-phone.

During the development, by allowing leaky abstractions the set-top box application could easily print diag-nostic information needed for development, that was hard to receive from the sender side, especially if thephone crashed.

9.2 Build process

When implementing an iterative development cycle, the project is built several times per day. This meansthat the build and deploy time greatly affects the delay between writing code and getting feedback. Theproblem is multiplied by the number of platforms.

44

The cordova build process takes quite a while to complete, and it is not always obvious which parts need tobe recompiled at any given point. Sometimes small changes in the GUI can prove to be more time consum-ing then anticipated, since more often than not at least one platform requires special treatment.

In addition there were manual steps in the deploy process that was addressed early in the project (by loadingfiles dynamically from a web server rather than from a USB stick). There are several additional process op-timizations that could improve this situation, maybe even different build systems, but optimizing this wasdeemed out of scope for this thesis. Also, the company uses Cordova in other projects, therefore replacing itwas not considered.

9.3 Development process

During the development of the product, the user centered design methodology was utilized. Since the usersthat were readily available, and used to briefly test the application during development, were employees ofZenterio Services, the selection might be biased, and even though it aided the development the result mighthave been better with a more neutral user section, i.e. a distribution more similar to that of potential users.

The current process allow incorporation of many hard earned experiences into the development process,but might have disregarded things like intuitiveness and learnability. Looking at the results from the properuser tests, this problem seems to be very limited.

Another open question is whether users can be "reused", many of the users polled during development werepicked from a rather limited pool, meaning that learnability factors were only really addressed the first fewiterations. As the users added more and more feedback, they might even have become invested in certainfunctionality or mechanics.

One telling example is Method C, where the range of input was dictated primarily by one user during devel-opment, but during user testing it turned out that maybe the movement range should be narrowed to allowfor more precise, fine-tuned, wrist-only movements.

9.4 STB specific DOM issues

The browser in the STB sometimes works just a little bit differently than any other browsers, this includesthings like websockets and CSS/DOM property updates and similar things. Giving rise to a number of issueswhere the code runs fine on a "normal" browser but not at all in the STB.

For example, attempts to increase the size of the item to be highlighted did not work out as expected. On a PCbrowser this worked as expected, but on the STB, the object lost its position in the window when scrolling.No errors were shown, and it turned out that this was a known bug for this particular browser.

Since these situations usually do not cause errors they are very time consuming to troubleshoot, there areno stack traces to follow, and no documentation or even on-line communities to consult.

45

9.5 Documentation for javascript libraries

Many JavaScript languages are not fully documented and the developer is left to follow unofficial tutorialsor just searching for instructions on their use.

This results in many problems where solutions are found, but may not provide idiomatic solutions, or inother cases solutions that are deprecated in the current version. Sometimes even when there is documen-tation available, it does not match the use case well.

One good example of this is scrolling functionality, where there are several methods available, but eachmethod tested exhibits some problem on at least one of the platforms.

9.6 Git

One of Git’s greatest strengths is to manage versions in a multi-user environment, where several users areworking on the project simultaneously and them commit their changes. Git then helps detect conflicts, andhelp merge changes to get a unified result. Of course all intermediate steps are saved and can be rolled backas desired.

Since this project only had one developer, much of this functionality remains unused. The one feature thatwas used frequently was stash (go back temporarily to last, hopefully working, commit).

One constant issue that was observed was that many features tended to span multiple files and end upaffecting a large part of the project. As a result, most commits affected most of the project files and werehard to roll back independently of other commits.

46

10 | Conclusion

The results show that all interaction techniques presented and tested can be used as a substitute for a remotecontrol. The concept of the application was appreciated, and further development of these methods couldbe a potential product that can be integrated into JetUI. Out of the three methods integrated, Method A wasmost liked by the users and would be used as a substitute over a remote control. However, a combination ofMethod A and B may prove to be more beneficial in the long run.

The answers to the problem statements for this thesis project are as follows:

1. How can a touch based application implemented on a smartphone become an effective substitute

for a remote control, whilst enabling the user to use the application without looking down on the

phone screen? Two of the methods explored, A and B, do solve this problem. Since there is no GUI onthe phone, there is no need to look at it. All feedback is given by the receiving device (TV), and couldpotentially also be given in non-visual form on the sending device (phone).

2. Can other interaction techniques, such as voice and gesture facilitate the use of the application?

Voice interaction was deemed infeasible early on in the project, but gesture based interaction wastested in Method C. This method was the least liked and least efficient method, however some userswould have liked to have this as a substitute for a remote control. Since the overall assessments of thismethod was so negative, it can not be recommended at least not in its current form.

3. What feedback should be given to the user to facilitate the use of the application? A significant por-tion of the perceived problems with any of the Methods A-C seems to stem from insufficient feedbackin one form or another. This is interpreted to indicate that having enough feedback is important.

Two types of feedback were identified. The first category concerns clear visual response on the TVscreen identifying the highlighted item, or the cursor. The second category is feedback to confirm thatan interaction was identified on the client, like a short vibration or sound to confirm that an actionwas completed.

Both these categories should be used to provide the best feedback to the user.

4. How should the graphical interface on the smartphone be designed in order to ensure that users

know how to use it whilst not attracting to much attention to itself when being used. Based on pre-vious research, presented in Chapter 3.1, using a remote control interface on a smart-phone, withbuttons representing real buttons on a remote control, draws the visual attention of the user to thesmart-phone, since no tactile feedback is given on where the different buttons are located.

In this thesis, the UI was formed to accept touch interactions on a plain surface, without any visualinformation given. This worked fine, as the necessary instruction was given by the test leader. In a realsetting, including similar instructions on the phone itself should be considered.

47

11 | Future work

For the application derived for this thesis, some improvements that can benefit the product and that can beneeded to do before implementing into JetUI are described in this chapter.

11.1 Sensitivity

For all interaction methods, some improvements concerning the sensitivity needs to be performed. ForMethod A, the sensitivity for separating a swipe from a tap needs to be tweaked. The decision to not useany external library for detecting touch events, since it is supported on all major mobile platforms, mighthave been a wrong assessment. The problems that occurred with the distinction of swipes and taps mighthave been prevented, and thereby minimized the number of errors, by using a library instead. This problemcould in some cases also be seen for Method B, and therefore using an external library might have proven tobe beneficial.

For Method C, a lot of improvements needs to be done with its sensitivity. Two major problems with thismethod can be seen. The first being how smooth the cursor moves on the other screen, and the secondbeing how the range of the angles used for the method.

The first one, as mentioned in Chapter 7.2.2.2, was noticed by the participants during the user tests, in whichthe cursor stuttered. This behavior can be explained by the threshold value that was added to the MethodC, in order to remove the small movements that is registered when holding the phone in the hand. Sincethis approach resulted in that the stuttered cursor movements another approach, by using a low-pass filter,might have proven to be a better solution.

The second problem for the sensitivity for Method C, was connected to how the majority of the users usedsmall wrist movements to control the cursor on the receiving device. Increasing the sensitivity, and therebydecreasing the angles that shall be represented on the other screen, could become an improvement. How-ever, future tests that should be performed is to determine which movements, (wrist or elbow and shouldermovements) that are best suited for the intended users.

11.2 Communication and pairing

When developing the application for this thesis, the security for the pairing and communication betweenthe sender and receiver device was not developed.

Currently when creating a connection between the sender and receiver device, the ip-adresses for bothdevices have to be entered manually. This process is not the most efficient one, and future work for thiswould be to create a secure pairing process. Another feature that then would make this more efficient wouldbe to add the functionality of remembering units that previously have been paired.

In order to develop this application into a full product, future work is to extend the application and develop

48

a secure connection between the sender and receiver.

11.3 Feedback

Some future work that needs to be done for the application is mainly to enhance the feedback given fromthe receiving device but also from the sender device.

As showed from the user test, the current feedback given from the receiving device needs to be improvedin order to more clearly visualize which item that is selected. This could be done, as previously mentioned,using contrasting colors for the highlight, but also more clearly improve the banner displaying the movietitles.

For the sender device, the feedback could be enhanced by using small vibrations or sounds when perform-ing actions. A good approach might be to start with adding vibrations in order to give a tactical feedback,however future work could also be to investigate which of the following approaches that would be mostappreciated by the user.

11.4 Future tests

The current tests done only values the end result for the cursor based methods (i.e. where the user clicks),not the cursor movement to get there. This means that a method where the user moves the cursor all overthe screen gets as good a result as one where the user moves distinctly from one position to the next. Aninteresting test for fine tuning these methods would be to test how easily and accurately the user can movealong a specific path.

As for testing methods, now that the major interaction models are validated, A/B testing could be used on alarger number of users for fine tuning the variables of the different methods, for example using small versuslarger angle ranges in Method C.

As discussed in Chapter 8.2, information density in the UI of the receiving device might affect the usabilityof the different interactions. Future tests on how strong this correlation is might be valuable.

49

References

APPLE. (2016). Use your Siri Remote or Apple TV Remote with Apple TV (4th generation). https://support.apple.com/en-us/HT205305. Retrieved: 10/03/16.

textscBackbone.js. (2016). Backbone.js. http://backbonejs.org. Retrieved 11/04/16.

BENYON, D., TURNER, P. & TURNER, S. (2005). Designing Interactive Systems - People, Activities, Contexts,

Technologies. Pearson Education Limited, England.

BERNHAUPT, R., BOUTONNNET, M., GATELLIER, B., GIMENEZ, Y., POUCHEPANADIN, C. & SOUIBA, L. (2012).A set of recommendations for the control of IPTV-systems via smart phones based on the understanding of

users practices and needs. Proceedings of the 10th European conference on Interactive tv and video, p.143-152, New York, USA.

BLENDER FOUNDATION. (2008) Big Buck Bunny. https://peach.blender.org/press/. Retrieved: 11/11/16.

DHIR, A.(2004). The Digital Consumer Technology Handbook: A Comprehensive Guide to Devices, Stan-

dards, Future Directions, and Programmable Logic Solutions. Elsevier, USA.

DREAMSTIME 2017 Hands holding phone and tablet vector icons set. https://www.dreamstime.com/stock-illustration-hands-holding-phone-tablet-vector-icons-set-monochrome-communication-click-smartphone-gesture-finger-image57257617. Retrieved: 01/17/17.

DUMAS, J.F. & REDISH, J.C. (1993). A Practical Guide to Usability Testing. Greenwood Publishing GroupInc., USA.

ENNS, N. R. N., & MACKENZIE, I. S. (1998). Touchpad-based Remote Control Devices. Companion Pro-ceedings of the ACM Conference on Human Factors in Computing Systems - CHI ’98, pp. 229-230. NewYork: ACM.

FONTELLO. (2017). Fontello - icon fonts generator. http://fontello.com/. Retrieved: 06/06/17.

FORSELL & COOPER. (2010). A guide to scientific evaluation in visualization. In Proceedings of the 14thInternational Conference Information Visualisation(IV’10), pp. 162-169. IEEE.

GAMMA, E., JOHNSON, R., HELM, R. & VLISSIDES, J.. (1994). Design Patterns: Elements of Reusable Object-

Oriented Software. Pearson Education, USA.

ISO9241-11. (1998). Ergonomic requirements for office work with visual display terminals, part 11. Guid-ance on usability. Geneva. International Organization for Standardization.

JAAKKOLA, H. & THALHEIM, B..(2011) Architecture-driven modelling methodologies. Proceedings of the2011 conference on Information Modelling and Knowledge Bases XXII. Anneli Heimbürger et al. (eds). IOSPress. p. 98

50

KANGOVI, S. (2016). Peering Carrier Ethernet Networks. Morgan Kaufmann, USA.

LAZAR, J. (2005). Web Usability: A User-Centered Design Approach. Addison-Wesley Longman PublishingCo., Inc., USA.

LÖWGREN, J. 1993 Human-computer Interaction: What Every System Developer Should Know. Studentlit-teratur. Sweden.

MEDLEJ, J. (2014). Human Anatomy Fundamentals: Flexibility and Joint Limitations. https://design.tutsplus.com/articles/human-anatomy-fundamentals-flexibility-and-joint-limitations--vector-25401,Retrieved: 01/23/17.

MOZILLA DEVELOPER NETWORK. (2016). Touch events. MDN - Mozilla Developer Network. https://developer.mozilla.org/en-US/docs/Web/API/Touch_events, Retrieved: 07/27/16.

NIELSEN, J. (2012). Usability 101: Introduction to Usability. https://www.nngroup.com/articles/usability-101-introduction-to-usability/, Retrieved: 09/20/16.

OSMANI, A. (2012). Learning JavaScript Design Patterns. O’Reilly Media, Inc., Sebastopol, CA, USA.

PERCIVAL, J. (2013). HTML5 Advertising. Apress, USA.

SAFFER, D. (2009). Designing for Interaction: Creating Innovative Applications and Devices. New RidersPublishing. USA.

ZENTERIO. (2016). Zenterio. https://se.linkedin.com/company/zenterio-ab. Retrieved: 10/04/16.

ZENTERIO MARKETING. (2016a) Zenterio releases updated versions of JetUI and Zenterio OS. http://www.zenterio.com/zenterio-releases-updated-versions-jetui-zenterio-os/. Retrieved: 08/14/16.

ZENTERIO MARKETING. (2016b) ACCESS to showcase JetUI at Interop Tokyo. http://www.zenterio.com/access-showcase-jetui-interop-tokyo/. Retrieved: 08/14/16.

51

Appendix A. | Results

Table A1: Average time [s] for completing the tasks using the different interaction techniques

Remote Control [s] Method A [s] Method B [s] Method C [s]

1 6.1 4.0 4.4 5.9

2 5.5 7.2 6.1 6.1

3 6.8 5.8 5.1 8.8

4 8.5 6.9 6.1 6.6

5 6.3 6.0 5.6 5.0

6 5.0 5.3 3.9 3.6

7 5.7 4.5 6.0 4.7

8 5.9 7.5 10.4 12.6

9 5.6 5.1 5.7 7.6

Total average: 6.2 5.8 5.9 6.8

Table A2: Table displaying responses on which of the methods that the users would like to use instead of a

remote control.

Yes Yes, if improved No

Method A 55.6% 11.1% 33.3%Method B 22.2% 33.3% 44.5%Method C 22.2% 11.1% 66.7%

Table A3: All time[s] measured in the user tests.

Remote Control [s] Method A [s] Method B [s] Method C [s]

1 9.3, 7.8, 4.6, 6.0 5.2, 4.1, 3.8, 4.1 7.8, 5.1, 3.6, 4.6 11.0, 7.8, 5.0, 4.9

2 10.1, 5.8, 5.2, 5.5 13.2, 6.5, 7.6, 7.4 8.4, 7.9, 5.4, 4.9 7.6, 7.8, 4.9, 5.5

3 10.4, 7.2, 6.1, 7.0 5.5, 9.0, 4.2, 4.1 7.1, 5.7, 5.0, 4.6 9.8, 13.9, 6.9, 5.6

4 11.2, 8.9, 8.4, 8.2 13.9, 9.4, 6.1, 5.3 9.7, 5.6, 6.1, 6.5 9.1, 8.0, 5.3, 6.5

5 7.3, 7.2, 5.7, 6.1 6.8, 7.9, 5.0, 5.2 6.9, 6.0, 5.7, 5.1 8.8, 6.5, 3.8, 4.6

6 9.5, 5.4, 4.2, 5.4 5.5, 7.7, 4.3, 3.8 5.6, 4.6, 3.5, 3.6 3.4, 4.1, 3.3, 3.4

7 9.8, 6.6, 5.1, 5.3 7.0, 5.6, 4.0, 4.0 8.2, 5.3, 6.3, 6.4 6.7, 5.0, 4.0, 5.0

8 8.8, 6.5, 4.8, 6.5 10.6, 5.6, 7.9, 8.9 11.9, 9.5, 9.9, 11.8 8.3, 9.5, 16.8, 11.4

9 8.1, 6.5, 5.2, 5.1 7.8, 5.7, 5.2, 4.3 7.6, 4.6, 6.9, 5.6 14.4, 7.7, 7.7, 7.5

52

A.1 ANOVA and Post-hoc test - MATLAB

In this appendix the code for determining the ANOVA and post-hoc utilizing MATLAB is displayed.

A.1.1 ANOVA1 and post-hoc calculated for Chapter 7.2

1 % I f the user f e l t they had looked on the c o n t r o l l e r ( remote/phone)2 % With the answers yes = 1 , no = 0 .3 Y = 1 ;4 N = 0 ;5

6 % Remote Control7 m1 = [N, N, N, Y , N, N, N, N, N] ;8 % Method A ( swipe , tap )9 m2 = [N, N, N, N, N, N, Y , N, N] ;

10 % Method B ( tap , drag )11 m3 = [N, N, N, N, N, N, N, N, N] ;12 % Method C ( gesture , tap )13 m4 = [N, N, N, N, N, Y , N, Y , N] ;14

15 group = { ’Remote Control ’ , ’Method A ’ , ’Method B ’ , ’Method C ’ } ;16 r e s u l t s = [m1’ , m2’ , m3’ , m4’ ]17 [p , tbl , s t a t s ] = anova1 ( r es u l ts , group ) ;18 multcompare ( s t a t s ) ;

A.1.2 ANOVA1 and post-hoc calculated for Chapter 7.2.1.1

1 % Remote Control2 m1 = [ 9 . 3 , 1 0 . 1 , 1 0 . 4 , 1 1 . 2 , 7 . 3 , 9 . 5 , 9 . 8 , 8 . 8 , 8 . 1 ] ;3 % Method A ( swipe , tap )4 m2 = [ 5 . 2 , 1 3 . 2 , 5 . 5 , 1 3 . 9 , 6 . 8 , 5 . 5 , 7 , 1 0 . 6 , 7 . 8 ] ;5 % Method B ( tap , drag )6 m3 = [ 7 . 8 , 8 . 4 , 7 . 1 , 9 . 7 , 6 . 9 , 5 . 6 , 8 . 2 , 1 1 . 9 , 7 . 6 ] ;7 % Method C ( gesture , tap )8 m4 = [ 1 1 , 7 . 6 , 9 . 8 , 9 . 1 , 8 . 8 , 3 . 4 , 6 . 7 , 8 . 3 , 1 4 . 4 ] ;9

10 group = { ’Remote Control ’ , ’Method A ’ , ’Method B ’ , ’Method C ’ } ;11 r e s u l t s = [m1’ , m2’ , m3’ , m4’ ]12 [p , tbl , s t a t s ] = anova1 ( r es u l ts , group ) ;13 multcompare ( s t a t s ) ;

A.1.3 ANOVA1 and post-hoc calculated for Chapter 7.2.1.2

53

1 % Remote Control2 m1 = [ 7 . 8 , 4 . 6 , 6 , . . .3 5 . 8 , 5 . 2 , 5 . 5 , . . .4 7 . 2 , 6 . 1 , 7 , . . .5 8 . 9 , 8 . 4 , 8 . 2 , . . .6 7 . 2 , 5 . 7 , 6 . 1 , . . .7 5 . 4 , 4 . 2 , 5 . 4 , . . .8 6 . 6 , 5 . 1 , 5 . 3 , . . .9 6 . 5 , 4 . 8 , 6 . 5 , . . .

10 6 . 5 , 5 . 2 , 5 . 1 ] ;11 % Method A ( swipe , tap )12 m2 = [ 4 . 1 , 3 . 8 , 4 . 1 , . . .13 6 . 5 , 7 . 6 , 7 . 4 , . . .14 9 , 4 . 2 , 4 . 1 , . . .15 9 . 4 , 6 . 1 , 5 . 3 , . . .16 7 . 9 , 5 , 5 . 2 , . . .17 7 . 7 , 4 . 3 , 3 . 8 , . . .18 5 . 6 , 4 ,4 , . . .19 5 . 6 , 7 . 9 , 8 . 9 , . . .20 5 . 7 , 5 . 2 , 4 . 3 ] ;21 % Method B ( tap , drag )22 m3 = [ 5 . 1 , 3 . 6 , 4 . 6 , . . .23 7 . 9 , 5 . 4 , 4 . 9 , . . .24 5 . 7 , 5 , 4 . 6 , . . .25 5 . 6 , 6 . 1 , 6 . 5 , . . .26 6 , 5 . 7 , 5 . 1 , . . .27 4 . 6 , 3 . 5 , 3 . 6 , . . .28 5 . 3 , 6 . 3 , 6 . 4 , . . .29 9 . 5 , 9 . 9 , 1 1 . 8 , . . .30 4 . 6 , 6 . 9 , 5 . 6 ] ;31 % Method C ( gesture , tap )32 m4 = [ 7 . 8 , 5 , 4 . 9 , . . .33 7 . 8 , 4 . 9 , 5 . 5 , . . .34 13.9 , 6 . 9 , 5 . 6 , . . .35 8 , 5 . 3 , 6 . 5 , . . .36 6 . 5 , 3 . 8 , 4 . 6 , . . .37 4 . 1 , 3 . 3 , 3 . 4 , . . .38 5 , 4 ,5 , . . .39 9 . 5 , 1 6 . 8 , 1 1 . 4 , . . .40 7 . 7 , 7 . 7 , 7 . 5 ] ;41

42 r e s u l t s = [m1’ , m2’ , m3’ , m4’ ]

54

43 [p , tbl , s t a t s ] = anova1 ( r e s u l t s ) ;44 multcompare ( s t a t s ) ;

A.1.4 ANOVA1 and post-hoc calculated for Chapter 7.2.1.3

1 % Remote Control2 m1 = [ 0 , 0 , 0 , 0 , 1 , 0 , 0 , 0 , 0 ] ;3 % Method A ( swipe , tap )4 m2 = [ 0 , 0 , 2 , 2 , 0 , 1 , 1 , 5 , 0 ] ;5 % Method B ( tap , drag )6 m3 = [ 0 , 0 , 0 , 0 , 0 , 0 , 0 , 0 , 1 ] ;7 % Method C ( gesture , tap )8 m4 = [ 1 , 0 , 1 , 0 , 1 , 0 , 0 , 0 , 0 ] ;9

10 group = { ’Remote Control ’ , ’Method A ’ , ’Method B ’ , ’Method C ’ } ;11 r e s u l t s = [m1’ , m2’ , m3’ , m4’ ]12 [p , tbl , s t a t s ] = anova1 ( r es u l ts , group ) ;13 multcompare ( s t a t s ) ;

A.1.5 ANOVA1 and post-hoc calculated for Chapter 7.2.1.3

1 % remote2 m1 = [ 0 , 0 , 0 , 0 , 1 , 0 , 0 , 0 , 0 ] ;3 % mA4 m2 = [ 0 , 0 , 2 , 2 , 0 , 1 , 1 , 5 , 0 ] ;5 % mB6 m3 = [ 0 , 0 , 0 , 0 , 0 , 0 , 0 , 0 , 1 ] ;7 % mC8 m4 = [ 1 , 0 , 1 , 0 , 1 , 0 , 0 , 0 , 0 ] ;9

10 % l a b l e s11 group = { ’Remote Control ’ , ’Method A ’ , ’Method B ’ , ’Method C ’ } ;12

13 % ANOVA14 r e s u l t s = [m1’ , m2’ , m3’ , m4’ ] ;15 [p , tbl , s t a t s ] = anova1 ( r es u l ts , group ) ;16

17 % Post−hoc ( tukey−kramer − default )18 %[ c ,m, h ,nms] = multcompare ( s tats , ’ alpha ’ , . 0 5 , ’ ctype ’ , ’ tukey−kramer ’ ) ;19 [ c ,m, h ,nms] = multcompare ( sta ts , ’ alpha ’ , . 0 5 , ’ ctype ’ , ’ lsd ’ ) ;

A.1.6 ANOVA1 and post-hoc calculated for Chapter 7.2.1.4

55

1 % How hard/ easy the part ic ipants perceived i t to see which item that washighlighted

2 % Remote3 m1 = [ 5 , 5 , 2 , 3 , 3 , 3 , 5 , 5 , 4 ] ;4 % Method A5 m2 = [ 5 , 5 , 3 , 3 , 3 , 3 , 5 , 5 , 4 ] ;6 % Method B7 m3 = [ 5 , 5 , 5 , 5 , 3 , 4 , 5 , 4 , 4 ] ;8 % Method C9 m4 = [ 5 , 5 , 5 , 5 , 3 , 4 , 5 , 4 , 4 ] ;

10

11 r e s u l t s = [m1’ , m2’ , m3’ , m4’ ] ;12 group = { ’Remote Control ’ , ’Method A ’ , ’Method B ’ , ’Method C ’ } ;13 [p , tbl , s t a t s ] = anova1 ( r es u l ts , group ) ;14 multcompare ( s t a t s ) ;

A.1.7 ANOVA1 and post-hoc calculated for Chapter 7.2.2.2, Table 5.

1 % How hard/ easy the part ic ipants perceived i t to go to a movie item2 % Remote3 m1 = [ 5 , 4 , 5 , 5 , 5 , 5 , 5 , 4 , 4 ] ;4 % Method A5 m2 = [ 5 , 3 , 4 , 4 , 5 , 4 , 5 , 5 , 5 ] ;6 % Method B7 m3 = [ 5 , 5 , 5 , 4 , 4 , 4 , 5 , 3 , 5 ] ;8 % Method C9 m4 = [ 3 , 5 , 2 , 2 , 3 , 4 , 4 , 3 , 2 ] ;

10 group = { ’Remote Control ’ , ’Method A ’ , ’Method B ’ , ’Method C ’ } ;11 r e s u l t s = [m1’ , m2’ , m3’ , m4’ ] ;12 [p , tbl , s t a t s ] = anova1 ( r es u l ts , group ) ;13 multcompare ( s t a t s ) ;14 %saveas ( gcf , ’ move_to_item . pdf ’ )

A.1.8 ANOVA1 and post-hoc calculated for Chapter 7.2.2.2, Table 6.

1 % How hard/ easy the part ic ipants perceived i t to play , pause and return to the menu.2

3 % Remote4 m1 = [ 5 , 4 , 5 , 5 , 5 , 5 , 5 , 5 , 5 ] ;5 % Method A6 m2 = [ 5 , 3 , 5 , 5 , 5 , 5 , 5 , 4 , 5 ] ;7 % Method B8 m3 = [ 5 , 5 , 4 , 5 , 2 , 4 , 3 , 2 , 3 ] ;

56

9 % Method C10 m4 = [ 4 , 5 , 3 , 3 , 3 , 4 , 4 , 3 , 3 ] ;11

12 r e s u l t s = [m1’ , m2’ , m3’ , m4’ ] ;13 group = { ’Remote Control ’ , ’Method A ’ , ’Method B ’ , ’Method C ’ } ;14 [p , tbl , s t a t s ] = anova1 ( r es u l ts , group ) ;15 multcompare ( s t a t s ) ;

A.1.9 ANOVA1 and post-hoc calculated for Chapter 7.2.2.2, Table 7.

1 % Method B2 m1 = [ 5 , 5 , 5 , 5 , 5 , 4 , 5 , 3 , 3 ] ;3 % Method C4 m2 = [ 3 , 4 , 2 , 2 , 3 , 4 , 4 , 4 , 2 ] ;5

6 r e s u l t s = [m1’ , m2’ ] ;7 group = { ’Method B ’ , ’Method C ’ } ;8 [p , tbl , s t a t s ] = anova1 ( r es u l ts , group ) ;9 multcompare ( s t a t s ) ;

A.1.10 ANOVA1 and post-hoc calculated for Chapter 7.2.3, Table 8.

1 % Grades2 % Remote3 m1 = [ 4 , 4 , 5 , 4 , 3 , 4 , 4 , 4 , 3 ] ;4 % Method A5 m2 = [ 4 , 3 , 5 , 5 , 3 , 4 , 4 , 5 , 4 ] ;6 % Method B7 m3 = [ 5 , 5 , 4 , 3 , 3 , 3 , 5 , 2 , 3 ] ;8 % Method C9 m4 = [ 2 , 5 , 2 , 2 , 2 , 4 , 3 , 3 , 2 ] ;

10

11 r e s u l t s = [m1’ , m2’ , m3’ , m4’ ] ;12 group = { ’Remote Control ’ , ’Method A ’ , ’Method B ’ , ’Method C ’ } ;13 [p , tbl , s t a t s ] = anova1 ( r es u l ts , group ) ;14 multcompare ( s t a t s ) ;

A.1.11 ANOVA1 and post-hoc calculated for Chapter 7.2.3, Figure 24 and Table A2.

1 % I n i t i a l i z i n g values that represents the answers yes , yes i f improved and2 % no .3 y = 2 ;4 y_i = 1 ;

57

5 n = 0 ;6

7 % Method A ( swipe , tap )8 m1 = [n , n , y , y , y_i , y , n , y , y ] ;9 % Method B ( tap , drag )

10 m2 = [ y , y , y_i , n , y_i , n , y_i , n , n ] ;11 % Method C ( gesture , tap )12 m3 = [n , y , n , n , n , y , n , n , y_i ] ;13

14 group = { ’Method A ’ , ’Method B ’ , ’Method C ’ } ;15 r e s u l t s = [m1’ , m2’ , m3’ ] ;16 [p , tbl , s t a t s ] = anova1 ( r es u l ts , group ) ;17 multcompare ( s t a t s ) ;

58

Document Title:

Zenterio Zense Revision:

Date:

2016-07-08

Zenterio “Zense”

Zenterio ”Zense” är ett koncept där vi vill undersöka hur man kan använda de alternativa

input-möjligheter som en smart telefon erbjuder för att fjärrstyra en TV applikation.

Touch

Med hjälp av telefonens touch skärm ska man kunna förflytta ett fokus på TV skärmen.

Man ska även kunna välja det fokuserade elementet och även scrolla listor med flera

element.

Man kan även tänka sig att touch används för att utföra gester på skärmen för att rita

siffor och på så sätt hitta rätt TV kanal. Att dubbel-tappa på skärmen kan betyda att ta

upp en huvudmeny exempelvis.

Här gäller det att hitta en lösning där touch känns naturligt och som fungerar lika bra

eller bättre som ersättare till en vanlig fjärrkontroll. Det bör exempelvis vara enklare att

snabbt förflytta sig mellan många element då man slipper trycka många ggr på en pil-

tangent.

Motion/Orientation

Med hjälp av att känna av hur telefonen är riktad vill vi testa om den kan användas som

en sorts pekpinne för att styra en fokuspunkt på en TV skärm. Touch används här mest

för att välja eller scrolla.

Man kan också tänka sig att man drar nytta av att veta om telefonen ligger med

ovansidan nedåt eller uppåt för att aktivera eller avaktivera TV tjänsten, Spela/pausa en

video eller liknande.

Man skulle även kunna detektera en lätt skakning av telefonen vilket skulle kunna

betyda att visa en huvudmeny eller liknande.

Voice

Genom att använda rösten kan man utför saker utan att behöva styra med fingrar eller

rörelser. Det kan ibland vara enklare och mer effektivt än att navigera sig fram till olika

funktioner. I detta koncept vill vi testa hur enkelt det är att använda sig av befintliga

tekniska lösningar för att testa detta. Det kan betyda att kvalitén blir lidande av de

begränsningar de lösningar vi har tillgång till kan medföra

Appendix B. | Product description (in Swedish)

59

Document Title:

Zenterio Zense Revision:

Date:

2016-07-08

Övriga sensorer

Det vore även intressant om tid ges att utnyttja ljussensor på telefonen om möjligt. Den

skulle kunna användas för att anpassa UI på TV till rätt ljusförhållanden exempelvis.

Demonstration av lösning För att kunna granska dessa olika metoder behöver ett par appar skapas som tester och

demos kan utföras på. Det behövs en klient-app i telefonen som tar in all input och

skickar denna data till en server. Det behövs även en klient-app på TVn som tar emot all

data via servern och visualiserar navigering på en TV-skärm.

Design Exempel

Telefon App

60

Document Title:

Zenterio Zense Revision:

Date:

2016-07-08

Användartester Som en del forskningen av dessa koncept vor det intressant att utföra tester med en

grupp av testpersoner från olika åldersgrupper. I denna grupp skulle man kunna testa

dels traditionell användning av fjärrkontrollen och dels användning av ”zense” som

alternativ metod.

61

Appendix C. | User Test Protocol

C.1 Introduction

Hej! Tack så mycket för att du vill vara med i detta användartest vars resultat kommer vara en del i mitt examensarbete. Det som du ska testa är en demoapplikation som jag har utvecklat som del av mitt examensarbete på företaget Zenterio i Linköping. Zenterio är ett företag som framför allt arbetar med att producera tv-boxar och applikationer som kan visas på smart-tv. Utöver det så arbetar de också med utveckla tv-tjänster som kan användas på andra enheter såsom smart-phones, surfplattor eller webbläsare. I mitt examensarbete har möjligheten att använda en smart-phone till att styra en annan skärm undersökts, där fokus har legat på att testa hur man kan göra det med olika interaktionstekniker. Syftet med dessa tekniker är att användaren ska kunna styra den andra skärmen utan att behöva kolla ner på sin telefon. Till detta har 3 alternativ tagits fram, och det jag vill undersöka här idag är hur väl de 3 interaktionsteknikerna fungerar, och om någon av dem skulle kunna vara ett alternativ till en fjärrkontroll. Min roll här är att ge ett antal uppgifter som du ska utföra och jag kommer att mäta tiden som krävs för att slutföra dem för respektive interaktionsteknik. Efter varje test kommer jag ställa några frågor om upplevelsen. Din roll i detta användartest är att utföra dessa uppgifter och därefter besvara frågorna. Användartestet är beräknat att ta en timme. Jag kommer börja med att samla in lite bakgrundsinformation om dig och därefter ge en liten introduktion kring vad det är du ska testa.

62

C.2 Background information

63

C.3 Questions - Remote control

64

C.4 Questions - Touch swipe and tap

65

C.5 Questions - Touch with cursor

66

C.6 Questions - Orientation and tap

67

C.7 Final questions

68