activity recognition in resource-constrained pervasive...

LICENTIATE T H E S I S

Department of Computer Science and Electrical EngineeringDivision of Computer Science

Activity Recognition in Resource-Constrained Pervasive SystemsISSN 1402-1757

ISBN 978-91-7583-485-6 (print)ISBN 978-91-7583-486-3 (pdf)

Luleå University of Technology 2015

Niklas K

arvonen Activity R

ecognition in Resource-C

onstrained Pervasive Systems

Niklas Karvonen

Activity Recognition

in

Resource-Constrained Pervasive Systems

Niklas Karvonen

Department of Computer Science and Electrical EngineeringLuleå University of Technology

SE–971 87 LuleåSweden

December 2015

Supervisors

Kåre Synnes, Professor, Luleå University of TechnologyJosef Hallberg, Ph.D., Luleå University of Technology

Peter Parnes, Professor, Luleå University of Technology

Printed by Luleå University of Technology, Graphic Production 2015

ISSN 1402-1757ISBN 978-91-7583-485-6 (print)ISBN 978-91-7583-486-3 (pdf)

Luleå 2015

www.ltu.se

Abstract

There is an increasing need for personalised and context-aware services in our everyday livesand we rely on mobile and wearable devices to provide such services. Data collected fromthese devices includes important information about users’ movements, locations, physiolog-ical status, and environment. This data can be analysed in order to recognize users’ activ-ities and thus provide contextual information for services. Such activity recognition is animportant tool for personalising and adapting assistive services and thereby increasing theusefulness of them.

This licentiate thesis focuses on three important aspects for activity recognition usingwearable, resource constrained, devices in pervasive services. Firstly, it is investigated howto perform activity recognition unobtrusively by using a single tri-axial accelerometer. Thisinvolves finding the best combination of sensor placement and machine learning algorithm forthe activities to be recognized. The best overall placement was found to be on the wrist usingthe random forest algorithm for detecting Strong-Light, Free-Bound and Sudden-Sustainedmovement activities belonging to the Laban Effort Framework.

Secondly, this thesis proposes a novel machine learning algorithm suitable for resource-constrained devices commonly found in wearable and pervasive systems. The proposed al-gorithm is computationally inexpensive, parallelizable, has a small memory footprint, and issuitable for implementation in hardware. Due to this, it can reduce battery usage, increase re-sponsiveness, and also make it possible to distribute the machine learning task, which enablesbalancing computational costs against data traffic costs. The proposed algorithm is shown tohave a comparable accuracy to that of more advanced machine learning algorithms mainlyfor datasets with two classes.

Thirdly, activity recognition is applied in a personalised and pervasive service for im-proving health and wellbeing. Two monitoring prototypes and one coaching prototype wereproposed for achieving positive behaviour change. The three prototypes were evaluated ina user workshop with 12 users aging between 20 and 60. Participants of the workshop be-lieved that the proposed health and wellbeing app is something people are likely to use on apermanent basis.

By applying results from this thesis, systems can be made more energy efficient andless obtrusive while still maintaining a high activity recognition accuracy. It also shows thatpervasive and wearable systems using activity recognition have the potential of relievingsome problems in health and wellbeing that society face today.

iii

Contents

Abstract iii

Publications ix

Acknowledgments xi

1 Thesis Introduction 1

1.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

1.2 Motivation of thesis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

1.2.1 Healthcare . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

1.2.2 Behaviour change . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

1.2.3 Sports and leisure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

2 Thesis Overview 7

2.1 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

2.1.1 Ubiquitous and pervasive computing . . . . . . . . . . . . . . . . . . . 9

2.1.2 Constraints on pervasive systems . . . . . . . . . . . . . . . . . . . . . 9

2.1.3 Machine learning on resource constrained pervasive systems . . . . . . 10

2.1.4 Activity recognition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

2.1.5 Effort classification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12

2.1.6 Laban movement analysis . . . . . . . . . . . . . . . . . . . . . . . . . 12

2.1.7 Theory for behaviour change . . . . . . . . . . . . . . . . . . . . . . . . 13

2.1.8 Motivational ICT applications . . . . . . . . . . . . . . . . . . . . . . . 13

2.2 Related Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

2.2.1 Activity recognition using accelerometers . . . . . . . . . . . . . . . . 15

2.2.2 Quantifying physical activities of everyday life . . . . . . . . . . . . . 16

v

vi Contents

2.2.3 Implementation of classifiers on resource-constrained devices . . . . . 16

2.2.4 ICT for positive behaviour change . . . . . . . . . . . . . . . . . . . . . 17

2.2.5 Pervasive mobile systems for positive health behaviour change . . . . 18

2.3 Research questions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20

2.4 Delimitations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21

2.5 Research methodology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22

2.6 Overview of included articles . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23

3 A Computationally Inexpensive Classifier Merging Cellular Au-tomata and MCP-neurons 25

3.1 Abstract . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27

3.2 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27

3.2.1 The McCulloch-Pitts neuron model . . . . . . . . . . . . . . . . . . . . 28

3.2.2 Cellular Automata . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28

3.2.3 Research questions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29

3.3 Related work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30

3.4 The CAMP algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31

3.5 Training of the classifier . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33

3.5.1 Chromosome encoding . . . . . . . . . . . . . . . . . . . . . . . . . . . 33

3.5.2 Fitness calculation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33

3.5.3 Selection of individuals . . . . . . . . . . . . . . . . . . . . . . . . . . . 33

3.5.4 Crossover and mutation . . . . . . . . . . . . . . . . . . . . . . . . . . . 34

3.6 Results and evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34

3.6.1 Test setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35

3.6.2 Learning curve . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35

3.6.3 Testing of accuracy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35

3.7 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38

3.8 Conclusions and future work . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39

4 Analyzing Body Movements within the Laban Effort Frameworkusing a Single Accelerometer 41

4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43

4.2 Related Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45

4.3 Data Collection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47

4.3.1 Definition of Activities . . . . . . . . . . . . . . . . . . . . . . . . . . . 47

4.3.2 Collecting Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49

Contents vii

4.4 Data Processing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50

4.4.1 Feature Extraction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50

4.4.2 Feature Selection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52

4.4.3 Classification models . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53

4.5 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53

4.6 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55

4.7 Conclusions and Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58

4.8 Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58

4.9 Author Contributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58

4.10 Conflicts of Interest . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59

5 Designing ICT for Health and Wellbeing - An Allostatic,Behavioral-Change Approach to a Monitoring and Coaching App 61

5.1 Abstract . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63

5.2 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63

5.2.1 The allostatic model in health and wellbeing . . . . . . . . . . . . . . . 63

5.2.2 Transtheoretical model of behavioral change . . . . . . . . . . . . . . . 64

5.3 The monitoring and coaching platform . . . . . . . . . . . . . . . . . . . . . . . 66

5.4 Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67

5.4.1 Initial findings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67

5.5 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69

5.6 Conclusions and Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69

5.7 Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70

6 Discussion, Conclusions and Future Work 71

6.1 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73

6.1.1 How can activity recognition be performed unobtrusively in a wear-able system with resource constraints? . . . . . . . . . . . . . . . . . . 73

6.1.2 How can classification be performed in resource-constrained devices? 74

6.1.3 How can a mobile activity recognition system be designed to achievepositive behaviour change in health and wellbeing? . . . . . . . . . . . 74

6.2 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75

6.3 Future work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76

Publications

This licentiate thesis consists of an introduction and three publications. The introductionprovides an overview of the papers and their relation to each other. All papers are publishedin international peer-reviewed journals or conferences except Paper 1 that is submitted to aninternational peer-reviewed journal.

Paper 1 Niklas Karvonen, Basel Kikhia, Lara Lorna Jimenez, Miguel Gomez, Josef Hall-berg "A Computationally Inexpensive Classifier Merging Cellular Automata and

MCP-neurons", Submitted to IEEE Transactions on Neural Networks and Learning

Systems, ISSN 2162-237X, 2015.

Paper 2 Basel Kikhia, Miguel Gomez, Lara Lorna Jimenez, Josef Hallberg, Niklas Kar-vonen, and Kåre Synnes. "Analyzing Body Movements within the Laban Effort

Framework Using a Single Accelerometer.", In Sensors, Volume 14, Issue 3, Pages5725-5741, Publised by MDPI, ISSN 1424-8220, DOI 10.3390/s140305725, 21 March2014.

Paper 3 Anders Hedman, Niklas Karvonen, Josef Hallberg, Juho Merilahti "Designing ICT

for Health and Wellbeing - An Allostatic, Behavioral-Change Approach to a Mon-

itoring and Coaching App", In Proceedings of the 6th International Workshop on

Ambient Assisted Living (IWAAL), Lecture Notes in Computer Science (InformationSystems and Applications, incl. Internet/Web, and HCI), Ambient Assisted Living andDaily Activities, Volume 8868, ISSN 0302-9743, Belfast, UK, 2-5 December 2014.

ix

Acknowledgments

First, I would like to thank Emelie and my family for the love, support, and understanding youconstantly show me. And to Oskar: You are and will always be the most important person inmy life. I can not imagine a life without anyone of you.

Secondly, I would like to thank my supervisors Josef Hallberg, Kåre Synnes and PeterParnes for believing in me. Their principles, ideas and advice has made all the differenceon this journey. I hope to continue sharing the good times, the hard times, and the times inbetween with you guys in the next chapter as well.

Thirdly, I would like to thank all my friends and colleagues out there, you know who youare. Thank you for making my life awesome and for constantly inspiring me. Let us stayyoung and hungry forever!

Luleå, November 2015

Niklas Karvonen

P.S. Holla to da Swedish gov for handin’ me tha G’s dat pay my billz.

xi

Part 1

Thesis Introduction

1

Thesis Introduction 3

1.1 Introduction

We live in an age where mankind has surrounded itself with computer systems. Everythingbetween small single-purpose computational devices to large-scale infrastructure systems,like the Internet, are now all around us and some of these devices are even crucial for oursociety to work. During recent years, mobile phones, wearable sensors, cameras and GPSdevices have made their way into our everyday lives. These devices are now starting tochange the way humans interact with computers, and what we are seeing is the dawn of aparadigm shift in computer science.

The use of pervasive and wearable computing enables the creation of context-aware ap-plications. Worn sensors can provide important information about the user’s activities andsurroundings that can be analyzed in order for applications to react automatically to impor-tant events. Due to this, it is nowadays increasingly common for applications to initiate thecommunication with the user instead of the other way around. Our behaviour and actions arethus, in a way, becoming the output from these applications rather than the input to them.This makes context-aware applications an interesting tool that can be used in a variety of ar-eas, such as elderly care, health-care and wellbeing, where our behaviour could benefit frombeing monitored or changed in a positive way.

An important part of context-awareness is the ability to recognize a user’s activities. Thisthesis focus on how to perform recognition in an everyday setting using mobile and pervasivedevices. Such devices typically have resource constraints on computational power, memoryand battery lifetime. There are also other resource constraints that arise if activity recognitionis to be performed in everyday life, such as the number of sensors that can be used withoutobstructing the user, and the size, placement and connection of devices. Activity recognitionperformance is also highly dependent on the data available, making it a challenge to designminimalistic systems that still have reasonable performance.

Resource limitations in mobile pervasive devices can also introduce difficulties for activ-ity recognition. Activity recognition is commonly performed by using machine learning tech-niques, and these often involves computationally expensive calculations or algorithms. Thiscan make it unfeasible or sometimes even impossible to implement certain machine learn-ing configurations on such devices. Activity recognition using accelerometers in resource-constrained mobile pervasive systems has therefore been a focus of this thesis. It presentsmethods for solving some of the problems within this area as well as a prototype applicationfor applying these results for positive behavioural change. The goal of the presented researchis not to provide a complete solution but rather to provide insight and tools that can be adaptedto fit the user’s needs.

4 Thesis Introduction

1.2 Motivation of thesis

There are numerous areas of research and application domains that could benefit from usingactivity recognition. The benefits can be both on an individual as well as on a societal level.The remainder of this chapter will give examples from three different areas where a pervasivesystem using accelerometers for activity recognition could have a great impact.

1.2.1 Healthcare

One of the most important areas for activity recognition is healthcare. A pervasive activityrecognition system could become an important tool for preventing, treating and reducing theimpact of various medical conditions. The consequences of a stroke, for example, is highlydepending on when it is detected, since it needs to be treated as early as possible. During astroke, the patient typically loses some motor skills in her left or right side of the body. Asystem recognizing movement patterns could detect such an anomaly early on, and alarm theuser or caregiver. Such a system could also be a useful tool in the rehabilitation process aftera stroke [45].

There are also many other conditions that cause diversions from normal movement pat-terns where activity recognition could be used: Parkinson’s disease [59], Dementia [61],fall-detection [25], epilepsy seizures [66], just to mention a few. Early detection of thesesymptoms could improve the reaction time for treatment, which in some emergency condi-tions could mean the difference between life and death.

1.2.2 Behaviour change

Another interesting area for activity recognition is behavioural change. Since many be-haviours are linked with specific activity patterns, activity recognition applications could beused in order to detect wanted or unwanted behaviours and give the user instant feedback.This can be a powerful tool if the application is designed in line with behavioural therapyprinciples [83]. A smoking cessation application, for example, could use an accelerometeron the dominant wrist in order to detect when the user is smoking a cigarette or even open-ing the cigarette package, and give the user in-time motivation to stop. An application forprevention or treatment of depression could recognize changes in activities suggesting theperson is feeling low and engage the user to interact with friends or perform other actionsthat help the user to stop worrying. This would provide an intervention at the time where itmatters the most, something that is difficult to achieve in traditional treatment. Such an inter-vention could possibly break the otherwise downwards spiralling behaviour that can lead todepression.

1.2.3 Sports and leisure

Consistent movements plays a key role in many recreational activities. By training an activ-ity recognition system to recognize the correct movements for an activity, the user can get

Thesis Introduction 5

feedback on the quality of the movements performed in comparison to the correct ones. Insome situations it might even be possible to provide the user with feedback on which partof each movement that needs to be adjusted. In sports this can serve as an important tool inanalyzing swimming strokes, racquet swings, martial arts moves, bowling throws and more[71, 7, 51, 98]. For music, it could also help in practice for instruments such as the violin[87] and drums which are also highly affected by correct and consistent movements. Variantsof systems like these could also be useful in other fields like performance dancing, Tai-Chi,theatre acting or even video games.

6 Thesis Introduction

Part 2

Thesis Overview

7

Thesis Overview 9

2.1 Background

This section presents background information relevant to the research presented in this thesis.Some of the important terms and concepts will be explained. Figure 2.1 explains how thedifferent parts fit together.

2.1.1 Ubiquitous and pervasive computing

The terms ubiquitous computing and pervasive computing are closely related areas of re-search and they are often used interchangeably. While the general idea for both is to havecomputers surrounding us in the real world, there are, however, subtle differences. Ubiqui-tous computing, as described by Mark Weiser’s vision in the early 1990’s [106], is to havecomputers available anywhere at any time. Pervasive computing extends this definition tohaving computers and computer services everywhere while hiding this technology from theuser.

Mobile phones and wearable devices such as smartwatches, are examples of deviceswhich are commonly used in pervasive computing system environments today. These de-vices are commonly worn by users in everyday life and provides a great platform for a widevariety of pervasive applications. This platform has therefore played an important role inwriting this thesis.

2.1.2 Constraints on pervasive systems

It is important to consider the usability when designing applications for wearable computingand pervasive devices. The size and weight of devices is one essential consideration whendesigning a wearable system. Big and heavy equipment will generally have a negative effecton the usability of the system since the user might be fatigued by the weight or be obstructedby the worn devices. It is therefore important that the size and weight of the devices thesystem uses are kept small if the system is to be used in everyday life.

Even if devices are small and lightweight, one should keep the number of devices min-imal. A higher number of devices typically require more maintenance and effort from theuser such as taking the devices on and off, charging their batteries, connecting them to otherequipment etc. Furthermore, having a large number of devices also generally requires a morecomplex software system and can result in higher production costs overall. It is therefore im-portant to consider the tradeoff between information gain and resource-usage when designingwearable systems.

Having a small system is not enough in order for a system to be unobtrusive. The usabilityof the system can also be negatively affected if the placement of devices is uncomfortable,intrusive or may cause the user to experience social stigma. In the case of activity recognitionusing accelerometer data, the position of the sensor on the subject’s body will not only affectthe usability but also the performance of the classification. It is therefore important to find alocation that is unobtrusive while still being able to provide a reasonable accuracy.

10 Thesis Overview

Figure 2.1: Overview of a wearable pervasive system with resource constraints.

From the discussion above, it is clear that a wearable system for everyday use should con-sider the device positioning and minimizing the size and weight of the included devices inorder to ensure usability of the system. Limiting the size of devices will, however, generallyintroduce challenges for implementation activity recognition such as limitations in computa-tional power or memory. For cases such as low-level microcontrollers, this could, however,mean that implementation of more complex algorithms or calculations will not be feasible ormay even be impossible.

2.1.3 Machine learning on resource constrained pervasive systems

Many machine learning configurations (the combination of algorithm and feature selection)have a high computational cost that can negatively affect for example battery life and responsetime of a pervasive system. This could be solved partially by using hardware implementa-tions, such as FPGA:s, for the classifier. These are generally efficient in terms of powerconsumption and also make parallel processing possible. Unfortunately, many of the com-mon machine learning algorithms are not easily implemented in FPGA:s or other low-leveldevices due to their structures, use of floating point arithmetics, and other computations thatcan be complex to implement.

Thesis Overview 11

One example of a common machine learning algorithm that is difficult to implement onlow-level devices is the "traditional" feedforward neural network, using three fully intercon-nected layers with floating point weights associated with the connections and the sigmoid orhyperbolic tangent activation function. Implementing such a network on a limited device willrequire special workarounds such as the use of pseudo floating point arithmetics, adaptationsof the activation functions or other complex and possibly degrading principles [34].

From this discussion, it should be clear that machine learning is not always easily imple-mented in pervasive systems. Most activity recognition research so far has ignored this factand instead focused on the performance of different machine learning configurations in termsof accuracy. This is unfortunate since it means that many results available today can not bepractically applied in real-world applications.

2.1.4 Activity recognition

Activity recognition is a field of research within the bigger framework of context-awareness.In activity recognition, a set of user activities are classified (often with machine learning) byanalysing sensor data, commonly from pervasive and mobile devices. The set of activitiesto be recognized and the sensor data available will decide the complexity of the activityrecognition problem. Most research in the area so far has focused on detecting a set ofeveryday tasks such as walking, running, going up and down stairs, sitting and lying. It hasbeen shown that these can be recognized with a small amount of unobtrusive sensors, such asaccelerometers [31].

In order to perform activity recognition in an everyday scenario it is important to find asensors that can provide rich information while still being unobtrusive. Tri-axial accelerom-eters are well-suited sensors for this purpose since they are small, cheap and low-powered,precise sensors. Furthermore, body-worn tri-axial accelerometers can provide informationabout a subject’s body movements. These combined characteristics makes this kind of sen-sors useful for activity recognition in pervasive systems.

Creating accurate models of accelerometer patterns for activity recognition is a difficultproblem since individuals have different physical attributes and movement patterns. Most ac-tivity recognition research using accelerometers has therefore relied on supervised machinelearning in order to perform classification. This is commonly performed by first derivinga set of features from the time and frequency domains using a sliding window overlappingthe raw data. These features can then be used to construct a feature vector for the machinelearning process. The problems with activity recognition using accelerometers mainly con-sists of finding the combination of sensor placements, feature selection and machine learningalgorithm that yields the best classification performance for the selected set of activities. It isimportant to note that both the features selected and the position of a sensor are both affectingthe underlying model that a machine learning algorithm is trying to learn. This means thatfor any change in one of these parameters, a new classification problem will be created.

12 Thesis Overview

2.1.5 Effort classification

Although many everyday activities have been shown to be possible to classify with a rea-sonably high accuracy, this classification only gives information about what activity is beingperformed but not how it is performed. A person, for example, typically walks or runs atdifferent speeds depending on the occasion. If activity recognition is ignoring intensity level,important information about how activities are performed will be lost. One solution to thisproblem, is to divide each activity class into sub-classes based on intensity, e.g. slow walk,medium walk, fast walk, but this solution has limitations. It could possibly have a negativeeffect on classification accuracy due to some classes becoming "closer" to each other, it re-quires more training data to be gathered and thresholds for the different intensity levels mustbe decided. This solution is also activity-specific, meaning that it needs to be applied to eachclass where intensity information is needed.

Another solution would be to measure the intensities of movements that are part of activ-ities rather than the intensities of activities themselves. This could possibly provide a moregeneric way of measuring effort of activities. As a step towards this, the combination of activ-ity recognition and Laban Movement Analysis is examined in this thesis in order to classifyeffort of daily activities.

2.1.6 Laban movement analysis

Most physical activities are composed of body postures and movement patterns. In the light ofthis, activity recognition can be considered a kind of higher-level movement analysis. Bodyposture and movement patterns are widely studied in areas like dancing, sports and musicin order to enhance performance. This has led to the creation of different techniques andframeworks for analyzing human movement. One such framework is the Laban MovementAnalysis [64].

Laban Movement Analysis (LMA) was originally introduced by Rudolf Laban as amethod and language for describing, visualizing, interpreting and documenting varieties ofhuman movement. It provides a rich overview of movement possibilities, and it is consideredas "a formal language for movement description" [19]. LMA is divided into four categories:Body, Effort, Shape, and Space [2]. The Effort category, which is used in this work, dividesmovements further into four subcategories: Strong - Light, Sudden - Sustained, Bound - Freeand Direct - Indirect. It recognizes that many activities performed by humans have simi-lar body organizations or movement patterns associated with them. Reaching for a glass orthrowing a punch, for example, both include an extension of the arm in a similar fashion. Theefforts required for these two activities are, however, very different. LMA analyzes posture,accelerations and velocity in order to find the effort required for a certain movement. Theclassification in the LMA is performed by a human expert, which means there is no way ofstandardizing the classification in the way it is performed today.

LMA, and especially the Effort category, is interesting for activity recognition since theparameters required for this classification can easily be obtained from an accelerometer. Moreinterestingly, as explained above, certain body movements can often be found in many dif-ferent activities. This means that by finding a proper subset of such movements, it could

Thesis Overview 13

be possible to create an LMA effort classifier that can estimate the effort of these activitieswithout training the classifier specifically for each activity.

2.1.7 Theory for behaviour change

Knowing a user’s everyday activities and effort of these can be important in several areas,such as health and wellbeing. By utilising activity recognition for context-awareness, onecan create applications aimed at helping users achieve positive behaviour change. Changinga person’s behaviour is, however, a complex task since there is usually multiple psychologicalprinciples that affect a certain behaviour [72].

Behavioural science and psychology research has proposed several learning theories foreffective behaviour change such as the Social Cognitive Theory [15], Transtheoretical modelfor behaviour change [81], Theories of Reasoned Action [41] and Planned Behaviour [10]and the Health Action Process Approach [90]. These models benefit from having accurateinformation about the user’s current state and when, where and how the user engages incertain activities. While this information can be gathered manually by the user in diaries andlogs, such self-reporting have shown not only to be subject to different kinds of errors (e.g.subjective bias and recall errors) [97] but they also impose significant workload upon the user.

A mobile pervasive system that automatically logs user data can reduce the problems re-lated with data acquisition for behavioural therapy. Such a system can provide higher amountsof data with less subjective influence, something that is essential to the health-care profession[97]. Also by using technology for data evaluation and interventions, factors such as shameor guilt that can sometimes be experienced by subjects when self-reporting data [92] couldbe reduced due to the non-personal nature of computers.

2.1.8 Motivational ICT applications

There is a significant amount of research on using motivational ICT-systems for both phys-ical and mental wellbeing purposes [32, 36, 11, 65]. So far, results have been mixed andthe positive effects of motivational applications and ICT interventions have varied. However,despite that the knowledge about how to design effective ICT interventions for sustainable be-haviour change is still limited [74], the use of ICT for interventions has shown great promisein various domains of health. These domains include psychotherapeutic use [11, 18], dis-ease management [46, 42], health behaviour change [105, 76, 79, 77] and mental well-beingthrough positive psychology [74].

Studies using in-time interventions in motivational applications are, however, still scarce.Most studies on motivational applications so far, requires the user to manually input relevantdata at some convenient time. By using a pervasive system instead of manual input, the usercould be made aware of an unwanted behaviour at the time of interest. This use of in-timefeedback or interventions with different media (text, sound, video, vibrations) could have apositive impact on behaviour change when combined with health behavioural models [83].

Although behavioural therapy research has suggested many principles and methods onhow to achieve positive behaviour change, there is still much to learn about how to trans-

14 Thesis Overview

late this knowledge to computer applications [83]. Given the complex nature of man, it ishighly likely that many of these design choices must be found by trial-and-error explorations.Therefore, this thesis proposes a prototype application for positive behavioural change that isevaluated in a user workshop.

Thesis Overview 15

2.2 Related Work

The papers included in this thesis contain related work to the research questions addressedin each paper respectively. Therefore, this section instead presents related work from theperspective of the whole thesis. Focus will be on activity recognition using accelerometers,implementation of classifiers on resource-constrained devices, quantifying physical activitiesof daily life and using pervasive systems for positive behaviour change.

2.2.1 Activity recognition using accelerometers

A significant amount of activity recognition research has focused on finding the optimal ac-curacy by combining different sensor positions, feature selections, and machine learning al-gorithms. Bao and Intille [16] presented early seminal work in this field where subjects worefive accelerometers on different locations; upper arm, lower arm, hip, thigh and ankle. Thedata was used to construct features from both the time and the frequency domain in order totrain a suite of machine learning algorithms to detect 20 different everyday activities. Theirwork showed that these activities could be recognized with over 80% accuracy in a non-laboratory environment and without subject-specific training. They also investigated how thenumber of sensors affected the classification accuracy and found that thigh + wrist and hip+ wrist pairs showed less than a 5% decrease in recognition rate from results using all fiveaccelerometer signals. The best sensor placement for a single sensor was found to be thethigh while the second best placement was the hip.

Olguin and Pentland [78] studied the use of one, two and three accelerometer configu-rations for a Hidden Markov Model classifier in order to classify eight different everydayactivities. Their study included sensor placements on the right wrist, left hip and chest. Ob-servation vectors was formed by combining the mean and variance in the x, y, and z axes fromeach accelerometer. They found that the improvement of the global classification accuracyusing two or three sensors compared to having a single one was 20% and 30% respectively.The best single sensor placement was found to be the left hip and the overall accuracy ofusing a single sensor was about 60%.

Cleland et. al [31] studied the optimal placement of sensors in order to detect whole bodyactivities and postures; walking, jogging on a motorized treadmill, sitting, lying, standingand walking up and down stairs. 57 different combinations of sensor placements commonin previous research (chest, wrist, lower back, hip, thigh and foot.) were investigated in thisextensive study. Their results showed that the hip was the best placement of a single ac-celerometer using a Support Vector Machine trained on 26 different features from time andfrequency domain. It was also found that using two sensors improved the accuracy signifi-cantly compared to using only one, but that using more than two sensors did not significantlyimprove the accuracy. In their work, they also concluded that the difference between using asingle sensor and multiple sensors can be small when detecting high level activities such aswalking, sitting and lying which are grossly different from each other in terms of accelerom-eter signals.

Preece et. al [80] made an extensive survey on activity recognition using accelerometers.Their study found that most work varied considerably in the choice of sensor placements and

16 Thesis Overview

the activities being analyzed, which makes comparison of the results from different studiesdifficult. Due to this they suggested that future work in activity monitoring may benefit fromthe use of a standard predefined set of activities and sensor placements.

The vast majority of activity recognition research using accelerometers requires that clas-sifiers are trained for a given set of activities. In some applications, however, the need toknow the actual underlying activity is secondary to knowing for example what physical effortthe activity requires or whether or not the current behaviour is normal (anomaly detection).In these cases, movement analysis could possibly be used instead of activity recognition.By recognizing a set of movements that are common in everyday activities, one could solvemetadata-related problems such as finding physical effort or finding deviating behaviour fromactivities, without specifically training the classifier for each activity.

2.2.2 Quantifying physical activities of everyday life

Physical activity is an important factor for health and wellbeing. Unfortunately, it is difficultto quantify the physical activities of everyday life. Due to this, some activity recognitionresearch has used accelerometers and activity recognition techniques in order to find energyexpenditure for everyday activities. The signal magnitude area (SMA) for accelerometersignals (high pass filtered) have been used to quantify the level of intensity for a physicalactivity into a unit now known as activity counts. Linear models have been found that relatethe number of activity counts to the metabolic energy expenditure (MET) [26, 53, 101, 100].While this provides a way to measure the physical activities of everyday life, it provides littleinformation on how activities are performed. A system using activity counts could thereforebe improved by adding movement analysis in order to find more metadata about activities,for example whether movements have been harmful or not (e.g. repetitive movements, un-balanced activities).

2.2.3 Implementation of classifiers on resource-constrained devices

Although activity recognition research has found sensor placements, feature selections andmachine learning algorithms that yield high accuracies, the works that considers how to im-plement these in real-world scenarios are scarce. Karantonis et al. [57] implemented anaccelerometry-based, real-time movement classifier on a 16-Bit Ultra-Low-Power Microcon-troller with 2Kb memory. Their system distinguished between periods of activity and rest,recognized the postural orientation of the wearer, and detected events such as walking andfalls to a reasonable degree of accuracy. The algorithm they developed involves a hierarchi-cal binary structure where broad classifications are made in the top levels of a decision tree,and more detailed subclassifications are made at lower levels. Their results suggested thatthe implementation of a real-time movement classifier is feasible, but that their performancecould have been increased given a larger memory capacity. They also argued that processorspeed would become an issue if more elaborate processing techniques were to be employedto achieve more accurate and/or detailed classifications, such as using a neural network forclassifications.

Thesis Overview 17

Junker [56] examined how the sampling rate and bit resolution of accelerometer dataaffected the performance of C4.5 Decision tree, kNN and Naive Bayes classifiers. They useda sliding window approach to calculate the mean variance and the RMS feature from 4 sensorsignals and used those features (12 in total) for the classification. Sampling rates and signalresolutions were then varied from 10 to 100 Hz and 1 to 16 bits respectively. Their resultsshowed that the performance of the classifiers was stable over a wide range of samplingfrequencies and resolutions. They only observed a significant degradation when the samplingrate and the resolution were set to 10 Hz and 1 bit, respectively. Another interesting find wasthat the recognition performance for the Naive Bayes classifier even increased for smallerresolutions.

Krause [63] analyzed the trade-off between power consumption and prediction accuracyfor the eWatch, a wrist-worn sensor platform. They used a SVM classifier for classifica-tion on features from time and frequency domain separately. For frequency-domain-basedclassification, they used the absolute values of the frequency components using the FFT forboth accelerometer axes separately. For time-domain-based classification, they computed theempirical means, variances, square root of the uncentered second moment and the medianabsolute differences. In their work they suggested a selective sampling method based on aMarkov chain for transition probabilities between activities. This method showed that a re-duction to 40% of the number of used samples still resulted in a 95% classification accuracywhile improving their device battery lifetime by a factor of two. They also found that a sam-pling rate of of 6 Hz from accelerometers gave almost the same accuracy as achieved withmuch higher sampling rates.

Similarly as Krause, Yan [111] proposed a strategy for varying the the sampling frequen-cies but their method also varied the feature selection. These selections were based on aconfidence vector returned by the classifier, a vector containing the probability of the currentactivity recognition belonging to a each class. The highest scoring class in the confidencevector was chosen and that class’ optimal sampling frequency and feature selection was se-lected for the classifier and sensor. Should the highest confidence value drop below a certainthreshold, the classifier selected the highest sampling frequency and the full set of featuresuntil an acceptable value in the confidence vector was found. Their results showed that thisapproach could save 20-25% of battery life compared to a non-adaptive approach.

Although efforts have been made to reduce the resource consumption of activity recogni-tion, these works have focused mainly on finding methods for making the feature extractionand data acquisition more efficient. They have also typically chosen classifiers that have alower computational complexity, but there is little work performed in creating new classi-fiers or modifying existing ones that are targeted specifically towards resource-constrainedpervasive devices.

2.2.4 ICT for positive behaviour change

There has been a significant amount of research studying the use of ICT for achieving pos-itive behavioural change. Andrews et al. [11] studied 22 randomized controlled trials ofcomputerised Cognitive Behavioural Therapy (CBT) for major depression, social phobia,panic disorder or generalized anxiety disorder. They found the effect sizes being substantial

18 Thesis Overview

and that the results from these studies indicate both short term and long term benefits fromcomputerised CBT. They also found that although the patients have significantly reduced timewith the clinician, they were still adhered to and were satisfied with computerised CBT.

[18] reviewed 92 studies of online therapy and found an average effect size of 0.53 whichis considered to be a medium effect. The effect size showed significant variations among theproblem categories and variations on average effects ranged from a minimum of -0.10 to amaximum of 1.68. Their work suggested that problems that are more psychological in nature,that is dealing with emotions, thoughts and behaviours, are better suited for online treatmentsthan problems that are physical or somatic in nature, such as blood pressure or weight. Itshould be noted, however, that some problems (e.g., weight loss) are less effectively treatedby quite a few traditional face-to-face therapeutic approaches too [50, 91]. This implies thatthe effectiveness of ICT for behavioural change (and coaching in general) rely heavily on thenature of the problem addressed as well as the design of the application.

Gibbons et al. [46] did a meta-analysis of 146 articles related to Consumer Health Infor-matics (CHI) to find evidence of positive effects. These articles results showed great differ-ences in outcome depending on the targeted problem (e.g. only 36% studies demonstratedpositive effects on obesity while 57% showed positive effects for smoking cessation) andthere were also mixed results between studies relating to the same problem. These findingsalso showed that the outcome of CHI depends both what problem/condition is addressed andhow the application is designed.

2.2.5 Pervasive mobile systems for positive health behaviour change

Research on using pervasive applications for positive and healthy behaviour change are stillscarce. Consolvo et al. [32] created a mobile application system, UbiFit Garden, that used on-body sensing from a device containing a 3-d accelerometer, a barometer, a humidity sensor,visible and infrared light sensors, a temperature sensor, a microphone, and a compass, to logphysical activities. The system was designed to increase users’ physical activities by usinga metaphor of a garden that blooms throughout each week. For each activity, a flower isadded to the garden that is displayed on the background screen of the user’s mobile phone.Their results showed that there is a challenge in inferring a wider range of activities and thatthe form factor of on-body sensors, even if incorporated into a mobile phone, is an issue insome common situations. They also found that traditional error metrics used to describe theeffectiveness of activity inference systems do not capture important subtleties of how usersperceive inference errors, and that the usefulness and credibility of such systems is improvedby allowing users to manipulate and add to inferred data.

The Motivate! system developed by Lin et al. [65] used the user’s profile, location,agenda (weekly information about the user’s schedule), weather information, time of dayand information of local events in order to generate tailored advice for increasing the user’sphysical activity. There were in total 34 pieces of advice including 20 kinds of activities withdifferent constraints and the suitability of advice was calculated using IF-THEN rules for thedifferent parameters. Their findings suggested that recommending "simple-to-do" activitiesthat required fewer efforts could trigger more behavior change.

Thesis Overview 19

Another everyday health system, BALANCE, was proposed by Denning et al. [36]. BAL-ANCE is a mobile phone-based system for long term wellness management that automati-cally detects the user’s caloric expenditure via sensor data from a Mobile Sensing Platformunit worn on the hip. Users manually enter information on foods eaten via an interface ona mobile phone while the user’s gross motor patterns such as sitting, walking, running andbicycling are automatically detected. This information is then used to present a "fuel gage"to the user that informs how to balance the food and activities during a day.

While the research on mobile motivational systems for behaviour change is still limited,the interest for these kinds of applications is still high. This can be seen, for example, by thevarious commercial applications that are emerging in this field [4, 3, 5, 1]. It is likely thathealth and wellbeing applications will become further integrated with our current, alreadyubiquitous, computational environments.

20 Thesis Overview

2.3 Research questions

This thesis addresses some important considerations when performing activity recognition inresource-constrained pervasive and wearable systems. Activity recognition can become animportant tool in for example healthcare due to its ability to provide information about a userand its current context. This enables applications to respond to user movement patterns andbehaviour in various ways, such as detecting unwanted behaviour, logging movements foranalysis, performing anomaly detection and giving in-time feedback. If such systems are tobe used in everyday life it is, however, important that they are unobtrusive. The research inthis thesis therefore focuses on research questions related to performing activity recognitionin wearable, resource-constrained pervasive devices. This thesis also examines how to applyactivity recognition results in order to achieve positive behaviour change.

How can activity recognition be performed unobtrusively in a wearable system with

resource constraints?

This research question examines the classification accuracies for a suite of common machinelearning algorithms using features from a single accelerometer when placed on different bodylocations.

How can classification be performed in resource-constrained devices?

This research question examines how to perform classification on low-level devices suchas microcontrollers and FPGA:s. These kinds of devices can have resource-constraints thatmakes it a challenge to implement some of the common machine learning algorithms.

How can a mobile activity recognition system be designed to achieve positive behaviour

change in health and wellbeing?

This research question explores how principles from behavioural psychology can be com-bined with a pervasive mobile system in order to motivate the user to positive behaviourchange. It also examines how users would experience such a system.

Thesis Overview 21

2.4 Delimitations

Security and privacy are important considerations when creating pervasive applications.These need to be considered also when gathering data by means of sensor technology, andstoring and exchanging data. Security and privacy are both significant research areas on theirown, and have not been a focus of the work presented in this thesis.

Even though results in this thesis presents a prototype system for health and wellbeing,the main focus is still activity recognition. Furthermore, human-computer interaction is aninterdisciplinary research topic of its own and while generally known heuristics have beenapplied when designing the prototypes, no specific research on interface design is mentionedin this work.

22 Thesis Overview

2.5 Research methodology

The research presented in this thesis has been conducted with the ambition to use computerscience to solve actual problems found in the real world. This has been accomplished byworking in research projects that involve interdisciplinary research with both academia andindustry. The used research methodology was thus practical in nature and has resulted in pro-totypes, methods and artefacts that can be applied in real-life applications. These have givenvaluable insight about important potentials and problems in applied activity recognition. Thisthesis includes both quantitative and qualitative research and the problems addressed wereidentified primarily by observation of user needs, user behaviour and existing services.

The thesis investigates the classification accuracy for different machine learning algo-rithms when applied to features from a single accelerometer sensor placed on different bodypositions. Subjects for this study were randomly selected and were asked to perform a set ofpredefined movements in a controlled environment using wearable accelerometers. The needfor a controlled environment was due to the difficulties in requesting the subjects to self-annotate data together with the risk that such annotation might be performed erroneously,which would introduce uncertainty into the model. Evaluation was performed by quantita-tive evaluation of a suite of machine learning algorithms’ performances on the different bodypositions.

In this thesis, a mobile pervasive system motivating positive behaviour change in healthand wellbeing was developed. First, explorative studies of existing research and applicationsin the field of motivational support for health and wellbeing were performed. After this, dis-cussions of different ideas was held both with researchers and persons from the outside. Thiswas followed by paper prototyping to reach the two design choices of the prototypes. Thesewere implemented in HTML/Javascript for user evaluation. A workshop with 12 users agingbetween 20 and 60 was put together where the users answered a questionnaire individuallyand then explored the prototypes in groups of four. The results of the workshop was used inorder to select the most appropriate candidate prototype and that candidate was used, togetherwith feedback from the workshop, to create the mobile application prototype.

While implementing the mobile application prototype it became clear that the battery-consumption of the wrist-worn device would become a problem. Hence, the need for a com-putationally inexpensive machine learning algorithm that could be run on a low-level micro-controller was raised. The methodology used to address this problem was to explore existingresearch on this problem, and then proceed with developing a proof-of-concept algorithm.Evaluation was performed by testing the algorithm with real machine learning problems andcomparing its results to other common machine learning algorithms in order to assess itscapabilities.

Thesis Overview 23

Figure 2.2: Overview of included articles.

2.6 Overview of included articles

In this section, each publication included in the thesis will be briefly presented. The overalltheme is to find methods and principles for applying activity recognition in real-world appli-cations for pervasive systems. Such systems commonly have different resource-constraintsthat makes activity recognition a challenge. The included publications propose solutions tosome of these problems, and will be presented in a bottom up approach. (See figure 2.2)

Paper 1: A Computationally Inexpensive Classifier Merging Cellular Automata and

MCP-neurons In the first paper, a novel machine learning algorithm is presented that istargeted for low-level computational devices that are common in wearable pervasive systems[58]. The size of such systems (assumed battery driven) is partly determined by requirementson working memory, CPU functionality and power consumption. CAMP has a small memoryfootprint, uses only binary logic and has a low computational complexity, and can therebypossibly reduce the size of battery-driven pervasive systems.

Paper 2: Analyzing Body Movements within the Laban Effort Framework Using a

Single Accelerometer The second paper examines how activity recognition principles canbe applied on movement analysis in order to estimate the effort of everyday activities [60].Movements were classified according to the Laban Movement analysis framework using asingle accelerometer. By using only a single accelerometer and movement analysis it ispossible to create a minimalistic activity recognition system that can still provide importantdata about the user’s physical activities.

Paper 3: Designing ICT for Health and Wellbeing The third paper explores how to de-sign a mobile pervasive motivational system for health and wellbeing with minimal resources[52]. An activity recognition system consisting of a wrist-worn accelerometer and a heart-rate monitor was used together with a mobile phone application in order to motivate usersto positive behaviour change in four key areas: physical activity, stress, sleep, and social ac-tivity. Two prototype applications based on the Transtheoretical Model of behaviour changewere proposed and evaluated in a user workshop. The results of the workshop reveals someimportant considerations when designing a motivational system for health and wellbeing.

24 Thesis Overview

Part 3

A Computationally Inexpensive

Classifier Merging Cellular

Automata and MCP-neurons

25

27

3.1 Abstract

The algorithm presented in this paper, named CAMP, has been developed to obtain a classifierthat is suitable for resource-constrained devices such as FPGA:s, ASIC:s or microcontrollers.The algorithm uses a combination of the McCulloch-Pitts neuron model and Cellular Au-tomata in order to produce a computationally inexpensive classifier with a small memoryfootprint. The algorithm consists of a sparse binary neural network where neurons are up-dated using a Cellular Automata rule as the activation function. Output of the classifier isdepending on the selected rule and the interconnections between the neurons. Since solvingthe input-output mapping mathematically can not be performed using traditional optimiza-tion algorithms, the classifier is trained using a genetic algorithm. The results of the studyshow that CAMP, despite its minimalistic structure, has a comparable accuracy to that ofmore advanced algorithms for the datasets tested containing few classes, while performingpoorly on the datasets with a higher amount of classes. CAMP could thus be a viable choicefor solving classification problems in environments with extreme demands on low resourceconsumption.

3.2 Introduction

In 1943 neurophysiologist Warren McCulloch and mathematician Walter Pitts wrote a re-search paper that would highly influence computer science and artificial intelligence researchfor decades to come. The paper posed a model on how neurons in the human brain mightwork, modelled with electronic circuits. Their paper can be seen as the starting point for thenow well-known machine learning family of artificial neural networks. Although there existmany types of neural networks now, in this paper we will use the term neural network torefer to the perhaps most common type of neural networks: fully interconnected feedforwardneural networks.

Neural networks are algorithms that have proven to perform with great accuracy in awide variety of applications [85, 54, 102]. They have important and useful characteristics,such as the ability to approximate nonlinear functions, to use different activation functions,to perform both regression and classification, and to be executed in parallel. Recent advancesin deep neural networks training have also shown that this type of algorithm can be used inhighly complex classification problems by utilizing different levels of abstraction.

The basic concept of a neural network is as follows: A neuron in a neural network layerhas a connection from its output feeding into all the inputs of the neurons in the next layer.Each of these connections have an associated floating point weight. To calculate a neuron’sactivation level in the next layer, all the incoming connections are multiplied by the corre-sponding weight and then summed together. This sum is then used as input to an activationfunction, such as the sigmoid or hyperbolic tangent function, that decides the output level ofthe neuron.

Neural networks require that all weights between the layers are stored in memory. In ad-dition, these weights are floating point values that need to be multiplied and summed togetherfor use in a discretization of a continuous function. This makes neural networks a challenge to

28

implement in hardware or in resource-constrained devices such as FPGA:s. Even for more so-phisticated devices like microcontrollers, this can pose problems if there are hard constraintson memory or real-time response.

In order to reduce the complexity of implementing a neural network in hardware, theproblems of memory consumption, computational complexity and the use of floating pointvalues must be addressed. This could, for example, be achieved by simplifying the neuralnetwork by using integer or even binary values and reducing the interconnectivity betweenneurons. The complexity of implementing the activation function in hardware is of coursedepending on the function used, but it is clear that both the sigmoid and the hyperbolic tangentare not easily implemented in devices such as an FPGA:s [115]. This paper aims to find asimplified model addressing all these problems in order to produce a neural network that issuitable for hardware implementation, or for use on devices with extreme limits in resources.

3.2.1 The McCulloch-Pitts neuron model

The neuron model introduced by McCulloch-Pitts is an interesting choice for our problem,since it uses only binary values for both neuron output and network weights (fig. 3.1). Inputs,that are of either inhibitory or excitatory type, are summed together and compared to a thresh-old value, which determines the output of the neuron. Inhibitory inputs override excitatoryinputs, meaning that if there are any inhibitory inputs active, the neuron will always outputzero.

Due to its binary-only values, the architecture of the MCP-network is much easier toimplement in hardware than a traditional neural network. Although being structurally well-suited to run on primitive devices, there are two problems using the MCP-network for efficientand resource constrained classification:

1. It can only model linearly separable problems [86]

2. The computational complexity of three layered MCP-networks is quadratic

This paper examines if principles from binary Cellular Automata could be used in orderto overcome these limitations of the MCP-network model.

3.2.2 Cellular Automata

Cellular Automata (CA) is known to work at the edge of chaos and can show complex be-haviour emerging from seemingly simple rules [84]. A binary one-dimensional CA consistsof binary cells on a lattice. The cells’ states are updated in timesteps in the following way:the next state for each cell is calculated by looking at the cell’s own state together with itsleft and right neighbourhood, called a configuration. This configuration is used to look up thecell’s next state in a rule table that holds all possible combinations of configurations. In thecase of, for example, 3-neighborhood binary CA, one can construct 256 possible rules. Rulesare commonly named by first permuting the possible configurations in an orderly fashion and

29

Figure 3.1: The McCulloch-Pitts neuron model. Weights W, inputs I and the output ofneurons, are all binary-valued.

Figure 3.2: Rule 110 for a 1-dimensional binary cellular automata. The rule is named bytranslating the output pattern to an integer (0b01101110 = 110). At the bottom is an exampleof a CA using the rule to evolve one time step.

then constructing a bit pattern from the according next states that is translated to an integernumber [110]. (See fig. 3.2)

Despite its deceitful simplicity, CA has been shown to be turing complete even for a 3-neighbourhood binary cellular automata using rule 110 [33], making it interesting to studythis setup and its applications further.

3.2.3 Research questions

The hypothesis assumed in this paper, is that using a CA with rule 110 as an activation func-tion could increase the performance of the McCulloch-Pitts model while reducing the numberof needed interconnections between neurons (fig. 3.3). The research questions addressed are:

30

Figure 3.3: The CAMP neuron. Each neuron has r incoming connections (here r=3). Bothinputs and outputs are binary-valued.

1. Can a 3-neighbourhood binary cellular automata using rule 110 be used as an acti-vation function in a simplified MCP-network to create a computationally inexpensiveclassifier?

2. What is the accuracy of such an algorithm compared to common machine learningalgorithms?

The paper is structured as follows: Section 3.3 reviews related work, section 3.4 describesthe CAMP classifier, section 3.5 explains how CAMP is trained using a Genetic Algorithm,section 3.6 shows the performance of the classifier, section 3.7 discusses the results, andsection 3.8 propose future work and concludes the paper.

3.3 Related work

3-neighbourhood binary cellular automata has shown promising results in pattern recogni-tion. Chaudhuri et al. have done extensive work on additive cellular automata [30]. Amongtheir work we find a multi-class pattern classifier using Multiple Attractor Cellular Automata(MACA). This classifier saved on average 34% of computer memory using pseudo-exhaustivepatterns compared to the straight-forward approach of storing the class of each pattern di-rectly [29]. Maji et. al performed pattern recognition with CA:s using General MultipleAttractor Cellular Automata (GMACA). They proposed an associative memory using CAthat was found to be 33% more effective than a hopfield network [68].

The pattern recognition algorithms using (G)MACA:s are highly computationally effi-cient, but they only act as content-addressable memories and do not create generalised modelsover data. They also require that the learnt patterns are close to each other in Hamming Dis-tance [44], which is problematic since data from e.g sensors and such need to be encoded into

31

binary patterns in a suitable way (e.g. gray code [22], thermometer encoding [35] or scattercode [93]). The CAMP algorithm, however, does not require such binary encoding of inputdata due to its non-constant neighborhood configuration inspired by traditional feedforwardneural networks.

Combining principles from Neural Networks with Cellular Automata is not in itself anovel approach. Cellular Automata Neural Networks (CNN:s, note this is not to be confusedwith Convolutional Neural Networks), for example, are also neural networks using principlesfrom CA. The focus of CNN research has however mainly been to improve the accuracy ofclassification and regression. In a CNN, decimal values are used for weights and neuronoutputs like in a traditional neural network. Cells are often connected in a 2-dimensional gridwith an 8-neighbor configuration, and are updated using their neighbor’s values as inputs to anoutput function. Cells’ outputs are connected to other cells and/or the output of the algorithm.CNN:s are promising and have shown to be successful in, for example, image processing [21],fluid dynamics [62] and statistical physics [38]. The CAMP algorithm differs from a CNNin the way that each cell can connect to any r cells in the CA, not just adjacent ones, and ituses only binary values and relies on binary logic operations instead of floating point valuecalculations.

Another field of research related to the work in this paper is chaotic neural networks.There is research showing that chaotic dynamics may exist biological neurons [9, 8, 20, 37]which have led to research with using chaotic activation functions in neural networks. Thesetypes of networks can have advantages regarding computational time and memory usage,since the complex dynamics of the neurons are described by simple and deterministic differ-ence equations rather than models of differential equations or stochastic processes. [6]

3.4 The CAMP algorithm

This work tries to utilize the fact that CA works at the edge of chaos and is turing completeeven with a binary neighborhood of three cells in order to create a simplistic neural network.In traditional Cellular Automata, cells are only connected to adjacent cells, but in order tocreate the proposed algorithm, this principle is loosened to allowing cells to get their neigh-bourhood from any r = 3 cells, not just adjacent ones. Each cell’s neighbours are stored ina connection-vector and this vector is allowed to change between timesteps. In essence, thismeans a CA where the neighbourhood (now allowing any cells) for the cells change overtime.

Another way of looking at the algorithm, is considering it to be a MCP-neural network,where each neuron has r = 3 connections to neurons in the previous layer and all connectionsare of excitatory types (unit weight) only. It should, however, be noted that although theweights’ functionality in the original MCP-model may seem to be disregarded, the CA rulecan yield both inhibitory and excitatory effects on the output from a given input.

A CAMP neuron gets updated in the following two steps:

1. Determination of its neighbourhood configuration.

2. Calculation of the next state using the CA rule for that configuration.

32

In essence, the CAMP algorithm works in the same manner as a feed-forward neuralnetwork.

As can be noted from above, the update procedure of a neuron is identical to that of aCA cell. This is because they are in fact equivalent, and the algorithm can be viewed eitheras a CA with neighbourhood configurations changing over time, or as a sparsely connectedMCP-network. Through the rest of this paper, the MCP-network view will be used for thesake of clarity.

Since the aim of this paper is to keep resource consumptions at a minimum, a three layerednetwork structure was used, with an input layer, a hidden layer and an output layer. The inputlayer and hidden layer are of equal size, while the number of neurons in the output layer isequal to the number of classes to classify. To perform the classification, the output layer’spattern is compared to chosen output patterns for the different classes.

The network structure and state can be represented using only two sequences of bits. Thefirst sequence contains all connections between neurons and is created by concatenating allthe neurons’ connection-vectors using a lexicographic order. A connection-vector is simplya binary representation of what other neurons in the previous layer that a neuron is connectedto. The second sequence holds the states of all neurons and is created by concatenatingtheir binary values in the same lexicographical order as the connection-vectors’. Running theclassifier demands only the addition of another bit sequence where the neurons’ next states(updated states) will be stored.

Since each neuron is only connected to a constant number of neurons in the previous layer,the computational complexity of the algorithm is linear. The actual CPU instructions requiredto update each neuron can easily be calculated due to the simplistic structure of the classifier(assuming AND, OR and NOT are part of the instruction set and are atomic operations). Theworkload consists of the algorithm consists of evaluating the boolean expression that is theactivation function. For rule 110 this expression is:

q = (¬a∧b)∨(b∧¬c)∨(¬b∧c)Where q is the next state for a neuron with incoming connections a, b, c. In the case

of rule 110, each update requires 8 instructions which means that in order to calculate theactivation functions of n neurons we must perform 8n CPU instructions.

As we can see, the CAMP algorithm has several interesting properties that are importantfor machine learning algorithms running in resource-constrained environments:

1. It has a small memory footprint.

2. It is computationally inexpensive.

3. It does not make use of floating point values, making it faster to compute on less so-phisticated processors [34].

4. Updates of neurons are done locally, i.e. does not need access to a global state makingparallel computations possible.

5. It has a simple structure and logic making it suitable for FPGA or ASIC implementa-tion.

33

3.5 Training of the classifier

The output of the classifier depends solely on the set of connections between the cells, andtraining the classifier consists of finding the best set of connections among the nodes that willmap an input set to an desired output set. The chaotic behaviour of the CA, however, makes itproblematic to solve this mapping mathematically. Exhaustive search is not an option either,since there are C possible combinations of connections for a three layer network with I, J, Knumber of nodes in each layer:

C = ( I!r!(I− r)!)

J

( J!r!(J− r)!)

K

(3.1)

This means that even for a limited neighbourhood of r = 3, the number of permutations ofconnections escalates quickly. In order to find a connection set, we instead rely on a geneticalgorithm to converge towards a solution. This is unfortunate, since this does not guaranteefinding the optimal solution. The following sections describes the genetic algorithm’s foursteps:

1. Chromosome encoding.

2. Fitness calculation.

3. Selection.

4. Crossover and mutation.

3.5.1 Chromosome encoding

The input connections to a neuron from the previous layer, is represented using a bit string,or a connection-vector, where each bit corresponds to a neuron in the previous layer. A setbit indicates there’s a connection to the corresponding neuron. Each connection-vector isconsidered a gene, and chromosomes are created by concatenating all the connection-vectors(genes) into one binary string using a lexicographical order.

3.5.2 Fitness calculation

The fitness F for each chromosome is simply calculated by running the algorithm with thechromosome’s connections and counting the amount of correct classifications among the ntraining examples.

F = n∑1(classi f ierOut putn∧correctClassn) (3.2)

3.5.3 Selection of individuals

Roulette wheel selection was used to select mating individuals from the current populationbased on their fitness results. The fitness for each individual thus determines its probability to

34

Figure 3.4: Crossover of connection vectors

be chosen to breed with another selected individual. The 10 most fit individuals were addedto the next generation unmodified (elitism). This ensures that there can be no degrading fromone generation to the next but with the tradeoff for a less versatile next generation.

3.5.4 Crossover and mutation

When two individuals have been selected to breed, a crossover algorithm is used to createthe offspring. Crossover is performed by stepping through an empty chromosome, gene bygene, and randomly selecting which parents’ gene (node connection-vector) to choose. Thisprinciple ensures that an offspring is a combination of the parents connection-vectors whilethe connection limit r = 3 is preserved (fig. 3.4).

After the crossover the offspring is mutated with a mutation rate of 0.5%, meaning thata chromosome has a 0.5% chance of being selected to mutate. It is important that the mu-tation does not add or remove any connections from any neuron since this would break the3-neighbourhood configuration. If a chromosome is chosen to mutate, a gene from that chro-mosome is randomly selected and two bits are swapped in that gene. In essence, this meansthat one of the neurons in the neural network will randomly move one of its incoming con-nections to another neuron in the previous layer. This modifies the offspring while ensuringthat the connection limit r = 3 will still hold.

3.6 Results and evaluation

In order to evaluate the CAMP algorithm the WEKA platform was used [109]. WEKA isan open source tool for machine learning and data mining, that supports several standardmachine learning algorithms of various types along with other useful tools such as validationmethods, pre-processing, visualization, and feature selection. The CAMP algorithm wasimplemented as a classifier in WEKA since that allowed for the algorithm to be benchmarked

35

against validated implementations of other machine learning algorithms. This also makes iteasier for other researchers to put the results presented here into perspective.

Six different classification datasets that are shipped with WEKA were used in order toevaluate the algorithm’s performance (see Table 3.1). These were all the available classi-fication datasets with numeric values only. WEKA:s datasets were chosen since these arepublically available and have all been part of published scientific work, making it possiblefor researchers to compare our results with other work on the same datasets. The differentstages of the testing will now be explained.

3.6.1 Test setup

The numeric values in each training example were converted to 16-bit values, where eachbit was fed to an input neuron. The number of output neurons was set to the number ofclasses and each output neuron corresponds to a choice of a class. For example, a three classdataset would have the output configurations; 100 = class A, 010 = class B, 001 = class C. Ifmore than one output neuron is activated, the classification is regarded as unclassifiable. Thenumber of hidden neurons was set to be three times the number of output neurons due to theconnection limit r = 3.

For each dataset, the three highest ranked features were selected using InfoGain [23] toavoid overfitting. Training was performed by running the genetic algorithm for 1000 gener-ations with a population size of 100 and with a 10 individual elite. Testing was performedusing 10-fold cross validation.

3.6.2 Learning curve

A learning curve was created using the diabetes dataset in order to analyze the learning abil-ities of the algorithm. The number of training examples ranged from 10-90% of the datasetwith 10% intervals after which the curve was smoothed. It is important to note here thatthe training of CAMP is nondeterministic, due to its stochastic optimization training (geneticalgorithm). The curve, plotted in fig. 3.5, shows that the algorithm generally improves as thenumber of training examples are increased. This supports the hypothesis posed in this paper.

3.6.3 Testing of accuracy

To put the CAMP performance in perspective, five of WEKA’s standard machine learningalgorithms of different types (Bayesian, Tree, Neural Networks, Ensemble) were used as acomparison. Since this was done only to give an indicative of CAMP’s performance, all thecomparative algorithms used their default settings in WEKA in order for the experiments tobe easily repeated. This means neither CAMP nor the comparative algorithms were adapted

36

Tab

le3.1

:D

escriptionof

thetested

datasets.

Da

taset

Cla

ssesIn

stan

cesF

eatu

resselected

Descrip

tion

Crea

tor

Glass

7214

Mg,A

l,KD

atasetfor

classification

oftypes

ofglass

motivated

bycrim

inologicalin-

vestigation.

B.

Germ

an–

Central

Research

Estab-

lishmentH

ome

Offi

ceForensic

Science

Service

Alderm

aston,R

eading,B

erk-shire

RG

74P

NIonosphere

2351

a05,a06,a33R

adardata

classifyinggood

orbad

radarreturns.

Good

radarreturns

arethose

showing

evidenceofsom

etype

ofstructure

inthe

ionosphere.B

adreturns

arethose

thatdo

not;their

signalspass

throughthe

ionosphere

Space

Physics

Group

Applied

Physics

Laboratory

JohnsH

opkinsU

niversityS

igillito,V

.G

.,W

ing,S

.P.,

Hutton,

L.

V.,and

Baker,K

.B.(1989)

Segm

ent-C

hallenge7

1500Intensity-m

ean,raw

red-mean,

rawgreen-m

ean

The

instancesw

eredraw

nrandom

lyfrom

adatabase

of7

outdoorim

ages.T

heim

agesw

erehand

segmented

tocreate

aclassifi

cationfor

everypixel.

Each

instanceis

a3x3

region

Vision

Group,

University

ofM

as-sachusetts

Labour

257

Wage-increase-

first-year,

wage-increase-

second-year,statutoru-holidays

Data

includesall

collectiveagreem

entsreached

inthe

businessand

personalservices

sectorfor

localsw

ithat

least500

mem

bers(teachers,nurses,univer-

sitystaff,

police,etc)

inC

anadain

87and

firstquarter

of88.

Collective

Barganing

Review

,m

ontlypublication,

Labour

Canada,

IndustrialR

elationsInform

ationS

ervice,O

ttawa,

Ontario,K

1A0J2,C

anada

Diabetes

2768

plas,mass,age

Adiagnostic,

binary-valuedvariable

isinvestigated

toshow

whether

thepa-

tientshow

ssigns

ofdiabetes

accordingto

World

Health

Organization

criteria.

National

Instituteof

Diabetes

andD

i-gestive

andK

idneyD

iseases,V

incentS

igillito(vgs@

aplcen.apl.jhu.edu)R

e-search

Center,

RM

IG

roupL

eaderA

p-plied

Physics

Laboratory

The

JohnsH

opkinsU

niversity

37

Tab

le3.2

:T

hecl

assi

fica

tion

resu

lts

for

the

algo

rith

ms

onth

edi

ffer

entd

atas

ets.

CA

MP

Nai

veB

ayes

Bay

esN

etM

ulti

-lay

ered

Per

cept

ron

K-s

tar

J48

Ran

dom

Fore

stG

lass

32.2

4%

49.0

6%63

.55%

63.0

8%64

.49%

64.4

9%67

.29%

Iono

sphe

re85.4

7%

86.3

2%89

.46%

87.4

6%89

.17%

90.6

0%90

.03%

Seg

men

t-ch

alle

nge

41.9

3%

60.6

0%70

.53%

78.4

7%76

.20%

81.0

0%86

.00%

Lab

our

61.4

0%

84.2

1%78

.90%

84.2

0%80

.70%

80.7

0%84

.20%

Unb

alan

ced

98.6

0%

98.6

0%98

.60%

98.6

0%98

.60%

98.6

0%97

.90%

Dia

bete

s73.4

4%

76.4

3%74

.22%

76.4

3%72

.40%

74.6

1%72

.53

%

38

Figure 3.5: Learning curve for diabetes dataset.

in any way to fit the datasets and it is therefore important to note that the results presented hereare not to be considered the best performance possible for neither CAMP nor the comparativealgorithms. The results from the different classifiers are shown in Table 3.2.

3.7 Discussion

The learning curve and the results in Table 3.2 show that the CAMP algorithm works as aclassifier and with results comparable to those of some conventional algorithms for someof the datasets. This is promising and shows that the algorithm could prove beneficial, atleast for certain kinds of classification problems. The datasets for which the algorithm hasthe highest performance have in common a low number of classes. Similarly, the algorithmperforms poorly on datasets with a higher amount of classes. This suggests there could bea connection between the performance and number of classes in the classification problemand should be studied further. It is possible, for example, that reducing the single multiclassclassification problems into multiple binary classification problems (similar to e.g. multiclassSupport Vector Machines [108]), could improve the accuracy of the algorithm.

Since no efforts were made to setup the algorithm to fit each dataset, the accuracy pre-sented here could most likely be improved. By using different feature selections and chang-ing parameters such as bit resolution, the algorithm could be better matched against a certaindataset. Also, since the number of output neurons is limited to the number of classes andthere’s only one output pattern for matching a class, the solution space for the input-outputmapping is limited. It is therefore possible that increasing the number of output neuron’s andallowing for multiple output patterns per class could improve the accuracy, since that wouldallow for a higher number of solutions for reaching each class. Using expanded output pat-terns would also make it possible to use techniques, such as the hamming distance, to findthe closest match for a non-matching pattern instead of discarding it.

39

Another possible way of increasing the accuracy of CAMP would be to use an expandedneighbourhood size in order to allow for more complex calculations. This would, however,result in both a larger memory footprint and a higher computational complexity, since thenumber of connections between neurons would increase. Using a deeper network, with twoor more hidden layers, could also allow for more complex calculations, but with the sametradeoff as an expanded neighborhood.

A final way that could improve the classification performance, would be to use differentrules. The CA rule 110 was chosen for this work based on its ability of universal calculation,but further studies of the algorithm should include the use of other rules as well. Anotherinteresting study would be to expand the algorithm to use a non-uniform cellular automata.Each neuron would then have its own update rule, thereby expanding the solution space forthe input-output mapping. This would, however, increase the memory usage of the algorithmand could increase the number of atomic operations needed to update the neurons.

Even if the general performance of CAMP would prove to be inferior to existing algo-rithms, it does not mean the algorithm is not useful. CAMP has proven to have a goodaccuracy for some datasets, while being extremely efficient in terms of memory and CPUusage. This makes it a viable choice for applications that have a higher demand for low re-source consumption than for a high classification accuracy. CAMP could also prove usefulas a pre-processing stage for more advanced algorithms in order to save resources.

3.8 Conclusions and future work

This paper proposes a binary-valued classifier based on a combination of Cellular Automataand the McCulloch-Pitts neuron model. The classifier is designed to be computationallyinexpensive and highly suitable for mobile phones, microcontrollers, FPGA:s or ASIC:s.Evaluation was performed using six different datasets with their three highest ranked featuresselected using InfoGain. Results show that the classifier performed better for the datasetstested with a low number of classes, while performing worse on datasets with a higher amountof classes.

Future work should include studying how reducing the single multiclass classificationproblems into multiple binary classification problems affect the accuracy of the algorithm.Another study would be to allow a higher amount of connections between neurons, as thisallows for more complex input-output calculations. Furthermore, the use of Hybrid CA’s(each neuron/cell having its own update rule) could also be studied, but both these studieswould make the algorithm itself more complex and thus less suited for the initial purpose.

Part 4

Analyzing Body Movements

within the Laban Effort

Framework using a Single

Accelerometer

41

43

4.1 Introduction

Analyzing Body Movements is receiving an increasing amount of attention from context-awareness researchers. This attention is motivated by the wide range of applications thatrely on the analysis of human motion. For instance, analysis of dancer/athletic performance,medical diagnosis [94], and recognizing emotions based on the movements of the body [14].The movements of the body usually form a synchronized pattern when performing activities.Those patterns can be analyzed and described in the framework of Laban Movement Analysis(LMA). LMA is a method for describing and interpreting all varieties of human movements.It provides a rich overview of movement possibilities, and it is considered as “a formal lan-guage for movement description” [19]. LMA is divided into four categories: Body (total-body connectivity), Effort (Energetic dynamics), Shape, and Space [2]. This work focuses onanalyzing body movements with regard to the Effort category, which is also divided into foursubcategories: Strong—Light, Sudden—Sustained, Bound—Free and Direct—Indirect [64].

It is demonstrated in [70] that even an expert cannot categorically determine whether amovement is Strong, Light, Sudden, Sustained, Bound, Free, Direct or Indirect. This classifi-cation may even vary from one expert to another. It is therefore important to define each typeof movement beforehand, so it is possible to choose the right activities that represent eachmovement when collecting data. The authors have defined the types of movement within thisstudy as follows:

• Strong: A movement is considered to be Strong when a person needs to make a con-siderable effort to perform an activity.

• Light: A Light movement is such that a person could perform the activity effortlessly.

• Free: A movement is considered to be Free when it is characterized by open postureswhere the extremities of the body, mainly upper body limbs, are kept mostly away fromthe body.

• Bound: A Bound movement is a controlled movement performed with the extremitiesclose to the body.

• Sudden: A Sudden movement is a swift movement that does not follow any particularpattern. It generates a change in velocity, that is, a spontaneous acceleration.

• Sustained: A Sustained movement is a continuous movement that follows a specificpattern where the velocity is maintained.

• Direct: A movement is considered to be Direct when the route a person follows over acertain period of time is on average a straight path.

• Indirect: A movement is considered to be Indirect when a person follows, over a certainperiod of time, an oblique route.

Naturally, we perform all these movements during our daily activities. For instance, mostpersons walk effortlessly, and the required movement is considered to be Light, but carrying

44

heavy objects while walking will increase the activity’s Effort level, and the movement wouldbe considered as Strong. To recognize activities of daily living, many researchers have usedwearable sensors for the task of human activity recognition. In particular, machine-learningtechniques have been utilized for the purpose of using accelerometers to detect daily activitiessuch as walking, running, sitting and lying [112, 31, 13, 17]. The small size of accelerome-ters and their low power consumption make them well suited to wearable applications [26].However, the purposes of the classified data from movement analysis and activity recognitionare different. The primary purpose of movement analysis is to determine the movement ef-fort, either for the use on its own or to be combined with other contexts to clarify the currentsituation. For instance, differentiating between strong exercise and strong emotions when themovement classification is coupled with a galvanic skin response sensor that measures thesubject’s stress [104]. On the other hand, the primary purpose of classical activity recogni-tion classification is to gain direct insight into the specific type of activity.

In this article, the authors present an experiment to categorize the body movements ofthe subject using wireless tri-axial accelerometers placed at the chest, wrist and thigh. Thoselocations have shown positive results for detecting activities of daily living in [31, 47, 12, 82].Figure 4.1 below illustrates the placement of the accelerometers at the chosen locations.

Figure 4.1: Selected placement locations for the accelerometers (chest, wrist and thigh). Thewrist and thigh sensors are placed at the dominant side of the body

45

Having multiple sensors will increase the complexity of the monitoring system and makeit more cumbersome for the subject [48]. The authors investigate the best location betweenchest, wrist and thigh, to place a single accelerometer for the purpose of detecting each typeof movement. The aim of the work is to answer the following research questions:

1. What level of accuracy can be achieved in detecting body movements within the Effortcategory using a single accelerometer?

2. Which are the best machine-learning techniques and the best placement for an ac-celerometer to accurately classify each type of movement within the Effort category?

The results of the presented work in this article will give an indication of how to estimatethe physical level of the body movements. This estimation can be employed in differentapplications. For instance, physiotherapists can get an estimation of the body movements’level of their patients throughout the therapy, and dance teachers can get an estimation of thebody movements’ level of their students while dancing. Note that this work does not cover theclassification of the Direct and Indirect elements within the Effort category. Those elementsgenerally require a non-accelerometer external subsystem in order to capture them, such asGPS subsystem.

The rest of this paper is organized as follows: Section 4.2 presents state-of-the-art re-lated work. Section 4.3 discusses the data collection, which includes the chosen activitiesto represent each type of movement, and the data collection process by the participants ofthe study. Section 4.4 presents how the collected data has been processed, this includes thefeatures that have been extracted and selected, and the classification models that have beenbuilt and tested. Section 4.5 shows the obtained results, and section 4.6 discusses the researchquestions. Finally, Section 4.7 concludes the paper and presents future work.

4.2 Related Work

Some previous studies classified movements performed by subjects within the Laban Move-ment Analysis framework (LMA). For example, Fagerberg et al. [40] classified the bodymovements within LMA to find the connections between the mental state of the subject andthe movements being performed. The traditional methodology was employed for this pur-pose based on the observation of the movements by either a Laban expert [70] or movementexperts, such as choreographers or expert dancers [27]. Some researchers have tried to cap-ture the movements of individuals using the human interaction with a system. Mentis et al.[70] used video data captured by a Kinect camera to analyze the movement qualities. Themovement qualities were calculated based on acceleration, pathways, velocity, levels and re-lationship of limbs to the body. The movements captured from the subject were contrastedwith the opinion of various Laban experts for the purpose of seeing whether the system wasable to recognize the movements or not. The study provided indications of how movementqualities can be detected using a static video camera, and how these qualities can be inte-grated into the design of interactive systems. However, this system showed weakness whenthe recognition of the movements is conducted in a real world situation, since the system is

46

not portable and it requires a controlled environment. Another study presented by Foroudet al. [43] that focused on analyzing the movements of rats instead of human beings. Themovements created by rats when interacting with each other have been collected and storedin videotape. Using the traditional methodology, the videotape was analyzed and the move-ments were classified within the LMA Effort factors.

Godfrey et al. [48] presented a review about measuring the human movements by theuse of accelerometers. The reason of focusing on accelerometers was the low cost, weightand power consumption. Other techniques were discussed for collecting the movements suchas the use of diaries, questionnaires or observation. These techniques were rejected becausethey present disadvantages when a continuous analysis of the movements is required. Thereview presented a compromise between the number of accelerometers and the data obtainedfrom them. The authors in [48] affirmed that a smaller number of accelerometers makes themonitoring of movements less complex, whereas the amount of information is reduced. Theresults also showed that the use of accelerometers is a good way to collect movements. Inaddition, the authors concluded that accelerometry is a non-intrusive mean to access ambu-latory movements, postures, postural transitions and intensity of movements. Another studywas done by Veltink et al. [103] to distinguish between postures and movements of the body.The authors used two uni-axial accelerometers placed at the trunk and the upper legs for thedetection purpose. The study showed that it is possible to distinguish between postures andmovements by simply checking the sensors signal. If the signal is not changing over time, itcan be assumed that the body segment is not moving. However, the discrimination betweenvarious movements was not analyzed.

Most of the previous works in this field have focused on the use of accelerometers forthe purpose of recognizing daily activities. The method that was generally implemented wasto collect data using accelerometers and then use machine-learning techniques to processit. For instance, activity recognition has been done in [31, 17, 47, 12, 82]. Some of thesestudies use tri-axial accelerometers [31, 47, 12, 82] placed at different parts of the body suchas chest, wrist, thigh, waist and ankle, while others use bi-axial accelerometers [17]. Olguinand Pentland [78] used accelerometers attached to the chest, hip and wrist to detect activitiesof daily living and the accuracy rate was up to 92%. Another study done by Ravi et al.[82] used a single accelerometer attached to the waist to detect a range of activities and theaccuracy of detection was up to 64%. One major topic that is discussed in related works isthe optimal placements when using accelerometers. Cleland et al. [31] investigated the useof five tri-axial accelerometers placed at the chest, wrist, lower back, hip, thigh and foot todetect seven activities of daily living. Data collected from the hip yielded the best resultsfor detecting activities using a Support Vector Machine classifier. Bao and Intille [17] usedfive biaxial accelerometers worn on the user’s right hip, dominant wrist, non-dominant upperarm, dominant ankle and non-dominant thigh when collecting data. Their work suggestedthat using an accelerometer at the dominant wrist and thigh may be able to detect commoneveryday activities. Different sensor locations have been studied by for example Cleland etal. [31], Gjoreski et al. [47] and Atallah et al. [12]. These studies analyzed the differentlocations and their performances as a data source for detecting a set of everyday activities.

The authors in this article investigate the possibility of classifying body movements withinthe Laban Effort category using a single accelerometer. To the best of the authors’ knowledge,

47

this was not investigated before. In addition, the presented work discusses the sensors’ place-ment for the purpose of detecting body movements instead of activities, which complementsand expands previous works in this field.

4.3 Data Collection

4.3.1 Definition of Activities

It may be sufficient to detect the intensity of the performed activities in order to indicate thetype of movement included. A set of activities is defined for each type of movement withinthe Effort category. The subject is expected to perform some actions during each activity andthose actions will give an indication about the body movements. The total number of chosenactivities is 23. Some of the activities were repeated for different types of movement withslight variations in the way the activity was performed.

Sustained, Light, Bound and Strong Activities

For the first three elements (Sustained, Light and Bound), the subject was asked to performseven activities, namely walking, running, stairs up, stairs down, sitting, lying and standing.Each activity was performed for one minute. The element Strong contains a variation ofsome of the mentioned activities. For Strong movements, the subject was asked to carrysome heavy objects in a box and also in a backpack while performing walking, running,stairs up and stairs down. The purpose is to increase the activity’s Effort level so the subjectwould feel it as a “strong activity”. In addition, the subject was asked to perform a cyclingactivity with first gear for Light, second gear for Sustained and Bound, and third gear withheavy items for Strong.

Sudden Activities

Each activity performed in this element should be comprised of a series of sudden move-ments. Each activity was performed for one minute. The defined activities are the following:

• Finding a cell phone: A cell phone was hidden in a room. The phone was called andthe subject had to find it. When doing this activity the subjects perform abrupt bodymovements, particularly involving the upper body, in their haste to find the cell phone.

• Putting shoes on and taking them off: The subject produces swift and non-patternedmovements, especially hand movements.

• Getting dressed: the subject was asked to put on a winter cap, a pair of gloves, a scarfand a coat, and then take them off. The movements a subject performs while puttingon each item are fast during a short period of time.

• Simon says, a kid’s game: In this game the researcher stated an activity (i.e., walk-ing/running) and the subject has to perform it. The intent is to alternate rapidly between

48

the activities performed and therefore, generating sudden movements. These activitieswere: Walking, running, jumping, hopping right leg, hopping left leg and stopping.

• Cleaning: In an open room, some trash was placed on the floor and on a table so thesubject would have to sweep the floor and use a cloth to clean the table. Some noteswere written on a whiteboard so that with the help of an eraser, it had to be cleaned bythe subject. The subjects produced sudden movements due to the nature of the activityof cleaning.

• Making a sandwich: a few ingredients were scattered around a room to create sud-den and non-patterned movements. The subjects were asked to make a sandwich bychoosing the ingredients they saw fit.

Free Activities

Three activities were defined for this element. The activities were chosen because they gen-erate a series of free movements when performing them:

• Dancing: For this activity each subject was granted a private space in an effort to notmake the individual feel self-conscious and therefore negatively impacting the datacollection of this activity. Dancing has twice the samples the rest of activities do (twominutes), in order to cover some of the sample shortage in this category.

• Running like “Homer in Land of Chocolate”: This combination of activities was ex-tracted from “The Simpsons” TV show where one of the main characters was gener-ating free movements throughout the activity. It is an activity in between running andjumping as it is shown in Figure 4.2 This activity was performed for one minute.

• Walking like “Sound of Music”: This activity is based on the beginning of the classicfilm “The Sound of Music”. The idea is to collect free movement as shown in Figure4.3 This activity was performed for one minute.

Figure 4.2: Homer in the Land of Chocolate.

49

Figure 4.3: The Sound of Music.

4.3.2 Collecting Data

Ten healthy subjects participated in the data collection. The participants were students atLulea University of Technology. Information about the subjects is presented in Table 4.1below.

Table 4.1: Subjects’ information

Attribute Mean Standard DeviationHeight (m) 1,75 0,07Weight (kg) 70,97 4,03Age (years) 27,20 4,80Right handed 1,00 0,00Body Mass (kg/m2) 23,11 1,95

GenderFemale 4Male 6

The subjects were equipped with three Shimmer wireless sensors (Shimmer 2R, RealtimeTechnologies, Dublin, Ireland) placed at the chest, thigh and wrist, as shown in Figure 4.1.The Shimmer sensors were fixed to the body using elasticized strapping and holsters. Thisis a common method of attachment in activity recognition studies [26]. The thigh and wristsensors were placed depending on whether the subject was left handed or right handed (inthe dominant side of the body), and also aligned to the chest sensor. The subjects were alsoequipped with a smartphone using a sport’s armband on the non-dominant arm, and with astopwatch to notify them when to stop performing the activity. The shimmer sensor allowsranges of acceleration of +-1.5 g and +-6 g. It is stated in [39] that +-2 g was insufficient todetermine vigorous exercise, while other studies have shown that the use of +-6 g accelerationrange can be sufficient [114, 113]. The range of acceleration was therefore set to +-6 g. Thesampling rate of the Shimmer sensor was set to 10 Hz, which is considered to be sufficientfor detecting daily activities from accelerometer data [12, 82].

50

A video of 3 min was prepared beforehand to demonstrate the activities that the subjecthad to perform. The focus of the video was on the free movements as it is not common toperform such movements on a daily basis. The video was played to each subject separately,and then the subject was asked to perform the 23 activities of one minute each (except for thedancing activity, which is performed for 2 min). The data was transmitted via Bluetooth tothe mobile device that the subject had attached to the arm, where it was saved for later offlineanalysis. The raw acceleration data was labeled after each activity based on the performedmovement. The subjects were asked from time to time to give feedback on the position ofthe sensors on their bodies. In addition, the position of the sensors was checked after eachactivity, and a correction was made when needed. The list of the performed activities by eachsubject is summarized and shown in Table 4.2 below. The number of samples refers to theones collected from each location (chest, wrist and thigh).

4.4 Data Processing

4.4.1 Feature Extraction

As shown in Table 4.2, the total number of acceleration samples was around 23,400 per sub-ject from each location. Having 10 subjects produces around 234,000 samples for all typesof movement from chest, wrist and thigh. Features were extracted from the raw accelerationdata using a window size of 128 samples with 64 samples overlapping between consecutivewindows. This represents 12.8s of data per window. A 50% window overlap has been deemedsufficient to compute the features [17], and the window size was selected in a trade-off be-tween fast computation of the data and the ability to accurately determine cyclic movements[112, 24, 57, 80]. Eighteen features were extracted from each window, giving a total of 34attributes. Table 4.3 below presents all the extracted features.

The extracted features are a combination of time-domain (features 1 to 14) and frequency-domain (features 15 to 18). Time-domain features grant the possibility of differentiating dy-namic movements from static ones [47, 80]. Frequency-domain features are essential foridentifying patterns within acceleration data, which aids in discriminating vigorous frommoderate movements [17, 107].

Features 1–14 are standard statistical metrics. For feature 15, it has been concluded in [80]that FFT components have a greater assessing ability than other features. The authors in [80]analyzed which number of FFT components maximizes the classification accuracy attained,where the first five FFT components were used. Feature 16 (Spectral energy) is calculated asthe sum of the squared FFT components within the chosen window. Afterwards, the resultobtained is normalized employing the window length. This feature is particularly relevantwhen assessing the energy expenditure of a subject while moving [31]. For the last feature(feature 18), it is well known that when a subject performs an activity, a certain range of

51

Tab

le4.2

:A

ctiv

itie

spe

rfor

med

byea

chsu

bjec

t

Lab

an

Ele

men

tA

ctiv

itie

sT

ime

(min

)

per

act

ivit

y

Sam

ple

sp

er

act

ivit

y(a

p-

pro

x.)

Tota

lsa

mp

les

per

elem

ent

from

each

loca

tion

(ap

pro

x.)

Sud

den

Fin

ding

ace

llph

one

-P

utti

ngsh

oes

onan

dta

king

them

off

-G

etti

ngdr

esse

d-

Sim

onsa

ys–

Cle

anin

g-

Mak

ing

asa

ndw

ich

160

036

00

Sus

tain

edW

alki

ng–

Run

ning

–S

tand

ing

–S

itti

ng-

Sta

irs

up-

Sta

irs

dow

n–

Lyi

ng-

Cyc

ling

gear

2

160

048

00

Str

ong

Car

ryin

gH

eavy

Stu

ffan

da

back

-pa

ckan

dpe

rfor

min

g:W

alki

ng–

Run

ning

–S

tair

sup

-S

tair

sdo

wn

–C

ycli

ngge

ar3

160

030

00

Lig

htW

alki

ng–

Run

ning

–S

tand

ing

–S

itti

ng-

Sta

irs

up-

Sta

irs

dow

n–

Lyi

ng-

Cyc

ling

gear

1

160

048

00

Fre

eD

anci

ng-

Run

ning

like

"Hom

erin

the

Lan

dof

Cho

cola

te"

-W

alki

ngli

ke"T

heS

ound

ofM

usic

"

1,ex

cept

2fo

rda

nc-

ing

600,

exce

pt12

00fo

rda

ncin

g24

00

Bou

ndW

alki

ng–

Run

ning

–S

tand

ing

–S

itti

ng-

Sta

irs

up-

Sta

irs

dow

n–

Lyi

ng-

Cyc

ling

gear

2

160

048

00

Tota

ln

um

ber

of

sam

ple

sp

ersu

bje

ctfo

rall

6el

emen

tsfr

om

each

loca

tion

23400

52

Table 4.3: Features extracted from each window of raw acceleration data

Feature no. Feature description1 Acceleration Vector Magnitude (Length) value over 3 axes2 Mean value for each axis (x, y, and z)3 Average Mean value over 3 axes4 Mean value over Length attribute5 Root Mean Squared (RMS) value for each axis (x, y, and z)6 Average RMS value over 3 axes7 RMS value over Length attribute8 Standard Deviation (STD) value for each axis (x, y, and z)9 Average STD value over 3 axes10 STD value over Length attribute11 Skewness value for each axis (x, y, and z)12 Average Skewness value over 3 axes13 Kurtosis value for each axis (x, y, and z)14 Average Kurtosis value over 3 axes15 First 5 Fast Fourier Transform (FFT) value for each axis (x, y, and z)16 Spectral Energy value for each axis (x, y, and z)17 Average Spectral Energy value over 3 axes18 Principal Frequency value for each axis (x, y, and z)

frequencies is generated [57]. Making use of the main frequency feature may prove useful indetermining various types of movements.

4.4.2 Feature Selection

To avoid redundant or irrelevant features, feature selection was done over the set of the ex-tracted features. A filter method (InfoGain) was used to obtain the best set of features thatprovides the highest accuracy and minimizes overfitting [55]. To do that, the collected datawas first organized based on the location of the accelerometer and the subcategory withinLMA. For instance, acceleration data collected from the chest sensor labeled as Strongor Light (for all 10 subjects) was grouped together in 1 file. This was repeated for eachlocation (chest, wrist, thigh) in combination with each subcategory (Strong—Light, Sud-den—Sustained, Bound—Free). This procedure created nine files that represent (location-subcategory).

InfoGain method creates a ranking of the most relevant features based on the informa-tion obtained with respect to the class evaluated [109]. For each combination (location-subcategory), the 34 attributes were ranked using the InfoGain method. The features werethen divided into different subsets recursively based on the previous raking using the Bisec-tion algorithm [95]. Those subsets were tested later using the classification models to findthe subset that gives the best accuracy. All attributes were used if there was no subset thatgave better results. Finding a subset with the most relevant features increases the accuracyand decreases the computation time.

53

4.4.3 Classification models

Eight machine-learning classifiers have been evaluated based on their classification accu-racy: Naïve Bayes [47, 82], C4.5 [26, 82, 69], Logistic Regression [69], Support Vector Ma-chine (SVM) [39, 67], K-Nearest Neighbour [12, 80, 67], Random Forest [67, 73], Boosting[82, 67] and Bagging [67, 73]. These classifiers have been implemented in previous works,and they have shown good results that vary based on the nature of the study. The WEKAdata mining software (Version 3.6.7, University of Waikato, Hamilton, New Zealand) hasbeen used to build the classifiers. WEKA supports several standard data mining tasks, morespecifically, data pre-processing, clustering, classification, regression, visualization, and fea-ture selection [49].

The extracted features were tested with the chosen classifiers to select the best featuresfor each combination (location-subcategory). After selecting the best set of features for eachcombination, Leave-one-subject-out cross validation (LOOCV) [17] was used to train eachclassifier on nine subjects and uses the subject excluded for testing. This procedure was re-peated ten times (one per subject) excluding a different subject each time. The average ofthe results was calculated to give the accuracy of the classifier in detecting the right move-ment. Leave-one-subject-out cross validation guaranties that there will be no overlappingdata between the training set and the test set, which will generally give more realistic results.

4.5 Results

The eight classifiers were tested using Leave-one-subject-out cross validation (LOOCV) foreach subcategory within the Effort category. The collected data by the 10 subjects was usedfor training and testing each classifier. The level of accuracy was different among variousclassification methods and locations. Table 4.4 presents the results for (Strong—Light), Ta-ble 4.5 for (Sudden—Sustained) and Table 4.6 for (Bound—Free). Each row in each tablerepresents the level of accuracy obtained when using a single accelerometer placed at a spe-cific location on the body.

The best classifier for each location, in terms of accuracy, is marked with bold text in theprevious tables. In addition to the classification accuracy, the F-measure value was calculated.The F-measure was used as a performance index to evaluate how reliable these results are. Itis a combined metric that combines precision and recall as presented in Equation 4.1:

F −measure = 2 ⋅ precision ⋅ recall

precision+ recall(4.1)

Precision is the proportion of instances which truly have class x among all those whichwere classified as class x. For instance, a precision 0.85 means that 85% of the returneddocuments were relevant. Recall is the ratio of relevant documents found in the search resultto the total of all relevant documents. A higher value of recall indicates that relevant docu-ments are returned more quickly [109]. Ideally the best classifier would give an accuracy of

54

Tab

le4.4

:C

lassification

accuracyfor

(Strong—

Light)

usingL

OO

CV

Naïve

Bayes

C4.5

Logistic

Regression

SV

MK

-Nearest

Random

ForestB

oostingB

agging

Chest

64.12%70.39%

68.94%68.35%

66.00%72.35%

62.25%70.52%

Wrist

74.24%79.92%

77.85%78.17%

78.94%83.05%

79.53%79.62%

Thigh

60.77%72.13%

66.55%64.85%

66.84%73.78%

63.85%76.34%

Tab

le4.5

:C

lassification

accuracyfor

(Sudden—

Sustained)

usingL

OO

CV

Naïve

Bayes

C4.5

Logistic

Regression

SV

MK

-Nearest

Random

ForestB

oostingB

agging

Chest

78.15%82.85%

86.89%87.46%

85.15%87.51%

84.44%84.55%

Wrist

78.14%79.69%

81.66%82.26%

83.39%84.34%

78.84%78.25%

Thigh

49.20%71.87%

66.03%64.78%

75.39%73.97%

72.45%73.86%

Tab

le4.6

:C

lassification

accuracyfor

(Bound—

Free)

usingL

OO

CV

Naïve

Bayes

C4.5

Logistic

Regression

SV

MK

-Nearest

Random

ForestB

oostingB

agging

Chest

80.94%81.15%

85.13%85.46%

84.27%85.81%

81.99%83.19%

Wrist

76.87%80.63%

87.19%87.31%

82.33%86.20%

83.04%84.58%

Thigh

69.73%72.63%

79.70%80.05%

75.98%80.23%

80.31%74.16%

55

100% and an F- measure value of 1. Table 4.7 below presents the best classifiers based onthe previous tables, including the F-measure values. In light of these results, Table 4.8 belowsummarizes the best placement of an accelerometer for detecting each subcategory within theEffort category.

Based on Table 4.7 above, it is also possible to calculate the average accuracy of each lo-cation for detecting all subcategories within the Effort category. Table 4.9 shows the averageaccuracy that can be obtained from a single accelerometer for detecting all subcategories.

4.6 Discussion

This section discusses the results of the efforts with respect to the research questions. Firstly,it is important to note that even though the aim of LMA is to standardize the classificationof human movement, there is room for interpretation within this framework. Differentiat-ing between Strong—Light, Sudden—Sustained, and Bound—Free movements is a matterof subjective opinion. While the baseline may differ between individuals, the difference be-tween movement types should be easily separable. Making the baseline configurable shouldtherefore reveal quantifiable thresholds that would allow differentiating between movementtypes. The presented results in this article discuss the analysis of human movement basedon the definition of movements, listed in Section 4.1, and the definition of activities, listed inSection 4.3.1. These results might differ if an expert has a different opinion of what each typeof movement could be. In addition, movement-based recognition differs from the commonactivity-based recognition systems. There are many examples of applications that can benefitof using LMA instead of using activity-based recognition systems. While an activity-basedsystem focuses on the exact activity performed by the subject, LMA can give an overview ofthe subject’s state and situation by analyzing the movements of the body. For instance, ana-lyzing body movements of patients with physical injuries to help doctors follow the progressof the therapy.

The introduction section presented two research questions that have been discussed andaddressed in this article. The first addressed question is: “What level of accuracy can beachieved in detecting body movements within the Effort category using a single accelerome-ter?”

As demonstrated in Tables 4.4, 4.5 and 4.6, the level of accuracy differs between classi-fication methods and locations. In fact, the contents of these tables answer the first researchquestion. For example, it is presented in Table 4.4 that the level of accuracy for detectingStrong—Light from the chest sensor range from 62.25% to 72.35% depending on the usedclassifier. This range is different when the aim is to detect other subcategories within the Ef-fort category. As shown in Tables 4.5 and 4.6, the level of accuracy obtained from the chestsensor range from 78.15% to 87.51% for Sudden—Sustained and from 80.94% to 85.81%

56

Tab

le4.7

:T

heaccuracy

andthe

F-m

easurevalue

forthe

bestclassifiers

foracceleration

dataobtained

fromeach

location

Strong-L

ightS

udden-Sustained

Bound-F

reeB

estclassifier

Accuracy

F-m

easureB

estclassifier

Accuracy

F-m

easureB

estclassifier

Accuracy

F-m

easureC

hestR

andomForest

72.35%0.722

Random

Forest87.51%

0.875R

andomForest

85.81%0.8559

Wrist

Random

Forest83.05%

0.8259R

andomForest

84.34%0.8422

SV

M87.31%

0.8694T

highB

agging76.34%

0.7566K

-Nearest

75.39%0.7462

Boosting

80.31%0.7877

Tab

le4.8

:B

estlocationto

placea

singleaccelerom

eterto

detectsubcategoriesw

ithinthe

Effortcategory

Strong—

Light

Sudden—

Sustained

Bound—

Free

BestL

ocationW

rist(R

andomForest

83.05%,

F-m

easure=

0.8259)

Ch

est(R

andomForest

87.51%,F

-measure

=0.875)

Wrist

(SV

M87.31%

,F

-m

easure=

0.8694)

57

Table 4.9: The average accuracy for each location to detect all subcategories within the Effortcategory

Strong—Light Sudden—Sustained Bound—Free Average accuracyChest 72.35% 87.51% 85.81% 81.89%Wrist 83.05% 84.34% 87.31% 84.90%Thigh 76.34% 75.39% 80.31% 77.35%

for Bound—Free, respectively. It is also noted that the worse/best classifier differs dependingon the selected location and the target movement. The achieved level of accuracy leads toanswer the second research question.

The second addressed research question in this article is: “Which are the best machine-learning techniques and the best placement for an accelerometer to accurately classify eachtype of movement within the Effort category?”

As it is presented in Table 4.7, the Random Forest classifier yielded the best results forthe chest sensor in detecting all subcategories. The Random Forest was also the best classifierfor the wrist sensor in detecting Strong—Light and Sudden—Sustained, while the best onefor detecting Bound—Free movements is the SVM classifier. Even though SVM gave betterresults than Random Forest in detecting Bound—Free from the wrist data, the differencewas small with a slight difference (87.31%, F-measure = 0.8694, for SVM, and 86.20%, F-measure = 0.8687, for Random Forest). It is therefore possible to use the Random Forestclassifier with the chest and wrist sensors for detecting all subcategories within the Effortcategory. However, the best classifiers to classify data from the thigh sensor were Baggingfor Strong—Light, K-Nearest for Sudden—Sustained and Boosting for Bound—Free. Thechoice of the classifier should therefore depend on the location of the accelerometer and thetarget movement. In addition, the classifiers’ level of accuracy gives an indication of which isthe best/worse suitable location to place an accelerometer for the purpose of detecting bodymovements.

For the best accelerometer placement between (chest, wrist and thigh), it is shown inTable 4.8 that the thigh location did not yield a best result for detecting any of the movements.The wrist was the best location for detecting Strong—Light and Bound—Free movements.However, the chest sensor gave the best results for detecting Sudden—Sustained. As a result,a single accelerometer should only be placed at the chest if the aim of the study is to detectSudden—Sustained, and at the wrist for the other two subcategories. However, the wristwould be the best place if the aim were to detect each one of the subcategories simultaneouslywithin the Effort category using a single accelerometer. This is shown in Table 4.9, as thewrist data gave the best average accuracy for detecting all subcategories simultaneously, witha value of 84.90%. In conclusion, the Random Forest classifier would be the best choicewith an accelerometer placed at the wrist for the aim of detecting body movements within theEffort category.

The study shows that it is feasible to use a single accelerometer to analyze body move-ments within the Laban Effort Framework. This result is good because using a single ac-celerometer will reduce the burden of carrying multiple sensors in wearable systems, which

58

makes such systems easier to use. For instance, a lifelogging system can utilize a single ac-celerometer placed at the wrist to analyze the body movements of the subject. This analysiscan contribute into logging the life of the subject using a minimal set of equipment.

4.7 Conclusions and Future Work

This paper investigated the level of accuracy obtained when classifying body movementswithin the Effort category of Laban Movement Analysis using a single accelerometer. Forthis purpose, eight classifiers were tested on three separate body locations. Results showthat in order to separately classify each subcategory (i.e. Strong—Light) within the Effortcategory, both the best location and classifier vary from one subcategory to another. Thewrist placement and the Random Forest classifier are the “best location” and “best classifier”respectively, since they are both the best at classifying body movements in two of the threesubcategories. The results also demonstrate that the most successful classification is achievedfor Sudden—Sustained, followed by Bound—Free and then Strong—Light.

The obtained results show that body movement can be classified within three subcat-egories of the Laban Movement Analysis Effort category with reasonable accuracy whenusing a single accelerometer. Future work should investigate this classification in an un-controlled environment, where the subject would perform the movements naturally withoutinstructions. Future work should also investigate the classification of body movements withinthe Direct—Indirect subcategory that has been left out of this study. For this purpose, an ad-ditional GPS subsystem should be employed. It is also important to focus on implementingthe presented results within a lifelogging system that can take advantages of the analysis ofbody movements using a single accelerometer.

4.8 Acknowledgments

The authors would like to thank the Dem@care project (www.demcare.eu) for funding partof this work. The Dem@care project has received funding from the European Community’sSeventh Framework Programme (FP7/2007-2013) under grant agreement 288199. The au-thors also thank the EIT ICT Labs for supporting this work through the HWB Social Wellbe-ing activity. The authors would like to acknowledge the researchers at the Swedish Instituteof Computer Science (SICS) for their support and encouragement of this work.

4.9 Author Contributions

B.K., M.G. and L.L.J. designed the study. B.K., M.G. and L.L.J. B. analyzed the data. B.K.,M.G., L.L.J., J.H. N.K. and K.S. wrote the manuscript.

59

4.10 Conflicts of Interest

The authors declare no conflict of interest.

Part 5

Designing ICT for Health and

Wellbeing - An Allostatic,

Behavioral-Change Approach to a

Monitoring and Coaching App

61

63

5.1 Abstract

We are developing a monitoring and coaching app for health and wellbeing based on (1) anallostatic model of adaption combined with (2) behavioural change theory and (3) user-oriented design. The (1) allostatic model comes from stress research and was introduced toexplain how human health and wellbeing can be maintained. It suggests that human healthand wellbeing is a complex multidimensional phenomenon that needs to be understood holis-tically. We have used this model to incorporate the dimensions of human health and wellbeingthat are key for stress reduction: physical and social activity and sleep. The allostatic modelcan allow us to understand human health and wellbeing but it does not tell us how to supportthe behavioural changes needed in order to reach a healthy state of allostasis. For this we relyon (2) theory of behavioural change. This article describes how we have integrated (1-3) intothe system design and reports from an initial workshop with users.

5.2 Introduction

In westernized societies diseases related to sedentary life styles and stress constitute a majorhealth and wellbeing challenge. We present a mobile application designed to motivate peopleto engage in physical and social activities. The application design is an attempt to go beyondthe state of the art in apps for health and wellbeing through a combination of an allostaticmodel of human adaption [96, 88, 89] and the Transtheoretical model of Behaviour changedeveloped by Dr. Prochaska. Although effects of the latter has been debated, we believeit is a good fit for this application. To measure progress, the system uses external sensors(heart rate monitor, accelerometer and microphone) to monitor the user’s physical activity,stress level, social interactions and quality of sleep. The user’s activities are then comparedover time with challenges defined by both the system and by the user. Challenges are basedon achieving a certain amount of units in activities, for example “Walk 10.000 steps” or“Spend 30 minutes talking to a friend”. When a challenge is completed, the user gets positivevisual and audio feedback to reinforce the behaviour. The user also gets “health points” forcompleted challenges and these can be used to get discounts at 3rd party partners (e.g., localgyms, sports retailers), further motivating the use of the applica-

5.2.1 The allostatic model in health and wellbeing

Health and wellbeing depends on maintaining physiological and psychological balance.Much of our understanding of how to maintain such balance comes from the research onstress. The stress research began with a focus on homeostasis [28]. Stress was thought ofin terms of a generalized stress response [28]. Today the field of stress research has movedfrom a generalized model of stress to one based on allostasis [96, 88, 89]. We have chosenan allostatic model because it is more realistic for modelling stress than older homeostaticmodels. According to the allostatic model, a human being’s stress tolerance depends onmany factors, such as genetic disposition, individual biology, and developmental history.Thus, the tolerance is affected by multiple interactive systems that can become unbalanced.

64

According to the allostatic model of stress, how well a person can deal with the stressors oflife and thus maintain health and wellbeing, depends on:

a. Genetic factors

b. Social activity patterns

c. Sleep and resting patterns

d. Physical activity patterns

e. Stress coping techniques

f. Diet and nutrition habits

We have chosen to look more closely at (b-e) and to a limited extent on (f). Genetic fac-tors (a) are important for understanding health and wellbeing. We can expect that much workwill be done in the future on genetic screening. In the not so distant future, preventive geneticscreening for health and wellbeing may become common. (b) Our social life is importantfor understanding stress. Too little social interaction can be a source of stress and the wrongforms of social interaction can lead to immense stress while nurturing social relationships canhave the opposite effect. (c) Sleep and activity patters modulate our stress levels. If we getinsufficient sleep our bodies respond with increasing levels of stress hormones. Increasinglevels of stress hormones make us sleep less and we can find ourselves in a stress spiral. (d)Physical activity can make us more stress resilient. Someone who is physically fit typicallyhas a greater ability to rebound from stressful situations. The levels of stress hormones goback to normal faster than for someone who is less fit. (e) Stress reduction techniques suchas deep breathing and somatic quieting can help to bring down stress levels. Someone whopractices these techniques may cope better with stressors than one who doesn’t. (f) Diet andnutrition can be important factors in determining how well we cope with stress. E.g., it hasbeen shown that strict vegetarians become less likely to suffer from cardiovascular disease.Our allostatic model for health and wellbeing takes into account all dimensions above apartfrom genetic factors. It could be extended to such factors in the future but at present we havechosen to leave it out until genetic screening techniques become more commonplace. Ac-cording to our allostatic model how well a person can maintain health and wellbeing dependson how well the person is doing on b-f. More concretely this means that we have systemat-ically attempted to construct a health and wellbeing monitoring and coaching platform andapp that allows the user to do better in areas b-f.

5.2.2 Transtheoretical model of behavioral change

The Transtheoretical Model of behavioural change developed by Dr. Prochaska is based onmore than 25 years of research measuring behaviour change for a variety of health behaviours.This model serves as a blueprint for effecting self-change in health behaviours and can bereadily applied in health, fitness and wellness coaching [75, 81]. The model consists of fivestages of change:

65

I. Pre-contemplation

II. Contemplation

III. Preparation

IV. Action

V. Maintenance

These stages sequentially approach behaviour change with the first stage being furthestfrom change and the fifth stage being maintaining an already achieved behaviour. We nowpresent these 5 stages further together with our suggested solutions for how our applicationcould influence the user to progress through the stages.

Pre-contemplation (Stage I) The subject is not yet thinking about changing behaviour.The person may not admit they have a problem or think that change is possible [75]. Itis unlikely that a person in this stage would engage with our application on their own, butthis stage is still important since application users in higher stages might fall back into thisstage during their behaviour change process. Persons in the pre-contemplation stage mayget the application out of curiosity or through recommendation. To move the subject to thenext stage, a simple challenge is presented to the user at the first use of the application.The challenge consists of answering simple questions about themselves, their health andgoals. Between each question there are brief assertions of how better health can be attainedwith little effort. Upon completing the first challenge the user receives rewarding feedbackof having taken a step in changing behaviour and becoming healthier. This feedback canhopefully help the user move to the contemplation stage.

Contemplation (Stage II) Subjects in this stage are thinking about changing be- haviourand are considering taking action. They are generally aware of the positive effects a be-haviour change could yield but may perceive making the change as difficult. Moore et al.[75] suggests that increasing awareness of compelling reasons to change and getting peopleto connect with others that have successfully changed is important. This is done in our appli-cation by showing information splash screens from other successful users (that have agreedto this). Seeing others completing challenges and reaching results could motivate the user totake action and reach the preparation

Preparation (Stage III) Subjects have overcome most negative feelings about the changeprocess, strengthened their motivation and are planning to take action. Subjects also exper-iment with possible solutions [75]. Moore et al. suggest assisting subjects with their plansand encouraging them to write down a statement of what they are committed to do. Theyalso suggest identifying small steps to put into action. By allowing the user to choose oreven create his/her own challenges our suggested application helps the subject through thepreparation stage. The premade challenges in our application are simple to complete so themajority of users can finish them in a day. After the challenges are chosen, the applicationmonitors progress and notifies the user with positive feedback as challenges are completed.This hopefully keeps the subject motivated to make positive behaviour changes, and if moti-vated enough move to the action stage.

66

Action (Stage IV) Subjects are consistently working towards their goals through practic-ing new behaviours including building new relationships [75]. Moore et al. state that assistingsubjects in developing new relationships with people who share their interests and goals canmake a significant difference. In our application this is addressed by enabling chatting withother users. Users can also add their custom challenges to the public challenge database tofurther encourage user-to-user involvement. It is also possible for users to add real-life eventssuch as meet-ups and competitions. If the user is active in the action stage for a longer periodof time (6 months [75]) he/she is considered to have changed and is considered to be in themaintenance stage.

Maintenance (Stage V) This stage begins when the new behaviour has turned into ahabit. Coaching in this stage is focused on retaining the positive habit and prevent- inglapses. A lapse occurs when the subject temporarily abandons the new behaviour. Whenthis happens, the subject may need assistance to set new goals and refocus [75]. By detectingchanges in habits, our suggested application could recommend the user to enter a competition,challenge a friend or participate in an event to get the user back on track. The applicationwill also periodically encourage the user to share his/her results with others since this has apositive effect on motivation [75].

5.3 The monitoring and coaching platform

The monitoring and coaching platform is being built in modules, parallel to the design of theinterfaces. The primary focus has been on monitoring the four dimensions of our allostaticmodel: social activity, sleep and resting, physical activity, and stress levels. The architectureis easily extendable with new sensors and devices, and improved measurement techniquesas they become available. A number of quantifiable DataUnits have been defined for thefour dimensions. These DataUnits allow aggregation and fusion of data to vary as long asthe quantified value is produced as a result. This also allows for many different devices tobe used, given that a sensor module is developed to capture the data and that the DataUnitssupport the data type being captured. Social interactions are manually input by the user,but the user can also choose to have them automatically detected using the mobile phonemicrophone and voice activity detection algorithms. It should be noted that the accuracy ofthis feature is highly dependent on the placement of the device.

Sleep and resting is sensed through heart rate and accelerometer-data from the wrist-worndevice. The algorithm detects movement in the night, coupled with the heart-rate variabilityto detect quality of sleep, such as resting, sleep and deep sleep. Physical activity is calculatedfrom movement and heart rate. Movement is sensed using Laban Movement analysis of datafrom the wrist-worn accelerometer, which has been shown as the best placement of a singleaccelerometer to detect movement intensity [60]. This analysis is complemented with heart-rate data for complementary detection of activity intensity.

Stress levels are detected through heart-rate variability and accelerometer-data from awrist-worn device. The wrist-worn device uses an optical sensor for detecting heart rate,which yields the accurate heart-rate detection required for heart-rate variability calculations.

67

It has been shown that heart rate variability can be used to determine stress during a numberof different activities, such as while sitting, standing, and walking [99].

The sensing modules are currently under development and will soon be integrated into aworking prototype together with the interfaces that are evaluated in this paper. Furthermore,the detected aspects will be used by the coaching application to provide automatic and help-ful advice and information to motivate users to perform healthy behavioural changes. Forexample, the application can provide information on the positive effects of an active physicallife, and offer suggestions on activities to improve quality of life, such as what to do to im-prove the quality of sleep. These advices will be developed in collaboration with healthcareprofessionals and mined from common- and best practices.

5.4 Evaluation

Two monitoring prototypes and one coaching prototype (see fig. 5.1) were evaluated in auser workshop. The monitoring prototypes contained the proposed main monitoring interfacerelated to activity tracking, and the coaching prototype contained the proposed user interfacefor a coaching component. 12 users participated in the test and their ages ranged from 20-60. The test participants were first briefed about the project. A short demonstration wasthen given of the prototypes. The participants were then asked to explore the prototypes forthemselves in groups of four people. The two monitoring prototypes were web-based andwere presented on iPod touch devices. The coaching prototype was presented on a laptop.Each group had one iPod and one laptop. The coaching prototype was in PowerPoint formatand lent itself better for a laptop device. As the users explored the prototypes they were alsoasked to fill in a questionnaire. The users were also given post-it notes to jot down notes andany reflections they might have about the prototypes. These notes were then saved along witheach questionnaire to aid in the analysis. All prototypes had many pages that are not shownhere.

5.4.1 Initial findings

We found the following in our evaluation of the first prototypes.

1. The second monitoring prototype was the one we should choose for further develop-ment. It was perceived as easier to navigate and allowed the users to get a better senseof what the capabilities were of the app and how it could be used. From the responsesto the questionnaire we saw that the first monitoring prototype was received slightlynegatively with respect to how easy it was for the users to understand what could bedone with the application. The same was true for how the information was presented.Monitoring prototype two fared better and was, on the whole evaluated in positiveregard. When asked which prototype they preferred most users said monitoring proto-type two. Some of the reasons given were that it was: “simpler, better with horizontalbars”, “clearer—if you target a group around 40 years old prototype one might be dif-

68

Figure 5.1: Prototypes tested in the first user workshop

ficult to handle” and that “the information is easier to understand”. It was clear that theprototype to continue development with was the second monitoring prototype.

2. The users thought that the coaching component ought to be included in the app andthat it was important. The users held the coaching prototype in positive regard.

3. The monitoring parts of the app were also perceived favourably.

4. Our users were not in favour of having a social sharing component. They did not wantto share their results with other users through the app.

5. They wanted the coaching feedback to be personalized—not just general advice.

6. Most users preferred to be coached via the app and not via interaction with peers.

7. Most users believed that people are likely to use a health and wellbeing app for a longperiod of time.

These findings are based on our questionnaire and also supported by the user’s notes andour discussions with them. From reading the notes and discussing with the users we alsofound that:

8. User opinions diverged greatly. Some users in the test wrote largely negative commentsand reviewed the apps unfavourably while others were highly positive and reviewed theprototypes favourably.

69

Here it is interesting to note that the users revealed a preference for having a coachingcomponent in the application. They also thought that the coaching should be personalized.However, the users neither wanted to share the information within the coaching app with theirfriends nor to be coached through social peer-to-peer coaching. Lastly, it is interesting to notethat they believed that a health and wellbeing app is something that people would be likelyto use on a more permanent basis. Our initial evaluation of the prototype was only meant toprovide rapid initial feedback for guiding system design. We plan to conduct further studiesonce we have a working system. To understand how the system would work in the real worldit would be best to perform our future evaluation in everyday life situations, i.e., having usersuse the system as intended.

5.5 Discussion

From our user study it became evident that some users were highly critical of our prototypeswhile others were highly positive. Most users fell somewhere in between those extremes.What made some users so critical? Some raised concerns about the validity of the monitoringmeasures and soundness of coaching suggestions. Perhaps these users would be more sat-isfied with an improved coaching and monitoring app. However, there might be a group ofusers who would not be satisfied with any monitoring and coaching app. After all, a humancoach can offer things that a computational system cannot such as the presence of a caringand empathic professional.

The users did not object to our multi-dimensional model of health and wellbeing. Indeedthey seem to support such a model. If they had not supported it, then they would have objectedto the inclusion of one or more dimensions of monitoring and coaching. Implicitly then, onecould argue that the allostatic model was supported. It was natural for the users to think ofwellbeing as the result of interplay of social activity, sleep and physical activity patterns. Inthe future we might extend this model with a nutritional model in order to strengthen theallostatic model further. Another possibility would also be to include lessons or other contenton the allostatic model within the coaching module. It would also be possible to includecontent on the behavioural change model. Whether we do this or not in the future couldbe decided after our next user workshop. According to the behavioural change model it isimportant that users want to understand the mechanisms of change. It is a model that involveschanging behaviour, but it is also one that can be further strengthened through understanding.

To some extent this could be said about the allostatic model as well. Once users under-stand how it works it allows them to change their behaviour more easily. If a user sees that heor she is not doing so well in one category (such as sleep or physical activity) then this canbe a strong incentive to change behaviour within that category.

5.6 Conclusions and Future Work

We learned from our first user study how to proceed with the development of our health andwellbeing app. We will continue with the second monitoring prototype and we will include

70

the coaching component. The fact that the users preferred to have a coaching componentincluded supported our underlying allostatic, behaviouralchange model. If the users had notpreferred to have a coaching model it would be unclear how the behavioural-change modelcould be included. As it is we will continue to work with the coaching component. We willcontinue to work with the allostatic model as well. It is a model that fits well with the resultsfrom the first user workshop.

Moreover, it is also compatible with the behavioural-change model. Indeed the two mod-els can work as mutually supporting. The allostatic model gives support from a fundamentalbio-psycho-social perspective while the behavioural-change model works at a higher psycho-logical level to motivate change. In the next user workshop we plan to explore whether or notlearning about these underlying models is something that the users would want.

5.7 Acknowledgements

This work has been supported by the European Institute of Innovation & Technology (EUEIT ICT Labs) within the HWB Cognitive Endurance activity.

Part 6

Discussion, Conclusions and

Future Work

71

Discussion, Conclusions and Future Work 73

6.1 Discussion

This licentiate thesis presents techniques and principles that can be used in order to performactivity recognition in resource-constrained pervasive systems. The focus of this research hasbeen to perform practical studies on how to design a minimalistic wearable system that canbe used for activity recognition in an everyday scenario. Since all studies have been practicalin nature, the results are directly applicable in real-world problems. The following sectionprovides a discussion on the research questions found in section 2.3.

6.1.1 How can activity recognition be performed unobtrusively in a

wearable system with resource constraints?

The size of devices, the placement of devices and the number of devices are three very im-portant factors for unobtrusiveness. These three factors have all been addressed in this thesisas can be seen in the following discussion:

Size of devices

Most of this work rely on the use of tri-axial accelerometer sensors for activity recognition.Accelerometers are sensors that can be made very small, as can be seen from their commonuse in smart-watches, mobile phones, cameras etc. They are also power efficient sensorsadhering to more strict resource constraints on battery capacity. These two traits togethermakes it possible to create small accelerometer devices that, despite their size, can providerich movement data. This makes them a suitable choice for unobtrusive activity recognition.

Number of devices

In this thesis it is shown that classifying body movements within the LMA effort category canbe detected with a high accuracy using only a single accelerometer. This result is interestingsince it, together with the fact that the Laban effort framework can be applied across a rangeof applications, suggests that a wearable activity recognition system can be designed usingfew resources.

Placement of devices

This work examines the classification accuracy for a suite of machine learning algorithms onaccelerometer data gathered from the chest, wrist and thigh. These locations are unobtrusiveto the user while at the same time being able to provide a reasonable accuracy for classifi-cations [31, 12]. Each classifier was trained and evaluated on each location since machinelearning algorithm performance can vary heavily based on the input data. The classifiersaccuracies were then compared and the results showed that the overall best placement of asingle accelerometer sensor for classifying movements in the LMA effort category was thedominant wrist when using the random forest algorithm.

74 Discussion, Conclusions and Future Work

6.1.2 How can classification be performed in resource-constrained de-

vices?

This thesis presents a novel algorithm, named CAMP, that merges Cellular Automata withfeedforward neural networks in order to create an efficient classifier that is suitable for low-level microcontrollers. The classifier utilizes the fact that cellular automata works at the edgeof chaos in order to maintain performance while creating a sparse network consisting of onlybinary values. Its simple structure and its use of binary logic makes it suitable for use inlow-level microcontrollers and other less sophisticated computational devices. Evaluation ofthe classifier shows that, despite its simplistic nature, it has an accuracy that is comparableto that of more complex algorithms for some of the datasets tested while performing poorlyon others. This means that for at least some classification problems, the CAMP algorithmcould serve as an efficient classifier even under extreme limitations on computational powerand memory.

6.1.3 How can a mobile activity recognition system be designed to

achieve positive behaviour change in health and wellbeing?

This thesis proposes a mobile system designed for positive behaviour change in four impor-tant health-related areas: sleep, stress, social activity and physical activity. The system uses asingle wrist-worn device with an tri-axial accelerometer and a heart-rate monitor. This deviceis connected to a mobile phone application in order to perform activity recognition in all ofthe key areas, except for social activity that is manually annotated by the user.

A literature study was performed on results from behavioural and psychology researchin order to find important principles and models for behavioural change. Two prototypeapplications were then developed using principles from the Transtheoretical Model (TTM).The workings and design choices were, with best effort, made to address the 5 stages and 8processes of change declared by the TTM in order to motivate users to a healthier lifestyle.

The two prototypes were evaluated in user workshop with 12 users with ages rangingfrom 20 to 60. This workshop included both a questionnaire and an explorative study of theprototypes in groups of four. Results from this workshop reveals that the users generally likedthe idea of a monitoring and coaching application, and many believed that such an applicationis something people would be likely to use on a more permanent basis and for long periodsof time. Users did not, however, like the parts in the prototypes that involved sharing theirresults with other users, nor did they like the idea of being coached by other peers. Insteadthey wanted to be coached by via the app using personalized advice.

Discussion, Conclusions and Future Work 75

6.2 Conclusions

This thesis discusses how to create applications that adapt and react to the user’s current situa-tion while having low resource requirements. This includes methods for activity recognition,such as a novel classifier targeted for resource constrained devices commonly found in perva-sive systems and an implementation for estimating effort in everyday activities. Furthermore,it holds insights on design choices for pervasive motivational systems, based on applied con-cepts from context-awareness and behavioral science, that has been evaluated with real users.

The results presented in this thesis are advancements in providing technology for worndevices, and is suitable for adoption in new types of interactive services, such as health-careservices and motivational services for lasting positive behavioural change. Society is facinga problem of an aging population with an increased amount of chronic diseases such as dia-betes, dementia, cardiovascular disease and hypertension. These will, if let be, have a greatnegative impact both on an economical level as well as on an individual one. Many of theseconditions are highly affected by lifestyle choices and behaviour, making it paramount to en-gage the public in achieving positive behaviour change. This means that pervasive computingand motivational applications will likely play an important role in future health-care, wherethe demand for self-management is likely to increase.

76 Discussion, Conclusions and Future Work

6.3 Future work

This research has shown how activity recognition can be used in order to motivate users toperform positive behavioural changes. While the proposed system design does use principlesfrom behavioural psychology, these are mostly applied without taking into consideration thepersonal differences between users. Since people generally come from different backgroundsand respond to different incentives, there is most likely much to gain from personalizinghow, when and where a motivational system interacts with the user. It would therefore be aninteresting study to use machine intelligence to analyse user behaviour in order to create suchpersonalizations. By finding specific interactions that are successful when applied to userswith certain personal traits, this could not only have an impact on motivation results but alsobe important for behavioural psychology research as well.

Another interesting study would be to try to make LMA quantifiable. Although people’sphysical traits and condition varies, it should be possible to find approximate measurementsthat fits the general public. By measuring for example accelerometer values on different partsof a subject’s body such as wrists, elbows, ankles, knees, hips, chest and head, one should beable to find a “ground truth” for the different categories of LMA, thereby removing the needfor a human expert to perform the movement classifications.

Bibliography

[1] AppleWatch. http://www.apple.com/watch/. Accessed: 2015-11-11.

[2] EUROLAB european association for laban/bartenieff movement studies website.http://laban-eurolab.org/index.php?option=com_content&view=article&id=45. Ac-cessed: 2014-01-16.

[3] Fjuul. http://www.fjuul.com/. Accessed: 2015-11-11.

[4] Nike+. urlhttp://www.nikeplus.com.br/. Accessed: 2015-11-11.

[5] Runkeeper. https://runkeeper.com/. Accessed: 2015-11-11.

[6] M. Adachi and K. Aihara. Associative dynamics in a chaotic neural network. Neural

Networks, 10(1):83–98, 1997.

[7] A. Ahmadi, D. Rowlands, and D. A. James. Towards a wearable device for skill assess-ment and skill acquisition of a tennis player during the first serve. Sports Technology,2(3-4):129–136, 2009.

[8] K. Aihara and G. Matsumoto. Chaotic oscillations and bifurcations in squid giantaxons. Chaos, 12:257–269, 1986.

[9] K. Aihara and G. Matsumoto. Forced oscillations and routes to chaos in the Hodgkin-

Huxley axons and squid giant axons, pages 121–131. Chaos in biological systems.Springer, 1987.

[10] I. Ajzen. From intentions to actions: A theory of planned behavior. Springer, 1985.

[11] G. Andrews, P. Cuijpers, M. G. Craske, P. McEvoy, N. Titov, et al. Computer therapyfor the anxiety and depressive disorders is effective, acceptable and practical healthcare: a meta-analysis. PloS one, 5(10):e13196, 2010.

[12] L. Atallah, B. Lo, R. King, and G.-Z. Yang. Sensor placement for activity detectionusing wearable accelerometers. In Body Sensor Networks (BSN), 2010 International

Conference on, pages 24–29. IEEE, 2010.

[13] L. Atallah, B. Lo, R. King, and G.-Z. Yang. Sensor positioning for activity recognitionusing wearable accelerometers. Biomedical Circuits and Systems, IEEE Transactions

on, 5(4):320–329, 2011.

77

78 Bibliography

[14] A. P. Atkinson, W. H. Dittrich, A. J. Gemmell, A. W. Young, et al. Emotion percep-tion from dynamic and static body expressions in point-light and full-light displays.Perception-London, 33(6):717–746, 2004.

[15] A. Bandura. Social foundations of thought and action: A social cognitive theory.

Prentice-Hall, Inc, 1986.

[16] L. Bao and S. S. Intille. Activity recognition from user-annotated acceleration data. InPervasive computing, pages 1–17. Springer, 2004.

[17] L. Bao and S. S. Intille. Activity recognition from user-annotated acceleration data. InPervasive computing, pages 1–17. Springer, 2004.

[18] A. Barak, L. Hen, M. Boniel-Nissim, and N. Shapira. A comprehensive review anda meta-analysis of the effectiveness of internet-based psychotherapeutic interventions.Journal of Technology in Human Services, 26(2-4):109–160, 2008.

[19] I. Bartenieff and D. Lewis. Body Movement: Coping with the Environment 1st ed.

Routledge: New York, NY, USA, 2002.

[20] E. Basar. Chaos in Brain Function: Containing Original Chapters by E. Basar and TH

Bullock and Topical Articles Reprinted from the Springer Series in Brain Dynamics.Springer Science & Business Media, 2012.

[21] A. Basturk and E. Gunay. Efficient edge detection in digital images using a cellularneural network optimized by differential evolution algorithm. Expert Systems with

Applications, 36(2):2645–2650, 2009.

[22] J. R. Bitner, G. Ehrlich, and E. M. Reingold. Efficient generation of the binary reflectedgray code and its applications. Communications of the ACM, 19(9):517–521, 1976.

[23] V. Bolon-Canedo, N. Sanchez-Marono, and A. Alonso-Betanzos. A review of featureselection methods on synthetic data. Knowledge and information systems, 34(3):483–519, 2013.

[24] A. G. Bonomi, A. Goris, B. Yin, and K. R. Westerterp. Detection of type, dura-tion, and intensity of physical activity using an accelerometer. Med Sci Sports Exerc,41(9):1770–1777, 2009.

[25] A. Bourke, J. O’brien, and G. Lyons. Evaluation of a threshold-based tri-axial ac-celerometer fall detection algorithm. Gait & posture, 26(2):194–199, 2007.

[26] C. V. Bouten, K. Koekkoek, M. Verduin, R. Kodde, and J. D. Janssen. A triaxialaccelerometer and portable data processing unit for the assessment of daily physicalactivity. Biomedical Engineering, IEEE Transactions on, 44(3):136–147, 1997.

[27] A. Camurri, S. Hashimoto, K. Suzuki, and R. Trocca. Kansei analysis of dance per-formance. In Systems, Man, and Cybernetics, 1999. IEEE SMC’99 Conference Pro-

ceedings. 1999 IEEE International Conference on, volume 4, pages 327–332. IEEE,1999.

Bibliography 79

[28] W. B. Cannon. The wisdom of the body. 1932.

[29] S. Chattopadhyay, S. Adhikari, S. Sengupta, and M. Pal. Highly regular, modular,and cascadable design of cellular automata-based pattern classifier. Very Large Scale

Integration (VLSI) Systems, IEEE Transactions on, 8(6):724–735, 2000.

[30] P. P. Chaudhuri. Additive cellular automata: theory and applications, volume 1. JohnWiley & Sons, 1997.

[31] I. Cleland, B. Kikhia, C. Nugent, A. Boytsov, J. Hallberg, K. Synnes, S. McClean,and D. Finlay. Optimal placement of accelerometers for the detection of everydayactivities. Sensors, 13(7):9183–9200, 2013.

[32] S. Consolvo, D. W. McDonald, T. Toscos, M. Y. Chen, J. Froehlich, B. Harrison,P. Klasnja, A. LaMarca, L. LeGrand, R. Libby, et al. Activity sensing in the wild:a field trial of ubifit garden. In Proceedings of the SIGCHI Conference on Human

Factors in Computing Systems, pages 1797–1806. ACM, 2008.

[33] M. Cook. Universality in elementary cellular automata. Complex Systems, 15(1):1–40,2004.

[34] N. J. Cotton, B. M. Wilamowski, and G. Dundar. A neural network implementationon an inexpensive eight bit microcontroller. In Proc. Int. Conf. Intelligent Engineering

Systems INES 2008, pages 109–114, 2008.

[35] P. Crook, S. Marsland, G. Hayes, U. Nehmzow, et al. A tale of two filters-on-linenovelty detection. In Robotics and Automation, 2002. Proceedings. ICRA’02. IEEE

International Conference on, volume 4, pages 3894–3899. IEEE, 2002.

[36] T. Denning, A. Andrew, R. Chaudhri, C. Hartung, J. Lester, G. Borriello, and G. Dun-can. Balance: towards a usable pervasive wellness application with accurate activityinference. In Proceedings of the 10th workshop on Mobile Computing Systems and

Applications, page 5. ACM, 2009.

[37] D. W. Duke and W. S. Pritchard. Proceedings of the Conference on Measuring Chaos

in the Human Brain, April 3-5, 1991, at the Supercomputer Computations Research

Institute, Florida State University, Tallahassee, FL. World Scientific, 1991.

[38] M. Ercsey-Ravasz, T. Roska, and Z. Néda. Statistical physics on cellular neural net-work computers. Physica D: Nonlinear Phenomena, 237(9):1226–1234, 2008.

[39] M. Ermes, J. Parkka, J. Mantyjarvi, and I. Korhonen. Detection of daily activities andsports with wearable sensors in controlled and uncontrolled conditions. Information

Technology in Biomedicine, IEEE Transactions on, 12(1):20–26, 2008.

[40] P. Fagerberg, A. Ståhl, and K. Höök. Designing gestures for affective input: an analysisof shape, effort and valence. In In Proceedings of the 2nd International Conference on

Mobile Ubiquitous and Multimedia, 2003.

[41] M. Fishbein. A theory of reasoned action: some applications and implications. 1979.

80 Bibliography

[42] B. S. Fjeldsoe, A. L. Marshall, and Y. D. Miller. Behavior change interventions de-livered by mobile telephone short-message service. American journal of preventive

medicine, 36(2):165–173, 2009.

[43] A. Foroud and S. M. Pellis. The development of “roughness” in the play fight-ing of rats: A laban movement analysis perspective. Developmental Psychobiology,42(1):35–43, 2003.

[44] N. Ganguly, P. Maji, B. K. Sikdar, and P. P. Chaudhuri. Design and characterizationof cellular automata based associative memory for pattern recognition. Systems, Man,

and Cybernetics, Part B: Cybernetics, IEEE Transactions on, 34(1):672–678, 2004.

[45] N. Gebruers, C. Vanroy, S. Truijen, S. Engelborghs, and P. P. De Deyn. Monitoring ofphysical activity after stroke: a systematic review of accelerometry-based measures.Archives of physical medicine and rehabilitation, 91(2):288–297, 2010.

[46] M. C. Gibbons, R. F. Wilson, L. Samal, C. U. Lehmann, K. Dickersin, H. P. Lehmann,H. Aboumatar, J. Finkelstein, E. Shelton, R. Sharma, et al. Consumer health infor-matics: results of a systematic evidence review and evidence based recommendations.Translational behavioral medicine, 1(1):72–82, 2011.

[47] H. Gjoreski, M. Luštrek, and M. Gams. Accelerometer placement for posture recogni-tion and fall detection. In Intelligent Environments (IE), 2011 7th International Con-

ference on, pages 47–54. IEEE, 2011.

[48] A. Godfrey, R. Conway, D. Meagher, and G. ÓLaighin. Direct measurement of hu-man movement by accelerometry. Medical engineering & physics, 30(10):1364–1386,2008.

[49] M. Hall, E. Frank, G. Holmes, B. Pfahringer, P. Reutemann, and I. H. Witten. The wekadata mining software: an update. ACM SIGKDD explorations newsletter, 11(1):10–18,2009.

[50] W. Hardeman, S. Griffin, M. Johnston, A. Kinmonth, N. Wareham, et al. Interventionsto prevent weight gain: a systematic review of psychological models and behaviourchange methods. International journal of obesity, 24(2):131–143, 2000.

[51] J. Harding, J. W. Small, and D. A. James. Feature extraction of performance variablesin elite half-pipe snowboarding using body mounted inertial sensors. In Microelec-

tronics, MEMS, and Nanotechnology, pages 679917–679917. International Society forOptics and Photonics, 2007.

[52] A. Hedman, N. Karvonen, J. Hallberg, and J. Merilahti. Designing ict for health andwellbeing-an allostatic, behavioral-change approach to a monitoring and coaching app.In IWAAL, pages 244–251, 2014.

[53] D. Hendelman, K. Miller, C. Baggett, E. Debold, and P. Freedson. Validity of ac-celerometry for the assessment of moderate intensity physical activity in the field.Medicine and science in sports and exercise, 32(9 Suppl):S442–9, 2000.

Bibliography 81

[54] K. J. Hunt, D. Sbarbaro, R. Zbikowski, and P. J. Gawthrop. Neural networks for controlsystems—a survey. Automatica, 28(6):1083–1112, 1992.

[55] A. Janecek, W. N. Gansterer, M. Demel, and G. Ecker. On the relationship betweenfeature selection and classification accuracy. In In JMLR Proceedings, Workshop and

Conference, pages 90–105, 2008.

[56] H. Junker, P. Lukowicz, and G. Troster. Sampling frequency, signal resolution andthe accuracy of wearable context recognition systems. In Wearable Computers, 2004.

ISWC 2004. Eighth International Symposium on, volume 1, pages 176–177. IEEE,2004.

[57] D. M. Karantonis, M. R. Narayanan, M. Mathie, N. H. Lovell, and B. G. Celler. Imple-mentation of a real-time human movement classifier using a triaxial accelerometer forambulatory monitoring. Information Technology in Biomedicine, IEEE Transactions

on, 10(1):156–167, 2006.

[58] N. Karvonen, B. Kikhia, L. Lorna Jiménez, G. Simón, and J. Hallberg. A computa-tionally inexpensive classifier merging cellular automata and mcp-neurons, 2015.

[59] N. L. Keijsers, M. W. Horstink, and S. C. Gielen. Ambulatory motor assessment inparkinson’s disease. Movement Disorders, 21(1):34–44, 2006.

[60] B. Kikhia, M. Gomez, L. L. Jiménez, J. Hallberg, N. Karvonen, and K. Synnes. Ana-lyzing body movements within the laban effort framework using a single accelerome-ter. Sensors, 14(3):5725, 2014.

[61] K.-J. Kim, M. M. Hassan, S. Na, and E.-N. Huh. Dementia wandering detection andactivity recognition algorithm using tri-axial accelerometer sensors. In Ubiquitous

Information Technologies & Applications, 2009. ICUT’09. Proceedings of the 4th In-

ternational Conference on, pages 1–5. IEEE, 2009.

[62] S. Kocsardi, Z. Nagy, A. Csik, and P. Szolgay. Simulation of 2d inviscid, adiabatic,compressible flows on emulated digital cnn-um. International Journal of Circuit The-

ory and Applications, 37(4):569–585, 2009.

[63] A. Krause, M. Ihmig, E. Rankin, D. Leong, S. Gupta, D. Siewiorek, A. Smailagic,M. Deisher, and U. Sengupta. Trading off prediction accuracy and power consumptionfor context-aware wearable computing. In Wearable Computers, 2005. Proceedings.

Ninth IEEE International Symposium on, pages 20–26. IEEE, 2005.

[64] R. Laban and F. C. Lawrence. Effort: Economy in body movement 2nd ed., 1974.

[65] Y. Lin, J. Jessurun, B. de Vries, and H. Timmermans. Motivate: context aware mobileapplication for activity recommendation. In Ambient Intelligence, pages 210–214.Springer, 2011.

[66] J. Lockman, R. S. Fisher, and D. M. Olson. Detection of seizure-like movements usinga wrist accelerometer. Epilepsy & Behavior, 20(4):638–641, 2011.

82 Bibliography

[67] M. Luštrek and B. Kaluža. Fall detection and activity recognition with machine learn-ing. Informatica, 33(2), 2009.

[68] P. Maji, N. Ganguly, S. Saha, A. K. Roy, and P. P. Chaudhuri. Cellular automata

machine for pattern recognition, pages 270–281. Cellular Automata. Springer, 2002.

[69] A. Mannini and A. M. Sabatini. Machine learning methods for classifying humanphysical activity from on-body accelerometers. Sensors, 10(2):1154–1175, 2010.

[70] H. M. Mentis and C. Johansson. Seeing movement qualities. In Proceedings of

the SIGCHI Conference on Human Factors in Computing Systems, pages 3375–3384.ACM, 2013.

[71] F. Michahelles and B. Schiele. Sensing and monitoring professional skiers. Pervasive

Computing, IEEE, 4(3):40–45, 2005.

[72] S. Michie, M. Johnston, J. Francis, W. Hardeman, and M. Eccles. From theory to inter-vention: mapping theoretically derived behavioural determinants to behaviour changetechniques. Applied psychology, 57(4):660–680, 2008.

[73] V. Mircevska, M. Luštrek, and M. Gams. Combining machine learning and expert

knowledge for classifying human posture. 2009.

[74] J. Mitchell, D. Vella-Brodrick, and B. Klein. Positive psychology and the internet: Amental health opportunity. Sensoria: A Journal of Mind, Brain & Culture, 6(2):30–41,2010.

[75] M. Moore. Coaching psychology manual. Wolters Kluwer Health/Lippincott,Williams & Wilkins, Philadelphia, 2010.

[76] L. M. Neville, B. O’Hara, and A. Milat. Computer-tailored physical activity behaviorchange interventions targeting adults: a systematic review. International Journal of

Behavioral Nutrition and Physical Activity, 6(1):30, 2009.

[77] L. M. Neville, B. O’Hara, and A. J. Milat. Computer-tailored dietary behaviour changeinterventions: a systematic review. Health Education Research, 24(4):699–720, 2009.

[78] D. O. Olguın and A. S. Pentland. Human activity recognition: Accuracy across com-mon locations for wearable sensors. In Proceedings of 2006 10th IEEE International

Symposium on Wearable Computers, Montreux, Switzerland, pages 11–14. Citeseer,2006.

[79] D. B. Portnoy, L. A. Scott-Sheldon, B. T. Johnson, and M. P. Carey. Computer-delivered interventions for health promotion and behavioral risk reduction: a meta-analysis of 75 randomized controlled trials, 1988–2007. Preventive medicine, 47(1):3–16, 2008.

[80] S. J. Preece, J. Y. Goulermas, L. P. Kenney, and D. Howard. A comparison of featureextraction methods for the classification of dynamic activities from accelerometer data.Biomedical Engineering, IEEE Transactions on, 56(3):871–879, 2009.

Bibliography 83

[81] J. O. Prochaska and W. F. Velicer. The transtheoretical model of health behaviorchange. American journal of health promotion, 12(1):38–48, 1997.

[82] N. Ravi, N. Dandekar, P. Mysore, and M. L. Littman. Activity recognition from ac-celerometer data. In In Proceedings of the 17th Conference on Innovative Applications

of Artificial Intelligence, volume 5, pages 1541–1546, 2005.

[83] W. T. Riley, D. E. Rivera, A. A. Atienza, W. Nilsen, S. M. Allison, and R. Mermelstein.Health behavior models in the age of mobile interventions: are our theories up to thetask? Translational behavioral medicine, 1(1):53–71, 2011.

[84] P. Sarkar. A brief history of cellular automata. ACM Computing Surveys (CSUR),32(1):80–107, 2000.

[85] J. D. Schaffer, D. Whitley, and L. J. Eshelman. Combinations of genetic algorithmsand neural networks: A survey of the state of the art. In Combinations of Genetic

Algorithms and Neural Networks, 1992., COGANN-92. International Workshop on,pages 1–37. IEEE, 1992.

[86] M. Schmitt. On the size of weights for mcculloch-pitts neurons. In Proceedings of the

Sixth Italian Workshop on Neural Nets WIRN VIETRI-93, pages 241–246. Citeseer,1994.

[87] E. Schoonderwaldt, N. Rasamimanana, and F. Bevilacqua. Combining accelerome-ter and video camera: Reconstruction of bow velocity profiles. In Proceedings of

the 2006 conference on New interfaces for musical expression, pages 200–203. IR-CAM—Centre Pompidou, 2006.

[88] J. Schulkin. Rethinking homeostasis: Allostatic regulation in physiology and patho-

physiology. MIT Press, 2003.

[89] J. Schulkin. Allostasis, homeostasis, and the costs of physiological adaptation. Cam-bridge University Press, 2004.

[90] R. Schwarzer. Modeling health behavior change: How to predict and modify theadoption and maintenance of health behaviors. Applied Psychology, 57(1):1–29, 2008.

[91] K. Shaw, P. O’Rourke, C. Del Mar, and J. Kenardy. Psychological interventions foroverweight or obesity. Cochrane Database Syst Rev, 2(2), 2005.

[92] G. Sinoff and L. Ore. The barthel activities of daily living index: Self-reporting versusactual performance in the old-old (+75 years). Journal of the American Geriatrics

Society, 45(7):832–836, 1997.

[93] D. Smith and P. Stanford. A random walk in hamming space. In Neural Networks,

1990., 1990 IJCNN International Joint Conference on, pages 465–470. IEEE, 1990.

[94] D. Smith and P. Stanford. Human motion analysis: A review. In In Proceedings of the

IEEE Nonrigid and Articulated Motion Workshop, pages 90–102. IEEE, 1997.

84 Bibliography

[95] K. Srinivasan and D. Fisher. Machine learning approaches to estimating softwaredevelopment effort. Software Engineering, IEEE Transactions on, 21(2):126–137,1995.

[96] P. Sterling and J. Eyer. Allostasis: a new paradigm to explain arousal pathology. ineds. fisher s, reason j. Handbook of Life Stress, Cognition and Health, pages 629–649.

[97] A. A. Stone, C. A. Bachrach, J. B. Jobe, H. S. Kurtzman, and V. S. Cain. The science

of self-report: Implications for research and practice. Psychology Press, 1999.

[98] C. Strohrmann, H. Harms, G. Tröster, S. Hensler, and R. Müller. Out of the lab andinto the woods: kinematic analysis in running using wearable sensors. In Proceedings

of the 13th international conference on Ubiquitous computing, pages 119–122. ACM,2011.

[99] F.-T. Sun, C. Kuo, H.-T. Cheng, S. Buthpitiya, P. Collins, and M. Griss. Activity-awaremental stress detection using physiological sensors. In Mobile computing, applica-

tions, and services, pages 211–230. Springer, 2012.

[100] A. M. Swartz, S. J. Strath, D. R. Bassett, W. L. O Brien, G. A. King, and B. E.Ainsworth. Estimation of energy expenditure using csa accelerometers at hip andwrist sites. Medicine and Science in Sports and Exercise, 32(9; SUPP/1):S450–S456,2000.

[101] P. Terrier, K. Aminian, and Y. Schutz. Can accelerometry accurately predict the energycost of uphill/downhill walking? Ergonomics, 44(1):48–62, 2001.

[102] A. Vellido, P. J. Lisboa, and J. Vaughan. Neural networks in business: a survey ofapplications (1992–1998). Expert Systems with Applications, 17(1):51–70, 1999.

[103] P. H. Veltink, H. B. Bussman, F. Koelma, H. M. Franken, W. L. Martens, and R. C.Lummel. The feasibility of posture and movement detection by accelerometry. In In

Proceedings of the 15th annual international conference of the IEEE, Engineering in

Medicine and Biology Society. IEEE, 1993.

[104] M. V. Villarejo, B. G. Zapirain, and A. M. Zorrilla. A stress sensor based on galvanicskin response (gsr) controlled by zigbee. Sensors, 12(5):6075–6101, 2012.

[105] T. Webb, J. Joseph, L. Yardley, and S. Michie. Using the internet to promote healthbehavior change: a systematic review and meta-analysis of the impact of theoreticalbasis, use of behavior change techniques, and mode of delivery on efficacy. Journal of

medical Internet research, 12(1):e4, 2010.

[106] M. Weiser. The computer for the 21st century. Scientific american, 265(3):94–104,1991.

[107] G. J. Welk, J. A. Differding, R. W. Thompson, S. N. Blair, J. Dziura, and P. Hart.The utility of the digi-walker step counter to assess daily physical activity patterns.Medicine and science in sports and exercise, 32(9 Suppl):S481–8, 2000.

Bibliography 85

[108] J. Weston and C. Watkins. Multi-class support vector machines. Technical report,Citeseer, 1998.

[109] I. H. Witten and E. Frank. Data Mining: Practical machine learning tools and tech-

niques. Morgan Kaufmann, 2005.

[110] S. Wolfram. A new kind of science, volume 5. Wolfram media Champaign, 2002.

[111] Z. Yan, V. Subbaraju, D. Chakraborty, A. Misra, and K. Aberer. Energy-efficientcontinuous activity recognition on mobile phones: An activity-adaptive approach. InWearable Computers (ISWC), 2012 16th International Symposium on, pages 17–24.Ieee, 2012.

[112] J.-Y. Yang, J.-S. Wang, and Y.-P. Chen. Using acceleration measurements for activityrecognition: An effective learning algorithm for constructing neural classifiers. Pattern

recognition letters, 29(16):2213–2220, 2008.

[113] T. Zhang, J. Wang, P. Liu, and J. Hou. Fall detection by embedding an accelerometerin cellphone and using kfd algorithm. International Journal of Computer Science and

Network Security, 6(10):277–284, 2006.

[114] T. Zhang, J. Wang, L. Xu, and P. Liu. Fall detection by wearable sensor and one-classsvm algorithm. In Intelligent Computing in Signal Processing and Pattern Recogni-

tion, pages 858–863. Springer, 2006.

[115] X. Zhen-zhen and Z. Su-yu. A non-linear approximation of the sigmoid function basedfpga. In Proceedings of the 2011, International Conference on Informatics, Cyber-

netics, and Computer Engineering (ICCE2011) November 19–20, 2011, Melbourne,

Australia, pages 125–132. Springer, 2012.

activity recognition in resource-constrained pervasive...

Documents