automatically identifying lifestyle behaviours from sensecam images (invited talk at the london...
Post on 24-May-2015
185 Views
Preview:
DESCRIPTION
TRANSCRIPT
When using wearable cameras in ethnography,
we should analyse the data in 2 stages:
1) Iden<fy episodes/ac<vi<es
2) Categorise their behavioural type & context
British Heart Foundation Health Promotion Research Group
Aiden Doherty’s talk in 1 slide...
Microsoft Early career in computer science...
SenseCam Hodges, Ubicomp conf., 2006
My interest is now in public health...
Something from department? Maybe Charlie Cochrane reviews? Maybe
Paul WHO document? Maybe Heart Stats?
This talk reflects the work of many...
The Lancet 2013, 380(9859), pp. 2095-2128
Global burden of disease...
Use recent Lancet website
The Lancet 2013, 380(9859), pp. 2095-2128
Main diseases are lifestyle related...
Highlight main diseases are lifestyle related
Sallis 2000, Ann. Behav. Med. 22(4):294-298
Behavioural epidemiology framework...
Sallis & Owen, highlight measurement
Troiano; Med Sci Sport Exer 2008; 40(1):181-8 Craig; NCSR 2008 – Health Survey for England
There is a big measurement problem...
Self report : 50% Accelerometer: 5%
Self-report : 38% Accelerometer: 5%
% adults meeting physical activity recommendations
Kelly, IJBNPA 2011, 8:44 Armstrong, Sprts Med 06, 36:1067–1086
Self-report has limitations... Recall error...
Comprehension problems... Social desirability error...
SenseCam Hodges, Ubicomp conf., 2006
Current objective measures limitations...
Show acc != behaviour slide … also GPS/GIS slides too
SenseCam Hodges, Ubicomp conf., 2006
Wearable cameras identify behaviours...
Video of wearable camera data
Mann, Computer 1997, 30; 25-32 Tano, ICME conf 2006; 649-652
Past efforts focused on hardware storage and miniaturisation...
Vicon Autographer
New devices are now smaller...
Memoto
New devices are now smaller...
Google Glass
Big companies like Google are now producing wearable cameras...
Gurrin 2013; Am J Prev Med 44(3):308-313
Smartphones too...
SenseCam
SenseCam Hodges, Ubicomp conf., 2006
Microsoft’s SenseCam is the most popular in health research...
- Active travel - Sedentary behaviour - Nutrition - Environment exposures - Contextualising other sensors
Doherty 2013, Am J Prev Med 43(5), 320 – 323
Wearable cameras in health...
Kelly, Intl J Behav Nutr Phys Act 2011, 8:44
Active travel – U.K. NTS diary...
Kelly, Intl J Behav Nutr Phys Act 2011, 8:44
Journey time = 20 minutes
Active travel – self-report entry...
Kelly, Intl J Behav Nutr Phys Act 2011, 8:44
journey time = 12 min 38 sec Active travel – wearable camera entry...
All journeys +2 min 34 sec (S.E. 32 sec)
Car +2 min 08 sec (S.E. 60 sec)
Walk +1 min 41 sec (S.E. 45 sec)
Bike +4 min 33 sec (S.E. 64 sec)
Kelly, Intl J Behav Nutr Phys Act 2011, 8:44
Active travel – adults’ self-report error...
test
Then show Paul Journey2School...
Kelly, 2012, Am J Prev Med 43(5), 546 – 550
Self-report error in adolescents' travel...
test
Then show Paul Journey2School...
Kerr, 2013, Am J Prev Med 44(3), 290 – 296
Sedentary behaviour...
test
Then show Paul Journey2School...
Kerr, 2013, Am J Prev Med 44(3), 290 – 296
Sedentary behaviour... Images were classifıed as “un-codeable” when the cameralens was obstructed.
Data AnalysisSenseCam codes were aggre-gated to the minute level. Avalid minute was defıned ashaving the same posture andactivity codes for the fırst andlast image within the minute.SenseCam minute-level andaccelerometer minute-leveldata were merged using thetime-stamp data for each unit.Time spent in sedentary be-havior postures and activitiesas coded by the SenseCamwere compared to the classifı-cation from the accelerome-ter. Minutes in each behavior,minutes under the 100-cpmthreshold, and mean countsin each behavior type werecalculated. The sensitivity andspecifıcity of the 100-cpm cut-point was compared with theSenseCam-derived classifıca-tion of sedentary behavior, calculated for the whole data set andeach participant. All analyses were performed using SPSS,version 19.
ResultsA total of 40 participants completed the study. Each par-ticipant contributed a mean of 4 days of data; 70% weremale; more than one third reported an annual householdincome of!$100,000; average agewas 36 years (SD!12);and 85% were Caucasian. A total of 170 days were coded,including 86,109 valid minutes and 364,841 images. Atotal of 8546 minutes of images (8.2%) were classifıed as“uncodeable,” such as when the camera lens was ob-scured by an item of clothing or body part. It took trainedcoders approximately 2 hours to code 1 day of data in-cluding the six postures and 12 activity types. Participantswere compliant with the wear instructions and did notreport any problems with the devices. Several partici-pants deleted a small number of images in the reviewprocess. Many participants were fascinated by the tech-nology and research applications.
The coded image data were compared to the accelero-meter data using the 100-cpm threshold. On average, theaccelerometer cutpoint classifıed participants in seden-tary behavior for 331.2minutes per day (SD!135.8). Theaverage minutes coded as sedentary per day from theSenseCam imageswas 302.2 (SD!130.0). Themajority ofminutes were spent in a sitting posture (Table 1).
The accelerometer 100 cpm correctly identifıed “sit-ting” 90% of the time. However, when the SenseCamimage indicated “standing without movement” or“standing with movement,” the accelerometer recorded"100 cpm 72% and 35% of the time, respectively. Elevenpercent of SenseCam bicycling had "100 cpm on theaccelerometer. Overall, the sensitivity and specifıcity forthe whole data set were 90% and 67%, respectively. Themean sensitivity across participants was 89% (SD!7%),and the mean specifıcity was 69% (SD!11%).
Table 2 presents the various activities that occurredwhen the SenseCam images were coded as “sitting.” Themost-prevalent behavior categories, as determined by thenumber of minutes recorded by the SenseCam, were(1) other screen use; (2) administrative activity; (3) TVwatching; (4) eating; and (5) riding in a car. TV viewingconstituted 11% of the total sedentary time, compared withother screen use, which made up 46% of total observedsedentary time. The accelerometer cutpoint of 100 cpmwasaccurate almost 90% of the time in classifying each of thesefour individual behaviors. At least 26%of the time observedin a car had accelerometer counts#100 cpm.
DiscussionThis study is the fırst to assess the convergent validity of awidely used accelerometer-based cutpoint for classifyingsedentary behavior over several days of free-living behav-ior in adults using a discrete observation method. The
Table 2. Minutes of coded sedentary posture from Microsoft’s SenseCam by activitycategory
Image code Minutes
Percent timein
accelerometercpm "100
Interquartilerange of
accelerometercpm
Meanaccelerometer
cpm
Sports 0 — 1525–2900 2330
Self care 85 60 2–361 284
Manual labor 202 35 41–669 432
Conditioning exercise 230 21 85–1405 1262
Household activity 244 58 10–305 260
Riding in other vehicle 409 82 0–50 90
Leisure 428 81 0–73 91
Riding in car 4,653 74 8–103 101
Eating 5,250 92 0–4 54
TV watching 5,407 89 0–11 46
Administrative activity 9,546 92 0–10 55
Other screen use 22,881 93 0–0 34
cpm, counts per minute
4 Kerr et al / Am J Prev Med 2013;xx(x):xxx
www.ajpmonline.org
test
Then show Paul Journey2School...
Kerr, 2013, Am J Prev Med 44(3), 290 – 296
Sedentary behaviour... Images were classifıed as “un-codeable” when the cameralens was obstructed.
Data AnalysisSenseCam codes were aggre-gated to the minute level. Avalid minute was defıned ashaving the same posture andactivity codes for the fırst andlast image within the minute.SenseCam minute-level andaccelerometer minute-leveldata were merged using thetime-stamp data for each unit.Time spent in sedentary be-havior postures and activitiesas coded by the SenseCamwere compared to the classifı-cation from the accelerome-ter. Minutes in each behavior,minutes under the 100-cpmthreshold, and mean countsin each behavior type werecalculated. The sensitivity andspecifıcity of the 100-cpm cut-point was compared with theSenseCam-derived classifıca-tion of sedentary behavior, calculated for the whole data set andeach participant. All analyses were performed using SPSS,version 19.
ResultsA total of 40 participants completed the study. Each par-ticipant contributed a mean of 4 days of data; 70% weremale; more than one third reported an annual householdincome of!$100,000; average agewas 36 years (SD!12);and 85% were Caucasian. A total of 170 days were coded,including 86,109 valid minutes and 364,841 images. Atotal of 8546 minutes of images (8.2%) were classifıed as“uncodeable,” such as when the camera lens was ob-scured by an item of clothing or body part. It took trainedcoders approximately 2 hours to code 1 day of data in-cluding the six postures and 12 activity types. Participantswere compliant with the wear instructions and did notreport any problems with the devices. Several partici-pants deleted a small number of images in the reviewprocess. Many participants were fascinated by the tech-nology and research applications.The coded image data were compared to the accelero-
meter data using the 100-cpm threshold. On average, theaccelerometer cutpoint classifıed participants in seden-tary behavior for 331.2minutes per day (SD!135.8). Theaverage minutes coded as sedentary per day from theSenseCam imageswas 302.2 (SD!130.0). Themajority ofminutes were spent in a sitting posture (Table 1).
The accelerometer 100 cpm correctly identifıed “sit-ting” 90% of the time. However, when the SenseCamimage indicated “standing without movement” or“standing with movement,” the accelerometer recorded"100 cpm 72% and 35% of the time, respectively. Elevenpercent of SenseCam bicycling had "100 cpm on theaccelerometer. Overall, the sensitivity and specifıcity forthe whole data set were 90% and 67%, respectively. Themean sensitivity across participants was 89% (SD!7%),and the mean specifıcity was 69% (SD!11%).Table 2 presents the various activities that occurred
when the SenseCam images were coded as “sitting.” Themost-prevalent behavior categories, as determined by thenumber of minutes recorded by the SenseCam, were(1) other screen use; (2) administrative activity; (3) TVwatching; (4) eating; and (5) riding in a car. TV viewingconstituted 11% of the total sedentary time, compared withother screen use, which made up 46% of total observedsedentary time. The accelerometer cutpoint of 100 cpmwasaccurate almost 90% of the time in classifying each of thesefour individual behaviors. At least 26%of the time observedin a car had accelerometer counts#100 cpm.
DiscussionThis study is the fırst to assess the convergent validity of awidely used accelerometer-based cutpoint for classifyingsedentary behavior over several days of free-living behav-ior in adults using a discrete observation method. The
Table 2. Minutes of coded sedentary posture from Microsoft’s SenseCam by activitycategory
Image code Minutes
Percent timein
accelerometercpm "100
Interquartilerange of
accelerometercpm
Meanaccelerometer
cpm
Sports 0 — 1525–2900 2330
Self care 85 60 2–361 284
Manual labor 202 35 41–669 432
Conditioning exercise 230 21 85–1405 1262
Household activity 244 58 10–305 260
Riding in other vehicle 409 82 0–50 90
Leisure 428 81 0–73 91
Riding in car 4,653 74 8–103 101
Eating 5,250 92 0–4 54
TV watching 5,407 89 0–11 46
Administrative activity 9,546 92 0–10 55
Other screen use 22,881 93 0–0 34
cpm, counts per minute
4 Kerr et al / Am J Prev Med 2013;xx(x):xxx
www.ajpmonline.org
Images were classifıed as “un-codeable” when the cameralens was obstructed.
Data AnalysisSenseCam codes were aggre-gated to the minute level. Avalid minute was defıned ashaving the same posture andactivity codes for the fırst andlast image within the minute.SenseCam minute-level andaccelerometer minute-leveldata were merged using thetime-stamp data for each unit.Time spent in sedentary be-havior postures and activitiesas coded by the SenseCamwere compared to the classifı-cation from the accelerome-ter. Minutes in each behavior,minutes under the 100-cpmthreshold, and mean countsin each behavior type werecalculated. The sensitivity andspecifıcity of the 100-cpm cut-point was compared with theSenseCam-derived classifıca-tion of sedentary behavior, calculated for the whole data set andeach participant. All analyses were performed using SPSS,version 19.
ResultsA total of 40 participants completed the study. Each par-ticipant contributed a mean of 4 days of data; 70% weremale; more than one third reported an annual householdincome of!$100,000; average agewas 36 years (SD!12);and 85% were Caucasian. A total of 170 days were coded,including 86,109 valid minutes and 364,841 images. Atotal of 8546 minutes of images (8.2%) were classifıed as“uncodeable,” such as when the camera lens was ob-scured by an item of clothing or body part. It took trainedcoders approximately 2 hours to code 1 day of data in-cluding the six postures and 12 activity types. Participantswere compliant with the wear instructions and did notreport any problems with the devices. Several partici-pants deleted a small number of images in the reviewprocess. Many participants were fascinated by the tech-nology and research applications.
The coded image data were compared to the accelero-meter data using the 100-cpm threshold. On average, theaccelerometer cutpoint classifıed participants in seden-tary behavior for 331.2minutes per day (SD!135.8). Theaverage minutes coded as sedentary per day from theSenseCam imageswas 302.2 (SD!130.0). Themajority ofminutes were spent in a sitting posture (Table 1).
The accelerometer 100 cpm correctly identifıed “sit-ting” 90% of the time. However, when the SenseCamimage indicated “standing without movement” or“standing with movement,” the accelerometer recorded"100 cpm 72% and 35% of the time, respectively. Elevenpercent of SenseCam bicycling had "100 cpm on theaccelerometer. Overall, the sensitivity and specifıcity forthe whole data set were 90% and 67%, respectively. Themean sensitivity across participants was 89% (SD!7%),and the mean specifıcity was 69% (SD!11%).
Table 2 presents the various activities that occurredwhen the SenseCam images were coded as “sitting.” Themost-prevalent behavior categories, as determined by thenumber of minutes recorded by the SenseCam, were(1) other screen use; (2) administrative activity; (3) TVwatching; (4) eating; and (5) riding in a car. TV viewingconstituted 11% of the total sedentary time, compared withother screen use, which made up 46% of total observedsedentary time. The accelerometer cutpoint of 100 cpmwasaccurate almost 90% of the time in classifying each of thesefour individual behaviors. At least 26%of the time observedin a car had accelerometer counts#100 cpm.
DiscussionThis study is the fırst to assess the convergent validity of awidely used accelerometer-based cutpoint for classifyingsedentary behavior over several days of free-living behav-ior in adults using a discrete observation method. The
Table 2. Minutes of coded sedentary posture from Microsoft’s SenseCam by activitycategory
Image code Minutes
Percent timein
accelerometercpm "100
Interquartilerange of
accelerometercpm
Meanaccelerometer
cpm
Sports 0 — 1525–2900 2330
Self care 85 60 2–361 284
Manual labor 202 35 41–669 432
Conditioning exercise 230 21 85–1405 1262
Household activity 244 58 10–305 260
Riding in other vehicle 409 82 0–50 90
Leisure 428 81 0–73 91
Riding in car 4,653 74 8–103 101
Eating 5,250 92 0–4 54
TV watching 5,407 89 0–11 46
Administrative activity 9,546 92 0–10 55
Other screen use 22,881 93 0–0 34
cpm, counts per minute
4 Kerr et al / Am J Prev Med 2013;xx(x):xxx
www.ajpmonline.org
Arab, Eu J Clin Nutr 2011;65(10):1156–62 O’Loughlin 2013 Am J Prev Med 44(3), 297 – 301
Nutrition...
O’Loughlin 2013 Am J Prev Med 44(3), 297 – 301
Nutrition...
diary that matched the information obtained from theSenseCam.Mean total calorie intake amounts using diary alone
when compared with the combination of the diaryplus SenseCam were as follows: 2349!827.9 kcal vs2631!893.4 kcal for the trainee jockeys; 2600!521.9 kcalvs 3190!770.2 kcal for the Gaelic footballers; and2237!318.5 kcal vs 2487!404.6 kcal for the universitystudents (Figure 2). Differences among measurementmethods of 10.7% (p!0.001); 17.7% (p!0.001); and10.1% (p!0.01) were reported for the trainee jockeys,Gaelic footballers, and university students, respectively(Table 1).
DiscussionThe results of this initial study suggest that the SenseCammay provide useful benefıts in terms of augmenting estab-lished techniques for recording energy intake. The imagesobtained from the SenseCam provide additional informa-tion regarding dietary intake patterns and highlighted a sig-nifıcant under-reporting of calorie intake, ranging from
10% to 18% in the popula-tions studied. To the au-thors’ knowledge, no pre-vious research has beenpublished examining thepotential use of the Sense-Cam for dietary assess-ment. The current studythus stands as a proof ofconcept, indicating the po-tential value of the Sense-Cam in dietary analysis.The automatic nature
of the SenseCam offersmany advantages over
studies using a user-activated camera to log foods. Re-questing that individuals manually photograph before–after images ofmeals is dependent on subject compliance,is potentially intrusive, and requires the individual toremember not only the camera but also to take photo-graphs at appropriate times. Previous studies13–15,24 havereported no general advantage in terms of accuracy inusing a conventional camera to capture dietary intakeover food diaries because of participant burden and in-complete photographic food records.The process of recording food consumption has been
shown previously to influence habitual dietary practicesto the extent that alternate choices may be made, as par-ticipants are aware that their diary will be analyzed.25During the current study, each subject wore the Sense-Cam and simultaneously completed a food diary for a1-day period. When conducting dietary analysis overmore extended periods, the SenseCammay becomemorevaluable because of decreased enthusiasm for diary log-ging by participants over time. Previous studies pub-lished by the current authors’ research group have used a7-day food diary to gain insight into the dietary practicesof jockeys. However, as with all dietary assessment meth-ods, under-reporting was evident.21 Use of the SenseCamprovides a mechanism to check the accuracy of the diaryand, because it works automatically, is not subject toreporting biases.
Figure 1. Images from the Microsoft SenseCam showing food consumed by the subjectNote: (a) an individual’s dinner, allowing the extra condiment about to be put on the plate to beremembered; (b) a breakfast image allowing questions regarding portion size
4000
3500 C
alor
ie in
take
3000
2500
2000
1500
1000
0 500
UniversityTraineejockeys
Gaelicfootballers students
Diary alone Diary and SenseCam
****
*
Figure 2. Comparison of energy intake using two assess-ment methods: diary alone, and diary with MicrosoftSenseCam*p!0.01, **p!0.001
Table 1. Mean difference in energy intake between twodietary assessment methods, M!SD
Mean difference(kcal)
Mean difference(%)
Trainee jockeys 282!164 10.7**
Gaelic footballers 591!304 17.7**
University students 250!216 10.1*
*p!0.01,**p!0.001
O’Loughlin et al / Am J Prev Med 2013;xx(x):xxx 3
Month 2013
O’Loughlin 2013 Am J Prev Med 44(3), 297 – 301
Nutrition...
diary that matched the information obtained from theSenseCam.
Mean total calorie intake amounts using diary alonewhen compared with the combination of the diaryplus SenseCam were as follows: 2349!827.9 kcal vs2631!893.4 kcal for the trainee jockeys; 2600!521.9 kcalvs 3190!770.2 kcal for the Gaelic footballers; and2237!318.5 kcal vs 2487!404.6 kcal for the universitystudents (Figure 2). Differences among measurementmethods of 10.7% (p!0.001); 17.7% (p!0.001); and10.1% (p!0.01) were reported for the trainee jockeys,Gaelic footballers, and university students, respectively(Table 1).
DiscussionThe results of this initial study suggest that the SenseCammay provide useful benefıts in terms of augmenting estab-lished techniques for recording energy intake. The imagesobtained from the SenseCam provide additional informa-tion regarding dietary intake patterns and highlighted a sig-nifıcant under-reporting of calorie intake, ranging from
10% to 18% in the popula-tions studied. To the au-thors’ knowledge, no pre-vious research has beenpublished examining thepotential use of the Sense-Cam for dietary assess-ment. The current studythus stands as a proof ofconcept, indicating the po-tential value of the Sense-Cam in dietary analysis.
The automatic natureof the SenseCam offersmany advantages over
studies using a user-activated camera to log foods. Re-questing that individuals manually photograph before–after images ofmeals is dependent on subject compliance,is potentially intrusive, and requires the individual toremember not only the camera but also to take photo-graphs at appropriate times. Previous studies13–15,24 havereported no general advantage in terms of accuracy inusing a conventional camera to capture dietary intakeover food diaries because of participant burden and in-complete photographic food records.
The process of recording food consumption has beenshown previously to influence habitual dietary practicesto the extent that alternate choices may be made, as par-ticipants are aware that their diary will be analyzed.25During the current study, each subject wore the Sense-Cam and simultaneously completed a food diary for a1-day period. When conducting dietary analysis overmore extended periods, the SenseCammay becomemorevaluable because of decreased enthusiasm for diary log-ging by participants over time. Previous studies pub-lished by the current authors’ research group have used a7-day food diary to gain insight into the dietary practicesof jockeys. However, as with all dietary assessment meth-ods, under-reporting was evident.21 Use of the SenseCamprovides a mechanism to check the accuracy of the diaryand, because it works automatically, is not subject toreporting biases.
Figure 1. Images from the Microsoft SenseCam showing food consumed by the subjectNote: (a) an individual’s dinner, allowing the extra condiment about to be put on the plate to beremembered; (b) a breakfast image allowing questions regarding portion size
4000
3500
Cal
orie
inta
ke
3000
2500
2000
1500
1000
0 500
UniversityTraineejockeys
Gaelicfootballers students
Diary alone Diary and SenseCam
****
*
Figure 2. Comparison of energy intake using two assess-ment methods: diary alone, and diary with MicrosoftSenseCam*p!0.01, **p!0.001
Table 1. Mean difference in energy intake between twodietary assessment methods, M!SD
Mean difference(kcal)
Mean difference(%)
Trainee jockeys 282!164 10.7**
Gaelic footballers 591!304 17.7**
University students 250!216 10.1*
*p!0.01,**p!0.001
O’Loughlin et al / Am J Prev Med 2013;xx(x):xxx 3
Month 2013
Arab, Eu J Clin Nutr 2011;65(10):1156–62 O’Loughlin 2013 Am J Prev Med 44(3), 297 – 301
Assessment of environment features...
that individuals may encounter during walking or cyclingjourneys. Although the current dataset was derived from alimited number of participants and geographic area, allhypothesised features of importance identified from theaudit tools were identified from the images captured. Thetendency to under-report journey duration is in contrastwith previous SenseCam research [22,32] and Global Posi-tioning System (GPS) studies [33], possibly due the smallconvenience sample and focus on work related walkingand cycling trips only. We found significant differences inthe presence of specific features between walking andcycling modes, suggesting preliminary support for the con-tent validity of this approach. For example, a significantlygreater proportion of footpaths, pedestrians, and pedes-trian crossings were found for walking trips, while a higher
prevalence of car presence was found for cycling journeys.With the exception of cycle lanes, all significant differencesbetween features identified by walking and cycling were inthe expected direction. The lack of cycle lanes in the studyareas may explain this finding somewhat, whereby manycycling journeys were completed on roads without cyclelanes. Improving on existing audits that do not reflect tem-poral exposure, use of the SenseCam data enabled the cap-ture of factors that individuals actually encountered duringactive transport journeys, such as traffic density; weatherconditions; presence of pedestrians, cyclists, and dogs; andtemporary obstructions to walking or cycling.Almost a quarter of data were lost due to images being
too dark to enable coding of features. In part, this is likelydue to the study being conducted during winter, with
Cars driving, pedestrians, pedestrian crossing, rain, road good condition, trees
Cars driving, cycle lane, dark, other lights, pedestrian crossing, road good condition
Trees, walkway
Trees, walkway Cars driving, cycle lane, footpath, road goodcondition
Congested traffic, cars driving, footpath, footpath good condition, grass verge, grass
verge maintained, residential, retail buildings, road good condition, trees
Figure 1 Sample images and exemplar coding of features present. Note: Data were collected in Auckland, New Zealand, in June 2011.
Table 3 Journey duration characteristicsTrip duration (minutes) Cycling (n = 8) Walking (n = 21) Total (n = 29)
Mean (min, max) Mean (min, max) Mean (min, max)
Reported duration 20.1 (15.0, 53.0) 22.0 (10.0, 45.0) 21.5 (10.0, 53.0)
SenseCam duration 21.3 (9.9, 56.6) 22.3 (9.6, 60.0) 21.7 (9.6, 56.6)
Difference (Reported – SenseCam) !1.1 (!3.6, 5.2) !0.2 (!11.1, 8.6) !0.42 (!11.1, 8.6)
Notes: Data were collected in Auckland, New Zealand, in June 2011.n = number of journeys.One walking journey was extracted for the comparison between reported and SenseCam trip duration as this was not recorded on the travel diary and wouldhave biased the comparisons.
Oliver et al. International Journal of Health Geographics 2013, 12:20 Page 5 of 7http://www.ij-healthgeographics.com/content/12/1/20
Oliver, 2013 Int J Health Geographic 12(20)
Assessment of environment features...
on journeys between home and workplace only was a prag-matic decision to ensure manageability of data treatmentand acknowledging the significant contribution work-relatedtravel makes to overall travel behaviours. Travel diaries wereconsidered the criterion for occurrence and mode of trips
undertaken. Although GPS data can be used to identifywalking and cycling journeys, trip purpose is not captured,therefore the travel diary was deemed an appropriate mea-sure of work-related journey occurrence for the purposes ofthe current study. SenseCam images were considered the
Table 1 Description of environmental features present in walking and cycling journeysFeature Description Cycling
(n = 599†)Walking
(n = 1150†)Total
(n = 1749†)
n (%) n (%) n (%)
Bus stop Bus stop visible in photo 44 7.3 69 6.0 113 6.5
Cars driving Cars in motion or in traffic lanes on road 388 64.8 674 58.6* 1062 60.7
Cars in carpark Cars parked in car park wholly or more than2/3 partially visible
68 11.4 110 9.6 178 10.2
Cars parked Cars parked on side of the road 190 31.7 151 13.1** 341 19.5
Commercial Commercial or institutional buildings visible 281 46.9 648 56.3** 929 53.1
Congested traffic More than 6 stationary cars in driving lanes 4 0.7 10 0.9 14 0.8
Cycle lanes Designated cycle lane on road or footpath 16 2.7 247 21.5** 263 15.0
Cyclists Any person/people riding cycles other than the participant 6 1.0 8 0.7 14 0.8
Dark Image indicates journey conducted in darkness(e.g., dusk or dawn, streetlights on) but features stillvisible and image codeable†
120 20.0 209 18.2 329 18.8
Dogs Dogs or a lead in participant hand visible 0 0.0 4 0.3 4 0.2
Footpath Footpath visible (not walkway/pathway) 338 56.4 761 66.2** 1099 62.8
Footpath good condition No cracks or potholes visible 327 54.6 759 66.0** 1086 62.1
Graffiti Graffiti visible 0 0.0 2 0.2 2 0.1
Grass verge Any area of grass either beside road or footpath 270 45.1 504 43.8 774 44.3
Grass verge maintained No obvious weeds or overgrown grass 262 43.7 454 39.5 716 40.9
Litter Litter present (e.g., paper, food wrappings, etc.) 1 0.2 1 0.1 2 0.1
Other lights Lights from houses, buildings or cars in photos 247 41.2 348 30.3** 595 34.0
Pedestrian crossing Zebra crossings and traffic light pedestrian crossings visible 82 13.7 240 20.9** 322 18.4
Pedestrians Any person/people in the photo other than the participant 63 10.5 272 23.7** 335 19.2
Permanent obstructionsto cycling
Tree, signage, or other permanent structure in cycleway 2 0.3 0 0.0 2 0.1
Permanent obstructionsto walking
Tree, signage, or other permanent structure onfootpath/walkway
2 0.3 0 0.0 2 0.1
Rain Rain visible 63 10.5 54 4.7** 117 6.7
Residential Private homes visible 155 25.9 229 19.9** 384 22.0
Retail buildings Buildings with retail/shop-fronts visible 141 23.5 165 14.3** 306 17.5
Road good condition No cracks or potholes visible 462 77.1 820 71.3** 1282 73.3
Street lighting Street lights visible (not including traffic lights) 209 34.9 531 46.2** 740 42.3
Temporary obstructionsto cycling
Rubbish bins, parked cars, roadworks, etc. in cycleways 9 1.5 11 1.0 20 1.1
Temporary obstructionsto walking
Rubbish bins, parked cars, roadworks, etc. onfootpath/walkway
14 2.3 41 3.6 55 3.1
Trees Any trees visible in photo including from a distance 441 73.6 842 73.2 1283 73.4
Walkway Journey occurring in walkway/pathway(not road or footpath)
45 7.5 200 17.4** 245 14.0
Notes: Data were collected in Auckland, New Zealand, in June 2011.n = number of images.*p < 0.05; **p < 0.01 significant difference in features present between walking and cycling journeys; †If photo was too dark to code individual features then it wascoded as uncodeable and not included here; %, percentage of walking or cycling images where feature was present; n, number of images.
Oliver et al. International Journal of Health Geographics 2013, 12:20 Page 3 of 7http://www.ij-healthgeographics.com/content/12/1/20
Example features: Bus stop Cycle lanes Graffiti Grass verge Rain Street lighting Trees
49 participants wore SenseCam for 3 days each, 311 accelerometer bouts randomly selected 79% of episodes could be coded (n=311)
- 14% had no associated image data (n=57) - 3% unsure of activity from images (n=10) - 2% images too dark to code (n=7)
Doherty, Intl J Behv Nutr Phys Act 2013 10(22)
Wearable cameras contextualising accelerometer data
Doherty, Intl J Behv Nutr Phys Act 2013 10(22)
Wearable cameras with accelerometers
311 episodes coded - 12 PA Compendium categories identified - 114 PA compendium subcategories identified
59% outdoors, 39% indoors 33% leisure time, 33% transportation, 18% domestic, 15% occupational 45% episodes non-social, 33% direct social, 22% social/not-engaged
Doherty, Intl J Behv Nutr Phys Act 2013 10(22)
Wearable cameras contextualising accelerometer data
Doherty, Intl J Behv Nutr Phys Act 2013 10(22)
Activity type... Activity type...
Doherty, Intl J Behv Nutr Phys Act 2013 10(22)
Activity context... Activity context...
“… Manual coding of the images is also time-consuming and coding errors can occur … ” 150 participants, 7 days of living, 2000 images per day = 2.1 million images !!!
Kerr 2013; Am J Prev Med 44(3): 290-296
How to manage all these images?...
Framework based on organising data into events
Human memory:
“… segmenting ongoing activity into distinct events is important for later memory of those activities …“ (Zacks 2006, Psychology and Aging 21:466-482)
Video retrieval:
“… automatic shot boundary detection (SBD) is an enabling function for almost all automatic structuring of video…” (Smeaton 2010, Computer Vision and Image Understanding 114(4):411-418)
Wearable accelerometers:
“… for comparison with physical activity recommendations, 10-min activity bouts were defined as 10 or more consecutive minutes above the relevant threshold …” (Troiano 2008, Med Sci Sport Exer 40(1):181-8)
Early lifelogging:
“… continuous recordings need to be segmented into manageable units so that they can be efficiently browsed and indexed …” (Lin 2006, Proc. SPIE MM Content Analysis, Management, and Retrieval)
British Heart Foundation Health Promotion Research Group
Wearable camera data management?...
Doherty 2013. IEEE Pervasive 12(1); 44-47
Processing wearable camera data...
Doherty 2013. IEEE Pervasive 12(1); 44-47
Processing wearable camera data...
Doherty 2013. IEEE Pervasive 12(1); 44-47
Processing wearable camera data...
Doherty 2011; Memory 19(7):785-795
Event segmentation overview...
Take WIAMIS / Yahoo! Research slides showing the nuts ‘n bolts of this…
Doherty 2011; Memory 19(7):785-795
Event segmentation overview...
Data divided into training and test sets with thousands of different combinations evaluated
From groundtruth we noticed: Average of 22 events groundtruthed per day
Approach Recommended: Quick segmentation (sensor values only)
Performance: F1 score of 60% against users’ semantic boundaries
Doherty 2011; Memory 19(7):785-795
Event segmentation – how good is it?...
Take Mediamill SVM slides showing the nuts ‘n bolts of this… highlight groundtruth construction, etc.
Staudenmayer 2012; MSSE 44(1):S61-67
Event identification overview...
Take Mediamill SVM slides showing the nuts ‘n bolts of this… highlight groundtruth construction, etc.
Staudenmayer 2012; MSSE 44(1):S61-67
We could use accelerometer signals...
Take Mediamill SVM slides showing the nuts ‘n bolts of this… highlight groundtruth construction, etc.
Staudenmayer 2012; MSSE 44(1):S61-67
We could use accelerometer signals...
Video of image classification using a support vector machine …
Snoek 2009; Fnd Tnd Inf Ret; 2(4):215-322
Could also use images too...
Take Mediamill SVM slides showing the nuts ‘n bolts of this… highlight groundtruth construction, etc.
Doherty 2011; Com Hm Behv 27(5):1948-1958
Event identification overview...
Doherty; Am J Prev Med 2013; 43(5), 320 – 323
http://ajpmonline.wordpress.com/2013/04/15/using-wearable-cameras-in-your-research/
When using wearable cameras in ethnography, we should: 1) Iden<fy episodes/ac<vi<es & 2) Categorise their behavioural type & context
top related