slice&dice: recognizing food preparation activities using embedded accelerometers cuong pham...
TRANSCRIPT
![Page 1: Slice&Dice: recognizing food preparation activities using embedded accelerometers Cuong Pham & Patrick Olivier Culture Lab School of Computing Science](https://reader036.vdocuments.us/reader036/viewer/2022062517/56649eda5503460f94be9a80/html5/thumbnails/1.jpg)
Slice&Dice: recognizing food preparation activities using embedded accelerometers
Cuong Pham & Patrick OlivierCulture Lab
School of Computing ScienceNewcastle University
![Page 2: Slice&Dice: recognizing food preparation activities using embedded accelerometers Cuong Pham & Patrick Olivier Culture Lab School of Computing Science](https://reader036.vdocuments.us/reader036/viewer/2022062517/56649eda5503460f94be9a80/html5/thumbnails/2.jpg)
Overview
Introduction Instrumented utensils Activity Recognition framework Experiment
Data collection & annotation Evaluation
Reflections
![Page 3: Slice&Dice: recognizing food preparation activities using embedded accelerometers Cuong Pham & Patrick Olivier Culture Lab School of Computing Science](https://reader036.vdocuments.us/reader036/viewer/2022062517/56649eda5503460f94be9a80/html5/thumbnails/3.jpg)
Introduction: Ambient Kitchen project
Goal: help people with dementia live more independent by providing situated services and prompting based on context recognition
Kitchen context: what people are doing objects people are interacting (i.e. food
ingredients) user locations etc.
![Page 4: Slice&Dice: recognizing food preparation activities using embedded accelerometers Cuong Pham & Patrick Olivier Culture Lab School of Computing Science](https://reader036.vdocuments.us/reader036/viewer/2022062517/56649eda5503460f94be9a80/html5/thumbnails/4.jpg)
Introduction: Ambient Kitchen project
Ambient kitchen: a lab-based ambient intelligence environment, designed using high fidelity prototype.
![Page 5: Slice&Dice: recognizing food preparation activities using embedded accelerometers Cuong Pham & Patrick Olivier Culture Lab School of Computing Science](https://reader036.vdocuments.us/reader036/viewer/2022062517/56649eda5503460f94be9a80/html5/thumbnails/5.jpg)
Introduction: prior work
Sensors worn on different parts of users body [Bao2004, Tapia2007, Ravi2005].
Detected outdoor activities such as running, walking, climbing, cycling etc. or high level activities[Wu2007]
Data collected under laboratory [Ravi2005] or semi-realistic conditions [Bao2004]
People with dementia needed fine-grained prompts to complete low-level activities [Wherton2008]
![Page 6: Slice&Dice: recognizing food preparation activities using embedded accelerometers Cuong Pham & Patrick Olivier Culture Lab School of Computing Science](https://reader036.vdocuments.us/reader036/viewer/2022062517/56649eda5503460f94be9a80/html5/thumbnails/6.jpg)
Introduction: system requirements
Detect low-level activities Sensors hidden from users No wires The cost & ease of deployment Comfortable-to-use Reasonable accuracy
![Page 7: Slice&Dice: recognizing food preparation activities using embedded accelerometers Cuong Pham & Patrick Olivier Culture Lab School of Computing Science](https://reader036.vdocuments.us/reader036/viewer/2022062517/56649eda5503460f94be9a80/html5/thumbnails/7.jpg)
Instrumented utensils: Wii ADXL330
A thin, low power, 3-axis accelerometer
Signal conditioned voltage outputs Dynamic acceleration can be
measured motion, shock and vibration
Acceleration can be measured in a range of ±3g
![Page 8: Slice&Dice: recognizing food preparation activities using embedded accelerometers Cuong Pham & Patrick Olivier Culture Lab School of Computing Science](https://reader036.vdocuments.us/reader036/viewer/2022062517/56649eda5503460f94be9a80/html5/thumbnails/8.jpg)
Instrumented utensils
Modified Wii Remotes were embedded in the kitchen utensils
![Page 9: Slice&Dice: recognizing food preparation activities using embedded accelerometers Cuong Pham & Patrick Olivier Culture Lab School of Computing Science](https://reader036.vdocuments.us/reader036/viewer/2022062517/56649eda5503460f94be9a80/html5/thumbnails/9.jpg)
Activity Recognition Framework
Data Communication & Processing acceleration data X, Y, Z sent to the computer
through a bluetooth device pitch and roll were computed for each triple
X,Y,Z Data Segmentation
data stream were segmented into 32, 64, 128, 256, and 512 sample windows
50% overlap between two consecutive windows.
![Page 10: Slice&Dice: recognizing food preparation activities using embedded accelerometers Cuong Pham & Patrick Olivier Culture Lab School of Computing Science](https://reader036.vdocuments.us/reader036/viewer/2022062517/56649eda5503460f94be9a80/html5/thumbnails/10.jpg)
Activity Recognition Framework
Feature Computation Mean Standard deviation Energy Entropy
Classification algorithms (from Weka Lib) Decision Tree C4.5 Bayesian Networks Naïve Bayes
![Page 11: Slice&Dice: recognizing food preparation activities using embedded accelerometers Cuong Pham & Patrick Olivier Culture Lab School of Computing Science](https://reader036.vdocuments.us/reader036/viewer/2022062517/56649eda5503460f94be9a80/html5/thumbnails/11.jpg)
Experiment: data collection
20 subjects 5 IP cameras 4 utensils: 3 knives and one serving spoon Given ingredients: potatoes, tomatoes,
lettuce, carrots, onions, kiwi fruit, grapefruit, peppers, bread, and butter
No instruction and no time-constrained to the subjects
Task: prepare a mixed salad and sandwich
![Page 12: Slice&Dice: recognizing food preparation activities using embedded accelerometers Cuong Pham & Patrick Olivier Culture Lab School of Computing Science](https://reader036.vdocuments.us/reader036/viewer/2022062517/56649eda5503460f94be9a80/html5/thumbnails/12.jpg)
Experiment: data annotation
Collected videos were annotated using Anvil Multimodal Tool [Kipp2001]
![Page 13: Slice&Dice: recognizing food preparation activities using embedded accelerometers Cuong Pham & Patrick Olivier Culture Lab School of Computing Science](https://reader036.vdocuments.us/reader036/viewer/2022062517/56649eda5503460f94be9a80/html5/thumbnails/13.jpg)
Experiment: example
![Page 14: Slice&Dice: recognizing food preparation activities using embedded accelerometers Cuong Pham & Patrick Olivier Culture Lab School of Computing Science](https://reader036.vdocuments.us/reader036/viewer/2022062517/56649eda5503460f94be9a80/html5/thumbnails/14.jpg)
Experiment: example
![Page 15: Slice&Dice: recognizing food preparation activities using embedded accelerometers Cuong Pham & Patrick Olivier Culture Lab School of Computing Science](https://reader036.vdocuments.us/reader036/viewer/2022062517/56649eda5503460f94be9a80/html5/thumbnails/15.jpg)
Experiment: data annotation
Dataset B annotated by one coder
Dataset A independently annotated by three coders only regions where all there coders agreed
were extracted Dataset B is larger than dataset A, but
dataset A is more consistent than dataset B
![Page 16: Slice&Dice: recognizing food preparation activities using embedded accelerometers Cuong Pham & Patrick Olivier Culture Lab School of Computing Science](https://reader036.vdocuments.us/reader036/viewer/2022062517/56649eda5503460f94be9a80/html5/thumbnails/16.jpg)
Experiment: subject independent evaluation
Trained 19 subjects Tested the remaining one Repeated the process for 20 subjects Finally, aggregated the results Subject to test was not included in the
training dataset
![Page 17: Slice&Dice: recognizing food preparation activities using embedded accelerometers Cuong Pham & Patrick Olivier Culture Lab School of Computing Science](https://reader036.vdocuments.us/reader036/viewer/2022062517/56649eda5503460f94be9a80/html5/thumbnails/17.jpg)
Experiment: evaluation results
Algorithm Dataset A Dataset B
Decision Tree 82.9 77.2
Bayesian nets 78.9 71.3
Naïve Bayes 52.4 73.5
Best accuracies were achieved on window size of 256-sample
![Page 18: Slice&Dice: recognizing food preparation activities using embedded accelerometers Cuong Pham & Patrick Olivier Culture Lab School of Computing Science](https://reader036.vdocuments.us/reader036/viewer/2022062517/56649eda5503460f94be9a80/html5/thumbnails/18.jpg)
Experiment: evaluation analysis
Peeling and stirring were highly distinctive (more than 90%)
Chopping, slicing, coring, scooping performed really good (around 80-90%)
Eating, spreading, shaving, scraping and dicing were below 80%: eating sometimes misclassified as scooping spreading sometimes misclassified as shaving
and coring dicing often misclassified as chopping
![Page 19: Slice&Dice: recognizing food preparation activities using embedded accelerometers Cuong Pham & Patrick Olivier Culture Lab School of Computing Science](https://reader036.vdocuments.us/reader036/viewer/2022062517/56649eda5503460f94be9a80/html5/thumbnails/19.jpg)
Reflection
Low-level food preparation activities can be reliably recognized using sensors embedded in kitchen utensils
Our work will continue with finding features most impact on algorithm
performance detecting objects developing Models
![Page 20: Slice&Dice: recognizing food preparation activities using embedded accelerometers Cuong Pham & Patrick Olivier Culture Lab School of Computing Science](https://reader036.vdocuments.us/reader036/viewer/2022062517/56649eda5503460f94be9a80/html5/thumbnails/20.jpg)
Thank you for your attention
Q&A