sensor analysis – part ii
DESCRIPTION
Sensor Analysis – Part II. A literature based exploration Thomas Plötz [material taken from the original papers and John Krumm “Ubiquitous Computing Fundamentals”. sports: training / reviewing movements after the “fact” - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Sensor Analysis – Part II](https://reader035.vdocuments.us/reader035/viewer/2022062812/5681637c550346895dd459e3/html5/thumbnails/1.jpg)
Sensor Analysis – Part II
A literature based explorationThomas Plötz
[material taken from the original papers and John Krumm “Ubiquitous Computing Fundamentals”
![Page 2: Sensor Analysis – Part II](https://reader035.vdocuments.us/reader035/viewer/2022062812/5681637c550346895dd459e3/html5/thumbnails/2.jpg)
![Page 3: Sensor Analysis – Part II](https://reader035.vdocuments.us/reader035/viewer/2022062812/5681637c550346895dd459e3/html5/thumbnails/3.jpg)
![Page 4: Sensor Analysis – Part II](https://reader035.vdocuments.us/reader035/viewer/2022062812/5681637c550346895dd459e3/html5/thumbnails/4.jpg)
![Page 5: Sensor Analysis – Part II](https://reader035.vdocuments.us/reader035/viewer/2022062812/5681637c550346895dd459e3/html5/thumbnails/5.jpg)
• sports: training / reviewing movements after the “fact”• daily routine: efficiency (where do you waste time?) – quantified self movement
![Page 6: Sensor Analysis – Part II](https://reader035.vdocuments.us/reader035/viewer/2022062812/5681637c550346895dd459e3/html5/thumbnails/6.jpg)
vanKasteren2008-AARvanKasteren2010-TKOPatterson2005-FGA Perkowitz2004-MMO
![Page 7: Sensor Analysis – Part II](https://reader035.vdocuments.us/reader035/viewer/2022062812/5681637c550346895dd459e3/html5/thumbnails/7.jpg)
Loga
n200
7-AL
T
![Page 8: Sensor Analysis – Part II](https://reader035.vdocuments.us/reader035/viewer/2022062812/5681637c550346895dd459e3/html5/thumbnails/8.jpg)
Ward2006-ARO
![Page 9: Sensor Analysis – Part II](https://reader035.vdocuments.us/reader035/viewer/2022062812/5681637c550346895dd459e3/html5/thumbnails/9.jpg)
ADL well studied and recognized segmentation + classification variety of application domains (health, smart homes, …)
- no fine-grained analysis (read: trivial? Beyond proof-of-concept?)- no quality assessment (How good? vs. What?)
… your conclusions here!
![Page 10: Sensor Analysis – Part II](https://reader035.vdocuments.us/reader035/viewer/2022062812/5681637c550346895dd459e3/html5/thumbnails/10.jpg)
![Page 11: Sensor Analysis – Part II](https://reader035.vdocuments.us/reader035/viewer/2022062812/5681637c550346895dd459e3/html5/thumbnails/11.jpg)
![Page 12: Sensor Analysis – Part II](https://reader035.vdocuments.us/reader035/viewer/2022062812/5681637c550346895dd459e3/html5/thumbnails/12.jpg)
inexpensive non-obtrusive, privacy respecting (typically …) even wearable
- overkill?
… your conclusions here!
![Page 13: Sensor Analysis – Part II](https://reader035.vdocuments.us/reader035/viewer/2022062812/5681637c550346895dd459e3/html5/thumbnails/13.jpg)
![Page 14: Sensor Analysis – Part II](https://reader035.vdocuments.us/reader035/viewer/2022062812/5681637c550346895dd459e3/html5/thumbnails/14.jpg)
• Activity Recognition of Assembly Tasks Using Body-Worn Microphones and Accelerometers• classical Activity Recognition – ISWC-style
![Page 15: Sensor Analysis – Part II](https://reader035.vdocuments.us/reader035/viewer/2022062812/5681637c550346895dd459e3/html5/thumbnails/15.jpg)
![Page 16: Sensor Analysis – Part II](https://reader035.vdocuments.us/reader035/viewer/2022062812/5681637c550346895dd459e3/html5/thumbnails/16.jpg)
Sound Intensity Analysis for Segmentation of Activities – analysis of two microphone signals (I1 – wrist ; I2 – upper arm)
Sound Classification (which tool used?) based on spectral analysis and templates (after smoothing)
![Page 17: Sensor Analysis – Part II](https://reader035.vdocuments.us/reader035/viewer/2022062812/5681637c550346895dd459e3/html5/thumbnails/17.jpg)
• Spectral representation of acceleration data, calculated using sliding window technique
• HMM-based classification of acceleration data
• different fusion techniques for integrating hypotheses from acoustic acoustic recognition and accelerator-based processing
![Page 18: Sensor Analysis – Part II](https://reader035.vdocuments.us/reader035/viewer/2022062812/5681637c550346895dd459e3/html5/thumbnails/18.jpg)
pioneering work combination of modalities• explicit segmentation step• focus on classification techniques and fusion (NOT on feature representation)
![Page 19: Sensor Analysis – Part II](https://reader035.vdocuments.us/reader035/viewer/2022062812/5681637c550346895dd459e3/html5/thumbnails/19.jpg)
• Fine-Grained Activity Recognition by Aggregating Abstract Object Usage• object-centered sensing and recognition (in contrast to ISWC-style)
![Page 20: Sensor Analysis – Part II](https://reader035.vdocuments.us/reader035/viewer/2022062812/5681637c550346895dd459e3/html5/thumbnails/20.jpg)
Independent HMMs connected HMMs Object-centered HMMs
[DBN with aggregates]
![Page 21: Sensor Analysis – Part II](https://reader035.vdocuments.us/reader035/viewer/2022062812/5681637c550346895dd459e3/html5/thumbnails/21.jpg)
![Page 22: Sensor Analysis – Part II](https://reader035.vdocuments.us/reader035/viewer/2022062812/5681637c550346895dd459e3/html5/thumbnails/22.jpg)
![Page 23: Sensor Analysis – Part II](https://reader035.vdocuments.us/reader035/viewer/2022062812/5681637c550346895dd459e3/html5/thumbnails/23.jpg)
![Page 24: Sensor Analysis – Part II](https://reader035.vdocuments.us/reader035/viewer/2022062812/5681637c550346895dd459e3/html5/thumbnails/24.jpg)
![Page 25: Sensor Analysis – Part II](https://reader035.vdocuments.us/reader035/viewer/2022062812/5681637c550346895dd459e3/html5/thumbnails/25.jpg)
• Transferring Knowledge of Activity Recognition across Sensor Networks
• What if we change the environment?
![Page 26: Sensor Analysis – Part II](https://reader035.vdocuments.us/reader035/viewer/2022062812/5681637c550346895dd459e3/html5/thumbnails/26.jpg)
• list of ADLs is the same for each house, but sensor networks differ
1. How to deal with differences in sensor networks (due to different layouts of houses)?2. How to learn model parameters that respect differences in behavior of the inhabitants?
![Page 27: Sensor Analysis – Part II](https://reader035.vdocuments.us/reader035/viewer/2022062812/5681637c550346895dd459e3/html5/thumbnails/27.jpg)
![Page 28: Sensor Analysis – Part II](https://reader035.vdocuments.us/reader035/viewer/2022062812/5681637c550346895dd459e3/html5/thumbnails/28.jpg)
• Modeling approach: HMM• “A prior distribution is learned from the source houses and used to provide a sensible initial value for the model parameters of the target house.”• conjugate prior: posterior is of same functional form as prior• procedure:
1. learn model parameters for source house (using ML)2. Learn hyperparameters for initial state, and transition is straightforward using
numerical estimation3. Learn hyperparameters for observations requires feature space mapping and
numerical estimation
![Page 29: Sensor Analysis – Part II](https://reader035.vdocuments.us/reader035/viewer/2022062812/5681637c550346895dd459e3/html5/thumbnails/29.jpg)
• Finally: EM-based estimation of target model parameters exploiting estimated priors
![Page 30: Sensor Analysis – Part II](https://reader035.vdocuments.us/reader035/viewer/2022062812/5681637c550346895dd459e3/html5/thumbnails/30.jpg)
• Mining Models of Human Activities from the Web
• What if we don’t have enough (or none) sample data?
“ […] requires models of the activities of interest, but model construction does not scale well: humans must specify low- level details, such as segmentation and feature selection of sensor data, and high-level structure, such as spatio-temporal relations between states of the model, for each and every activity.”
![Page 31: Sensor Analysis – Part II](https://reader035.vdocuments.us/reader035/viewer/2022062812/5681637c550346895dd459e3/html5/thumbnails/31.jpg)
• “[…] we show how to mine very large libraries of human activities from the web, instead of analyzing sensor data.”• “PROACT assumes that “interesting” objects in the environment contain RFID tags.”• “Users employ RFID tag readers to track tag objects they interact with.”• “As users go about their daily activities, the readers detect tags that (a) users touch, (b) are close to them, or (c) are moved by them, and thereby deduce which objects are currently involved in an activity. PROACT uses the sequence and timing of object involvement to deduce what activity is happening.”
![Page 32: Sensor Analysis – Part II](https://reader035.vdocuments.us/reader035/viewer/2022062812/5681637c550346895dd459e3/html5/thumbnails/32.jpg)
• Modeling based on Hidden (semi) Markov Models
• general model description via web-mining
• extract initial parameters (probabilities)using Google conditional probabilities (GCP)
P(o, i, l) = GoogleCount(“l”+o) / GoogleCount(l)
P(o, i, l): prob. that object o is involved in step i of the activity labeled l.
![Page 33: Sensor Analysis – Part II](https://reader035.vdocuments.us/reader035/viewer/2022062812/5681637c550346895dd459e3/html5/thumbnails/33.jpg)
![Page 34: Sensor Analysis – Part II](https://reader035.vdocuments.us/reader035/viewer/2022062812/5681637c550346895dd459e3/html5/thumbnails/34.jpg)
![Page 35: Sensor Analysis – Part II](https://reader035.vdocuments.us/reader035/viewer/2022062812/5681637c550346895dd459e3/html5/thumbnails/35.jpg)
Problem: Skewed distributions (one very dominant class)
![Page 36: Sensor Analysis – Part II](https://reader035.vdocuments.us/reader035/viewer/2022062812/5681637c550346895dd459e3/html5/thumbnails/36.jpg)
Problem: Fragmentation; or: Does it really make sense to evaluate sample wise?
Idea: Event-based evaluation
![Page 37: Sensor Analysis – Part II](https://reader035.vdocuments.us/reader035/viewer/2022062812/5681637c550346895dd459e3/html5/thumbnails/37.jpg)
![Page 38: Sensor Analysis – Part II](https://reader035.vdocuments.us/reader035/viewer/2022062812/5681637c550346895dd459e3/html5/thumbnails/38.jpg)
![Page 39: Sensor Analysis – Part II](https://reader035.vdocuments.us/reader035/viewer/2022062812/5681637c550346895dd459e3/html5/thumbnails/39.jpg)
Bibliography• van Kasteren et al. Accurate activity recognition in a home setting. Proc. Int. Conf. on Ubiquitous Computing
(2008)
• Ward et al. Activity recognition of assembly tasks using body-worn microphones and accelerometers. IEEE Trans. Pattern Analysis and Machine Intelligence (2006) vol. 28 (10) pp. 1553--1567
• van Kasteren et al. Transferring Knowledge of Activity Recognition across Sensor Networks. Proc. Int. Conf. on Pervasive Computing (2010) pp. 283--300
• Perkowitz et al. Mining models of human activities from the web. Proc. 13th Int. Conf. on World Wide Web (2004) pp. 573--582
• Ward et al. Performance metrics for activity recognition. ACM Trans. on Intelligent Systems and Technology (2011) vol. 2 (1)
• Logan et al. A long-term evaluation of sensing modalities for activity recognition. Proc. Int. Conf. on Ubiquitous Computing (2007) pp. 483--500
• Patterson et al. Fine-Grained Activity Recognition by Aggregating Abstract Object Usage. Proc. IEEE Int. Symp. on Wearable Computers (2005)