non-invasive techniques for human fatigue monitoring
DESCRIPTION
Non-invasive Techniques for Human Fatigue Monitoring. Qiang Ji Dept. of Electrical, Computer, and Systems Engineering Rensselaer Polytechnic Institute [email protected] http://www.ecse.rpi.edu/homepages/qji. Visual Behaviors. Visual behaviors that typically reflect a - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Non-invasive Techniques for Human Fatigue Monitoring](https://reader035.vdocuments.us/reader035/viewer/2022070423/568167c8550346895ddd1533/html5/thumbnails/1.jpg)
Non-invasive Techniques for Human Fatigue
MonitoringQiang Ji
Dept. of Electrical, Computer, and Systems Engineering
Rensselaer Polytechnic [email protected]
http://www.ecse.rpi.edu/homepages/qji
![Page 2: Non-invasive Techniques for Human Fatigue Monitoring](https://reader035.vdocuments.us/reader035/viewer/2022070423/568167c8550346895ddd1533/html5/thumbnails/2.jpg)
![Page 3: Non-invasive Techniques for Human Fatigue Monitoring](https://reader035.vdocuments.us/reader035/viewer/2022070423/568167c8550346895ddd1533/html5/thumbnails/3.jpg)
Visual BehaviorsVisual behaviors that typically reflect
a person's level of fatigue include
– Eyelid movement – Head movement – Gaze – Facial expressions
![Page 4: Non-invasive Techniques for Human Fatigue Monitoring](https://reader035.vdocuments.us/reader035/viewer/2022070423/568167c8550346895ddd1533/html5/thumbnails/4.jpg)
Eyelid Movements Tracking Eyes
Develop techniques that can robustly track eyes under different face orientations, illuminations, and large head movements.
Compute Eye movement parameters PERCLOS Average Eye Closure/Open Speed
(AECS)
![Page 5: Non-invasive Techniques for Human Fatigue Monitoring](https://reader035.vdocuments.us/reader035/viewer/2022070423/568167c8550346895ddd1533/html5/thumbnails/5.jpg)
Eyes tracking demo
![Page 6: Non-invasive Techniques for Human Fatigue Monitoring](https://reader035.vdocuments.us/reader035/viewer/2022070423/568167c8550346895ddd1533/html5/thumbnails/6.jpg)
PERCLOS measurement over time
![Page 7: Non-invasive Techniques for Human Fatigue Monitoring](https://reader035.vdocuments.us/reader035/viewer/2022070423/568167c8550346895ddd1533/html5/thumbnails/7.jpg)
Average Eye Closure Speed Over time
![Page 8: Non-invasive Techniques for Human Fatigue Monitoring](https://reader035.vdocuments.us/reader035/viewer/2022070423/568167c8550346895ddd1533/html5/thumbnails/8.jpg)
![Page 9: Non-invasive Techniques for Human Fatigue Monitoring](https://reader035.vdocuments.us/reader035/viewer/2022070423/568167c8550346895ddd1533/html5/thumbnails/9.jpg)
![Page 10: Non-invasive Techniques for Human Fatigue Monitoring](https://reader035.vdocuments.us/reader035/viewer/2022070423/568167c8550346895ddd1533/html5/thumbnails/10.jpg)
Gaze (Pupil Movements)
Real time gaze tracking No calibration is needed and
allows natural head movements !.
Gaze parameters Spatial gaze distribution overtime Ratio of fixation time to saccade time.
![Page 11: Non-invasive Techniques for Human Fatigue Monitoring](https://reader035.vdocuments.us/reader035/viewer/2022070423/568167c8550346895ddd1533/html5/thumbnails/11.jpg)
Gaze distribution over time while alert
![Page 12: Non-invasive Techniques for Human Fatigue Monitoring](https://reader035.vdocuments.us/reader035/viewer/2022070423/568167c8550346895ddd1533/html5/thumbnails/12.jpg)
Gaze distribution over time under fatigue
![Page 13: Non-invasive Techniques for Human Fatigue Monitoring](https://reader035.vdocuments.us/reader035/viewer/2022070423/568167c8550346895ddd1533/html5/thumbnails/13.jpg)
![Page 14: Non-invasive Techniques for Human Fatigue Monitoring](https://reader035.vdocuments.us/reader035/viewer/2022070423/568167c8550346895ddd1533/html5/thumbnails/14.jpg)
Head Movement Real time head pose tracking
Perform 3D face pose estimation from a single uncalibrated camera.
Head movement parameters Head tilt frequency over time Percentage of side views (PerSideV)
![Page 15: Non-invasive Techniques for Human Fatigue Monitoring](https://reader035.vdocuments.us/reader035/viewer/2022070423/568167c8550346895ddd1533/html5/thumbnails/15.jpg)
![Page 16: Non-invasive Techniques for Human Fatigue Monitoring](https://reader035.vdocuments.us/reader035/viewer/2022070423/568167c8550346895ddd1533/html5/thumbnails/16.jpg)
![Page 17: Non-invasive Techniques for Human Fatigue Monitoring](https://reader035.vdocuments.us/reader035/viewer/2022070423/568167c8550346895ddd1533/html5/thumbnails/17.jpg)
Facial Expressions Tracking facial features
Recognize certain facial expressions related to fatigue like yawning.
Building a database of fatigue expressions.
![Page 18: Non-invasive Techniques for Human Fatigue Monitoring](https://reader035.vdocuments.us/reader035/viewer/2022070423/568167c8550346895ddd1533/html5/thumbnails/18.jpg)
Facial expression demo
![Page 19: Non-invasive Techniques for Human Fatigue Monitoring](https://reader035.vdocuments.us/reader035/viewer/2022070423/568167c8550346895ddd1533/html5/thumbnails/19.jpg)
Fatigue Modeling• Knowledge of fatigue is uncertain and
from different levels of abstraction.
• Fatigue represents the affective state of an individual, is not observable, and can only be inferred.
![Page 20: Non-invasive Techniques for Human Fatigue Monitoring](https://reader035.vdocuments.us/reader035/viewer/2022070423/568167c8550346895ddd1533/html5/thumbnails/20.jpg)
Overview of Our Approach
Propose a probabilistic framework based on Bayesian Networks (BN) to
model fatigue.systematically integrate various
sources of information related to fatigue.
infer and predict fatigue from the available observations and the relevant contextual information.
![Page 21: Non-invasive Techniques for Human Fatigue Monitoring](https://reader035.vdocuments.us/reader035/viewer/2022070423/568167c8550346895ddd1533/html5/thumbnails/21.jpg)
Bayesian Networks Construction
• A BN model consists of target hypothesis variables (hidden nodes) and information variables (information nodes).
• Fatigue is the target hypothesis variable that we intend to infer.
• Other contextual factors and visual cues are the information nodes.
![Page 22: Non-invasive Techniques for Human Fatigue Monitoring](https://reader035.vdocuments.us/reader035/viewer/2022070423/568167c8550346895ddd1533/html5/thumbnails/22.jpg)
Causes for Fatigue
Major factors to cause fatigue include: Sleep quality. Circadian rhythm (time of day). Physical conditions. Working environment.
![Page 23: Non-invasive Techniques for Human Fatigue Monitoring](https://reader035.vdocuments.us/reader035/viewer/2022070423/568167c8550346895ddd1533/html5/thumbnails/23.jpg)
Bayesian Network Model for Monitoring Human Fatigue
![Page 24: Non-invasive Techniques for Human Fatigue Monitoring](https://reader035.vdocuments.us/reader035/viewer/2022070423/568167c8550346895ddd1533/html5/thumbnails/24.jpg)
Interface with Vision Interface with Vision ModuleModule
An interface has been developed to connect the output of the computer vision system with the information fusion engine.
The interface instantiates the evidences of the fatigue network, which then performs fatigue inference and displays the fatigue index in real time.
![Page 25: Non-invasive Techniques for Human Fatigue Monitoring](https://reader035.vdocuments.us/reader035/viewer/2022070423/568167c8550346895ddd1533/html5/thumbnails/25.jpg)
Conclusions
• Developed non-intrusive real-time computer vision techniques to extract multiple fatigue parameters related to eyelid movements, gaze, head movement, and facial expressions.
• Develop a probabilistic framework based on Bayesian networks to model and integrate contextual and visual cues information for fatigue monitoring.