Design and Evaluation of an Automated Coaching System for Elderly PopulationFerda Ofli +, Gregorij Kurillo +, Holly Jimison *, Misha Pavel *, Ruzena Bajcsy +

+Department of Electrical Engineering and Computer Science, University of California at Berkeley, Berkeley, CA* College of Computer and Information Science at Northeastern University, Boston, MA

Background and Motivation• Positive effects of exercise on the well-being and quality of independent living for older adults are

well-accepted.• Many elderly individuals, however, lack access to exercise facilities, or the skills and motivation to

perform exercise at home.• To address these issues, we developed an automated interactive exercise coaching system geared

towards elderly users based on the Microsoft Kinect camera.

New System Design• Re-design of user interface (UI) that features high contrast and visibility.• Reduction of information presented on each screen.• Richer selection of exercises, grouped by targeted body parts and difficulty level.• In-exercise feedback includes silhouette and skeleton captured by the Kinect.

Acknowledgements• National Science Foundation (NSF): Award Number 1111965: SHB: Large: Collaborative Research: Integrated Communications and Inference

Systems for Continuous Coordinated Care of Older Adults in the Home.

• Authors would like to thank Jennifer Marcoe and Paul Gorman of OHSU for feedback and contributions, Sue Scott for designing the exercise program, and the pilot study participants.

References• Š. Obdržálek, G. Kurillo, E. Seto, R. Bajcsy, "Architecture of an Automated Coaching System for Elderly Population", Stud Health Technol Inform. 184:309-11, 2013.

• Š. Obdržálek, G. Kurillo, F. Ofli, R. Bajcsy, E. Seto, H. Jimison, M. Pavel, "Accuracy and Robustness of Kinect Pose Estimation in the Context of Coaching of Elderly Population",EMBC, 34th International Conference of the IEEE Engineering in Medicine and Biology Society, San Diego, California, August 2012

• S. Obdržálek, G. Kurillo, J. Han; T. Abresch, R. Bajcsy, "Real-time human pose detection and tracking for tele-rehabilitation in virtual reality," Stud Health Technol Inform. 173: 320-4, 2012.

Figure 1: Elderly user interacting with the developed automated exercise coaching system (left). Overview of the data processing pipelinefor automated coaching (right). Kinect joint data are processed into higher-level performance measures which are in turn used to verifyexercise execution, count repetitions, and provide feedback on performance.

Conclusions• Deploying technology in homes of elderly poses many challenges.• Lessons learned from our pilot study include: (1) limitations on living space with respect to deployed

technology, (2) usability issues with the camera and software, (3) usability issues with the userinterface, (4) selecting form of visual and auditory feedback, (5) interpretation of physicalmeasurements from the captured kinematics in relation to clinical outcome measures and standardcomponents of physical fitness.

Figure 2: Example of the exercise analysis for Buddha’s Prayer: (a)performance evaluation is done by extracting primary measurementsrelated to the exercise; (b) exercise verification provides correctivefeedback; (c) stage detection tracks number of repetitions.

Figure 3: Example of extracted primary measurement (wristlift) for Buddha’s Prayer with stage detection (dark/lightshades) and corresponding key skeletal frames.

Pilot Study• We deployed initial coaching system into homes of six independently-living elderly individuals.• We examined issues related to the in-home system setup, user tracking, feedback, and exercise

performance evaluation to guide the design of the next version of the coaching system.

TABLE IIREPORTED USER FEEDBACK AND OBSERVATIONS FROM ACQUIRED DATA

Figure 4: User interface control flow displaying the main screens: (a) home screen, (b) daily survey, (c) exercise selection, (d) exercisepreview/instructions, (e) in-exercise feedback, (f) session summary.

(a) (b) (c)

(d) (e) (f)

Figure 5: Comparison of in-exercise feedback between the original coaching system (used in the pilot study) and the re-designed interface.