1

2015-07-10 Slide 1 of 60

Designing for Seniors Prof. Ron Baecker The Technologies for Aging Gracefully lab (TAGlab) Department of Computer Science, University of Toronto Thanks to many present and past members of the lab and to our collaborators from across Canada and the U.S. HCII Invited Course Los Angeles, 4 August 2015

Research for the Journey through Life

!  The Technologies for Aging Gracefully lab (TAGlab) designs and develops tech to make seniors & families “smarter, i.e., more capable, resourceful, independent "  Not … making machines “smarter” to “watch over” seniors

!  Method … focus on needs "  Identify where technology could serve human needs

and enable greater inclusion in life and in society "  Envision, design, build, test, improve, & commercialize

solutions for conditions such as AD, MCI, stroke, MS, vision loss … and for normally aging senior citizens

2

2015-07-10

Identify real needs, then design and develop !  Diagnostic instruments that detect problems !  Prostheses that assist with or compensate for a loss !  Rehabilitative systems to strengthen or restore a facility !  Preventative interventions, e.g., to delay mental aging !  Ideally, technology interventions are informed by

medicine and the health sciences, and can in turn also contribute to the development of such science

The Problem and The Opportunity !  The good news: we are living longer (U.N., 2012)

"  1950: 5% of world population is over 60 "  2013: 12% "  2050: 21% "  2100: 28%

"  Years to enjoy family/friends, convey wisdom, reap rewards

!  The bad news "  Alzheimer’s disease (AD), aphasia from stroke, Parkinson’s,

blindness, deafness, poor mobility, etc. "  Sensory, motor, cognitive challenges "  Isolation, loneliness, vulnerability, depression

!  Can technology assist seniors? How?? … i.e., In response to what needs?

3

2015-07-10

Advances in Assistive Technology !  Visual challenges # Display substitution, re-creation

"  Screen readers, audio description, descriptive video, sonification "  Haptic displays "  Nuances: Acuity /= glaucoma /= macular degeneration

!  Hearing challenges # Sound substitution, re-creation "  Captions, sign language "  Haptic displays, e.g., of music (Tactical Audio Displays)

!  Mobility and wayfinding "  Location tracking, mobile guidance systems "  Obstacle avoidance, e.g, smart canes "  Transport systems, e.g., smart wheelchairs

!  Great stuff! But … the most critical needs for seniors??

Maslow Hierarchy of Human Needs

!  Self-actualization "  Need for a cause, calling,

vocation, fulfillment

!  Esteem "  Need to feel satisfied, self confident, valuable; to have

meaningful work and activities; to develop personally

!  Love "  Love, affection, sense of belonging, family, friends

!  Safety "  Feeling of safety, freedom from danger or perceived danger

!  Physiological "  Oxygen, food, water, warmth, health, fitness, seeing, hearing,

mobility

4

2015-07-10

Physiological Needs

!  Seeking health information online !  Seeking health information via social media !  Keeping physically fit !  Maintaining a good diet !  Dealing with low vision !  Dealing with poor hearing !  Dealing with restricted mobility !  Facilitating breathing

2015-07-10

Safety Needs

!  Avoiding falls !  Getting up and about every day !  Avoiding environmental obstacles !  Turning off the stove !  Avoiding medication errors !  Dealing with multiple chronic diseases

5

2015-07-10

Love Needs

!  Staying connected with friends !  Staying connected with family !  Reconnecting to distant family and lost friends !  Making new friends !  Continuing to participate in community

2015-07-10

Esteem Needs

!  Being able to remember names !  Being able to speak !  Being able to read !  Remaining cognitively proficient !  Not forgetting where you have left an object !  Being able to write !  Being able to create something, e.g., music

6

2015-07-10

Meaning in Life

!  Understanding where you came from !  Telling your story, helping create your family’s history !  Thinking about and preparing for the end of life !  Giving something back !  Finding meaning in life

2015-07-10 Slide 12 of 60

Physiological Needs

!  Seeking health information online !  Seeking health information via social media !  Keeping physically fit !  Maintaining a good diet !  Dealing with low vision !  Dealing with poor hearing !  Dealing with restricted mobility !  Facilitating breathing

7

Sample Health Websites: PubMed Health, Mayo Clinic, WebMD

2015-07-10

An Example of Social Media for Health

2015-07-10

8

FitBit to Monitor Physical Activity for Health

2015-07-10

Seniors Keeping WiiFit (or Kinected)

2015-07-10

9

2015-07-10

Questions For Discussion

!  Do these technologies empower seniors? !  How do they change the way medicine is practiced? !  What is sufficient evidence for technology adoption?

2015-07-10 Slide 18 of 60

Safety Needs

!  Avoiding falls !  Getting up and about every day !  Avoiding environmental obstacles !  Turning off the stove !  Avoiding medication errors !  Dealing with multiple chronic diseases

10

Fall Detection (commercial products, Alex Mihailidis)

2015-07-10

Electricity/Water Flow Monitor (Shwetak Patel)

2015-07-10

11

2015-07-10

Crowdsourcing for Safety (Jeff Bigham et al.)

!  Large-scale information gathering !  Example 1: Improving public transit accessibility for

blind riders by crowdsourcing bus stop landmark locations with Google Streetview

!  Example 2: Answering visual questions with conversational crowd assistants, e.g., "  Do you see picnic tables across the parking lot? "  What temperature is my oven set to? "  Can you please tell me what this can is?

2015-07-10

Questions For Discussion

!  Are there disciplines other than computer science relevant to safety?

!  Why is medication safety such a complex issue?

12

2015-07-10 Slide 23 of 60

Love Needs

!  Staying connected with friends, e.g., Facebook !  Staying connected with family !  Reconnecting to distant family and lost friends !  Making new friends !  Continuing to participate in community

The Problem !  A personal story

!  Many individuals … "  Live alone, with little family and small social networks "  May have sensory and motor impairments "  May have little control over how they feel at a particular time

and when they are available for social contact

!  Examples … "  Seniors living alone, in retirement homes, in long-term care "  People in long-term hospitalization, rehab, quarantine "  Individuals with chronic pain, MS, TBI, ALS "  People in hospice care

"  7/24 home-bound caregivers

13

2015-07-10

Social Isolation and Loneliness

!  10 to 43% of community-dwelling older adults are socially isolated (Nicholson, 2012)

!  A cohort stury of 1,604 older adults in the U.S. shows that 43% feel lonely (Perissinotto, Cenzer, & Covinsky, 2012).

Consequences

!  Health effects "  Depression, morbidity, stress, functional decline, death

(Edelbrock et al., 2001; Perissinotto et al., 2012; Steptoe et al., 2013)

"  Health risks comparable to the dangers of smoking cigarettes and obesity (Cornwell & Waite, 2009)

!  Loneliness kills !! "  A recent 2010 meta-analysis of 148 studies reported

“50% increased likelihood of survival for participants with stronger social relationships” (Holt-Lunstad, Smith, & Layton, 2010)

!  Socio-economic burden for older adults, families, social institutions, government, taxpayers

14

Connecting Seniors … Bringing, Keeping Families Together !  Synchronous video chat

"  Skype, Google Hangouts, …

!  Asynchronous messaging "  Conventional email software "  Modern messaging apps, e.g., WeChat, WhatsApp

!  But we (TAGlab) thought we could do better … the result we call “InTouch”

Field Studies, Prototype Technology

!  Interview and diary studies with seniors "  Home dwellers in chronic pain "  Patients in complex continuing care hospitals "  People in retirement residences "  People in long-term care facilities "  Home health care patients

!  First deployment: home dwellers in chronic pain

15

Design Implications for InTouch

!  Design appliances, not software or interfaces !  Leverage pictures of family !  Focus on asynchronous messaging !  Support multimedia messaging, no need to type !  Use iconic communication, not verbal !  …

!  Currently, 4th version of the technology

InTouch

!  Touch frame, & send ‘wave’ (‘hi there’, request for contact), voice, photo, video messages; see messages from loved one

16

First Mixed Methods Pilot Study

!  1 older adult living in a retirement community (65+, F)

!  5 frail oldest old living in a long-term care

facility (average age = 87.2, 3 F and 2 M).

Preliminary Results

!  High perceived usefulness of InTouch for their interaction with relatives

!  Reduced feelings of being ‘left out’

!  Increased perceived levels of interaction with relatives (e.g., grandchildren)

17

Opportunities and Challenges

!  Ergonomic and digital literacy challenges

!  Need for social support and family buy-in

!  Different kinds of adoption and use, varying social and cultural expectations

!  Christie Gardens, Circle of Care, other studies

!  Opportunity: Supporting not just social goals, but also health goals and practical goal, i.e., enabling seniors to stay at home (famli.net Communications Inc.)

2015-07-10

Questions For Discussion

!  How do the needs for love and for health interact? !  How do various stakeholders and actors contribute to

increasing the health and happiness of seniors? !  Please share your experiences with respect to this

area.

18

2015-07-10 Slide 35 of 60

Esteem Needs

!  Being able to remember names !  Being able to speak !  Being able to read !  Remaining cognitively proficient !  Not forgetting where you have left an object !  Being able to write !  Being able to create something, e.g., music

2015-07-10

Friend Forecaster

!  Predict the names of individuals who you are comparatively likely to meet in a location.

19

2015-07-10

MyVoice Inc.

!  Aid for word/phrase recall and articulation

Accessible, Large-print, Listening and Talking e-Book (ALLT e-book) for Reading

20

39

ALLT

$  Accessible, Large print, Listening and Talking e-reader

$  Combats social isolation — reading together in the same place at different times �  Intelligent assistive recording & reading

$  Collaborative and assistive access to critical information �  Support older adults and their caregivers

40

Reading together

$  Audio e-book read in a familiar voice, recreating a pleasant moment �  Reading together “in the same place at different times”

$  Machine learning / speech recognition / interface design �  Facilitates transparent recording for the caregiver or family

member reading to an older adult �  Learning of reading speed �  Automatic synchronization �  Reduces playback/record

control buttons

21

41

Collaborative information access

$  Support for assisting family members and caregivers with helping older adults process critical information �  Rich annotations �  Interactive sticky notes �  Multimedia (text, audio, video)

personal messages and reminders

Brain Training Exercises Are Believed to Aid Cognitive Health and Delay Decline

!  Can we slow down cognitive aging, and delay the onset of Alzheimer�s disease?

!  A commercial frenzy ($250M in 2008 ==> up to $5B projected for 2015), huge promise, but huge controversy

22

Can We Slow Mental Aging? !  Evidence for brain plasticity in animal studies and

in rehab contexts suggests yes

!  �The concept of cognitive reserve (CR) suggests that innate intelligence or aspects of life experience like educational or occupational attainments may supply reserve, in the form of a set of skills or repertoires that allows some people to cope with progressing AD pathology better than others.� (Scarmeas and Stern, J. Clin. & Exp. Neuropsychology 25(5), 2003)

"  But … these are impacts of life experience accumulated over the long-term, over a lifetime

Controversy

!  Many studies do not disentangle effects of recent activities from those of lifelong activities

!  Only a few studies evaluate medium-term impacts "  ACTIVE cognitive training for memory, reasoning, or speed of

processing (Ball et al.) "  Baycrest cognitive rehabilitation for memory, goal management, and

psychosocial skills (Winocur, et al.)

!  Also, little evidence of transfer of training effects from the game task to other tasks

!  Thus we need more prospective research studies "  But they are very difficult and costly

23

2015-07-10

Questions For Discussion

!  Please share your experiences with respect to this area. !  What about “feeling OK with technology, i.e., not

helpless” as an esteem need for seniors? !  Will computer literacy and “mastery” go away as a

problem in a decade or two?

2015-07-10 Slide 46 of 60

Meaning in Life

!  Understanding where you came from (ancestry.com) !  Telling your story, helping create your family’s history !  Thinking about and preparing for the end of life !  Giving something back !  Finding meaning in life

24

Multimedia Biographies for AD (Baecker and Marziali)

!  Problem: Degenerative brain disease impacts memory, executive functioning, identity "  Who you were fades from your memory and families� memory "  Huge caregiver burden, enormous toll on families

!  Possible users "  400K Canadians with AD in 2010 ==> 750K in 2031 "  36M with dementia worldwide ==> 66M in 2030 (AD Int�l.)

!  Solution: Digital life histories to aid reminiscing "  For individuals with moderate or mild AD, or MCI "  To help the individual with AD AND the family to remember

and to re-experience, to connect and to feel close "  A practical production method to produce multimedia bios

Ms F (91, mid-stage AD)

25

Ms Z (85, mid-stage AD) and daughter

Research Finding and Challenges !  Findings

"  Participants enjoy the viewings "  Viewings enhance reminiscing "  Group viewing stimulates family conversation "  Repeated viewings help rekindle sense of identity

"  Biographies provide joy & emotional support to family members

"  Biography improves AD caregivers��ability to understand and empathize with the biographee

!  Challenges "  Gathering media, incl. SenseCam, Glass, social media "  Leveraging family resources to help transform

memories and experiences into stories

26

2015-07-10

Thanotosensitive Design (Mike Massimi)

!  Ferguson et al., Technology in hospice, CSCW2014 !  Massimi, Online bereavement support, CSCW2013

2015-07-10

Questions For Discussion

!  Does technology distract us from the meaning of life, e.g., perhaps we should read more books instead.

27

Slide 53 of 60

Lessons Learned !  It�s not just about memory or cognition, but about

identity and efficacy and self-worth

!  It�s not just for individuals, but for family, community, caregivers, and clinicians

!  The goal is not just prosthetics, but also rehabilitation of health and preservation of health

!  It’s highly multidisciplinary "  Computer science, neuropsychology, neurology, social work,

speech/language pathology, nursing, graphic design, …

!  Observing and studying real use by real users in real environments is the key to inspired design

2015-07-10 April 27, 2011 | Slide 54 of 52

Baecker Design Framework: Can Look Systematically for Design Opportunities

!  Goal !  Place in Maslow Hierarchy !  Cognitive/social process !  Users !  Other relevant stakeholders !  Use model, usage style !  Design method !  Technology

28

2015-07-10

Discussion Example: InTouch

!  Goal !  Place in Maslow Hierarchy !  Cognitive/social process !  Users !  Other relevant stakeholders !  Use model, usage style !  Design method !  Technology

2015-07-10

Research Challenge and Design Exercise

!  Remembering important objects, finding lost objects

29

2015-07-10

Research Challenge and Design Exercise

!  Detecting signs that warn of possible falls

2015-07-10

Synthesis Lectures on Assistive, Rehabilitative, & Health-Preserving Technologies (M&C)

30

Empowering Seniors for Additional Years of Contributions

2015-07-10

Thanks for your attention and contributions!! !  Email: ron@taglab.ca !  URL: http://taglab.utoronto.ca/

!  Thanks to past students & collaborators and to financial supporters "  Alzheimer�s Association (+ Intel Corp.) "  Connaught Innovation Fund, University of Toronto "  Google Research "  GRAND Network of Centres of Excellence of Canada "  Microsoft Research "  MyVoice Inc. "  NSERC "  OCE+CC "  Revera Inc.