headset control with assistive ai for limited mobility users
DESCRIPTION
Headset Control with Assistive AI for Limited Mobility Users. Frankie Pike Adviser: Dr. Franz Kurfess. Why I’m interested. Why I’m interested. But that’s EASY!. GPS = where we are Cameras = what we don’t want to hit Mind control = magic. Architecture. But that’s EASY!. - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Headset Control with Assistive AI for Limited Mobility Users](https://reader030.vdocuments.us/reader030/viewer/2022033104/56816150550346895dd0d9e2/html5/thumbnails/1.jpg)
HEADSET CONTROL WITH ASSISTIVE AI FOR LIMITED MOBILITY USERSFrankie Pike Adviser: Dr. Franz Kurfess
![Page 2: Headset Control with Assistive AI for Limited Mobility Users](https://reader030.vdocuments.us/reader030/viewer/2022033104/56816150550346895dd0d9e2/html5/thumbnails/2.jpg)
Why I’m interested
![Page 3: Headset Control with Assistive AI for Limited Mobility Users](https://reader030.vdocuments.us/reader030/viewer/2022033104/56816150550346895dd0d9e2/html5/thumbnails/3.jpg)
Why I’m interested
![Page 4: Headset Control with Assistive AI for Limited Mobility Users](https://reader030.vdocuments.us/reader030/viewer/2022033104/56816150550346895dd0d9e2/html5/thumbnails/4.jpg)
But that’s EASY!
GPS = where we are Cameras = what we don’t want to
hit Mind control = magic
![Page 5: Headset Control with Assistive AI for Limited Mobility Users](https://reader030.vdocuments.us/reader030/viewer/2022033104/56816150550346895dd0d9e2/html5/thumbnails/5.jpg)
Architecture
Consensus
Obstacle Detectio
n
User Commands
(EEG)GPS Path
Plotting
![Page 6: Headset Control with Assistive AI for Limited Mobility Users](https://reader030.vdocuments.us/reader030/viewer/2022033104/56816150550346895dd0d9e2/html5/thumbnails/6.jpg)
But that’s EASY!
![Page 7: Headset Control with Assistive AI for Limited Mobility Users](https://reader030.vdocuments.us/reader030/viewer/2022033104/56816150550346895dd0d9e2/html5/thumbnails/7.jpg)
But that’s EASY!
![Page 8: Headset Control with Assistive AI for Limited Mobility Users](https://reader030.vdocuments.us/reader030/viewer/2022033104/56816150550346895dd0d9e2/html5/thumbnails/8.jpg)
But that’s EASY!
![Page 9: Headset Control with Assistive AI for Limited Mobility Users](https://reader030.vdocuments.us/reader030/viewer/2022033104/56816150550346895dd0d9e2/html5/thumbnails/9.jpg)
Architecture
Consensus
Obstacle
Detection
User Commands
GPS Path
Plotting
![Page 10: Headset Control with Assistive AI for Limited Mobility Users](https://reader030.vdocuments.us/reader030/viewer/2022033104/56816150550346895dd0d9e2/html5/thumbnails/10.jpg)
eBay!
Multi-Agent Decision EngineAgents
Auctioneer
Message Feed
Auction
Timer
Bidders
OAS
3D Cameras
Long Range Planner
GPS Compass
User
EEG Filter Override
NavClasses
![Page 11: Headset Control with Assistive AI for Limited Mobility Users](https://reader030.vdocuments.us/reader030/viewer/2022033104/56816150550346895dd0d9e2/html5/thumbnails/11.jpg)
Agent Lifecycles
Compute
Decay
Start Auction
Receive Bids
Issue Directiv
e
Store Last
Directive
Call NavCla
ss
Evaluate
Solution
Submit Best Bid
Bidder Monitor
![Page 12: Headset Control with Assistive AI for Limited Mobility Users](https://reader030.vdocuments.us/reader030/viewer/2022033104/56816150550346895dd0d9e2/html5/thumbnails/12.jpg)
Architecture
Consensus
Obstacle
Detection
User Commands
GPS Path
Plotting
![Page 13: Headset Control with Assistive AI for Limited Mobility Users](https://reader030.vdocuments.us/reader030/viewer/2022033104/56816150550346895dd0d9e2/html5/thumbnails/13.jpg)
Architecture
Consensus
Obstacle
Detection
User Commands
GPS Path
Plotting
![Page 14: Headset Control with Assistive AI for Limited Mobility Users](https://reader030.vdocuments.us/reader030/viewer/2022033104/56816150550346895dd0d9e2/html5/thumbnails/14.jpg)
Architecture
Consensus
Obstacle
Detection
User Commands
GPS Path
Plotting
![Page 15: Headset Control with Assistive AI for Limited Mobility Users](https://reader030.vdocuments.us/reader030/viewer/2022033104/56816150550346895dd0d9e2/html5/thumbnails/15.jpg)
Architecture
Consensus
Obstacle
Detection
User Commands
GPS Path
Plotting
Partially Observable Environment
![Page 16: Headset Control with Assistive AI for Limited Mobility Users](https://reader030.vdocuments.us/reader030/viewer/2022033104/56816150550346895dd0d9e2/html5/thumbnails/16.jpg)
POMDPs Partially Observable Markov Decision
Processes Allow modeling with uncertainty
Uncertainty in state Uncertainty in transition
PSPACE-Hard (Go)
![Page 17: Headset Control with Assistive AI for Limited Mobility Users](https://reader030.vdocuments.us/reader030/viewer/2022033104/56816150550346895dd0d9e2/html5/thumbnails/17.jpg)
Xavier (Carnegie Mellon)
One of the first POMDP based 3000 states Periodic re-evaluation
60km of testing in offices
Compartmentalized Nav Landmark: 80% POMDP: 93%
Resilient to failure
![Page 18: Headset Control with Assistive AI for Limited Mobility Users](https://reader030.vdocuments.us/reader030/viewer/2022033104/56816150550346895dd0d9e2/html5/thumbnails/18.jpg)
RHINO
Tested in a museum Builds model of
environment Decentralized processing Markov models for
localization
V1: Glass is invisible to lasers What else is?
![Page 19: Headset Control with Assistive AI for Limited Mobility Users](https://reader030.vdocuments.us/reader030/viewer/2022033104/56816150550346895dd0d9e2/html5/thumbnails/19.jpg)
Collaborative Wheelchair Control Guided Follow, not push
Side by side Pass through door Follow behind
Heuristics for dense environments
![Page 20: Headset Control with Assistive AI for Limited Mobility Users](https://reader030.vdocuments.us/reader030/viewer/2022033104/56816150550346895dd0d9e2/html5/thumbnails/20.jpg)
HaWCoS (Hands-free Wheelchair Control System)
Non-autonomous Modest hardware (P3) Timed Tasks
EEG = 48% longer
Better with AI?
![Page 21: Headset Control with Assistive AI for Limited Mobility Users](https://reader030.vdocuments.us/reader030/viewer/2022033104/56816150550346895dd0d9e2/html5/thumbnails/21.jpg)
Doable and worthwhile