spatial routines for a simulated speech-controlled...
TRANSCRIPT
![Page 1: Spatial Routines for a Simulated Speech-Controlled Vehiclecs.brown.edu/people/stellex/publications/Tellex-Roy-HRI06-presentation.pdf · Generous Stingy Go left 90% 80% 0.12 Go right](https://reader034.vdocuments.us/reader034/viewer/2022042106/5e858b53ebb4a14e79496b43/html5/thumbnails/1.jpg)
Spatial Routines for a Simulated
Speech-Controlled Vehicle
Stefanie Tellex and Deb Roy
MIT Media Lab Cognitive Machines Group
February 27, 2006
![Page 2: Spatial Routines for a Simulated Speech-Controlled Vehiclecs.brown.edu/people/stellex/publications/Tellex-Roy-HRI06-presentation.pdf · Generous Stingy Go left 90% 80% 0.12 Go right](https://reader034.vdocuments.us/reader034/viewer/2022042106/5e858b53ebb4a14e79496b43/html5/thumbnails/2.jpg)
“Go left.”
![Page 3: Spatial Routines for a Simulated Speech-Controlled Vehiclecs.brown.edu/people/stellex/publications/Tellex-Roy-HRI06-presentation.pdf · Generous Stingy Go left 90% 80% 0.12 Go right](https://reader034.vdocuments.us/reader034/viewer/2022042106/5e858b53ebb4a14e79496b43/html5/thumbnails/3.jpg)
“Go left.”
![Page 4: Spatial Routines for a Simulated Speech-Controlled Vehiclecs.brown.edu/people/stellex/publications/Tellex-Roy-HRI06-presentation.pdf · Generous Stingy Go left 90% 80% 0.12 Go right](https://reader034.vdocuments.us/reader034/viewer/2022042106/5e858b53ebb4a14e79496b43/html5/thumbnails/4.jpg)
“Go left.”
![Page 5: Spatial Routines for a Simulated Speech-Controlled Vehiclecs.brown.edu/people/stellex/publications/Tellex-Roy-HRI06-presentation.pdf · Generous Stingy Go left 90% 80% 0.12 Go right](https://reader034.vdocuments.us/reader034/viewer/2022042106/5e858b53ebb4a14e79496b43/html5/thumbnails/5.jpg)
“Go right.”
![Page 6: Spatial Routines for a Simulated Speech-Controlled Vehiclecs.brown.edu/people/stellex/publications/Tellex-Roy-HRI06-presentation.pdf · Generous Stingy Go left 90% 80% 0.12 Go right](https://reader034.vdocuments.us/reader034/viewer/2022042106/5e858b53ebb4a14e79496b43/html5/thumbnails/6.jpg)
Execution Trace
“Go across the room.”
![Page 7: Spatial Routines for a Simulated Speech-Controlled Vehiclecs.brown.edu/people/stellex/publications/Tellex-Roy-HRI06-presentation.pdf · Generous Stingy Go left 90% 80% 0.12 Go right](https://reader034.vdocuments.us/reader034/viewer/2022042106/5e858b53ebb4a14e79496b43/html5/thumbnails/7.jpg)
Execution Trace
“Go across the room.”go(across(room))
![Page 8: Spatial Routines for a Simulated Speech-Controlled Vehiclecs.brown.edu/people/stellex/publications/Tellex-Roy-HRI06-presentation.pdf · Generous Stingy Go left 90% 80% 0.12 Go right](https://reader034.vdocuments.us/reader034/viewer/2022042106/5e858b53ebb4a14e79496b43/html5/thumbnails/8.jpg)
“Go across the room.”
![Page 9: Spatial Routines for a Simulated Speech-Controlled Vehiclecs.brown.edu/people/stellex/publications/Tellex-Roy-HRI06-presentation.pdf · Generous Stingy Go left 90% 80% 0.12 Go right](https://reader034.vdocuments.us/reader034/viewer/2022042106/5e858b53ebb4a14e79496b43/html5/thumbnails/9.jpg)
“Go across the room.”
![Page 10: Spatial Routines for a Simulated Speech-Controlled Vehiclecs.brown.edu/people/stellex/publications/Tellex-Roy-HRI06-presentation.pdf · Generous Stingy Go left 90% 80% 0.12 Go right](https://reader034.vdocuments.us/reader034/viewer/2022042106/5e858b53ebb4a14e79496b43/html5/thumbnails/10.jpg)
“Go across the room.”
![Page 11: Spatial Routines for a Simulated Speech-Controlled Vehiclecs.brown.edu/people/stellex/publications/Tellex-Roy-HRI06-presentation.pdf · Generous Stingy Go left 90% 80% 0.12 Go right](https://reader034.vdocuments.us/reader034/viewer/2022042106/5e858b53ebb4a14e79496b43/html5/thumbnails/11.jpg)
“Go across the room.”
![Page 12: Spatial Routines for a Simulated Speech-Controlled Vehiclecs.brown.edu/people/stellex/publications/Tellex-Roy-HRI06-presentation.pdf · Generous Stingy Go left 90% 80% 0.12 Go right](https://reader034.vdocuments.us/reader034/viewer/2022042106/5e858b53ebb4a14e79496b43/html5/thumbnails/12.jpg)
“Go across the room.”
![Page 13: Spatial Routines for a Simulated Speech-Controlled Vehiclecs.brown.edu/people/stellex/publications/Tellex-Roy-HRI06-presentation.pdf · Generous Stingy Go left 90% 80% 0.12 Go right](https://reader034.vdocuments.us/reader034/viewer/2022042106/5e858b53ebb4a14e79496b43/html5/thumbnails/13.jpg)
“Go across the room.”
![Page 14: Spatial Routines for a Simulated Speech-Controlled Vehiclecs.brown.edu/people/stellex/publications/Tellex-Roy-HRI06-presentation.pdf · Generous Stingy Go left 90% 80% 0.12 Go right](https://reader034.vdocuments.us/reader034/viewer/2022042106/5e858b53ebb4a14e79496b43/html5/thumbnails/14.jpg)
“Go across the room.”
![Page 15: Spatial Routines for a Simulated Speech-Controlled Vehiclecs.brown.edu/people/stellex/publications/Tellex-Roy-HRI06-presentation.pdf · Generous Stingy Go left 90% 80% 0.12 Go right](https://reader034.vdocuments.us/reader034/viewer/2022042106/5e858b53ebb4a14e79496b43/html5/thumbnails/15.jpg)
![Page 16: Spatial Routines for a Simulated Speech-Controlled Vehiclecs.brown.edu/people/stellex/publications/Tellex-Roy-HRI06-presentation.pdf · Generous Stingy Go left 90% 80% 0.12 Go right](https://reader034.vdocuments.us/reader034/viewer/2022042106/5e858b53ebb4a14e79496b43/html5/thumbnails/16.jpg)
RobChair NavChairStop StopGo forward ForwardGo backward Left/right (turn 30o)Rotate right/left Turn left/right (turn continuously)Hard right/left (turn 20o) Pass doorSoft right/left (turn 10o) Approach desk
Follow wall
Wheelesley Current SystemStop StopForward Go straightBack Face left/right (turn 90o)Left/right Go left/right
Turn aroundGo across the roomGo to the objectGo to the left/right of the object
![Page 17: Spatial Routines for a Simulated Speech-Controlled Vehiclecs.brown.edu/people/stellex/publications/Tellex-Roy-HRI06-presentation.pdf · Generous Stingy Go left 90% 80% 0.12 Go right](https://reader034.vdocuments.us/reader034/viewer/2022042106/5e858b53ebb4a14e79496b43/html5/thumbnails/17.jpg)
Visual Routines
◮ Ullman’s visual routines (Shimon Ullman, Visual Routines,1983)
◮ Rao’s thesis (Satyajit Rao, Visual Routines and Attention,1998)
![Page 18: Spatial Routines for a Simulated Speech-Controlled Vehiclecs.brown.edu/people/stellex/publications/Tellex-Roy-HRI06-presentation.pdf · Generous Stingy Go left 90% 80% 0.12 Go right](https://reader034.vdocuments.us/reader034/viewer/2022042106/5e858b53ebb4a14e79496b43/html5/thumbnails/18.jpg)
Datatypes - Grid
![Page 19: Spatial Routines for a Simulated Speech-Controlled Vehiclecs.brown.edu/people/stellex/publications/Tellex-Roy-HRI06-presentation.pdf · Generous Stingy Go left 90% 80% 0.12 Go right](https://reader034.vdocuments.us/reader034/viewer/2022042106/5e858b53ebb4a14e79496b43/html5/thumbnails/19.jpg)
Datatypes - Mask
![Page 20: Spatial Routines for a Simulated Speech-Controlled Vehiclecs.brown.edu/people/stellex/publications/Tellex-Roy-HRI06-presentation.pdf · Generous Stingy Go left 90% 80% 0.12 Go right](https://reader034.vdocuments.us/reader034/viewer/2022042106/5e858b53ebb4a14e79496b43/html5/thumbnails/20.jpg)
Datatypes - Paths
![Page 21: Spatial Routines for a Simulated Speech-Controlled Vehiclecs.brown.edu/people/stellex/publications/Tellex-Roy-HRI06-presentation.pdf · Generous Stingy Go left 90% 80% 0.12 Go right](https://reader034.vdocuments.us/reader034/viewer/2022042106/5e858b53ebb4a14e79496b43/html5/thumbnails/21.jpg)
Evaluation
![Page 22: Spatial Routines for a Simulated Speech-Controlled Vehiclecs.brown.edu/people/stellex/publications/Tellex-Roy-HRI06-presentation.pdf · Generous Stingy Go left 90% 80% 0.12 Go right](https://reader034.vdocuments.us/reader034/viewer/2022042106/5e858b53ebb4a14e79496b43/html5/thumbnails/22.jpg)
Evaluation
◮ 5 subjects
◮ 20 examples of robot’s planned path and actual path.◮ Focus of work is planned path.
◮ Chosen from uniform distribution, two radii from walls.◮ Orientation chosen from uniform distribution.
◮ Each example marked as “correct”, “incorrect”, “n/a”.
◮ Report:◮ Generous - Samples with at least one correct.◮ Stingy - Samples where most subjects marked it correct.◮ Light’s Kappa
![Page 23: Spatial Routines for a Simulated Speech-Controlled Vehiclecs.brown.edu/people/stellex/publications/Tellex-Roy-HRI06-presentation.pdf · Generous Stingy Go left 90% 80% 0.12 Go right](https://reader034.vdocuments.us/reader034/viewer/2022042106/5e858b53ebb4a14e79496b43/html5/thumbnails/23.jpg)
Evaluation Samples - “Go left.”
![Page 24: Spatial Routines for a Simulated Speech-Controlled Vehiclecs.brown.edu/people/stellex/publications/Tellex-Roy-HRI06-presentation.pdf · Generous Stingy Go left 90% 80% 0.12 Go right](https://reader034.vdocuments.us/reader034/viewer/2022042106/5e858b53ebb4a14e79496b43/html5/thumbnails/24.jpg)
Evaluation Samples - “Go right.”
![Page 25: Spatial Routines for a Simulated Speech-Controlled Vehiclecs.brown.edu/people/stellex/publications/Tellex-Roy-HRI06-presentation.pdf · Generous Stingy Go left 90% 80% 0.12 Go right](https://reader034.vdocuments.us/reader034/viewer/2022042106/5e858b53ebb4a14e79496b43/html5/thumbnails/25.jpg)
Evaluation Samples - “Go across the room.”
![Page 26: Spatial Routines for a Simulated Speech-Controlled Vehiclecs.brown.edu/people/stellex/publications/Tellex-Roy-HRI06-presentation.pdf · Generous Stingy Go left 90% 80% 0.12 Go right](https://reader034.vdocuments.us/reader034/viewer/2022042106/5e858b53ebb4a14e79496b43/html5/thumbnails/26.jpg)
![Page 27: Spatial Routines for a Simulated Speech-Controlled Vehiclecs.brown.edu/people/stellex/publications/Tellex-Roy-HRI06-presentation.pdf · Generous Stingy Go left 90% 80% 0.12 Go right](https://reader034.vdocuments.us/reader034/viewer/2022042106/5e858b53ebb4a14e79496b43/html5/thumbnails/27.jpg)
Planned Path
Command % Correct Light’s KappaGenerous Stingy
Go left 90% 80% 0.12Go right 95% 70% 0.12Go across 50% 35% 0.53the room
Actual Path
Command % Correct Light’s KappaGenerous Stingy
Go left 90% 50% 0.15Go right 90% 50% 0.11Go across 45% 0% 0.09the room
![Page 28: Spatial Routines for a Simulated Speech-Controlled Vehiclecs.brown.edu/people/stellex/publications/Tellex-Roy-HRI06-presentation.pdf · Generous Stingy Go left 90% 80% 0.12 Go right](https://reader034.vdocuments.us/reader034/viewer/2022042106/5e858b53ebb4a14e79496b43/html5/thumbnails/28.jpg)
Command Average Correct Standard Deviation
Go left. 61% 30Go right. 56% 27.25Go across 36% 7.42the room
![Page 29: Spatial Routines for a Simulated Speech-Controlled Vehiclecs.brown.edu/people/stellex/publications/Tellex-Roy-HRI06-presentation.pdf · Generous Stingy Go left 90% 80% 0.12 Go right](https://reader034.vdocuments.us/reader034/viewer/2022042106/5e858b53ebb4a14e79496b43/html5/thumbnails/29.jpg)
“Go right.”
![Page 30: Spatial Routines for a Simulated Speech-Controlled Vehiclecs.brown.edu/people/stellex/publications/Tellex-Roy-HRI06-presentation.pdf · Generous Stingy Go left 90% 80% 0.12 Go right](https://reader034.vdocuments.us/reader034/viewer/2022042106/5e858b53ebb4a14e79496b43/html5/thumbnails/30.jpg)
“Go across the room.”
![Page 31: Spatial Routines for a Simulated Speech-Controlled Vehiclecs.brown.edu/people/stellex/publications/Tellex-Roy-HRI06-presentation.pdf · Generous Stingy Go left 90% 80% 0.12 Go right](https://reader034.vdocuments.us/reader034/viewer/2022042106/5e858b53ebb4a14e79496b43/html5/thumbnails/31.jpg)
“Go left.”
◮ The two worst samples for “Go left.”
◮ (Each had 4 incorrect, 1 n/a)
![Page 32: Spatial Routines for a Simulated Speech-Controlled Vehiclecs.brown.edu/people/stellex/publications/Tellex-Roy-HRI06-presentation.pdf · Generous Stingy Go left 90% 80% 0.12 Go right](https://reader034.vdocuments.us/reader034/viewer/2022042106/5e858b53ebb4a14e79496b43/html5/thumbnails/32.jpg)
“Go left.”
◮ The two second worst samples for “Go left.”
◮ (Only one subject marked each of these correct.)
![Page 33: Spatial Routines for a Simulated Speech-Controlled Vehiclecs.brown.edu/people/stellex/publications/Tellex-Roy-HRI06-presentation.pdf · Generous Stingy Go left 90% 80% 0.12 Go right](https://reader034.vdocuments.us/reader034/viewer/2022042106/5e858b53ebb4a14e79496b43/html5/thumbnails/33.jpg)
Lessons Learned
◮ Worked well for some people some of the time.
◮ Poor interannotater agreement.
◮ Not very realistic.◮ Empty environment.◮ Top down situation view.
◮ Better methods:◮ Movies of paths. (1st or 3rd person.)◮ Richer environment.◮ Paths generated by people (By drawing or driving.)
![Page 34: Spatial Routines for a Simulated Speech-Controlled Vehiclecs.brown.edu/people/stellex/publications/Tellex-Roy-HRI06-presentation.pdf · Generous Stingy Go left 90% 80% 0.12 Go right](https://reader034.vdocuments.us/reader034/viewer/2022042106/5e858b53ebb4a14e79496b43/html5/thumbnails/34.jpg)
Future Work
◮ Applying spatial routines to real time strategy games.
◮ Learning routines using grammar induction/Earley parser.
◮ Using routines to label object trajectories.
![Page 35: Spatial Routines for a Simulated Speech-Controlled Vehiclecs.brown.edu/people/stellex/publications/Tellex-Roy-HRI06-presentation.pdf · Generous Stingy Go left 90% 80% 0.12 Go right](https://reader034.vdocuments.us/reader034/viewer/2022042106/5e858b53ebb4a14e79496b43/html5/thumbnails/35.jpg)
Real Time Strategy Games
![Page 36: Spatial Routines for a Simulated Speech-Controlled Vehiclecs.brown.edu/people/stellex/publications/Tellex-Roy-HRI06-presentation.pdf · Generous Stingy Go left 90% 80% 0.12 Go right](https://reader034.vdocuments.us/reader034/viewer/2022042106/5e858b53ebb4a14e79496b43/html5/thumbnails/36.jpg)
Future Work
◮ Applying spatial routines to real time strategy games.
◮ Learning routines using grammar induction/Earley parser.
◮ Using routines to label object trajectories.
![Page 37: Spatial Routines for a Simulated Speech-Controlled Vehiclecs.brown.edu/people/stellex/publications/Tellex-Roy-HRI06-presentation.pdf · Generous Stingy Go left 90% 80% 0.12 Go right](https://reader034.vdocuments.us/reader034/viewer/2022042106/5e858b53ebb4a14e79496b43/html5/thumbnails/37.jpg)