touch-enabled interfaces ming c. lin department of computer science university of north carolina at...
TRANSCRIPT
![Page 1: Touch-Enabled Interfaces Ming C. Lin Department of Computer Science University of North Carolina at Chapel Hill lin lin@cs.unc.edu](https://reader035.vdocuments.us/reader035/viewer/2022070408/56649e6c5503460f94b6b28e/html5/thumbnails/1.jpg)
Touch-Enabled Interfaces
Ming C. Lin
Department of Computer Science
University of North Carolina at Chapel Hill
http://www.cs.unc.edu/~lin
![Page 2: Touch-Enabled Interfaces Ming C. Lin Department of Computer Science University of North Carolina at Chapel Hill lin lin@cs.unc.edu](https://reader035.vdocuments.us/reader035/viewer/2022070408/56649e6c5503460f94b6b28e/html5/thumbnails/2.jpg)
Human Computer InteractionHuman Computer Interaction
![Page 3: Touch-Enabled Interfaces Ming C. Lin Department of Computer Science University of North Carolina at Chapel Hill lin lin@cs.unc.edu](https://reader035.vdocuments.us/reader035/viewer/2022070408/56649e6c5503460f94b6b28e/html5/thumbnails/3.jpg)
Human Computer InteractionHuman Computer Interaction
• Visual (graphics, vision, etc)
• Auditory (sound)
• Haptic (touch-enabled)
• Others
![Page 4: Touch-Enabled Interfaces Ming C. Lin Department of Computer Science University of North Carolina at Chapel Hill lin lin@cs.unc.edu](https://reader035.vdocuments.us/reader035/viewer/2022070408/56649e6c5503460f94b6b28e/html5/thumbnails/4.jpg)
Common Touch-based InterfacesCommon Touch-based Interfaces
![Page 5: Touch-Enabled Interfaces Ming C. Lin Department of Computer Science University of North Carolina at Chapel Hill lin lin@cs.unc.edu](https://reader035.vdocuments.us/reader035/viewer/2022070408/56649e6c5503460f94b6b28e/html5/thumbnails/5.jpg)
Other Touch-based InterfacesOther Touch-based Interfaces
![Page 6: Touch-Enabled Interfaces Ming C. Lin Department of Computer Science University of North Carolina at Chapel Hill lin lin@cs.unc.edu](https://reader035.vdocuments.us/reader035/viewer/2022070408/56649e6c5503460f94b6b28e/html5/thumbnails/6.jpg)
BenefitsBenefits
• Augment other senses
• Inherent 3D interfaces
• Physically-based interaction
• Assisted Technology
• Natural & Intuitive
![Page 7: Touch-Enabled Interfaces Ming C. Lin Department of Computer Science University of North Carolina at Chapel Hill lin lin@cs.unc.edu](https://reader035.vdocuments.us/reader035/viewer/2022070408/56649e6c5503460f94b6b28e/html5/thumbnails/7.jpg)
What Is Haptic Rendering?What Is Haptic Rendering?
Simulation
HapticDevice
Robot
Virtual Reality
Master-Slave Systems
Human
Force Feedback
Tactile Feedback
Human-in-the-Loop
![Page 8: Touch-Enabled Interfaces Ming C. Lin Department of Computer Science University of North Carolina at Chapel Hill lin lin@cs.unc.edu](https://reader035.vdocuments.us/reader035/viewer/2022070408/56649e6c5503460f94b6b28e/html5/thumbnails/8.jpg)
Inter-disciplinary ResearchInter-disciplinary Research
ComputerScience
ElectricalEngineering
MechanicalEngineering
Haptic Rendering Control and actuators
Mechanical design
Computation of the forces output by the device
![Page 9: Touch-Enabled Interfaces Ming C. Lin Department of Computer Science University of North Carolina at Chapel Hill lin lin@cs.unc.edu](https://reader035.vdocuments.us/reader035/viewer/2022070408/56649e6c5503460f94b6b28e/html5/thumbnails/9.jpg)
Control of Haptic DevicesControl of Haptic Devices
• Impedance Devices • Admittance devices
6-DOF Phantom COBOTs
![Page 10: Touch-Enabled Interfaces Ming C. Lin Department of Computer Science University of North Carolina at Chapel Hill lin lin@cs.unc.edu](https://reader035.vdocuments.us/reader035/viewer/2022070408/56649e6c5503460f94b6b28e/html5/thumbnails/10.jpg)
Engine Close-UpEngine Close-Up
Boeing VPS System
![Page 11: Touch-Enabled Interfaces Ming C. Lin Department of Computer Science University of North Carolina at Chapel Hill lin lin@cs.unc.edu](https://reader035.vdocuments.us/reader035/viewer/2022070408/56649e6c5503460f94b6b28e/html5/thumbnails/11.jpg)
Collaborative Haptic Design ReviewCollaborative Haptic Design Review
![Page 12: Touch-Enabled Interfaces Ming C. Lin Department of Computer Science University of North Carolina at Chapel Hill lin lin@cs.unc.edu](https://reader035.vdocuments.us/reader035/viewer/2022070408/56649e6c5503460f94b6b28e/html5/thumbnails/12.jpg)
Other ExamplesOther Examples
• A Haptic Hybrid Controller for Virtual Prototyping of Vehicle Mechanisms (Ford, BMW, etc)
• 3-DOF Cobot for Engineering Design
(Northwestern University and Ford Automobile)
![Page 13: Touch-Enabled Interfaces Ming C. Lin Department of Computer Science University of North Carolina at Chapel Hill lin lin@cs.unc.edu](https://reader035.vdocuments.us/reader035/viewer/2022070408/56649e6c5503460f94b6b28e/html5/thumbnails/13.jpg)
Medical SimulatorsMedical Simulators
• Endoscopy simulator - Bronchoscopy and upper and lower gastrointestinal procedures on a single platform
• Endovascular simulator - Percutaneous coronary and peripheral interventions and cardiac rhythm management
• Hysteroscopy simulator - Skills assessment and myomectomy
• Laparoscopy simulator - Skills, cholecystectomy, sterilization, ectopic pregnancy, and myomectomy suturing
• Vascular access simulator - Adult, geriatric, and pediatric IV; PICC; phlebotomy; and skills assessment
![Page 14: Touch-Enabled Interfaces Ming C. Lin Department of Computer Science University of North Carolina at Chapel Hill lin lin@cs.unc.edu](https://reader035.vdocuments.us/reader035/viewer/2022070408/56649e6c5503460f94b6b28e/html5/thumbnails/14.jpg)
Virtual Endoscopic Surgery TrainingVirtual Endoscopic Surgery Training
VEST System One (VSOne) Technology
• 3 haptic (force-feedback) devices as mock-up endoscopic instruments
• 1 virtual endoscopic camera
• three new Basic Task Training (BTT) exercises - Find tubes/touch points/follow path
![Page 15: Touch-Enabled Interfaces Ming C. Lin Department of Computer Science University of North Carolina at Chapel Hill lin lin@cs.unc.edu](https://reader035.vdocuments.us/reader035/viewer/2022070408/56649e6c5503460f94b6b28e/html5/thumbnails/15.jpg)
Laparoscopic SurgeryLaparoscopic Surgery
• MIT Touch Lab
![Page 16: Touch-Enabled Interfaces Ming C. Lin Department of Computer Science University of North Carolina at Chapel Hill lin lin@cs.unc.edu](https://reader035.vdocuments.us/reader035/viewer/2022070408/56649e6c5503460f94b6b28e/html5/thumbnails/16.jpg)
Molecular DynamicsMolecular Dynamics
• VMD: Visual Molecular Dynamics
Humphrey, 1996
![Page 17: Touch-Enabled Interfaces Ming C. Lin Department of Computer Science University of North Carolina at Chapel Hill lin lin@cs.unc.edu](https://reader035.vdocuments.us/reader035/viewer/2022070408/56649e6c5503460f94b6b28e/html5/thumbnails/17.jpg)
Haptic Vector FieldHaptic Vector Field
• Lawrence, Lee, Pau, Roman, Novoselov
– University of Colorado at Boulder
• 5 D.O.F. in
• 5 D.O.F. out
Lawrence, 2000
![Page 18: Touch-Enabled Interfaces Ming C. Lin Department of Computer Science University of North Carolina at Chapel Hill lin lin@cs.unc.edu](https://reader035.vdocuments.us/reader035/viewer/2022070408/56649e6c5503460f94b6b28e/html5/thumbnails/18.jpg)
dAb: Haptic Painting SystemdAb: Haptic Painting System
![Page 19: Touch-Enabled Interfaces Ming C. Lin Department of Computer Science University of North Carolina at Chapel Hill lin lin@cs.unc.edu](https://reader035.vdocuments.us/reader035/viewer/2022070408/56649e6c5503460f94b6b28e/html5/thumbnails/19.jpg)
inTouch: 3D Haptic PaintinginTouch: 3D Haptic Painting
Painted Butterfly (~80k triangles)
http://gamma.cs.unc.edu/inTouch
![Page 20: Touch-Enabled Interfaces Ming C. Lin Department of Computer Science University of North Carolina at Chapel Hill lin lin@cs.unc.edu](https://reader035.vdocuments.us/reader035/viewer/2022070408/56649e6c5503460f94b6b28e/html5/thumbnails/20.jpg)
inTouch: Multiresolution Modeling with Haptic InterfaceinTouch: Multiresolution Modeling with Haptic Interface
http://gamma.cs.unc.edu/inTouch
![Page 21: Touch-Enabled Interfaces Ming C. Lin Department of Computer Science University of North Carolina at Chapel Hill lin lin@cs.unc.edu](https://reader035.vdocuments.us/reader035/viewer/2022070408/56649e6c5503460f94b6b28e/html5/thumbnails/21.jpg)
ArtNova: Touch-Enabled 3D Model DesignArtNova: Touch-Enabled 3D Model Design
• Interactive texture painting• User-centric viewing• Realistic force response
http://gamma.cs.unc.edu/ArtNova
![Page 22: Touch-Enabled Interfaces Ming C. Lin Department of Computer Science University of North Carolina at Chapel Hill lin lin@cs.unc.edu](https://reader035.vdocuments.us/reader035/viewer/2022070408/56649e6c5503460f94b6b28e/html5/thumbnails/22.jpg)
FreeForm DesignFreeForm Design
Model Galleryhttp://www.sensable.com/freeform-models.htm
![Page 23: Touch-Enabled Interfaces Ming C. Lin Department of Computer Science University of North Carolina at Chapel Hill lin lin@cs.unc.edu](https://reader035.vdocuments.us/reader035/viewer/2022070408/56649e6c5503460f94b6b28e/html5/thumbnails/23.jpg)
Manipulating GearsManipulating Gears
![Page 24: Touch-Enabled Interfaces Ming C. Lin Department of Computer Science University of North Carolina at Chapel Hill lin lin@cs.unc.edu](https://reader035.vdocuments.us/reader035/viewer/2022070408/56649e6c5503460f94b6b28e/html5/thumbnails/24.jpg)
Basic PipelineBasic Pipeline
Simulated Environment
Grasped Object
Scene
Haptic Device
User
motionmotion
![Page 25: Touch-Enabled Interfaces Ming C. Lin Department of Computer Science University of North Carolina at Chapel Hill lin lin@cs.unc.edu](https://reader035.vdocuments.us/reader035/viewer/2022070408/56649e6c5503460f94b6b28e/html5/thumbnails/25.jpg)
Basic PipelineBasic Pipeline
Simulated Environment
Scene
Haptic Device
User
motionmotion
position
Grasped Object
![Page 26: Touch-Enabled Interfaces Ming C. Lin Department of Computer Science University of North Carolina at Chapel Hill lin lin@cs.unc.edu](https://reader035.vdocuments.us/reader035/viewer/2022070408/56649e6c5503460f94b6b28e/html5/thumbnails/26.jpg)
Basic PipelineBasic Pipeline
Simulated Environment
Scene
Haptic Device
User
motionmotion
position
Grasped Object
![Page 27: Touch-Enabled Interfaces Ming C. Lin Department of Computer Science University of North Carolina at Chapel Hill lin lin@cs.unc.edu](https://reader035.vdocuments.us/reader035/viewer/2022070408/56649e6c5503460f94b6b28e/html5/thumbnails/27.jpg)
Basic PipelineBasic Pipeline
Simulated Environment
Scene
Haptic Device
User
motionmotion
position
Grasped Object
![Page 28: Touch-Enabled Interfaces Ming C. Lin Department of Computer Science University of North Carolina at Chapel Hill lin lin@cs.unc.edu](https://reader035.vdocuments.us/reader035/viewer/2022070408/56649e6c5503460f94b6b28e/html5/thumbnails/28.jpg)
Basic PipelineBasic Pipeline
Simulated Environment
Scene
Haptic Device
User
motionmotion
position
command
Grasped Object
![Page 29: Touch-Enabled Interfaces Ming C. Lin Department of Computer Science University of North Carolina at Chapel Hill lin lin@cs.unc.edu](https://reader035.vdocuments.us/reader035/viewer/2022070408/56649e6c5503460f94b6b28e/html5/thumbnails/29.jpg)
Basic PipelineBasic Pipeline
Simulated Environment
Scene
Haptic Device
User
motionmotion
position
command
forces
Grasped Object
![Page 30: Touch-Enabled Interfaces Ming C. Lin Department of Computer Science University of North Carolina at Chapel Hill lin lin@cs.unc.edu](https://reader035.vdocuments.us/reader035/viewer/2022070408/56649e6c5503460f94b6b28e/html5/thumbnails/30.jpg)
Basic PipelineBasic Pipeline
Simulated Environment
Scene
Haptic Device
User
motionmotion
position
command
forces
Grasped Object
DIRECT RENDERING
![Page 31: Touch-Enabled Interfaces Ming C. Lin Department of Computer Science University of North Carolina at Chapel Hill lin lin@cs.unc.edu](https://reader035.vdocuments.us/reader035/viewer/2022070408/56649e6c5503460f94b6b28e/html5/thumbnails/31.jpg)
Haptic Rendering LoopHaptic Rendering Loop
Fuser
forcecommand
Fdevice
position
![Page 32: Touch-Enabled Interfaces Ming C. Lin Department of Computer Science University of North Carolina at Chapel Hill lin lin@cs.unc.edu](https://reader035.vdocuments.us/reader035/viewer/2022070408/56649e6c5503460f94b6b28e/html5/thumbnails/32.jpg)
Problem of Haptic RenderingProblem of Haptic Rendering
1. The user becomes part of the simulation loop.
2. 1KHz is necessary so that the whole system doesn’t suffer from disturbing oscillations.
• Think of the analogy with numerical integration of a system with spring, mass and damper, where the frequency of the haptic loop sets the integration step.
3. The Phantom haptic devices run their control loop at 1KHz.
4. Consequence: we are very limited on the amount of computation that we can do.
![Page 33: Touch-Enabled Interfaces Ming C. Lin Department of Computer Science University of North Carolina at Chapel Hill lin lin@cs.unc.edu](https://reader035.vdocuments.us/reader035/viewer/2022070408/56649e6c5503460f94b6b28e/html5/thumbnails/33.jpg)
Haptic Rendering LoopHaptic Rendering Loop
Human-in-the-loop
• High sensitivity to instabilities!!
• High update rates required!! (kHz for high stiffness)
![Page 34: Touch-Enabled Interfaces Ming C. Lin Department of Computer Science University of North Carolina at Chapel Hill lin lin@cs.unc.edu](https://reader035.vdocuments.us/reader035/viewer/2022070408/56649e6c5503460f94b6b28e/html5/thumbnails/34.jpg)
Key ChallengesKey Challenges
• Collision Detection
– Choice of representation and algorithm
• Interaction Paradigm
– Penalty forces vs. constraint-based optimization
– Virtual coupling vs. direct rendering
– Newtonian dynamics / Quasi-static approximation
– Single user vs. collaboration
![Page 35: Touch-Enabled Interfaces Ming C. Lin Department of Computer Science University of North Carolina at Chapel Hill lin lin@cs.unc.edu](https://reader035.vdocuments.us/reader035/viewer/2022070408/56649e6c5503460f94b6b28e/html5/thumbnails/35.jpg)
Additional IssuesAdditional Issues
• Decouple haptic and simulation loops?
– Use intermediate representations?
• Force type and quality
– How hard does hard contact feel?
– How free does free-space feel?
• Repulsive forces?
• Force artifacts / stability considerations