doru talaba, professor robotics and virtual reality ......doru talaba, transilvania university of...
TRANSCRIPT
![Page 1: Doru Talaba, Professor Robotics and Virtual Reality ......Doru TALABA, Transilvania University of Brasov nsil 1 Doru Talaba, Professor Robotics and Virtual Reality Research Lab (RVR)](https://reader030.vdocuments.us/reader030/viewer/2022040201/5e59cac601e14951923a6399/html5/thumbnails/1.jpg)
Doru TALABA, Transilvania University of Brasov 1 nsilnsil
Doru Talaba, Professor
Robotics and Virtual Reality Research Lab (RVR)
Transilvania University of Brasov, Romania
![Page 2: Doru Talaba, Professor Robotics and Virtual Reality ......Doru TALABA, Transilvania University of Brasov nsil 1 Doru Talaba, Professor Robotics and Virtual Reality Research Lab (RVR)](https://reader030.vdocuments.us/reader030/viewer/2022040201/5e59cac601e14951923a6399/html5/thumbnails/2.jpg)
Virtual Reality
! One of the core technologies of the modern Human Machine Interface (HMI) science
! Human Machine Systems – one of the 14 technical committees of IFToMM
" A multi disciplinary science with roots in Computer Graphics and Robotics
Doru TALABA, Transilvania University of Brasov 2
![Page 3: Doru Talaba, Professor Robotics and Virtual Reality ......Doru TALABA, Transilvania University of Brasov nsil 1 Doru Talaba, Professor Robotics and Virtual Reality Research Lab (RVR)](https://reader030.vdocuments.us/reader030/viewer/2022040201/5e59cac601e14951923a6399/html5/thumbnails/3.jpg)
Virtual Reality (II)
! The science of Human Machine communication via natural sensorial channels
! Usually the machine outputs signals that are perceived through the human senses
! Multi – modal interface " several senses
" Virtual Reality is a technology for OUTPUT ?
Doru TALABA, Transilvania University of Brasov 3
![Page 4: Doru Talaba, Professor Robotics and Virtual Reality ......Doru TALABA, Transilvania University of Brasov nsil 1 Doru Talaba, Professor Robotics and Virtual Reality Research Lab (RVR)](https://reader030.vdocuments.us/reader030/viewer/2022040201/5e59cac601e14951923a6399/html5/thumbnails/4.jpg)
VR #" 3 “I”
! Immersion
! Imagination
! Interaction
! Burdea and Coiffet (2004):
but
! INTERACTION = INPUT and OUTPUT
! Conclusion: Virtual Reality = INPUT and OUTPUT
! We call them “modalities”. Doru TALABA, Transilvania University of Brasov 4
![Page 5: Doru Talaba, Professor Robotics and Virtual Reality ......Doru TALABA, Transilvania University of Brasov nsil 1 Doru Talaba, Professor Robotics and Virtual Reality Research Lab (RVR)](https://reader030.vdocuments.us/reader030/viewer/2022040201/5e59cac601e14951923a6399/html5/thumbnails/5.jpg)
VR modalities ! OUTPUT: Most known and developed VR technologies
! One sense = one “modality”
! Really addressing the 5 human senses.
Most common “modalities”:
– - Visual sense
– - Touch (Haptic feedback)
– - Audio
– - smell
– - taste
Doru TALABA, Transilvania University of Brasov 5
![Page 6: Doru Talaba, Professor Robotics and Virtual Reality ......Doru TALABA, Transilvania University of Brasov nsil 1 Doru Talaba, Professor Robotics and Virtual Reality Research Lab (RVR)](https://reader030.vdocuments.us/reader030/viewer/2022040201/5e59cac601e14951923a6399/html5/thumbnails/6.jpg)
VR INPUT Modalities
! The human communicates to the machine
! The “modalities” do not correspond to human senses
…but to “machine sensing”
! Still they must be NATURAL and easy for the human
"Input modality = NATURAL human communication mean
! Examples of input modalities:
– - Speech
– - Gestures
– " we name them: “explicit input”
Doru TALABA, Transilvania University of Brasov 6
![Page 7: Doru Talaba, Professor Robotics and Virtual Reality ......Doru TALABA, Transilvania University of Brasov nsil 1 Doru Talaba, Professor Robotics and Virtual Reality Research Lab (RVR)](https://reader030.vdocuments.us/reader030/viewer/2022040201/5e59cac601e14951923a6399/html5/thumbnails/7.jpg)
implicit INPUT
! Tracking based input: most attractive input modalities
! The user is tracked: head, hands, whole body, eyes, electrical signals (including neuro) etc
" A much larger range of “sources”
! Minim cognitive load for the user
! Simple to use " more complex to develop
! Examples of implicit input modalities:
– - Navigation
– - Pointing
Doru TALABA, Transilvania University of Brasov 7
![Page 8: Doru Talaba, Professor Robotics and Virtual Reality ......Doru TALABA, Transilvania University of Brasov nsil 1 Doru Talaba, Professor Robotics and Virtual Reality Research Lab (RVR)](https://reader030.vdocuments.us/reader030/viewer/2022040201/5e59cac601e14951923a6399/html5/thumbnails/8.jpg)
Multi-Modal Interfaces
! The big concept in 2000’s
– - Input modalities
– - Output modalities
! Many challenges and research topics:
– - fusion of modalities (for input)
– - fission of modalities (for output)
– - conversion from one modality to another "
– - compensation of one another
Doru TALABA, Transilvania University of Brasov 8
![Page 9: Doru Talaba, Professor Robotics and Virtual Reality ......Doru TALABA, Transilvania University of Brasov nsil 1 Doru Talaba, Professor Robotics and Virtual Reality Research Lab (RVR)](https://reader030.vdocuments.us/reader030/viewer/2022040201/5e59cac601e14951923a6399/html5/thumbnails/9.jpg)
9
Example: Immersive interfaces
! For output: – 3D visualisation
– Haptic devices
– 3D audio
! For input: – Tracking
– Gesture
– Speech
Doru TALABA, Transilvania University of Brasov
![Page 10: Doru Talaba, Professor Robotics and Virtual Reality ......Doru TALABA, Transilvania University of Brasov nsil 1 Doru Talaba, Professor Robotics and Virtual Reality Research Lab (RVR)](https://reader030.vdocuments.us/reader030/viewer/2022040201/5e59cac601e14951923a6399/html5/thumbnails/10.jpg)
10
Typical core interface
Doru TALABA, Transilvania University of Brasov
![Page 11: Doru Talaba, Professor Robotics and Virtual Reality ......Doru TALABA, Transilvania University of Brasov nsil 1 Doru Talaba, Professor Robotics and Virtual Reality Research Lab (RVR)](https://reader030.vdocuments.us/reader030/viewer/2022040201/5e59cac601e14951923a6399/html5/thumbnails/11.jpg)
11
Typical VR interface for
existing applications
!
Doru TALABA, Transilvania University of Brasov
![Page 12: Doru Talaba, Professor Robotics and Virtual Reality ......Doru TALABA, Transilvania University of Brasov nsil 1 Doru Talaba, Professor Robotics and Virtual Reality Research Lab (RVR)](https://reader030.vdocuments.us/reader030/viewer/2022040201/5e59cac601e14951923a6399/html5/thumbnails/12.jpg)
VR applications in Medical Robotics
Doru TALABA, Transilvania University of Brasov 12
" Multi-modal interfaces :
- simulation, planning and training
- remote medicine, tele-presence
" Individual modalities used to
cmpensate the lost or degradation of
one sense or human ability
- Rehabilitation
- Ambient Assistive Living
![Page 13: Doru Talaba, Professor Robotics and Virtual Reality ......Doru TALABA, Transilvania University of Brasov nsil 1 Doru Talaba, Professor Robotics and Virtual Reality Research Lab (RVR)](https://reader030.vdocuments.us/reader030/viewer/2022040201/5e59cac601e14951923a6399/html5/thumbnails/13.jpg)
VR modalities that are important for
Medical Robotics
Doru TALABA, Transilvania University of Brasov 13
- Haptic: mainly rehabilitation, but also
surgical training)
- 3D Visual: pre-operatory planning,
augmented reality
- Tracking: Ambient assisting living,
remote medicine
" a number of projects completed,
ongoing or just started at UTBv
![Page 14: Doru Talaba, Professor Robotics and Virtual Reality ......Doru TALABA, Transilvania University of Brasov nsil 1 Doru Talaba, Professor Robotics and Virtual Reality Research Lab (RVR)](https://reader030.vdocuments.us/reader030/viewer/2022040201/5e59cac601e14951923a6399/html5/thumbnails/14.jpg)
“Haptic modality”
! Admitance control – genuine haptic input: the user force is measured and sensed by the machine
– - Not very often used
! Traditionaly " haptic feedback = output (impedance
control)
! This could be also equivalent to haptic input because of
Newton’s “action-reaction” principle
! The force feedback is “resisted” by the human operator or viceversa
! Many haptic applications in medical robotics: rehabilitation, robotic surgery and training. Most important: rehabilitation
Doru TALABA, Transilvania University of Brasov 14
![Page 15: Doru Talaba, Professor Robotics and Virtual Reality ......Doru TALABA, Transilvania University of Brasov nsil 1 Doru Talaba, Professor Robotics and Virtual Reality Research Lab (RVR)](https://reader030.vdocuments.us/reader030/viewer/2022040201/5e59cac601e14951923a6399/html5/thumbnails/15.jpg)
• Able people can always be resisted (to simulate grasping of an object, for example)
• Disabled workers need sometimes to be assisted, and as
they progress to be resisted. – A transition between assistance
and resistance is needed for rehabilitation, which is case-
dependent;
• Could/should be done by intelligent agent software
programmed/prescribed by the medical specialist.
• Another technique is by haptic disturbances - make control
more difficult – thus improving motor control capabilities
• Sometimes disturbances are designed to create useful after
effects .
Why is haptic control important
in rehabilitation?
![Page 16: Doru Talaba, Professor Robotics and Virtual Reality ......Doru TALABA, Transilvania University of Brasov nsil 1 Doru Talaba, Professor Robotics and Virtual Reality Research Lab (RVR)](https://reader030.vdocuments.us/reader030/viewer/2022040201/5e59cac601e14951923a6399/html5/thumbnails/16.jpg)
Typical methodology - When the patient is unable to move, the robot
moves the patient s arm to the target
- If the patient moves inappropriately, the robot
guides the arm towards a nominal trajectory to
the target – like a haptic channel
- As the patient improves, the robot provides less
assistance
![Page 17: Doru Talaba, Professor Robotics and Virtual Reality ......Doru TALABA, Transilvania University of Brasov nsil 1 Doru Talaba, Professor Robotics and Virtual Reality Research Lab (RVR)](https://reader030.vdocuments.us/reader030/viewer/2022040201/5e59cac601e14951923a6399/html5/thumbnails/17.jpg)
haptic gloves for hand rehabilitation
Rutger Master, 2002 Univ. Transilvania, 2006
Project VEGA
Most important features: max force, lightweight, workspace
![Page 18: Doru Talaba, Professor Robotics and Virtual Reality ......Doru TALABA, Transilvania University of Brasov nsil 1 Doru Talaba, Professor Robotics and Virtual Reality Research Lab (RVR)](https://reader030.vdocuments.us/reader030/viewer/2022040201/5e59cac601e14951923a6399/html5/thumbnails/18.jpg)
Performances
- The VEGA glove uses a sensing/feedback exoskeleton, producing 9 N per finger (16 N - Rutger Master).
- Similar weight
- Workspace limitations (for Rutgers)
!
![Page 19: Doru Talaba, Professor Robotics and Virtual Reality ......Doru TALABA, Transilvania University of Brasov nsil 1 Doru Talaba, Professor Robotics and Virtual Reality Research Lab (RVR)](https://reader030.vdocuments.us/reader030/viewer/2022040201/5e59cac601e14951923a6399/html5/thumbnails/19.jpg)
Multi-modal, collaborative VR:
“Project MERVI: Collaborative Pre-
surgery Planning in a Tele-immersive
Environment using VR Technology”
![Page 20: Doru Talaba, Professor Robotics and Virtual Reality ......Doru TALABA, Transilvania University of Brasov nsil 1 Doru Talaba, Professor Robotics and Virtual Reality Research Lab (RVR)](https://reader030.vdocuments.us/reader030/viewer/2022040201/5e59cac601e14951923a6399/html5/thumbnails/20.jpg)
The goal
- Collaboration between remote medical
teams
- Remote assisted surgery
- Reduced time of bone surgery using a
pre-operatory planning
![Page 21: Doru Talaba, Professor Robotics and Virtual Reality ......Doru TALABA, Transilvania University of Brasov nsil 1 Doru Talaba, Professor Robotics and Virtual Reality Research Lab (RVR)](https://reader030.vdocuments.us/reader030/viewer/2022040201/5e59cac601e14951923a6399/html5/thumbnails/21.jpg)
Surgery Room Simulation
![Page 22: Doru Talaba, Professor Robotics and Virtual Reality ......Doru TALABA, Transilvania University of Brasov nsil 1 Doru Talaba, Professor Robotics and Virtual Reality Research Lab (RVR)](https://reader030.vdocuments.us/reader030/viewer/2022040201/5e59cac601e14951923a6399/html5/thumbnails/22.jpg)
3D bone fracture model
! Using SIEMENS SOMATOM CT
![Page 23: Doru Talaba, Professor Robotics and Virtual Reality ......Doru TALABA, Transilvania University of Brasov nsil 1 Doru Talaba, Professor Robotics and Virtual Reality Research Lab (RVR)](https://reader030.vdocuments.us/reader030/viewer/2022040201/5e59cac601e14951923a6399/html5/thumbnails/23.jpg)
Bone fracture evaluation
![Page 24: Doru Talaba, Professor Robotics and Virtual Reality ......Doru TALABA, Transilvania University of Brasov nsil 1 Doru Talaba, Professor Robotics and Virtual Reality Research Lab (RVR)](https://reader030.vdocuments.us/reader030/viewer/2022040201/5e59cac601e14951923a6399/html5/thumbnails/24.jpg)
Engineering of implants
![Page 25: Doru Talaba, Professor Robotics and Virtual Reality ......Doru TALABA, Transilvania University of Brasov nsil 1 Doru Talaba, Professor Robotics and Virtual Reality Research Lab (RVR)](https://reader030.vdocuments.us/reader030/viewer/2022040201/5e59cac601e14951923a6399/html5/thumbnails/25.jpg)
Virtual testing of implants
![Page 26: Doru Talaba, Professor Robotics and Virtual Reality ......Doru TALABA, Transilvania University of Brasov nsil 1 Doru Talaba, Professor Robotics and Virtual Reality Research Lab (RVR)](https://reader030.vdocuments.us/reader030/viewer/2022040201/5e59cac601e14951923a6399/html5/thumbnails/26.jpg)
Doru TALABA, Transilvania University of Brasov 26
Most input modalities are based on tracking
One of the most important: Navigation
! Zero cognitive load
! Essential function: moving in virtual environment
! Currently: navigation by mouse, gesture etc.
! VR challenge: natural navigation
“Tracking”
![Page 27: Doru Talaba, Professor Robotics and Virtual Reality ......Doru TALABA, Transilvania University of Brasov nsil 1 Doru Talaba, Professor Robotics and Virtual Reality Research Lab (RVR)](https://reader030.vdocuments.us/reader030/viewer/2022040201/5e59cac601e14951923a6399/html5/thumbnails/27.jpg)
Project TRIMA: Eye tracking
Doru TALABA, Transilvania University of Brasov 27
Where and What the user gazes at ?
Eye based interaction: blinks and winks
![Page 28: Doru Talaba, Professor Robotics and Virtual Reality ......Doru TALABA, Transilvania University of Brasov nsil 1 Doru Talaba, Professor Robotics and Virtual Reality Research Lab (RVR)](https://reader030.vdocuments.us/reader030/viewer/2022040201/5e59cac601e14951923a6399/html5/thumbnails/28.jpg)
Intent identification with Hidden
Markov Chains
13 possible transitions
O(t)! O(t+1)!
S1 (O1)! ID1! ID2!
S2 (O2)! ID1! ID4!
S3 (O3)! ID1! ID3!
S4 (O4)! ID4! ID5!
S5 (O5)! ID1! ID5!
Correct intent identification rate 94%
![Page 29: Doru Talaba, Professor Robotics and Virtual Reality ......Doru TALABA, Transilvania University of Brasov nsil 1 Doru Talaba, Professor Robotics and Virtual Reality Research Lab (RVR)](https://reader030.vdocuments.us/reader030/viewer/2022040201/5e59cac601e14951923a6399/html5/thumbnails/29.jpg)
Applications
Desktop
applications
Car industry Direct command of
assisting robots
![Page 30: Doru Talaba, Professor Robotics and Virtual Reality ......Doru TALABA, Transilvania University of Brasov nsil 1 Doru Talaba, Professor Robotics and Virtual Reality Research Lab (RVR)](https://reader030.vdocuments.us/reader030/viewer/2022040201/5e59cac601e14951923a6399/html5/thumbnails/30.jpg)
Interaction metaphors
! Pointing with the eyes
! Selection by blinks
! Various other gestures by combinations
of blinks and winks
Doru TALABA, Transilvania University of Brasov 30
![Page 31: Doru Talaba, Professor Robotics and Virtual Reality ......Doru TALABA, Transilvania University of Brasov nsil 1 Doru Talaba, Professor Robotics and Virtual Reality Research Lab (RVR)](https://reader030.vdocuments.us/reader030/viewer/2022040201/5e59cac601e14951923a6399/html5/thumbnails/31.jpg)
Technical Description of ASL
Eye-Track 6 H6-BN
6 DOF Head Tracking
120/240/360Hz
0.5° to 1°
50° Horizontal, 45° Vertical
or more
0.5° visual angle or less
0.1° of visual angle
Video based Eye
Tracking with Bright Pupil
illumination
Automatic and with 5 to 9
points
Head Tracker
Sampling Rate:
Gaze Position Accuracy:
Tracking Range:
System Accuracy:
Resolution:
Technology:
System Calibration:
![Page 32: Doru Talaba, Professor Robotics and Virtual Reality ......Doru TALABA, Transilvania University of Brasov nsil 1 Doru Talaba, Professor Robotics and Virtual Reality Research Lab (RVR)](https://reader030.vdocuments.us/reader030/viewer/2022040201/5e59cac601e14951923a6399/html5/thumbnails/32.jpg)
Testing the precision of the eye
tracker on a plane surface
The cloud of points are the correspondent of fixations on a certain object.
The red and cyan points are the projection on the gazed objects of each eye
gaze lines.
The lines connecting the dots show the path followed by the eyes.
![Page 33: Doru Talaba, Professor Robotics and Virtual Reality ......Doru TALABA, Transilvania University of Brasov nsil 1 Doru Talaba, Professor Robotics and Virtual Reality Research Lab (RVR)](https://reader030.vdocuments.us/reader030/viewer/2022040201/5e59cac601e14951923a6399/html5/thumbnails/33.jpg)
Testing the precision of eye
tracking in tridimensional space
! Precision of the
convergence point
measurement is less
accurate at distances
greater than 2 m.
! a) The plane is
placed at 0.5 m away
from the user
! b) The plane is
placed at 1.5 m away
from the user
a) b)
![Page 34: Doru Talaba, Professor Robotics and Virtual Reality ......Doru TALABA, Transilvania University of Brasov nsil 1 Doru Talaba, Professor Robotics and Virtual Reality Research Lab (RVR)](https://reader030.vdocuments.us/reader030/viewer/2022040201/5e59cac601e14951923a6399/html5/thumbnails/34.jpg)
Testing the precision of eye
tracking in tridimensional space
! Precision of the
convergence point
measurement is less
accurate at distances
greater than 2 m.
! a) The plane is
placed at 0.5 m away
from the user
! b) The plane is
placed at 1.5 m away
from the user
a) b)
![Page 35: Doru Talaba, Professor Robotics and Virtual Reality ......Doru TALABA, Transilvania University of Brasov nsil 1 Doru Talaba, Professor Robotics and Virtual Reality Research Lab (RVR)](https://reader030.vdocuments.us/reader030/viewer/2022040201/5e59cac601e14951923a6399/html5/thumbnails/35.jpg)
Selection of virtual objects: by gaze
![Page 36: Doru Talaba, Professor Robotics and Virtual Reality ......Doru TALABA, Transilvania University of Brasov nsil 1 Doru Talaba, Professor Robotics and Virtual Reality Research Lab (RVR)](https://reader030.vdocuments.us/reader030/viewer/2022040201/5e59cac601e14951923a6399/html5/thumbnails/36.jpg)
Selection of virtual objects by
head movements
![Page 37: Doru Talaba, Professor Robotics and Virtual Reality ......Doru TALABA, Transilvania University of Brasov nsil 1 Doru Talaba, Professor Robotics and Virtual Reality Research Lab (RVR)](https://reader030.vdocuments.us/reader030/viewer/2022040201/5e59cac601e14951923a6399/html5/thumbnails/37.jpg)
3D Puzzle by Optical Finger Tracking
![Page 38: Doru Talaba, Professor Robotics and Virtual Reality ......Doru TALABA, Transilvania University of Brasov nsil 1 Doru Talaba, Professor Robotics and Virtual Reality Research Lab (RVR)](https://reader030.vdocuments.us/reader030/viewer/2022040201/5e59cac601e14951923a6399/html5/thumbnails/38.jpg)
Ambient assisting living application
![Page 39: Doru Talaba, Professor Robotics and Virtual Reality ......Doru TALABA, Transilvania University of Brasov nsil 1 Doru Talaba, Professor Robotics and Virtual Reality Research Lab (RVR)](https://reader030.vdocuments.us/reader030/viewer/2022040201/5e59cac601e14951923a6399/html5/thumbnails/39.jpg)
“Point and command” interaction metaphor for AAL
Doru TALABA, Transilvania University of Brasov 39 Doru TALABA, Transilvania University of Brasov
- Multi-modal input for Assistive humanoid robots
- Modalities involved: speech, tracking (kinect) and natural gesture
- Goal: Natural interaction with a robot-assistant in home/office environment
![Page 40: Doru Talaba, Professor Robotics and Virtual Reality ......Doru TALABA, Transilvania University of Brasov nsil 1 Doru Talaba, Professor Robotics and Virtual Reality Research Lab (RVR)](https://reader030.vdocuments.us/reader030/viewer/2022040201/5e59cac601e14951923a6399/html5/thumbnails/40.jpg)
Layout of the room
Doru TALABA, Transilvania University of Brasov 40 DoDoDoDorurururururururururururururururu T T T T T T T T T T T T T T T T T T T T TALALALALALALALALALALALALALALALALALALALALALABABABABABABABABABABABABABABA,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,A, T T T T T T T T T T T T T T T T T T Transilvania Universitytytyty of Brasasasasasasasasovovovovovovov 40
The pointing angle
The assisted room
![Page 41: Doru Talaba, Professor Robotics and Virtual Reality ......Doru TALABA, Transilvania University of Brasov nsil 1 Doru Talaba, Professor Robotics and Virtual Reality Research Lab (RVR)](https://reader030.vdocuments.us/reader030/viewer/2022040201/5e59cac601e14951923a6399/html5/thumbnails/41.jpg)
Robot communication: voice and
gestures
Behaviors like « shruged » or « confused »
41
![Page 42: Doru Talaba, Professor Robotics and Virtual Reality ......Doru TALABA, Transilvania University of Brasov nsil 1 Doru Talaba, Professor Robotics and Virtual Reality Research Lab (RVR)](https://reader030.vdocuments.us/reader030/viewer/2022040201/5e59cac601e14951923a6399/html5/thumbnails/42.jpg)
Objects used for experiment
Doru TALABA, Transilvania University of Brasov 42
![Page 43: Doru Talaba, Professor Robotics and Virtual Reality ......Doru TALABA, Transilvania University of Brasov nsil 1 Doru Talaba, Professor Robotics and Virtual Reality Research Lab (RVR)](https://reader030.vdocuments.us/reader030/viewer/2022040201/5e59cac601e14951923a6399/html5/thumbnails/43.jpg)
43
NAO searching for the red ball
NAO grabbing the ball
![Page 44: Doru Talaba, Professor Robotics and Virtual Reality ......Doru TALABA, Transilvania University of Brasov nsil 1 Doru Talaba, Professor Robotics and Virtual Reality Research Lab (RVR)](https://reader030.vdocuments.us/reader030/viewer/2022040201/5e59cac601e14951923a6399/html5/thumbnails/44.jpg)
Doru TALABA, Transilvania University of Brasov 44
Nao brings the ball
![Page 45: Doru Talaba, Professor Robotics and Virtual Reality ......Doru TALABA, Transilvania University of Brasov nsil 1 Doru Talaba, Professor Robotics and Virtual Reality Research Lab (RVR)](https://reader030.vdocuments.us/reader030/viewer/2022040201/5e59cac601e14951923a6399/html5/thumbnails/45.jpg)
Gaze tracking by
ElectroOculoGraphy (EOG)
![Page 46: Doru Talaba, Professor Robotics and Virtual Reality ......Doru TALABA, Transilvania University of Brasov nsil 1 Doru Talaba, Professor Robotics and Virtual Reality Research Lab (RVR)](https://reader030.vdocuments.us/reader030/viewer/2022040201/5e59cac601e14951923a6399/html5/thumbnails/46.jpg)
EOG ! EOG=measuring the resting potential of the retina
by electrodes (Deng et al., 2010; Lv et al., 2010).
! The human eye can be seen as an electrical dipole:
– positive pole =cornea
– negative pole at the retina
! By measuring the voltage in reference positions
around the eyes, " the electric signal varies as the
eye-movement changes (Barea et al., 2002a;
Venkataraman et al., 2005).
Doru TALABA, Transilvania University of Brasov 46
![Page 47: Doru Talaba, Professor Robotics and Virtual Reality ......Doru TALABA, Transilvania University of Brasov nsil 1 Doru Talaba, Professor Robotics and Virtual Reality Research Lab (RVR)](https://reader030.vdocuments.us/reader030/viewer/2022040201/5e59cac601e14951923a6399/html5/thumbnails/47.jpg)
Based on feature identification
Doru TALABA, Transilvania University of Brasov 47
!
Postelnicu C., Girbacia F., Talaba D., 2012
![Page 48: Doru Talaba, Professor Robotics and Virtual Reality ......Doru TALABA, Transilvania University of Brasov nsil 1 Doru Talaba, Professor Robotics and Virtual Reality Research Lab (RVR)](https://reader030.vdocuments.us/reader030/viewer/2022040201/5e59cac601e14951923a6399/html5/thumbnails/48.jpg)
Doru TALABA, Transilvania University of Brasov 48
!
!
![Page 49: Doru Talaba, Professor Robotics and Virtual Reality ......Doru TALABA, Transilvania University of Brasov nsil 1 Doru Talaba, Professor Robotics and Virtual Reality Research Lab (RVR)](https://reader030.vdocuments.us/reader030/viewer/2022040201/5e59cac601e14951923a6399/html5/thumbnails/49.jpg)
From brain to Computer: bio-
signals tracking with BCI
!! !
![Page 50: Doru Talaba, Professor Robotics and Virtual Reality ......Doru TALABA, Transilvania University of Brasov nsil 1 Doru Talaba, Professor Robotics and Virtual Reality Research Lab (RVR)](https://reader030.vdocuments.us/reader030/viewer/2022040201/5e59cac601e14951923a6399/html5/thumbnails/50.jpg)
Doru TALABA, Transilvania University of Brasov 50
Project:
Interface between human brain and computer to explore "just-
by-THINKing" Computer Aided Design and Engineering
(Ongoing project, 2014 - 2017)
! brings research to the very edge of HCI/BCI, focusing on
the concept “just by thinking”
! extremely limited voluntary muscle movement to be
considered (just the eyes muscles).
! The holistic solution proposed in this project is targeting
the needs of people that cannot move at all.
![Page 51: Doru Talaba, Professor Robotics and Virtual Reality ......Doru TALABA, Transilvania University of Brasov nsil 1 Doru Talaba, Professor Robotics and Virtual Reality Research Lab (RVR)](https://reader030.vdocuments.us/reader030/viewer/2022040201/5e59cac601e14951923a6399/html5/thumbnails/51.jpg)
Concept and methodology
Doru TALABA, Transilvania University of Brasov 51
![Page 52: Doru Talaba, Professor Robotics and Virtual Reality ......Doru TALABA, Transilvania University of Brasov nsil 1 Doru Talaba, Professor Robotics and Virtual Reality Research Lab (RVR)](https://reader030.vdocuments.us/reader030/viewer/2022040201/5e59cac601e14951923a6399/html5/thumbnails/52.jpg)
INPUT from wearable devices,
sensor and intelligent “Things”
! A ICT branch of research of which Virtual
Reality and Robotics is taking unlimited
benefits
![Page 53: Doru Talaba, Professor Robotics and Virtual Reality ......Doru TALABA, Transilvania University of Brasov nsil 1 Doru Talaba, Professor Robotics and Virtual Reality Research Lab (RVR)](https://reader030.vdocuments.us/reader030/viewer/2022040201/5e59cac601e14951923a6399/html5/thumbnails/53.jpg)
Project “Spine”
Doru TALABA, Transilvania University of Brasov 53 Doru TALABA, Transilvania University of Brasov 5
![Page 54: Doru Talaba, Professor Robotics and Virtual Reality ......Doru TALABA, Transilvania University of Brasov nsil 1 Doru Talaba, Professor Robotics and Virtual Reality Research Lab (RVR)](https://reader030.vdocuments.us/reader030/viewer/2022040201/5e59cac601e14951923a6399/html5/thumbnails/54.jpg)
Intelligent
clothes
Project “SPINE”
(Ongoing project, 2014 - 2017)
![Page 55: Doru Talaba, Professor Robotics and Virtual Reality ......Doru TALABA, Transilvania University of Brasov nsil 1 Doru Talaba, Professor Robotics and Virtual Reality Research Lab (RVR)](https://reader030.vdocuments.us/reader030/viewer/2022040201/5e59cac601e14951923a6399/html5/thumbnails/55.jpg)
How it works
![Page 56: Doru Talaba, Professor Robotics and Virtual Reality ......Doru TALABA, Transilvania University of Brasov nsil 1 Doru Talaba, Professor Robotics and Virtual Reality Research Lab (RVR)](https://reader030.vdocuments.us/reader030/viewer/2022040201/5e59cac601e14951923a6399/html5/thumbnails/56.jpg)
Control of posture – similar principles as for
haptics
• Not always final posture could be prescribed from the first
time – a transition between initial and final posture could be
prescribed in a multi-phase treatment, which is case-
dependent;
• Should be done by a specialized software application,
based on a “medical prescription”. The prescribed “therapy” may consider progress parameters, measured
continuously.
• Another possible prescription " posture disturbances -
forcing limit postures – thus improving control capabilities
• Disturbances could be designed to create useful after
effects .
![Page 57: Doru Talaba, Professor Robotics and Virtual Reality ......Doru TALABA, Transilvania University of Brasov nsil 1 Doru Talaba, Professor Robotics and Virtual Reality Research Lab (RVR)](https://reader030.vdocuments.us/reader030/viewer/2022040201/5e59cac601e14951923a6399/html5/thumbnails/57.jpg)
• Healthy ageing: detection of pre-disposition
before it is too late
• Efficient treatment " avoid systematically
harmful postures, stimulate therapeutic ones
• Information communication between the doctor
and patient system: progress monitoring, multi-
phase treatments, etc.
Prevent and cure
![Page 58: Doru Talaba, Professor Robotics and Virtual Reality ......Doru TALABA, Transilvania University of Brasov nsil 1 Doru Talaba, Professor Robotics and Virtual Reality Research Lab (RVR)](https://reader030.vdocuments.us/reader030/viewer/2022040201/5e59cac601e14951923a6399/html5/thumbnails/58.jpg)
Doru TALABA, Transilvania University of Brasov 58
Conclusions
! Most of the VR Output modalities (3D visual,
haptic and audio) – already technology
! Input modalities development is the new focus in
Virtual Reality
! Challenging issues related to the human behavior
! Interdisciplinary research: Mechanical & Cognition
& IT & Electronics etc