virtual and augmented reality 2016 miect, mieet, mei...
Post on 21-Jul-2020
2 Views
Preview:
TRANSCRIPT
Virtual and Augmented Reality – 2016
MIECT, MIEET, MEI
Introduction to course
Universidade de Aveiro
Departamento de Electrónica,
Telecomunicações e Informática
Realidade Virtual e Aumentada Beatriz Sousa Santos, 2016/2017
2
Virtual and Augmented (Mixed) Reality
• Has been expanding from a research field into a commercially viable
technology.
• is highly interdisciplinary, combining work in diverse disciplines:
- more technology oriented research:
computer graphics, display technology, computer vision,
sensors, signal processing and computer networking.
and
- human centered research:
interaction techniques, design, human factors,
wearable computing and mobile computing.
• The growing interest in VAR applications is creating new challenges
for research in all of these areas
3
What is Virtual Reality?
“For better or worse, the label virtual reality stuck to this particular branch of computer graphics.
I define a virtual reality experience as any in which the user is effectively immersed in a responsive virtual world. This implies user dynamic control of viewpoint.”
“A high-end user-computer interface that involves real-time simulation and interaction through multiple sensorial channels (vision, sound, touch, smell, taste)”.
(Fred Brooks, 1999)
4
(Burdea et al., 2004)
Virtual Reality in practice for years – examples
Industry
Jaguar / Land Rover
Ford
http://www.youtube.com/watch?v=j3qcnvgVlNk
https://www.youtube.com/watch?v=BEFfp2QhHZ
U
5
Medicine
http://www.youtube.com/
watch?v=zJmrcEM-uvA
7
Entertainment
http://www.oculusvr.com/
Oculus Rift
2014; ~300 USD
8
http://www.youtube.com/watch?v=N8uuDT5AYts
“AR is a novel human machine interaction tool that overlays computer-
generated information on the real world environment. The information display
and image overlay are context-sensitive, which means that they depend on the
observed objects.” (Ong et al., 2008)
10
What is Augmented Reality?
• According to Azuma Augmented Reality systems have the following
characteristics:
“1) Combine real and virtual
2) Interactive in real time
3) Registered in 3D”
11
http://www.youtube.com/watch?v=8UxWkZtUKaI
Early Augmented Reality – examples
Lego
12
Ray Ban Virtual Mirror
13
http://www.youtube.com/watch?v=Ag7H4YScqZs
http://www.ray-
ban.com/usa/virtual-mirror
Augmented Reality Sandbox
14
http://idav.ucdavis.edu/~okre
ylos/ResDev/SARndbox/
http://www.techradar.com/reviews/wearables/microsoft-hololens-1281834/review
The Future of AR?
Hololens
2016; ~3000 USD?
https://www.microsoft.com/microsoft-hololens/en-us
R. Furlan,”The future of augmented reality: Hololens - Microsoft's AR headset
shines despite rough edges”, IEEE Spectrum, 2016, 53 (6), p. 21
Mixed Reality: “Continuum”
“Augmenting natural feedback to the operator with simulated cues”
16
(Milgram et al., 1994)
Mixed Reality (MR)
Real Envinoment Augmented Reality Augmented Virtuality Virtual Reality
(Steinicke et a., 2009) http://www.youtube.com/watch?feature=endscreen&v=UgQfo7eNFdw&NR=1
Team
Beatriz Sousa Santos + Paulo Dias
Other courses by this team:
Human-Computer Interaction, Visual Computing, Information
Visualization, Computer Graphics, 3D Modeling and Visualization
Interactive Computer Systems
Dissertations:
Medical and Scientific Data/Information Visualization, Virtual Reality,
Augmented Reality, Interaction with large displays …
18
We thank all the people involved:
• Guilherme Campos, Samuel Silva, Carlos Ferreira
• MSc and PhD Students:
Mário Cruz, Paulo Moreira; Angela Pimentel; Gustavo Corrente; Ana
Cantanhede; Jan-Willem Baggerman, Pauline van der Horst, Frank van
Huussen; Ricardo Casaleiro; Ricardo Seco; Paulo Nascimento, Mauro
Ribeiro; Bruno Morais; Nuno Veloso; Danilo Sousa, João Pinto, Sérgio
Eliseu, João Cardoso, João Pinto, Luís Afonso ...
• And all participants in demos, tests and usability studies...
20
Topics
• Introduction to Virtual Reality (VR) and Augmented Reality (AR)
Definition, historical perspective and evolution, examples
• Modeling of Virtual Environment (VE)
• Input and output devices, tracking, cameras
• 3D user interfaces and interaction techniques
• Human factors in VR and AR
• Frameworks for VR and AR
• Traditional and emerging applications
http://sweet.ua.pt/bss
21
Lectures and Lab classes (tentative)
1 (13/09/16) - Introduction to the course
2 (20/09/16) - Introduction to Virtual and Augmented Reality
Paper presentation Guidelines
Frameworks para RV e RA / VR and AR Frameworks
3 (27/9/16) - Introduction to Virtual and Augmented Reality (cont.)
Unity
4 (4/10/16) - Input Devices / Paper presentation
Unity
5 (11/10/16) – Visit to Centro Rovisco Pais (to be confirmed)
6 (18/10/16) - Output Devices (visual) - Stereoscopy / Paper presentation
Unity - Practical assignment
22
7 (25/10/16) – Unity - Practical assignment
8 (8/11/16) - Output Devices (other) - Paper presentation
Unity – Practical assignment
9 (15/11/16) - Interaction in VR and AR - Paper presentation
Introduction OpenCV
10 (22/11/16) – Cameras and world registration - Paper presentation
Camera calibration with OpenCV
10 (29/11/16) - Human Factors in VR and AR - Paper presentation
Toolkits for AR (ARToolkit and others)
12 (06/12/16) – Hardware Demo
13 (13/12/16) – Assignments: demo and presentation
14 (20/12/16) - Physics modelling/ Paper presentation
23
Lectures and Lab classes
• ~1h30 min – lecture; including paper presentation and discussion
• ~1h 30 min – lab
• 2 sessions devoted to presentation and demo of the assignments
Unity, OpenCV, ARToolkit ....
24
Main bibliography
• Jerald, J., The VR Book: Human-Centered Design for Virtual Reality, ACM and
Morgan & Claypool, 2016
• Schmalstieg, D., Hollerer, T., Augmented Reality: Principles and Practice (Usability).
Addison-Wesley Professional. (2016).
• Craig, A., Sherman, W., Will, J., Developing Virtual Reality Applications:
Foundations of Effective Design, Morgan Kaufmann, 2009
• Burdea, G., P. Coiffet, Virtual Reality Technology, 2nd ed., Wiley, 2003
• Bowman, D., E. Kruijff, I. Poupyrev, J. LaViola, 3D User Interfaces: Theory
and Practice, Addison Wesley, 2004
• Anthes, C., M. Wiedemann, D. Kranzlmüller, “State of the Art of Virtual Reality
Technology”, IEEE Aerospace Conference, 2016.
• Journal and Conference Papers
25
Other bibliography
• Kim, G., Designing Virtual Reality Systems: The structured approach, Springer 2005
• Mullen, T., Prototyping Augmented Reality, Sybex, 2011
• Furht (ed.), Handbook of Augmented Reality, Springer, 2011
• Stanney, K.(ed.), Handbook of Virtual Environments, Design, Implementation and Applications, Lawrence Erlbaum Associates, 2002
• Journal and Conference Papers
26
- Exam – 40%
- Programming assignment – 20% + 30%
- Paper presentation and discussion – 10%
• Working students must decide on the type of assessment and inform us until September, 30
Assessment
27
Practical assignment (general characteristics)
• Groups of two students
• Mini-project
• Development of a simple application using specific devices and
libraries
• Existing code may be used, provided that it is explicitly identified
and its source adequately cited
29
This year’s “client”:
30 http://www.tsf.pt/portugal/saude/interior/hospital-rovisco-pais-e-central-na-recuperacao-de-
doentes-com-avc-4486208.html
Our devices
31
Analyze and present a conference/ journal paper:
1 student – short paper (VR2016, ISMAR2015)
2 students– long paper (SIGGRAPH2016 or journal)
Each student must:
- Decide if he/she will present alone or with a colleague
- select two papers
or send preferences to (bss@ua.pt)
until September, 29
- Propose another paper
- Read the paper presentation guidelines (presentation guidelines)
- Make a 12 mim (short paper) or a 25 min (2 students - long paper) presentation
- Send the slides to (bss@ua.pt)
33
Papers to read, present and discuss
http://ismar.vgtc.org/
http://ieeevr.org/2015/
34
http://ieeexplore.ieee.org/Xplore/
http://dl.acm.org/
http://s2016.siggraph.org/
http://www.springer.com
/computer/image+proce
ssing/journal/10055
Discuss papers
• After each paper presentation:
- All students vote on their colleagues presentation
(through googleforms)
- Two students will discuss the paper
35
https://docs.google.com/forms/d/1rXAb2LlJTaj74-
xWMhhYMwWcNoUlzBUiyY9gZp2OFT0/viewform
This year’s “client”:
36 http://www.tsf.pt/portugal/saude/interior/hospital-rovisco-pais-e-central-na-recuperacao-de-
doentes-com-avc-4486208.html
• Students’ profile?
• Questions?
37
Interesting recent papers and links
• Rod Furlan,”The future of augmented reality: Hololens - Microsoft's
AR headset shines despite rough edges”, IEEE Spectrum, 53 (6), p.
21, DOI: 10.1109/MSPEC.2016.7473143
38
top related