the effects of visual realism on search tasks in mixed reality simulations-ieee transaction paper...

1

THE EFFECTS OF VISUAL REALISM ON SEARCH TASKS

IN MIXED REALITY SIMULATION

Under the gu idance o f RA J I R . P ILLA I (Asst. Professor)

Yadhu Kiran

Roll No. 63

Seminar on :

2

INTRODUCTION

• Experiments in AR Domain are difficult.

- Difference in Display Systems used. (Result are not generalizable)

- Difficult to conduct Controlled Experiments in Unpredictable Environment Conditions.

3

DISPLAY SYSTEMS

Head Mounted Display ( H. M. D. )Data Glove

Binocular Omni-Orientation Motor

Cave Automatic Virtual Environment ( CAVE )

4

SOLUTION

Hint : Real Display systems can be used only in Real Environment.

Hence use, Mixed Reality Simulations.

• A concept of using High Fidelity Virtual Reality systems to simulate environment from MR Continuum.

• Different display systems are simulated by varying the level of immersion.

5

FIDELITY OF MR SYSTEMS

Display Interaction SimulationFidelity Fidelity Fidelity

(Depends on (Depends on (Depends on Sensory Fidelity) Interactions) Environment &

Fidelity of Objects)

6

VISUAL REALISM

• The degree to which images on simulated environment is perceived by user in the real world.

• The concept of visual realism is very important for understanding and verifying Fidelity of Simulations.

• In this paper, the aim is to conduct experiments in different Simulated Environments rather than Real World Environment by varying the level of Visual Realism in each Simulation.

7

What level of Visual Realism is required for replicating AR Experiments?• Search Experiments were conducted and

performance was analyzed in 3 MR Simulations and Real World.

Increasing Order of Visual Realism

Level 1 Level 2 Level 3 Real World

8

VISUAL REALISM FACTORS

• Shadow Softness

• Surface Softness

• Lighting

• Geometry

Analyzing these, it was found that Soft Shadows and Surface Type affects perception of Visual Realism.

9

CREATING VIRTUAL MODELS

• Different formats and techniques are used for creating Virtual Models, each resulting in different levels of Visual Realism.

Model Geometry Color Information

Lighting Techniques

• Image Based

• Point Based

• Stored Polygonal Data

• Simple Colors

• Coloring Material Properties

• Textures

• Ray Casting

• Ray Tracing

10

THE EXPERIMENT

1. Verify the validity of MR Simulations by checking whether Visual Realism has any impact on Search Task Performance. Or Simply, Does Visual Realism impacts results?

2. If so, How?

There were 2 goals to be achieved :

11

Steps :

• Design outdoor AR experiment involving common tasks.

• Choose a Real World location for experiment.

• Use 3 Models and Simulate them for conduction experiment indoors.

12

TASK & ENVIRONMENT

Low Fidelity Medium Fidelity High Fidelity

• Polygons for major scenes only.

• Neglected minor features.

• Plain color was used but no textures.

• Increased number of polygons for Geometry.

• Simple low resolution Textures.

• No change in lighting.

• High polygon count.

• High resolution images instead of textures.

• Baked Lighting.

Virtual Models were designed using 3D Modeling Techniques.

13

THE TASK

• To search for both ‘Virtual’ and ‘Physical’ information based on both ‘Virtual’ and ‘Physical’ Information Criteria.

( The most recognizable AR Experiments which require very little training. )

They created Virtual Information on the scene such that it must be suitable for the current environment where a particular user might want to do it.

14

The high-fidelity model and the real environment, both with virtual annotations.

Miscellaneous Information

Information about Office

Classroom

Laboratory

15

• Created 16 questions.

• There were mix of questions that required the user to find both virtual information and physical information based on both virtual and physical information context.

• For each task question the participant was required to find certain pieces of information and verbally report that information.

The Method :

Target information refers to the exact nature of the response required by the task question.

Physical information is inherent in the real world while virtual information needs to be provided by the virtual icons and the text they contain.

Criteria information refers to what information the user is using to search the scene

16

DISPLAY SYSTEM USED

NVis SX111 HMD

• Pointgrey USB3 Flea camera

• 102° Horizontal FOV

• 64° Vertical FOV

• 1600 x 1024 pixels resolution

• 60 FPS

+ InterSense IS900 Tracking System.

17

Additional Hardwares and Softwares used :

• Windows 7 PC with a Quadro 5600

• Intel Core2 2.4 GHz Duo-Core CPU

• 2 GB Memory

• WorldViz’s VR Toolkit Vizard 4.0

18

EXPERIMENT DESIGN

Dependent Variable for each task was the Search Time.

The 3-Time Question Sessions :

• Q1 – Q5 : Single item search

• Q6 – Q11 : Multiple item Search

• Q12 – Q16 : Comprehensive

Avoided ‘Learning Effect’ by giving 2 Minute breaks after each session.

19

THE PROCEDURE

• Color Vision Test for Participants.

• Training with Icons using HMD.

• Screening Participants.

20

THE QUESTIONS :

21

ISSUES WITH REPLICATING AN OUTDOOR ENVIRONMENT

• Accuracy of Models

• Real Location undergone changes.

• Geometrical difference between CAD plan and Real Buildings.

• Weather change affected lighting.

22

RESULTS

1. Difference in Task Times :

• For the analysis of the time results from our experiment, ANOVA test was used to determine if the level of realism had a significant effect on task time.

• In the second stage a post-hoc analysis of all task pairs was performed.

23

A plot of the mean task times, 95 % confidence intervals, and significance P values for each level of realism for all 16 task questions

Result of ANOVA :

24

P - Value :

• These result were reported as P-Values.

• P-Value is a measure how likely this spot will be obtained if no real difference existed. A smaller P-Value indicates more significant is the difference between groups. (Small usually means 0.05).

Only four of the 16 task questions revealed a significant difference between the realism conditions.

Q3 (a door under bridge)

Q6 (a tree) For these tasks, A Tukey post-hoc analysis was used to determine

Q8 (trash bin) pairwise difference.

Q12 (a comprehensive search)

25

A Tukey

post-hoc

analysis

of Q3, Q6,

Q8 and

Q12. Pair-

wise

significan

t

difference

s between

each level

of visual

realism are

highlighted

in red

26

Observations from Tukey Post-hoc test :

• For Q3, Q6, Q8 the difference in significant only in Real Conditions.

• Camera artifacts, vegetation, and lighting conditions, affected the visibility of the real-world objects much greater than the virtual objects.

• Cluttered environment might have increased task time when the search criteria was relatively small.

27

2. Equivalence in Task Times :

• All other tasks did not reveal significant difference for level of realism.

• For further investigation, the Two-one-sided t-test (TOST) analysis was used to look for equivalent task time performance within the groups.

• All other task questions produced at least one instance where two of the realism conditions were equivalent except Q12 and Q13.

What type of courses are generally taught in North Hall ?

Which professor has the office which is located the furthest from their lab ?

Comprehensive

28

3. Notable Task Questions :

• Q8 - the only task with results indicating any obvious performance trend with respect to task time.

• Q12 - by far the most difficult one. the results here reflect the difficulty of the task and the effects from strategy more than from level of realism.

29

4. Task type and Visual Realism :

• Participants were quicker to respond when the search target type was virtual information.

• Search criteria type is shown to be significant in all four conditions while search target type has a similar effect on the high and the real conditions. This suggests that the high level of realism performs similarly to the real conditions.

• The similarity of these effects support the validity of MR Simulation.

30

FUTURE WORK

• Clarify the exact cause of the differences between the AR case and the simulated AR cases.

• Improving the actual AR experience for better ways to present AR imagery to users.

• Exploring Mixed Reality Simulation for different task types (not just search, but also browsing and annotation and interaction tasks) and for different training environments to simulate.

31

REFERENCES

• D. A. Bowman, C. North, J. Chen, N. F. Polys, P. S. Pyla, and U. Yilmaz. Information-rich virtual environments: theory, tools, and research agenda. In Proceedings of the ACM symposium on Virtual reality software and technology, VRST ’03, pages 81–90, New York, NY, USA, 2003. ACM.

• M. Elhelw, M. Nicolaou, A. Chung, G.-Z. Yang, and M. S. Atkins. A gaze-based study for investigating the perception of visual realism in simulated scenes. ACM Trans. Appl. Percept., 5(1):3:1–3:20, Jan. 2008.

• J. L. Gabbard, J. E. Swan, II, and D. Hix. The effects of text drawing styles, background textures, and natural lighting on text legibility in outdoor augmented reality. Presence: Teleoper. Virtual Environ., 15(1):16–32, 2006.