1

Advanced Educational Interfaces:Teaching and Learning with Augmented Reality

Brett E. Shelton

Ph.C. Educational TechnologyCollege of EducationUniversity of Washingtonbshelton@u.washington.edu

PETTT – Program for Educational Transformation Through Technology

2

Presentation Overview

The Problem Research

Overview Focus Methods

Results Learning Assessment Interaction Analysis

Discussion of Trends Future Directions Doctoral Consortium

Issue – Video Ex.

3

The Problem

Traditional instruction uses many different representations, always involving a significant level of abstraction

Interface constraints Students have difficulty accommodating

spatially related knowledge Instructors are challenged to represent spatial

phenomena

4

The Augmented Reality (AR) Interface

Interaction with the real environment Augmented with virtual 3D objects Exist in 3D space

Consists of: Display Mode of manipulation for 3D objects Accompanying HW and SW

5

Research Overview

Building on recent work in AR, cognition, vision science, educational technology

Examine AR for viewing and manipulating 3D objects for classroom-based curriculum

What is happening? In what ways do students’ understandings

change? Resources?

Interface design Cognitive advantages

6

Research Focus

Introductory physical science (Geography, Astronomy, etc.) Working knowledge of physical landscape

and environmental processes Exist at different spatial scales Essential for more sophisticated concepts

Ex. A Private Universe film Earth closest to sun –> summer

7

Phase 1 Methods

Introductory Geography at a 4-year university 34 students, summer 2002 3 to 6 animated models of earth, sun,

supporting models and annotations Replaced traditional instruction

covering this material

8

Model #2

9

Model #2

10

Model #5

11

Phase 1 Methods: Procedure and Sample

Students participated one at a time over a 2 day period

Pre-assessment and post-assessment worksheets 3 questions, 1 per topic Written and sketch explanation Assessed on a 0-5, novice-expert scale

Students received participation grades, examination the following week

12

Phase 1 Methods: Data Gathering

Pre-assessment, intervention, post-assessment

Intervention: Subjects given a brief introduction of the

interface and how to manipulate virtual objects View each model, ask questions, answer

questions Sessions were audio/videotaped from 2

perspectives

13

3rd Person Perspective

14

1st Person Perspective

15

Assessment of Learning with AR

How did students’ performance change from pre-assessment to post-assessment?

Which students improved and which did not?

Which topics was student performance affected?

16

Phase 1 Results:Assessment Sheets

General improvement of conceptual and factual understanding

Misrepresentation of factual information was greatly reduced

17

Phase 1 Results:Interaction analysis

Short adjustment time to interface familiarity

“Explorers” were more comfortable with content, “Watchers” less, regardless of interface familiarity

18

Results – Interaction analysis

Physical inspection of the content was a key to understanding how multiple elements worked together Able to switch their attention to

different things happening in the same model

Either moving the model, or depending on the chosen angle, switching their area of focus

19

Results – Interaction analysis

Control over content Students had difficulty in articulating

this as a factor Use a “diagram” in a way they could

control what they were looking at

20

Discussion of Trends

Getting beyond abstraction No need for orange-earth

AR may provide more powerful learning opportunities We can represent 3D objects as they exist in

relative 3D space for learning Augmented reality allowed students to view and

interact with sophisticated phenomena in real-world setting Time, position, movements and angles In the classroom may benefit quality in curriculum

involving complex 3D spatial phenomena and concepts

21

Future Directions(Phase 2: Oct-Nov 2002)

Interview data gathering for pre- and post-assessment

Videotape analysis of experts for assessment comparison

Methodologically tie student learning through their activity with AR

Help inform new studies of AR Help inform AR design for use with

specific learning applications

22

Issue Focus

Video analysis of augmented reality activity

Link visual behaviors with information acquisition 1st and 3rd perspectives together Tracing specific student action

After specific instruction As an “overall” activity

23

Interaction Analysis(Jordan & Henderson, 1995)

Considers: Physical activity with interface Cognitive activity with content Verbal exchanges

Within (in addition to) IA How might I identify those resources within the

interface the student is using? In what ways do these resources impact the

cognitive processes involved in learning? How can I further trace student activity to

derive meaningful insights about what and how students are learning?

24

Video Samples…

25

Issue Revisited

Further ideas in how I might… Link students’ visual behaviors with

information acquisition

Identify useful interface resources Link these resources to cognitive

learning processes