feedback and presentation in the microsoft distributed classroom project richard anderson university...

Feedback and Presentation in the Microsoft Distributed Classroom

ProjectRichard Anderson

University of Washington and Microsoft Research

Richard Anderson UW & MSR 2

t

Overview

• Presentation system– Develop system for lecture delivery that

increases instructor flexibility

• Feedback system– Develop system that supports increased in

class communication

• Design experiment– Experiment with system in actual classroom


t

Educational Technology

…in the winter of 1813-1814 … I attended a mathematical school kept in Boston…on entering his room, we were struck at the appearance of an ample Blackboard suspended on the wall, with lumps of chalk on a ledge below, and cloths hanging at either side. I had never heard of such a thing before. [Mr. May, 1866]


t

Classroom Technology

• Black board

• White board

• Overhead projector

• Transparencies– Persuasion, PowerPoint, LaTeX

• PC with data projector


t

Research goal

• Develop presentation system for PC/Data Projector– Increase flexibility in lecture delivery– Support greater communication

• Design for instructional use at the university level

• Evaluate based on pedagogical impact


t

Scenarios

• Distributed classroom– Multi site video conferenced classes– Data projectors in each room– Possible desktop access

• Lecture hall– Public projected display– Wireless enabled class– Possible student devices

• Laptop, tablet, handheld


t

Teaching styles

• Wide range of teaching styles and opinions• Instructors will use technology in different ways• Instructors will use technology in unexpected

ways• We are not comparing between different

technologies– PC delivery already established– Some situations require PC delivery


t

DISC Project

• Internet based video conferenced classes

• Multicast technology to support multiple sites

• Deploy system on a PC

• Simple startup (the big red button)


t

Preliminary Study

• UW Professional Master’s Program– Video conferenced class– Netmeeting for PowerPoint distribution– Smartboard ™ Electronic whiteboard

• Instructor interviews

• Student interviews

• Classroom observations

• Review of video archive (5 years of data)


t

Study conclusions

• Need to support handwriting and slides• Current electronic whiteboards not

adequate• Important to integrate whiteboard and

slides– Every observed use of whiteboard was in the

context of a slide

• Use pen based device as presenter console


t

Electronic whiteboard comments

• "The whiteboard would be better if: both it and the slides could be seen at the same time the handwriting was captured more accurately; if I write in anything other than slowly-drawn block letters, it looks like a heap of pick-up sticks."

• "The problem with the whiteboard is that, unless you draw very carefully, half of it won't come out, and similarly when you erase. This is enough of a hassle that I wound up not using it much."


t

PowerPoint comments

• "I could do some of what I wanted with PowerPoint animations, but these took a huge amount of time to prepare. Previously I had used slide overlays (transparencies) a lot, frequently presenting a slide that was partially filled in (some background and a question) and then using class discussion to fill it in."

• "PowerPoint is also a pain for the same reason it's a pain in non-distance classes - the slides impose a rigid structure on the lecture, and make it more difficult to adjust to the interactions that occur during it. "


t

Presentation system

• Key idea – integration of ink and slides

• Lecture from tablet pc– Mouse input adequate for gesturing, but not

for writing

• Write on PowerPoint

• Synchronized display of slides

• Real time transmission of ink


t

Implementation

• Export slides as images, broadcast images– Advantages: easy and it worked. Viewer does

not need to have PowerPoint. Easier support of a range of devices.

– Disadvantages• Loss of fidelity• Modifying intermediate form• Loss of animations, transitions


t

Technical Issue: Slide distribution

• Current approach– Broadcast slide deck using multicast– Interpacket delay introduced to avoid network

saturation– Rebroadcast on request

• Issues– Dealing with clients joining sessions– Large images (instructor’s baby pictures)

• Recognized as a challenge for distributed presentations


t

Whiteboard features

• Scrollable overlays (virtual mylar)– Simulate overhead projector

• Minimize or hide slide for whiteboard use

• Ink and transparent ink

• Region highlight

• Remote pointer


t

Classroom use

• Fujitsu tablet

• UW Distance master's class (Professional Master's Program, PMP)– Database– UW Lecturer, MSR classroom– 3 hour evening lecture for 10 weeks

• Several sessions in introductory computer programming


t

Observation results

• Used for both highlighting and examples

• Lecturers used "dense PowerPoint"– Presentations not designed for ink annotation

• Novelty wore off quickly– Instructors soon became comfortable

• Used from podium

• Some difficulty writing on Fujitsu tablet

• User interface issues to be resolved


t

Unexpected use

• Alon Halevy: "Could I get volunteers to come up and write their sample queries on the tablet"

• Martin Dickey. TV Talk show style. Took the tablet to the audience, and had them write on it. Wireless tablet.

• Used the device to allow new interaction. Technology to enhance classroom collaboration.


t

Current experiment

• Use presentation tool in Introductory Computer Programming, Summer 2002

• Evaluate tool in sustained, large lecture use

• Deeper questions– Does the presentation tool enable a better

educational experience?– Does the presentation tool alter the

pedagogy?


t

Research questions

• Gestures for lecturing from tablet– Specific technology very important– Limited experience with tablet pc– Specific gestures for presentation– Constraints imposed by “performance

environment”

• Opportunities for manipulation of slide content


t

Feedback System

• Motivation and challenge– Increase level of interaction in large lecture

classes– Give instructor information on level of

understanding of lecture

• Real time feedback– Students annotation of materials on personal

devices– Real time summarization for instructor


t

Prior work

• Telep (and others)

• Active class (UCSD)

• Class talk (UMass)

• Classroom 2000

• University of Maryland

• Results mixed


t

Challenges to Interaction in Large Classes

• Student apprehension– Concerns about interrupting class for

“unimportant” questions

• Comment verbalization– Difficulty in expressing comments

• Feedback lag– Questions not formed until topic has been

changed

• Single speaker paradigm


t

Design Principles

• Non-verbal communication

• Anonymity

• Shared context for feedback

• Rapid, automatic synthesis of feedback

• Simple interfaces

• Value to all users


t

Important questions

• Will instructors be willing to use system?

• Will instructors be able to take advantage of feedback during lecture?

• Will students be willing to use system?

• Will students benefits to students outweigh the costs of using the system?


t

Feedback System

• Viewers select annotations from context menus

• Annotations associated with slide geometry

• Real time summary of annotations provided to viewer

• Potential to deploy on a wide range of devices– Take advantage of existing student devices


t

Feedback categories

• Low impact to make comments• Allow presenter to solicit particular

feedback– State of mind (I’m confused, I understand)– Suggested action (Explain, Elaborate,

Example, Define)– Pacing (Slower, Faster, Just Right)

• Presenter may set categories in a preprocessing phase


t


t

Summarization of feedback

• Presenter needs summarization if there is significant feedback

• Summarization done on "semantic regions"– Identify slide geometry - rectangles– Summarize comments in rectangles

• Alternate summarization methods– Intensity or Histogram

• Comment menus also linked to regions


t

CSE 100 Experiment

• Design experiment• Done in actual class• Understand use and effect in real context• Engineering a learning environment by

introducing an intervention• Study environment to guide design• Conduct pen and paper feedback • Use laptops in class for feedback


t

Study

• CSE 100: Fluency in information technology• Large, diverse introductory class

– 150 students, large lecture format

• Observations– Very limited student participation– Most students surveyed reported speaking in class

less than once per week– Instructor reported few questions, so she could not

gauge student understanding


t

Pencil and paper study

• Group of students given paper copies of the slides

• Students asked to annotate slides from predefined categories– Slow Down, Explain, Question, Cool topic

• Simulate student experience

• Evaluate type of feedback delivered


t

Classroom study

• Limitations:– One class period– Seven laptops (8 of 100+ students

participated in study)– Novelty of system and participation in a study

likely influenced how students used system– Instructor had a TA mediate feedback

(intended use of the system is for instructor to get feedback directly)


t

Study data

• Logging of system actions

• Classroom observations

• Student questionnaires

• Instructor interviews

• Draw qualitative conclusions from multiple sources of data


t

System usage

• Students using laptops gave between 1 and 9 annotations

• Students using paper gave between 7 and 15 annotations

Category type

Laptop Paper

Slow Down

2 7

Question 2 21

Explain 12 19

Cool topic 16 13


t

Instructor response

• Change in presentation style• Altering pace of lecture in response to slow

down comments• Providing additional explanation in response to

question or explain• Examples of terms annotated

– Tracking number – forward reference, clarified later– Batch processing – old term which students weren’t

familiar with


t

Reactions

• Positive instructor interaction– “[The students] can actually say something

right away, and if the teacher has the wherewithal or time, they can actually make small improvement or small changes that will treat those anxieties or concerns doing what they’re doing that the students like”


t

Student reactions

• Students completing survey said that the instructor responded to their comments

• “Further explanations were given when asked to”

• “The computer guy [the mediator] told [the instructor] that there was a question and she did respond to it”


t

Effect on classroom

• Student participation on day of experiment was consistent with previous classes– 6 student initiated interactions on day of

experiment, 11 teacher initiated interactions– Average of 6 student initiated interactions in

earlier classes, 6 teacher initiated interactions

• New style of interaction– Teacher response to feedback creating

discussion


t

Conclusions

• Positive evidence from design experiment– Students used system– Instructor was able to take advantage of

information– Linking feedback to content was important– Selected categories appropriate– Positive response from instructor and

students– Level of interaction did not drop


t

Future work

• System development

• Larger field tests– Introductory computing, summer 2002– Direct instructor use of system– Sustained use of feedback system

• Feedback infrastructure as platform for “structured interaction”


t

Acknowledgments

• UW– Tammy Vandergrift, Steve Wolfman, Ken

Yasuhara

• MSR DISC Project– portal.learningwebservices.com/disc

feedback and presentation in the microsoft distributed classroom project richard anderson university...

Documents

pc delivery slide

handheld slide

actual classroom slide

data projector slide

slide use pen

microsoft research slide

years of data slide

presenter console slide