shining a flashlight on teaching and learning with the web dr. craig m. ross department of...

Shining a Flashlight on Teaching and Learning with the Web

Shining a Flashlight on Teaching and Learning with the Web

Dr. Craig M. RossDepartment of Recreation & Park Administration

cmross@indiana.edu

Rising Use of Technology in Instruction

59.3

42.7

30.7

0

10

20

30

40

50

60

%

Email WebResources

Course WebPage

Percentage of courses using IT resourcesCampus Computing Project 2000

1995199819992000

“...incoming college students expect to receive technology in the classroom just as much as they expect to receive a library card to use the library.”

“New technologies seldom support old working practices...they tend to undermine existing practices and demand new ones.”

(Gavin, 1999)

Presentation Overview

Background • Pieces of the puzzle• Teaching, learning, and technology

Research purpose Seven Principles of Good Practice Flashlight Program overview Study overview Methodology Results Conclusions Discussion

Pieces of the Puzzle

Teaching

HPER

Technology

Learning

Research Purpose

To study the impact of web-enhanced course delivery methods on undergraduate teaching and learning from a student and faculty perspective• A web-enhanced course was defined as a course that is taught in

a classroom and is supplemented with course related information delivered on the internet. These include lectures/notes on the web, on-line gradebook and quizzes, chat rooms, discussion boards, web links, e-mail, etc.

Specifically:• Impact on teaching and learning practice across a

number of courses in School of HPER• Foster Seven Principles for Good Practice • Perceived impact on learning – students/faculty• Learner satisfaction• Most and least useful features• Greatest barriers/benefits to faculty

1. Frequent Student-Faculty Contact• “Frequent student-faculty contact in and out of

classes is the most important factor in student motivation and involvement.”

2. Cooperation Among Students• “Learning is enhanced when it is more like a team

effort than a solo race.”

3. Active Learning• “Learning is not a spectator sport. Students do not

learn much just sitting in classes listening to a teacher, memorizing pre-packaged assignments, and spitting out answers.”

4. Give Prompt Feedback• “Knowing what you know and don’t know focuses

learning. Students need appropriate feedback on performance.”

Seven Principles For Good Practice in Undergraduate Education

(Arthur Chickering & Zelda Gamson, AAHE Bulletin, March 1987)

Seven Principles For Good Practice in Undergraduate Education (Cont’d)

5. Time on Task• “Time plus energy equals learning. There is no

substitute for time on task.”

6. Communicate High Expectations• “Expect more and you will get it. High expectations

are important for everyone.”

7. Respect Diverse Talents and Ways of Learning• “There are many roads to learning. People bring

different talents and types of learning to college.”

(Arthur Chickering & Zelda Gamson, AAHE Bulletin, March 1987)

Flashlight Program Model

(http://www.tltgroup.org/programs/flashlight.html)

HPER Students and Faculty

Students• # of students: 1,515• 53% male 47% female• BS Degree Programs: 3 with 19

undergraduate major emphases• Avg credit hour load: 13.39 hrs/semester • Avg class size: 40 students

Faculty• #: 62 full-time

Departments• Applied Health Science• Kinesiology• Recreation & Park Administration

Study Population and Methodology

Spring 2002 Student survey:

• 53 item in-class survey• 18 sections of students in courses of 13 instructors• Total number of students enrolled: 1,050

Items drawn from Flashlight Current Student Inventory

Focus groups Faculty survey:

• Full-time faculty• Currently using web enhancements• 24 item mail survey

Focus groups

Student Demographics

737 survey respondents (70.2%)• Applied Health Science – 17.5% (129)• Kinesiology – 19.5% (144)• Recreation & Park Adm – 20.2% (149)• Other – 42.7% (315)

Gender• Female: 68.9% Male: 31.1%

Students by Class Level (n=737)

Class Level Percentage

Freshman 12.6

Sophomore 24.2

Junior 29.9

Senior 33.3

Students by Age (n=737)

Age Percentage

18 5.8

19 16.8

20 22.5

21 27.4

22 17.2

23 or older 10.2

Student Computer Skills

23.5

12.9

69.2

53.5

7.3

33.6

0

10

20

30

40

50

60

70

%

Better Same Worse

vs Peersvs Instructor

Prerequisites for Using Technology

Strongly Agree Agree Disagree

Strongly Disagree

I am at a disadvantage because I do not possess adequate keyboard/typing skills. (91.6)

2.6 5.9 50.6 41.0

I am at a disadvantage because I do not possess adequate computer skills. (91)

2.1 6.9 52.8 38.2

I am at a disadvantage because I do not have Internet access at home. (87.2)

3.8 9.0 52.5 34.7

How Often Students Log Into Course Web Site

22.9

35.0 34.9

4.7

2.4

0

5

10

15

20

25

30

35

%

<Once perwk

Once perwk

Fewtimes/ wk

Once perday

>Once perday

WWW Browser Used

14.7

83.6

1.6

0

10

20

30

40

50

60

70

80

90

%

Netscape InternetExplorer

Other

Personal Computer Owned by Students

72.3

22.6

9.0 10.9

0

10

20

30

40

50

60

70

80

%

Desktop Laptop Both Neither

Overall Usefulness of Web-Enhancements

Very Useful Useful

Somewhat

Useful

Not atAll NA

Online gradebook (82.1) 57.3 24.8 3.7 1.1 13.0

Access to course assignments (88.8)

54.9 33.9 6.6 .4 4.2

Announcements (82.3) 53.2 29.1 7.1 .8 9.7

Access to lecture notes/ PowerPoint (68.8)

42.7 26.1 7.0 1.5 22.7

Access to course readings (67.2)

37.3 29.9 8.5 2.6 21.8

E-mail links to students and instructors (57.6)

25.5 32.1 13.5 5.2 23.6

External course-related websites (51.5)

22.3 29.2 12.6 4.7 31.2

Online quizzes/tests 17.5 15.2 6.3 3.6 57.4

Discussion board 11.0 10.2 7.8 6.9 64.1

Active Learning

MoreLikely

About the

SameLess

Likely

Think about a similar course you have had that relied primarily on paper and pencil assignments and face-to-face lectures and discussions. Compared to that course, because of the way this course uses web-enhancements, how likely are you to:

Use the Internet as a source of course-related information.

81.763.7

15.618.2

2.718.2

Search for answers to your questions, rather than ask your instructor or other students.

68.336.4

27.027.3

4.736.4

Spend more time investigating and studying course concepts on your own.

57.760.0

35.610.0

6.730.0

Apply what you are learning to “real world” problems.

46.6 46.9 6.5

Refer to the course syllabus. 66.672.8

28.90

4.527.3Blue=faculty response

Student/Faculty Collaboration

MoreLikely

About the

SameLess

Likely

Think about a similar course you have had that relied primarily on paper and pencil assignments and face-to-face lectures and discussions. Compared to that course, because of the way this course uses web-enhancements, how likely are you to:

Discuss ideas and concepts taught in the course with your classmates.

36.5 50.9 12.6

Work on assignments with other students.

35.450.0

47.712.5

16.837.5

Communicate and interact with your instructor.

57.954.5

32.218.2

9.927.3

Time on Task

Strongly Agree Agree Disagree

Strongly Disagree

Think about a similar course you have had that relied primarily on paper and pencil assignments and face-to-face lectures and discussions. Compared to that course, because of the way this course uses web-enhancements, how strongly do you agree/ disagree with this statement:

I am learning to make study time more productive

8.6 55.9 31.8 3.7

I spend more time on course-related work.

8.7 47.7 41.4 2.1

I spend too much time trying to gain access to a computer.

3.4 12.3 61.4 22.9

I spend too much time “surfing” the web instead of studying.

6.4 28.8 47.7 17.1

Perceived Impact on Learning

Strongly Agree Agree Disagree

Strongly Disagree

Think about a similar course you have had that relied primarily on paper and pencil assignments and face-to-face lectures and discussions. Compared to that course, because of the way this course uses web-enhancements, how strongly do you agree/ disagree with this statement:

I have access to a greater variety of learning materials.

27.6 60.7 10.9 .7

I am better able to understand the ideas and concepts of this course.

15.08.3

63.183.3

20.20.0

1.78.3

I seem to always know what is going on in this course.

23.88.3

53.991.7

18.60.0

3.70.0

I am more likely to review lecture notes.

20.537.5

51.150.0

25.412.5

3.00.0

Other Student Perspectives

Strongly Agree Agree Disagree

Strongly Disagree

I would like to have web-enhancements used in other courses.

29.1 58.9 9.1 2.9

The course was suited to my learning style.

18.0 66.8 12.4 2.7

I put more thought into my comments on-line since my classmates will be reading them.

10.6 45.0 37.5 6.9

Online discussions encourage more thinking about course concepts.

10.2 41.3 40.4 8.1

There was not enough social interaction.

5.7 25.3 56.1 12.9

Faculty Demographics

Number of faculty: 12 Department:

• Applied Health Science: 5• Kinesiology: 3• Recreation & Park Adm: 4

Academic rank:• Professor: 1• Associate Professor: 5• Assistant Professor: 6

Tenure status:• Pre-tenure: 4• Tenured: 5• Non-tenure track: 3

Faculty Demographics

Average number of years teaching: 12.5 years Own a personal computer: 100.0% Internet access at home: 100.0% Update course web site: 66.6% at least once per/wk

Faculty Computer Skills

Faculty member skills compared to colleagues and students

66.7

50

33.3

50

0

10

20

30

40

50

60

70

%

Same Better

ColleaguesStudents

Time Spent Using Web-Enhancements

58.3

8.3

33.3

0

10

20

30

40

50

60

%

Higher Lower About theSame

+3.08 hrs/wk

+3.08 hrs/wk

The Seven Principles Revisited....

16.7

41.7

33.3

8.3

0

5

10

15

20

25

30

35

40

45

%

Much MoreLikely

SomewhatMore Likely

About theSame

Much LessLikely

Do Web-Enhancements in Your Course Encourage Student - Faculty Contact?

Faculty Perspectives

The Seven Principles Revisited....

16.7

33.3

41.7

8.3

0

5

10

15

20

25

30

35

40

45

%

Much MoreLikely

SomewhatMore Likely

About theSame

Much LessLikely

Do Web-Enhancements in Your Course Encourage Collaboration Among Students?

Faculty Perspectives

The Seven Principles Revisited....

16.7

50.0

25.0

8.3

0

5

10

15

20

25

30

35

40

45

50

%

Much MoreLikely

SomewhatMore Likely

About theSame

Much LessLikely

Do Web-Enhancements in Your Course Encourage Active Learning?

Faculty Perspectives

The Seven Principles Revisited....

33.3

41.7

16.7

8.3

0

5

10

15

20

25

30

35

40

45

%

Much MoreLikely

SomewhatMore Likely

About theSame

Much LessLikely

Do Web-Enhancements in Your Course Encourage Prompt Feedback?

Faculty Perspectives

The Seven Principles Revisited....

25.0

16.7

41.7

16.7

0

5

10

15

20

25

30

35

40

45

%

Much MoreLikely

SomewhatMore Likely

About theSame

Much LessLikely

Do Web-Enhancements in Your Course Emphasize Time on Task?

Faculty Perspectives

The Seven Principles Revisited....

25.0

50.0

16.7

8.3

0

5

10

15

20

25

30

35

40

45

50

%

Much MoreLikely

SomewhatMore Likely

About theSame

Much LessLikely

Do Web-Enhancements in Your Course Communicate High Expectations?

Faculty Perspectives

The Seven Principles Revisited....

8.3

50.0

33.3

8.3

0

5

10

15

20

25

30

35

40

45

50

%

Much MoreLikely

SomewhatMore Likely

About theSame

Much LessLikely

Do Web-Enhancements in Your Course Respect Diverse Talents and Ways of

Learning?

Faculty Perspectives

Faculty Perspective...

Greatest barrier(s) to you as an instructor using web enhancements?

Time involved in maintaining course material on the Web (58.3%)

Time involved in responding to student email (50.0%)

Time involved in redesigning teaching approaches (41.7%)

Preference for face-to-face student-instructor interactions (41.7%)

Student’s inexperience with computers and FTP (40.1%)

Faculty Perspective...

Greatest benefit(s) to you as an instructor using web enhancements?

Keeping the course organized (91.7%)

Ability to keep students informed about the course (83.3%)

Convenience to using the Web (83.3%)

Access to enriched teaching materials (75.0%)

Time – responding to email

Faculty Perspective...

Did Your Approach to Teaching or Your Teaching Practices Change as a Result of the

Way You Used Web-Enhancements?

• “Yes, more prep time involved but worth the time for student learning and communication.”

• “Yes. I’m much more aware of how students receive and access information.”

• “Yes. My expectations of student familiarity with the course material is higher. Increase in the time that I spend in class on key concepts as opposed to descriptive material.”

• “Web-enhancements allow a variety of teaching/learning styles to be successful.”

• “Allows students to be proactive.”• “Better able to track student progress from anywhere.”• “Email from students can be overwhelming!”• “Made assumption that students had web notes/overheads.”• “I think my lectures are more boring to students since they have

access to the notes....I think it makes me more dependent on the technology.”

Faculty Perspective...

How Did Your Use of Web-Enhancements Affect the Students’ Learning Experience?

• “Able to communicate to all students outside of class time.”• “My course content includes a large amount of terminology

which is a barrier to learning the concepts. The web-enhancements that I implemented tends to increase the exposure of the student to familiarity and terminology so the key concepts are easier taught.”

• “Web-enhancements have affected student learning by improving students’ abilities to organize course information through on-line lecture notes.”

• “By having lecture outline/materials available to students before class reduces copying time and increases listening time in class.

• “Better able to focus on discussion rather than taking notes.”

• “Less face-to-face interaction with students.”• “Student access to lecture notes seem to negatively impact

attendance....don’t seem to pay attention.”• “Students appear to be neglecting non-web based educational

materials…..anything not quick!”

Positive

Negative

Conclusions

Student computer skills• adequate keyboard/typing skills• adequate computer skills

Student computer access• Adequate Internet access at home• Desktop/laptop ownership• Computer cluster support

Preferred web browser• Internet Explorer

Majority of students were satisfied with the web-enhancements used in their course

Conclusions

Student perspective

Most Useful

???

Online quizzes,

discussion, chat room

Notes, course readings, external

links to course related content

Communication in terms of online gradebook, assignments, and

announcements

Conclusions

Faculty perspective:

• While still positive, not as strongly supportive of learning outcomes as students

• Enhanced several of the Seven Principles. However, more study needed on:

• Collaboration among students• Time on task• Diverse talents and ways of learning

• Also experienced the value of better communication with the students

What Did We Learn/Gain?

Hands-on experience in using the Flashlight tool Positive results regarding the use of web-enhanced

course delivery methods Useful findings to share with other faculty and

instructional support staff to improve practices and outcomes

Ideas for future studies• Part-time faculty• Non-web users• Use of Flashlight online survey instrument• Graduate education• Cost analysis• Other uses of technology (ie Oncourse, PowerPoint, etc.)

Development of a website for faculty to improve online teaching (See Virginia Tech)

“Effective Uses of Online Course Tools”Virginia Tech University

Discussion, Comments, ....

Thank You