A Look at Online Teaching and Learning at UCF

Charles D. Dziuban

Patsy D. Moskal

University of Central Florida

Levels of faculty development

IDL 6543Mixed-mode course to design, develop, & deliver

M or W course

ADL 5000Online course modeling delivering existing M or

W course

EssentialsOnline course for faculty who want to supplement

their F2F course without reducing any class time.

Growth of UCF online learning environments

6461538

3096

5668

8710

12443

6026

951310558

12070

17640

02000400060008000

10000

1200014000160001800020000

AY9697 AY9798 AY9899 AY9900 AY0001 AY0102

Enr

ollm

ent

Academic Year – Summer, Fall, Spring

M+E courses

W courses

The Evaluation

Principles that guide our evaluation• Evaluation must be objective.

• Evaluation should conform to the culture of the institution.

• Uncollected data cannot be analyzed.

• Data do not equal information.

• Qualitative and quantitative approaches must complement each other.

• We must show an institutional impact.

• Our results may not be generalized beyond UCF.

Distributed Learning Impact Evaluation components

Reactive behaviorpatterns

Success ratesAttitudes

Demographicinertia

Withdrawal rates

Strategies forsuccess

Students

Online programs

Real-time surveysWriting project model

Large online classes

Faculty

Modified instructionaltheories

Student evaluation ofinstruction

Student Results

What we have found regarding online students

• The majority of students enrolled in fully online (W) courses are also enrolled in face-to-face courses,

• The distribution of students by ethnicity is approximately the same for all modalities,

• Fully online (W) courses consistently have more females than other modalities,

• On the average, students enrolled in W courses are oldest, followed by those in M sections then face-to-face.

Student satisfaction in fully online and mixed-mode courses

0

5

10

15

20

25

30

35

40

45

50

39% Fully online (N = 1,526)Mixed-mode (N = 485)

41%

11% 9%

Very SatisfiedUnsatisfiedSatisfied

Neutral

38%

44%

9%

Very Unsatisfied

3%5%

1%

Success rates by modalitySpring 01 through Spring 03

9193

91 90

94

91 9191

97

94

91

97

92 9189

93

9092 92 92 91

75

80

85

90

95

100

Spring 01

Summer01

Fall 01

Spring 02

Summer02

Fall 02

Spring 03

F2FMTotal N= 139,444 studentsW

Per

cent

A segment model depicting success in course by college, gender, and modality

93.0%n=31,124

94.3%n=20,858

90.5%n=10,266

94.5%n=62,403

93.1%n=9,226

97.4%n=2,989

89.8%n=9,024

95.7%N=1,242

Females Males

F2F W M F2F,W M

College of Health & Public Affairs

Withdrawal rates by modalitySpring 01 through Spring 03

6 3 4 5 3 3 56 2 5 5 2 6 510 6 8 8 6 6 7

0102030405060708090

100

Spring 01

Summer01

Fall 01

Spring 02

Summer02

Fall 02

Spring 03

F2FMTotal N= 147,194 studentsW

Per

cent

Student Behavior Types

Research on reactive behavior patterns

• Theory of William A. Long, University of Mississippi

• Ambivalence brings out behavior patterns

• Provides a lens for how “types” react to different teaching styles

Resources

• Personality

• Emotional maturity

• Sophistication level

• Level of intellect

• Educational level

• Character development

A description of Long behavior types

• Aggressive Independent• high energy• action-oriented• not concerned with approval• speaks out freely• gets into confrontational

situations• Passive Independent

• low energy• not concerned with approval• prefers to work alone• resists pressure from authority

• Aggressive Dependent

• high energy

• action-oriented

• concerned with approval

• rarely expresses negative feelings

• performs at or above ability

• Passive Dependent

• low energy

• concerned with approval

• highly sensitive to the feelings of others

• very compliant

A description of Long behavior traits

• Phobic

• exaggerated fears of things

• often feels anxious

• often sees the negative side

• doesn’t take risks

• Compulsive

• highly organized

• neat, methodical worker

• perfectionist

• strongly motivated to finish tasks

• Impulsive

• explosive

• quick-tempered

• acts without thinking

• frank

• short attention span

• Hysteric

• dramatic and emotional

• more social than academic

• artistic or creative

• tends to overreact

Distribution of Long types and traits for fully online students

AI21%

PI18%

AD54%

PD7%

51%

75%

26%

30%

(N=1,437) (N=1,520)

Distribution of Long types and traits for mixed-mode students

AI17%

PI23%

AD52%

PD8%

54%

76%

23%

32%

(N=472)

Distribution of Long types and traits for Composition I students

AI20%

PI23%

AD44%

PD14% 50%

53%

38%

40%

(N=1,054)

Long types and traits for Web, mixed-mode, and general education students

Web(N=1,533)

Mixed-mode(N=491)

Comp I(N=1,054)

Aggressive

Dependent

54% 52% 44%

Passive Dependent

7% 8% 14%

Compulsive 74% 76% 53%

Impulsive 26% 23% 38%

Typ

esT

rait

s

Faculty Results

A lot more time

Time to develop course as compared with a comparable face-to-face section

Morework

Equalto or

less than Wn=56

MN=43

Modality

A little more time

About the sameA little less timeA lot less time

2%

52%

21%

5%

77%

43%

2%

A lot more time

Time in weekly course administration activities as compared with a comparable face-to-face section

Morework

Equalto or

less than Wn=55

MN=42

Modality

A little more time

About the sameA little less timeA lot less time

4%

43%

15% 19%

60%

38%20%

30%

A lot more time

Time in weekly course delivery activities as compared with a comparable face-to-face section

Morework

Equalto or

less than

Wn=55

MN=42

Modality

A little more time

About the sameA little less timeA lot less time

20%

5%

28%

29%

9%

37%

13% 15%

15%

Amount of interaction in Web classes compared to comparable F2F sections

Moreinteraction

Equalto or

less thanW

n=55M

N=40

Modality

13%

45%

16%

15%

62%

30%

2%7%

8%3%

IncreasedSomewhatincreasedAbout the sameSomewhatdecreased

Decreased

Quality of interaction in Web classes compared to comparable F2F sections

Betterinteraction

Equalto or

less thanW

n=55M

N=43

Modality

22%

30%

33%

19%

35%

37%

9%2%

14%

IncreasedSomewhatincreasedAbout the sameSomewhatdecreased

Decreased

Very satisfied

Faculty satisfaction with their varying course modalities

Positive

Neutralor

negativeW

n=55M

N=43F2F

N=64

Modality

SatisfiedNeutralUnsatisfied

Very unsatisfied

6%

44%

44%

5%

58%

5%

49%

38%

38%

7%

7%

Faculty willingness to teach Web courses in the future

Positive

Neutralor

negativeW

n=71M

N=53

Modality

81%

16%

2%

69%

13%

10%6% 4%

DefinitelyProbably

Probably not

Definitely not

Relationships of faculty satisfaction with class interaction and workload (TAU-b)

W M(n=53) (n=38)

Amount of interaction .39** .34*

Quality of interaction .43** .51**

Time to develop .16 .09

Time to administer .10 .01

Time to deliver .06 .10

*p<.05; ** p<.01

Student Ratings

Student Ratings by Modality

VeryModality Excellent Good Good Fair Poor

F2F 42.00 29.50 19.00 7.20 2.40(N=628,623)

E 44.00 29.10 17.40 6.90 2.60(N=6,632)

M 40.60 28.60 20.60 7.70 2.40(N=11,450)

W 55.40 25.20 12.10 4.90 2.50(N=5,435)

Facilitation of learning

Communication of ideas

Excellent Very Good Good Fair Poor

Then...

The probability of an overall rating of Excellent = .93 &

The probability of an overall rating of Fair or Poor =.00

If...

A decision rule based on student evaluation responses and the probability of faculty receiving an overall rating of Excellent

A comparison of excellent ratings by college unadjusted and adjusted for instructors satisfying Rule 1

College Unadjusted % Adjusted %Arts & Sciences 41.6 92.4

Business 34.9 90.9Education 56.8 94.8

Engineering 36.2 91.3H&PA 46.1 93.9

(N=441,758) (N=147,544)

A comparison of excellent ratings by course modality--unadjusted and adjusted for instructors satisfying Rule 1

F2F 42.0 92.2E 44.0 92.3M 40.6 92.0W 55.4 92.7ITV 20.9 86.7

CourseModality Unadjusted % Adjusted %

N=709,285 N=235,745

Research Initiative for Teaching Effectiveness

The transition from online to face-to-face courses

DistributedLearningImpact

Evaluation

ResearchInitiative

forTeaching

Effectiveness

Research Initiative for Teaching Effectiveness

For more information contact:

Dr. Chuck Dziuban(407) 823-5478

dziuban@mail.ucf.edu

Dr. Patsy Moskal(407) 823-0283

pdmoskal@mail.ucf.edu

http://pegasus.cc.ucf.edu/~rite