1

The Use of Standards for Peer Review of Online Courses

Barbara Battin Little, DNP, MPH, RNbblittle@ufl.edu

2

Background & Review of Literature

• Rapid expansion of online education – Concerns about quality

• Standards established – Programs - accrediting organizations– Courses - universities

• Peer review process– Faculty & instructional designers

3

Purpose

Pilot the use of two sets of online course standards for

peer review of web-based nursing courses

4

Pilot Design• Sample – two RN to BS courses

• Instruments – 2 sets of online course standards– USF College of Public Health – Quality Matters (QM)

• Peer review process– individually by nurse educator & instructional

designer– jointly finalized scores

5

College of Public Health Online Course Standards

• 53 standards in 3 domains:– Communication: Instructor to

Student

– Course Delivery, Organization and Design

– Instructional Elements

• Scoring: – Met– Exceeded – Comments

6

Quality Matters Peer Course Review Rubric

40 standards in 8 categories:1. course overview and introduction2. learning objectives3. assessment and measurement4. resources and materials5. learner interaction6. course technology 7. learner support 8. ADA compliance

Scoring: – Pre-assigned numerical value based on importance

• 1, 2, or 3 points (3 = most important, required)– Written comments– 85% required for QM recognition

85 %

7

Results

1. Peer Review of Course – Total % of standards met on each tool– % of each individual standard met

2. Usefulness of Standards – Strengths and areas for improvement of the

tools

8

Percent of COPH Standards Met

57

71 69

100

70

43

65

77

100

70

0

20

40

60

80

100

120

1 Communication:Instructor to Student

2 Course delivery,organization and

design

3 Instructionalelements

4 Course Evaluation Total

Educational Transitions

Leadership & Management

9

Total Quality Matters Points

4249

68

80

0

10

20

30

40

50

60

70

80

90

EducationalTransitions

Leadership &Management

Points Requiredfor QM

Recognition

Possible Points

(85%)

10

Percent of Each QM Standard Met

020406080

100120

Educational Transitions Leadership and Management

11

Strengths of Standards

COPH Online Course Standards

Quality Matters

•measured important quality issues

•included grading criteria in standard 1.2d

•measured important quality issues

•easier to use and more consistency among reviewers •references•accreditation standards•broader use

12

Discussion

• Courses did not meet standards– 50% do not meet on first review– standards not in place– Varying levels of faculty expertise in online

education

13

Recommendations

Adoption of Course Standards

Peer Review Processes

Faculty Development

14

Adopt QM standards to guide course development

– assess course quality in key areas

– provide feedback to faculty course developer

– provide guidance to instructional design support team

Adoption of Course Standards

15

• Develop peer review process– Faculty driven process– Focus on improving quality

Peer Review Processes

16

• Provide training for faculty– Standards

– Peer review process

– Model best practices for online education

– QM reviewer and train-the-trainer programs

Faculty Development

17

Selected References

Quality Matters (2005). Peer course review rubric FY05/06. Retrieved April 1, 2007 from www.qualitymatters.org.

University of South Florida (2006). COPH online course standards. Retrieved May 27, 2008, from http://itt.usf.edu/publichealth/standards/standards/std_012606/COPH_Online_Course_Standards_v5.doc, http://itt.usf.edu/publichealth/standards/standards/std_012606/1.0_Communicaton.htm, http://itt.usf.edu/publichealth/standards/standards/std_012606/2.0_Course_Organization.htm, http://itt.usf.edu/publichealth/standards/standards/std_012606/3.0_Instructional_Elements.htm