1
The Use of Standards for Peer Review of Online Courses
Barbara Battin Little, DNP, MPH, [email protected]
2
Background & Review of Literature
• Rapid expansion of online education – Concerns about quality
• Standards established – Programs - accrediting organizations– Courses - universities
• Peer review process– Faculty & instructional designers
3
Purpose
Pilot the use of two sets of online course standards for
peer review of web-based nursing courses
4
Pilot Design• Sample – two RN to BS courses
• Instruments – 2 sets of online course standards– USF College of Public Health – Quality Matters (QM)
• Peer review process– individually by nurse educator & instructional
designer– jointly finalized scores
5
College of Public Health Online Course Standards
• 53 standards in 3 domains:– Communication: Instructor to
Student
– Course Delivery, Organization and Design
– Instructional Elements
• Scoring: – Met– Exceeded – Comments
6
Quality Matters Peer Course Review Rubric
40 standards in 8 categories:1. course overview and introduction2. learning objectives3. assessment and measurement4. resources and materials5. learner interaction6. course technology 7. learner support 8. ADA compliance
Scoring: – Pre-assigned numerical value based on importance
• 1, 2, or 3 points (3 = most important, required)– Written comments– 85% required for QM recognition
85 %
7
Results
1. Peer Review of Course – Total % of standards met on each tool– % of each individual standard met
2. Usefulness of Standards – Strengths and areas for improvement of the
tools
8
Percent of COPH Standards Met
57
71 69
100
70
43
65
77
100
70
0
20
40
60
80
100
120
1 Communication:Instructor to Student
2 Course delivery,organization and
design
3 Instructionalelements
4 Course Evaluation Total
Educational Transitions
Leadership & Management
9
Total Quality Matters Points
4249
68
80
0
10
20
30
40
50
60
70
80
90
EducationalTransitions
Leadership &Management
Points Requiredfor QM
Recognition
Possible Points
(85%)
10
Percent of Each QM Standard Met
020406080
100120
Educational Transitions Leadership and Management
11
Strengths of Standards
COPH Online Course Standards
Quality Matters
•measured important quality issues
•included grading criteria in standard 1.2d
•measured important quality issues
•easier to use and more consistency among reviewers •references•accreditation standards•broader use
12
Discussion
• Courses did not meet standards– 50% do not meet on first review– standards not in place– Varying levels of faculty expertise in online
education
13
Recommendations
Adoption of Course Standards
Peer Review Processes
Faculty Development
14
Adopt QM standards to guide course development
– assess course quality in key areas
– provide feedback to faculty course developer
– provide guidance to instructional design support team
Adoption of Course Standards
15
• Develop peer review process– Faculty driven process– Focus on improving quality
Peer Review Processes
16
• Provide training for faculty– Standards
– Peer review process
– Model best practices for online education
– QM reviewer and train-the-trainer programs
Faculty Development
17
Selected References
Quality Matters (2005). Peer course review rubric FY05/06. Retrieved April 1, 2007 from www.qualitymatters.org.
University of South Florida (2006). COPH online course standards. Retrieved May 27, 2008, from http://itt.usf.edu/publichealth/standards/standards/std_012606/COPH_Online_Course_Standards_v5.doc, http://itt.usf.edu/publichealth/standards/standards/std_012606/1.0_Communicaton.htm, http://itt.usf.edu/publichealth/standards/standards/std_012606/2.0_Course_Organization.htm, http://itt.usf.edu/publichealth/standards/standards/std_012606/3.0_Instructional_Elements.htm