RUBRIC FOR IMPROVING THE QUALITY OF ONLINE COURSES

Denise Nelson, Virtual Campus

2

OUTCOMES

Share strategies used to measure the quality of online courses.

Discuss the impact of a quality assurance system on institutional operations and stakeholders.

3

PLAN

SIAST Context & background Rubrics...rubrics...rubrics...

Adaptation Use

Discuss quality assurance processes & results

Future plans?

4

VIRTUAL CAMPUS REVIEW

Development and Delivery on Online Courses

5

VIRTUAL CAMPUS REVIEW

External review of online courses Stakeholder focus groups Email interviews Web survey mySIAST – Intranet call for feedback

6

RUBRICS...

Used existing rubrics as guides to develop online courses CNIE Criteria for

Award of Excellence in Instructional Design BlackBoard Exemplary Course Rubric Blood-Siegfried et al.

A Rubric for Improving the Quality of Online Courses

Blood-Siegfried et al. adapted to SIAST Context

7

ANALYSIS OF EXISTING RUBRICS RELEVANT FOR ONLINE COURSE

1. California State University, Chico - What does a high quality online course look like?http://www.csuchico.edu/tlp/resources/rubric/rubric.pdf (Committee for Online Instruction, 2003)

2. Towson University, Maryland - Towson Faculty Peer Review of Online Courseshttp://www.towson.edu/~mcmahon/peerreview/On-linerubric.pdf (Ashcraft, McMahon, Lesh, & Tabrizi, 2003)

3. University of West Georgia - Assessing Interactive Qualities of Online Courseshttp://www.westga.edu/~distance/roblyer32.html (M D Roblyer & Ekhaml, 2000; M. D. Roblyer & Wiencke, 2003)

4. Grant McEwan College - Criteria for Evaluating the Quality of Online Courses (Wright, 2004)http://elearning.typepad.com/thelearnedman/ID/evaluatingcourses.pdf777

5. University of Wisconsinhttp://www.uwex.edu/disted/conference/Resource_library/handouts/Forum1.pdf (Swan, Bishop, Wisher, & Trollip, 2003)

6. Robert Wood Johnson Partnerships for Traininghttp://pftweb.org/reach/reach_home_frameset.htm (Partnership for training, 2003)

7. Measuring Success: Evaluation Strategies for Distance Educationhttp://www.educause.edu/ir/library/pdf/eqm0213.pdf (Lockee, Moore, & Burton, 2002)

8

BLOOD-SIEGFRIED ET AL. RUBRIC CRITERIA

1. Organization and Design 2. Course Content3. Instruction4. Interaction5. Evaluation and Assessment6. Student Support Services - added

9

USE OF RUBRIC FOR EXTERNAL COURSE REVIEW

Adapted rubric Obtained course context information

nature of target audience design specifications student/instructor interactions pace development and delivery schedule

Faculty team available to respond to inquiries Provided assessment items and related data

not housed with course (e.g., examinations)

10

DISTANCE COLLABORATION

11

PROCESS AND DELIVERABLES OF THE REVIEW

Reviewer received documentation & access Context and rubric documentation Access to Master version of online course Access to archived online course (demonstrates

interaction, grades) Reviewer conducted review within ~2

weeks Reviewer emailed completed review

documentation Reviewer and stakeholders held debriefing

mtg

12

EXTERNAL REVIEWER COMMENTS ABOUT USE OF RUBRIC

Lori AtkinsonInstructional Designer

13

CHOICES...

14

RATE THE USEFULNESS OF THE RUBRIC FOR COURSE EVALUATION?

Highly useful The criterion creates a checklist that will

demonstrate whether the online course provides the best shot at a quality student experience.

Overall the 7-13 objectives within each criteria captured the major areas of best practice for online learning.

I would recommend that the rubric is used for the basis of a course evaluation or course development tool. (Lori Atkinson, 2009)

15

RATE THE USEFULNESS OF THE RUBRIC TO INFORM ONLINE COURSE DEVELOPMENT?

Highly useful for instructor mediated courses

This rubric should be adapted for self-paced and blended courses to reflect different student to content (and media), student to student, and student to instructor interaction strategies. (Lori Atkinson, 2009)

16

WHAT ARE THE STRENGTHS OF THE RUBRIC?

Well grounded in research, reflected in structure and content (Lori Atkinson, 2009)

17

WHAT ARE THE LIMITATIONS OF THE RUBRIC?

Requires an additional document to synthesize major themes for improvement(Lori Atkinson, 2009)

18

ARE THERE OTHER CRITERIA/RUBRICS THAT SHOULD BE INCORPORATED?

A section that provides evaluation of the tie between the outcomes, instruction and assessment. This would help to demonstrate areas of the

course that are content heavy and not really related to the course outcomes.

As a development tool this would be helpful to assess what technology intervention is really a match for the learning outcome. (Lori Atkinson, 2009)

Learning Outcome

Learning Activities

Assessments

Write resume Practice writing resume

Write resumeCritique resume

19

LORI ATKINSON, EXTERNAL REVIEWER

20

RESULTS OF USING RUBRIC

Identify professional development needs Instructional design process Alignment of outcomes, learning

activities, and assessment Evidence for resource allocation Identify important questions (e.g.,

Should this course be instructor led, self-paced, blended?)

Improve learning experience Review debrief facilitates connection

with other SIAST teams and examples of high quality SIAST courses

21

DISCUSSION AND FUTURE PLANS?

Professional development to increase awareness of benefits of using rubric to guide development and evaluation

Dialogue re improvements needed in course development processes (e.g., dealing with heavy course content; alignment)

...

22

CONTACT

Denise NelsonInstructional DesignerSIAST Virtual Campus

Phone: 306-775-7683Email: denise.nelson@siast.sk.caPPT available at

http://programs.siast.sk.ca/nelson/#presentations