The Quality Matters Review Process

Presentation by Michael Wilder

Introduction

The Quality Matters review process:• Involves nine major steps.• Requires communication between Faculty

Developer, Master Reviewer, Institutional Representative, and at least two Peers Reviewers.

• Involves a multitude of online and digital forms.

Overview

Steps in a QM Course Review1. Instructor worksheet submitted2. Reviewer team selected3. Pre-review

• Pre-review conference call

4. Course review5. Final report submitted

• Post-review conference call• Post-review survey

6. Communication with faculty developer as needed7. Course amendment form submitted8. Changes reviewed/approved9. Course recognized

Steps

Step 1: Instructor worksheet submitted

From the Course Review Management System (CRMS), the faculty developer initiates the process by submitting a Course Review Application.

Step 1: Instructor worksheet submittedAs part of the online application process, the faculty developer creates a “Course Worksheet” (also known as “Faculty Developer Worksheet”).This worksheet identifies such details as course title and course-level learning objectives.

Step 1: Instructor worksheet submittedThe QM Review Coordinator provides the Faculty Developer with instructions to complete the Faculty Developer Worksheet. Once the Faculty Developer Worksheet and the Course Review Tracking Spreadsheet have been successfully completed, the Review Coordinator makes the course available for review in the CRMS.

Step 2: Reviewer team selectedSuccessful submission of the Course Review Application also alerts the QM Review Coordinator who identifies the Master Reviewer. If the course to be reviewed is initiated by a subscriber institution, it is possible that the Master Reviewer has already been selected.

Step 2: Reviewer team selected

The Quality Matters Review Coordinator sends the Course Review Tracking Spreadsheet to the Master Reviewer when he or she confirms availability.

Step 2: Reviewer team selectedThe Master Reviewer uses the “Peer Reviewer Search Tool” (located at MyQM) to identify potential members of the review team per QM standards. Once reviewers confirm availability, the Master Reviewer informs the Review Coordinator via an updated Course Review Tracking Spreadsheet.

Step 2: Reviewer team selected• The Master Reviewer is responsible for choosing a

team on a QM-managed Review. • Subscriber-managed reviews generally have chosen

the team ahead of time. • In either case, the Master Reviewer will receive

access to the Instructor Worksheet, and should retrieve the forms from MyQM in the Master Reviewer area to set up and conduct the pre-review conference call (next step).

Step 2: Reviewer team selectedThe Master Reviewer lets the Institutional Representative and the faculty developer know who is on the review team and who is the subject matter expert. This information should be posted on the spreadsheet and sent to each team member.

Step 3: Pre-reviewUsing the “Email Introduction Template” as a guide, the Master Reviewer sends customized “Review Assignment Instructions” to reviewers, the faculty developer, and the Institutional Representative. The “Team Calendar,” “QM Principles,” “QM Roles,” and “Writing Recommendations” documents are included as attachments.

Step 3: Pre-reviewThe Faculty Developer (or Institutional Representative) opens the online course to the Peer Reviewers and provides login information to the Master Reviewer. Directions and login information to enter the course are sent to the Peer Review team by the Master Reviewer.

Step 3: Pre-reviewThe Master Reviewer develops the Team Calendar, setting timelines for the pre-review conference call, active review, post-review, and revision procedures. Generally the entire process takes about twenty weeks. The Master Reviewer sends a copy of the Team Calendar to the Peer Review team, the Faculty Developer, and the Institutional Representative.

Step 3: Pre-reviewUsing the “Pre Review Conference Call Agenda Template” as a guide, the Master Reviewer initiates contact with the Peer Review team, communicates instructions and responsibilities for the pre-review conference, shares contact information, and requests that the review team enter the course to ensure no immediate navigation difficulties.

Step 3: Pre-reviewPeer review team members review the Faculty Developer Worksheet prior to the Pre-Review Conference Call.

Step 3: Pre-reviewThe Master Reviewer convenes the pre-review conference. At this conference, the Master Reviewer describes the review process, discusses the Team Calendar, communicates time expectations, identifies LMS access concerns, elicits questions regarding the Faculty Developer Worksheet, explains change limitations, and discusses confidentiality.

4. Course reviewUsing the Quality Matters Rubric and the Course Review Management System, the Review Team individually provides feedback and recommendations on the Course Worksheet.

4. Course reviewThe Master Reviewer will have access to review course worksheets in progress. It is the Master Reviewer’s responsibility to communicate with the other reviewers if recommendations are not meeting the criteria for helpful feedback. Recommendations should be constructive, specific, measurable, sensitive and balanced. Master Reviewers should coach peer reviewers if changes need to be made.

5. Final report submissionOnce Peer Reviewers complete all recommendations, but before the final report is submitted, the Master Reviewer conducts a post-review discussion. The Peer Review team discusses differences in scoring and any remaining questions for the Faculty Developer.

5. Final report submissionThe Master Reviewer schedules a "Post-Review" Conference Call so the reviewers can discuss "essential” standards that did not meet expectations. The MR will encourage the reviewers to be sure each recommendation is clear and detailed enough that the faculty developer/instructor knows what needs to be done and encourage team members to include positive comments. Team members can still change their decision (met or not met) at this point on any of the standards.

5. Final report submissionIndividual reviews are revised based on team discussion and finalized.Peer Reviewers submit their revised report in CRMS after editing for spelling and clarity. Reviewers may edit, save and resubmit up until the final report is submitted. The Master Reviewer submits the Final Report in the CRMS.

5. Final report submissionOnce the Master Reviewer submits the final Reviewer Report Form, the CRMS automatically notifies the Faculty Developer, IR and QM review coordinator with the review outcome and provides directions to the Faculty Developer for completing the online Faculty Response Form.If QM standards are met, the process moves to Step 6.

5. Final report submissionIf QM standards are not yet met, the Faculty Developer receives the outcome email instructing them to view the report and submit the online Faculty Response Form within two weeks of the final report submission indicating if changes will be made to the course. The submission of the Faculty Response Form will generate a Course Amendment Form in the CRMS. The CRMS will determine the amendment due date depending upon the review start date. The re-review needs to be completed within the 20-week window from review start date unless the Institutional Representative petitions Quality Matters for an extension to the amendment date.

5. Final report submissionThe Faculty Developer completes the online Course Amendment Form to record the changes he or she made in the course in order for it to now meet expectations. Upon submission, the Master Reviewer will be automatically notified by email that the amendments are complete.

5. Final report submissionThe CRMS provides directions to the Faculty Developer for completing the online Faculty Response Form. All reviewers, the faculty developer, and the institution representative are asked to take a post-review survey. The CRMS will automatically send a link to this survey once the final report is submitted.If the course has met QM standards, the process moves to Step 9.

6. Communication with faculty developer as neededThe Institution Representative follows up with the Faculty Developer to provide instructional design resources and to ensure that the amendment deadline is met.

7. Course amendment form submittedOnce the amendments to the course are completed, the Faculty Developer submits the online Amendment Form in the CRMS.

8. Changes reviewed/approvedThe Master Reviewer reviews the changes in the course and approves the online Amendment Form if the course now meets expectations. Once amendments are approved and submitted, the CRMS will notify the Faculty Developer, IR and QM review Coordinator that the course now meets standards (if applicable).

The Master Reviewer sends the completed Course Review Tracking Spreadsheet to the QM Review Coordinator at the close of the review.

8. Changes reviewed/approvedIf the course does not meet QM standards after amendment, the Master Reviewer does not approve the online Amendment Form.

The CRMS automatically notifies the Faculty Developer, IR and QM review Coordinator that the course does not meets standards.

The Master Reviewer sends the completed Course Review Tracking Spreadsheet to the QM Review Coordinator at the close of the review.

Step 9: Course recognizedOnce the course meets Quality Masters Standards, it is formally recognized in the QM CRMS.

Questions?

This presentation made by:

Michael WilderInstructional Design Coordinator

E-mail | Web Site