06.1AANNUAL PROGRAMME REVIEW TEMPLATE, ACTION PLAN AND

GUIDANCE NOTES

Academic Handbook 2017/18 – Volume 2 - 06.1A – Annual Programme Review Template, Action Plan and Guidance Notes – Introduced 28.10.14, modified 14.05.15, 04.08.15, 11.10.17

Cardiff Metropolitan UniversityAnnual Programme Review

Programme(s) Active in the Academic Year 2016-2017

Cardiff School of

Title of Programme(s) and any pathways (please include previous programme title if this is still being taught out):

Name of Programme Director(s):

Link to Supporting Documents:

Summary of Measures of SuccessRecruitment Retention Award

outcomesCareer

DestinationsStudent Opinion

External Examiner

Red

Amber

Green

Summary of teaching quality (TEF) (home undergraduate programmes only)

NSS Teaching NSS Assessment and Feedback

Red

Amber

Green

Date Considered by Programme Committee

Date Considered by Learning & Teaching Committee

Dated Signature of Programme Director:

Dated Signature of Student Representative:

Section 1 – Previous Action plan

Number Issue Source Evaluation and Impact(indicate evidence base)

16.01Resolved?YES/NO16.02Resolved?YES/NO16.03Resolved?YES/NO16.04Resolved?YES/NO

Section 2 – Reflection on Last Year’s Action Plan

If your programme has not been through a Periodic Review or Validation, go to Section 4

Section 3 – Response to Recommendations from Periodic Review/Validation

If your programme has not made any modifications in the last academic year, go to Section 5

Section 4 – Modifications last academic year

If your programme does not have a PSRB, go to section 6

Section 5 - PSRB

Section 6

Data Population (by School Administrator)

  6.1. Recruitment

  6.2. Retention    6.3. Award outcomes  

 6.4. Career destination

6.5. Student Opinion

 

Section 7

Data Reflection & Commentary (1 Page max) (by Programme Director)

Recruitment -

Retention -

Award Outcomes -

Career Destination –

Student Opinion -

Strengths

Weaknesses

Opportunities

Threats

Section 8 – SWOT Analysis

Section 9

Number

KPIIssue Source Actions taken/planned By Whom Timescale Measurement of success

APR Guidance DocumentContentsInitial Section...................................................................................................................................................................2

Initial information........................................................................................................................................................2

Link to Supporting Documents, i.e. Supporting Evidence............................................................................................2

Summary of Measures of Success...............................................................................................................................3

Undergraduate Measures of Success......................................................................................................................3

Postgraduate Measures of Success.........................................................................................................................3

Signatures and Dates...................................................................................................................................................4

Section 1: Previous Action Plan and Evaluation and Impact of last year’s actions..........................................................4

Section 2: Reflection on last year’s action plan...............................................................................................................4

Section 3: Response to Outcomes from Periodic/Elective Review..................................................................................4

Section 4: Reflection on Modifications to Programme....................................................................................................4

Section 5: Reflection on PSRB activity.............................................................................................................................5

Section 6: Data Population..............................................................................................................................................5

Section 7: Data Reflection & Commentary......................................................................................................................5

Section 8: SWOT analysis................................................................................................................................................5

Section 9: Action Plan Going Forward.............................................................................................................................5

MIS - Origins of the APR data.........................................................................................................................................7

Key Performance Indicators........................................................................................................................................7

Recruitment.............................................................................................................................................................7

Student Progression................................................................................................................................................7

Award Outcomes.....................................................................................................................................................7

Career Destinations.................................................................................................................................................7

Student Opinion......................................................................................................................................................7

Survey Data.................................................................................................................................................................7

UKES........................................................................................................................................................................7

SSS...........................................................................................................................................................................7

PTES.........................................................................................................................................................................7

Remaining data sheets................................................................................................................................................7

Criteria for Rating External Examiner Reports.................................................................................................................8

Green..........................................................................................................................................................................8

Amber..........................................................................................................................................................................8

Red..............................................................................................................................................................................8

Initial SectionInitial informationDesignated School administrative colleagues complete the name of the School, the overarching title of the programme and the name of the programme director. If the programme contains pathways, then these should be listed in a separate WORD document and included as part of your evidence (02_ClusterList – see supporting evidence below).

Link to Supporting Documents, i.e. Supporting EvidenceAdministrative colleagues set up the folder to host the evidence for the APR and ensure that the Programme Director (PD) and team and ASQU have permissions to access the folder. Administrative colleagues to include on the APR template the link to the OneDrive/SharePoint site. Administrative colleagues to populate the shared folder with the evidence documents using the following naming conventions:

Please note for ALL file names there should be no spaces and the first letter of each word should be capitalised for ease of reading.

Use the prefix of 01_CompletedAPR for the completed template for the programme once signed off by both PD and student representative.

Use the prefix of 02_ClusterList for the file containing the list of programmes/pathways being included in the APR. Use the prefix of 03_ExternalExaminerReport for each external examiner report and add the external examiner’s

initials at the end of the file name. For example, if you have external examiners Joe Bloggs and Ann Other, you should have 2 reports named:

o 03_ExternalExaminerReportJBo 03_ExternalExaminerReportAO.

Use the title 04_TeamResponseToExternalExaminerReport for the team’s response to the external examiner. Use the prefix of 05_ModuleEvaluationSummary for each module evaluation and add the module code at the end of

the file name. For example, if you are including modules with codes MAE7001 and MAE7035, you should have 2 reports named:

o 05_ModuleEvaluationSummaryMAE7001o 05_ModuleEvaluationSummaryMAE7035.

Use the prefix of 06_ProgrammeEvaluationSummary for the summary of the programme evaluation and add the programme code at the end of the file name. If there are several pathways included in the APR, you may have separate evaluations. Append the pathway/programme codes to the end of the file name. For example, if you have two pathways, coded 10359 and 10771, you should have 2 reports named:

o 06_ProgrammeEvaluationSummary10359o 06_ProgrammeEvaluationSummary10771

If you are including other programme type surveys such as NSS and PTES in your evidence body, add them to this section with the 06_ prefix, e.g.:

06_NSSSummary 06_PTESSummary.

Use the prefix of 07_ProgrammeCommitteeMinutes for each set of programme committee minutes you are including. Append the dates of the meetings to the end of the file names. For example, if you had 3 meetings, dated 181016, 250317 and 240517, you should have 3 sets of minutes named:

o 07_ProgrammeCommitteeMinutes181016o 07_ProgrammeCommitteeMinutes250317o 07_ProgrammeCommitteeMinutes240517.

If you have undergone an elective or period review, please include the response from the review panel. Use the filename:

o 08_ProgrammeReviewDocumentation. If your programme is professionally endorsed and you have a report from your PSRB, use the filename:

o 09_PSRBDocumentation. Each School should have a record of all external staff development undertaken, along with the staff member’s

evaluation of the training. Members of academic staff should send details to the School Administrator of all internal staff development undertaken during the previous academic year. The details should be brief and recorded under the headings: Surname, Forename, Activity, Delivered by and Evaluation. The School Administrator should report the programme team’s staff development in a file named:

o 10_SummaryOfStaffDevelopment. If the programme team has undertaken any external engagement, for example, with employers, a summary of the

activity should be recorded as: o 11_ExternalEngagement.

If there are additional files deemed appropriate to add as evidence, then the numbering and naming conventions should be followed. For example, 12_AdditionalEvidence.Your final folder in the SharePoint site should look something like:

Summary of Measures of SuccessThis section will be completed by administrative colleagues and PD. Several of the Measures of Success will have a Red or Green rating available in QlikView. This will differ for undergraduate and postgraduate programmes, so the guidance is a little different for each. Management Information Systems (MIS) colleagues have provided information on the QlikView application data and this is available at the end of this document. Loosely, most of the ratings are based on the 16/17 Measures of Success stipulated in the new Corporate Strategy. For Undergraduate programmes, these Measures of Success are: Retention (based on the HESA Performance Indicator tracking students from Year 1 to Year 2), Award Outcomes (% Good Honours), Career Destinations (DHLE data: Graduate Employment or Further Study), Student Opinion (NSS: Overall Satisfaction).

The rating for the external examiners’ reports, for both UG and PG programmes, will be provided by ASQU who will have reviewed the External Examiners’ reports and given them a Red/Amber/Green rating. (Details of the criteria for this are available at the end of this document).

Undergraduate Measures of Success Administrative colleagues will be able to populate the Recruitment, Retention (i.e. Student Progression), Award outcomes, Career Destinations and Student Opinion Measures of Success. Administrative colleagues should place an X in the Red/Amber/Green area to reflect the information available in QlikView relating to the below categories:

Green: Meets 2022/23 Measure of Success Amber: Meets 2016/17 Measure of Success but not 2022/23 Red: Fails to meet 2016/17 Measure of SuccessFor example, with the following information available in QlikView for Student Progression (Retention), the administrative colleague would RAG code Retention Green:

With respect to the TEF metrics, PDs should select the TEF Metrics tab in QlikView and make a judgement based on the graphical information available for NSS Teaching and NSS Assessment and Feedback. They should then place an X in the Red/Amber/Green area to reflect their judgement.

Postgraduate Measures of SuccessFor postgraduate programmes the only measure of success RAG rated in QlikView is Career Destinations (DHLE data: Graduate Employment or Further Study). Administrative colleagues should place an X in the Red/Amber/Green area to reflect the information available in QlikView relating to this category.

To remain consistent with last year, the % of Distinctions and Merits is also presented. With the exception of the External Examiner category (where ASQU will provide the RAG rating), PDs should RAG the remaining measures of success based on their judgement about the information presented in QlikView and the evidence gathered in support of the APR and available in the SharePoint site.

Signatures and DatesAdministrative colleagues to add the date that the APR was considered at Programme Committee and at Learning and Teaching Committee.Programme director needs to sign and date the form and ensure that there is student representation present at the programme committee meeting that considers the APR.

Section 1: Previous Action Plan and Evaluation and Impact of last year’s actionsThe evaluation of last year’s APR actions needs to be completed at this point. Administrative colleagues will add the numbers, issues and sources from the previous year’s APR.Number: Each issue should be numbered sequentially, prefaced by the year the action plan was written, in the format: 16.01; 16.02 etc. Administrative colleagues should put the numbers in place. Issue: A brief description of the issue should be provided, copied from the previous year’s APR.Source: The source(s) of the issue should be identified, e.g. external examiner feedback.The Evaluation and Impact of the previous year’s action plan should be completed by the PD.Evaluation and Impact: PDs need to indicate whether the issue has been resolved by deleting YES/NO as appropriate. PDs, please note the phrase ‘impact’ in the final column. Although impact on student experience has always been implicit in the evaluation of actions taken, it has been included to emphasise that it is a primary focus. Please also identify the evidence base, e.g. external examiner report, improved module evaluations, programme statistics.If, following PD’s evaluation, an issue has not been resolved effectively, it should be carried over to this year’s action plan. Administrative colleagues will carry it forward to the START of the new action plan in Section 9, ensuring that the number remains the same.

Section 2: Reflection on last year’s action planPDs need to reflect briefly on last year’s action plan, indicating any successes and noting why certain actions are being carried forward. Please try to limit the reflection to 250 words.

Section 3: Response to Outcomes from Periodic/Elective ReviewIf the programme underwent an elective/periodic review then the PD needs to reflect briefly on the event, in particular the recommendations from the panel. The panel’s final report should also be available in the OneDrive/SharePoint site discussed earlier. Please try to limit this to 500 words.

Section 4: Reflection on Modifications to ProgrammeIf the programme underwent modifications then the PD needs to note them and reflect briefly on them. Please try to limit this to 500 words.

Section 5: Reflection on PSRB activityIf the programme is professionally accredited and there have been events during the academic year, the PD needs to reflect briefly on these events. Any supporting PSRB documentation should also be available in the SharePoint site discussed earlier. Please try to limit this to 500 words.

Section 6: Data PopulationThis section is populated by administrative staff using the data available in QlikView. This year, the data will all be available in Section 6 of the APR and the PD will comment in Section 7. Next year, we will revisit this format as some colleagues have

queried keeping data presentation in a separate section to reflection. Where programmes are clustered together, the clustered selection should be made to create the graphs; otherwise, the graphs for a programme with several pathways will require several pages of the APR. However, if there are particular weaknesses in individual pathways, this should be highlighted.The QlikView APR application is available and accessible to all PDs and designated School administrative colleagues. Those who do not have access to the data can email APR@cardiffmet.ac.uk, copied to their DDLT, to request access. The application has undergone redevelopment since last year in order to include further data sets and simplify the aesthetics. Greater emphasis has been put on trends over academic years and comparisons against school and university populations.

Additional data for this year include:

Survey data: UKES; SSS; PTES; PRES Franchise data: module completion rates

Data removed for this year include:

Franchise data that are not relevant or reliable: disability; ethnicity.

Section 7: Data Reflection & CommentaryThe PD reflects on the data included in the previous section. PDs encouraged to reflect on the three year trends in programme data. There is a limit of one page (Please try to limit this to 500 words).

Section 8: SWOT analysisThe PD identifies briefly the key strengths, weaknesses, opportunities and threats to the programme. There is a limit of one A4 page to include all components (bullet points are preferred). It is important to base these points on the evidence base included in evidence folder. Areas to consider (this is not an exhaustive list) include:

Anything from the data set; Curriculum currency; Student experience; Student voice; On-going actions; Alignment to the institutional strategic plan – key theme for 2016-17 is Retention.

Strengths Weaknesses

Opportunities Threats

Section 9: Action Plan Going ForwardNumber: Each issue should be numbered sequentially, prefaced by the year the action plan is written, in the format: 17.01; 17.02 etc. Administrative colleagues should put the numbers in place. Remember to start the list by including any items that need to be carried forward from previous APR action lists; the original numbering of the entry should be used so that it is clear the issue was raised in a previous year. Actions should be listed in order of priority, and be decided by the Programme Team, at Programme Committee, with the highest priority action being recorded first.The Measures of Success associated with the issue should be identified by the PD (Recruitment/Student Progression (Retention)/Award Outcomes/Career Destinations/External Examiners/Student Opinion/Staff Opinion/PSRB/NSS Teaching/NSS Assessment and Feedback). The remainder of the action plan should be completed by the PD.Issue: A brief description of the issue should be provided.Source: The source(s) of the issue should be identified, e.g. external examiner feedback.Actions taken/planned: Ensure that action(s) already taken and those planned are clearly defined and are capable of addressing the issue identified. By Whom: An indication of those responsible for undertaking the specified action(s) should be provided, e.g. Programme Director; Deputy/Assistant Dean Learning and Teaching. Please do not name individuals. You will need to discuss with your DDLT when assigning actions outside of the programme team.Timescale: A realistic timescale for completion of each action should be provided.Measurement of success: Indicate how the team will recognise/measure that an issue has been resolved (e.g. increased recruitment, fewer unfair practice cases).

MIS - Origins of the APR data

Key Performance IndicatorsRecruitmentFor the chosen programme or cluster of programmes, this is the number of new students enrolled in 16/17.

Undergraduate Degree student numbers do not include students who have progressed from the Foundation Leading to… programmes.

Student ProgressionThis replicates (within the limits of the current Qlikview application) the Higher Education Statistic Agency (HESA) Performance Indicator which measures the proportion of first year 15/16 Undergraduate Degree students who re-enrol in 16/17. Students who withdraw before 1st December in 15/16 are not included. Students who withdraw after 1st December in 15/16 but gain an exit award are classed as ‘retained’.

Award OutcomesThe proportion of students who graduated in 16/17 with a First Class or Second Class Upper degree.

Career DestinationsThe results of the 15/16 Destination of Leavers from Higher Education (DLHE) survey, which questions students who left in the 15/16 Academic Year.

Student OpinionThe results of the 16/17 National Student Survey (NSS), which questions students who were in their penultimate year of study in 15/16.

Survey DataUKESThe United Kingdom Engagement Survey (UKES) is run by the Higher Education Academy and questions undergraduate students in their first year of study in 15/16 (threshold or 5 or more students).

SSSThe Student Satisfaction Survey (SSS) is a Cardiff Metropolitan Survey and questions undergraduate students in their second year of study in 15/16 and third year top-up students (threshold of 10 or more students).

PTESThe Postgraduate Taught Experience Survey (PTES) is run by the Higher Education Academy and questions postgraduate students (threshold of 5 or more students).

Remaining data sheetsAll of the remaining data relate to students enrolled (on the Student Record System) in the 2016/2017 academic year.

Criteria for Rating External Examiner ReportsGreen

Exceptional – recorded as such

No issues raised – recorded as such

Minor issues raised in no more than two sections (Section A, Section B etc).

Minor issues raised in no more than four sub-sections (e.g. B2.1, D3.3 etc).

Note – ‘Partly’ answers to be excluded from the assessment unless accompanied by an explanatory statement.

AmberAll other reports not rated green or red.

RedMajor procedural concerns

- Not receiving an invitation to attend an examination board- Not receiving material explicitly requested (unless that material is protected)- Not adhering to Cardiff Metropolitan University regulations (modification, assessment approval)- Not receiving moderation pack in a timely manner

Implied criticism that could lead to reputational damage

- Negative comments on the management of academic standards - Negative comments of the structure of the curriculum (programme not module level)

Statements of a personal nature

Statements that identify individuals in a negative context

Student complaints

Not provided an opportunity to meet with students following a specific request

Implied threats to invoke an escalation process

Statements that reiterate issues that have not been addressed since previous report

Note:

1. The above list is not intended to be exhaustive and an element of interpretation will always be evident in the process.

2. Staff can request a review of the rating by emailing the Academic Standard and Quality Unit at ASQU@Cardiffmet.ac.uk.

Annual Programme Review Staff Development SessionsSchedule 2017/18: Home Provision and Collaborative Provision (Welsh partners)

Date Campus Room Time Who should attend?Thursday 20.07.17 Llandaff O1.09 10.30 - 12.30 Link Tutors and Moderators

If unable to attend this session Moderators and Link Tutors should attend one of the following sessions:

Date Campus Room Time Who should attend?Wednesday 12.07.17 Llandaff Boardroom, Llandaff 10.30 -12.30 School Administrators

and DDLTsMonday 17.07.17 Llandaff Boardroom, Llandaff 10.30 -12.30 Programme DirectorsTuesday 18.07.17 Cyncoed V007 14.00 – 16.00 Programme DirectorsThursday 07.09.17 Llandaff Boardroom, Llandaff 10.30 -12.30 Programme DirectorsWednesday 13.09.17 Cyncoed Conference Room 1 10.30 -12.30 Programme Directors

Annual Programme Review/External Examiner ReportsSchedule 2017/18: Home Provision and Collaborative Provision (Welsh partners)

Task Home Provision and Collaborative Provision (Welsh partners) Undergraduate, Taught Postgraduate and PGCE

Statistical data for Home programmes is ‘live’ in QlikView and PDs can engage with it ANY time from 25 July 2017. Retrieval data will be available post-retrieval boards.

From 25/07/17To 26/10/17

NSS programme-specific data will be ‘live’ in QlikView and PDs can engage with it any time from 1 August 2017.

01/08/17

A/DDLTs to quality assure developing APRs and provide support to improve quality as necessary (This should be prior to student sign off at programme committee)

During APR write-up period July to 13/10/17

Draft APRs considered in full and approved by Programme Committee. This includes external examiner reports and the programme response. Please note that student representatives must be part of this process.

By 20/10/17

APR summary and analysis completed by A/DDLTs. 1 week prior to Learning and Teaching Committee

Learning and Teaching Committee to consider and approve (subject to amendments as necessary):

i) APR key points and action plans (normally presented by PDs);

ii) APR summary and analysis (presented by A/DDLTs).

Full APRs (including external examiner reports) should be made available to the L&T Committee under Agenda B and can be moved to Agenda A for full scrutiny as necessary.

Please note that student representatives must be part of this process.

By 03/11/17

Schools to submit electronic copies of APRs/evidence base and A/DDLT APR summaries and analyses, to ASQU and the Learning & Teaching Board Secretary, via SharePoint Repository.

06/11/17

Learning & Teaching Board receives and APR Panel discusses APR summaries/analyses and corporate/over-arching themes.

29/11/17

Learning & Teaching Board discusses Dean of Q&S’ analysis of external examiner reports.

10/01/18

Please note that A/DDLT analyses of home provision and Collaborative Provision (Welsh partners) undergraduate and postgraduate APRs will go through L&T Board in November 2017.

Annual Programme Review Staff Development SessionsSchedule 2017/18: Collaborative Provision (TNE)

Date Campus Room Time Who should attend?Thursday 20.07.17 Llandaff O1.09 10.30 - 12.30 Link Tutors and Moderators

If unable to attend this session Moderators and Link Tutors should attend one of the following sessions:

Date Campus Room Time Who should attend?Wednesday 12.07.17 Llandaff Boardroom, Llandaff 10.30 -12.30 School Administrators and

DDLTsMonday 17.07.17 Llandaff Boardroom, Llandaff 10.30 - 12.30 Programme DirectorsTuesday 18.07.17 Cyncoed V007 14.00 - 16.00 Programme DirectorsThursday 07.09.17 Llandaff Boardroom, Llandaff 10.30 -12.30 Programme DirectorsWednesday 13.09.17 Cyncoed Conference Room 1 10.30 -12.30 Programme Directors

Annual Programme Review/External Examiner ReportsSchedule 2017/18: Collaborative Provision (TNE)

Task Collaborative Provision (TNE) Undergraduate, Taught Postgraduate and PGCE

Statistical data capture from QlikView (including any relevant NSS data) and dispatch to partners with APR proformas (ASQU).

24/11/17

Link Tutors and Moderators to quality assure developing APRs and provide support to improve quality as necessary (This should be prior to student sign off at programme committee).

During APR write-up period until 19/12/17

Draft APRs considered in full and approved by Programme Committee. This includes external examiner reports and the programme response. Please note that student representatives and Link Tutors/Moderators must be part of this process.

Deadlines to be confirmed by Schools

APR summary and analysis completed by A/DDLTs. 1 week prior to Learning and Teaching Committee

Learning and Teaching Committee to consider and approve (subject to amendments as necessary):

i) APR key points and action plans (normally presented by PDs);ii) APR summary and analysis (presented by A/DDLTs).

Full APRs (including external examiner reports) should be made available to the L&T Committee under Agenda B and can be moved to Agenda A for full scrutiny as necessary.

Please note that student representatives must be part of this process.

By 23/02/18

Schools to submit electronic copies of APRs/evidence base and A/DDLT APR summaries and analyses, to ASQU and the Learning & Teaching Board Secretary, via the APR Repository.

05/03/18

Learning & Teaching Board receives and APR Panel discusses APR summaries/analyses and corporate/over-arching themes.

14/03/18

Learning & Teaching Board discusses Dean of Q&S’ analysis of external examiner reports.

10/01/18

Please note that A/DDLT analyses of Collaborative Provision (TNE) undergraduate and postgraduate APRs will go through L&T Board in March 2018.