1. Good Practice in ValidationProcesses: Is going online the wayforwardMatthew WatsonCathy ShawQuality Officer (Validation & Review) Assistant Registrar (Quality Management)Quality and Academic Partnerships Academic Standards & Quality UnitUniversity of Northampton University of Wolverhampton

2. Background to idea for this session National conference on Good Practice inValidations held at the University of Northamptonin May 2011 The introduction of QAA Institutional Review andthe QAA UK Quality Code The funding regime a much less benignenvironment! 3. Outline of sessionDefinition of validation for the purposes of the session(QAA definition & requirements of the UK Quality Code)1. Validation processes at UoN2. Validation processes at UoW3. Debate: Pros & Cons of each methodology against 6 key principles4. Conclusions Is going on-line the way forward? 4. Validation Processes at theUniversity of NorthamptonThe more traditional style validation 5. Key Elements to validation process (UoN) All validations, changes of approval and Periodic Subject Reviews are managedcentrally by Quality & Academic Partnerships (QAP). Validations are based on the element of risk the higher the perceived risk themore complete the validation: i.e. a new programme with more than 50% newmodules is considered high risk so there is a full validation with externals inattendance. All Programme Specifications, Module Specifications and Award Maps areproduced, maintained and provided by a central Curriculum Team which is partof QAP. Prior to the real validation event, documentation is signed-off by the proposingSchool at an Internal School Meeting. The process is based on the end point meeting giving approval (or not) Validation reports are audited by central committees. 6. Flowchart! 7. DocumentationAll validation Panels are provided with: Curriculum documents: Programme Specifications, Module Specifications and Award Maps Rationale document from the Programme Team outlining the whys, whens and hows of the programme Draft Student Handbook The CVs of the Programme Team Memorandum of Co-operation (if collaborative) Supporting Statement from Library & Learning Services Indicative reading lists Panel Briefing Note from the Officer (usually from QAP) 8. Roles within Curriculum DesignProgramme Teams normally work with the following during the curriculumdesign process: Academic Advisor (one in each School): an academic with expertisein the University Modular Framework Curriculum Team representative: will produce, process and providethe Programme Specifications, Module Specifications and AwardMaps from the Academic Database System (ADS) Deputy Dean in the School (normally has the remit for QualityAssurance) Officer from QAP: will provide procedural advice External examiners (in some cases) 9. Timelines (a typical example)For a normal high risk validation event (involving a full Panel): September: Development Approval Form approved by University ExecutiveTeam and goes onto validation schedule October: QAP organises an initial information meeting with the ProgrammeTeam October to December: Programme Team working on validationdocumentation January: Internal School Meeting documentation signed-off Early February: Final documentation sent out to validation Panel Early March: Validation event held Late April: Response to conditions submitted May: Panel signs off response to conditionsBut it can be done a lot quicker (or slower) than this! The School decides 10. Validation Processes at theUniversity of WolverhamptonThe Virtual Validation 11. Key Elements to validation process (UoW) Validation embedded within curriculum development No end point validation meeting and introduce a process for validation throughdevelopment stages External Advisers to work with Teams in development Involvement of other University Departments at the earliest stages in curriculumdevelopment Only key documents required Module and Course Specifications availableonly on-line / electronically All documents presented to be written for students not for validation soinformation can be reused Validation / approval granted when all members of the validation panel are clearthat all requirements / actions have been fulfilled Validation reports are audited by central committees 12. Flowchart! 13. Roles within Curriculum Design External Adviser Central Registry ASQ OfficerServices School Proposing Institute forManagement TeamCurriculum TeamLearning(Associate Dean of Enhancement (ILE) School)LearningValidation Chair &Information Panel membersServices (LIS) 14. Documentation Tools used for viewing on-line documents and discussion Submission of drafts on e:Vision and WOLF Modules on e:Vision, other documents on WOLF All documentation is written for a student audience, notfor validation. CVs required if new area for the University If collaborative provision include CVs of staff based withcollaborative partner plus MoC and Operations Manual Academic Planning documentation can be madeavailable for reference, this includes market demanddata, provision of resources etc. 15. Timelines (a typical example)University level academic planning approvalOn approval: ASQ Officer set up IPM with Chair and Developmental LeadDeadlines set for: Receipt of nomination for External Adviser Date for visit from External Adviser (recommend 1st draft received) Process points for receipt of draft documents Process points for feedback from Validation PanelDiscuss a possible completion date for full approvalBut it can be done quickly (or slowly) - the School decides 16. Debate Six key themes 1. Accountability 2. Peer Review, Externality &Constructive Exchange 3. Timeliness, Efficiency andEconomies 4. Documentation 5. Curriculum Design, Approval &Sustainability 6. Quality of Product & AcademicStandards 17. Accountability Who has overall accountability for a validation process: the Panel Chair, theQA Officer, external or internal Panel members, the Programme/CourseTeam? In an online validation approach will the roles, responsibilities and levels ofaccountability change? How are issues resolved (does it differ between online and traditionalapproaches?) Is the Programme/Curriculum Team more or less accountable in an onlineapproach? What is the output from the validation - a detailed report or a confirmationthat the process is complete? Does it matter?The administrative tail is wagging the academic dog! 18. Peer Review, Externality & Constructive Exchange To what extent is the input adding to the process and to the end product? Which approach (online or traditional) offers the better opportunity for constructive exchanges? To what extent do the two approaches involve student input? How do the approaches facilitate a truly two-way conversation? How do the approaches impact on the nature and clarity of language used? In an online validation approach do the externals have a formative rather than a summative role (or both)? Validations are always so confrontational! 19. Timeliness, Efficiency and Economies Robustness versus responsiveness: in the two validation approaches is itpossible to have both or is one at the expense of the other? So is online really quicker, more efficient and cheaper? How does the role of the QA administrator/officer change in an onlineapproach (more tasks or less or just different)? Does an online approach to validations increase capacity even morevalidations possible! Is this a good thing? Which process allows Programme/Course Teams to explore new ideas and tomake changes to their product? Does an online approach enable greater input from students and otherstakeholders or is this unnecessarily slowing things down?Validation slows everything up! 20. Documentation What is the purpose of the validation documentation (is it just for thevalidation itself or beyond)? Does one approach generate more paperwork than the other? Should validation documentation be just for approval purposes? Or doesit need to be student friendly? What do validation Panels really need to see? Which validation approach creates a more appropriate QAA audit trail?Is this what we should be aiming for? Is the richness of detail and contextual information lost in the onlinevalidation approach?I had to produce so much documentationand it is only a minor change! 21. Curriculum Design, Approval and Sustainability Who should be involved in the curriculum design process? Consultation with external Panel members in an online validationprocess is it at an appropriate level? How can the QA administrator/officer most effectively manage all thecontributions to curriculum design in an online validation process? How do the two validation approaches ensure both the fitness forpurpose of the new programme as well as its long term sustainability? At what point in the validation process should the Programme/CourseLeader have the final say? We have 50 students all waiting to take this new programme honestly! 22. Quality of Product & Academic Standards Does the validation process add to the quality of the final product in terms ofLearning, Teaching and Assessment Strategies? Or is the validation processprincipally one of signing off paperwork? To what extent do the two validation approaches emphasise that validation issimply a first stage of the Quality Assurance process? How does the validation process ensure the production of high quality publicinformation? To what extent do the two validation approaches help to ensure the reputationof the University? Does either approach better highlight the role of the QA administrator/officeras a facilitator in the validation process (i.e. more than just a minute taker!)Have you got everything minuted? Is there teaand coffee booked & are we having lunch? 23. Debate Six key themes 1. Accountability 2. Peer Review, Externality &Constructive Exchange 3. Timeliness, Efficiency andEconomies 4. Documentation 5. Curriculum Design, Approval &Sustainability 6. Quality of Product & AcademicStandards 24. Accountability Regardless of type of process accountability lies with the Panel Chair supported by the Quality Officer/Administrator. In return the Panel Chair and QualityColleagues, it is 4.45 and Officer/Administrator must have a a decision needs to beprocess that supports them. made 25. Peer Review, Externality & Constructive ExchangeConstructive but critical peer review(both external and internal) has to beat the core of any process and bereceived at a point in that processwhere it can have an impact and bestI dont like blogging because I dont want to appearensure the highest quality product for negative.students. 26. Timeliness, Efficiency and Economies In a perfect world we could define a standard timeframe for validation. However, curriculum design requires thought, research, consultation and evaluation. Institutions do notIt took six weeks just to get an assessment employ staff as just curriculumchanged. designers. 27. DocumentationDocumentation should beappropriate for both student andvalidation processes so as to avoidunnecessary duplication.Documentation should not beWe will validate what you wasted on validation. are going to deliver nowyou deliver what we havevalidated. 28. Curriculum Design, Approval and SustainabilityCurriculum design and approval aremore effective as an integratedprocess, allowing wider stakeholder Everyone we spoke to before the validationinvolvement with no nasty surprisessaid it was fine, yet why at the end. have we been set conditions? 29. Quality of Product & AcademicStandards Our validation processes must be answerable to external quality agencies and this is unavoidable. Should this stop us from doing things Will this stand up to differently?muster when the QAA audits us? 30. Conclusions?