contents web viewthe nature of the project ... academic staff are interested in the possibility of...

Contents

Contents 1

Introduction, aims and purpose 2

Participating institutions 3

Case studies 14Co-creating the curriculum through student academic partnerships 14Sharing views and making links: student perceptions of outstanding teaching and exceptional learning (SPOut) 18Enhancing The Learning Experience Through Evidence-Informed Curriculum Design

21Students Overseas: promoting learning opportunities abroad for UK students and engaging more effectively with incoming students 25Understanding student experience in a changing world - rationalising and using the evidence 29Data for risk assessment and enhancement 33Developing VLE approaches for an enhanced student learning experience 38Transition to and during the first year of study - an integrated institutional approach

42Enhancing Impact of Evidence-Informed Changes to Assessment and Feedback 46Assessment Strategies and student degree attainment 49

Evidence-informed quality improvement programme (EQUIP) 2010-11 An overview of the programme, participating institutions and case studies

Introduction, aims and purposeEQUIP is a change programme that aimed to help higher education providers take an evidence-informed approach to quality enhancement at departmental, faculty or institutional level. It enabled participating institutional teams to identify, understand and resolve key issues affecting the quality and effectiveness of the student learning experience.

Taking an evidence-informed approach to quality enhancement, the programme enabled institutional teams to examine the evidence available on the learning experiences of their students. This evidence might include data generated by the institution (such as external examiner reports, internal surveys) and comparative external data, including the National Student Survey (NSS) and QAA institutional audit.

EQUIP focused on using evidence to improve the quality of learning and teaching processes and offered participants the opportunity for participants to address areas for development such as: curriculum design and development; the creation of learning environments; responding to issues raised in institutional audit and external examiner reports; developing a quality enhancement strategy; addressing issues raised through student surveys, e.g. the National Student Survey

(NSS).

The programme consisted of the following key events:

Start-up meeting – 14 September 2010 Two-day residential – 7-8 December 2010 Final meeting – 23 March 2011

Participating institutionsThe following institutions took part in the programme. The next section provides a summary of each team’s initiative and a link to a case study where available. The case studies can also be viewed as a collective in the final section of this publication. Click on an institution name to jump to a summary.

Birmingham City University

De Montfort University

Edge Hill University

Newcastle University

The Open University

University of Derby

University of Huddersfield

University of Stirling

University of Winchester

University of Worcester

Birmingham City UniversityCo-creating the curriculum through student academic partnerships

Birmingham City University is focused on generating a vibrant and integrated learning community that fully involves staff and students in its academic and research development. This focus, through specific activities, has the potential to significantly impact upon institutional culture and the quality of the student learning experience and is driven through a stimulating partnership between the University and Birmingham City Students’ Union. A major part of this activity has seen the creation of a Student Academic Partners (SAP) scheme which strives to co-create curriculum innovations through this new partnership. The first iteration of the SAP scheme in 2010 resulted in over 60 project applications and 23 funded projects, involving 35 students. Engineering, Art, Business, Law, Media and Nursing students engaged in the scheme as employed partners not assistants and were empowered to be active leads of project teams. A further iteration of the SAP scheme will be launched in the autumn of 2010 and the project team hope that participation in EQUIP will help it put in place measures that enable the project’s educational rigour and evidence base to be placed beyond question and to use this evidence to inform and guide growth of the scheme and facilitate institutional change.

Team leader – Luke Millard

A case study of this initiative is available below – click here to jump to it

De Montfort UniversitySharing views and making links: student perception of outstanding teaching and exceptional learning (SPOut)

The SPOut project aims to build capacity within the institution by taking a holistic view of DMU's approach to quality improvement. DMU already has strong processes in place for putting into practice recommendations for improvements: we make significant use of the internal and external data and processes available to us and central departments work with the students' union to take forward learning and teaching projects.The SPOut project will further enhance and embed these processes. It will also link with work which has investigated staff perceptions of what outstanding teaching and exceptional learning might look like across subject disciplines together with other research which looks at the ways subject staff define good practice in learning and teaching. Both pieces of research show that whilst there may be links and similarities across subject areas, these are often unexplored due to the predominance in higher education of the subject ethos. The SPOut project aims to focus on the students' views of outstanding teaching and exceptional learning to investigate whether there is an echo of the findings found amongst academic staff groups.The identification of examples of outstanding teaching and exceptional learning and an evaluation of what characterises them will help stimulate institution-wide debate around learning and teaching, leading not to a slavish copying or unthinking transfer of practice from one context to another but a deeper critique of the practices within subjects which will promote the process of identifying similar examples of excellence within individual subject teams.

Team leader – Nick Allsopp


Edge Hill UniversityEnhancing The Learning Experience Through Evidence-Informed Curriculum Design

An Undergraduate Framework (UGF) is already in situ at Edge Hill and contains a series of 'design lenses' congifured as a set of enabling, yet constraining questions which must be responded to by proposing teams at programme approval. Linked to these is a Directory of Good Practice (DoGP) which collates exemplars from the University's quality management processes with the aim of brokering contacts between practitioners and spreading good practice. This project will aim to:

Secure and strengthen appreciation of the UGF architecture as it rolls out through programme approval across the three Faculties;

Review, within the EQUIP framework, the fitness-for-purpose of the UGF design lenses against emerging research literature;

Explore ways of harvesting and disseminating components of the Directory of Good Practice more effectively to encourage better interaction between practitioners, including the development of an online interactive resource bank containing practical 'real world' exemplars;

Better engage students in curriculum design and delivery as advisors and co-producers of case studies using an 'appreciative enquiry' approach;

Embed the input thus obtained from students in teaching and learning development for academic staff;

Extend, where appropriate, the approach adopted to the UGF to the emerging parallel Postgraduate Taught Framework within Edge Hill.

A key outcome of the project will be to create a reportable case study for change and development. This should be of key interest to other institutions which share our objective of enhancing the student experience, retention, satisfaction and, crucially, academic success.

Team leader – Tony Turjansky


Newcastle UniversityStudents Overseas: Promoting learning opportunities abroad for UK students and engaging more effectively with incoming students

This project has twin aims, to: Increase the likelihood of UK-originating Newcastle University students undertaking a

period of study overseas and the range and depth of learning from such experiences; and

Enhance the experience of international students at Newcastle University, specifically in promoting links with UK-originating students.

The core team comprises members from an academic faculty (Team Leader), central service units and Newcastle University Union Society. A larger group, including the other faculties, will support this team to enable them to embed the outcomes of their work within wider policies and practices. The project combines: (a) evidence from UK undergraduate students at Newcastle University about perceived barriers to participation in learning opportunities abroad; and (b) evidence from UK and international students, as well as staff, about how they have reduced or overcome such barriers. Based on this evidence, the project will lead to: Policies, marketing approaches and practices leading to increased uptake of

opportunities for learning abroad; Enhanced integration of international and UK students at Newcastle University; and A ‘toolbox’ of guides, examples and staff development provision that will help with

creating and maintaining international partnerships.

Team leader – Gerard Corsane


The Open UniversityUnderstanding student experience in a changing world - rationalising and using the evidence

The Open University routinely analyses student retention and progress data, and collects extensive quantitative and qualitative information from students about their experience of our modules and qualifications. These data are the focal point for formal module, faculty and institutional review processes that inform quality assurance and enhancement processes. This project will evaluate the different sources of evidence available and how such data can be used:

1 to better understand the student learning experience, and2 to explore how it can drive student-focused personalised quality

services/enhancement. The expected deliverables would be:

1 the identification of a coherent information set and methodology that can be used consistently and efficiently to support regular reflection on the experience of our students and to support decision making about enhancement of our teaching and learning practices. Critical to the success of both is clear evidence that reliably points to successful and less successful practice from the perspective of the student.

2 more specifically, an evidence base from the data to be used to inform decisions on personalisation of services as an enhancement to the student experience.

The project aims to create a framework which might be of use to other HEIs in evaluating the student experience in their own institution.

Team leader – Josie Taylor


University of DerbyData for risk assessment and enhancement

This project will assist the University in (a) devolving the responsibility for (and understanding of) the assessment of risk to a wider range of staff within the Faculties, and (b) extending the range of data for risk assessment purposes. The data will include statistics generated through OASIS (an on-line facility linked to the student record system), NSS outcomes, the reports of external examiners and, crucially, the evidence of student satisfaction obtained from programme and module questionnaires. The project will:

1 Identify the information needed by programme teams, Faculties and Academic Board for quality assurance and enhancement purposes;

2 Identify student priorities for enhancement and the associated data requirements;3 Ensure student access to data and support student representatives in the effective

use of such data;4 Agree minimum datasets to be made available to programme teams and to Faculty

staff with risk assessment responsibilities;5 Review the form and content of programme and module questionnaires;6 Assist staff and students in developing their skills and confidence in the analysis and

interpretation of data (both qualitative and quantitative) for risk assessment purposes;7 Enable the University to further develop its risk assessment methodology as the

integrating hub of its quality assurance, enhancement, resource allocation and planning processes.

The project links to both the University’s Learning, Teaching and Assessment and Student Experience strategies as well as the Enhancement Policy. The provision of relevant and valid data, accessed efficiently, is the foundation for improving the effectiveness of decision-making and ensuring efforts are focused where the greatest impact will be felt.

Team leader – Colin Fryer


University of HuddersfieldDeveloping VLE approaches for an enhanced student learning experience

The nature of the project is to focus on the institutional and School-based use of Virtual Learning Environments (VLEs: both our proprietary VLE Blackboard and other non-proprietary VLEs eg Web2.0 tools) in enhancing the student learning experience. This project underpins institutional change to the use and integration of VLEs and assists with developing agile responses to a changing HE environment. It builds upon evidence provided by a recent Thematic Review of our proprietary VLE, Blackboard, incorporating other VLEs such as Web2.0 tools, and develops consistent, integrative and interactive use of VLEs in teaching and learning approaches This builds on the University's Teaching and Learning Strategy and enables further development of pedagogic approaches with the use of technology and a clearer focus for the quality enhancement of the student learning experience. The project also aims to develop procedures whereby the consistency and appropriateness of a better integrated use of VLEs can be more effectively and rigorously explored with course teams at key events in quality assurance and enhancement procedures, for example, at programme validation and annual evaluation of programme provision.

Team leader – Wilma Teviotdale


University of StirlingTransition to and during the first year of study - an integrated institutional approach

The University of Stirling, through its Induction Working Group, aims to develop a collaborative, evidence-based Institutional strategic plan which ensures that induction/transition activities are integrated across the University. Through the EQUIP project we aim to bring about strategic change which will impact on our approaches to induction/transition and benefit the student learning experience. We aim to: Enhance the University’s approach to induction/transition by developing a strategy

which will have cross-institutional impact; Implement a coherent and co-ordinated approach to induction/transition with an

emphasis on induction/transition as a student focussed, core activity; Ensure the strategy and associated activities are evidence-led and embed a quality

enhancement approach; Create an audit tool to benchmark induction/transition activities across the University

and provide a framework for enhancing induction/transition processes and practices. These aims are aligned with the University’s current ‘Learning and Teaching Quality Enhancement Strategy’ in which transition is clearly identified as a priority area, with a specific aim to develop and enhance the induction process for all students. The intention is that the final induction/transition strategy will be integral to the mainstream learning and teaching activities of the institution and will be adopted as a key component of the University’s approach to the enhancement of the student learning experience. To meet the project objectives, a cross-university team (service areas, academics and the Students Union) will work together and draw upon both external and internal documents as well as qualitative and quantitative data from students and staff.

Team leader – Mark Wilkinson


University of WinchesterEnhancing Impact of Evidence-Informed Changes to Assessment and Feedback

The University of Winchester have always been passionate about assessment and feedback and the subsequent impact on the student learning experience. Following a trend of poor assessment scores on the NSS, a series of small internal and external projects funded by the institution were conducted. From this platform, the Academy awarded Winchester an NTFS project - TESTA (Transforming the Experience of Students Through Assessment, www.testa.ac.uk), funded over 3 years. Participation within the EQUIP programme would allow the dissemination of a tried and tested methodological approach to enhancing the programme-level assessment and feedback experience of students. Since Phase 1 of TESTA, which included the collection of data from seven programmes across four institutions, more than eight additional programmes from partner institutions have expressed interest in being TESTA'd. Participation in the EQUIP programme would facilitate a greater impact of the TESTA research through wider engagement, dialogue, and strategic discourse around the change processes. The successful embedding of the methodology within validation and re-approval processes would add a bigger impact from the evidence and statements of programme teams currently involved (than the initial project predicted). The culmination of these developments will encourage the project leaders and EQUIP programme team to draw together some of the key conceptualisations in reflection on the programmatic/institutional change process.

Team leader – Yaz El-Hakim


University of WorcesterAssessment strategies and student degree attainment

The proportion of good (1 and 2:1) degrees awarded varies significantly between institutions, subject groupings, and has been shown to be related to student characteristics. This project seeks to make use of a range of evidence to ascertain the reasons for relatively low performance in assessment outcomes in two subject areas and to inform the development of improvement strategies. The key stages of the project are:1 analysis of student attainment data, external examiner reports, student feedback data

for specific courses and modules, including benchmarking with other HEI data2 audit of course assessment strategies3 .staff and student focus groups4 review of assessment policies and practices5 design and development of revised assessment strategies. Anticipated outcomes of the project will be:

1 modelling of the use of evidence for the purposes of reviewing assessment strategies2 advancing knowledge and understanding of relatively low student attainment as

measured by degree class3 strengthening of staff-student partnerships for quality enhancement at course leve4 improved understanding of how assessment strategies at course level are likely to

impact on student learning experiences and attainments.

Team leaders – Will Bowen-Jones and Ian Scott


Case studiesCo-creating the curriculum through student academic partnershipsBirmingham City University

The context within which your initiative is located

Birmingham City University is focused on generating an integrated and effective learning community that fully involves staff and students in its academic and research development. This focus, through activity, has the potential to significantly impact upon institutional culture and the quality of the student learning experience. A major part of this activity has seen the creation of the Student Academic Partners scheme in partnership with Birmingham City Students' Union. Further details can be found at http://www.birminghamcitysu.com/saps.

The University wide scheme aims to integrate students, through paid employment, into the teaching and pedagogic research communities of faculties. The intention is to develop collaboration between students and staff, generating a sense of ownership of programmes and a greater sense of being part of a learning community, whilst at the same time delivering outcomes which make a significant contribution to the enhancement of the student learning experience.

The University engaged with the EQUIP team and the HEA to assist and guide the evaluation of the project. In addition, the University team sought support in creating publications and reviewing the additional benefits that could be accrued by students, staff and the University.

Aims/ Objectives of the initiative

The SAP team at Birmingham City University resolved to:

create a plan for the production of journal articles submit at least one journal article by summer 2011 explore accreditation opportunities at the University determine the value of such accreditation to students and staff to disseminate learning from the SAP scheme to the HE sector

Description

The major challenge that the SAP team face is the continual evolution of the scheme and the way it operates. It challenges ways of working and assumptions around student and staff engagement. Students and staff continue to be innovative and challenging and the SAP team need to be flexible enough to meet these needs.The SAP initiative is advertised as having a partnership based approach and this is demonstrated through its mode of operation. This approach will be sustained through the activities we develop with the EQUIP team.The EQUIP focus on accreditation offers a case in point as it was assumed that students and staff would be very interested in having their experiences recognised through an accreditation scheme. However, student opinion was split. Many were not very interested in writing a reflective piece that would have to be assessed and would receive 15 credits. They were much more interested in receiving an award that was not necessarily credit bearing. An award that recognised their participation and the skills

http://www.birminghamcitysu.com/saps

that they had developed was identified as being key. However, for some a credit bearing award was of interest so we have pursued a mixed economy approach to the issue.

Academic staff are interested in the possibility of employing the curriculum development aspects as part of a Masters level piece of work and the project team agreed to explore this possibility.

Outcomes

Intended

The Student Academic Partner initiative features in two journal articles which have been submitted.

An accredited module at level 6 is being written to meet the needs of students requiring credits.

A level 7 module is being developed for staff who would wish to seek accreditation for the learning from the SAP project

A dissemination event will be held as the main part of the University’s Learning and Teaching Festival at which student academic leadership certificates will be provided for all student participants

Learning from the SAP project has been shared through a variety of external conference presentations at JISC, ALD in HE, NAIRTL and other student engagement conferences. In addition, an internal dissemination event hosted by the Students’ Union has been employed to encourage dissemination amongst university students and staff.

Unintended

The accredited module at level 6 is being developed through a Student Academic Partner project in the Business School. At the same time as the SAP team was enquiring of student attitudes to accreditation, a proposal was received from a student and staff team in the Business school to develop a module that would recognise student achievement outside of the classroom. It was always intended that the SAP team would lead on this work, but it is more fitting with the scheme that a SAP project team have identified a similar need and will now undertake the development work.

Impact

on staff

92 members of academic staff are engaged in SAP projects across the University. Every one of the six faculties is represented and each has a minimum of ten SAP projects.

on students

164 students are engaged in SAP projects across the University.

on the HEI

The University now has 250 change agents working across the University. Through a variety of joint events we are trying to make this cohort of change agents have more of an effect that the sum of their parts. In particular, we are seeking to encourage SAP projects to share ideas and work with other SAP teams in different faculties.

From a university perspective we are trying to understand what makes these projects work and identify any difficulties they face in having an impact. The creation of video snapshots of enablers and barriers has proven useful and is aiding our dissemination activities and internal learning.

Important moments in the initiatives

The most important moment in the initiative was probably one over which we had no control as in November 2010 the SAP scheme was awarded the Times Higher Education award for outstanding support for students. The external recognition this provided also ensured substantial internal support and has now resulted in Vice-Chancellor level support for expansion of the SAP scheme as a leading arm of the student engagement work at the University.

This has also assisted with our dissemination of the SAP ethos to the wider HE sector and has resulted in many universities consulting with the SAP team to see if the ideas in operation at BCU can be transferred. We are aware of three other universities that have mimicked the SAP scheme and we continue to meet with others, almost on a monthly basis.

One of the most interesting aspects of the SAP scheme has been the developing relationship with our own Students’ Union. Collaboration between the university and the Students’ Union is now viewed as a given and this message is one that many other Students’ Unions and universities have found to be of great interest.

Lessons learned

Personal level

The SAP project is wonderful as you can really see it change the lives of staff and students. Staff become excited and invigorated by the new ideas of students. Students have their lives changed through the development of new skills, but perhaps more importantly through a change in perception of their role. The ability to work with academics as equals is of high importance to the students we work with and one which seems to make a real difference to the way they perceive themselves and their university.

Professional level

Professionally it is very pleasing to be working with a team and an idea that receives external recognition within the sector. Having folk from other universities interested in your idea is a rare event and one that reassures us and the university that we are heading in the right direction.

Team level

I have been supported by a great team that offers me either student centred and/or faculty specific advice. The creation of the EQUIP team at BCU was important as this

offered a dissemination tool in its own right and has helped take the SAP message to students and faculties.

Institutional level

This is the first time the University has won a THE award. It now appears on all university advertising and branding. As a result the University has now bought into student academic engagement in a far greater way than we could have foreseen. This has resulted in increased funding and a move, through the HEA’s Change Academy, to explore greater opportunities for student engagement through a much a larger student employment scheme.

Future Work

Short Term

Budget talks are now taking place to see how we can grow the SAP scheme over the coming years.

Medium Term

The SAP scheme will form a key part of the Change Academy initiative we will be undertaking to explore a much broader scheme of enhanced student engagement through employment.

Long TermWe want Birmingham City University to be seen in the sector as one of the leading universities in the field of student academic engagement and student employment. We are confident we are heading in the right direction, but we are impatient to have a greater impact within our university and to share these ideas with others.

Key Messages for others

We were surprised at how wary some staff were of working with students. However, once the success of the SAP scheme had been established a momentum developed.

Word of mouth between students and staff has become our greatest friend. We now have students and staff who apply each year, but we also have the contagion effect of these enthusiasts infecting the rest of the student and staff body with their challenging ideas.

Come and talk to us, we want to offer our insights and learn from you!

Sharing views and making links: student perceptions of outstanding teaching and exceptional learning (SPOut)De Montfort University


The SPOut project builds on capacity within the institution by taking a holistic view of DMU's approach to quality improvement. DMU already has strong processes in place for putting into practice recommendations for improvements. For example the university makes significant use of data gathered internally and externally and as a consequence central departments work with the Students' Union to take forward learning and teaching projects.

The SPOut project further enhances and embeds these processes. It also links with work which has investigated staff perceptions of what outstanding teaching and exceptional learning might look like across subject disciplines and with other research which looks at the ways subject staff define good practice in learning and teaching. Both pieces of research show that whilst there may be links and similarities across subject areas, these are often unexplored due to the predominance in higher education of the subject ethos. The SPOut project focused on the students' views of outstanding teaching and exceptional learning to investigate whether there is an echo of the findings found amongst academic staff groups.


develop a clear and precise idea of what students think characterises good teaching and learning;

identify the similarities and differences across subject disciplines; identify links and gaps with staff views; and work to embed this knowledge in our developments for learning and teaching.

Description

We started by designing an online questionnaire to gather as wide a range of responses from students as possible and to help with the construction of the questions for the focus groups that followed. We ran the questionnaire and gathered the names of students interested in contributing to a series of focus groups. We analysed the results of the online questionnaire and formed the questions for the focus groups. Having run the six focus groups we again analysed the results and formed conclusions using all of the data available to us. We compared our tentative conclusions with the findings of the sister project which considered staff responses to the issue of outstanding teaching and learning and were able to draw up a final set of findings and recommendations which we are putting to the institution for consideration and, where appropriate, implementation.

Outcomes

The research project has produced a report which details its findings thus:

Teaching: students believe that outstanding teachers enable them to perform their best as students.

More, and more diverse, students: students tend to regard large teaching groups more as a challenge than an asset.

Curriculum relevance: the overwhelming majority of students considered the content of their programme to be up-to-date with developments in their field.

Transitions: students have experienced different degrees of challenge in their transition into university from school or college, often according to the subject chosen for their course.

Assessment: almost all students have been informed about the methods of assessment used on their course and considered the variety to be appropriate to allow them to effectively communicate their skills, knowledge and understanding.

Support for learning: students generally were very positive about the range and quality of support for their learning provided by the university.

We intend to publish our full report to an internal audience and to organise a series of dissemination events.We also intend to write a number of articles which bring together the research findings of this project and its predecessor in order to find links.

Impact

on staff:

Not involved in this project directly. However they will become involved through an engagement with the research findings.

on students

Students have been the central focus of this work. Members of the Students’ Union executive have been integral members of the research team, constructing and conducting the questioning we have undertaken. Students interviewed reported thatthey felt it a positive experience to be asked their views and for their views to be listened to.

on the HEI:

Impact will be most acutely felt when the results of this research project are combined with the previous research to form an overview of both staff and students.Important moments in the initiatives

We were pleased with the initial take up and participation in the online survey but then disappointed by the low student participation in voluntary focus groups following the survey. We salvaged this situation by asking directly for help from staff in organising a second set of focus groups.The results from the focus groups have provided us with a rich data set which will enable us to understand more fully the student view and to react to the student voice. We feel that overall the project has significantly added to our understanding and will enable us to take forward work based on solid research evidence.

Lessons learned

Comments below come from the Students’ Union executive members of the research team and help to explain the ways of working and the significance of the outcomes.

Personal level

“It has been an enjoyable experience working alongside members of the university on this project, and we were pleased to be asked to help.

- It has been a real learning experience as sabbatical officers as we’ve been able to see the responses given from students to members of the University, in an honest way, which sometimes might only be said to the Students’ Union.” (DSU)

Professional level

“The final report will be extremely useful for our Deputy President Education to look at and use for developments within their role. They will see the differences between faculties and the strengths of and weaknesses of each. The differences between staff and students perceptions will also be very useful for this member of our team, and across the institution.” (DSU)

Team level

“The team worked really well as a whole, and it was a great idea to include the Students’ Union and student voice in the team, as we could offer first hand opinions of what it’s like to be taught at De Montfort University and had an idea of what students may say.- It was also really good for us to be invited to the residential training, and be included in all the developments on the project, so that we had input at all stages of the research.” (DSU)

Institutional level

“We learnt a lot about student perceptions from the focus groups, and how they differ between faculties. This information, along with the differences between student and staff perceptions of what they think exceptional teaching and learning is can be used across the institution and we believe it will be a very valuable report for De Montfort University, and key to its education developments.” (DSU)

Future Work

As above, we intend to consider the research findings from this project alongside the previous project which looked at staff views of outstanding learning and teaching. Together these projects will form an important set of data upon which to base further action and to develop institutional approaches.


The use of members of the Students’ Union was key to the success of this project. We also had confirmed the levels of positive engagement with the process from the general student body. In general we have found that our students appreciate being asked their opinions and are prepared to engage in a serious dialogue which might (or might not) have a direct effect upon their own learning. In other words our students are not the surface and instrumental learners or consumers that popular myth seems to suggest.

Enhancing The Learning Experience Through Evidence-Informed Curriculum DesignEdge Hill University


The Undergraduate Degree Framework (UGF) has been in place at Edge Hill since 2008, integral to which is a review architecture containing ‘lenses' focused on different aspects of the student experience such as academic skills development (‘learning literacies’); inductions and transitions; teaching, learning and assessment; personal tutoring; personal development planning; technology-enhanced learning; work-related learning and work-based learning; and the international student experience. These are accompanied by a set of enabling yet constraining questions which programme developers are required to address at validation http://www.edgehill.ac.uk/aqu/StudentLearningExperience.htm. Also in place is a Directory of Good Practice (DoGP) in which exemplars from validation, monitoring, periodic review and external examiner reports are collected and collated with the aim of brokering institutional contacts and spreading good practice http://www.edgehill.ac.uk/aqu/documents/DirectoryGoodPractice_2010.pdf. By bringing these together in the form of a wiki-based repository with supporting administrative and technical infrastructure it was hoped to increase the impact of the UGF and DoGP.


By using evidence from validation, review and other processes to identify exemplars of good practice, the aim was to evaluate their potential for dissemination and work with the originating areas to develop a series of online toolkits which would secure and strengthen appreciation of the UGF architecture across the institution.

Description

A project team was convened at the start of the year comprising representatives of academic departments, support services and the Students’ Union with the aim of scoping the content and format of the proposed wiki (Phase 1). A pilot version was developed and launched for internal university use during early 2011 containing exemplars of good practice harvested, evaluated and processed by ‘task teams’ drawn from academic and support staff across the institution, each assigned to a different lens of the UGF architecture. Phase 2 of the project, which is ongoing, aims to expand and enhance the content of the wiki through a second round of task team activity while university and faculty-level committees will continue to scrutinise the reports of validations, reviews, external examiners etc. to evaluate good practice citations for possible inclusion in the wiki. Three Senior Learning and Teaching Fellows (SL&TFs) have been appointed to lead enhancement activity within their faculties and will bridge and co-ordinate the activities of task teams and committees. SL&TFs also have links to the institution’s SOLSTICE (‘Supported Online Learning for Students using Technologies for Innovation and Communications in their Education’) CETL where multimedia and interactive applications for the wiki will also be explored.

Outcomes

Intended

The task team approach involves making direct interventions at department level and has shown itself capable of generating concentrated bursts of activity, yielding

reasonably quick results across all of the UGF review lenses. By contrast, working through institution and faculty-level committees to identify enhancement potential tends to be a more protracted process and may not generate exemplars across the whole of the UGF architecture. Nevertheless, both approaches are meant to complement each other as while task team activity is by its nature short-term and time-limited, the enhancement foci of committees are expected to be long-term, continuous and sustainable. It will also be desirable to review and refresh the constitution of task teams over time to increase their reach into faculties and departments.

Unintended

The appointment of Senior Learning and Teaching Fellows promises benefits that extend beyond the UGF wiki project, for example leading on activities to develop capacity and capability in learning and teaching development within their faculties and leading and encouraging support for learning and teaching research, scholarship and knowledge transfer activities both within and outwith the institution.

Impact

on staff:

Task team activity associated with development and implementation of the wiki has started to raise the institutional profile of the UGF. However, awareness remains highest amongst a limited though influential constituency of deans, associate deans, heads of department and programme leaders and more needs to be done to bring the framework and wiki to the wider academic audience. Reconstitution of the task teams to include more representatives of academic departments should help facilitate this.

on students:

Student engagement with the wiki has thus far been limited and there is scope to makefuller use of module and programme-level evaluations as potentially rich sources of evidence of good practice. The recent introduction by the university of institutional Teaching Awards for staff, nominated and voted on by students, provides further opportunities to identify and harvest exemplars.

on the HEI

Task team leaders are, in many cases, institutional Learning and Teaching Fellows whose fellowship plans include conference presentation and journal publication as well as contributing to internal staff development in learning and teaching. Senior Learning and Teaching Fellows in particular are expected to publish and present the university’s pedagogic developments, research and evaluation of projects at regional, national and international conferences.


Key challenges addressed by the project group included:

Evaluation of good practice: Good practice citations, even those of independent external examiners, are not always sufficiently detailed or robust to merit inclusion in the wiki without further verification and in the majority of cases some additional investigation has been necessary. Evaluation also needs to take into account the importance of exemplars being transferable to other contexts and disciplines beyond

the areas in which they were originally developed if wider application is going to be practical.

Format and content: Accessibility was a key consideration in the design of the wiki interface and the adoption of a linear structure linking each lens of the UGF to a set of of Resources, Guidance documents and Key contacts was determined to provide an appropriate balance between visual sophistication and usability.

Ensuring sustainability: While the twin approaches of task teams and committees provide the mechanisms for generating and updating content, it was also necessary to secure long-term administration and technical support for the wiki. Locating this within the university’s Teaching and Learning Development Unit has provided a logical base for operations as well as facilitating ongoing communication with faculties and academic departments.

Lessons learned

Personal level

While the psychometric testing offered by the HEA provided some interesting insights into individuals’ leadership and team-working capabilities, in practice the assigning of responsibilities within the project group was determined mainly by members’ substantive roles within the institution.

Professional level

The decision to populate the project group with representatives of academic departments, support services and the Students’ Union was ultimately beneficial in providing the widest range of perspectives from which to draw.

Team level

Working both individually and collectively, the project group was able to lobby a range ofinstitutional stakeholders from deans of faculty to directors of support services and up to PVC level in order to secure institutional buy-in. Where challenges were encountered these were mostly of an organisational nature, for example securing individuals’ availability for project meetings and HEA awaydays, and better forward planning of diaries may have alleviated this. It also became clear that student input to the project was going to be more realistic at the evaluation stage, rather than during development, and this might have been anticipated when producing the project plan.

Institutional level

Interim progress reports were submitted to university committees for learning and teaching and the student experience and helped maintain the visibility and profile of the project. Close co-operation with the Teaching and Learning Development Unit was essential to ensure that initiatives taken by the project group were co-ordinated with work taking place on technical development of the wiki. A key challenge going forward will be encouraging departments under pressure to free up time and resource to produce materials showcasing their good practice to a wider audience and the institution will wish to consider possible means of incentivising such activity.

Future Work

Short Term

The project team is cognizant that evaluation should extend beyond mere statistical measurement of engagement with the wiki. A comprehensive 360° survey should involve all identified stakeholders and while written contributions may be appropriate for academic and support staff, student opinion may be better secured by communicating in ‘real time’ through focus groups, questionnaires or other personal testimony.

Medium Term

In twelve months we will consider the wiki to have been effective if there is clear evidence of it having influenced practice through module and programme evaluations and reports of validation, annual monitoring, periodic review and external examiners.

Long Term

Successful implementation will be demonstrated by extensive population of the wiki and wide take-up by academic and support staff. Most importantly, the institution will seek evidence that adoption of the UGF principles has enabled a more coherent and holistic approach to programme design than that which preceded it.


The project reinforced for us the importance of team constitution, planning and organisation. Although existing systems and structures provided good routine communication between faculties and services, managing the UGF wiki project under the EQUIP banner gave it a profile and impetus that it may otherwise have lacked and where we have concerns these are mainly around sustaining that profile as the wiki becomes embedded in the institution’s routine systems and structures. Project planning including advance scheduling of meetings is essential if momentum is to be maintained and some form of shared access to diaries may be helpful here. The setting of milestones and interim deadlines is also important in keeping the project on track. Because of the nature of our project, user evaluation was always going to follow later and in hindsight it would be preferable if some form of interim evaluation could have been conducted before the EQUIP deadline.

Students Overseas: promoting learning opportunities abroad for UK students and engaging more effectively with incoming studentsNewcastle University


A period of study overseas can be one of the most effective means of supporting the development of the various attributes that equip students for global citizenship. Yet nationally 25 students come to the UK for each one going abroad [THES, 3 June 2010].

The team hopes to change the situation at Newcastle University by further developing policies, marketing strategies, staff and student ‘toolkits’, enabling and supportive practices and staff development. Through all of this it is hoped to increase uptake of opportunities for learning abroad by UK-originating students and enhance integration of international and UK students at home. This will help to enhance the learning experiences of all Newcastle University students.


This project aims to:

Increase the likelihood of UK-originating Newcastle University students undertaking a period of study overseas and the range and depth of learning from such experiences;

Enhance the experience of international students at Newcastle University, specifically in promoting links with UK-originating students.

Description

We collected evidence in policies, strategies and plans from across the university. A significant survey completed with all 1st/2nd year students in the Humanities and Social Sciences Faculty ascertained students’ views on a range of study abroad opportunities and the barriers that they would need to overcome. In the Faculty of Medical Sciences information is being collected from undergraduates and potential applicants at open days. The project intends to continue collecting information from students so that a better understanding can be gained about what are viewed as useful international experiences and the perceived and actual barriers that students face when considering taking up different opportunities. Further evidence will be collected from staff members involved in supporting incoming and outgoing students who have undertaken, or intend to take up, opportunities of gaining experience in another country.

It is hoped that by collecting and analysing this evidence, we will be better placed to draft strategies and plans to help develop:

New marketing approaches and supporting practices for arriving UK students, leading to increased awareness, confidence and ability to prepare for and benefit from learning opportunities abroad of all types, thus enhancing their career prospects;

Enhanced co-ordination and development of arrangements for mentoring and ‘buddying’ by international and UK-originating students.

Mitigation and removal where possible of barriers to study abroad, for example to do with accommodation,;

Creation, building on existing examples, of 'toolboxes' for students and staff, with the latter supporting development and maintenance of partnerships, including staff development provision;

An optional undergraduate credit-bearing module around students' preparation for learning opportunities abroad; and

A cultural shift towards a more inherently international outlook by students and staff.

Outcomes

Intended

The tangible intended outcomes which are being worked upon are:

A strategy and implementation plan, marketing approaches and practices intended to help UK-originating students to increasingly consider and take up opportunities for learning abroad;

Enhanced integration of international and UK students at Newcastle University through finding ways to bring international and home students together through events, mentoring and buddy schemes,; and

A ‘toolbox’ of guides and examples for staff and student development provision that will help to facilitate more students making decisions to take up the different opportunities to gain experience in another country.

Unintended

The timing of the project has coincided with the Project 2012 at Newcastle University. This means that staff at all levels will be more receptive and looking for ways to engage with the topic, as everyone looks for ways to enhance the student experience. People with whom the project has been discussed are all supportive of the proposed outputs. However, this development of the University’s agenda has meant that the changes envisaged have to be dovetailed with a range of other developments. Together with new demands on the project leaders, this has entailed delays in actions and thus impact.

Impact

on staff:

This has been limited at the moment. However, the impact will be more measurable once more information about the project has been disseminated and the proposed outcomes and products have been developed.

on students:

The above statements are also true regarding the current impact on students. However, as the project continues, the impact for students will become more obvious. Currently, those students surveyed may be more conscious of the opportunities available.

on the HEI:

The topic and the project have started to be seen by key decision-makers in the institution, especially with work that is being done in relation to Project 2012 and what can be done to enhance the student experience.Important moments in the initiatives

Important moments have been:

Conceptual development of the project and proposed products by the core team during the residential event;

Decided on a list of grouped actions to be undertaken during the life of the project; Work started on drafting a strategy and implementation plan with Student Union

involvement; Profiling of the project amongst stakeholders and decision-makers through

committees and smaller meetings; Influencing people and agendas, especially as many at the university are considering

Project 2012 and how to enhance the student experience; Data gathering is continuing – and we hope to expand across other Faculties; and Continued development of potential ‘study abroad’ opportunities and partnerships in

HaSS.

The challenges faced and overcome:

Time – team working, division of labour and working independently; Identification of members for a larger team of stakeholders; Plans for setting up student fora; Drafting of strategy and implementation plan is underway; and Work has started on a new website, within the framework of a university-wide review.

The main remaining challenges are:

Competing and rapidly changing priorities; Funding and capacity for the development of online resources for students and staff

(the value of the ‘EQUIP badge’ is clearly important to the project here); Realisation that the project needs to be considered as more long-term.

Lessons learned

Personal level

Personally some of the group have learned that there is no quick solution and that theproject will need to be ongoing and longer-term than originally thought. There are so many potential stakeholders and there is a wealth of evidence already accumulated, with opportunities for further gathering of data. The project definitely needs more time.

Professional level

The sessions and supporting material provided during the EQUIP sessions were very useful and have helped to shape the project. The team found much of this useful in terms of professional lessons learned – the involvement of our individual Academy-nominated staff member was especially useful in identifying key questions and issues.

Team level

We have learned how helpful it is to have a smallish team with a clear brief; especially one that involves student participation.

Institutional level

The learning for the institution is not yet clear to us, but we hope it will become so as the outcomes are increasingly achieved.Future Work

Short Term

Continue collecting evidence and as the project develops involve others. It will also be important to continue ‘selling’ the topic to people at all levels in the organisation.

Medium Term

To extend the core team and to undertake further work on drafting documentation that supports the initiative.

Long Term

To develop the ‘toolkits’ and work towards getting more material online for staff and students alike. Once the material has been developed and outcomes and outputs achieved, there will be a need to put in place a system to evaluate the impact of the project in terms of quantitative and qualitative assessment.


With study-abroad opportunities, UK HEIs have identified a problem in that the number of incoming international / EU students far outweighs the number of UK-originating students going overseas to enrich their studies and future career prospects by taking up international opportunities. This imbalance also has an impact on the UK generally. "Without people who are comfortable working in an international context, the UK's international influence is bound to decline over time. […] Universities should consider what kind of study-abroad experience would attract more UK students to sign up.” [THES, 3 June 2010]This ongoing project will promote learning opportunities abroad and hopefully help staff and students to understand perceived and actual barriers, thereby helping to find ways to reduce or remove these barriers. We will continue to be happy to be approached by colleagues at other institutions who would like to know more about further outcomes of this project (email: [email protected]).

mailto:[email protected]

Understanding student experience in a changing world - rationalising and using the evidenceThe Open University


This project was an evaluation of different sources of evidence available and how such data can be used (i) to better understand the student learning experience, and (ii) to explore how it can drive student-focused personalised quality services/enhancement. The drivers for change were: (i) responding to our audit recommendation that the University might "consistently analyse in greater depth the extensive management information it gathers and use it systematically to inform qualitative strategic analysis, academic development and quality enhancement"; (ii) we needed to better understand the student experience, particularly important in a distance learning setting; (iii) we needed to foster improved cross-unit working in a university which has separate units responsible for curriculum, student support and the production of learning materials.


1 To consider the sources of student data available, and how they might be used to understand the student experience and deliver quality enhancement.

2 To foster more effective cross-unit working to optimise use of data.3 To examine how personalised services can be driven by this data.

Description

Originally, we thought the project would be undertaken by a team from the University’s Institute of Educational Technology and its Quality Office, as well as the Chair of the University’s Student Experience Advisory Group. It would draw on information and expertise from other areas of the University which support data and information gathering, and academic and support areas involved in developing courses, course materials, teaching and supporting students.

However, at the same time, our Knowledge Media Institute obtained funding from JISC for a project which looked at combining activity data (as opposed to questionnaire responses) from students using systems such as email, the VLE etc., combined with statistical data about courses, submitted assessments and student background, to see if it is possible to reliably predict students at risk.

This in turn coincided with the emergence of learner analytics as a newly emerging research area across the world, and an interest in understanding how activity data can inform learners and teachers alike, and hence drive useful, engaging behaviours in students.

Consequently, we have contributed to the drawing together of all these initiatives into a much larger University project on learning analytics, sponsored by the PVC (Learning, Teaching and Quality). Staff will be hired to take the project forward.

Outcomes

Intended

We have helped to stimulate the debate and discussion with the OU about the role and value of this approach to data, and have thrown our support behind moves to improve

data warehousing. We have also engaged in two major strands of work which feed into the learning analytics project. The first is a learning design approach to course development, using ‘at-a-glance’ views of student data. The second is the triangulation of data points to support Deaneries in their understanding of multiple data streams, including course-related student data, and student activity data from the VLE.The integration of these strands of work has been a major success for us – in a large organisation, it is tempting for units to provide services as a ‘cottage industry’, leading to bespoke solutions which simply don’t scale. Streamlining data collection and warehousing is clearly a better approach, but historically we have been left with a legacy problem. The move amongst staff to resolve these difficulties has been a major achievement, but requires constant monitoring as systems develop.

Unintended

The team did not anticipate that the University would respond to these moves by instigating a major strategic project. This illustrates that we were clearly building on a developing wave of recognition that we needed to change our approach.

Impact

on staff:

We have supported moves to integrate and streamline data gathering, data warehousing, and provided a strong rationale for why this should happen. This has had an enabling effect.

on students:

Students have yet to feel the impact from this effort. They will benefit from the system initially within 12 months, but more realistically, it will take 2 to 3 years for the full benefit to be realised.

on the HEI:

The most powerful impact at this point in our development is on the Institution. The project will entail re-design of some of our basic systems, and ways of thinking about course design and provision. The data which will then be available will drive our business thinking, and our ability to respond to student demand both in terms of curriculum, but also in terms of levels of service and student support. These will be important factors in the years to come, as fees increase and students become more focused on getting what they want from what they are paying for.


The project, undoubtedly like many others, has been hampered somewhat by the changing world in which we live. All participants in the project have significant roles in the University, and in dealing with the current financial situation, have been required to deliver to those roles as a priority. The project leader was also absent on sick leave for almost 3 months.

However, the important moment in the project came when we met, rather despondently, to discuss what little progress we thought we had made specifically on the EQUIP project, only to find that each of us had in fact been contributing very effectively through a variety of other channels. By pulling it all together, we could then see the net effect,

and identify the contribution from our project expressed as strands through other initiatives. We are delighted that the University has instigated a strategic project around this topic, and would see that as an outcome for us.

Lessons learned

Personal level

Lesson learned: don’t give up just because you don’t have time.We have all struggled with time management in the project. It is easy then to feel guilty for not finding/making the time to do the work, and the temptation is to give up, to close the project down. As a team, we have resisted doing this, despite our difficulties. The ‘branding’ of the project gives us time to bring into focus things that we might not have profiled in quite the same way if we hadn’t been engaged in it, and the team have definitely learned more from one another than we would have done in our normal professional roles.

Professional level

Lesson learned: You can’t do everything - make sure you have the right support in placeOur team members all occupy significant roles in relation to student data management in the University, so we were able to use our influence at key points in the debate and decision making processes. We were also able to identify where in the University such discussions might be going on, and were able to engage with them. This strategic role was very important. However, in a future project, we would consider engaging a project officer who had time set aside to support the project work and to promote it as a project in its own right.

Team level

Lesson learned: know when to protect project boundaries, and when to let them dissolve.It is generally considered good practice to establish the boundaries of a project, and try to protect them. This helps prevent scope-creep, and allows one to know whether a project has actually delivered what it should have done. We began by trying to protect our project, but eventually realised that doing so would also limit the effectiveness of our own efforts. Hence, we changed tack, particularly given our own time limitations, to both engage other initiatives and to encourage others to step forward with contributions. This would not always be an advisable strategy, but, in this case, has resulted in a much better outcome than we could have predicted.

Institutional level

Lesson learned: Don’t underestimate how much persistence is required to bring a good idea to fruition.The Learning Analytics project has come together through the combined efforts of various staff groupings/teams across the University, all working on different aspects of the problem. Pulling them together into strategic coherence required a great deal of advocacy, discussion, problem resolution and persuasion. This has turned out to be a contribution of our project team, working with many others in the Institution who also recognised the need.

Future Work

Short Term

The Learning Analytics project will take up where we have left off in terms of system development and deployment across the University.

Medium Term

We will all stay close to the development to ensure it delivers what we are expecting, and to see how we can further exploit the data in the future.

Long Term

We hope to see measurable improvements in the student experience as the systems begin to facilitate our understanding of student behaviour.

Key Messages for othersOur experience in this project has illustrated how important it is to continually scan the environment to identify emerging projects/activities that can help progress a project’s aims. It might seem that there has been a certain amount of serendipity in our situation, but we also recognise our influence on those emerging ideas – this co-evolution of problem identification and the bringing forward of potential solutions is an important institutional process which we think should be acknowledged and recognised.

Data for risk assessment and enhancementUniversity of Derby


Risk assessment lies at the heart of the University’s enhancement-led strategy, integrating the information generated by the various quality assurance procedures and ensuring that they inform enhancement-focused action by staff at all levels within the Institution. Risk assessment has previously been undertaken by the University's Centre for Quality and it has been based on a limited range of essentially qualitative data.

This project was established to assist the University in: devolving the responsibility for (and understanding of) the assessment of risk to a

wider range of staff within the faculties, and; extending the range of data for risk assessment purposes.

The data will include statistics generated through OASIS (an on-line facility linked to the student record system), NSS outcomes, the reports of external examiners and, crucially, the evidence of student satisfaction obtained from programme and module questionnaires.


To identify the information needed by programme teams, faculties and Academic Board for quality assurance and enhancement purposes.

To agree minimum datasets to be made available to programme teams and to faculty staff with risk assessment responsibilities.

To agree on appropriate benchmarking data and key performance indicators by which to judge performance.

Description

Assessing the health and vitality of provision, analysing situations, evaluating problems and making decisions are reliant, in the first instance, on having the right information to hand. Coupled with this, is the need to assist staff in developing their skills and confidence in the analysis and interpretation of data. Whilst the two are mutually inclusive, our assumption is that if the institution is unclear about what data is required for decision making/risk assessment to support specific activities, then data literacy diminishes in value.Our approach has therefore been to test out this assumption by:

Reviewing the adequacy and effectiveness of the current suite of statistical reports, particularly at the programme level.

Discussions with programme leaders via focus group meetings to identify the key datasets and obtain constructive feedback on the reports available through OASIS in relation to format, layout, structure, user friendliness, etc. Fitness for purpose was an important discussion thread at the meetings to inform proposed changes to the suite of statistical reports.

Discussions with heads of school to capture their requirements and identify areas for improvement in relation to data access, minimum datasets and the form and content of school level statistical reports.

Redesigning the programme level statistical reports used for annual monitoring and measuring the effectiveness of the changes via feedback from programme leaders.

Proposing changes to the school level statistical reports based on the outcomes of a review of the revised data reports at the programme level.

Outcomes

Intended

In consultation with programme staff, a minimum dataset was agreed for the purpose of annual monitoring at the programme level. The dataset provides statistical information on enrolments, retention and achievement.

The reports produced at the programme level incorporate for the first time National Student Survey (NSS) data, where appropriate, and specific University benchmark data published as part of the key performance indicators. Such data includes benchmarks for retention and degree classification achievements.

The format of the reports has been completely revised to ensure that the information is presented clearly and in a style that programme leaders find easy to use.

The number of reports that programme leaders are required to access from OASIS has been significantly reduced from eight to two. This has been achieved by removing duplicate information wherever possible and re-formatting the data so that it is re-packaged in such a way as to make the best use of space on the printed page.

OASIS has been further developed to incorporate data from different systems, for example NSS information and University key performance indicators as part of a drive to develop an integrated management information system where data access is dynamic.

Unintended

The development of revised reports for programmes has been instrumental in engaging further discussion about data requirements at both the school and University level.

A recognition that a ‘one size fits all’ approach to statistical reporting will always be limiting in terms of staff engagement with data as the system lacks the ability to be dynamic and tailored for individual need.

End users of reports require further guidance on data definitions, for example retention, and that the reports should also make specific reference to the provenance of the data.

The need to reinforce the message that the reports available from OASIS are primarily for the purpose of annual monitoring whilst recognising that they may also be of value in other aspects of programme management.

Impact

on staff:

Through the focus group meetings staff have had the opportunity to contribute to the development of the revised statistical reports at the programme level and agree a minimum dataset for annual monitoring purposes. There is now a greater ownership of data by programme leaders than in the past, reinforced via open discussions with teams and their involvement throughout the process of revising the report templates.

Staff have commented on the improvements in relation to the presentation of data and how the new reports are easier to use when completing the annual monitoring programme report.

The inclusion of NSS results and University benchmarking data has been instrumental in engaging staff in the assessment of risk as well as extending the range of data for risk assessment purposes.

on students:

Although the impact on students has not been an explicit feature of the project, staff are now better placed, through the provision of easier to use statistical reports, in developing their skills and confidence in the analysis and interpretation of data and hence are better placed in drawing conclusions regarding the student learning experience. In this way, staff are enabled to better identify ‘at risk’ students at an earlier stage than previously so that support interventions can be brought in to play sooner.

From September 2011, the annual monitoring programme report (and its enhancement plan) will be prepared by the programme committee at which student representatives attend. Students will therefore have the opportunity consider the data and comment.

on the HEI:

Following on from this project, the University has established a Data Journey Group whose remit is to review the University’s systems for capturing data at all levels and initiate developments as appropriate; for example, an interim review has taken place of the data provided at assessment boards and subsequent data provided to inform annual monitoring and how this is accessed by external examiners. It is intended that this will support initial intervention to ensure that external examiners receive all of the information that they have requested in a timely fashion.


Whilst the external environment is rapidly changing, it is assumed that the nature and type of data that HEIs will be engaged with will remain fairly constant, located within the returns made for HESA and HEIDI. The risk is that as HEIs become more business focused the type of data used in the decision-making process will change markedly from the current position and hence the project’s effectiveness could become diminished. As a consequence, the project brief was revisited and sensible decisions were made concerning the scope of the project so that the focus in relation to risk assessment was in the context of annual monitoring and the data required by programme leaders. This approach had the benefit of providing more structure to the project and helping us to define more clearly what would be achievable within the timeframe.

A real positive from the project has been the engagement with programme leaders in helping to define the minimum dataset for annual monitoring programme reports. Establishing a consensus in relation to programme level data represents a significant step forward and provided the baseline for further discussions on revising the report templates. Conflating several data reports into two reports, where the goal was to achieve templates that were easy to use and set a premium on high presentation values, was challenging. Identifying duplicate information and streamlining the data were essential to achieving the defined outcomes.

Lessons learned

Personal level

Having been involved in the development of OASIS in recent years this project has provided me with the opportunity to reflect on how users engage with the system, its effectiveness, and importantly how it could be improved. It has been quite illuminating in discussions with programme leaders to gain a better understanding of how they use the statistical reports for annual monitoring and what works well, and

also what could be improved. Also there has been a recognition that the existing reporting system had become too sophisticated and resource intensive and hence a more straightforward approach was required.

Whilst this has been an ambitious project and its objectives have had to be reviewed over the twelve month period, what has been achieved will be of benefit to programme leaders and has informed developments going forward, most notably the concept of an integrated management information system.

Professional level

Reflection on practice has been at the heart of this project particularly on how decisions taken centrally impact on staff in the faculties. Listening to colleagues on how they engage with data in assessing risk and evaluating the student learning experience helps enormously in reflecting on one’s professional practice. Working through the project is in itself a vehicle for change both individually and collectively through such activities as staff development, additional guidance materials and changes to processes such as annual monitoring that actively involve student participation. Consideration of data and risk is but one single point that makes a connection with a wider network of related activities.

Team level

The team for this project was relatively small, consisting of five members of staff from central departments and faculties each with specific expertise to complement one another. Whilst each member was able to contribute to the project, the demands on individuals were sometimes in flux as a result of having to revisit the project brief on a number of occasions to review and revise the objectives. The team was at its most effective when it had sufficient time to consider progress and formulate actions having debated fully the issues, for example at the workshop in Leeds organised by the HEA.

Institutional level

The revised programme level statistical reports have been endorsed by the University through consideration by the institution’s Quality Enhancement Committee. The statistical reports are made available to programme leaders as part of the annual monitoring process and further developments are planned for similar reports at the school level.

The project has identified that standardising datasets across the institution is only one aspect of a complex equation. It is recognised that data is held on different systems within the institution and accessing this information is currently very difficult. Considering the recent changes in the sector and the volatile external environment, the University is alive to the benefits of developing an integrated management information system, and to this end has established a Data Journey Group to spearhead future developments.

Future Work

Short Term

To identify the information needed by schools, faculties and Academic Board for quality assurance and enhancement purposes.

To agree minimum datasets to be made available to programme teams and to faculty staff with risk assessment responsibilities.

To agree on appropriate benchmarking data and key performance indicators (University and/or sector-wide as appropriate) by which to judge performance.

Medium Term

Review the form and content of programme and module questionnaires so that they are fit for purpose and take account of both the formal (programme committees, NSS, etc) and informal feedback mechanisms.

To further develop the OASIS suite of statistical reports around the concept of an integrated management system with the key building blocks having been identified.

Ensure that staff have the skills and confidence in the analysis and interpretation of data (both qualitative and quantitative) for risk assessment purposes.

Long Term

A fully integrated on-line reporting system that provides effective access to relevant and valid data that is directly linked to decision-making and the assessment of risk.

To have developed an effective risk assessment methodology as an integrating hub of the University’s quality assurance, enhancement, resource allocation and planning processes.


It may seem like a statement of the obvious but be realistic in what is achievable within the planned timeframe for the project, recognising that the external environment is redefining priorities almost on a day-to-day basis.

Do not be afraid to ask the big questions but do not expect to always achieve all of the answers.

Think carefully about the key stakeholders and how you can engage them in such a way as to maximise the benefits from the project.

Celebrate success.

Developing VLE approaches for an enhanced student learning experienceUniversity of Huddersfield


In 2001 the University of Huddersfield was at the leading edge of technological innovation and teaching, becoming one of the first universities to begin using a VLE. Despite the early adoption, Huddersfield has tended to mirror the findings of Browne et al (2006) who, in their survey of VLE adoption and use, found that universities had been slow to move beyond the management function to exploit the instructional capabilities of the systems. The EQUIP project was seen as a catalyst that could support our desire for institutional change and academic curriculum innovation in the University of Huddersfield.

In May 2010 a thematic review of the VLE at the university recommended “an expanded and more detailed articulation of the University‟s strategy regarding learning technology” and in making a number of specific recommendations suggested that a project such as this might be the best way of assessing how to move forward.


To provide direction for further development of pedagogic approaches with the use of technology and a clearer focus for the quality enhancement of the student learning experience so that it is truly innovative and inspirational.

To develop procedures, embedded in our established quality assurance and validation processes, whereby the consistency and appropriateness of a better integrated use of VLEs can be more effectively and rigorously explored with course teams.

Description

A process of consultation, encompassing university senior managers, Student Voice 2010 conference, staff questionnaire administered by our Teaching and Learning Institute and team meetings has led to a much more centrally focussed approach at University level. A VLE Steering Group has been established to support the move to our new VLE platform, Unilearn, and the development of a staff „driving licence‟ to support the institutional change required.

Outcomes

Intended

Restructuring of the core VLE committee structure. The number of groups, forums and committees with advisory or executive roles had become complex. The EQUIP project identified institutional structure problems and worked with senior management to provide a new structure with a clearer focus.Obtained senior management buy-in of a VLE driving licence suggested by the EQUIP team. The evidence from an external survey of Huddersfield‟s academics showed high levels of engagement with learning technology, however the respondents were self-selecting and therefore may have been those most interested in learning technology. Data gathered from a staff planning day in the Business School showed that, although a large minority of staff were resistant to the use of learning technology, the majority of staff either used the technology or were keen to use it. However, very few expressed confidence in using the University VLE for assessment. By establishing a VLE driving

licence it is envisaged that confidence and willingness to engage in learning technology will be raised.

Unintended

The EQUIP process has encouraged a stronger focus on learning technology when addressing teaching and learning research. There are now a number of internal projects running that may not have been conceived had it not been for the influence of the project.

Unexpected

Funding has been provided by senior management to allow the use of placement students in each of our seven academic schools to further support the embedding of technology in the student learning experience.A Learning Technology Development Group was set up to support and implement the recommendations of the VLE steering group. The overall structure has added formality to the change process and allowed the organisation to be flexible and agile in its provision and more importantly in response to HE innovations.

Impact

on staff:Raised awareness of the urgency of developing a clear strategy by holding staff planning days centred around learning technology, making learning technology the focus of teaching and learning seminars and by promoting other university initiatives and projects.

on students:The EQUIP process put the wheels in motion to address concerns raised by students in the VLE thematic review.

on the HEI:The establishment of the VLE Steering Group (executive) and the Learning Technology Development group (advisory).


When the project began we were able to draw on the VLE thematic review which had been conducted earlier in May 2010. This was an important body of evidence from which to begin the project.The residential in Leeds was valuable in enabling the team to clarify the issues and what the barriers to progress might be. It was following this that the project leaders were able to approach senior management to recommend a review of the number of groups involved in learning technology with a view to creating a more coherent strategy for the development of the VLE and the establishment of a staff development strategy.The collection of evidence during the project was a constant challenge. While the VLE thematic review was an important starting point a broader body of evidence was required if the project was to be confident that it could reliably inform senior management. Evidence from two staff surveys proved useful; however only a small amount of further data was obtained from the student body.

Lessons learned

Personal levelAs project leader, I have been reminded of the power of „time out‟ and the freedom of peer discussions. Group working and the synergies from this are real catalysts for innovation.

Professional levelWe know plenty of academics who are already using technology; many have been encouraged to share their practices via the staff development opportunities which exist. The enthusiasm of these academics needs to be harnessed in order to bring more on board.

This year we were able to use a Staff Planning Day to find out how the staff in the Business School felt about learning technology. The responses supported the notion that for most the VLE is a management tool with around a quarter seeing the VLE as an assessment tool and around a third seeing it as a place for collaborative learning.

Academic staff are encouraged to implement the use of learning technology through a two pronged approach driven by top-down and bottom-up initiatives.

Team level

It is considered that all members of the team benefited from the residential work and the free and supportive exchange of ideas and opinions.

Institutional level

There was better student engagement in the data collection process, allowing staff to get a perspective of their views and ideas. What the students think is relevant and there is consistent evidence from the VLE thematic review through to recent discussions with students that while many students appreciate the benefits of VLE use, their experiences vary from module to module. To achieve more consistency, we need to develop and support staff – the „driving licence‟ will be an important aspect of this.

Academics need support to be empowered; our evidence shows a large minority who currently have no interest in pursuing learning technology as part of the student learning experience. Institutions need to provide support from the top level, encouraging academic usage - in order to empower staff to achieve „bottom up‟ development.

Future Work

Short Term

The current project can at most provide clarity and direction to a much larger process of institutional change. Therefore a short-term priority is the securing of additional funding to continue with the work of the project.The university is about to unveil a new VLE and opportunities for staff to learn about how to use it and how to extend its use into learning and assessment need to be implemented and supported on a on-going basis. The introduction of the new system is an opportunity to present new ideas to staff about the use of the VLE as a tool for learning and assessment.

Medium Term

We will establish a VLE benchmark which is used as a measurement tool for VLE minimum requirements. These minimum requirements are based on a clear remit from the student voice. Evidence gathered from the project is that students feel that the VLE enhances their learning experience but that it needs to be used consistently.

We aim to establish a procedure to ensure that the student voice is heard clearly and regularly.

Long Term

We aim to develop and implement an institution-wide Driving Licence which is used as a method of staff development.

We aim to establish institutional agility so that the institution will find it easier in the future to progress with developments in learning technology


It has been said before, but change does not happen on its own. Whilst progress was being made in the direction of institutional change, the project has helped the institution to refocus and consider how the process might be improved.

No academic can be left behind in this process, but at the same time no single approach is going to bring everyone on board. This does not mean that all academics will need to exploit learning technology to the full in all modules. It does mean that all academics are empowered to make pedagogically sound decisions with regard to the learning which takes place. If the academics are excluded from this choice then the students become excluded also.

Finally, do not ignore the student voice.

Reference

Browne, T., Jenkins, M., & Walker, R. (2006). A longitudinal perspective regarding the use of VLEs by higher education institutions in the United Kingdom. Interactive Learning Environments, 14(2), 177-192.

Transition to and during the first year of study - an integrated institutional approachUniversity of Stirling


The University’s ‘Induction Working Group’ (IWG) had been successful in driving forward a number of initiatives which enhanced the student experience. However, there had been limited integration with the more ‘academic’ elements of induction and transition.

We therefore aimed to create and implement an evidence-based institutional strategic plan which ensured that induction/transition is fully integrated across all aspects of the student experience.

In the context of learning and teaching, the University’s mission is operationalised through the current Learning and Teaching Quality Enhancement Strategy (LTQES 2010-2015). In the LTQES, transition into, and through, the University is clearly identified as a priority area with a specific aim to develop and enhance the induction process for all students. This project therefore directly addressed a key priority area for enhancement, and participation in the EQUIP programme would enable participants to take forward a strategic priority in an evidence-informed way. An additional key driver for participation is the institution’s aim to develop ‘one of the best student experiences in the UK.’


We aimed to:

Enhance the University’s approach to induction/transition by developing a clear strategy;

Implement a coherent and co-ordinated approach to induction/transition with an emphasis on this as a student focussed activity;

Ensure the strategy and associated activities are evidence-led and embed a quality enhancement approach;

Create an audit tool to benchmark induction/transition activities across the University and provide a framework for enhancing processes and practices.

Description

The project team decided part way through the project to revise our timescales and by September 2012 we plan to have:

Enhanced the University’s approach to induction/transition by developing a clear strategy which would have cross-institutional impact;

Implemented a coherent and co-ordinated approach with an emphasis on induction/transition as a student focussed, core activity;

Ensured that the strategy and associated activities were evidence-led and utilised a quality enhancement approach;

Created a self assessment tool kit to benchmark induction/transition activities across the University’s Schools which would provide a framework for enhancing induction/transition processes and practices;

Identified and promoted examples of good practice across the Institution in order to encourage appropriate adoption.

We have used the HEA EQUIP development meetings to build our team, clarify and refine our aims and approaches and to agree on timescales and milestones.

Team members have contributed in various ways to the project to date, and these include: Desk research; Literature reviews; Liaison with other universities to identify resources and model strategies; Considering and embedding good practice at the level of School plans and strategies; Undertaking focus group and discussion sessions with students and staff; Raising awareness of the project in the University (for example, QAA sponsored

Student Engagement event, Edufair, a Learning and Teaching staff development event).

Outcomes

Intended

Continued and focussed discussions at various levels across the University and within the Student Union on the purpose of induction/transition. These discussions will inform the University’s and School transition strategies.

Gathering of new (and collation of previous) student views on the induction experience.

Embedding of a strategic approach to induction/transition within the School of Nursing, Midwifery and Health’s Student Experience Strategy.

Collation of a ‘library’ of resources in one central place relating to induction and transition.

Adaptation of a self-audit tool for academic use to benchmark, review and evaluate induction and transition and development of a ‘good practice’ template to gather examples of effective practice.

Presentations at internal events – Student Engagement event (University of Stirling, May 2011) and also Edu Fair (University of Stirling, May 2011)

Unintended

The EQUIP group have developed good working relationships and will continue (and may add additional team members) to work on this project over the next 12-15 months.

All team members have had an opportunity to take discussions about the project and about the purpose and value of effective induction/transition into many various contexts and situations thus stimulating wider interest and debate about this topic.

Impact

The links between a successful induction/transition into Higher Education and an enhanced student learning experience are well documented in the research literature and have been communicated by group members via a number of activities and meetings – these include the University’s Induction Working Group, the Quality Enhancement Committee, the annual University teaching and learning event – EduFair, via discussions with students within the Student Union, and amongst academic colleagues in the School of Nursing, Midwifery and Health and the Stirling Management School. It also remains an important focus within the University’s efforts to prepare for Enhancement Led Institutional Review later in 2011.

This project team recognise and have communicated that for transition into the first year of study to have a positive impact on the learning experience, it is necessary for activities to be integrated across the University. The continued work and outcomes from this project will provide the framework within which to implement this approach. The intention remains that the induction/transition strategy will be integral to the mainstream learning and teaching activities of the institution and will be adopted as a key component of the University’s approach to the enhancement of the student learning experience. As such, it will be adopted by the University’s Quality Enhancement Committee; this Committee is chaired by the Deputy Principal (Learning and Teaching) and is responsible for all academic policy and strategy including the LTQES.

It is our intention to evaluate the revised arrangements in place in the School of Nursing from September 2011 and to identify the impact these have on successful transition to the University for this student cohort. We will also identify the impact of University wide revised arrangements for the wider intake in September 2011. This data will used to complete the process of drafting the University wide Induction and Transition strategy which will be in place for September 2012.


In addition to attending the HEA EQUIP residential the project group noted the following as key moments:

Early realisation that the scale and ambition of the project and the barrier of identifying sufficient time meant that our timescales had to be stretched – we are now working on plans for a final strategy for September 2012.

That the presumption that most institutions already had in place a similar strategy was wrong.

Lessons learned

Personal level

Keep an eye on the range of competing interests and time demands for all project participants and build in flexibility for all parties. Maintain communication across the time span of the project even when diverted to other priorities.

Professional level

Developing interdisciplinary, interprofessional, multi-agency relationships is enriching and rewarding. Collaborating with peers who would not necessarily work together on other project work broadens perspectives and insight into the student experience.

Team level

The focus of this project offered an opportunity to develop closer working relationships with colleagues, learning new skills, meeting and making friends that can help shape and develop practice, protocol and policy for the future.Discovering new ways of working and sharing ideas, concepts and developing knowledge transfer activity across academic and service areas is enriching and rewarding.

Institutional level

Enhancing existing practices with available evidence and taking more of a 360 degree approach to problem solving to ensure that the student experience meets the expectations of today’s students and also that of key stakeholders – this model of working together can be replicated within the institution on a wide variety of development areas.

Future Work

Short Term

The EQUIP project group will continue to meet across the Summer period 2011 and will finalise our self audit tool and good practice template. We will report on progress and negotiate the next steps with the University’s Induction Working Group.

We will complete the report on students’ views on induction/transition from the surveys already undertaken. We aim to prepare and pilot in September 2011 a student expectation and perception survey methodology.

Medium TermOver the period September 2011 we will evaluate with the help of the School of Nursing, Midwifery and Health the innovative practices now embedded in their student experience strategy and use this data to complete a draft Institutional Strategy for Induction and Transition (Institutional wide by September 2012)

Long Term

Continued monitoring and further enhancement by the Induction Working Group of the University’s agreed strategy.


Do not underestimate the time required to provide a contextualised evidence base to support institutional change.

The power of student engagement with projects should not be underestimated. Keeping in regular communication both within the team and with the Institution are

crucial to keep the momentum going. More than worthwhile, vital even, that there is inter-disciplinary collaboration on any

improvement project work that is undertaken. Benefits to the team and the institution are immeasurable when more than one

academic and service area work together and share commonalities and understand differences and similarities.

Time is a crucial factor in that there needs to be time for teams (project groups) to sit and work through developments on a face to face basis.

Sharing ideas with like-minded individuals in other institutions is vital in scoping and extending knowledge and understanding, and finding potential solutions to problems, either real or perceived.

Enhancing Impact of Evidence-Informed Changes to Assessment and FeedbackUniversity of Winchester


Our initiative developed from the TESTA project, which looks at programme level assessment regimes at 4 universities including the University of Winchester. The TESTA methodology uses evidence collected from student questionnaires and focus groups, reviews of programme documentation and discussion with programme teams to form plans of action to improve assessment across programmes. The positive response from programme teams and the possibilities for influencing programme assessment regimes at a fundamental level encouraged us to think hard about how to embed elements of the TESTA procedure into the programme revalidation process. Our drivers were a desire to increase student satisfaction through 1) making their experience of learning deeper and more cohesive, and through 2) including them in the revalidation process via their feedback influencing programme development. A third driver was an institutional move towards Learning and Teaching and Quality working more closely together.


As defined in our aspirations document: “By the end of the programme we will have embedded a collegiate yet rigorous social and dialogic process for programmes where periodic review is more strongly evidence informed so as to encourage more thorough debates around pedagogy and assessment systems within and through programmes”.

Description

Our initial plans for embedding these practices into institutional procedures were fairly traditional; we intended to set up an Action Group with key stakeholders from across the University, garner support from programme leaders and senior management through summer 2011, with a target of beginning to change policy in the academic year 2011-12. However, circumstances changed and the project actually moved much faster than we envisaged, probably as a result of the drivers outlined above becoming a much higher priority across the university (especially the QAA institutional review‟s focus on student input into quality procedures), and as a result this stage became redundant. We moved straight to small meetings with the Quality team to plan how to implement the project on the ground. From this and the experience of two TESTA‟d programmes being revalidated, we now have a clear idea of how the process will work in practice. We are currently finalising who will be involved in data collection and analysis, as this will place the limit on the number of programmes that can be put through the process each year.

Outcomes

Intended:

The process will become embedded in the programme revalidation procedures. Increased institutional focus on programme level student experience, and the

importance of assessment in student learning. The beginnings of a closer working relationship between Learning &Teaching and the

Quality Management Office.

Unintended:

The immediate impact that working closely with the Quality Management Office has had – we are now meeting routinely, and starting working on other initiatives together.

The inclusion of the initiative in the implementation plan of the planned Academic Development Strategy (a combined L&T and Quality strategy) for 2011-14, as a clear marker demonstrating institutional commitment to the aspirations of the project.

Impact

on staff:

There is a closer working relationship between different departments (LTDU and Quality), meaning that other initiatives are starting to develop. By its inclusion in the Academic Development Strategy more staff will become aware of the initiative, and all programme leaders will know about it. It is envisaged as more and more programmes use the EQUIP/TESTA process of gathering evidence to inform revalidation, that this will become a key institutional method of highlighting the importance of the programme level student experience to staff.

on students:

So far, students have not yet been impacted by this initiative, although those that are on programmes that have been revalidated as test cases in the initiative will be impacted as these programmes are rolled out in 2011/12. However, it is intended that in the long term it will have a major impact on their experience, both through their involvement in feeding back and changing procedures that affect them, and on their experiences of assessment and thus their learning as a whole.

on the HEI

Via the dissemination of the TESTA project we are able to discuss how we are embedding the process through EQUIP. It is envisaged that there will be further dissemination work on EQUIP specifically once it is fully implemented.


A key issue has been identifying who will do the data collection and analysis, as this is the limiting factor on the number of programmes that could be looked at in a year. We have agreed in principal with Deans of Faculties that the current faculty-based Research-informed Teaching project officers can do this data collection. This has yet to be finalised. However, it is hoped that there will be enough capacity to do three programmes per year per faculty, so that 12 programmes per year would go through the process.

An important challenge which has not been completely overcome is making sure that Programme Leaders are won around to being involved in the process. As eight programmes at Winchester have undergone TESTA, and increasingly they are using this data for revalidation in cycles as they come up (to our knowledge five programmes at Winchester are doing this), and there is much positive institutional discussion about the “TESTA Effect”, there is not a high risk of programme leaders being unwilling to participate.

Lessons learned

Personal level

Fiona: I‟ve enjoyed the EQUIP residential and meeting, and talking to colleagues from across the UK. I was impressed at the way what seemed like very disparate projects share many common characteristics in terms of implementation – which is I guess the purpose of the EQUIP programme!

Carol Smith: The space that the residential gave to allow us to work as an uninterrupted team was immeasurable.

Professional level

Fiona: I‟ve learned that good ideas will shine through, and that having an evidence based claim is a powerful force for change, especially when that evidence is local, recent and can be seen to have an immediate impact.

Carol Smith: It has reinforced my strong belief that is worthwhile doing things „on top‟ of all the others pulls on your time. I know that this process will facilitate the experience of other students and staff.

Team level

The activities at the EQUIP residential in particular showed how well we can work together as a team, and these will continue to be tested as the project rolls out. We are also conscious that the team will expand and therefore change as people involved in the data collection join.

Institutional level

Working across departments, especially when it involves interfering with others‟ processes, can be challenging, but solutions can be found when ideas are communicated well, and become shared goals.

Future Work

Short Term

We defined the short term as by the end of 20010/11. By this time everyone involved in the EQUIP process will know how it impacts on them and how they will contribute.

Medium Term

In 3 years‟ time the programme will have been rolled out and embedded. Approximately 30 undergraduate programmes will have gone through the EQUIP process as they go through revalidation. The project will have been disseminated through publications and presentations, and will contribute to the wider movement towards looking at assessment at a programmatic, rather than modular, level. It will be evaluated via feedback from students (e.g. NSS results), programme leaders (various committees), and QAA Institutional Review.

Long Term

8 years‟ time – most of the undergraduate programmes will have gone through the EQUIP process and be completely embedded in institutional culture, with evaluation similar to that outlined above.


At the beginning of the EQUIP process we had a set of data that everyone thought was very interesting and academically had a lot to contribute to debates about assessment. Finding a way of using the results of the TESTA project to make changes at an institutional level seemed a much greater challenge than gaining excellent reviews of the research project, because it involved changing processes that were out of our control and that responded to the standards set by external agencies. Our key message is therefore very simple: change is possible, and will be taken up, even enthusiastically, by others if it is communicated effectively and you can create a shared set of goals.

Assessment Strategies and student degree attainmentUniversity of Worcester


This project took place at the University of Worcester within two of its constituent academic institutes, namely the Worcester Business School (WBS) and the Institute of Sport and Exercise Science (ISES). The University is a relatively small and new university. There is limited provision of higher education within the University’s hinterland and widening participation amongst groups under-represented in HE is a significant aspect of our mission.

Within the institutes taking part in the project the proportion of students obtaining good (1 and 2:1) degrees has historically been lower than that for other subject areas at the University. This project initially sought to make use of a range of evidence to ascertain the reasons for relatively low performance in assessment outcomes and to inform the development of improvement strategies. As the project developed we refocused the project less on assessment itself and more on student engagement with assessment.


To help students to increase their self-efficacy in relation to assessments and therefore improve the likelihood that they will achieve the results that they are capable of.

Description

Differing approaches were taken within the two institutes involved. Within the Institute of Sport and Exercise an approach was taken based on goal orientated tutorials. Such an approach was consistent with the coaching philosophy that is taught within the Institute. Within the Worcester Business School the intervention was based on raising the reflexivity of students on their own assignment work by using a variety of self-assessment techniques. The intention with each of these is that they would involve relatively simple mechanisms in order to be more likely to engage both students and staff by not being too time-intrusive. The main vehicle eventually used in WBS was a ‘Student Feedback Coversheet’, which enabled students to identify the strengths and weaknesses of their own work and identify specific areas where they felt feedback would be most helpful. Students were assured that their comments would not adversely affect grading and that the use of this approach was endorsed by the National Students Union. The intervention took place in both semester 1 and 2. In addition to these interventions the EQUIP team had regular meetings so as to enhance the level of information exchange and networking between the institutions involved.

Outcomes

Intended

We have a better understanding of how some specific interventions can enhance the achievement of students. We increased, to some extent, the level of cross-institute dialogue. We have successfully involved students in helping to design interventions. We have engaged staff in dialogue on how enhanced engagement can generate better student outcomes, and that the engagement is an outcome of both staff and student intervention.

Unintended

The intervention, during semester 1, was not widely adopted in WBS by students, in that relatively few handed in the feedback cover sheet (5%), although in discussion substantially more claimed to have engaged. Students who did not complete the form seem to poorly understand its purpose and we need to explore how this meta-learning approach is better integrated into the assessment process. However, the semester 2 (second) intervention with many of the same students, delivered a much improved response rate of 44% when the exercise was presented as ‘compulsory’. The returns also indicated an ability to evaluate learning in a meaningful way, though students were less skilled in being able to identify what specific feedback would be helpful.

In respect to the Institute of Sport, an unintended outcome is that the project has stimulated our thoughts about how we enquire into student engagement. This has led to the development of an appreciative inquiry based version of the NSS.

There have been presentations of the outcomes of the ISES side of the project to the University of Worcester Learning and Teaching Conference. In addition a paper has been accepted at the EDULEARN 2011 conference (Barcelona).

Impact

on staff:

With respect to the intervention on ISES, feedback from the staff involved was that it had been a very valuable process and that following the intervention, students seemed more focused during lectures and were more aware of what they needed to do in order to achieve their goals.

on students:

For the intervention in ISES, most students spoke positively about the opportunity to have a goal-focused meeting with their tutor. Preliminary data indicate that following the intervention more students progressed from year one to year two and that module grades have increased. Preliminary data for the WBS indicate that there has been no change in module results as a result of the intervention. This is not surprising given the relatively low number of students that engaged. The intervention was not readily adopted by students; however it did push many students to engage in self-evaluation, which in the long term is likely to be positive.

on the HEI:

The institution has undertaken a review of its tutorial system, the project has fed into this review and the lessons learnt are being incorporated into the new system.


The most significant moment of the project was the realisation that in considering the question of ‘how do we increase the proportion of good degrees?’ we were being overly instrumental and that the more important question was how to encourage greater engagement and self-efficacy. Unfortunately for the WBS intervention, this realisation was probably too late and the intervention was possibly too instrumental. The most significant challenge was to try to describe a central concept that united the two projects. Both projects aimed to generate greater engagement, reflexivity and forward thinking from students about their assessments, however in retrospect the projects were

only weakly linked. A significant difference between the interventions was that in ISES was high investment, personal and face to face.

Lessons learned

The key lesson learnt was that it is possible to influence student behaviours and their own self-belief in relation to assessment but that this requires significant effort, resource and encouragement for students. The outcomes from the resource-intense approach used by ISES and the ‘quick’ fix used by WBS are rather striking; this is despite the fact that the method used by WBS is supported in the literature. It may be that the success of methods such as those used by WBS crucially depends on the local ways of working and possibly the nature of students that take these subjects. The approach taken by ISES was aligned to methods that are taught and espoused in that Institute and are therefore likely to be more efficacious. It is also noteworthy that we entered the EQUIP programme with a firm notion of the approach to be trialled in ISES and indeed some preliminary data to suggest its efficacy. With respect to WBS, plans were far less advanced, and this was the first attempt to use a co-ordinated intervention across a range of students. The considerable improvement between the levels of engagement shown with respect to semester 1 and semester 2 suggest that the approach may take time to embed and become accepted by the student and academic community. We should be cautious therefore in anticipating that enhancement projects will produce results rapidly or that outcomes from one context will readily translate into another.

Future Work

Short Term

We will disseminate the findings of the initiatives. We will use the understanding gained to inform the ways we undertake other projects. We will continue to monitor data to assess the impact of the project. We will continue to work to enhance the engagement and success of students with their assessments and wider studies.

Medium Term

We will publish guidance in relation to how to generate successful outcomes from learning and teaching change projects.

Long Term

The University will evolve its tutorial system using the lessons learnt during this EQUIP process.


Enhancement initiatives need to be aligned to the local culture. Published and evidence-based methods to enhance outcomes do not necessarily translocate to new situations and locations. Care needs to be taken to appreciate the circumstances in which the intervention was evolved and used and how these circumstances relate to the new situation to which they are to be applied.

Contact us

The Higher Education AcademyInnovation WayYork Science ParkHeslingtonYorkYO10 5BR

+44 (0)1904 [email protected]

© The Higher Education Academy, 2014

The Higher Education Academy (HEA) is a national body for learning and teaching in higher education. We work with universities and other higher education providers to bring about change in learning and teaching. We do this to improve the experience that students have while they are studying, and to support and develop those who teach them. Our activities focus on rewarding and recognising excellence in teaching, bringing together people and resources to research and share best practice, and by helping to influence, shape and implement policy - locally, nationally, and internationally.

The HEA supports staff in higher education throughout their careers, from those who are new to teaching through to senior management. We offer services at a generic learning and teaching level as well as in 28 different disciplines.

Through our partnership managers we work directly with HE providers to understand individual circumstances and

priorities, and bring together resources to meet them.

The HEA has knowledge, experience and expertise in higher education. Our service and product range is broader than any other competitor.

www.heacademy.ac.uk|www.twitter.com/heacademy

The views expressed in this publication are those of the author and not necessarily those of the Higher Education Academy. No part of this publication may be reproduced or transmitted in any form or by any means, electronic or mechanical, including photocopying, recording, or any storage and retrieval system without the written permission of the Editor. Such permission will normally be granted for educational purposes provided that due acknowledgement is given.

To request copies of this report in large print or in a different format, please contact the communications office at the Higher Education Academy: 01904 717500 [email protected]

The Higher Education Academy is a company limited by guarantee registered in England and Wales no. 04931031. Registered as a charity in England and Wales no. 1101607. Registered as a charity in Scotland no. SC043946.

The Higher Education Academy and its logo are registered trademarks and

should not be used without our permission.

contents web viewthe nature of the project ... academic staff are interested in the possibility of...

Documents