Visit Report on Buckingham Medical School

This visit is part of the 2016/17 visit cycle.

Our visits check that organisations are complying with the standards and requirements as

set out in Promoting Excellence: Standards for medical education and training.

Summary

Medical School Buckingham Medical School

University University of Buckingham

Programmes MBChB

Date of visits

Objective Structured Selection Examination (OSSE):

8 September 2016

Main visit:13-14 September 2016

Objective Structured Clinical Examination (OSCE): 6

January 2017

Second year qualifying OSCE: 21 February

Progression examination board: 1 March 2017

Key Findings

1 The visit carried out in September 2016 was the fifth

visit to Buckingham medical school (the school) since

agreeing to commit resources to begin the process of

a multi-year quality assurance review. We will

continue the rolling cycle of annual quality assurance

visits to the school following the first cohort of

students through to graduation and their first year of

practice.

2 At the time of our visit the school was in phase one

of its curriculum delivery with advanced preparatory

Buckingham Medical School final visit report for 2016/17

2

plans for the delivery of phase two. In line with the

school’s progression through the programme, the

school is growing and currently has 79 students in

year one, 60 students in year two and is planning to

recruit another 85 students commencing in January

2017.

3 During our main visit we met with year one and two

students and a range of staff from different teams

across the school. We observed several teaching

sessions and we visited Milton Keynes University

Hospital NHS Foundation Trust (MKH).

4 The school has made considerable progress since our

last visit in addressing their open requirements and

recommendations. We have therefore closed two

requirements and three recommendations and noted

that two recommendations are partially met. One

requirement and three recommendations remain

open from previous cycles for further monitoring.

5 We have identified an area of good practice and 15

areas that are working well in the school. We found

that students are positive about their experiences at

the school and that staff at the school and MKH are

dedicated and enthusiastic.

6 However, we have also identified requirements and

recommendations for the school to address,

including: streamlining reporting mechanisms from

local education providers (LEPs) to the school,

accelerating the appointment of educational

supervisors for phase two, enhancing the training for

simulated patients and examiners in the OSSEs,

improving communication with students around

clinical placements and encouraging the school to

continue to evaluate their assessment processes.

3

Areas of good practice

We note good practice where we have found exceptional or innovative examples of work

or problem-solving related to our standards. These should be shared with others and/or

developed further.

Number Theme Good practice

1 Theme 3 (R3.2) We heard from all of the groups that we spoke

to about the responsiveness of the school and

the high level of educational and pastoral

support available to students.

Paragraphs 58-62

Areas that are working well

We note areas where we have found that not only our standards are met, but they are

well embedded in the organisation.

Number Theme Areas that are working well

1 Theme 1 (R1.17) The school promotes a culture of learning and

collaboration with other healthcare professions

in the delivery of the curriculum as students

have adequate opportunities to learn from

other healthcare professionals.

Paragraphs 20-22

2 Theme 1 (R1.21) The learning environment is positive with

robust structures in place for educational

support. Students commended the high quality

of support available at the school and noted

high levels of accessibility to teachers and

tutors.

Paragraphs 28-29

3 Theme 2 (R2.1) The school has an effective system of

educational governance which is responsive to

feedback from both learners and educators.

Paragraphs 30-32

4 Theme 2 (R2.3) We note that there has been a significant

4

improvement in the way in which the patient

and public group is constituted and run. The

group is engaged and motivated.

Paragraphs 34-35

5 Theme 2 (R.2.8) We were impressed with the strong working

relationship that the school has fostered with

Health Education England working across

Thames Valley in order to enable their

students to access the Professional Support

Unit.

Paragraphs 45-46

6 Theme 2 (R2.20) The OSSE stations tested a wide range of skills

and attributes which we note are blueprinted

against Good Medical Practice.

Paragraphs 53-56

7 Theme 2 (R.2.20) The school’s selection processes are open and

transparent.

Paragraphs 53-56

8 Theme 2 (R.2.20) The OSSE was well organised, with a high level

of staff involvement and commitment.

Paragraphs 53-56

9 Theme 3 (R3.7) Students were well prepared for the format of

their OSCE. It is evident that they have been

appropriately briefed and supported.

Paragraph 70

10 Theme 4 (R4.1) Educators are well trained and supported in

their roles to meet their education and training

responsibilities.

Paragraphs 75-77

11 Theme 5 (R5.3) Small group teaching is working well and was

commended by students.

Paragraph 88

12 Theme 5 (R5.5) The OSCE stations tested a wide range of skills

and attributes which we note are blueprinted

against outcomes for graduates.

5

Paragraphs 99-101

13 Theme 5 (R5.6) The school’s investment in assessment and

high level of analysis is to be commended.

Paragraph 114

14 Theme 5 (R5.8) We found the OSCE to be well organised, with

a high level of staff involvement and

commitment.

Paragraphs 115 -116

15 Theme 5 (R5.12) We observed clear evidence that reasonable

adjustments have been implemented for the

OSCE by the School.

Paragraphs 117-118

6

Requirements

We set requirements where we have found that our standards are not being met. Each

requirement is:

targeted

outlines which part of the standard is not being met

mapped to evidence gathered during the visit.

We will monitor each organisation’s response and will expect evidence that progress is

being made.

Number Theme Requirements

1 Theme 1 (R1.3) The school must ensure that systems are in

place so that any concerns reported through

placement providers are forwarded in a timely

fashion to a known individual in the LEP who is

responsible for relaying concerns back to the

school.

Paragraph 7-9

2 Theme 4 (R4.2) The school must expedite the appointment of

educational supervisors for phase two to

ensure they are trained and that their job

plans reflect their new role.

Paragraphs 78-80

7

Recommendations

We set recommendations where we have found areas for improvement related to our

standards. They highlight areas an organisation should address to improve, in line with

best practice.

Number Theme Recommendations

1 Theme 2 (R2.20) In OSSE settings, consideration should be

given to further develop consistency in the

simulated patients’ performance, examiners’

interactions with students, examiner training in

relation to the nature of the global score and

also the reliability of the selection event.

Paragraph 56-57

2 Theme 3 (R3.7) The school should improve communication

with current and future cohorts to ensure that

students receive timely and accurate

information about their clinical placements.

Paragraphs 67-69

3 Theme 5 (R5.6) The school should continue to evaluate their

assessment processes through the ongoing

assessment review.

Paragraphs 109-114

8

Update on open requirements and recommendations

Open requirements update Status

1 The school must improve the

provision of inter-professional

learning and explore more

opportunities for students to

learn from, and with, other

healthcare professionals and

students.

It is evident that the school has

been working hard to address this

requirement through the

implementation of the inter-

professional learning strategy.

During the visit we identified that

students have adequate

opportunities to learn from other

healthcare professionals as an area

that is working well. In addition,

we found that the school promotes

a culture of learning and

collaboration with other healthcare

professions.

See R1.17 paragraphs 20-22

Closed

2 Given its relatively small size, the

school must ensure the

committee structure excludes

those formally involved in student

support and progress from

making fitness to practise

decisions.

The school has demonstrated that

student support and the Fitness to

Practise Committee are

independent bodies with different

roles. The Fitness to Practise

Committee has no common

membership with the Concerns

Group and includes several

external members. We are

therefore happy that this

requirement has been met and can

therefore be closed.

See R3.2 paragraph 64

Closed

3 The school must ensure the

contingency plans with the

University of Leicester Medical

School continue to be feasible.

The Buckingham curriculum must

align with the new Leicester

curriculum to ensure no students

The school has been working

towards meeting this requirement,

but have noted it is not possible

for them to maintain an exact

alignment with Leicester over the

next three years, as the new

course at Leicester is implemented

Open

9

would be placed at a

disadvantage in terms of

assessment.

in stages. A transfer of students

from Buckingham to Leicester in

phase one would not put students

at a disadvantage. However, we

were made aware that changes to

the Leicester phase two curriculum

are greater and the finer details

are less well developed. We note

that while parts of the curriculum

are similar others are not and

acknowledge that alignment is

challenging. However, potential

issues with transfer points have

been noted and are on the school’s

radar. We will therefore continue

to monitor this item during future

QA cycles.

See R5.3 paragraphs 89-90

Open recommendations update Status

1 We would like to see evidence of

how the medical school building

and the new education centre will

accommodate two cohorts of

students.

During the visit we found that

while the medical school building is

crowded during lesson

changeover, there is adequate

space for teaching.

Building work on the education

centre at MKH was due to

commence in August 2016, with an

occupation date of September

2017. At the time of our visit in

September 2016, the site had been

cleared but building had not yet

begun. The school told us that

contingency plans are in place for

March to August 2017 when phase

two students will be on placement

at MKH. We will continue to

monitor this recommendation.

See R1.19 paragraphs 23-26

Open

10

2 We recommend to set out clear

guidance of the thresholds that

would trigger fitness to practise

procedures

The school has been working to

refine their FtP procedures and

thresholds in advance of receiving

a case. Since our last visit

Buckingham has considered GMC

guidance on Professionalism and

has incorporated it into their FtP

training and procedures and they

now have a framework by which

evidence and reports from

hearings will be collected and

presented. We will keep this item

open for further monitoring until

the process has been utilised more

to ensure changes are sustainable.

See R2.18 paragraph 52

Open

3 We recommend that in the OSCEs

the school requests examiner

written feedback for all students,

not only for the ones assigned a

failing grade

During the year two OSCE we

found that examiners now

complete written feedback for all

students. Therefore, this item can

now be closed.

See R3.13 paragraph 73

Closed

4 The school should increase the

number of options available as

student selected components

(SSCs) and provide clearer links

to how they meet the learning

outcomes defined in Tomorrow’s

Doctors

During the 2015/16 cycle of visits

the visit team concluded that this

recommendation was partially met

as the school has mapped all of its

SSCs to the learning outcomes in

Tomorrow’s Doctors and we noted

that we will continue to monitor

the increase in SSC options.

Since our last visit it is clear that

the school has been working

towards meeting the rest of this

recommendation through working

on increasing the number of SSCs

available to their students. The

school currently offers eight taught

SSCs and 73 library projects.

Additionally, there are five new

SSCs in development with further

proposals for new units under

Partially

met

11

scrutiny. We will continue to

monitor this recommendation.

See R5.3 paragraphs 91-94

5 The school should develop robust

procedures to manage patient

participation at general practices,

as they will be heavily relied upon

during the first year.

The 2016/17 cycle of visits found

no concerns in relation to

procedures around patient

participation at general practices.

The second year students that we

met told us that their GP

placements are going well, they

are well organised and they are

enjoying the narrative medicine

strand. In addition, the school

noted that there has been no

evidence of outstanding issues

from their quality information. This

recommendation can therefore be

closed.

See R5.3 paragraph 95

Closed

6 We understand the small group

activities will be evaluated - and

we recommend careful

consideration is given to how

often the groups should be

changed

The school has taken steps to

evaluate their small group

activities. Student representatives

were asked to consult informally

with the student groups to identify

initial thoughts and feelings

amongst the student body. At the

time of our visit in September 2016

the school planned to formally

consult with the student body

regarding potential changes to the

small activity groups, aiming to

implement an outcome in early

2017. We will therefore follow up

on this recommendation in the

next cycle of visits.

See R5.3 paragraphs 96

Partially

met

7 The ratio of new questions and

tested questions should be

reviewed in the initial runs of the

assessments throughout the

programme.

The school has been addressing

this item and has generated large

numbers of questions for each

assessment and these are

progressively filing their question

bank, such that Leicester questions

Closed

12

are now seldom used. The school

has quality systems in place,

including external scrutiny of their

assessments. Overall the

psychometric analysis of written

and OSCE examinations

constructed with the school’s own

items are satisfactory and the

school is responding to the few

questions with less favourable

psychometric characteristics. We

are therefore satisfied that this

item has now been met.

See R5.6 paragraph 103

8 In an OSCE setting, consideration

should be given to further

develop consistency in the

simulated patients’ performance,

examiners’ interactions with

students and in the calibration of

students’ marks/outcomes.

The school has taken steps to

address standardisation and has

developed an examiner briefing

clarifying that simulated patients

should not contribute to marking in

any way. The school has also

taken steps to develop consistency

in examiners’ interactions with

students and we note the

calibration exercise is a welcome

development.

Although we acknowledge that the

school has taken steps to address

the issues, this item will be kept

open for further monitoring.

See R5.6 paragraphs 104-108

Open

13

Findings

The findings below reflect evidence gathered in advance of and during our visit, mapped

to our standards.

Please note that not every requirement within Promoting Excellence is addressed. We

report on ‘exceptions’, e.g. where things are working particularly well or where there is a

risk that standards may not be met.

Theme 1: Learning environment and culture

Standards

S1.1 The learning environment is safe for patients and supportive for learners and educators. The culture is caring, compassionate and provides a good standard of care and experience for patients, carers and families.

S1.2 The learning environment and organisational culture value and support education and training so that learners are able to demonstrate what is expected in Good medical practice and to achieve the learning outcomes required by their curriculum.

Raising concerns (R1.1) Dealing with concerns (R1.2) Learning from mistakes (R1.3)

7 The school is making good progress towards meeting these requirements. We found

that there is transparency around raising concerns at both the school and MKH.

Students are encouraged to raise any concerns relating to patient safety or education

and training and told us that they feel comfortable in doing so. Moreover, students

are aware of how to report a patient safety concern and understand their

responsibility to do so.

8 The senior management team at MKH told us they have processes in place for

students to raise concerns about patient safety or education and training. Students

are encouraged to speak to a clinician in the first instance, but if this is not possible

then students are encouraged to raise a DATIX. The school told us they have an

agreement in place with MKH which stipulates that all concerns raised by students

must be relayed back to the school. MKH confirmed they send all DATIX reports from

students back to the school.

9 Although we found the foundations for raising concerns are in place at both the

school and MKH, we identified that concerns raised at MKH are not collated by a

known individual at the trust who is responsible for reporting concerns back to the

school. We encourage the school to work with its placement providers to ensure that

feeding concerns back to the school is streamlined.

Requirement 1: The school must ensure that systems are in place so that any

concerns reported through placement providers are forwarded in a timely fashion to a

14

known individual in the LEP who is responsible for relaying concerns back to the

school.

Seeking and responding to feedback (R1.5)

10 During the visit we heard from a number of individuals about how the school

encourages and responds to feedback from learners and educators. We also heard

about the subsequent changes.

11 The quality management team told us that the quality unit design feedback forms for

students to complete. In order to do this education provider unit (EPU) leads are

asked to suggest three questions that are specific to them. Each EPU receives their

own individual feedback but there is also an overview of feedback to allow EPUs to

see how their feedback sits in comparison to others. Additionally, the quality

management team noted they have changed the order of some of the units in

response to student feedback.

12 Academic teachers confirmed that students complete feedback forms in order to

identify areas for development and supervisors at MKH told us they also have the

opportunity to feedback to the school on the delivery of the curriculum. The

curriculum team noted the school has regular meetings with placement providers to

scrutinise student feedback and produce a formal action plan.

13 The patient and public group told us they receive student feedback on their sessions.

They added the process is evolving and that sessions are designed in response to

student feedback. We were also told the patient and public group feel confident in

putting forward their suggestions and feel listened to by the school.

14 The selection team noted that following the school’s selection events, the team meet

in order to identify how stations can be developed. Additionally, during our

observation of the OSSE, the school provided evidence that they collect feedback

from candidates and noted they plan to collect feedback from current students.

15 Finally, students confirmed they have ample opportunities to provide feedback to the

school, and year two students told us there is a student forum whereby students can

raise concerns and give their feedback to the school on a wide range of topics

including: academic, facilities, placement and assessments.

Appropriate capacity for clinical supervision (R1.7) & appropriate level of clinical

supervision (R1.8)

16 During our meeting with the senior management team at MKH it was noted that while

the trust has some hard to recruit to posts in certain specialties, they have

appropriate supervision plans to cover these posts. We were also told the clinical staff

body at MKH has grown significantly, meaning there is adequate support for

students.

15

17 The year two students we met told us their placements at MKH are enjoyable as

consultants are welcoming, supportive and keen to teach. The students did not raise

any concerns about supervision or working beyond their competency.

Induction (R1.13)

18 Upon arrival at the school students undergo an induction week in order to fully

introduce them to the course. Induction week covers a wide range of topics including:

careers, student support and fitness to practise.

19 Senior management at MKH told us that they are developing their induction

programme with the school. We were told that most of the induction is face-to-face

and that students have the opportunity to meet with the chief executive (CEO) during

their induction. Key topics including undermining and bullying and support are

explored during student induction at MKH.

Multiprofessional teamwork and learning (R1.17)

20 The school has an open requirement relating to this standard which can now be

closed. Previously we identified that the school must improve the provision of inter-

professional learning and explore more opportunities for students to learn from, and

with, other healthcare professionals and students.

21 It is evident that the school has been working hard to address this requirement

through the implementation of their inter-professional learning strategy. The key

features of this strategy are a set of experiences for students in phase one, plus a

minimum of one inter-professional learning event in each main block in phase two,

and a requirement to reflect on multi-professional working repeatedly in the e-

portfolio.

22 During the visit we identified that students have adequate opportunities to learn from

other healthcare professionals as an area that is working well. We observed a

procedural skills session for year one students which was led by two mid-wives, and

supervisors at MKH told us that students receive input into their learning from a

range of healthcare professions. Students also told us about an inter-professional

learning day which aims to cover working in a multi-disciplinary team. We were told

that there is a range of healthcare workers at these days, including diabetes nurses,

physiologists and radiographers.

Area working well 1: The school promotes a culture of learning and collaboration

with other healthcare professions in the delivery of the curriculum as students have

adequate opportunities to learn from other healthcare professionals.

16

Capacity, resources and facilities (R1.19)

23 The school has an outstanding recommendation relating to this requirement. During

our last visit we highlighted that we would like to see evidence of how the medical

school building and the education centre at MKH will accommodate two cohorts of

students.

24 The school is currently in partnership with MKH to build a new building for education

purposes at MKH. Work was due to commence in August 2016, with an occupation

date of September 2017. At the time of our visit in September 2016, the site had

been cleared but building had not yet begun. The school told us that contingency

plans are in place for March to August 2017 when phase two students will be on

placement at MKH. During our visit to MKH the senior management team advised

they have rented office space to house hospital back office functions such as, IT, HR

and finance, which in turn will make physical space for phase two students.

Additionally, during the visit we examined the physical space of the current education

centre, which while outdated is fit-for-purpose.

25 During the visit the senior management team told us the medical school building is

successfully accommodating two cohorts of students. We heard there is adequate

room to accommodate scheduled teaching but that there is limited space for private

study in the medical school building. We were also told the building is busy during

changeover times. Year two students supported this and told us that the medical

school building is busy but added it functions. They also told us the lack of private

study space in the medical school building is not an issue as they study in the library

or at home. Academic teachers told us despite growing pressures on office space in

the school that all of teachers have their own desk. They noted they have access to

meeting rooms should they want to meet with the students in private.

26 The senior management team at MKH told us they have no concerns about

placement capacity. MKH is also a placement provider for Oxford Medical School, but

it was noted there is no competition for resources between the different groups of

students as there is only a small number of Oxford students on site. Additionally, we

were told by the trust that they are confident they can accommodate increases in

Buckingham student numbers up to 96 students per cohort, and planning has begun

to accommodate this number of students. We will therefore continue to monitor this

recommendation until the education centre at MKH has been built and students are

occupying it.

Accessible technology enhanced and simulation-based learning (R1.20)

27 The curriculum team at the school confirmed that students will have access to

technology enhanced and simulation based learning opportunities during their time at

the school. They added that several units include simulation activities to solve clinical

problems; additionally there is also a simulation suite at MKH that students have

access to, to support their learning.

17

Access to educational supervision (R1.21)

28 The school’s personal tutor system ensures that students receive ongoing routine

support to facilitate their development. The personal tutor system is separate to the

concerns process and pastoral support group which are discussed in more detail

under R3.2. The school has 21 personal tutors who meet with students regularly in

order to oversee their personal and academic development, as well as their e-

portfolio.

29 During our visit students told us they meet with their personal tutor once a semester

but that personal tutors are accessible and responsive should students wish to

arrange additional meetings. Students noted the personal tutor system at the school

is very effective and they feel supported in their studies. They noted that reflecting

on their e-portfolio in meetings with their personal tutor is helpful. Additionally,

students with whom we met at MKH told us they are well supported on placements,

their personal tutors are accessible and they know who to contact should they have

an issue.

Area working well 2: The learning environment is positive with robust structures in

place for educational support. Students commended the high quality of support

available at the school and noted high levels of accessibility to teachers and tutors.

18

Theme 2: Education governance and leadership

Quality manage/control systems and processes (R2.1)

30 The school has operational and functional educational governance systems and

processes in place to manage and control the quality of education and training. EPUs

are a group of staff responsible for parts of the curriculum delivered in the university,

at an NHS or other body delivering clinical education and are responsible for quality

control. EPUs feed into the quality group responsible for quality management. The

risk registers and action plans from this group are combined into a live global

document and feed into the board of studies which is the body responsible for the

governance of quality for the programme.

31 The quality management team noted there is an understanding and engagement with

the school’s educational governance system amongst staff at the school. Additionally,

quality processes are continuing to develop in line with GMC developments as the

school’s risk register has been configured to match the standards outlined in

Promoting Excellence.

32 The school told us feedback is central to their quality management processes and

during our visits we heard examples of changes that have been made in response to

feedback. Please see R1.5 which details the different types of feedback that the

school collects.

Area working well 3: The school has an effective system of educational

governance which is responsive to feedback from both learners and educators.

Accountability for quality (R2.2)

33 The school is clearly demonstrating accountability for educational governance through

their processes and structures. The director of medical education (DME) holds

responsibility for quality of the MB ChB programme and is accountable to the vice

chancellor. Responsibility for quality management processes are delegated to the

quality unit which checks that each EPU has adequate quality control mechanisms in

Standards

S2.1 The educational governance system continuously improves the quality and outcomes of education and training by measuring performance against the standards, demonstrating accountability, and responding when standards are not being met.

S2.2 The educational and clinical governance systems are integrated, allowing organisations to address concerns about patient safety, the standard of care, and the standard of education and training.

S2.3 The educational governance system makes sure that education and training is fair and is based on principles of equality and diversity.

19

place. The body responsible for the governance of quality is the board of studies and

they convene at least 3 times a year. The board of studies reports to the Learning

and teaching committee and the university senate.

Considering impact on learners of policies, systems, processes (R2.3)

34 It is clear that the school takes into account the views of patients and the public

when considering the impact of systems, policies and processes. The patient and

public group noted they have been recruited to work with the students to give their

perspective as patients currently going through the healthcare system. The group has

grown since our last visit and now consists of 20 members. There is a diverse mix of

experience in the group and we found those in the group to be enthusiastic and

content with their roles.

35 The patient and public group told us processes are evolving in line with student

feedback. The group added they feel confident in raising their suggestions and

listened to by the school. In the 2013-14 visit report we set a recommendation that

the school should increase the level of patient and public involvement in quality

management. We note the progress the school has made in this area and we now

consider the patient and public group to be an area that is working well.

Area working well 4: We note that there has been a significant improvement in the

way in which the patient and public group is constituted and run. The group is

engaged and motivated.

36 As noted in R2.1, feedback from learners and educators is central to the school’s

quality functions and during the visit we heard examples of how the school takes into

account the views of learners and educators. This is discussed in more detail under

R1.5.

Evaluating and reviewing curricula and assessment (R2.4)

37 There are clear governance structures around the school’s curriculum and assessment

frameworks. The school’s curriculum is managed by the curriculum executive, led by

the DME and formally governed by the board of studies. Management of the school’s

assessment is the responsibility of the assessment lead and decisions about

assessment outcomes for individual students are made by the board of examiners.

38 During the visit we heard evidence that the school evaluates their curriculum and

assessment frameworks in order to drive improvement in the quality of education and

training. It was noted that the quality lead works with the curriculum and assessment

teams to ensure that the quality of the programme is monitored, reviewed and

evaluated. Additionally, the curriculum implementation group meets regularly to

coordinate curriculum delivery and make recommendations for enhancement. The

phase two implementation group focuses on the delivery of clinical education in

phase two and acts as an interface between education providers and the school.

20

39 Senior management and supervisors at MKH told us they have the opportunity to

feedback on how the curriculum is working on the ground. They noted there are

regular face-to-face meetings, as well as more formal meetings to discuss the

school’s curriculum and to ensure supervisors at MKH are well informed about

changes in the curriculum. As noted previously feedback is central to the school’s

quality functions of evaluating and reviewing the success of their program.

Collecting, analysing and using data on quality and on equality and diversity (R2.5)

40 The school has analysed all summative assessments in order to examine student

learning, identify common gaps in knowledge, misunderstandings and mistakes. They

also conduct analysis on assessment results in relation to protected characteristics

and socio economic factors.

Systems and processes to monitor quality on placements (R2.6)

41 The school has Service Level Agreements (SLAs) with current provider hospitals, MKH

and Bedford Hospital NHS Trust, which outline the level of service expected from the

education providers. At the time of our visit the school noted that they are engaging

in discussions around securing additional placements with Buckinghamshire

Healthcare NHS Trust and South Warwickshire NHS Foundation Trust. The

discussions are in advanced stages with Memorandum’s of Understanding in place.

The school is aiming to finalise SLAs with Buckinghamshire Healthcare NHS Trust and

South Warwickshire NHS Foundation Trust by the end of 2016.

42 During the visit the school told us about systems and processes in place to monitor

the quality of placements. One of the ways in which the school monitors quality is

through their quality review visits. Frequency of these visits is determined by a risk

based assessment, and visits to LEPs will normally happen every year, but there must

be one at least every two years. Following a visit, a report and action plan is

produced.

43 The visiting process to monitor general practices is different in that visits are scaled

to reflect the greater number of general practices in comparison to LEPs. General

practices will be visited on average every five years and likewise with the quality

review visits to LEPs a report and action plan will be produced following a visit.

44 Outside of quality review visits, senior management at MKH noted there is a good

flow of information about quality matters between the trust and the school. The

quality management team at the school told us there is a quality management

meeting every month at the trust, which the school quality lead attends.

Sharing and reporting information about quality of education and training (R2.8)

45 Senior management at MKH noted they have a good working relationship with the

school and communicate on a day- to-day basis at all levels. MKH formally engage

21

with the school through the curriculum governance process, a joint executive

committee and joint quality committees. It is through these groups that information

pertaining to the quality of education and training at the trust is shared with the

school.

46 The school also told us they are continuing to develop their relationship with Health

Education England working across Thames Valley (HEETV). The school has an active

SLA with HEETV for students to access careers guidance and the Professional Support

Unit, and to play a part in the broader support of student learning. The school added

that HEETV services are utilised by their students and students confirmed they are

aware of the services provided by HEETV.

Area working well 5: We were impressed with the strong working relationship that

the school has fostered with Health Education England working across Thames Valley

in order to enable their students to access the Professional Support Unit.

Collecting, managing and sharing data with the GMC (R2.9)

47 The school is transparent in their reporting to the GMC. They have adequate

governance systems in place to collect and manage all the necessary data required by

the GMC. They have begun submitting a medical school annual return detailing

quality information and they are responsive to all other document requests from the

GMC.

Systems and processes to ensure a safe environment and culture (R2.11) & Educators for

medical students (R2.13)

48 The school has systems and processes to ensure that learners have appropriate

supervision. They hold SLAs with each of their education providers which outline that

each student should have named educational and clinical supervisors in each block.

The phase two implementation group has approved a set of standards for educational

supervision in line with Promoting Excellence outlining what the school expects from

educational supervisors in LEPs. Likewise, the school has also developed a set of

standards for clinical supervision which are also tailored to Promoting Excellence.

49 The school follow up on access to appropriate clinical and educational supervision

through their quality processes, such as quality review visits and also through

feedback from learners and educators.

Managing concerns about a learner (R2.16)

50 The school has systems in place to identify, support and manage concerns about a

learner. Senior management at the school noted that student support is active and

quality management systems are functional in identifying students who require

additional support. Concerns can be identified through a variety of ways including:

attendance, performance in assessment and concerns raised by staff and students.

22

Once a concern has been identified the concerns group coordinates support for

students and the pastoral support team delivers it either directly or by referral to

specialist providers. There is a low threshold for referral in order to enable early

identification of students in potential difficulty. Please see R3.2 for a more

comprehensive explanation of resources the school provides to support the health

and wellbeing of their learners.

Sharing information of learners between organisations (R2.17)

51 We were assured the school has a transfer of information system in place to

adequately share information about a learner between sites. Student support told us

transfer of information forms are generated by the support lead, signed by the

student and occupational health physician and then passed to the LEPs. Senior

management at the school told us that transfer of information has been

communicated to the LEPs in preparation for phase two, and senior management at

MKH confirmed there is an adequate transfer of information process in place and

communication flow is working well.

Requirements for provisional/full registration with the GMC (R2.18)

52 During the 2015/16 cycle of visits we recommended the school should set out clear

guidance on the thresholds that would trigger fitness to practise procedures. Before

the visit the school told us they have been working to refine their FtP procedures and

thresholds in advance of receiving a case. Since our last visit the school has

considered GMC guidance on Professionalism and has incorporated it into their FtP

training and procedures. They now have a framework by which evidence and reports

from hearings will be collected and presented. We will keep this item open for further

monitoring until the process has been utilised more to ensure changes are

sustainable.

Recruitment, selection and appointment of learners and educators (R2.20)

53 The school’s selection processes operate in two stages. The first stage is an academic

selection and the second is the OSSEs, which are a series of stations with different

tasks to test abilities related to Good Medical Practice. During our OSSE observation

we found that the ten stations were varied and tested a diverse range of skills. It was

also apparent the school has blueprinted the stations against Good Medical Practice.

We therefore noted this as an area working well.

Area working well 6: The OSSE stations tested a wide range of skills and attributes

which we note are blueprinted against Good Medical Practice.

54 The OSSE commenced with assessor and candidate registrations, before leading into

a comprehensive assessors briefing and station allocation. As part of the process

parents and partners of candidates have the opportunity to have an informal chat

23

with the school to ask any questions they may have and also tour the campus and

medical school once candidates have finished their examination.

55 We observed the school’s selection process to be open and transparent. The process

was communicated clearly to all and assessors had an in-depth briefing before

commencing. The rotation around the OSSE stations also accommodated candidates

who required reasonable adjustments of extra time. Additionally, the OSSEs were well

organised and ran smoothly and there were ample staff committed to excellence and

delivering the OSSEs successfully. We therefore identified below as areas that are

working well.

Area working well 7: The School’s selection processes are open and transparent.

Area working well 8: The OSSE was well organised, with a high level of staff

involvement and commitment.

56 Despite observing areas of the OSSEs working well, we observed inconsistencies with

the role of the simulated patients involved in the examinations. During some of the

stations assessors and simulated patients consulted when scoring candidates, while at

other stations assessors did not consult with simulated patients. We also observed

that some examiners interacted with candidates after a station had finished whilst

others did not and that there were discrepancies in examiners’ understanding of the

nature of the global score.

57 The school reviews their selection processes each time they are run. The selection

team told us stations are developed and candidate feedback is analysed with the aim

of making improvements. Psychometric analysis is also completed following each

selection cycle. Previous psychometric reports highlight that Cronbach’s Alpha for the

ten stations is lower than desired. The school has recently increased the number of

OSSE stations to ten but they acknowledge that their Alpha is below the 0.80 desired

thresholds for reliability. For the 2016 intake, 225 candidates sat the OSSEs over five

days. The school has reported diversity in Cronbach’s Alpha across the five days and

therefore plan to consider reasons for stations performing poorly across the different

selection days. While the visiting team accepts that achieving a value greater than

0.80 for Cronbach’s Alpha for a new assessment can be challenging, we encourage

the school to continue to develop this assessment to maintain its aims while also

improving its reliability. We have therefore set a recommendation for the school to

address.

Recommendation 1: In OSSE settings, consideration should be given to further

develop consistency in the simulated patients’ performance, examiners’ interactions

with students, examiner training in relation to the nature of the global score and also

the reliability of the selection event.

24

Theme 3: Supporting learners

Standard

S3.1 Learners receive educational and pastoral support to be able to demonstrate what is expected in Good medical practice and achieve the learning outcomes required by their curriculum.

Learner's health and wellbeing; educational and pastoral support (R3.2)

58 Throughout our visit we were told about the excellent educational and pastoral

support the school offers by both students and staff. The school told us that in order

to deliver highly effective support to students that two main groups interact; the

concerns group and the pastoral support team. The concerns group coordinates all

support for students for concerns such as academic and health or conduct. Concerns

can be identified through a variety of ways including: attendance, performance in

assessment and concerns raised by staff and students. The concerns group does not

deliver support directly, but develops a structured action plan which is implemented

through the pastoral support team. The pastoral support team provides support

either directly or by referral to specialist providers.

59 Referrals to specialist providers include:

Occupational health services at MKH

coaching and mentoring services by the pastoral team

learning support including an assessment by an educational psychologist

academic skills support

direction to the big white wall service, which is an anonymous online platform

guided by trained professionals aiming to help those with mental health

conditions through seeking support from each other by sharing their

experiences

specialist services provided through St Andrew’s outpatient clinic and HEETV.

60 In addition to the school providing their own support services, and as mentioned in

R2.8, students have access to careers guidance and the Professional Support Unit at

HEETV. HEETV services are utilised by Buckingham students and during the last

academic year HEETV saw a number of Buckingham students for careers guidance

and training in language and communication.

61 As well as signposting students to HEETV career services, the school has developed

their own career advice initiatives. Students told us about an interactive lecture on

career options they received. They added staff at the school are very supportive in

helping the students explore their interests. Additionally, we were told student

25

support has created a Moodle page for careers which details information such as:

careers fairs and workshops, professional development conferences and blogs from

various students.

62 Overall, we identified this level of support as an area of good practice. The school

offers a wide range of high standard support to students. We heard consistently from

students that educational and pastoral support at the school is excellent. They told us

staff are accessible, responsive, approachable and that they feel comfortable raising

any issues with them.

Good practice 1: We heard from all of the groups that we spoke to about the

responsiveness of the school and the high level of educational and pastoral support

available to students.

63 We identified student perception of the concerns group is that it is a group aiming to

help students overcome any issues they may be having. The students told us they

feel comfortable accessing the group and are clear the group is wholly supportive and

independent of FtP processes.

64 During the 2015/16 cycle of visits we set a requirement that the school must ensure

the fitness to practise committee structure exclude those formally involved in student

support and progress from making fitness to practise decisions. The school has

demonstrated that student support and the fitness to practise committee are

independent bodies with different roles. The fitness to practise committee has no

common membership with the concerns group and is comprised of several external

members. We are therefore happy that this requirement has been met and can

therefore be closed.

Undermining and bullying (R3.3)

65 When asked about undermining and bullying, supervisors at MKH told us the trust has

a policy on undermining and bullying and they have no examples of such behaviour.

They told us staff and students are able to raise their concerns without fear of

adverse consequences and students are introduced to the policy during their

induction.

Student assistantships and shadowing (R3.6)

66 The school told us they are planning for four week student assistantships in a

secondary care environment in the final year and they will commence shortly after

electives finish. We will further explore the school’s planning for assistantships in the

2017/18 cycle of visits.

26

Information about curriculum, assessment and clinical placements (R3.7)

67 During the visit year one students told us the school has not communicated where

their clinical placements will take place during phase two. They added as there are no

year three and four students in the school they do not know where they are likely to

be placed for their clinical placements. They noted that this situation is causing

difficulties with arranging accommodation for coming years as they are expected to

sign annual housing leases when they do not know where they are going to be

placed.

68 As mentioned in R2.6, the school is currently engaging in discussions around securing

additional placements with Bedford Hospital NHS Trust and Buckinghamshire NHS

Trust and South Warwickshire Foundation Trust. They added once the school

confirms if these placements will go ahead or not then they will be able to confirm

where the year one students will be placed in phase two. They added they are hoping

for contracts to be signed by the end of 2016. In the meantime, all year two students

will be placed at MKH and year one students will be told where they are going for

clinical placements at the end of the first term in year two.

69 While we recognise there are fixed constraints on when year one students can expect

to find out where they are being placed in phase two, we encourage the school to

improve communication with current, as well as future cohorts to ensure students

receive timely and accurate information about their clinical placements.

Recommendation 2: The school should improve communication with current and

future cohorts to ensure that students receive timely and accurate information about

their clinical placements.

70 During our visit observing the year two OSCE we found the students to be well

prepared for the format of the exam. The school submitted documentation noting

students are informed about the OSCE in a variety of ways, including presentations

and on Moodle. They also shared the written brief that students receive before the

exam detailing information on OSCE organisation, OSCE marking and feedback and

general guidance for preparing for the exam. This exam operated smoothly and it

was evident that students had received accurate information prior to their exam.

Area working well 9: Students were well prepared for the format of their OSCE. It

is evident that they have been appropriately briefed and supported.

Out of programme support for medical students (R3.9)

71 The school told us a six week elective period will take place following the final

professional exam. Students then go on to complete their assistantship. Please see

R3.2 for further information on the support that the school offers. In future visits we

will explore the support offered to those students specifically on electives.

27

Feedback on performance, development and progress (R3.13)

72 During the visit we found learners are receiving regular, constructive and meaningful

feedback on their performance, development and progress. The facilitators we met at

MKH told us they give the students constructive feedback following tasks. They noted

they highlight specifics that may have been missed, areas for improvement, any

positives and address professionalism. We heard students also have the opportunity

to reflect on their own performance for development. Students at MKH told us they

are receiving enough feedback to enable their development and they can access

materials such as patient notes to enable their development.

73 Following the 2015/16 cycle of visits and observations, we set the school a

recommendation to request examiner written feedback for all students in their OSCE

and not only for those assigned a failing grade. During the year two 2017 OSCE we

found that examiners provide brief comments on the observation sheets regarding

student performance at the different stations. We are therefore satisfied that this

recommendation has been met and it can now be closed.

Career support and advice (R3.16)

74 During our visit the senior management team told us that the school’s exit degree has

been approved by the university. They noted that it makes up 360 credits and

students can opt to receive the degree at the end of phase one if they are not able to

complete a medical qualification.

28

Theme 4: Supporting Educators

Standards

S4.1 Educators are selected, inducted, trained and appraised to reflect their education

and training responsibilities.

S4.2 Educators receive the support, resources and time to meet their education and

training responsibilities.

Induction, training, appraisal for educators (R4.1)

75 The academic teachers we met told us they have received an induction to their role

and the curriculum. We were told the school has provided training in their roles in

areas such as, delivering group teaching and equality and diversity. It was made clear

that professional development is valued, with some of the academic teachers

highlighting they are studying for a postgraduate certificate in education. Additionally,

academic teachers told us they receive an annual appraisal with the DME.

76 The patient public group told us they have received numerous guidance documents

to support them in their roles. They noted they have had the opportunity to

undertake training in equality and diversity, and added they receive feedback from

students following their sessions. Some told us their sessions have been observed

and noted the feedback and observations contribute to their personal development as

they are able to develop and improve their sessions based on the feedback they

receive.

77 Supervisors at MKH supported the above and told us they have received training for

their roles. They added the trust fund and support those involved in education to

undertake additional training such as a masters and a PGCE.

Area working well 10: Educators are well trained and supported in their roles to

meet their education and training responsibilities.

Time in job plans (R4.2)

78 At the time of our visit the school did not have a full complement of educational

supervisors in post for the delivery of phase two. They added they are hoping to

recruit seven to eight educational supervisors per block and confirmed allocated

educational supervisors will be identified by the end of 2016 in order to undergo

training before starting in post for the delivery of phase two. Additionally, we were

told induction into the curriculum for new supervisors is planned and accounted for.

79 Undergraduate supervisors at MKH told us finding supervisors is a challenge and

supervisors do not all have formal time for teaching embedded into their job plans.

They noted time in job plans is not universal and highlighted discrepancies between

those with teaching responsibilities. Additionally, there is also an uncertainty around if

29

newly appointed supervisors will have adequate time allocated in their job plans for

teaching.

80 Whilst we recognise the school plan to have educational supervisors appointed by the

close of 2016, we encourage the school to expedite the appointment of educational

supervisors for phase two.

Requirement 2: The school must expedite the appointment of educational

supervisors for phase two to ensure they are trained and that their job plans reflect

their new role.

Educators' concerns or difficulties (R4.4)

81 Supervisors at MKH noted there is a process in place with the school to support

educators in dealing with learners in difficulty while on placement. Supervisors told us

if such a situation was ever to arise they would build up a picture of student

performance through documentation and relay this back to the school for support.

Academic teachers also noted they are kept updated on reported concerns regarding

students.

Working with other educators (R4.5)

82 Academic teachers told us they are encouraged to liaise with each other and work as

a team. They added academic teachers at the school support each other and also

interact with clinical teachers that teach at the LEPs the school works with.

Recognition of approval of educators (R4.6)

83 The school has met the formal GMC milestone for the recognition and approval of

educators. The senior management team at MKH supported this and told us that all

their supervisors have completed training in relation to the recognition and approval

of trainers, and that there is a phase two implementation working group that

monitors trainers are compliant with requirements.

30

Theme 5: Developing and implementing curricula and assessments

Informing curricular development (R5.2)

84 The school’s curriculum is informed by a range of stakeholders. During the visit

examples of developments and changes to the curriculum as a result of stakeholder

input were provided. The school also told us about the role of the curriculum

implementation group, who convene regularly and focus on improving the quality of

the school’s curriculum. Members of this group are drawn from across the medical

school and include academic, clinical and administrative staff.

85 The school also uses feedback from learners and educators in order to inform the

curriculum. Supervisors at MKH told us they have the opportunity to provide feedback

to the school on the curriculum through the phase two implementation group. The

GPs we met noted they have found the phase two implementation group a useful tool

to discuss the curriculum. Students also told us they have the opportunity to provide

feedback to inform the curriculum at the end of each unit.

Undergraduate curricular design (R5.3)

86 All of the students we met are content with the organisation and breadth of their

curriculum. Second year students commended the wide range of learning

opportunities and noted they enjoy the early patient contact in phase one as it

enables them to link the theory they learn to practice.

87 The school delivers a unit in narrative medicine which aims to teach students to see

patients as humans with individual stories experiencing illness in their social context.

During the narrative medicine unit students are assigned a patient whom they see

once a term. Students spoke highly of the narrative medicine unit as it offers them

the opportunity to gain insight into the longitudinal journey of a patient.

88 All of the students we met valued the small group teaching sessions. They told us

they are effective learning tools as they identify areas for improvement. Students

noted they enjoy applying their learning through example exam questions and case

studies in the small group teaching sessions. Students also praised the small group

sessions for anatomy noting they are effective in developing their learning. We

therefore identified the small group teaching sessions as an area that is working well.

Standard

S5.1 Medical school curricula and assessments are developed and implemented so that medical students are able to achieve the learning outcomes required by graduates.

S5.2 Postgraduate curricula and assessments are implemented so that doctors in training are able to demonstrate what is expected in Good medical practice and to achieve the learning outcomes required by their curriculum.

31

Area working well 11: Small group teaching is working well and was commended

by students.

89 The school has one open requirement and three open recommendations that fall

under this requirement. During the 2014/15 cycle of visits we set a requirement for

the school to ensure that contingency plans with the University of Leicester continue

to be feasible, and that the Buckingham curriculum must align with the new Leicester

curriculum. The school has been working towards meeting this requirement, but have

noted that it is not possible for them to maintain an exact alignment with Leicester

over the next three years as the new course at Leicester is being implemented in

stages. Phase one of the course in Buckingham is the same length as phase one in

the Leicester curriculum and the phase one clinical course at Leicester is also similar

to that at Buckingham. The school noted a potential transfer of students from

Buckingham to Leicester in phase one would not put them at a disadvantage.

90 The Leicester curriculum changes to phase two are greater and the finer details are

less well developed. While parts of the curricula are similar others are not, however,

potential issues with transfer points have been noted and are on the school’s radar.

We will therefore continue to monitor this item during future QA cycles.

91 The 2013/14 cycle of visits identified the recommendation that the school should

increase their number of SSCs and provide clearer links to how they meet the

learning outcomes defined in Tomorrow’s Doctors. During the 2015/16 cycle of visits

the visit team concluded this recommendation was partially met as the school has

mapped all of its SSCs to the learning outcomes in Tomorrow’s Doctors.

92 SSCs in phase one of the curriculum run in terms five and six and since the 2015/16

cycle of visits the term five SSCs have now run for the first time. SSCs are organised

into two components, taught units and library projects and students must complete

one of each in either term five or six. In term five the school delivered three taught

SSCs and students had a choice of 73 university sourced library projects. Students

also had the option to source their own library projects. Students are already

allocated to their term six SSCs with the school planning to deliver five taught SSCs

and the option of 73 library projects. In total the school has 8 taught SSCs across

term five and six.

93 Year two students who had completed their term five SSCs at the time of our visit

told us that the choice of SSCs complement the course and their learning. Feedback

on their SSCs was generally positive and students noted they were clear and well-

structured and supervisors were easy to access throughout. Some students however

noted they would prefer greater diversity in the taught SSCs. The senior management

team told us there are up to five new SSCs in development with additional proposals

for new units under scrutiny.

94 Since our last visit it is clear that the school has been working towards meeting the

rest of this recommendation through working towards increasing the number of SSCs

32

on offer to their students and delivering the first round of SSCs. In addition, feedback

from year two students on their SSCs was generally positive but some students made

reference to the diversity available for the taught SSCs. We will continue to monitor

this item until the additional SSCs in development have been finalised.

95 In our 2013-14 visit report we made a recommendation for the school to develop

robust procedures to manage patient participation at general practices. During the

2015/16 visit report we noted that general practices have been provided with detailed

scheduling templates to allow them to plan patient participation. However, there had

been limited patient contact at the time of the visit. During the 2016/17 cycle of visits

no concerns were highlighted in relation to procedures around patient participation at

general practices. The year two students we met told us their GP placements are

going well, they are well organised and they are enjoying the narrative medicine

strand. In addition, the school told us there is no evidence of outstanding issues from

their quality information. This recommendation can therefore be closed.

96 The 2015/16 cycle of visits recommended that the school should carefully consider

how often small activity groups are changed. Currently students stay in the same

peer groups for the whole of phase one. However, the school planned to carry out an

evaluation of the small group activities to enable consideration into changing the

student groups. Student representatives were asked to consult informally with the

student groups and identified mixed views, in that some groups have formed strong

bonds, while others would welcome a change. At the time of our visit in September

2016 the school planned to formally consult with the student body regarding potential

changes to the small activity groups with an aim to implement an outcome in early

2017. We will therefore follow up on this recommendation in the next cycle of visits.

Undergraduate programmes and clinical placements (R5.4)

97 The school currently has SLAs with MKH and Bedford Hospital NHS Trust and are

hoping to secure additional placements with Buckinghamshire Healthcare NHS Trust

and South Warwickshire NHS Foundation Trust. The LEPs that the school works with

are working to different timelines in terms of receiving students. During phase one

students undertake placements at MKH. Phase two students will commence

placements at St Andrews in early 2017, and during our meeting with representatives

from St Andrews they noted they are familiar with Buckingham’s curriculum and they

are prepared for students to arrive. It’s planned that students will begin clinical

placements at Bedford Hospital during 2018. The school told us the recruitment of

GPs to accommodate phase two has been successful and there have been no issues

with attracting GPs.

98 Year two students told us both their primary and secondary care placements are

enjoyable and well organised and structured. They noted that placements are well

linked to small group activity sessions and support their learning at the school.

Additionally, they noted both consultants and GPs are keen to teach the students and

make them feel welcome.

33

Assessing GMC outcomes for graduates (R5.5)

99 The school utilises a whole course assessment blueprint to outcomes for graduates to

ensure that students experience outcomes in different contexts across assessments

from the first to the final year. At the time of our visit the school was in the advanced

stages of planning for the delivery of phase two. They told us they have reviewed the

outcomes of each block and mapped them to the outcomes for graduates to ensure

outcomes are explicitly met in each block.

100 The school are continuously refining their assessment processes as they continue to

deliver and perform psychometric analysis on their assessments. However, it is clear

the school is aware of the importance of mapping their assessments to outcomes for

graduates as they note the whole course blueprint is the starting point for the

development of all the written and OSCE assessments.

101 The second year OSCE that we observed tested learning from the whole of phase one

and is clearly blueprinted to outcomes for graduates. We found the 12 station OSCE

samples a wide range of the outcomes in a variety of patient presentations and many

of these stations are advanced for the students’ stage of learning. We have therefore

noted this as an area that is working well.

Area working well 12: The OSCE stations tested a wide range of skills and

attributes which we note are blueprinted against outcomes for graduates.

Fair, reliable and valid assessments (R5.6)

102 The school takes adequate steps to scrutinise and ensure their assessments are fair,

reliable and valid. They use the borderline regression method of standard setting for

the end of year OSCEs and the modified Angoff method of standard setting for the

qualifying exam OSCEs. They use Cronbach’s Alpha as a measure of internal

consistency of examinations and aim for a sore of 0.80 or greater. Previous

assessment reports for the OSCEs have found that Cronbach’s Alpha is either slightly

above or slightly below 0.80, and closer to 0.90 for end of term assessments.

Additionally, item-total correlation is used to evaluate performance on the different

questions in written examinations. So far, the school has identified several questions

with poor item-total correlations which have been reviewed by the assessment team.

103 The school has two open recommendations relating to this requirement. The 2014/15

cycle of visits identified that the ratio of new and tested questions should be reviewed

during the initial run of the assessments throughout the process. To begin with the

school used Leicester’s assessment items; however, Leicester questions are now

seldom used and the school has developed an approach that generates large

numbers of questions which are progressively filing their question bank. Staff

involved in teaching input by suggesting questions, a paper is then created by the

deputy assessment lead, scrutinised by the appropriate groups of staff before being

submitted to an external examiner for comment. The school noted that psychometric

34

analysis of previous examinations found that assessments constructed with the

school’s own items are robust and reliable and the school is responding to the few

questions with less favourable psychometric characteristics. We are therefore satisfied

that this item can now be closed.

104 The 2015/16 OSCE observation identified that consideration should be given to

further develop consistency in the simulated patients’ performance, examiners’

interactions with students and the calibration of students’ marks and outcomes. The

second year 2017 OSCEs highlighted that the school has begun to take steps to

address this. The OSCE began with a briefing for examiners and simulated patients

which highlighted the importance of portraying their roles in a consistent fashion,

scoring candidates and general house rules for the OSCE. This was followed by a

calibration and familiarisation exercise during which assessors and simulated patients

practised their roles. This aimed to ensure consistency in simulated patient acting

between the two loops, and develop a shared common understanding of the marking

system amongst the assessors of the different stations and loops.

105 However, we observed some discrepancies between the examiners and simulated

patients in the OSCE marking process, with some of the simulated patients

contributing considerably to marking whilst others did not. This varied across the

stations and between the loops. We also observed that some of the examiners

intervened by stopping the students to ensure there was adequate time to answer

the ‘optional’ questions at the end of a station whilst others did not.

106 We fed this back to the school after our observation of the first year two OSCE, and it

was evident that the school had taken steps to further address these discrepancies

for the year two qualifying exam OSCE. We observed a clearer briefing to examiners

which emphasised that simulated patients should not contribute to marking in any

way. Following this we observed that simulated patients did not contribute to the

marking process or attempt to influence the marking process as advised in the

morning training session. The school also changed the briefing and documentation to

make clear that the questions at the end of the station are not optional and the

student should be stopped with adequate time to answer the questions.

107 During the 2016/17 second year OSCE we observed the competency and global

marking and discussed this with some examiners. We are concerned that this may

not be borderline regression for standard setting because it appears that the

competency marks are being used directly to calculate the global score. The school

had taken steps to address this in the second year qualifying exam OSCE and we feel

there has been some improvement. The school has clarified the independence of the

global score and the competency marks with examiners and we encourage the school

to continue their efforts in ensuring that competencies are objectively marked and

regressed against the global score.

108 We also found inconsistencies amongst examiners in their understanding of the

thresholds for passing. Some described the global score of pass being equivalent to a

35

competent FY1 and a global score of excellent being equivalent to an excellent FY1.

However, other examiners described pass as not quite at FY1 level and excellent as

equivalent to a competent FY1. Therefore, there may not be a shared vision of what

constitutes a pass and also it may be unrealistic to expect a year two student to

perform at FY1 level in order to pass. At the second year qualifying exam OSCE the

standards expected for the global scores of pass and excellent were covered

thoroughly in the early morning briefing and we observed an understanding amongst

examiners in the thresholds for passing. Although we have observed some

improvements with this issue and note the school are addressing concerns we will

keep this recommendation open to ensure that changes made are sustainable.

109 The school told us they are conducting a review of their assessment processes in

order to continuously enhance the way in which assessments are delivered. For a

given type of assessment the school has decided that students must pass a set

number of short answer questions or OSCE stations in order to pass the assessment

overall. The methodology used to decide the number of questions needed to pass, is

not explicit. The visit team encourage the school to consider this in their assessment

review.

110 The school currently combine the OSCE and multiple choice questions to decide if a

pass or fail should be awarded in the qualifying exam; however they do not adopt

this method for term exams. The school has noted they are keeping the differences

between the term and qualifying exam method under review. Additionally, by the end

of the academic year students who fail a component, or cannot take an exam due to

mitigating circumstances are required to sit the qualifying exam that covers the full

year, rather than resit only the exams they have failed. Again we encourage the

school to keep this under review.

111 The school analyses item-total correlations to assess student performance across

different stations or questions in an assessment. The visit team notes that some

items in both written exams and OSCEs have had a low item-total correlation and

encourages ongoing analysis of this in their assessment review.

112 The board of examiners meeting that we observed discussed if students who had

taken the second year qualifying exam were able to progress to phase two of the

course. High performing students from the whole of phase one were also considered

for an award. Overall, the meeting was well run with input from appropriate areas of

expertise. The school had adequate external representation, including a lay

representative and an external examiner who had considered the assessment analysis

for the qualifying exam and concluded that it is reliable. From the meeting and

papers it is apparent that the school undertakes rigorous scrutiny of its exams

including previously mentioned psychometric analysis and double marking of short

answer questions with moderation/checking.

113 However, we consider the school is awarding a large number of distinctions and

merits, out of keeping with the sector norms. In 2016, 20% distinctions and 40%

36

merits were awarded to year one and two students. The external examiner also noted

this as an area for consideration. The school is aware of this and is looking to address

it in future cohorts. Considering that these points contribute to a possible honours

degree we encourage the school to address this and the above four areas listed in

paragraphs 109-113 through their assessment review.

Recommendation 2: The school should continue to evaluate their assessment

processes through their ongoing assessment review.

114 The year two OSCEs identified that the school invests heavily in their assessment

processes. They conduct psychometric analysis following each examination to test the

reliability of their assessment processes. Results are then considered at the board of

examiners meetings. The visit team noted this as an area that is working well in the

school and we encourage the school to continue to develop their assessment

processes.

Area working well 13: The school’s investment in assessment and high level of

analysis is to be commended

Examiners and assessors (R5.8)

115 Assessors are selected from medical school staff, clinical staff and partner

organisations such as general practices and LEPs. All assessors undertake a

comprehensive training session related to the OSCEs which covers OSCE scoring,

standard setting, bias and video calibration. The school also use recorded stations

with candidates being role played by others, for training and calibration of marking by

examiners.

116 During the 2016/17 second year OSCEs we observed the assessors briefing followed

by the calibration exercise. Following this we made some suggestions to the school

and during the qualifying exam it was evident that the school had improved their

assessors briefing. Additionally, the calibration exercise is a welcome development to

ensure a high level of consistency and judgement between assessors.

Area working well 14: We found the OSCE to be well organised, with a high level

of staff involvement and commitment.

Reasonable adjustments in the assessment and delivery of curricula (R5.12)

117 The school has a process in place for dealing with students who require reasonable

adjustments. An educational psychologist reassesses all students accepted onto the

programme who have disclosed a learning support need at application. Reasonable

adjustments are confirmed following a discussion between the educational

psychologist and occupational health services. Those who have not disclosed a

learning support need at application are referred to the educational psychologist

through the concerns group. Following the identification of a support need, the

37

assessment teams, unit leads and personal tutors monitor student performance with

regard to the adjustment.

118 During our observation of the OSCE it was evident that reasonable adjustments had

been implemented as some students had extra reading time before commencing a

station. We noted this as an area that is working well.

Area working well 15: We observed clear evidence that reasonable adjustments

have been implemented for the OSCE by the School.

38

University of Buckingham Medical School’s response to the GMC’s visit

report

The University of Buckingham Medical School (UBMS) thanks our GMC visiting team

for a reasonable and constructive approach to our ongoing quality assurance process.

The School is content to accept the conclusions of the report, and is heartened by the

recognition of considerable strengths in our developing provision. We are already

addressing the few requirements and recommendations made and look forward to

demonstrating this to the team in the next cycle.

It is always our aim to meet, and preferably exceed, the standards for medical

education prescribed by the General Medical Council. We welcome constructive

feedback that will enable us to do so, and we aim to act upon that feedback as

quickly as we can. In reality, we believe, most issues identified in this report were

already addressed before we received it.

We will continue to cooperate as best we can with our ongoing quality assurance

process.

39

Team leader Dr Nick Bishop

Visitors Dr Bruno Rushforth

Dr Carol Gray

Mr Faisel Alam

Prof Helen Cameron

Dr Jessie Sohal-Burnside

GMC staff Emily Saldanha (QA Programme Manager)

Jessica Ormshaw (Education Quality Analyst)

Evidence

base

Doc 161: UBMS term two risk register – October 2015

Doc 162: UBMS update on the academic building at MKH – November

2015

Doc 163: UBMS update on clinical capacity – November 2015

Doc 164: Examples of narrative medicine reflective accounts –

November 2015

Doc 165: UBMS substantive term three OSCE report – December 2015

Doc 166: UBMS substantive term three written paper report –

December 2015

Doc 167: UBMS term three OSCE examiner correlations – December

2015

Doc 168: UBMS updated global risk register – June 2016

Doc 169: Quality management & enhancement in year two – June 2016

Doc 170: Progress report on open issues – August 2016

Doc 171: Progress report on resources and staffing – August 2016

Doc 172: Progress report on the implementation of Phase two – August

2016

Doc 173: Progress report on assessments - August 2016

Doc 174: Progress report of student support – August 2016

Doc 175: Briefing document for the year two OSCE observation –

January 2017

Doc 176: Briefing document for QE2OSCE observation – February 2017

40

Doc 177: Exam board report MED15 QE year two – March 2017

Doc 178: Exam Board meeting agenda – March 2017

Doc 179: Board of examiners minutes for MED16, ETA3 and QE

Doc 180: OSSE performance report 2015 selection cycle