submission for accreditation australian medical council ... · 6. monitoring and evaluation ......

Submission for Accreditation Australian Medical Council

July 2014

ACCRM Reaccreditation Submission to AMC July 2014

Table of Contents

Executive Summary...................................................................................... vi Contact Details............................................................................................ xii 1. Programs Context .................................................................................. 1 1.1. Governance ...................................................................................................................... 1 1.2 Program Management ..................................................................................................... 5 1.3 Educational Expertise and Exchange ............................................................................ 13 1.4 Interaction with the Health Sector ............................................................................... 18 1.5 Continuous Renewal ...................................................................................................... 21 1.6 AMC Recommendations (BB, AA) ................................................................................. 22 1.7 Other Matters to be Addressed .................................................................................... 24

2. Purpose and Outcomes ........................................................................ 26 2.1 Purpose of the Training Program .................................................................................. 26 2.2 Graduate Outcomes ....................................................................................................... 29 2.3 AMC Conditions (7) ........................................................................................................ 32 2.4 Other Matters to be Addressed .................................................................................... 36

3. Curriculum Content...............................................................................37 3.1 Curriculum framework .................................................................................................. 37 3.2 Curriculum components ............................................................................................... 41 3.3 Research in the Training Program ................................................................................ 42 3.4 Flexible training ............................................................................................................. 44 3.5 The Continuum of Learning .......................................................................................... 46 3.6 AMC Conditions (9) ....................................................................................................... 49 3.7 Other Matters to be Addressed ................................................................................... 50

4. Teaching and Learning .........................................................................52 4.1 Teaching and Learning methods .................................................................................. 52 4.2 AMC Recommendations (KK) ....................................................................................... 55 4.3 Other matters to be addressed .................................................................................... 56

5. Assessment of Learning ........................................................................58 5.1 Assessment approach ................................................................................................... 58 5.2 Feedback and performance.......................................................................................... 62 5.3 Assessment Quality ....................................................................................................... 64 5.4 Assessment of Specialists Trained Overseas ............................................................... 65 5.5 AMC Recommendations (OO) ...................................................................................... 67 5.6 AMC Conditions (13) ..................................................................................................... 68


5.7 Other Matters to be Addressed ................................................................................... 69

6. Monitoring and Evaluation.................................................................... 81 6.1 Ongoing monitoring ....................................................................................................... 81 6.2 Outcome evaluation ...................................................................................................... 87 6.3 AMC Conditions (15,18) ................................................................................................ 91 6.4 Other matters to be addressed .................................................................................... 92

7. Trainees...............................................................................................98 7.1 Admission policy and selection ..................................................................................... 98 7.2 Trainee participation in the training organisation governance ............................... 100 7.3 Communication with trainees .................................................................................... 102 7.4 Resolution of training problems and disputes .......................................................... 105 7.5 Other Matters to be discussed ................................................................................... 107

8. Program Delivery ................................................................................111 8.1 Supervisors, assessors, trainers and mentors ........................................................... 111 8.2 Clinical and other educational resources .................................................................. 114 8.3 AMC Conditions (22,23,25) ........................................................................................ 118 8.4 Other matters to be addressed .................................................................................. 123

9. Continuing Professional Development ...............................................126 9.1 Continuing Professional Development Programs ...................................................... 126 9.2 Retraining...................................................................................................................... 133 9.3 Remediation ................................................................................................................. 134 9.4 Recommendations (TT) ................................................................................................ 135 9.5 Other Matters to be Addressed .................................................................................. 136

Glossary of Terms .....................................................................................138

Attachments

Section 1: Program Context

1.1 Information Memorandum on Proposed Changes 1.2 ACRRM Memorandum of Association 1.3 ACRRM Articles of Association 1.4 ACRRM Committees Terms of Reference 1.5 ACRRM Committees Current Membership 1.6 ACRRM Organisational Chart 1.7 ACRRM External Committee Representation List 1.8 Curriculum Comparison Review Report

Section 2: Purpose and Outcomes

2.1 ACRRM Member Survey 2013


2.2 ACRRM Annual Report 2012-13 2.3 ACRRM Registrar Survey 2013 2.4 ACRRM Vocational Training Exit Survey 2014 2.5 ACRRM Consumer Feedback Survey Questions 2.6 ACRRM Consumer Feedback Survey (Key Stakeholder Mailing List) Section 3: Curriculum Content

3.1 ACRRM Primary Curriculum 4th

edition 3.2 ACRRM Advanced Specialised Training Curricula 3.3 ACRRM Procedural Logbook 3.4 Primary Curriculum summary of changes 3.5 ACRRM Academic Practice AST Draft curriculum 3.6 ACRRM Policy Statements for CCT; PRRT; and, AST 3.7 ACRRM Policy for Completion of Training in PRRT 3.8 List of completed ACRRM Registrar Projects 3.9 AGPT Academic Posts Guide 3.10 ACRRM Policy on Leave from Vocational Training 3.11 ACRRM Policy on Part-time Vocational Training

3.12 ACRRM Policy on RPL in Vocational Training

Section 4: Teaching and Learning

4.1 List of RRMEO Modules 4.2 IP Workshop Programs (2013, 2014)

4.3 IP Teaching and Learning Blueprint 4.4 Virtual Classroom Program 2014 4.5 Revised Emergency Medicine Course Requirements Section 5: Assessment of Learning

5.1 ACRRM Fellowship Assessment Handbook 2014 5.2 ACRRM Assessment Blueprint 2013 5.3 ACRRM Special Consideration Policy 5.4 ACRRM Registrar Assessment Reports (MSF, MiniCEX, MCQ, StAMPS) 5.5 Assessment Program Publications and Presentations 5.6 ACRRM Assessment Review Day Program 5.7 Presentation on StAMPS and MCQ Section 6: Monitoring and Evaluation 6.1 Revised ACRRM Evaluation Plan 2013-14 6.2 Vocational Training and Assessment Evaluation 2013-14 6.3 ACRRM Evaluation Report 2014 Section 7: Trainees 7.1 AGPT Handbook 2015 7.2 RVTS Application Guide 2014 7.3 IP ‘How to Apply’ Guide 2015 7.4 IP Selection Process 7.5 RMA Workshop Programs and Presentations 7.6 Events attended by ACRRM Staff 2010-2014 7.7 ACRRM Grievance Policy 7.8 ACRRM Appeals Policy 7.9 IP Training Agreement 2013


Section 8: Program Delivery 8.1 ACRRM Standards Supervisors and Teaching Posts (PRRT, CCT, AST Curricula) 8.2 Post Accreditation Management Agreement with RTPs 8.3 RTPs’ Post Accreditation Annual Report 2013 8.4 Example of collated feedback from StAMPs Examiners and Candidates 2014 8.5 ACRRM Standards for RTPs 8.6 ACRRM RTP Accreditation: Principles and Outcomes 8.7 Bi-College RTP Accreditation Program Guide Section 9: Continuing Professional Development 9.1 ACRRM PDP Member Handbook 2014 9.2 ACRRM PDP Evaluation Report 2013 9.3 ACRRM PDP Remediation Policy

Tables and Figures Section 1: Program Context Table 1.1 ACRRM Staff involved in educational activities (2014) Table 1.2 ACRRM Staff List (2014) Table 1.3 New College Enrolments 2010 - Present Table 1.4 New College Fellowships 2010 - Present Figure 1.1 College Governance Structure Section 2: Purpose and Outcomes Table 2.1 Number of registrars achieving Fellowship (Breakdown by Program) Table 2.2: Fellows by ASGC-RA and training pathway in 2013

Section 3 Curriculum Content Table 3.1 Number of Registrars applying for and granted RPL (Jan 2011–Jul 2014) Section 5: Assessment Table 5.1 Summative Assessment requirements for each AST Discipline Table 5.2 Formative Assessment requirements for each AST Discipline Table 5.3 Vocational Programs Assessment results for 2010-2014

Table 5.4 AST Assessment Results 2010-2014 Table 5.5 Number of attempts required to pass assessments in 2010 Table 5.6 Number of attempts required to pass assessments in 2011 Table 5.7 Number of attempts required to pass assessments in 2012 Table 5.8 Number of attempts required to pass assessments in 2013 Table 5.9 Number of attempts required to pass assessments in 2013

Table 5.10 Registrars reviewed by Registrar Review Panel and Outcomes 2010-2014 Table 5.11 Number of Registrars Withdrawing from Pathway Jan 2010–Jul 2014

Figure 5.1 Miller’s Pyramid


Section 6: Monitoring and Evaluation Table 6.1 ACRRM Currently Operational Evaluation Tools Table 6.2 ACRRM College Evaluation activities since 2010 Table 6.3 ACRRM Program Review activities since 2010 Table 6.4 ACRRM Policy Review activities since 2010 Table 6.5 ACRRM Standards Review activities since 2010 Figure 6.1 College Evaluation Cycle Figure 6.2 Evaluation Process relationship to Program Improvement Section 7: Trainees Table 7.1 Number of ACRRM Registrars entering FACRRM pathways 2011-2013 Table 7.2 Number of ACRRM Registrars entering AGPT pathways 2011-2013 (Breakdown by RTP)

Section 8: Program Delivery Table 8.1 Primary Rural and Remote Training Posts for Re-accreditation Table 8.2 Advanced Specialist Training Posts for Re-accreditation Section 9: Professional Development Program Table 9.1 2011-2013 Triennium Compliance Table 9.2 2011-2013 Enrolment by Reportable Advanced Skills Area Table 9.3 2014-2016 Triennium Compliance Table 9.4 2014-2016 Enrolment by Reportable Advanced Skills Area Table 9.5 2014-2016 PDP Courses Accredited

ACRRM Reaccreditation Submission to AMC vi

July 2014

Executive Summary The Australian College of Rural and Remote Medicine (ACRRM) is one of two Colleges accredited to provide Fellowship training and professional development; and, certify standards in the Specialty of General Practice. It currently has around 1500 fellows, over 200 of who have been trained and fellowed since 2010. It has a further around 600 registrars currently in vocational training and a total membership of around 3700. Additional to its core educational roles, it provides services including international medical graduate assessment, undergraduate rural placement support, telehealth, and advocacy in rural medical education issues. The College fellowship and vocational training programs are designed to prepare and certify doctors for general practice anywhere while ensuring that they have attained the special skills set to underpin quality practice in rural and remote settings. The College’s ultimate mission is to contribute to meeting the healthcare needs of rural and remote communities. It seeks to achieve this through provision of a sufficient number of trained and certified, general practitioners who are motivated and able to provide the high quality, breadth, depth, and mode of rural practice services essential to meeting these needs. The College was founded and continues to thrive due to the passionate support of its membership and their commitment to this essential mission. It was established by a national plebiscite of rural doctors in recognition that the pre-existing models of professional education and accreditation were not equipped to fully accommodate the medical needs of rural and remote communities. The College has evolved to develop the skills-base of fellows, and foster their necessary sense of professional identity to function as community of practice committed to improving the health of rural Australians. ACRRM APPROACH

ACRRM is committed to enabling provision of safe, quality medical care to Australia’s rural and remote communities. Quality care is only attainable if a sufficient number of appropriately skilled doctors are available and willing to serve in those communities. Supplying and supporting a rural medical workforce with an appropriately broad scope of practice are at the core of what the College stands for. Intrinsic to this commitment; is a focus on innovative delivery solutions to make training and certification possible for doctors who are distant from central educational facilities and who may have limited locum support. This often requires a distinctive approach to traditional medical college frameworks. The 2006 Productivity Commission Report to Council of Australian Governments (COAG) specifically noted that:

“…it (the rural and remote Australian health workforce) has been an ‘incubator’ for evolution in job design and other workplace innovation…. Improved (workforce) outcomes are attainable, especially if broader reform frameworks make explicit provision to address rural and remote issues.”

1

ACHIEVEMENTS OF THE ACRRM APPROACH Eighty percent of ACRRM members are located in regional, rural and remote settings and as members, receive specially-tailored vocational training and Continuing Professional Development (CPD).

2 Furthermore, 82% of ACRRM’s trainees fellowed in 2013 intended to practice rurally (one

third of them in the country’s most remote classification zones).3

1 Productivity Commission. Australia’s Health Workforce, Research Report, Canberra: 2005. (Ch. 10, Pg. 204) 2 ACRRM Member Survey 2013. 3 ACRRM Vocational Trainee Exit survey. 2013

ACRRM Reaccreditation Submission to the AMC July 2014

vii

ACRRM has consistently advocated for development of advanced and procedural skills for rural general practitioners. Through the increasing popularity of ACRRMs vocational training and CPD programs; and, such developments as the national network of Rural Generalist (RG) training pathways and the Rural Procedural Grants Program (both initiatives proposed and developed by ACRRM). As a result the rural procedural skills base is slowly rebuilding (the most recent national data shows a 10% increase in the number of rural proceduralists from 2011-2012 ).

4

The ACRRM pathway model was adapted by Queensland Health to deliver the initial RG pathway, the Queensland Rural Generalist Pathway (QRGP). The model was subsequently adopted to varying degrees by other States and Territories. The QRGP, with around 97% of its trainees on the ACRRM fellowship program,

5 has achieved a 92% reduction in critical rural hospital vacancies; and has a

projected long-term rural retention rate among its trainees of 62%; entire rural hospitals that would otherwise have been closed; are now being run by GP proceduralists, typically with dual community/hospital appointments.6

For nearly two decades, ACRRM has provided the engine for driving innovation, research and development into new ways to deliver safe quality training and certification for rural and remote doctors. These now form the fabric of Australia’s rural medical training framework. - The College is engaged by the Commonwealth to provide the national online telehealth platform

used by around 25,000 clinicians annually; to deliver national telehealth programs; and produce the national telehealth guidelines (which have been incorporated into the education programs of medical colleges and key professional bodies).

- Its online seminar facilities are regularly used for national events including the General Practice Education and Training (GPET) annual research conference.

- The College proposed, developed, and has for over a decade managed the national undergraduate rural placement support programs; and, the procedural grants program. It developed the original model for the Remote Vocational Training Scheme (RVTS) and the prevocational training placements program.

- ACRRM’s Rural Medicine World Summit attracted around 800 delegates from 27 countries and defined an international consensus statement on Rural Medicine and its best applications to medical services.

- ACRRM’s ‘RRMEO’ was the world’s first online learning platform designed to support rural and remote general practice training. It continues to lead the way in introducing new learning support functionalities; such as, virtual classrooms, the learning planner system, smartphone accessible clinical guidelines (as decision support systems); and the College continues to innovate, research and develop new mechanisms particularly in the area of practice-based education and assessment.

Alignment with the principles of the College is one of the top three reasons for joining the College. General satisfaction is demonstrated by the fact that 92% of all surveyed members intend to renew membership, 90% are proud of their ACRRM membership and overall the College received an 84% satisfaction rate amongst its membership.

7

4 RHWA Combined Rural Workforce Agencies National Minimum Data Set Report as at Nov 2012. Melbourne: 2013. 5 RMA Conference 2013. Presentation by Dr Denis Lennox, QRGP Clinical Director. 6 As reported in February 2012 in Ernst and Young. Evaluation and Investigative Study of the Queensland Rural

Generalist Program. Queensland Health, Office of Rural and Remote Health. February 2013. 7 ACRRM Members’ Survey, 2014.


viii

FUTURE ARRANGEMENTS FOR GENERAL PRACTICE TRAINING It is important to note that this accreditation process is proceeding against a backdrop of major change to the structures and funding arrangements for general practice training including the winding up of GPET and the retendering and reduction of the numbers of numbers of Regional Training Providers (RTPs). At the time of writing the roles and responsibilities of the College with respect to leading these changes is unclear. The College is actively engaging with the Government and the Department of Health as a matter of urgency to address this. 1. Project Context: Firm foundations Since 2010, ACRRM has undergone a process of reflection and adaptation to ensure that its entire corporate structure reflects corporate best practice; ensuring appropriate representation, expertise and professional integrity. This has included revision of its constitution; standing committees and its staff management structure. This process is ongoing. During this period ACRRM’s programs have substantially expanded and the organisational structure has been reconstituted to meet its wider scale of operations. Its capacity to meet its responsibilities is demonstrated by a 170% increase in provision of trained, credentialed Fellows over the accreditation period and registrar enrolments continue to rise. New Fellowships (i.e. trained with ACRRM) increased by 38% last year and are anticipated on current projections to increase by a further 15% this year. The general organisational approach has been maintained as the appropriate model for an organisation committed to providing training, professional development and ultimately quality general practitioners for rural and remote communities. The model is a highly distributed one, with a major focus on innovative delivery solutions; engaging fellows’ to contribute to program delivery; and, strategic collaborations with academia and the wider health sector. 2. Purpose and Outcomes: Meeting its Mission The College has a clear mission to meet the general practice needs of rural and remote communities particularly through a unique education and standards framework specially designed and constantly reviewed and adjusted. Meeting this mission involves the graduate outcome of producing a competent, appropriately trained rural and remote medical general practice workforce and it extends beyond this, to ensuring provision of quality services. This involves a measure of innovation to enable provision of effective training and professional development in rural and remote locations, with minimal disruption to services provided by supervisors, trainees, and College Fellows. A threefold approach has been adopted to ensure the College keeps faith with its essential mission and continues to understand the changing status of medical services and community needs in rural and remote Australia. Firstly, by providing effective stakeholder communication and collecting and processing feedback, via the College Evaluation Plan; secondly, through the operation of a widely representative committee structure; and thirdly, through ongoing engagement in wider policy forums, and scoping of current literature and information. 3. Curriculum Content: Training Matched to Community Need The College released its revised 4

th Primary Curriculum in 2013. Efforts are now focussed on

reviewing the AST curricula; and the development of an additional AST program in Academic Practice. ACRRM contends that general practice in rural and remote locations (without easy access to the full urban suite of specialised services) requires a distinctive collection of skills and competencies over and above those that would be typically required for urban community practice. Defining requisite preparation for that scope should be led by experts in the field. Accordingly, the 4th edition Primary


ix

Curriculum review process was guided by College Fellows with expertise in key areas; assisted by feedback from the wide cross-section of its membership, selected experts/specialists and educationalists, and informed through a process of stakeholder engagement. The Curriculum has been designed to ensure Fellows are adequately prepared for broad scope community-based, continuing care; as well as for hospital based secondary and emergency care. To ensure the former, a minimum of six months training in community primary care is required for Fellowship. The Primary Curriculum also covers the essential requirements of hospital-based service, emergency care and population health. The AST curricula specify extended professional abilities in an area of focused cognitive and/or procedural practice as required to sustain needed rural health services locally among a network of colleagues. Of newly fellowed members surveyed in 2013, 77-79% considered the program had provided the knowledge and skills required for independent general practice in rural and remote communities.8 The congruence of skills provision and community need is demonstrated by the seven advanced skills ranked by rural communities as their ‘most needed’; all being among the top ten choices for PDP activity9. The College actively promotes the concept of structured ‘training pipeline’ for rural practitioners – an approach that the literature indicates is the most effective way to produce long-term rural practitioners. Accordingly the College takes a comprehensive, multifaceted approach to maximise vertical integration; through its involvement in activities stretching from medical school and throughout medical careers. Along with ‘pipeline’, mechanisms exist to maximise program flexibility and facilitation of lateral entry for doctors with diverse backgrounds, through rigorous recognition of prior Learning (RPL) and assessment processes. 4. Teaching and Learning: Imparting the skills and knowledge ACRRM has become a national leader in the area of innovative online delivery of medical education activities and telehealth. ACRRM provides training services to the remotest practices in the country. The vast majority of its members are located in rural and remote locations and it services members in Antarctica, Norfolk Island, Bamaga and the Kimberley. ACRRM registrars commonly face the dual challenge of geographical remoteness and heightened workload and responsibilities. A major focus of the Colleges’ energies has always been to pioneer creative delivery solutions to enable quality training and minimise disruption to trainees. The delivery of the College program is underpinned by a core group of medical educationalists on staff together with 215 trained health professionals (mostly members) who support delivery, contributing to training, mentoring and supervision, assessment development and provision. One important strategy has been to provide a diverse suite of teaching and learning modes of delivery and delivery timing. The range includes in-person workshops, virtual classrooms, tele/videoconferencing, and self-directed online learning activities. The College has recognised that as its educational offerings become ever wider and more complex it becomes increasingly important to provide an online interface with expert coordination, navigability and consistency of language. To this end, expert consultants have been engaged and constructed an integrated single interface system: the Customer Relations Management (CRM) system. This will be operational in the forthcoming months.

8 ACRRM Vocational Training Exit Survey 2013. 9 ACRRM Evaluation Report 2014.


x

5. Assessment: Ensuring Quality The College approach aims to apply a wide range of tools to provide a full picture of each trainee’s competencies for safe practice measured against the curriculum outcomes. Innovative approaches have been applied to enable workplace based assessment wherever practicable and to maximise registrar support. Each tool has been developed to international academic standards; and is regularly and rigorously tested for reliability and validity as well as for user satisfaction. Assessment results are regularly collated and examined by the relevant committees to identify trends. The College contends that safe and quality care requires provision of generalist doctors in rural and remote contexts with a scope of practice that is more advanced and broader than that typically necessary for general practice in urban settings. For this reason assessment expectations are higher and more challenging than those typical of general practice Colleges internationally. In recent years, concerted efforts have been made to provide additional coaching and support to help trainees prepare for assessment; and one-on-one remediation programs are in place for trainees who fail summative assessments. These arrangements appear to be positively affecting trends. 6. Monitoring and Evaluation: Commitment to continuous Improvement The College engages in a comprehensive and continuous cycle of measuring and evaluating its programs. Evaluations assess the direct educational outcomes of the program; as well as their implications in the wider community context. A body of data has now been assembled; such that both immediate effects as well as longitudinal trends can be identified and addressed. As an organisation committed to driving new solutions to the delivery of quality-assured medical education in rural and remote locations, every new tool introduced to the program is evaluated and engaged in evaluation cycle. In additional to statistical database gathering, and occasional research projects; the Evaluation Plan incorporates 45 regular and ongoing evaluations into program activities; 17 program reviews, 12 policy reviews, and 16 standards reviews. The Evaluation framework is directed to program improvement. Data is collected and processed. Recommendations for operational improvement arising from these are developed are presented to program management and College governance for consideration. 7. Trainees: Nurturing the next generation ACRRM registrars are able to enrol in three pathways. For trainees who enrol in the Australia General Practice Training (AGPT) and the Remote Vocational Training Service (RVTS) pathways selection and program delivery is delegated to the respective training providers. ACRRM is actively seeking to influence proposed future changes to General Practice training in Australia to enable greater engagement of the College in registrar selection. The College’s Independent Pathway (IP) program is delivered internally. All programs must be delivered in compliance with ACRRM’s standards and curriculum. The College has directed considerable energies in recent years to increasing trainee communication and support. Many additional contacts with trainees; RTPs and other supervisors, and instructional workshop activities have been introduced; and a variety of new peer networking platforms have also been established. The College is working with the Commonwealth to determine how training for ACRRM registrars in the AGPT pathway can best be provided under the impending new arrangements for AGPT.


xi

8. Program Delivery: Commitment to Excellence in Teaching and Teaching Facilities The College has a comprehensive set of policy statements defining standards for teaching and teaching facilities provision in its training programs. These are all regularly reviewed. The College has focussed its energies in recent years to ensure that the education services delivered through training providers in its delegated delivery framework is compliant with College standards and curriculum. This has been achieved most significantly through the Bi-college Accreditation Program that it has developed collaboratively with GPET, RTPs and RACGP. Progress has also been made through increased communication and interaction with RTPs and supervisors across all three programs. 9. Continuing Professional Development: Maintaining the skills base in communities The ACRRM Professional Development Program (PDP) is the College’s longest running program. This reflects its importance to the College as a means of maintaining the essential skills base that rural and remote practitioners are able to safely provide. In the last triennium ACRRM provided PDP services for 1718 doctors, 1634 of them being ACRRM Fellows and provided them with 2906 accredited courses. ACRRM contends that ‘full scope’ general practice in rural and remote areas demands broader set of competencies than typically required in office-based urban general practice. For ACRRM the PDP is especially important to preserve high quality, safe healthcare in rural and remote communities by developing and maintaining skills required to deliver services. Among ACRRM PDP participants surveyed, 96% felt that their PDP activities usefully contributed to their daily practice in their community. This is further evidenced by the fact that the seven ‘most needed services’ identified by surveyed rural community representatives; were all within the top ten PDP courses that were undertaken by ACRRM members.10

10 ACRRM Evaluation Report 2014.


xii

Contact Details Australian College of Rural and Remote Medicine Level 4 410 Queen Street Brisbane QLD 4000 Chief Executive Officer: Ms Marita Cowie Telephone: 07 31058200 Fax: 07 31058299 Email: [email protected] Website: www.acrrm.org.au Officer to contact: Ms Marita Cowie Telephone: 07 31058200 Email: [email protected] Training Program Offered Fellowship: Australian College of Rural and Remote Medicine (FACRRM). ACRRM is the awarding body.

mailto:[email protected]

http://www.acrrm.org.au/


ACRRM Reaccreditation Submission to AMC 1

July 2014

1. Programs Context 1.1 Governance

1.1.1 The education provider’s governance structures and its education and training, assessment and continuing professional development functions are defined.

The scope of the College’s structure and functions are defined by the ‘Memorandum of Association’ and the ‘Articles of Association’; and are implemented through the ACRRM Board and committees structure (see Figure 1.1 below).

The ACRRM Board is responsible for setting all policy and standards. Oversight of these and other clinical governance matters are delegated to the Education Council. The Chief Executive Officer, College Censor-in-Chief, Education Director and all College Committees are all appointed by the Board.

During this last accreditation period, independent consultants, ‘Board Matters’ have been engaged by the College to undertake a comprehensive review of its constitutional processes and provide advice and professional assistance as required. Their brief has been to ensure that all College processes, reflect current organisational best practice. They have produced revised versions of both the ‘Articles of Association’ and ‘Memorandum of Association’. Further changes to the structure and the Constitution have been developed and approved by the Board for presentation and vote at its forthcoming College General Meeting. These would maintain the current Committees structure but adjust the structure of the top tier of Governance replacing the current Board with a larger, representative College Council that will be responsible for electing a five-member Board from among its membership. Figure 1.1: College Governance Structure

ACRRM Reaccreditation Submission to the AMC 2 July 2014

Constitutional Changes since 2010 and prospective Future Changes The Constitutional Review process has prompted changes to the College’s governance structures. Changes have also been undertaken to satisfy AMC recommendations. The key areas that have been identified for consideration and subsequent outcomes are outlined below.

1. Membership Classifications Simplified

In response to recommendations by Board Matters and consistent with corporate best practice, it was determined that the wording in the Memorandum of Association related to membership classifications was unnecessarily complicated. The Board approved changes to the Memorandum removing references to several membership classifications that were considered redundant for the purposes of the document. This was subsequently approved at the College Annual General Meeting in 2013 and enacted. 2. Definition of College within Specialty of General Practice

In recognition of the Specialty’s accreditation status and as per the recommendations of the AMC, the Board approved the redrafting of the Memorandum of Association to ensure that the definitions of the specialty it contained were consistent with its accredited status as an accredited College within the Specialty of General Practice. It should be noted and was explicitly drawn to the attention of ACRRM members at the AGM that these changes did not signify and changes to ACRRM’s overall purposes, rather the objects were strengthened to set out ACRRM’s role in the specialty of general practice as broadly defined by the College. This was subsequently approved at the College Annual General Meeting in 2013 and enacted. 3. Leadership Transition arrangements

In response to recommendations by the AMC, the Board has given detailed consideration to its framework for elections to leadership positions. There was concern that as the terms of half the directors of the board lapse each year at the same time and as positions were open to all eligible members; there was potential for three key leadership (President, Vice-President and Treasurer) positions being filled by directors potentially without Board experience. The Board noted firstly that although this was technically correct, it was contrary to the experience of the Board since the inception of the College. In practise, the current governance framework has facilitated a generally smooth transitional process of departing and new Directors. It was noted that the departure of the three key positions every two years did create significant change to the executive leadership at that time. It was felt that this risk was partly ameliorated by incorporating the Immediate Past President as an ex-officio and non-voting member of the Board. A proposal has been put forward to the membership that will involve a larger College Council that elects a smaller board with consideration to transitional issues. As below the proposed changes were tabled at the 2013 Annual General Meeting and the membership (including those not in attendance), have been given time to consider these. The proposal will be put to an Extraordinary General Meeting in 2014.

4. Representation of membership

Board Matters has recommended changes to the constitution as it related to member representation and executive leadership. The rationale being that they are in line with current corporate best practice to deliver maximal member representation while also ensuring the attainment of the necessary executive skills set.


An Information Memorandum was prepared in order to engage with members regarding the more substantial changes proposed for governance architecture. This document was distributed to members and discussed at the 2013 AGM. A subsequent consultation period with members has ensued in order to enable members who cannot be present in person to provide their views. A variety of written and online mechanisms have been developed to seek feedback from members regarding the proposed changes. A new constitution is being drafted and presented to members for consideration of adoption at an Extraordinary General Meeting in 2014.

(See Attachment 1.1: ACRRM Memorandum of Association)

(See Attachment 1.2: ACRRM Articles or Association.)

(See Attachment 1.3 Information Memorandum on proposed changes to College Structure tabled at 2013 AGM.)

1.1.2 The governance structures describe the composition and terms of reference for each committee and allow all relevant groups to be represented in decision-making.

In addition to the ‘Memorandum of Association’ and ‘Articles of Association’, all College Committees have clearly articulated Terms of Reference incorporating specific reference to their respective composition. These are reviewed every 3 years. Some of the specific undertakings to ensure representation in decision-making by relevant groups are listed below:

All members are entitled to nominate and/or vote for the positions of President, Vice President and Treasurer.

Members are entitled to nominate and/or vote for their respective State Director to the Board. These Directors have an important role in using both formal and incidental meetings of the membership and other critical stakeholders within their area to provide feedback to the Board based on these communications.

Registrar representation is requisite for the Board, Education Council, Research Committee, Assessment Committee, and Vocational Training Committee. (For further information on Registrar representation: See Section 7.2).

All committees (with the exception of the Board and the Executive Committee) include appropriately qualified non-executive members. The entire membership is notified of position vacancies and encouraged (where eligible) to submit an expression of interest for a position for the consideration of the Board.

Representation on the International Medical Graduate (IMG) Assessment Committee, Vocational Training Committee and Post-Fellowship Training Committee includes respectively; IMGs, Vocational Trainees (one from each training pathway), and Post-Fellowship program trainees.

Additionally, as a general principle selection of non-executive members will consider balance of gender, geography and age and (where appropriate) experience and expertise.

(See Attachment 1.4 College Committees Terms of Reference.)

(See Attachment 1.5 College Committees Current Membership)


1.1.3 The education provider’s internal structures give priority to its educational role relative to other activities.

The College’s charter is to define and provide education and standards for general practice particularly for rural and remote settings. ACRRM was formed by a plebiscite of members of the Rural Doctors Association of Australia (RDAA) specifically to enable delivery of training and standards that defined and prepared practitioners toward best practice, rural and remote general practice. It was envisaged (and continues to be the case) that the industrial aspects of supporting rural and remote doctors would remain the province of the parent organisation, RDAA. The Board includes an RDAA representative which facilitates continuity of this division of organisational roles. The ‘Articles of Association’ that established the organisation and articulate its functions clearly reflect the prioritisation of its educational role. Of the 19 ‘Objects’ listed, 16 deal directly with education issues the remaining three, indirectly. The effectiveness of the College’s internal structures to ensure its education focus is well evidenced.

The role of Censor-in-Chief is in fact, and in common perception within the organisation, a central leadership role. Typically, (and as is currently the case) it is held by office bearers who have served previously, or have gone on to serve, as College President.

Seventy-five percent of all ACRRM core staff (81% of total FTE) are engaged directly in education, education support and standards services delivery. Of the 61 core staff that ACRRM currently employs, 45 (42.1 FTE) have roles directly related to delivery of educational programs, standards and support. Additional to this percentage, the College contracts certified practitioners to contribute to its education, assessment and certification activities. Last year 215 such practitioners contributed to the College’s activities. Table 1.1: ACRRM Staff involved in educational activities

(See Attachment 1.6: College Organisational Chart. (NB This does not include contracted assessors and medical educators)).

ACRRM STAFF

Directly involved in education, standards and support

Overall Management

Business services, membership, marketing

CEO and support staff 3

Membership services 2

Marketing services 4 Corporate services (Director) 1

Financial services 3 Policy and strategic projects 3 Program Operations (Director) 1

Evaluation coordinator 1 Censor-in-chief 1

Fellowship services 8 Prevocational Training services 8 Vocational Training services 13

Online services 8

IMG operations 5 TOTAL 46 15


1.2 Program Management

1.2.1 The education provider has established a committee or committees with the responsibility, authority and capacity to direct the following key functions:

planning, implementing and reviewing the training program(s) and setting relevant policy and procedures

setting and implementing policy and procedures relating to the assessment of overseas trained specialists

setting and implementing policy on continuing professional development and reviewing the effectiveness of continuing professional development activities.

Training Programs The ACRRM Board maintains ultimate responsibility for setting all policy and standards which it regularly reviews. The Education Council oversees these standards and clinical governance matters on behalf of the Board. Many operational aspects are further delegated to the Education Council’s respective subcommittees (i.e. Assessment Committee, IMG Assessment Committee, Post-Fellowship Education Committee, Board of Examiners, Professional Development Committee, and, Vocational Training Committee). All these subcommittees are required to regularly report to the Education Council/College Censor, providing advice and feedback. Their respective Chairs are all members of the Education Council. Overseas-trained Specialists Assessment ACRRM has in place an IMG Assessment Program for the express purpose of assessing the credentials of IMG’s wishing to practice in Australia. The IMG Assessment Committee has responsibility for developing and recommending IMG Assessment policy and standards. These are ratified by the Education Council and where appropriate the ACRRM Board. The Committee reports to the Censor-in-Chief after every meeting. The Assessment program and the IMGA Committee are supported by the IMG Operations Coordinator, a medical educator, a work-place based assessment coordinator and four additional dedicated staff. Contracted assessors are engaged to deliver structured assessments.

Continuing Professional Development The Professional Development Committee has responsibility for developing and implementing policy and standards in the PDP. The Committee is authorised to report on PDP activities and certify compliance. The Committee reports regularly to the Education Council. It is a term of reference for the Committee to: “ensure that appropriate systems are in place to collect data, which will enable the committee to monitor and evaluate the effectiveness of the PDP over a period of time.” The PDP and Professional Development Committee are supported by dedicated College staff in the Fellowship Services unit. The capacity of these programs is enhanced by the College’s expansive suite of online services.

(As above See Attachment 1.4: College Committees Terms of Reference.)


1.2.3 The education provider’s education and training activities are supported by appropriate resources including sufficient administrative and technical staff.

Appropriate Administrative Model ACRRM is unique among Australia’s specialist Colleges. It exists in order that a need be met for the doctors that practice in rural and remote communities across Australia: (1) to be adequately prepared and certified to meet the distinct medical exigencies of their communities; and, (2) able to access the training and educational support that this engenders with minimal loss of services to their communities. Both these challenges required a paradigmatically distinctive model of training and program delivery. Given Australia’s geography and transport infrastructure, the major capital city hubs model that is common among Australia’s medical professional colleges is a poor fit. Our members in the Kimberley for instance are over 2000 kilometres from a potential hub in Perth; and even in smaller more populous states like Victoria, the practicalities of accessing a state hub in Melbourne can be prohibitively time-consuming. Consistent with its mission, the approach that ACRRM has taken is to develop a highly distributed model. This allows the College to provide certified training to members in locations as remote as Macquarie Island, Antarctica. The central College secretariat is augmented by an extensive network of rurally-based experts drawn largely on its membership; and, a major and continuing investment into developing innovative solutions to maximise the capabilities of distributed program delivery. Many of the College’s key administrative staff are also distributed in various locations across the country and the College has established videoconferencing and remote collaboration arrangements to connect staff in a virtually connected environment. Outcomes of the ACRRM Approach This approach has been so effective that much of what is now considered the essential architecture of Australia’s remote medical education/professional development and telehealth framework is a direct result of ACRRM innovations. Some of the programs ACRRM has conceived, piloted and/or delivered include:

Remote Vocational Training: The concept of the Remote Vocational Training Scheme was developed based on an ACRRM submission.

John Flynn and bonded scholarships schemes : ACRRM secured initial funding for these programs and helped to establish their function as not just opportunities for experience but also for peer-networking, personal support and to build rural professional identity, inspiration and aspiration.

Rural Generalist Training Pathways: ACRRM has promoted and advanced this concept from inception. ACRRM was the lead College in the partnership that founded the initial program in Queensland. That program was built on the ACRRM curriculum, and many of its key developers/directors are executive College Fellows. The Queensland program has provided the model for all subsequent programs.


Prevocational community-based training: ACRRM prepared the initial proposal that led to establishment of the Prevocational General Practice Placement Program (PGPPP) and as such to prevocational community-based training.

Workplace based assessments and virtual classrooms: ACRRM continues to trial workplace based clinical assessment tools with commissioned educationalist teams and with the support of the Department of Health. These, when fully developed and accredited, will be another world first innovation for the College. The College has also pioneered virtual classrooms; online teaching session delivery certified to equivalence with face-to-face vocational training and professional development. Modules are available now across most related disciplines. Candidates as remote as Thursday Island have been able to meet their face-to-face requirements toward professional development certification.

Telehealth: ACRRM has been a national pioneer in this area. It is contracted by the Commonwealth to deliver the nation’s largest telehealth online platform in a virtual community that was accessed by over 25,000 clinicians in 2013. In partnership with a wide range of stakeholders, ACRRM developed the national Telehealth Standards framework that is used by other medical colleges and professional organisations including the Royal Australian College of Physicians (RACP), RACS (Royal Australian College of Surgeons) and NACCHO (National Aboriginal Community Controlled Health Organisation). The College has operated ‘Tele-Derm’ a pioneering national specialist dermatology referral and educational service for over a decade using store-and-forward technology.

Rural Procedural Grants Program: The College conceptualised; advocated for; and, for over a decade managed this national program that has been a vital tool in ensuring the continuing provision of quality-assured procedural services for rural and remote Australian communities.

Staff and Resources

ACRRM currently has a total core staff of 61 and further staff on casual and contractual arrangements. The overall management structure includes the CEO, strategic project officers, general administration, and membership staff, and the Directors of Corporate Services and Operations.

ACRRM FTE core staff capacity to manage different program areas are as follows:

Vocational Training and Assessment: 11.6 FTE

Prevocational Training: 8 FTE Fellowship Services: 8 FTE

IMG Operations: 4.1 FTE

Online Services: 7.4 FTE

Whole-of-College Operations: 17.4 FTE For major projects as appropriate the College may engage external consultancies and staff on contractual arrangements. The College delegates delivery of AGPT program vocational training and some aspects of the training post accreditation to ACRRM-accredited RTPs. This is consistent with its decentralised approach. Similarly, registrars on the RVTS are delivered by the staff of this program.


Program Teams The operation of ACRRM programs is guided and directed by the Director (Operations). The College has three program teams: Prevocational Training; Vocational Training and Assessment; and Fellowship Services. There are also more loosely constructed units for International Medical Graduate Operations and e-health. Each is led by a Program Manager and incorporates administrative and technical staff including one or more designated medical educators to support delivery. The Program Manager also leads liaison with the related College Committee/s. The team draws upon the services and resources of the marketing and communications unit and online services and contracted clinicians to support delivery. Medical Educators ACRRM has in place a core team of six qualified medical educators to support the development and delivery of educational programs. These include the Director of Education and Censor-in-chief. It is in the process of making an additional appointment. (Two of these are employed under contract). Their role extends to training Fellows to enable them to contribute to the program in educational roles. Contracted Educators and Contributing Clinicians The permanent staff members are assisted in their program delivery by a select group of trained, certified clinicians who are predominantly College Fellows based in rural and remote practices across the country. 215 of these contributed to programs last year and a further 380 provide services as mentors in the Colleges prevocational programs. The utilisation of community-based, trained Fellows is consistent with the distributed-model approach of the College. (As outlined above this is essential to meeting its charter to advance Rural and Remote Medicine). The model is sustained by a process of recruiting and nurturing aspiring educationalists among its membership across the country to contribute to College programs. The Director of Education and also other medical educators on staff have a role in this training. Online Services Technology assisted delivery of programs is an essential element of the ACRRM model. Accordingly ACRRM has assembled some of the leading minds in telehealth and remote medical education delivery in the country. The value of ACRRM’s expertise in these areas is well recognised by the wider medical and health service profession:

Its Telehealth Directory and discussion platform was utilised by over 25000 clinicians in 2013.

It has developed Telehealth Standards for multiple specialty colleges and other health organisations.

It provides national delivery of the annual GPET Online Conference “Research Week”

It has been contracted by the Commonwealth to deliver its national Telederm program – providing specialist dermatology services to health services anywhere in the country.


The College has pioneered and continues to lead development of world-first technology and instruments to enable and enrich medical training activities in rural and remote locations. The College is constantly exploring and innovating new ways to maximise the learning possibilities with minimal dislocation for practitioners. Customer Relationship Management Project The Customer Relationship Management (CRM) project aims ultimately to improve the quality and quantity of services deliverable within the College’s staff and resource base. The College has invested significant resources into establishing this system which will improve information and services. A range of expert consultants have been contracted for extended periods by the College to assist with development of the new system (e.g. Project Manager, Web Developers, and, Business Analysts). (For more information: See Section 4.2).

(See Attachment 1.6: College Organisational Chart.)

Table 1.2: ACRRM Staff List Program Area No. of Core Staff No. of contracted Medical

Educators

Overall program management

CEO and support staff 3

IT Services 8 Policy and Strategic Projects 3

Direction (Corporate Services) 1

Financial Services 3 Director (Operations) 1

Marketing 4

Membership services 2

Evaluation 1 Censor-in-chief 1

Total 27 (24.8 FTE) Specific Programs

Prevocational Training 8 (8 FTE) 48

Total 8 (8 FTE) 48

Vocational Training 13 (11.6 FTE)

Bi-College Accreditation 10

MiniCEX 18

StAMPS 32

Exam Preparation 8

Study Group 9

Virtual Classroom 9

Selection 4

Total 13 (11.6 FTE) 90

Professional Development 8 (8 FTE) 48

Total 8 (8 FTE) 48

IMG Operations 5 (4.1 FTE)

PESCI 77

Total 6 (5.1 FTE) 77

Total contracted educators 215

Total core staff 61 (56.5 FTE)


Structural Expansion and Adjustment since 2010

Since 2010 the popularity of ACRRM fellowship training and PD programs has increased substantially. Its vocational training pathway enrolments have increased by 40% creating associated demands on the PDP program. The College has not only expanded its staff base from 51 (45 FTE) to 61 (56.5 FTE) to meet these needs but has made fundamental structural changes to its composition. Since the last accreditation the College has:

Created distinct educational program areas with dedicated managers, technical and administrative staff to support educational delivery;

Appointed a full time Evaluation Coordinator;

Put in place a communications team to enhance information provision to members/stakeholders;

Created an executive level role ‘Director of Strategic Programs’ which strengthens resources and provides strategic direction to innovation.

Appointed a Director (Operations) and Director(Corporate) to assist in guiding growth and to provide improved succession planning and risk mitigation for executive roles within the organisation;

Put in place an expansive online services team to support all activities and drive development of innovative new educational tools;

Created two permanent paid positions for educationalists to offer high level input and guidance to educational activities;

Put in place a CRM team using long-term contracted arrangements to enhance operational effectiveness through establishment of a single online platform framework.

Key new senior management positions created since 2010 are listed below: 2014

Medical Educator (Independent Pathway) – to be appointed. (0.4 FTE) 2013

Bi-College Program Manager – Joan Burns (1 FTE) This position is based in Adelaide and co-funded with RACGP and is responsible for national management of the Bi-College RTP accreditation process.

2012

Education Director – Dr Sandra Mendel (0.4 FTE)

Censor in Chief – A/Prof David Campbell (0.25 FTE) instituted as a paid positions Director of Strategic Programs: e-health - Vicky Sheedy (1 FTE)

2011

Director (Corporate Services) – Darryl Perkins (1 FTE)

Director (Operations) – Jeremy Simmons (1 FTE) Evaluation Coordinator (1 FTE)

Demonstrated Capacity to Meet Educational Needs

The adequacy of the staff and resources is best demonstrated by their ability to meet their core educational requirements.

Expanded demand: The general satisfaction with the programs provided is demonstrated by their increasing popularity. Enrolments in all three programs continue to climb. The IP managed entirely by the College was oversubscribed for the past two years. IP enrolments have increased by over a third and are projected to increase again for 2014. (See Tables 1.3 and 1.4 below).


Demonstrated meeting of demand: The program delivery infrastructure has and continues to meet this demand and transform its enrolled registrars into fully-accredited, broad scope rurally-proficient general practitioners. New Fellowships (of ACRRM trained doctors) increased by 38% last year and are anticipated on current projections to increase by a further 15% this year. The increased operational demands of more Fellows requiring CPD have been met with a solid compliance rate of 95% over the last triennium. It is also noteworthy that the gap between the number of vocational training enrolments and the number of Fellowships is closing; (from 82% in 2010 to an anticipated 65% in 2014).

Indicators of adequacy of staff and resources

- Accredited Training Posts have expanded 32% over the accreditation period from 574 in 2010 to 757 anticipated for 2014.

- The PDP programs enrolments have increased by around 4% over the

accreditation period and there are 925 more accredited activities available to participants.

- Assessment has expanded both in the number of assessments undertaken

(increasing 53% over the accreditation period; from 187 in 2010 to 354 in 2013); and in the sophistication of the suite of tools available. (These innovations are important as they not only enhance validity and reliability but also improve their practical accessibility to registrars’ in rural and remote locations).

- General satisfaction with the College’s provision of training and education is

demonstrated by the 86% satisfaction rating in its 2013 Member Survey. It is also noteworthy that Registrars reported 85-96% percent satisfaction with the RRMEO Online modules provided11. The 2013 Registrar and Members Surveys also point to some general areas for improvement as outlined below.

Challenges for program resourcing

- The principle challenges for staff and resourcing are also implicit in the College’s raison deter. Doctors pursuing professional development and training in rural and remote locations encounter issues of isolation and relative lack of local staff and resources which require an energetic and tailored response.

- The College recognises that addressing the sense of isolation that doctors

experience and its extenuating issues: (e.g. problems of connectivity; lack of peer communication; and, a lack of in-person support when problems arise); is of utmost importance for rural and remote doctors. The College is committed to driving innovation and generating new solutions to continuously improve the educational experience it is able to provide in rural and remote locations.

- Considerable resources have been invested in ACRRM’s strategic development

projects such as its e-health offerings; online services generally and its systemic improvements through the CRM system.

11 ACRRM Registrar Survey 2013


- As outlined above major efforts have been put toward recruiting and training members across the country to contribute to its programs. (Nearly 80 Fellows contributed in 2013 to our vocational and CPD programs and an additional around 180 provided services to our prevocational programs).

- The College has actively sought to address social isolation, increasing social

interaction opportunities including creating social networking platforms; provision of virtual classroom facilities for both peer and supervised interactions; and more face-to-face interactions. (For further details: See Section 7.3)

- Another major focus area has been toward improving the workability of the

College’s relationship with the RTP's to ensure that cooperatively the best possible educational experience is provided to members. Considerable efforts have been made in this area. It should be noted that major changes to the AGPT system have been foreshadowed. The College is engaging in talks with the Commonwealth to ensure these optimally support the programs the College is able to deliver. (For further details: See Section 8.3 Condition 23)

- Specific areas are being addressed in response to Evaluation Feedback. The

Registrar and Supervisor Surveys both pointed to a need to improve the ease of finding training and other information on ACRRM websites. Improvements in this area are a specific brief of the CRM project. (For further details: See Section 4.2 Recommendation KK)

- Specific issues related to Registrar Assessment support are also being

addressed. (See Section 7.2).

Table 1.3: New College Vocational Training Enrolments 2010 – Present**

Pathway 2010 2011 2012 2013 % Increase Mid-2014 VPP 77 78 102 120 56 68 (126)*

RVTS 16 22 24 20 25 32 (64)*

IP 63 49 45 68 8 35 (70)*

Total 156 149 171 208 33% 136 (272)*

*Projected total for full year ** All enrolments as at June each year.

Table: 1.4: New College Fellowships 2010 – Present**

Pathway 2010 2011 2012 2013 % Increase Mid-2014 VPP 12 14 27 31 160 23 (46)*

RVTS 3 6 4 5 66 2 (4)*

IP 12 17 22 37 208 17 (34)*

Total 27 37 53 73 170% 42 (84)*

*Projected total for full year ** All enrolments as at June each year.


1.3 Educational expertise and exchange

1.3.1 The education provider uses educational expertise in the development, management and continuous improvement of its education, training, assessment and continuing professional development activities.

The ACRRM operates in the rural and remote medical training milieu in which collaboration of available resources is essential and common practice. It has co-evolved with (and has been instrumental in the development of) the national infrastructure of academic rural and remote medicine: including rurally-based medical schools such as at James Cook University (JCU); the network of rural clinical schools and university departments of rural health; and, academic departments of rural and remote medicine. The College has naturally drawn upon these networks (including many of their key players who have held leadership positions in the College) in the development of all its educational and assessment programs. The model of having medical educators on staff and amongst our membership working collaboratively with the existing expertise in Universities and other academic institutions; and, the profession, is appropriate to the Specialty as it enhances vertical integration and interdisciplinary cooperation essential to the viability of Rural and Remote Medicine. For these reasons as the College matures it continues to adhere to this approach.

Expertise within the Membership: The College leadership has consistently included members with educational expertise. The College’s governance frameworks ensure continuity of this representation. The Board includes an Academic Director who must hold a senior position in medical academia; and who has ex-officio membership of the Research Committee and Education Council. The College Censor-in-chief as mentioned above also requires academic standing and holds one of the Colleges senior-most leadership roles. The Censor oversees the activities of all education related committees. The College actively nurtures academic careers amongst its membership and has a team of six medical educators who facilitate this process. This is also encouraged through the activities of the Research Committee and more recently through its development of the Advanced Skills Training module of Academic Practice.

Expertise within Permanent Staff: The permanent staff of the College includes six medical educators and ACRRM will soon be appointing an additional educator to staff. These contribute to development and delivery of all educational programs and have a role in training Fellows to provide educational services.

Expertise drawn from Collaborations and Consultancies

Prof Tarun Sen Gupta was part of the international panel of experts that developed the College Assessment framework and published extensively on their work. As Chair of the College’s Assessment Committee he continues to offer the highest of credentials to its ongoing development and oversight. (See Attachment 5.6: ACRRM Assessment Program Publications).

Professor Anthony Maeder, University of Western Sydney (UWS) also affiliated with Standards Australia (Telehealth); assisted in the technical aspects associated with the development of the ACRRM Telehealth Standards Framework.


In 2011, the College worked with a senior academic from Flinders University to develop its Rural Emergency Obstetrics Training (REOT) course.

The JCU School of Medicine has partnered with ACRRM to develop an online orientation and training package for registrars. The School contributes to training for formative assessors and summative examiners in the Structured Assessment of Multiple Patient Scenario (StAMPS) examination and is actively involved in the ongoing development of the face-to-face examiner training for StAMPS.

1.3.2 The education provider collaborates with other educational institutions and compares its curriculum, training program and assessment with that of other relevant programs.

Collaboration with other Colleges There is ample evidence of ACRRM’s general approach of working collaboratively to meet its College goals.

In 2013 ACRRM successfully commenced the Bi-College Accreditation Program for the AGPT program; the result of a lengthy and ongoing collaborative effort between the College, RACGP and GPET. This has established an important, clear line of accountability between the College and the RTPs that deliver its program. The unique program design enables the Colleges to make independent decisions regarding an RTP against their respective standards. It is understood to be the only such program in the world involving collaboration of two independent Colleges.

ACRRM is represented on, and contributes to all key GPET Committees. It’s an active member of the Committee of Presidents of Medical Colleges (CPMC). It is an active member of the United General Practice Australia (UGPA).

The College has been actively engaged with GPET and the RACGP in reviewing the current selection mechanisms for applicants to the AGPT program. More recently the College is also engaged in some initial discussions with the Commonwealth regarding future directions for these processes under the forthcoming new framework.

As ACRRM embarks on its scheduled review of its AST curriculum it will be working with its Joint Consultative Committees (JCCs) in Anaesthetics; Surgery; and, Obstetrics and Gynaecology. The JCC process is a tripartite arrangement between ACRRM, the RACGP and the relevant colleges, which considers curriculum and assessment requirements for general practitioners.

ACRRM was commissioned to produce a Report into the development of a national approach to rural generalist training. The Report, “The Rural Way” was produced collaboratively with representatives from the RTP network (ACE), the rural clinical schools peak body, (FRAME), the Confederation for Postgraduate Medical Education Councils (CPMEC), and, the RDAA.

Certificate of Primary Care Dermatology was developed in conjunction with the Australasian College of Dermatology in 2011. This is one of numerous educational programs that have been developed by or in conjunction with other medical specialties or craft groups (and accredited by the College), and made available as part of the suite of educational programs on RRMEO.

ACRRM has developed a Telehealth Standards framework which has been applied by key medical organisations including the NACCHO, the RACS, the RACP; and the College has been commissioned to develop and deliver online education based on these standards.


International Relationships

In 2012, a formal Memorandum of Understanding was signed establishing mutual recognition of Fellowship between ACRRM and the College of Family Physicians Canada (CPC).

Similarly, recognition arrangements are in place to allow FACRRMs to be recognised by the Division of Rural Hospital Medicine in New Zealand (RHMNZ).

In-principle agreements have now been reached with the Royal New Zealand College of General Practitioners (RNZCGP).

The Memorandum of Understanding signed with the University of the Witwatersrand and Africa

Health Placements enables ACRRM registrars to take up training placements in South Africa and Fellows interested in teaching, to work for periods of time in South Africa.

Underlying all ACRRM programs is the aspiration to define and refine the scope of practice that can ultimately lead to doctors in locations remote from urban facilities enabled to provide the best possible service to their communities. ACRRM’s World Summit in 2013 lead to the development of an initial Consensus Statement and has kick-started an ongoing international conversation in which the College continues to work with senior members of their specialty area in nine different countries to refine and develop the consensus position.

ACRRM has an established internationally-based network of like-minded esteemed academics to provide input to and critique its educational activities. Some key individuals among these that have contributed to ACRRM’s educational development include:

- Professor Roger Strasser (Dean, Northern Ontario Medical School, Canada), - Associate Professor Michael Jong (Associate Professor of Family Medicine, Memorial

University of Newfoundland, Canada), - Professor Karl Stobbe (Assistant Dean of Medicine, McMasters University, Canada,), - Professor Ian Couper (Professor of Rural Health, University of Witwatersrand, South Africa), - Dr John Wynn-Jones (Director, Institute of Rural Health, UK), - Professor Richard Hays (Chair of Medical Education and Head of School, School of Medicine,

Keele University, UK), - Dr Tim Malloy (Chair, Rural Faculty, Royal New Zealand College of General Practice, NZ) - Dr Garry Nixon (Chair, Division of Rural Hospital Medicine, NZ), - Professor Desmond Gorman (Dean, School of Medicine, University of Auckland, NZ), - Dr Ruth Ballweg, (Director, Physicians Assistance Program, University of Washington, USA), - Professor Roger Rosenblatt (Professor of Family Practice, University of Washington, USA), - Professor Arthur Kauffmann (University of Mexico, Mexico).

Comparison with other relevant programs As part of its quality assurance and continuous improvement program, ACRRM conducted a desk top review of general practice postgraduate training programs and compared curricula, training program content, accreditation, assessment, evaluation and outcomes with the general practice postgraduate vocational training programs of Royal Australian College of General Practice (RACGP), RNZCGP, DRHMNZ, Royal College of General Practitioners (RCGP), College of Physicians of Canada and American Board of Family Practice (ABFP). Curricula have been identified for a range of general practice programs. A comparison of these was undertaken as part of the review of the ACRRM Primary Curriculum. (For further details of curriculum review: See Section 3).


Investigation into other related curricula overseas has underpinned the subsequent process that the College has undertaken; to endeavour to establish quality assured mutual recognition arrangements with a range of international specialty Colleges; (See Section 1.3.2: International Relationships). Continuing Curriculum Review and development has been informed by the following findings regarding some of the more closely related curricula.

Based on a review of other programs, the length of ACRRM training compares well with programs of a similar scope.

The combination of FRACGP and FARGP if completed concurrently, require the same total length of postgraduate experience/training to complete as FACRRM i.e. one year postgraduate experience and four years vocational training.

Fellowship of RNZCGP and FDRHMNZ if completed concurrently, is one year longer for the total length of postgraduate experience/training i.e. six years total, two years postgraduate experience and four years vocational training. However vocational training alone is four years which is the same as FACRRM.

The Membership of RCGPs has been revised moving the minimum for UK doctors from four to five years post graduate experience/training at the completion of vocational training.

Canada only requires two years vocational training and in the USA the length of the Family Medicine residency is generally three years.

ACRRM has taken a similar approach to the other programs reviewed. Minimum and/or maximum time requirements are set to ensure that registrars have exposure to appropriate environments to enable them to achieve the range of skills, knowledge and experience required for the scope of practice. ACRRM has three pathways to Fellowship and documented RPL pathways, this compares well with options and flexibility provided in other programs.

ACRRM’s assessments compare favourably with those in other programs. ACRRM uses five different summative assessment modalities, many other College’s programs reviewed only use two modalities of summative assessments.

Furthermore, ACRRM’s assessment modalities enable registrars to opt to participate in summative assessment in their home/practice location. This provides additional flexibility for registrars and workforce benefits for Australian rural and remote communities.

(See Attachment 1.8 Curriculum Comparison Review Report)


1.4 Interaction with health sector

1.4.1 The education provider seeks to maintain constructive working relationships with relevant health departments and government, non-government and community agencies to promote the education, training and ongoing professional development of medical specialists.

1.4.2 The education provider works with healthcare institutions to enable clinicians employed by them to contribute to high quality teaching and supervision, and to foster peer review and professional development.

Rural and Remote Medicine Paradigm Working collaboratively with the healthcare providers and communities to enable program delivery is inherent to all the College’s educational activities. Training with the ACRRM prepares Fellows for a full scope of generalist practice. This entails the ability to work in a wide range of community settings, hospitals, emergency departments and to working in flexible ways with the wider healthcare delivery team, using all available local resources. The complex, collaborative nature of leadership in Rural and Remote Medicine is nowhere better reflected than in the multiple roles held by the College’s leadership team. For example: the College President is also Dean of regional medical school that incorporates a Rural Clinical School and a University Department of Rural Health; is a past Chair of the Federation of Rural Australian Medical Educators (FRAME); on the Board of a regional Health and Hospital Service; and on multiple national health policy boards. The College Censor is a founding Board member of a Rural Workforce Agency, as well as a medical academic, the director of a Rural Clinical School, Board member of a Regional Training Provider, a rural generalist procedural practitioner, and is a vocational training instructor, mentor and supervisor. Other leaders have similarly diverse roles across multiple agencies with a common theme of serving rural and remote communities. The ACRRM is defined by its imperative to achieve skilled rural workforce outcomes and, in this respect interaction with the government and community agencies to ensure that training and accreditation frameworks are able to meet community needs is also necessarily part of the College’s core business. Demonstration of constructive approaches to collaboration

The College works extensively with health services at regional and state levels with Medicare Locals, Local Hospital Network Boards and individual hospitals that provide training for general practitioners. It contributes to the operation of health services through provision of formal submissions, briefings and delegations and participation in formal committees and conferences. Building strong relationships with all of these networks is a priority and represents a significant challenge given the very considerable number of such organisations across the country. Contributing at the Policy and Advocacy Level The College regularly contributes to, and promotes dialogue on health policy in areas of importance to its membership and mission.

It regularly makes submissions to formal enquiries as well as providing informal briefings and delegations as required. It made submissions importantly in 2012 to the Parliamentary Inquiry into registration processes and support for overseas trained doctors 2012; and the


Inquiry into factors affecting the supply of health services and health professionals in rural areas. 2012. It contributed strongly to the development of the Mason Report. It took a lead role in the development over 2012-2013 of the HWA National Rural Medical Generalist Framework.

The College advocated for, and in 2014 was commissioned to produce a seminal Report to the Department of Health on a National Rural Generalist Framework. As outlined below, it has also been instrumental in advocating for, and establishing RG Training Pathways across the country.

ACRRM contributed to a wide range of Australian Health Practitioners’ Registration Authority (APHRA) consultations including registration standards, international medical graduate and telehealth guidelines including; preparing formal submissions to Minister Dutton’s review of the Personally Controlled Electronic Health Records and the Review of Medicare Locals.

The College actively participates as a member organisation of the CPMC and its educational subcommittees.

Contributing at Operational Forums

ACRRM meets regularly with all health departments ensuring effective collaboration with relevant personnel and networks for its education, training and professional development activities.

The College regularly participates as a formal member of key state health department committees and reference groups (see list below). Additionally ACRRM Fellows participate as members of clinical privileging committees and within local Divisions of General Practice to contribute to local networks of education, training and professional support for the specialty.

The role of the State Directors (who commonly represent the College on the committees below) is an important mechanism for managing and monitoring interactions with the health sector. State Directors provide feedback on grassroots issues to the Board through their regular and ad hoc contact with members of the College. In turn these directors contribute to policy deliberations and use their experience to inform debate.

ACRRM actively participates in jurisdictional meetings between state health departments and peak bodies for general practice in Queensland, New South Wales, Western Australia and South Australia (e.g. ACRRM, RACGP, Divisions of General Practice/Medical Locals, Rural Workforce Agencies, and, Association of Academic General Practice). These committees generally meet on a quarterly basis. Similar committees do not exist in other jurisdictions at this time.

The College contributes extensively to GPET and RTP network which includes 17 independent companies. This extends to Board level membership and engagement from executive through to operations level.

An example of ACRRM’s ability to enhance service delivery in rural and remote areas through collaboration is the Department of Health sponsored Telehealth initiative. This is led by ACRRM and involves 25 national organisations (including; the Department, Medicare Locals, Standards Australia, Australasian Telehealth Society, the National Rural Health Alliance, and national representatives of nurses, midwives, and practice managers). It provides the principle platform linking general practitioners to telehealth specialist services, and provides information, discussion forums, and standards.

ACRRM worked closely with the Queensland Government to co-found and develop its pilot Rural Generalist (RG) training program which provided the national blueprint. The College


subsequently joined working groups to establish RG programs (leading to Fellowship of ACRRM and recognition in the specialty of general practice) in New South Wales, Victoria, Northern Territory and Tasmania. Colleges are not represented in RG working groups in Western Australia or South Australia. In those jurisdictions, ACRRM liaises regularly with relevant personnel.

(See Attachment 1.7 ACRRM Representation on External Groups (Full List)).


State-based Committees with College representation (since 2010)

New South Wales: NSW Ministerial Advisory Committee on General Practice

NSW General Practice Advisory Council

Rural Generalist Medicine Program Governance Committee General Practice Advisory Network Primary Care Working Group Better Cardiac Care for Indigenous Peoples Working Group NSW GP Unity Group NSW Rural Training Forum NSW Melanoma Network

Northern Territory: Rural Generalist Medicine Reference Group

Queensland: Rural Generalist Medicine Reference Group State Recognised Practice Committee for Rural Generalist Medicine State Recognised Practice Committee on Generalist Emergency Medicine State Recognised Practice Committee on Mental Health State-wide Telehealth Governance Committee State Recognised Practice Committee on Rural Generalist Surgery Primary Care Billing Reference Group Radiation Advisory Council Queensland Health Physician Assistant Steering Committee General Practice Advisory Council Northern District Credentialing & clinical Privileging Committee Cairns Health Service District Credentialing Committee Fraser Coast Clinical Privileges Committee Professional Conduct Review Panel Annual Peer Review Panel for MSRPP advancement to senior status

South Australia: Roads to Rural Practice Reference Group (Rural Generalist Medicine) Primary Care Advisory Group SA Health GP Forum SA Institute of Medical Education (PGMEC) Rural Training Pathway Expert Committee

Tasmania: Rural Generalist Medicine Reference Group Tasmanian GP Unity Group After Hours Primary Care Technical Working Group

Victoria: GP Procedural Services working group Nurse Practitioner Advisory Committee Rural Birthing Services Planning Reference Group Victoria Seymour Credentialing of VMO’s Committee Cobram District Hospital Credentialing and Privileging committee Rural Workforce Agency Victoria Postgraduate Medical Council Victoria

Western Australia: Rural Practice Pathway Reference Group (Rural Generalist Medicine) General Practice Network Postgraduate Medical Council of WA – Education Committee GP Stakeholder Group WA Country Health Services For Planning Delivery of Clinical Services – Clinical Reference Group Medical and Dental Workforce Council of Western Australia Clinical and Vocational Reference Group

Mental Health Nurse Incentive Program Expert Reference Group


1.5 Continuous Renewal

1.5.1 The education provider reviews and updates structures, functions and policies relating to education, training and continuing professional development to rectify deficiencies and to

meet changing needs.

ACRRM continuously reviews and adjusts its structures, functions and policies to ensure its activities keep pace with the changes in the primary care sector. It also aims to maintain their relevancy to the needs of its members and their communities. It responds to new opportunities, goals and targets to improve access to quality care for people living in rural and remote and under served communities. (For more information; see Section 6) Internal Program Needs The College standing committees undertake constant informal review, making adjustments and clarifications as needed. Evaluation Surveys are in place for all aspects of program delivery. Reports on each evaluation are submitted to line managers to table with the relevant committee for consideration and action as appropriate. Regular formal review processes are also applied to each program. These include opportunities for stakeholders and communities to contribute to the quality improvement of offerings. External Policy and Social Shifts The College Board and committees engage in a continuous dialogue, assessment and decision-process regarding the wider strategic and reform agendas that have the potential to impact on general practice standards or requirements, particularly in the rural and remote contexts. Ultimate decision-making on these matters are the preserve of the College Board.

Continuous renewal occurs formally through an annual review of the College’s Strategic Plan. This review includes an environmental scan and analysis of changes in the political, organisational, technological and professional environment in which the College operates. Key issues are referred to the relevant standing committees for policy and/or operational considerations. Ultimate decisions lie with Board.

This also occurs informally: in response to issues arising from key policy fora, and, regulatory and operational external committees on which ACRRM is represented. It also occurs incidentally from the College’s operations and from informal feedback from members and their communities. Some examples of the College responding to this information include: - In response to the Australian Safety and Quality Goals for Health Care, ACRRM has taken

a consumer led approach to the development of standards, curricula and education. The College has engaged with the National Rural and Remote consumers of Australia (e.g. they participate as a members of the ACRRM and Telehealth advisory committee and, the telehealth education resources incorporate the viewpoint of the patient and endorsement of this rural and remote consumer group).

- The College has developed a module and updated clinical guidelines to ensure that general practitioners have access to education relevant to management of transient ischemic attack or stroke in line with national clinical standards.

- ACRRM has been undertaking a range of initiatives to expand its educational offerings

and to encourage member interest in the area of Mental Health which has been recognised by the Commonwealth as a key area of need (particularly in rural and remote communities).


1.6 AMC Recommendations

AA Develop and implement strategies to engage a wider range of fellows in ACRRM governance and decision making. (Standard 1.1)

The College has extended its range of Evaluation Reports and has introduced a requirement where possible/appropriate to feed back outcomes of reports to the surveyed group. These reports provide an important mechanism by which all members have an opportunity to influence decision-making. Additionally there is a feedback link permanently available on the website.

Social networking sites encouraging peer-networking have been established to encourage across College feedback particularly to elected representatives. The College has developed Social Media Guidelines for members and a staff. ACRRM has established an e-health site/virtual community to serve the growing numbers of fellows adopting technology to enhance services to patients. This site includes a range of discussion forums including an active clinicians working group forum which provides ACRRM with advice to inform its contribution to the National E-Health Transition Authority (NeHTA) and Departmental consultation regarding the clinical usability of the Personally Controlled Electronic Health Record (PCEHR) Including pathology and diagnostic image integration.

The College has continued to successfully employ a range of communication strategies to allow its geographically disbursed membership to engage meaningfully in the process of policy development, critique and decision making. Communications have been increased in their frequency across all College programs both through the traditional means (e.g. publications, newsletters, meetings, workshops, conferences and committees), and also through the use of information technology to engage and inform with its members and stakeholders. The virtual classroom facility has created an invaluable tool to enhance the effectiveness of remote communications. Technologies such as Facebook, Twitter, YouTube, instant messaging, blogs and secure discussion groups within RRMEO all allow our members to engage in a two-way, easy, timely and personal manner. Social media is proving to be increasingly effective and popular across all age groups within the College but particularly amongst student and registrar members.

Communications regarding elections of representatives are advertised through Countrywatch, FACRRM Fundamentals and other social networking platforms.

ACRRM policy requires the organisation to advertise and encourage a wide range of fellows to participate in various clinical and organisational governance roles both within and external to the College. For example, the College advertises all potential leadership and representational roles to its fellows prior to selecting its nominees. Similarly fellows, registrars and members are encouraged to consider opportunities to join committees in line with relevant tenure provisions contained in committee terms of reference. Similarly, all key policy and standards related matters (e.g. Primary Curriculum review) have involved consultation with fellows, registrars, members and stakeholders using a variety of communication methods.

As outlined above the College is currently considering changes to its constitution to broaden the representation base of its Executive. These would increase the number of fellows and members involved in policy development and decision making. These have been formally presented at an AGM and given time for broader member consideration; and will be considered by at the forthcoming general meeting. (See Section 1.1).


The College employs a range of protocols, and communication strategies that allow its geographically disbursed membership to engage meaningfully in the process of policy development, critique and decision making. As reported previously, the College has employed a specific Social Media strategy for reporting on governance events to complement its traditional communication techniques. This continues to grow in popularity and use and allows Fellows to contribute to issues in a timely manner.

BB Review the electoral rules and timing of Board Director Appointments to provide a smoother cycle of new and departing Board Directors for business and corporate continuity. (Standard 1.1)

See Section 1.1 above.


1.7 Other matters to be addressed

a. A Brief review of the College’s governance structure.


b. A list of committees with roles in the College’s training, assessment and continuing professional development activities, any changes since the 2010 Accreditation and outline of plans for further development.

College Committees with direct involvement in relevant activities:

- Education Council - Assessment Committee - Board of Examiners - Vocational Training Committee - Professional Development Committee - Post-Fellowship Education Committee - International Medical Graduate Assessment Committee

Significant changes since 2010:

In 2012, the ‘Educational Council’ replaced the ‘Censors Committee’. The new body is authorised to standards on behalf of the Board, and review and align education policy. The Censors Committee previously ‘set’ professional standards and fellowship requirements. It is agreed that the focus of the Committee should be to act as a forum for discussion and review of education and training policies to ensure integration and consistency. Also the chair of the new Education Council is an elected Board Member (with the Censor an ex-officio member).

In 2012, the role of Honorary Director of Education was replaced with a (paid) Director of Education role. The key focus of this role has been on implementation and support for education and training program across the College. The role continues to require an experienced educationalist of considerable standing in the medical profession.

Associated with the above appointment the role of National Censor was reviewed and redesigned. The new role of Censor in chief is more focussed on the adjudication of individuals’ performance against requisite College standards, rather than its historical focus on standards development and revision.

The ‘Assessment Committee’ has delegated its role ‘to provide oversight of assessment marking and grading’ to its sub-committee, the newly formed Board of Examiners, which was formed in April 2010.

There have been no changes to the Vocational Training Committee terms of reference.

The requirement that all College state directors were represented on the IMG Assessment Committee has been removed.

In 2012, the Post-Fellowship Education Committee revised its Terms of Reference in line with its three-year review cycle. It has incorporated an additional role to its terms of reference to


identify and develop additional programs. Minor changes were also made to remove certain procedural responsibilities from the committee (e.g. reviewing course accreditation lists) in line with well-established operational procedures within the College. Its membership has expanded to specify the inclusion of a registrar engaged in training.

The Quality and Safety Committee is no longer operational, its functions have been subsumed by the general roles of the Board and the Education Council.

See Figure 1: College Governance Structure

See Attachment 1.4 College Committees Terms of Reference

See Attachment 1.5 College Committees Current Membership

c. College’s assessment of its resources available to support the accredited pathways to fellowship and other educational activities

See Section 1.2.3 above

d. Developments since 2010 in the College’s relationships with health departments to promote the education, training and ongoing professional development of specialist general practitioners.

See Section 1.4:1-2 above.

e. The college’s interactions with the other key stakeholders in general practice education and training.

See Section 1.4:1-2 above.

f. Any other planned developments that relate to Standard 1.

All major planned developments are outlined above. It is envisaged that the burgeoning numbers of medical graduates seeking places is likely to lead to increasing challenges for all Colleges in finding training posts for their trainees. ACRRM has always needed to use creative and collaborative solutions to enabling training particularly outside the tertiary hospital system. Its organisational experience and the cooperative arrangements already in place are likely to assist in addressing this issue. Increasing efforts to expand the already considerable number of members contributing to the educational program will also contribute. It is noted however that this will be a considerable challenge moving forward. These issues are further detailed below. (See Section 8.2.4).


2. Purpose and Outcomes 2.1 Purpose of the training program

2.1.1 The purpose of the training organisation includes setting and promoting high standards of medical practice, training, research, continuing professional development, and social and community responsibilities.

ACRRM was founded by practising doctors with the specific goal of providing general practice standards, training and continuing professional development in the context of rural and remote practice in Australia. The College accepts responsibility for serving rural and remote communities and actively considers and promotes this in all aspects of its activities within the profession. The Articles of Association and objects of ACRRM define that the College will set professional medical standards for training, assessment, certification and continuing professional development in the specialty of General Practice particularly for rural and remote contexts:

Developing standards and applying those standards in the education and training of medical practitioners in the practice of medicine in rural and remote areas;

Establishing and maintaining high standards of knowledge, experience, competence, learning, skills and conduct of General Practice in rural and remote areas by particularly for rural and remote contexts;

Developing curricula for the training of medical practitioners at traineeship and post-graduate levels;

Developing curricula for the training of medical students at undergraduate levels;

Recognising and accrediting the previous education and experience of medical practitioners in the field of rural and remote medicine;

Developing, designing and implementing education programs in the field of rural and remote medicine at the undergraduate, traineeship and post-graduate levels to improve knowledge and skill or to extend knowledge and raise standards of patient care;

Accrediting organisations, programs, individuals, posts and medical practices to train medical practitioners in the field of rural and remote medicine;

Offering awards or by giving of fellowships to medical or health practitioners or suitably qualified persons in recognition of their competence in the field of rural and remote medicine;

Liaising and communicating with organisations committed to the health and medical care of people in rural and remote communities;

Conducting research and evaluation of medical practice and education in rural and remote areas;

Co-operating with other organisations on matters related to the health of people in rural and remote areas;


Facilitating the careers of health practitioners in rural and remote areas; and,

Taking any action which is incidental to and supportive of objects of the Company.

ACRRM provides general practitioners with a framework for educational activities, service delivery and systematic, continuous quality improvement that in turn, support quality care and patient safety. ACRRM’s framework for education defines the knowledge and skills that doctors require to practise safely and competently, it covers vocational training and professional development. The framework for vocational training incorporates all facets of education and training to the point when a general practitioner becomes vocationally competent to practise unsupervised and independently often in relative professional or geographic isolation. The vocational training program reflects the training requirements for attaining the education and training objectives of ACRRM. (For further details: See Section 3). The PDP expresses the College’s professional development requirements for retention of appropriate skills to continue unsupervised, independent general practice across a broad range of settings, including rural and remote locations. (For further details: See Section 9). The ACRRM Board undertakes a continuous strategic planning process to ensure that its education, training and continuing professional development activities continue to maintain high standards of general practice and promote social and community responsibility. As part of the strategic planning process the Board reviews the purpose, values and goals of the College and develops strategies that chart a course for the College and guides the operational activities. The three year Strategic Plan is reviewed annually and is designed to be flexible and responsive to changes in the economy and the industry.

2.1.2 In defining its purpose, the training organisation has consulted fellows and trainees, and relevant groups of interest.

The original articles were prepared by a group of like-minded rural doctors following a national plebiscite involving hundreds of medical practitioners. Considerable consultation was undertaken at that time among interested rural doctors in the development of the initial framework. The recent process of consideration of changes to the original constitution the Memorandum of Association has been made available to all members over a period of time. It has been clarified that the changes to the wording do not in any way change the purpose of the organisation. The purpose is made very clear to the members in all documentation that is supplied through the enrolment process at the point at which members are deciding whether or not to join the organisation. Members are surveyed in an ongoing process with regard to their support for the general principles of the organisation. The reports are fed back to ACRRM administration and Standing Committees for ongoing review. Members continue to express their support for the underlying principles of the program as demonstrated in the recent 2013 Members’ Survey which reported that two of the three most common motivations for joining ACRRM were for its relevancy to general practice needs and because of their shared beliefs and values. (See Appendix 2.1: ACRRM Member Survey) The initial Primary Curriculum was written by rural and remote doctors documenting the skills and knowledge they required in order to meet the needs of their patients and communities. Feedback was sought from members of the newly formed college and other wider stakeholders. Broad


consultation (both internal and external) has been part of every subsequent revision process; and, this is now a standard process for all curricula review cycles. Additionally, the curriculum content is informed by the College’s assessment of community need as identified through internal evaluated feedback, current research and government health priorities. (For further details: See Section 2.2) (For an outline of the consultation process for the Primary Curriculum 4

th Edition: See Section 3.1).


2.2 Graduate outcomes

2.2.1 The training organisation has defined graduate outcomes for each training program including any sub-specialty programs. These outcomes are based on the nature of the discipline and the practitioners’ role in the delivery of health care. The outcomes are related to community need.

2.2.2 The outcomes address the broad roles of practitioners in the discipline as well as technical and clinical experience.

ACRRM defines Rural and Remote Medicine as the application of the full spectrum of skills and knowledge encapsulated in the medical specialty of general practice. Rural and remote medical practitioners provide comprehensive and continuing medical care for patients, their families and communities. This care is typically provided in private community based practice facilities and hospitals; however it can also occur on roadsides, in remote clinics, jails, Aboriginal Medical Services or via telephone or e-health systems. ACRRM’s vocational training program specifies outcomes for registrars regardless of which training pathway they take. These are clearly articulated in the Primary Curriculum 4th Edition as detailed below. By completing the ACRRM Vocational Training Program, registrars will be able to practice as safe, confident and independent general practitioners in a full and diverse range of healthcare settings across Australia, with a particular focus on rural and remote settings.

- Acquire the knowledge, skills and behaviours to practise safe, independent and comprehensive medicine as general practitioners with a focus on practising in rural and remote communities:

- Attain Fellowship of ACRRM by successfully completing the training and assessment pathway requirements based on the ACRRM Primary Curriculum; and

- Commit to maintain and enhance competency after attaining Fellowship through participating in a structured continuing PDP.

The Abilities and Domains of Practice associated with achieving these aims are as follows:

1. Provide medical care in the ambulatory and community setting 2. Provide care in the hospital setting 3. Respond to medical emergencies 4. Apply a population health approach 5. Address the health care needs of culturally diverse and disadvantaged groups 6. Practise medicine within an ethical, intellectual and professional framework 7. Practise medicine in the rural and remote context

The goals are achieved through attainment of the knowledge, skills and broader professional encapsulated in the domains and further detailed in the Primary Curriculum and AST Curricula. The rationale section of the Primary Curriculum sets out the community needs that have guided the assignment of learning outcomes in the curriculum. See Attachment 3.1 Primary Curriculum.

https://www.acrrm.org.au/curriculum


2.2.3 The training organisation makes information on graduate outcomes publicly available.

Requisite Graduate Outcomes: These are clearly articulated in the ACRRM Curriculum which is openly available on the ACRRM website, in html and a PDF. See Attachment 3.1 Primary Curriculum.

Producing Credentialed Practitioners: ACRRM makes information on new fellows publicly available on a progressive as well as annual basis. Statistics on graduate numbers are included in its Annual Report. The Annual Report is circulated broadly in hard copy and is also available on the ACRRM website. Table 2.1 below provides details of the number of registrars achieving Fellowship for each pathway over the past four years. Table 2.1: Number of registrars achieving Fellowship (Breakdown by Program)*

Pathway 2010 2011 2012 2013 Total % increase

VPP 11 10 27 31 182%

RVTS 3 6 4 5 67%

IP 12 17 22 37 208%

Total 26 33 53 73 181% * Figures as at Dec in each year.

ACRRM provides a formal notice to each State Medical Board following the admission of new fellows. This information is also provided to Medicare for vocational registration certification purposes. Training providers and GPET are advised when registrars in their program are awarded Fellowship. These organisations typically advertise these achievements in their newsletters and award ceremonies. The Medical Training Review Panel Annual Report presents medical training information at the university, prevocational and vocational levels. ACRRM provide data to the Medical Training Review Panel each year on graduate outcomes which is subsequently published and is publicly available. (See Attachment: 2.2: ACRRM Annual Report 2012-13)

Graduates Outcomes consistent with College Mission The College Evaluation Framework in its entirety is based on nine core questions designed to garner information on how effectively program outcomes are addressing the College’s central mission. These include questions such as: ‘How is ACRRM contributing to the healthcare needs of rural and remote Australians?’; and, ‘What are the professional characteristics (KSA) of an ACRRM registrar and what do registrars do differently a as a result of ACRRM training?’. The annual evaluation program includes surveys of current registrars in training and supervisors. These surveys are carried out across the AGPT, RVTS and IP and discretely measured. Within these surveys there are specific questions in regard to their experience and delivery of the broad scope of specialist general practice.


https://www.acrrm.org.au/acrrm-annual-reports


The Vocational Training Exit survey for those trainees who are awaiting Fellowship ratification or have just Fellowed clearly references the outcomes for the whole of the individual registrars’ training experience. Within this survey there are specific questions to address both the relevance of their experience and delivery to the broad scope of General Practice but also that this experience and delivery suits the community need. More specifically in terms of the opportunity for training and experience in delivering continuity of care, managing chronic disease, organising and undertaking disease prevention and understanding the responsibilities for population health.

The results of surveys are wherever practicable communicated to the relevant stakeholders. This is generally done through newsletters (i.e. Countrywatch and FACRRM Fundamentals), and social media networks as appropriate.


2.3 AMC Conditions

Condition 7 Provide evidence of processes for regularly reviewing the graduate outcomes of each training pathway in relation to community need. (Standard 2.2)

Graduate Outcomes consistent with the mission of the College are broad, multifaceted and heavily influenced by external factors. Accordingly a complex of approaches has been applied toward provision of reliable measures of the Colleges performance. The College Evaluation framework engages a range of tools to the review and refinement of each program. These are each processed toward program improvement including with regard to their continuing alignment with community need. Literature: Assessing the broader context of Community Need ACRRM maintains its knowledge of the latest literature relating to the status of rural and remote communities, their medical needs; and best practice for training and credentialing them. Many executive members are themselves researchers and significant contributors to this. From time to time the College itself engages in research undertakings encompassing literature scoping exercises. These all inform ongoing development. For example the College has learned from undertakings such as: 2014 Scoping Study produced for the Department of Health into establishment of a National Rural Generalist Pathway. This was a collaborative exercise with representatives of the peak bodies for the Rural Clinical Schools, Postgraduate Medical Councils, and RTPs, and RDAA. A Joint Paper with the ANU, Australian Primary Healthcare Research Institute, titled, “The Expanding Role of the Rural Generalist in Australia: A Systematic Review; and, A College Paper on Barriers to Procedural Skills in Rural and Remote Medicine. Program Evaluation Multiple evaluation tools are in operation for each program area. The tools most relevant to assessing community need, their relevant survey sections, feedback outcomes and the College responses to each of these are outlined below.

(For a full list of the 45 evaluation survey tools in operation: See Section 6.1).

Programs Evaluation Surveys

Member Survey This is undertaken annually to all members and to date these have been conducted in 2012, and 2013.

Relevant Questions: - What was your principle motivation to join ACRRM? Two of three top responses given were

that it was most relevant to general practice in rural and remote areas; and that it was aligned with members’ own beliefs and values.

- What are the College’s most important functions? Responses identified that member priorities were also well aligned with community need (e.g. 95% for ‘provision of information


relevant to rural practice’; 95% for ‘advocacy for rural and remote medical professionals’; 88% for ‘promoting rural and remote medical careers’.)

College Response: Feedback on both of these questions is viewed as an encouraging indicator that the College mission and associated programs are consistent with the perspectives of its membership.

(See Attachment 2.1: Member Survey 2013)

Registrar Survey

This is undertaken annually to all ACRRM registrars and to date these have been conducted in 2012, 2013, 2014. Results below relate to the 2013 survey. Relevant questions: - To what extent does the College curriculum identify the knowledge and skills to practice as a

rural and remote general practitioner? (This also allows free comment on this general topic.) Responses were generally positive (62-85%); no significant free text feedback was received regarding community need issues.

College Response: - These were considered to be supportive of current arrangements. It was noted that the

RVTS was less supportive than the other 2 programs.

(See Attachment 2.3: Registrar Survey 2013)

Vocational Trainees Exit Survey

This is undertaken annually to all ACRRM registrars and to date these have been conducted in 2012, 2013, 2014. Results below relate to the 2014 Survey. Relevant questions: - Does the ACRRM vocational training (PRRT) program provide registrars with the opportunity

to experience and deliver continuity of care, manage chronic disease, as well as undertake disease prevention and understand their responsibilities for population health? Responses were positive. Provided the knowledge (82%); skills (80%); were adequately prepared to provide continuity of care and manage chronic disease (77%); and to undertake disease prevention and understand responsibilities for population health (72%).

- What type of practice will you be undertaking? Respondents indicated that they would

practice in hospital and/or general practice (79%); general practice only (61%); AMS (18%); Retrieval Service (10%).

- In which ASGC-RA location will you be practicing? Responses indicated as follows; 82% would

practice in ASGC-RA 2-5 (i.e. regional/rural/remote areas); and notably 32% of those in ASGC-RA 4-5 (i.e. Australia’s most remote locations).

- Which advanced skills will you be using in your next role? 52% of respondents will use their

Emergency Medicine skills; 37% Anaesthesia; 27% Obstetrics and Gynaecology; 18% Remote Medicine; 18% Adult Internal Medicine. From (6-15%) will use AST skills in the remaining areas.

College Responses (in order):


- The vast majority of respondents considered that the program prepared them for the most important areas of community need.

- College considers ‘intended type of practice’ results to be a positive affirmation of their

program aims and consistent with community need. They reflect key College aims to produce graduates to provide services across the breadth of work settings and perform advanced skills as well as primary care services; and the College contends (supported by considerable evidence in the literature ) that this is consistent indeed essential to meeting Community Needs in rural and remote areas.

- The vast majority of graduates will be providing services in rural and remote Australia. This is

viewed by the College as a critical indicator of successfully meeting community need and (by extension) its mission.

- The College welcomes the focus on provision of emergency medicine, anaesthesia and

obstetrics and gynaecology services which it considers to be consistent with community needs. (This position is verified by considerable external and internal literature). It has been noted by the College that relatively few registrars (13%) are providing advanced Mental Health skills and this is a key area of unmet community need. The College is undertaking a range of strategies to build coursework and interest in Mental Health.

(See Attachment 2.4: Vocational Trainees Exit Survey 2014)

Community Feedback and Needs Analysis

This is an ongoing and annual evaluation tool. The first survey is currently being undertaken. This has been posted to a wide range of community groups and relevant stakeholders and additionally is available to any interested persons via the ACRRM webpage. Results are pending.

Relevant questions include:

- What are the main health issues affecting your community? - Can you identify any gaps in your community’s medical resources or health services? - What personal attributes should GP trainees exhibit to be healthcare providers in your

community? - What knowledge and skills and attitudes should trainees learn during their General Practice

training? And what methods and strategies should be used to teach these? - Respondents are asked to identify the most important areas of medicine within their

community? See Attachment 2.5: Community Feedback and Needs Analysis: Questions

See Attachment 2.6: Community Feedback and Needs Analysis: Mail out list

Statistics for Annual Evaluation

The College collates specific data to assess its performance against its specific mission to produce advanced and broad scope general practitioners (including proceduralists) and doctors who serve in rural and remote communities. The annual evaluation of training includes monitoring the work location of alumni and the advanced specialised skills being maintained. To identify the extent ACRRM programs are contributing to the development of rural and remote medical workforces and to the provision of advanced skills within them. (For further details: See Section 6). These are incorporated into the Vocational Training Annual Evaluation. The 2013 Evaluation found, 80% of ACRRM Fellows who undertook FACRRM training on


one of the accredited training pathways are currently practising in rural or remote Australia. These results are fairly consistent across the three pathways (VPP = 82%; IP = 77%; RVTS = 77%).

Table 2.2: Fellows by ASGC-RA and Training Pathway in 2013

ASGC-RA VPP IP RVTS Total %

RA1 20 19 8 47 20%

RA2 29 31 8 68 28%

RA3 38 24 14 76 32%

RA4 15 12 4 31 12%

RA5 12 4 1 17 7%

OS 0 2 0 2 1%

Total 114 92 35 241 100%

External Verification of Meeting Community Need The Queensland Rural Generalist Program (QRGP) provides a good measurable example of the College program outcomes matching the Colleges’ stated goals. The program has been built on the ACRRM Primary Curriculum and 97% of its trainees are in the FACRRM pathway. 12 The Health Workforce Australia commissioned report into its operations reported that within two years of the programs operation (2010-2012):

critical vacancies in rural/remote hospitals were reduced by 96%; and

62% of surveyed trainees reported intending to become long-term rural practitioners.

The pathway has been consistently oversubscribed. 13 Rural hospitals previously at risk of closure such as the Kingaroy Hospital are now serviced entirely by the new cadre of general practitioner proceduralists commonly providing joint private practice and hospital services.14

12 RMA Conference 2013. Presentation by Dr Denis Lennox, QRGP Clinical Director. 13 As reported in February 2012 in Ernst and Young. Evaluation and Investigative Study of the Queensland Rural

Generalist Program. Queensland Health, Office of Rural and Remote Health. February 2013. 14 As per the advice of Darling Downs Hospital and Health Service. March, 2014.


2.4 Other matters to be addressed

a. Changes to purpose or mission of the College.

.

The College’s purpose and Mission in unchanged. It should be noted that the wording to the articles of Association has been changed slightly to clarify its consistency with the College’s accreditation status within the Specialty of General Practice. These changes were widely disseminated to members for consideration and voted upon at the annual AGM. It was made clear in the information provided that the changes did not signify any change to the essential purpose of the College.

b. Changes to college statement of graduate outcomes for training programs .

For the purposes of simplification for users, the Primary Curriculum are substantially unchanged;

“The ACRRM Vocational Training Program aims to produce Fellows who can function as safe,

confident and independent general practitioners in a full and diverse range of healthcare settings across Australia, with particular focus on rural and remote settings.

The Program has a number of goals for registrars:

1. Acquire the knowledge, skills, and behaviours to practise safe, independent and

comprehensive medicine as general practitioners with a focus on practising in rural and remote communities;

2. Attain Fellowship of ACRRM by successfully completing the training and assessment pathway requirements based on the ACRRM Primary Curriculum; and

3. Commit to maintain and enhance competency after attaining Fellowship through participating in a structured continuing professional development program.”

There are a few very minor differences in the 3rd Edition:

Goal 2 reads: “Attain Fellowship of ACRRM by successfully completing the training and assessment pathway requirements.” Goal 3 reads: “Commit to maintain competency after attaining Fellowship through participating in a structured professional development program.”

(See Attachment 3.4: Explanation of Changes to Primary Curriculum)

c. Any other planned developments that relate to Standard 2

.

All significant changes related to Standard 2 are outlined above. The College anticipates that the Community Feedback Survey tool will provide a useful resource in guiding future developments. While still in information collection phase, it has already elicited considerable feedback and interest from community organisations. The initial feedback is already providing useful information; pointing to areas where the College has been successfully meeting its charter and also to areas to focus further development efforts.

(For further information see Section 3.1 and Section 6.2.2).



3. Curriculum Content The ACRRM Primary Curriculum Fourth Edition and ACRRM AST Curricula define the standards and outcomes for ACRRM vocational training and professional development programs.

See Attachment 3.1: Primary Curriculum.

See Attachment 3.2: AST Curricula.

3.1 Curriculum framework

3.1.1 For each of its education and training programs, the training organisation has a framework for the curriculum organised according to the overall graduate outcomes. The framework is

publicly available.

ACRRM Primary Curriculum

The ACRRM Primary Curriculum makes explicit the core learning abilities which must be attained to produce graduates that can function as safe, confident and independent doctors in the Australian health system across a full and diverse range of general practice including in rural and remote environments. The Primary Curriculum underpins and acts as a baseline for AST Curricula. The ACRRM Primary Curriculum is organised on the following basis:

11 Principles: these form the conceptual and practical foundation for the Curriculum;

7 Domains: different contexts of general practice, particularly in the rural and remote context;

73 Generic Abilities: these define the abilities that registrars must demonstrate, organised within the seven domains of rural and remote general practice; and

18 Curriculum statements: these describe the relevant content in the major medical disciplines or practice areas. The curriculum statements contain abilities organised within the seven domains of rural and remote general practice and essential knowledge and skills.

Procedural and Clinical Skills The ACRRM Primary Curriculum clearly identifies essential psychomotor procedural and clinical skill competencies for registrars. The skills are distilled in the ACRRM Procedural Logbook. The AST curricula identify additional procedural skills required in that discipline.

The ACRRM Primary Curriculum and ACRRM Procedural Logbook are available on the ACRRM website, and are available to training providers in hard copy. (See Attachment 3.3 ACRRM Procedural Logbook)


https://www.acrrm.org.au/advanced-specialised-training-curricula

https://www.acrrm.org.au/assessment


Primary Curriculum Review

The Primary Curriculum fourth and current edition (2013) has resulted from a major review of both content and structure involving key stakeholders conducted between October 2009 and January 2010.

The College invited and encouraged input from a wide range of individuals and organisations including College members, other specialist colleges, General Practitioner Training Providers, State Health Departments, RG Programs, relevant international organisations. The organisations and individuals that contributed are listed in the acknowledgements at the back of the curriculum. A summary of the changes to the curriculum is also available on the ACRRM website. The webpage also provides blueprint templates to assist in the implementation of the revised curriculum.

The following methods were employed to communicate the iterative developments with stakeholders:

Regular articles in ACRRM e newsletters Country Watch and FACRRM Fundamentals https://www.acrrm.org.au/facrrm-fundamentals

Advice via individual email to stakeholder groups

ACRRM website

Virtual classroom live and recorded sessions made available on the ACRRM website

Conference presentations

Regular attendance at training provider CEO and Director of Training meetings

At the release of the new curriculum the following methods were employed to publicise and explain the implications of the new curriculum:

Ongoing liaison and discussion at the Regional Training Provider CEOs network

Ongoing liaison and discussion at the Directors of Training network

Regular virtual classroom sessions that ACRRM convenes for educators and supervisors

Sending information through newsletters and direct mail to supervisors and registrars; and

Updating the Curriculum page on the College website to provide an opportunity for direct feedback and a document summarising changes made.

(See Attachment 3.4: Summary of changes to Primary Curriculum)

Continuous Primary Curriculum Review: Update The review cycle for the curricula is five years; however the document will also be refined as required in response to ongoing feedback and needs identified by the College. To facilitate ongoing feedback the html version of the Primary Curriculum gives readers the opportunity to provide feedback as they read the document.

The recent consultation round focused mainly on feedback from general practitioner and other medical specialists. College has now established a process to seek broader feedback from consumers and allied health providers on the College curricula (Primary and AST Curriculum). Feedback is being encouraged through an online survey. This is permanently available on the College website. (For further details: See Section 6.2.2).



(See Attachment 2.5: Community Stakeholder Feedback Survey (information and survey link)).

The College is encouraging feedback to the online survey through an annual invite as was sent in June this year. The College hopes the survey will encourage general feedback from a broad range of individuals and organisations and additionally identify those who will want to be involved in more specific feedback on the College curricula and other areas where input is being sought.

The College intends to review the feedback received by October 2014 and consider what adjustments to the curriculum are required for the release for beginning of 2015.

Advanced Specialised Training Curricula

ACRRM has identified 11 extended or advanced areas of specialised training that it believes represent the required, appropriate and necessary skills for general practice in the rural and remote context. These skills enable the graduate to contribute to a full range of medical services and workforce collaboration models to provide appropriate local care particularly for rural and remote communities.

The disciplines are:

Aboriginal Health

Academic Practice

Adult Internal Medicine

Anaesthetics

Emergency Medicine

Mental Health

Obstetrics and Gynaecology

Paediatrics

Population Health

Remote Medicine

Rural Generalist Surgery

Academic Practice was approved by the ACRRM Board as an ACRRM AST in 2012, and an advanced draft of the curricula is provided. Registrars are currently able to undertake posts in Academic Practice AST providing they seek approval prospectively and provide the College with the following information:

A set of learning outcomes for the discipline area, which demonstrate a clear extension beyond the content of the ACRRM Primary Curriculum;

Details of the proposed training post to allow accreditation of the post to take place;

Details and documents verifying the registration and qualifications of the proposed supervisor; and,

A clear plan of how the discipline will be assessed, and by whom.

(See Attachment 3.5 Academic Practice Draft Curriculum)

AST Curriculum Review Curricula for all AST determined by ACRRM are being reviewed in 2014 -2015 to align them with the format of the revised Primary Curriculum and to ensure that they remain current and relevant to best practice for general practice training in the relevant discipline. As part of the review CPD requirements will also be set as appropriate to the curriculum.



Curriculum, accreditation and assessment requirements for Anaesthetics; Surgery and, Obstetrics and Gynaecology have been established through Joint Consultative Committees (JCC) processes involving collaboration between ACRRM, the RACGP and the relevant specialist college. The JCCs are operational and their contribution will form the basis of reviews in their respective AST area. (NB. The JCC for Surgery was only recently re-established under the new title of JCC for General Practice Procedural Surgery). The other AST curricula do not have corresponding JCC structures. ACRRM nonetheless determines the curricula in consultation with specialist and generalist colleges and colleagues. The general process for collaboration in developing these AST curricula is outlined below.

A working group is established with members who are expert in the relevant field and a lead clinical author appointed by the Board.

The working group meets to establish the learning outcomes, syllabus, assessment and accreditation requirements.

The lead author works with ACRRM educational staff to draft the document. The working group uses an iterative process of review and editing until they are satisfied that

the document meets their objectives.

The draft document is reviewed by the Vocational Training Committee and Assessment Committee then subsequently the Education Council to ensure clarity and consistency in methodology across all AST curricula.

The document is provided to ACRRM members, stakeholders and the relevant specialist college/authority for further comment, input and/or endorsement.

The draft curriculum is adjusted to address feedback where appropriate and is reconsidered by the Vocational training and assessment committee and Education Committee.

The final draft is submitted to the ACRRM Board for endorsement. The AST curriculum is uploaded to the ACRRM website and implemented.

Progress At the time of writing this submission; all curricula have been transcribed into the revised format. Consultation with a wide range of organisations and individuals is underway and will continue until 30 September 2014. A webpage including an online form has been established to facilitate feedback.

The relevant Specialist Colleges are being contacted to arrange a meeting to discuss the review of the curricula related to their discipline i.e. College of Physicians (Adult Internal Medicine, Paediatrics and Population), College of Emergency Medicine and College of Psychiatrists.

(See AST Curriculum Development website).



3.2 Curriculum Components

3.2.1 For each component or stage, the curriculum specifies the educational objectives and outcomes, details the nature and range of clinical experience required to meet these objectives, and outlines the syllabus of knowledge, skills and professional qualities to be

acquired.

Clinical training requirements

ACRRM registrars are required to complete a minimum of four years of vocational training in accredited posts. This includes:

12 months Core Clinical Training (CCT) in accredited metropolitan, regional or rural hospital recognised for intern training. CCT provides baseline competencies for safe clinical practice.

24 months Primary Rural and Remote Training (PRRT) in rural and remote ACRRM accredited private practices, Aboriginal Medical Services, small rural hospitals or community based facilities. This period of training must include a minimum of: - 12 months Rural and Remote Context, - 6 months community primary care and population health and - 6 months hospital and emergency care

12 months AST in ACRRM accredited posts in one approved discipline (note Rural Generalist Surgery requires 24 months training) (see 3.1.1 for list).

Educational objectives, requirements and outcomes are outlined in the ACRRM Primary Curriculum for CCT and PRRT training; and, for AST, in the individual AST discipline curricula. ACRRM training is delivered through three pathways, Vocational Preparation Pathway (VPP), Remote Vocational Training Scheme (RVTS) and Independent Pathway (IP). All pathways have the same requirements, standards and assessments.

(See Attachment 3.6: Policy statements for each stage of training: CCT, PRRT, AST)

(For further detail on these pathways: See Section 4.)

Community-based Primary Care Experience The Completion of Training process requires the registrar and training provider to verify that all requirements as outlined above have been met prior to the award of Fellowship. The requirements for a minimum of 6 months community primary care and population health has been implemented and enforced through the associated training policies. All registrars who have commenced training from 2012 will not be awarded Fellowship until this requirement is met. The large majority of registrars commencing training prior to 2012 will also meet this requirement. (See Attachment 3.7: Policy for Completion of Training)

(See Attachment 8.1: Standards for Supervisors and Teaching Posts)

3.2.2 Successful completion of the training program must be certified by a diploma or other formal award.

Fellowship Award

ACRRM certifies successful completion of its vocational training program through the award of Fellowship.

https://www.acrrm.org.au/operational-policies


https://www.acrrm.org.au/teaching-posts


3.3 Research in the Training Program

3.3.1 The training program includes formal learning about research methodology, critical appraisal of literature, scientific data and evidence-based practice, and encourages the trainee to participate in research.

Curriculum incorporates Research Competencies

The ACRRM Primary Curriculum describes the abilities, knowledge and skills required in research and evidence based medicine Registrars are required to be able to explain the nature of research particularly in rural and remote general practice, access medical literature and other sources of information, critically appraise information, understand basic concepts of clinical epidemiology and undertake clinical audit. The 4

th edition of the Primary Curriculum extends on

the syllabus statement on Research and Evidence Based Medicine in the previous curriculum to include abilities, knowledge and skills for teaching, mentoring and supervision. The ACRRM assessment blueprint ensures registrars are assessed on their research and evidence based medicine competencies.

In the VPP and RVTS pathways training providers are responsible for delivering learning activities to meet the College curricula. The College requires these training providers to demonstrate at accreditation that all registrars have been provided with education on research and critical appraisal.

(See Attachment 8.6: Bi-College Accreditation: Principles and Outcomes.)

ACRRM as the training provider for the IP includes educational sessions on survey design, and statistical analysis in the program’s education program. These are provided in person and online to enable viewing by the widest possible audience. Self-directed learning and support options are also available for registrars who wish to pursue research skills.

Consideration is being given to including an option for counting up to 6 months FTE toward the PRRT Program in an area of study such as Medical Education and Research.

Programs encourage research participation

Registrars who undertake research projects (as part of their AST) are offered support through the Research Committee and are encouraged and supported to publish and to present their project, including at Rural Medicine Australia (the College’s Annual Conference).

(See Attachment 3.8: List of completed Registrar Projects.)

An “Evidence Based Medicine” PDP Module is currently underdevelopment to be delivered on the RRMEO platform. This is an initiative of the Research Committee and is being developed by appropriately qualified Fellows in conjunction with the Committee.

Research Opportunities through External bodies

ACRRM has maintained active representation on GPET’s Registrar Research Committee, which is

designed to oversee the functional and policy issues regarding registrar academic posts.

https://www.acrrm.org.au/training-providers-tool-kit


ACRRM hosts GPET’s annual Research Week, an online conference open to all AGPT trainees and to all ACRRM members. The conference is delivered via RRMEO. It aims to provide learning opportunities and encourage a research culture among vocational trainees.

AGPT has a Registrar Scholarship and Research Fund which aims to encourage research within the field of general practice. This fund provides project funding for registrars wanting to complete a research project to take up a 12-month part-time academic registrar position.

3.3.2 The training program allows appropriate candidates to enter research training during specialist education and to receive appropriate credit towards completion of specialist training.

A range of mechanisms are in place to ensure that research opportunities are available and accredited toward Fellowship in ACRRM vocational training programs.

Amendments are currently being considered which would allow ACRRM registrars to undertake research during PRRT and have this count towards training. (The research period is limited to 3-6 months).

Academic posts are part of the Australian General Practice Training (AGPT) program and are therefore available to registrars on this pathway. Registrars seeking to undertake an academic post need to obtain prior approval from ACRRM to ensure that it is undertaken in a way that complements their training and overall attainment of training outcomes and to decide to which stage of training PRRT or AST it will contribute.

(See Attachment 3.9: AGPT Academic Posts Guide.)

The Academic Practice AST provides an opportunity for registrars to develop advanced research skills while in training. Additionally, the AST’s in Population Health, Remote Medicine and Aboriginal and Torres Strait Islander Health all have a project as their main summative assessment requirement. The project may be a research study.

http://www.gpet.com.au/GP-registrars/Training-Posts/Academic-Training/Academic-Posts


3.4 Flexible training

3.4.1 The program structure and training requirements recognise part-time, interrupted and other flexible forms of training.

General practice is a medical specialty that offers a high degree of flexibility, and general practice in rural and remote environments offers even greater levels of flexibility in terms of scope of professional practice and workforce context. The College’s program structure and training requirements attempt to reflect and maximise this flexibility, while providing structure and rigour to ensure standards and efficiency through the program. The vocational training program has a number of features that provide flexibility to suit differing registrar circumstances, profiles and lifestyle choices. The three training pathways target the different needs and situations of the wide variety of registrars that train for general practice. The IP in particular provides a very flexible individualised program option for experienced and self-directed learners. There is also flexibility in how clinical training requirements are met within the structure of each of the vocational training pathways. At the commencement of training registrars are encouraged to develop a learning and assessment plan for their four year program. This builds understanding between the medical training advisor and the registrar, as well as an opportunity to discuss family and other support that may be required to manage progress through the program. Learning and assessment plans are structured to provide a progressive, integrated approach to skills development and competencies, that compliments career and family position. The order in which registrars progress through training terms is also flexible and can be negotiated with agreement of their medical educators. For example, registrars can undertake their AST terms immediately after CCT provided they can demonstrate that they have achieved the prerequisite competencies that apply and have plans to move to posts that will allow them to use those advanced specialised skills after training. The College program also allows registrars to undertake a mixture of PRRT and AST terms simultaneously. In such cases there must be clear documentation and a delineation of the respective experience gained, including assessment. This model provides variety and flexibility for training as well as responds to local workforce or resource limitations with respect to local hospitals and case loads. There are flexible ways for registrars who have not gained one or more of the core clinical rotations to gain these skill sets as they progress through training. In all cases, ACRRM encourages and facilitates negotiation of flexible training arrangements between local medical educators, registrars and practices/hospitals, then recommendations are formally considered and determined by the ACRRM Censor.

The ACRRM assessment program also provides training flexibility in each of the five assessment components which can be attempted progressively as the registrar wishes rather than at prescribed periods or the end of training.

ACRRM supports registrars who wish to take extended periods of leave or undertake training part-time. Registrars are also able to undertake training on a part-time basis, subject to approval from their training provider. Training time is calculated and accumulated on a pro rata basis.


Currently there are five registrars in the IP approved to work part-time and six on leave from the program. Registrars on the VPP and RVTS pathways apply to their training provider for part-time training and extended leave.

(See Attachment 3.10: ACRRM Policy on Leave from Training.)

(See Attachment 3.11: ACRRM Policy on Part-Time Training.)

3.4.2 There are opportunities for trainees to pursue studies of choice, consistent with training

program outcomes, which are underpinned by policies on the recognition of prior learning. These policies recognize demonstrated competencies achieved in other relevant training programs both here and overseas, and give trainees appropriate credit towards the requirements of the training program.

ACRRM has provision for RPL and experience for its registrars. It is possible to complete other qualifications and have these recognised as contributing to the training requirements. For example, ACRRM recognises the Advanced Diploma of the Royal Australian and New Zealand College of Obstetricians and Gynaecologists (DRANZCOG Advanced) for the purposes of an Advanced Specialised Training (AST). Registrars on all training pathways apply to ACRRM for assessment of their RPL. The RPL policy was revised in June 2013 to introduce a two stage process for RPL for which trainees apply, prior to training. The first stage involves desk top review of information and evidence. The second stage requires a clinical interview. The revision process also clarified; that RPL applications submitted after enrolment in training must include a recommendation by the training provider.

(See Attachment 3.12: ACRRM’s Vocational Training RPL Policy.)

Table 3.1 (below) provides details on the number of registrars applying for and granted RPL between 2011– 2014 (includes up to end June 2014). Table 3.1: Number of Registrars applying for and granted RPL (Jan 2011–Jul 2014)

Pathway Applied for RPL Granted RPL

IP 252* 197

RVTS 17 17

VPP 87 87

Total 356 301

* The difference in figures relates to applicants who applied for RPL but did not meet eligibility requirements.

http://www.acrrm.org.au/operational-policies




3.5 The continuum of learning

3.5.1 The education provider contributes to articulation between the specialist training program and prevocational and undergraduate stages of the medical training continuum.

Vertical integration is a core pillar of the ACRRM approach. It is well established in the scientific literature that establishing a vertically integrated ‘rural training pipeline’ provides the best possible results in terms of rural workforce retention.15 As such all College programs and activities are consistent with this approach and the College has taken a lead role in advocating for the institutionalisation of structures to support it. ACRRM’s undertakings to promote articulation between undergraduate, prevocational and vocational training for general practice, are summarised below. Advocacy for Vertical Integration

The College has many Fellows (including its current President) who are national leaders and advocates for vertical integration of general practice education particularly as it pertains to building a rural and remote workforce. They pursue this through their roles as directors/deans of health and medicine faculties; medical schools; University Departments of Rural Health; and, Rural Clinical Schools; and, membership of professional bodies such as the Federation of Rural Academic and Medical Educators, and Postgraduate Medical Councils.

The College is specifically represented on key integration committees including: the Commonwealth Department of Health Prevocational Advisory Committee; the GPET Prevocational Advisory Committee; most state Postgraduate Medical Councils or equivalents; and the Rural Australian Medical Undergraduate Scheme (RAMUS) Appeals Committee.

The College since its inception has advocated for establishment of the Rural Generalist training pathway; their most salient feature being the provision of an integrated training experience from medical school through to professional development. It co-founded the pathway in Queensland and lent its programs as the basic structure. The resultant Queensland Rural Generalist Program has provided the basic blueprint for the subsequent pathways established across the country. Fellows provide Integrated Program Delivery Over 180 ACRRM Fellows supervise and mentor across the spectrum of undergraduate, prevocational and vocational education and training providing important links across the continuum of learning and peer support. For example, the John Flynn Placement Program (JFPP) requires students to visit the same practice for two weeks each year for four years. During this time the students are mentored by the same rural doctor and supported by community mentors. Around 390 ACRRM fellows participate as mentors in this program. Many junior doctors, who were JFPP students, have returned to complete their PGPPP placement with their JFPP mentor enabling them to continue to build on the clinical skills learnt during the undergraduate years. ACRRM is conducting a longitudinal study to trace JFPP students through the stages of their career and the linkage between their supervisor/mentor and their career stage.

15 Norris TE. The universal importance of the ‘rural pipeline’. Aust. J. Rural Health. 2005; 13:203-204.


Vertically articulated Undergraduate and Prevocational programs

The College has devised, advocated for, and now manages some of the Government’s key programs to promote integration of undergraduate and prevocational experience for students interested in rural general practice careers to rural vocational training and professional development. The (then newly formed) College proposed, developed and delivered the first national prevocational placement program for general practice fourteen years ago. This program was originally developed to support an articulated pathway for rural practice and was called the Rural and Remote Area Placement Program. Due to the success of the program the government expanded the program in 2006 to include outer metropolitan placement opportunities and re-titled the program PGPPP. It was been co-managed between ACRRM and RACGP from 2005 to 2010; when its operation was taken over by GPET.

The College was instrumental in establishing supported undergraduate rural placements; and currently manages the following undergraduate programs:

John Flynn Placement Program (JFPP), managed by ACRRM since 2001

Medical Rural Bonded Scholar Support Scheme, established by ACRRM and managed since 2003; and

Bonded Medical Places Student Support Scheme, established by ACRRM and managed since 2004.

The fact that the College over a decade later continues to administer these programs (which are awarded regularly on a competitive basis), demonstrates its ongoing attainment of program outcomes. Vertically-integrating Curriculum and Standards

ACRRM undertakes a number of activities that support the articulation between curricula and standards at the pre-vocational, vocational and post-vocational levels.

Curriculum:

The College contributes both informally and formally, to medical school and medical undergraduate processes (e.g. curriculum reviews, placement processes, community advisory boards). This provides an opportunity to contribute to undergraduate curriculum and policy regarding experience and requirements for general practice, particularly in rural and remote environments.

ACRRM also promotes vertical integration of learning through curriculum development. The ACRRM curricula provide a central point of reference to ensure continuum of learning between vocational and post-vocational programs (See Section 3.1).

JCU and Flinders University have also requested ACRRM provide mutual recognition and articulation for substantial elements of its Masters of Rural and Remote Medicine program.

Standards

ACRRM works with the CPMEC to ensure integration of emerging national curricula and assessment with ACRRM formative assessment and curricula requirements.

The College worked with CPMEC and a range of other stakeholders to determine a national framework for teaching post accreditation which cross-mapped prevocational and vocational training accreditation requirements for general practice. The College utilises the framework in the ACRRM Core Clinical Training Standards for supervisors and Teaching Posts.


The College has also set standards for Queensland Health to benchmark the required skills set for prevocational doctors training in the Rural Generalist training program. These trainees progress to ACRRM vocational training and their prevocational achievements are credited against their vocational training requirements.

Vertically Integrated Learning Planners

RRMEO, the College’s web-based learning management system, enables recording and tracking of members’ education and learning activity through the ‘Learning Planner’ tool. The planner has been deliberately structured to allow participants to maintain ongoing, cumulative learning records from undergraduate medical studies through the duration of their medical career.

Participants in JFPP can also use the learning planner to record achievements and skills mastered. Doctors who enter ACRRM vocational training continue to utilise the same learning planner through vocational training and into PDP. Registrars are able to use the learning planner as evidence when applying for RPL and to demonstrate professional development to a third party.


3.6 AMC Conditions

Condition 9 For ACRRM curriculum statements for specialist discipline areas, engage with the standards setting bodies for the medical specialities to ensure that learning objectives are appropriate and supported with appropriate clinical experience and supervision, and to provide a sustainable system for curriculum review. (Standard

3.2).

Approach to Collaboration The rural and remote practitioner; as one of very few doctors (if not the only doctor) to whom their community may have easy access; needs to provide (or partially provide) many of the services that would in cities ordinarily be the province of other specialties; often applying distinctive delivery models using available local health staff/resources and possibly remote urban based specialists. For these reasons, collaboration is at the heart of rural and remote medical services delivery and has been reflected in our curriculum development process and all educational undertakings; not just with other medical specialities but also with allied health professions, NACCHO and other relevant experts. ACRRM’s mission and the concordant graduate outcomes it strives to attain are defined by the needs and circumstances of the communities in which its Fellows will be able to practice. The scope must be both broad and flexible and nuanced by the distinctive nature of medical service, performed remotely from an urban suite of staff and resources. The knowledge, skills and professional competencies that this engenders are distinct and their curricula requires guidance by experts in the field. Over the past 15 years ACRRM has harvested the expertise and experience of the country’s foremost minds in the practice of rural remote generalist practice, in particular, of those applying advanced skills and broad scope practice in rural and remote practice. The College has augmented this with the accumulated knowledge developed across the world to identify and continuously redefine best practice in the field. The College has now acquired a cumulative body of knowledge and experience which it continuously adjusts to embrace new advances and address social shifts. Despite its uniqueness, its FACRRM credentials are recognised internationally through mutual recognition arrangements in a number of countries. Its leadership in the area of rural and remote medicine is widely recognised among like-minded colleagues both in Australia and across the world as was demonstrated by the over 800 doctors from almost 30 countries who attended its World Summit on Rural Generalism in 2013. It should be noted also that in Australia, the number of experienced practitioners in specialist and sub-specialist services engaged in delivery of rural and remote services in many disciplines is very small and decreasing; and the pool of rural and remote practice expertise from which curriculum advice can be drawn is correspondingly limited. Collaboration Process The College’s planned collaboration process as it embarks on reviews of its AST curricula (and development of its Academic Practice AST) is outlined in detail above (See Section 3.1). For the AST areas which have associated JCCs the review will receive input from their respective Committees.


3.7 Other Matters to be Addressed

a. A summary of how the Colleges’ primary curriculum and advanced specialist training curricula have evolved since 2010.

Primary Curriculum A summary of the changes to the Primary Curriculum is given on the ACRRM website. (See Attachment 3.1) Advanced Specialist Training Curricula At the time of the last review the AST Emergency Medicine and Remote Medicine curricula were complete and other AST curricula were under development. Aboriginal health, Adult Internal Medicine, Population Health were completed 2010, Mental Health, Rural Generalist Surgery, Paediatrics were completed in 2011.

(For a summary of the changes to assessment in the AST: See Section 5.7 (a).)

b. The College’s critical analysis of its progress against its own plans for curriculum development, curriculum challenges remaining and strategies to address them.

The Primary Curriculum has been successfully completed in accordance with its five year review cycle. The College is now engaged in reviews of the eight AST under the independent governance of ACRRM. In these, consultations have commenced with community and allied health professionals around all curricula. ACRRM has established a JCC for General Practice Procedural Surgery. This group will provide input into the curriculum review.

(For further details: See Section 3.1.1).

c. Comment on how the College has implemented the requirement (for registrars commencing in 2012) fora minimum of 6 months full time equivalent experience that meets the requirement “the registrar manages undifferentiated acute and chronic health problems in an unreferred patient population, providing care to all age groups, male and female, with continuity of care and preventative activities for individuals and families and organised care for practice populations.” How has this requirements been communicated and assessed for individual registrars?

See Section 3.2.1 above.

d. Information on the numbers and % of trainees in each pathway who seek and are granted RPL

and the periods of RPL granted.


e. An outline of plans for further development of the pathways to fellowship.

The College has identified two stages of training that require further development.



1. The CCT stage requires registrars to obtain rotations in General Medicine, Surgery,

Emergency, Paediatrics, Anaesthetics and Obstetrics and Gynaecology. It is becoming increasing difficult for registrars to obtain these rotations. For those who are successful the rotations are less than 10 weeks and may not provide adequate clinical experience. The College allows flexibility for registrars to meet the requirements through alternative ways of developing the skill set as they progress through training.

(See Attachment 3.6: Policy Statement on CCT.)

2. AST is also requiring further development. The College has identified eleven areas where

advanced skills are required to meet rural community needs. The College is receiving feedback that this does not meet the needs of rural communities in all States and Territories. An area that has and continues to be of particular focus is the development of skills capacities in Mental Health.

Work is underway and the College will discuss progress on these areas at the review visit.



4. Teaching and Learning

4.1 Teaching and learning methods

4.1.1 The training is practice-based involving the trainees’ personal participation in relevant aspects of the health services and, for clinical specialties, direct patient care.

ACRRM requires three stages of learning and clinical experience: Core Clinical Training (CCT), Primary Rural and Remote Training (PRRT) and Advanced Specialised Training (AST) to ensure that a registrar is exposed to the range of learning environments required to meet the ACRRM primary and AST curricula. Core Clinical Training

The aim of the CCT year is to provide a foundation of clinical competence across the major areas of hospital-based clinical practice relevant to both rural/remote and urban medicine. At the completion of CCT, the registrar is expected to be able to function competently as a junior doctor with significant responsibility for making patient care decisions, under broad supervision, across a range of specified medical disciplines. CCT provides baseline competencies for safe clinical practice. The following domains from the ACRRM Primary Curriculum are the focus during this foundation phase of training:

Provide medical care in the hospital setting; and

Respond to medical emergencies. Primary Rural and Remote Training

PRRT comprises 24 months experience in ACRRM accredited training posts including general practice, hospital, community, and other posts. PRRT aims to progressively build a registrar’s clinical and procedural skills, particularly in the context of rural and remote general practice. After completing this training period, the registrar is expected to be able to independently provide comprehensive and continuing care for individuals, families, and communities across the primary and secondary care continuum particularly in a rural and remote setting. By completion of CCT and PRRT the registrar would normally cover the full scope of learning outlined in the ACRRM Primary Curriculum. Advanced Specialised Training AST involves a minimum of 12 months training in one of eleven specified disciplines; (See Section 3.1.1). AST provides an opportunity for a registrar to extend skills and knowledge beyond the ACRRM Primary Curriculum learning outcomes in one specialised discipline that is relevant to general practice in the rural and remote context. The aim is to ensure that a doctor who attains Fellowship of ACRRM is able to contribute to specialised medical services, and work with medical colleagues, to ensure that communities are afforded access to a full range of medical services. (See Attachment 3.6: Policy Statements on CCT, PRRT and AST).

(See Attachment 3.2: AST Curricula).




4.1.2 The training program includes appropriately integrated practical and theoretical instruction.

Training Pathways to FACRRM

Three pathways are available for registrars to complete the ACRRM vocational training program. All pathways are based on an enhanced apprentice ship model that includes integrated practical and theoretical instruction and have exactly the same standards, curriculum and assessments. The ACRRM vocational training program (all pathways) requires all registrars to complete:

Education program provided by training provider (see further information below);

All formative and summative assessment modalities; (See Section 5); Required training time in accredited and appropriately supervised teaching posts this

includes requirements to participate in education activities at the post;

At least two emergency courses accredited by ACRRM; and

Completion of a minimum of four Green Flagged ACRRM online education modules through RRMEO.

The ACRRM vocational training program does not include university or other formal courses (with the exception of emergency medicine training courses outlined above). However as stated previously it is possible to complete other qualifications and have these recognised as contributing to the training requirements. ACRRM offers a range of online education modules and assessment preparation activities such as study groups and mock exams. The Vocational Preparation Pathway (VPP) and RVTS pathways are delivered by training providers accredited by ACRRM against the ACRRM Standards for RTPs Recognition, through the Bi-College Accreditation Principles and Outcomes framework. The Independent Pathway (IP) is delivered by ACRRM.

(For an outline of the accreditation process: See Section 8).

(See Attachment 4.1: List of RRMEO Modules).

Vocational Preparation Pathway

The VPP is the predominant pathway particularly, for new graduates. This pathway is funded by the Australian Government. GPET has been managing the AGPT program on behalf of the Australian Government; training is delivered by a network of 17 RTPs. The 15 regional and remote based RTPs are accredited by ACRRM against defined standards for program delivery. The rural RTPs have been fully accredited as training providers by ACRRM since 2008. ACRRM also maintains contact with those metropolitan RTPs who have an interest in staying abreast of the ACRRM training and issues. It is envisaged that this framework will be undergoing considerable changes over the next 12 months and the College is engaged in talks with the Commonwealth regarding these changes. The 2015 AGPT Handbook provides further information in regard to integrated practical and theoretical instruction included in the VPP.

(See Attachment 7.4: AGPT Handbook 2015).

http://www.gpet.com.au/Junior-doctors/Australian-General-Practice-Training--AGPT--program/New-Applicants


Remote Vocational Training Scheme Pathway

The RVTS pathway provides a vocational training option for general practitioners who would otherwise need to leave their rural or remote communities in order to gain vocational training. The program is delivered almost entirely through distance-based supervision and regular peer learning workshops. This pathway is funded by the Australian Government and is delivered by the RVTS organisation. RVTS was fully accredited as a training provider by ACRRM in February 2009. The RVTS Registrar Handbook 2014 provides further information.

(See Attachment 7.5: RVTS Registrar Handbook 2014)

Independent Pathway

The IP is generally more suitable for doctors who have a broader and more extensive range of work experience or a strong focus on self-directed learning than those who might elect to pursue training through the VPP or the RVTS. Admission to the IP is administered directly through ACRRM. The pathway is a full fee paying pathway and is not subsidised or funded by the Australian Government. The Independent Pathway ‘How to Apply’ Guide provides further information.

(See Attachment 7.6: ‘How to Apply to the Independent Pathway’ Guide 2015.)

4.1.3 The training process ensures an increasing degree of independent responsibility as skills and knowledge grow.

Three features of the vocational training program facilitate increasing responsibility:

1. The Standards for Teaching Posts and Teachers in Primary Rural and Remote Training establish an expectation that registrars are provided with an increasing degree of independence and responsibility as they progress through the training pathway and gain competency. Guidance is provided as to what may be an appropriate level of supervision, for example in the first 12 months of training the supervisor undertakes direct observation or review of the registrars taped consultations. Supervisors are required to provide ongoing appraisal and assessment of the registrar in accordance with the learning stage of the registrar. Registrars are required to have a learning plan and supervisors are encouraged to assist the registrar with developing and reviewing this plan.

2. The three stages of learning and experience: CCT, PRRT and AST facilitate increasing levels of

responsibility in general practice, (For further details: See Section 4.1.1.)

3. The programmatic approach to assessment using repeated formative measurements of the registrar over time facilitates succession through the training program. For example, the formative miniCEX is conducted regularly providing the registrar and supervisor with structured information about the registrar’s progress through training. The summative assessment modalities are also distributed across the four years of the program; this reflects the need for registrars to view their learning as evolving. (For further details: See Section 5).

(See Attachment 8.1: Standards for Teaching Posts and Teachers in PRRT)

http://www.rvts.org.au/training/handbook

https://www.acrrm.org.au/ipapplication-2013




KK Through the College IT team increase the liaison with registrars to continue improvement in the website’s capacity to support registrars’ learning plans. (Standard 4.1.2)

ACRRM’s new Customer Management Relationship (CRM) System will enable the College to obtain a 360 degree view of the Registrar’s interactions with the College from one data repository. All information will be kept in a single system streamlining communication with Registrars. A new online portal will be available allowing Registrars to register in training; track their learning activities; and, enrol in courses and events. Registrars will have additional functionality allowing them to share Learning activities with their Supervisors. The College will also offer an enterprise search engine providing Registrars with increased intuitive searching capability accessing and retrieving information from a number of repositories. Registrars will be able to nominate what information should be targeted for them, so the information they receive from the College matches their area of expertise and interests. The CRM System development commencement has been extended. The College is working towards a ‘go live’ date in the later part of 2014. The College expects to be able to demonstrate the new system to the review team in October. Meanwhile registrars continue to be able to track training progression and log procedures on the existing system through RRMEO. A number of processes have been put in place to ensure support for registrars’ learning plans:

The College provides access for all registrars at all times to their learning plans. Where problems arise, registrars have access to an email (or phone if necessary) help desk based at the ACRRM Secretariat (available 9-5 Monday to Friday).

IP Registrars are required to undertake Induction Workshops held twice a year which among other purposes provide an opportunity for ACRRM staff to provide instruction on the online learning plans and other aspects of the RRMEO platform. It is an expectation that this service will be provided for ACRRM registrars in the AGPT and RVTS programs by their respective training providers.

Additionally, occasional workshops are held to provide instruction in the RRMEO platform and particularly in the use of the learning planners as appropriate.



a. Summarise the teaching and learning methods used in the different components of the program and in different pathways and identify any changes since 2010.

The training providers on all pathways have continued to develop teaching and learning methods. At the time of writing this submission five training providers have been reviewed in the current round of accreditation. The College has noted considerable developments by these providers with increased range of education content in response to College curricula and regional needs. In addition in many providers the delivery methods have become more flexible and accessible to registrars across their training footprint. Some examples are development of training nodes with local education delivery, rural community training hubs involving training of all health professionals, greater use of virtual classroom and online delivery, blended delivery involving face-to-face workshops with online information and activities pre and post workshop, peer learning. Many training providers have also invested in learning management systems which allow registrars, educators, and supervisor’s ongoing access to resources from workshops. The IP has also made many improvements. The structure of the education remains the same but the content tutorials and workshops have developed in response to registrar feedback. In particular there are more procedural skills activities in the face-to-face workshops. The education program is more closely aligned with the curriculum through the development of a teaching and learning blueprint and each education session being mapped to the curriculum.

All training providers are required to map education activities to the curriculum. Compliance with this requirement is monitored through the Bi-College RTP Accreditation Program (Outcome 2.4). (See Attachment 8.6: Bi-College RTP Accreditation Program; Principles and Outcomes (Outcome 2.4)) (See Attachment 4.2: IP Workshop Program 2013 and 2014) (See Attachment 4.3: ACRRM Teaching and Learning Blueprint)

(See Attachment 4.4: Virtual Classroom Program 2014)

b. Comment on the range of educational activities available to registrars and their accessibility, with emphasis on any developments since 2010.

There are now 103 education modules available. The range of online modules on RRMEO has expanded with 183 being added since 2010. The College is prioritising areas for future online module development, this is informed in part by identification of assessment results areas where registrars are not performing well. The Emergency Medicine Course Requirements policy was revised in response to feedback from registrars. Previously all courses had to be current at the time of awarding Fellowship, the policy is now in line with the Training Time Policy.

The accredited Tier 1 or 2 courses referred to above must have been undertaken within 10 years of Fellowship (maximum training time allowed as defined in Training Time Policy). These courses must be accredited by ACRRM for vocational training.



The registrar is required to ensure that at least one course covering the Advanced Life Support (ALS) skills and knowledge has been completed within three years of Fellowship. This requirement is consistent with ongoing professional development requirements as described in the ACRRM Professional Development Program. Providing that the vocational training EM course requirements have been met, the EM course designed to maintain skills may be chosen from courses accredited for the ACRRM vocational training or the ACRRM Professional Development Program.

The number of Emergency Courses accredited as suitable for training requirements has increased. This occurs through registrars and training providers identifying courses that are suitable and requesting that the College consider accreditation. The College has significantly increased the assessment support programs available for all registrars. These include assessment information sessions, mock exams and study groups. Information is available at https://www.acrrm.org.au/assessment. (See Attachment 4.5: Revised Emergency Medicine Course Requirements Policy)

c. Report on the trainees access to Rural and Remote Medical Education Online – how the College has monitored this since 2010, any difficulties encountered by trainees, and the College response to them.

The College has monitored registrar’s access to RRMEO and is not aware of any ongoing issues around access. All registrars are provided with information and RRMEO login details once enrolled with the College. RRMEO modules are promoted regularly to all registrars through email when a new module is released, and regularly via Country Watch and FACRRM Fundamentals. Online modules are also routinely recommended as way to meet individual learning needs. It is worthy of note that satisfaction with RRMEO Modules as demonstrated through the College’s Evaluation Programs is consistently very high; with from 85-96% of survey registrars in the 2013 Registrar Survey nominating it as relevant and useful (i.e. 85% in VPP, 85% in RVTS, 96% in IP). Similarly, 80% of respondents in the 2014 Vocational Training Exit Survey rated RRMEO modules as ‘highly effective’.

d. Any other planned developments that relate to Standard 4.

The College continues to monitor and review all modes of delivery. Of particular focus will be the impacts of the CRM system on the various programs. The College will continue to expand and refine its online education module offerings.



5. Assessment of Learning 5.1 Assessment approach ACRRM views assessment as an ongoing and integral part of the learning process. Assessment assists registrars in identifying and understanding their strengths and weaknesses, provides guidance, and enables registrars to become competent, confident and safe general practitioners. The three core assessment principles that are used are:

1. The content of assessment reflects the curriculum; 2. Assessment has been developed by clinically active Fellows and other medical practitioners; and 3. Registrars have the option to participate in the assessments within the locality where they live

and work.

5.1.1 The assessment program, which includes both summative and formative assessments,

reflects comprehensively the educational objectives of the training program.

The ACRRM assessment process incorporates a variety of best practice and evidence based modes of assessment:

Multiple choice question (MCQ) examination;

Structured assessment using multiple patient scenarios (StAMPS);

Procedural Skills Logbook (Logbook);

Mini Clinical Evaluation Exercise (miniCEX) formative and summative;

Multi-Source Feedback (MSF); and

Supervisor reports and

Projects (some AST assessments)

The assessment modalities are described in more detail in the Handbook for Fellowship Assessment. Collectively, these assessment modalities embrace all four levels of Millers Pyramid so that registrars are required to demonstrate that they ‘know’, ‘know how’, ‘show how’, and finally, what the registrar actually ‘does’ in the workplace. Figure 5.1: Miller’s Pyramid


A core feature of the ACRRM assessment process is the ‘programmatic approach’ (i.e. the assessment is integrated into all aspects of the curriculum and essentially a ‘program’ across the entire four years of training, rather than a specific instrument or examination). Registrars are required to achieve a minimum of a ‘Satisfactory Completion’ grade in each of the five summative assessment modalities to ensure competency is achieved across the curriculum. The programmatic approach allows ACRRM to combine assessment methods allowing for a combination of practice based and ‘external’ examinations. Each examination measures a different aspect of the registrar’s clinical skills. Similarly, the MSF and the miniCEX measure different attributes of the registrar’s professional behaviour, one as perceived by patients and colleagues and the other through direct examiner observation. The ACRRM vocational training program and assessment processes are directly structured around the ACRRM Primary Curriculum and AST curricula. The assessment blueprint articulated in the ACRRM Primary Curriculum ensures that ACRRM assessment processes are directly mapped against the curriculum domains and abilities to ensure that abilities are assessed at least once during the four-year program. Alignment of the educational objectives of the training program and assessment has been further enhanced through the operational and logistical structure of these areas within the ACRRM office design. Management of assessment is divided into four separate but interrelated roles: logistical management is with the Vocational Training and Assessment Manager, Clinical and Academic management is with the Principal Examiners, ratification of assessment results is with the Board of Examiners and oversight and consultative support and program review from the membership is with the Assessment Committee. This process allows for close integration between training and assessment, while also recognising the important separation of training delivery and assessment. (See Attachment 5.1 Fellowship Assessment Handbook 2014) (See Attachment 5.2 Assessment Blueprint) (See Attachment 3.1 Primary Curriculum 4th Edition) (See Attachment 3.2 AST Curricula)

5.1.2 The education provider uses a range of assessment formats that are appropriately aligned to the components of the training program.

All registrars training towards FACRRM must complete the following summative assessments aligned with the Primary Curriculum:

1. MCQ: Pass grade 2. miniCEX: Pass grade 3. StAMPS: Pass grade 4. MSF: Satisfactory completion 5. Procedural Skills Logbook: Satisfactory completion

Registrars are also required to obtain a Pass grade in each of the assessments for their chosen AST discipline as described below.



https://www.acrrm.org.au/misc/curriculum/Default.htm



Table 5.1: Summative Assessment requirements for each AST Discipline

DISCIPLINE SUMMATIVE ASSESSMENT

StAMPS Project Supervisor Report Logbook

Aboriginal & Torres Strait Islander Health

Adult Internal Medicine Anaesthetics Conducted by JCCA: Case commentaries, viva voce and a project

Emergency Medicine

Mental Health Obstetrics Conducted by RANZCOG: MCQ, oral examination, supervisor reports,

logbook, workplace-based validations, case syntheses

Paediatrics

Population Health

Remote Medicine

Surgery The standard for a successful outcome in each modality is that of a doctor practising safely and independently at Fellowship level.

Summary of Formative Assessment requirements

All registrars training towards FACRRM must complete the following PRRT formative assessments:

miniCEX of at least six consultations (compulsory for candidates commencing training from 2012 and strongly encouraged for all others)

Six monthly supervisor reports.

Registrars are also required to complete the following formative assessment specific to their chosen AST discipline. Table 5.2: Formative Assessment requirements for each AST discipline

DISCIPLINE

FORMATIVE ASSESSMENT MiniCEX

(5 Consultations) Supervisor Report

(at 6 Months) Project

(1500-2000 Words)

Aboriginal & Torres Strait Islander Health

Adult Internal Medicine Emergency Medicine

Mental Health Paediatrics Population Health

Remote Medicine

Surgery

Those AST that already have formal credentialed external examinations (e.g. the DRANZCOG examination and anaesthetic training) continue to use the JCC examination.

The key check points in the assessment process are the completion of the primary training summative assessments and completion of the AST year summative assessment.


Eligibility requirements

The eligibility criteria specified below must be satisfied before enrolment for assessment will be accepted. 1. All applicants must have current medical registration with Australian Health Practitioner

Regulation Agency (AHPRA) and be current financial members of ACRRM.

2. Applicants must be enrolled in one of the following pathways to enrol in any ACRRM assessment:

a) One of the three ACRRM training Pathways ACRRM Independent Pathway VPP; or RVTS; or

b) IMG Specialist Pathway.

3. Candidates on a training pathway: a) Prior to enrolling in PRRT assessment, candidates enrolled on a training pathway must have

completed: one year of ACRRM training or have received one year of RPL, prior to enrolling for the

MCQ and MSF (i.e. in year two, three or four of training). two years of ACRRM training or have been awarded two years for RPL, prior to

enrolling for summative miniCEX and StAMPS (i.e. in year three or four of training). b) Prior to enrolling in an Advanced Specialised Training assessment it is required that

candidates are undertaking, have completed training in the discipline, or have received RPL for training in the discipline. It is recommended that the assessments are taken in the later part of training. It is not a prerequisite to complete all primary training summative assessment before

undertaking the AST assessments.

4. Candidates on IMG Specialist Pathway

a) Prior to enrolling in assessment, doctors enrolled in the specialist pathway must have completed a portion of their peer review period as specified in their requirements.

5.1.3 The education provider has policies relating to disadvantage and special consideration in assessment, including making reasonable adjustments for trainees with a disability.

ACRRM has a Special Considerations policy. Special consideration may be granted to accommodate a disadvantage suffered by a candidate which is beyond his/her control and which is likely to, or has affected performance in assessment. The Special Considerations policy was developed in June 2013, it expands on and replaces the former Disability Policy. ACRRM also has a comprehensive Code of Conduct for all participants in the assessment process included in its Assessment Handbook. As outlined, the key foundations for the policy are: respect for people, integrity, diligence, and economy and efficiency.

(See Attachment 5.3: ACRRM Special Considerations Policy)

(See Attachment 5.1: ACRRM Fellowship Assessment Handbook (Appendix 3))




5.2 Feedback and performance

5.2.1 The education provider has processes for early identification of trainees who are under performing and for determining programs of remedial work for them.

5.2.2 The education provider facilitates regular feedback to trainees on performance to guide learning.

The ACRRM assessment approach maintains a commitment to a balance of formative assessment for feedback purposes and summative assessment to determine progression. Structured formative feedback is provided through two paths:

Formalised formative assessments; and

Feedback from summative assessments. The programmatic approach (See Section 5.1.1), directly lends itself to the early identification of registrars who are under performing. The results of formative and summative assessment are shared with all of the registrar’s supervisors, training and assessment providers. This vertically integrated structure facilitates the early identification of registrars who require educational assistance and the creation of tailor-made remediation programs that target specific areas of weaknesses. The structured formative processes also provide a formal mechanism to measure the effectiveness of remedial programs. ACRRM uses miniCEX as a formative and summative assessment. A minimum of six formative miniCEXs are required during training. The ACRRM MSF modality provides a detailed report that includes qualitative and quantitative results as well as comparison with international normative values. The registrar is required to complete a reflective exercise looking at the results of the MSF and then discuss this with their medical educator. This level of detail greatly assists in structured feedback to the registrar and informs remediation when required. The Logbook provides a record of the registrar’s developing procedural skills, and therefore assists in identifying registrars requiring additional assistance. The registrar is required to document that they have discussed their progress with their supervisor at least every six months, a process monitored by ACRRM through written documentation in the Logbook. The Logbook is used formatively and summatively. Registrars have opportunities to gain experience in assessment methods formatively before they experience them in a summative mode. The MCQ practice exam is marked thereby providing feedback to registrars prior to undertaking the summative MCQ. ACRRM offers mock StAMPS allowing the registrar to practice the assessment under exam conditions and receive feedback and an indication of readiness for enrolment in the next summative assessment. The summative miniCEX, MCQ and StAMPS are each reported as grades (i.e. pass or fail); and, the summative MSF is marked as ‘Satisfactory Completion’ as appropriate. These are all also broken down into a series of scores across a range of important attributes. Registrars who are unsuccessful in assessment are offered a feedback session with an examiner. They are encouraged to have their supervisor and/or Medical Educator attend the session as well.


The training providers are delegated to deliver and manage ACRRM training. Training providers are required to have processes in place that monitor and assess performance of doctors in training and provide feedback on training progress (as per Bi-College RTP Accreditation Principles and Outcomes (Outcome 4.5). Compliance with this outcome is monitored through the Accreditation Program. Training providers monitor information from the range of ACRRM requirements outlined above in addition to other sources including:

Supervisor reports at the end of each 6 months

Medical Educator visits to the registrars in practice

Dedicated medical educator or education staff member allocated to each registrar Other formative assessments set by the provider

Participation in education sessions (For examples for reports received by Registrars: See Attachment 5.4: Registrar Assessment Reports

(MiniCEX, MSF, MCQ, StAMPS))

(See Attachment 8.6: Bi-College RTP Accreditation Principles and Outcomes)

5.2.3 The training organisation provides feedback to supervisors of training on trainee performance where appropriate.

Each registrar is required to sign a release statement which allows sharing of all of the registrar’s formative and summative assessment data with their supervisors and training providers (i.e. RTP, RVTS or ACRRM). Training Providers are provided with the assessment outcomes for their registrars. As outlined above, where a registrar has not met the standard in an assessment they are offered a feedback session by the College. Registrars are encouraged to invite the supervisor and or Medical Examiner to attend these feedback sessions.



5.3 Assessment Quality

5.3.1 The training organisation has a policy on the evaluation of the reliability and validity of assessment methods, the educational impact of the assessment on trainee learning and the feasibility of the assessment items. It introduces new assessment methods where required.

Reliability ACRRM formally evaluates the reliability of each assessment modality on an ongoing basis. The MCQ and StAMPS examinations have formal statistical testing after each exam and miniCEX each 6 months using standard statistical methodology. All assessments have appropriate Cronbach alpha scores. The MCQ examinations have achieved Cronbach’s alpha score ranging between 0.7 and 0.8. Primary Curriculum and AST EM StAMPS Cronbach’s alpha scores are consistently high and range from 0.90 to 0.99. MiniCEX Cronbach alpha scores are calculated each 6 months and the scores have ranged between 0.88 and 0.92. Standard setting ACRRM uses internationally recognised processes for standard setting and definition of the cut point between pass and fail in each of the summative assessment modalities. Standard setting for the MCQ examination is based on the modified Angoff method. This involves setting a standard score for test items prior to the test using judgements by experts based on the projected performance of ‘borderline registrars’. The pass mark for each examination is calculated from the average Angoff score with consideration for an adjustment by removal of questions that have not performed well and/or standard error of measurement. Standard setting for the summative miniCEX assessment is established by the modified Rothman method, this method uses global judgements of borderline registrars made during the test. All examiners are trained in the method prior to conducting the assessment. Standard setting for the summative StAMPS assessment is also established by the Rothman method described above. The summative MSF assessment modality is performed under licence by Client Focused Evaluations Program (CFEP). An extensive international normative value database is held by this organisation. Assessment items development The questions used in the MCQ and StAMPS examinations are developed by practising rural general practitioners. There is a small “expert team” developing assessment items with input form a larger group of practising rural doctors. Following the examination, standard question reliability statistics such as Cronbach’s alpha are considered, with reliable questions placed in the repository for future examinations or to be included in publicly released practice exams. Those with poor reliability are referred back to the editorial process for consideration of redevelopment or retiring. Assessment items validity The assessment program including StAMPS has been presented to the wider international academic medical education audience through both peer reviewed publications and conference presentations. A list of publications is provided.

(See Attachment 5.5: ACRRM Assessment Program Publications and Presentations)


5.4 Assessment of specialists trained overseas

5.4.1 The processes for assessing of specialists trained overseas are in accordance with the principles outlined by the AMC and the Committee of Presidents of Medical Colleges Joint Standing Committee on Overseas Trained Specialists (for Australia) or by the Medical Council of New Zealand (for New Zealand).

Specialist Pathway Since 2010 the College has received formal accreditation of the assessment process for Overseas Trained General Practitioners (OTGP) and has developed and established processes and systems to support the implementation of the Colleges policy in accordance with the principles outlined by the AMC/CPMC Standing Committee on Overseas Trained Specialists “Assessment of Overseas Trained Specialists – Guidance for Colleges”. The accredited policy detailed the assessment requirements and processes, for OTGP applicants for specialist recognition, appointment to an Area of Need position and Fellowship of ACRRM under the Specialist Pathway. This policy also provided details of the assessment criteria for comparability and set out the requirements for the Professional Review period which is required to be undertaken before ACRRM will recommend recognition as a specialist to AMC and Fellowship of ACRRM. Since the Colleges initial accreditation under the Specialist Pathway, ACRRM has applied and been granted accreditation for an ad eundum gradum pathway to fellowship of ACRRM for holders of two comparable fellowships. In December 2011 following comprehensive research and consultation, ACRRM submitted a proposal to AMC for the establishment of an ad eundum gradum pathway for holders of Certification College of Family Physicians of Canada (CCFP) (post 1992), Fellowship of the Royal New Zealand College of General Practitioners (FRNZCGP) and Fellowship of the Division of Rural Hospital Medicine New Zealand (FDRHMNZ). The proposal set out the specific requirements for granting fellowship under an ad eundum gradum pathway. At the November 2012 meeting of the AMC Specialist Education Accreditation Committee (SEAC) agreed that the FRNZCGP satisfied the requirements for this pathway. The SEAC meeting of May 2013 considered further information regarding ACRRM’s proposal and agreed that the CCFP post 1992 also meet the requirements of the pathway. The committee was not convinced the FDRHMNZ did. As an outcome of the AMC accreditation ACRRM has established processes and systems to support the implementation of this pathway. At the ACRRM Board meeting of April 2013 the Board reviewed the Colleges Codified List for equivalence for overseas trained general practitioners for the purpose of ACRRM’s Specialist Pathway. The review was precipitated by changes that had occurred since the list was submitted with ACRRM’s accreditation application. Changes included the establishment of reciprocal arrangements between ACRRM and the College of Family Physician Canada (CFPC) and the AMC approval for the establishment of an ad eundum gradum pathway for holders of the CCFP and FRNZCGP. This revised list was submitted to AMC in April 2013. Standard Pathway

At the time of the last report ACRRM had developed specific assessment processes to assist in determining the quality and relevance of prior training and experience of IMG’s on the Standard


Pathway. The Medical Board of Queensland (MBQ) had agreed to implement Pre-Employment Structured Clinical Interviews (PESCI’s) as a mandatory component of registration for IMG’s seeking registration in Queensland on the Standard Pathway and Competent Authority Pathway (CAP).

The purpose of the PESCI is to establish whether an IMG has the knowledge, skills and experience to practise safely and effectively in the position for which registration is being sought. The MBQ at its July meeting considered ACRRM’s proposal to conduct the PESCI’s in Queensland and subsequently appointed ACRRM to conduct PESCI’s on their behalf. In preparation for mandatory PESCI’s in Queensland, ACRRM established an interview structure and commenced the development of a bank of questions and scenarios. ACRRM has also conducted three trial PESCI interviews which were observed by the Medical Advisor to the MBQ. The purpose of the trial was to ensure the processes and systems established meet the requirements of the MBQ and that the interview outcomes reported provide sufficient information to the MBQ to assist it in determining registration matters. With the event of a national registration system in April 2012, ACRRM applied for accreditation to conduct PESCI’s in other states and territories’ using the structure’s, processes and protocols developed and based on the Queensland experience. As an outcome of that application ACRRM was granted accreditation to conduct PESCI’s for each state and territory in Australia. In response ACRRM has recruited and trained additional panellist’s, conducted a review of the Colleges scenario bank and has engaged a consultant to review the operational processes and protocols to ensure ACRRM continues to develop and implement best practice in this critical assessment process. Competent Authority Pathway Since ACRRM’s last report the College has sought and gained AMC accreditation to undertake Workplace Based Assessment (WBA) for CAP doctors working in general practice. Doctors holding the AMC Advance Standard Certificate entering general practice with limited registration were required to enrol in ACRRM accredited assessment process. The assessment process required the CAP doctor to successfully complete a summative Multi-source Feedback (MSF) and provide three satisfactory supervisor reports during their 12 month period of supervised practice. All results were ratified by the ACRRM Board of Examiners before being reported to AMC for their consideration. The recent review of the CAP conducted by the Medical Board of Australia (MBA) will see significant change to the CAP as of 1 July 2014. As of that time all CAP doctors will be able to apply directly to the MBA for provisional registration hence eliminating the need for WBA for those working in general practice. Governance

As previously reported to support the College’s expanded activities in regard to OTGP and IMG assessment ACRRM reviewed its governance and operational structures and established an International Medical Graduate Assessment Committee as a committee of the Board. A major role of the committee was to overview the development, implementation and review of the IMG assessment program and policies within ACRRM. This committee continues to meet bi-monthly and has been proactive in the ongoing development of this important component of the Colleges work. The establishment and ongoing development of an operational unit within ACRRM’s support structures to ensure effective and efficient administration and operational management of the College’s IMG Assessment program has been important to the Colleges ability to expand its IMG related activities. Also important to this has been the development of online documentation and information to support the application process.



00 Enhance capacity to assess registrars’ practical and procedural skills (Standard 5.1.2)

The ACRRM Primary Curriculum procedural skills logbook was revised as part of the Primary Curriculum revision. The revised logbook is structured according to curriculum statement. The revised version is available in printed form and as a PDF on the website. See ACRRM website: ‘Summary of the Changes to the 3

rd Edition’ and also ‘Assessment Changes for

2014’. ACRRM continues to assess physical examination skills as part of the summative miniCEX. The mandatory requirements for physical examination are five detailed physical examinations from at least three of the following:

Cardiovascular;

Respiratory;

Abdominal; Neurological;

Endocrine;

Musculoskeletal (region);

Mini-mental state examination (cognition testing), (maximum of one);

Neonatal/paediatric (maximum of one); or

Antenatal (maximum of one). Nine cases must be seen or 4.5 hours of assessment time, whichever occurs first. If physical examinations have not been performed on at least three of the required systems, the examiner must instruct the candidate to perform an examination on one of the systems specified. If there are insufficient patients then the registrar will be required to conduct a consultation on an available staff member to ensure the required number of cases. No more than one such consultation can be performed in lieu of a patient. This consultation must address any shortfall in the mandatory history taking and physical requirements and must be complimentary to the actual patients the candidate has seen. Any such consultation is to be directed by the examiner accordingly. ACRRM has found assessing physical examination in the context of a real consultation during miniCEX is valuable and it assesses what the doctor does do, rather than what they can do. It is also allows the assessment of physical examination as part of a consultation rather than in isolation. Registrars’ practical and procedural skills are also assessed through formative miniCEX and six monthly supervisor reports throughout training including advanced specialised training. The College has given consideration to establishing a separate assessment to assess physical examination and other procedural skills, however it has been concluded that the current level of assessment adequately covers the curriculum and that increasing assessment requirements would create a significant barrier for registrars. The assessment of physical examination as part of miniCEX also allows for a grater number and range of systems to be assessed than if it where part of another assessment for example StAMPS. The Assessment Committee will continue to review these considerations.





5.6 Conditions

Condition 13 Report on the implementation of the ‘Independent Pathway Active Training Policy’ and other mechanisms to monitor the progress of registrars on the Independent Pathway. (Standard 5.2)

There have been a large number of developments made to the IP, these include:

More robust selection process. (See Section 7.5 (a))

Changes to RPL process with provisional RPL awarded on paper based assessment of experience and performance followed by final RPL awarded following selection interview which includes clinical scenarios. (See Section 3.4.2)

Introduction of the training agreement.

(See Attachment 7.7: ACRRM IP Training Agreement)

A six-monthly review process for all registrars to check training is progressing appropriately according to training agreement and Active Training policy and records are accurate. Those not progressing are worked with to set training goals and timeframes.

Registrars who are unsuccessful in an assessment are reviewed by the Director of Education, Medical Examiner and Training Coordinator and an individualised learning plan is established. This may also lead to a revision of RPL.

In addition registrars are encouraged to take advantage of the assessment support programs now offered by ACRRM.


5.7 Other matters to be addressed (relating to 5.1-3)

a. An outline of the College’s plans for development and review of its assessment strategies as they were in 2010 and how these have evolved since then. Comment on challenges remaining for the College and strategies to address them.

Assessment is reviewed in an ongoing way using feedback from registrars, examiners, assessment item writers and invigilators after each assessment. As described previously a range of quality statistics are considered following exam assessment. A face-to-face Assessment Review day is held every two years, with the most recent review day was held on 20 August 2013. This incorporated presentations overviewing StAMPs and MCQ. The next review day is being held on 1 August 2014.

(See Attachment 5.6: Assessment Review Day Program)

(See Attachment 5.7: Presentation on StAMPS and MCQ in the ACRRM program 2012.)

ACRRM has undertaken work to further develop and refine its formative and summative assessment processes. The main changes to assessment that have been made since 2010 are:

MiniCEX:

2012 Introduction of mandatory formative miniCEX– minimum of 6 consultations

Reduction from a large pool of miniCEX examiners to a smaller experienced team of examiners

2013 improving initial training of examiners and adding 6-monthly examiner updates by Principal Examiner

2013 combining marking categories for Clinical management and rural and remote context into one category Clinical management in the rural and remote context

2014 pass grade required in physical examination as well as overall

2014 introducing a feedback survey after each miniCEX for candidate, examiner and practice staff.

2014 Principal examiner reviews every miniCEX report prior to presentation at Board of Examiners and examiners followed up if any anomalies. Previously only reports where the candidate failed where reviewed by Principal Examiner.

MCQ

2009 in response to requests by registrar MCQ offered in a central exam centre as well as by arranging own venue

2012 Introduction of road testing the exam with a recent registrar 2014 September reduce MCQ answer options from five to four

StAMPS

2012 in response to registrar requests StAMPS offered by video conferencing and face-to-face each year

2012 increasing the rigour of examiner training by new examiners co-marking with experienced examiners for their first exam

2012 Introduction of study groups, coaching and then in 2014 mock exams 2013 combining marking categories for Clinical management and rural and remote context

into one category Clinical management in the rural and remote context

2013 October running StAMPS in two centres concurrently due to an increase in numbers. This was very successful and is being repeated in 2014.

2014 Using a single Community Profile as the setting for all StAMPS assessments for the year


Primary Curriculum Procedural skill logbook

2012 in response to registrar feedback to options for certifying procedures; these can now either be signed by doctor or alternately the registrar provides the name and contact information of the certifier (this enables ACRRM to verify the validity of the logbook if there are concerns).

2012 Rural Clinical School medical students are able to have some procedures certified in final two years (see front of logbook for list of procedures)

2014 Revision in alignment with the Primary Curriculum 4th

edition AST assessments

2013 Changed the main summative assessments for ASTs in Mental Health, Paediatrics and Adult Internal Medicine from miniCEX to StAMPS. This was primarily done because of the difficulty of ensuring that the candidate was in a suitable environment to demonstrate the skills in the AST.

2013 the first AST STAMPS were held for Surgery, Paediatrics and Adult Internal Medicine late and in early 2014 for Mental Health. The assessments were delivered remotely using Go to Meeting. Item writers and Examiners included a Specialist from the relevant discipline and a FACRRM with Advanced Skills in the area. Feedback from all stakeholders was positive.

MSF

2012 MSF changed from requiring a pass grade to requiring candidates demonstrate satisfactory completion of at least one MSF.

Satisfactory completion requires submission to ACRRM of:

A completed MSF report covering the two components;

A completed reflective exercise; and

Evidence of discussion with a Medical Educator and remediation if required.

Assessment writers and examiners

Considerable refinement has been made to the development and assessment items for StAMPS and MCQ. This has been achieved through development of a team of doctors who are developing their skills through experience and provision of feedback on performance of the assessment items. There also continues to be a wider pool of doctors providing the cases from practice that are then refined.

The pool of examiners and writers also continues to expand and the College ensures that all key positions have a succession plan.

Challenges The main ongoing challenge is to ensure examiner and item writing capacity keeps pace with demand. The College has been active in involving more Fellows in all aspects of the College including in training and assessment activities. Registrars are identified during training as potential examiners. Doctors are encouraged to assist with study groups and then if suitable become an examiner. Eight new examiners have been recruited for the October StAMPS assessment. They are undergoing training with the Principal Examiner and through working alongside an experienced examiner in the July mock exam.


b. Provide a table showing the number and percentage of candidates who passed various summative assessments at their first, second, third and subsequent attempts for the period 2010 to 2014.

Outcomes of assessment are provided in the tables below. Table 5.3 provides the results of registrars undertaking assessment in 2010 to 2014. Tables 5.5-5.11 provide the number and percentages of attempts required to pass assessment modalities from 2010 through to 2014. Table 5.3: Vocational Programs Assessment results for 2010-2014

Modality Year Result Program Total

AGPT RVTS

Independent Pathway

Specialist Pathway

2010

Pass 15 7 32 0 55**

Fail 4 0 3 0 8**

MSF 2011

Pass 13 6 46 4 69

Fail 5 2 11 2 20

2012*

Pass 4 2 4 2 12

Fail 1 3 0 0 4

Satisfactory Completion

7 3 21 0 31

2013

Pass 0 0 0 12 12

Fail 0 0 0 1 1


39 4 27 0 70

2014

Pass 0 0 0 5 5

Fail 0 0 0 0 0


14 3 15 0 32

2010

Pass 20 7 17 0 44

Fail 3 4 19 0 26

MCQ 2011

Pass 22 9 43 0 74

Fail 3 5 14 0 22

2012

Pass 27 7 35 1 70

Fail 4 0 12 0 12

2013

Pass 34 8 29 0 71

Fail 6 5 12 0 23

2014

Pass 19 1 14 0 34

Fail 0 1 12 0 13 * MSF changed from a Pass/Fail to Satisfactory Completion in 2012 with the exception of Specialist Pathway candidates. ** Total includes 1 Registrar with Advanced Standing


Table 5.3: Vocational Programs Assessment results for 2010-2014 (Continued)

Modality Year Result

Program Total

AGPT RVTS Independent

Pathway Specialist Pathway

Total

2010 Pass 11 4 11 0 28***

Fail 3 1 4 0 9**

miniCEX 2011 Pass 15 6 22 1 44

Fail 2 0 5 0 7

2012 Pass 24 9 32 2 67

Fail 3 0 4 0 7

2013 Pass 33 7 49 8 97

Fail 1 0 7 0 8

2014

Pass 14 0 14 3 31

Fail 1 0 1 1 3

2010

Pass 18 6 24 0 48

Fail 2 2 7 0 11 StAMPS

2011 Pass 15 6 16 0 37

Fail 6 7 27 0 40

2012

Pass 24 10 29 0 63

Fail 11 2 31 0 44

2013 Pass 32 4 24 1 61

Fail 8 1 30 2 41

2014

Pass 14 1 11 2 28

Fail 2 1 8 1 11

* MSF changed from a Pass/Fail to Satisfactory Completion in 2012 with the exception of Specialist Pathway candidates. ** Total includes 1 registrar with Advanced Standing *** Total includes 2 registrars with Advanced Standing.


Table 5.4: AST Assessment Results 2010 - 2014

*AST miniCEX no longer is an assessment modality for AST - it was replaced by StAMPS in 2014.

Modality Year Result Program

AGPT RVTS Independent

Pathway Specialist Pathway

Total

2012 Pass 0 0 1 0 1

Fail 0 0 0 0 0

AST miniCEX*

2013 Pass 0 0 1 0 1

Fail 0 0 0 0 0

2014 Pass NA NA NA NA 0

Fail NA NA NA NA 0

AST Emergency Medicine StAMPS

2010 Pass 3 1 3 0 7

Fail 3 0 6 0 9

2011 Pass 8 1 8 0 17

Fail 0 0 13 0 13

2012 Pass 3 1 6 0 10

Fail 1 1 12 0 14

2013 Pass 2 1 4 0 7

Fail 9 0 8 0 17

2014 Pass 3 0 4 0 7

Fail 4 1 10 0 15

AST Paediatrics

StAMPS

2014 Pass 0 0 0 0 0

Fail

0 0 1 0 1

AST Adult Internal

Medicine StAMPS

2013 Pass

1 0 0 0 1

Fail 1 0 0 0 1

AST Surgery StAMPS

2013 Pass 2 0 0 0 2

Fail 0 0 1 0 1

AST Mental Health

StAMPS 2014

Pass 0 0 2 0 2

Fail 0 0 0 0 0

AST Projects 2012

Pass 2 1 2 0 5

Fail 0 0 0 0 0

2013 Pass 2 1 2 0 5

Fail 0 0 0 0 0

2014 Pass 0 0 6 0 6 Fail 0 0 0 0 0


Table 5.5: Number of attempts required to pass assessments in 2010

2010

Attempt

1 2 3 4 1 2 3 4

Number of registrars passed Percentage of registrars passed

MCQ 41 2 1 0 59% 29% 50% 0%

MSF 53 1 - - 89% 0% - -

miniCEX 28 - - 77% - - -

StAMPS 45 3 - - 82% 60% - -

AST EM StAMPS

7 - - - 44% - - -

Table 5.6 Number of attempts required to pass assessments in 2011

2011

Attempt

1 2 3 4 1 2 3 4


MCQ 61 12 0 1 82% 60% 0% 100%

MSF 93 2 - - 71% 50% - -

miniCEX 39 5 - - 86% 80% - -

StAMPS 33 2 2 - 52% 40% 66% -

AST EM StAMPS

3 3 - - 21% 43% - -


2012

Attempt

1 2 3 4 1 2 3 4


MCQ 62 3 5 - 86% 43% 71% -

MSF* 12 - - - 33% - - -

miniCEX 63 4 - - 94% 100% - -

StAMPS 50 11 2 - 85% 52% 25% - AST EM StAMPS

9 1 - - 45% 100% 0% -

AST miniCEX

1 - - - 100% - - -

AST Projects

5 - - - 100% - - -



2013

Attempt

1 2 3 4 1 2 3 4


MCQ 61 7 2 1 85% 78% 50% 100%

MSF* - - - - - - - -

miniCEX 92 5 - - 95% 83% - -

StAMPS 42 10 7 2 59% 59% 54% 66%

AST EM StAMPS

3 4 1 17% 66% 0% -

AST miniCEX

1 - - - 100% - - -

AST Adult Internal Medicine

1 - - - 100%

AST Surgery StAMPS

2 - - - 66% - - -

AST Projects

5 - - - 100% - - -


2014

Attempt

1 2 3 4 1 2 3 4


MCQ 30 3 1 - 79% 37% 100% -

MSF* - - - - - - - -

miniCEX 28 3 - - 90% 100% - -

StAMPS 22 5 1 - 79% 45% 100% -

AST EM StAMPS

5 2 0 - 42% 20% 0% -

AST Paediatrics StAMPS

0 - - - 0% - - -

AST Mental Health StAMPS

2 - - - 100% - - -

AST Projects

6 - - - 100% - - -

*MSF changed from a Pass/Fail to Satisfactory Completion in 2012 with the exception of Specialist Pathway candidates.


c. Data provided by ACRRM indicate that participants in the Independent Pathway underperform in some assessment modalities. Outline how ACRRM monitors, analyses the cause, and addressees the differences between pathways in performance in the assessments.

Assessment outcomes are routinely broken down by training pathway and number of attempts. This information is presented to the Board of Examiners and Training and Assessment Team. Annually this information is reported in the Evaluation report and shared with Assessment Committee and Vocational Training Committee. Detailed analysis of the performance of doctors on the IP was undertaken in 2012. This helped to identify issues and prompted the improvements in IP selection and training outlined earlier. As reported earlier (See Section 5.6 Condition 13); each IP candidate that is unsuccessful in an assessment is reviewed individually to identify causes and put in place learning activities.

d. The number of registrars considered under College’s Candidate Review Policy by pathway and outcomes.

Table 5.12: Registrars reviewed by Registrar Review Panel and Outcomes 2010-2014

Year No. of Registrars Reviewed

Outcomes

2010 Nil Nil

2011 5 2 registrars were given a remediation plan and allowed to continue with training; (1 of these has now Fellowed; and, the other withdrew). The other 3 were withdrawn from training (because 1 registration had lapsed and the other 2 had lapsed for non-compliance).

2012 Nil Nil

2013 1 This registrar was withdrawn due to non-compliance

2014 1 A remediation plan was set. So far the registrar has met all requirements to remain in the program.

Three more registrars are awaiting review by the registrar review panel; three for non-compliance with training requirements and the third is not currently registered. The Registrar Review Panel will meet in August.

e. The progress of the processes designed to improve IP registrars’ assessment performance:

Study groups (due to commence October 2012)

Commencing 2013 cohort applicants who are awarded full RPL will be required to attend a study group prior to undertaking assessment

Medical educators to assist the registrars to know when they are ready to sit assessment

Remediation program (due to commence September 2012)

Coaching on exam technique (Commenced June 2012)

The College has developed and refined a range of assessment support activities: These are open to registrars on all pathways and include:

Assessment Information sessions offered twice a year prior to close of assessment enrolments via virtual classroom. The first sessions were held in July 2014.


Mock exam for StAMPS held a number of times per year (this replaced the previous coaching workshop)

Study groups (via virtual classroom) held for eight weeks prior to each Primary Curriculum and AST Emergency Medicine StAMPS.

Assessment information on website has been revised. (See ACRRM website Assessment Page)

Effectiveness At each assessment the College has analysed the outcomes according to the assessment support programs that the candidate has participated in. The numbers are small and results need to be interpreted with caution. The Study Groups have been running since October 2012, while there has been strong demand and the groups have been well received by candidates a clear correlation between attendance at study groups and assessment outcome has not yet been established. Coaching workshops which were held during 2012 and 2013 were also well received by candidates but a clear correlation between attendance at coaching workshop attendance and assessment outcome is yet to be statistically evidenced. StAMPS Mock Exams have been run twice and in the last StAMPS exam were able to see a positive relationship between those who attended the Mock Exam and the assessment outcome. Those who attend both study groups and mock exam appeared to benefit more. The improvements were noticeable for both resits and registrars on the IP. In 2013 five IP registrars completed an eight-week remediation program involving one-to-one tutoring by a Medical Educator. They had been referred to remediation as a result of being unsuccessful in StAMPS on three occasions. Two have now completed assessments and Fellowed, two have not resat assessment and one withdrew from training. In 2014 five registrars are currently undergoing remediation and two are soon to commence.



f. Provide a table for the period 2010 to 2014 showing the numbers of candidate who withdrew from the program before completion and a summary of the reasons for withdrawal.

Table 5.13: Number of Registrars Withdrawing from Pathway (Jan 2010–Jul 2014)

Pathway Number withdrawing Summary of reasons for withdrawal (by pathway)

Independent Pathway 51 Non-payment of fees Change to another specialist College Ill health/death Registration issues Moved overseas Non-compliance with training requirements Unable to get post accredited Financial issues Moved RVTS Unhappy with assessment outcome Time constraints

RVTS 32 Non-payment of resubs and/or Not continuing with FACRRM Ill health AST project too difficult to complete May join IP later to finish fellowship Changed to physician training Unknown

VPP 138 Non-payment of resubs and/or Not continuing with FACRRM Advised by RTP – no reason given Failing assessment Want to finish FRACGP first. May come back to ACRRM later via IP Post not accredited for ACRRM training Decided FRACGP or FARGP would suit better / already finished and don’t see point of both Personal circumstances /commitments Changed to different discipline training Mandatory rotations / logbook requirements Couldn’t complete training in preferred location Withdrew and joined IP / RVTS ACRRM too restrictive Difficulty combining ACRRM with ADF Moved to Tasmania – FACRRM too difficult to undertake there Lack of knowledge and support from RTP Living overseas Lack of clinical work – may re-join later Only joined to get access to ACRRM resources Transferred to general / urban pathway

Total 221


g. Append the document(s) provided to candidates that explain/s the assessment policy, the nature of the assessments and the standards of performance required.

See Attachment 5.1: Fellowship Assessment Handbook 2014

h. Any other planned developments that relate to Accreditation Standards 5.1 and 5.3

The Assessments for ASTs will be reviewed as part of the AST curricula review, however there is not expected to be any significant changes.

Assessment preparation programs including Assessment Information Sessions, Study Groups and Mock StAMPS will continue to be provided.

(See Attachment 5.6: Assessment and Assessment Preparation Programs dates for 2015 ).



Other matters to be addressed (relating to 5.4)

a. The College’s response to the nationally agreed policy for the assessment of international medical specialists, including actions to implement AMC recommendations and/or alternative approaches agreed by the College.

The recent changes to the Specialist Pathway nationally agreed policy for IMG specialists, precipitated a review of ACRRM’s Specialist Pathway. This review has contributed to the refinement and enhanced of operational processes that ensure ACRRM’s capacity to provide a robust program that conforms to, and complies with the national policies and guidelines. As ACRRM’s accreditation for its Specialist Pathway is reasonably new, the review showed that our current accredited policy and procedures for the pathway complied with the national requirements. What the review identified was the need to change the College’s administrative and operational aspects of the program to ensure ACRRM was able to take on its new roles and functions. ACRRM has been active in the AMC/MBA convened workshops regarding changes and has contributed to the development of multi-college materials to ensure consistent standards across colleges in the administration of the pathway (e.g. application forms, PD and CV templates etc).

b. Report on the number of applications considered from overseas-trained specialists and the outcomes of their applications.

Since the commencement of the pathway, ACRRM has received 97 applications from OTGPs to enter the pathway. Of this number 21 applications did not proceed. Of those assessed 24 were found to be partially comparable, 31 substantially comparable and 14 not comparable. The remaining seven applications are currently in the process of being assessed.

c. An outline of plans for further development.

Given the changes to the pathway as reported above, ACRRM’s IMG Assessment Committee will monitor the new processes which in turn will inform further development of the program policies, protocols and operational processes as and if required. The area of further development which is currently being planned is the result of the recent consultation conducted by the MBA regarding short term training in a medical specialty for IMG’s who are not qualified for general or specialist registration. The consultation paper included draft guidelines, these guidelines were considered by ACRRM and the College has agreed to expand its Specialist Pathway to include a specialist in training component. ACRRM is developing a policy and operational framework for the introduction of this pathway which will comply with the standards set out by the MBA. The College has been closely examining trends in registrar assessment performance across the range of assessment modalities and the Assessment Committee will continue to consider its options in terms of the continued application of each modality.


6. Monitoring and Evaluation

6.1 Ongoing monitoring

6.1 1 The education provider regularly evaluates and reviews its training programs. Its processes address curriculum content, quality of teaching and supervision, assessment and trainee progress.

The College continuously reviews and evaluates all aspects of its operations. It embraces innovation and creative problem-solving in pursuing its mission to serve the communities of rural and remote Australia and their changing needs and circumstances. As a necessary extension of this approach, it is a general policy of the College, that every new aspect of its programs is evaluated as part of its development and implementation, and then continuously reviewed. Since 2012 monitoring and established evaluation activities in the College have been guided by an overall Evaluation Plan and directed by a dedicated coordinator with qualifications in Education, Psychology, and Applied Social Research appointed in 2011. The Evaluation Report is itself regularly reviewed to improve and meet the changing environment and a revised ACRRM Evaluation Plan 2012-14 has been produced. This plan entails a continuous circuit of evaluation, processing of feedback; guiding program improvement; evaluation plan review and adjustment; and, re-evaluation. This evaluation cycle is detailed in Figure 6.1 below. The College reviews every aspect of its programs and services at the operational level. All program areas are subject to a review cycle. These cycles notwithstanding, the College also has in place processes to enable it to respond to emerging issues drawn to its attention through monitoring and evaluation feedback or from the wider environment whenever necessary.

Each individual activity undertaken by the College is continuously reviewed.

The various components of each area are monitored and evaluated. Evaluation results are regularly combined to deliver integrated program reports. These reports pool information from relevant stakeholder groups such as the Registrar Survey and the Consumer Feedback Survey; relevant additional statistical information; and specific activity reports as appropriate. (For example, the Vocational Training and Assessment Evaluation for 2013-14).

A regular review cycle is in place for all College curricula. The College also has governance arrangements in place to enable it to respond and adjust as appropriate to issues as they arise. In this sense curriculum development is constantly guided by information arising from the breadth of evaluation reports, feedback and emergent statistics.

Teaching Standards and Training Policies are also subject to a regular review cycle. As with the curriculum these reviews are also constantly informed by the information arising from the overall evaluation program.

Overall College Evaluation Reports are routinely produced to provide an integrated view of the College performance and trends. Evaluation databases for the preceding two years are drawn upon and collated in response to the core evaluation questions contained in the Evaluation Plan (i.e. the bigger picture questions).

(See attachment 6.1: Revised College Evaluation Plan (2012-14))


(For an example of an integrated program report; see Attachment 6.2: Vocational Training and

Assessment Evaluation 2013-14.)

(See Attachment 6.3 for the ACRRM Evaluation Report 2012-2013) Figure 6.1: College Evaluation Cycle

Evaluation Framework directed to Program Improvement The ultimate aim of the College Evaluation Framework is to lead continuous program improvement. Each evaluation report contains a recommendations section derived from the evaluation results. Each report, including the recommendations, is firstly discussed with the relevant program manager/teams. The program manager will then present the report’s findings to the SMG and relevant committees. It is the role of the project manager/team in conjunction with the relevant committees to then take actions to respond to the recommendations outlined. Where feasible, evaluation results are also communicated to participants (via e-newsletters), to close the evaluation loop.


Figure 6.2: Evaluation Process relationship to program improvement

Curriculum Content In accordance with the review cycle an extensive review of the Primary Curriculum involving all stakeholders concluded in 2013. The Primary Curriculum is available on the website as html file and PDF. A summary of the changes is also provided. The review process involved a comprehensive program of stakeholder engagement. Consultation incorporates an iterative cycle of feedback with:

The wider College membership

Teams drawn from the membership particularly incorporating members identified from databases as having relevant qualifications

Supervisors including representatives of RTPs

Relevant community groups

Related medical colleges

Representatives of health and other related services As part of the review program planning is now underway for the review of eight AST Curricula which are operated independently by the College. (The curriculum for the 11th AST, Academic Practice is still being developed.) Review for the ASTs in Obstetrics, Surgery and Anaesthesia will be undertaken collaboratively with input from their respective JCCs. The College Curriculum is viewed as a living document and appropriately must respond to needs as they arise. These may be as demonstrated by program delivery outcomes revealed through the program evaluation process and also in response to external events. College Governance arrangements are in place to ensure that the College is able to respond to issues as and when they become apparent. All program evaluation materials feed into this process; specifically activity reports and feedback from stakeholder groups via the Registrar Survey, Supervisor Survey and Community Feedback Survey. (See Attachment 3.1: ACRRM Primary Curriculum.)



Teaching Standards Cycles of Review also are in place for all College documentation related to Teaching Standards. In accordance with this schedule, during the review period the following activity has occurred around development and review of standards for supervisors and teaching posts:

2013: Revision of standards for PRRT

2013: Development of Guidance for off site supervision 2011: Development of standards for CCT (This allows accreditation of suitable PGY2 posts or

disciplines that are not currently accredited by the Postgraduate Medical Councils) 2011: Development of standards for supervisors and teaching posts for AST posts in the

following disciplines: Emergency Medicine, Aboriginal and Torres Strait Islander Health, Adult Internal Medicine, Mental Health, Population Health, Remote Medicine and Surgery and Paediatrics.

Assessment and Trainee Performance

As detailed previously, the Assessment Blueprint contained in the ACRRM Primary Curriculum ensures alignment between what is taught, learnt and assessed. Accordingly the blueprint is evaluated as part of the overall curriculum review. Following each assessment registrars, examiners, invigilators, writers and editors are given the opportunity to provide anonymous feedback through an online survey. The ACRRM processes for ongoing evaluation of assessment modalities includes consideration of the educational impact of the assessment on registrar learning. Assessment processes have been developed and improved using continuous quality improvement principles. The process of producing exam items involves a group of doctors who have developed expertise in producing MCQ and/or StAMPS items. After each assessment the writers examine how the items perform in order to improve the individual item if required and more generally improve skills in developing items. All StAMPS scenarios are road tested prior to the assessment by a registrar who has completed this assessment. In addition to the standardised program of assessment the College from time to time, draws on the resources of its Evaluation Office, to undertake specific research into particular areas as required.

6.1.2 Supervisors and trainers contribute to monitoring and to program development. Their feedback is systematically sought, analysed and used as part of the monitoring process.

There is a range of tools and mechanisms available to supervisors and trainers involved in ACRRM programs to enable and encourage them to provide informed comment to the College; and by which the college collates and analyses this toward organisational and program improvement.

A Supervisor Survey is undertaken annually and its outcomes processed through the College administration and committees structure. The survey includes questions to address the core questions which provide ‘bigger picture’ information about the way the programs and their outcomes are perceived.

As outlined above, following provision of each assessment registrars, and assessment personnel may provide anonymous feedback through an online survey.


The Curriculum Review process involved a comprehensive consultation process; including interactions with supervisors and training providers. Representatives from each RTP were invited to comment on the revised format and also to assist in the content revision for the individual curriculum statements.

As part of the new Bi-College Accreditation arrangements, RTPs now submit an Annual Report to the College detailing their supervisor and teaching post accreditation activities and experiences against established standards.

Each newly accredited IP Supervisor now receives a direct initial consultation with ACRRM staff and a follow up consultation at the submission of each successive Registrar Report.

Regular attendance at the RTP CEOs and Director of Training meetings and also the annual GPET convention has provided opportunities to engage and gain additional verbal feedback and comment from these organisations.

As a general policy, the College has redoubled its efforts in recent years to offer as many opportunities for direct interface with RTP representatives as possible.

(For full detail on these: See Section 8.3 Condition 23).

6.1.3 Trainees contribute to monitoring and program development. Their confidential feedback on the quality of supervision, training and clinical experience is systematically sought, analysed and used in the monitoring process. Trainee feedback is specifically sought on proposed changes to the training program to ensure that existing trainees are not unfairly disadvantaged by such changes.

The Registrar Committee and registrar representatives on the Vocational Training and the Assessment Committees assist ACRRM to obtain registrar feedback. College staff communicate with registrars in a range of ways also providing opportunities to gain feedback. There is a dedicated email address for the registrar chair; [email protected]. A Facebook page was established in May 2014. (For more information on College communication with Registrars; See Section 7.1). AGPT and RVTS undertake annual surveys of registrars training on their pathways. ACRRM is able to view the AGPT survey report but to date has not had access to RVTS survey reports. (NB It is unknown whether these opportunities will continue under the new arrangements for the AGPT system). ACRRM undertakes an annual registrar survey plus an exit survey when registrars have completed training. ACRRM has a feedback facility on the website; it includes the option for registrars to provide anonymous feedback. Registrars have been involved in the review of Primary Curriculum at all stages. This was achieved through the involvement of the Registrar Committee, and encouragement and invitations through FACRRM Fundamentals and Countrywatch newsletters. Registrars and Fellows identified on the College’s membership database as having a special interest, skills or training in a particular discipline were invited to provide feedback on the relevant statements. This method was the most effective in providing feedback.

mailto:registrarchair%40acrrm.org.au


The Registrar Committee has also been asked to provide specific advice to the College about communication regarding the changes to the curriculum once the revision has been completed. The review of the PRRT Policy to specify a minimum amount of time in specific training environments involved consultation with all stakeholders including registrars, supervisors and training providers plus the broader ACRRM membership. The College will as a matter of course engage Registrars in policy developments of importance to them. For example, in 2013 the College was commissioned by the Department of Heath to develop a report into a National Rural Generalist Framework. This led to further refinement of College policies in this area. The Report process involved discussion and an approval process with the Registrar Committee Chair as well a dedicated survey process to all Registrars to gauge their perspectives on the relevant issues.


6.2 Outcomes evaluation

6.2.1 The education provider maintains records on the outputs of its training program, is developing methods of measuring outcomes of training and is collecting qualitative

information on outcomes.

As outlined above the ACRRM Evaluation Plan was established in 2012 and was revised and extended in 2013 to its current form. The Evaluation Process commenced with the establishment of a baseline bank of data which can provide a benchmark for each incremental evaluation cycle. The available databank from which progress can be measured builds over time and enables more nuanced analysis. At the same time the evaluation tools applied are continuously assessed for relevance and new tools introduced as appropriate. The College Evaluation Framework in its entirety is based on nine core questions designed to garner information on how effectively program outcomes are addressing the College’s central mission these are: 1. How is ACRRM contributing to the healthcare needs of rural and remote Australians and

specifically in relation to its approach to the specialty of General Practice? 2. What are the professional characteristics (KSA) of an ACRRM registrar and what do registrars do

differently a as a result of ACRRM training? 3. What is the skill mix (KSA) of an ACRRM fellow to improve and sustain the healthcare of rural and

remote Australian Communities? 4. How does a FACRRM qualification in rural and remote areas benefit the Australian health system

in terms of cost, efficiency and time? 5. What is the career path of a rural doctor? 6. Is ACRRM positioned as an authority/leader in rural and remote health national and

internationally? 7. How is ACRRM leading the way in innovative medical education? 8. How efficiently are staff and resources being used? 9. How can ACRRM programs and services be improved? As previously outlined integrated reports such as the Program Reports and the Overall College Report are specifically collated from the perspective of assessing how the data and feedback selectively measure against these core questions. Data Collection toward measuring Graduate Outcomes The RRMEO online platform hosts databases with considerable information to inform evaluation of both outputs and outcomes. They include:

Enrolments and participant personal data

Award of Fellowships (including attainment of advanced skills and clinical privileges) Program assessments and assessment outcomes

Fellow rurality including longitudinal tracking of rural retention

IMG enrolments; interviews, assessments and determinations

PDP participation and enrolments in RRMEO modules

Number and scope of accredited PDP activities Participation in academic research (i.e. lists of Fellows’ requests to participate in programs)

FACRRM’s accredited as supervisors


Other Useful Information Sources for Outcomes measurement Some other evaluation tools incorporated into the Evaluation Plan which provide important information about program outputs are:

Measurement of number of research collaborations

Membership on external committees Mutual recognition agreements (both internal and international)

RMA Conference and World Summit (outcomes and participation rates) Reviews of academic literature and rural medical workforce databases

Comparison with other College programs with overlapping program areas Evaluation Surveys and Reports The evaluation process draws on the data and processed feedback arising from the 45 survey instruments currently in place which provide qualitative information regarding the way programs and their outcomes are perceived by stakeholders. (See Table 6.1 below.) Table 6.1: Currently operational evaluation tools Evaluation Tool

Frequency of distribution

Vocational Training & Assessment

ACRRM Registrar survey Annually

ACRRM Supervisor survey Annually

Vocational Training exit survey At completion of training

ACRRM Registrar Withdrawal survey As required

Community Feedback and Needs Analysis survey Annually (and permanently available via web)

Reliability analysis of ACRRM assessments Quarterly or as required

Virtual Classroom surveys After each session

Workshop surveys After each workshop

Study groups After study group complete

Mock StAMPS exam After Mock exam

VT Assessment Evaluation surveys After each assessment IP Assessment tools evaluation As required

IP Selection Process evaluation After IP Selection Interviews

GPET Registrar survey results* Annually by GPET

RTP accreditation information Every 3 years and monitoring of unmet outcomes.

PDP PDP Member survey Biannually

PDP Member Records Audit Annually

Education Provider survey** Biannually

ACRRM Events Feedback surveys (ALS, REST, Ultrasound, Mental Health)

Annually

PDP Evaluation Forms Annually

RMA Feedback Survey and Workshop Evaluation Forms Annually IMG TBC

IMG Recruiter survey TBC

PESCI Feedback survey (candidates and panellists) TBC

Specialist Pathway survey (candidates and assessors) TBC

Reliability analysis of ACRRM assessments TBC

Online Services

Tele-Derm User survey Annually

PDA Clinical Guidelines survey Biannually

RRMEO Module Evaluation surveys Ongoing

Virtual Classroom survey Annually

Online Services Client survey TBC


e-health Participant survey TBC

Business Services, membership and marketing

ACRRM Member survey Annually (first two six-monthly)

Business Services Client survey TBC

Staff Survey Annually

RMA Trade evaluation Annually

Bonded Scholarship Program

BSP Annual Evaluation survey Annually

BSP Conference Evaluation Survey 6 monthly

BSP Virtual Classroom Survey 6 monthly BSP Career Options Presentation and Luncheon survey 6 monthly

BSP Networking Nights survey 6 monthly

JFPP

JFPP Student Evaluation survey Continual

JFPP Mentor Evaluation survey Continual

JFPP Community Contact evaluation Continual

JFPP Longitudinal Tracking survey Annually

JFPP Exit survey Continued

* To be confirmed whether this will be continuing under new arrangements. ** To be confirmed what form these will take under new arrangements

Measuring Outputs

ACRRM maintains records of the inputs and outputs of each of the training pathways and assessment modalities. The information is analysed to enable trend data to be recorded and monitored.

Outputs of training are represented principally by the number of registrars achieving Fellowship (see Table 2.1) and the assessment outputs are provided (see Tables 5.3 to 5.9).

6.2.2 Supervisors, trainees, health care administrators, other health care professionals and consumers contribute to evaluation processes.

Supervisors’ Contributions to Evaluation Process

These are outlined at Section 6.1.2 above.

Registrar Contributions to Evaluation Process

These are outlined at Section 6.1.3 above.

Healthcare Administrators, Professionals and Consumers Contribution to Evaluation Processes As a general approach, wider stakeholder groups are involved with all major reviews of College Activity. The Curriculum Review process involves seeking input from a wide range of stakeholders including community groups (See Section 3). Teaching Standards policy development process also incorporated engagement with relevant stakeholders particularly health administrators. Consumer Feedback and Needs Analysis Survey The ACRRM Evaluation Plan has been extended to incorporate a Consumer Feedback and Needs Analysis Survey which will continue to its continuous improvement. The survey seeks comment on the College Curricula (Primary and AST) as they pertain broadly to graduate and medical workforce outcomes and the overall performance of the College in terms of meeting community needs.


The survey is specifically distributed broadly to key rural organisations (with a role in allied health provision or as representatives of health consumers) who are invited to contribute feedback of the organisation as a whole and their individual members. The first round of invitations was sent out in June 2014. Additionally the website incorporates a link which is available to any interested persons. Respondents are required to provide details of their role (be it as consumer or provider) in the healthcare system to inform analysis of feedback received. Feedback is to be specifically sought and processed on an annual basis but relevant stakeholders have the opportunity to submit comment or participate in surveys at all times via a specified link on the College website.

(See Attachment 2.5: List of Consumer Groups specifically requested to give feedback to Consumer Feedback and Needs Analysis Survey).

(See Attachment 2.6: Consumer Feedback and Needs Analysis Survey (Questions))

(See Link to the Consumer Feedback and Needs Analysis Survey on the College Website)

https://www.acrrm.org.au/forms/feedback-on-vocational-training-and-assessment


6.3 AMC Conditions

Condition 15: That the College implement a systematic framework for program evaluation, with a particular focus on the use of evaluation information for program improvement. (Standard 6.1)


Condition 18: Implement processes for regularly obtaining comment on the curriculum from consumers, supervisors and non-medical health professional and involving them in more formal program review. (Standard 6.2)




a. Give details of evaluation activities undertaken since the 2010 Accreditation assessment.

Table 6.2 Program Evaluation Activities since 2010

Program Evaluations Frequency of reporting

Operations progress

Intended audience

John Flynn Placement Program

JFPP Student Placement Questionnaire

Six-monthly Operational from outset of programs and ongoing

JFPP Manager/team; DoH; SMG; marketing

JFPP Mentor Placement Questionnaire

Six-monthly

Operational from outset of programs and ongoing


JFPP Community Contact Placement Questionnaire

Six-monthly

Operational from outset of programs and ongoing


JFPP Participation Statistics Six-monthly

Operational from outset of programs

and ongoing


JFPP Exit Survey Annually Undertaken 2012,

2013, 2014 and ongoing

JFPP Manager/team; DoH;

SMG; marketing

JFPP Longitudinal Tracking Project

Annually Operational since 2005 and ongoing


Bonded Support Program

BSP Conference Evaluation Six-monthly Commenced at outset of program in 2010 and ongoing

JFPP Manager/team; DoH; SMG

BSP Networking Evaluation Six-monthly Commenced at outset of program in 2010 and ongoing


BSP Career Options Evaluation

Six-monthly Commenced at outset of program in 2010 and ongoing


BSP Virtual Classroom Evaluation

Six-monthly Commenced at outset of program in 2010

and ongoing


BSP Annual Survey Annually Undertaken 2011,

2012, 2013 and ongoing

JFPP Manager/team; DoH;

SMG; BSP participants

Vocational Training & Assessment

Registrar Feedback Survey

Annually

Undertaken 2012, 2013, 2014 and ongoing

VT Manager/team; SMG; VT Committee; Registrar Committee; survey participants; AMC; marketing

Supervisor Feedback Survey Annually

Undertaken 2012, 2013, 2014 and ongoing

VT Manager/team; SMG; VT Committee; survey participants; AMC; marketing

Vocational Training Exit Survey

Annually Undertaken 2012, 2013, 2014 and

ongoing

VT Manager/team; SMG; AMC; VT Committee

Registrar Withdrawal Survey Annually

Undertaken 2010, 2011, 2012, 2013, 2014 and ongoing. (Program improvements introduced in 2012)

VT Manager/team; SMG; AMC

Community Feedback & Needs Assessment Survey

Annually

Undertaken 2014 and ongoing



Reliability analysis for StAMPS, MCQ, MiniCEX

After each assessment

Commenced at introduction of each modality and ongoing


Vocational Training and Assessment Statistics

Annually Operational from outset of programs and ongoing

VT Manager/team; VT Committee; AMC

Fellowship Services

PDP Member Survey Biannually

Undertaken 2010, 2012 and ongoing

FS Manager/team; SMG; PD Committee; survey participants; AMC; marketing

Education Provider Survey Biannually Undertaken 2013 and

ongoing

FS Manager/team; SMG;

PD Committee; survey participants

PDP Records Audit Tri-annually Undertaken 2012, 2014 and ongoing (Program improvement introduced in 2014 – scientific sampling)

FS Manager/team; SMG; PD Committee; AMC

PDP Participant Statistics Annually Operational from outset of program and ongoing

FS Manager/team; SMG; PD Committee; AMC

ALS, REST, and Emergency Course Evaluations

Annually Operational from outset of program and

ongoing. Process improvements introduced in 2014

FS Manager/team; SMG; Course Facilitator

Mental Health Course Evaluations

Annually Operational from outset of program in 2008/9 and ongoing. Process improvements introduced in 2014.

FS Manager/team; SMG; PD Committee

RPGP & Rural LEAP Progress/Final Reports

Six-monthly Operational from outset of program and ongoing

FS Manager/team; DoH; SMG; PM Committee

RPGP & Rural LEAP Participant Survey

Tri-annually Operational from outset of program and

ongoing

FS Manager/team; DoH; SMG; PM Committee

International

Medical Graduate

PESCI Panelist Feedback

Survey

Annually Undertaken 2013 and

ongoing

IMG Manager/team; SMG;

survey participants; marketing

IMG Recruiter Feedback Survey

Annually Undertaken 2013 and ongoing

IMG Manager/team; SMG; survey participants

PESCI Candidate Feedback Survey (pending)

TBC Planned for 2014 and ongoing

IMG Manager/team; SMG; PESCI panellists

IMG Program/Participant statistics

Annually Operational from outset of program and ongoing. Expanded reporting database under-development

IMG Manager/team; SMG

Online Services Tele-Derm User Survey Biannually Undertaken 2012 and ongoing.

OS Manager/team; SMG; survey participants; marketing

Clinical Guidelines PDA Survey

Biannually 2011 and 2013 and ongoing.

OS Manager/team; SMG; survey participants;

marketing

Virtual Classroom Survey (Overall program)


OS Manager/team; SMG

Online Education Modules Evaluation

Annually Undertaken 2012, 2013, 2014

OS Manager/team; SMG; marketing

Research Week Evaluation Annually Undertaken 2014 OS Manager/team; SMG; Louise Stone

Online Services Statistics Annually Operational from services’

OS Manager/team; SMG; AMC


commencement and ongoing

eHealth eHealth data reports Monthly Undertaken since 2011 and ongoing

eHealth Manager/team; SMG; marketing

eHealth education module reports

Monthly Undertaken since 2013 and ongoing

eHealth Manager/team; partner organisations; funding bodies

Telehealth provider/mentor surveys

To be determined

Under-development eHealth Manager/team; SMG; partner organisations; funding bodies

Membership ACRRM Member Survey Annually Undertaken 2012, 2013 and ongoing

SMG; ACRRM Board; ACRRM members; marketing & membership

Re-subscription Data Annually Undertaken 2014 and ongoing

SMG; ACRRM Board; marketing

Resigned Member Survey Annually Planned for 2014 and ongoing

Membership; SMG; Board

Corporate services

Staff Survey Annually

Ongoing SMG; staff

RMA conference

Workshop evaluations


Events team; workshop presenters; marketing

Trade evaluation

Annually

Undertaken 2013 and ongoing

Events team; marketing

RMA online survey Annually Undertaken 2013, and

planned for 2014

Events team; marketing

Table 6.3 Program Review activities since 2010

Program Progress Process Important Changes Primary Curriculum 2013- Full review completed Reviewed internally with

extensive member and external stakeholder engagement.

Information updated and restructured.

AST:

Emergency Medicine To be reviewed internally with broad consultation.

Aboriginal and Torres Strait Islander Health

To be reviewed internally with broad consultation.

Adult internal medicine

To be reviewed internally with broad consultation.

Mental health To be reviewed internally with broad consultation.

Population health To be reviewed internally with broad consultation.

Remote Medicine To be reviewed internally with broad consultation.

Surgery To be reviewed internally with broad consultation.

Paediatrics To be reviewed internally with broad consultation.

Obstetrics and

gynaecology

Reviewed with input from

Conjoint Committee for the Diploma of O & G.

Anaesthetics To be reviewed with input from JCC Anaesthetics

Rural generalist surgery

To be reviewed with input from JCC General Practice Procedural Surgery.

Academic Practice 2013: Initial Draft Curriculum established.

Draft distributed for consultation and adjustment to concord with regulatory frameworks.

New AST.


Table 6.4 Policy Review activities since 2010

ACCRM Policies Last

review

Next Review

Date

Advanced Specialised Training Policy Dec-12 Dec-15

Appeals policy Jun-10 Jun-13

Completion of Training Policy Dec-13 Dec-16

Core Clinical Training Policy Dec-12 Dec-15

Emergency Medicine Course Requirements Policy Dec-13 Dec-16

Independent Pathway Active Training Policy Jul-13 Jul-16

Independent Pathway Remediation Policy Jun-13 Jun-16

Leave Policy Jun-13 Jun-16

Overseas Training Policy Jun-13 Jun-16

Primary Rural and Remote Training Policy Dec-12 Dec-15

Recognition of Prior Learning Policy 2009 Jun-13 Jun-16

Registrar Review Policy Jun-12 Jun-15

RRMEO Module Completion Policy Dec-11 Dec-14

Special Considerations Policy Jun-13 Jun-16

Training Time Policy Jun-13 Jun-16

Vocational Training Program Enrolment and Participation Policy May-12 May-15

Grievance Policy Sep-12 Sep-15

Table 6.5: Standards Review activities since 2010

ACRRM Standards Published

Review Date

Supervisors and Teaching Posts

Core Clinical Training 2011 2014

Primary Rural and Remote Training 2013 2016

Advanced Specialised Training

Aboriginal and Torres Strait Islander Health 2011 2014

Adult Internal Medicine 2011 2014

Emergency Medicine 2011 2014

Mental Health 2011 2014

Paediatrics 2011 2014

Population Health 2011 2014

Remote Medicine 2011 2014

Generalist Emergency Medicine 2011 2014

Post Fellowship

Generalist Emergency Medicine 2011 2014

Training Providers

Standards for Regional training Provider Recognition 2007 2013


b. Please report in 2014 on how the college is responding to feedback reported in recent registrar surveys (as detailed in AMC response to 2013 ACRRM progress report).

Please provide a summary of evaluations undertaken, the main issues arising from trainee evaluations and supervisor evaluations and the college’s response to them.

The 2013 Registrar Survey outlined a series of recommendations to address issues identified through the survey tool. Below is listed each recommendation and the Colleges subsequent response.

1. Devise strategies to increase the percentage of affirmative responses for all statements, especially for any questions with significantly lower satisfaction ratings. College Action: As this is a generic reference, related actions are generally listed below.

2. Review the working of training and assessment guidelines (including web content) to present requirements in clear, simple language with little use of jargon and confusing acronyms. College Action: A process was undertaken to simplify language in relevant documents. Additionally as part of the CRM process the College website is currently being completely reworded to ensure consistency and simplicity of the language used.

3. Establish closer working relationships with RTPs to increase awareness of FACRRM training and assessment requirements. College Action: The College views this as a priority issue and major efforts have been invested in making improvements in this area. - Considerable effort has been invested in improving the quality and the frequency of

communications with supervisors and RTP administration. Regular meetings have been established with five training providers; this has been very positive and will be rolled out to include all training providers. The College CEO meets regularly with the RTP CEOs and the Director of Training meets with RTP Directors of Training.

(For further details: See Section 8.3 Condition 23).

- Additional training workshops have been introduced and an instructional program for supervisors on applying the ACRRM program is currently under development.

- It is hoped that the Bi-College Accreditation processes and the attendant policies that have been put into place to link compliance with College procedures and curriculum to maintenance of accreditation; will over time have a positive influence upon compliance. (At the time of survey, these were only in their infancy).

- Furthermore the College is engaging in discussions with the Commonwealth to ensure that the new arrangements act to improve the workability of the system.

4. Increase Registrar support for assessment preparation, including provision of: clearer

information about the assessment process and requirements; and study materials (e.g. practice questions) and online sessions/workshops. College Response: Assessment information sessions, study groups and mock exams have been established. All assessment information has been reviewed.


5. Review the current process for provision of post-assessment feedback to ensure that

registrars receive clear advice regarding areas of improvement. College Response: In addition to the written feedback provided to all registrars. Those that are unsuccessful in assessment are offered a feedback session with an examiner.

6. Invite ACRRM Fellows (particularly those who have undergone the vocational training and assessment program) to act as mentors and/or medical educators for current registrars. College Response: The College has invested considerable efforts in encouraging members to participate in medical education and support roles. These efforts have been fruitful and 73 members have been recruited and are actively contributing to the vocational training and assessment programs.

7. Develop a strategy to increase registrar involvement with the College/medical educators/ supervisors and with each other (e.g. promote the role of the ACRRM Registrar Committee and encourage registrars to offer regular feedback to the College; help registrars to offer regular feedback to the College; help registrars to establish online study and exam preparation groups). College Response: The College has endeavoured to foster peer networking through such mechanisms as Facebook groups; increasing the frequency and detail in the FACRRM Fundamentals (Registrars’) e-Newsletter; online study groups; using virtual classrooms as a networking tool.

8. Publish a statement of survey findings in the FACRRM Fundamentals e-newsletter and communicate to registrars any plans for program improvement based on survey feedback. College Response: Survey results have been published in FACRRM Fundamentals. This is an area in which the College will be endeavouring to provide more frequent communications.

9. Include in the FACRRM Fundamentals e-newsletter ongoing updates regarding progress of program improvement strategies. College Response: Information is provided regularly through e-newsletters and direct email. FACRRM Fundamentals has increased in frequency from every three months to every two months. The website identifies when a document has changed and a summary of significant changes is provided. As above this is an area in which the College will be endeavouring to extend its efforts.

10. Re-administer the ACRRM Registrar Survey in April or May 2014 to allow for comparison of finding between previous years. Consider offering an incentive to encourage participating in the survey.

College Response: Surveys are conducted annually. Plans for the next Registrar Survey are underway.


7. Trainees

7.1 Admission Policy and Selection

7.1.1 A clear statement of principles underpins the selection process, including the principle of merit-based selection.

7.1.2 The process of selection into the training program:

are based on the published criteria and the principles of the training organisation concerned are evaluated with respect to validity, reliability and feasibility

are transparent, rigorous and fair

are capable of standing up to external scrutiny include a formal process for review of decisions in relation to selection, and information on the

process is outlined to registrars prior to the selection process.

7.1.3 The education provider documents and publishes its selection criteria. Its recommended weighting for various elements of the selection process, including previous experience in the discipline, is described. The marking system for the elements of the process is also described.

7.1.4 The education provider publishes the requirements for mandatory experience, such as periods of rural training, and/or for rotation through a range of training sites. The criteria and process for seeking exemption from such requirements are made clear.

7.1.5 The education provider monitors the consistent application of selection policies across training sites and/or regions.

The guiding principles for all trainee selection activities are defined by the College Selection Framework. The three training pathways to FACRRM have their own documented eligibility criteria and selection processes. The AGPT and RVTS pathways are delivered through different organisations and the delivery organisation holds primary responsibility for the selection process. ACRRM conducts selection for the Independent Pathway. All three pathways have a selection process that is merit based and nationally consistent. ACRRM contributes to the development and review of the AGPT and RVTS policies and processes. The selection criteria are made available through the websites of the respective organisations; the ACRRM website provides a link from the vocational training page to the AGPT and RVTS websites. Selection to the AGPT has presented considerable challenges for the College in terms of ensuring compliance with their Guiding Principles. Major steps forward have been made in this regard through the RTP Bi-College Accreditation process. It is unclear at this stage how the new arrangements will affect the Colleges’ future selection process. This is a subject for continuing talks between the College


and the Commonwealth. Of particular concern to the College in this dialogue is ensuring the College’s ability to certify compliance under future arrangements specifically, ensuring selection processes are concordant with the College Selection framework and as such optimally contribute to its ultimate graduate outcomes. Vocational Preparation Pathway

The VPP selection criteria and processes are set by the GPET National Selection Committee; ACRRM has representation on this committee. Selection into AGPT is based on a nationally consistent process involving checking eligibility, national assessment and regional training provider selection and placement offers. The AGPT selection criteria are comparable to the ACRRM criteria, they can be found in the AGPT Applicant Guide. In 2015 there will be 1500 places AGPT places available.

(See Attachment 7.1: AGPT Application Guide (Selection Criteria)).

Remote Vocational Training Scheme

The RVTS pathway selection process is run by the RVTS organisation. ACRRM is involved in setting and implementing selection policy through consultation with the organisation, ACRRM membership on the Board of the RVTS, and participation by Fellows on selection interviews. The RVTS eligibility and selection criteria and processes can be found in the RVTS Applicant Guide. The RVTS pathway has a national quota of 22 registrars in any one calendar year for training toward FACRRM and FRACGP. In 2013 RVTS added an additional 10 places to train doctors in ACCHS.

(See Attachment 7.2: RVTS Application Guide.)

Independent Pathway

ACRRM delivers the IP and admission to this pathway is administered directly through ACRRM. The Independent Pathway How to Apply Guide is available on the ACRRM website and can be posted on request. This document provides guidance on the application and selection process. This pathway is a full fee-paying pathway, it is not subsidised or funded by the Australian Government. ACRRM sets the maximum number of new training places each year according to capacity to train. Over the past three years the quota has been set at around 40 per year.

(See Attachment 7.3: Independent Pathway ’How to Apply’ Guide)

(See Attachment 7.4: Independent Pathway Selection Process)

http://www.gpet.com.au/Junior-doctors/Australian-General-Practice-Training--AGPT--program/New-applicants/Application-Process

http://www.rvts.org.au/applicants/applications




7.2 Trainee participation in governance

7.2.1 The education provider has formal processes and structures that facilitate and support the involvement of trainees in the governance of their training.

Internal Governance Structures

ACRRM governance structures support the involvement of registrars through representational membership on the:

Registrar Committee;

Program Committees; and

ACRRM Board. The ACRRM Board of Directors includes a Registrar Director who is elected by registrar members. The Registrar Committee consists of at least six registrars; including a minimum of one from each training pathway and gives consideration to balance gender, geography and age. The ACRRM Registrar Committee’s role is to:

Represent the broad interests of the registrar members of the College;

Provide direction and guidance regarding the impact of College policies on registrars; and Provide direction and guidance regarding information and communication strategies directed

at the registrar population. The committee is chaired by the Registrar Director and meets a minimum of six times per year primarily via teleconference. The committee reports to the ACRRM Board via the Registrar Director following each committee meeting. To facilitate communication with registrars the Registrar Committee is accessible to stakeholders through a dedicated email address

[email protected] The Vocational Training, Assessment, Research and PDP committees also include at least one member who is a registrar in training. All Registrar Committee positions are filled by calling for applications from the College membership. The ACRRM Board reviews the applications and makes a decision about inclusion on the committees. Registrars are full members of the committees and hold the same rights as other members. The Registrar Committee nominates members to join other College committees such as for Vocational Training. ACRRM pays the cost of travel and accommodation for registrars to attend any committee or board related activities. Additional to the strategic framework the College uses the Governance framework as well as wider scoping approaches to ensure Registrar involvement in addressing important issues/projects as they arise. For example the College was engaged by the Commonwealth to provide a Report into the development of a National Rural Generalist Pathway. This exercise was extended to the development a policy position by the College on the issue. The Registrar Committee chair was engaged in the Report’s development, the Report was considered by the Registrar Committee; and all Registrars were given the opportunity to provide feedback into the report’s development through a dedicated survey.

mailto:registrarchair%40acrrm.org.au


Representation in External Organisations

ACRRM registrar representatives participate in meetings with the General Practice Registrars Association (GPRA). They also attend functions on behalf of ACRRM and act as guest speakers. All training providers accredited by ACRRM must demonstrate that they have mechanisms for involving doctors in training in the review and design of their training program (Outcome 1.5). This is monitored through the Bi-College RTP Accreditation program.

(See Attachment 8.6: Bi-College RTP Accreditation Principles and Outcomes)



7.3 Communication with trainees

7.3.1 The education provider has mechanisms to inform trainees about the activities of its decision-making committees, in addition to communication by the trainee organisation or trainee representatives.

Important information is emailed directly to training providers, supervisors and registrars. Decisions that affect all registrars are published on the ACRRM website. When standards and policies are revised or new a flag indicating this is included and if changes are significant these are summarised and made available on the website. (For an example see website advice on Primary Curriculum Changes; and, Assessment Changes for 2014). The bimonthly newsletter reiterates changes in policy. This newsletter is distributed to all training and assessment stakeholders and is available on the website. The Registrar Committee provides input into each edition of FACRRM Fundamentals. (See ACRRM website at: FACRRM Fundamentals). Two Facebook groups have been set up in May 2014; one for IP registrars and the other, all ACRRM registrars (with nearly 2800 likes). 46 IP registrars are members of both groups and a further 74 RVTS and AGPT registrars have joined the group for all registrars. ACRRM also has established Twitter forums and other social media platforms such as You Tube, for registrar interaction. ACRRM has increased direct contact with registrars over the past two years. This includes holding orientation sessions (via virtual classroom) for registrars, medical educator facilitated study groups (via virtual classroom) prior to each STAMPS assessment and also mock exams for StAMPS. Information is also communicated to the training providers to assist them to communicate to registrars, training and assessment requirements, and keep up to date with changes. In 2012 ACRRM held monthly information sessions (via virtual classroom) for training providers. There was strong attendance for the first six months then attendance decreased. Therefore another approach has been implemented. In 2013 and 2014 face-to-face workshops have been held for training providers. These workshops are held twice a year. One involves a series of workshops at the RMA conference and the second a stand alone two workshop for training providers. The last workshop was held in Adelaide in April 2014. These workshops have been well attended and feedback from participants positive. ACRRM intends to continue running two face-to-face workshops each year.

(See Attachment 7.6: RMA Workshop Programs and Presentations.)

Virtual classroom sessions continue to be held when significant changes are implemented for example the revision of the Primary Curriculum or when areas are identified that require further communication. ACRRM staff members continue to visit training providers and attend education functions around Australia to provide information on FACRRM training and answer questions. Staff members attend the GPET conference each year and are available at the trade booth for the duration. ACRRM also attends a number of other conferences and activities where there is likely to be a registrar presence. In the 2014 registrar and supervisor surveys the College asked for information on the how respondents preferred to receive information from the College. Both groups ranked direct email as the strongest preference, followed by e-newsletters, meetings and website. This information reinforces the methods that have been applied but suggests that there should be an increase in the




http://www.acrrm.org.au/facrrm-fundamentals


amount of direct email contact. Comparatively few respondents rated social media around 4% for supervisors and 8% for registrars. The next survey will consider specific types of social media (e.g. Facebook, Linkedin etc). The College also ensures that the same information is distributed using a number methods noting the different preferences.

(See Attachment 7.7: Events ACRRM Vocational Training staff members have attended since 2010)

7.3.2 The education provider provides clear and easily accessible information about the training program, costs and requirements, and any proposed changes.

Communication with registrars and other stakeholders is a challenge for the College due to the delivery environment for general practice training. The environment includes many stakeholders including AGPT, GPET, 15 accredited RTPs, RVTS, 789 training posts/supervisors and 533 registrars. In response ACRRM is refining and improving its communication strategy to try to overcome this complexity. ACRRM has reviewed its website and has commenced a re-development project (CRM) which will provide greater levels of interactivity and two-way communication facilities. ACRRM uses a range of methods to communicate with registrars including through:

Direct e-mail

Making program information available on the website and in printed formats;

E-newsletter FACRRM Fundamentals is distributed every two months;

providing a dedicated email: [email protected] and free-call number: 1800 223 226;

Registrar involvement in College governance structures;

Regular contact with the RTPs;

Workshops throughout the ‘Rural Medicine Australia’ conference and a standalone workshop for training providers;

Other events such as careers events;

ACRRM registrars participate in the GPRA; and

Handbooks are being developed for IP registrars, AGPT and RVTS registrars and Training providers. The handbooks will bring together existing information into one document for easy reference.

ACRRM undertakes a wide range of activities to promote general practice training. This includes running programs at undergraduate and prevocational levels (See Section 3.5), distribution of marketing materials, attendance at careers events, conferences and rural health club meetings, engagement with the General Practice Students Network and speaking at their events. ACRRM receives many ad hoc requests for information and advice which are responded to by phone and email.

7.3.3 The education provider provides timely and correct information to trainees about their training status to facilitate their progress through the training requirements.

Training Providers are responsible for providing information on training status for registrars training with them. This is an area that is reviewed as part of Training Provider Accreditation. (See Attachment 8.6: Bi-College RTP Accreditation Principles and Outcomes ((Outcome 2: Records)




In addition RRRMEO provides a Learning Planner/Portfolio to manage and document learning goals and outcomes including logbook requirements, module completion and formative assessment outcomes. Through this facility registrars and their supervisors can view training progress at any time. This process is working well for IP where the Learning Plan/Portfolio is consistently used by registrars and ACRRM. However, RTPs and RVTS (and therefore VPP and RVTS registrars) use alternate systems to record their training information. In addition to this GPET has a separate and mandatory data collection and management system which it requires RTPs to use for reporting purposes. As a result ACRRM has experienced some difficulty in aligning the various learning management systems and data systems and has needed to introduce alternative means for training data transfer for those accredited providers that do not currently use RRMEO. ACRRM and GPET have a data MOU which defines what information is shared between the two organisations. ACRRM provides GPET with information on assessment enrolments and outcomes, RMMEO modules completion and award of FACRRM. The intention was for ACRRM to be able to access training data for ACRRM registrars. This process has started but has not yet delivered meaningful information about ACRRM registrar training. To date GPET gives ACRRM access to ACRRM registrar training data on enrolments and withdrawals and completion of AGPT via dashboard. It is uncertain how these will progress with the cessation of GPET. As described above registrars on the IP have access to information on their training progress via RRMEO. Additionally registrars who are selected for positions on the IP are provided with a letter advising them of their individualised learning plan which takes into account RPL. Early in the program registrars participate in learning planning sessions, these are repeated every six months during training. This is also an opportunity for training staff to confirm with the registrar that training records are accurate and current. Training providers issue a completion of training form when all the requirements of training are completed. This information is verified by ACRRM staff, the Censor and endorsed by the ACRRM Board prior to awarding Fellowship.


7.4 Resolution of training problems and disputes

7.4.1 The education provider has processes to address confidentiality problems with training supervision and requirements.

7.4.2 The education provider has clear impartial pathways for timely resolution of training – related disputes between trainees and supervisors or trainees and the organisation

Stakeholders are encouraged to make contact early to seek clarification or resolve issues. Ease of contact is facilitated through a free call number, a dedicated email and online feedback facility. The Vocational Training and Assessment Manager is responsible for ensuring all issues are managed appropriately. This includes ensuring that issues are resolved and records maintained on the individual member file. An underlying culture of continuous improvement ensures that information, and processes are updated in response to systematic issues identified. RTPs and RVTS are responsible for resolving training issues for their registrar cohort. Therefore registrars and supervisors are encouraged to contact their training provider first to resolve issues. However ACRRM encourages training provider staff, medical educators and registrars to seek clarification from ACRRM when required. The Bi- College training provider accreditation process monitors that grievance processes are in place and are effective. The ACRRM Grievance policy was developed in 2012 to provide guidance to registrars and supervisors on how to resolve training grievances. . The Grievance policy is available on the ACRRM website. (See Attachment 7.7 ACRRM Grievance Policy)

7.4.3 The education provider has reconsideration, review and appeals processes that allow trainees to seek impartial review of training-related decisions, and makes its appeals policies publicly available.

Appeals Process The ACRRM Chief Executive Officer views and records all appeals, giving the ability to identify any trends. The appeals are incorporated into the monthly reports to the Board who have a secondary opportunity to identify systemic issues. Where a GPET appeal relates to an ACRRM registrar, ACRRM is included in the appeal panel. To date there have been two appeals relating to ACRRM registrars on AGPT. Complaints Process ACRRM has established a Complaints register where staff are encouraged to log any complaints received to enable identification of areas for improvement. For example the Emergency Medicine Course policy was revised follow a complaint from a registrar. The Vocational Training team has ‘Complaints’ as a standing agenda item for their monthly meetings. The specific details of the complaint and resolution are recorded in the registrar file.



The ACRRM website invites feedback through the use of an online form. There is an option to provide their name and contact details or remain anonymous. (See ACRRM website at: ‘Feedback’)

(See Attachment 7.8: ACRRM Appeals Policy)

(See Attachment 7.1 AGPT Application Guide (Appeals Policy))

(See Attachment 7.2 RVTS Application Guide (Appeals Policy))

7.4.4 The training organisation has a process for evaluating de-identified appeals and complaints to determine if there is a systems problem.

The number of applications for appeal received is not on a large scale making it difficult to draw

strong general inferences regarding the overall program from individual cases.

For example in 2014 of the around 600 registrars enrolled, only three applications for appeal were registered (and one was subsequently withdrawn) the other two did not demonstrate sufficient grounds for the appeal to proceed. In 2013, eight applications for appeal were received and of those four demonstrated grounds and two of those four led to original decisions being overturned.

All formal appeals are registered, and the management process is tracked and recorded. These records are maintained and available for longitudinal systemic review.

https://www.acrrm.org.au/forms/feedback






a. Report on the further development and evaluation of the selection process for all pathways since the 2010 assessment and comment on the success of the changes made. For pathways not managed by ACRRM comment on ACRRMs contribution to policy development and review. If relevant append policy.

Independent Pathway Measurable improvements have occurred in recent years to registrars’ academic progress through training and assessment. There has been a significant reduction in the number of registrars identified as requiring additional support. Some key changes that have been introduced in these years include: From 2013:

Clinical questions were included in the interview process.

ACRRM processes ensured that the selection panel included doctors with experience in general practice training and assessment.

In addition to the individual criteria, the interview panel were required to give a global score for applicants. This is used to set the cut off score for those offered training positions.

From 2014:

The College increased the validity of references through obtaining a verbal reference from referees, using a standardised proforma.

Delivery of selection interview changed form teleconference to video conference. AGPT and RVTS In 2010, RTPs undertook a selection locally, a range of tools and processes were used. Some RTPs piloted a selection process that involved a Situational Judgement Test and Multiple Mini Interviews. Evaluation showed this to be valid, fair method for selection. In 2011 GPET rolled this process out nationally. Since 2011 there has been a nationally consistent process for selection to AGPT. GPET has commissioned research to look at the effectiveness of the selection process. The research involves comparing selection outcomes and compares these with Fellowship assessment outcomes. This research is in the final stages of analysis. In 2013 GPET commissioned a General Practitioner job analysis project to evaluate the validity of the selection criteria. This project is in the final stages of analysis. ACRRM contributions to the ongoing development of the RVTS and RTP selection processes are outlined above. (See Sections 7.1.1-5).


b. Provide copies of information available to prospective applicants on :

The selection process The nationally available opportunities for entering the training programs any quotas ad other

limits , such as the number of training positions

Mandated training experiences to complete ACRRM pathways to fellowship

See Attachment 7.3: Independent Pathway ‘How to Apply’ Guide. See Attachment 7.2: RVTS Applicant Guide 2015.

See Attachment 7.1: AGPT Application Guide 2015.

c. Provide information on the number of applicants and number of registrars entering ACRRM pathways in each of the last three years (for each year include the number and distribution across regional training providers and pathways of both continuing and newly commencing registrars)

Table 7.1: Number of ACRRM Registrars entering FACRRM pathways 2011-2013*

Pathway 2011 2012 2013

Enrol total enrol Total Enrol Total

Independent

pathway

49 104 45 120 45 142

AGPT (by RTP) 73 225 102 276 115 309

RVTS pathway 22 65 23 80 21 82

Total by Year 144 394 170 476 181 533

* Figures as at Dec each year. Table 7.2: Number of ACRRM Registrars entering AGPT Pathways 2011-2013 (Breakdown by RTP)*

RTP 2011 2012 2013

Total Enrolments Total Enrolments Total Enrolments

Adelaide to Outback GP Training Program (SA)

13 7 13 3 22 12

Beyond Medical Education (NSW + VIC)

12 2 11 2 11 4

Bogong Regional Training Network (VIC)

9 4 6 0 4 1

Coast City Country (NSW) 5 1 9 5 17 13

GP Training Queensland (QLD)

17 6 34 17 32 8

GP Synergy (NSW) 1 0 3 2 6 2

General Practice Training Tasmania Inc (TAS)

10 5 7 2 7 4

General Practice Valley to Coast (NSW)

3 2 1 0 3 2

North Coast GP Training (NSW)

4 1 3 0 2 2

Northern Territory GP Education (NT)

13 1 25 13 27 9

Sturt Fleurieu GP Education & Training (SA)

48 12 58 22 57 14





Tropical Medical Training (QLD)

15 7 12 2 14 8

3 0 4 1 3 0

48 20 61 24 79 27

VMA (VIC) 0 0 0 0 0 0

WAGPET (WA) 17 5 24 9 22 7

WentWest 0 0 0 0 1 1

No RTP 7 5 0 2 0

TOTAL 225 73 276 102 309 115

* Figures as at Dec each year.

d. Outline the formal processes and structures that facilitate and support the involvement of registrars in the governance of their training, highlighting any changes since the 2010 accreditation assessment.

The processes are summarised above (See Section 7.1.1). The key changes since 2010 include following:

Bi-College Accreditation Process incorporates a requirement for registrar representation in RTP governance and program development structures.

Registrars must be represented on the (newly established) Education Council, Research Council, and all vocational-training related Education Subcommittees.

More generally the role of the Registrar Committee chair has been made more representative, through the establishment of communication links and more regular provision of meetings.

e. Outline developments in relation to College communication with, and support of registrars since 2010 and comment on the success of these developments.

Key developments are listed below. Evaluation Tools

Development of multiple evaluation tools which are processed and their results and recommendations are submitted to relevant committees and programs teams toward program improvement. These include the Registrar, Member Survey, and, the Vocational Training Exit Survey. The 2014 Registrar Survey included a specific question about registrar preferences for communications they receive from the College.

Relationship with RTPs

Improving the effectiveness of its relationship with RTPs has been a key focus of the College’s time and energies and major steps forward have been made in this area.

The Bi-college Accreditation framework is a crucial step forward as it provides for the first time a

procedural mechanism requiring accountability of RTPs to the College with respect to the passing on of its communications.

(For further details: See Section 8 Condition 23).


Assessment Support

Development of a range of assessment support programs including assessment information sessions, study groups and mock exams have been introduced. These have been a measurable success as there is an improved pass rate for those participating.

General Communications

Orientation Sessions for registrars on all pathways via virtual classrooms have been introduced. The effectiveness of these is demonstrated by the strong attendance, with 100 registrars enrolled to attend.

The E- newsletter for registrars, FACRRM Fundamentals has been made more extensive and more frequent and the Registrar Committee now contributes to its content. The effectiveness of the newsletter is explicitly evaluated in the Registrar Survey.

The College has established social networking fora in Facebook and twitter as well as developing the virtual classrooms tool for communication. The effectiveness of these is also assessed in the Registrar Survey.

An Email link for communication with the Registrar Committee Chair has been established. RRMEO

Development of a tracker on RRMEO to enable registrars to track training completion. Registrar Surveys have recorded some improvement in satisfaction levels in this area.

The number of RRMEO modules available has increased and emergency medicine courses have been accredited. (Over 100 education modules are currently available).

Curriculum and Program Structure

Revision and development of training policies to provide greater clarity and flexibility.

Completion of AST curricula and addition of an Academic Practice as another option for AST.

f. Provide data on appeals by registrars and/or overseas-trained specialists since 2010 and the outcome of the appeals.


g. Any other planned developments that relate to Accreditation Standard 7.

The redevelopment of the ACRRM website and implementation of a Customer Relations Management system are expected to improve the ease with which important information can be found, and the consistency and functionality of the language used.


8. Program Delivery 8.1 Supervisors, assessors, trainers and mentors

8.1.1 The training organisation has defined the responsibilities of hospital and community practitioners who contribute to the delivery of the training program and the responsibilities of the training organisation to these practitioners.

ACRRM has defined the responsibilities for supervisors in the’ Standards for Supervisors and Teaching Posts’ documents. There is a set of standards for each stage of training and each AST. The standards relating to supervisors outline the qualifications, experience and abilities required and evidence of a commitment to teaching and supporting registrars.

(See Attachment 8.1: Standards for Supervisors and Teaching Posts (CCT, PRRT, AST Curricula)).

8.1.2 The education provider has processes for selecting supervisors who have demonstrated appropriate capability for this role. It facilitates the training of supervisors and trainers.

ACRRM works with training providers, state health and other college partnerships to identify suitable supervisors and posts. ACRRM is responsible for the accreditation of supervisors and teaching posts, this role is undertaken in collaboration with the relevant training provider. AGPT Pathway On the AGPT pathway RTPs are funded for accreditation visits in their core funding. In 2011 ACRRM delegated to RTPs the responsibility to collect information and evidence against the supervisor and teaching post standards and make an accreditation recommendation to the College. The information and evidence against these standards is not required by ACRRM at the time of accreditation but must be stored by the training provider and provided if audited. 10% of accreditations are audited each year to ensure appropriate evidence is available. This delegation applies to the PRRT stage only. For CCT and AST stages ACRRM must be provided with information and evidence to support the recommendation. The delegation of Post Accreditation is formalised through a formal management agreement signed by ACRRM and the training provider. The Post Accreditation Management Agreement with the training providers requires submission of an annual report to ACRRM. This report is sent to ACRRM each July and provides the College with an opportunity to engage with RTPs to discuss quality improvement opportunities based on survey data that has been received from registrars. The processes and records are also checked as part of the Bi-College Accreditation site visit. RVTS and Independent Pathway ACRRM continues to be responsible for reviewing evidence and accrediting against the standards on the RVTS and IP.

(See Attachment 8.2: Post Accreditation Management Agreement with RTPs))



8.1.3 The education provider routinely evaluates supervisor and training effectiveness including feedback from trainees and offer guidance in their professional development in these roles.

The training providers hold primary responsibility for evaluating supervisor and training effectiveness. ACRRM monitors the effectiveness through the Bi-College RTP accreditation process. The teaching post standards require that the post consents to the registrars providing feedback to the training provider on the training environment provided by the posts and supervisors (see attachment below). The Training Providers are required to have processes in place to collect information on supervisor performance. Registrars provide feedback to the provider at the end of each training placement. In addition:

AGPT RTPs administer a registrar survey or utilise information from the GPET Annual Survey. These surveys include questions relating to supervisor performance.

RVTS also surveys registrars annually.

The ACRRM annual Registrar Survey for all registrars also includes questions on supervisor and training effectiveness.

On the AGPT pathway prior to delegation to the RTPs of post accreditation ACRRM required all providers to demonstrate how they evaluate supervisor performance. Now, ACRRM requires an annual report from each training provider against a set of key performance indicators. A summary of feedback from registrars on their supervision is required, plus any issues that were identified and how they were dealt with. Feedback obtained may also take the form of compliments, specific problems or accreditation related issues. These matters are primarily dealt with by the responsible training provider; however, in some cases ACRRM and the training provider may discuss and agree on a way of dealing with the issue and providing feedback. This evaluation information feeds into supervisor accreditation and reaccreditation cycles. (See Attachment 8.1: Standards for Supervisors and Teaching Posts; PRRT Policy - Indicator 8.8.2).

(See Attachment 8.2: Post Accreditation Management Agreement with RTPs).

(See Attachment 8.3: RTPs’ Post Accreditation Annual Report).

8.1.4 The training organisation has processes for selecting assessors in written, oral and performance-based assessments who have demonstrated relevant capabilities.

A professional team of qualified writers, editors and examiners are led by two Principal Examiners, one for Primary Curriculum Assessment and the other for AST assessments. Expressions of interest to join these groups are invited through the national Country Watch e-newsletter to enable as broad as possible geographic and demographic membership representation. Team members are required to hold FACRRM and be vocationally registered. Initially ACRRM has appointed assessors who have known assessment and training expertise. These assessors had often developed expertise through training and experience from other jurisdictions (e.g. pre-registration medical schools, other medical colleges). The expertise of the team has grown with new members appointed as the program has developed. Fellows, who are not known to the assessment or training teams, are assessed for



suitability as assessors through involvement first in other areas such as study groups or facilitating education sessions. All assessors receive training and mentoring prior to commencing assessment roles.

8.1.5 The education provider has processes to evaluate the effectiveness of its assessors/examiners in this role.

ACRRM uses a number of processes to evaluate the effectiveness of general practitioners who contribute to assessment modalities. Post exam feedback from registrars, assessors, invigilators and others involved in assessment occurs routinely after each assessment. Previously this was in place for MCQ and StAMPS, recently miniCEX and MSF feedback surveys were also introduced. This information is reviewed by the Principal Examiner and if relevant feedback to the assessors.

(See Attachment 8.4: Example of Collated Feedback from StAMPS 2014 Examiners and Candidates)

Exam writing and editing involves more than one person working on the same question, this allows for feedback between writers. Writers also receive feedback after the exam regarding the performance of the assessment items. At each StAMPS assessment centre one examiner is appointed as the QA examiner. The QA examiners role is to ensure that the assessment is delivered in a fair and consistent way. The QA examiner is rostered to observe all examiners over the exam session; they replace the examiner when there is a conflict of interest and will review the recording of scenarios when requested by the examiner, when there has been a need for special consideration or borderline cases. The QA examiner provides feedback to examiners on areas noted for improvement. At the end of each StAMPS rotation examiners attend a debrief session. This includes an opportunity for examiners to calibrate their marking. An external examiner may also attend the StAMPS assessment to observe and give feedback. Prof Tarun Sen Gupta attended the May 2014 exam. MiniCEX is delivered by a small group of experienced examiners who have previously demonstrated their assessment skills. Training is provided by the Principal Examiner, staff and through a RRMEO module on formative miniCEX. In May 2014 a new quality assurance process has been implemented for miniCEX. Feedback is sought from candidates, venues and examiners following each miniCEX. The Principal Examiner reviews each miniCEX report prior to ratification by the Board of Examiners. Concerns raised through this process or through candidate feedback are followed up immediately if significant. Collated feedback is provided to the miniCEX examiners at the end of each semester (6 months). The Principal Examiner also conducts a session with examiners every 6 months, to give a receive feedback and to provide information on any changes or reinforce areas where required.


8.2 Clinical and Other Educational Resources

8.2.1 The education provider has a process and criteria to select and recognise hospitals, sites and posts for training purposes. The accreditation standards of the training organisation are publicly available.

As described in Section 8.1, the standards for posts are contained within ‘The Standards for Supervisors and Teaching Posts’. The standards are available on the ACRRM website and in hard copy on request.

All training towards Fellowship of ACRRM must take place in an accredited post. Training providers are responsible for arranging posts for the registrars training with them. Registrars on the IP are responsible for finding their own accredited post or having the post they are already working in accredited. CCT posts must be accredited by the State or Territory Post Graduate Medical Council or by ACRRM against the Standards for CCT. PRRT and AST must take place in a post which is accredited against the ACRRM Standards for Teaching Posts and Teacher’s in Rural and Remote Medicine. As outlined in Section 8.1 above, in 2011 ACRRM delegated to training providers on the AGPT pathway the responsibility to collect information and evidence against the supervisor and teaching post standards and make an accreditation recommendation to the College. This delegation applies to the Primary Rural and Remote sphere of training only. For CCT and AST stages ACRRM must be provided with information and evidence to support the recommendation. Table 8.1 below provides information on the number of posts accredited for ACRRM vocational training.

8.2.2 The education provider specifies the clinical and/or other practical experience, infrastructure and educational support required of an accredited hospital/training position in terms of the outcomes for the training program. It implements clear processes to assess the quality and appropriateness of the experience and support offered to determine if these requirements are met.

ACCRM defines these requirements in the following documents:

The ACRRM Standards for RTPs Recognition outlines the requirement for training providers providing training on ACRRM training pathways. (See Attachment 8.5 ACRRM Standards for RTPs Recognition).

The Standards for Supervisors and Teaching Posts for each stage of training plus the

Management Agreement. (See Attachment 8.1: Standards for Supervisors and Teaching Posts (PRRT, CCT, AST curricula)).

(See Attachment 8.2: Management Agreement with RTPs)

ACRRM training policies. (See Section 3).

ACRRM Curricula (See Attachment 3.1: ACCRM Curriculum)

(See Attachment 3.2: AST Curricula)





On AGPT and RVTS programs, Training Providers are accredited to deliver training. Training providers are assessed against the above ACRRM standards during accreditation. ACRRM is directly responsible for delivery of training against these standards on the IP. ACRRM and RACGP work together to accredit training providers on AGPT and RVTS pathways. The Bi-College RTP Accreditation program provides one streamlined process for accreditation against each College’s standards. To facilitate the Bi-College accreditation the Colleges have agreed on set of principles and outcomes as a framework to structure the review process. The ACRRM standards listed above are assessed through this framework. (See Attachment 8.6: Bi-College Accreditation: Principles and Outcomes.)

RTPs seeking ACRRM accreditation are reviewed every three years as part of the Bi- College RTP Accreditation Program. The RTP is required to provide a written submission against the Bi-College Principles and Outcomes framework and submit the report to the Bi-College program. An Accreditation Review Team is appointed consisting of two GP reviewers with knowledge of both College standards and a senior education staff member from each College. The review team is supported by the Bi-College Program Manager. The review team seeks evidence to demonstrate meeting College standards, through the Principles and Outcomes framework. An Accreditation Review Team Report is written detailing the findings of the review and assesses the degree to which each outcome is met. The Colleges are provided with a copy of the Accreditation Review Team report and the review team views on recommendations and conditions that should be applied. The RTP’s Bi-College Accreditation Review Team Report and Recommendations Report once received by the College are initially reviewed by an Internal Governance Team comprising of the Censor in Chief, Director of Education and Director of Operations. Their role is to ensure that any recommendations and/or conditions suggested by the Review Team match the assessment of risk for the College and the reputation of its training program. The outcome of this process is the formal proposal to the Board regarding Accreditation of the training provider in the form of a ‘draft’ ACRRM Accreditation Outcomes Report. Training providers fully complying are granted ACRRM accreditation for three years. RTPs that have not fully met the standards are awarded accreditation with conditions or recommendations. The accreditation period may be reduced from three years if there are significant concerns. The RTP is provided with a report outlining actions required and timeframe for compliance. This is monitored by the College to determine when the outcome is met. (See Attachment 8.7: Bi-College RTP Accreditation Program Guide).

Information to assist in monitoring training provider performance is obtained through a range of sources:

Registrar surveys conducted by AGPT, RVTS and ACRRM.

Post Accreditation Annual Report.

Complaints and compliments about training providers received by the College from registrars and supervisors

Registrar RPL and Completion of training applications received from training providers

Monitoring information on training provider websites




This information feeds into the three yearly mandatory quality review and accreditation process described above.

8.2.3 The education provider’s accreditation requirements cover; orientation, clinical and/or other experience, appropriate supervision, structured educational programs, educational and infrastructure supports such as access to the internet, library, journals and other learning facilities, continuing medical education sessions accessible to the trainee, dedicated time for teaching and training and opportunities for informal teaching and training in the work environment.

The aspects described above are covered in the Principles and Outcomes for training and providers. Principle 3: Education and Assessment. In addition the practical experience, infrastructure and educational support required of posts are clearly outlined in the Standards for Supervisors and Teaching Posts for each stage of training they define requirements for:

The location of the post;

Training resources;

Clinical and office equipment;

A teaching plan;

Provision of a range of learning opportunities;

Structured dedicated teaching times for training and study;

Adequate organisational management arrangements; and

Structured evaluation.

(See Attachment 8.6: Bi-College RTP Accreditation: Principles and Outcomes).

8.2.4 The education provider works with the health services to ensure that the capacity of the health care system is effectively used for service-based training, and that trainees can experience the breadth of the discipline. It uses an appropriate variety of clinical settings, patients and clinical problems for training purposes, while respecting service functions.

The standards for supervisors and teaching posts describe the clinical learning opportunities required for each stage of training or AST discipline. A significant review of clinical experienced required during PRRT was undertaken during 2012. This resulted in a refinement of the Standards for Supervisors and Teaching posts for PRRT. In particular 8.1 Clinical Learning Opportunities, the definition of posts and an appendix that provides guidance on accrediting teaching posts and in particular any restrictions that may apply. This work then informed an update to the teaching post standards and training policy for PRRT. Capacity Training posts capacity is monitored through the Bi-College RTP Accreditation Program (see Program Principles and Outcomes: Outcome 6.4). There have been significant increases in AGPT training places over the past few years. The training providers report that they are starting to experience challenges around a providing adequate numbers of training posts for all registrars. This has been exacerbated by the recent announcement of an additional 300 training places for 2015. All training providers are working to address this challenge through utilising practices accredited for PGPPP, existing practices taking more registrars and accrediting more posts.



Innovative methods of supervision and in practice teaching are being developed and accredited by ACRRM. ACRRM standards allow for different models of supervision providing that standards are met. The Standards for Supervisors and Teaching Posts in PRRT (Appendix 3) includes guidance for offsite supervision and outlines the information to be provided to ACRRM in order to be accredited. ACRRM is holding a preconference forum prior to RMA 2014 looking at innovative models to increase training capacity in rural areas. ACRRM is soon to commence a campaign with Fellows of ACRRM to encourage them to become accredited for FACRRM training.


8.3 AMC Conditions

Condition 22 Ensure that the supervisors accredited to deliver ACRRM training, have the necessary resources and support, and access to appropriate professional development and training. (Standard 8.1.1)

The Standards for Supervisors and Teaching Posts (Indicator 7.2.10) requires supervisors to participate in supervisor training and other activities to further develop teaching/mentoring. ACRRM provides a Supervisor Guide and offers workshops at the annual RMA conference for supervisors. In addition an online module for supervisors is being developed and is expected to be complete by the end of 2014. The Supervisor Guide is available on the website. On the AGPT and RVTS pathways the training providers are required to provide training for supervisors. This is monitored through the Bi-College RTP accreditation process.

(See ACCRM website: ‘PRRT Guide for Supervisors’).

(See Attachment 8.1: Standards for Supervisors and Teaching Posts)

Condition 23 Communicate actively with regional training providers and supervisors to ensure they are informed about training requirements, standards or supervision, and expectations of the supervisors as they currently apply and when changes are made. (Standard 8.1.1).

The College recognises and prioritised the need to improve the relationship with its RTP partners toward ensuring that its Registrars are able to receive the highest possible standards of training. The College has made major steps forward in this undertaking over the last 3 years.

The most significant step forward has been that the Bi-College Accreditation Process has established a clear mechanism between compliance with College curricula and policies and continuance of accreditation.

The College has increased the frequency of its interactions with training providers. These include regular attendance at the RTP CEOs and Director of Training meetings. Regular meetings have been established with five training providers; this has been very positive and will be rolled out to include all training providers. The College has also been redoubling its efforts to maximise its informal engagement with RTPs including at occasions such as the annual GPET convention.

Each RTP has been individually visited by the College as part of its accreditation process. These visits will become a regular part of the ongoing accreditation process.

The College has increased the number of workshops available to provide training in ACRRM standards, curricula and procedures; including, at its Annual Conference (Rural Medicine Australia). This is one of the times when a significant number of program supervisors will be in attendance. It has also introduced night session workshops to enhance convenience. The workshops give supervisors an opportunity to learn about ACRRM learning systems and tools and provide an additional opportunity for them to have a direct interface with the College. Regular workshops are held for training providers twice a year.




The College has appointed MS Nicole Lind as Vocational Training and Assessment Operations, who brings to the role her considerable experience of working in the RTP network. Additionally a Supervisor Liaison Officer has also been appointed.

Condition 25 Report on its contribution to ensure that all posts have current accreditation and to clear the backlog of posts awaiting accreditation and reaccreditation. (Standard 8.2)

Under the Bi-College Accreditation arrangements, the College’s role in the accreditation of training posts is to contact the training provider requesting action whenever accreditation is overdue; occasionally these require follow up contacts. The training providers, then follow up and either withdraw the post or arrange to expedite the re-accreditation. Delays can often be due to RTPs opting to process accreditations in groups. In these instances the College does not receive the necessary paperwork until the entire group is complete, by which time some are marginally overdue.

There are currently 6 provisionally accredited posts awaiting reaccreditation. The College has received necessary documentation for one of these and it is expected to be accredited in the very near future. The remaining five are not actively training and may not continue with accreditation. There are currently 27 fully accredited posts in the PRRT overdue for re-accreditation. The majority of these do not have ACRRM registrars at present. 2 posts have received their requisite site visit and the College is awaiting documentation which should lead to accreditation in the near future. The remaining 25 visits are currently being arranged by training providers. Of these nearly all are only very recently overdue (21 lapsed within the last 3 months), the remainder are no longer than 6 months overdue (See Table 8.1 below).

AST post accreditation is an area that is still under development. There are currently 22 provisionally accredited AST posts that are overdue for re-accreditation. Most of these have lapsed in the last 12 months; (all bar 8). The majority of these are currently being worked on and many do not currently have a registrar. Some for a range of reasons will not progress to full accreditation (See Table 8.2 below). Three of these posts are accredited through RVTS.

Nambour General Hospital (Adult Internal Medicine)– Due May 2013 – was originally accredited for a specific RVTS registrar so may not be re-accrediting – will check with relevant RTP to see if interested in continuing

John Hunter Hospital / Calvary Mater Hospital (Adult Internal Medicine) – Due Jun 2013 – was originally accredited for a specific RVTS registrar so may not be re-accrediting – will check with relevant RTP to see if interested in continuing

Joyce Palmer Health Service (ATSIH) – Due May 2014 - was originally accredited for a specific RVTS registrar so may not be re-accrediting – will check with relevant RTP to see if interested in continuing.

Table 8.1: Primary Rural and Remote Training Posts for Re-accreditation

Accreditation Status

Posts overdue for re-accreditation

Progress

Fully Accredited NCGPT (2)

NSW Meadows Medical Due Oct 2013 - RTP has undertaken site visit – sending in recommendation for re-accreditation

Tintenbar Medical Centre

Due Mar 2014 – RTP has undertaken site visit – sending in recommendation for re-accreditation

CCCT (2)


Griffith Medical Centre

Due Jun 2014 – RTP working on this

Sapphire Coast Medical Practice

Due Jun 2014 – RTP working on this

GP Synergy (2)

Barraba Medical Centre

Due Jun 2014 - RTP working on this

Barney Street Medical Due Jun 2014 – RTP working on this NTGPE (1)

NT Ramingining Community Heath Centre

Due Oct 2013 – RTP currently working on this

QRME (3)

QLD Highfields & District Medical Centre

Due May 2014 – RTP currently working on this

Millbank Medical Centre


Mackenzie House Medical Centre

Due Jun 2014 – RTP currently working on this

GPTQ (3) Wilsonton Medical

Centre Due Dec 2013 – RTP currently working on this

Murgon & Wondai Family Medical Practice

Due Jun 2014 – RTP sending in paperwork to have these two practices re-accredited in their own right

Sturt-Fleurieu

SA The medical Clinic Millicent

Due November 2013 – RTP currently working on this

Kincraig Medical Clinic

Due April 2014 – RTP currently working on this

Ferrers Medical Clinic Due April 2014 – RTP currently working on this Southern Fleurieu

Family Practice Due April 2014 – RTP currently working on this

Strathalbyn Medical Clinic


Gumeracha Medical Practice


The Medical Clinic – Angaston Medical Centre


Beyond Medical

VIC Stawell Medical Due Jan 2014 – RTPL currently working on this Centre Gisborne

Medical Centre Due June 2014 – RTP currently working on this

Bogong Training

Beechworth Surgery Due Apr 2014 – RTP currently working on this

Southern GP Training

Heyfield Medical Centre

Due May 2014 - RTP currently working on this

Maffra Medical Group Due May 2014 - RTP currently working on this RAAF Base Sale –

Base Medical Facility Due May 2014 - RTP currently working on this

Sale Medical Centre Due June 2014 - RTP currently working on this

WAGPET


WA KAMSC Due April 2014 – RTP currently working on in conjunction with AST post

Provisionally Accredited

QRME (5)

QLD Doctors @ Beaudesert

Due May 2014. QRME looking in to this.

Maleny Hospital Due 2013. Paperwork received and under discussion.

Morton ATSICHS, Deception Bay

Due May 2014. QRME checking whether they wish to re-accredit.

Moreton ATSICHS, Strathpine.

Due May 2014. QRME checking on whether they wish to re-accredit.

Murri Medical Due Sept 2013. QRME checking on whether they wish to re-accredit.

GPTQ (1)

Urangan Medical Centre

Due Jan 2014. GPTQ checking on whether they wish to re-accredit.

Table 8.2: Advanced Specialised Training Posts for Re-accreditation

Accreditation Status

Posts overdue for re-accreditation

Progress

Fully Accredited WAGPET (1)

Kimberley Aboriginal Medical Services (Emergency Med)

Due Apr 2014. From WAGPET advice anticipate recommendation for re-accreditation in near future.

TMT (1)

ATSIH RFDS Cairns (Remote Medicine)

Due Dec 2013 – RTP working on now that AST in Remote Medicine has been finalised.

Provisionally Accredited

QRME (2)

QLD Doctors @ Beaudesert Due May 2014. QRME looking in to this.

Maleny Hospital Due 2013. Paperwork received and under discussion.

AOGP (1)

Beyond Medical (1)

NSW Dubbo Base Hospital (Emergency Medicine)

Due Apr 2011 – RTP currently working on this

CCCT (1) Wagga Wagga Base

Hospital (Emergency Medicine)

Due Jan 2011 - RTP working on currently

NTGPE (1)

NT Centre for Disease Control (ATSIH)

Due Dec 2012 – RTP have just sent in request to change this to PRRT post

GPTQ (3)

QLD Ipswich General Hospital (Adult Internal Medicine)

Due Jan 2014 - RTP currently working on this

Kalwun Health Service (ATSIH)

Due Feb 2014 - was originally accredited for a specific registrar so they may not be re-accrediting

Gladstone Hospital (Emergency Medicine)

Due Mar 2014 – RTP currently working on this

QRME (6)

Ipswich General Hospital (Paediatrics)

Due Oct 2011, currently on hold at request of RTP

Toowoomba Base Hospital (Paediatrics)

Due Dec 2012 – RTP currently working on this

Ipswich General Hospital (Emergency Medicine)

Due Oct 2012 – RTP currently working on this

Toowoomba Base Hospital (Surgery)



Moreton ATSICHS – Deception Bay (ASTIH)


Moreton ATSICHS – Strathpine (ATSIH)


AOGP (1)

SA Pika Wiya Health Service Aboriginal Corporation (ATSIH)

Due Apr 2014 - was originally accredited for a specific registrar so they may not be re-accrediting

Southern GP Training (3)

Vic Mount Gambier Hospital (Emergency Medicine)

Due Feb 2011 – RTP currently working on this

Ballart Base Hospital (Emergency Medicine)


South West Healthcare (Emergency Medicine)

Due Apr 2014 – RTP currently working on this

Beyond Medical (2)

St John of God Hospital (Emergency Medicine)

Due May 2013 – RTP working on currently

Echuca Regional Health/Bendigo Health

Due Oct 2013 – chasing up with RTP

GPTT (1) TAS North West Regional

Hospital (Emergency Medicine)

Due Feb 2014. RTP has advised that post won’t be re-accredited until a registrar indicates that they want to go there. Will be withdrawing it temporarily.



Pertaining to Standard 8.1:

a. College’s role in selecting and setting standards for supervisors, with particular emphasis on any review or changes since the 2010 accreditation. How the college works with other bodies, such as RTPs, in setting and assessing standards for supervisors and ensuring supervisors are meeting ACRRM standards and assessing registrars against ACRRM

requirements.

The standards for supervisors in Primary Rural and Remote Training were revised in June 2013. The following changes were made when the standards were revised to allow urban posts to be accredited for up to 6 months for an individual registrar.

7.1.2 Where the teaching post is in an urban setting and supervisors meeting the above criteria are not available, a suitably qualified (Fellowship of GP or other Speciality College with not less than five years full-time experience) onsite supervisor is acceptable providing that the registrar also has an offsite mentor meeting the above criteria. The role of the mentor would be to put information into a rural context. The mentor would be expected to meet (via phone or other virtual means) with the registrar bimonthly.

Offsite or remote supervision has always been allowed for in the ACRRM standards and is used routinely in the RVTS pathway. In 2013 requests were received to accredit other offsite supervision models on the AGPT pathway. A Guidance document was developed in response. It is included as Appendix 3 in the Standards for Supervisors and Teaching Posts in PRRT.

The Supervisor Survey includes a question regarding the achievability of the ACRRM standards and gives the opportunity to offer suggestions for improvements. In the 2014 survey 78% of supervisors rated the standards as relevant and achievable. No suggestions were made for improvements to the standards. The College regularly receives feedback from training providers regarding standards. The changes described above have been made in response to their feedback.

b. Comment on processes for review of supervisor performance.

See Condition 22 above.

c. Describe the College’s processes for informing supervisors about changes to the curriculum and assessment methods and any supervisor training activities undertaken or planned.

Support and training is provided to training providers and supervisors to assist them to understand the requirements of FACRRM, this support is particularly relevant with training providers and posts being familiar with FRACGP requirements but still learning about FACRRM requirements and how they differ.

The following are examples of training and support provided by ACRRM:

Direct email, e-newsletters and the website

Information sessions via virtual classroom when changes are made or areas for improvement identified;

workshops during RMA conference;

Other workshops for training providers:


Regular meetings with training providers

Ongoing training and coaching with training providers individually, often occurring in the context of accrediting posts, case conferencing issues arising with individual registrars.

d. Report on other developments relevant to Standard 8.1.

All developments are reported above.

Pertaining to Standard 8.2:

a. Summarise the evolution of the policy ‘Primary Rural and Remote Training’ (PRRT) and when changes made take effect. Indicate how the College informs RTPs and accredited practices of policy change and how it is monitoring the implementation of the changes.

The PRRT Policy was revised in 2012. This revision defined minimum training times in three areas:

Community primary care and population health: minimum 6 months full-time equivalent experience

Hospital and emergency care: minimum 6 months full-time equivalent experience

Rural and remote context: Minimum 6 months full-time equivalent experience This policy applies to all registrars commencing training from 2012.

This policy change was made following extensive consultation with all stakeholders and ACRRM membership. The changes were communicated to Registrars and other stakeholders through:

FACRRM Fundamentals

Direct email

Meetings with training provider Directors of Training and CEOs

Presentations at conferences and workshops

The College Website There was also a change in the PRRT Supervisor and Teaching Post Standards providing definition to posts that require restriction due to limited scope of practice. Compliance with the PRRT policy is enforced through RTP accreditation and for individual registrars through checking that required experience has occurred prior to Completion of Training.

b. Append a copy of the College’s policy for accreditation of training posts/units. Please provide a summary of the changes made since the 2010 Accreditation. Indicate how the College has informed RTPs and accredited practices of change in the policy and process and how it is monitoring the implementation of the changes.

A significant review of clinical experienced required during PRRT was undertaken during 2012. This resulted in a refinement of the Standards for Supervisors and Teaching posts for PRRT. In particular ‘8.1 Clinical Learning Opportunities’, the definition of posts and an appendix that provides


guidance on accrediting teaching posts and in particular any restrictions that may apply. This work then informed an update to the teaching post standards and training policy for PRRT.

Standards for Supervisors and Teaching posts in CCT and each AST have been developed since 2010.

(See Attachment 8.1: ACRRM Standards for Accreditation of Supervisors and Teaching Posts (PRRT))

c. Summarise the College’s accreditation activities since the 2010 AMC assessment, and the outcomes of these activities including settings of posts accredited. Indicate any posts given limited or restricted accreditation, including posts for PRRRT not offering experience in community primary care which have been given restricted accreditation.

The activities are given at Section 8.3 Condition 25, and figures given at Tables 8.1 and 8.2.

d. Outline mechanisms for registrars or supervisors to raise concerns about the training environment outside accreditation process. Indicate how many concerns have been referred to the College since 2010 and how it has addressed them.


e. Outline any other challenges or plans the College is addressing relation to Accreditation Standard 8.

The processes are now in place for accrediting training providers through the Bi-College RTP Accreditation program. All training providers will have been reviewed by end of 2015.

The monitoring process for training providers has been devised and will commence shortly as the first RTP to be accredited reaches first reporting requirement. The College will continue to monitor and audit to ensure compliance with supervisor and teaching posts accreditation processes. ACRRM is engaging in discussions with the Commonwealth toward ensuring that the impending changes to the AGPT and GPET framework will optimally facilitate general practice training particularly for doctors in rural and remote locations and their communities.



9. Continuing Professional Development 9.1 CPD Programs

9.1.1 The education provider’s professional development programs are based on self-directed learning. The programs assist participants to maintain and develop knowledge, skills and attitudes essential for meeting the changing needs of patients and the health care delivery system, and for responding to scientific developments in medicine as well as changed societal expectations.

The ACRRM PDP aims to ensure that Fellows and members access and benefit from a range of continuing educational activities that enhance their clinical, management and professional skills. In turn, this ensures that the profession strives towards the delivery of the highest possible standards and quality of care to patients and communities. ACRRM is committed to life long learning for all members and encourages individual continuing professional development that is relevant to a member’s profile of professional practice and performance. ACRRM contends that there is a core set of skills and competencies over and above those typically required in urban based general practice necessary for the preservation of quality and safe healthcare in rural and remote communities. For ACRRM the PDP is especially important as it contributes towards the development and maintenance of these and the continued delivery of the associated services.

The program comprises of assessed clinical experience and educational activities. Educational activities are provided directly by ACRRM and also by a wide range of delivery organisations. The program is self-directed with Fellows and members being directly responsible for determining and planning what educational activities are relevant to their professional and community needs. ACRRM’s education platform Rural and Remote Medical Education Online (RRMEO) facilitates self-directed learning by assisting with identification of workshops and courses, and by providing specifically designed modules, electronic resources and online management of training records and PDP points. The PDP is the most mature of the College’s programs and has been delivering high quality professional development to Fellows and members for eleven years. Participation in and compliance with PDP is mandatory for all Fellows of ACRRM and the program is freely available to all members.

9.1.2 The education provider determines the formal structure of the CPD program in consultation with stakeholders, taking account of the requirements of relevant authorities such as the Medical Board of Australia and the Medial Council of New Zealand.

During 2011 the ACRRM’s Professional Development Committee, in consultation with stakeholders undertook a review of the program to ensure that the College continues to provide best practice models of CPD and that the program continues to deliver an appropriate range of activities to support quality and safe practice in changing environments. Additionally, that it meets the Maintenance of Professional Standards (MOPS), Vocational Recognition maintenance, credentialing and clinical privileging requirements of members. An outcome of the review was the refinement of the ‘points allocation’ framework for the 2014 -2016 triennium. The framework articulated in the ACRRM PDP Member Handbook 2014 - 2016 sets out the PDP activities for which points can be claimed and the value of points each activity attracts. This


indicates to Fellows, members and education providers the educational quality/weighting ACRRM gives to those activities. The framework includes an activity points capping system and aims to encourage participants to undertake a range of activities that supports their practice and the provision of high quality health services to the community. To be compliant participants must gain a minimum of 100 points in the 2014 - 2016 triennium accumulated through participation in the three categories of accredited activities:

Advanced Basic Life Support (10 points)

Practice Reflective Professional Development (30 points)

Core Continuing Professional Development (60 points) The introduction of the mandatory category of Advanced Life Support for Fellows into the 2008-2010 triennium requirements was a direct response by the ACRRM Board to a call by the NSW Medical Board to ensure all rural and remote doctors had the necessary skills to save a life. This has continued to be a strong theme in the Scheme since that time. The Practice Reflective Professional Development has been matured from previous parts of the scheme to ensure that a major portion of the continuing medical education of Fellows and members is of a reflective nature and leads to improved practice and clinical skills.

(See Attachment 9.1: ACRRM PDP Member Handbook 2014-2016)

9.1.3 The process and criteria for assessing and recognising CPD providers and/or the individual CPD activities are based on educational quality, the use of appropriate educational methods and resources, and take into consideration feedback from participants.

Activities Assessment To ensure CPD providers deliver activities to a high standard and level of relevance to the ACRRM curriculum, the ACRRM PDP requires education providers to gain accreditation for each training activity. This application process requires details of the educational activity including: learning objectives; facilitator details; topics; relevance to ACRRM’s educational domains and curriculum areas; and assessment requirements. ACRRM reviews completed applications to ensure all details are correct and the activities are relevant to the ACRRM curriculum and educational domains. The Professional Development Committee provides a clinical reference group for staff to ensure that applications are relevant to the Fellows and member’s actual requirements. Where necessary the Professional Development Committee reviews applications and determines accreditation status and points allocation. Providers and members are advised and details are posted on ACRRM’s website. The following criteria are used by ACRRM to assess educational activities:

1. Is there evidence in the application that the planned educational activity is based on a rural medical practitioner’s educational needs?

2. Does the activity relate to one or more of the ACRRM educational domains?

3. Were rural medical practitioners involved in the planning and implementation stages of the educational event/activity?

4. Does the educational event/activity have clear, specific learning objectives?

5. Are the planned educational strategies based on adult learning principles?

https://www.acrrm.org.au/pdp


6. Does the educational event/activity include a combination of educational interventions, which predispose, enable and reinforce behaviour change?

7. Does the application show that some impact evaluation (changes in knowledge, skill, attitude, practice or patient outcome) is planned?

(For further details of the process and criteria for assessing activities for accreditation purposes: See the College website, PDP page.)

Program Evaluation

To ensure that ACRRM continues to accredit and provide programs based on educational quality that are relevant, appropriate and supportive to ongoing educational needs of PDP participants, ACRRM conducts an annual PDP Evaluation Survey and a PDP Education Provider Evaluation Survey. The report for the last surveys was created as at 30 June 2013. The survey provides important data to assist ongoing program planning, development and resourcing. The outcomes of the regular survey program have determined the priority of educational activities to be offered at this year’s ACRRM conference (Rural Medicine Australia). The survey identified challenges relating to RRMEO. In response the College developed a number of strategies to improve the system and support member usage, including:

A simplification of the protocols for RRMEO use;

Small group workshops via teleconference; Online tutorial modules using simulation software such as Adobe Captivate; and

Face-to-face workshops at conferences such as ACRRM’s annual conference. (See Attachment 9.2: Annual PDP Evaluation Report 2013)

Generalist Emergency Medicine and Generalist Mental Health Training In response to identified community need the College has focussed on development of programs in two critical areas of need particularly in rural and remote areas these are in the area of emergency response and mental health. ACRRM has developed both Mental Health Level 1 and Level 2 accredited training online. As a priority area particularly for remote practices these were the first programs to be delivered using the virtual classroom facility. This enabled face-to-face equivalent training to doctors unable to attend in person workshops. ACRRM has in the past five years engaged in the development of a comprehensive advanced life support Emergency Medicine program which was delivered at the ACRRM annual conferences and now as a regular series of courses delivered at locations appropriate to Fellows and members. To further support educational opportunities in Emergency Medicine, ACRRM has assumed the responsibility for the ongoing development and delivery of the Rural Emergency Skills Training (REST) program which was developed by the Rural Workforce Agency Victoria (RWAV) in 2001 and which was successfully delivered by RWAV in both Victoria and Northern Territory since that time. The REST program is a two day emergency medicine life support course designed to meet the demand of International Medical Graduate (IMG’s) recruited to general practice and for general practitioners generally who wish to enhance or maintain their emergency medicine skills. ACRRM has also developed and delivered a one-day Rural Emergency Obstetrics Training (REOT) program that was delivered at the 2012 and 2013 ACRRM annual conference and will now be

https://www.acrrm.org.au/pdp


delivered on an ongoing basis in relevant locations. This course aims to provide non-obstetric healthcare practitioners with knowledge and skills to manage normal labour and other less common obstetric emergencies that may occur in a rural practice. To add to the suite of Emergency Medicine courses for rural practitioners ACRRM partnered with FujiFilm SonoSite Australasia Pty Ltd to create an Introduction to Ultrasound Use in Rural Emergency Medicine workshops. These workshops are aimed at the participant with little or no ultrasound experience - through to the more experienced requiring refresher courses. Again these workshops have been delivered at the ACRRM annual conference and are delivered on an ongoing basis in relevant locations.

9.1.4 The training organisation documents the recognised CPD activities of participants in a systematic and transparent way, and monitors participation.

ACRRM provides participants with 24 hour access to their PDP activity status via the College’s online education platform, RRMEO. Each participant’s accredited activities are collected in his or her online ‘Learning Planner’ which is organised accordingly to currently enrolled and completed activities with the number of allocated points clearly shown. All PDP participants receive unofficial activity statements annually. An unofficial statement can also be downloaded from the learning planner at any time as RRMEO automatically tracks compliance status for the current triennium. The activity statements present a complete list of all completed PDP activities undertaken during the current triennium, with the relevant PDP points listed according to category (Advanced Life Support, Extended or Core Skills). Participants’ status for MOPS is also listed, where applicable. The participants’ overall PDP status is presented clearly according to skills category and shows the number of points remaining in order to be considered PDP compliant for the triennium. All PDP records are monitored on an ongoing basis and formal end of triennium reports on participation are prepared for Medicare and RACGP (for its Fellows and members participating in ACRRM’s program) for VR purposes. ACRRM also monitors and reports at the end of the triennium on participation for MOPS purposes. As ACRRM Fellows must be PDP compliant each triennium; six months prior to the end of a triennium, all Fellows who have not provided sufficient evidence of CPD are identified as ‘at risk’. Each of these Fellows is contacted directly by the ACRRM PDP team to ascertain if there are special circumstances that may apply for their exemption and/or to offer assistance in identifying activities to assist them meeting compliance requirements. If 90 days prior to the end of the triennium ‘at risk’ Fellows have not shown progression they will be formally contacted by the Fellowship Services Manager and will again be offered assistance to meeting their requirements before the end of the triennium. At the end of the triennium those Fellows who have failed to provide sufficient certification of continuing professional development to comply with the requirements of ACRRM PDP will be identified by the PD Committee as “non-compliant” and will enter a three month period of remediation. (For further details see Section 9.1.6 below) A summary of data collected over the last Triennium and data to this point in the new Triennium is given below. Table 9.1, shows compliance figures for the 2011-2013 triennium as at the start of May. Table 9.2 shows this data for the newly commenced, current triennium (2014-2016) as at 2 June 2014.


2011-2013 Triennium Final Report Report as at 01 May 2014

1718 total registered for PDP 1634 enrolled in PDP with PDP learning planners

1377 are Fellows

excludes locked accounts Total courses accredited: 2906

2014-2016 Triennium Progress Report as at June 2014

Report as at 02 June 2014

1735 total registered for PDP 1690 enrolled in PDP with PDP learning planners

1439 are Fellows

excludes locked accounts Total courses accredited (as at Jun 2014): 495

Table 9.1: 2011-2013 Triennium Compliance:

Participants Category % Fellows (1377) ALS

Compliant 1310 95%

Uncompliant 20 1.5%

Exemption/ Extension 36 2.5%

Retired/ Other 2 0.5%

Will let FACRRM lapse 9 0.5%

Non-fellows (257) BLS

Compliant 229 89%

Uncompliant 1 1%

Exemption/ Extension 2 1%

No reporting requirement 25 9%

PRPD Compliant 1547 95%

Uncompliant 9 0.5%

Exemption/ Extension 39 2% Retired/ Other 5 .5%

No reporting requirement 25 1.5%

Will let FACRRM lapse 9 0.5 Core Compliant 1546 95%

Uncompliant 10 0.5%

Exemption/ Extension 39 2% Retired/ Other 5 0.5%

No reporting requirement 25 1.5%

Will let FACRRM lapse 9 0.5

Overall PDP Compliant or

reporting not needed

Uncompliant

Fellows 1305 24

Non-fellows 226 1


Exemption/ Extension 39

Retired/ Other 5 No reporting requirement 25

Will let FACRRM lapse 9

1609 (98.5%) 25 (1.5%)

Table 9.2: Enrolment by Reportable Advanced Skills Area (2011-2013 Triennium)

Table 9.3: 2014-2016 Triennium Compliance

Participants Category % Fellows (1439) ALS Compliant 107 7%

Uncompliant 1332 93%

Non-fellows (251) BLS

Compliant 16 6%

Uncompliant 235 94%

PRPD

Compliant 279 17%


Core

Compliant 43 3%


Overall PDP Compliant Uncompliant

Fellows 31 1408

Non-fellows 3 248

34 1656 Table 9.4: Enrolment by Advanced Skills Reportable Area (2014-2016 Triennium)

JCCMA JCCA CCDOG Radiology Mental Health (FPS CPD)

Enrolled 27 245 286 111 49 Compliant 25 233 262 80 36

Not Compliant 2 12 24 31 13

JCCMA JCCA CCDOG Radiology Mental Health (FPS CPD)

Enrolled 31 255 296 113 54

Compliant 0 33 35 3 0 Not Compliant 31 222 261 110 54


Course Accreditations: Total courses accredited for 2014-2016 to date is 495 Table 9.5: Courses Accredited for 2014-16

ALS BLS PRPD Core

65 20 212 220

JCCMA JCCA CCDOG Radiology Mental Health

(FPS CPD)

2 102 54 42 4

9.1.5 The training organisation has mechanisms to allow doctors who are not Fellows to access relevant continuing professional development and other educational opportunities.

PDP participation is open to all ACRRM members as part of their membership fee. The only difference in policy and procedures relating to Fellows and non-Fellows is that PDP participation is compulsory for all Fellows of ACRRM. ACRRM also offers a range of continuing education opportunities to doctors who are not ACRRM members. For example, ACRRM offers both Level 1 and Level 2 interactive Mental Health training online to all general practitioners, plus a certificate course in Dermatology and Tele-derm/Radiology to any rural or remote doctor. ACRRM also recognises that many of its members have multiple and various levels of recognition and certification requirements to maintain. So as not to overburden members, ACRRM has allowed for substantial cross-accreditation of educational programs with similar programs run by other medical colleges, JCCs and other educational providers.

9.1.6 The training organisation has processes to counsel Fellows who do not participate in ongoing professional development programs.

As previously stated, ACRRM has processes in place to monitor the activities of PDP participants and systems to advise participants of their progress. (See Section 9.1.4 above) As also stated six months prior to the end of the triennium participants who have not provided sufficient evidence of PDP participation will be identified as ‘at risk’. A list of ‘at risk’ members by state is compiled by staff for the Fellowship Services Manager. The Fellowship Services Manager contacts those on the list to establish if there are extenuating circumstances which are limiting the members’ capacity to participate in the program and to offer support and assistance to them. After consultation with the participant an action/support plan will be developed and staff will be requested to support the member in its implementation and monitor outcomes. If after a three month period progress is unsatisfactory counselling will be provided by the Chair of the PD Committee and/or Censor. If three months after the conclusion of the triennium no progress has been made the ACRRM remediation policy is activated. (For further detail on the ACRRM Remediation Policy; See Section 9.3)


9.2 Retraining

9.2.1 The training provider has processes to respond to requests for retraining of its Fellows.

The ACRRM retraining plan has been developed to support ACRRM Fellows who wish to return to active general practice following a prolonged absence. This policy relates to Fellows who have either identified themselves, or have been identified by a Regional Health Board, Medical Board or Medical Council, as requiring retraining. Fellows requiring retraining in order to return to safe general practice will be required to enrol in ACRRM’s retraining module on RRMEO. The retraining module contains self-directed learning activities specific to the individual’s retraining needs. The fellow will be required to develop a retraining Learning Plan with timelines and submit for approval to the Chair of the PD Committee. Approval of the plan is required prior to the fellow commencing the retraining program. If deemed necessary by the PD Chair a mentor will be assigned to support the Fellows’ progress. At an agreed review date, the fellow’s retraining outcomes will be assessed by the Professional Development Committee Chair. Fellows who have successfully completed retraining will continue their professional development with ACRRM via PDP participation. Those who fail to meet retraining requirements may be granted a retraining extension. Fellows who continue to fail to meet the requirements without valid reason will be referred to the Censor and may have their Fellowship of ACRRM suspended or rescinded.


9.3 Remediation

9.3.1 The training organisation has processes to respond to requests for remediation of its Fellows who have been identified as under performing in a particular area.

There has been no change to the remediation policy since initial accreditation in 2007. The Professional Development Committee have commenced a discussion on further refinement of the remediation policy for the 2014–2016 triennium to explore the question of revalidation and how this may be achieved for rural and remote Fellows. (See Attachment 9.3: Remediation Policy)



Recommendation TT: Introduce a requirement for a compulsory audit of procedures undertaken by Fellows on a regular basis (Standard 9.1.2 and 9.1.3)

The College is currently considering the practicality and utility of introducing mandatory audit for members as recommended by the AMC. The ACRRM process currently in place has been specifically assessed and approved by the AHPRA as acceptable evidence for their audit purposes. The College actually accredits educational activities before they are added to a Fellow’s or member’s learning planner the fundamentals of the learning activity are audited before they are assigned to the individual. Additionally a certain percentage of records assessed each triennium are audited. An actual audit of members’ records will not necessarily add value to these arrangements. The Professional Development Committee is however reviewing the process regularly and will introduce a different audit regime if deemed appropriate. If supported, the earliest that such a process could be practically introduced by the College would be the next PDP triennium (2017-19).



a. Outline how the College’s CPDP programs have developed since the 2010 AMC Accreditation.

The College has made a range of changes to its PDP for the 2011-13 triennium as outlined above, and also reviewed the program to ensure it was consistent with the MBA’s continuing professional development registration standard. In the 2011–2013 triennium the PDP’s second component was changed from ‘Extended Skills’ to ‘Planned Reflective Professional Development’ (PRPD). This component has always been mandatory for compliance and always comprised a major component of the ACRRM program (amounting to 30 points of the overall 100 required for compliance). Typically, the most common activities that have met this requirement had been Clinical Audits and Peer Review activities. The introduced changes added a more enriched set of activities that required planning and reflection to ensure proper learning outcomes. In the 2014-2016 triennium the PRPD was further refined. It was designated its current title and its activities were reviewed to ensure that ‘only’ those that required reflective learning were included. A standard Clinical Audit form has been designed to ensure the participants outline how the process has changed their practice, indicating the level of their reflection and learning. The PDP scheme was also adjusted to incorporate the new primary curriculum and the new domains. As outlined previously, forthcoming reviews of AST curricula will include development of defined statements of their respective CPD requirements. (See Section 3.1) Opportunities to continue to improve the quality and range of these types of activities will continue to be explored by the College. ACRRM will continue to monitor the MBA’s standards and make adjustments to the program each triennium as required.

b. An outline of plans for further development.

The new CRM customer interface that will be available through the College web portal is expected to provide additional assistance to Fellows and members and simplify the process of keeping track of their on-going CPD. This will be available in the second half of 2014. (For further details of the CRM system: See Section 4.2) At the same time the online learning material available to Fellows and members is growing and older material being reviewed to ensure it is still educationally and clinically relevant. In addition, the Professional Development Committee (informed by the ongoing program of Evaluation Reporting) will continue to review the nature of the PDP scheme to ensure it meets the needs of the participants and complies with the Australian Medical Board’s standards.


c. Please provide data showing the number and proportion of fellows participating in the College’s continuing professional development programs over the last three years.

For the 2011-2013 triennium; at the end of the recording period the College had 1377 Fellows participating in PDP; and 257 non-Fellows; and a combined total of 1634. These participants are all tracked for compliance purposes. There were also an additional 84 non-Fellows enrolled for record-keeping purposes only. This group tend to be enrolled in other programs (e.g. IMGs, registrars etc.).

d. Outline how the College’s retraining and remediation policies have developed since the 2010 AMC accreditation.

There were no changes to the retraining and remediation policies made in the 2011–2013 triennium. It was considered that the existing policies met the current needs. At the end of that triennium the PDP committee made slight amendments to the policies to refine the wording; and in particular updating references to the new online system which is in redesign. In the light of the MBA’s conversation on revalidation, the Professional Development Committee will review the models available in New Zealand and adopted by other Colleges to explore how this may be applied to Fellows and members who practice in rural and remote locations. It is assumed that this discussion will provide some background research that will enable the College to reach a policy position when the MBA releases it position paper, expected in the second half of this year. It is also planned to review the retraining policy once the new web based portal is available to fully utilise this new facility.

e. Any other planned developments that relate to Accreditation Standard 9.

The College is awaiting further detail of impending changes to its accreditation framework from the MBA and will respond accordingly.


Glossary of Terms ABFP American Board of Family Practice ACD Australasian College of Dermatology ACE Association of Chief Executives ACEM Australasian College of Emergency Medicine AMAC Australian Medical Acupuncture College ACRRM Australian College of Rural and Remote Medicine AGPT Australian General Practice Training AHS Aboriginal Health Services ALS Advanced Life Support AMC Australian Medical Council AMSA Australian Medical Students Association AMS Aboriginal Medical Services AST Advanced Specialised Training BLS Basic Life Support CAP Competent Authority Pathway CCDOG Conjoint Committee for the Diploma of Obstetrics and Gynaecology CCT Core Clinical Training CFPC Certification of the College Family Physicians Canada CFEP Client Focused Evaluation Program CPD Continuing Professional Development CPMC Council of Presidents of Medical Colleges CPMEC Confederation of Postgraduate Medical Education Councils CFPC College of Family Physicians of Canada COP Completion of Training CRM Customer Relationship Management (system) DRHMNZ Division of Rural Hospital Medicine New Zealand DRANZCOG Advanced Diploma of Royal Australian and New Zealand College of

Obstetricians and Gynaecologists DOHA Department of Health and Ageing ECTV External Clinical Teaching Visit FACRRM Fellowship of Australian College of Rural and Remote Medicine FARGP Fellowship Australian Rural General Practice FRAME Federation of Rural Medical Educators GEM Generalist Emergency Medicine GP General Practitioner GPET General Practice Education and Training GPRA General Practice Registrars Association GPMHSC General Practice Mental Health Standards Collaboration IMG International Medical Graduate IP Independent Pathway IT Information Technology JCC Joint Consultative Committee JCCA Joint Consultative Committee (Anaesthesia) JCCMA Joint Consultative Committee Medical Acupuncture JCU James Cook University JFPP John Flynn Placement Program MBA Medical Board of Australia MBQ Medical Board of Queensland MCQ Multiple Choice Question MiniCEX Mini Clinical Evaluation Exercise


MSOAP Medical Specialist Outreach Assistance Program MOPS Maintenance of Professional Standards MSF Multi Source Feedback NACCHO National Aboriginal Community-Controlled Health Organisation NeHTA National E–Health Transition Authority NRHSN National Rural Health Students Network OSCE Observed Structured Clinical Examination OTGP Overseas Trained General Practitioner PCEHR Personally Controlled Electronic Health Records PDP Professional Development Program PESCI Pre-Employment Structured Clinical Interview PGMEC Post Graduate Medical Education Committee PGPPP Prevocational General Practice Placements Program PGY Post Graduate Year PMC Procedural Medicine Collaboration PRRT Primary Rural and Remote Training PVT Prevocational Training Programs QA Quality Assurance QRGP Queensland Rural Generalist Program RACGP Royal Australian College of General Practitioners RACP Royal Australasian College of Physicians RACS Royal Australasian College of Surgeons RANZCP Royal Australian and New Zealand College of Psychiatrists RANZCOG Royal Australian and New Zealand College of Obstetricians and

Gynaecologists REOT Rural Emergency Obstetrics Course REST Rural Emergency Skills Training RDAA Rural Doctors Association of Australia RNZCGP Royal New Zealand College of General Practitioners RRMA Rural, Remote, Metropolitan Areas RPL Recognition of Prior Learning RRMEO Rural and Remote Medical Education Online RLO Registrar Liaison Officers RTP Regional Training Provider RVTS Remote Vocational Training Scheme RWAV Rural Workforce Agency Victoria StAMPS Structured Assessment using Multiple Patient Scenarios VPP Vocational Preparation Pathway VR Vocational Recognition WBA Workplace Based Assessment WAVE Western Australia Vocational Education

submission for accreditation australian medical council ... · 6. monitoring and evaluation ......

Documents