report to department of education and training...

REPORT TO

DEPARTMENT OF EDUCATION AND TRAINING

8 JUNE 2017

NEW COLOMBO PLAN

INDEPENDENT EVALUATION FINAL STAGE 3 REPORT

ACIL ALLEN CONSULTING PTY LTD

ABN 68 102 652 148

LEVEL FIFTEEN

127 CREEK STREET

BRISBANE QLD 4000

AUSTRALIA

T+61 7 3009 8700

F+61 7 3009 8799

LEVEL ONE

15 LONDON CIRCUIT

CANBERRA ACT 2600

AUSTRALIA

T+61 2 6103 8200

F+61 2 6103 8233

LEVEL NINE

60 COLLINS STREET

MELBOURNE VIC 3000

AUSTRALIA

T+61 3 8650 6000

F+61 3 9654 6363

LEVEL ONE

50 PITT STREET

SYDNEY NSW 2000

AUSTRALIA

T+61 2 8272 5100

F+61 2 9247 2455

LEVEL TWELVE, BGC CENTRE

28 THE ESPLANADE

PERTH WA 6000

AUSTRALIA

T+61 8 9449 9600

F+61 8 9322 3955

161 WAKEFIELD STREET

ADELAIDE SA 5000

AUSTRALIA

T +61 8 8122 4965

ACILALLEN.COM.AU

RELIANCE AND DISCLAIMER THE PROFESSIONAL ANALYSIS AND ADVICE IN THIS REPORT HAS BEEN PREPARED BY ACIL ALLEN CONSULTING FOR

THE EXCLUSIVE USE OF THE PARTY OR PARTIES TO WHOM IT IS ADDRESSED (THE ADDRESSEE) AND FOR THE PURPOSES SPECIFIED IN IT. THIS

REPORT IS SUPPLIED IN GOOD FAITH AND REFLECTS THE KNOWLEDGE, EXPERTISE AND EXPERIENCE OF THE CONSULTANTS INVOLVED. THE

REPORT MUST NOT BE PUBLISHED, QUOTED OR DISSEMINATED TO ANY OTHER PARTY WITHOUT ACIL ALLEN CONSULTING’S PRIOR WRITTEN

CONSENT. ACIL ALLEN CONSULTING ACCEPTS NO RESPONSIBILITY WHATSOEVER FOR ANY LOSS OCCASIONED BY ANY PERSON ACTING OR

REFRAINING FROM ACTION AS A RESULT OF RELIANCE ON THE REPORT, OTHER THAN THE ADDRESSEE.

IN CONDUCTING THE ANALYSIS IN THIS REPORT ACIL ALLEN CONSULTING HAS ENDEAVOURED TO USE WHAT IT CONSIDERS IS THE BEST

INFORMATION AVAILABLE AT THE DATE OF PUBLICATION, INCLUDING INFORMATION SUPPLIED BY THE ADDRESSEE. UNLESS STATED OTHERWISE,

ACIL ALLEN CONSULTING DOES NOT WARRANT THE ACCURACY OF ANY FORECAST OR PROJECTION IN THE REPORT. ALTHOUGH ACIL ALLEN

CONSULTING EXERCISES REASONABLE CARE WHEN MAKING FORECASTS OR PROJECTIONS, FACTORS IN THE PROCESS, SUCH AS FUTURE MARKET

BEHAVIOUR, ARE INHERENTLY UNCERTAIN AND CANNOT BE FORECAST OR PROJECTED RELIABLY.

ACIL ALLEN CONSULTING SHALL NOT BE LIABLE IN RESPECT OF ANY CLAIM ARISING OUT OF THE FAILURE OF A CLIENT INVESTMENT TO PERFORM

TO THE ADVANTAGE OF THE CLIENT OR TO THE ADVANTAGE OF THE CLIENT TO THE DEGREE SUGGESTED OR ASSUMED IN ANY ADVICE OR

FORECAST GIVEN BY ACIL ALLEN CONSULTING.

© ACIL ALLEN CONSULTING 2017

C O N T E N T S

EXECUTIVE SUMMARY I

1

Introduction 1 2.1 Context 1 2.2 This report 1 2.3 Report structure 3

2

Design elements 4 3.1 Overview NCP’s design elements 4 3.2 Nomination and application processes 5 3.3 Eligibility criteria 5 3.4 Guidelines and guidance 8 3.5 Funding arrangements 10 3.6 Internships and mentorships 11 3.7 Short term vs semester based programs 12 3.8 Alumni and post-NCP student activities 13 3.9 Summary of stakeholder feedback 16

3

Administrative arrangements 19 4.1 Overview of NCP’s administrative arrangements 19 4.2 Timeframes for nominations and applications 19 4.3 Online application and management system 21 4.4 Communication between Government and universities 23 4.5 Responsiveness to university feedback 23 4.6 Summary of stakeholder feedback 24

4

Impacts 25 5.1 Impact on students 25 5.2 Impact on Australian universities 30 5.3 Impact on host locations 33

5

Conclusion 34 6.1 Overall summary 34 6.2 Response to Stage 3 evaluation questions 36

A

Stakeholders consulted A–1 A.1 Australian universities consulted A–1 A.2 Host location consultations A–2 A.3 Student workshops A–2

C O N T E N T S

B

Consultation questions 3 B.1 Home university consultation questions 3 B.2 Host location consultation questions 4 B.3 Student consultation questions 5

FIGURES FIGURE 2.1 LOGIC OF THE NCP EVALUATION—STAGES 1, 2 & 3 2 FIGURE 3.1 MOST USEFUL ELEMENTS OF THE ALUMNI PROGRAM 14 FIGURE 3.2 PROFESSIONAL DEVELOMENT AS PART OF THE

ALUMNI PROGRAM 14 FIGURE 3.3 PARTICIPATION IN ALUMNI ACTIVITIES 15 FIGURE 3.4 MOST VALUED USES OF AUSTRALIAN GLOBAL ALUMNI

NETWORK 15 FIGURE 3.5 ALUMNI PROMOTING THE NCP 16 FIGURE 5.1 SKILLS GAINED 26 FIGURE 5.2 IMPACT ON SCHOLARSHIP STUDENTS 27 FIGURE 5.3 DEVELOPMENT OF CONTACTS/LINKS 27 FIGURE 5.4 INFLUENCE ON RECIPIENTS CAREER 28 FIGURE 5.5 INFLUENCE ON RECIPIENTS FUTURE PLANS 29

TABLES TABLE 3.1 RELEASE DATE OF PROGRAM GUIDELINES SINCE THE

PILOT PHASE 8 TABLE 3.2 LENGTH OF PROGRAM GUIDELINES—MOBILITY AND

SCHOLARSHIP 10 TABLE 3.3 SUMMARY OF UNIVERSITY AND STUDENT FEEDBACK

ABOUT NCP’S DEISGN FEATURES 17 TABLE 4.1 SUMMARY OF UNIVERSITY FEEDBACK ABOUT NCP’S

ADMINISTRATIVE ARRANGEMENTS 24 TABLE 5.1 BENEFITS OF THE NCP EXPERIENCE FOR

PARTICIPANTS 25 TABLE 5.2 EXAMPLES OF OUTBOUND MOBILITY MENTIONED IN

UNIVERSITY STRATEGIC PLANS 32 TABLE A.1 CONSULTATIONS WITH AUSTRALIAN UNIVERSITIES

FOR STAGE 3 A–1

BOXES BOX 3.1 FUNDING AVAILABLE FOR THE SCHOLARSHIP AND

MOBILITY PROGRAMS 10 BOX 4.1 STUDENT DETAILS AND OTHER INFORMATION

REQUIRED REUQIRED BY NCP 21 BOX 5.1 INTERNATIONAL EXCHANGES AT AUSTRALIAN

NATIONAL UNIVERITY 31 BOX 5.2 INTERNATIONAL ENGAGEMENT AT CURTIN

UNIVERSITY 31 BOX 5.3 GRIFFITH UNIVERSITY 32

NEW COLOMBO PLAN INDEPENDENT EVALUATION i

E X E C U T I V E S U M M A R Y

Background and context

In early-2014 ACIL Allen was engaged by the Departments of Education (the Department) and Foreign Affairs and Trade (DFAT) to undertake an independent evaluation of the recently established New Colombo Plan (NCP). The evaluation was intended to provide early and ongoing feedback on the NCP’s implementation, with a particular focus on the efficiency, effectiveness and appropriateness of the Mobility and Scholarship pilot programs. The pilot programs offered approximately 1,300 Australian undergraduate Mobility students and 40 Australian undergraduate Scholarship students with opportunities to study in Hong Kong SAR, Indonesia, Japan and Singapore throughout 2014-2015.

Since that time, ACIL Allen has undertaken a number of evaluation activities and delivered two final reports for the consideration of Government. These activities and reports have informed the ongoing implementation and continued expansion of NCP, which now provides opportunities for Australian students to study in some 40 countries across the Indo-Pacific Region.

This report presents the main findings associated with the third and final stage of the evaluation. Each stage of the evaluation has followed the student journey:

— from application through to pre-departure during the pilot phase (Stage 1)

— in-country NCP experience (Stage 2)

— post-NCP experience and return (Stage 3, this report).

The logic of the evaluation is depicted in Figure 2.1 in the body of the report.

Due to the three year length of the evaluation and the number of refinements made to the NCP since 2014, the focus of this report is broader than the pilot programs. In particular, some aspects of the report are intentionally forward looking (i.e. whether NCP is appropriately designed to meet its ongoing implementation, expansion and how best to engage the alumni), and not just confined to questions about the efficiency, effectiveness and appropriateness of the pilot programs. As such, the report draws on the lessons and insights of the past while providing analysis useful to future decision making.

Overall summary

The overall feedback from the participating students and home university staff, was overwhelmingly supportive of the design and administration of the NCP, and of the responsiveness of NCP to addressing stakeholder feedback year on year. The body of the report documents these findings extensively.

NEW COLOMBO PLAN INDEPENDENT EVALUATION ii

Given this, and noting the ongoing efforts towards improving the NCP, the executive summary focuses mainly on residual issues raised about the NCP’s design and administration, and on insights and analysis that may assist to address these.

Design elements

The evaluation focused on seven design elements of the NCP—nomination and application processes; eligibility criteria; guidelines and guidance; funding arrangements; internships and mentorships; short term vs semester based programs; and alumni support. Universities positively viewed the design elements of the Mobility program and the refinements that have been made following the pilot, such as relaxing the age restrictions for applicants. Universities were similarly positive about the Scholarship program’s design elements, and felt they remained an appropriate way of meeting policy objectives and supported the continued roll out of the NCP.

Specific matters that require further consideration are:

— Communication of the NCP expectations and decisions is vital to stakeholders. It was widely noted that improvements have been made in the information for applicants, including to the FAQ, and that the program guidelines are generally clear and comprehensive. However, for some universities it is important to be mindful of the timeliness of changing the Program Guidelines, and avoid increasing the length or complexity of the Program Guidelines. Relatedly, universities are seeking more individualised feedback on unsuccessful applications, both to support students through the process and to retain the interest of academics that invest considerable effort in developing and ranking the applications.

— While most universities are comfortable with NCP’s eligibility and selection criteria, for some regional universities with high numbers of mature students, the restrictions on age are reported to be having an undesirable discriminatory effect. Indeed, for universities with certain student demographics and course offerings (such as dual sector universities), relaxing the age requirement may boost and broaden their participation in NCP.

— Funding arrangements are generally thought to be fit for purpose, yet too generous for some locations where Scholars undertake their studies, internships or mentorships. An opportunity exists to tailor funding support to better reflect the living costs in different host locations.

— Universities and students are seeking a balance to be maintained between short term and semester based programs. Universities appreciate the flexibility afforded to them in selecting programs that meet the preferences of their students and the types of programs faculties wish to pursue.

— Arrangements to support Mobility students to engage in an internship or mentorship could be further improved, particularly where universities do not have institutional relationships to leverage opportunities for students, nor access to the internship/mentorship portal to assist students to identify opportunities. Further, for those universities that source specialised advice about these opportunities, the fee for service reduces the financial aid available to students decreasing if not completely negating the attractiveness of the program.

— Universities appear to be doing little to formally monitor the activities of, and hence impact on, returning participants. While the NPC Secretariat is developing opportunities to maintain professional connections and engage alumni in advocacy and promotional activities (which students report is well designed), there appears to be some disconnect between the universities alumni activities and those of NCP. Enhanced linkages between the NCP alumni activities and those of the universities could assist Government to better meet its NCP alumni objectives.

Administrative arrangements

The evaluation focused on four administrative elements of the NCP—timeframes for nomination and applications; the online application and management system; communications between students, participating universities and government; and the program’s responsiveness to feedback.

Feedback was particularly positive about the Government’s openness to feedback and the willingness of program administrators to improve all aspects of program delivery. The only two administrative arrangements identified for further attention are:

NEW COLOMBO PLAN INDEPENDENT EVALUATION iii

— The administrative functionality of the online application and management system could be improved. Participating Australian universities identified two main concerns—first, that managing input of data about students participating in the Mobility program is difficult and time consuming, particularly for information that is unknown before departure or often changing, such as destination phone numbers; and second, streamlining administrative processes, such as the introduction of electronic signatures and the ability to leave queries on specific fields, would improve the functionality of the system for administrators.

— The flow of information back to universities and students could be improved. Participating Australian universities highlighted both students’ and academics’ desire for feedback about decisions on individual applications. While acknowledging the effort that this may involve, universities and students contended that this flow of information is key to maintaining high levels of support and ongoing engagement in the NCP.

Impacts

Many of the NCP’s impacts are long term in nature and may not be fully evident for some time. In the short to medium term, perhaps the best indicators of impact are based on the level of Mobility activity generated and the visible changes that have been observed in and by the students participating. These include:

— the provision of relevant work experience and cementing of career goals through a placement opportunity that is not otherwise as readily available in undergraduate study

— the gaining of independence and confidence in working with co munities and professionals through exposure to officials and organisations as part of Mobility projects

— the raising of interest in developing countries and other cultures

— the promotion of overseas travel, to locations that may not have been the first port of destination, and for some at all

Impacts of the NCP were also two-way, with benefits for host location students and organisations also observed. These benefits included a transfer of skills and culture from Australian students to host location students/organisations through community activities such as fundraising and volunteering.

From the perspective of universities, NCP has delivered a myriad of benefits which are there for all to see, but which are also difficult to quantify across the sector. All universities consulted were confident in their assessment about the benefits to students arising from NCP. However the universities consulted for this report are not tracking how NCP benefits a student’s academic and professional progress, but believe they could be. All universities consulted were also confident that NCP has made a lasting impact on the relationships formed between Australian universities and host location institutions. For these universities consulted, the type or scope of the relationship was not an important benefit observed, it’s the fact that NCP provides an incentive for academic faculty members to explore and secure relationships based on teaching activities (which also have benefits to research activities) that is the outstanding benefit from NCP.

The Stage 2 evaluation reported survey data that suggests host locations are highly supportive of NCP and its objectives, and believe that it is well designed. The Stage 3 evaluation has not identified any evidence which contradict these findings.

Looking to the future

The evidence and findings from this evaluation suggest that NCP is well positioned for the future.

For the overwhelming majority of universities consulted, NCP has either met or exceeded their expectations since the 2014 pilot phase (where some early teething issues were identified by stakeholders). These expectations have been shaped by the willingness of Government’s program administrators to improve the way NCP is delivered, and in key decisions to expand the number of eligible host locations and to relax restrictions in facing participation.

In 2017, many stakeholders believe that NCP is suitable for being scaled up over time, and have confidence in Government’s ability to manage such growth. However, any further expansion of NCP should be undertaken in close consultation with universities so as to optimise future rollout.

NEW COLOMBO PLAN INDEPENDENT EVALUATION 1

2 I N T R O D U C T I O N

1

Introduction

2.1 Context

In early-2014 ACIL Allen was engaged by the Departments of Education (the Department) and Foreign Affairs and Trade (DFAT) to undertake an independent evaluation of the recently established New Colombo Plan (NCP). The evaluation was intended to provide early and ongoing feedback on the NCP’s implementation, with a particular focus on the efficiency, effectiveness and appropriateness of the Mobility and Scholarship pilot programs. The pilot programs offered approximately 1,300 Australian undergraduate Mobility students and 40 Australian undergraduate Scholarship students with opportunities to study in Hong Kong SAR, Indonesia, Japan and Singapore throughout 2014-2015.

Since that time, ACIL Allen has undertaken a number of evaluation activities and delivered two reports for the consideration of Government. These activities and reports have informed the ongoing implementation and continued expansion of NCP, which now provides opportunities for Australian students to study in some 40 countries across the Indo Pacific Region.

2.2 This report

This report presents the main findings associated with the third and final stage of the evaluation. Each stage of the evaluation has followed the student journey:

— from application through to pre-departure during the pilot phase (Stage 1)

— in-country NCP experience (Stage 2)

— post-NCP experience and return (Stage 3).

The logic of the evaluation is depicted in Figure 2.1.

Due to the extended frames of this evaluation and the number of refinements made by Government to the NCP since 2014, the focus of this report is broader than the pilot programs. Some aspects of this report are intentionally forward looking (i.e. whether NCP is appropriately designed to meet its ongoing implementation, expansion and how best to engage the alumni), and not just confined to questions about the efficiency, effectiveness of the pilot programs. This approach draws on the lessons and insights of the past while providing Government with analysis that is meaningful and useful to future decision making.


FIGURE 2.1 LOGIC OF THE NCP EVALUATION—STAGES 1, 2 & 3

Note: Builds on similar a figure presented in the Stage 2 Final Report.

SOURCE: ACIL ALLEN

As Figure 2.1 shows, the NCP rollout has experienced substantial growth since its inception in 2014. Within this context, the report identifies some of residual issues which may impact on NPC’s continued roll out and possible future expansion.

The key research question that have informed the evaluation are outlined below. These key questions are an adaptation of the evaluation questions originally outlined in a strategy document developed for the overall evaluation during early-2014.

1. To what extent do the Scholars/Mobility grant recipients perceive their NCP experience enhanced their studies, employment prospects and understanding/connections to the Indo Pacific region?

a) Mobility-focused question—To what extent have universities enriched their existing relationships and or/created new relationships or partnerships in host locations from short and semester long Mobility projects?

b) Scholarship-focused question—To what extent have universities enriched their existing relationships and or/created new relationships or partnerships in host locations from the Scholarship Pilot Program?

2. What longer term benefits and challenges have participating Australian universities experienced through their involvement in the NCP?


a) Mobility-focused question— What longer term benefits and challenges have Australian universities experienced through their approach to developing short and semester long Mobility projects?

3. What longer term benefits and challenges have participating host universities (in the pilot locations) experienced through their involvement in the NCP?

4. What longer term benefits and challenges have participating businesses observed through their involvement in the NCP?

5. To what extent have Scholars/Mobility grant recipients participated in community development/advocacy activities on their return to Australia?

To address the Stage 3 evaluation questions the following evaluation activities were undertaken by ACIL Allen:

— Consultations with Australian universities and a small selection of NCP students.

― Consultations were undertaken with NCP contact officers from 22 universities for Stage 3. To ensure continuity in the consultation process, many of the contact officers consulted during previous stages of the evaluation were consulted for this stage of the work.

― Consultations with a small selection of students. Focus group style discussions were held between ACIL Allen and NCP students at the Deakin University and the University of Melbourne.

― Attempts were made to re-consult with the Stage 2 representative sample of Mobility and Scholarship host location partners (including businesses, universities, not-for-profit organisations who hosted students) but the overwhelming feedback from host location stakeholders is that they provided detailed feedback during the Stage 2 evaluation and had few additional insights for this evaluation report.

— Analysis of DFAT’s alumni survey of NCP participants (2015). This survey included questions directly relevant to the evaluation and captured information from approximately 197 of NCP students who had completed their programs.

— Analysis of student completion reports and other documentation held by Government.

— Comparison of feedback and information gained during Stage 3 to the information collected during stages 1 and 2 of the evaluation.

2.3 Report structure

The report is structured as follows:

— Chapter 2 considers the NCP’s core design features and any potential implications for continued roll out of NCP. This chapter draws heavily on data collected during consultations with NCP contact offices of Australian universities, as well as survey data from NCP students.

— Chapter 3 considers the efficiency, effectiveness and appropriateness of NCPs administrative arrangements during and after the pilot programs. This chapter also draws heavily on the data collected during consultations with NCP contact officers of Australian universities, and the other consultations undertaken for Stage 3.

— Chapter 4 discusses some of the impacts (i.e. benefits and challenges) associated with participation in NCP, as considered from the perspectives of selected stakeholders.

— Chapter 5 summarises the outcomes of Stage 3 against the key evaluation questions set for this stage.


3 D E S I G N E L E M E N T S

2 Design elements

This chapter examines the design features of the NCP, while the next chapter addresses its administration arrangements. The focus of each is to provide insights into the NCP’s strengths and challenges, and for enhancing or maintaining those arrangements which are most highly valued by students, universities and host partners.

3.1 Overview NCP’s design elements

The Stage 1 evaluation report considered the suitability of NCP’s core design features, identified as:

— Nomination and application processes.

― The Mobility Pilot Program comprises a single application process, whereby universities would assemble and submit a consolidated application for funding support for Mobility projects across faculties, academic teaching departments and schools.

― The Scholarship Pilot Program comprises a nomination and an application process, whereby the universities nominate a selection of students who proceed to the application phase once they have been deemed to meet NCP’s eligibility criteria.

— Eligibility criteria, which include age criteria, study and enrolment criteria, residency criteria amongst other criteria and the number of host locations eligible and participating in NCP.

— Guidance and guidelines to home universities and students.

— The funding model used to support universities and students which/who participate the Mobility and Scholarship pilot programs.

— Internships and mentorships for students who are in-country and seeking to expend their NCP experiences beyond the classroom.

— Program duration, namely the distinction between short term and semester based study options for students.

— Alumni and post-NCP activities with students and scholars.

Since the NCP’s implementation in early 2014, the Departments of Education and Foreign Affairs and Trade have made ongoing refinements to these design features. Within this context, the following sections provide consolidated feedback from consultations with 23 universities which participated in Stage 1 of the evaluation. A very small number of universities which participated in the Stage 1 evaluation chose either not to participate or felt their knowledge and insight of NCP was not sufficient to provide feedback for this Stage 3 evaluation report.

Note also that many of the comments reported relate to NCP more generally, as opposed to the specific programs. Where a comment is pertinent to Scholarships or Mobility it is identified in the text below.


3.2 Nomination and application processes

3.2.1 Scholarship

Feedback from universities suggest an overwhelming level of support for the design of the nomination and application process for the Scholarship program.

Since the pilot phase, all universities consulted have introduced and embedded internal processes that support the efficient identification of potential nominees across faculties, and centralised decision making processes to arrive at a final list of nominees. However, the internal processes used by universities for identifying scholars varies widely. That is, some universities allow faculties to take a stronger role in the nomination process, while others have centralised processes (often overseen by a high level nominations committee. The majority of NCP contact officers interviewed valued the autonomy afforded to universities to nominate their most prospective candidates for a Scholarship.

The use of the dual nomination and application process provided universities the ability to identify students who not only met the academic requirements of NCP, but also demonstrated the values and reputation universities were seeking to demonstrate. These universities believe that such autonomy outweighs the internal administrative burden required to support an internal nomination process.

The autonomy to nominate students also provides an incentive for universities to support each nominee throughout the application process. A large number of the universities consulted for Stage 3 provided students with a combination of written feedback on applications, coaching for the interview process, and support in securing internship and mentorship opportunities while in-country.

Many of the universities consulted preferred this approach to the alternative of an open and direct application process for individual students, as it gave them greater control over the calibre of applicants. This is an important aspect of NCP’s design, with universities required to ensure that applicants participate in credit bearing activities that contribute to the progression of their degrees.

At least half of the universities consulted, however, expressed concerns about the level of communication following the submission of a nominee’s application. Most universities seek to support students who have been shortlisted, but some reported losing track of their students as they progressed through the application stage. Email notification from the NCP Secretariat at the same time a student is notified about their progress would remedy this.

3.2.2 Mobility

The design of the Mobility program’s application processes were also viewed positively by the universities consulted. Significant improvements to the internal application development processes of universities have occurred since the pilots, and most universities report having timetabled and systematic internal processes for developing applications across faculties.

An area of concern raised by universities was the provision of tailored feedback by the Department of Education on unsuccessful applications. These concerns were at two levels.

First, in many universities, high level review and ranking committees have been established to increase the level of rigour and strategic oversight of applications. As such, decisions which overturn the project rankings in each application have the potential to undermine the investment and ongoing support for NCP at senior levels within universities.

Second, universities noted that some faculty members who submitted unsuccessful applications and did not receive an explanation were losing enthusiasm for the NCP. Consultations consistently noted situations where faculty members had sought feedback on project proposals, but the NCP contact officers were not in a position to provide any response.

3.3 Eligibility criteria

While a number of universities (at least half consulted) were generally supportive of the eligibility criteria and see them as entirely appropriate design elements, this area has remained a source of concern for more than half of the universities consulted during Stage 3 of this evaluation.


Feedback relating to the most frequently discussed criteria are outlined below.

3.3.1 Age criteria

Section 3.1 of the 2018 Scholarship guidelines state that applicants must be:

… between 18 and 28 years of age (inclusive) on 1 January 2017.

While Section 3.2.1 of the 2018 Mobility guidelines state:

At least 70 per cent of Students participating in any given Mobility Project must be aged 18 to 28

inclusive at the commencement of their Mobility Project. Indigenous students aged over 28 are not

subject to this requirement, and for the purposes of calculating the 70 per cent, are to be counted as

students aged under 28. Universities having difficulty filling places on Mobility Projects due to age

requirements may make a case to the Department of Education of Training for an exemption prior to

projects commencing. These will be considered on a case-by-case basis.

University feedback about the age requirements of NCP fell into the following themes.

First, there was broad consensus that relaxing of the age requirement for Mobility has been a positive change to the program. For example, stakeholders welcomed changes since the 2014 pilots that allowed 10 per cent of students to be above the 28 year age limit (in 2015) and 30 per cent of students on a project to be above 28 years (in 2016) and even further relaxation in 2017 which removed age barriers on a case-by-case basis.

Second, while the relaxation of age requirements has been well-received by the sector there remain approximately 50 per cent of the universities consulted which viewed the requirement as unnecessary. These universities generally, but not exclusively, had the following characteristics:

— Dual sector responsibilities. These universities typically have students who are enrolled in Associate Degrees in academic disciplines which align with a standard mobility experience. For example, one university consulted identified that its Asian language studies courses were offered as Associate Degrees and Diplomas, and as a consequence older students who were enrolled in these courses were ineligible to participate in NCP.

— Regional universities with proportionally larger mature age student populations. Regional universities make up approximately 18 per cent of Australia’s student enrolments1 and are typically characterised by student populations that are above the age of 30 years. As a consequence, a large number of potentially worthy applicants are excluded from NCP. For example, one university consulted suggested that its humanities degrees are “filled mostly with students above 28 years and up to 90 per cent of students is some humanities courses are ineligible to undertake an NCP Mobility project.”

The universities consulted which demonstrated these characteristics sought for there to be no age restriction.

Third, most universities consulted were silent on the age restriction for Scholarships. This is principally because Scholarships are offered to a small number students and universities rarely experience difficulties in identifying an appropriate number of suitable candidates for nomination that meet the age criterion.

3.3.2 Student criteria (enrolment and study criteria)

Section 3.2 of the 2018 Mobility guidelines states that students must meet the following citizenship, enrolment and study criteria to participate in NCP:

– be an Australian citizen. Students cannot receive a Student Grant to undertake a Mobility Project in

a Host Location in which they have dual citizenship or residency rights;

– be enrolled in an on-shore campus of an Australian University throughout the duration of the

Mobility Project;

– be undertaking a Bachelor Degree (Level 7) or Bachelor Honours Degree (Level 8), as defined by

the Australian Qualifications Framework at the commencement of their Mobility Project;

– …. not have already commenced a period of overseas study in the same Host Location as the

Mobility Project; and

1 DET 2016, ‘2016 First half year student statistics, Summary of Student Numbers’, https://docs.education.gov.au/node/43241


– not receive more than one short-term and one Semester grant (with or without an internship) during

their Australian course of study.

These criteria are mirrored in Section 3.1 of the Scholarship guidelines which set out the eligibility of nominees:

– be an Australian citizen. Students cannot receive a Student Grant to undertake a Mobility Project in

a Host Location in which they have dual citizenship or residency rights;

– be enrolled in an on-shore campus of an Australian University throughout the duration of the

Mobility Project;

– be undertaking a Bachelor Degree (Level 7) or Bachelor Honours Degree (Level 8), as defined by

the Australian Qualifications Framework at the commencement of their Mobility Project;

– …. not have already commenced a period of overseas study in the same Host Location as the

Mobility Project; and

– not receive more than one short-term and one Semester grant (with or without an internship) during

their Australian course of study.

These criteria have remained largely unchanged since the introduction of NCP’s pilots in 2014.

For the majority of universities consulted these criteria were neither contentious nor presented challenges to their participation in NCP. Most universities consulted reported a level of support for the criteria which limit participation to Australian citizens, who are enrolled in an onshore campus and undertaking an undergraduate level degree. Restricting students to one short term and one semester based Mobility grant was also acknowledged as providing opportunities for a large number of students to experience the benefits of both short and long term Mobility grants and ensures equity in the distribution of grants across the student population. At the same time it was noted that it was important for short term Mobility grantees, once exposed the personal and career benefits, to have the opportunity for at least one further, and potentially longer, experience.

A small number of universities consulted expressed long held concerns about the undergraduate criterion. These universities are typically dual sector institutions or institutions with a large number of mature age students enrolled in Associate Degrees, Graduate Diplomas and professionally-focused postgraduate degrees. These universities hold strong views that such criterion are unnecessarily and detrimentally restrictive.

3.3.3 Host locations

The pilot phase offered participants in the Scholarship and Mobility programs opportunities to study in four locations: Hong Kong SAR; Singapore; Indonesia and Japan. Feedback gained from universities during the pilot phase recognised that while the selection of these countries was a practical starting point for NCP, many universities did not have existing relationships with these host locations at that time. The process for establishing new formal study based relationships with host location partners was considered by many universities to be beyond the timeframes of the pilot phase applications.

Since 2015, the number of eligible host locations has expanded rapidly. Today, the 2018 program guidelines identify some 40 eligible host locations across the Indo-Pacific (see Section 3.2 (Scholarship) and Section 3.4 (Mobility)). The overwhelming majority of universities consulted for Stage 3 supported the decision to expand the number of host locations eligible under the NCP. Expansion provided opportunities for universities to leverage a wide-range of relationships between faculty members and hosts, and to explore new relationships when opportunities were presented to academic faculty. To quote a number of NCP contact officers:

Expansion have been a great development for NCP. It allowed our university to deepen its relationships

at the faculty level with a broad range of existing teaching and research partners. This would not have

happened otherwise.

Capital city based university.

Expansion has been welcome. It has given faculty members considerable choice.

Group of Eight university.

Expansion has been great for our university. The university was founded on Asian studies and has

many relationships across the Indo-Pacific that can now be built on in every way imaginable.


Innovative Research Universities Australia university.

Pilots were far too narrow for our purposes. The university did not have established relationships in any

of the host locations offered under the pilots. The expansion gives the university access to China where

many of the existing relationships are. Some faculty members have actively pursued their Chinese

contacts and been effective in establishing new programs for our students. Expansion has only

delivered benefits to the university.

Regional Universities Network university.

Two universities, by contrast, raised some concerns about the expansion into host locations where the personal safety of students was in question. For example, one university reported that private security guards were required in a host location for students travelling between the host institution and their accommodation. In addition to the safety issue, the private security was both expensive and considered to be not adequately funded by NCP.

The program guidelines state clearly, however, that the responsibility for identifying and then managing the risks associated with an in-country experience rest with the students and their institutions. Section 3.2 for Scholarship and Section 3.4 for Mobility outline the basis upon which students can travel to specific host-locations:

Scholars must not travel to a Host Location or region within a Host Location for which the Australian

Government's Smartraveller travel advice recommends 'Do not travel' (see www.smartraveller.gov.au).

Host Locations for which the Smartraveller travel advice recommends 'Reconsider your need to travel'

are eligible, but applicants should inform themselves carefully of the risks and think seriously about the

need to travel to that country or region before applying. Applicants must acknowledge in their

Scholarship agreement (see Section 6) that they have considered the risks carefully and investigated

safety precautions before deciding to travel.

3.4 Guidelines and guidance

3.4.1 Release of guidelines

NCP’s guidelines (for Mobility and Scholarship) outline the process for selecting students for each round, and the conditions of the scholarships and Mobility grants. Applicants are encouraged to read the guidelines in conjunction with the ‘Advice to Applicants’ published on the NCP website (see below).

The timing of the release of the guidelines attracted considerable attention during discussions with universities for Stage 3 of this evaluation, particularly in relation to their calendar proximity to other major international events that involve many NCP contact officers. The commonly held view was that universities would benefit from advance warning of changes in the guidelines, and the guidelines themselves, so they can communicate with faculty members and start preparing internal stakeholders for the impending round.

The release data of program guidelines following the pilot phase is presented in Table 3.1 below.

TABLE 3.1 RELEASE DATE OF PROGRAM GUIDELINES SINCE THE PILOT PHASE

Round (year) Release date

2015 round August 2014

2016 round March 2015

2017 round March 2016

2018 round January 2017

SOURCE: NCP WEBSITE, ‘PREVIOUS ROUNDS’, HTTP://DFAT.GOV.AU/PEOPLE-TO-PEOPLE/NEW-COLOMBO-PLAN/SCHOLARSHIP-PROGRAM/PAGES/PREVIOUS-ROUNDS.ASPX

Table 3.1 shows that with the exception of the 2015 round, program guidelines are released within the first three months of each calendar year. Feedback from consultations consistently suggested that universities are seeking as much advanced warning as possible to help prepare internal stakeholders for nominations and applications. Such lead time is considered critical for coordinating the application


process between the key internal university stakeholders who often have conflicting teaching and research commitments. The change to the release date for the 2018 round guidelines directly addresses this feedback.

3.4.2 Advice to applicants and FAQs

Since the 2017 application round (i.e. conducted in early 2016), the NCP website has included a section that offers ‘Advice to Applicants’. The section replaces the Frequently Asked Questions (FAQs) (used in the 2015 and 2016 rounds) with more direct advice to applicants about critical changes/components of the NCP, and typically provides two types of advice to applicants:

— ‘Key changes’ in the round—identifies the key enhancements that have been made to the program guidelines and the application and assessment processes that support the guidelines.

— ‘Lessons learned from previous rounds to assist’ applicants—provides generic advice to assist universities in preparing applications. It clarifies the key eligibility and assessment criteria as well as examples of how successful applicants met individual criterion to a high level.

While this section of the website does not fully address all of the concerns noted earlier regarding feedback on unsuccessful applications, it did receive considerable praise from universities during consultations. A number of NCP contract officers noted the clarity and timeliness in which the information is provided, for example:

Changes in guidelines are well-managed and clearly communicated. The department does a good job at

this, which continues to improve as the NCP rolls out.

Regional non-aligned university.

Process for changing guidelines is well-communicated. I don’t have any cause to complain.


Guidelines are fine and process for updating and releasing them is also fine.


There was a high level of consensus amongst universities consulted about the benefit in retaining if not expanding this part of the website.

3.4.3 Clarity and usefulness of the guidelines

An overwhelming majority of universities found the guidelines met the needs of students, faculty members and NCP contact officers. However, it is important to note that many NCP contact officers prepare summary documents of the program guidelines to ensure faculty members and students understand core elements of the nomination and application process, and the eligibility and selection criteria.

Within this context, the following quotes by NCP contact officers indicate the typically held views of university stakeholders:

The guidelines are clear and relatively easy to understand. Most facilities don't really read them so most

of the interpretation occurs within the Mobility Office.

Regional Universities Network university

The guidelines are robust—we have no issues understanding them and communicating key messages

to our faculties.


The guidelines are generally good. A cheat sheet for new NCP contact officers would be handy, and

some guidance for new faculty members (especially about who to contact if they have a question) would

also be handy.


A small number of NCP contact officers consulted identified issues with the length and complexity of guidelines. These universities believed that program guidelines for Mobility and Scholarship had


gradually increased in size and complexity, which in turn increased the burden on faculty members and students when preparing applications. These universities also identified an increased work load on the NCP contact officer (in terms of additional questions from faculty members and students) as a result of the complexity.

The actual number of pages in each year’s guidelines is shown in Table 3.2. While the number of pages included in the guidelines is only a proxy for complexity, it shows that, the length of the guidelines has remained remarkably consistent since the pilot phase. It is possible that the concern expressed by NCP contact officers may reflect growth in demand for NCP activity within universities, than an actual increase in the requirements placed on universities and students through the program guidelines.

TABLE 3.2 LENGTH OF PROGRAM GUIDELINES—MOBILITY AND SCHOLARSHIP

Year Number of pages

Mobility Scholarship

2014 round (pilot phase) 18 15

2015 round 20 20

2016 round 26 20

2017 round 28 20

2018 round 26 20

Note: Page count taken from PDF document and excludes cover and contents pages.

SOURCE: NCP PROGRAM GUDELINES 2014-2018. HTTP://DFAT.GOV.AU/PEOPLE-TO-PEOPLE/NEW-COLOMBO-PLAN/SCHOLARSHIP-PROGRAM/PAGES/PREVIOUS-ROUNDS.ASPX

3.5 Funding arrangements

The 2018 Scholarship and Mobility program guidelines outline the funding available to students under the NCP. Box 3.1 outlines the funding available for the Scholarship and Mobility programs.

BOX 3.1 FUNDING AVAILABLE FOR THE SCHOLARSHIP AND MOBILITY PROGRAMS

Scholarship program funding is provided for:

— Tuition fees for up to one Academic Year (including student service and

amenities fees), where Scholars are studying as fee-paying international

students, up to a maximum of $20,000 ($10,000 per semester or $6,666

per trimester). Fees are payable directly to the Host Institution and only if

there is no exchange agreement between the home university and host

institution.

— In-country language training tuition fees of up to $1,500.

— A travel allowance of $2,500 to support travel to and from the host

location.

— An establishment allowance of $2,500 to contribute to the costs

associated with the Scholar settling into their Host Location.

— A monthly stipend of $2,500 to support basic living costs while in-country.

— Health and travel insurance.

— An addition benefit of $1,000 to purchase study materials for the top-

ranked Scholar in each location.2

Mobility program grant funding is provided for:

— Short-term grants enable students to participate in Mobility projects

that are a minimum of two weeks in duration, and count towards the

course requirements for their home university. A short-term grant

provides between $1,000 and $3,000 per student.

— Semester grants enable students to enrol in full-time study overseas

for a semester that counts toward the course requirements for their

home university. Semester grants provides between $3,000 and

$7,000 per student.

— Internship grants are available for semester-based Mobility projects

that include an internship component in addition to the study.

Internship grants will provide an addition $1,000 per student

— Administration support funding offered to home universities and is

calculated at 10 per cent of the total funding offered for each project.3

SOURCE: 2018 SCHOLARSHIP AND MOBILITY PROGRAM GUIDELINES

2 Section 2—Scholarship Benefits, 2018 Scholarship Program Guidelines. 3 Section 2.2—Mobility Projects and Student Grants, 2018 Scholarship Program Guidelines.

http://dfat.gov.au/people-to-people/new-colombo-plan/scholarship-program/Pages/previous-rounds.aspx

http://dfat.gov.au/people-to-people/new-colombo-plan/scholarship-program/Pages/previous-rounds.aspx


Feedback from universities about the funding arrangements was generally very positive.

The overwhelming majority of NCP contact officers consulted noted that while the Scholarship funding is generous (relative to other scholarship programs) the funding provides a commensurate contribution to the student’s in-country costs and provides incentives for students to participate in NCP. The breadth of funding available effectively supported a broad range of potential living, study and travel costs associated with the Scholarship experience. The feedback consistently identified strong support for the flexibility embedded in the current funding model for Scholarships.

A small number of universities believed that the funding for some host locations is overly generous and could be reduced to provide the capacity for additional Scholars. These universities believe that the $2,500 monthly living allowance is appropriate for Japan, South Korea, Hong Kong SAR and Singapore where average living costs are amongst the highest in the Indo-Pacific region. Such feedback is consistent with the results of the Economist Intelligence Unit’s 2017 cost of living index survey which found that:

…. Singapore was 20% more expensive than New York and 5% pricier than Hong Kong, which lies in

second place. A sustained recovery in the strength of the Japanese yen has led to rising costs in Osaka

and Tokyo. Asia now hosts five [Singapore #1; Hong Kong #2; Tokyo #4; Osaka #5; and Seoul #7] out

of the seven most expensive cities in the world.4

However, they believe that funding allocated to students in other locations should be adjusted to better reflect the costs of those locations. Such adjustments could be based on the results of the worldwide cost of living survey.

Feedback relating to Mobility funding was similarly positive. Universities identified that the funding levels for a short term and semester-based Mobility project of up to $3,000 and $7,000 per student respectively, were an appropriate contribution to the students’ cost of participation in the project. Student focus groups identified that many students had personally incurred between $1,000 and $2,000 in costs, but had been happy to do so given the benefit of their experience. In this regard, student focus groups also indicated that there was a wide range of awareness of additional funding support that may have been be available from the university itself, whether through travel and insurance assistance or other related global mobility programs.

The funding offered for internships was by contrast an area that received some critical feedback from NCP contact officers. These contact officers suggested that $1,000 was an insufficient incentive to encourage take up of an internship, especially if students were required to remain in-country following the completion of their Mobility project for the internship. A number of contact officers consulted suggested that an incentive of up to $3,000 depending on the nature of the internship opportunity would provide the incentive required for more students to undertake an internship while in-country.

3.6 Internships and mentorships

Internships and mentorships are considered an important aspect of the NCP. For the Scholarship program applicants are encouraged to:

— Undertake a professional work experience in which a student has learning goals relevant to their academic qualification and professional development, in the form of an internship. Internships can include clinical placements or practicums.

— Participate in a developmental relationship in which a more experienced or more knowledgeable person helps to guide a less experienced or less knowledgeable person, relevant to work, career, or professional development, in the form of a mentorship.

Internships and mentorships do not need to be confirmed at the time of application and Scholars can undertake more than one internship/mentorship during their Scholarship.

Under the Scholarship program, students are granted access to the NCP Internship and Mentorship Network to help them identify opportunities made available by registered organisations, but can also be arranged through other means.5

4 Economist Intelligence Unit 2017, ‘Measuring the Cost of Living 2017’, http://www.economist.com/blogs/graphicdetail/2017/03/daily-chart-13 5 Section 1.3—Scholarship Conditions and Eligible Activities, 2018 Scholarship Guidelines.


As discussed above, Mobility students are offered a $1,000 incentive to undertake an internship for semester based projects.

Feedback from NCP contact officers identify that this design element of the NCP remains a significant challenge for universities and students, despite some growth in the element since the pilot phase. Several reasons were cited for this by universities.

First, the overwhelming majority of universities consulted do not have the institutional agreements and faculty level relationships to leverage internship and mentorship opportunities for students. This means that the responsibility for organising internships and mentorships falls to students, who typically do not have the knowledge or expertise to secure an internship or mentorship.

Second, universities reported that they do not have access to NCP’s online Internship and Mentorship Network portal. This means that NCP contact officers and faculty members seeking to help a student identify an opportunity in the Network do not have access to the information, unless the student provides her/his log in details.

Third, around half of the universities consulted (approximately 12) have in place agreements with third party providers who specialise in sourcing internship and mentorship opportunities for their students. However, these third party providers operate on a fee-for-service basis payable by students not the universities. Consultations identify that the $1,000 Mobility internship grant is an insufficient incentive for students to engage the services of a provider, and therefore do not typically access the services on offer.

3.7 Short term vs semester based programs

The Mobility program offers funding for short term and semester based projects. Short term projects must be minimum of 14 days in duration and count towards a course, while semester based projects must entail full time study overseas and also count towards a course.6

There was some confusion amongst universities consulted as to whether implicitly prioritise applications for semester based programs over those applications for short term projects.

For those universities who felt that semester based program applications were being favoured over short term project applications, there was concern that institutions could not easily respond to the change in priorities.

Examples, of these concerns include:

The university finds it very difficult to develop semester long Mobility programs. The development costs

are larger, and often require travel to the host location during the development phase. The requirements

of some host universities are quite strict, as they are looking for long term partnerships to meet various

credit and academic requirements which can't be guaranteed under NCP because the funding is often

not multi-year or long term funding. Eighty five per cent of the programs developed by our university are

short term for these reasons.


The university now has a target for short term Mobility in the strategic plan. The focus within the

university is now very much on short term Mobility and semester based programs are likely to receive

less attention in the future (until the target changes).


The university would not like to see one preference over the other. The university likes the flexibility and

wants the program to be neutral about this.

Non-aligned university.

6 Section 2.2—Mobility Projects and Student Grants, 2018 Mobility Program Guidelines.


Semester based programs are only taken up in small numbers. Most students are looking for the short

term opportunities because of their work, family or personal commitments. Our university has found it

relatively difficult to secure student interest in the semester based programs. This is unlikely to change

in the future.


The majority of universities consulted expressed a preference for NCP to maintain a neutral blend of short term and semester based programs.

Student focus groups, on the other hand, identified that many students found semester long projects difficult, both in relation to the impact on their part-time jobs as well as well as the maintaining the intensity required of a project over that period of time. They saw semester long projects as a natural progression for a student first exposed to a short two week project, and then in a better position to understand and identify their preferred personal and career interests, and to justify a longer term commitment.

3.8 Alumni and post-NCP student activities

Developing an alumni community that engages recipients and provides the opportunity for them to share their experiences and promote the NCP was an operational objective of the pilot phase. The 2018 Mobility and Scholarship program guidelines and the whole of government Global Alumni Engagement Strategy (2016-2020) recognise alumni as key to achieving the NCP’s aims of promoting relationships and strengthening business connections between Australia and the region.

Following the pilot phase, DFAT bought together 60 alumni from 28 participating universities to help with the design of an alumni support program. Opportunities for alumni to maintain and foster relationships both on and offline were canvassed. Early initiatives to use social networking platforms (such as LinkedIn and Facebook) to bridge connections between NCP alumni in Australia and in host locations have since become signature initiatives of the Global Alumni Engagement Strategy.

Universities consulted for Stage 3, on average, had very little awareness of the activities of NCP alumni and were largely detached from the detail. As such, there are no real findings of substance (outside this very point) to record from the consultations. General university alumni programs encourage all alumni to share their experiences with the student community but make little distinction between the NCP recipients and the range of other international educational experiences on offer.

Key insights into the NCP alumni are can be drawn from a 2015 survey of the participants of the pilot phase.7 While only 16 percent (197 students) responded to the alumni survey, two thirds (67 per cent) of alumni indicated that job or work opportunities would be the most useful to them as part of the NCP Alumni program. Professional networking (57 per cent) and professional development opportunities (54 per cent) are also popular elements, as indicated in Figure 3.1.

7 The online survey was conducted by Australian Survey Research Pty Ltd and published in December 2016. Of the 1,324 alumni invited to participate, 197 completed it in full (16 per cent). Most commonly, these alumni undertook their program in Indonesia or Japan, had an honours or undergraduate degree.


FIGURE 3.1 MOST USEFUL ELEMENTS OF THE ALUMNI PROGRAM

SOURCE: NEW COLOMBO PLAN SURVEY (2016), AUSTRALIAN SURVEY RESEARCH SURVEY

More specifically, survey results suggest that for alumni of the Mobility program, social networking with other alumni (17 per cent) and job placements with an organisation associated with the NCP (10 per cent) would be very useful alumni activities, while for alumni of the Scholarship program, social networking was also important (20 per cent), but participating in mentoring opportunities (20 per cent) was considered more valuable than job placements (15 per cent).

It has opened several academic and professional doors. The program influenced my choice to do honours, the region of study and the connection with my supervisor, which has subsequently lead to more professional work.

2015 survey participant

With regard to professional development, the survey results also indicated that alumni are keen to develop leadership and management skills (75 per cent), followed by development of communication skills (64 per cent) (see Figure 3.2 below).

FIGURE 3.2 PROFESSIONAL DEVELOMENT AS PART OF THE ALUMNI PROGRAM

SOURCE: NEW COLOMBO PLAN (2016), AUSTRALIAN SURVEY RESEARCH

NCP alumni were able to participate in alumni or advocacy activities on return to Australia. While only 29 per cent of all alumni participated in an alumni event, 88 per cent of these are likely to attend another event, and 54 per cent of those that had not attended indicated they were likely to do so in the future. As shown in Figure 3.3 below, alumni were particularly motivated by opportunities for social and professional networking.


FIGURE 3.3 PARTICIPATION IN ALUMNI ACTIVITIES

SOURCE: NEW COLOMOBO SURVEY (2016), AUSTRALIAN SURVEY RESEARCH

On return, opportunities for engagement online were also available to alumni. These may have been in their infancy, or not well known to alumni, as only 4 per cent of alumni participated in these options. Under half of survey respondents (45 per cent) joined the NCP Alumni LinkedIn Group and most (84 per cent) had not used or did not know of the Australia Global Alumni Network.

As shown in Figure 3.4 and illustrated by the quotes below, for those that had engaged with online resources, alumni identified discussion forums, the availability of news articles and connections with other alumni as the most useful elements of an online alumni community.

FIGURE 3.4 MOST VALUED USES OF AUSTRALIAN GLOBAL ALUMNI NETWORK

SOURCE: NEW COLOMOB PLAN SURVEY (2016), AUSTRALIAN SURVEY RESEARCH


Social networking and friendship building was a major highlight for myself during the NCP program. You

make connections from across the globe, and not just with the people in the host country.

2015 survey participant

I plan to keep in contact with people I met in Hong Kong. I am using technology to help me do this,

using LinkedIn, WhatsApp, Facebook and Skype to keep in contact with students and professionals I

met while in Hong Kong.

Scholarship student, Hong Kong8

Despite enthusiasm for NCP and the benefits it provided, survey results indicated that opportunities to “give back” to NCP by mentoring or providing training to new participants, or participating in NCP’s development, was not a high priority for the alumni: third for Scholars and sixth for Mobility students. However, as shown in Figure 3.5 below, the majority of the alumni were motivated to continue their engagement with NCP and to participate in networking, training and promotional activities into the future. Specifically, the Scholarship cohort (55 per cent) and to a lesser extent the Mobility cohort (39 per cent) actively recommend the program to Australian students and promote the NCP.

I definitely want to help promote NCP—particularly in my home university. I have already had the

opportunity to communicate with current applicants of the NCP scholarship, sharing my experiences and

advice on what the opportunity involves and demands from its participants. My plan is to reach out to

the Macquarie University Merit Scholars group that facilitates the nomination of applicants, and

potentially hold an event for it.

Scholarship student, Japan9

FIGURE 3.5 ALUMNI PROMOTING THE NCP

SOURCE: NEW COLOMBO PLAN SURVEY (2016), AUSTRALIAN SURVEY RESEARCH SURVEY

3.9 Summary of stakeholder feedback

Table 3.3 provides a summary of the feedback provided by universities about NCP’s design features. The table provides an indication of the issues identified by universities and students for each design feature and the strength or frequency with which the issues were raised. Additional consideration of this feedback is also in the conclusions to the report.

8 Quote taken from a Scholarship Student Final Report. 9 Quote taken from a Scholarship Student Final Report.


TABLE 3.3 SUMMARY OF UNIVERSITY AND STUDENT FEEDBACK ABOUT NCP’S DEISGN FEATURES


Finding Frequency Finding Frequency

Nomination and application arrangements

The current application process is an effective

and appropriate process for the Program

Overwhelming

majority

The use of a nomination and an application

process is an effective and appropriate process

for the Program

Overwhelming

majority

Introduction of high level committees within the

university to oversee applications from the

faculties, schools and departments

Majority Maturity of internal processes which call for

nominations across faculties, schools and

departments

Overwhelming

majority

Increased support, coaching and mentoring of

students to enhance the success of nominees

Majority

Increased communication during the Scholars’

application phase sought by universities

Majority

Eligibility criteria

High degree of comfort in the program’s

eligibility criteria—including citizenship criteria,

undergraduate criteria, short term and semester-

based criteria

Majority High degree of comfort in the program’s eligibility

criteria—including citizenship criteria,

undergraduate criteria, short term and semester-

based criteria

Majority

Continued relaxation of age criteria has been a

welcome evolution in the program design

Overwhelming

majority

Relaxation of the age eligibility criteria are

required

Minority

Relaxation of undergraduate criteria are

required

Minority Relaxation of undergraduate criteria are required Minority

Expansion of countries has provided universities

with additional opportunities to pursue existing

partnerships across the Indo Pacific region that

could not be taken advantage of during the

Pilots

Overwhelming

Majority


with additional opportunities to pursue existing


could not be taken advantage of during the Pilots

Overwhelming

Majority


with additional opportunities to pursue new


could not be taken advantage of during the

Pilots

Overwhelming

Majority


with additional opportunities to pursue new


could not be taken advantage of during the Pilots

Overwhelming

Majority

Concern about the personal security costs of

students who attend host locations where

personal safety is a potential issue

Small Minority

Guidelines and guidance

Content, clarity and robustness of the guidelines

are viewed favorably by NCP contact officers

Majority Content, clarity and robustness of the guidelines

are viewed favorably by NCP contact officers

Majority

Process for making changes to the guidelines

and communicating those changes to

universities is well-managed

Majority Process for making changes to the Program

guidelines and communicating those changes to

universities is well-managed

Majority

Lessons learned and advice to applicants is

well-received by stakeholders

Majority Lessons learned and advice to applicants is well-

received by stakeholders

Majority

Programs guidelines are becoming increasingly

complex and lengthy

Small minority Programs guidelines are becoming increasingly

complex and lengthy

Small minority

Funding

Funding levels for short term Mobility projects

(i.e. $1,000-$3,000/student) are appropriate

Overwhelming

Majority

Funding levels are appropriate and reflective of

the prestigious nature of the award

Majority




Funding levels for short term Mobility projects

(i.e. $3,000-$7,000/student) are appropriate

Small Minority Funding supports a broad range of student costs

while in-country

Majority

Internship funding is insufficient to incentivise

students to undertake internships.

Majority More students could be supported with the same

overall funding if the funding was reduced per

student in low cost of living host locations

Minority

Internships and mentorships

Universities engage third party providers to

assist students. Payment of providers is the

responsibility of students

Even split Significant improvement in the ability of

universities to support the internship and

mentorship ambitions/requirements of Scholars

Majority

Universities have been unsuccessful in

leveraging faculty, school or department level

contacts to support internships and mentorships

Majority Universities have been successful in leveraging

faculty, school or department level contacts to

support internships and mentorships

Even split

Universities have chosen not to pursue these

aspects of the NCP

Small Minority

Short term vs semester based programs

Maintain a balance between short term and

semester based projects

Overwhelming

majority

N/A N/A

Alumni and post-NCP student activities

Level of visibility and interest displayed by

universities in Government’s NCP alumni

program is limited

Overwhelming

majority

Level of visibility and interest displayed by

universities in Government’s NCP alumni

program is limited

Overwhelming

majority

Design of Alumni program is appropriate and

valued by students

Majority (of

students

surveyed)

Design of Alumni program is appropriate and

valued by students

Majority (of

students

surveyed)

SOURCE: CONSULTATIONS WITH HOME UNIVERSITIES AND STUDENTS SURVEYED


4 A D M I N I S T R A T I V E A R R A N G E M E N T S

3 Administrat ive arrangements

This chapter considers the administrative arrangements of NCP and the degree to which they represent a benefit or challenge to its continued rollout and expansion. Once again, the chapter draws heavily on consultations with universities.

The most important issues relating to program administration are discussed and analysed in detail below.

4.1 Overview of NCP’s administrative arrangements

The Stage 1 evaluation report considered the suitability of NCP’s administrative arrangements, identified as:

— Publicity of NCP at the university level.

— Call for nominations and applications from universities.

— Support and guidance to universities (including questions about eligibility).

— Assessment of university nominations and applications against eligibility and selection criteria.

— Student interviews (Scholarships only).

— Selection of successful Mobility projects and Scholars.

— Notification of outcomes to universities and students.

— Liaison/communication with universities, students and host locations following selection.

Consultations with universities identify that there are only a small number of issues relating to the administrative arrangements outlined above, and it is only the issues of material nature which are discussed in this chapter. Some of the issues relating to guidance, applications and nominations, and the notification of assessment outcomes have been discussed in the previous chapter.

4.2 Timeframes for nominations and applications

A significant issue identified during the Stage 1 evaluation centred on the timing and timeframes associated with the nomination and application window. A summary of the two key issues about timing identified by universities during consultations is provided below.

4.2.1 Overlap with the NAFSA annual conference

NAFSA is the world's largest not for profit association dedicated to international education and exchange. NAFSA has more than 10,000 members located at more than 3,500 educational institutions worldwide. Between late-May and early-June of each year NAFSA holds an international mobility conference in the USA. A large number of NCP contact officers attend this conference annually, which coincides with the submission of nominations and applications for NCP. The overlap between the


NAFSA annual conference and the submission of NCP nominations and applications cause problems, which is best summed up by one NCP contact officer who noted:

The application round opens 1 April and closes 3 June. This the worst time of the year for most ILOs to

prepare and submit an application for a major government program, as most are either presenting at or

attending the NAFSA conference. Last year I spent one evening and into the early hours of one morning

liaising with faculties and my staff at home to finalise the university’s applications. It was really very

difficult to do this from a hotel room in North America. It has not happened to me, but I know that some

ILOs have had issues with their IT while at the conference which has made it even more difficult to

finalise and submit applications. The Department should think about moving the submission date so it

does not clash with the last few days of NAFSA. This is a problem that could be easily avoided.


The feedback gained from universities on this point relates to the 2014, 2015, 2016 and 2017 nomination and application rounds. The feedback identified a nearly even split between the universities about how the overlap could be addressed by Government. That is, approximately 50 per cent of NCP contract officers identified a preference for the nomination and application window to be brought forward, while approximately 50 per cent of NCP contact officers suggested that the timeframes should be move to a point later in the year.

It is important to note that the 2018 version of the Mobility and Scholarship program guidelines has addressed this issue, and nominations and application closed on 19 May 2017 to avoid this overlap.

Overlap with Endeavour Scholarships and Fellowships Program

The intersection between NCP and the Endeavour Scholarships and Fellowships Program was a consistent theme of consultations with NCP contact officers. The 2018 Endeavour Program Guidelines state that applications open in April and close on 30 June 2017, while nominations and applications for NCP open in early April and close in early June 2017.

For a majority of NCP contact officers the overlapping timeframes for NCP and Endeavour Scholarships and Fellowships Program creates some level of confusion amongst faculty members seeking to submit applications for NCP while also supporting their students’ applications for the Endeavour Scholarships and Fellowships Program. Even though the Endeavour Scholarships and Fellowships Program is clearly focused on supporting postgraduate research, vocational education and training and professional development for graduates, compared to NCP which is focused on supporting undergraduate study, the NCP contact officers consulted reported that greater separation or coming together of the application windows would better enable faculties to effectively support all applications. The quotes below illustrate the point made by many universities:

Endeavour and NCP timeframes are an issue for some faculty members who are preparing applications

for a Mobility grant while also helping those students who are applying for the Endeavour program. I

often hear academics question why the timeframes are similar, but not the same or further apart.

Capital city based university

Our academics find NCP and Endeavour application windows confusing—they overlap but are not

exactly the same. A single application window for both programs, with a single portal for uploading both

applications, would be ideal for all parties concerned.

Innovative Research Universities Australia university

Separate NCP and Endeavour processes would assist our facilities and students submitting

applications. My suggestion is to bring the nomination and application window for NCP forward so as to

minimise the overlap and confusion that seems to occur every year.

Non-aligned university

Move the timeframes for Endeavour and NCP as far apart as possible part. There are no synergies in

having them close together, as one [Endeavour] focuses on postgrad research, VET students and

professional development, and the other [NCP] is for undergrads…. They should be moved a part to

reduce confusion amongst faculties, students and everyone involved in both programs.

Group of Eight university


It has been noted that for the 2018 nomination and application round, the timing of nominations and applications has been brought forward in an attempt to address the feedback from provided universities and relayed to Government by the ACIL Allen evaluation team in late 2016.

4.3 Online application and management system

Section 5 of the 2018 Mobility Guidelines state that applicants are required to apply for funding and accept or decline funding offers through ISEO.10 Under the guidelines universities can nominate multiple people to have access to the ISEO system; however, the nominated NCP contact officer is the only person who can submit an application in the system.

Once funding has been offered, a university will use ISEO to input student details, vary Mobility projects (where necessary), and acquit Mobility Projects. In addition, universities must ensure that all details in ISEO are current and accurate:

Funding Recipients must ensure the following information is always current in ISEO: [This includes

ensuring that] start and end travel dates, host Location(s), student numbers, project partner(s), home

University contact point for the Mobility Project (name, position, email, phone number),

Internship/Mentorship details (if applicable) [are current].

Section 5 ISEO—2018 Mobility Program Guidelines.

By contrast, students must be nominated by their university in order to apply for a Scholarship. Each university conducts an internal process to nominate up to 10 unranked students. Universities are required to make nominations accessible to all eligible students enrolled at that university. Nominations are submitted through the NCP online website. The Department then provides a nominated NCP contact officer with instructions on how students can submit applications.11

The overwhelming majority of feedback provided by universities relates to the ISEO system used to submit and then manage applications for the Mobility program. The online system for the Scholarship program attracted no critical comment from universities consulted, and is therefore not discussed any further in the following sections.

Updating student and travel details

The requirement to maintain current and accurate details of students and their travel arrangements is a significant issue for a large majority of the universities consulted. Under the Mobility Guidelines universities are required to upload and update the information provided in Box 4.1 below.

BOX 4.1 STUDENT DETAILS AND OTHER INFORMATION REQUIRED REUQIRED BY NCP

Student Details including:

— Student Identification Code

— Title, First name and Family name

— Gender

— Date of birth

— City of residence (optional)

— Campus

— Email (personal email if available)

— Phone

— Destination phone.

Travel Details including:

— Location

— Destination city

— Field of study

— Qualification level

— Start date

— End date

— Project Partners(s)

— Australian University project contact name

— Australian University project contact phone

— Study experience/s.

Note: Bolded and underlined Details were identified by universities as needing to be optional rather than mandatory requirements In ISEO. SOURCE: SECTION 5—2018 MOBILITY PROGRAM GUIDELINES

10 See: https://iseo.education.gov.au/ 11 Section 5.1—Nominations, 2018 Scholarship Guidelines.


An overwhelming majority of the universities felt that providing and maintaining all of the student and travel details outlined in the Mobility program guidelines was administratively cumbersome and difficult within the timeframes required by the Department of Education. Specifically, universities find it difficult and time consuming to gain an accurate “destination phone number” as many students do not know this information prior to leaving Australia. Also the need to identify a “location” and a “destination city” can be difficult information for NCP contact officers to capture efficiently. This is because some programs involve field work in regional locations that may not be city-based or specific to a particular city by a regional area. The result is that the information provided by NCP contact officers is by their own admission sometimes partial or sometimes incorrect. The following quotes provide supporting evidence of the feedback gained from universities about these issues.

Uploading some student details can be time consuming and is quite a manual process. It is also difficult

to keep track of the details if a program extends beyond a calendar year, and involves multiple cohorts

of students.

Innovation Research Universities Australia university.

Under the rules a student can't be awarded project funding for two short term Mobility projects. It is a

very time consuming and manual process to check this. Also, it can be time consuming to upload

student details. Some of these details can't be easily found (such as the host location contact phone

number) and I find myself spending a lot of time trying to track information down. On occasions, I have

just used a generic contact number and not the exact one to ensure all of the ISO requirements are met.


Some student details captured (such as host location phone numbers) are difficult to find and not often

used by us or the Department. I don’t understand why it is a requirement to capture this information over

such a large student population.

Innovation Research Universities Australia university.

In short, universities would appreciate some more flexibility to update these data any time prior to departure, and not by the current three week deadline. The request for this flexibility is consistent with the feedback the Department itself collected from universities (during October 2015) and shared with ACIL Allen for this report.

Variations, invoices and acquittal

A small number of universities consulted (less than ten) identified that ISEO limits the ability of NCP contact officers to use the online system efficiently and effectively. These universities indicated that the system and supporting processes for variations, invoicing and acquittal are unnecessarily burdensome and require streamlining, as shown in the quotes below.

ISEO is not an intuitive system. It can be difficult to upload documents (in particular, invoices). A live

chat function or some other way to leave a query and have the Department respond to a particular field

within the system would be very useful.


Signing statutory declarations project variations and acquittals and uploading them for into ISEO system

is very, very painful. As an ILO I need to find an appropriately authorised witness, which can be difficult

at the best of times within the university. I’m not allowed to attach electronic signatures to these

documents which makes it a manual process and generates extra administration for me.


Variations can be a little cumbersome, and involve a lot of manual processing. This is fine for major

variations, but for relatively modest ones, ie change of host partner, change in student numbers or

change in course focus, ISEO is not very user friendly. Changes to project dates in the online system, in

particular, are not easy to make.

Non-aligned university.

These quotes indicate that universities would like to see the introduction of an electronic signature function in ISEO, additional flexibility with respect to minor changes in ISEO and the ability to leave a query within ISEO (which relates to specific fields) which can be answered by the Department.


4.4 Communication between Government and universities

4.4.1 Scholarship

The Stage 1 NCP evaluation identified that universities were concerned about the level of communication that occurs between the university, the nominated student and the Government during the application process (i.e. following acceptance of the university’s nominations). This communication was considered vital to ensure that applicants understood the selection criteria and were well prepared for Scholarship interviews.

The administration of Scholars happens mostly outside the hands of the university. This can make it

difficult to keep track of Scholars unless they actively make contact with ILOs. We see this as an

ongoing gap in NCP’s administration as the universities have responsibilities for student welfare etc,

which we can’t meet if we aren’t aware of key issues.


In 2016-2017, consultations have identified that this is still an issue for many universities. In short, universities are seeking to be included in all written correspondence (email and other formal correspondence) so they can assist students during the application process, and keep senior-level university staff and faculties informed about the progress of their students.

4.4.2 Mobility

The Stage 1 evaluation report also identified a strong interest amongst universities for tailored feedback about the quality of applications. This feedback was considered by universities as being vital to ensure faculty members who invest time and effort in unsuccessful applications understand why the applications were unsuccessful and have opportunities to apply these learnings to future application rounds.

To remedy these complaints, the Government has published FAQs and more recently Advice to Applicants on the NCP website, in lieu of providing individual feedback to applicants. It is largely acknowledged amongst Departmental staff responsible for the Mobility program that they do not have the resources (staff time and financial resources) to provide feedback to large number of applications submitted each round.

However, some three years later, universities still express concerns about the nature of the feedback provided on unsuccessful applications. A number of universities consulted identified that there is a growing level of disinterest amongst some faculty staff who are repeatedly unsuccessful. These universities are seeking a system established which provides tailored feedback to academic faculties regarding successful and unsuccessful project/program applications.

4.5 Responsiveness to university feedback

The responsiveness to suggested improvements in NCP’s administration was a significant theme of Stage 3 consultations. The overwhelming majority of universities consulted provided highly positive feedback about the openness of the Department and DFAT to constructive feedback about how to improve all aspects of administration. They noted that the attitude and willingness of program administrators to improve the NPC’s delivery was a significant factor in its success, and are seeking that the same level of openness and receptiveness about administrative improvements be maintained in the future.

Department is relatively receptive to all recommendations aimed at improving the online system. There

are some problems with the process of uploading documents (i.e. invoices) which make the system

clunky.


Overall the administration of NCP has been pretty good. There are no real complaints about its

administration.

Australian Technology Network university.


The Department takes feedback well and is open to system and process improvement suggestions. The

NCP people are always easy to deal with.

Capital city-based university.

We have no real concerns about NCP’s admin. The online system and other processes are fairly user

friendly, and the staff are as helpful as they can be.

Non-aligned university

The administration processes and system for NCP are a lot better than in 2014. The processes and

systems keep improving. Government has been very receptive to the changes suggested which is

pleasing and a good sign for the future.

Regional university.

4.6 Summary of stakeholder feedback

Table 4.1 provides a summary of the feedback provided by universities about NCP’s administrative arrangements. The table provides an indication of the issues identified by universities and students for each design feature and the strength or frequency with which the issues were raised by stakeholders. Additional consideration of this feedback is provided in the conclusions to the report.

TABLE 4.1 SUMMARY OF UNIVERSITY FEEDBACK ABOUT NCP’S ADMINISTRATIVE ARRANGEMENTS



Timeframes for nominations and applications

Application window coincides with a international

conference and when many NCP university contact

officers are overseas

Majority Application window coincides with an

international conference and when many NCP

university contact officers are overseas

Majority

Overlap with the Endeavour Scholarships and

Fellowships Program generates confusion amongst

faculties that could be avoided

Majority Overlap with the Endeavour Scholarships and

Fellowships Program generates confusion

amongst faculties that could be avoided

Majority

Online application and management system

Universities find it difficult to maintain current and

accurate details of students and their travel

arrangements and are seeking some relaxation of this

requirement

Majority N/A

Universities are seeking the continued evolution of

ISEO to reduce the administration effort required to

manage Mobility grants

Majority N/A

Communication between Government and universities

Universities are seeking additional feedback about

unsuccessful applications to ensure faculty members

responsible for developing these applications better

understand why they were unsuccessful

Overwhelming

Majority

Universities are seeking to be included in all

written correspondence between the Government

and students

Majority

The inclusion of the Advice to Applicants has

provided welcome insight into the design of

successful applications

Majority

Responsiveness to university feedback

Government highly responsiveness to feedback about

administrative improvements at all stages of program

administration

Overwhelming

majority

Government highly responsiveness to feedback

about administrative improvements at all stages

of program administration

Overwhelming

majority

SOURCE: CONSULTATIONS WITH HOME UNIVERSITIES


5 I M P A C T S

4 Impacts

This chapter considers the impacts on the stakeholders who are core to NCPs objectives. The impacts (both positive and negative) are considered from the perspectives of the students who participate in NCP, the universities which facilitate and manage these students’ participation and the institutions which have hosted and will continue to host NCP students into the future. The chapter seeks to address elements of all research questions considered for Stage 3.

5.1 Impact on students

5.1.1 Student perspectives of impact

Analysis of the student experience undertaken in Stage 2 of the evaluation highlighted that exposure to different cultures, study development and work development were generally the most valued aspects of participation in the NCP. Evidence from the alumni survey, together with evidence from Scholarship students program reports that were completed on conclusion of their placement, further confirmed that the NCP has an overwhelmingly positive impact on students.

As indicated in Table 5.1, for both Mobility and Scholarship students the most beneficial elements of the program clearly align with the strategic objectives NCP, namely to:

— Lift knowledge of the Indo-Pacific in Australia.

— Deepen Australia’s people-to-people and institutional relations with the region.

— Establish study in the Indo-Pacific as a rite of passage for Australian undergraduate students.

— Increase the number of work-ready Australian graduates with regional experience.

TABLE 5.1 BENEFITS OF THE NCP EXPERIENCE FOR PARTICIPANTS

Objective Measure Strongly agree or agree

(% of respondents)

Lift knowledge of the Indo-Pacific in Australia by

increasing the number of Australian

undergraduate students undertaking study and

Internships in the region

I gained useful knowledge and

understanding of my host

location(s)

97

Deepen Australia’s people-to-people and

institutional relationships with the region, through

the engagement of students, universities,

businesses and other stakeholder networks in

the Program

I gained personal connections

that have helped in my career

59

Increase the number of work-ready Australian

graduates with regional experience.

I am confident in my ability to

engage with people in the Indo-

Pacific region

87


Objective Measure Strongly agree or agree

(% of respondents)

I gained professional

experience that has helped in

my career

68

I am more likely to seek out

work relating to the Indo-Pacific

region

83

SOURCE: NEW COLOMBO PLAN SURVEY 2016, AUSTRALIAN SURVEY RESEARCH

The alumni survey also highlighted the specific insights and skills-sets that NCP participants believe they gained. As shown in Figure 5.1 below, overwhelmingly participants strongly agreed or agreed that the main benefits to flow from their experience was gaining knowledge of their host location, cross-cultural skills and the confidence to engage with both communities and professionals in the region.

FIGURE 5.1 SKILLS GAINED

SOURCE: NEW COLOMBO PLAN SURVEY (2016), AUSTRALIAN SURVEY RESEARCH

The alumni survey indicates that for Mobility students’ knowledge of the host location, cross cultural skills and the confidence to engage were the top three benefits of the NCP, whereas participants of the Scholars highlighted professional experiences as particularly valuable. This is confirmed in the final reports of pilot program Scholars. As shown in Figure 5.2 and the quotes below, overwhelmingly, Scholars believe that participation in the NCP greatly enhanced their cultural awareness and skills, while also improving their understanding of their host location and their professional connections.


FIGURE 5.2 IMPACT ON SCHOLARSHIP STUDENTS

Note: n= 35 pilot program Scholarship students

SOURCE: PROGRAM COMPLETION REPORT FROM SCHOLARSHIP STUDENTS

Singapore is a dense salad bowl of cultures. To live here makes understanding and adaption to others a

necessity. I have made friends from so many different backgrounds, and living within such a culturally

diverse place just feels normal to me now. It’s going to feel very strange leaving it!

Scholarship student, Singapore12

Improved cultural awareness and greater comfort levels are foundational to improved business and institutional relationships—from which flow opportunities to advance Australia’s broader national interests. Only a small percentage of participants did not report gaining professional experiences or improve personal connections following participation in the program.

One of the enduring impacts reported of participation in the NCP was establishing connections with Australian and host institutions, as well as other participants and new-found professional and personal contacts in host locations. While 16 per cent of survey respondents indicated that they had not maintained any significant links, 84 per cent maintained at least one and 28 per cent had maintained at least four. As shown in Figure 5.3 and illustrated in the quote below, for the majority of program participants these links played a significant role in their understanding of the region, and greatly enhanced their study opportunities and employment prospects. Only a small proportion of survey participants did not believe their NCP contacts provided these ongoing benefits.

FIGURE 5.3 DEVELOPMENT OF CONTACTS/LINKS


I made so many fantastic friends and professional contacts during my time in Indonesia. During my

internship at the Australian Embassy I met many Australian diplomates interested in bilateral

12 Quote taken from a Scholarship Student Final Report.

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Experience of living abroad enhanced my cultural awarenessand skills

Internship contributed to my understanding of the host location

Internship helped me to create professional connections

Not at all To a minor extent To a moderate extent To a great extent


relationships, as well as Indonesian government counterparts from other development diplomate

stakeholders in Indonesia. I also attended several conferences in the duration of the NCP experience,

all of which introduced me to people around the world interested in Indonesia and the ASEAN region.

Finally and most importantly, my day to day study, work and general life in Indonesia allowed me to

meet so many great people and I know I will continue to foster connections with into the future.

Scholarship student, Indonesia13

The NCP also had a positive influence on the majority of participants’ career choices. For both Mobility and Scholarship participants, around four in five indicated that the experience influenced their choices to a great or moderate extent, while nine in ten respondents believed their experience had made a positive contribution to their career prospects (see Figure 5.4 and the quotes below). Participation in the program was particularly valuable to the career prospects of scholarship students with 81 per cent indicating that it had a strong positive impact.

I believe I have made the connections for future job opportunities and cross-country collaboration.

These were mostly formed through either working with my supervisors or interviewing various woman

executives, who through casual conversation offered opportunities or offered to link me with others who

could help me achieve my goals.

.Scholarship student, Hong Kong SAR14

This internship hugely increased my network as I met and worked with many researchers from RIKEN

Brain Science Institute, and if I continue my research in this field, I will be able to collaborate again with

RIKEN, and there is even a possibility of returning to work at RIKEN in a few years’ time.


FIGURE 5.4 INFLUENCE ON RECIPIENTS CAREER

SOURCE: SOURCE: NEW COLOMBO PLAN SURVEY (2016), AUSTRALIAN SURVEY RESEARCH

The survey also identified that participants generally were highly motivated to engage in further study or seek employment. While the direct connection between these choices and participation in the NCP was not assessed, as shown in nearly half of participants intend to seek employment in the Indo-Pacific region. Evidence of heightened levels of comfort and familiarity with the region and its cultures among participants would further support this.

Working in the Telstra office, which currently has over 150 staff members, allowed me to constantly

meet new people, network and form connections during my 7 week internship. I was also able to

develop closer working relationships with my immediate teams and worked on an extremely important

project which lead to my being offered a job at Telstra Singapore.

Scholarship student, Singapore16

13 Quote taken from a Scholarship Student Final Report. 14 Quote taken from a Scholarship Student Final Report. 15 Quote taken from a Scholarship Student Final Report. 16 Quote taken from a Scholarship Student Final Report.


FIGURE 5.5 INFLUENCE ON RECIPIENTS FUTURE PLANS


I definitely will keep in touch with my internship organisation - having already connected with them

through multiple social media platforms. As I head deeper into my career, I know that some key

individuals, such as my supervisor/boss Jeffrey Char, has been a great mentor to me and someone I will

continue to seek advice from as I encounter different challenges in my life. I know that as Australia

continues to improve its ties with Asia, inevitably, my desire to drive innovation in Australia, and pursue

an entrepreneurial career will take be back to this network I have built in these past ten months.


Mobility student focus groups

Mobility students cited a range of impacts from their participation in NCP Mobility projects. Examples were that NCP:

— provided relevant work experience and cemented career goals

— exposed them to officials and organisations that enabled them to be more comfortable engaging with professionals

— helped them gain independence and confidence through a placement opportunity that is not otherwise as readily available in undergraduate study

— formed ongoing relationships, with some students continuing to correspond with the host organisations and contacts made

— developed language skills that were now being used in part-time (retail) jobs back in Australia

— inspired their interest in developing countries and other cultures

— promoted overseas travel, some who “wouldn’t have left Melbourne” without the grant, others to locations where students may not have otherwise gone.

One student also observed a distinct impact on host location students—that “shy” local students “opened up” as a result of seeing the dedication and teacher-student relationship amongst the Mobility students.

Other students raised specific business impacts for their host location, including being involved in fundraising activities and local volunteering, and developing policy papers for their host country government organisation.

5.1.2 University perspectives of student impact

All universities consulted for Stage 3, without question, identified that personal, study and professional benefits flow to students who participate in the NCP. However, the feedback gained from universities about student perceptions is highly anecdotal. This is because universities (with the exception of one university which has commissioned a long term study of NCP students that will be finalised in 2018) do not appear to directly measure the impact of Mobility programs on their students’ academic and professional performance.

17 Quote taken from a Scholarship Student Final Report.


That being said, even though university feedback is anecdotal and largely based on observations from discussions between NCP contact officers and the faculty members who support students’ participation in NCP, it is universal in nature and highly consistent with feedback gained from universities across each stage of the NCP evaluation. Examples of the comments provided by universities in Stage 3 are provided below. These comments show that NCP has also benefited students from a broad range of faculties and fields of study, which are beyond traditional mobility programs which focus on international studies and foreign languages.

There has been a broader uptake of NCP across the university. It is not just Asian language studies, but

now includes the sciences and other disciplines. This is an unexpected benefit from NCP.


Our university hasn’t even considered the need to quantify the benefits of NCP to our students.

However, I hear a lot of good news stories from faculties and the students I meet about NCP. I would be

useful to have some systematic evidence to back this up, but we don’t at this stage.

Large metropolitan university.

The university is seeing more interest by students and faculties in Asia.


Overall we receive a lot of positive feedback, although no data to support this.

Regional university.

5.2 Impact on Australian universities

5.2.1 New vs existing relationships

The majority of NCP contact officers consulted reported that involvement in the NCP has both strengthened existing and encouraged the development of new relationships with universities in the Indo-Pacific region, particularly since the NCP expanded to include more host locations. For many of the universities consulted, the distinction between new and existing relationships is of marginal interest. For example, some universities reported already having too many relationships and seeking to focus only on a smaller number of these relationships. These universities indicated that they are being selective among the large number of offers to partner on a regular basis. On the other hand, other universities are constantly seeking new partners to lift their profile in the Indo-Pacific region where it aligns with the strategies of universities or faculty members.

In short, the primary objective of all universities consulted was to develop “good relationships” as opposed to focusing on new or existing ones. By good relationships these universities refer to relationships that support a broad range of university activities (which include joint research initiatives), and NCP is just one component in the relationships between universities.

While the extent to which the NCP has led to inter-institutional relationships being strengthened, extended or newly forged has not been tracked by universities, the following comments provide qualitative evidence of improved relationships as an outcome of NCP:

Expansion has been great for the university. The university was founded on Asian studies and has

many relationships across the Indo-Pacific.


The university has developed a handful of new exchange partnerships because of NCP. These build on

existing relationships, but in new areas of study.


Pilots were far too narrow. The university did not have established relationships in these host locations.

The expansion gives the university access to China where many of the relationships are. The university

has developed a handful of new exchange partnerships because of NCP. These build on existing

relationships, but in new areas of study.



Expansion of countries was a good thing. This has allowed for some interesting

partnerships/relationships to foster in unusual countries. For example, to establish a program with the

government of Bhutan to look at agricultural improvement options.

Large regional university.

NCP has delivered an enhanced relationship between the Mobility office and faculties. There was no

mechanism to bring Mobility together across the university and NCP has done this. The university has

seen many new programs come on line and will continue to do so over the short/medium term.


The feedback obtained from consultations with universities is supported by evidence contained in university annual reports. As the boxed examples (Box 5.1, Box 5.2 and show, it is common for universities to showcase their students’ involvement in the NCP in the context of other successful initiatives to forge research and exchange relationships with universities in the Indo-Pacific region.

BOX 5.1 INTERNATIONAL EXCHANGES AT AUSTRALIAN NATIONAL UNIVERITY

ANU reports that the New Colombo Plan facilitated learning abroad for 228 students in Cambodia, China,

Colombia, Hong Kong, Indonesia, Japan, Korea, Mexico, Myanmar, Singapore, Taiwan, Turkey and Vietnam,

while a further six students were awarded scholarships, including two that became NCP Fellows. These

achievements sit alongside reports of a new framework for research cooperation and exchange between

ANU’s Research School of Chemistry and the Institute for Protein Research at Osaka University, and a new

National Parliamentary Fellowship Program that offers three-month placements in parliamentary offices in

India and Japan for postgraduate students, among many other examples.

SOURCE: ANU ANNUAL REPORT, 2015).

BOX 5.2 INTERNATIONAL ENGAGEMENT AT CURTIN UNIVERSITY

For Curtin University participation in the New Colombo Plan—which included funding for 109 students to

undertake industry internships, semester long exchange, study tours and clinical placements in the region—

sits alongside reports of successes in international engagement in 2016, including establishing 59 new

international agreements for student and staff exchange agreements that focus on collaborative research. For

example, Curtin University and Nanjing University in China recently signed a memorandum of understanding

for the establishment of the Sino-Australia Collaborative Innovation Alliance. Joining Curtin and Nanjing are

four universities including Tongji University, Harbin Institute of Technology, Huazhong University of Science

and Technology, and Shanghai Jiao Tong University.

SOURCE: CURTIN UNIVERSITY ANNUAL REPORT, 2016.


BOX 5.3 GRIFFITH UNIVERSITY

For Griffith University, the New Colombo Plan is associated with the university delivering on the ‘Griffith 2020’

plan which commits the university to ensure that students are well prepared for professional life post-

graduation. The university aims to achieve this through work-integrated-learning courses with the not-for-profit

sector that challenge students to address real issues faced by a community. In 2016, the university was

successful in obtaining New Colombo Plan funding for 57 students over three years to undertake a

multidisciplinary project in India. The team of staff who managed this were awarded an Australian Learning

and Teaching Citation.

SOURCE: GRIFFITH UNIVERSITY ANNUAL REPORT, 2016)

5.2.2 Mobility and university strategic plans

A core benefit of NCP, according to the universities consulted for Stage 3, has been the elevation of outbound mobility to the highest levels of prominence within the universities. This reaffirmed the Stage 1 finding that prior to NCP, global mobility did not hold as prominent a position in some universities.

Furthermore, consultations have identified that NCP has been both a catalyst for outbound mobility to be included in the strategic plans and priorities of universities (many reporting that this has occurred for the first time), and an important source of funding to help universities meet these objectives. In addition to numerous supporting quotes and comments provided by NCP contact officers, a review of selected university strategic plans covering the duration of NCP (as shown in Table 5.2) is further evidence that NCP (as a major driver of out bound mobility) is now an established aspect of the university sector’s strategic horizon and priorities.

TABLE 5.2 EXAMPLES OF OUTBOUND MOBILITY MENTIONED IN UNIVERSITY STRATEGIC PLANS

University Plan Example of strategy/statement about outbound mobility

University of

Canberra

Breakthrough: The

University Of

Canberra’s Strategic

Plan

International experiences—By 2018 at least a third of our graduating

cohort (including both domestic and international students) will have

had an international experience that includes industry-based study

tours, work integrated learning, intensive language and culture

programs, and for-credit study at partner institutions.

University of

Wollongong

University of

Wollongong Strategic

Plan

Goal 3: connection communities—Re-double our efforts to increase

outbound student mobility, especially by focussing on opportunities in

Asia, and

significantly increase the movement of staff and students between

UOW’s on—and off-shore campuses

Griffith

University

Griffith 2020 Overarching Goals—To enhance our engagement with the Asia–

Pacific region and to consolidate our reputation as one of Australia’s

most Asian-engaged universities

RMIT

University

Ready for Life

Strategic Plan 2015-

2020.

Goal 7 (Global research and outlook), Priority 1—Preparing students

for the globalised world of work. Our global presence will enrich the

student experience to prepare for life and work in increasingly

globalised labour markets. Curriculum design, digital tools, teaching

strategies and opportunities for student mobility will all reflect and

embed our global outlook. How we will be known in 2020? Our

students successfully develop cross-cultural skills and

competencies.

SOURCE: VAROUS WEBSITES WHERE STRATEGIC PLANS ARE PUBLISHED


5.3 Impact on host locations

As part of the agreed Stage 3 evaluation activities, ACIL Allen sought consultations with those businesses and host location organisations which hosted pilot program students and were interviewed for the Stage 2 report (in 2015). The 2015 consultations identified a high degree of satisfaction with the majority of NCP students (Scholarship and Mobility) hosted, which was also supported by survey results of a hosts.

An interesting finding from Stage 3 of the evaluation is that all host location organisations contacted did not feel that they had any insight into the workings of NCP or the quality of students that had not already been provided.

In addition, consultations with universities identified that host locations reported high levels of satisfaction with NCP students. However, the university representatives consulted were often in direct contact with host locations (unless there was an urgent or pressing issue to address) and were therefore not in a position to provide detailed information about the impacts that NCP has had on host partners. Universities also noted that many of the impacts that NCP has on hosts will be longer term and beyond the scope of this evaluation (i.e. the impacts of NCP are likely to be realised over a 5-10 year time horizon).


6 C O N C L U S I O N

5 Conclusion

6.1 Overall summary

As noted in the introduction, the evaluation of the NCP was designed to track the program from its inception and launch during the pilot phase (Stage 1), through the in-country experience for recipients and participating universities (Stage 2), and the post-NCP experience for both students and participating universities/organisations (Stage 3). Throughout the evaluation there have been continued adjustments to the administration and delivery of the program that have been largely welcomed by the stakeholders.

Noting the ongoing efforts towards improving the program, Stage 3 of the evaluation focuses on residual issues raised regarding the NCP’s design and administration, and on insights and analysis that may assist to address these.

6.1.1 Design elements

The evaluation focused on seven design elements of the NCP—nomination and application processes; eligibility criteria; guidelines and guidance; funding arrangements; internships and mentorships; short term vs semester based programs; and alumni support. Universities positively viewed the design elements of the Mobility program and the refinements that have been made following the pilot, such as relaxing the age restrictions for applicants.

Specific matters that require further consideration are:

— Communication of the NCP expectations and decisions is vital to stakeholders. It was widely noted that improvements have been made in the information for applicants, including to the FAQ, and that the program guidelines are generally clear and comprehensive. However, for some universities it is important to be mindful of the timeliness of changing the Guidelines, and avoid increasing the length or complexity of the Guidelines. Relatedly, universities are seeking more individualised feedback on unsuccessful applications, both to support students through the process and to retain the interest of academics that invest considerable effort in developing and ranking the applications.

— While most universities are comfortable with NCP’s eligibility and selection criteria, for regional universities with high numbers of mature students, the restrictions on age are reported to be having an undesirable discriminatory effect. Indeed, for universities with certain student demographics, relaxing the age requirement may boost their participation in NCP.

— Funding arrangements are generally thought to be fit for purpose, yet too generous for some locations. An opportunity exists to tailor funding support to better reflect the living costs in different host locations.


— Universities and students are seeking a balance to be maintained between short term and semester based programs. Universities appreciate the flexibility afforded to them in selecting programs that meet the preferences of their students and the types of programs faculties wish to pursue.

— Arrangements to support Mobility students to engage in an intern/mentorship could be improved, particularly where universities do not have institutional relationships to leverage opportunities for students, nor access to the internship/mentorship portal to assist students to identify opportunities. Further, for those universities that source specialised advice about these opportunities, the fee-for-service reduces the financial aid available to students decreasing if not completely negating the attractiveness of the program.

— Universities appear to be doing little to formally monitor the activities of, and hence impact on, returning participants. While the NPC Secretariat is developing opportunities to maintain professional connections and engage alumni in advocacy and promotional activities, the considerable experience and investment of universities in their existing alumni programs could also be utilised to this end.

6.1.2 Administrative arrangements

The evaluation focused on four administrative elements of the NCP—timeframes for nomination and applications; the online application and management system; communications between students, participating universities and government; and the program’s responsiveness to feedback.

Feedback was particularly positive about the Government’s openness to feedback and the willingness of program administrators to improve all aspects of program delivery. The only two administrative arrangements identified for further attention are:

— The administrative functionality of the online application and management system could be improved. Participating Australian universities identified two main concerns—first, that managing input of data about students participating in the Mobility program is difficult and time consuming, particularly for information that is unknown before departure or often changing, such as destination phone numbers; and second, streamlining administrative processes, such as the introduction of electronic signatures and the ability to leave queries on specific fields, would improve the functionality of the system for administrators.

— The flow of information back to universities and students could be improved. Participating Australian universities highlighted both students’ and academics’ desire for feedback about decisions on individual applications. While acknowledging that the effort that this may involve, they contended it is key to maintaining high levels of support and ongoing engagement in the program.

6.1.3 Impacts

Many of the impacts of participation in Scholarship and Mobility programs will be long term in nature and may not be fully evident for some time. In the short-to-medium term, perhaps the best indicators of impact are based on the level of Mobility activity generated and the visible changes that have been observed in and by the students participating. These include:

— the provision of relevant work experience and cementing of career goals through a placement opportunity that is not otherwise as readily available in undergraduate study

— the gaining of independence and confidence in working with co munities and professionals through exposure to officials and organisations as part of Mobility projects

— the raising of interest in developing countries and other cultures

— the promotion of overseas travel, to locations that may not have been the first port of destination, and for some at all

Impacts of the NCP were also two-way, with benefits for host location students and organisations also observed, including through transfer of skills and culture, and through direct community activities such as fundraising and volunteering.


6.2 Response to Stage 3 evaluation questions

6.2.1 Evaluation questions 1

EQ 1. To what extent do the Scholars/Mobility grant recipients perceive their NCP experience enhanced their studies, employment prospects and understanding/connections to the Indo-Pacific region?

The results of the alumni survey clearly show that both Scholars and Mobility grant recipients believe their NCP experience greatly enhanced their studies, employment prospects and understanding/connections with the Indo-Pacific region. For the alumni surveyed, the most valuable elements of the NCP experience was gaining an understanding of the host locations (97 per cent) and cross cultural skills that have aided career development (88 per cent), followed closely by greater confidence in their ability to engage with people in the region (87 per cent). On this basis, 47 per cent frequently recommend the project to other Australian students.

The extent to which NCP participants benefited personally and professional from their experience is also evident from the following findings:

— The majority of alumni (56 per cent) believe the experience contributed positively to their future career prospects and now intend to undertake further study (59 per cent) or seek employment in Australia (51 per cent) or the region (42 per cent)

— The majority of the alumni reported that the contacts they made through the NCP greatly (53 per cent) or moderately (30 per cent) enhanced their understand and connections of their host location, while many also strongly agreed (48 per cent) or agreed (40 per cent) that they gained cross cultural skills that have helped their career.

The survey also highlighted areas in which the perceived benefits could be enhanced, including:

— On maintaining links with people in the host location, the majority (84 per cent) of surveyed alumni only maintained one, while fewer (28 per cent) maintained at least four. While for some recipients, these links enhanced their understanding of the region (53 per cent) and their study opportunities (29 per cent), for others it did not achieve these goals: for 16 per cent the experience did not enhance their employment prospects, and for 12 per cent it did not enhance their study opportunities and outcomes.

— A large proportion of the NCP alumni did not take up the opportunities to connect with other alumni online, either by joining the Global Alumni Network (49 per cent) or the Alumni LinkedIn group (45 per cent). While alumni indicated that social and professional networking were useful activities to engage in, their knowledge of (and the availability of) these opportunities appeared low.

EQ 1. a) To what extent have universities enriched their existing relationships and/or created new relationships or partnerships in host locations from short and semester long Mobility projects? EQ 1. b) To what extent have universities enriched their existing relationships and/or created new relationships or partnerships in host locations from the Scholarship Pilot Program?

Universities have provided strong feedback that NCP has had a positive impact on the relationships between Australian universities and institutions in host locations. The formation of these relationships is both historical and buried through the faculties where they are operationalised through NCP funding and other joint initiatives, such as teaching and research.

Whether these relationships are new or existing is really moot point amongst all universities consulted for Stage 3. This is because some universities have too many existing relationships to effectively manage and others are in need of more formal agreements in host locations which they have not traditionally operated. What is important is that NCP provides a mechanism and a source of ongoing funding to support and nourish the relationships that universities want to pursue and feel there is value in pursing. This means the number of relationships supported or developed with the assistance of NCP is not and should not be the critical measure of NCPs benefit to a university’s engagement with the Indo-Pacific region. It is the flexibility afforded to universities by NCP which is the critical measure and by this measure, NCP has undoubtedly enriched relationships across the region.


6.2.2 Evaluation questions 2

EQ 2. What longer term benefits and challenges have participating Australian universities experience through their involvement in the NCP?

This phase of the evaluation has highlighted that there are significantly more benefits than challenges facing Australian universities which participate in NCP.

Most of the NCP’s design features are viewed favourably amongst universities consulted and pose few barriers to their ongoing participation. The design of the nomination and application arrangements, the number of eligible host locations, the guidelines and guidance provided and the funding arrangements are all features that will support effective university engagement over the longer term. Other design features such as the internships/mentorship arrangements and the design of the alumni program are fundamentally well configured, however, will take some additional time to bed down. The uptake of these design features is likely to increase as NCP matures.

Feedback given for Stage 3 about the NCP’s administration arrangements has been similarly positive. The overwhelming majority of universities hold the view that NCP is well managed by Government and have the basic systems and processes in place to deliver it efficiently and effectively. Universities have also noted that Government’s willingness to improve administration arrangements places NCP on a sound footing and means that program administration is unlikely to be a significant challenge in future participation.

That being said, there are a number of areas where universities believe that NCP’s design and administration arrangements present challenges for their ongoing participation. Most notably, a number of universities continue to argue for the removal of the age eligibility restrictions. Regional and dual sector universities, in particular, see the age requirement (albeit it having been relaxed over time) as a barrier to participation.

A number of universities also view the feedback given on unsuccessful applications, as an ongoing challenge to the NCP’s administration. These universities stress that such feedback is vital in helping universities to maintain the enthusiasm generated by NCP at the faculty level that is vital to ongoing engagement.

EQ 2. a) What longer term benefits and challenges have Australian universities experienced through their approach to developing short and semester long Mobility projects?

Consultations with universities have highlighted that many of the early challenges in the development of short term and semester based Mobility projects have now been resolved. This is most clearly evidenced by the large growth in student numbers, participating faculties, and universities since the pilot phase. The flexibility afforded to universities to apply for short and semester based program funding is considered a critical factor in developing programs which meet the objectives of universities and needs of students. A key take out is that the ability for a faculty member to choose a short or a semester based study option for his/her students is a design feature that should be preserved into the future.

In addition, some universities have cited the funding provided by NCP as an important source of assistance in developing short and semester based programs. The administration support funding offered to home universities (calculated at 10 per cent of the total funding offered for each project) was regularly used to support the development costs of programs. While all universities are naturally more funding than is offered, there was widespread recognition that the funding is an important element of program that should be preserved.

6.2.3 Evaluation question 3

EQ 3. What longer term benefits and challenges have participating host universities (in the pilot locations) experienced through their involvement in the NCP?

Evidence here is the most limited in the evaluation. It is perhaps revealing that no host partners who were consulted during Stage 2 were able to participate in Stage 3 consultations, so that it is not possible comment on the lasting impacts of their hosting activities. While home universities reported that NCP has helped them to strengthen their ongoing relationships with host locations, the Mobility student feedback obtained indicated that host location awareness was predominantly through the


home university faculties and lecturers involved rather than the NCP itself, with some not at all understanding the relationship between the students and NCP. As such, strengthening and formalising the linkages between the host locations and the NCP could be an area on which to focus in the future.

6.2.4 Evaluation question 4

EQ 4. To what extent have Scholars/Mobility grant recipients participated in community development/advocacy activities on their return to Australia?

An early NCP objective was to develop an alumni community that engages grant recipients and provides an opportunity for them to share their experiences, promote the NCP and continue to develop knowledge of professional links with the Indo Pacific region. Evaluation reports for stages 1 and 2 noted that it was not possible to assess the contribution of alumni at these early phases, and that evidence would be gathered and analysed in Stage 3.

The survey of pilot phase participants provides some insight into recipient’s participation in development and advocacy activities, and the issues and opportunities the alumni program could address.

Participation in alumni activities

Survey responses show that alumni were mainly engaged in social networking (which may be offline, online or inter-personal) with other NCP students and professional networking associated with an NCP alumni event. Alumni were less engaged in opportunities for mentoring, presentations and training events and job placement activities. The high levels of ‘don’t know/no applicable’ in the survey results suggests that alumni were not made aware that activities were available to them, or that availability varied across the country.

Alumni participation in online opportunities was low, with only 4 per cent of survey recipients engaging in these offerings. Given, however, that the suite of online options has matured considerably since the pilot project, current surveys may find alumni’s interest in social and professional networking are better catered to. The department’s investment in using online tools to meet the objectives of the Global Alumni Engagement Strategy (2016-2020) is a clear indication that digital engagement is the preferred method of supporting and continuing to build the alumni community. Efforts to link to and share content with the online initiatives of established alumni programs in participating universities in Australia and host locations may help to grow this further. Alumni could be further encouraged to participate or advance the NCP by requiring a commitment to do so in the program selection criteria or incentivising it through a small grant for digital content production on return.

The survey results suggest that the most useful activities to alumni are job placements, professional networking and professional development opportunities. Social networking, access to mentors and participation in program development, such as delivering training, were considered less useful. This suggests alumni are strongly motivated by activities that enable them to capitalise on and advance the professional dimensions of their experience in the region, and that contributing to the development of the program or assisting other participants may be considered a less effective way of achieving this.

Advocacy and promotion

The survey found that alumni are generally motivated to promote the NCP, particularly to Australian students, and do so frequently. Over half of the alumni indicated a willingness to speak at schools, mentor NCP students, help with pre-departure briefings and get involved in media opportunities.

Given however that alumni readily promote the program to students, and generally favoured professional development opportunities over ‘giving back’ to the program, the program could be further promoted as providing professional development opportunities, such as public speaking with the business or research community.

NEW COLOMBO PLAN INDEPENDENT EVALUATION A–1

A . S T A K E H O L D E R S C O N S U L T E D

A Stakeholders consulted

A.1 Australian universities consulted

Table A.1 lists the stakeholders from Australian universities who were consulted for Stage 3. These universities were consulted in Stage 1 of the evaluation.

TABLE A.1 CONSULTATIONS WITH AUSTRALIAN UNIVERSITIES FOR STAGE 3

Date Stakeholder Institution

25 November 2016

Ashley Tanks

Peter Day

University of Wollongong

Jeannette Geesmann Murdoch University

29 November 2016

Victoria Heron

Brianna Watt

Deakin University

Glen Stafford

Chris Hoffmann

University of Adelaide

1 December 2016

Daniel Mather

Takayo Yoshida

Flinders University

John McKinnon University of New England

Jess Gallagher

Heidi Benjaminson

University of Queensland

Jessica Gordon

Eva Chye

Joel Wittwer

University of Western Australia

2 December 2016

Tilly Hickenbothan

Emma McCullough

University of Canberra

Jogvan Klein RMIT University

Jess Cronin

Kristina Tan

Khanh Hoang

Australian National University

6 December 2016 Cara Bonnington

Leonie Patrick

Sydney University

NEW COLOMBO PLAN INDEPENDENT EVALUATION A–2

Date Stakeholder Institution

7 December 2016

Stewart Collins

Mervyn Chong

Swinburne University of Technology

Kayla Warner

Aiko Fujino

Bond University

8 December 2016 Tiffany Forbes Federation University

9 December 2016 Steven McDonald

Jo Byng

Western Sydney University

21 December 2016 Julie Lambert Griffith University, Gold Coast Campus

23 December 2016 Belinda Price

Benita Ho

Melbourne University

1 February 2017 Rachna Chandra

Wendy Tran

Monash University

15 February 2017 Shevahn Telfser Charles Sturt University

27 February 2017 Steven Morris Charles Darwin University

3 March 2017 Marnie Slaghuis University of Tasmania

A.2 Host location consultations

As part of the evaluation plan, ACIL Allen agreed to consult with the same host location organisations consulted for the Stage 2 report. Interview requests to these host location contacts were sent to the following institutions on 11 January 2017, although an initial contact had been made late-2016:

— Nanyang Technological University, Singapore (Scholarship and Mobility)

— University of Hong Kong, Hong Kong SAR (Scholarship and Mobility)

— University of Tokyo, Japan (Scholarship)

— University of Gadjah Mada, Indonesia (Scholarship)

— Nagoya University, Japan (Mobility)

— Asian Women’s Welfare Association, Singapore (Mobility)

The request was also sent to the following “back up contacts” on 5 August:

— University of Singapore, Singapore (Scholarship)

— Okayama University, Japan (Mobility)

— Third party provider, Indonesia (Mobility).

University of Tokyo and Asian Women’s Welfare Association responded that they did not wish to participate.

A follow up request was sent on 17 February 2017, but no further responses were received.

A.3 Student workshops

The following on-campus workshops were held with NCP students for Stage 3:

— University of Melbourne—28 March 2017—8 Mobility students.

— Deakin University—19 May 2017—1 Mobility student and 1 Global Mobility staff member.

B . C O N S U L T A T I O N Q U E S T I O N S

B Consultation questions

Three consultation guides were developed and used for Stage 3 consultations with universities, host locations and students. The consultation guides were developed in consultation with Government which provided feedback on draft questions before they were cleared for use in the evaluation.

The consultation guides offered participants across the stakeholder groups an opportunity to consider similarly themed questions and provide a response to each question from different perspectives. In some instances, this approach provided ACIL Allen with opportunities to triangulate consultation feedback across the stakeholder groups with survey data. In other instances it offered opportunities to explore similar issues from a range of different perspectives. The questions were semi-structured in nature to allow stakeholders the flexibility to explore issues which were not readily apparent from the consultations. Each consultation guide used for this stage of the evaluation is provided below.

B.1 Home university consultation questions

Over the past two years ACIL Allen consulting has been undertaking an independent evaluation of the pilot phase of the New Colombo Plan (NCP) for the Departments of Education and Training and Foreign Affairs and Trade. As part of the evaluation, we are meeting with a selection of university NCP contact officers and senior representatives to capture their reflections on the pilot phase, particularly now the majority of the 2014 pilot students have completed their programs.

The purpose of each meeting is to understand the experiences of your university, your students, and their host location and other partners in relation to the pilot programs. The areas where we are particularly seeking feedback are outlined below, though there will also be an opportunity for open discussion on other relevant matters where required.

Please note that we are treating all consultations confidentially. We will report the outcomes of the meeting at a thematic level only.

As a key stakeholder to the NCP and someone who may have participated in the pilot programs, we are seeking your views on the NCP as we head into the final stages of the evaluation.

B.1.1 Understanding

6. Please outline your role and interactions with the pilot phase and the NCP more broadly.

7. Was this for the Mobility or Scholarship pilot (or both) programs?

B.1.2 Program design

The pilot programs each had a number of important design elements, which are: eligibility criteria; nomination processes; application processes; selection criteria and processes; guidance/guidelines; and funding.

8. Did the design of the pilot programs effectively support the introduction of NCP and its continued roll-out and evolution? If yes/no, why?

B.1.3 Administration

9. How would you characterise the effectiveness, efficiency and appropriateness of administrative arrangements of NCP during and after the pilot programs?

10. What, if any, changes are needed in the NCP’s administration to support its continued roll-out and evolution?

B.1.4 Benefits

11. What do you see as the most (or least) valuable aspects from the university’s participation in the pilot programs? [Please also reflect on the feedback given by the students and host location contacts who participated in the pilot programs, if available]

12. Were there any unexpected benefits or outcomes from participation in the pilot phase and the broader roll-out of the NCP?

13. Were there any benefits that were missed/not captured that could have been? These could include benefits to students, universities, host locations and businesses who participated in the pilot phase and the broader roll-out of the NCP?

B.1.5 Lessons

14. Did your assessment and expectations of the pilot phase change throughout the pre-departure, in-country and return phases?

15. What do you consider to be the key lessons from the pilot programs?

16. Do you believe these have been factored into the broader rollout of NCP or still require further emphasis?

B.1.6 Other

17. Do you have any other any views or perspectives on the pilot programs not already covered?

B.2 Host location consultation questions

B.2.1 Understanding

1. Please outline your role and interactions with the NCP since its introduction in late 2014?

2. Was this for the Mobility or Scholarship (or both) components of the NCP?


The NCP has a number of important design elements, which are: eligibility criteria; nomination processes; application processes; selection criteria and processes; guidance/guidelines; and funding.

3. Has the NCP’s design effectively supported your organisation’s engagement with the Mobility and/or Scholarship programs? If yes/no, how?


4. How would you characterise the effectiveness, efficiency and appropriateness of administrative arrangements of NCP?

5. What, if any, changes are needed in the NCP’s administration to support its continued roll-out and evolution?

B.2.4 Benefits

6. What do you see as the most (or least) valuable aspects from your organisation’s participation in the NCP? [Please also reflect on the feedback given by the students and Australian universities who participated in the NCP, if available]

a) Are any of these benefits able to be quantified?

7. Were there any unexpected benefits or outcomes from participation in the NCP?

8. Were there any benefits that were missed/not captured that could have been?

B.2.5 Lessons & expectations

9. Has participation in the NCP met, not met or exceeded your organisation’s expectations? If yes/no, why?

10. What do you consider to be the key lessons from participation in NCP?

B.2.6 Other

11. Do you have any other any views or perspectives on the pilot programs not already covered?

B.3 Student consultation questions


The pilot programs each had a number of important design elements—eligibility criteria; nomination processes; application processes; selection criteria and processes; guidance/guidelines; and funding.

1. Were these design elements appropriate for the initial application, undertaking of your Mobility project and post-NCP alumni activities? Could they be improved in any way?


2. How would you characterise the effectiveness, efficiency and appropriateness of administrative arrangements of NCP during and after your Mobility project?

3. What, if any, changes would help improve administrative support for the Mobility projects?

B.3.3 Benefits

4. What do you see as the most (or least) valuable aspects of your participation in NCP?

5. Were there any unexpected benefits or outcomes from your participation?

6. Were there any benefits that were missed/not captured that could have been?

B.3.4 Lessons

7. Did your assessment and expectations of the Mobility project change throughout the pre-departure, in-country and return phases?

8. What do you consider to be the key lessons for the NCP from your Mobility project experience?

B.3.5 Other

9. Do you have any other any views or perspectives on your participation in the NCP not already covered?

ACIL ALLEN CONSULTING PTY LTD

ABN 68 102 652 148

ACILALLEN.COM.AU

ABOUT ACIL ALLEN CONSULTING

ACIL ALLEN CONSULTING IS ONE OF

THE LARGEST INDEPENDENT,

ECONOMIC, PUBLIC POLICY, AND

PUBLIC AFFAIRS MANAGEMENT

CONSULTING FIRMS IN AUSTRALIA.

WE ADVISE COMPANIES,

INSTITUTIONS AND GOVERNMENTS

ON ECONOMICS, POLICY AND

CORPORATE PUBLIC AFFAIRS

MANAGEMENT.

WE PROVIDE SENIOR ADVISORY

SERVICES THAT BRING

UNPARALLELED STRATEGIC

THINKING AND REAL WORLD

EXPERIENCE TO BEAR ON PROBLEM

SOLVING AND STRATEGY

FORMULATION.

report to department of education and training...

Documents