1/47

NATIONAL REPORT ON THE IMPLEMENTATION AND IMPACT OF ERASMUS+

NATIONAL REPORT – HUNGARY

Budapest, 30 June 2017

Ministry of Human Capacities

Contact:

Ministry of Human Capacities

1054 Budapest

Akadémia u. 3.

Hungary

Ref. Ares(2017)3349205 - 04/07/2017

2/47

Table of Content

1 Executive Summary ................................................................................................................. 3

2 Methodology for the preparation of the National Report, role of actors .................................... 5

2.1 ERASMUS+ Programme in Hungary ............................................................................... 5

2.2 Aim of the research ......................................................................................................... 5

2.3 Scope of the research...................................................................................................... 5

2.4 Research methodology of interim evaluation ................................................................... 6

2.4.1 Document and database analysis ............................................................................ 6

2.4.2 Empirical data collection .......................................................................................... 6

2.4.3 The preparation of the National Report .................................................................... 7

3 Answers to standard questions ................................................................................................ 7

3.1 Effectiveness ................................................................................................................... 8

3.2 Efficiency ....................................................................................................................... 17

3.3 Relevance ...................................................................................................................... 23

3.4 Internal and external coherence and complementarity .................................................. 25

3.5 European added value and sustainability ...................................................................... 27

4 Conclusions and suggestions for improvements to Erasmus+ and for a future programme .. 29

5 Annexes ................................................................................................................................. 32

3/47

1 Executive Summary

Erasmus+ is the EU Programme in the fields of education, training, youth and sport for the period 2014—2020 which was introduced in 2014 by the European Commission based on the results achieved by the European programmes on education, training, youth and sport of over 25 years. The aim of the Programme is to provide support for the Programme countries in the effective exploitation of the European talent and social capital within the perspective of lifelong learning by enhancing formal and informal learning in the fields of education, training, youth and sport. The Programme also creates further opportunities for cooperation and mobility between the partner counties.

The National Report was prepared by the Ministry of Human Capacities with the involvement of the National Agency (hereinafter referred to as: “NA”) and external experts. The assessment and analysis of the national reports will be followed by policy and finance consultations at the European Union level and the European Commission’s proposal for the new programme is expected to be made by the end of 2018.

The objective of the interim evaluation of the Programme is to examine the effectiveness of the integration of the predecessor programmes, and of the new programme structure in the achievement of objectives and intended impacts; furthermore, it aims to specify development proposals which — based upon the experience gained during implementation — may support the further development of the Programme as well as the planning of the successor programme.

The complexity of the Programme necessitated the sequential application of the following research methods during the research conducted in preparation for the interim assessment: the examination of yearly reports, evaluations, and participant’s reports within the framework of document analysis supplemented by online questionnaires for the project promoters, focus group interviews and individual interviews as well as SWOT analysis conducted by the NA. The report was made according to the structure defined by the Commission and contains answers to all the questions raised by the Commission.

Conclusions

Erasmus+ Programme is well known and very popular in Hungary. There is no international or national competitor to it.

The objectives of the Programme coincide with the objectives of education policy in Hungary and thus they influenced the legislative changes in HE (hereinafter referred to as: “HE”), especially in the area of strengthening the internationalisation of institutions in HE. The internationalisation of organisations at an institutional level in every sector has clearly expanded and in many cases the objectives of the Programme has been integrated in the strategic documents. All in all the general and specific objectives of the Programme have been successfully realised, however in Youth the realisation is less efficient - mainly due to difficulties regarding the inclusion of target groups with disadvantaged backgrounds.

There is no doubt that the predecessor programmes in Education and Training has been successfully integrated into the Erasmus+ Programme, however, in Youth the integration has not been complete.

The structure of the Programme is clear at the level of the applicants as well as at the level of the NA; the establishment of sectoral cooperations is underway.

At the same time, certain elements of KA2 strategic partnerships are not sufficiently clarified and thus difficult to interpret.

The administration of the Programme has been simplified and it has been confirmed by both the applicants and the NA.

Nevertheless, several problems have been identified, especially with regard to redundant questions in the application forms and reports. Furthermore, the ICT systems supporting the administration have been strongly criticised by both the applicants and the NA.

4/47

Currently the available funds are only sufficient for the implementation of the granted projects, however the quality threshold for funding is very high in several sectors, and applications of high quality cannot enter the system. For the time being resources are sufficient for the national level management of the Programme.

The implementation of the projects is largely assisted by the supportive attitude and activities of the NA.

Major suggestions

It is recommended that the Commission rely more on the experience of the NAs and involve them in the preparation of the policy decisions as well as in the planning of the successor programme in a planned manner.

The representation of some sector specific elements – especially in Youth and in Vocational Education and Training (hereinafter referred to as “VET” – are recommended.

The involvement of target groups with disadvantaged background needs more intense support.

When establishing the successor programme, the structure of Erasmus+ Programme is worth to be preserved and the system should only be ‘fine-tuned’ based on its focal points.

Based on the needs of Youth the activity “Transnational youth initiatives” should be represented in the successor programme again; KA107 needs to be significantly simplified and approximated to the operation of KA103.

It should be laid down that the initial funding for a sector/action should not be less than the amount provided for the same sector/action in the last year of the previous programme.

It is also worth reviewing the distribution of funding among the sectors and/or providing further funding to achieve the objectives in sectors where the demand significantly exceeds the available funds. Furthermore, it is unavoidable to revise and rethink lump sums.

The administrative procedures of the NA should be made more coherent and the types of data collected should be rationalised and then used to create indicators which are easy to understand for the applicants. This is in particular important at the beginning of the new programme phase.

The review of the ICT systems is strongly recommended. The necessary changes should be made based on the needs assessment of the applicants and the NA as well as on the advantages of the current systems (without developing a completely new system). The tools should be adjusted to the required contents and not the other way around.

Applicants need to be supported in sustaining the results.

5/47

2 Methodology for the preparation of the National Report, role of actors

2.1 ERASMUS+ Programme in Hungary

In Hungary the NA operating within the framework of Tempus Public Foundation (hereinafter referred to as “TPF”) is responsible for the implementation of ERASMUS+ Programme (hereinafter referred to as “the Programme”) in each sector (education and training, youth). The Programme is supervised by the National Authority, that is the Ministry of Human Capacities (hereinafter referred to as “MHC”). The work of the National Authority and the NA is supported by an independent audit body selected through public procurement.

The following table shows the decentralised actions of the Programme in Hungary.

Field Action

KA1 KA2 KA3

Higher Education Mobility between Programme Countries (KA103)

Strategic partnerships in HE (KA203)

-

Mobility between Programme and Partner Countries (KA107)

-

Higher Education VET mobility/Mobility in VET (KA102)

Strategic partnerships in HE (KA202)

-

School Education Teacher mobility (KA101) Strategic partnerships in school education (KA201)

-

- Strategic partnerships for schools only(KA219)

-

Adult Learning Mobility for adult education staff (KA104)

Strategic partnerships in adult education (KA204)

-

Youth Youth mobility (KA105) Strategic partnerships in Youth (KA205)

Structured dialogues (KA347)

TPF commissioned an external expert body through public procurement to carry out the research related to the preparation of the Report. The expert body analysed the achievements in the implementation of the Programme in Hungary and made a proposal for future developments. The findings of the research were validated by experts from TPF and MHC and the National Report was prepared as a result.

2.2 Aim of the research

The aim of the research was to map out the extent to which the Programme achieved its original objectives during its implementation in Hungary as well as how the Programme relied on the results of its predecessor programmes (Lifelong Learning Programme – hereinafter referred to as “LLP” and Youth in Action Programme – hereinafter referred to as “YiA”) and to clarify the ways how such results can be used to further develop the Programme at national and EU level.

2.3 Scope of the research

The interim evaluation report covers the beneficiaries of the decentralised actions of the Programme available in Hungary as well as the organisations responsible for the coordination and management of the Programme.

6/47

2.4 Research methodology of interim evaluation

During the research conducted to prepare the Report the following methods were applied:

2.4.1 Document and database analysis

The documents and databases used for the survey are listed in Annex 1.

2.4.2 Empirical data collection

2.4.2.1 Data collection through online questionnaire

In order to collect data from the organisations taking part in the implementation of the Programme a user friendly automated online questionnaire was launched. By means of a well-structured data collection providing an amount of data which was sufficient for statistical analysis, the continuous, quick and easy availability of the data was ensured.

The online questionnaire’s scope covered all the organisations that have taken or are taking part in the implementation of the Programme as beneficiaries (100% coverage).

The questionnaire was anonymous and voluntary, available on the website of the expert body, however it contained questions which made it possible to identify the sector and the type of the measure in which the respondents were involved on an annual basis.

TPF invited the project coordinators of the beneficiaries to fill in the questionnaire thus ensuring the expected return rate (of 30%<). The expert body carrying out the research provided online and telephone support to assist coordinators in filling in the questionnaire and to answer the questions that may arise.

The electronic version of the questionnaire consisting of 21 questions was uploaded on the website on 29 March 2017. Following the testing, the questionnaire was first fully filled in on 5 April 2017 and the last, 309th full completion dated 30 April when data collection was closed. The return ratio thus is 33%.

The raw data from the questionnaires processed in the first round served as input information for the SWOT analysis as well as for the focus group and individual interviews. See Annex II. for the detailed analysis of the answers given to the questions in the questionnaire.

2.4.2.2 Focus group discussions

In order to clarify the result of the questionnaires and to obtain more detailed information 10 focus group discussions were conducted with the involvement of the most important stakeholders of the different actions. Annex III. contains the list of focus groups.

The relevant staff of TPF observed the focus group discussions which made it possible to use the shared information in the SWOT analysis (see the next section below).

2.4.2.3 SWOT analysis

A SWOT analysis was conducted together with the managers of TPF involved in the different actions with the aim of evaluating the Programme’s structure and implementation and specifying proposals for development.

2.4.2.4 Individual interviews

Individual interviews were made with the following actors responsible for the implementation of the programme:

representatives of MHC;

the chairman of TPF’s Board of Trustees;

the director of TPF;

managers at TPF responsible for the implementation of ERASMUS+ Programme.

7/47

The objectives of the individual interviews were to evaluate how the Programme built on the predecessor programmes (LLP and YiA), to identify the Programme’s role in the policy, and to specify development proposals at programme level. To this end, a general guide was provided for the interviews with actual questions regarding the three topics mentioned before.

2.4.2.5 Phone (Skype) interviews

For the evaluation of the Programme it was important to collect data from unsuccessful applicants and from those who did not apply. Three actors from each sector were interviewed via phone or Skype.

2.4.2.6 Consultations

During the research process the experts held two consultations with the experts of TPF and MHC on order to have the findings and proposals in the report validated.

2.4.3 The preparation of the National Report

Based on the results of the research the National Report was compiled according to the requirements set in the English document entitled “National Reports on the Implementation and Impact of Erasmus+”.

3 Answers to standard questions

The document analysis shows that there is an overall satisfaction with the implementation of the Programme in Hungary, highlighting its high, in certain cases excellent quality level. With no exception, the European Commission gave positive feedback to all the yearly reports by the NAs (with regard to years 2014 and 2015, the assessment of the yearly report for 2016 is still in progress) and confirmed to the Hungarian NAs that the implementation of the Erasmus+ Programme is well on track.

The feedback by the European Commission with regard to the Programme’s management can be summed up by the following quotation: “Based on the information provided in the Yearly NA Report, the NA has managed the programme properly including the high level of funds absorption and overall compliance with the programme requirements.”

1

The attainment of programme objectives was also similarly confirmed with regard to both 2014 and 2015.

It is all further supported by statistics which show that the implementation of the Programme in Hungary is highly successful: its absorption is almost full which is unique compared to such ratios of other Programmes. It is as much the result of the effective dissemination of information and the supportive attitude towards the applicants by the NA as it is of the more flexible financial structure of the Erasmus+ Programme.

Next, we will describe the implementation of the Programme through the answers given to the 21 questions put forward by the Commission.

1 Yearly Report 2014—regarding E&T and Youth Programmes

8/47

3.1 Effectiveness

1. To what extent have Erasmus+ and its predecessor programmes contributed to the realisation of the Erasmus+ specific objectives (as listed in point B.2 in annex 3) in your country? Are there differences across fields? Please provide, where relevant, your assessment for each of the specific objectives and provide evidence and examples where possible.

Based on the reviewed documents (reports by TPF and their evaluations by the EC) the objectives were in general realised.

The project promoters addressed by the questionnaire had to assess whether the objectives included in the documents related the general and specific objectives of the Programme (see Annex 4) are relevant – and if they are, to what extent – in their own projects. They also had to evaluate the realisation of the objectives they found relevant during the implementation of the projects.

It was the expansion of the institution’s internationalisation that received the highest evaluation as a realised specific objective with highest average value - in all the fields but Youth. Three fields (VET, School Education, Adult Learning) found the sharing of good practices the most relevant objective.

Project promoters in general ranked the relevance of objectives higher than the realisation of them. This shows that the objectives of the Programme remain relevant, however, at the level of project implementation these objectives are not met in every case.

The objectives of internationalisation in Adult Learning and in Youth the improvement of the level of key competences and skills needed for social inclusion and employment, sharing good practices, and the improvement of the teaching and learning of languages create exceptions - regarding these objectives project promoters ranked realisation higher than relevance.

The well-defined and established target structure – which is a result of a long development – completely covers the target groups and provides individual learning opportunities for each participant as it is confirmed by both the focus group discussions and the individual interviews.

According to questionnaires filled in by participants in 2014-2016 the Programme almost entirely supported the improvement of the skills of the participants. The satisfaction level regarding the development of the general (problem solving, creative, analytical), practical, and professional skills of the participating pupils and students is above 90% in general - although it differs. There is a similar level of satisfaction among teachers, trainers, and youth specialists participating in Mobility.

They have learnt field specific or practical skills relevant for their professional improvement, they have expanded their knowledge in the field of new methods for evaluating and acknowledging competences and skills acquired through formal/informal learning, and their sensitivity to issues related to social inclusion of disadvantaged people has clearly developed.

Another important result is the high level of satisfaction about the development of independent learning and language skills in both sectors (Education and Training, and Youth) and in both target groups (pupils, teachers, and experts).

2. To what extent has the progress on the realisation of the specific objectives contributed to the realisation of the Erasmus+ general objectives (as listed in point B.2 in annex 3) in your country?

In the questionnaire – similarly to the previous question – project promoters had to evaluate the objectives set in the documents related to the general objectives of the Programme (see Annex 4) from the point of view of relevance and realisation. They found the objectives relevant - typically, the level of relevance was similar to that of the actual realisation in their projects.

The project promoters in general gave fewer scores for the relevance and realisation of objectives with regard to general questions - compared to those of specific objectives. It shows that the project promoters found it more difficult to interpret the general objectives.

9/47

Even among the generally low level of relevance values, the attitude of HE towards two objectives – namely reducing the rate of early leavers aged 18-24 from education and training and ensuring that people aged 30-34 complete some form of higher education – is noteworthy, especially because these two objectives can be related to HE. Yet the average value of the relevance of the former objective is significantly higher in VET. The reasons for this may be subject to further examinations.

Among the general objectives we have not found any with a significantly higher evaluation regarding relevance in any field. In HE “graduates should spend some time studying or training abroad”, in VET and School Education “the improvement of teachers’ and trainers’ competences in methodology”, in Adult Learning “providing opportunities for participation in lifelong learning”, and in Youth “providing more and equal opportunities for young people in education and the labour market” were indicated as the most relevant factors along well identifiable sectoral interests.

The focus group discussions confirmed that general and specific objectives can only be interpreted together, and from this respect – apart from Youth – the vast majority of the sectors are satisfied with the objectives and find them relevant.

None of the reports have indicated any related problems in Education and Training. The summary of the youth projects referred to before, however, also indicates some shortcomings: according to the NAs the objectives of the Programme are too general and abstract, and cannot be sufficiently related to the practical youth policies. This problem could be best addressed in Hungary by providing an opportunity for applying national priorities.

3. To what extent have Erasmus+ actions influenced policy developments in the domains of education and training, youth and sport in your country? Which actions were most effective in doing so? Are there marked differences between different fields?

Due to the international nature of HE the reintroduction of national priorities into this sector is not justified, however, within the domain of youth the policy of the Hungarian government emphasises the importance of supporting families and enhancing employment; these significant national priorities may contribute to the integration of the Programme into the Hungarian policy.

Higher Education

Based on the yearly reports and other documents as well as on the results of the empirical data collection conducted as part of the research preparing the report it is clear that the HE field has been embedded in policy the most. It is partly the result of the fact that the Erasmus Programme has been active in Hungary for almost 20 years and has significantly contributed to the internationalisation of HE institutions which tendency has been further strengthened by the Bologna process.

The internationalisation of HE institutions has been also supported by the introduction of European instruments (such as ECTS) and by the establishment of the European HE Area. The delivery of the Leuven objectives also strengthens the commitment of HE and the national implementation of the Programme: “In 2020, at least 20% of those graduating in the European HE Area should have had a study or training period abroad.”

2

The government involved TPF – responsible for the implementation of the Programme – into the development of the new concept of HE (Change of Pace in HE—Mid-term Policy Strategy 2016

3)

thus the objectives of the Programme are clearly represented in this document.

At the same time the Mobility Committee4 was established on 1 April 2016 to develop and

improve the criteria of a unified scholarship policy for mobility in HE. The Committee had six

2 Communiqué of the Conference of European Ministers Responsible for Higher Education, Leuven and Louvain-

la-Neuve, 28-29 April 2009 3 Government Decision 1785/2016 (XII.16.) on the adoption of „Change of Pace in Higher Education–Mid-term

Policy Strategy 2016” 4

Government Decree 1193/2015 (III.31.) on the establishment of Mobility Committee

10/47

members delegated from MHC, the Hungarian Rectors' Conference, the Hungarian Academy of Sciences, and the National Association of Student Governments.

The introduction of the mobility window5 also shows synergy with the Programme: the provision of

an international mobility window embedded in the curriculum for students to participate in a study or training period abroad is a prerequisite for launching a bachelor or master programme.

The NA received grants from the European Social Fund to enhance the internationalisation of HE under two governments in succession (Campus Hungary and Campus Mundi programmes).

From the beginning of academic year 2017/2018 the education and output requirements have to be expressed in learning outcomes

6 and this can definitely be interpreted as the success of the

EU policy (thus indirectly as the success of the Programme).

Foreign language learning has been brought into the forefront by the government upon the recognition that Hungarian young people cannot join international programmes without the necessary foreign language competences.

School Education

The popularity of the Programme in the institutions of shcool education is indicated by the amount of funds applied for exceeding the available resources (see Question 5).

Vocational Education and Training

Making VET attractive as one of the objectives of the European Union is strongly promoted in Hungary as well (e.g. by strengthening practical training and putting emphasis on competences relevant in the labour market). VET mobilities are extremely popular among VET institutions. The two main reasons for this are that, on the one hand, these institutions can participate in the Leonardo da Vinci Programme since 1995 and, on the other hand, the spirit of the Programme has permeated the sector through ECVET and other instruments (e.g. EQAVET) - (LLP and ERASMUS+) because mobilites have provided opportunities for trying and getting to know these instruments.

Adult Learning

The adult learning sector is open to identifying the possible connections with the policy, however due to its characteristics (e.g. many actors are involved) it can only be embedded in the policy in the long run.

Youth

Feedback from Youth is rather negative here - the structures of KA2 and KA3 are criticised in their current forms in the Amsterdam Summary referred to above as not being suitable for serving the objectives in general, and based on the focus group discussions and individual interviews this is also the case in Hungary.

Community development as one of the priorities of the European Union is also of importance for the Hungarian government. The National Youth Strategy (2009-2024)

7 adopted in 2009

authorised the Minister responsible for the area to draw up the scheme of voluntary and international mobility programmes aiming at acquiring work experience within the organised framework of child and youth organisations and public education institutions.

Sport

The integration of resources allocated for sport into the Programme was a significant milestone of the EU sport policy development. In Hungary the resources of the Programme are used to support the delivery of certain national sport policy objectives. The sport related calls for

5 Government Decree 87/2015. (IV.9.) on the implementation of certain provisions of Act CCIV of 2011 on

National Higher Education 6 EMMI Regulation 18/2016. (VIII.5.) on the amendment of EMMI regulation 8/2013. (I.30.) on the education and

output requirements of vocational trainings and bachelor and master programmes in higher education, and on the common requirements for teacher training and the education and output requirements of certain teacher training programmes 7

Parliament Resolution 88/2009. (X.29.) OGY

11/47

proposals of the Programme can contribute to the development of the international network of the Hungarian sport organisations and to the adaptation of the good practices in Hungary.

4. What specific approaches (such as co-financing, promotion or others) have you taken in order to try to enhance the effects of Erasmus+ in your country? To what extent have these approaches been effective? Can any particular points for improvement be identified?

According to the feedback, the key to the enhancement of the effects of the Programme is to provide thorough preparation and ongoing support for the applicants.

The technical work and applicant friendly attitude of TPF coordinating the implementation in Hungary has significantly contributed to the enhancement of the impacts of the Programme. The focus group discussions confirmed that TPF is characterised by a problem-solving attitude, they are also ready to help and give advice, and they undertake ’indirect’ activities related to the implementation of the Programme in Hungary.

Respondents have highlighted the seminars and events on Programme implementation and they found them well-structured using varied methodologies and helpful for the quality implementation of the projects. Several sectors have indicated though that beside the events held in the capital more easily accessible country events would also be welcomed.

The youth sector welcomes the three deadlines a year where only there are only a few granted applications but the calls are open all year round which makes planning easier for them.

The NA has identified the following areas for further improvement: strengthening the institutional approach in KA1, planning and measuring institutional impacts, further deepening of knowledge with regard to the application of learning outcomes. The quality assurance of courses in School Education is a crucial point; the platform School Education Gateway contributes to the improvement of the situation but there is still a lot do.

In some cases the competence of the experts assessing the applications and professional feedback were indicated by the project promoters as areas to be improved, especially in Youth, where they do not always find the professional reasons explaining the refusal justified and they miss detailed feedback for the final reports.

5. Do you consider that certain actions of the programme are more effective than others? Are there differences across fields? What are the determining factors for making these actions of the programme more effective?

If we want to define effectiveness by reaching the target groups and their involvement in the application processes, it can be shown from the ratio of the number of applying institutions to the number of finally granted institutions. In each sector the funding needed by applications submitted exceeds the available budget (the only exception is KA103). The relevant data are contained in Annex 6.

The participant satisfaction in KA1 is outstanding, over 95%, in each sector. The detailed information can be found in Annex 7.

Regarding the actual influence on policy development and the impacts on the target group, it is HE that comes first. The difference among the sectors only shows when we examine the clarity of each action. However, the structure of the Programme (KA1, KA2, and KA3) does not contribute to the prevalence of sectoral differences: mobility, partnerships, and policy reforms can be separated in broad terms, however these are technical categories and they are difficult to communicate to the applicants.

According to the NA KA1 activities are clear, coherent, and can be similarly interpreted in each sector. (However, it is difficult to compare KA103 to other actions because here the applicants form a closed circle of institutions and a significant percentage of the grants are awarded to this certain target group and activity).

A similar, however slower, trend can be traced regarding the VET Mobility Charter. The charter-type application process—being available for VET and HE—ensures planning for the long term,

12/47

makes implementation easier, and its influence is more significant. It is mobility that can create greater impact and more visible results through the participants.

Due to their complexity the project results of KA2 are more difficult to communicate, furthermore, the number of awarded institutions is smaller and it takes a longer time for the projects produce any results. The impact of KA3 (available in Youth) cannot be larger either, because the number of awarded organisations cannot be high.

6. To what extent has the integration of several programmes into Erasmus+ made the programme more effective in your country? Do you see scope for changes to the structure of Erasmus+ or its successor programme that could increase effectiveness?

The integrated programme has the advantage that it applies simpler and more coordinated requirements, it has streamlined the overgrown activities of the previous programme, and activities outside Europe can be implemented within one programme.

The above mentioned characteristics, however, have not necessarily made the system as a whole easier to understand for the applicants, yet the harmonisation of the activities of mobility and strategic partnerships independent of the sectors have clarified the Programme. Decentralisation and thus making certain opportunities (e.g. developments) more available to the applicants are also welcomed. It is beneficial for the NA because of the need for only a single agreement (Delegation Agreement), while the access to certain activities (Europass Mobility Certificate/VET Mobility Charter), the expansion of the geographical focus, and the simplified system of grants mean benefits for the applicants. The NA supported the strengthening of mainstreaming (namely applications by institutions replace individual applications), although the applicants’ opinions in this matter are divided.

Based on the documents integration during the first year meant many challenges for both the applicants and the NA implementing the Programme. The target group still has difficulties with comprehending and understanding the programme scheme brought under one umbrella and with using the Erasmus+ Programme Guide and it was made more difficult by the limited language access, especially in 2014.

There are fundamental problems with the structure of KA2. It has been mentioned with regard to previous questions as well that, unlike in LLP

8, significantly fewer institutions can become

successful applicants due to the consortium system.

The reason for this is that the more complex a project is (with a higher number of partners and resulting in more/higher quality intellectual output), the higher the amount applied for is. As a result fewer applications can be funded from the same budgetary framework. At the same time the structure of this action channels the applicants towards the large projects thus again fewer organisations can be supported.

The youth field—especially at the level of the NAs—however is not satisfied with the impacts of the integration; according to the interim reports provided by the NAs

9 responsible for Youth the

programmes targeted at the youth cannot effectively reach out to and address the target group in the current structure. Their perception is that the current programme has taken over the structure and IT-system of the former LLP thus it is more difficult for them to connect and integrate.

There is more than one sector that is not satisfied with certain elements of the structure and in several cases the integration is perceived as a constraint. It is especially true with regard to HE where the new programme ruled out a previously well established and proven grant structure; there are questions with regard to youth and the applicants still find ‘blurred spots’ in the activities of KA2 (for details see Question 11). Activity KA3 within Youth has become devoid of substance—due to the limited budget and activities—and only a few applicants can be funded.

8 National Agencies for E+ programme, in the field of Youth, Common input for the midterm evaluation 2017

9 National Agencies for E+ programme, in the field of Youth, Common input for the midterm evaluation 2017

13/47

7. Is the size of budget appropriate and proportionate to what Erasmus+ is set out to achieve? Is the distribution of funds across the programme’s fields and actions appropriate in relation to their level of effectiveness and utility?

In general the size of the budget seems to be sufficient, however the planned budget growth is expected by all the sectors.

As it has been described before that the demand for the grants provided by the Programme in Hungary is significantly larger than the available funds. Consequently, very high quality requirements have to be set for successful applicants.

In other words, based on the figures no field can be identified where the demand for funding is less, on the contrary: each sector would be able to absorb more funding. The only exception to this trend is the mobility in HE because it is funded from other sources as well. Without exception all the other sectors would demand a larger budget size. The realisation of the objectives cannot be called into question—although under-resourcing prevents these objectives from sufficiently saturating the institutions and becoming embedded in them. Furthermore, the support of certain target groups—primarily the inclusion of disadvantaged people—cannot be adequately emphasised or coherently represented in the Programme.

At the start of the Programme the decrease of resources in an area which clearly had had a history in the predecessor programme created a major problem. It especially created incoherence in the field of Youth in the first year where the funding level of the last year of the predecessor programme has still not been achieved. (“The level of decentralised funding available in KA347 is still 25% lower [40% lower with actual reallocation] than in the last year of the previous programme [YiA 2013].”— Yearly Report 2015). In the field of Youth reallocation within KA1 is a further issue, where—according to the Agency’s considerations—more flexibility would be needed.

KA2 in school education sector has a funding framework of an approximately similar size to that of the funding available under the former LLP, although more actions have to be financed from this amount.

HE had a similar complaint with regard to the funding of student mobility and the interim assessment is actually meant to find a solution to that complaint.

(“We take note of funding issues in student mobility. We expect that the increased grant amounts selected by the NA for the 2016 Call will improve the situation. The appropriateness of grant levels will be analysed in a mid-term review of simplified grants, which may result in adaptations where needed.” Yearly Report 2015).

KA2 applicants gave feedback about the structure of the budget: costs allocated to project management are found insufficient and intellectual outputs or mobilities/multiplier events absorb a large part of the funds.

The applicants are usually able to manage the projects without providing own financial contributions, however in the field of human capacity extra resources are often needed for successful implementation.

8. What challenges and difficulties do you encounter while implementing the various actions of Erasmus+? What changes would need to be introduced in Erasmus+ or its successor programme to remedy these?

To map out the problems we used PDCA10

logic with the aim of being able to identify all the problems arising throughout the project beginning with planning through implementation till evaluation. The answers given to Questions 7 and 13 contain the details of the financial and administrative challenges encountered during the implementation of the Programme. All in all, project implementation has not been a problem for the organisations. According to the survey only 9% of the applicants indicated that they faced serious challenges during the implementation of their projects.

10

Plan-Do-Check-Act

14/47

Adaptation of project objectives to the strategic objectives of your own institution

The Programme does not replenish the resources of the applying organisations in Hungary but provides additional opportunities. This is a definite strength of the Programme because the organisations have their own strategy which they amend or complement with the objectives of the Programme (for more details consult Question 1).

Drafting and submitting applications

The focus group discussions revealed that the drafting of the applications meant serious difficulties at the beginning even for institutions that had been successful in the predecessor programmes. Today, however, there is an established circle of applicants with sufficient experience to submit high quality applications. (This may also be interpreted as a weak point of the Programme because experienced applicants ’crowd out’ the new ones.) 11% of the respondent project promoters indicated that submitting the applications was more of a problem than not. Institutions of School Education and VET in KA1 consider it desirable to change the deadline for submission for the beginning of March because this way they would have more time to draft the applications after the winter holiday and the end of the midterm.

It was difficult to find an appropriate interpretation of ’intellectual output’ for KA2 applications because the relevant requirements are not clear. The application form does not make the applicants define intellectual output clearly. It is very difficult to describe the output itself as well as its quality. Furthermore, according to the applicants’ experience, the different NAs provide different instructions for the definition of the output.

This flexibility can be interpreted as an advantage, because the ’novelty’ as output has to be placed in an international environment.

The stakeholders acknowledge that this way more freedom is given in project planning, however they would like to have a Commission guideline outlining the main framework (such as the definition of qualitative indicators).

Here we have to refer to the fact the institutions previously got used to work package based programme implementation (WP) which made planning their activities easier and the transition to a result-based approach meant a serious challenge for them. WP-based planning forced the applicants to more thoroughly consider their applications.

Nevertheless, respondents acknowledged that by such product development a ‘for-profit approach’ can be introduced to the project and forces them to plan their projects in a more cost-effective way.

The respondents actually felt the lack of the former requirement to attach a work plan to the submitted application; on the one hand, the new requirements make it easier to draft the application, yet, on the other hand, they delay the submission deadline for the project plan to a later date within the actual project period which poses serious risks regarding the implementation of the project.

Furthermore, the financing of the final output does not motivate the teachers because the institution cannot support financially the activities carried out during project implementation. The organisations has overcome this problem by the introduction of so called ’packages’ (output components).

Reaching and attracting focus groups and partners

In KA1 the insufficient language skills of the target groups create a problem. The teachers’ and trainers’ lack of self-confidence is an obstacle of participating in professional development opportunities abroad. Surprisingly enough, the same issue prevails in HE among the students. Attracting (and preparing) the students imposes an additional burden on the institutions. It is VET in particular where supported preparatory language courses would be needed because OLS (Online Linguistic Support) is not suitable for the preparation of the target group of VET pupils. One-to-one language education would also be more useful for young people with disadvantaged background (however this approach is not a viable option with smaller languages). More

15/47

emphasis should be put on the support of language learning and thus becoming familiar with other cultures.

Those participating in EVS (European Voluntary Service) work in a close cooperation, yet due to the lack of language skills they can integrate less in the local environment.

It is considered good practice, that there are institutions which organise mobility for teachers (or the heads of the institutions) to assess partner institutions so that they can make decisions about student mobility in view of their experience.

In order to prepare successful partnerships it would also be useful to provide support for preparatory visits in KA2 as well. More experienced institutions having participated in predecessor programmes have created their partner networks and work with selected and reliable partners. They tend to launch new projects with established partners and then to involve new partners in those projects. There are cases where receiving EVS volunteers has had a history while in other cases the need arises during cooperation with the partners (local indication that can be fulfilled by the applicant via the foreign partner network). Regrettably, some costs are not granted (e.g.: there is no grant for an evaluation meeting).

In Youth KA3 can attract many local young people into the Programme and the target group can be reached depending on the preparedness and network of the applicant as well as on the size of the grant.

The problem of selecting partners and cooperating with them arises mainly in KA107 (mobility with Programme and partner countries). Among the different actions this causes the most difficulties for the applicants. This is partly so because the applicants—having been socialised in Europe and having participated in predecessor programmes—know and understand the Programme’s requirements which in certain cases have to be explained to partner countries from outside Europe by them. On the other hand, regarding this action in case of each country a separate detailed justification has to be provided to explain why that certain country is considered a strategic partner by the applicant.

Each country has to be evaluated separately; the above action is complicated at the levels of both the applicants and the coordination. The categories used should not be countries but regions. Based on the focus group discussions and the individual interviews it has become clear that the requirements of KA107 should be approximated to those of KA103 (mobility with Programme countries).

92% of the respondents to the questionnaire are rather satisfied with the cooperation with the partners (50% are fully satisfied), however a strong need is expressed for the development of a European database containing quality characteristics and evaluations related to the partners.

Quality assurance of the projects

Although the Erasmus+ Programme Guide for the applicants developed by TPF in Hungarian contains proposals for the quality assurance of the projects as well as for their internal evaluation (using qualitative and quantitative performance indicators) and these tasks have to described in the application form, the internal project evaluation entirely depends on the institutions. Only very few organisations have procedures which ensure the monitoring including regular interim evaluations, and the quality assurance of the projects. However—depending on the organisational culture—, every beneficiary has developed ways to evaluate the projects and manage the problems during the actual project implementation. Even if it means extra costs, the development of a single quality assurance guide containing general indicators regarding intellectual output should be considered.

9. To what extent are the approaches and tools that are used for disseminating and exploiting the results of Erasmus+ and its predecessor programmes in your country effective? Where can you see the possibilities for improvements?

The forms, channels, and methods of dissemination applied by the Hungarian NA are acknowledged by different actors. Based on the reports the Agency owns a varied ‘toolbox’ at the

16/47

level of documents which—with its quality included—is acknowledged and appreciated by the European Commission.

This activity is clearly successful in the field of education and training as well as in the field of youth. According to the feedback provided by the EC “We appreciate that by efficient communication strategy and use of variety of tools, the NA ensures both balances country participation in the country as well as the dissemination and exploitation of programme's results.”

“We appreciate events aiming at the creation of networks and supporting finding project partners. It is also very positive that among the communication and dissemination channels, the NA uses social media and cooperates with the press.”

Examining the dissemination materials (http://tka.hu/kiadvanyok, http://tka.hu/rendezvenyek, and http://tka.hu/sajtoszoba) available on the website of the NA we can see that the Agency goes beyond the minimum requirements regarding mandatory events and organises special events (e.g. on the use of IT-tools or on special financial rules for state-owned schools) meeting the needs of the target group as well as based on the possible national features.

There is also a wide range of different publications – transcending the strict limits regarding application guidelines – available on the website of the NA such as technical summaries, compilations of good practices, studies, special guidelines and researches.

TPF prepares yearly reports for the MHC and the general public that provide information about the progress and results of the Programme, although not all of these reports are available on the website of TPF.

However, on the positive side, there are more and more informative and well-designed infographics regularly published on the website and the Agency actually created a separate subheading for them.

There are infographics which sum up information while others aim at providing special information for certain target groups - primarily students in HE.

The number of publications in English is insignificant compared to those in Hungarian. (There are two documents published in English every year: Users’ Guide to Hungary and a collection of quality award-winning projects). The Commission has not issued any guideline or requirement with regard to making information available in foreign languages thus it falls into the competence of the given country’s NA (i.e. its human capacity, financial resources, and relevant strategic decision if such exists) to decide what to publish for foreign readers and in what depth.

The answers to the questionnaire ranked the dissemination channels with regard to their effectiveness from the point of view of the applicants as follows:

website;

project information event;

press article;

newsletter;

flyers.

On the negative side, it was mentioned by the Hungarian NA and in the focus group discussions that there is insufficient time and energy for ensuring sustainability and for the dissemination of results. Dissemination as such is still not one of the strong points of the Hungarian institutions: they are not necessarily able to break down the results of a project based on the characteristics of a given target group and it is often a problem how to represent themselves in a foreign language. (“Further guidance should be provided for the preparation of the summaries of the applications and reports in English.” quoted form an interview.) They understand the requirements regarding dissemination at the end of the project, however they cannot really interpret this requirement for the duration of the development process of the intellectual output. Furthermore, the national media are not really interested - these forums can mostly be activated only at local levels.

17/47

Conclusion (Hungary)

Due to the lack of publications in English, the activities of the Hungarian Agency and the good results of the Erasmus+ Programme are not really visible internationally; TPF does not represent itself sufficiently in this area in the European region. It would be useful to at least make an English translation of the summary of the results (e.g. infographics).

Conclusions (Europe):

Funding of the projects’ sustainability should be reconsidered horizontally within the framework of the Programme. There is just a minimum requirement in the Call for Proposals in this respect and it is not embedded in the structure of the Programme, consequently the NA has no tools for monitoring. Furthermore, each application/project is evaluated separately by the experts thus the development curves of the previous projects and their relations with the latest ones, i.e. their sustainability, are difficult to assess.

If the EC would to see more significant results in this area, resources would be advised to be allocated to this activity at the level of the applicants and that of the NAs.

3.2 Efficiency

10. To what extent is the system of cooperation and division of tasks between the Commission, Executive Agency, National Agencies, European Investment Fund, Independent Audit Bodies, and Erasmus+ Committee help the efficient and well-functioning implementation of the Programme from the point of view of your country? What are the areas for possible improvement or simplification in the implementation of Erasmus+ or a successor programme?

At the start of the Programme, cooperation between TPF and the European Commission was hindered by the considerable fluctuation at the latter. Appointment of a new Director-General in late 2015 brought about positive changes: liaising between the parties improved considerably. The fact that policy or operative changes affecting the Programme are often introduced abruptly and that NA is not informed of the developments in detail and in a timely manner (e.g. Erasmus Pro) still presents some difficulties. It would be reasonable to assign the NAs a stronger role in the preparation of decisions, as was the case with the predecessor programmes.

With respect to the management, making VET and Adult Learning in Education and Training the responsibility of the Directorate-General for Employment, Social Affairs, and Social Inclusion adds further complication.

The NA has no regular working relationship with EACEA, even though both engage in very similar activities. Operation of the EACEA and the projects they manage is out of sight, despite both of them working in the same sector and having similar workflows. It would be reasonable to develop better cooperation with EACEA, and to stage and support communication and information exchange events centrally.

Operational cooperation with the MHC is regular.

Hungary is in a special situation in that the Youth sector was formerly the remit of a separate agency under the National Institute for Family, Youth and Population Policy.

This institute has been wound up without any successor, and with effect from 1 September 2016 the Youth sector was integrated into TPF which is in charge of coordinating the Education and Training.

This means that since last year coordination has been unified under the Programme, but the introduction of a new area (and organisational unit) certainly creates problems in the life of an organisation even if the two agencies had cooperated before. A Youth Advisory Committee will be set up in the near future to help strengthen external expert support.

Nevertheless, these issues can be tackled in the short term building on the decades of

18/47

experience TPF amassed as well as its long-standing practice and expertise in Youth. By exchanging good practices and tools, the two areas can help each other, for example by encouraging TCA-type activities.

TPF takes on a proactive role in the cooperation between NAs: it shares its experience with other agencies at formal and informal meetings, and adopts good practices to improve its own functioning. Activities of this sort, which facilitate horizontal learning, should be more actively supported at the EU level in the future.

Generally, the Independent Audit Body detects only a small number of financial or arithmetic errors during the annual audits, and these are always corrected as proposed. Cooperation is appropriate.

It is important to note that cooperation between TPF and the applicants was exemplary. Based on the questionnaire survey, satisfaction with the NA is 94 %. This has been confirmed by other sources as well (focus group discussions and personal interviews).

The reason is that compared with other Hungarian calls, applicants are assisted instead of focusing the communication on calling to account and sanctioning.

TPF’s more than 20 years of experience in this field makes Programme implementation much easier, its responsibilities involved the implementation of the predecessor programmes, hence it has a solid base of resources to ensure the professional realisation of projects, including assisting applicants.

11. To what extent has the integration of several programmes into Erasmus+ resulted in efficiency gains or losses for the implementation of the programme in your country, both at the level of the National Agency and on the beneficiaries' and participants' level? Do you see scope for changes to the structure of Erasmus+ or its successor programme that could increase efficiency?

In its reports the NA mentioned several difficulties, smaller and larger problems, the lack of certain documents, or the delayed access to different background tools which—with the exception of two such examples— were solved or normalised by the current phase of the Programme. This in itself does not show whether the integration of the programmes has been efficient or not, however it is clear that the processes work well in practice by now. Certain elements resulting from integration—such as the simplified funding system, the principle of lump sums, the option for cooperation between the Agencies, and the strengthening of the cross-sectoral nature—have been welcomed.

Furthermore, in its yearly reports the European Commission acknowledges the efforts made by the Agency as well as the successful use of past experience and the achievement of set objectives and indicators.

Apart from the positive feedback there are certain smaller or larger scale elements that need to be fine-tuned according to both the Agency and the applicants. Also, the sectors in Hungary are significantly divided in their extent of satisfaction with the integration.

The following three areas meet major challenges regarding the integration. These issues have not yet been fully solved since the launch of Erasmus+ and although the system operates, considerable tension has been created for those who operate it.

With regard to HE it is the project based type financing of KA1 mobility and the single funding system, where the project promoters feel, that integration resulted in efficiency loss.

The involvement of graduates is welcomed and there was a great deal of enthusiasm about activity KA107 in the beginning but that enthusiasm has decreased due to the new, previously not used processes and the fact that it is difficult to work together with totally different cultures, and the structure of the programme is found not flexible enough by the relevant actors.

The youth sector is explicitly dissatisfied with the integration according to the Amsterdam report

11—although some positive points of the changes are acknowledged—the Programme

11

Summary of the results of the remarks during the world cafe at the Business meeting in Amsterdam on the

19/47

structure, certain actions, and reaching the target group are questioned in several cases. The document referred to before acknowledges the benefits of the integration but is of the opinion that those benefits prevail mostly in offices/agencies that used to work together under the same umbrella or for those applicants who can apply in several actions. These opinions were supported by the focus group discussion thus they can be considered relevant for Hungary as well. The interview was organised with the participation of experienced applicants but even they said that it was very difficult for new applicants to get into the system because the competition is very strong.

It is the issue of intellectual output in the case of activity KA2 that has not been thoroughly clarified and raises questions in several sectors (youth, adult education). (See Answer 8)

12. Do you consider that the implementation of certain actions of the programme is more efficient than others? Are there differences across fields? What good practices of these more efficient actions of the programme could be transferred to others?

The Programme is very popular in Hungary and the demand largely exceeds the available funding.

The commitment rate of the resources is very high in every sector—close to 100%—thus it is difficult to answer the question regarding the efficiency of sectors or activities. We can only give information about actual use with regard to projects in KA1 started in 2014 however in some of those cases the financial report assessment is still in progress. At the time of the preparation of the National Report the ratio of unused sources is 10-23% according to the yearly report for 2014. (For data see Annex 5.)

There was one decisive change concerning the management of the programmes: Youth was integrated into the organisation (TPF) responsible for education and training in the autumn of 2016. The operation was continuous and the transition from the former LLP programme was smooth from this respect, there was no technical break and the processes had been established at a certain level. In Youth had been managed by the same office for quite some time, however due to the changes in maintainer there were several uncertainties (such as fluctuation resulting from short term contracts for staff); finally, following the completion of the integration process and with the Programme being brought under one umbrella hopefully these instabilities will be balanced out. Both of the organisations that merged have many years and decades of experience and they continuously strive for renewal focusing on increasing and supporting the quality of projects.

Regarding KA1 both the documents and the NA have confirmed that the activities are transparent and clear and attract many institutions and participants. The numbers are increasing with the increase of the available funding, however it is difficult for new institutions to enter the strong competition and this opinion is supported by the interview with the office as well as by the focus groups. At this point the Programme has the advantage that the focus on institution building is further emphasised by the European Development Plan and by the introduction of the VET Charter what further helps planning for a longer term. However, in HE, with regard to budget elements in KA103 and activities in KA107, flexibility could be further enhanced.

There are several questions and doubt arising with regard to KA2, in other words the application system has exposed both the applicants and the NA to several challenges. This action has not in every sector had an antecedent (e.g. it wasn’t included in Youth) and/or was realised in a different way (in vocational training Partnerships and Innovation Transfer were merged), or it was the consortium form that meant totally new challenges (e.g. in School Education). Certain elements of the action are still not completely clear: “The first year of the Erasmus+ programme presented a great challenge to the institutions applying in the school sector, … and the concept of intellectual output, strategic thinking and the content of priorities have been completely innovative and new to most applicants.” The quotation refers to the school education field, however similar difficulties were confirmed by the adult education or Youth where the concept of intellectual output had no predecessor either.

questions of the Midterm evaluation

20/47

Furthermore, we have to examine the volume of the reached target group which will definitely change regarding the LLP-Erasmus+ relation due to the modified structure and composition of action KA2. Those small-scale projects which had required smaller funding were replaced by large consortia which often carry out developments, in other words the depths of use of funds in KA2 are significantly different.

13. To what extent has the system of simplified grants resulted in a reduction of the administrative burden for NAs and programme beneficiaries and participants? Are there differences across actions or fields? What elements of the programme could be changed to further reduce the administrative burden, without unduly compromising its results and impact?

The system of simplified grants and lump sum-based budget are mentioned in every single document as beneficial modifications. These changes have clearly decreased the programme related administration for both the NAs and for the applicants. The midterm report by the NAs in the field of youth puts it this way: “Lump sums and unit costs, which have broadly been introduced in the new programme, are very much relieving the administrative burden of applicants and NAs.”

The answers to the questionnaire show that with regard to KA1-KA2 two-third of the respondents are satisfied with the impact of the simplified grants on relieving administrative burden.

However if we examine the sectors separately the results differ: for VET and School Education the ratios of satisfaction are two-third and three-fourths respectively, in the youth and adult education sector approximately half of the respondents are ‘rather satisfied’, while in the sector of HE only 38,7% of the respondents were positive about the change. The background of these results will be detailed later.

Project promoters—of course— also have to comply with the national legislation and accounting regulations where the Programme has no influence. It is a relief for the applicants anyway that they do not have to compile long lists of invoices and ‘packages’ of financial reporting documents.

There are several critical remarks about the system with regard to the sums—on one hand there is the lack of coherence and on the other hand there are country and system specific factors which would necessitate the introduction of smaller or larger sums per field or country.

The different approaches to the standardised unit cost grant scheme are well illustrated by the following example: the amount allocated for organisational support in Youth is deemed to be very small (because more activities have to be financed from this amount in comparison with the other sectors) while in HE this amount is deemed to be excessive. Lump sums are also welcomed with regard to KA1 and KA2, however the relevant amounts regarding KA2 activities, or rather the principles for their calculation need to be revised.

According to the feedback provided by the focus groups the level of administration is higher in Hungary than in other countries.

The applicants find that there are redundancies along certain components (e.g. ‘Detailed impacts’). They also note that in the reports the questions are not focused enough, often the same issue is asked about several times while certain components (see sustainability) are left out and they feel they have to produce a lengthy text.

The applicants are familiar with the Erasmus+ Programme Guide but they all find it too complex and confusing. The terminology used in the Programme presents a challenge even for the experienced applicants and it makes the newcomers uncertain. Applicants would welcome further practical guidances and advice as well as templates for the implementation—especially in KA2.

The staff costs regarding this activity are frustratingly low for Hungary compared to those for other countries and the right experts cannot be financed.

The amount allocated for project management is also deemed insufficient. The composition of the budget headings for activity KA2, and the focuses designated by the available funds and their calculation methods are criticized by many and in their opinion, this does not help the implementation.

21/47

No objection was received with regard to the structure of KA1 with which the applicants are satisfied. In HE the subsistence costs calculated in days resulted in many problems, furthermore, the too heavy administrative burden in KA107 could be improved.

The applicants appreciate that they can personally contact the NA and most of the times they give positive feedback even about the monitoring visits and audits because they feel that besides control the objective of such meetings is providing assistance.

The NA notes that follow-up in its current form is not adequately ensured in KA2 because there is no requirement for midterm report in projects with a duration shorter than 24 months, and even if such a requirement exists it is not connected to the financial report thus it is difficult to follow the implementation in line with the financial compliance.

14. To what extent are the IT tools provided by the Commission adequate for the efficient management and implementation of the programme in your country? Do they answer your needs? Give specific examples where they can be improved. Is the set of IT tools appropriate or should it cover more/less elements of the programme implementation?

IT tools have been mentioned particularly often during the evaluation by all actors and levels.

At this point document analysis relies first and foremost on the annual reports of the NA. The European Commission usually acknowledges the efforts of the Hungarian Agency.

Although keeping the EPlus Link up-to-date was a challenge in the first year (in its response to the 2014 report the EC gave a warning in this respect), there are no additional remarks of this kind to suggest that the Hungarian Agency has failed to adapt its activities to the expectations of the EC.

The EC spoke of the Hungarian Agency in high terms for its IT-related efforts, the assistance it lends to applicants, and for organising events with a special focus on IT.

Considering the feedback given by the Agency, delays in and the slowness of the development of IT tools are a general problem similarly to the time- and energy-intensity of acquiring the necessary skills. Being a resource-intensive task, neither the Commission nor the Agency could make proper plans and get prepared. The situation has more or less stabilized by now, but it is quite telling of the scale of development and of the way it takes place that there are still programmes and activities where certain IT tools are used for the first time because when the corresponding projects started the element concerned was still under development (for example, reporting).

Regarding the forms provided by the Commission, redundancies in the application forms and being short of space and characters in the forms and the reports were mentioned besides noting that having the same number of characters available for each question did not allow applicants to attach different weights to different parts of the project.

The applicants are generally satisfied with the tools, but are not able to grasp the system as a whole—even the Agency claimed that there were too many different (!) tools to be used during the project cycle.

It would be sensible to filter out the redundancies, to transfer the data requested in one system to the other one (e.g. ECAS data transferred to the report, transfer of participant data to the Mobility Tool), and to adopt a more user-friendly approach (e.g. if one partner institution is removed, other partner institutions need to be deleted too, because only the last entry can be deleted).

Apart from stressing the need for fine-tuning, applicants also cited practical, well-functioning elements. Opinions differ on whether this application form is simpler or more complex compared with other application schemes.

Overall, the form was found to be good, its verification function and the possibility to attach files were deemed useful. Nonetheless, these elements can be further improved. For example: other than EVS, programme attachments (Excel) cannot be used, and the Mobility Tool is easy to manipulate (entering an e-mail address that is not that of the participant).

22/47

Although its usefulness is generally appreciated, the Mobility Tool requires too much effort on the part of project promoters.

On the one hand, it is too demanding to remind participants to fill out the questionnaire, on the other hand, the questionnaire itself is considered inappropriate by several sectors: too abstract and very general with no room for elaboration or actual reflection on the project, it is not target group-specific, and a separate evaluation is to be made, but this is too much for the participants, especially when it comes to disadvantaged, digitally less literate target groups whose competences (or the lack thereof) are ignored in the harmonized system.

The Mobility Tool data could be exploited also for statistical purposes; however, users are yet to see its benefits as the system does not seem to be functional in this respect.

For KA3 in the Youth sector, the Mobility Tool seems incompatible with the Programme. Recording the data of programme participants demands much administration and commits substantial resources of the applying organisations which, in the absence of queries and statistics, do not see the benefits. Although the online reporting system has superseded a great bulk of paperwork and photocopying in the Youth sector, it still fails to be fully compatible with the sector, and in particular with KA3. The Youth sector does not regard the IT tools as fitting.

HE should develop its own system, especially because of the high degree of mobility, as the process in this sector entails many administrative steps rendering the added value of IT tools imperceptible. In fact, this sector is obliged to use and develop new ones.

Concerning OLS, the fact that coordinators have no authorisation to access the site presents many difficulties as they are required to present OLS to pupils or students.

In general, focus group members were not familiar with the dissemination site, although they had been informed in connection with their contractual obligation.

Furthermore, every new IT tool demands extra effort on the NA’s part: in addition to getting the guides translated—which, by the way, do not give enough information to applicants—, they need to help project promoters over the phone or face-to-face.

Besides the above, applicants noted a lot of other things of minor importance (ECAS registration, updating ECAS data, it is difficult to enter the PIC code if the training is provided in another country) which require further fine-tuning.

15. To what extent is the level of human and financial resources that is available for the implementation of the programme in your country adequate? What steps did you take to optimise the efficiency of the resources deployed for the Erasmus+ implementation in your country?

TPA regularly organises introductory sessions, further training, seminars, and experience sharing events to support applicants in project implementation. The necessity and usefulness of these events have been clearly confirmed by the answers to the questionnaire, the focus group discussions.

Professionalism and quickness are both highlighted in general but in one sector: the Youth sector is an exception where—especially at the start of the Programme—a sense of insecurity was felt more profoundly and the pieces of information applicants were given were contradictory, but these inconsistencies have been eliminated by now.

Many stressed that their personal contact with the Agency made communication and cooperation smoother.

In general, the work of the NA can be summed up in the following quotation: “TPF operates as a service provider, not as an authority” (focus group member).

Overall, applicants have a positive image of the NA, yet there are some sharp minority opinions worth clarifying when experienced in practice (e.g. The huge and totally needless administrative burden together with the late money transfers made by the NA and other difficulties made the use of own funds necessary. Extract from the responses to the questionnaire; “Ambivalent

23/47

relationship”—extract from the focus group discussion in School Education).

The rate of the Hungarian contribution provided by the MHC for management costs has not changed since 2014, but as EU funds have increased, so far this has not created any problems. To sum it up, currently there are adequate financial resources available to implement the projects.

At the institutional level expert groups have formed which are capable of dealing with the projects throughout the entire cycle, from submission to completion. In the field of education, substitution is a problem for state-owned schools because it is not eligible for financing. Moreover, parents are also displeased if a professional is absent for two weeks without any competent substitute.

Based on the empirical survey and the document analysis it can be concluded that, subject to minor corrections, the funds allocated to the specific project types are basically sufficient to cover the implementation of granted projects (see Question 7), yet the quality of projects and the number of applicants call for the increase of the total budget (see the extra applications not covered by the budget under Question 5).

3.3 Relevance

16. To what extent do the Erasmus+ objectives continue to address the needs or problems they are meant to solve? Are these needs or problems (still) relevant in the context of your country? Have the needs or problems evolved in such a way that the objectives of Erasmus+ or its successor programme need to be adjusted?

The objectives are relevant in every field (see Questions 1, 2, and 17). Their organisational embedding is further enhanced by them being included in the strategic documents—in HE they are included in the Institution Development Plans and in School Education and VET they are incorporated in the so called ‘pedagogical programmes’) of the organisations taking part in the Programme (especially internationalisation, the upgrading of language competences, and the enhancement of mobilities). At the beginning of 2016 TPF issued a methodological guidance for public (and vocational) schools regarding the development of institutional internationalisation strategies.

Furthermore, the Campus Mundi Programme coordinated by TPF also enhances the strengthening of the international network of Hungarian HE by supporting its internationalisation thus contributing to the government objective of increasing the number of foreign students arriving to Hungary from 25,000 to 40,000 by 2021.

17. To what extent are needs of different stakeholders and sectors addressed by the Erasmus+ objectives? How successful is the programme in attracting and reaching target audiences and groups within different fields of the programme's scope? Is the Erasmus+ programme well known to the education and training, youth and sport communities? In case some target groups are not sufficiently reached, what factors are limiting their access and what actions could be taken to remedy this?

The Programme is known in all sectors, and helped the development of the linguistic and intercultural competences of target groups to a great degree.

Although the Programme specified the target groups properly, all respondents agreed that regarding KA2 projects, partners could have a different understanding in the case of specific projects. Therefore, when planning the project all partners should define who constitutes the target groups in the given country.

School Education

The project promoters interviewed designated teachers (81 %) as the target group easiest to reach. Supporting international mobility is an excellent motivation tool for the management of the institution. It is a positive thing that teachers are not required to submit applications individually and hence are exempted from writing applications themselves. One advantage is that institutional

24/47

development becomes possible by merging individual (teachers’) and organisational (the school’s) interests. In addition, because of the separate teacher training options more colleagues can be involved in the Programme. These two elements definitely represent progress relative to LLP.

However, it is a problem that teachers are overburdened which causes personal tension within the institution and the way it is addressed depends on the head of the institution (and the organisational culture of the school concerned).

Vocational Education and Training

VET mobility is one of the most popular project types in Hungary. It makes VET institutions more attractive both to the students and the parents if they have international mobility on offer upon admission.

Developing foreign language competences is a serious challenge for students. Attracting them requires considerable effort from schools. Even so, the ratio of IVET students participating in the Programme is relatively high (69 %). Since in VET professionals who are non-school employees (e.g. consultant, career guidance counsellor) have ever fewer chances of mobility, it would be reasonable to involve them in mobility projects. Based on the questionnaires returned, the involvement rate of VET trainers and VET professionals is only 70 % and 61 %, respectively.

Adult Learning

In the Adult Learning sector attracting the target group is rather problematic, according to the respondents the success rate of reaching the target group in their own projects was 52 %.

This is probably because this sector has multiple players which makes it hard to spot the target group.

Higher Education

In HE the Programme covers all stakeholders, and as opposed to the predecessor programmes, graduates and non-academic staff (e.g. Registrar’s Office staff, librarians) can also participate in mobility projects.

However, based on the questionnaire the involvement rate was only 46 % for the latter which shows that in the case of this target group further promotional actions are needed.

The attraction rate of teachers is 69 %. The fact that there are fewer university students constitutes a challenge. In the wake of the economic crisis parents’ funds are now more limited and there have been negative changes in student attitude as well.

The special structure of the Hungarian credit system (many subjects yielding a small number of credits), the difficulties of credit recognition, teachers’ attitude in respect of the recognition of skills acquired abroad, a growing number of students wishing to study abroad for a degree (the most mobile segment), and, in many cases, inadequate support from the top management of the institution all contribute to the decline in students’ willingness to enrol in mobility programmes. Launch of the Campus Mundi Programme may be another reason why it is more difficult to recruit students for the Programme.

Youth

The biggest problem in this sector is reaching and attracting the original target group, i.e. disadvantaged young people. Their involvement is greatly hindered by a lack of self-confidence and “defeatism”.

Disadvantaged youth should be reached in a special and targeted manner, yet applicants do not have enough time and energy to invest in this. At the same time, there should be an organisation behind this target group which is able to manage the projects as without such an organisation young people cannot start out. EVS is very competitive, disadvantaged youth have little chance of participating in this programme without help.

The questionnaires reveal that the rate of young people attracted under the Youth programme is

25/47

65 %, whereas for the target group of youth specialists/youth workers this figure is 62 %.

Under KA3 applicants contact those concerned directly: they visit them the questionnaires and hold focus group discussions on the spot. Student councils constitute another area of improvement.

Sport

In the opinion of the NAU, the Programme is little known in the Hungarian sports community. In order to raise awareness of the Programme among sport organisations, each year the MHC and Tempus Public Foundation organise an Erasmus+ Sport information day together. In 2017 this event was staged for the fourth time and was attended by numerous organisations engaged in sport.

The NAU thinks that with respect to Erasmus+ Sport, the introduction of small cooperation partnerships in 2016 represented a breakthrough as by relieving the administrative burden and offering lower grant amounts, smaller leisure sport federations had a better chance to succeed in submitting an application. In the first two years of the programme only the Hungarian School Sport Federation was granted funds under Erasmus+.

After a call had been issued for the first time in 2016 for small cooperation partnerships, other Hungarian sport organisations also received grants under Erasmus+ Sport.

3.4 Internal and external coherence and complementarity

18. To what extent are the various actions that have been brought together in Erasmus+ coherent? Can you identify any existing or potential synergies between actions within Erasmus+? Can you identify any tensions, inconsistencies or overlaps between actions within Erasmus+?

The integration of the former programmes basically enhanced the more efficient and successful realisation of the Programme’s objectives. However there are certain identifiable elements of integration that have not been fully exploited and can be further emphasised within the objectives of the Programme’s next phase.

The cross-sectoral approach which used to be stronger in the beginning is one of the less exploited potentials, not so much used by the applicants. Neither the NA nor the experts are prepared for its management (e.g. which sector should provide the funding, is the sector emphasised enough where the funding is submitted). There is transition among the applicants of the fields Youth, School Education, and Adult Education which could be made more beneficial by the strengthening of intersectorality.

There is a clear tension in HE caused by the complexity of KA107 in comparison with KA103.

The introduction of the concept of Charter in other fields would also strengthen integration and the international activities of the institutions (thus may also influence the international character of their own strategies), furthermore, for the experienced and high performing applicants it would make international activities more sustainable and easier to plan.

The inclusion of (socially) disadvantaged people remains a challenge. Currently it is made possible by the Programme but not in a coherent way. There is a separate budget in HE while in VET funding can be requested as exceptional costs, and reaching young people in Youth encounters serious problems.

The obvious losers of the new Programme phase are the organisations who previously established partnership type, small scale cooperations (in ‘mobility for learning purposes’).

These were almost entirely left out from KA2, and in general in this field the number of supported projects has dropped significantly and this phenomenon is a rather demotivating element in the system (see Question 6).

The possibilities for integration and/or stronger coherence are worth examining separately in the set of School Education and VET (e.g. providing language courses for teachers in vocational education).

26/47

The disadvantage of integrating the previous programmes into one is the disappearance of national characteristics; country specific issues are not focused on within the Programme. Previously it was not easy to record these data, however it is worth considering to raise the profile of national contexts to a certain extent.

27/47

19. To what extent does Erasmus+ complement other national and international programmes available in your country? Can you identify any tensions, inconsistencies or overlaps with other programmes?

Considering its professional complexity and scope, there is no alternative to the Programme in Hungary in gaining international experience. Such a wide focus group cannot be reached or involved at the European level by any other programme. There are only a few programmes that encourage knowledge sharing on an international scale.

The Vocational School Development Programme implemented in VET between 2003 and 2009 can be cited as an example: this programme supported study visits abroad and exchange of experience, but was never an alternative to the Leonardo da Vinci Programme. However, at the moment there is no such programme in VET.

Programmes promoting internationalisation have been running primarily in HE.

The National Excellence Programme — Campus Hungary, a convergence programme aimed at the development of a personal grants system for international student mobility in R&D projects and training programmes was carried out under a call for proposals

12 and was awarded EU funds

of some EUR 13 million in 2012—2015.

In the framework of the Campus Hungary Programme, more than ten thousand persons received a grant to study abroad. Moreover, the programme supported the internationalisation processes of Hungarian HE to raise the number of foreign students studying in Hungary by providing opportunities for joint appearances for HE institutions at roadshows, education fairs, and conferences.

The scholarship programme Stipendium Hungaricum was launched in 2013 with a similar objective; it was established in response to the East and South Initiatives of Hungarian foreign policy. Besides facilitating the internationalisation of Hungarian HE and improving its quality, this programme encourages Hungarian academia to forge additional international relationships and boosts the cultural diversity of HE institutions conveying the image of a competitive Hungarian HE abroad. This programme has a budget of nearly EUR 55 million for 2017.

In HE, the Programme is complemented by the Campus Mundi programme which awards approximately EUR 30 million to promote student mobility and the development of international competitiveness of Hungarian HE in the period 2016—2021. Campus Mundi builds on the Erasmus+ programme, and partly uses the institutional relationships of and the mobility headcounts defined for institutional cooperation under the latter.

3.5 European added value and sustainability

20. To what extent Erasmus+ and its predecessor programmes produce effects that are additional to the effects that would have resulted from similar actions initiated only at regional or national levels in your country? What possibilities do you see to adjust Erasmus+ or its successor programme in order to increase its European value added?

Added value of the Programme

The Programme offers a unique learning opportunity for all participants to gain experience abroad, to test good practices, and to get acquainted with other cultures and so promotes the spread of intercultural knowledge among individuals (expertise) and in society in the field of education and training as well as in the Youth sector.

We are not aware of any project opportunities with a scope similar to that of Erasmus+ which cover the entire spectrum of lifelong learning and support cooperation on a European scale, and this holds particularly true for the sectors listed (with the exception of HE).

93 % of the respondents believed that their own project helped raise intercultural awareness, 92 % of them claimed that it encouraged the internationalisation of institutions, and 94 % were of the opinion that it fostered the exchange of good practices.

12

New Széchenyi Plan TÁMOP-4.2.4.B/2-11/1-2012-0001

28/47

According to the respondents, the projects implemented had a high rate (83 %) of achieving one of the key objectives of the Programme, that is, the promotion of European values (respect for human dignity and human rights, freedom, democracy, equality, the rule of law, pluralism, non-discrimination, tolerance, justice, solidarity, equality between men and women).

As for mobility projects, having participated in the Programme more than four-fifth of the participants (pupils, students, teachers, and professionals) “felt more European” and took ownership of the European values.

An important benefit of the Programme is that, as the focus group discussions testify, these values are not only reflected in the attitude of Programme participants, but also have an impact on their parents, friends, colleagues, and immediate social environment (local communities).

Sustainability

The sustainability of project results is a crucial point of the Programme: above all, the calls focus on project implementation, and even though dissemination is an obligation of the institutions, such activities usually subside after the projects are over.

Ensuring sustainability is the responsibility of applicants; however, neither the institutions nor the NAs have any related central expectations, guidance or tools at their disposal. Unfortunately, recording, measuring, and monitoring project results also pose difficulties, and as these activities could provide valuable input for planning sustainability, this is indeed a problem.

Although TPF regularly organises workshops to present the project results (see Answer 9) to promote horizontal learning, these events cannot contribute directly to sustainability.

It is definitely reasonable to develop a central sustainability guide for applicants and for the NAs under the Programme.

21. To what extent Erasmus+ will be able to absorb in an effective way the sharp increase in the budget that is foreseen in the coming years up to 2020 in your country? Could the programme use even higher budgets in an effective way? Do you see challenges to effectively use more money for particular actions or fields of the programme?

The continuous need for mobilities justifies the increase of the budget of KA1. The only exception may be HE due to the dominant share of the domain (about 40 %).On the other hand, reaching the 20 % ratio of HE students with experience gained in foreign education

13 justifies the further

encouragement of mobilities.

If there were a chance for smaller and simpler calls in KA2A, that would also necessitate the increase of the budget.

In Youth it is especially important to readjust the decreased budget in KA347.

Compared to the former period the scale of the European Union funding provided for the implementation of the Programme is sufficient and predictable. In comparison with the minimum budget increase in the period 2014-2016 a sharp increase is foreseen for 2017-2020. A question has arisen whether such increase in budget volume can be maintained from 2021 in order to reach a more balanced growth dynamics for the next programme.

Because of the increase of funding marked out for calls the gap is going to grow between the frameworks of calls and the frameworks of operation. It is important to note that the increased amount allocated for application resources does not necessarily mean that the management costs will rise on the same scale. The growing number of calls does not lead to an equivalent increase in management costs either, although in some areas (such as preparatory seminars, expert assessments, administration regarding the finances, etc.) an increase of costs can definitely be projected.

There is no need for changing the structure of the Programme thus the involvement of further participants can be ensured by fine-tuning its budget and by increasing the resources.

13

The Bologna process 2020 – Communiqué issued by the Conference of European Ministers responsible for Higher Education held in Leuven and Louvain-la-Neuve on 28 and 29 April 2009

29/47

4 Conclusions and suggestions for improvements to Erasmus+ and for a future programme

The findings about the implementation of the Programme in Hungary are detailed in the answers given to the questions in Chapter 3 above. All in all, we can say that the Programme is widely known and popular in the country. The integration of predecessor programmes generated basically positive changes and resulted in a more efficient realisation of the objectives. The structure of the Programme is clear and there is no need for any radical changes; however, fine-tuning the processes, simplifying administration (IT tools included) underlying project implementation, and overviewing some content elements are indispensable during the development of future programmes. The following proposals provide feedback to and help such development work.

Policy

A key factor of Programme implementation is for NAs to have adequate information about the Programme itself and its development.

The Commission is advised to better involve the NAs in the preparation of policy-related decision-making (drawing up the successor programme included) as these Agencies have a wealth of practical experience in project implementation and are deeply familiar with the problems and expectations of applicants, and such experience may well prove valuable in improving operation and to future development efforts.

For the Programme to be successful and its results to be utilised, Agencies should adopt an applicant-friendly attitude and a supportive mindset which are greatly appreciated by the applicants. Such philosophy of operation and organisational culture definitely contribute to the reputation and success of the Programme. Therefore, promoting horizontal learning between the NAs and a central exchange of good practices are highly recommended.

Adding field-specific elements to the Programme—mainly in the Youth and VET—is also desirable as these could ensure that the European objectives reflect the specific national contexts more appropriately.

Objectives, target groups

The Programme covers the target groups that have a stake in the realisation of the objectives completely.

However, in order to reach disadvantaged target groups directly, opportunities should be provided consistently by granting extra funding for projects aimed at the target groups (as is currently the case in HE).

Structure

The next Programme phase should definitely build on the strengths of Erasmus+ and only those aspects should be altered and fine-tuned that actually need to be changed, i.e. the principle of “Evolution, not revolution” should apply. Based on our experience here, in Hungary, four aspects have been identified regarding Programme structure which are worth reconsideration.

The institutional approach could be strengthened. TPF believes there is room for more support on the Commission’s part, the Agency thinks certain project elements are “neglected”. These include impact and its measurement—it is necessary to record the expected (long-term) results and to make these measurable at the start of a new programme. This would require stronger expectations in terms of sustainability, and the promotion of sustainability within the project cycle.

This aspect also implies considering the possibility of launching more charter projects. It is worth examining what other sectors—besides HE and VET—are ready for charter projects. On the institution’s part this assumes the existence of appropriate strategic thinking, while the NA has to cater for programme management. The possibility of integration and/or that of stronger cohesion should be addressed separately at the intersection of School Education and VET—e.g. offering courses and language courses to VET professionals.

Under KA1, mobility has come under criticism only in HE: project-specific reporting under KA103 and the flexibility of KA107 should both be reviewed (a greater degree of flexibility is required).

30/47

With respect to project activities, as long as certain elements of the strategic partnerships under KA2 are unclear, these will fail to contribute materially to the success of the Programme. Both the Applicants and TPF struggled a lot with interpretation, the questions, and the guide. More clues and harmonised European guidelines are needed to clarify the concept of intellectual output as the countries’ different interpretations and their shift of focus lead to inconsistencies in the system. Applicants miss uniform templates and models (e.g., timesheet) which could facilitate their work and serve as guidance. For this project, impact should be measured in terms of the number of institutions reached (funded): the obvious “losers” of the new Programme phase are the organisations who previously established partnership-type, small-scale cooperation (aimed at learning or the exchange of experience). These were almost squeezed out of KA2: here, in general the number of supported projects dropped significantly and this phenomenon is a rather demotivating element in the system. This evaluation was not meant to go into such details, therefore, these should be subject to a separate impact assessment, especially in the sectors where this project type has little or no record at all. Even when there is no leeway to do this horizontally, it is worth “digging deep” in certain sectors in the following order of priority:

Youth;

Adult Learning (the concept of intellectual output is not clearly understood here);

School Education (new players entered this sector, schools are not the only ones to go into partnership with);

HE;

VET.

However, the success of integration in the Youth sector should be examined separately. The documents show that NAs were dissatisfied with implementation, it has serious concerns about the visibility of the sector and the fulfilment of the objectives. Beneficiary applicants seemed to be less dissatisfied; nonetheless, a certain segment of the target group is more difficult to reach and attract, and this should be the subject of further analysis.

What has been indeed confirmed is that the former “youth initiatives” are absent from the Programme, and it would be useful to reintroduce these in the successor programme.

The Programme holds great potential in allowing for cross-sectorality, yet this potential has remained untapped relative to the first year. By overcoming administrative burdens and offering assistance (e.g. ‘supportive type’ application forms, designation of the grant-providing field, sufficient emphasis on the field for which the application is submitted…etc.) the cross-sectoral character could gain new momentum.

For example, even now there are certain overlaps between applicants in the Youth, School Education, and Adult Learning sectors which could be better exploited through cross-sectoral cooperation.

Budget

It should be stipulated that at the start of the next programme the initial grant amount allotted for a given sector/activity should by no means be less than in the last year of the previous programme.

In the light of applications that could not be covered by the budget and in order for the objectives to be met, it might be helpful to take a look into the distribution of funding across the sectors and/or provide extra funding for the sectors where the available budget fails to finance the applications submitted.

The figures reveal that it is in the Youth and the Adult Learning sectors as well as in the case of KA2 in general and VET mobility under KA1 where the number of non-financed applications is the highest. This means that these sectors/project types would be more efficient if awarded more funding.

It is necessary to review the grant amounts given to certain sectors (see student mobility in HE and the eligible daily lump sum). In addition, the options of redistribution in certain sectors should be more flexible (see: redistribution within KA1 in the Youth sector).

The process of central clearing under KA2 (strategic partnerships for schools only) should be made more transparent.

31/47

Under KA2 countries can share their available budgets in the Youth, VET, School Education, and Adult Learning sectors; the effects should be monitored at the European level though, as countries can adopt different practices which, in turn, may affect the number of eligible projects in other countries.

Flat rate financing is definitely beneficial to both applicants and the Executive Agency. Nevertheless, the target groups indicated a need for examining and revising the lump sums for the specific sectors and activities—especially with regard to KA2—because several sources claimed that:

Traveling to distant countries is problematic—the amount specified for the given km range is not sufficient.

Teachers from Western Europe are not interested in strategic partnerships because of quotas that are narrow for them. Hungarian staff costs are too low. In the Youth sector EUR 33/day/person does not necessarily meet the costs.

For shorter term mobility, the course fees are not fully covered by the amount granted, so reallocation is necessary. In “more expensive” Member States even the course fees are higher.

The system “pays off” only once a certain number of EVS volunteers have been received.

The successor programme ought to have dedicated support for language learning to ensure an appropriate standard of language skills as OLS is unsuitable for some target groups (e.g. IVET students, disadvantaged youth).

Overall, integration and simplification have many benefits, yet when it comes to the budget, generalisation fails to bring satisfaction. Here, the differences between the sectors and/or the countries make the Programme fall apart. Therefore, it is advisable to design a system which is more tailored to the specific sectors/target groups/countries instead of complete generalisation.

Administration and IT

If the new Programme phase were to run in an integrated IT environment, it is important that it is massively built on the previous ones rather than being a new system set up from scratch. It should be a natural, more advanced continuation of the current system which better reflects the needs of applicants and the responsibilities of the NAs. IT should follow the technical content and expectations, and not vice versa.

However, the current IT system is very complex, applicants “get lost”, and TPF also has a lot of trouble using the various IT systems. Although the IT tools offer many services, project promoters are required to use many more different (!) systems during a project cycle than in the former Programme phase. Even now, in the fourth year of the Programme, there are functions that hardly operate or do not operate at all (e.g. validation of KA2 partners, modifications after financial audits in the Mobility Tool, queries about project budget by activity type in EPLink). On the whole, there are too many systems operated in connection with a project cycle both at the level of applicants and of the NAs. It would be sensible to harmonise and integrate these. Besides, more resources should be dedicated to operation on the part of the European Commission and the NAs alike, especially at the start. So in the next Programme phase the NAs should definitely be involved already in the planning phase.

Regarding KA3 in the Youth sector, the use and/or the form of the Mobility Tool should be reconsidered as in its current form it places a disproportionately heavy burden on applicants.

With respect to administration, there is a need to create coherence between the procedures of the NAs. In the context of KA2 it has been brought to light several times that there are marked differences between the administrative requirements by the NAs.

As regards data collection, it would suffice to gather a smaller but well-defined range of data using an administrative interface. These could serve as an indicator which should be made accessible and available for query by granting different authorisations to applicants and to the NAs.

32/47

5 Annexes

Annex 1 Documents and databases used for the survey

Yearly report E&T – 2014

EC evaluation, conclusion letter for E&T – 2014

Yearly report Y – 2014

EC evaluation, conclusion letter for Y – 2014

Yearly report E&T – 2015

EC evaluation, conclusion letter for E&T – 2015

Yearly report Y – 2015

EC evaluation, conclusion letter for Y – 2015

Yearly report E&T&Y– 2016

NAU October report E&T – 2015

NAU October report Y – 2015

NAU October report E&T – 2016

NAU October report Y – 2016

NAs for E+ programme, in the field of Youth, Common input for the midterm evaluation 2017

Summary of the results of the remarks during the world cafe at the Business meeting in Amsterdam on the questions of the Midterm evaluation

Participant reports of learners and staff (2014-2016)

33/47

Annex 2 Results of the online questionnaire

The tables and graphs bellow contain the distribution of answers given to the online questionnaire. The analysis of these results and answers given to the open questions are detailed in the Report. Annex 2 does not contain the answers given to the open questions and the Report does not deal with the background variables of respondents.

34/47

35/47

36/47

37/47

General objectives

Released Relevance Released Relevance Released Relevance Released Relevance Released Relevance

Reducing the rates of early school

leaving below 10%

    3,99 4,08 3,85 3,75        

At least 95% of children (from 4 to

compulsory school age) should

participate in early childhood

education

        1,98 2,07        

Fewer than 15% of 15-year-olds

should be under-skilled in reading,

mathematics and science

    2,46 2,29 4 4,03        

The rate of early leavers from

education and training aged 18-24

should be below 10%

3,69 3,25 4,19 4,62            

At least 40% of 30-34–year-olds 

completing third level education

3,18 3,28                

At least 15% of adults should

participate in lifelong learning

4,84 5,24 4,44 4,16 4,57 4,53 5,69 5,85    

At least 20% of higher education

graduates and 6% of 18-34 year-olds

with an initial vocational qualification

should have spent some time

studying or training abroad

5,26 5,52 4,62 4,29            

The sustainable development of

partner countries in the field of

higher education;

4,9 5                

The improvement of teachers’ and 

trainers’ competences in methodology

5,24 5,29 5 5,07 5,81 5,87 4,75 4,42    

To provide more and equal

opportunities for young people in

education and the job market

4,86 5,16 5,04 5,04 4,91 4,83 4,44 4,4 4,86 5,2

Youth

Objective

HE VET School education Adult learning

38/47

Specific objectives

Released Relevance Released Relevance Released Relevance Released Relevance Released Relevance

Promoting equity, social cohesion,

and active citizenship

                5,35 5,43

The Union is founded on the values

of respect for human dignity,

freedom, democracy, equality, the

rule of law and respect for human

rights, including the rights of persons

belonging to minorities. These values

are common to the Member States in

a society in which pluralism, non-

discrimination, tolerance, justice,

solidarity and equality between

women and men prevail.

4,65 4,73 4,69 4,51 5,2 5,24 4,18 4,67 5,29 5,45

To improve the level of key

competences and skills, with

particular regard to their relevance

for the labour market and their

contribution to a cohesive society, in

particular through increased

opportunities for learning mobility and

through strengthened cooperation

between the world of education and

training and the world of work;

5,27 5,42 5,46 5,52 4,93 5,01 5,17 5,23 5,24 5,27

To foster quality improvements at

the level of education and training

institutions

4,71 5,1 4,8 4,9 5,16 5,4 4,58 5,23 3,71 3,68

To foster innovation excellence at

the level of education and training

institutions

4,57 4,93 4,69 4,72 5,03 5,35 4,42 4,5 3,67 3,84

To foster internationalisation at the

level of education and training

institutions

5,27 5,71 5,5 5,59 5,64 5,67 5,31 5 4,74 4,76

to promote the dissemination of

good practices

5,21 5,56 5,32 5,54 5,75 5,83 5,6 5,69 5,22 5,17

To promote the emergence and raise

awareness of a European lifelong

learning area designed to complement

policy reforms at national level and to

support the modernisation of

education and training systems

4,4 4,77 4,2 4,48 4,1 4,22 4,44 4,78 2,96 3,13

To improve the teaching and learning

of languages and to promote the

Union's broad linguistic diversity

4,93 5,26 5,17 5,38 5,66 5,7 4,22 4,33 5 4,76

To promote intercultural awareness 5,28 5,53 5,3 5,31 5,7 5,81 5,29 5,07 5,62 5,68

Objective

HE VET School education Adult learning Youth

39/47

40/47

Annex 3: Focus groups in the survey

Focus group

Action

1 Mobility within programme countries (KA103)

Mobility within programme countries and partner countries (KA107)

Strategic partnerships for HE (KA203)

2 VET mobility (KA102) VET learner and staff mobility

3 Strategic partnerships for HE (KA202)

4 School education staff mobility (KA101)

5 Strategic partnerships for School Education(KA201)

Strategic partnerships for schools only(KA219)

6 Adult education staff mobility (KA104)

Strategic Partnerships for Adult Education (KA204)

7 Youth mobility (KA105)

Structured dialogue (KA347)

8 Strategic partnerships for youth (KA205)

9 Organisations that applied for both LLP and Erasmus+

10 Organisations that applied for both KA1 and KA2

41/47

Annex 4 General and specific objectives of the Programme

General objectives and benchmarks

The Programme shall contribute to the achievement of:

(a) the objectives of the Europe 2020 strategy, including the headline education target:

Reducing the rates of early school leaving below 10%;

At least 40% of 30-34–year-olds completing third level education.

(b) the objectives of the strategic framework for European cooperation in education and training ('ET 2020'), including the corresponding benchmarks:

At least 95% of children (from 4 to compulsory school age) should participate in early childhood education

Fewer than 15% of 15-year-olds should be under-skilled in reading, mathematics and science

The rate of early leavers from education and training aged 18-24 should be below 10%

At least 40% of people aged 30-34 should have completed some form of higher education

At least 15% of adults should participate in lifelong learning

At least 20% of higher education graduates and 6% of 18-34 year-olds with an initial vocational qualification should have spent some time studying or training abroad

The share of employed graduates (aged 20-34 with at least upper secondary education attainment and having left education 1-3 years ago) should be at least 82%

(c) the sustainable development of partner countries in the field of higher education;

(d) the overall objectives of the renewed framework for European cooperation in the youth field (2010-2018):

To provide more and equal opportunities for young people in education and the job market;

To encourage young people to actively participate in society.

(e) the objective of developing the European dimension in sport, in particular grassroots sport, in line with the Union work plan for sport; and

(f) the promotion of European values in accordance with Article 2 of the Treaty on European Union:

The Union is founded on the values of respect for human dignity, freedom, democracy, equality, the rule of law and respect for human rights, including the rights of persons belonging to minorities. These values are common to the Member States in a society in which pluralism, non-discrimination, tolerance, justice, solidarity and equality between women and men prevail.

42/47

Specific objectives - Education and Training

(a) to improve the level of key competences and skills, with particular regard to their relevance for the labour market and their contribution to a cohesive society, in particular through increased opportunities for learning mobility and through strengthened cooperation between the world of education and training and the world of work;

(b) to foster quality improvements, innovation excellence and internationalisation at the level of education and training institutions, in particular through enhanced transnational cooperation between education and training providers and other stakeholders;

(c) to promote the emergence and raise awareness of a European lifelong learning area designed to complement policy reforms at national level and to support the modernisation of education and training systems, in particular through enhanced policy cooperation, better use of Union transparency and recognition tools and the dissemination of good practices;

(d) to enhance the international dimension of education and training, in particular through cooperation between Union and partner-country institutions in the field of VET and in higher education, by increasing the attractiveness of European higher education institutions and supporting the Union's external action, including its development objectives, through the promotion of mobility and cooperation between the Union and partner-country higher education institutions and targeted capacity-building in partner countries;

(e) to improve the teaching and learning of languages and to promote the Union's broad linguistic diversity and intercultural awareness;

(f) to promote excellence in teaching and research activities in European integration through the Jean Monnet activities worldwide.

42/47

Annex 5.: Details regarding the use of funds for grant support

SUMMARY FINANCIAL AND CONTRACTUAL

2014

Erasmus+ Financial Statement on the use of funds for grant support

Period covered from 01/01/2014 to

31/12/2016

NA: HU01 NA Name: Tempus Public Foundation Financial Year: 2014 Reference to NA Agreement:

EAC-2014-0045

Description

Total: E&T & Youth

Total E&T

HE HE School Education Adult Education

Total Youth

Youth

KA1 KA2

Total HE

KA1 KA2

Total VET

KA1 KA2

Total SCH

KA1 KA2

Total ADU

H1 H1 H1 H1 H1 H1 H1 H1 Total KA1

Total KA2

Total KA3

I.6 Total of Erasmus+ and National funds (I.4+I.5)

30 708 475,00

26 419 328,00

12 728 388,00

654 941,00

13 383 329,00

6 060 621,00

1 872 184,00

7 932 805,00

1 039 585,00

2 849 334,00

3 888 919,00

210 513,00

1 003 762,00

1 214 275,00

4 289 147,00

3 410 781,00

760 261,00

118 105,00

II.6 net value of signed grant agreements

30 637 010,83

26 336 551,86

12 727 689,00

611 139,00

13 338 828,00

6 060 621,00

1 862 758,86

7 923 379,86

1 039 585,00

2 820 484,00

3 860 069,00

210 513,00

1 003 762,00

1 214 275,00

4 300 458,97

3 448 856,00

733 487,97

118 115,00

II.7 Commitment Rate (%)

99,69% 99,99% 93,31% 99,67% 100,00% 99,50% 99,88% 100,00% 98,99% 99,26% 100,00% 100,00% 100,00% 100,26% 101,12% 96,48% 100,01%

IV. Determining the final grant amounts (resulting from the final reports)

IV.1 Total final grant amounts (realised)

19 373 982,46

16 368 699,19

9 844 813,91

0,00 9 844

813,91 5 429

331,82 27

639,86 5 456

971,68 935

785,00 0,00

935 785,00

130 355,57

773,03 131

128,60 3 005

283,27 2 792

893,09 173

714,18 38

676,00

IV.2 Realisation Rate (%) 63,24% 62,15% 77,35% 0,00% 73,81% 89,58% 1,48% 68,87% 90,02% 0,00% 24,24% 61,92% 0,08% 10,80% 69,88% 80,98% 23,68% 32,74%

SUMMARY FINANCIAL AND CONTRACTUAL

43/47

Erasmus+ Financial Statement on the use of funds for grant support

2015

Period covered from 01/01/2015 to

31/12/2016

NA: HU01 NA Name: Tempus Public FoundationFinancial Year: 2015 Reference to NA

Agreement: EAC-2015-0100

Description

Total: E&T & Youth

Total E&T

HE HE School Education Adult Education

Total Youth

Youth

KA1 KA2

Total HE

KA1 KA2 Total VET

KA1 KA2 Total SCH

KA1 KA2 Total ADU

H1 H4 H1 H1 H1 H1 H1 H1 H1

Total KA1

Total KA2

Total KA3

I.6 Total of Erasmus+ and National funds (I.4+I.5)

33 704 638,00

29 395 051,00

12 564 318,00

2 680 024,00

769 800,00

16 014 142,00

6 022 028,00

1 842 309,00

7 864 337,00

997 383,00

3 315 874,00

4 313 257,00

192 304,00

1 011 011,00

1 203 315,00

4 309 587,00

3 309 238,00

836 013,00

164 336,00

II.6 net value of signed grant agreements

33 704 730,34

29 373 463,84

12 572 828,24

2 677 891,76

769 800,00

16 020 520,00

6 022 019,00

1 838 009,30

7 860 028,30

981 907,00

3 296 659,54

4 278 566,54

203 338,00

1 011 011,00

1 214 349,00

4 331 266,50

3 304 417,50

862 513,00

164 336,00

II.7 Commitment Rate (%)

99,93% 100,07% 99,92% 100,00% 100,04% 100,00% 99,77% 99,95% 98,45% 99,42% 99,20% 105,74% 100,00% 100,92% 100,50% 99,85% 103,17% 100,00%

IV. Determining the final grant amounts (resulting from the final reports)

IV.1 Total final grant amounts (realised)

4 681 614,37

3 235 442,55

29 530,00

0,00 0,00 29

530,00 2 699

760,71 16

759,30 2 716

520,01 480

931,00 8

461,54 489

392,54 73

224,00 0,00

73 224,00

1 446 171,82

1 241 942,56

204 229,26

0,00

IV.2 Realisation Rate (%)

13,89% 11,01% 0,23% 0,00% 0,00% 0,18% 44,83% 0,91% 34,56% 48,98% 0,26% 11,44% 36,01% 0,00% 6,03% 33,39% 37,58% 23,68% 0,00%

SUMMARY FINANCIAL AND CONTRACTUAL

2016

Erasmus+ Financial Statement on the use of funds for grant support

44/47

Period covered from 01/01/2016 to

31/12/2016

NA: HU01 NA Name: Tempus Public FoundationFinancial Year: 2016 Reference to NA

Agreement: EAC-2016-0035

Description

Total: E&T & Youth

Total E&T

HE HE School Education Adult Education

Total Youth

Youth

KA1 KA2

Total HE

KA1 KA2 Total VET

KA1 KA2 Total SCH

KA1 KA2 Total ADU

H1 H4 H1 H1 H1 H1 H1 H1 H1

Total KA1

Total KA2

Total KA3

I.6 Total of Erasmus+ and National funds (I.4+I.5)

34 464 463,68

30 097 123,68

12 885 902,00

2 993 053,68

781 821,00

16 660 776,68

6 140 515,00

2 125 113,00

8 265 628,00

1 076 810,00

2 809 951,00

3 886 761,00

146 424,00

1 137 534,00

1 283 958,00

4 367 340,00

3 290 761,00

926 934,00

149 645,00

II.6 net value of signed grant agreements

31 854 674,22

29 030 358,23

12 684 664,58

2 739 489,67

781 821,00

16 205 975,25

6 064 381,00

1 648 559,96

7 712 940,96

1 076 810,00

2 806 544,02

3 883 354,02

146 424,00

1 081 664,00

1 228 088,00

2 824 315,99

2 200 256,99

524 906,00

99 153,00

II.7 Commitment Rate (%)

96,46% 98,44% 91,53% 100,00% 97,27% 98,76% 77,58% 93,31% 100,00% 99,88% 99,91% 100,00% 95,09% 95,65% 64,67% 66,86% 56,63% 66,26%

IV. Determining the final grant amounts (resulting from the final reports)

IV.1 Total final grant amounts (realised)

239 980,38

3 692,98

0,00 0,00 0,00 0,00 0,00 2

193,96 2 193,96 0,00

1 499,02

1 499,02 0,00 0,00 0,00 236

287,40 103

393,89 132

893,51 0,00

IV.2 Realisation Rate (%)

0,75% 0,01% 0,00% 0,00% 0,00% 0,00% 0,00% 0,13% 0,03% 0,00% 0,05% 0,04% 0,00% 0,00% 0,00% 8,37% 4,70% 25,32% 0,00%

45/47

Annex 6. Ratio of granted applications to all applications submitted

Fields years 2014, 2015, 2016

Ratio of granted applications to all applications submitted

(%)

Adult Learning KA104 32,5%

KA204 17,9%

Higher Education KA103

14 97,9%

KA107 62,3%

KA203 16,0%

Youth KA105 31,3%

KA205 11,9%

KA347 22,6%

School Education KA101 56,9%

KA201 28,4%

KA219 26,3%

Vocational Education and Training

KA102 57,8%

KA202 24,7%

Cross-sectoral KA200 18,2%

14

There is no professional assessment for this action, and HE institutions apply for headcounts. This is the reason for the extremely high number of granted projects.

46/47

Annex 7 Overall satisfaction of participants about mobilities(2014-2016)

Education and Training

Adult Education Higher Education School education

Vocational Education and

Training

staff students staff students staff staff students

Very satisfied 81% 86% 83% 61% 87% 84% 75%

Rather satisfied 18% 13% 17% 34% 13% 14% 21%

Rather unsatisfied 0% 1% 0% 1% 0% 0% 1%

Very unsatisfied 0% 0% 0% 0% 0% 0% 0%

Cannot decide 1% 0% 0% 4% 0% 1% 2%

Youth Exchange Programme Young people

Excellent 67%

Good 25%

Average 5%

Bad 2%

Very bad 1%

European Voluntary Service Volunteers

Very satisfied 88%

Rather satisfied 10%

Neither satisfied nor unsatisfied 2%

Youth mobility Youth workers

Very satisfied 72%

Rather satisfied 24%

Neither satisfied not unsatisfied 3%

Rather unsatisfied 1%

Very unsatisfied 0%