the ex-post evaluation of the pre-esf 2000 hrd grant schemefile/note6n4cgm.pdf · the ex-post...

The ex-post evaluation of the pre-ESF 2000 HRD grant scheme

Institute of Educational SciencesRomanian National Observatory

WP 2005-11-3 Institution Building Support the development of local capacity to evaluate ESF types of activities

RO/PHARE/2005/SSTA 10 Support for the evaluation of the PHARE HRD 2000 programme

Authors: Magdalena Balica Cesar Birzea Ciprian Fartuşnic Mihaela Jigău Data analyses: Cornelia Novak Lucian Voinea Andreea Marutescu Translation: Maria Ana Dumitrescu Irina Anca Ionescu Consultants: Douwe Grijpstra (Research voor Beleid) Arjen Deij (European Training Foundation) Outi Kärkkäinen (European Training Foundation)


RO/PHARE/2005/SSTA 10 Support for the evaluation of the PHARE HRD 2000 programme

Summary Executive Summary 1 1. Evaluation methodology 13

1.1. Objective of the evaluation 1.2. Evaluation model 1.3. Methods of investigation 1.4. Experiences with gathering data 1.5. Methodology of statistical analysis

2. Context 16

2.1. Analysis of the actual state in the field of employment and training in Romania 2.2. Convergence of the program objectives with the European and national policies 2.3. Former programmes 3. Objectives and priority issues 25 4. Resources 28

4.1. Funds by regions 4.2. Profile of project promoters 4.3. Partners 5. Implementation/operations 41 5.1. Program priorities addressed 5.2. Program measures addressed 5.3. Problems in initiation and implementation 6. Costs 46 6.1. Expenditure on equipment 6.2. Training costs 7. Output 52

7.1. Number of successful and unsuccessful projects 7.2. Funds absorption capacity 7.3. Creation of different types of institutional capacity building 7.4. Characteristics of beneficiaries 7.5. Output of training courses

8. Effect 72

8.1. Viability of the training centres 8.2. Completion and drop-out at the beneficiary level 8.3. Satisfaction of beneficiaries

9. Efficiency 80 10. Impact 80

10.1. Dissemination of results 10.2. Sustainability of effects on beneficiaries 10.3. Impact in promoters opinion

11. Conclusions and recommendations 83 Annexes 94

Annex 1. Methods of investigation Annex 2. Methodology of the data statistical analysis Annex 3. Report of the Seminar “Impact Evaluation of Phare HRD 2000 Scheme” Annex 4. Additional tables and charts


RO/PHARE/2005/SSTA 10 Support for the evaluation of the PHARE HRD 2000 programme 1

Executive Summary 1. Background

The implementation of ESF type of activities in Romania represents an important opportunity for the process of modernisation of the vocational education and training system and HRD and employment policy in the next years. This opportunity presents a challenge to the national and regional authorities, which need to administer the allocation of “ESF type resources” according to agreed policy requirements and new planning tools. It also presents a challenge to social partners and to all those stakeholders that are involved in initiating quality assurance and evaluation ESF type of activities.

Learning lessons from EU experience and the evaluations of the former Phare programs is crucial for the success of the future ESF type of activities. From the start up of Pre-ESF activities in Romania, ETF has been actively supporting institution building measures that should enable the Romanian stakeholders to make better use of the opportunities offered trough ESF and Pre-ESF activities.

2. Learning from Phare for ESF

The findings of former evaluation of Phare program in Romania draw attention to different specific issues.

The Phare 97 Pre-HRD scheme focused on relevant measures to promote HRM in companies. The evaluation showed that in spite of many good individual examples, the impact at system and enterprise level has been limited.

The Phare 98 HRD grant scheme evaluation showed that CVT has been stimulated by the call for applications, and there has also been a positive impact on employment in many cases. However, the evaluation has not been able to demonstrate how structural these effects have been. The 98 evaluation did show that local partnerships should be reinforced to enhance the structural impact of the ESF measures.

The grant schemes in Romania were reviewed with an overview of the 1998-2000-2002-2003-2004-2005-2006 grant schemes (total value approx.150 MEURO).

The 1998 ex-post evaluation focused on the selection process, project outputs and recommendations for future schemes. It managed to provide a clear overview of how the scheme was implemented and provided valuable feedback for the implementation of the 2002 scheme, which is visible of what we know now of the 2002 scheme, but was limited due to the limitations of the PRAG (Practical Guide for Phare, obliging standardised procedures for selection and monitoring not adapted to HRD activities). The evaluation followed a constructivist approach; the main weak points were lack of triangulation and objectivity, focus on outputs with no real assessment of impact.

The RICOP evaluation of Active Employment measures used a quasi-experimental model, based on a comparison of counselling and training measures from RICOP with a randomly chosen control group from the NAE register. Weak points in the evaluation were the presentation of the findings, and the fact that the actual positive impact was within the error margins (given the size of the sample).

On request of the EC Delegation in Romania, ETF organised the ex-post evaluation of the Pre-HRD Phare 97 Progress scheme, promoting HRM in enterprises and the ex-post evaluation of the Pre ESF Phare 98 HRD grant scheme. In 2004, ETF has also been requested to organise the ex-post evaluation of the Pre-ESF 2000 HRD grant scheme. The Romanian National Observatory – Institute of Educational Sciences carried out the evaluation of Phare 2000 HRD activities. The evaluation has been based on a training session in the evaluation of ESF type of activities. A five-day seminar, starting with the general principles of



ESF ex-post evaluation in EU Member States took place in December 2004. In order to ensure the participatory approach, staff from the Ministry of European Integration, the Ministry of Labour and the National Agency for Employment were also attending the training. The general conclusion of the seminar was that Phare grants schemes should be used better in order to prepare and learn for ESF.

3. Objectives of the evaluation

The present report focuses on the PHARE 2000 ECONOMIC AND SOCIAL COHESION PROGRAMME – Human Resources Development (HRD) in the context of industrial restructuring. The report presents the outcomes of the ex-post programme evaluation. The main objectives have been to evaluate the impact of Phare projects in the framework of the HRD component, while considering each of the three priorities of the component and to make recommendations on the implementation of future HRD structural projects.

The Phare 2000 HRD evaluation focused more on the leverages of structural change, understanding better the role that service providers, beneficiaries, target groups and other groups play in the success of ESF measures. The evaluation provides focused feedback for all stakeholders and in particular the MLSSF, the NAE and the EC DEL.

4. Phare 2000 ECONOMIC AND SOCIAL COHESION PROGRAMME – HRD in the context of industrial restructuring

The main goal of the programme was to support the Romanian Government in implementing an integrated regional development policy in line with the economic and social cohesion strategy adopted within the context of the pre-accession process, through regional development investment projects in priority sectors (Human Resource Development, SMEs and local and regional infrastructure) identified in Romania’s National Development Plan and in Romanian Regional Plans. Support to these sectors aims at developing, through an integrated approach, the indigenous economic and social potential of the development regions.

Two types of projects have been financed from the Grant Scheme: � Large Projects: projects that have a regional coverage, of at least two counties;

complex projects envisaging integration of several activities; projects encouraging partnerships between education and training providers, enterprises, social partners. Minimum grant amount of €100,000 and a maximum amount of €300,000

� Small Projects: projects that have an impact at municipality or commune level and which address at least one of the eligible activities. A minimum grant amount of €10,000 and a maximum amount of €50,000

The three priority themes of the project are: Priority 1: Qualification and re-qualification of the work force in order to make it more

responsive to the evolving needs of the labour market. Priority 2: Enhancement of active employment measures as systemic tools to foster

employment. Priority 3: Promotion of social inclusion for disadvantaged groups

5. Methodology of evaluation

The evaluation of the HRD Component of the PHARE 2000 ESC Programme has taken place based on a complex methodology, with the use of diverse investigation methods. Quantitative and qualitative methods of investigation have been mixed. The survey has addressed to the project promoters and the direct beneficiaries of training activities.



The survey of the project promoters was exhaustive, as each of the promoters received a questionnaire. Out of 371 promoters, 171 answered the questionnaire. Investigated organizations promoted 185 out of 393 projects developed in the HRD component, representing 47%.

Individual interviews with 30 promoters have been applied, selected on the following criteria: development region, type of organization, project size. They were selected from the counties of Alba, Arad, Călăraşi, Cluj, Constanţa, Dolj, Neamţ, and the City of Bucharest, as these represented all the development regions.

The selection of the sample of persons representing the target groups for the HRD projects used the lists with the participants in the training programmes developed in the framework of the projects.

The selection criterion was the category of beneficiaries: employed / unemployed / disadvantaged categories. The resulting sample counted 1,799 subjects and consisted of 3 sub-samples, of a relatively equal size. The 1,799 individuals (55% women and 45% men) were mailed a questionnaire. 822 participants in training programmes answered the questionnaire. Thus, 822 participants in training programmes (64,5% women and 34,5% men) were subject to evaluation out of which 551 are employed and entrepreneurs (67%), 125 are unemployed (15,2%) and 146 are disadvantaged persons (17,8%). The number of respondents represents 45,7% of the total selected sample.

The evaluation model used in the survey is based on the EU evaluation framework for structural funds. The evaluation model was the framework for the report structure, based on key evaluation issues presented in the following figure.

Linkages between the key evaluation issues

IMPLEMENTATION

RELEVANCE EFFICACY / EFFICIENCY Source: European Commission, DG REGIO

6. Main findings

The key statistical data resulted by the evaluation will be presented below, according to the evaluation model based on the following structure: resources – operations – costs – output- effect – impact.

Context IMPACT

Results Output ResourceObjective

Operations



6.1. Resources Funds by regions • Given that the allocation of funds took into consideration two categories of regions

(Priority Regions Category I and Priority Regions Category II) and the different funding resources (the European Community and the national budget, respectively) they benefited, the volume of allocated funds differed from one region to another. Thus, the highest funding level has been recorded in the regions Northwest - 20.7% and Southeast – 20.3% of the total allocated funds while the lowest level has been recorded in the regions Southwest Oltenia – 5.0% and Bucharest-Ilfov – 5.4%

• At the project level, it can be observed also:

- A lower funding level for companies, as compared to the other types of organizations, at each development region level, an understandable situation considering the low number of companies among the total of promoters in the sample investigated;

- The higher average budget per company project, at each region level, as compared to other types of promoting organization (per total regions, the average budget – from Phare sources / national budget – of a company amounted to almost €45,600; in the case of the other organizations, it was €42,000).

Profile of project promoters • The participation of different types of organisation to the HRD program is well balanced.

The ratio of organizations specialized in human resources development— which include NGOs, companies as well as research institutes, training centres, education an training institutions and Chamber of Commerce —amounts to almost a third of the total number of promoters surveyed.

• Companies (enterprises, consultancy firms, SMEs, etc.) make up the major category of project promoters of over one third of the total projects. The majority of the surveyed companies are small, with less than 50 employees and a rather long period of operation (about 10 years).

• Non-governmental organizations (associations, foundations, trade unions, and employers’ organizations), with a share around 25% are also well represented, followed by education and training institutions (universities, high schools, TVET schools -16%), local authorities (12%), and other public institutions (county employment agencies, county school inspectorates etc. - 11%).

Partners • One in four promoters indicated the absence of any project partner in the case of their

project. For projects with at least one partner, the mean partner number per project was 2.2 (for small projects) and 3.4 (for large projects).

• Local public administration holds a share close to the sum total of the rest types of partner organisations while social partners were nominated by less than 10% of promoters surveyed;



6.2. Implementation/operations Program priorities and measures addressed • The distribution of projects by objectives highlights the fact that the majority of project

promoters have preferred training activities, these activities accounting for around two thirds of the total measures.

• Regarding the priorities addressed, projects exclusively addressing to active employment measures (objective 2) and social inclusion (objective 3) amount to less than 10% of the total projects. In almost 60% of the cases, the promoters surveyed mentioned to have focussed on only one priority.

• In the case of surveyed projects, a total of 31732 persons have benefited from project’s activities, the employees accounting for around half of the total while unemployed and disadvantaged categories for around 20%. In the case of training activities, the differences between the categories of beneficiaries are even higher, the employees accounting for almost two thirds of the total.

• More than 80% of the companies surveyed organised training activities; the number of trainees in the case of projects promoted by companies represent almost one third of the total number of trained people; however only 5% of these trainees were disadvantaged persons.

Problems in initiation and implementation Main difficulties stated by project promoters related to project initiation and implementation are the following: the delays in securing financing, difficulties in continuing the project after Phare financing ceased and difficulties related to the recruitment of target groups. The majority of promoters surveyed declared that they had little difficulties in following the existing procedures in accessing different sources of information and in communication with RDA or MEI.

Other issues mentioned by project promoters during field interviews were related to: difficulties in cooperation with project partners (i.e. formal involvement, underachievement of objectives etc.); difficulties in relation with project beneficiaries (i.e. low motivation of disadvantaged persons to attend training activities); financial (i.e. losses due to exchange rates fluctuations, slow procedures related to obtaining the official approval of budget revision and other decisions of the Treasury, Ministry of European Integration and Regional Development Agencies); changes in legislation (i.e. accreditation of training providers, fiscal code etc.), acquisitions (i.e. difficulties in purchasing equipments with EU origin certificate). Another important difficulty to which the project promoters referred to was securing their own contribution, issue mentioned especially by public institutions representatives.

6.3. Costs Expenditure on equipment • The rate of equipment acquisition expenditures from project total budget represented

about 28%.

• The highest rate of the funds allocated for equipment acquisition occurred in the framework of small-scale projects (34.5%, as against 25% in large-scale project) and of those developed by companies (30%, as against 26.7% - other organizations). In point of regional development, the highest rate occurs in Northwest (32%), Southeast, and Northeast (around 30%). Possibly, owing to a better endowment of the Bucharest-Ilfov and Centre regions, their expenditures on equipment purchasing were the lowest: 21% and 20%, respectively, of the project budgets.



• There is a hypothesis concerning the intention of companies to develop their production capacity by production equipment acquisition. The hypothesis can be confirmed by the data on:

- share of total budget allocated to equipments: 30%, as against about 26% - other organizations;

- use of equipment purchased for other purposes than training (for production / specific activity), as well – about 39%, as against 24%;

- the degree at which the equipment purchased is used for training activities: 55% of the companies use the equipment, at the highest degree, for training purposes; in the case of the other organizations, the corresponding percentage is 58%;

- the share of production equipments in the total budget allocated to all types of equipments purchased: companies – 20%, other organizations – 4%.

Training Costs • The average cost per participant is significantly higher compared to the courses

organised by companies and investigated through a NIS survey at national level1: 210 Euro as against 75 Euro. However the important differences for the average cost per participant is entailed by the long duration of courses in the case of HRD Phare 2000 program: 73 hours/course (INS), as compared to 190 hours/course (HRD component).

• The cost calculated by training hour is similar in both cases: € 1,0 (INS) and € 1,2 (HRD Program).

6.4. Output Number of Successful and Unsuccessful Projects • Only 5.5% of the total number of promoters has failed to finalize their projects, the

companies being the main project promoter category with cancelled contracts. Fund Absorption Capacity • In general, the projects in the HRD component demonstrated an average capacity of fund

absorption (around 85%). A slightly higher absorption capacity can be observed in the case of small projects – around 87%, compared with large projects - 84%.

• In point of regions of development (taking into account all projects finalised within HRD program), projects developed in North-West (89%) and Bucharest-Ilfov (87.5%) have the highest indicator value (higher than the average); in point of types of organization (in the sample evaluated), the highest indicator value is that of projects developed by companies (86.4, against 85% in the case of other organizations). A more careful monitoring, and assistance granted all along project development could increase project absorption capacity.

Creation of different types of institutional capacity building • More than half of the number of projects (52.4%) implemented two, three, or four of the

measures targeted at institutional capacity building, namely: creation of a new training centre, or development of training capacity (new classrooms, new training equipment); enhancing the training offer (new training programmes, new target groups); training the trainers.

• The specifics and the number of the measures having been implemented differ in function of the type of promoter. Thus, companies have been more interested than the other organizations in creating their own training centres (34% as against 29%); the other organizations saw as priorities the measures concerning the enhancement of their

1 Characteristics of CVT in Romania, NIS, Bucharest, 2001.



training offer (65% as against the companies’ 45%) or the training of trainers (23%, and 15%, respectively).

• An unbalanced distribution by region of development could be noticed among the projects that developed institutional capacity, particularly, creation of new training centres. The situation could be generated by such factors as: a different level of socio-economic development of the regions; the different funding available in one region or another; the different fund absorption capacity; characteristics of the trainer market (number of trainers, diversity of offers, quality of trainers, etc.); the training needs at regional level of the various target population groups, etc.

• Most of the companies having created their training centres use a completely private capital (81.3%), are small-sized enterprises (62,5%), and have been operating for about 10 years (62,5%).

Characteristics of Beneficiaries • Employees are the most important group of beneficiaries, accounting half of the total

target-categories. The distribution of the direct beneficiaries, by type of assistance received2 indicates that the employees are 50% of the total target-categories. However, the rate of the entrepreneurs (actual or potential), the unemployed, and the socially disadvantaged could be judged as high, compared to the rate of projects targeted on measures specific to priority 2 and 3. The explanation relates to the accidental inclusion of these categories in the activities of the project (not being strategically targeted by project promoters) and, to the way the project priorities and measures have been formulated.

• Most important share of the trainees belongs to the employees (64,(%). 17% of the beneficiaries of the analysed courses are unemployed persons and 13.1% are disadvantaged categories. The entrepreneurs account for 5.4% of the total trainee numbers.

• The survey data obtained from the respondent promoters show a total number of 21,902 individuals who have attended the training courses. Most of the beneficiaries attended at least one course. Keeping in mind that the same person attended several courses within the same project, the total figure is 23,741 graduates of different courses.

• Beneficiaries with a high level of education benefit more of the training courses. Depending on the participants’ educational level over 72% of the courses were addressed to the high school level or higher education level persons, while only ¼ of the courses were meant for lower educated people.

Developed Training Courses • The training measures have occupied a very important place within the PHARE HRD

2000 projects. According to the present survey, out of a total amount of 185 investigated projects, almost 89% have developed training programmes. The training courses developed in the surveyed projects cover all the development regions, in different proportions. The promoters from Bucharest region developed most of the courses included in the survey. This is influenced also by the answering rate in Bucharest region, higher than others. West and South regions are also well represented in the lot of projects that developed training courses. The rest of the regions register low proportions between 2, 4% and 7,3% from the total analysed training courses.

• The courses focused more on competence upgrading or acquiring new competences and less on qualification. Out of the 445 training courses included in the research, 34.4% were training and re-conversion courses in different occupations, over 65% were designed to update the old or acquire new knowledge in specific fields.

2 For referring to the type of assistance offered within the project, it will be also used the term measure; this has a distinct meaning compared to the programme measures.



• The service sector was the main focus of the training courses developed. More than half of the courses organised within the investigated projects were targeted on the service sector, followed by the industrial sector (17.8%). Only 5.1% of the courses have trained human resources for agriculture, while 3.1% addressed the building sector. The interest of the promoters to develop training courses in the service sector is part of the economic development strategy of Romania, within which the development of the service sector is a priority. It is worth mentioning that a relatively high percentage (15.5%) of the courses are designed for public administration and defence, a field undergoing staff restructuring during the PHARE HRD 2000 projects.

• There is high diversity of the course duration, depending on the theme, scope and economic field of the training course. The average duration of a training course calculated in the research sample is 190 hours/course. The average duration of the vocational training/re-conversion courses is 374 hours/course, while the vocational development courses have an average duration almost 4-times shorter (97 hours /course). However, standard deviation data shows a high diversity of the course duration, depending on different variables. Therefore it is difficult to find a specific trend in the average length of the courses developed. The reason could be that each training provider organised the course duration according to its specific needs.

6.5. Effect Viability of Training Centres • Most of the training centres created via the HRD Component projects have received

authorization / are in the course of authorization (44%) or at least, they intent to pursue for authorization (36%). In addition, most of the training centres continue to use identical or even enhanced programme content, as compared to the one in the project (80%).

• In some cases, about 14% of the newly created training centres, promoting organizations having created them do not intend to pursue their accreditation. Moreover, the content of their training offer is limited or even significantly different as compared to the one developed in the framework of the projects. As a result, at least for some of them, project feasibility and sustainability may be questionable.

Completion and drop-out at beneficiary Level • The beneficiary dropout rate is about 4%; the unemployed registered the highest dropout

rate (8%). On the other hand, 2.4% of the employees have dropped out. As for the disadvantaged categories, the highest drop out rate is in young graduates.

• According to the beneficiaries surveyed, almost 67% of them received at the end of the course a graduation/qualification certificate recognised on the labour market and 17.1% received a certificate recognised at the company level.

Beneficiary satisfaction • In general, the direct beneficiaries of the training courses declared their satisfaction with

the relevance of the training content. They seem to be even more content with the quality of trainers. Almost 68% declared that they are satisfied to a large extent with the trainers, while 24% had a good enough opinion on the quality of teachers. Similar appreciations have been registered in the case of beneficiaries’ opinion on the methodologies for teaching and training that have been used during the course.

• The highest level of dissatisfaction is declared on the quality of learning aids and equipments and the opportunities offered to use learning facilities during the course. Data shows that 7% of the questioned beneficiaries were satisfied to a small extent or not at all with the learning equipments and aids provided by the training centre, while 12,6% of the beneficiaries declared that they didn’t have the opportunity to use the learning facilities at all or only to a small extent.



• Regarding the interests of beneficiaries for further training courses, most of the questioned persons shown their willing to attend similar training programs in the future.

6.6. Impact Effects on Beneficiaries • The employees having completed training courses record positive changes in their career

and professional status. An important percentage of the employees have advanced in their career (21.8%). Although over half of the employees failed to register any changes in their company jobs, almost 90% have stated that their activity at the work place has improved in terms of work quality, organization of tasks, and responsibility taken. However, such effects are not unexpected, knowing that most of the employees would have a higher educational level and qualifications before taking part in the course.

• The relatively high percentage of the unemployed and young graduates hired after having participated in the course (45% for an undetermined period and 7.9% as temporaries) represents one of the program’s successful points. It is also remarkable that 5.3% of the unemployed and young graduates have declared that they started their own business.

Impact Level in Promoters’ Opinion • Most promoters have pointed out the importance of the project impact at local and

organizational level. Only one fifth of the respondents have indicated an impact at the regional level, a point of concern from the perspective of the program general objective, i.e., support granted to the implementation of the integrated measures on regional development. The fact should be considered that, both at the level of all projects and at that of the sample project, the ratio of large-size projects has been highly reduced.

• Taking into account the high rate of the projects that developed training courses in the service sector and processing industry, a higher impact of projects in those sectors can be expected.

• The opinion of most project coordinators interviewed is that the project impact was significant, both on the target population groups and on the organization. The questionnaire survey indicates that the companies also appreciate the project effects; only 3% of them have considered the project effects as unsatisfactory. However, one out of four promoters showed some difficulties in evaluating the project impact.

• More than one third of the surveyed companies have registered a positive evolution since the project launch till the survey, both in terms of productivity, turnover, or profit margin, and of number of employees or level of investment in new equipment / technologies. In addition, an important number of companies show an increased budget allocation to training and increased numbers of employees participating in training courses.

• Only about one fourth of the surveyed companies have declared that the project has also aimed at developing the human resources management at the institutional level. The high number of companies having acknowledged project importance in view of the institutional development (e.g., for the improvement of the human resources management system) proves that the training activities have been designed without taking into account the real project potential and failed to involve all decision-makers at organizational level.

Dissemination of Results • Most of the investigated promoters have used diversified ways of dissemination. Among

them, the publication and distribution of project-related information material was the most frequent way used. Other ways of dissemination frequently mentioned were: the publication of information articles in mass media and the organisation of seminars and



conferences. Almost 38% of the promoters think that dissemination was assured by the publication of information materials in electronic form, while almost 22% of them have published training modules. Only 21.6% of the promoters have built up project websites.

• The most important level where the project results have been disseminated is the local one, as over 60% of the investigated promoters mentioned that they had carried out dissemination activities at this level. The dissemination at the regional level has been carried out only in 34% of the investigated projects. Only 15% of the projects involved national level dissemination.

7. Preliminary Conclusions • The Phare HRD 2000 program stimulated the interest for the field of human resources

development and the participation of various organizations, most of which has a limited prior experience in the HRD matter. The program represented a special opportunity for the participating companies, the percentage of which is dominant among the promoters. It is also remarkable that, of the total companies, the best represented were the small-sized companies, operating with less than 50 employees and with a history of about 10 years. This is due to the fact that, at project launching, the small companies had rather limited other funding opportunities in the field of HRD.

• The partnership seems to be rather limited in terms of number and profile of partner organisations; while a significant number of projects failed to have at list one partner. Social partners have been involved in a low degree in partnerships. The situation had negative implications in terms of regional and local impact. The local public administration was the preferred type of organisation as project partner; taking into account also the share of projects promoted by local administration, it appears that this type of organisation is one of the main beneficiaries of the program.

• In the case of all priorities, promoters demonstrated a low interest for addressing complex measures linked to well-defined target groups. Just a limited number of projects focused exclusively on active employment measures (objective 2) and social inclusion (objective 3); however it should be noticed that in the case of many projects surveyed, it was difficult to identify the main priority addressed. The analysis of data on the target groups provided by promoters has revealed that the promoters had some difficulties in explicitly defining and reporting them, particularly the group of the disadvantaged persons (envisaged by the social inclusion priority), both at the project initiation and during their implementation. The number of unemployed and socially disadvantaged categories strategically targeted and benefiting from the HRD measures (training, counselling, mediation for job etc.) was lower than expected.

• The promoters’ participation in the program has been an opportunity for them to build their own institutional capacity, irrespective of the type of organization. They created a number of 131 new training centres, much higher output that was expected, taking into account that in the project fiche the foreseen number was a minimum of 14 modernised training centres. Assuring the viability and sustainability of the newly created centres, subsequent to the termination of funding, remains an open issue. In addition, beneficiary statements on the use of the new equipment in the training process draws a signal as to the degree in which such equipment has been efficiently put to use.

• The training measures have occupied a very important place within the PHARE HRD 2000 projects. Employees benefited the most from the training measures and the courses focused on competence upgrading or acquiring new competences and less on training of low qualified persons or disadvantaged target groups. The focus of training on employees with a higher education level is typical for CVT programmes within Romanian companies. Regarding this, it can be stated that the program was less effective in reaching objective 3 – targeted at socially excluded persons.

• As a rule, the training fields envisaged by the projects correspond with some of the priorities for development at national level, such as services or fields undergoing reorganization at project launch (public administration, defence).



• As far as the impact of the training courses on the participants is positive – as shown in the statements made by an important part of them – (new competencies, career advancement and/or other changes, finding a job), we may appreciate course efficiency as relatively high.

• In terms of impact on training beneficiaries, program results have been positive. A significant number of the formerly unemployed and young graduates, who have participated in the training courses, found a job (temporary or for an unlimited period), and most of the employees have signalled significant changes in their activity or professional status.

• In general, the projects in the HRD component demonstrated an average capacity of fund absorption. The fund absorption level of 85% could be assessed as satisfactory, as far as a series of former programs developed in Romania registered, in general, lower external funds absorption. This indicator helps to demonstrate, to some extent, the quality of the promoters’ selection process and the stage of awareness raising within the programme.

• The data gathering, recording, processing, and reporting system related to the implemented projects has posed major difficulties in the evaluation of the program impact. Therefore, the lack of information in some cases affected the relevance of the comparison or made the calculation of key indicators for evaluation. In addition, the absence of synthetic input information made it necessary for the data to be collected directly from project promoters.

8. Preliminary Recommendations • Paying special attention to partnership size in the evaluation process, while stressing the

involvement of the social partners in HRD activities.

• Modification of the project selection procedures, aimed at achieving a better equilibrium in function of the target groups (stronger emphasis on the priorities related to improving active employment measures and promoting the social inclusion of the disadvantaged groups).

• A more careful project selection and monitoring, with relevant indicators, which should allow for the reduction of the number of insecure projects in terms of their viability and sustainability subsequent to the cessation of the external financing.

• Assistance should be granted all along project implementation, with a view to increasing fund absorption capacities and ensuring the centres’ viability and sustainability subsequent to the termination of the external funding. The opportunity to offer consultancy to project promoters in the area of financial management for project funded under European programs should be further examined.

• The monitoring and evaluation systems specific to HRD Phare 2000 programme require in-depth revision for all levels of implementation (local, regional, and national). Specifically, the following areas should be considered:

- The monitoring and evaluation systems used at region of development level call for consistency, to allow for the aggregation of different statistical data and to make available the information on program implementation at national level. There is a need for diversifying the categories of information, increasing information accuracy, and inclusion of relevant indicators in the project monitoring and evaluation, which should allow for the evaluation of the training courses’ efficiency, by category of target group.

- Activities should be developed with a view to making promoters aware of their key role in project monitoring and in supplying accurate, detailed, and complete data on the main objectives of their activities

- Developing a set of key indicators agreed by all stakeholders and designing successive stages in data gathering should facilitate the calculation of such indicators, observation of progress made, and the evaluation of correction



interventions at the critical time; emphasis should fall on the recommendations made during mid term evaluations.

- Devising new data gathering tools and procedures will permit accurate data collection, and identification and correction of erroneous data, as well; in point, the publication of resource materials (definitions of process and result indicators, ways of data gathering, etc.) will be appropriate. Development of a computerized system of data gathering and reporting of results concerning the monitoring and evaluation of the activities developed within the projects.



1. Evaluation methodology

1.1. Objective of the evaluation The present report focuses on the PHARE HRD component in the pre-ESF 2000 grant scheme and presents the outcome of its ex-post evaluation. The main objectives of the evaluation have been:

• Evaluation of the impact of Phare projects in the framework of the HRD component, while considering each of the three priorities of the component;

• Recommendations to be made on the implementation of future HRD structural projects.

The specific objective of the evaluation has been the identification of performance indicators and key data to improve on monitoring and evaluation system of future ESF type of programs in Romania.

1.2. Evaluation model The evaluation model used in the survey is based on the EU evaluation framework for structural funds. The evaluation model was the framework for the report structure, based on key evaluation issues presented in the following figure.

Figure 1.Linkages between the key evaluation issues IMPLEMENTATION

RELEVANCE EFFICACY / EFFICIENCY Source: European Commission, DG REGIO According to this model, context and baseline indicators are defined in relation to the needs which it is intended that the PHARE HRD 2000 programme addressed. Output, result and impact indicators are defined in relation to the objectives of the programme. A comparison between the actual and planned outputs, results and impacts indicates the effectiveness of the programme. A comparison between outputs, results and impacts and the costs (financial inputs) indicates the efficiency of the programme. The extent to which the outputs and results

Context IMPACT

Results Output ResourcesObjectives

Operations



are sustained following the end of the programme indicates the sustainability of the programme. The relationship between the impacts of the programme and the context and baseline indicators relating to the perceived needs gives an indication of the relevance of the programme, while the change in these context and baseline indicators due to the programme’s impacts indicates the utility of the programme.

1.3. Methods of investigation The evaluation of the HRD Component of the PHARE 2000 ESC Programme has taken place based on a complex methodology, with the use of diverse investigation methods.

• Analysis of the documents related to the HRD projects • Quantitative methods of investigation:

- Questionnaire mail survey of project promoters - Questionnaire mail survey of the direct beneficiaries of HRD projects

• Qualitative methods of investigation (interview survey of project promoters)

Sample The survey of the project promoters was exhaustive, as each of the promoters received a questionnaire. Out of 371 promoters, 171 answered the questionnaire. Investigated organizations promoted 185 out of 393 projects developed in the HRD component, representing 47%. Individual interviews with 30 promoters have been applied, selected on the following criteria: development region, type of organization, project size. They were selected from the counties of Alba, Arad, Călăraşi, Cluj, Constanţa, Dolj, Neamţ, and the City of Bucharest, as these represented all the development regions.

The selection of the sample of persons representing the target groups for the HRD projects used the lists with the participants in the training programmes developed in the framework of the projects. The selection criterion was the category of beneficiaries: employed / unemployed / disfavoured categories. The resulting sample counted 1,799 subjects and consisted of 3 sub-samples. The 1,799 individuals (55% women and 45% men) were mailed a questionnaire. 822 participants in training programmes answered the questionnaire. Thus, 822 participants in training programmes (64,5% women and 34,5% men) were subject to evaluation out of which 551 are employed and entrepreneurs (67%), 125 are unemployed (15,2%) and 146 are disadvantaged persons (17,8%). The number of respondents represents 45,7% of the total selected sample.



1.4. Experiences with gathering data

The information collected and available at program level was very scarce and that situation had a direct influence on evaluation activities. The Ministry of European Integration (MEI) provided a statistical overview of the program including five indicators estimated and accomplished (number of trained people, number of training courses, number of training centres created, number of equipments purchased and number of new created jobs). It is not clear how these indicators were calculated taking into account the fact that MIE provided to the research team a list with projects funded under Phare 2000 HRD scheme containing data only on name and code of the project, name and address of the promoter, the size of the grant duration and status of the project and the regional agency monitoring the project. Moreover, the information provided was not complete and updated and was not structured as a database. In order to gather input data on all promoters participating to the program, it was necessary to create within research activities a database in parallel with the survey by post and the field interviews. The project promoters, the monitors from regional developing agencies and from MIE assisted the research team in checking, updating and increasing the existing input information at program level, but this process involved important additional resources

1.5. Methodology of statistical analysis The techniques of statistical processing have been determined by the nature of the research, which reflects the summing up type evaluation rather than to a cause-effect relation. Under these conditions, depending on the nature of the analysed variables, we have determined: For the qualitative variables (nominal or ordinal), the most numerous in the case of a questionnaire:

- The Occurrence rate of the concerned variants, as well as the structure of the responses (percentage);

- The Distribution of the results in the contingency tables; - The Comparative evaluation of the distribution of the main indicators according to the

two basic types of promoters, namely companies and other categories of promoters. For the quantitative variables - concerning the material and financial resources of the project (the budget and its main expenditure items), the number of beneficiaries of the training programmes and number of the training courses organised – the following have been determined:

- total and average levels by categories, as well as the minimum and maximum levels; - structure indicators concerning the number of beneficiaries, by categories, including

the expenditures by main budget categories and within a category; - derived indicators representing unitary levels, such as: average cost by category of

beneficiaries, average duration of a course (average number of days) by type of course or fields in which it was organised, etc.

- bi-varied correlation between the distribution by the two categories of promoters and number of courses they organised, and between the two categories of promoters and the total number of training courses beneficiaries.



2. Context

2.1. Analysis of the actual state in the field of employment and training in Romania

• Education and training This chapter is focused on the main figures regarding the participation of young and adults in education and training in Romania. The analyses is based on a set of relevant indicators in a comparative perspective with a number of neighbouring countries. Rate of participation at all levels of education (ISCED 1-6) of the population aged 15-24 Romania is on an inferior position, compared to both the EU member countries, as well as the countries that have recently joined the Union, regarding the participation at all levels of education of the population aged 15-24: 41,9% compared to 57,5% the EU average (chart 2.1.1).

Chart 2.1.1. The rate of participation at all levels of education (ISCED 1-6) of the

population aged 15-24 – comparative data

010203040506070

Rep

.C

ehă

Est

onia

Ung

aria

Leto

nia

Litu

ania

Pol

onia

Slo

vaci

a

Slo

veni

a

Bul

garia

Rom

ânia

Med

ieU

E

Source: Progress towards the Lisbon Objectives in Education and Training. 2005 Report, Brussels, Commission of the European Communities, 2005. The rate of the population aged between 20-24 years, which finalised at list superior secondary level In 2004, the secondary higher education graduation rate of people with ages between 20 and 24 with respect to the whole population within same age segment was 76.4% in the European Union. Romania has a similar rate to the EU. The highest values come from Slovakia, the Czech Republic and Poland, with over 90% (Key Data on Education).



Chart 2.1.2. The rate of the population aged between 20-24 years, which finalised at list

higher secondary level

70 75 80 85 90 95 100

Czech Republic

Estonia

Hungary

Letonia

Lituania

Poland

Slovakia

Slovenia

Bulgaria

Romania

UE average

Source: Progress towards the Lisbon objectives in education and training. 2005 Report, Brussels

Early school dropout rate of youth aged 18-24 Comparative data concerning the indicator of early school dropout rate of youth aged 18-24 places Romania way below the level of the member states or of the states that have recently joined the EU, except for Malta. Thus, in 2004, the value of the indicator for Romania recorded 23.6%, as compared to countries such as Slovenia – 4.2%, the Czech Republic – 6.1%, Bulgaria – 21.4%, EU members – 15.9%.

Chart 2.1.3. Early school dropout rate of youth aged 18-24 – comparative data

0

5

10

15

20

25

Czech

Rep

ublic

Estonia

Hunagary

Letonia

Lituan

ia

Poland

Slovak

ia

Sloven

ia

Bulgaria

România

EU-15

Source: Progress towards the Lisbon Objectives in Education and Training. 2005 Report, Brussels, Commission of the European Communities, 2005.



Participation of adults in lifelong learning From 2002 up to 2004 Romania has not recorded any significant progress regarding the participation of adults in the lifelong learning. Next to Bulgaria, Romania is still among the last countries in Europe regarding the percentage of adults aged 25-64 who take part in the education and training process. According to the latest data given by the Labour Force Survey, in 2003, only 1.3% of the Romanian adults, aged 25-64, participate in the education or training program, while the EU-25 average is about 9%, with the best countries reporting participation levels of over 18%. It is therefore obvious that it is important to make every effort into this direction if we take into consideration the fact that the European objectives anticipate that the medium level of participation in the permanent education throughout the EU to be of at least 12.5% out of the population aged 25-64, by the year 2010.

0

5

10

15

20

Czech

Repub

lic

Estonia

Hunga

ryLatv

ia

Lituan

ia

Poland

Slovak

ia

Sloven

ia

Bulgari

a

Roman

ia

UE avera

ge

Chart 2.1.4. Participation rate of adults to education and training (25-64 years old). Comparative data: 2002 and 2004

20022004

Source: Progress towards the Lisbon Objectives in educational and Training, 2005 Report, Brussels, Commission of the European Communities, 2005

In Romania, the adults' participation in permanent education is still among the last in Europe. The result of the second Eurostat survey regarding the continuing training in enterprises showed that, in 1999, only 11% of the enterprises provided professional training for their employees, while an National Employment Agency report showed that a mere 2.5% of the unemployed had attended professional training courses in 2002. According to Eurostat study, the percentage of participation in these continuous training programs in Romania is much smaller than that of the Czech Republic (69%), Slovenia (48%) or Hungary (37%). In the Member states, 70% of the enterprises provide continuous professional training for their employees, except for Spain (36%) and Portugal (22%), whose percentages are smaller. In Romania, the rate of participation with the enterprises that offered such courses was of only 20% (for both men and women), situating itself below the participating countries average (30%) and much lower that that of the Member States. This



data emphasizes an accentuating tendency among the discrepancies between the workforce of Romania and those from the rest of Europe. The Country Monograph on Education and training and employment policies in Romania, done in 2003 by ETF, have shown that adults' participation in continuous training is still polarized, with the participants to such training programs usually being highly qualified persons, with a higher level of education that have management or middle-management positions in a company. Thus, continuous professional training programs still remain much inaccessible for those categories that primarily need continuous training: people with a low level of education, unemployed, underprivileged categories.

• Trends in employment The Country Monograph on Education and training and employment policies in Romania shows that the labour market developments in Romania have been quite different than in other Future Member States. Thus, the employment rate is falling 62.4% in 2001, (down by 1% per year since 1997) and 57.6% in 2002 (EU-15 average 64.3% and the average of 9 Acceding Countries (AC) 55.98%1). The decline in the urban employment has been reflected in job growth in agriculture. Unemployment has not risen sharply. However, in 2002 more than 35% of the employed persons worked in agriculture2 (with 31% in industry and construction and 34% in services). The higher employment rate in Romania compared to the AC-10 average is linked with the high employment rate in agriculture, much of it based on subsistence agriculture. The employed in this sector include many unpaid family members and almost 40% of the employed persons are females. The unemployment rate was estimated at 7.0% in 2002, one of the lowest rates among 13 Future Member States (FMS), slightly lower then the EU-15 average (7.7%) and far below the average for 9 Acceding Countries (14.8%). However, despite the industrial restructuring, hidden unemployment in less productive activities continues to be important. Long-term unemployment as well as youth unemployment figures for 2002 (3.8% and 18.5% respectively) was close to the EU-15 average (3.0% and 15.1% respectively). As a large country, Romania experienced also strong regional disparities in terms of labour market performance. In 2001, regional variations in the unemployment rates are sizeable in Romania: 8% in the Southeast region as opposed to 4.8% in the Northwest region. The rate was at a very low level (4%) in the capital city. At the level of year 2002, can be observed a diminuation of differences between regions according to this indicator: Southeast and South – 4,7% and, respectively, West – 3,2%. The regions are highly heterogeneous in composition. Regional data are therefore only an approximate indicator. County data show much stronger disparities within the country.

1 Source: Eurostat, New Cronos database (October 2003) 2 Based on Labour Force Survey data. Note that the definition used in AMIGO (the Romanian LFS) for employment in agriculture is based on at least 15 h employment in the four weeks before the survey, whereas the EU LFS definition uses 1-hour employment as a basis. Thus employment in agriculture would be even higher using the common EU definition.



2.2. Convergence of the Phare 2000 Program Objectives with the European and national policies The Phare 2000 ESC program objectives closely relate to the objectives of HRD public policies, both at European and national levels. Actually, the major European and national documents related to HRD policies take account of the three priorities of the Phare 2000 ESC, HRD Component.

The analysis focuses on policy documents relevant for the period of the HRD program implementation (2000-2003). Among these, is should be mentioned the following:

• At European level, the European Employment Strategy, launched by the European Council in Luxemburg (November 1997), developed on a four pillars approach: i) active labour market policies; ii) adaptability, lifelong learning; iii) employability, national inclusion; iv) equal opportunities, entrepreneurship. Among these, the first three are explicitly followed in Phare HRD 2000 scheme while the fourth pillar (entrepreneurship) is included in the overall objectives of Phare 2000 ESC Program.

• At national level, the period 2000-2003 witnessed important HRD developments at policy level, reflected in strategy documents including: The National Development Plan, The National Action Plan for Employment, Joint Memorandum on Social Inclusion, National Anti-Poverty and Promotion of Social Inclusion Plan.

The policy documents, both at national and European level, have guided the process of designing the objectives and priorities of Phare 2000 HRD scheme. However, a number of differences in terms of explicit ratio and content can be observed, as further analysed.

This is how the major European and national public policy documents cover the Phare 2000 objectives:

Priority 1: Training and re-training of the labour force under employment

At European level The European Employment Strategy relies on four pillars, two of which, at least, (adaptability and entrepreneurship) are in agreement with Priority 1 as set in the Phare 2000 program. One of the ‘employment guidelines’ explicitly refers to the development of human resources and to a series of specific measures found among the actions proposed in the Applicant Guide (item 2.1.3.).

At national level The chapter, Social and human resource policies in the National Development Plan (2000-2002) explicitly provides for continuous training and learning as a priority of the Government Program over 2001-2004. The National Action Plan for Employment contains ten guidelines, three of which explicitly direct to the Phare 2000, HRD Component objectives:

- Supporting adjustability and mobility on the labour force market; - Encouraging the development of human capital and lifelong learning;



- Ensuring labour proficiency while providing matching incentives. The Joint Memorandum on Social Inclusion sets up the decade’s objectives aiming at solving the serious structural problems of the 1990s (extreme poverty, long-term unemployment, social exclusion, and mass emigration of the young labour force). Of these, the following objectives refer to the actions included in Priority 1 of the Phare 2000 ESC, HRD Component:

- Rising the quality of employment in rural areas and in agriculture via investments in vocational training and guidance of the rural population, and supporting entrepreneurship;

- Expanding youth capacity in turning to account employment opportunities on the labour force market by encouraging partnerships between the systems of education and the business communities, aiming at vocational training and employment;

- Encouraging company investments in human resources designed to increase labour productivity and job quality. A chapter in the Joint Memorandum is dedicated to participation in employment and the HRD role in this sector.

The National Anti-Poverty and Promotion of Social Inclusion Plan contains a ‘Social Program’, which integrates the objectives of the social policy over the 2002-2003 years. Although they do not specifically refer to the field covered by Priority 1 in the Phare 2000 ESC, HRD Component, the objectives are implicitly present in most of the directions for action of the social policies. For example:

Objective 6 – Ensuring access to social services (employment included); Objective 11 – Community development and social partnership (meaning, the human

resources of the social policies). In point of sector analysis, The National Anti-Poverty Plan dedicates a whole chapter to employment. Among the objectives established for the sector (in terms of fighting poverty and social exclusion), one objective is significant in point:

Objective 6 – Improvement of vocational training, with its two components (vocational education and training and continuous vocational training).

Priority 2: Development of the active employment measures for the

unemployed, as a systemic tool designed to encourage labour force employment

At European level The European Employment Strategy identifies ‘employment’ as an essential pillar and the role of active measures in ensuring social cohesion and fighting unemployment.

At national level A paragraph—which clearly coincides with the content of Priority 2 in the Phare 2000 ESC, HRD component—in the chapter on “Social Policies and Human Resources” of the National Development Plan (2000-2002), acknowledges:

The Government program aims at increasing the employment of the population and at harmonising internal legislation on access to the labour market with the relavant community



acquis. In this sector, the policy will gradually pass on from passive to active measures, with less stress on the financial provision of the unemployed and more stress on their reconversion and job creation.

In turn, the National Action Plan for Employment also contains objectives closely related to the meaning of Priority 2, that is:

Guideline 1 – ‘Active and preventive measures for the unemployed and inactive’

Guideline 5 – ‘Increasing the labour market supply and promoting active old age’

Guideline – ‘Converting black market work into employment’

The Joint Memorandum on Social Inclusion stipulates the following measures in agreement with the Priority 2 active measures:

• Creation of new jobs in the high added value economic sectors, particularly in the field of services;

• Focussing programs of active measures on long-term unemployment and target social groups with a high risk of social exclusion

The chapter dedicated to participation in employment mentions:

• At present, the major directions for action are the fight against structural unemployment and encouragement of key groups’ participation in the labour market (young unemployed aged 15-24 and long-term unemployed). Specific programs of active measures apply in the fight against the effects of structural unemployment.

• Objective 5 of the Government Priorities 2004-2004, included in the National Anti-Poverty and Promotion of Social Inclusion Plan includes active measures related to anti-poverty fight and social inclusion, namely:

- Raising the financing ratio of active programs for employment (22% of the Unemployment Fund in 2003);

- Increased program efficiency.

Priority 3: Support for the Social Inclusion of Underprivileged Groups At European level One of the four pillars of the European Employment Strategy sends to equal opportunities. The content of Priority 3 is also among the objective of the European Social Fund.

At national level The National Development Plan specifies:

• The reform in the field of social policies concerns population groups facing difficulties during the economic transition (the handicapped, disfavoured old aged persons, etc.) and directs to an enhanced pension system, active employment measures and protection of the unemployed, harmonisation of the legislation with the European Union norms, the fight against poverty and expansion of child care reforms.



• The Plan continues, “The new economic environment of Romania makes the younger population vulnerable”, and the subsequently designed National Action Plan for the Youth aims at the protection of this category.

With reference to the Rroma population, the Plan includes the “Strategy on Improving the Status of the Rroma Population in Romania”.

• Three guidelines out of ten in the National Action Plan for Employment connect to activities included in Priority 3 of the Phare 2000 ESC, HRD Component, namely:

- Guideline 6 – Gender equality;

- Guideline 7 – Integration and the fight against labour market discrimination of the disfavoured;

- Guideline 10 – Fight against regional disparity in employment.

• The Joint Memorandum on Social Inclusion establishes the following priorities in the field of social and employment policies:

- Reducing the after effects of structural unemployment, particularly in support of population groups with difficulties in accessing the labour market (the long-term unemployed, the rural population);

- Mounting efforts aimed at the inclusion in economic activities of some risk population groups: young Rroma persons, the handicapped, young persons coming from foster homes and other minor care centres;

- Diversified ways aimed at increasing the job numbers, particularly in economic crisis areas (mining and subsistence agricultural areas).

• The National Anti-Poverty and Promotion of Social Inclusion Plan includes some objectives similar to those indicated in eligible measures for Priority 3, namely:

- Objective 2 – Eradication of morally unacceptable social situations in a civilized society: children of the street, abandoned institutionalised children, traffic of human beings, home violence, family neglected and/or abused children; development of a victim support system;

- Objective 4 – Alleviation of regional gaps, recovery of neglected zones, and prevention of regional disparities

- Objective 5 – Promoting an inclusive society characterised by a high degree of social cohesion;

- Objective 6 – Ensuring the access of all members of the society to basic social services: health care, education, employment, and social assistance.

The analysis of the documents on public policies reveals two clear-cut trends, traceable among the three priorities of the Phare 2000 ESC program, HRD component.:

• First, the harmonisation of the national directions and priorities with the European ones, above all with the provisions of the European Employment Strategy;

Second, employment is a major pillar in terms of social integration and participation, which explains the priority that has been granted to the vocational training of the employed, the



active measures targeted at the unemployed, and the equal opportunity and social insertion policies, focused on vulnerable persons and population groups.

2.3. Former programmes

To learn from EU experience and the evaluations of the former Phare programs is a crucial success factor for the implementation of future ESF type of activities. From the start up of Pre-ESF activities in Romania, ETF has been actively supporting institution building measures that should enable the Romanian stakeholders to make better use of the opportunities offered trough ESF and Pre-ESF activities. The findings of former evaluation of Phare program in Romania drown attention to different specific issues.

The Phare 97 Pre-HRD scheme focused on relevant measures to promote HRM in companies. The evaluation showed that in spite of many good individual examples, the impact at system and enterprise level has been limited.

The Phare 98 HRD grant scheme evaluation showed that CVT has been stimulated by the call for applications, and there has also been a positive impact on employment in many cases. However, the evaluation has not been able to demonstrate how structural these effects have been. The 98 evaluation did show that local partnerships should be reinforced to enhance the structural impact of ESF measures.

The grant schemes in Romania were reviewed with an overview of the 1998-2000-2002-2003-2004-2005-2006 grant schemes (total value approx.150 MEURO).

The 1998 ex-post evaluation focused on the selection process, project outputs and recommendations for future schemes. It managed to provide a clear overview of how the scheme was implemented and provided valuable feedback for the implementation of the 2002 scheme, which is visible of what we know now of the 2002 scheme, but was limited due to the limitations of the PRAG (Practical Guide for Phare, obliging standardised procedures for selection and monitoring not adapted to HRD activities). The evaluation followed a constructivist approach; the main weak points were lack of triangulation and objectivity, focus on outputs with no real assessment of impact.

The RICOP evaluation of Active Employment measures used a quasi-experimental approach based on a comparison of counselling and training measures from RICOP with a randomly chosen control group from the NAE register. Weak points in the evaluation were the presentation of the findings, and the fact that the actual positive impact was within the margins of error (given the size of the sample).

On request of the EC Delegation in Romania, ETF organised the ex-post evaluations of the Pre-HRD Phare 97 Progress scheme, promoting HRM in enterprises and the Pre ESF Phare 98 HRD grant scheme. In 2004, ETF has also been requested to organise the ex-post evaluation of the Pre-ESF 2000 HRD grant scheme.



3. Objectives and priority issues The main goal of the PHARE 2000 ECONOMIC AND SOCIAL COHESION PROGRAMME – Human Resources Development (HRD) in the context of industrial restructuring was to support the Romanian Government in implementing an integrated regional development policy in line with the economic and social cohesion strategy adopted within the context of the pre-accession process, through regional development investment projects in priority sectors (Human Resource Development, SMEs and local and regional infrastructure) identified in Romania’s National Development Plan and in Romanian Regional Plans. Support to these sectors aims at developing, through an integrated approach, the indigenous economic and social potential of the development regions. The Human Resource Development component of the programme had three main priorities: 1. The qualification and re-qualification of the work force in order to make it more responsive

to the evolving needs of the labour market. 2. The enhancement of active employment measures as systemic tools to foster

employment 3. The promotion of social inclusion for disadvantaged groups. The following minimum and maximum amounts applied to the grants for individual projects financed under the HRD component:

� small projects - a minimum grant amount of EURO10,000 and a maximum amount of EURO50,000

� large projects - a minimum grant amount of EURO100,000 and a maximum amount of EURO300,000

The eligible applicants included both the non-profit making and the profit-making organisations, being possible the association in a consortium with other partners in the framework of the same project. The three priority themes of the project were:

Priority 1: Qualification and re-qualification of the work force in order to make it more responsive to the evolving needs of the labour market.

Priority 2: Enhancement of active employment measures as systemic tools to foster employment.

Priority 3: Promotion of social inclusion for disadvantaged groups Activities had to take place in one of the eight development regions in Romania. The programme envisaged a large type of activities, including the following, but not limited to:



Priority 1: Qualification and re-qualification of the work force in order to make it more responsive to the evolving needs of the labour market

The activities eligible for support under this priority were targeted at the employed labour force • Improving the quality of continuing vocational training, in accordance with identified labour

market needs, including through the development of training packages; • Training of managers and staff, particularly in SMEs to maximise the use of new

production technologies, development of new products, communication technology, marketing practices, human resources management and leadership and networking;

• Training in management skills including innovation, environmental, quality control, logistics, communication, especially in SMEs;

• Training for entrepreneurs and would-be entrepreneurs, including entrepreneurs in the social sector, to support business start-ups, development and sustainability. Particular attention should be given to the promotion of women SMEs;

• Modernisation of VET systems and innovation in the field of vocational education and training with the aim of strengthening the skills and competencies relevant to the needs of the labour market in order to enhance the employability of young people (development of curricula packages, training of teachers, head teachers, social partners and acquisition of related didactic materials);

• Enhancing Human Resources Development in the context of industrial restructuring; good practice in SMEs through, for example, developing personnel and training strategies and plans; in-company key personnel training (manager/company training officer/personnel staff/trade union leaders) to develop a lifelong learning culture for all employees within enterprises and HRD good practice networks (provision of high quality counselling and guidance);

• Developing, delivering and evaluating training programmes and courses for trades related to tourism, handicraft, agro-food etc, to foster rural diversification in accordance with regional development plans;

• Vocational training to address specific innovation and technology skills shortage; • Projects aimed at implementing “Stabilisation of employment” and “paid released for

training” plans. Priority 2: Enhancement of active employment measures as systemic tools to

foster employment. The activities eligible for support under this priority are targeted at the unemployed labour force. • Identification of current and future regional and local training needs and skill shortages

required by the labour market, development of a system of assessing the potential of individuals and of tracking their career development, to be used as tools by the employment offices and relevant HRD service providers



• Improvement of the communication and information systems between the institutions with responsibilities in the field of training and employment;

• Development of training and retraining programmes of the short-term unemployed, which address skill shortage opportunities;

• Supporting employment of the long-term unemployed through specific training, guidance and placement schemes;

• Range of support measures for unemployed people, especially young unemployed, to return or enter the labour market (including professional counselling and vocational guidance, job search, job clubs, graduate training initiatives, work placements, continuing vocational training and retraining, mentor support, etc.).

• Advice and guidance for setting up and running community income generating activities to support the development of the local economy, i.e. through revitalisation of local/regional craft trades so as to promote employment;

• Vocational guidance and training in business-related services for identified potential entrepreneurs in order to promote entrepreneurship and job creation.

Priority 3: Promotion of social inclusion for disadvantaged groups The activities eligible for support under this priority are targeted at disadvantaged categories of population, including: people with physical, mental and social disabilities (drugs addicted, ex-prisoners etc), Roma population, young people leaving child care institutions, women returning to the labour market, unemployed aged over 45, single parents, young people who dropped out from education without a basic qualification. • Diagnostic studies, surveys, etc. with a view to identifying and assessing of the needs of

socially excluded people, in particular in relation to the access to the labour market; • Compensatory training and apprenticeship schemes for young people who drop-out of

education without a basic qualification; special attention will be given to the Roma population and to young adults from foster institutions;

• Training of trainers, especially social trainers, for developing and adapting programmes to the needs of specific groups, supporting the social inclusion;

• Setting up counselling, guiding and information (especially “legal”) services aimed at social inclusion;

• Developing and providing training, including tailor-made modules in accordance with the needs of the participants;

• Advice on setting up and running community based income generating activities and support of the entrepreneurship of the disadvantaged groups;

• Supporting Employers’ initiatives in the sheltered employment sector on behalf of the disabled;

• Placement of people from disadvantaged groups in companies or NGOs for on-the-job training through temporary work, in order to allow these people to get (new or additional) professional experience.



4. Resources

4.1. Funds by regions The Social and Economic Cohesion Programme received funding within the framework of the Joint Memorandum of the Government of Romania and the European Commission on the PHARE 2000 National Program ratified by the Government Ordinance no. 51 / 2001. The general program objective was the support granted for the implementation of the integrated regional development programs in a number of priority regions in Romania. Fund allocation – the HRD Component included – took place as follows, while considering two categories of regions: 1. Priority regions category I, including regions with complex development problems —

high unemployment rates; high dependency on agricultural activity; poor development of the social and technical infrastructure. These regions – South-East, North-East, South Muntenia, and North-West – benefited from the entire amount allocated by the European Union via the PHARE SEC program.

2. Priority regions category II, which includes the remaining regions – South-West Oltenia, West, Centre, Bucharest-Iflov – benefited from the funds allocated from the national budget, funds representing Romania’s joint financing joining to the amount allocated by the European Union for the projects implemented in Priority regions category I (according to the agreement with the European Commission).

Owing to the criteria considered, the funding volume differs from one region to another. Thus, the highest funding level from Phare/national budget sources has been recorded in the regions North-West - 20.7% and South-East – 20.3% of the total allocated funds while the lowest level has been recorded in the regions South-West Oltenia – 5.0% and Bucharest-Ilfov – 5.4% (Table 4.1.1.).

Table 4.1.1. Level of funding from Phare source / national budget, by development region

Level of funding Region of development

Abs val. % by region

North-East 3306752 15.8 South-East 4253932 20.3 South Muntenia 3790637 18.1 South-West Oltenia 1045789 5.0 West 1633340 7.8 North-West 4323649 20.7 Centre 1429415 6.8 Bucharest-Ilfov 1128759 5.4 Total 20912273 100.0

Obs: Data on 416 projects, including the ones not finalized

Source: Calculations based on information supplied by the MEI.



Subsequent to the different funding level from one region to another, the sample evaluated also presents a series of differences, in point of total funding and average value of the budget awarded to each project at region level (Tables 4.1.2 and 4.1.3.).

TABLE 4.1.2. Level of funding and average budgets awarded to projects by development

region

Funding level Average budget per project Funding source

Region of development Companies Other

organizations Total Companies Other organizations Total

PHARE / national budget

North-East 143180 970094 1113274 47727 32336 33736

South-East 828121 1099536 1927657 103515 57870 71395

South Muntenia 265260 576884 842144 37894 33934 35089

South-West Oltenia 217310 367927 585237 36218 30661 32513

West 261800 420962 682762 26180 35080 31035

North-West 355355 1181822 1537177 32305 62201 51239

Centre 109876 272272 382148 36625 22689 25477

Bucharest-Ilfov 374798 534023 908821 46850 66753 56801

Total 2555700 5423520 7979220 45638 42043 43131

Own contribution North-East 62700 442409 505109 20900 14747 15306

South-East 323222 326628 649850 40403 17191 24069

South Muntenia 143128 190471 333599 20447 11204 13900


West 132320 211481 343801 13232 17623 15627

North-West 130953 643878 774831 11905 33888 25828

Centre 38273 116402 154675 12758 9700 10312

Bucharest-Ilfov 218553 213515 432068 27319 26689 27004

Total 1196072 2399899 3595971 21358 18604 19438

TOTAL North-East 205880 1412503 1618383 68627 47083 49042

South-East 1151343 1426165 2577508 143918 75061 95463

South Muntenia 408388 767355 1175743 58341 45139 48989


West 394120 632443 1026563 39412 52704 46662

North-West 486308 1825700 2312008 44210 96089 77067

Centre 148149 388674 536823 49383 32390 35788

Bucharest-Ilfov 593351 747538 1340889 74169 93442 83806

Total 3751772 7823420 11575191 66996 60647 62569

Source: Calculations based on information supplied by the MEI.

Data collected also indicate:

- A lower funding level for companies, as compared to the other types of organizations, at each development region level (Table 4.1.3.), an understandable situation considering the low number of companies among the total promoters in the sample investigated;



- The higher average budget per company project, at each region level (Table 4.1.2.), as compared to other types of promoting organization (per total regions, the average budget – from Phare sources / national budget – of a company amounted to almost EURO45,600; in the case of the other organizations, it was EURO42,000).

TABLE 4.1.3. Distribution of allocated budget by region of development and type of

promoter

Percent from project total budget Percent from budget per region Region of development

Companies Other organizations Total Companies Other

organizations Total

North-East 5.5 18.1 14.0 12.7 87.3 100.0 South-East 30.7 18.2 22.3 44.7 55.3 100.0 South Muntenia 10.9 9.8 10.2 34.7 65.3 100.0 South-West Oltenia 9.7 8.0 8.5 36.9 63.1 100.0 West 10.5 8.1 8.9 38.4 61.6 100.0 North-West 13.0 23.3 20.0 21.0 79.0 100.0 Centre 3.9 5.0 4.6 27.6 72.4 100.0 Bucharest-Ilfov 15.8 9.6 11.6 44.3 55.7 100.0 Total 100.0 100.0 100.0

The projects funded from the grant scheme were of two types: - Large-scale projects: with regional impact covering two counties at least, these

projects assume the integration of more activities and encourage social partnerships among education and training providers, companies, and social partners; the funding grants amount from EURO100,000 to EURO300,000;

- Small scale projects: with local impact (towns or counties) and covering at least one eligible activity; the grants awarded vary from EURO10,000 to EURO50,000.

At the first bidding session, promoters from all regions of development could present their project tenders in either of the two categories. At the second bidding session, however, in the West and Centre regions, only small-scale projects received funds, amounting to a maximum of EURO50,000.

By project size, out of the total, 93% projects entered the category of small-scale projects, while only 7% represented large-scale projects. Projects in the latter category developed particularly in the development regions of the South-East (the bulk), North-West, and Bucharest Iflov. Function of the promoting organization, similar projects were developed by organizations other than companies (62%, as against 38%) - Tables 4.1.4 and 4.1.5.



TABLE 4.1.4. Project types by region of development

Large-scale projects Small-scale projects Total Region of development Number Percent Number Percent Number Percent

North-East 1 7.7 32 18.6 33 17.8 South-East 5 38.5 22 12.8 27 14.6 South Muntenia 1 7.7 23 13.4 24 13.0 South-West Oltenia 0.0 18 10.5 18 9.7 West 0.0 22 12.8 22 11.9 North-West 3 23.1 27 15.7 30 16.2 Centre 0.0 15 8.7 15 8.1 Bucharest-Ilfov 3 23.1 13 7.6 16 8.6

Total 13 100. 0 172 100. 0 185 100.0

TABLE 4.1.5. Projects by types of promoters

Large-scale projects Small-scale projects Total Promoting organization Numbers Percent Numbers Percent Numbers Percent

Companies 5 38.5 51 29.7 56 30.3 Other organizations 8 61.5 121 70.3 129 69.7

Total 13 100.0 172 100.0 185 100.0

The higher share of small projects entailed a higher financing level in the case of the surveyed promoters, both per total and at the level of company or other organization projects (Table 4.1.6).

TABLE 4.1.6. Distribution of allocated budget by project types and promoters Percent from project total budget Percent from budget per region

Project type Companies Other organizations Total Companies Other

organizations Total

Small-scale project 69.6 68.4 68.8 32.8 67.2 100.0

Large-scale project 30.4 31.6 31.2 31.6 68.4 100.0

Total 100.0 100.0 100.0 32.4 67.6 100.0

4.2. Profile of project promoters Typology of project promoters The present section contains synthetic information on promoter profiles and the projects they have coordinated. We shall identify the main categories of organizations having developed projects in the framework of the Phare 2000 ESC, HRD Component, their geographical distribution, and main activity fields. In addition, the section also investigates the partnership structure and partner level of co-operation.

Following two bidding sessions, 1,862 project proposal have been received in the framework of the Human Resources Component. Of these, 416 projects were eligible for funding. The number of project proposals reflects an increased interest on the part of eligible institutions: the indicator registers a significant increase as compared to the previous similar program



(Phare 1998), the first large scale program with European funding in the field of human resources development in Romania. A total of 371 organisations coordinated the projects selected for financing. Most of the institutions developed one project only, while around 10% developed concomitantly more projects. It should be noticed also the fact that some of the promoting institutions are county or regional branches of national organizations: e.g., chambers of commerce, trade unions, professional associations, NGOs, etc. also participating to the program.

The analysis of the types of co-ordinating institutions reveals that economic agents (companies, enterprises, consultancy firms, SMEs, etc.) make up the major category of promoters of over one third of the total projects. Non-governmental organizations (associations, foundations, trade unions, and employers’ organizations), well represented, hold 25% of the total. Educational institutions make up 16% of the total projects, with pre-university and higher education institutions holding equal footing. Public administration (local and county councils) and other public institutions (county school inspectorate, county agencies for employment, public training centres, etc.) promoted 12%, and 11%, respectively, of the total projects (Chart 4.2.1).

As illustrated in Chart 4.2.2, the distribution of the main categories of promoter institutions in the case of the sample investigated by the present research nears total distribution. Although smaller by six percentage points than the per-total distribution, economic agents represent the main category (30%). For the rest of the categories, differences are small – one or two percentage point – except for public administration institutions: in their case, there is a slight over-representation by 5%.

CHART 4.2.1. Project distribution by type of promoting organization (total projects)

36%

25%

12%

8%

8%

11%Economic agents

NGOs

Public administration

Pre-university education institutions

Universities

Other public institutions

Source: The Ministry of European Integration (MEI) of Romania



CHART 4.2.2. Project distribution by type of promoting organization (sample)

30%

24%

17%

10%

9%

10% Economic agents

NGOs


Pre-university educationinstitutions

Universities


Considering that, at project launch, HRD grant accession opportunities were scarce, economic agents were highly motivated to participate in the program. This explains the large number of project proposals and that of projects implemented by companies. However, no data is available at program level on companies characteristics (form of ownership, field of activity, number of employees, or length of operation, etc.) in order to get a clearer representation of the general profile of this type of coordinators.

The questionnaire survey however succeeded in supplying some information on companies participating to the program. Charts 4.2.3, 4.2.4, 4.2.5 and 4.2.6. offer the graphic representation of companies distribution by form of ownership, field of activities1, size, and length of operation.

CHART 4.2.3. Ownership

86%

7%7%

private capital

mostly statecapital

mostly privatecapital

CHART 4.2.5. Staff

42%

18%

13%

7%

13%

7%

less than 10

between 10and 50between 51and 250between 251and 500over 500

no answer

1 The economic fields taken into consideration were the followings: Agriculture, game and forestry, Processing industry, Electric and thermal power, gas and water, Constructions, Sale and repair of cars and household goods, Hotels and restaurants, Transport, warehousing and communications, Financial transactions, Real estate and business transactions, Public administration and defence, Education, Health and social protection, Other services



CHART 4.2.4. Field of activity

20%

2%4%2%

13%

45%

14%processingindustryconstructions

commerce andrepair workstransport

education

other services

no answer

CHART 4.2.6. Period of operation

20%

30%

43%

7%

less than 5 years

5 to 10 years

over 10 years

no answer

The projects coordinated by NGOs (about a quarter of the total) also hold an important ratio, due to the rich experience NGOs hold in preparing project proposals, participation to bids being their major self-financing source. Given the rigors of the grant competition, NGOs were quite successful in their endeavour. Professional association, trade unions, and employers’ organizations (institutions having been included in the category) also play a significant role as promoters of training programs in their economic sector or branch.

Other categories include local administration institutions (local and county councils) and other public institutions (county employment agencies, school inspectorates, research institutes, etc.).These institutions enjoyed a privileged position within the project, some of them being directly represented in the RDA councils and having an easy access to information on this program.

Numerous pre-university or university education institutions have been involved also in the program as project promoters, holding a share around 16% in the total projects. At pre-university level, most of the institutions, coming from SAM (School of Arts and Trades) succeeded to extend their offer to continuous vocational training and capitalize on the experience acquired in the Phare VET reform program. In terms of higher education, universities with different areas and from diverse regions of development also took part in the program. However, it is interesting to notice that only public higher education organisations participated in the program.

Geographical distribution of projects

The geographic distribution is in direct relation with the funds allocated to each region of development (Chart 4.2.7). The development regions with the highest number of selected projects include the North-West, South, and North-East. At the opposite end lie South-West and Bucharest Ilfov.



CHART 4.2.7.Distribution of projects by region (total and sample)

14,913,7

15,9

7,7

11,3

21,8

10,6

4,1

17,8

14,613,1

9,7

11,9

16,2

8,1 8,6

0

5

10

15

20

25

NE SE S SW W NW C BI

Number of projects per region(total)Number of projects per region(sample)

Source: Data calculated based on information supplied by the MEI and project promoters.

We should notice that the MEI system of registration focuses on the project and not on the promoter, considering a project belonging to the region of development where the project was implemented2. Taking into consideration the geographical area of the organisation coordinating a HRD project, the distribution modifies slightly (i.e. the number of promoters in Bucharest Ilfov region increases).

Promoter’s areas of activity

There are no centralized data on the promoters’ field of activity. Detailing three of the institutional categories considered in our sample (NGOs, public institutions, and public administration institutions), it was possible however to cast a light on this issue. The examination of the general data on surveyed organisations reveals that HRD program was of interest for a wide variety of institutions, with different areas of activity (Chart 4.2.8).

2 The data made available to research team by MEI on project promoters was structured by RDA monitoring the project and county where the project was implemented (8 Excel files including 42 sheets)



CHART 4.2.8. Distribution of project promoters by type of institution (sample)

0 10 20 30 40 50 60

Consultancy

Association/Foundation

Employer Organisation

Trade union

Universities

Training centre

Schools (preuniversity)

Research institute

County School Inspectorates

Company

Chambers of Commerce

County Employment Agency

Local public administration

Source: Data calculated based on MEI reports and research

In the case of our sample, the ratio of organizations specialized in human resources—which include both NGOs and companies the explicit field of activity is HRD training and consultancy, as well as research institutes, training centres, education institutions, and Chamber of Commerce business schools—amounts to 32%. This percentage is even higher if the category would also include the County Employment Agencies (CEA) and the County School Inspectorates (CSI), public institution important in the development of education and training programmes.

Project size The 416 projects contracted divide into two main categories. A number of 46 projects selected are representing the category of large-scale projects pursuing, as already pointed out, integrated activities, targeted at two counties, at least, and involving a form of extended partnership. The budget of such projects includes a grant amounting from EURO100,000 up to EURO300,000. All these conditions were mandatory in the selection process. Chart 4.2.9 presents project distribution, per total and per sample, of the two project categories.

CHART 4.2.9 Project distribution by size

46

370

13

172

0 50 100 150 200 250 300 350 400

Large-scale projects

Small-scale projectsSampleTotal

Source: MEI and project promoter survey.



4.3. Partners No centralized data are available on partnership in the HRD Component projects. Consequently, the number of partner institutions, their type, and / field of co-operation cannot be appreciated.

Data on these aspects resulted from processing the questionnaire survey data on promoters. Excepting the organisation that have declared not having any partner (more than a quarter in total), the mean partner number per project was 2.2. In the case of large-scale projects, the mean was higher, reaching 3.4 partners per project. When taking into account the project objectives and funding level, we appreciate that partnerships failed the expected level. More than a third of the investigated organisations indicated to officially involved only one partner, as illustrated in Chart 4.3.1.

CHART 4.3.1 Project distribution by number of partners

27%

35%

22%

10% 6% No partnerOne partnerTwo partnersThree to four partnersMore than four partners

The situation is similar in the case of companies, with an average of 2.1 partners per project in the case of small projects and 2.6 partners per project in the case of large projects. Chart 4.3.2. illustrates the partnership structure by main types of promoters and it should be noticed the fact that companies and other public institutions are having the highest number of projects with none or maximum one partner.



CHART 4.3.2 Project distribution by type of promoter organisation and number of partners

The analysis of the main categories of partner institutions having participated in the program reached the same conclusion. Table 4.3.1 displays the percentage of institutions among the total partner organizations.

TABLE 4.3.1. Partner institutions involved in project implementation (in percentages)

Public local administration 44.36

County employment agencies (CEA) 5.04

Chambers of Commerce 2.16

Companies 12.95

School inspectorates 5.28

Research institutes 1.68

School units 6.24

Continuing vocational training centres 3.84

Universities 3.12

Trade unions 1.20

Employers’ organisations 1.44

Non-governmental organisations 6.24

Consultancy firms 3.60

Other 2.88

The analysis of the types of the main partners involved in projects funded in the framework of the HRD programme indicates that local public administration was the preferred partner by project promoters. Public administrations hold a percentage close to the sum total of the rest of institutions, which prompt the conclusion that they played a key role in project implementation. Together with the companies, this type of institution was the main beneficiary of the program, being both a training supplier (particularly to the public servants employed in local administration) and a supplier of assistance to other types of institutions having organized HRD activities.

0% 20% 40% 60% 80% 100%

Companies

NGOs Public institutions


Pre-university education

Higher education

none One two three



Investigation instruments helped the researchers to identify, only in part, project promoters’ motivation, and expectations from their partnerships with the public administration. Among the main reasons for partnership with the local administration, interviewed promoters included:

- Facilities obtained in term of contract and project involvement of persons from target groups;

- Logistics support received aimed at organizing project activities;

- The dissemination, at local level, of project results.

Considering that in over 70% of the investigate projects there existed at least one local administration partner, this involvement is questionable, as it may have been to the detriment of other HRD specialized institutions.

Although ranking as the second type of organization, in terms of frequency, the number of companies taken as partners in the financed project of the program was lower than expected. The category also includes consultancy firms. Only one third of the companies investigated had two or more partners, most of them being local administration institutions or other companies.

A significant number of public institutions (e.g., CEA, CSI) were also partners in the project. This is significant, knowing the important role such institutions play in policy-making and programme development in the field of vocational education and training. What is surprising is the low number of training centres having been partners in HRD projects finance by the program. They represent only 3.8% of the total partner institutions.

TABLE 4.3.2. Distribution of partnership by type of promoter

Promoter type Companies NGOs Public

administration


Higher education

Other public

institutions Total

Companies 19 5 7 4 3 15 56

NGOs 9 10 7 1 2 15 45 Public administration

3 9 15 0 1 13 31


7 2 6 1 0 9 19

Higher education 5 2 2 0 2 6 16 Other public institutions

1 2 2 2 1 6 18

Total 44 30 39 8 9 64 185

Partnerships with public institutions aside, we could notice that, in general, promoters preferred to select institutions of their own type as project partners. Cases in which companies and NGOs or educational institutions and companies have been partners are rarer than expected, if we consider the program objectives.

All investigated promoters (73.5%) appreciate their co-operation with partner institutions as good or excellent. A small number of answers (less than 2%) label the partnership relationship as unsatisfactory. Similarly, less than 3% of the promoting companies are unsatisfied with their partners’ involvement in the project.

The high number of cases (79%), in which promoters declare that they continue the co-operation with all their former project partners, confirms the above. Some promoters (16%)



have ceased co-operation with some of the former partners and less than 5% declared that they had given up collaboration with all their project partners.

Chart 4.3.3. is a synthetic presentation of the co-operation area in the case of the partnerships still active. As shown, the most frequently met fields are development of training programmes in partnership and analysis of continuous training needs. Relatively frequently mentioned are the development of a strategy on human resources; evaluation of the company human resources, and the development of training offers.

CHART 4.3.3. Distribution of active partnerships by area of cooperation

18,4

51,5

59,6

16,2

21,3

7,4

15,4

11

0 10 20 30 40 50 60 70

Developing a HR strategy

Analysis of the needs for continuous training

Training programmes in partnership

Evaluation of human resources

Preparing of training offers

Development of a strategy on HR management

Economic relations

Other situations

The main reasons stated by promoters for ceasing the partnership relations refer to: a negative experience during the project development – e.g., failing to observe one’s tasks, failing to achieve given quality standards, difficulties in communication; changes occurred after project finalization – e.g., partner organization has ceased operation / changed structure / changed location; the absence of new common interests. In other cases the ceased collaboration can be based on the success of a project, which does not need any kind of continuation.

Although the number of social partners is rather low (less than 3% of the total partner institutions), more than one quarter of the investigated promoters declare that, in turn they have also involved social partners in their project activities. However, only 16.2% of the promoters admit having involved (greatly or sufficiently) their social partners; on the other hand, more than half of the promoters declare that they have not involved any social partners in their projects. The number of projects in which no social partners participated is higher, if considering the significant number of no-answers recorded in the case of this item (Chart 4.3.4).

CHART 4.3.4. Involvement of social partners

6% 10%

12%

51%

21% Greatly involved

Sufficiently involved

Small-scale involvement

Not involved

NA



5. Implementation/operations

5.1. Program priorities addressed

This section contains the synthetic presentation of general information about the measures and priorities of projects funded in the framework of the Phare 2000 ESC, HRD Component. We shall try to identify the measures proposed, project spreading by each program objective, and the differences registered in the case of small- and large-scale projects. In addition, the chapter is to present the main categories of project beneficiaries and methods to identify target groups and their needs.

We have failed to receive data on project distribution by priority at programme level. In the case of our sample, labour force training and re-training / reconversion represented the promoters’ priority in the case of around 88% of total implemented projects (Table 4.1.1.).

TABLE 5.1.1. Project distribution by program priority

Total Companies Program priority

Number Percent Number Percent

1. Train and re-train workforce, to help it adjust readily to constantly changing labour market demands

162 87.6 54 96.4

2. Improve active employment measures to use as a systemic instrument to encourage employment

73 39.5 20 35.7

3. Promote social inclusion of socially-disadvantaged categories of people

56 30.3 12 21.4

Taking into account the number of priorities addressed by each project, an interesting image on project distribution can be highlighted. As seen, in almost 60% of the cases, investigated projects focussed on one priority only (Chart 5.1.1.).

CHART 5. 1.1. Distribution of projects by number of priorities addressed

48.1

6.5

16.8

4.91.1

17.81.1

3.8

Projects focussed on priority(1) only

Projects focussed on priority (2) only

Projects focussed on priority (3) only

Projects focussed on priorities (1) and (2)

Projects focussed on priorities (1) and (3) only

Projects focussed on priorities (2) and (3)

Projects focussed on all priorities

NA



We can also note that the number of projects focussed on priority (2) and (3) exclusively amount to less than 10% of the total projects. Although the rate of the projects focussed on all priorities is notable (17.8%), the number of integrated projects is lower than expected in the case of the investigated sample.

Table 5.1.2. presents project Distribution function of priorities pursued and type of promoter. Thus, we notice that projects promoted by the public administration focus on priority (2) in a smaller degree, while the economic agents addressed priority (3) in the smallest. In the first case, an indicator points to the public administration as being insufficiently prepared to participate in actions promoting active measures to encourage employment. The second case is indicative of the relatively low level of social responsibility manifested by Romanian companies, particularly in relation to socially excluded persons.

TABLE 5.1.2. Distribution of priorities by category of promoters (in percentages)

Promoters Priority 1 –

Training

Priority 2 –

Active employment measures

Priority 3 -

Social inclusion

Economic agents 96.4 35.7 21.4 NGOs 80.0 44.4 35.6 Public administration 74.2 19.4 38.7 Pre-university education institutions 89.5 52.6 36.8 Higher education institutions 100.0 37.5 25.0 Other public institutions 88.9 61.1 27.8

5.2. Program measures addressed The distribution is uneven in point of focus on priority measures and different promoting institutions. As shown in Table 5.2.1, the ratio of projects focussed on development, provision, and evaluation of training course programmes in tourism, handicrafts, agricultural food products, etc., aiming at encouraging rural diversity or projects focussed on implementing plans on staff maintenance is significantly lower than that of more general measures focuses on employee training or improved training quality. Differences are more significant in the case of companies, as none of the investigated projects focussed on training programmes and courses designed to encourage rural diversity, although rural development is an important component of all development policies in Romania.



TABLE 5.2.1. Project distribution by target measures. Priority 1 – Training (in percentages)

Target measures Total promoters Companies

Improved quality of continuous vocational training 58.0 53.7

Employee training 58.0 64.8

Manager training and development of managerial competences 35.2 33.3

Training entrepreneurs and future entrepreneurs 18.5 7.4

Modernized vocational training systems and innovation in the field of vocational education and training

13.6 11.1

HDR support in the context of industrial reform for developing and implementing HRD strategies and plans

21.0 27.8

Development, provision, and evaluation of training programmes and courses in tourism, handicrafts, agricultural food products, etc., designed to encourage rural diversity

4.9 0.0

Projects for the implementation of staff maintenance plans 3.7 1.9

Other 4.3 3.7

In the case of projects focussed on priority (2), the measures least focussed on were those related to promotion of entrepreneurship, work placement of the long-term unemployed, and assistance for income-making community activities. This fact strengthens the conclusion about promoters’ insufficient competence and low interest in more complex activities in the program and assistance offered to specific target groups (e.g., the long-term unemployed, local entrepreneurs, etc.). Companies are no exception, since the ratio of their activities focussed on such measures is lower than the general average (Table 5.2.2.).

TABLE 5.2.2. Project Distribution by target measures. Priority 2 – Active employment measures (%)

Target measures Total promoters

Companies

Identifying actual and future, local and regional needs for vocational training and the deficit of competence market demand

45.2 45.0

Development of an evaluation system of individual potential and supervision of vocational development

17.8 10.0

Enhanced communication and information among institutions tasked with vocational training and support with a view to employment

20.5 20.0

Development of specific training programmes for the re-training of the short-term unemployed

26.0 30.0

Supporting the employment of the long-term unemployed via specific training, guidance, and placement schemes

16.4 20.0

Support measures for the unemployed, particularly young, aimed at their integration or re- integration on the labour market through counselling and vocational guidance, job finding, job clubs, job placement, etc.

32.9 20.0

Vocational training for the unemployed, particularly young, through training and re-training programmes

38.4 45.0

Consultancy on setting up and developing income-making community activities through, for instance, revival of local/regional crafts

16.4 10.0

Vocational training and guidance in services related to the business environment aimed at identifying potential entrepreneurs and thus, promoting entrepreneurship and creating new job

12.3 10.0

Other 1.4 0.0



As in the case of the other priorities, with priority 3, we notice a low degree of interest in measures aimed at a well-defined target group. In this case, measures selected are more general ones related to training, counselling, guidance, and information of the socially excluded or running the risk of social exclusion (i.e. Development and provision of training programmes adjusted to the specific needs of trainees). It is striking that no company in our sample focussed their project on some measures (i.e. employment of the persons with a handicap, training the trainers), or studies on the training needs of the socially excluded groups. A possible explanation is related to the uneven distribution in complexity among measures but also to in the need for highly qualified trainers that could develop and teach training programs from these fields. The individual interviews offered additional information of the justification of avoiding selecting measures directly targeted to socially disadvantaged population: low motivation of trainees to attend training courses, lack of prior experience and basic competences for participation to training activities (literacy and arithmetic skills, communication etc.). In specific cases difficulties were mentioned in training people with little/no experience in following a strict schedule, dressing properly or following basic rules of personal hygiene.

TABLE 5.2.3. Project Distribution by target measures. Priority 3 – Social inclusion (in percentages)

Target measures Total promoters

Companies

Studies to identify and evaluate the needs of the socially excluded, in particular for labour market access

17.9 0.0

Education in compensation and apprenticeship schemes for young persons having dropped out of the system of education without a basic qualification, focussed on Rroma ethnics and foster home residents

14.3 8.3

Training the trainers aimed at programme developing and adjustment to target groups needs designed to increase social integration

10.7 0.0

Creation of counselling, guidance, and information services with a view to increasing social integration

28.6 16.7

Development and provision of training programmes adjusted to the specific needs of trainees

48.2 50.0

Consultancy on establishing and developing profit-making community activities and support for entrepreneurship among the socially-disfavoured groups

19.6 8.3

Support offered to employer initiatives in the sector of protected employment for the benefit of the handicapped

7.1 0.0

Placement of the socially disadvantaged in companies or NGOs for on-the-job training

17.9 25.0

Other 17.9 16.7

In the case of projects investigated, almost 90% of them have organized training activities. The number of projects promoted by companies and that did not include training programmes is relatively higher than that of the rest of promoters (Table 5.2.4). In the chapter on output of the projects a more detailed analysis on training activities will be presented.



TABLE 5.2.4. Projects distribution by type of promoter organization and training programme (%)

Company Training programmes 83.9

Other services 16.1

Other providers Training programmes 90.7

Other services 9.3

Total Training programmes 88.6

Other services 11.4

The analysis of measures definition and formulation, in the light of research outcomes, highlights the fact that for future similar measures a more clear division between objectives is needed, with a special focus on defining specific measures for each objective. Moreover, for an increased accountability in evaluation, it is needed to identify a core of detailed priority measures and to formulate more broadly directions for action.

5.3. Problems in initiation and implementation Other issues mentioned by project promoters during field interviews were related to the following issues: difficulties in cooperation with project partners (i.e. formal involvement, underachievement of objectives etc.); difficulties in relation with project beneficiaries (i.e. low motivation of disadvantaged persons to attend training activities); financial (i.e. losses due to exchange rates fluctuations, slow procedures related to obtaining the official approval of budget revision and other decisions of the Treasury, Ministry of European Integration and Regional Development Agencies); changes in legislation (i.e. accreditation of training providers, fiscal code etc.), acquisitions (i.e. difficulties in purchasing equipments with EU origin certificate). Another important difficulty to which the project promoters referred to was securing their own contribution, issue mentioned especially by public institutions representatives.

CHART 5.3.1. Main difficulties in project initiation and implementation indicated by project promoters

6%29%

26%

25%

8%

6%

Delays in securing financing

Difficulties in continuing after Phare financingceasedDifficulties in recruiting target groups

Difficulties related to the existing procedures

Difficult acces to sources of information

Difficulties in communication with RDA/MEI



6. Costs

6.1. Expenditure on equipment Information on the budget allocated to equipment acquisition has been available for 174 out of the 185 projects evaluated based on the questionnaire survey on promoters (94%). In the rest of the cases, according to data supplied, no equipment has been purchased. In the case of the projects having purchased equipment, the costs amounted to around 28% of the total project budgets (Table 6..1.1.).

Differences in terms of the rate of expenditure on equipment acquisition occur among the development regions. The highest cost rate, of the regional budget size, occurs in the North-West (32%), South-East and North-East regions (about 30%); the lowest, occurs in the Bucharest-Ilfov (21%) and Centre (20%) regions. The differences in distribution by region for the share of funds allocated to equipment acquisition can be explained, partly, by taking into account the differences between promoters organisations related to these types of resources existing at the beginning of the programme.

TABLE 6.1.1. Rate of expenditure on equipment acquisition, by region of development

Expenditure with equipment Development region Total budget Absolute value Percent of total budget

North-East 1,552,530 459,932 29.6

South-East 2,232,637 689,427 30.9

South Muntenia 115,6861 295,466 25.5

South-West Oltenia 924,876 239,770 25.9

West 942,582 236,617 25.1

North-West 2,312,008 743,631 32.2

Centre 452,428 89,540 19.8

Bucharest-Ilfov 1,340,889 284,156 21.2

Total 10,914,810 3,038,539 27.8

Purchase of new equipment is a measure related to those aimed at promoter institutional capacity building analysed in the present report, namely: creation of new training centres, development of training offers (new training programmes, new target population groups); development of the initial training capacity (new classrooms, new training equipment). In the case of projects having supposed the application of more such measures aimed at institutional capacity building, the share allocated to the purchase of equipment from project budgets was higher. As shown in Table 6.1.2., in the case of almost 59% of the projects having implemented two such measures, and of almost 67% of those having implemented three capacity building measures, respectively, the share exceeded 25%. According to promoter statements, about 51% of the projects having implemented only one measure – equipment purchasing – attained a similar level in point of funds allocated to the measure in question.



TABLE 6.1.2. Amount of the project budget allocated to equipment purchasing by number of measures related to institutional capacity building

Number of institutional capacity building measures implemented Budget percentage

allocated to equipment purchasing 1 measure

addressed 2 measures addressed

3 measures addressed

Total

Less than 10% 12.2 12.2 13.3 12.3 10-25% 36.6 29.3 20.0 32.6 25-50% 50.0 56.1 40.0 50.7 Over 50% 1.2 2.4 26.7 4.3 Total 100.0 100.0 100.0 100.0

Note: Data refer to a number of 138 projects of the evaluated 185, namely, those having developed training activities and allocated a given budget to equipment purchasing.

When considering other analysis criteria – type of project and type of promoter – the highest level of budget allocation designed for equipment acquisition occurs in the case of small-scale projects – 34.5%, and in that of companies – 30% (Tables 6.1.3. and 6.1.4). In the second case, it is possible for the promoters having represented companies to have born in mind, ever since the project launch, the development of the infrastructure of their own organization.

TABLE 6.1.3. Rate of expenditure on equipment acquisition by types of projects

Expenditure with equipment Type of project Total budget

Absolute value Percent of total budget

Large-scale projects 7,645,041 1,911,921 25.0

Small-scale projects 3,269,769 1,126,618 34.5

Total 10,914,810 3,038,539 27.8

TABLE 6.1.4. Rate of expenditure on equipment acquisition, by types of promoters

Expenditure with equipment Type of promoter Total budget

Absolute value Percent of total budget

Companies 3,672,435 1,102,828 30.0

Other organizations 7,242,375 1,935,711 26.7

Total 10,914,810 3,038,539 27.8

Such a hypothesis corroborates with other data presented in the report. According to the data, at present, a higher rate of former promoting companies than other organizations use the equipment for training purposes and for production / specific activity, as well – about 39%, as against 24%.

A similar interpretation is valid for the data indicating a slightly lower rate, as compared to the representatives of other organizations (55% as opposed to 58%), for the companies which currently use the purchased equipment at maximum capacity for training activities (Table 6.1.5.).



TABLE 6.1.5. Capacity at which the equipment is used

At maximum capacity

At 75% or less capacity

Promoter is no longer in

possession of the equipment

No answer Type of promoter

Number Percent Number Percent Number Percent Number Percent

Companies 30 55.6 18 33.3 2 3.7 4 7.4

Other organizations

70 58.3 41 34.2 3 2.5 6 5.0

Total 100 57.5 59 33.9 5 2.9 10 5.7

The surveyed project promoters mentioned a series of reasons to motivate their not using the equipment purchased through the project at full capacity for training activities, such as:

- lack of funds needed for supply acquisition; - lack of funds needed for equipment maintenance; - decrease of the number of participants in the training courses as a consequence

of the decreasing demand for the concerned training offer; - decreased demand for the concerned training offer, because the potential

beneficiaries had difficulties in supporting training costs; - the changed content of the training offer; - current training space is no longer compatible; - equipment handover to promoter staff, to be used in specific activities; - the delayed authorization of the training centre caused the break of the training

activity.

Of the reasons mentioned, the most frequent are those referring to the potential beneficiaries’ difficulty in supporting the training cost and the decreased demand for the training offer developed through the project.

The detailed analysis of the information on the equipment reveals that the highest expenditures, in order of their rate, were made for the acquisition of IT equipment - 49%, furniture – 14, 4% and production equipment – around 9%.

TABLE 6.1.6. Expenditures on equipment acquisition, by region of development

Of which, by type of equipment and endowment (%) Development region

Project total

budget

Budget allocated

to equipment acquisition

Furniture Training equipment

IT equipment

Teaching support

materials

Equipment simulating

work processes

Production equipment Other

North-East 1471464 26,3 9,8 1,2 67,3 2,0 0,1 5,2 14,3 South-East 2195824 27,0 5,7 0,7 41,6 2,0 4,3 8,8 37,1 South Muntenia

1067336 25,5 9,8 1,1 55,4 4,7 0,3 15,3 13,5

South-West Oltenia

874523 26,2 5,8 2,0 63,2 0,0 7,8 3,5 17,6

West 722708 24,2 7,2 1,5 72,6 3,0 0,0 4,5 11,3 North-West1 2079541 32,6 34,3 0,8 32,2 21,7 0,6 6,2 4,1 Centre 452428 19,5 11,0 0,5 61,3 3,7 0,0 13,4 10,0 Bucharest-Ilfov

1120382 20,9 7,3 0,6 44,1 1,5 3,9 28,7 13,8

Total 9984205 26,6 14,4 1,0 49,1 7,2 2,2 9,4 16,6

1 The atypical distribution of the budget for equipment in the case of North West region compared with the other regions is entailed by two large projects implemented within this region, with a corresponding total budget of Euro 471,000 and Euro 499,000. The first project has allocated for furniture around 20% of the total budget while the second allocated 24%. By excluding these projects from our analysis the budget distribution for North West region becomes similar with the other ones, namely: furniture – 4,6%, training equipments – 1,7%, IT equipment – 64,2%, teaching support materials– 3,1%, equipment simulating work processes – 1,4%, production equipment– 14,9%, others – 9,9%.



Differences on the respective expenditures relate to such criteria as project size and type of promoter. Therefore, within large projects, the highest expenditures were allocated for IT equipments (almost two thirds of the total equipment budget) and for production equipment (10,4%) while in case of small projects, besides IT equipments the main categories of acquisitions were related to furniture and teaching support materials (Table 6.1.7.). The companies purchased, as expected, more production equipments compared to other categories of project promoters – almost 20% compared with 4% (Table 6.1.8.).

TABLE 6.1.7. Distribution of equipment acquisition expenditure by types of projects

Of which, by type of equipment and endowment (%)

Project

total budget

Budget allocated

for equipment acquisition


IT equipment

Teaching support

materials


work processes


Large-scale projects

6714436 24,1 8,2 1,4 61,7 2,8 3,1 10,4 12,4

Small-scale projects

3269769 31,7 24,2 0,2 29,6 14,1 0,8 7,9 23,2

Total 9984205 26,6 14,4 1,0 49,1 7,2 2,2 9,4 16,6

TABLE 6.1.8. Distribution of equipment acquisition expenditure by types of promoters

Of which, by type of equipment and endowment (%)

Project

total budget

Budget allocated

for equipment acquisition


IT equipment

Teaching support

materials


work processes


Companies 3424063 27,2 5,4 1,0 44,9 1,7 3,3 19,5 24,2 Other organizations

6560142 26,3 19,3 1,0 51,5 10,2 1,6 4,0 12,5

Total 9984205 26,6 14,4 1,0 49,1 7,2 2,2 9,4 16,6

It can be noticed that the hypothesis concerning the intention of companies to develop their institutional capacity by production equipment acquisition is confirmed by the data on:

- share of total budget allocated to equipments,

- use of equipment purchased for other purposes than training

- the degree in which the equipment purchased is used for training activities.

- share of production equipments in the total budget allocated to all types of equipments purchased.

The data on type of production equipment purchased could have supported further this hypothesis. However, the quantity and the quality of the data provided by project promoters on this specific issue made impossible to verify the hypothesis from this perspective. Concretely, the promoters provided information on the total cost of production equipment purchased, but failed to specify the number and the type of the equipment. Taking that fact



into consideration, the Table 6.1.9. presents data only on other types of equipment than production.

Based on information supplied by questionnaire-surveyed promoters, it was possible to collect data on equipments, furniture, etc. purchased - Table 6.1.9. However, due to scarce information on this issue, the table does not include data on production equipments nor on equipment used for simulating production processes.

TABLE 6.1.9. Equipment and endowments purchased

Equipment and endowments Number

Study desks 766

Work desks 375

Office chairs 2037

Filing cabinets 226

Computer desks 229

Glass or wooden blackboard 11

Magnetic blackboard 20

Projection screen 26

Flipchart 35

Maps, illustrative charts 108

Computers 788

Server 31

Printers 212

Scanner 27

Video projector 87

Slide projector 20

Copier 63

Software 280

INTERNET equipment 11

Telephone / fax 11

Car 10

Heating installation / interior 2

Note: Data supplied by 153 projects having supplied the information.

6.2. Training costs Among the important indicators used in evaluating the results, the efficiency evaluation of the HRD Component projects, particularly of those having developed training activities, are those indicators that refer to training costs. Of these, the survey has used:

- The average cost of training courses per participant; - The average cost per hour of the training courses.

Since survey data did not allow for estimation of some expenditure, such as those with the training space (rent, electricity and water consumption, etc.), equipment exploitation, supplies, etc., the research calculated the values of the indicators applied by taking into account the following expenditures only:

- Expenditures with trainer remuneration (for their training activity proper); - Expenditures with payment of the training modules prepared.



According to this calculation method, the average cost of the training courses per participant amounted to EURO 210. The average cost was lower in the case of training courses organized by companies - 203 EURO - and slightly higher in the case of courses organized per projects developed by other organizations - 212 EURO. Significant differences (one to five rations) are registered among development regions (Table 6.2.1.).

Table 6.2.1. Average cost of the training courses per participant and average cost per

hour of the training courses, by development regions and categories of promoters

Development Region

Average cost of the training courses per participant Average cost per hour of the training courses

Companies Other organisations Total Companies Other

organisations Total

NE 86 173 164 2,56 1,24 1,31 SE 417 311 345 0,43 0,82 0,67 S 186 267 258 0,90 0,39 0,41 SW 151 272 235 1,18 2,44 2,29 W 98 168 141 0,35 0,82 0,68 NW 209 148 167 3,14 0,87 1,34 Centre 103 210 198 0,05 1,12 0,79 Bucharest Ilfov 161 124 145 1,10 0,66 0,98

Average value 203 212 210 1,05 1,01 1,02

Given that participants in training courses benefited from courses with various objectives (training, re-training, refresher courses, etc.) – a situation involving different course duration – a more relevant indicator in efficiency evaluation is the average cost per hour of the training courses. In this case, the calculated indicator value amounted to EURO 1, 2. The cost was higher in the case of training courses organized by companies – EURO 1.05 compared to EURO 1, 01. Again high differences among development regions are registered.

Considering the average cost per hour of the training courses, calculated based on information supplied by promoters of HRD Component projects, we could estimate the average costs of training courses attended in pursuit of different qualification levels (types of courses also developed in the framework of some of the projects evaluated):

• Level of qualification I – 360 hours: EURO 367,2; • Level of qualification II – 720 hours: EURO 864; • Level of qualification III – 1,080 hours: EURO 1296.

The value of the first indicator presented - the average cost of training courses per participant - above differs significantly from the value presented within the National Institute for Statistics survey in 2001 (with data for 1999)2: 1,197 thousand ROL (around EURO 75, at 1999 exchange rate, compared with 210 EURO in the case of projects organised within the projects). More than this, one must bear in mind that NSI survey indicators considered costs for: course fees, trainer remuneration (internal or external trainers), transport, daily allowance, and accommodation for course participants, training space expenditures (rent, utilities, etc.), supplies, etc. On the other hand, it should be noticed the high difference between the average length of courses presented by INS survey and the one specific to HRD program: 73 hours/course (INS) compared with 190 hours/course (HRD component). Concerning the second indicator - the average cost per hour of a training course – the value is relatively similar for both cases: 16 thousand ROL / EURO 1, 01 (INS) and EURO 1, 2 (HRD Program).

2 NATIONAL INSTITUTE FOR STATISTICS. Caracteristici ale formării profesionale continue în România [Characteristics of continuous vocational training in Romania]. Bucharest, 2001.



7. Output

7.1. Number of successful and unsuccessful projects The final evaluation reports prepared by RDA indicate 22 projects with terminated contracts due to failure to finalize. These projects hold a rather low ratio, 5.3%, in such a wide program. Table 7.1.1. presents the relatively balanced distribution of the terminated projects by region. The South-West region registers the highest rate (9.4%) of terminated projects, as opposed to the West region (2.12%).

TABLE 7.1.1. Total projects and cancelled projects (in numbers)

NE SE S SW W NW C BI Total

Total projects 62 57 66 32 47 91 44 17 416

Cancelled projects 3 3 4 3 1 6 2 1 23

Cancelling rate (%) 4,8 5,3 6,1 9,4 2,1 6,6 4,5 5,9 5,5

Source: RDA final evaluation reports.

The companies are the main institutions having failed to finalize their projects, with a ratio of 72.2% of the total institutions with cancelled contracts. As indicated by RDA reports and on site research, in general, contracts terminated at the request of those promoters who could not cover the individual financial contribution. Other indicated causes are management or partner co-operation difficulties. In case of RDA-initiated cancellations, the main reasons invoked refer to failure to organize planned activities inside the eligible period or problems related to project budget execution (Chart 7.1.1)

CHART 7.1.1. Distribution of projects by type of organisation terminating the contract

56%22%

22% At promoter request

Proposed by RDA andMEINA

Source: Calculations based on RDA final evaluation reports.



7.2. Funds absorption capacity In the case of total number of projects of HRD program, the difference between project available funds and the budget execution indicates an average capacity for fund absorption both at the level of all regions of development (over 85% of the Phare source / national budget) and at the level of each region. The highest differences in point are between the South-East region, with the lowest absorption capacity (around 82%) and the North-West region, with the highest (89%) - Table 7.2.1.

Tabel 7.2.1. Absorption capacity of grants (from Phare and national budget sources),

by all projects from HRD program, by regions

Total Large-scale projects Small-scale projects Grant Grant

Spent %

Spent Grant Grant Spent

% Spent Grant Grant

Spent %

Spent North-East 3074563 2600210 84,6 1440874 1180549 81,9 1633689 1419660 86,9 South-East 4130830 3369516 81,6 2907567 2369866 81,5 1223264 999650 81,7 South Muntenia 3631489 3167940 87,2 1621349 1420290 87,6 2010140 1747649 86,9

South-West Oltenia 897624 765059 85,2 0 0 897624 765059 85,2

West 1593340 1383773 86,9 164860 136719 82,9 1428480 1247054 87,3 North-West 4107215 3650102 88,9 1482418 1273682 85,9 2624797 2376420 90,5 Centre 1394516 1158582 83,1 297849 238279 80,0 1096667 920303 83,9 Bucharest-Ilfov 1093179 956139 87,5 708907 614939 86,7 384272 341200 88,8 Total 19922756 17051321 85,6 8623824 7234324 83,9 11298933 9816996 86,9

Obs. : Data referring all 393 finalised projects. Source: Data calculated from MEI information.

By project size, it can be observed a higher level of funds absorption in the case of small projects – 87% compared with 84% in the case of large projects.

As can be observed in the table 7.2.2. and 7.2.3., the findings concerning the indicator analysed are similar in the case of our sample. This referrers to the differences among regions related to the total amount of expenditures. At the same time, differences can be noticed in the case of average size of budget spent by project.

TABLE 7.2.2. Budget execution by region of development

Funds spent Average budget / project Funding source

Development region Companies Other

organizations Total Companies

Other organizations Total

PHARE / national budget

North-East 136666 849204 985870 45555 28307 29875

South-East 649217 968838 1618055 81152 50991 59928

South Muntenia 242191 478728 720919 34599 28160 30038


West 240147 359077 599224 24015 29923 27237

North-West 327226 973891 1301116 29748 51257 43371

Centre 85484 243896 329381 28495 20325 21959

Bucharest-Ilfov 333856 440273 774129 41732 55034 48383

Total 2208347 4612500 6820847 39435 35756 36869 Own contribution North-East 60466 385856 446323 20155 12862 13525

South-East 259644 281684 541328 32455 14825 20049

South 133016 162042 295058 19002 9532 12294



Muntenia


West 119782 181069 300851 11978 15089 13675

North-West 120253 514057 634310 10932 27056 21144

Centre 30477 104243 134719 10159 8687 8981

Bucharest-Ilfov 198148 171748 369896 24768 21468 23118

Total 1049667 1991090 3040758 18744 15435 16437

TOTAL North-East 197132 1235061 1432192 65711 41169 43400

South-East 908861 1250522 2159383 113608 65817 79977

South Muntenia 375207 640770 1015977 53601 37692 42332


West 359929 540146 900075 35993 45012 40913

North-West 447479 1487948 1935426 40680 78313 64514

Centre 115961 348139 464100 38654 29012 30940

Bucharest-Ilfov 532004 612020 1144025 66501 76503 71502

Total 3258015 6603590 9861605 58179 51191 53306

Source: Calculations based on information available from the MEI.

TABLE 7.2.3. Absorption capacity, by regions of development Expenditure from Phare sources / national budget (%)

Region of development Companies Other organizations Total

North-East 95,5 87,5 88,6 South-East 78,4 88,1 83,9 South Muntenia 91,3 83,0 85,6 South-West Oltenia 89,1 81,2 84,1 West 91,7 85,3 87,8 North-West 92,1 82,4 84,6 Centre 77,8 89,6 86,2 Bucharest-Ilfov 89,1 82,4 85,2 Total 86,4 85,0 85,5

In terms of type of promoting organization, companies register a higher absorption capacity (except for the South-East and Centre regions) as compared to other promoters. The finding is applicable particularly to projects developed by companies in the North-East (95.5% from total external funding, as opposed to 87,5%), North-West (92% as against 82%), and in South and West regions. Taking the total projects, the difference is insignificant (86.4% as against 85%).

In general, the projects in the HRD component demonstrated an average capacity of fund absorption. In point of regions of development (taking into account all projects finalised within HRD program), projects developed in North-West and Bucharest-Ilfov have the highest indicator value (higher than the average); in point of types of organization (in the sample evaluated), the highest indicator value is that of projects developed by companies. A more careful monitoring, and assistance granted all along project development aimed at increasing project absorption capacity.



7.3. Creation of different types of institutional capacity building With regard to the sample included in the evaluation, one of the targets was the degree at which the HRD Component projects have supported the promoters’ institutional building (establishment of new training centres included) i.e., of those having carried out training activities. A high percentage (88.6%) of the 164 projects having organized training courses has aimed at institutional building under various forms:

• Infrastructure and/or equipment endowment; • Development of the human resources (developing a pool of trainers); • Enlargement of the training offer.

In the first case, we can speak of either the setting up of new training centres, the case of over 30% of the projects, or the development of the training capacity, as against their initial capacity at project launch time (new classrooms, new training equipment) – about 49% of the projects.

A smaller rate (below 21%) of the evaluated projects pursued the training the trainers objective. On the other hand, about 60% of the projects achieved the development of the training offer (new training programmes, new target groups (Table 7.3.1.).

The data reveals the fact that developing new training offers represents the scheme applied by most of the promoted projects having developed training programmes. This relates to the statement of over 55% of the promoters that offering training courses has supposed the adjustment of earlier offers to the specific needs of the target groups or the development of a new offer.

TABLE 7.3.1. Project measures aimed at institutional capacity building

Total Companies Other organizations

Building institutional capacity measures Numbers Percent Numbers Percent Numbers Percent

1 Setting up a new training centre 50 30.5 16 34.0 34 29.1

2 Enhanced training offers (new training programmes, new target population groups)

97 59.1 21 44.7 76 65.0

3 Developing the training capacity (new classrooms, new training equipment)

80 48.8 23 48.9 57 48.7

4 Training trainers 34 20.7 7 14.9 27 23.1

164 47 117

Almost half of the projects (47.6%) focussed on only one of the four measures aimed at building institutional capacity. However, a large percentage (almost 16%) has implemented three or four of the measures (Table 7.3.2.). The implementation of one or more measures aimed at institutional capacity building relates to a series of factors, among which:

• capacity of concerned organization at project launch time in terms of infrastructure, equipment endowment, and human resources (trainers);

• prior experience in training activities; • available training offer;



• type of training programme the project advances; • target population groups, etc.

TABLE 7.3.2. Number and type of measures applied aiming at institutional capacity building

Total Companies Other organizations Number and type of measure Numbers Percent Numbers Percent Numbers Percent

1 19 11.6 11 23.4 8 6.8

2 37 22.6 9 19.1 28 23.9

3 18 11.0 9 19.1 9 7.7

4 4 2.4 0 0.0 4 3.4

Total projects having applied one measure

78 47.6 29 61.6 49 41.8

1; 2 5 3.0 1 2.1 4 3.4

1; 3 4 2.4 1 2.1 3 2.6

1; 4 3 1.8 0 0.0 3 2.6

2; 3 27 16.5 6 12.8 21 17.9

2; 4 4 2.4 1 2.1 3 2.6

3; 4 6 3.7 2 4.3 4 3.4

Total projects having applied two measures

69 29.8 11 23.4 38 32.5

1; 2; 3 9 5.5 1 2.1 8 6.8

1; 2; 4 1 0.6 0 0.0 1 0.9

1; 3; 4 2 1.2 1 2.1 1 0.9

2; 3; 4 8 4.9 2 4.3 6 5.1

Total projects having applied three measures

20 12.2 4 8.5 16 13.7

Total projects having applied four measures (1, 2, 3, 4)

6 3.7 1 2.1 5 4.3

No answer 11 6.7 2 4.3 9 7.7

164 47 117

Note: Figures 1 through 4 in the first column represent the four institutional building measures.

The analysis of the different types of measures aimed at institution capacity building reveals differences among the promoters. Thus, companies have been more interested than the other organizations in creating their own training centres (34% as against 29%); the other organizations saw as priorities the measures concerning the enhancement of their training offer (65% as against the companies’ 45%) or the training of trainers (23%, and 15%, respectively (Table 7.3.1., Chart 7.3.1.).



CHART 7.3.1. Project measures for institutional capacity building

010203040506070

new trainingcentres

new trainingoffers

trainingcapacity

trainingtrainers

companiesother organizations

The analysis of the measures aimed at institutional capacity building by detailed types of the promoters also reveals a series of differences among the latter. Of all organizations having created new training centres, most are companies (32%), followed closely by higher education institutions (16%), NGOs and public institutions (14%), institutions of the public administration and pre-university education units (12%) (Table 7.3.3).

Although holding a lower percentage among the organizations having created new training centres, the tendency is remarkable for pre-university education units towards developing their institutional capacity and expanding their training offer by getting involved in the lifelong learning process, in the general process of training human resources.

The involvement of the NGOs in the projects, that is, in the creation of training centres renders evident the decentralization of the vocational training system through the diversification of the training institution and of the offer in the field of continuous vocational training. Actually, the same type of organizations have applied the measure involving the development of the training offer has been applied to the greatest extent (33%) Table 7.3.3).

TABLE 7.3.3. Measures aimed at institution capacity building, by type of promoting organization

Type of applied measures 1 2 3 4 Type of promoting

organization Number Percent Number Percent Number Percent Number Percent

Economic agents (companies)

16 32.0 21 21.6 23 28.8 7 20.6

NGOs 7 14.0 32 33.0 12 15.0 4 11.8

Public institutions 7 14.0 7 7.2 10 12.5 5 14.7 Public administration

6 12.0 14 14.4 14 17.5 9 26.5

Pre-university education units

6 12.0 12 12.4 12 15.0 6 17.6

Higher education institutions

8 16.0 11 11.3 9 11.3 3 8.8

Total 50 100.0 97 100.0 80 100.0 34 100.0



In general, based on the data resulted from the investigation, we can appreciate that the organizations having tried to develop (as compared to other types of organizations considered by the analysis) the institutional capacity (new training centre, new training premises and training facilities and equipment) are the companies. This is a sign of employers’ increased interest in developing their own human resources and of their awareness of the need for developing and updating their competences, a process facilitated by the establishment of their own training centre. Actually, the analysis reveals that the projects promoted by companies and aimed at creating a training centre focussed on employees. Of the 16 training centres created by companies, 15 training centres developed training activities for employed persons only (internal or external employees), or for employed persons and other target groups (entrepreneurs / potential entrepreneurs, unemployed, and disfavoured persons); only one promoter developed training programmes focussed on other groups than its own employees.

Particularly the creation of new training centres, represented a key target of the projects developed in the framework of the PHARE 2000 ESC Program HRD Component. According to data provided by MEI, for all the projects developed in the framework of HRD component, at the beginning moment, the estimated number of new training centres to be developed was 147. Of these, 131 new centres have opened, that is, 33.2% from among the total implemented projects, a very high rate, considering the extent of the pace and its expected outcome1. An analysis by region of development reveals a disparity in the distribution of the number of new training centres. The NW and NE regions register the highest rate, 40.5%, and 16.8 percent, respectively; the lowest rate turns up in Bucureşti Ilfov and SE region (below 1%, and 3.8%, respectively). At a first glance, the disparity justifies, given the different number of projects developed by each region of development, a fact determined in turn by a series of factors, among which:

- The different level of regional socio-economic development; - The different level of funding available in one region or another; - The different power of fund absorption; - The characteristics of the trainer market (number of trainers, diversity of offers, quality

of trainers, etc.); - The training degree of the different target population groups, at regional level, etc.

However, we should mention the high differences in percentages among the regions, in terms of projects developed and of the newly set up training centres, respectively. The highest differences lie with the NW, NE, and W, on the one hand – in the case of which the rate of developed projects is about 22%, 15%, and 12%, respectively, of the total; for the new training centres, the percentages are 40%, 17%, and 14%, respectively of the total. On the other hand, for the SE and Bucharest Ilfov regions, the corresponding project percentages are 14%, and 4%, respectively; the training centres percentages indicate 4%, and 1%, respectively. Thus, the figures indicate that in the case of the first regions, the ratio of the newly set up training centres is higher than the ratio of the regionally developed projects; in

1 The percentage corresponds to the number of 393 developed projects, and leaves out the 23 cancelled projects.



the case of the second set of regions, the ratio of the new training centres is three to four times lower that that of developed projects (Table 7.3.4.).

TABLE 7.3.4.. Distribution of the new training centres, by region of development

Total projects implemented

New training centres (in numbers)

Created Development regions Number Percent Estimated Number Percent of total

training centres Percent of

total projects DR 1 North-East 59 15.0 37 22 16.8 37.3

DR 2 South-East 54 13.7 5 5 3.8 9.3

DR 3 South Muntenia 62 15.8 10 10 7.6 16.1

DR 4 South-West 29 7.4 13 13 9.9 44.8

DR 5 West 46 11.7 17 18 13.7 39.1

DR 6 North-West 85 21.6 53 53 40.5 62.4

DR 7 Centre 42 10.7 10 9 6.9 21.4

DR 8 Bucharest Ilfov 16 4.1 2 1 0.8 6.2

Total 393 100.0 147 131 100.0

Source: Calculations based on data supplied by MEI.

The disparity becomes more obvious if studying the figures pointing to the number of projects that have foreseen the creation of new training centres out of the total projects developed at each region level: NW –62%; W – 39%, and NE – 37%. At the opposite end, SE – 9%, and Bucharest Ilfov – 6% (Table 7.3.4.). The differences observed between development regions concerning the number of new training centres created through HRD program could be explained by the significant differences between the existing training markets and also by other criteria and dimensions presented above.

010203040506070

North-East

South-East

South South-West

West North-West

CentreBucharest

Chart 7.3.2. Number of projects that created training centres, by region

At the sample level, as we presented above, it can be noticed that the main organisations that developed new training centres was companies. At the same time,, it is interesting that the creation of a training centre by companies – a measure which, theoretically, at least, supposes a higher level of financing, given the costs of space rehabilitation / rental, equipment endowment, etc. – has been a measure to have been implemented (87.5%) particularly in the framework of the small scale projects, with reduced financing (Table 7.3.5.). That signifies both a higher capacity to use the funds more efficiently as well more



possibilities to support such measures with individual resources. At the same time, it has to be taken into account the unbalanced distribution of large and small projects, the focus on more general objectives than creating training centres in the case of large projects, the existing training centres before the program in the case of large project promoters and, last but not list, the fact that only around half of the total number of promoters has replied to our survey.

TABLE 7.3.5. Companies that created training centres, by project scale

Project type Numbers Percentage

Small scale project 14 87.5

Large scale project 2 12.5

Companies that have established training centres possess a series of characteristics:

- their available capital is entirely private – 81.3; - they are small-sized enterprises (the personnel number is below 50) – 62.5%; - they have been operating more than 10 years – 62.5%;

Chart 7.3.3. Distribution of companies that created training centres by type of ownership

81%

19%

Entirely private ow nership Mixt ownership

Chart 7.3.4. Distribution of companies that created training centre, by size

62%13%

25%

Small companies Medium size companies

Large companies

Chart 7.3.5. Distribution of companies that created training centre, by operating history

62%13%

25%

More than 10 years 6-10 years Under 5 years

The data presented can confirm, to some extent, the results of other studies on HRD and

company participation in the process. Such studies2 indicated that an actively supported HRD policy is generally identifiable in private companies with a longer period of operation (over 5 years). They are more mobile, flexible, and full of initiative in this respect. The results of the

2 NATIONAL ROMANIAN OBSERVATORY. Skills Audit Survey, Bucharest, 2003.



survey are also indicating that small size companies tend to build their own training capacities.

7.4. Characteristics of beneficiaries The distribution of the direct beneficiaries, by type of assistance received3 (Chart 7.4.1.) indicates the fact that the category of employees had benefited the most from the programme, accounting for 50% of the total target-categories.

CHART 7.4.1. Distribution of the main direct beneficiaries, by category

16069

2785

6582

6278

0 2000 4000 6000 8000 10000 12000 14000 16000 18000

Employees

Entrepreneurs

Unemployed

Socially disfavoured

However, the rate of the entrepreneurs (actual or potential), the unemployed, and the socially disadvantaged could be judged as high, compared to the rate of projects targeted on measures specific to priority 2 and 3. The explanation relates to the accidental inclusion of these categories in the activities of the project (not being strategically targeted by project promoters) and, as already pointed out in the previous chapter, to the way the project priorities and measures have been formulated.

Chart 7.4.2. presents the distribution of the main categories of direct beneficiaries by the type of measures. Employees represented the main category having to a high degree benefiting from training activities (63%) as opposed to entrepreneurs (4.4%). We could also notice that employees hold also the highest rate among the total beneficiaries of consulting activities, while the socially disadvantaged category hold the highest rate in point of counselling and guidance. As expected, job seekers benefited most of all of mediation services, while the employees assisted to find a job were primarily those in danger to become redundant.




CHART 7.4.2. Number of direct beneficiaries, by categories and type of measures

0 5000 10000 15000

training

guidance andcounselling

conultancy

mediation forjob placement

disadvantaged categoriesunemployedentrepreneursemployees

The distribution of project measures by categories of direct beneficiaries highlights the fact that the majority of project promoters has preferred training activities, these activities accounting for around two thirds of the total measures taken into account. Around 10% of the total number of direct beneficiaries have been involved in more than one measure.

TABLE 7.4.1. Distribution of measures by category of direct beneficiaries (companies)

Companies Training Guidance and counselling

Consultancy Mediation for job

placement Total*

Employees 5257 166 1098 15 6406 Current/potential entrepreneurs 123 45 364 0 487

Unemployed 1547 313 9 10 1753 Disadvantaged categories 419 653 120 8 1180

Total 7346 1177 1591 33 9826 * Calculated taking into account the number of persons benefiting from more than one measure

In the case of projects coordinated by companies, we should mention that, in addition to their employees, they also assisted job seekers, particularly in the case of projects involving training activities. Again, we notice the low rate of the socially disadvantaged having benefited from some assistance programmes (e.g., training courses, consultancy) – Table 7.4.1.

According to the data provided by the Ministry of European Integration the originally estimated number of trained people within the PHARE HRD 2000 Programme is 47,353. The same source shows that only 89.6% of this number was achieved.



Table 7.4.2. Distribution of the estimated and real numbers of trained people by Regional Development Agencies

Trained people Regional Development

Agency estimated achieved % achievement

RDA 1 North-East 7033 3441 48.9% RDA 2 South-East 6336 5364 84.7% RDA 3 South 5887 7254 123.2% RDA 4 South-West 2135 2288 107.2% RDA 5 West 5417 5307 98.0% RDA 6 North-West 8106 7686 94.8% RDA 7 Centre 7413 7608 102.6% RDA 8 Bucharest 5026 3510 69.8%

Total 47353 42458 89.7%

The data in the above table show that in some of the regions the number of trained people within the programme was overestimated, and the achievement rates were 48% (North-East Region) or 69.8% (Bucharest). At the same time, we can see that in some cases the estimated figures were exceeded, e.g. in the South Region (123%) or South-West Region (107.2%). Main categories of training beneficiaries The survey data related to the project promoters which have filled in the questionnaire show a total number of 21,902 individuals, who have attended the training courses. Most of the beneficiaries attended at least one course. Keeping in mind that the same person attended several courses within the same project, the total figure is 23,741 direct beneficiaries.

Table 7.4.3. Number of trainees by categories of beneficiaries

Categories of training beneficiaries

Individuals Beneficiaries of the

training courses

Theoretical number of attended courses by

person

employees 13842 15309 1.11 current/potential entrepreneurs 960 1275 1.33 unemployed 4004 4036 1.01 socially-disadvantaged people 3096 3121 1.01 total 21902 23741 1.08

The distribution by main categories of beneficiaries shows that the most important share of the trainees belongs to the employees (64.5%). 17% of the beneficiaries of the analysed courses are unemployed persons and 13.1% are disadvantaged categories. Entrepreneurs account for 5.4% of the total trainee’s number. Taking into account that the entrepreneurs are only a small part of the working population, it is remarkable that a rather large percentage of entrepreneurs that have been trained were subjects of more than one course.



Fig. 9.2.1.1. Distribution of beneficiaries, by main categories

64,5%5,4%

17,0%

13,1%employeesentrepreneursunemployeddisadvataged people

Concerning the promoting organisation we can see that the company promoters have had a higher number of employees trained, but also trained a considerable number of unemployed people. Companies account for lower shares compared to other types of promoting organisations, in the disadvantaged people category as well as in the entrepreneurs’ category

7.5. Output of training courses The training measures have occupied a very important place within the PHARE HRD 2000 projects. According to the data provided by the Ministry of European Integration, the estimated number of training courses was achieved by almost 105%.

Table 7.5.1.Distribution of estimated and real number of training courses, by development regions Training courses Regions

estimated achieved

% achievement

ADR 1 North-East 186 125 67,2% ADR 2 South-East 46 46 100,0% ADR 3 South 150 188 125,3% ADR 4 South-West 49 57 116,3% ADR 5 West 112 109 97,3% ADR 6 North-West 228 228 100,0% ADR 7 Centre 77 144 187,0% ADR 8 Bucharest 52 47 90,4%

Total 900 944 104,9%

Chart. 7.4.4. Beneficiaries of training programs, by main categories and type of promoting organization

0%

20%

40%

60%

80%

Employees Current/potentialentrepreneurs

Unemployed Disadvantaged categories

CompanyOthers



There are also regions where the number of courses was overestimated, the rate of achieved courses being 67,2% (North-West) or 90,4% (Bucharest). The rest of regions register rates between 97% (West) and 187% (Centre).

According to the present survey, out of a total amount of 185 investigated projects, almost 89% have developed training programmes addressed to the target groups. The distribution of courses at the regional level is presented in the table bellow (table 7.5.2.).

Table 7.5.2. Distribution of training courses, by Regional Development Agencies Training courses

Regions No. %

ADR 1 North-East 67 2,4% ADR 2 South-East 56 16,1% ADR 3 South 77 6,5% ADR 4 South-West 46 4,8% ADR 5 West 64 21,0% ADR 6 North-West 61 11,3% ADR 7 Centre 27 7,3% ADR 8 Bucharest 56 30,6%

Total 454 100,0%

The training courses developed in the surveyed projects cover all the development regions, in different proportions. Most of the courses included in the survey were developed by the promoters from Bucharest region. This is influenced also by the answering rate in Bucharest region, higher than others. West and South regions are also well represented in the lot of projects that developed training courses. The rest of the regions register low proportions between 2, 4% and 7,3% from the total analysed training courses.

The number of training courses developed within a project Within every project, which had developed training activities, a different number of training courses has been organised, the highest number of courses within one project being 17 courses. Almost half of the projects have had only one training course, while 14% of them have organised complex packages of over 5 courses.

Table 7.5.3. Number of course organised within one project

Courses / project Number of projects %

1 81 49.4 2 - 5 60 36.6 over 5 23 14.0

Total 164 100.0%

Depending on the type of the promoting organisation we can see a slight preference of companies for a lesser number of courses (1-2 courses). Thus the share of the companies, which have had more than 3 courses is 31.9% of the total number, while the corresponding percentage of other types of promoting organisations is over 40%, by almost 8 percentage points higher.



Table 7.5.4. Number of courses organised within a project, depending on the type of the promoting organisation

Number of courses organised within

one project Projects, whose promoter is a „company”

Projects, whose promoters are „other than companies”

no. % no. %

1-2 32 68.1 70 59.8 3-5 8 17.0 31 26.5

over 5 7 14.9 16 13.7

Total 47 100 117 100

Characteristics of the training courses

The research conducted on 185 projects counted a total amount of 445 training courses. The investigation by questionnaires followed the identification of the following characteristics in the training curses held within the projects: goal and fields of training, course duration, fields of economic activity related to the courses, target groups of the course.

• Goal and fields of training

Out of the 445 training courses included in the research, 34.4% were training and re-conversion courses in different occupations, over 65% were designed to update the old or acquire new knowledge in specific fields.

Chart 7.5.1. Course distribution according to the training goal

0 10 20 30 40 50 60 70

qualification/requalificationcourses

professional developmentcourses

The inventory of the qualification / re-qualification vocational courses shows that the project promoters oriented themselves towards classical occupations, only few some of them being new on labour market.

Table 7.5.5. Inventory of the qualifications for which training courses were organised textile worker canned food worker garment worker milk processing worker tailor – garment worker catering worker tailor textile garment waiter/ bar tender electrical mechanic baker welder / gas welder shop assistant mechanical shaper confectioner / pastry cook



mechanical hauler cook saw operator carpenter stoker receptionist crane operator topometry / cadastre worker fibber glass poly-esther operator agricultural tourism operator cutting operator driver brick-plaster layer car mechanic railway builder car electrician car tin smith

The courses for competence upgrading or for acquiring new competences were organised on the following themes: new technologies upgrading / specialising (45.9%), specialised management (22.6%), human resources management (11.6%), train the trainers / facilitators (5.5%), project management (5.1%), guidance and counselling (2.1%).

Chart 7.5.2 Course distribution according to the training field

50%

24%

13%6% 5% 2% upgrading/specialising in new technologies

specialised management

human resources management

train the trainers/facilitators

project management

guidance and counselling

• Target groups of the courses according to the education level

Depending on the participants’ educational level over 72% of the courses were addressed to the high school level or higher education level persons, while only ¼ of the courses were meant for lower educated people.

Table 7.5.6. Target groups of the courses according to the beneficiaries’ education level

Total Training goal/field

Low education

level High school and

higher education level No. %

Total 122 323 445 100 Vocational qualification/re-conversion in different occupations 116 37 153 34.4 Competences updating: 6 286 292 65.6

75.8% of the activities directed at the low educated persons were training/re-conversion courses, while the competence updating courses were almost exclusively targeted on the high school leavers and higher education graduates.



Table 7.5.7. Target groups of the competences updating courses according to the beneficiaries’ education level

Training goal/field

Low education level

High school and higher education

level Total - upgrading/specialising in new

technologies 4 130 134 45.9

- specialised management 1 65 66 22.6 - human resources management 34 34 11.6

- foreign languages 21 21 7.2 - train the trainers/facilitators 16 16 5.5

- project management 15 15 5.1 - guidance and counselling 1 5 6 2.1

The main training themes show a focus on upgrading new technologies skills in different fields and the management skills. Teacher training and guidance and counselling are also preferred domains of the training courses developed.

• Fields of economic activity addressed by the training programmes

More than half of the courses organised within the investigated projects are targeted on the service area, followed by the industrial area (17.8%). Only 5.1% of the courses have trained human resources for agriculture, while 3.1% addressed the building sector. The interest of the promoters to develop training courses in the service area is part of the economic development strategy of Romania, within which the services development is a priority sector.

It is worth mentioning that a relatively high percentage (15.5%) of the courses are designed for public administration and defence, a field undergoing staff restructuring during the PHARE HRD 2000 projects.

The high share (19.8%) of the non-responses to this item can be determined either by the fact that the project promoters did not specify clearly enough the field they were addressing in their course, or by the fact that the courses addressed a too wide area and the field of economic activity was more difficult to identify.



Fig. 7.5.3. Course distribution according to the field of economic activity

0 2 4 6 8 10 12 14

Agricultura

Processing industry

Electric and thermal power, gas and water

Constructions

Sale and repair of cars and household goods

Hotels and restaurants

Transport, warehousing and communications

Financial transactions

real estate and buiness transactions

Public administration and defence

Education

Health and social protection

Other services.

vocational training/reconversion courses competence upgrading/aquiring courses

Depending on the type of course, we can see that the training/re-conversion courses were developed mainly in the following fields: agriculture, hotels and restaurants, while the specialising courses and the courses for acquiring new competences have a more important share in fields like public administration and defence, education and financial transactions. The economic field where we can see a balanced distribution of the vocational training versus specialisation / new competence course are: the processing industry, health and social welfare, other services.

Duration of the courses

• Course duration, by days and hours per course

The duration of the courses organised within the analysed projects has been between 4 and 120 days (19-720 hours/course). The most important share (41.8%) is that of the courses with duration up to 60 hours /course. A similar situation is in the courses whose duration is between 61 and 120 hours/course, while 16.3% have duration of over 360 hours/course.



Table 7.5.8. Course duration, by type of promoter’s organization

Hours Duration (days) Companies Other organisations Total 6-18 1-3 11.3% 5.2% 6.8%

19-60 4-10 37.1% 34.2% 35.0% 61-120 11-20 23.4% 13.3% 16.1%

121-360 21-60 18.5% 26.4% 24.2% 361-720 61-120 8.9% 17.0% 14.8% over 720 over 120 0.0% 2.1% 1.5%

Non-response Non-response 0.8% 1.8% 1.5% Total 100.0% 100.0% 100.0%

Depending on the type of the promoting organisation we can find a higher preference of companies for courses whose duration is shorter, up to 60 hours /course, compared to other organisations. In the category of the courses whose duration exceeds 360 hours/course, the situation is reversed, namely the proportion of the courses organised within the projects promoted by the companies is 8.9% compared to 19.1 in the case of other types of promoting organisations.

Depending on the goal of the training course, we can find that almost 98% of the qualification courses have over than 120 hours duration. The professional development courses are balanced split between courses of 6-120 hours/course (43%) and longer courses over than 120 hours/course (57%).

• The average duration of a training course

The average duration of a training course calculated on the research lot is 190 hours/course. The average duration of the vocational training/re-conversion courses is 374 hours/course, while the vocational development courses have an average duration almost 4-times shorter (97 hours /course). In the case of the company project promoters the average duration of a training course is slightly shorter, namely 132 hours/course.

Depending on the field of economic activity of the course, the average duration of a training course can reach high values between 320 and 480 hours in fields such as: constructions, electric and thermal power, gas and water, transport, warehousing and communications, hotels and restaurants and agriculture. The shortest average duration of a course is in the following fields: real estate and business transactions (99 hours/course), education (102 hours/course) and public administration and defence (105 hours/course).

In the qualification training courses, almost all the fields of economic activity where courses have been conducted the average duration exceeds 360 hours/course, given the regulations on the authorisation of the training providers, which provide for minimum duration of the training courses, depending on the qualification level to be achieved (360 hours – level 1; 720 hours – level 2; 1080 hours – level 3).

The highest average durations are in the following fields: electric and thermal power, gas and water (551 hours/course), construction (492 hours/course), hotels and restaurants (470 hours/course), and at the other extreme we find the areas of services and agriculture.



Depending on the themes of training, the vocational development courses have higher average duration in fields like: training the trainers/facilitators (106 hours/course), guidance and counselling (106 hours/course), and foreign languages (103 hours/course). The lowest average duration is of 40 hours/course in human resources management, followed by the “specialised management” – 67 hours/course.

However, standard deviation data shows a high diversity of the course duration, depending on field of economic activity. Therefore it is difficult to find a specific trend in the average duration of the courses developed. The reason could be that each training provider organise the courses duration according to its specific needs. Also, no specific regulation have been recommended in the call fro proposal regarding this issue.

• Average, minimum and maximum number of hours per course

The comparative analysis of the minimum, average and maximum hours/course shows that there are a number of projects, which have opted for courses with a higher number of hours, much over the average. We can find this trend especially in the following fields: training the trainers/facilitators, project management, foreign languages, upgrading/acquiring new technologies, specialised management.

Table 7.5.11. Average, minimum and maximum number of hours of the professional development courses, by the training field

Duration in hours Training field of professional development courses average maximum minimum Standard deviation

upgrading/specialising in new technologies 71 360 30 151,57 specialised management 67 308 8 99,08 human resources management 40 120 16 62,32 foreign languages 103 360 6 74,07 train the trainers/facilitators 113 1160 6 96,92 project management 87 400 8 24,00 guidance and counselling 106 280 20 87,70

The highest number of hours/course is found in the training the trainers/facilitators courses, with a total amount of 1160 hours/course. But this course duration is an exception, because the average duration in this course field does not exceed 113 hours /course. A maximum number of 400 hours/course is found in the project management, where the average is 87 hours/course.

Standard deviation data shows again that there is high diversity of the course duration, depending on the theme of training course. Only in the field of project management, standard deviation shows that the average, minimum and maximum duration of the courses are relevant for the entire sample of investigated training courses.



8. Effect

8.1. Viability of the training centres Important for evaluating the results of the projects having conducted training activities and, in particular, of those seeking to build institutional capacity by creating new training centres, are the answers to these questions:

- Are the new training centres created still operational?

- Are training activities still taking place at these centres?

- What is the current content of the training offer by the new centre?

A first indicator to consider when answering these questions involves the status of the training centre: authorized / unauthorized / decommissioned (non-functional).

The data collected from the survey of the project promoters’ sample show that 44% of the created training centres has obtained or is seeking authorization and 36% of them intends to seek authorization (Table 8.1.1). This is the case of 62% of the centres created by companies and of 35% of the centres opened by other organizations. This situation supports the idea advanced all along the analysis about company interest in training human resources, including the creation of their training centres. However, the situation is debatable when considering 14% (possibly, the percentage is higher, given the 6% of non-answers) of the newly established centres, the promoters of which declare they do not intend to seek authorization. In these cases, the causes having determined such promoters to wave authorization are important. At the same time, at the end of external financing, such situations make project sustainability debatable.

TABLE 8.1.1. Status of established training centres, by type of promoter

Total Companies Other organizations Currently. centres are

Number Percent Number Percent Number Percent

Authorized 19 38.0 8 50.0 11 32.4 Undergoing authorization 3 6.0 2 12.5 1 2.9 Authorization intended 18 36.0 2 12.5 16 47.1 Authorization not intended 7 14.0 3 18.7 4 11.8 Non-answer 3 6.0 1 6.3 2 5.8 Total 50 100.0 16 100.0 34 100.0

The interviews have revealed that, in some cases, authorization of the new centres has been given up owing to the complicated procedures involved: We tried to prepare the needed documentation and obtain authorization but we gave it up because we lacked the fiscal certificates and the time has run up (Company, Bucharest).

In some other situations, the cause invoked was a changed legislation having occurred between project preparation and the project bidding date: I think that the legislation was highly dynamic and it coincided with the period when the program was being developed... When we prepared the project, NAE used to issue graduation certificates based on clear legal prerogatives; in January 2003, when we started the project activities proper, NAE certificates



had to be approved by the Centre and were issued only for vocations included in the Occupational Classifier of Romania (Company, Cluj).

While the legislation on the accreditation of the vocational training services was in the making, some training providers would appeal to partner institutions able to issue nationally recognized certificates. Some organizations, which did not want to become authorized providers of a specific training course, resorted to the same solution, explaining that it means to retain certain persons, some permanent lecturers, while the profile of our organization fails to include continuous training as a permanent activity (University, Alba Iulia).

In the same context, it is also significant the fact that an considerable ratio—of about 30%, even higher if we include the non-answers—of the organizations having developed training activities have also purchased some equipment, with this end in view. At present, they use the equipment both for training purposes, and for production and specific activities (Table 8.1.2). In the case of companies, considering the specifics of their activity, the ratio is higher than that of other organizations – over 40%, as against 25%.

TABLE 8.1.2. Use of equipment in the case of projects that developed training activities with equipment purchases

Total Companies Other organizations Equipment use

Number Percent Number Percent Number Percent

For training 52 65.0 12 52.2 40 70.2

For training and production / specific activity 23 28.8 9 39.1 14 24.6

No answer 5 6.2 2 8.7 3 5.3

Total 80 100.0 23 100.0 57 100.0

Another indicator of project feasibility and continuity is the actual content of the training offer. More precisely, it is about the degree at which the actual content of the training offer used in the training centre activities is identical / significantly different from that developed in the project.

In point, the analysis data reveal that most of the newly set up training centres (80%), the actual content of the training offer is either identical or improved as compared to the project one. However, there is a relatively important ratio of new training centres that use a shortened or significantly different content than the project training offer – 16% (excluding no-answers); the ratio is higher in the case of companies – over 31% and lower in the case of other organizations – 9% (Table 8.1.3).



TABLE 8.1.3. Status of the created training centres, by actual content of their offer, by type of promoter

Total Companies Other organizations Actual content of the training offer Number Percent Number Percent Number Percent

Identical to the one developed in the project

22 44.0 6 37.5 16 47.1

Enhanced, as compared to the one developed in the project, as includes new qualifications

18 36.0 4 25.0 14 41.2

More limited than the one developed in the project

5 10.0 3 18.8 2 5.9

Significantly different from that developed in the project

3 6.0 2 12.5 1 2.9

No answer 2 4.0 1 6.2 1 2.9

Total 50 100.0 16 100.0 34 100.0

In the case of companies, a limited training offer may actually signify its adjustment to the training needs of its employees and the implicit give up of the training programmes targeted at other target groups. In addition, from among the project promoters having created training centres without intending to have them authorized, almost half (43%) have significantly limited the content of their training offer, as against the one developed during the project. In such cases – it refers to 3 new training centres – we should appreciate that the limited offer and the absence of the intent for centre authorization is tantamount to the promoter’s dropping out the training activity.

The interviews with project promoters indicated among of the main reasons for reducing the training offer the fact that the number of trainees decreased as a consequence of a lower demand on labour market for the competences developed through the course. This situation raises questions about the accuracy of training needs evaluations, both for local and regional level.

8.2. Completion and drop-out at the beneficiary level According to the data reported by the investigated project promoters the completion rate of the training courses is 96.2%. By categories of beneficiaries the highest rate is in employees (97.7%) and the lowest in unemployed people (92.0%).

Table 8.2.1. Completion rate and drop outs by categories of beneficiaries

Categories of training beneficiaries

Total enrolment

completed the course

Drop-outs completion

rate drop-out

rate employees 15309 14953 356 97.7% 2.3%

current/potential entrepreneurs

1275 1197 78 93.9% 6.1%

unemployed 4036 3715 321 92.0% 8.0%

socially-disadvantaged people

3121 2962 159 94.9% 5.1%

TOTAL 23741 22827 914 96.2% 3.8%



The beneficiary survey on persons involved in training courses shown a lower completion rate of almost 92,3%, with a difference of 4,3 p.p. The rate of beneficiaries that declared that graduated the course and took part in the final evaluation of the course is even lower, being about 89%. A significant difference is noticed also between project promoter and direct beneficiaries in the drop-out rate reported. According to the project promoters, the drop-out rate is 3,8%, while the same rate registered by the beneficiaries survey is 5,9%. The difference between the drop out rate reported by the promoters and the beneficiaries can be explained trough different reasons as such:

- Lack of information on the beneficiaries with the promoters: - The specific sample of questioned beneficiaries could differ than the entire program

beneficiaries figures. Nevertheless, these kinds of figures always differ between promoters and the target groups. Only when comparing the information on the beneficiary level one can try to develop a more exact figure. Better of course is to monitor the projects on the beneficiary level. The highest drop-out rate reported by project promoters is in unemployed, namely 8%. At the other end, 2.4% of the employees have dropped out. In the other categories of beneficiaries the drop-out rate is 6-7%.

employeesentrepreneurs

unemployeddisadvantaged people

Fig.8.2.1. Enroled participants, completed course and drop outs, by categories of beneficies

drop-outscompleted the course

enroled

Table 8.2.2. Drop-out rate by categories of beneficiaries Categories of training beneficiaries Drop-outs Drop-out rate employees 375 2.4% current/potential entrepreneurs 78 6.6% unemployed 299 8.0% socially-disadvantaged people 203 5.8%

total 1532 3.8%

If we consider sub-categories of beneficiaries, among the employees those who have had higher drop-out rates belong to the following groups: craftsmen and skilled workers in arts and crafts, machines and equipment maintenance workers. As for the disadvantaged categories, the highest drop out rate is for young graduates



Table 8.2.3. Drop-out rate, by sub-categories of beneficiaries

Employees drop-out

rate Socially-disadvantaged people

drop-out rate

Managers and executives in public administration and social-economic units 0.89% Young graduates 13.47%

Intellectuals and scientists (e.g. engineers, teachers, trainers, career guidance counsellors, etc.)

2.16% Young people coming from children’s homes

5.76%

Technicians, foremen and assimilated 2.10% Roma 4.17%

Civil servants 0.98% People with disabilities 5.00%

Employees in services, commerce and assimilated 4.60%

People who have served time in prison

0

Farmers and skilled workers in agriculture, forestry and fishery 2.40% Long-term unemployed 2 4.82%

Craftsmen and skilled workers in arts and crafts, machines and equipment maintenance workers

11.28% Other 2.55%

Machines and equipment operators, assembly-line workers 6.34% Total 5.09%

Unskilled workers 3.98%

Other 2.20%

Total 2.33%

Reasons for drop-out

From the viewpoint of the project promoters the main reason for the drop-out is the time constraints of the beneficiaries. Given the high average duration of the courses organised within the projects, this argument seems to be an important one. The possible lack of flexibility in organising the courses by days or modules can also result in time constraints for the beneficiaries. A similar reason is mentioned also by the beneficiaries. Almost 16% of beneficiaries that dropped out the course consider that the course schedule was unsuitable for them (too long, too many classes per day, unsuitable timing, unsuitable periods of time).

According to the project promoters, another important reason for drop-out is the low interest of the beneficiaries in attending a course. This reason care highlight either the lack of confidence of the beneficiaries in the training measures as a condition for success on the labour market / work place or the lack of attractiveness o the courses. From the beneficiaries perspective, almost 13 % of the beneficiaries that dropped-out the course consider that the course was useless for their needs, explaining their low interests in attending the course.

Table 8.2.4. Reasons for drop-out mentioned by the promoters and beneficiaries

Reasons mentioned by the promoters % Reasons mentioned by beneficiaries

that dropped out the course %

lack of time 31,5 Personal reasons (change of residence, family obligations, health reasons etc.) 30,5

low interest in attending the course 16,7 High costs of bed, board, transport etc. 22,0

changing / finding a workplace 14,8 Unsuitable course schedule (too long, too

many classes per day, unsuitable timing, unsuitable periods of time)

16,9



high level of course 11,1 Too high level of knowledge required for the course 16,9

personal problems / health reasons 9,3 Training course useless for trainee’s needs 13,6

lack of financial resources 9,3

other reasons (organisational shortcomings, non-functioning equipment, computer, internet access)

7,4

Changing / finding a workplace during the course is also a reason, which made the beneficiaries to drop out the course. This kind of motivation draws the attention to the participants’ expectations from the benefits of the training. On the short term, a training course might be a better chance to find a job or to be promoted in the career, but long term benefits seam to be more difficult to identify by the beneficiaries.

The high level of the training course is mentioned by the project promoters as a reason for drop-out. The beneficiaries mentioned also in a high percent the same reason. In this case, it was probably necessary to make more efforts to adapt the training courses to the specific needs of the target population, so it has more chances of success.

Other reasons mentioned by the promoters relate to the beneficiaries’ personal problems. Among them the lack of financial resources to attend the course is also mentioned. According to the direct beneficiaries’ declarations, the main reasons for drop-out are personal causes as: change of residence during the course period, family obligations, health reasons etc.). The second reason is mentioned by almost 17% of the beneficiaries that drop out was the high costs of connected expenses to attend the course, as accommodation, transport and others.

Other reasons have also been mentioned by promoters relating to the way the training course was organised. Some of the beneficiaries dropped it out because of some organisational deficiencies. The lack of training equipment (computer, internet access) has contributed, according to the promoters, to discouraging the beneficiaries to complete the course.

Certification According to the beneficiaries surveyed, almost 67% of them received at the end of the course a graduation/qualification certificate recognised on the labour market and 17.1% received a certificate recognised at the company level.

Chart 8.2.2. Distribution of training beneficiaries, by type of certificate issued at the end of the course

4,7%nonRl

11%no certificationt

17,1%graduation/qualification certificate recognised at the company level

67,2%graduation/qualification certificate recognised on the labour market



8.3. Satisfaction of beneficiaries Almost 57% of the beneficiaries declared themselves satisfied to a large extent with the training content, while 4,7% of them are not satisfied at all or only to a small extent.

Direct beneficiaries of the training courses seem to be even more content with the quality of trainers. Almost 68% declared that they are satisfied to a large extent with the trainers, while 24% had a good enough opinion on the quality of teachers. Similar appreciations have been registered in the case of beneficiaries’ opinion on the methodologies for teaching and training that have been used during the course.

The highest level of dissatisfaction is declared on the quality of learning aids and equipments and the opportunities offered to use learning facilities during the course. Data shows that 7% of the questioned beneficiaries were satisfied to a small extent or not at all with the learning equipments and aids provided by the training centre, while 12,6% of the beneficiaries declared that they didn’t have the opportunity to use the learning facilities at all or only to a small extent.

In spite of high level of training equipment acquisition, a significant number of beneficiaries declared themselves unsatisfied with the opportunities to use training facilities during the course.

Chart 8.3.1. Satisfaction level of beneficiaries regarding the relevance of the training content, quality of trainers/teachers/instructors, opportunity to use the learning aids

and equipment during the course

50%

55%

60%

65%

70%

75%

80%

85%

90%

95%

100%

relev

ance

of tra

ining co

ntent

quali

ty of

trainers

oport

unity

to us

e the

equip

ments

nonR

I do not know

not at all

to a small extent

w ell enough

to a large extent



Most of the beneficiaries have been satisfied with the support provided by the organisers of the course. Only a small percent of the beneficiaries declared that the support was inadequate or even they had no contact with the organiser.

Table 8.3.1.. Satisfaction level regarding the support provided by the organisers of the course

Very good 61,6% Good 25,9% Adequate 3,7% Inadequate 1,6% I had no contact with the organisers 1,9% NonR 5,3% Total 100,0%

Regarding the interests of beneficiaries for further training courses, most of the questioned persons shown their willing to attend similar training programs in the future.

Chart 8.3.2. The interest of beneficiaries in attending similar training programmes in the future

1,2%not at al interestedl 5%

NonR5,4%ry interested

14,4%terested

74%highly interested



9. Efficiency The information collected through the survey din not allowed the evaluation of the training programmes effects for the courses participants and, as a result, neither the relation between costs and effects. The data regarding the costs for training activities are provided in chapter 6.2. (Training costs). 10. Impact

10.1. Dissemination of results

Most of the investigated promoters have used diversified ways of dissemination. Among them, the publication and distribution of project-related information material was the most frequent way used. Other ways of dissemination frequently mentioned were: the publication of information articles in mass media and the organisation of seminars and conferences.

Chart 10.1.1. Dissemination methods

0% 10% 20% 30% 40% 50% 60% 70% 80% 90%

Informative materials (leaflets, brochures,posters)

Articles in the mass-media

Organise seminars and conferences

On-line informative and training materials

Set up a website

Publishing training modules

Other

Almost 38% of the promoters think that dissemination was assured by the publication of information materials in electronic form, while almost 22% of them have published training modules. Only 21.6% of the promoters have built up project websites.

Dissemination levels

0%10%20%30%40%50%60%70%

organisation local regional national

Chart 10.1.2. Dissemination levels



The most important level where the project results have been disseminated is the local one, as over 60% of the investigated promoters mentioned that they had carried out dissemination activities at this level. The dissemination at the regional level has been carried out only in 34% of the investigated projects. Only 15% of the projects involved national level dissemination.

10.2. Sustainability of effects on beneficiaries In case of the majority of the surveyed beneficiaries, the moment of investigation took place at 6 month to 2 years since graduation. In this period of time some changes occurred in their carer or professional status. Training graduates who were employed at the starting time of the course mentioned the following changes:

- An important share got a promotion in the company (21.8%).

- Moving on to another job in the same company is a change that occurred in the case of 10% of the surveyed employees;

- A lower percentage found a job with another company.

Most of the surveyed employees trained in the program (61,1%) declared that their position in the company did not change at all. Even so, they noticed that, in general, their activity in the workplace improved after the participation in training in terms of quality of work, responsibility and efficiency in organising the work. 3,6% failed to find any significant progress, while almost 5% cannot evaluate this issue. A very small percent of the surveyed employees at the moment of training course lost their jobs and are currently unemployed.

Training graduates, who were unemployed or fresh graduates at the time of the start of the course, mentioned in the survey the following changes:

- 45% of them are currently employed for an indefinite period of time (permanently employed)

- 7,9 % are currently temporary workers (employed for a limited period of time)

- 5,3 % set up a business

- almost 40% are still seeking a job.

Since graduation, 35,7% of the unemployed or fresh graduates surveyed didn’t find any job, while 8.5% had one/several short term jobs, but now are unemployed.

Since graduation, almost 48% of unemployed or fresh graduates found a job and have kept it to date or had one/several jobs and now are employed. Other 8,3% have recently found a job.

Almost half of the beneficiaries that find a job after the course are confident that the course attended helped to a large extent to their employment, while 22% of beneficiaries think that the course was not of help at all.



10.3. Impact in promoters opinion Most promoters have pointed out the importance of the project impact at local and organizational level. Only one fifth of the respondents have indicated an impact at the project regional level, a point of concern from the perspective of the program general objective, i.e., support granted to the implementation of the integrated measures on regional development. The fact should be considered that, both at the level of all projects and at that of the sample project, the ratio of large-size projects has been highly reduced.

The opinion of most project coordinators interviewed is that the project impact was significant, both on the target population groups and on the organization. The questionnaire survey indicates that the companies also appreciate the project effects; only 3% of them have considered the project effects as unsatisfactory. However, one out of four promoters showed some difficulties in evaluating the project impact.

More than one third of the surveyed companies have registered a positive evolution since the project launch till the survey, both in terms of productivity, turnover, or profit margin, and of number of employees or level of investment in new equipment / technologies. In addition, an important number of companies show an increased budget allocation to training and increased numbers of employees participating in training courses.

Only about one fourth of the surveyed companies have declared that the project has also aimed at developing the human resources management at the institutional level. The small number of companies which have not acknowledged the project importance in view of the institutional development (e.g., for the improvement of the human resources management system) proves that the training activities have been designed without taking into account the real project potential and failed to involve all decision-makers at organizational level.



11. Conclusions and recommendations

The key statistical data resulted by the evaluation will be presented above, according to the evaluation model based on the following structure: resources – operations – costs – output- effect – impact.

RESOURCES Funds by regions • Given that the allocation of funds took into consideration two categories of regions

(Priority Regions Category I and Priority Regions Category II) and the different funding resources (the European Community and the national budget, respectively) they benefited, the volume of allocated funds differed from one region to another. Thus, the highest funding level has been recorded in the regions Northwest - 20.7% and Southeast – 20.3% of the total allocated funds while the lowest level has been recorded in the regions Southwest Oltenia – 5.0% and Bucharest-Ilfov – 5.4%

• At the project level, it can be observed also:

- A lower funding level for companies, as compared to the other types of organizations, at each development region level, an understandable situation considering the low number of companies among the total promoters in the sample investigated;

- The higher average budget per company project, at each region level, as compared

to other types of promoting organization (per total regions, the average budget – from Phare sources / national budget – of a company amounted to almost EURO45,600; in the case of the other organizations, it was EURO42,000).

Profile of project promoters • The participation of different types of organisation to the HRD program is well balanced.

The ratio of organizations specialized in human resources development— which include NGOs, companies as well as research institutes, training centres, education an training institutions and Chamber of Commerce —amounts to almost a third of the total number of promoters surveyed.

• Companies (enterprises, consultancy firms, SMEs, etc.) make up the major category of project promoters of over one third of the total projects. The majority of the surveyed companies are small, with less than 50 employees and a rather long period of operation (over ten years).

• Non-governmental organizations (associations, foundations, trade unions, and employers’ organizations), with a share around 25% are also well represented, followed by education and training institutions (universities, high schools, TVET schools -16%), local authorities (12%), and other public institutions (county employment agencies, county school inspectorates etc. - 11%).

Partners • One in four promoters indicated the absence of any project partner in the case of their

project. For projects with at least one partner, the mean partner number per project was 2.2 (for small projects) and 3.4 (for large projects).

• Local public administration holds a share close to the sum total of the rest types of partner organisations while social partners were nominated by less than 10% of promoters surveyed;



IMPLEMENTATION/OPERATIONS Program priorities and measures addressed • The distribution of projects by objectives highlights the fact that the majority of project

promoters has preferred training activities, these activities accounting for around two thirds of the total measures taken into account the number of projects exclusively addressing to active employment measures (objective 2) and social inclusion (objective 3) amount to less than 10% of the total projects. In almost 60% of the cases, the promoters surveyed mentioned to have focussed on only one priority.

• In the case of surveyed projects, a total of 31732 persons have benefited from project’s activities, the employee accounting for around half of the total while unemployed and disadvantaged categories for around 20%. In the case of training activities, the differences between the categories of beneficiaries are even higher, the employees accounting for almost two thirds of the total.

• More than 80% of the companies surveyed organised training activities; the number of trainees in the case of projects promoted by companies represent almost one third of the total number of trained people; however only 5% of these trainees were disadvantaged persons.

Problems in initiation and implementation Main difficulties stated by project promoters related to project initiation and implementation are the following: the delays in securing financing, difficulties in continuing the project after Phare financing ceased and difficulties related to the recruitment of target groups. The majority of promoters surveyed declared that they had little difficulties in following the existing procedures in accessing different sources of information and in communication with RDA or MEI.

Other issues mentioned by project promoters during field interviews were related to: difficulties in cooperation with project partners (i.e. formal involvement, underachievement of objectives etc.); difficulties in relation with project beneficiaries (i.e. low motivation of disadvantaged persons to attend training activities); financial (i.e. losses due to exchange rates fluctuations, slow procedures related to obtaining the official approval of budget revision and other decisions of the Treasury, Ministry of European Integration and Regional Development Agencies); changes in legislation (i.e. accreditation of training providers, fiscal code etc.), acquisitions (i.e. difficulties in purchasing equipments with EU origin certificate). Another important difficulty to which the project promoters referred to was securing their own contribution, issue mentioned especially by public institutions representatives.

COSTS Expenditure on equipment • The mean of equipment acquisition expenditures from project total budget represented

about 28%.

• The highest rate of the funds allocated for equipment acquisition occurred in the framework of small-scale projects (34.5%, as against 25% in large-scale project) and of those developed by companies (30%, as against 26.7% - other organizations). In point of regional development, the highest rate occurs in Northwest (32%), Southeast, and Northeast (around 30%). Possibly, owing to a better endowment of the Bucharest-Ilfov and Centre regions, their expenditures on equipment purchasing were the lowest: 21% and 20%, respectively, of the project budgets.



• In the case of the companies, the intention to develop their institutional capacity by production equipment acquisition; the hypothesis is suggested by the data on:

- share of total budget allocated to equipments: 30%, as against about 26% - other organizations;

- use of equipment purchased for other purposes than training (for production / specific activity), as well – about 39%, as against 24%;

- the degree at which the equipment purchased is used for training activities: 55% of the companies use the equipment, at the highest degree, for training purposes; in the case of the other organizations, the corresponding percentage is 58%;

- share of production equipments in the total budget allocated to all types of equipments purchased: companies – 20%, other organizations – 4%.

Training Costs • The average cost per participant is significantly higher compared to the courses

organised by companies and investigated through a NIS survey at national level1: 210 Euro as against 75 Euro. However the important differences for the medium value cost per participant is entailed by the long duration of courses in the case of HRD Phare 2000 program: 73 hours/course (INS), as compared to 190 hours/course (HRD component).

• The cost calculated by training hour is similar in both cases: EURO 1,0 (INS) and EURO 1,2 (HRD Program).

OUTPUT Number of Successful and Unsuccessful Projects • Only 5.5% of the total number of promoters has failed to finalize their projects, the

companies being the main project promoter with cancelled contracts. Funds absorption Capacity • In general, the projects in the HRD component demonstrated an average capacity of fund

absorption (around 85%).

• In point of regions of development (taking into account all projects finalised within HRD program), projects developed in North-West (89%) and Bucharest-Ilfov (87.5%) have the highest indicator value (higher than the average); in point of types of organization (in the sample evaluated), the highest indicator value is that of projects developed by companies (86.4, against 85% in the case of other organization). A more careful monitoring, and assistance granted all along project development aimed at increasing project absorption capacity.

Creation of different types of Institutional capacity building • More than half of the number of projects (52.4%) implemented two, three, or four of the

measures targeted at institutional capacity building, namely: creation of a new training centre, or development the training capacity (new classrooms, new training equipment); enhancing the training offer (new training programmes, new target groups); training the trainers.

• The specifics and the number of the measures having been implemented differ, function of the type of promoter. Thus, companies have been more interested than the other organizations in creating their own training centres (34% as against 29%); the other organizations saw as priorities the measures concerning the enhancement of their

1 Caracteristics of CVT in Romania, NIS, Bucharest, 2001.



training offer (65% as against the companies’ 45%) or the training of trainers (23%, and 15%, respectively.

• Un unbalanced distribution by region of development could be noticed among the projects that developed institutional capacity, particularly, creation of new training centres. The situation could be generated by such factors as: a different level of socio-economic development of the regions; the different funding available in one region or another; the different fund absorption capacity; characteristics of the trainer market (number of trainers, diversity of offers, quality of trainers, etc.); the training needs at regional level of the various target population groups, etc.

• Most of the companies having created their training centres use a completely private capital (81.3%), are small-sized enterprises (62,5%), and have been operating for about 10 years (62,5%).

Characteristics of Beneficiaries • Employees had benefited the most from the programme, accounting half of the total

target-categories. The distribution of the direct beneficiaries, by type of assistance received2 indicates the fact that the employees are 50% of the total target-categories. However, the rate of the entrepreneurs (actual or potential), the unemployed, and the socially disadvantaged could be judged as high, compared to the rate of projects targeted on measures specific to priority 2 and 3. The explanation relates to the accidental inclusion of these categories in the activities of the project (not being strategically targeted by project promoters) and, to the way the project priorities and measures have been formulated.

• Most important share of the trainees belongs to the employees. The survey data obtained from the respondent promoters show a total number of 21,902 individuals who have attended the training courses. Most of the beneficiaries attended at least one course. Keeping in mind that the same person attended several courses within the same project, the total figure is 23,741 direct beneficiaries. The distribution by main categories of beneficiaries shows that the most important share of the trainees belongs to the employees (64.5%). 17% of the beneficiaries of the analysed courses are unemployed persons and 13.1% are disadvantaged categories. The entrepreneurs account for 5.4% of the total trainee numbers.

• Beneficiaries with a high level of education benefit more of the training courses. Depending on the participants’ educational level over 72% of the courses were addressed to the high school level or higher education level persons, while only ¼ of the courses were meant for lower educated people.

Developed Training Courses • The training measures have occupied a very important place within the PHARE HRD

2000 projects. According to the present survey, out of a total amount of 185 investigated projects, almost 89% have developed training programmes addressed to the target groups. The training courses developed in the surveyed projects cover all the development regions, in different proportions. The promoters from Bucharest region developed most of the courses included in the survey. This is influenced also by the answering rate in Bucharest region, higher than others. West and South regions are also well represented in the lot of projects that developed training courses. The rest of the regions register low proportions between 2, 4% and 7,3% from the total analysed training courses.




• The courses focused on competence upgrading or acquiring new competences and less on qualification. Out of the 445 training courses included in the research, 34.4% were training and re-conversion courses in different occupations, over 65% were designed to update the old or acquire new knowledge in specific fields.

• Service area was the focus of the developed training courses More than half of the courses organised within the investigated projects are targeted on the service area, followed by the industrial area (17.8%). Only 5.1% of the courses have trained human resources for agriculture, while 3.1% addressed the building sector. The interest of the promoters to develop training courses in the service area is part of the economic development strategy of Romania, within which the services development is a priority sector. It is worth mentioning that a relatively high percentage (15.5%) of the courses are designed for public administration and defence, a field undergoing staff restructuring during the PHARE HRD 2000 projects.

• There is high diversity of the course duration, depending on the theme, scope and economic field of training course. The average duration of a training course calculated on the research lot is 190 hours/course. The average duration of the vocational training/re-conversion courses is 374 hours/course, while the vocational development courses have an average duration almost 4-times shorter (97 hours /course). However, standard deviation data shows a high diversity of the course duration, depending on different variables. Therefore it is difficult to find a specific trend in the average duration of the courses developed. The reason could be that each training provider organise the courses duration according to its specific needs.

EFFECT Viability of Training Centres • Most of the training centres created via the HRD Component projects have received

authorization / are in the course of authorization (44%) or at least, the intent to pursue for authorization exists in their case (36%). In addition, most of the training centres continue to use identical or even enhanced programme content, as compared to the one in the project (80%).

• In some cases, about 14% of the newly created training centres, promoting organizations having created them do not intend to pursue their accreditation. Moreover, the content of their training offer is limited or even significantly different as compared to the one developed in the framework of the projects. As a result, at least for some of them, project feasibility and sustainability may be questionable.

Completion and drop-out at beneficiary Level • The beneficiary dropout rate is about 4%; the unemployed registered the highest dropout

rate (8%). On the other hand, 2.4% of the employees have dropped out. As for the disadvantaged categories, the highest drop out rate is in young graduates.

• According to the beneficiaries surveyed, almost 67% of them received at the end of the course a graduation/qualification certificate recognised on the labour market and 17.1% received a certificate recognised at the company level.

Beneficiary satisfaction • In general, the direct beneficiaries of the training courses declared their satisfaction with

the relevance of the training content. They seem to be even more content with the quality of trainers. Almost 68% declared that they are satisfied to a large extent with the trainers, while 24% had a good enough opinion on the quality of teachers. Similar appreciations



have been registered in the case of beneficiaries’ opinion on the methodologies for teaching and training that have been used during the course.

• The highest level of dissatisfaction is declared on the quality of learning aids and equipments and the opportunities offered to use learning facilities during the course. Data shows that 7% of the questioned beneficiaries were satisfied to a small extent or not at all with the learning equipments and aids provided by the training centre, while 12,6% of the beneficiaries declared that they didn’t have the opportunity to use the learning facilities at all or only to a small extent.

• Regarding the interests of beneficiaries for further training courses, most of the questioned persons shown their willing to attend similar training programs in the future.

IMPACT Effects on Beneficiaries • The employees having completed training courses record positive changes in their career

and professional status. An important percentage of the employees have advanced in their career (21.8%). Although over half of the employees failed to register any changes in their company jobs, almost 90% have stated that their activity at the work place has improved in terms of work quality, organization of tasks, and responsibility taken. However, such effects are not unexpected, knowing that most of the employees would have a higher educational level and qualifications before taking part in the course.

• The relatively high percentage of the unemployed and young graduates hired after having participated in the course (45% for an undetermined period and 7.9% as temporaries) represents one of the program successful points. It is also remarkable that 5.3% of the unemployed and young graduates have declared that they started their own business.

Impact Level in Promoters’ Opinion • Most promoters have pointed out the importance of the project impact at local and

organizational level. Only one fifth of the respondents have indicated an impact at the project regional level, a point of concern from the perspective of the program general objective, i.e., support granted to the implementation of the integrated measures on regional development. The fact should be considered that, both at the level of all projects and at that of the sample project, the ratio of large-size projects has been highly reduced.

• The opinion of most project coordinators interviewed is that the project impact was significant, both on the target population groups and on the organization. The questionnaire survey indicates that the companies also appreciate the project effects; only 3% of them have considered the project effects as unsatisfactory. However, one out of four promoters showed some difficulties in evaluating the project impact.

• More than one third of the surveyed companies have registered a positive evolution since the project launch till the survey, both in terms of productivity, turnover, or profit margin, and of number of employees or level of investment in new equipment / technologies. In addition, an important number of companies show an increased budget allocation to training and increased numbers of employees participating in training courses.

• Only about one fourth of the surveyed companies have declared that the project has also aimed at developing the human resources management at the institutional level. The high number of companies having acknowledged project importance in view of the institutional development (e.g., for the improvement of the human resources management system) proves that the training activities have been designed without taking into account the real project potential and failed to involve all decision-makers at organizational level.



Dissemination of Results • Most of the investigated promoters have used diversified ways of dissemination. Among

them, the publication and distribution of project-related information material was the most frequent way used. Other ways of dissemination frequently mentioned were: the publication of information articles in mass media and the organisation of seminars and conferences. Almost 38% of the promoters think that dissemination was assured by the publication of information materials in electronic form, while almost 22% of them have published training modules. Only 21.6% of the promoters have built up project websites.

• The most important level where the project results have been disseminated is the local one, as over 60% of the investigated promoters mentioned that they had carried out dissemination activities at this level. The dissemination at the regional level has been carried out only in 34% of the investigated projects. Only 15% of the projects involved national level dissemination.

PRELIMINARY CONCLUSIONS • The Phare HRD 2000 program stimulated the interest for the field of human resources

development and the participation of various organizations, most of which has a limited prior experience in the matter. The program represented a special opportunity for the participating companies, the percentage of which is dominant among the promoters. It is also remarkable that, of the total companies, the best represented were the small-sized companies, operating with less than 50 employees and for less than 10 years. This is due to the fact that, at project launching, the had rather limited funding in the field of human resources development.

• The partnership seems to be rather limited in terms of number and profile of partner organisations; while a significant number of projects failed to have at list one partner. Social partners have been involved in a low degree in partnerships. The situation had negative implications in terms of regional and local impact. The local public administration was the preferred type of organisation as project partner; taking into account also the share of projects promoted by local administration, it appears that this type of organisation is one of the main beneficiary of the program.

• In the case of all priorities, promoters demonstrated a low interest for addressing complex measures linked to well-defined target groups. Just a limited number of projects focused exclusively on active employment measures (objective 2) and social inclusion (objective 3); however it should be noticed that in the case of many projects surveyed, it was difficult to identify the main priority addressed. The analysis of the target groups has revealed some problems in explicitly defining them by project promoters, particularly the group of the disadvantaged persons (envisaged by the social inclusion priority), both at the project initiation and during their implementation. The number of unemployed and socially disadvantaged categories strategically targeted and benefiting from the HRD measures (training, counselling, mediation for job etc.) was lower than expected, this situation appears more clearly in the case of projects promoted by companies.

• Based on the evaluation results, we could appreciate that promoters’ participation in the program has been an opportunity for them to build their own institutional capacity, irrespective of the type of organization. They created a number of 131 new training centres, an output expected as stated in the project fiche being foreseen a minimum 14 modernised training centres. Assuring the viability and sustainability of the newly created centres, subsequent to the termination of funding, remains an open issue. In addition, beneficiary statements on the use of the new equipment in the training process draws a signal as to the degree in which such equipment has been efficiently put to use.



• The training measures have occupied a very important place within the PHARE HRD 2000 projects. Employees benefited the most from the training measures and the courses focused on competence upgrading or acquiring new competences and less on training of low qualified persons or disadvantaged target groups. The focus of training on employees with a higher education level is typical for CVT programmes within Romanian companies. Relative to this perspective, it can be stated that the program was less effective in reaching objective 3 – targeted at socially excluded persons.

• As a rule, the training fields envisaged by the projects correspond with some of the priorities for development at national level, such as services or fields undergoing reorganization at project launch (public administration, defence).

• As far as the impact of the training courses on the participants is positive – as shown in the statements made by an important part of them – (new competencies, career advancement and/or other changes, finding a job), we may appreciate course efficiency as relatively high.

• In terms of impact on training beneficiaries, program results have been positive. A significant number of the formerly unemployed and young graduates, who have participated in the training courses, found a job (temporary or for an unlimited period), and most of the employees have signalled significant changes in their activity or professional status.

• In general, the projects in the HRD component demonstrated an average capacity of fund absorption. This indicator helps to demonstrate, in some degree, the quality of the promoters’ selection process and of the stage of program awareness.

• The data gathering, recording, processing, and reporting system related to the implemented projects has posed major difficulties in the evaluation of the program impact. Therefore, the lack of information in some cases affected the relevance of the comparison or made impossible the calculation of key indicators for evaluation. In addition, the absence of synthetic input information made it necessary for the data to be collected directly from project promoters.

PRELIMINARY RECOMMENDATIONS • Paying special attention to partnership size in the evaluation process, while stressing the

involvement of the social partners in HRD activities.

• Modification of the project selection procedures, aimed at achieving a better equilibrium, function of the target groups (stronger emphasis on the priorities related to improving active employment measures and promoting the social inclusion of the disadvantaged groups.

• A more careful project selection and monitoring, with relevant indicators, which should allow for the reduction of the number of insecure projects in point of their viability and sustainability subsequent to the cessation of the external financing.

• Assistance should be granted all along project implementation, with a view to increasing fund absorption capacities and ensuring the centre viability and sustainability subsequent to the termination of the external funding. It should be examined further the opportunity to offer consultancy to project promoters in the area of financial management for project funded under European programs.

• The monitoring and evaluation systems specific to HRD Phare 2000 programme require in-depth revision for all levels of implementation (local, regional, and national). Specifically, the following areas should be considered:

- The monitoring and evaluation systems used at region of development level call for consistency, to allow for the aggregation of different statistical data and to make available the information on program implementation at national level. There is a need for diversifying the categories of information, increasing information accuracy,



and inclusion of relevant indicators in the project monitoring and evaluation, which should allow for the evaluation of the training course efficiency, by category of target group.

- Activities should be developed with a view to making promoters aware of their key role in project monitoring and in supplying accurate, detailed, and complete data on the chief objectives of their activities

- Developing a set of key indicators agreed by all stakeholders and designing of successive stages in data gathering should facilitate the calculation of such indicators, observation of progress make, and the evaluation of correction interventions at the critical time; emphasis should fall on the recommendations made during mid term evaluations.

- Devising new data gathering tools and procedures will permit accurate data collection, and identification and correction of erroneous data, as well; in point, the publication of resource materials (definitions of process and result indicators, ways of data gathering, etc.) will be appropriate. Development of a computerized system of data gathering and reporting of results concerning the monitoring and evaluation of the activities developed within the projects.

PROMOTERS RECOMMENDATIONS Based on research tools (questionnaires and interviews) it was possible to collect a set of recommendations stated by surveyed project promoters regarding the future HRD programs. The registered recommendations are related to the following issues: areas of intervention, implementation mechanisms, monitoring and evaluation systems and level of intervention. Areas of intervention for future HRD Programs The project promoters referred to the following recommendations on training activities that should be taken into consideration for future HRD programs:

- At regional level it is important to identify priority fields for training both on medium and long term and to take them into account in designing future call of proposals

- Creating new incentives for the participation of unemployed and disadvantaged categories to HRD projects.

The proposals made by the project promoters referred to the need to focus future programs mainly to specific target groups such as:

- population from rural areas; - socially excluded groups (with a special focus on institutionalised children), - youngsters; - public servants; - entrepreneurs; - unemployed (with a special focus on aged active labour force and on personnel with a

long history in state enterprises), etc. New fields proposed by project promoters are related to: the development of competences specific to the knowledge society, increasing the local capacity for initiating developing programs, communication/linguistic competences, transversal competences (i.e. conflict management, organisation of the working place), assessing the impact of economic activities on environment etc. Other proposals mentioned by project promoters are referring to a range of transversal priorities, such as:

- up-dating existing occupational standards, - improving the competences for labour market analysis and forecast, - implementing the aquis on specific activity areas, - diversifying assistance and expertise for SMEs in developing HRD systems.



Implementation mechanisms According to the promoters view, the involvement of RDAs implementation units proved to be a key factor for the success of the program. The future HRD programs should carefully assess the resources needed (both human and physical resources) for implementation units in order to assure a pro-active approach in assisting project promoters. The proposals made by investigated project promoters concerning the implementation mechanisms are referring to the following issues: Existing procedures

- keep the same procedures during the life-cycle of a project, - submitting the narrative and financial reports at local level and receiving feed-back in a

shorter period of time; - increasing the responsibilities of RDAs; - better communication between contracting and implementing authorities; - clearer indications on timeframes required in answering to notifications made by project

promoters); - better correlation between local, regional and national administrative procedures; - simplifying the acquisition procedures, eliminating the request for an open bid in the

case of public acquisitions with a value of less than EURO 1500; - reducing the number of copies requested for reports; - reducing the time allocated for project evaluation/notification/contracting

Partnership

- motivating the local authorities for a more active involvement in activities of the projects - developing management capacities for partners identification and activation

Assistance to project promoters

- management, reporting, clarifications on contracts, financial issues, acquisition rules, monitoring and evaluation etc.;

- guide for HRD project promoters (including best practices examples); - a higher involvement of RDAs in all the stages of the project and a more in-depth feed-

back during the monitoring in situ visits; - national and regional meetings with project promoters as arena for innovation and

transfer of successful experiences; Participation of public institutions to Phare programs

- changes in legislation in order to the public servants to be paid for management services;

- national database with information on organisations that are legally allowed and can secure finance for own contribution

Assistance in accreditation of the courses

- for the Phare projects developed within the HRD program the existing regulations related to accreditation should be more flexible;

- detailed and up dated statistics on LLL should be produced and available; - assistance in better correlation of local development strategies and project activities)

Recommendations on monitoring and evaluation

- Communication with monitoring institutions should improve. - Monitoring should take place at intervals specified in the contract, based on the

technical and financial reports prepared by the beneficiary. - The use of electronic forms should expand. - The setting up of local monitoring units should involve the direct beneficiaries.



- Agency evaluators should estimate the degree at which project objectives have been achieved and feed back regarding the priority intervention areas in the period left should be ensured.

- A commonly agreed on framework model should be observed in the case of reports (e.g., the use of standard forms).

Proposals concerning the level for future HRD programs (local, regional, sectoral) made by project promoters The majority of project promoters stated that the local level should be in the future the main approach of HRD programs, in close relation with the local development programs. Even in the case of regional approaches, the main aim should be creating and developing competences related to the introduction of new technologies in ”winning” economic sectors and mobilizing these resources for new regions. The justification for choosing the local level as main approach is based on a justification stating the existing local needs and management capabilities. The difficult cooperation with national authorities in implementing regional or sectoral projects is another difficulty mentioned by project promoters. The micro-regional approach (division of a region in a detailed set of sub-regions) is proposed as a viable alternative to the existing approach. Other proposals made by project promoters Besides the proposals already mentioned, the project promoters also referred to:

- better information of potential project promoters concerning the specific objectives and measures taken into account by the programme;

- the guide for applicants should contain clear and explicit information on procedures, not allowing interpretations (avoiding general statements);

- organisation of Q&A sessions before the official date for submitting proposals (pre-evaluation programs);

- more active role of the regional training centres (public); - creating incentives for attending training courses for specific categories of beneficiaries

(i.e. youngsters, disadvantaged population etc.); - all official documents to be translated also in Romanian language; - the evaluation process should be more transparent and the evaluation outcomes

should be in detail presented to both winning and losing applicants; seminars with unsuccessful promoters;

- a higher percentage allocated to equipments, taking into account that the competences are in relation with modern technologies, expensive technologies

- in cash contribution to be reduced; - reimbursement of training participants when the training is organised more than 100km

away form the trainees place of residence;



ANNEXES

Annex 1. Methods of investigation The evaluation of the HRD Component of the PHARE 2000 ESC Programme has taken place based on a complex methodology, with the use of diverse investigation methods.

a) Analysis of the documents related to the HRD projects available from the Ministry of European Integration of Romania, as follows:

- Regional Development Agencies reports on final evaluation; - List of cancelled projects in the programme; - Synoptic table on the Phare 2000 RO 0007.02 budget execution; - List of the synthetic project indicators; - Other evaluation reports on HRD Phare component (1998, 2000).

Document analysis has helped in gathering information on the:

- number of promoters and type of organisations they represent; - total number of developed projects and number of projects with cancelled contracts

and reasons for cancellation; - level of project funding (full amount, Phare funding, joint funding); - number of trained applicants, number of organised training courses, and number of

training centres set up during the projects; - quantity of equipment purchased; - number of new jobs set up, etc.

b) Quantitative methods of investigation:

• Questionnaire mail survey of project promoters

The questionnaire1 survey has aimed at rendering evident on:

- the promoters’ main characteristics (type of promoting organisation, promoters’ geographic distribution, and field of activity),

- the type of developed projects (size of project, function of budget; involved partners; priorities and measures envisaged; categories of beneficiaries);

- the training activity developed in the project framework and resources (training tenders and costs, equipment, development of institutional capacity);

- the training activity impact over the promoting organisation (development of main quantitative and qualitative indicators at company level, degree of project impact, impact over the company system of human resources management);

- ways of dissemination and recommendations on future ESF HRD projects.

• Questionnaire mail survey of the direct beneficiaries of HRD projects Aiming at a higher degree of credibility and objectivity of the conclusions, the evaluation also investigated the direct beneficiaries of the projects, that is, the participants in the training projects developed within the projects. The information requested the direct beneficiary to indicate 1 The data collected trough questionnaires intended to cover the missing information on promoters characteristics received from Ministry of European Integration



- general characteristics (age, sex, level of education, occupation prior to participation in the training course, type of certification issued by the training course);

- characteristics of the training courses (length of course, theme, qualification obtained); - opinion about the training courses (correlation with individual and professional needs,

quality of trainers and efficiency of teaching methods, quality of infrastructure, efficiency of the practical training);

- impact of training (career follow-up and quality of activity, over one’s occupational status, convergence between acquired competences in the training course and competences demanded at work after participation in the training project).

c) Qualitative methods of investigation

• Interview survey of project promoters The operators conducted individual interviews aimed at finishing the information collected through questionnaires. The interviews insisted on the following aspects:

- project initiation and development (motivation to participate in the project; expectations about the project impact over the development of the organization; estimation of applied procedures and assistance received in developing the projects; ways to identify target groups and their needs; ascertaining the directions for action and measures corresponding to identified target groups and their needs);

- project progress (opinion about the role and contribution of project partners; - preparation of tenders for training; - opinions about the difficulties come across during project development, particularly about

the training activity; - monitoring mechanisms used in training activities and appreciation of the co-operation with

Regional Development Agencies (RDAs)/central administration representatives during ongoing projects, of the monitoring mechanisms used by the latter, and of the intermediate and final report drafting in the project;

- judgment about the causes having determined drop-out situations at organised training courses);

- project impact (estimates on project effects over the organization and target groups; appreciation of the degree at which project priorities have matched the socio-economic realities specific of the project area; project added value in relation to the needs of the target groups; evaluation criteria for the project and training courses; dissemination impact of project outcomes and recommendations on the development of future HRD structural projects).

Sample of the promoters The survey of the project promoters was exhaustive, as each of the promoters received a questionnaire. Out of 371 promoters, 157 answered the mail investigation (initially, 120; later on, 37 more). On-site operators also succeeded to investigate 14 more promoters. Thus, evaluation operated on 171 organizations (46.1% of the total) that have developed projects. These organizations promoted 185 out of 393 projects developed in the HRD component (23 of the original 416 projects have been cancelled), representing 47%. A phone investigation conducted in the case of 40 promoters that have failed answering the survey (they were registered in the MEI database) has revealed the main reasons why they had failed to answer, namely:

- Promoter ceased activity (2 cases); - The financing contract was cancelled (5 cases);



- Lack of time resources needed to fill in the questionnaire (19 cases); - Difficulties in gathering the information requested by the questionnaire due to the

project manager’s leaving the institution, to communication deficiencies with the team having implemented the project at the local level, etc. (6 cases);

- Obstacles when filling in one of the items (the item requesting coordinates of the participants in the training schemes) (12 cases).

The operators conducted individual interviews with 30 promoters, selected on the following criteria: development region, type of organization, project size. From 30 interviewed promoters, 16 are respondents to the mail survey, and 14, non-respondents. They were selected from the counties of Alba, Arad, Călăraşi, Cluj, Constanţa, Dolj, Neamţ, and the City of Bucharest, as these represented all the development regions. Sample of the project target groups The selection of the sample of persons representing the target groups for the HRD projects used the lists with the participants in the training programmes developed in the framework of the projects. The lists were available from the respondents to the mail survey. The total number of persons indicated amounted to 5,007, of which only 3,728 had full contact data. Of these, 2,302 were employed before the training courses, 810 were unemployed, and 616 belonged to socially disfavoured groups (young graduates, former foster home youth, persons of rroma origin, disabled persons, former detainees, long-term unemployed, etc.). The selection criterion was the category of beneficiaries: employed / unemployed / disfavoured categories. From among the employed category, 576 persons were selected at random, at a pace of 4; from among the unemployed, 607 persons were selected, at a pace of 2, in two successive selection stages; from the socially disfavoured category, all individuals, 6162, indicated by the promoters were retained for the sampling. The resulting sample counted 1,799 subjects and consisted of 3 sub-samples, of a relatively equal size. The 1,799 individuals (55% women and 45% men) were mailed a questionnaire. The number of responses received by return mail was 566. To increase the answer rate, in a second stage, operators applied group or individual questionnaires to 256 persons. Thus, 822 participants in training programmes (64,5% women and 34,5% men) were subject to evaluation. Thus, 822 participants in training programmes were subject to evaluation out of which 551 are employed and entrepreneurs (67%), 125 are unemployed (15,2%) and 146 are disadvantaged persons (17,8%). The number of respondents represents 45,7% of the total selected sample.

Total number of

beneficiaries indicated by the

promoters

Total number of beneficiaries indicated by the promoters with

completed contact details

Number of beneficiaries

selected in the sample

Number of beneficiaries

answering the questionnaire

Employed 2302 576 551

Unemployed 810 607 125 Disadvantaged persons

616 616 146

Total 5007 3728 1799 822

2 According to terms of reference, the survey on beneficiaries foreseen a sample of 1800 persons, 600 persons for each target group.



Annex 2. Methodology of the data statistical analysis Techniques of statistical processing

The techniques of statistical processing have been determined by the nature of the research, which belong to the summing up evaluation rather than to a cause-effect relation.

Under these conditions, depending on the nature of the analysed variables, we have determined:

For the qualitative variables (nominal or ordinal), the most numerous in the case of a questionnaire:

- The Occurrence rate of the concerned variants, as well as the structure of the responses (percentage);

- The Distribution of the results in the contingency tables;

- The Comparative evaluation of the distribution of the main indicators according to the two basic types of promoters, namely companies and other categories of promoters.

For the quantitative variables - concerning the material and financial resources of the project (the budget and its main expenditure items), the number of beneficiaries of the training programmes and number of the training courses organised – the following have been determined:

- total and average levels by categories, as well as the minimum and maximum levels;

- structure indicators concerning the number of beneficiaries, by categories, including the expenditures by main budget categories and within a category;

- derived indicators representing unitary levels, such as: average cost by category of beneficiaries, average duration of a course (average number of days) by type of course or fields in which it was organised, etc.

- bi-varied correlation between the distribution by the two categories of promoters and number of courses they organised, and between the two categories of promoters and the total number of training courses beneficiaries.

The database and its processing were carried out in EXCEL and at the same time a SPSS database was created by export. As for the SPSS programme a short analysis (syntax) programme was designed for the main analysed indicators.

Characteristics of the database

The statistical analysis was carried out in two distinct databases: the main database which registers all the participating promoters and their characteristics (“PROMOTOR.sav” 185 entries) and a base derived from the complex QP19 item (QP19-program.sav”- 454 entries), designed to analyse the training programmes. This registers all the training programmes specified by the promoters with a reference to the promoters, who organised the course and some of its basic characteristics (taken from the main base).

The PROMOTOR.sav database includes information concerning the response variants to the items included in the questionnaires, as well as re-coded variables and derived indicators.



The second base includes processed information from the main base concerning the promoter which organised the course, as well as information on the training programme such as: themes, field of activity and the educational level addressed, duration of the course (in hours and days, generally 6 hours /day and the estimation in number of weeks).

Notations used

For an easy identification of the variable in the database with the questionnaire item it refers to, the following notation was used: qpXX, where XX is the item represented in the base.

For the items with multiple answers, each variant was considered as a binary independent sub-item, and the following variants were assigned:

- 1 – for the presence of the answer; - 0 – for the non-selected variants of the answer

In this category of items the notation qpXX_1, qpXX_2 etc. have been used in all possible variants of answer.

For the derived indicators the generic notations qpXX_YY have been maintained, followed by the YY notation, which indicates the nature of the new indicator.

In order to highlight the filter questions, to the indicators variants of the database a coded variant named 8 was added (or, depending on the number of variants: 88 / 888 / 8888), which represents the “non applicable” situation. The lack of answers was marked by code 9 (99 / 999 / 9999).

In defining the variables within the SPSS database, non-responses have been included in the list of the variants under “missing values” with the “non applicable” code. This distinction was operated in order to clearly identify the structure of the variants and the share of the non-answers.



Annex 3. Report of the Seminar “Impact Evaluation of Phare HRD 2000 Scheme”

INTERNATIONAL SEMINAR

“Impact Evaluation of Phare HRD 2000 Scheme”

Bucharest, 31 January, 2006

- Report of the Seminar -

The seminar was organized by the Romanian National Observatory in cooperation with the Delegation of the European Commission in Romania and the European Training Foundation.

The seminar aimed at presenting the findings and recommendations of the report prepared by the evaluation team at the end of the ex-post evaluation of the HRD projects funded through the Economic and Social Cohesion sub-programme of Phare 2000. The objective of this seminar was to verify and validate these findings and recommendations within the discussions and debates, with the view to support the evaluators’ team in refining the operational recommendation proposed for the implementation and better evaluation of the future HRD grant schemes. These debates were structured around four areas identified by the evaluators:

A. Social partnership and the sectoral dimension of the future HRD programmes.

B. Defining priorities and target groups, especially the disadvantaged ones. C. Absorption capacity for the future ESF fund and the projects sustainability. D. Monitoring, evaluation and reporting systems.

The seminar was attended by representatives of Ministry of European Integration and Regional Development Agencies, Ministry of Labour, Social Solidarity and Family, National Agency for Employment, National Council for Adult Training, National Center for VET Development, social partners. The seminar was also attended by officials from the Delegation of the European Commission in Bucharest, European Training Foundation from Torino and the international expert who coordinated the activities of the Observatory’s evaluators as well as the Team Leader of the Phare project advising the Ministry of Labour on the implementation of employment policies for ESF. The summary of the ex-post evaluation report of the 2000 HRD Phare projects and a questionnaire were sent in advance to all participants which included the issues to be discussed during the working groups. The agenda, the list of participants and the final version of the questionnaire are appended as Annexes 1, 2 and 3. After the presentations given by the evaluators’ team and the international expert who coordinated them, two working groups were established with participants in the seminar to



debate on the main findings and recommendations of the evaluation reports In order to facilitate the discussions in the working groups, the participants were handed a copy of executive summary of the ex-post evaluation of HRD Phare 2000, copies of the presentations made and a list of questions prepared to guide the discussions. The facilitators and the reporting persons of the two groups were selected from among the participants. This report summarises the conclusions and recommendations made by the participants during the seminar. A. Social partnership and the sectoral dimension of the future HRD programmes. The debates and recommendations in this are followed three directions:

1. There is a need for awareness on the role of the social partners in the whole process, including in the implementation of the grant schemes as they can play roles in various stages of the grant schemes management.

They can be involved in the identification of the needs at local, regional or sectoral levels through the use of the information existing with some of the social partners.

The information and communication channels of the partners could also be used to support the dissemination and promotion of the grant schemes (sectoral committees, representatives in various committees and commissions at local and regional level). They can take part in the preparation and dissemination materials for good practices, such as catalogues of good practices, success stories brochures.

2. Large projects, with higher budgets, more regional impact and

involvement of social partners are more important and should be encouraged. Projects bringing together more companies from the same sector and bringing also a social partners could produce more effects than a small project.

3. Better prepared and conducted awareness campaigns for grant schemes

to ensure that sufficient information and/or clarifications is provided to potential applicants by the regional helpdesks and or other relevant information events. Staff of the helpdesks should be well informed and trained and procedures should be in place on the functioning of the helpdesks to cover also the situations when the required information or support exceeds the competency of the staff.

With this view the contracting authority should prepare clear dissemination strategy and procedures.



B. Defining priorities and target groups, especially the disadvantaged ones.

1. There is a need for a better analysis and selection of eligible measures and target groups to ensure they cover also the inclusion of the disadvantaged groups and their needs.

2. The local public administrations (e.g. town halls) could be also involved in

the identification of the target groups and their needs as they have available information on the disadvantaged groups, which could be used for needs identification.

3. Separate measures are recommended for specific disadvanged groups,

especially for those who need to develop base competencies.

4. Integrated measures should be encouraged to increase the chances of the final beneficiaries to develop their existing skills or to acquire new skills so that they can access new jobs according to the labour market demands.

5. Simplification of the selection and evaluation procedures for grants is

limited by the constrained imposed by the Practical Guide. However the good knowledge and understanding of the procedures by the people involved in the grant schemes management will allow them to prepare tools (eligibility criteria and scoring system) for reaching the objectives of the grant schemes, tools for evaluators to ensure a fair and transparent evaluation and selection of thright projects and also for the monitoring and evaluation process.

C. Absorption capacity for the future ESF fund and the projects sustainability. Three elements have been considered:

1. An important aspect of the absorption capacity through the prospective of the ESF is the financial implication considering the payment system in the structural funds. In this case the payment is made through the reimbursement system, meaning that the applicant must advance the funds to cover for the costs of the projects. Therefore this will allow only the organizations with a very good financial situation to apply for such funding with direct impact on certain sectors such as education and HRD, where organizations with less financial capacity are involved (e.g. public institutions).



The experience of other European countries could be used to find solutions and prepare in time. An example is the Irish solution where the state got involved and created national funding systems to advance the necessary funds to the applicants for the implementation of the projects.

2. Types and size of projects are also elements to consider when analyzing the

absorption capacity and projects sustainability. Question was raised whether large projects and accredited organizations should be encouraged as they would ensure more absorption and sustainability of the funded actions. In such a case there is still the issue of the new entrants on the market - the new organizations. Also if we encourage large projects the needs of the smaller target groups at regional and local level would remain unsolved. How do we solve this?

Therefore in order to ensure a better absorption capacity for future ESF funds and projects sustainability there is a need for a balanced combination between large and small, national, regional and local projects according to several factors:

• type and capacity of promoters • location: regional and local according to needs; • target groups

Larger projects could be prepared by bigger organizations with more capacity and sufficient experience and expertise able to address larger target groups or projects of national or regional interest. Therefore, assistance should be provided for such structures to consolidate. Smaller projects can be prepared by smaller organizations even not accredited but encouraged to get accreditation during the project implementation. This could solve the needs of smaller target groups and enable new organizations to develop.

3. More intense promotion campaigns should be conducted by the funding

institutions and more support given to project promoters for preparing better quality project proposals. This would ensure a higher number of applications submitted and the reduction of the level of drop-outs during the contracting and implementation stages.

Particular attention should be paid however when supporting potential applicants in preparation of application in order to prevent infringement of important principles such as competitiveness and avoidance of conflict of interest. Therefore this kind of support should be organized neutrally, separately from the specific grant schemes.



D. Monitoring, evaluation and reporting systems. A monitoring system involves procedures, rules and conventions which make it possible to produce and circulate indicators and qualitative information on the implementation of the programme. HRD programmes require a monitoring system different from the standard Phare monitoring as regards the indicators: indicators are needed on individuals and project promoters. The contact information such as postal address, telephone number, e-mail address, social security number or identity code are important for tracing the beneficiaries after the training or counselling has taken place for the purpose of ex-post impact evaluation. Personal information such as age, gender; status on the labour market and ethnic group are also necessary. The feasibility and success of mid-term and ex-post evaluations depends on the monitoring system providing the necessary information. A monitoring system is a crucial source of information for evaluations. Therefore in addition to the information collected for management purposes (corrective actions), the monitoring system should be designed to allow for the collection of information which will be used at a later stage to assess the effectiveness, impact and sustainability of interventions. It is important that a monitoring and evaluation system is built in to the programme during the preparation stage. A list of indicators and the calculation methodology is to be defined during the preparation stage of the grant scheme when the objectives and the eligibility criteria are established (financial allocation, eligible measures, eligible applicants and partners, target groups). In order to ensure the ownership and compliance in gathering the necessary monitoring information, these indicators should be communicated to the project promoters both at the application stage and at the beginning of implementation of their projects: - during the application stage: - list of indicators included in the Guidelines for applicants; - indicators explained to potential applicants during the awareness campaign; - during the implementation:

- training of the project promoters on the obligation and use of the monitoring system

- type and frequency of the reports to be produced - as a first step before an on-line system, electronic report format and database

format (preferably in or transferable to Excel) to be provided to project



promoters in order to facilitate the integration of data by the PIUs and afterwards by the monitoring unit of the Ministry of Labour

- on-site - visits The monitoring procedures developed by the MIE and taken over and adapted by the MoLSSF are an important tool in the hand of the PIUs for better monitoring and supporting the project promoters in the proper implementation of their projects. These will allow for corrective actions by the project promoters, as identified together with the PIU’s monitors, to ensure the achievement of the stated objectives and results and at the same time they will give the monitors the necessary tools for actions in case of irregularities they find out ( e.g. during the on-site visits). Therefore it is very important that the MLSSF and the PIUs revise these procedures according to specific needs of the grant schemes and also amend them on the basis of lessons learned from the experience of implementation of previous projects. The findings of evaluation processes like this one are important information which the MLSSF can use to both amend the monitoring procedures and better prepare future HRD grant schemes. E. OPEN ISSUES The conclusions of the debates and the recommendations of the two groups were presented in a final plenary session and further debates took place with new ideals and proposals made. These proposals have been structured and summarised in Annex 3. of this report.



LIST OF PARTICIPANTS

No. Surname and Name Institution 1 Carmen NECŞESCU Ministry of European Integration 2 Marieta Enache Ministry of European Integration 3 Dana Sardarescu Ministry of Labour, Social Solidarity and Family 4 Aurica Leanca Ministry of Labour, Social Solidarity and Family 5 Cristina Mereuta Ministry of Labour, Social Solidarity and Family 6 Paulo Pedroso Team Leader - TA project to Ministry of Labour,

Social Solidarity and Family 7 Catalina Iacob National Agency for Employment 8 Madlen ŞERBAN National Center for VET Development 9 Daniela Bucsa National Center for VET Development 10 Douwe GRIJPSTRA Policy Research Netherlands 11 Simona Popescu Agency for Qualification and Partnership with the

Social and Economic Environment 12 Ana Elena COSTIN National Council for Adult Training 13 Victoria CAPRINI National Council for Adult Training 14 Gabriela Popescu RDA East 15 Ion Crangasu CIROM 16 Marius Staicu RDA Bucharest- Ilfov 17 Simona Gatu UGIR 1903 18 Ion Pirnã ACTMAR 19 Mihail Munteanu Sectoral Committee Construction 20 Mihaela JIGĂU Romanian National Observatory 21 Magdalena BALICA Romanian National Observatory 22 Ciprian FARTUŞNIC Romanian National Observatory 23 Lucian VOINEA Romanian National Observatory 24 Septimia DOBRESCU European Commission Delegation in Romania 25 Arjen DEIJ European Training Foundation 26 Outi Kärkkäinen European Training Foundation 27 Zamfira Balan IDCO Training& Consultancy 28 Marilena Tiprigan Interpreter



QUESTIONNAIRE FOR DEBATES

Subjects for debates in Group 1: A. Social partnership and the sectoral dimension of the future HRD programmes.

B. Defining priorities and target groups, especially the disadvantaged ones Questions for debates: 1. Which measures could we develop to assure a better correlation between the

training needs at the local and regional levels and the training offer of the HRD programs;

2. How could a more active participation of the social partners in the HRD programs be enhanced?

3. What kind of measures and which changes in the selection and implementation procedures of the projects can we develop so that the disadvantaged categories benefit more from the future programs?

4. What kind of measures and which changes in the selection and implementation procedures of the projects can we develop so that the target groups benefit of integrated measures (training, counselling, mediation, carrier plans etc.) with an emphasis on unemployed and disadvantaged categories?

5. How could the campaigns of promoting contracting, implementing and monitoring procedures at the local and regional levels become more efficient?

6. How can we assure that the new project promoters will be able to learn from the experience of the previous projects.

Subjects for debate in Group 2: C. Absorption capacity for the future ESF fund and the projects sustainability.

D. Monitoring, evaluation and reporting systems. Questions for debates: 1. What types of projects will be needed in the future in order to ensure the

absorption of the future funds? Would they be necessary projects 10 times larger or 10 times more projects of actual size?

2. Which measures for increasing the absorption capacity of the funds promoted from the ESF perspective can be promoted?

3. What kind of information and indicators could be developed in view of monitoring, evaluation and reporting the results of the future programs?

4. Which monitoring measures could we develop to increase the degree of equipment use in the training activities and to assure the match between the purchased equipment, the contents of the training offer and the trainers’ training?

5. What recommendations can you make in the light of the experience of the evaluation of Phare 2000 for the evaluation of Phare 2002.

SUM

MAR

Y OF R

ECO

MM

END

ATION

S FOR

FUR

THER

EVALUATIO

NS AN

D FU

TUR

E PRO

GR

AMM

ES

Evaluation of Phare HR

D

2002 G

rant scheme Phare

HR

D 2004

Grant schem

e Phare 2005/2006

TA to MLSSF

“ Support to MLSSF to

design and implem

ent em

ployment policies”

ESF

SOU

RC

ES OF

INFO

RM

ATION

•

training providers to be checked in the N

ATB

register of accredited training providers;

• can w

e check into the N

AE

whether sam

ple of beneficiaries for priorities 2/3 is on register?

• for control groups ask N

AE

to provide contact data – a m

uch larger control group ( 3000 persons) so that w

e can tailor it to the characteristics of the responding beneficiaries

• M

EI/ RD

As - additional inform

ation requested from

the projects im

plemented for P

hare 2002. N

ational

GU

IDELIN

ES FOR

APPLIC

ANTS

to consider the following

elements and create the

necessary tools: Training providers: •

enhance capacity of existing providers ( see 2002 evaluation questions);

• how

can we collect

information on the

existing training centers and use it during the evaluation process to better assess their capacity

3 •

build capacity at regional and sectoral levels

• partnership required; no longer projects just for one com

pany to train

• H

ow can w

e use the existing experience of the H

RD

grant schemes in

the planning exercise? Looking at w

hat has w

orked and what has not

worked;

• U

se network of prom

oters of 1998, 2000, 2002 and 2003 for planning local and/or regional partnerships;

• A

nalyse the role of the sectoral com

mittees in

assisting existing providers to im

prove the provision of training;

• U

se the conclusions of evaluations of P

hare H

RD

2000 and 2002 (second part of 2006) to identify the degree of integrated m

easures

Proposals m

ade by the TA

projects on: -

need for a clear vision on the learning experience

- quality of m

easures more

important than the norm

al speed w

hen evaluating and adjusting program

mes –

employm

ent culture -

development/im

provement of

a system of structures

- tripartite training centers;

- institutionalised partnerships;

- the need for a continuous evaluation on a sam

ple w

hich can input results in an ex-post evaluation

Establish a m

eeting between the

evaluators team and the TA

How

can we use the

existing experience of the H

RD

grant schem

es in the planning exercise?

3 Type of inform

ation to be collected: •

Capacity: how

many people they can train, sectoral profile coverage, geographic coverage;

• S

ustainability: how m

any contracts they have for training and expectations; existing links with the sector (e.g.: sectoral com

munities, social partners,

companies from

the sector, local authorities) •

Resources: how

many perm

anent teaching staff they have; how m

any temporary staff; w

here the temporary teaching staff is com

ing from; links w

ith training providers such as universities, com

panies, VE

T schools; •

Quality: accreditation of organization and/or training program

mes – under w

hat names; quality assurance certifications (e.g. IS

O)

Observatory w

ill send M

EI/R

DA

s an additional sheet for collecting contact details for final beneficiaries.

ADD

ITION

AL EVALUATIO

N

Activities •

degree and quality of integrated m

easures; •

competent institutions;

• staff involved;

• institutional form

s of the activities/link w

ith strategic developm

ent Final beneficiaries The low

skilled: •

what w

as their participation in the program

me?

• in w

hich type of activities? •

how effective w

here these in com

parison to the com

plete “population” of beneficiaries?

• dealing w

ith non-response of beneficiaries and see the characteristics of the respondents in com

parison to the responding beneficiaries using the N

AE

register. Training providers •

are they existing or new

institutions?

their own staff.

• need to dem

onstrate partnership w

ithin sector in term

s of several com

panies from one

sector/and/or links with

specialized social partner organisations, sectoral com

mittees, or

sectoral trade unions, or em

ployers organisations or professional associations

To consider: •

how can w

e improve the

roles played by the sectors?

• can w

e make a link

between com

panies w

ithin a sector and how?

• how

can we link them

to other social partners?

• could w

e consider sectoral training funds?

Establish and include list of

indicators in the Guidelines

(as annex) AW

AREN

ESS CAM

PAIGN

•

MLS

SF to establish

procedures and organise training of the helpdesks staff and experts for correct inform

ation/clarification/ support to applicants;

implem

ented and analyse and decide on the approach/tools to be used to encourage the projects proposing integrated m

easures (eligibility criteria and/or additional scoring)

• how

are they improving

their capacity? •

distribution in terms of:

- priorities,

- geographical coverage, types of program

mes,

links with sectoral

representatives ( e.g. sectoral com

mittees,

social partners, professional associations, com

panies/ from

organisations from a

specific sector) -

accreditation (accreditation register, other certifications),

- staff, size,

- links w

ith the world of

work.

Examples of good practice

• Identify a num

ber of exem

plary projects that can be used to prepare and prom

ote ES

F projects.

• m

onitoring indicators explained to potential applicants during the aw

areness campaign;

MO

NITO

RIN

G SYSTEM

( including procedures) -

adapt the monitoring

system to collect all the

necessary information

for both monitoring and

evaluation; -

train the PIU

s staff in using it;

- prepare m

anual for project prom

oters and training on the obligation and use of the m

onitoring procedures -

as a first step before an on-line system

, electronic report form

at and database form

at (excel?) to be provided to project prom

oters in order to facilitate the integration of data by the P

IUs and afterw

ards by the m

onitoring unit of the M

inistry of Labour

Annex 4. Additional tables and charts

Table 1. Data on estim

ated and accomplished outcom

es of Phare 2000 HR

D, provided by M

EI

trained people/no. training courses/no.

training centres/no. equipm

ents/no. new

created jobs/no. office equipm

ents/no. D

evelopment R

egion estim

ated accomplished estim

atedaccom

plished estimated

accomplished estim

atedaccom

plished estimated

accomplished

estimated

accomplished

RD

A 1 North East

7033 3441

186 125

37 22

563 387

861 284

563 387

RD

A 2 South East 6336

5364 46

46 5

5 622

601 23

23

RD

A 3 South Muntenia

5887 7254

150 188

10 10

1100 1119

140 117

340 346

RD

A 4 South West

2135 2288

49 57

13 13

261 198

217 183

RD

A 5 West

5417 5307

112 109

17 18

215 215

39 44

RD

A 6 North W

est 8106

7686 228

228 53

53 418

417 218

96 166

166

RD

A 7 Centre

7413 7608

77 144

10 9

62 62

698 304

RD

A 8 Bucharest Ilfov

5026 3510

52 47

2 1

527 495

200 100

TOTAL

47353 42458

900 944

147 131

3768 3494

2396 1151

1069 899

CHART 1. Project distribution by type of promoter (sample projects)

30%

24%17%

10%

9%10% Economic agents

NGOs


Pre-university educationinstitutionsUniversities


Source: Data calculation based on information supplied by the MEI

Table 2. Distribution of training courses, by the goal and duration

hours days Qualification / reconversion

Professional development

Total

6-18 1-3 0,0% 1,5% 0,5% 19-60 4-10 1,5% 19,0% 7,4% 61-120 11-20 0,7% 22,5% 8,2%

121-360 21-60 31,9% 40,4% 34,9% 361-720 61-120 55,9% 12,5% 41,1% over 720 over 120 9,9% 4,1% 7,9%

Total 100,0% 100,0% 100,0%

Table 3. Average duration of a course according to the field of economic activity

Average duration of a course (hours/course)

Standard deviation

vo

catio

nal

trai

ning

/re-

conv

ersi

on

Voca

tiona

l de

velo

pme

nt

Tota

l

voca

tiona

l tr

aini

ng/re

-co

nver

sion

Voca

tiona

l de

velo

pme

nt

Tota

l

Agriculture, game and forestry

304 390 322 107,63 35,4% 390,92 100,2% 208,68 64,8%

Processing industry 409 63 234 158,41 38,7% 64,31 102,1% 210,73 90,1% Electric and thermal power, gas and water

551 255 433 66,46 12,1% 225,00 88,2% 209,70 48,4%

Constructions 492 360 482 185,22 37,6% 0,00 0,0% 181,68 37,7% Sale and repair of cars and household goods

357 131 263 70,76 19,8% 117,53 89,7% 144,99 55,1%

Hotels and restaurants

470 114 347 339,76 72,3% 129,26 113,4% 331,52 95,5%

Transport, warehousing and communications

360 395 378 0,00 0,0% 325,00 82,3% 230,48 61,0%

Financial transactions

452 144 200 52,00 11,5% 79,17 55,0% 140,47 70,2%

Real estate and business transactions

- 99 99 99,99 101,0% 99,99 101,0%

Public administration and defence

361 67 105 139,93 38,8% 74,33 110,9% 131,10 124,9%

Education - 102 102 84,11 82,5% 84,11 82,5% Health and social protection

316 92 192 238,89 75,6% 64,72 70,3% 200,23 104,3%

Other services. 278 130 205 229,27 82,5% 153,91 118,4% 209,33 102,1% Total 374 97 190 217,13 58,1% 121,51 125,3% 206,37 108,6%

WP 2005-11-3 Institution BuildingSupport the development of local capacity

to evaluate ESF types of activities

IDCO Training & Consultancy24.03.2006

The content of this material does not necessarily represent the official position of the European Union

the ex-post evaluation of the pre-esf 2000 hrd grant schemefile/note6n4cgm.pdf · the ex-post...

Documents