1

CRITERIA FOR MEASURING ADULT EFFECTIVENESS FOR DISADVANTAGED GROUP

IMPADA PROJECT – OUTCOME 1

Contents1 Introduction.......................................................................................................................4

2 Analytical framework.........................................................................................................5

2.1 Key definitions............................................................................................................5

2.1.1 Adult education...................................................................................................5

2.1.2 Effectiveness of (adult) education.......................................................................5

2.1.3 Disadvantaged groups.........................................................................................6

2.2 Identification of disadvantaged groups......................................................................6

2.2.1 What do we know about PIAAC?.........................................................................6

2.2.2 Short typology.....................................................................................................8

2.2.3 Different approach to different disadvantaged group?.......................................8

2.3 Methodology of research...........................................................................................8

2.4 Comments on desk research......................................................................................9

2.5 Comments on focus group interviews......................................................................10

2.6 Comments on documentation analysis....................................................................11

3 Different outcomes of (adult) education.........................................................................12

3.1 Economic outcomes of adult education...................................................................12

3.2 Personal and social returns......................................................................................14

4 Factors influencing the effectiveness of adult education of disadvantaged groups........17

4.1 External vs. internal factors influencing effectiveness..............................................18

4.2 Participation in adult education...............................................................................19

4.2.1 Promotion.........................................................................................................20

4.2.2 Recruitment.......................................................................................................21

4.3 Persistence in adult education.................................................................................22

4.4 Quality of provision..................................................................................................26

4.4.1 Length of a course.............................................................................................28

4.4.2 Organization of learning....................................................................................28

4.4.3 Teachers quality................................................................................................30

4.4.4 Other factors.....................................................................................................32

5 Methods of assessment...................................................................................................33

6 Criteria on measuring adult education effectiveness.......................................................43

2

6.1 Criteria oriented directly on the expected outcomes (individual perspective):.......43

6.1.1 Objective measures...........................................................................................43

7 Key conclusions................................................................................................................51

8 Bibliography.....................................................................................................................52

3

1 Introduction

This report presents a review of findings from studies and publications about the effectiveness of adult education for disadvantaged groups. The main goal of the study was to recognize key criteria for measuring the effectiveness of adult education. Therefore, we focused the review around two questions: (i) what does it mean that education is effective and (ii) what are the ways to measure it. Answering those questions will help to understand the challenges of adult education for disadvantaged groups and – in consequence – to develop proper solutions of improving effectiveness of this education.

This report is first outcome of IMPADA project - Improving Adult Education of Disadvantaged Groups. The project aims to develop a Common Assessment Framework to evaluate the effectiveness of adult education for disadvantaged. Such Framework will help adult education providers in self-assessment, self-evaluation and – in consequence – improvement of provision of adult learning services for disadvantaged groups.

The report summarises two activities within the project:

A1 Desk research on factors that affect the effectiveness of adult education for disadvantaged groups

A2 Focus groups with adult education stakeholders aimed at identification of criteria influencing the effectiveness of AE for disadvantaged groups

First part of the report presents the definitions of key terms and methodology of the research. Second part presents considerations about how to understand the statement that the education is “effective” and general findings from the research on effectiveness of adult education. Third part presents key factors affecting the effectiveness of adult education disadvantaged groups, grouped in three areas: participation in adult education, persistence and outcomes of learning.

As seen in chapter 2.1.1, in the project adult education is defined as a non-vocational education. Linking this assumption with the focus on disadvantaged groups led us to pay particular attention to the “basic” or “foundation skills”(see Looney, 2008). Among them most important are literacy (e.g. reading and writing) and numeracy skills as well as soft and digital skills.

4

2 Analytical framework

This chapter presents the approach, methods, sources and definitions of terms used in the report.

2.1 Key definitions

The aim of the report is to find and analyse the criteria for measuring the effectiveness of adult education for disadvantaged groups. Therefore, the first step in our work was to define key terms: “adult education”, “effectiveness of (adult) education” and “disadvantaged groups”. This sub-chapter presents agreed definitions and comments on those three terms.

2.1.1 Adult educationThe term “adult education” is sometimes understood as any type of education that occurs after the theoretical age of completion formal education. In IMPADA project we use this term in a narrower sense and follow the definition of Erasmus+ guidelines, which states that “adult education is all forms of non-vocational adult education, whether of a formal, non-formal or informal nature” (European Commission, 2016).

In that sense learning of adult persons is divided into three types: higher education, vocational education and adult education. One can argue (and – as will be shown later in this report – there are many arguments for this hypothesis) that any type of education can have an impact on labour market status of a learner. We thus assume that “vocational education” is the one with direct impact on labour market status and the one where its improvement is a primary goal.

As a consequence, the adult education in this report usually (but not explicitly) refers to basic skills courses. Among them the most popular are those aimed at providing literacy and numeracy education, followed by soft skills, ICT and motor skills education. Those are also primary themes in the literature on adult education (see e.g. Vorhaus, Litster, Frearson, & Johnson, 2011; Windisch, 2015). Important characteristic of such education is that – contrary to vocational education – it does not always lead to a qualification, e.g. in “community learning” or “family literacy courses”.

2.1.2 Effectiveness of (adult) educationThe term “effectiveness of education” is usually narrowly understood in economic terms. Indeed, the impact of education on earnings and employability is well proven, and appears

5

to be one of the most important characteristics of education. But one must conclude that economic outcomes are not solely results of education. Among others, one can mention better health, citizenship, self-confidence, stronger relationships etc.

Therefore, in this report we understand the effectiveness of adult education as all outcomes of education that improve learner’s quality of life.

This statement is further discussed in chapter Error: Reference source not found.

2.1.3 Disadvantaged groupsBy the term “disadvantaged groups” we understand the adults, who lack the basic skills (literacy, numeracy, soft skills) in a way that disturbs their functioning in society. Such group is usually link with some “physical” characteristics as age, gender, race, state of health, or economic status. We therefore treat this term broadly, with no special focus on any of the more specific groups.

Such definition stays in accordance with other known from literature. E.g. (Jarvis, 2005) defines disadvantaged as “persons who, by reason of physical or social handicap, have not had equality of opportunity to achieve their potential through educational means”

(Anderson & Niemi, 1969) noticed that disadvantaged status is not only a consequence of economic (or: objective) factors, but also subjective determinants that shapes person’s opinion about herself. They also list a number of synonyms of this term: “hard-core poor”, “the lower socio-economic citizenry”, “low income people”, “the culturally deprived”, “the functionally illiterate”, “the educationally deficient”, “the hard-core unemployed”.

They assume, that disadvantaged learner has two characteristics: living in a poverty and gaps and limitations in the “cognitive environment”, what he understands as specific family status.

Next chapter shows short findings on the identification of disadvantaged adult learners.

2.2 Identification of disadvantaged groups

There are many definitions of disadvantaged groups. In the report we focus on low-skilled adults, since the low level of skills is usually correlated with other aspects of being disadvantaged. Windisch(2015) finds three main characteristics of low-skilled adults:

- Low skills are strongly correlated with low level of education (ISCED 0-2). A group of particularly high risk are drop-outs;

- Such correlation is the strongest in the group of young adults. Among older adults other factors start playing crucial role. These include for example: working in an

6

environment that requires the use of literacy and numeracy skills as well asusing the skills in family or social context (e.g. helping children with homework);

- Literacy and numeracy skills tend to deteriorate because of non-use.

German Ministry of Education advises to providers to use some hints in recognising the low-skilled adults. The most often factors appearing in CVs are: low level of formal education; frequent changes in education (different schools), unexpected breaks in education or employment (BMBF, 2012).

National strategies such as England’s Skills for Life Strategy, Australia’s National Foundation Skills Strategy for Adults, Indonesia’s AKRAB programme, France’s National Agency for the Fight against Illiteracy, Germany’s National Strategy for Literacy and Basic Education of Adults are often designed as to improve the situation of adult disadvantaged learners, thus they are an important source of knowledge.

2.2.1 What do we know about adults with low skills?Following the assumption that disadvantaged learners are those who lack basic skills and qualifications, we reviewed the results from the Programme of International Assessment of Adult Competencies (PIAAC). PIAAC is an international survey hold by OECD in order to recognise the structural factors influencing adult competencies and to diagnose main deficits. The survey was held among representative adults from different OECD countries, with the same questionnaire, including measurement of numeracy and literacy,, so it is possible to make the international comparisons based on a results of PIAAC. More than 160 thousand adults from 24 countries took part in the first edition of the survey (OECD, 2013a). The unique feature of PIAAC is that it measured skills levels among adult population using computer-based or paper tests, identifying those with lowest level of skills, i.e. those whose skills are at level 1 or below.

We used the PIAAC data to identify the vulnerable groups in four countries that are taking part in IMPADA project: Poland, UK, Estonia and Italy and participated in PIAAC (Greece was not included, as Greece was not participating in the PIAAC survey).

The low-skilled adults were there defined as those who performed less than 226 points in numeracy or literacy tests1. Such an absolute measure caused that the share of people in the total adult population (in age group 15-65) classified as low-skilled was different in different countries: Poland – 29,5%, UK – 25,8%, Italy – 37,8%, Estonia – 18,5%.

The analysis allowed identifying from 3 to 11 groups (clusters) of adults2, that are particularly at risk of being low-skilled. Those clusters are presented in the Table 1.

1 PIAAC scores are normalised, so the OECD-average is 500. Thus the score 226 may be interpreted as “no more than a half of an OECD-average score”. 2 Please note, that groups may overlap.

7

Table 1 Identification of vulnerable groups on PIAAC data

Country

Group Group as percentage

of total population

Percentage of a group

identified as low-skilled

Italy ISCED 1 or lower education; retired or working; disabled

6,3% 69,9%

Italy ISCED 1 or lower education; unemployed or housekeeping

4,5% 83,1%

Italy ISCED 2 education; possesses less than 10 books; unemployed for more than 5 years

6,6% 68,2%

Poland ISCED 2 or lower education; more than 47 years old; possesses less than 10 books, single

1,5% 84,2%

Poland ISCED 1 education; possesses less than 100 books

2,4% 68,2%

Poland ISCED 2 or lower education, low ISEI index3 (<22.16)

2,5% 64%

UK ISCED 2 or lower education; housekeeping or unemployed; possesses less than 25 books

2,9% 83,2%

UK ISCED 2 or lower education; low ISEI index (<28,5), low monthly earnings

1,9% 78,2%

UK ISCED 4 or lower education, age above 40 years; not working for 5 years or more

1,6% 78,6%

Estonia Unemployed or housekeeping; possessing less than 10 books

1,8% 58,2%

Source: own estimates based on PIAAC database (OECD, 2013a)

The table shows those groups, in which the share of low-skilled is above 50% and such statistics is found to be statistically significant. As one can notice, in almost all cases the key distinctive characteristic is low educational attainment (at ISCED level 2 or lower), but it is not always the case. Particularly interesting seems the Estonian case, where the unemployment and social environment (as one can understand the number of books) are sufficient factors leading to low level of skills, regardless the education.

2.2.2 Short typologyThe existing literature allows identifying some attributes of disadvantaged groups that are most likely to be identified as low-skilled. Such characteristics are:

3 International Socio-Economic Index of Occupational Status

8

- Age (older workers to be disadvantaged)- Level of educational attainment (those with lower formal qualifications to be

disadvantaged)- Labour market status (unemployed or low performers)- Social status (e.g. past criminals)- Geographical factors (rural or post-industrial areas)- Demographic status (minorities or migrants)- Physical characteristics (mental or physical disabilities).

2.2.3 Different approach to different disadvantaged group?There is no consensus in literature if the disadvantaged adults should be treated as one group. Some sources suggest that adult education programmes are more effective when persons from different subgroups are treated differently. If this were true, educational providers would have to design their programmes specifically for given groups (migrants, single mothers etc.). However, such evidence is limited. (Martin & Grubb, 2001) found that some measures are more effective for women re-entrants, while less effective for prime-age men with low initial education (formal classroom training and on-the-job training). Contrary, they see “special youth measures” such as training and employment subsidies appear not to be effective for disadvantaged youths, who seem to be main recipient (see also Carneiro & Heckman, 2003).

Since there is no clear evidence supporting this view, we address this report to all the adults classified as low-skilled, with an adequate commentary where the evidence suggesting different approach for a given group is available.

2.3 Methodology of research

There were three sources of information in this report. First of all, we did an extensive desk research of key reports and journal articles dealing with issues of adult education, effectiveness of education or disadvantaged groups. Secondly, the focus group interviews (FGIs) were conducted by all five partner institutions. Finally project partners: UPTER, DACES and ENAEA, who serve as adult education providers, shared their internal documentation that was analysed from the perspective of the main questions in the project. Figure 1 presents the methodology of the research. Desk research and focus group interviews are fully described in chapters 2.4 and 2.5, respectively.

9

Figure 1 Analytical framework of a research

Source: Authors

2.4 Comments on desk research

Since there are few publications directly focusing on the issue of measuring the effectiveness of adult education for disadvantaged groups, we propose the analytical framework that allows using relevant information from literature that is related to this topic. We start with the general topic of effectiveness of education, in order to find some general findings, and then we checked how it is applicable for adult education and education for disadvantaged groups. Figure 2 shows the analytical framework.

Figure 2 Analytical framework of desk research

Source: Authors

10

Effectiveness of educationEffectiveness of adult educationEffectiveness of adult education on

disadvantaged groups

There were numerous sources used in the research. Journal articles were obtained via databases such as: Google Scholar, Jstor, Science Direct EBSCO, with particular focus on such journals as: Adult Education Quarterly, Oxford Review of Education, Economics of Education Review, Studies in the Education of Adults, Journal of Adolescent & Adult Literacy. The keywords used in the analysis were e.g.: “Effectiveness of adult education”, “Effectiveness of literacy courses”, “Effectiveness of numeracy courses”, “Effectiveness of basic skills”, “disadvantaged adult education”, “quality of provision in adult education”, “quality of teachers in adult education” etc.

Additionally, we reviewed reports prepared by following institutions: OECD, UNESCO, National Institute of Adult Continuing Education, DeutschesInstitut für Erwachsenenbildung, Institutul Român Educație a Adulților, UKCES, National Research and Development Centre for Adult Literacy and Numeracy (NRDC), Bundesinstitut für Erwachsenenbildung.

The bibliography is presented in the last chapter of the report.

2.5 Comments on focus group interviews

Following the work plan of the IMPADA project, agreed methodology instructions and methodological tools, the partnership conducted 5 focus groups (of 1.30-2.30 h duration), in each partnership country. The participants of the focus groups where: a) adult education experts, and b) decision makers in the field of adult education, ideal for providing exclusive insight on the elaboration of criteria to evaluate the effectiveness of adult education for disadvantaged groups. Key individuals from the field of Adult Education were nominated and invited by the partnership organizations, taking into account their high level of expertise and familiarity with the topic.

All focus groups were conducted according to the agreed semi-structured scenario, which was necessary to gather coherent and comparable data among all project partners. The synthesized data analysis of all focus groups followed the content analysis approach and involved coding participants’ open-ended answers to main questions into closed categories. It allowed to identify issues that seem to be most important and relevant for the study as well as organize the data into various typologies (by comparing and contrasting collected information) (PROMEA, 2016). The conclusions from the analysis appear in the report.

2.6 Comments on documentation analysis

11

Project partners who practice adult education: DACES, UPTER and ENAEA made available their own assessment reports. That material was further analysed by IBE research team. The conclusions from the assessment reports are also a part of findings in this report.

12

3 Outcomes of (adult) education

The term “effectiveness of education” is usually associated with its impact on labour market status of learner. Since late 60’, there is a standard method in economics of education to measure the effectiveness of education with so called “Mincerian wage equations”, named after Jacob Mincer (see (Heckman, Lochner, & Todd, 2003). Those equations allow measuring the rates of returns to education, e.g. how much will the wage increase with an additional year of education. There is a vast amount of empirical literature proving, that there is strong, positive and statistically significant correlation between education and wages in virtually all the countries in the world4 (see e.g. (Card, 1999; Psacharopoulos & Patrinos, 2002).

However, the economic returns to education are not the only effects of education. McMahon (1999) basing on macroeconomic perspective, shows 7 “sectors” influenced by education: economic growth, population and health (net population growth, fertility, longevity, infant mortality), democratization, human rights and political stability, reduction of poverty and inequality, environment (forest and wildlife preservation, air pollution, water pollution), crime and drug use, labour force participation.

3.1 Economic outcomes of adult education

The outcomes of adult education are similar to those described in case of education system in general, but evidences are often less clear. Vorhaus et al. (2011) notice that – despite the fact that literacy and numeracy skills are strongly correlated with labour market status (employment status and earnings) – it is not obvious, if such relation is also true for literacy and numeracy qualifications. It appears that in case of adult learning signalling effect of actual literacy and numeracy skills5 is much less important than in case of formal education.

Similarly, (Champagne, 1987) examined the effects of adult education such as: educational participation, obtaining qualifications, lower dependence on public assistance (e.g. unemployment benefits), employability, quality of employment (temporality of contract, income, job-satisfaction), confidence. The author focused on the effectiveness of career counselling within adult education to prove, that there exists some moderate effect on all of those characteristics.

4 Even though, it is unclear if it is an effect of “signalling” or improvement in human capital.5 Signalling in economic literature is understood as a situation, in which having certificate of diploma is a “signal” for a potential employer that a person has a given set of skills and capabilities. In such understanding of education signals happen to be more important than an actual increase of knowledge, skills or competencies (Spence, 1973).

13

Broader criteria of effectiveness are also an important element of evaluation and assessment frameworks. American National Reporting System for Adult Education (NRS) lists three key measures: “educational gain”, entered or retained job and further education (e.g. entering post-secondary education). Educational gain is here defined as the difference between competencies of a learner at the beginning and at the end of a programme (National Reporting System for Adult Education, 2015).

NRS lists also four follow-up measures, based on formerly described criteria:

- Entering employment- Retaining employment within two years- Completing secondary education (obtaining GED certificate)- Entering post-secondary education.

(Vorhaus et al., 2011) identified several outcomes of adult education. First, they concluded that economic outcomes are important, but not crucial in adult education. There is some moderate influence on earnings, though it is smaller than expected by comparisons with “regular” types of education. There is also an influence on employment rates: adults who participate in adult education tend to have higher employment rates. That effect is not equal: for men the bigger effectiveness is obtained through literacy courses, while for women bigger impact have numeracy programmes.

However, the most important outcome of adult education recognised by Vorhaus et al. (2011) is the improvement of relative labour market status of a person via improvement of her job and job-seeking skills, such as:.

Job-seeking skills:

- Increased confidence in applying for a job- Confidence in getting a job- Increased aspirations (applying for a better job)- Increased motivation- Enhanced job search skills;

Job skills:

- Team work- Appropriate behaviour- Willingness in taking responsibility- Timekeeping- Better stocktaking skills.

Vorhaus et al. (2011) underline that

“However, over the course of the programmes, the participants’ optimism about the employment impacts of the programmes tended to decrease. Individuals reduced

14

their expectations about their future job prospects, and often became frustrated and discouraged.”(p. 36)

Some interesting and non-obvious findings are the element of Speckesser & Bewley (2006) report. They analysed three types of work-based learning for adults: “Basic Employability Trainings”, “Short Job-Focused Trainings” and “Longer Occupational Trainings” to examine their long-term (after 40 months) effects on employability and dependency on social benefits. Short Job-Focused Trainings are defined as a short-term (up to 6 weeks) programmes aimed at increasing the job match, e.g. training to obtain the health certificates required in applying for a job. Longer occupational trainings are the programmes up to 14 weeks designed to obtain some particular qualification6. Finally, Basic Employability Trainings are longer (up to 26 weeks) programmes addressed to adults who have a poor level of basic skills and focus on general literacy and numeracy skills.

According to their research, Short Job-Focused Training had the most significant impact: the employment rate among participants was increased by 5 percentage points and that effect seemed to be sustainable. Effect of Longer Occupational Trainings was similar, about 4 percentage points, but delayed: it is therefore important not to measure the impact immediately after the completion of the course, but also later in time. Finally, the Basic Employability Trainings also proved to have a positive impact on employment rates, however no impact was found on benefits dependency. That may be considered as a field that has to be better examined in order to find solutions and policies that would lead to reduced benefit dependency.

3.2 Personal and social returns

It is important to notice, as researchers suggest, that some of the literacy and numeracy programmes outcomes cannot be examined by a typical quantitative comparison of skills proficiency at the beginning and after the completion of the programme. There are some effects, such as improvements in confidence, teamwork, employee initiative and problem solving that cannot be measured in a typical way (Alkema & Rean, 2014).

This way of understanding the role of adult education was strongly visible among IMPADA focus groups' participants who – when asked what type of content should be included in adult education training programmes for disadvantaged groups – indicated priority focus on ‘soft’ skills and general well-being rather than just outcome qualifications (5/5 indications). Soft skills (e.g. confidence, social skills, self-belief, better routine) were often mentioned on the occasion of answering the question about what are the areas that could provide input for the assessment of the education provided (3/5 indications). Other often mentioned areas

6Consequently, they fall apart the definition of adult education presented in Chapter 2.1.1.

15

were: “progression and destinations”7, including ‘continuing further adult education programmes’ and ‘improvement and use of ICT skills’ (3/5) and “employability – life transformation” (3/5).

(Vorhaus et al., 2011) in their summary of research and publications list the following outcomes of adult education that are linked with psychical wellbeing of a person:

- Increased self-confidence- Confidence to try new things- Self-esteem- Belief in own ability- Belief in personal achievement- Reduced embarrassment or stigma- Improved health outcomes (physical and psychical)- Positive attitude.

Second field of non-economic outcomes found in the same report are improvements in family and everyday life. Those include:

- Increased ability to help children with their homework- Increased confidence to help children- Increased confidence to undertake the activities that involve literacy and numeracy

skills, such as signing the agreements- Ability to contribute to family life.

The adult education has also a positive impact on participation in further education. Furthermore, the effect of increased confidence and motivation to engage or re-engage to learning appears to be durable (Evans & Waite, 2008).

Ofsted’s report on literacy and numeracy of foreigners suggested that the programmes had positive impact on soft skills, ICT skills, self-confidence and self-efficacy. Some learners benefited also in their level of oral communication (Ofsted, 2003).

Other literature by (Windisch, 2015)or by (Mcintosh, 2004) also confirms the following (mentioned before) outcomes:

- Gains to self-confidence- Encouragement to get further education- Better family background, via helping children.

Centre for Literacy of Quebec (2010, p. 8) emphasized the impact of adult education on social capital:

“These studies, as well as related discussions (Sticht 1999, 2000, 2006, 2007), and literature surveys such as those produced by Benseman and Tobias (2003), Westell

7Broadly understood as learners’ participation in life-long learning (LLL).

16

(2005), Gray (2006), Hartley and Horne (2006), and Salomon (2009), point overwhelmingly to social capital as an almost ubiquitous outcome of adult learning and literacy interventions internationally in the past twenty years’ (…). Social capital outcomes include communicating and becoming involved with others, whether at work or home, with the family, or in the community.” (p. 8)

The fact that non-economic outcomes of adult education are sometimes equally or more important than economic outcomes is also visible among motivations of learners. As (Vorhaus et al., 2011) noticed,

"Learners are often motivated by a far wider range of factors than the wish to improve job performance, and managers by factors other than the desire to fill skill gaps or improve productivity" (p. 62).

That fact also should serve as a basis while discussing learners motivation on participation and persistence in programmes, issues that will be further discussed in chapters 4.2 and 4.3.

17

4 Factors influencing the effectiveness of adult education of disadvantaged groups

Even though some solutions are more effective than others, there is much evidence showing that basic skills courses are generally effective in terms of improving adult learners skills (Myers & Myles, 2005; Vorhaus et al., 2011; Windisch, 2015). Myers & Myles (2005) found that the least educated learners (or disadvantaged) are most likely to gain from the adult education. This effect is not equal for all disadvantaged groups, e.g. evidence suggests that adult women gain more than men (Heckman, 2003). The problem is that, at the same time, disadvantaged adults are least likely to participate in any type of education (Condelli, Kirshtein, Silver-Pacuilla, Reder, & Wrihley, 2010).

Vorhaus et al. (2011) states that

“Improved practice often precedes improved achievement: participation in ALN programmes leads to increased engagement in literacy and numeracy practices, and over time this can lead to gains in proficiency and achievement”. (p. 62)

To better understand the character and subtleties of adult education effectiveness, we started with the analysis of factors influencing adult education for disadvantaged groups. Factors from the first group (external factors) – generally – cannot be improved by the educational providers, so they seem to be outside the scope of this study. However, each educational provider has to be conscious of those factors. Such external factors sometimes serve as an obstacle, so educational provider has to adjust its programme or organization in order to fully utilize its potential and benefits from its own work. The second group are the institutional factors (Ofsted, 2015).

In general terms most of factors (either external or institutional) influence also other types of education (e.g. shape of curricula). However, there are two crucial differences:

1) adult learners from disadvantaged groups belong also to “hard-to-reach” groups, so the participation rates are much lower;

2) rates of achievement in adult education are non-linear, e.g. learners who completed the programme benefit (on average) more than twice more than those who dropped out in the middle of a programme (see e.g. Vorhaus et al., 2011).

Given the above, the third part of this chapter, focused on the specificity of adult education for disadvantaged groups (especially basic skills teaching), is divided into 3 sections, and each of them describe one of the issues particularly important in case of this type of education. It should be noted that identified areas are complex to assess, as they are strongly influenced by both external and institutional factors. These are:

18

- Participation in adult education that – apart from other impacts – depends on adequate and targeted promotion and on proper recruitment of adults from disadvantaged groups to adult learning;

- Persistence in adult learning that, according to subject literature – apart from other impacts – depends on actions aimed at enhancing adult learners to finish courses;

- Quality of provision that means solutions of improvement outcomes of the students who finish the course.

This approach will be followed in consecutive chapters. Each of sub-chapters follows the same structure: in the first part we list hypothetical factors that influence the effectiveness of adult education of disadvantaged groups. Second, we check the evidence in a literature and exclude those that have no proven impact – since they seem unimportant or insignificant. Finally, we find the criteria by which such factors may be measured.

This approach is graphically shown in the Figure 3.

Figure 3 Key aspects of effective adult education

Source: Authors

4.1 External vs. internal factors influencing effectiveness of adult education

Factors having positive or negative impact on effectiveness of adult education for disadvantaged groups can be generally divided into two types: external and internal. The latter can be improved by adult education providers on their own. The former cannot be controlled, but it is also important two analyse them. Full knowledge on external factors shaping adult education helps providers, how to adjust their programme in order to overcome the given obstacles.

There are three main types of external factors affecting adult education:

19

1. Public policies on education; this issue is not limited to policies on adult education, since it can also be affected by – say – formal elementary education;

2. Social policy – vast majority of programmes for adults from disadvantaged groups is addressed to those, who are also the beneficiaries of social policies. It is particularly crucial in case of activisation policies or different types of mandatory programmes.

The internal factors are then introduced in further sub-chapters.

4.2 Participation in adult education

Participation barriers – e.g. lack of awareness of learning opportunities as well as low motivation to learn – are one of the most important factors influencing the effectiveness of adult education (Windisch, 2015). Various types of such barriers apply both to unemployed and those whose workplaces don’t play the role of learning environments by engaging their employees in self-development and providing them basic education and training (Porras-Hernández & Salinas-Amescua, 2012).

The most common types of barriers to lifelong learning, identified in a lot of previous research and applicable also to adult education (Desjardins & Rubenson, 2013; Litster, 2007; Windisch, 2015), are:

i) Situational barriers, created by learner’s personal situation. Usually it is linked with lack of time, caused by family and job obligations. It may also refer to other physical barriers as inability to travel.

ii) Institutional barriers that occur when learner is unable to fill the requirements to participate in a programme (financial or formal, e.g. lack of prerequisite qualifications);

iii) Dispositional/psychological barriers when a learner has negative attitude to participate in a programme. In case of disadvantaged learners it is very often linked with bad formal education experiences;

iv) Informational barriers when a learner is unable to receive a proper information about the actual offer and potential benefits of a programme;

v) Financial constraints that are strictly linked with situational and institutional barriers.

(Desjardins & Rubenson, 2013) conclude that there are two key aspects of barriers to adult education: individual and structural barriers. Educational providers have to, therefore, apply different strategies to different sets of problems. It is particularly addressed to institutional barriers at the provider level.

20

From the OECD’s review based on data from years 1999-2004 (OECD, 2006) one can conclude that – in order to improve participation in LLL, including participation in adult education – educational authorities and policy makers should create and implement public policy focused on: (i) creating the structural preconditions for raising the benefits of basic skills learning; (ii) sharing information on learning opportunities and benefits; (iii) promoting well-designed co-financing arrangements; (iv) improving delivery and quality control; and (v) ensuring policy co-ordination and coherence by involving stakeholders such as social partners, employers, labour unions, and education providers.

Apart from the efforts that should be made by authorities, one can indicate activities necessary to implement by education providers from that promotion and recruitment seem to be the most important in the context of the need to increase disadvantaged adults’ participation in education.

4.2.1 PromotionFirst aspect of an effective engagement into adult education programmes is its proper promotion. Disadvantaged adults often lack the information on the possibilities of taking part in adult education. One of the most important reasons is that they use another media than those in which such programmes are promoted. Furthermore, even those who have information about the possibility to apply to a programme often lack the knowledge about the potential benefits of such programmes and, consequently, are not encouraged to apply. Finally, disadvantaged (or hard-to-reach) adults very often have bad formal school experiences, so they are discouraged from taking part in any other kind of education.

The latter issue is also an important factor in discussion the mandatory programmes (e.g. when participation in adult education is requirement for receiving unemployment benefits). People who are discouraged from any kind of education are very unlikely to achieve any significant progress, while participating in adult education. Furthermore, lack of progress only deepens the discouragement, creating a harsh vicious circle. Thus, the promotion of positive outcomes of adult education should also be directed to mandatory students.

All of those factors cause that the effective adult educational provider should implement a number of strategies to adequately promote its services. One can find a growing number of the literature on which strategies happens to be effective and why.

(Windisch, 2015) identifies some characteristics of promotion the hard-to-reach groups. First of all, they are far more easily reached in community context. Thus, one way to improve the participation rates is to work in community (Hamilton & Wilson, 2005). Similarly, (Mcintosh, 2004) suggests that the one-to-one approach is the most effective way to engage learners. Such approach may be best achieved in community context.

21

Communities are also good places to provide education programmes. Places such as community centres or workplaces are more accessible for potential learners, so it is much easier to convince them to try applying to the programme.

There are also another benefits of communities. If a person live in a community where education is believed to be profitable or valuable he/she is more likely to engage in any kind of formal or non-formal education. Thus, educational providers may try to change attitude of whole community.

There is also growing body of evidence on how the programmes are perceived by potential learners. Some researchers suggest that a name of a course is one of the factors increasing or decreasing participation rates. Programmes which names directly suggest that they are addressed to the low-skilled or disadvantaged learners (e.g. “Reading and writing”) are perceived by learners as a kind of stigma. To avoid such risks, it is recommended to use indirect names, such as “communication skills”.

Another aspect that must be analysed by an education provider is how a certain programme fits into specific type of disadvantaged learners. For instance, it is shown that adults who have higher numeracy levels are more likely to participate in on-the-job trainings (Parsons & Bynner, 2007). Such observations have to be taken into account while designing promotion and recruitment activities.

Even though adult education is understood as non-vocational type of education, one should not pass over the occupational context. As Vorhaus et al. (2011, p. 62) noticed:

“Workplace basic skills courses can reach people who are not normally involved in continuous education or training”. (p. 62)

This aspect was noticed by NAO (2008), who claims that the first crucial point of a successful educational offer addressed to adults from disadvantaged groups is creation of an offer integrated with a specific place of work (specific job). This can be explained by the fact that people who are away from education are likely to be interested in returning to learning, only if they believe that the knowledge they can gain will help them to take a chosen job. On the basis of positive examples one may identify specific factors that play a role in this process. These include enhanced working environment, professional support provided by a qualified tutor experienced in working with people with difficulties in learning and the possibility of validation and certification of informal and non-formal learning (e.g. skills acquired at work), particularly important because otherwise skills acquired in such ways would not be recognized by other (future) employers. Ambos stresses also that the low skilled prefer to acquire professional skills informally or in non-formal way, in particular through observation and practice as well as through training or apprenticeships in their workplace. At the same time, they demonstrate distance to classic education opportunities (such as courses organized outside their working environment) and to traditional methods of learning (such

22

as reading professional literature or learning via Internet) (Steiner, Voglhofer, Schneeweiß, Tamara Baca, & Fellinger-Fritz, 2012).

Individual counseling is one of the most important factors that contribute to arousing interest in further education among members of this group. With respect to people who are "away from education" jet still professionally active, it is recommended to organize such counseling at their place of work, because it allows to clearer understand their specific professional situation. Furthermore, in case of people that wish to learn and return to the labor market, individual counseling allows facing their fear for failure, which is often indicated as the main barrier to making decisions about further education. It applies especially to elder people who are often afraid that they will not be able to work with much younger coworkers (ibidem).

In order to reach this group, it is important that educational offer was easily accessible. Three aspects play here a key role: 1) easy access to education programme, without bureaucratic procedures, 2) lack of entrance exam that would generate high risk of failure, 3) easy access to the place where a process of education is being realized. Accessibility of educational offer is also connected with its costs. It is obvious that free offers are particularly attractive and thus easily accessible. Since people belonging to this group are usually low paid and pay little attention to education, the need to cover costs of education is one of the main barriers to learning. At the same time, these people often lack the ability to find a suitable offer via Internet. For these reasons, it is important to create opportunities to provide information about learning programmes with the use of methods alternative to Internet. One of such methods is posting ads in regional and local press (Steiner et al., 2012).

4.2.2 RecruitmentWindisch (2015) presents the approach designed by German Ministry of Education and Research (BMBF). According to this approach, recruitment of adults with low basic skills involves two key elements: recognition of such persons and proper talking to them. Apart from demographic or social characteristics (see chapter 2.2), it is advised to analyse certain behavioural characteristics and – on this basis – to recognise needs of adult learners. Typical recognised behaviours are:

- Refusals of reading or writing, using hardly possible excuses (such as the lack of glasses)

- Refusals of educational offers - Signing unread documents- Ignoring written materials, e.g. flyers- Inability to read or write a text longer than a single sentences.

23

(Windisch, 2015) finds also that screening and initial assessment are crucial. However, it has to be conducted sensitively and carefully in order to better meet the learners’ needs and not to demoralise them.

Good recognition of adults with low skills is crucial for successful recruitment. First of all, it allows a provider to place a learner in a proper programme – if the person is wrongly placed in the programme (e.g. he/she finds the programme as a throwing into deep water), he/she can be easily discouraged, drop out from the programme (see chapter Error: Referencesource not found) and, finally, refuse to participate in another programmes. Secondly, proper recognition of a disadvantaged person allows provider to adapt the recruitment process in order to increase the probability of convincing him/her to participate in a programme. BMBF lists five key aspects of adequate communication with low-skilled adults: sympathy, empathy, openness, patience and individual approach. Sensitiveness and respect to all learners may be crucial in encouraging them to participate in adult learning and in further successful completion of the program.

Next aspect identified by (Windisch, 2015) are partnerships of employers and trade unions. They are considered as an important and effective drivers of workplace based basic education programmes that help to support learners’ motivation and persistence.

In addition, it is typical for this group of people that they generally are not able to express concrete expectations towards educational offer, which is of their interest. It is often due to their lack of knowledge and awareness of the available educational opportunities. For this reason, institutions involved in adult education have to identify and meet interests and needs of these people and create offer, which will be most adapted to their needs. One of the possibilities of reaching this group is to place educational offer in group’s cultural and social environment (community context - see above). (ibidem)

The importance of proper recruitment was also stressed by the participants of IMPADA focus groups, especially when they answered the question about what could be improved in the adult education provided to disadvantaged groups in order to deal with the issue of group heterogeneity. In 4 out of 5 groups "targeted appropriate support and differentiation" (including - among others - 'recognition of prior knowledge') was pointed, in 3 out of 5 - "ethnographic approach" (including - among others - 'using of methods such as interviews, observation, cultural mediations etc., in order to understand each group of learners needs' as well as 'challenging stereotypes').

4.3 Persistence in adult education

As was mentioned in the introduction to this chapter, the key to successful adult education programme is persistence. Learners benefit the most when they complete whole

24

programme as well as such impact is correlated with attendance to particular lessons (Windisch, 2015).

However, adults from disadvantaged groups are not only less likely to participate in adult education, but also are more likely to drop out during the programme. (Coben et al., 2007) mention that average attendance in 150-hours courses was just 39 hours – such pattern is also a case in many other programmes. Therefore, keeping high motivation among learners is one of the key tasks for educational provider. There is no unequivocal evidence suggesting who is most likely to drop out during the course, mainly because hardly any provider monitors this issue which results in lack of proper data (Vorhaus et al., 2011).

Numerous sources (Alkema & Rean, 2014; European Commission, EACEA, & Eurydice, 2015; Vorhaus et al., 2011; Windisch, 2015) cite guidance services for disadvantaged adults as an important way to improve their motivation to participate in adult education. Though this factor seems to be crucial, there are hardly any European countries that provide guidance services for adults who do not participate in public employment services, and this group is limited and do not cover a large fraction of disadvantaged adults (European Commission et al., 2015). Windisch (2015), therefore, concludes that a structural guidance services aimed to support disadvantaged learners are not popular among European countries. In a report of (European Commission et al., 2015) there are just a few examples of such strategies on a national level: Austrian “Zentrale Beratgungsstelle für Basisbildung und Alphabetisierung”), German telephone guidance services, Luxembourg’s “Maison de l’orientation” and Portugal’s “Centros para a Qualificação e o Ensino Profissional”. In a majority of countries where such strategies are absent it is advised to prepare (out of necessity in a smaller scale) similar solutions by adult education providers.

(Windisch, 2015) finds appropriate information, advice and guidance as crucial factors increasing persistence rates in adult education of disadvantaged groups. Many disadvantaged students are unaware of potential benefits of education and therefore do not take advantage of them. That often contributes to the decision of dropping out the programme. American researchers suggest that the fraction of students actually benefiting from the different forms of guidance is very slight and even smaller among the disadvantaged adults. They therefore suggest implementation of Crockett’s idea of “intrusive advising” (Mcdonnell & Soricone, 2014).

It is important to note that lack of persistence does not always prove that the programme failed. Sometimes the changing circumstances in personal situation of a learner (such as sicknesses or personal problems) force him/her to give up from the programme. In such cases an educational provider should recognise such persons and provide them support in retaining to the programme, when the situation gets back to the normal. Such support should principally involve strengthening learner’s motivation and additional measures as distance and blended learning that helps learners to minimize losses (Vorhaus et al., 2011).

25

Supporting learner persistence means assisting learning during and beyond formal education. One interesting conclusion of Vorhaus et al. (2011, p. 14) is that

“breaking off from programmes is not always a programme failure but may be a more 'rational and positive response to changing circumstances' (...)”.

Taking it into account, adult education providers must ensure that - when those external circumstances change - a learner will not be discouraged from coming back into programme. Thus, the guidance and assistance for learners must occur not solely during the programme, but also beyond it.

Vorhaus et al. (2011) find that the key to learners’ persistence is a sensitive monitoring of progress. The lack of (self-reported) outcomes of adult education is often a reason why learners are discouraged from programmes. Such monitoring should also involve the “soft outcomes, such as improvements in self-confidence”.

Litster (2007) summarizes the number of surveys and British experiences in strategies aiming at improvement of learners’ persistence. She groups it into five groups: safe learning environment, community in the classroom, individualization of learning, goal-oriented education and supporting self-improvement.

The first group, a safe learning environment, is mainly addressed to teachers. It is crucial to create the relationship between teacher and learner that makes the latter more confident and willing to learn. She lists the examples such as:

- Finding common interests with learners- Knowing their names- Communication in a language of students (at least learning few words)- Understanding the learners’ personal situation and their problems with motivation

and persistence- Chatting with learners and giving informal advises- Cooperation with other teachers and staff who also deal with a given student.

Community in a classroom focuses on creating the cooperation among students that help them benefit from each other. She mentions following strategies:

- Controlling learners attendance and finding out the reason, with offering the help to solve problems that limit their participation

- Team working, working in pairs- Setting up a “buddy system”, in which one student is a support for the other- Creating possibilities to make up for those who miss a class- Engaging former students to talk about their motivation and successes- Engaging students to talk about their “goals, motivations, hopes and fears”.

26

Third element, individual approach to the learner, is complementary to former two. It helps a learner to understand that he/she can personally benefit from a programme and – therefore – improve his/her motivation to actively participate. There are three proposals of obtaining this goal:

- Establishing clear learning routes- Finding links to learners’ personal experiences and redesigning a programme as to

make it relevant to their lives- Creating activities that involve personal impact in the learning, such as writing the

diaries or personal histories.

Goal-oriented education is a strategy that is common for adult and regular education. In case of the former the proposed strategies involves:

- Explaining to students what does the goal-setting real mean- Finding out learners’ real goals- Asking students about the factors that might affect their performance and –

therefore – obtaining their goals- Revisiting goals on a regular basis- Using formative assessment (see later).

One of the most important goals of adult education is to encourage learners to continue their education and self-development also after the completion of one programme. The strategies that support self-learning and self-development are:

- Encouraging learners to take responsibility for their education by working independently, fulfilling homework etc.

- Solid assessment of homework- Giving the comments and advises to the homework, both verbal and written.

According to Porter, Cuban, & Comings (2005), education providers are challenged to value learning outcomes in a way that will allow to be sure of their quality and comparability for each learning pathway.

Another factor recognised by Windisch (2015) is supporting self-study. It helps to achieve additional goals as well as is one of the most efficient ways to support persistence: when a learner is prepared to self-study, it is much easier to make him/her to survive the period outside of education and henceforth come back to the programme.

Some strategies, especially those involving advisory, are common for all types of education and experiences in another schools may be also useful for adult education for disadvantaged groups. For instance, Bailey & Alfonso (2005) analysed the practices in (2-year) community colleges in United States and found 4 strategies that were proven to have a positive impact on the effectiveness of provision in those institutions. Those are:

27

1) Supporting learning with advisory, mentoring and counselling;2) Creating learning communities that help learners to better benefit from teachers’

experience (it is also shown that learning communities are far more effective, when the involved staff is employed full-time);

3) Creating special programmes or services for underprepared learners (this point is crucial from a point of view of analysing the adult education for disadvantaged groups);

4) Having a “college-wide” perspective on reforming and improving services: since the institutional elements of education programme are highly complementary, educational provider must not analyse learners' performances on a basis of one single aspect.

Very similarly, Siebert (2003) proposed five recommendations to maximise persistence in programmes:

- Counselling based on full, realistic and objective information- Recognition of learners’ interests and goals in the beginning of the programme- Ongoing assessment- Recognition of the causes on why some students do not persist, based on interviews

with both those who dropped out and those who completed a course- Enabling the flexible learning modes by modularisation of the programme.

Casey et al. (2006) found the embedded programmes to have a positive impact on learners’ persistence. They understand embedded as the programmes that integrate vocational and basic skills provision. Particularly, most effective embedding programmes were characterised by a strict collaboration between literacy, numeracy and vocational teachers.

In order to adjust an educational offer to the possibilities and needs of this group, it is also important that the form of learning and thus developing new professional skills does not refer to their past school failures. Regular feedback focused on made progress and implemented in the form of coaching may be here of help. Furthermore, one has to keep in mind that motivation is the key aspect not only before the start of learning, but it should be the integral part of the whole process (Steiner et al., 2012).

Depending on the kind of disadvantaged people (in terms of specific sub-groups), one should pay particular attention to the kind of conditions and support structures that should be created and used in case of this group. For those with the care responsibility it is important to provide childcare possibilities or to create family friendly places of learning. The most important aspect in this case is flexibility in terms of time of learning. Learning is problematic especially for women with the care responsibility. Financial issues should also be taken into account, since high tuition could be and often is a major barrier to making decisions about starting learning among members of this group. Moreover, it is strongly recommended to

28

create separate educational offers aimed at various sub-groups, such as migrants and older people (ibidem).

4.4 Quality of provision

Quality of provision in the case of learning programmes for disadvantaged adults is also important. There are five groups of most important factors that contribute to high quality of educational provision: organisation of the course, curricula design, organisation of learning, infrastructure and quality assurance.

The first group covers strictly organisational issues including: average length of a course, service intensity (e.g. average number of learning hours per week/month, total number of teaching hours, number of hours of instruction received by an average individual learner), and number of learners per course. The last issue was raised by some of IMPADA focus groups' participants who indicated the number of 8 to 12 participants - depending on characteristics of learners - as the optimal size of an average adult disadvantaged learners' class as 'accessible and attractive to such learners and providing enough support and time for those in most need').

In the case of curricula design issues that are visible in the literature include: curricula diversity, its flexibility (e.g. short optional courses as a part of a larger education programme), adequacy and responsiveness to the actual labour market’s needs (Swain et al., 2009).

Curricula design was also a topic that was raised directly during IMPADA focus groups. Participants of FGIs, asked about factors that should be included in the AE for the disadvantaged training programmes, indicated - among others:

- "improving learners' progression routes" (4/5 indications), including 'focusing on the big picture of each learner's journey', 'individualized approach' and 'extending the use of optional courses';

- "programming - flexibility: creating individual paths for learners" (3/5);- "focus on the labour market needs and Increased Joint Practice Development

between practitioners" (3/5).

The third factor of quality provision is the organisation of learning - use of certain practices, tools, settings, methods and approaches aimed at increasing learners’ engagement and motivation to learn as well as adjusting educational process to diagnosed learners’ needs, including:

- focus on identification of learners' needs;

29

- additional support for learners - if needed (e.g. childcare, financial support, adaptive equipment);

- establishing partnerships and working closely with different stakeholders in order to link formal and informal learning and learning in different environments as well as to identify local needs in the field of adult education.

Factors connected with the organisational site of learning were often mentioned during focus groups, especially :

- when discussing what could be improved in adult education programmes for disadvantaged groups in order to improve its effectiveness (5/5 indications in case of "identification of learners' needs", 3 - in cases of "additional support" and "partnerships");

- when discussing factors that should be included in the AE for the disadvantaged training programmes (3/5 indications in case of "using social pedagogy - supporting the process of learning, e.g. 'searching for <knowledge distributors> also within learners' social environment', 'working with members of families' etc.).

Learning infrastructure (including accommodation, resources, access to ICT, settings adjusted to programme’s specificity) also contributes to quality of educational provision. This factor was mentioned by the participants of almost all IMPADA focus groups (4/5 indications) in the context of naming characteristics crucial for training institutions that provide AE for the disadvantaged. During the discussions on this topic the following factors were mentioned: 'infrastructure adapted to the specifics of implemented actions and to the needs of their participants (trainers, teachers and learners)', 'small, accessible venues in the communities with the most disadvantaged learners', 'learning that is based where the learners are (not necessitating travel to a campus)', 'small scale and cozy buildings, less intimidating to disadvantaged than big institutions'.

Last but not least, quality assurance (internal QA systems implemented by training providers, external QA – depending on existing regulations, respected adequate QA procedures etc.) is crucial for the quality of outcomes.

Summarising, quality of provision depends on various factors, which can be shaped by a training provider. Knowing this, we asked IMPADA focus groups’ participants about characteristics that training institutions should have in order to provide effective adult education for the disadvantaged. Apart from infrastructure, the most often received answers to this question were:

- “good organisation qualities and characteristics” (5/5 indications), including ‘listening to learner's voice’, 'taking actions to make improvements based on feedback’, ‘using feedback mechanisms after end of programme and after ½ a year’, ‘using effective Quality Assurance Procedures’, ‘being characterised by respect and responsibility toward individuals’, ‘being characterised by innovation and reliability (ability to

30

demonstrate experience in effective management of educational projects)', ‘employing varied team (trainers and teachers) in terms of age, experiences and skills’, ‘being characterised by flexible approach in learning activities (programmes)’, ‘offering continuing education courses for teachers’;

- “experienced, dedicated and looked-after staff” (5/5 indications);- “successful training networks and partnerships”.

The following sub-chapters extend some of the abovementioned issues.

4.4.1 Length of a courseAs mentioned in the previous chapter, the key to positive outcome of adult education are attendance and completion of a course. One may ask, though, how long should be a course in order to have an impact on learner’s competencies. (Alkema & Rean, 2014) cite numbers of evidence to show that the answer to this question is unclear and hugely depends on the way of defining the word “improvement”. In USA, e.g., there is a term “measurable gain” which is understood as the improvement by a level (or gaining a GED certificate). In that case the evidence shows that there is the need of 100 hours of teaching. Contrary, studies in New Zealand and United Kingdom focused to measure “a significant gain” (understood as statistically significant improvement in competencies) and showed that even shorter courses, lasting 35-39 hours, may improve literacy and numeracy of learners.

Based on the analysed evidence, (Alkema & Rean, 2014) propose the useful distinction between analyses focusing on the issue of programme’s length and divide them into two categories: (1) courses that are focused on gaining new skills and (2) courses that focus on “brushing-up” existing skills, which is particularly often the case regarding to literacy and numeracy programmes. The latter appear to be shorter ones.

The report (European Commission et al., 2015) shows that there is a variety of approaches among European countries in designing the length of courses. In majority cases “basic education programmes” last between 100 and 300 learning hours or one year of learning. In a number of countries such programmes are divided into smaller unites as in case of Portugal where each “module” lasts 50 hours, or in Denmark where each “step” lasts between 30-60 hours (see also Windisch, 2015). Similarly, Coben et al. (2007) found that to achieve a progress of one level of qualifications framework, it is required to learn for 100-150 hours, but an observable progress is achieved by those who participate just for an average 39 hours.

To sum up, the length of a course is an important factor influencing the outcomes of adult education. In order to optimize its impact, a provider has to take into account following criteria:

- The needs of learners;

31

- Character of a course (gaining new skills or brushing-up existing ones);- Persistence of students.

4.4.2 Organisation of learningOne of the most important characteristics of adult education of disadvantaged groups is that learners often have bad experiences with formal class-based education and, therefore, negative attitude to learning. Such attitude affects not only participation and persistence of learners, but also their performance – discouraged learners are more passive during a programme and, consequently, less likely to succeed. Windisch (2015, p. 58) puts it simply:

“the learning context matters for success”.

National Research and Development Centre for Adult Literacy and Numeracy prepared series of reports analysing the effectiveness of particular types of basic skills education: writing (Grief, Meyer, & Burgess, 2007), reading (Brooks, Burton, Cole, & Szczerbiński, 2007) and numeracy (Coben et al., 2007).

According to the report on writing (Grief et al., 2007), the following strategies were proven to be effective:

- Using texts of different kinds, e.g. applying writing short stories, letters or e-mails, complaints or acknowledgments etc.;

- Using texts with meaningful contexts in that learner can find the links with real life, both in substance (e.g. stories on his own autobiography or family) and form (e.g. complaints on real products, as mentioned above);

- Discussions on writing, both with a teacher and with a group; - Individual feedback of teacher – marking of writing should involve not only grades,

but also teacher's comments on what should be improved.

Report of (Benseman, Sutton, & Lander, 2005) on reading focuses on slightly different approaches. According to them, the key to effective reading provision are skilled teachers who apply a variety of teaching strategies. The case of quality of teachers is discussed in one of the next chapters of this report. Second factor identified by Benseman, Sutton & Lander (2005) is – similarly to the case of writing – creating curricula linked to life. Provision is much more effective when the examples of texts that are read during the course are somewhat familiar for students, e.g. have reference to their professional experiences or living environments. Finally, the strategies that is proven to be very effective is implementing the reciprocal reading, involving the cooperation among students.

Additional factor identified by (Vorhaus et al., 2011) is the assistance for a teacher. As was stressed above, the effective provision of adult education must combine two elements: good subject knowledge and pedagogical preparation. Since subject teachers often lack the

32

competencies adjusted to the needs of disadvantaged adults, a teaching assistant may serve as an important factor of improving adult education effectiveness.

(Ofsted, 2007) points on the desirable character of such assistance. Competencies of a subject teacher and a support tutor should be highly complementary, and their work is most effective when they tightly cooperate, including the preparation of a programme and the design of curriculum. Situation in which a teaching assistant limits his/her engagement to in-class activities is, however, an often situation, and lowers overall programme effectiveness.

(Brooks et al., 2007) see the time spent in a classroom as a key to effective literacy provision. Their findings suggest that self-study may be an appropriate strategy to increase persistence (see chapter 4.3), but still it is less effective in improving literacy than actual working with a teacher or trainer. They therefore formulate 4 recommendations for adult education providers that seem to be important in effective provision:

- Programmes should be more intensive, so that shorter courses involve more hours of attendance;

- Providers should aim in decreasing learner:staff ratio;- Time of working alone appears to be too long in many cases;- Most practical ways to decrease the time of working alone is to enhance working in

groups or pairs (e.g. buddy system).

Third crucial type of basic skills education are the numeracy programmes. (Coben et al., 2007) examined their provision in UK to conclude that the most important factor is flexibility. That should involve: the possibilty to adapt a programme to diverse group of learners and changing during the course expectations on final outcomes. However, they noticed that among British providers such diversity was mediocre: they identified very limited use of practial tools and ICT. The organisation of learning was dominated by whole-class approach to learning and limited use of group or collaborative work.

4.4.3 Teachers qualityThe issue that is analysed in virtually all the publications on effectiveness of adult education is the quality of a teachers and trainers. There are two approaches to this case: one focuses on the (formal) qualifications of teachers, the second on soft skills.

The key conclusion of (Vorhaus et al., 2011) is that qualifications matter. There is an enormous evidence suggesting, that – contrary to some popular beliefs – provision is more effective when staff is trained and has pedagogical background.

(Benseman et al., 2005) stress that teaching is far more effective when staff is employed in full-time. It is an important conclusion, since the majority of staff of European AE training providers appears to be part-time employees or voluntary. (Ofsted, 2011) points that the

33

most successful providers are those with at least 1 senior member of staff with an excellent knowledge of literacy and its pedagogy.

Similarly, (Ofsted, 2011) in his report on literacy also underlines the role of teacher. According to his findings, the most effective providers were hiring at least one member of staff “with an excellent knowledge of literacy and its pedagogy”.

There is one exception from the following rule: overqualified teacher. In some cases teachers whose qualifications are above average appear unable to communicate with learners, which affects the effectiveness of provision and learners’ motivation. However, one may argue that such situation is an effect of lack of soft skills that help to overcome the barriers in teacher-learner relationship.

Given the above, the second characteristic of effective adult education teachers and trainers are soft skills. While dealing with disadvantaged adults, who often lack of motivation to learn (as shown in chapter Error: Reference source not found) and have bad experiences from formal education, adult education teachers have to implement the special approach to teaching.

(Ofsted, 2011) lists the following strategies that appear to be effective:

- teachers having high expectations about learners’ achievements;- adapting curricula to meet the needs of particular group of disadvantaged adult

learners; - intervention to a curricula or a programme, when learners’ expectations change or

previously assumed strategies appeared to be ineffective;- understanding barriers of disadvantaged learners, that are often different than those

of “regular” learners.

One conclusion is particularly important for providers of adult education for disadvantaged groups: some teachers tend to limit their expectations about their students and measure their progress by a comparison with a whole group, rather than with national average, which is presumably higher. Such strategy is found to be ineffective and makes the gap between an average and disadvantaged student to persist (Ofsted, 2011).

(MacLeod & Straw, 2010) underline the need of providing strong teacher-learner relationships. Informal contacts between teacher and learner help the latter to improve their provision and adapt the solutions described above.

Report of American Institutes for Research within the project “Promoting Teacher Effectiveness” proposed the useful framework to assess and identify teachers’ skills and competencies. The framework is based on 4 aspects: domains, competencies, performance indicators and sample illustrations. Table 2 presents their approach.

34

Table 2 Adult Education Teachers Competencies framework

Domains There are four domains that represent broad areas of activity for an adult education teacher:1. Monitors and manages student learning and performance through data 2. Plans and delivers high-quality, evidence-based instruction 3. Effectively communicates to motivate and engage learners 4. Pursues professionalism and continually builds knowledge and skills.

Competencies Within those 4 domains of activity 17 individual observable competencies represent the knowledge, skills, and abilities that an adult education instructor should possess to be effective within that domains. Each domain has four to five competencies.

Performance Indicators Each competency has a set of indicators that articulate what the performance of this competency looks like in an adult education context.

Sample Illustrations Each performance indicator is accompanied by a sample illustration that provides examples of the practices in different adult education settings (such as a multilevel English as a second language classroom, a basic literacy class for native).

Source: (American Institutes for Research, 2015)

(Yin, Condelli, Ogut, & Cronen, 2013) analysed the variety of aspects of adult education teachers’ performance. Some of them appeared to be less significant than it was assumed at the beginning of the survey, whereas others proved to be crucial factors of effective adult education provision. The following aspects were examined:

- Demography of teachers proved to have no impact on students’ performance. There were some suggestions that some demographical features may be important (e.g. gender or age of teachers), but there were no statistically significant results, or results found in one states were not proven in another;

- Teachers' professional development programmes hugely vary among different states, so it is impossible to give one simple answer on their effectiveness. However, there is a small, but significantly positive impact of the crude fact that one has participated in such programmes;

35

- Teacher's experience proved to be one of the most important factors. It has significantly positive impact on effectiveness in all analysed aspects: overall, language and mathematics post-tests.

4.4.4 Other factorsIMPADA focus groups' participants – asked about what kind of knowledge, skills, competences, qualifications and support do trainers for disadvantaged adult learners need in order to provide effective education - gave answers such as:

- "empathy and understanding" (5/5 indications), including 'building relationships with learners', 'awareness of learners diversity', 'high levels of tolerance and respect', 'communication and listening skills', 'stimulating adult learners to have faith in their ability to learn and change', 'creating relationships of trust with learners, as a pre-condition to the effectiveness of the learning process', 'reducing distance between trainer/teacher and projects' participants by using the appropriate appearance and behaviour', 'personal skills such as psychological resistance, inner-direction, sense of mission, ability to separate work from personal life, maturity, stability, high level of energy, mindfulness etc.';

- "teaching skills" (5/5), including 'ability to teach through active methodologies and methods adapted to adult learners', 'knowledge of different learning styles and learners' differences', 'knowledge of motivation factors', 'ability to understand and influence the group process', 'ability to create training situations conducive for the group engagement', 'ability to create alternative learning situations', 'coaching skills – ability to ask the right questions';

- "equality, diversity and inclusion skills" (3/5), including 'experience of working with people from a range of disadvantaged groups, e.g. autism, schizophrenia, bipolar, depression, anxiety etc.', 'compulsory training on Prevent, CSE, safeguarding';

- "organisational, fundraising and community skills" (3/5);- "support from colleagues and employers" (3/5), including 'work shadowing and

sharing other good practices', 'coaching support, counselling, weekly meetings to discuss current personal and training issues', 'encouraging to generate demand for and interest in initiatives to nurture staff creativity', 'providing to novice trainers/teachers many opportunities to participate in internships and perform real professional tasks with the support of experienced professionals'.

(Condelli et al., 2010) proposed the following strategies to be effective:

- Contextualization of basic skills instructions and strategies;- Blended learning;- Flexible teaching content that allows to be rearranged due to changing

circumstances.

36

5 Methods of assessment

Evaluating the effectiveness of basic skills programmes is challenging and needs to be conducted over an extended period of time, taking into consideration that learners facing difficulties with basic skills do not necessarily follow a direct or uninterrupted learning path (European Commission/EACEA/Eurydice, 2015a).

Measurement the effectiveness of education for disadvantaged is more difficult and nuanced than among average students. It is a very important note: if a disadvantaged person doesn't find a job after adult education, it does not always mean that the course was ineffective. Employability is just one of the aspects, but it may serve as a good illustration. Therefore, it is advised to measure "net" effect rather than "gross" – relative improvement since joining a programme. For instance, in Polish formal education system there is being implemented a method called “Educational value added” to measure, if the absolute improvement of a student is a merit of school or some external factors (such as innate capabilities of a student). In adult education there are no such tools, so it may be found as a niche to fill.

Education providers should also focus on measuring the effectiveness of awareness-raising and outreach activities, which is found to be particularly difficult (Windisch, 2015).

As noticed in (OECD, 2013b), the field of educational evaluation is constantly growing in all levels of education and both in formal and non-formal education. This is even more important, since there is a huge demand for more effective and equitable education. Furthermore,

“Although educational evaluation within school systems is not a recent concern, it has traditionally focussed mostly on the assessment of students. In recent years, countries are increasingly developing more comprehensive evaluation and assessment frameworks.” (p. 18).

Apart from the disputes on what does the effectiveness of education mean (as well as the quality or equity), the biggest problem is how to measure it. In general, there are two main sources for assessing the direct outcomes of education:

- Self-reported returns;- Objective measures.

Both have major limitations. Self-reported returns are biased by subjective feeling, especially among disadvantaged learners who usually have bad experiences from formal education (what can lead both to under or overestimation of real effects). On the other hand, objective

37

measures often require costly monitoring methods, which may be not efficient way to assess the impact. Furthermore, they often hide some indirect effects.

In case of adult education for disadvantaged groups, it is also important to note that lack of measurable economic effect does not always mean that the course was ineffective. For example, a disadvantaged person may have problems with finding a job after graduation despite the fact that his/her competencies and relative labour market value have been improved.

Table 3 Assessment methods of learning outcomes

Learning outcome Level of understanding Assessment taskBasic facts, definition of terms (example: state the cost of materials needed to make a brick wall)

Recall and recognition, some relational and multistructural

Multiple choice, close, short answer questions

Distinguish between differentconcepts, relate facts (relational and multi-structural)

Differentiate between fair and unfair dismissal

Assignment, presentation,comprehension test, case study

Demonstration of skill(search a database by fields)

Procedural knowledge Presentation, simulationexercise, audio/videotaping

Converse fluently in foreignlanguage (conduct a one-to-one conversation about theWeather)

Relational, functional knowledge

Role play, audio/videotaping

Develop an argument anddefend own views (specify how humour is used to portray stereotypes in ‘Jeeves’)

Evaluation, relational andextended abstract

Discussion, long essay, seminar presentation

Reflect on own learning(monitor own progress)

Extended abstract, relational Log/diary

Source: (Hillier, 2005)

The report (Hanover Research, 2015) on assessment of adult education programmes in USA presented some of the most common measures. All the states follow the National Reporting System framework, which focuses on five core indicators: skill level gains, educational

38

attainment (secondary or post-secondary), employment outcomes (unsubsidised employment and retention in employment).

Table 4 Types of assessment

Formative SummativeInformal Where are we so far? What

do we want to do next?

Example: a fitness class for people with arthritis

What do we learn?

Example: an introduction to art history class

Formal How much assessment specification/learning outcomes have we achieved?

Example: an NVQ portfolio development workshop

Did we achieve all the learning outcomes? What grade/level have he attained?

Example: AVCE/A level classes

Source: (Hillier, 2005)

(Looney, 2011) proposes the distinction on two types of assessment: “formative” and “summative”. Summative assessment is more regular and common one: it involves the assessment of learner’s performance based on her achievements, based on some tests or examinations taking place by the end of the programme. Summative assessment is also a part of a validation process, since it is a way to prove, that the student obtained the learning outcomes required for some qualification (certificate or diploma) and, therefore, it may be used to get access to higher levels of education or some specific occupation.

On the other hand, formative assessment is also based on student’s performance, but it has more ongoing character and another goal: the improvement of learning process. Thus, the formative assessment may serve as a way to identify learning needs and to adjust a teaching processes.

Looney sums up the difference, noticing that, “summative assessment is sometimes referred to as assessment of learning, and formative assessment, as assessment for learning”.

5.1 Formative assessment

Derrick & Ecclestone (2008) analysed the issue of formative assessment in detail. As was mentioned in previous chapter, formative assessment is believed to be a most effective

39

strategy to improve learners’ persistence. It also has a positive impact on outcomes of adult learning, since it helps to adjust the curricula to the changing motives of learners and their progress in learning.

They identified two aspects of adult learning:

“The first of these consists of practices and activities in which the purpose is to produce evidence for the planning of future learning and/or for constructive feedback and review: these might include activities such as assignments, tests, role-play, performances, observations, questioning, etc. It is important to note that these activities can also be used primarily for purposes that are not essentially formative or to support learning, for example to fulfil bureaucratic requirements or to serve the purposes of summative assessment. Secondly, formative assessment can take the form of learning activities which aim to develop the autonomy of the learner(s) by giving them practice in the use of relevant skills and knowledge in real-life contexts, and particularly in the assessment and evaluation of their performance and those of others, in those contexts.” (p. 71).

They also identified the following aspects of formative assessment that are crucial for an effective provision:

- Formative assessment cannot be used instead of summative assessment. The latter still exists as an essential strategy in adult education. However, its tools shall be “used formatively”. (Derrick & Ecclestone, 2008, p. 73) puts it:

- “this involves finding ways to get students to <<get beneath>> and <<go beyond>> the bald results of the summative assessment processes and try to understand how they work and reflect on what they mean.”

- Formative assessment may be used in planning activities, as to adjust the plans to actual needs.

- There should be an on-going evaluation of teachers’ communication skills, particularly those involving listening, empathy and understanding;

- Feedback should be constructive and practical. It also should not be taken by a learner personally, so it has to address the task;

- The assessment should aim at “improving motivation and confidence, autonomy, and citizenship”;

- Good cooperation between teachers should lead to an exchange of questioning techniques. The authors state that open questions are more effective than those as: double questions, leading questions, rethorical questions, closed questions.

- The assessment during all the stage of adult education programmes should involve peer assessment and self-assessment;

- The assessment techniques are often not understood by learners. One of the most popular reasons for that fact is that adult learners from disadvantaged groups use different language than teacher. In practice formative assessment should involve: “Encouraging learners to develop, discuss and evaluate their own assessment criteria

40

and assessment materials, as well as collectively designing „perfect‟ answers, will at the same time help them understand and critique the language of official assessment criteria. “ (Derrick & Ecclestone, 2008, p. 75)

- There is sometimes a tension between “letter” and “spirit” of formative assessment. Formative assessment is often understood in very narrow way, that is limited to defining the learning goals. Spirit of formative assessment means though, that it is “a way of engaging learners deeply with their learning in order to develop critical and cognitive autonomy.” (ibidem)

- Formative assessment is useful in all types of adult education.

41

6 Criteria on measuring adult education effectiveness

Evaluating the effectiveness of education is critical to education programmes accountability and, of course, it applies to adult education (including education for disadvantaged groups) in the same way as to general and higher education. Basing on intuition and common sense, we can tell about the strong positive influence that literacy programmes have on adults’ employment, family relations or other areas of life – referring to the fact that such influence is indicated in the case of literacy proficiency itself. At the same time, a clear indication of reliable ways to measure the actual results of education (specific education programmes), based on properly selected criteria and adequate methods of measurement, remains among common research challenges, especially when we talk about the long-term outcomes and focus on the specific group of disadvantaged adults. It is due to the lack of surveys based on scientific approach (e.g. with the use of comparison groups and conducted within a long period of time) as well as to mutually exclusive results obtained in the completed studies focused on this issue (Moore & Silva, 1999).

In this chapter we present the typology of the criteria on measuring the effectiveness of adult education for disadvantaged groups, based – on one hand – on our research on the specific characteristics and needs of this group, and on the other hand – on findings regarding various outcomes of (adult) education, factors influencing the effectiveness of such education, as well as methods of its assessment.

6.1 Criteria oriented directly on the expected outcomes (learner’s approach)

The first broad group of criteria focuses on direct, short- and long-term outcomes of learning, while the second one is based on inferencing about the effectiveness of learning process (education programme) from indicators characterising AE providers (and depending on their educational activities and achievements).

6.1.1 Objective measures (improving in/acquisition of new skills)In case of this group the word “objective” refers to the fact that the process of measuring the effectiveness of education based on these criteria requires using assessment methods, which are aimed at the observation of certain objective indicators, and not obtaining subjective opinions. Based on our desk research, we distinguished two types of criteria that belong to this group, from which those that we assigned to the first type can be used to

42

measure the effects of learning observable in a short term, while the criteria assigned to the second subgroup are useful in the assessment of long-term results of (adult) education.

Measuring during validation process (formative and summative assessment)

This group of criteria refers to measuring the improvement in/acquisition of new selected skills by learners, whose performances are compared to those that were observable at the time before starting a learning process. This approach focuses on the outcomes that are “visible” shortly after completing a certain education programme, and thus can be assessed during the validation process (which can be directed towards a certification of specific skills). As it was highlighted in the previous chapters, in case of the adults from disadvantaged groups skills that should be learned (and thus taught) in the first place are:

i) Literacy and numeracy;ii) Digital skills;iii) Soft skills, which can be grouped as in Table 5.

Table 5 Different types of soft skills and examples of evaluation criteria

Skills Learning outcomes (assessment criteria)awareness, self-care is able to be aware of personal attitudes, skills, vocations

and deep valuesis capable of developing self-esteem and self confidenceis capable of dealing with own emotions, physical sphere and interior life

acting autonomously is able to act within the big picture, developing critical thinkingis able to form and conduct life plans and personal projects (to demonstrate self-direction, self-discipline etc.)is able to assert own rights, interests, limits and needsis capable of learningis able to adapt and take initiative

using tools interactively and in a creative way

is able to use language, symbols and text interactivelyis able to use knowledge and information interactivelyis able to use technology interactivelyis able to create arts and enjoy creativityis capable of creating (handcrafts, cooking etc.)

interacting with others, also in heterogeneous groups

is able to build relations with othersis able to interact positively with people from various backgroundsis capable of taking care of others

social and civic skills is able to manage and resolve conflicts and create coexistence

43

Skills Learning outcomes (assessment criteria)is able to demonstrate global awareness, multicultural literacy, humanitarianismis able to cooperate with others and contribute to the communityis able to demonstrate civic, ethical, and social-justice literacy

Source: UPTER’s own documentation

Measuring of long-term outcomes

The second group of objective measures oriented directly on the expected outcomes of adult education for disadvantaged groups are measures possible to obtain by comparing group’s of learners (social, vocational, educational, financial) position/situation before starting a learning process (a certain educational activity) and after some time from completing it. These criteria can relate to areas such as:

- Social wellbeing/social exclusion, measured by various indicators and scales (e.g. learners’ better existence in local communities, better participation in civic society – e.g. by voting – better access to basic needs services such as good quality water and food, appropriate shelter, public transport, social and health services, greater participation in culture etc.); (Moore & Silva, 1999) notice that there aren’t a lot of credible research studies regarding the measurement of these outcomes;

- Family literacy, mentioned by Vorhaus et al. (2011) as one of the outcomes linked with psychical wellbeing of a person (see also p. 16 of this report);

- Employability (e.g. percentage of graduates who were employed for at least 6 months by the end of the first year after graduation), which – as was stressed in the previous chapters – is one of the most often quoted outcomes, even in the case of basic skills programmes, which – as a rule – are not directly aimed at improving learners’ situation on the labour market;

- Level of professional/vocational sufficiency (e.g. increased sufficiency measured by performance indicators), which – in case of employed learners – contributes to productivity at the company level; when it comes to this kind of outcomes (Moore & Silva, 1999) stress that

“(…) companies often lack the necessary data, or will not share these data beyond corporate management, to empirically establish the programs’ effect on indicators such as reductions in safety violations, reduced employee turnover, or increased employee attendance.” (p. 10),

so this issues require further investigation;

44

- Participation in LLL – continuous/further education (e.g. percentage of graduates who successfully participated in educational activities within the first 3 years after graduation), which – as it was repeatedly stressed in this report – is one of the key success factors in case of adult education, especially AE for disadvantaged groups;

- Finances (e.g. increase in average annual/monthly salary in examined group of learners).

6.1.2 Subjective measures (satisfaction surveys)The second broad group of criteria oriented on the expected outcomes of learning activities includes measures obtained by satisfaction quantitative and qualitative surveys (e.g. self-reported returns). Although – due to the cost effectiveness and relative ease of carrying out such surveys – this approach seems to be very popular, it is also controversial because of its limited applicability and low reliability of the obtained data. (Moore & Silva, 1999) notice that some studies based, e.g., on interviews with employers and employees who were part of certain educational activities, provide results contrary to results obtained on the basis of objective criteria of measurement and with the use of control groups (see also p. 38 of this report).

The participants of IMPADA focus groups emphasise that – on one hand

“Post training evaluation should evaluate (and somehow measure) the level of satisfaction and psychological fulfillment of learners, and not only their acquired knowledge, skills and competences.” (PROMEA’s synthesized focus group data form),

and – on the other hand – that the level of satisfaction (“reaction” – referring to Kirkpatrick's ‘Four-Level Training Evaluation Model’) should only be one of a couple indicators of education efficiency (IBE’s synthesized focus group data form).

Among groups of potential respondents in the researches aimed at measuring the effectiveness of adult education for disadvantaged groups on the basis of satisfaction surveys are:

- Learners – e.g. questions about satisfaction with educational experience – seeing positive influence on someone’s own life, seeing improvement in someone’s own self-esteem or feeling more capable across certain areas, but also questions about potential improvement of learner’s family life’s quality, increased frequency of reading etc.) (Moore & Silva, 1999);

- Employers (labour market’s perspective) – e.g. employers’ satisfaction with employees’ skills acquired during a certain education programme;

- Members of learners’ families (parents, spouses, children, siblings etc.) – e.g. spouses’ of learners satisfaction with the improvement of their family life’s quality achieved thanks to the skills acquired by learners during a family literacy programme;

45

- Teachers, trainers, tutors and head teachers – who can be asked about their level of satisfaction with chosen aspects connected with their own provision (e.g. aspects connected with the quality of provision – see chapter 4.4);

- Local authorities – who, e.g., can assess chosen learning activities in relation to selected local policies and strategies.

6.2 Criteria oriented on factors depending on the quality of provision (institutional approach)

In case of this group of criteria the effectiveness of adult education for disadvantaged groups is measured with the use of statistical data referring not to individual learners or their groups, but to training providers and quality of their provision. On the basis of these data it is possible to compare these institutions’ characteristics (see chapter 4) and achievements in the field of adult education using quantitative indicators such as:

- Learners’ average attendance at classes - Learners’ retention rate/percentage of drop-outs- Exam pass rates- Increases/decreases in numbers of participants of certain education programmes in

relation to the previous editions of this programmes etc.

46

7 ConclusionsThe report presented the current state of knowledge about effectiveness of adult education. The design of the study, combining the literature review with the knowledge obtained within focus group interviews with experts in adult education, allowed to get a full understanding of the issue.

The discussion on the effectiveness of adult education for disadvantaged groups showed, that there is no consensus in a literature, how the terms “adult education” and “disadvantaged” should be understood. We proposed our understanding, given the goals of whole project.

Disadvantaged adult learners appear to be “hard-to-reach”. That means that they are less likely to participate and to persist in the educational programmes. We therefore concluded, that it is not enough to create good curricula of adult learning, but it should be complemented by the strategies improving the participation rates. That lead to a proposed division of key aspects of effective adult education programme into three groups: participation (recruitment), persistence and quality of provision.

The last chapter of our report focused on the criteria for measuring the outcomes of adult learning. We divided them into two groups: objective and subjective. The proposed measures of effectiveness are believed to be a useful source of knowledge for adult education providers on how to understand their own performance.

Such analysis should be a useful inquiry to a design of a common assessment framework and to create the tools that help to improve the effectiveness of adult education for disadvantaged groups.

47

8 BibliographyAlkema, A., & Rean, J. (2014). Adult Literacy and Numeracy: An Overview of the Evidence.

Tertiary Education Commission (TEC), New Zealand Government.

American Institutes for Research. (2015). Adult Education Teacher Competencies.

Anderson, D., & Niemi, J. (1969). Adult Education and the Disadvantaged Adult. Retrieved from http://files.eric.ed.gov/fulltext/ED029160.pdf

Bailey, T. R., & Alfonso, M. (2005). Paths to persistence: An analysis of research on program effectiveness at community colleges. Indianapolis: Lumina Foundation for Education. Retrieved from http://www.luminafoundation.org/publications/PathstoPersistence.pdf

Benseman, J., Sutton, A., & Lander, J. (2005). Working in the light of evidence as well as aspiration: A Literature Review of the best available evidence about effective adult literacy, numeracy and language teaching. Auckland: Ministry of Education.

BMBF. (2012). Alphabetisierung und Grundbildung Erwachsener: Abschlussdokumentation des Förderschwerpunktes zur Forschung und Entwicklung 2007-2012. German Federal Ministry for Education and Research.

Brooks, G., Burton, M., Cole, P., & Szczerbiński, M. (2007). Effective teaching and learning: reading. Medical education (Vol. 43). London: National Research and Development Centre for Adult Literacy and Numeracy. http://doi.org/10.1111/j.1365-2923.2009.03427.x

Card, D. (1999). The causal effect of education on earnings. In O. Ashenfelter & D. Card (Eds.), Handbook of Labor Economics (Vol. 5, pp. 1801–1863). North-Holland. http://doi.org/10.1016/S1573-4463(99)03011-4

Carneiro, P., & Heckman, J. J. (2003). Human Capital Policy (No. 9495). NBER Working Papers Series (Vol. 53). Retrieved from http://www.mdpi.com/1996-1073/2/3/556/

Casey, H., Cara, O., Eldred, J., Grief, S., Hodge, R., Ivanic, R., … Mcneil, B. (2006). “You wouldn’t expect a maths teacher to teach plastering …” Embedding literacy , language and numeracy impact on learning and achievement. London: National Research and Development Centre.

Centre for Literacy of Quebec. (2010). Social Capital Outcomes of Adult Learning and Literacy Initiatives How Do We Measure Them? Literature Review.

Champagne, D. (1987). Disadvantaged Adult Learners: Can Career Counseling Enhance Adult Education Program Effectiveness? Adult Education Quarterly, 37, 63–77. http://doi.org/10.1177/0001848187037002001

Coben, D., Brown, M., Rhodes, V., Swain, J., Ananiadou, K., Brown, P., … Storey, V. (2007). Effective teaching and learning: Numeracy. London: National Research and Development Centre for Adult Literacy and Numeracy. Retrieved from http://dera.ioe.ac.uk/22306/1/doc_3349.pdf

Condelli, L., Kirshtein, R., Silver-Pacuilla, H., Reder, S., & Wrihley, H. (2010). Changing the

48

Odds: Informing Policy with Research on How Adult Learners Succeed. Washington, D.C.: American Institutes for Research. http://doi.org/10.2307/2654781

Derrick, J., & Ecclestone, K. (2008). English-language Literature Review. In Teaching, Learning and Assessment for Adults: Improving Foundation Skills. OECD Publishing. http://doi.org/http://dx.doi.org/10.1787/172251338713

Desjardins, R., & Rubenson, K. (2013). Participation patterns in adult education: The role of institutions and public policy frameworks in resolving coordination problems. European Journal of Education, 48(2), 262–280. http://doi.org/10.1111/ejed.12029

European Commission. (2016). Erasmus+ Programme Guide. http://doi.org/10.1109/CICEM.2013.6820130

European Commission, EACEA, & Eurydice. (2015). Adult Education and Training in Europe: Widening Access to Learning Opportunities. Luxembourg: Publications Office of the European Union. http://doi.org/10.2797/75257

Evans, K., & Waite, E. (2008). Adult Workers’ Engagement in formal and informal learning: insights into workplace basic skills from four UK organisations. Ottawa: Partnerships in Learning. Retrieved from http://www.tlrp.org/dspace/retrieve/3966/formal+informal+learning+paper.pdf

Grief, S., Meyer, B., & Burgess, A. (2007). Effective teaching and learning: reading. (Vol. 43). London: National Research and Development Centre for Adult Literacy and Numeracy. Retrieved from http://dera.ioe.ac.uk/22303/2/doc_3346.pdf

Hamilton, M., & Wilson, A. (2005). New ways of engaging new learners: Lessons from round one of the practitioner-led research initiative. London: National Research and Development Centre for Adult Literacy and Numeracy.

Hanover Research. (2015). MEASURING NEED AND EFFECTIVENESS IN ADULT EDUCATION, (October).

Heckman, J. J., Lochner, L. J., & Todd, P. E. (2003). Earnings Functions, Rates of Return and Treatment Effects: The Mincer Equation and Beyond (NBER Working Papers No. 9732). Retrieved from http://www.nber.org/papers/w9732

Hillier, E. (2005). Reflective Teaching in Further and Adult Education. London: Continuum. http://doi.org/10.1017/CBO9781107415324.004

Jarvis, P. (2005). International dictionary of adult and continuing education. Taylor & Francis.

Litster, J. (2007). What keeps learners on track. Reflect. The Magazine of NRDC, (7), 4–8.

Looney, J. (2008). Teaching, learning and assessment for adults: improving foundation skills . Retrieved from http://84.88.0.227/record=b1887676#

Looney, J. (2011). Integrating formative and summative assessment: Progress toward a seamless system? (OECD Education Working Paper No. 58).

MacLeod, S., & Straw, S. (2010). Adult basic skills. CfBT Education Trust. Reading: CfBT Education Trust.

49

Martin, J. P., & Grubb, D. (2001). What Works and for Whom: A Review of OECD Countries’ Experiences with Active Labour Market Policies. IFAU - Office for Labour Market Policy Evaluation. Retrieved from http://www.ssrn.com/abstract=348621

Mcdonnell, R., & Soricone, L. (2014). Promoting Persistence through Comprehensive Student Supports, (March).

Mcintosh, S. (2004). Literature Review of the Effects of Improvements in Adult Basic Skills Report Prepared for the National Assembly for Wales. National Assembly for Wales.

McMahon, W. W. (1999). Education and Development. Measuring the Social Benefits. Oxford: Oxford University Press.

Moore, M. T., & Silva, T. (1999). Evaluating the effectiveness of adult education programs.

Myers, K., & Myles, J. (2005). Self-assessed Returns to Adult Education: Life-long Learning and the Educationally Disadvantaged.

NAO. (2008). Skills for life: Progress in improving adult literacy and numeracy.

National Reporting System for Adult Education. (2015). Implementation Guidelines. Measures and Methods for the National Reporting System for Adult Education. Washington, D.C.: U.S. Department of Education.

OECD. (2006). Promoting Adult Learning. Innovations in Education and Teaching International (OECD Publi, Vol. 23). Paris, France. http://doi.org/10.1080/0033039860230310

OECD. (2013a). OECD Skills Outlook 2013: First Results from the Survey of Adult Skills. OECD Publishing. http://doi.org/10.1787/9789264204256-en

OECD. (2013b). Synergies for Better Learning. OECD Reviews of Evaluation and Assessment in Education. http://doi.org/10.1787/9789264190658-en

Ofsted. (2003). Literacy, Numeracy and English for Speakers of Other Languages: a Survey of Current Practice in Post-16 and Adult Provision. Office For Standards In Education. Retrieved from http://www.ofsted.gov.uk/assets/3410.pdf

Ofsted. (2007). Direct learning support in colleges: a survey of current practice. London: Office For Standards In Education, Children’s Services and Skills.

Ofsted. (2011). Removing barriers to literacy. London: Office For Standards In Education, Children’s Services and Skills. Retrieved from http://www.ofsted.gov.uk/sites/default/files/documents/surveys-and-good-practice/r/Removing barriers to literacy.pdf

Parsons, S., & Bynner, J. (2007). Illuminating disadvantage: Profiling the experiences of adults with Entry level literacy or numeracy over the lifecourse Samantha. London: National Research and Development Centre for Adult Literacy and Numeracy.

Porras-Hernández, L. H., & Salinas-Amescua, B. (2012). Nonpraticipation in Adult Education: From Self-Perceptions to Alternative Explanations. Adult Education Quarterly, 62(4), 311–331. http://doi.org/101177 074171361 i40698

50

Porter, K. E. ., Cuban, S., & Comings, J. P. (2005). “One day I will make it”. A Study of Adult Student Persistence in Library Literacy Programs. MDRC. Retrieved from http://www.mdrc.org/sites/default/files/full_3.pdf

PROMEA. (2016). IMPADA. Focus groups data analysis.

Psacharopoulos, G., & Patrinos, H. (2002). Returns to Investments in Education: A Further Update. World Bank Economic Policy Working Series (Vol. No. 2881).

Siebert, H. (2003). Didaktisches Handeln in der Erwachsenenbildung. Didaktik aus konstruktivistischer Sicht. München: Luchterhand Verlag.

Speckesser, S., & Bewley, H. (2006). The longer term outcomes of Work-Based Learning for Adults : Evidence from (DWP research report No. 390). Retrieved from http://www.dwp.gov.uk/asd/asd5/rports2005-2006/rrep390.pdf

Spence, M. (1973). Job Market Signaling. The Quarterly Journal of Economics, 87(3), 355–374. http://doi.org/10.2307/1882010

Steiner, K., Voglhofer, M., Schneeweiß, S., Tamara Baca, & Fellinger-Fritz, A. (2012). Methoden in der Berufs- und Arbeitsmarktorientierung für Bildungsferne. Wien: Arbeitsmarktservice Österreich. Retrieved from http://www.forschungsnetzwerk.at/downloadpub/AMS_PH_Bildungsferne_2012.pdf

Swain, J., Welby, S., Brooks, G., Bosley, S., Frumkin, L., Fairfax-Cholmeley, K., … Cara, O. (2009). Learning literacy together : the impact and effectiveness of family literacy on parents, children, families and schools. London: National Research and Development Centre for Adult Literacy and Numeracy.

Vorhaus, J., Litster, J., Frearson, M., & Johnson, S. (2011). Review of research and evaluation on improving adult literacy and numeracy skills (No. 61).

Windisch, H. C. (2015). Adults with low literacy and numeracy skills INTERVENTION. OECD Education Working Papers, (123). http://doi.org/10.1787/5jrxnjdd3r5k-en

Yin, M., Condelli, L., Ogut, B., & Cronen, S. (2013). The Importance of Teacher Background Qualifications for Student Learning.

51