addressing the completion challenge in portuguese higher ... · june 2017 addressing the completion...

Mossavar-Rahmani Center for Business & Government

Weil Hall | Harvard Kennedy School | www.hks.harvard.edu/mrcbg

M-RCBG Associate Working Paper Series | No. 81

The views expressed in the M-RCBG Associate Working Paper Series are those of the author(s) and do

not necessarily reflect those of the Mossavar-Rahmani Center for Business & Government or of

Harvard University. The papers in this series have not undergone formal review and approval; they are

presented to elicit feedback and to encourage debate on important public policy challenges. Copyright

belongs to the author(s). Papers may be downloaded for personal use only.

Addressing the Completion Challenge in

Portuguese Higher Education

Jonathan Williams

September 2017

ADDRESSING THE COMPLETION CHALLENGE IN

PORTUGUESE HIGHER EDUCATION

Technical Report

Jonathan Williams, Harvard Kennedy School of Government [email protected]

June 22, 2017

Prepared for: The Ministry for Science, Technology and Higher Education of Portugal and the

OECD Directorate for Education and Skills

This Policy Analysis Exercise reflects the views of the author and should not be viewed as

representing the views of the PAE's external clients listed above, nor those of Harvard University

or any of its faculty

Acknowledgements

I owe many thanks for the support I received on this project. Firstly, thank you to Pedro Barrias and the MCTES, and Thomas Weko and the Policy Analysis and Implementation team at the OECD Directorate for Education and Skills, for taking me on for this work and provide advice and assistance throughout the length of the project. Special thanks to Anna Pons and Cláudia Sarrico for your thoughtful feedback on document drafts. Secondly, muito obrigado to my interviewees in Portugal and at the OECD for your patience and thoughtfulness. I was sincerely impressed by the passion and commitment that each of you displayed towards addressing the challenges facing students in Portugal. Additional thanks to those who sent follow up information. Thirdly, thank you to my supervisor, Professor Josh Goodman, and Seminar Leaders Professors John Haigh and Julie Boatright Wilson. Sorry the report is so long. I am also thankful for financial support for the study visit from Degree Programs and Student Affairs and from the Malcolm Weiner Center for Social Policy at the Harvard John F. Kennedy School of Government. Fourthly, thanks to my colleagues at the Kennedy School for listening to my thoughts and sharing your own. And finally, thanks as always to my parents and Dima, for putting up with me and giving me solid advice throughout this project.

June 2017 Addressing the Completion Challenge in Portuguese Higher Education Notes

i

Table of Contents Abbreviations Guide .............................................................................................................. ii

Executive Summary ............................................................................................................... 1

Chapter 1: Introduction ......................................................................................................... 5

Chapter 2: Measuring and Conceptualizing Study Success..................................................... 19

Chapter 3: Admissions ......................................................................................................... 53

Chapter 4: Quality of Education Delivery .............................................................................. 77

Chapter 5: Financing .......................................................................................................... 123

Chapter 6: Options and Recommendations ........................................................................ 151

Appendix I: List of Interviewees ......................................................................................... 179

Appendix II: Summary of Analysis of Policy Options ........................................................... 181


ii

Abbreviations Guide

Abbreviation English Original Portuguese

A3ES Higher Education Evaluation and Accreditation Agency

Agência de Avaliação e Acreditação do Ensino Superior

AV Visual Arts Artes Visuais BE Study Bursary Bolsa de Estudo

CCSSE Community College Survey of Student Engagement

CEF Education and Training Programmes Cursos de Educação e Formação

CET Technological Specialisation Programmes

Cursos de Especialização Tecnológica

CH Scientific-Humanistic Científico-Humanístico CHEPS Centre for Higher Education Policy

Studies

CNA National Access Competition Concurso Nacional de Acesso CNAES National Commission for Access to

Higher Education Comissão Nacional de Acesso ao Ensino Superior

CNAPPES National Congress on Pedagogical Practices in Higher Education

Congresso Nacional de Práticas Pedagógicas no Ensino Superior

CS Social and Economic Sciences Ciências Socioeconómicas CT Science and Technology Ciências e Tecnologias

DGEEC Directorate-General for Education and Science Statistics

Direção-Geral de Estatísticas da Educação e Ciência

DGES Directorate-General for Higher Education

Direção-Geral do Ensino Superior

ECB European Central Bank

ECTS European Credit Transfer and Articulation System

EFA Adult Education and Training Programmes

Cursos de Educação e Formação de Adultos

ES Non-integrated higher education schools

Escolas Superiores Não Integradas

ESCS PISA index of economic, social and cultural status

EU European Union GDP Gross Domestic Product

IEFP Employment and Vocational Training Institute

Instituto do Emprego e Formação Profissional

IP Polytechnic Institute Instituto Politécnico


i

Abbreviation English Original Portuguese ISCED International Standard Classification

of Education

ISEG Lisbon School of Economics and Management

Instituto Superior de Economia e Gestão, Universidade de Lisboa

LH Languages and Humanities Línguas e Humanidades M23 Special admissions competition for

applicants over 23 years of age Mais de 23

MCTES Ministry of Science, Technology and Higher Education

Ministerio de Ciência, Tecnologia e Ensino Superior

NC Numerus Clausus Numerus Clausus NSS National Student Survey

NSSE National Survey of Student Engagement

OECD Organisation for Economic Cooperation and Development

PISA Programme for International Student Assessment

R&D Research and development

RGA General Access Stream Regime Geral de Acesso SES Socioeconomic status

TESP Higher Technical Professional Programmes

Cursos Técnicos Superiores Profissionais

UK United Kingdom

UKES UK Engagement Survey UNIVA Career Centres Unidades de Inserção na Vida Ativa

June 2017 Addressing the Completion Challenge in Portuguese Higher Education Executive Summary

1

Executive Summary Portugal has faced a significant economic crisis in recent years that is deeply rooted in aging demographics and low productivity. It is evident that strengthening the population’s skills is essential to improve competitiveness. Given this, the Government of Portugal has set an ambitious target for higher education attainment, which can only be achieved through both expanded access and higher completion. This study is seeking to answer the following research question: What steps can the Portuguese government take during its current mandate (2017 to 2020) to initiate a systematic increase in the higher education completion? At many points, we use the somewhat broader term of study success, but completion remains our emphasis.

Understanding Completion and Study Success OECD data on completion rates places Portugal somewhat below the average of other member countries, notwithstanding challenges in comparing data collected using different methodologies. At the national level, data on dropout after first year indicates significant differences between institution types, institutions and study programmes. Public institutions, universities, more competitive programmes, and Licenciado degrees tend to have lower dropout rates than private institutions, polytechnic institutes, less competitive programmes and Master’s degrees. Students typically drop out in first year.

International evidence indicates that while study success matters for individuals, families, institutions and society, the challenge is not merely to maximise the completion rate, or minimise dropout and time-to-completion. Some very high quality higher education systems have low completion rates by OECD standards, based on the roles they play within their specific societies. Efforts to raise completion must take into account trade-offs with access, student learning, and time-to-completion. Ultimately, Portugal should establish a target for completion rates that corresponds with its own economic and social circumstances and objectives. The international literature identifies intervening factors that influence study success, i.e. based on societal trends that are independent from higher education system policies but interact with these. These factors are (1) non-modifiable student characteristics (socioeconomic status,


2

gender, and ethnic origin), (2) student academic skills (based on their prior educational experiences), and (3) student motivations (relating to expectations, sense of belonging and labour market goals). We consider three main areas where higher education policies have an impact on completion: admissions, quality of education delivery, and financing.

Admissions We analyse how Portugal’s higher education admissions system promotes or hinders the effective matching of students, institutions and programmes, by balancing access and study success through smart selectivity, fostering accurate student expectations, and connecting students with programmes that meet their expectations. Portuguese students gain admission to higher education through a number of different pathways. At public institutions, most traditional-age students use the National Access Competition (Concurso Nacional de Acesso, CNA), which is based on their grades secondary education and especially from secondary education completion exams. Pathways for students at private institutions, as well as those for non-traditional students, are controlled at the institutional level pursuant to some national standards. The Government controls the total number of spaces in institutions and study programmes through Numerus Clausus (NC). We identify four ways in which Portugal’s admissions system may undermine study success. Alternative admissions streams lead to high dropout to such an extent that they may simply be setting students up to fail. The CNA promotes shallow matching based on grades that are unreliable and that provide a limited indication of student attributes at best. NC restricts access to sought-after programmes, which promotes dissatisfaction among some and undermines competition between institutions. Finally, there is insufficient information on the student experience, in terms of student engagement or student satisfaction survey results. International jurisdictions have adopted many different approaches to promote higher study success through improvements in admissions. These include efforts to expand criteria for admissions and improve information for students upon application and enrolment.

Quality of Education Delivery This chapter analyses how the quality of education delivery affects study success based on the following criteria: institutional commitment; tracking and monitoring of students; learning, teaching and assessment; flexibility; and social integration and support services. Overall, we find that the Portuguese higher education system is characterised by uneven institutional commitment in terms of strategic prioritisation and service offerings. Also evident is a widespread failure to identify and provide targeted support to students at risk of dropping out.


3

The system has a host of more strictly academic challenges. There is insufficient focus on teaching and serving students, for example in terms of strengthening pedagogy and being responsive to student views. Rates of academic failure are high, while many students who fail reenroll but actually do not attend classes or assessments. Rigid programme structures result from extensive required courses starting in first year and very few optional classes, affecting student satisfaction and flexibility. Students appear to have excessive class time, contributing to difficulties balancing schedules and responsibilities. The polytechnic role is undermined by the unclear binary distinction, which is most apparent in the requirement that IP faculty have doctoral degrees. Finally, the system offers inadequate remedial support for students with weaker academic preparation and is poorly adapted to the needs of mature students. On the non-academic side, the system has important weaknesses in first year student integration, but also suffers from limited student engagement more broadly. As well, the quality of student services is uncertain at best, and it is apparent specifically that supports for students with disabilities are underdeveloped compared with other jurisdictions. International jurisdictions have sought to strengthen the delivery of education to promote study success through policies we classify into three themes. The first is reshaping programme options through new degree structures, greater flexibility through credit transfer and recognition, or tighter restrictions on maximum enrolment periods. The second is improving academic and social supports, often focused especially on supporting students’ transitions into higher education. The last theme is improving the collection and use of information in terms of tracking students’ progression, integrating study success metrics in quality assurance processes, and building structures to share best practices.

Financing We analyse the adequacy of financial support to Portuguese higher education institutions and students and how the Government of Portugal’s funding and financial aid programmes affect institutional commitment to study success and student motivation. Portuguese higher education institutions are funded through a combination of government grants and private funds – namely tuition fees. The Government of Portugal also offers needs based grants to students and provides backing for private student loans, but take-up of loans is minimal. In addition, close to half of students pursue paid employment, and legislation and policies provide for many accommodations for these students. There is no question that institutional funding in Portugal is limited by international standards. However, there are also weak incentives in the institutional funding structure, as core funding is not tied to performance. The Government has provided only limited targeted funding for key initiatives to promote study success.


4

On the student side, many find it a struggle to balance studies and employment. This likely relates in part to the insufficiency of student financial aid. Financial aid eligibility criteria may also undermine completion, including by making it very difficult to regain eligibility once it is lost for academic reasons. Finally, Portugal provides negligible disability-related financial aid, such that it is certain that most students with disabilities do not receive any targeted support. International jurisdictions have sought to promote study success through institutional funding and financial aid measures. Governments have provided targeted institutional funding for support services, or linked funding with performance through a funding formula or performance agreements. Similarly, governments have expanded financial aid for students with need, but also introduced stronger conditions on funding amounts or repayment often based on study progress.

Analysis of Options and Recommendations We prioritise factors in study success using a heuristic based on each factor’s importance in determining overall completion rates, our confidence in related research findings, and our assessment of how susceptible the factor is to policy interventions by government. This assessment clearly emphasises the following concerns: Failure to assist students at risk of dropping out; weak incentives in the institutional funding structure; Uneven institutional commitment to study success; and high rates of academic failure. A second heuristic helps us categorise which factors the Portuguese Government can address directly (first-level), or through influencing the behaviour of institutions (second-level), or even institutional units or individual stakeholders (third-level). The Government has considerable direct control over financial factors and the admissions systems, but in general cannot address quality challenges directly. Institutional leaders, faculty, administrative staff, student representatives and quality assurance agency staff all must play a role in raising completion. We review 18 policy options under three themes: (1) Targeted Supports for Students at the Margins, (2) System Steering, and (3) Shaping Expectations of Student Success. Our analysis relies on four main criteria: impact on completion, risks, flexibility, and feasibility. We do not estimate costs of implementing our proposals, however we only consider policy options that we believe could be pursued with modest additional expenditure beginning in the short-term.

Our three primary recommendations for the Government of Portugal are:

1. Provide targeted institutional funding for services to identify and support students at risk of dropping out

2. Include performance criteria relating to study success within an institutional funding formula

3. Tighten rules regarding academic suspension and monitor institutional compliance Ten secondary recommendations aim to complement the primary recommendations and/or provide intermediate solutions as other policies are in development.

June 2017 Addressing the Completion Challenge in Portuguese Higher Education Chapter 1

5

Chapter 1: Introduction Portugal has faced a significant economic crisis in recent years that is deeply rooted in aging demographics and low productivity. It is evident that strengthening the population’s skills is essential to improve competitiveness. Given this, the Government of Portugal has set an ambitious target for higher education attainment, which can only be achieved through both expanded access and higher completion. This study seeks to answer the research question: What steps can the Portuguese government take during its current mandate (2017 to 2020) to initiate a systematic increase in the higher education completion? We describe the higher education system’s current performance in terms of completion and drop-out and identify key factors that undermine study success, relating to admissions, the quality of education delivery, and financing. We offer three primary recommendations to the Government of Portugal: (1) provide targeted institutional funding for services to identify and support students at risk of dropping out; (2) include performance criteria relating to study success within an institutional funding formula; and (3) tighten rules regarding academic suspension and monitor institutional compliance.


6

Chapter Guide

Context of the Report ............................................................................................................ 7

Research Question and Methods ......................................................................................... 10

Measures of Study Success .................................................................................................. 12

Factors in Study Success....................................................................................................... 13

Policy Options and Recommendations ................................................................................. 15

List of Figures Figure 1.1: Portugal’s projected population age profile, 2015-2080............................................ 8 Figure 1.2: Tertiary educational attainment of the 30-34-year-old population relative to the EU 2020 target, 2005-2015 ........................................................................................................... 10


7

Context of the Report Portugal was one of the five Euro-Members worst affected by the global financial crisis.1 Euro membership had allowed the country to access cheap credit, but Greece and Ireland’s debt-related problems in the aftermath of the global financial crisis made investors conscious of the fundamental weaknesses in the Portuguese economy, as well as those of Spain and Italy. Investor confidence fell sharply. Portuguese bonds were soon classified as junk (BBB-). Rates on ten-year bonds went from 4.5% in May 2009 to a peak of 14.7% in January 2014. in 2011 accepted a 78 billion Euro bailout from the European Central Bank (ECB), the European Commission and the International Monetary Fund (IMF), conditional on the adoption of a host of policies aimed at improving the country’s fiscal position. Interest rates have since fallen back to roughly 4%, as a result of monetary easing by the ECB and other factors. Public debt to GDP peaked in 2014 at 140%. Yet Portugal’s circumstances have been quite different from Greece, the most troubled of the Eurozone’s crisis economies. Whereas Greece’s public debt challenge is entrenched and longstanding, Portugal’s public-to-GDP ratio was only 62% in 2006, slightly up from 52% in 2000. The root of Portugal’s difficulties has been its inability to foster sustainable growth. Annual real GDP growth has averaged just 0.24% since 2000 and only exceeded 2% in a single year.2 The OECD suggests labour trends have been subtracting from GDP since 2006, due mainly to demographic decline.3 Population growth has been negative since 2011, and is projected to remain negative for the foreseeable future primarily due to Portugal’s low birthrate.4 Portugal is one of the oldest countries in the world, with an estimated median age of 43.6 in 2015, which is above the EU28 average and projected to rise to 49.5 by 2030. The proportion of Portuguese residents above the age of 65 has increased rapidly from 16.3% in 2000 to 20.8% in 2015. The proportion aged 15-64 peaked in 2000 and has since fallen by more than 2%. As indicated in Figure 1.1, moving forward these trends will build further, with the dependency ratio (the ratio of citizens over the age of 65 to citizens aged 14-64) rising steeply for the foreseeable future.5 The major cities of Lisbon and Porto will age less quickly, many rural areas will be even worse off.6 Such demographic challenges are common across Europe, but affecting Portugal especially deeply.

1 Mark Blyth, Austerity: The History of a Dangerous Idea (New York: Oxford University Press, 2013). 2 OECD Real GDP Growth Data. 3 OECD, “OECD Economic Surveys: Portugal 2017” (Paris: OECD Publishing, 2017), http://dx.doi.org/10.1787/eco_surveys-prt-2017-en. 4 European Commission, “Database - Eurostat,” Ec.Europa.Eu, March 5, 2017, http://ec.europa.eu/eurostat/data/database. 5 Ibid. 6 Hans Vossensteyn et al., “Dropout and Completion in Higher Education in Europe” (European Union, 2015).


8

Figure 1.1: Portugal’s projected population age profile, 2015-2080

Increased productivity is the foundation of competitiveness and economic prosperity, and practically the only means to expand wealth with a shrinking workforce. Portugal’s 2015 labor productivity, measured as GDP per hour worked (constant 2010 US Dollars, PPP), was 39% below the Eurozone average, 31.8% below the EU28 average and 30.6% below the OECD average.7 Of even greater concern, multifactor productivity has actually been shrinking since 2000 at an average annual rate of 0.1% and GDP per hour worked has grown just 1% per year since 2001%.8 Portugal faces significant challenges in generating investments in physical capital that could raise productivity. In the short term, this relates to the significant difficulties of its banking sector.9 The OECD’s most recent Economic Survey of Portugal argues that low skills levels are the critical factor in Portugal’s longer-term productivity challenge.10 One study estimated that low managerial skills alone account for around 30% of Portugal’s productivity gap relative to the US. Over two-thirds of Portuguese firms report that finding employees with required skills is an obstacle to their operations. In terms of basic education, Portugal’s share of the population aged 25-64 with upper secondary attainment is the third lowest in the OECD, whereas the rate of early school leaving among youths aged 18-24 is very high at 14%.11. Nevertheless, both these figures have improved considerably in the past few decades. For example, the rate of early school leaving was 63% in 1991. In higher education, the pattern of improvement but still low performance is much the same. Portugal is still well behind its peers in attainment of 25-34-year-olds even though its attainment advantage of 25-34-year-olds over 55-64-year-olds is greater than across the EU and the OECD, as shown in Figure 1.2.12

7 OECD, “Level of GDP per Capita and Productivity,” OECD.Stat, March 5, 2017, http://stats.oecd.org/index.aspx?DatasetCode=PDB_LV. 8 Ibid. 9 OECD, “OECD Economic Surveys: Portugal 2017.” 10 Ibid. 11 OECD, “Education at a Glance 2016: OECD Indicators” (Paris: OECD Publishing, 2016). 12 Ibid.

0%

10%

20%

30%

40%

50%

60%

70%

80%

2015 2020 2030 2040 2050 2060 2070 2080

0-14

15-64

65+

Dependency Ratio


9

Figure 1.2: Total tertiary attainment as a share of the adult population by age group, 2015

Employers are not providing on the job training to compensate for these formal education shortfalls. A 2015 survey found just 17.6% of Portuguese workers had received on the job training in the previous year, less than half the EU28 average of 41.9%.13 Among EU members only Hungary had lower workplace training participation. The Government of Portugal is committed to improving educational outcomes and attainment as part of its efforts to not only strengthen the country’s economy, but Portuguese society more broadly. Of special importance is the country’s goal to meet the Europe 2020 target that at least 40% of 30-34-year-olds should complete tertiary level education. At first glance, Portugal appears on pace to meet this target based on progress to date, as indicated in Figure 1.2.14 However, analysis completed by the Government of Portugal highlights the near impossibility of accomplishing the Europe 2020 through expanded access alone.15 A convergence scenario would require 20% increases in the number of graduates in relevant age groups in 2016 and 2017, followed by 40% annual increases from 2018 to 2020, as well as an improved balance of skilled international migration. These access and migration outcomes would be almost impossible to reach on such a short timeline. To successfully meet its goals for increased attainment, Portugal will need to not only to expand access, but to raise the proportion of students who complete their degrees.

13 OECD, “OECD Economic Surveys: Portugal 2017.” 14 “Europe 2020 in Portugal - European Commission,” accessed January 5, 2017, http://ec.europa.eu/europe2020/europe-2020-in-your-country/portugal/progress-towards-2020-targets/index_en.htm. 15 DGEEC, “Tertiary Education Attainment: 30 to 34 Years Old Population - Data and Projections” (Lisbon: Direção-Geral de Estatísticas da Educação e Ciência, 2016).

0%

5%

10%

15%

20%

25%

30%

35%

40%

45%

25-64 25-34 55-64

Portugal

EU-22

OECD


10

Figure 1.2: Tertiary educational attainment of the 30-34-year-old population relative to the EU 2020 target, 2005-2015

Study success has been on the policy agenda in Portugal for approximately 20 years, albeit unevenly.16 In 2013 the Assembly of the Republic discussed the issue in depth and notably recommended rigorous annual reporting on dropout. On June 3, 2016, the current government established a clear commitment to improving completion through a resolution of the Portuguese Council of Ministers that set the following goal: To ensure that higher education institutions […] make the following commitment during the current legislature: […] to reduce failure and drop-out rates to international reference levels.

Research Question and Methods The specific genesis of this report is the launching of an OECD Review of the Portuguese Science, Technology and Higher Education Systems and Policies, on request from the Ministry of Science, Technology and Higher Education (Ministerio de Ciência, Tecnologia e Ensino Superior, MCTES). The OECD Review’s terms of reference include the following research question: What steps can be taken at institutional and national levels to increase degree completion and reduce drop-outs? Based on this question and the Council of Ministers motion, our study seeks to answer the following research question: What steps can the Portuguese government take during its current mandate (2017 to 2020) to initiate a systematic increase in the higher education completion? At many points in the report, we use the somewhat broader term of study success, but completion remains our point of emphasis. To help develop our answer the primary research question, the study had to address four sub-questions:

16 DGEEC - Equipa de Estudos de Educação e Ciência, “Draft: Medidas de Promoção Do Sucesso Escolar Nas Instituições de Ensino Superior Públicas Em Portugal, Tal Como São Relevadas Nos Respetivos Sítios” (Lisboa: DGEEC, Unpublished).

0%

10%

20%

30%

40%

50%20

05

200

6

200

7

200

8

200

9

201

0

201

1

201

2

201

3

201

4

201

5

201

6

201

7

201

8

201

9

202

0

Portugal

EU

2020 Target

Linear (Portugal)

Linear (EU)


11

1. What data is available on trends in higher education completion across the system and at the institutional level? Chapter 2 provides an overview of the available data along with an overview of the concept of study success.

2. What are the drivers of non-completion in tertiary education in Portugal? Chapter 2 considers individual-level factors that affect completion independently from – but often in interaction with – policies of the higher education system itself. Chapters 3, 4 and 5 explore systemic factors in terms of admissions, quality of education delivery, and higher education finance.

3. What strategies and practices have other countries adopted to increase completion rates

in higher education? International approaches are noted in the closing sections of Chapters 3, 4 and 5.

4. What steering mechanisms can the Portuguese Government use to influence the

behaviour of the higher education system and increase programme completion? We focus specifically on measures that support system-wide changes of approach to address the factors in non-completion identified during the study, endeavouring to take into account constraints on administrative, financial and political feasibility. The analysis of policy options accompanies precedes our recommendations in Chapter 6.

We focus principally on the public higher education system, as a means to control scope and given data limitations regarding private sector institutions. The study relies on two key research methods. The first is an extensive literature review in English and Portuguese. We consulted academic journal articles, reports published by European higher education bodies, reports prepared by the Government of Portugal and associated agencies, and institutional surveys and presentations. Multiple reports rely on surveys of active students or students who dropped out of higher education. The most frequently cited surveys are from the the Universidade de Évora,17 and the Insituto Politécnico (IP) de Setúbal.18,19 Student survey responses are understood to reflect subjective perceptions, not necessarily precise facts. Another key source is a draft (unpublished) survey of institutional websites by the Directorate-General for Education and Science Statistics (Direção-Geral de Estatísticas da Educação e Ciência, DGEEC), which explores whether institutions are implementing 30 activities or services relevant

17 Rosalina Pisco Costa et al., “Identificação Das Causas Do Abandono Escolar Na Universidade de Évora” (Évora: Universidade de Évora, 2015). 18 IP de Setúbal, “O Abandono Nos Cursos de Primeiro Ciclo” (Setúbal: Instituto Politécnico de Setúbal, 2013). 19 IP de Setúbal, “O Abandono Nos Cursos de Segundo Ciclo” (Setúbal: Instituto Politécnico de Setúbal, 2014).


12

to completion. This survey’s calculations likely understate institutional activity given some activities may not be described online.20 We lean heavily on a review of study success policies across Europe published in late 2015 by the Centre for Higher Education Policy Studies (CHEPS).21 The report includes a literature review that provides much of the basis for our conceptual understanding of factors in higher education. It is also our key resources on policy approaches adopted in other jurisdictions. The second main method was interviews with stakeholders and experts within Portugal, as well as staff at the OECD’s Directorate for Education and Skills. A list of interviewees is provided in Appendix I. Conversations with OECD representatives occurred throughout the length of the project by telephone or Skype. Interviews with Portuguese representatives occurred primarily during a study visit from March 11 to March 18, 2017. The MCTES assisted in identifying interviewees and scheduling meetings. The study visit emphasised reality-testing findings from the literature review, exploring what institutions were doing to improve completion, and digging into the desirability and feasibility of different policy options. In advance, interviewees were provided a document introducing the project and indicating factors in non-completion identified through the literature review up to that point.

Measures of Study Success There are three key measures of study success: the completion rate, the drop-out rate, and time-to degree.22 These indicators complement each other to provide a more complete understanding of study success, but our focus is on the first two. The completion rate is the most comprehensive, tracking the proportion of students who begin a study programme at a higher education institution who complete the study programme successfully over a given period of time. The drop-out rate tracks the number of students in a study programme who fail to re-enrol in subsequent year and did not graduate, and allows for more immediate measurement of trends that feed into the completion rate given completion rates can take time to generate. The OECD provides the best comparative data on completion rates in higher education. In 2014, Portugal’s cross-cohort completion rate among Licenciatura (Bachelor’s) students was 65%, well below the 75% figure among OECD countries using this completion measure.23 However, only seven countries used the same cross-cohort methodology measure in 2014. Making a very imperfect comparison with completion rates using the true-cohort methodology after the length

20 DGEEC - Equipa de Estudos de Educação e Ciência, “Draft: Medidas de Promoção Do Sucesso Escolar Nas Instituições de Ensino Superior Públicas Em Portugal, Tal Como São Relevadas Nos Respetivos Sítios.” 21 Vossensteyn et al., “Dropout and Completion in Higher Education in Europe.” 22 Our focus in this study is on study success endogenous to a study programme, i.e. not considering the student’s subsequent pathway including employment. Other definitions may also emphasise grades, as a measure of student learning, but this falls outside our scope. 23 OECD, “Education at a Glance 2016: OECD Indicators.”


13

of the study programme plus three years, Portuguese Licenciatura students again fell behind the average across 15 other OECD countries (69%). Data comparability challenges and the details of factors in study success suggest that international reference levels are not terribly meaningful. Considering results across other jurisdictions, it is clear that systems can have low study success and still be successful, or high study success and be relatively low performing overall, depending on the role of higher education within the specific society and economy. Portugal should establish a target for completion rates that corresponds with the its own economic and social circumstances and objectives. For further detail in terms of study success across Portugal’s higher education system, we have to rely on students’ status at the end of their first year of study. The available measure of students’ status after one year of study can tell us about how study success varies between different institutions and study programmes. Institutional surveys in general, although not always, find highest levels of dropout in first year, but our measure remains limited given students can choose to continue, change or abandon their study programme throughout the programme’s duration.24 25 26 These data indicate that approximately one-in-ten Portuguese higher education students fail to re-enrol after their first year.27 By institution type, universities perform better than IPs, and public institutions better than privates. We also find considerable differences between individual institutions, especially in the private sector, as well as wide ranges in dropout rates at the programme level. These observations suggest dropout rates may be shaped as much by the types of study programmes that institutions offer as their performance.28 29 30

Factors in Study Success Study success is a complex phenomenon with numerous contributing factors. Most of this report aims to understand and prioritise these factors, so we can propose measures to address them.

24 Pisco Costa et al., “Identificação Das Causas Do Abandono Escolar Na Universidade de Évora.” 25 José António Sarsfield Cabral, “Algumas Notas Sobre o Acesso e Desempenho Dos Estudantes Admitidos Na U.Porto Em 2008/09, 2009/10 e 2010/11 (Pelo Regime Geral de Acesso Em 1s Ciclos e Mestrados Integrados)” (Seminário para o “Sucesso Académico,” Lisbon, May 12, 2015). 26 IP de Setúbal, “O Abandono Nos Cursos de Primeiro Ciclo.” 27 DGEEC, “Dados e Estatísticas de Cursos Superiores,” Infocursos, 2017, http://infocursos.mec.pt/. 28 António Firmino da Costa and João Teixeira Lopes, “Os Estudantes e Os Seus Trajectos No Ensino Superior: Sucesso e Insucesso, Factores e Processos, Promoção de Boas Práticas - Relatório Final” (Lisbon: Centro de Investigação e Estudos de Sociologia, 2008). 29 DGEEC, “Ensino Superior: Situação Em 2012-13 Dos Inscritos Pela Primeira Vez Em 2011-12” (Lisbon: Direção-Geral de Estatísticas da Educação e Ciência, 2013). 30 Firmino da Costa and Teixeira Lopes, “Os Estudantes e Os Seus Trajectos No Ensino Superior: Sucesso e Insucesso, Factores e Processos, Promoção de Boas Práticas - Relatório Final.”


14

We initially consider intervening factors, which interact with higher education policies but originate from outside of the system. These factors are (1) non-modifiable student characteristics (socioeconomic status, gender, and ethnic origin), (2) student academic skills (based on their prior educational experiences), and (3) student motivations (relating to expectations, sense of belonging and labour market goals). We then consider factors relating to higher education system policies, based largely on criteria from the Vossensteyn et al. review of study success policies across Europe.31 In Chapter 3, we identify four elements of Portugal’s admissions structure that do not appear to support strong matching of students and programmes, in terms of balancing access and study success through smart selectivity, fostering accurate student expectations, and connecting students with programmes that meet their expectations. In Chapter 4, we isolate 13 areas of concern in terms of the quality of delivery of education delivery, where policies and practices seem to fall short of expectations for: institutional commitment; tracking and monitoring of students; learning, teaching and assessment; flexibility; and social integration and support services. Finally, Chapter 5 finds seven areas of concern for promoting institutional commitment, providing adequate funding to institutions and students, and supporting student motivation. In Chapter 6 we use a heuristic to assess the appropriate prioritisation of each factor for policies to raise completion, based on our view of each factor’s importance in determining overall completion rates across the system, our confidence in our research findings regarding the factor, and our assessment of how susceptible the factor is to a policy intervention by government. Our assessment of the importance of different factors is with respect to the total population of students who drop out in a given year and is not intended to be an evaluation of the importance of particular students. Our confidence assessment recognises where our data is inadequate to help indicate which areas should become priorities with more information. Finally, our assessment of susceptibility is grounded in reflection on the financial, administrative, and political feasibility of possible government interventions to address each policy problem. This assessment clearly emphasises the following concerns: failure to assist students at risk of dropping out; weak incentives in the institutional funding structure; uneven institutional commitment to study success; and high rates of academic failure. A second heuristic helps us categorise which factors the Portuguese Government can address directly (first-level), or through influencing the behaviour of institutions (second-level), or even institutional units or individual stakeholders (third-level). The Government has considerable direct control over financial factors and the admissions systems, but in general cannot address quality challenges directly. Institutional leaders, faculty, administrative staff, student representatives and quality assurance agency staff all must play a role in raising completion.

31 Vossensteyn et al., “Dropout and Completion in Higher Education in Europe.”


15

Policy Options and Recommendations Chapter 6 contains our analysis of 18 policy options under three themes: (1) Targeted Supports for Students at the Margins, (2) System Steering, and (3) Shaping Expectations of Student Success. We are not in a position to estimate costs of implementing our proposals, however we only consider policy options that we believe could be pursued with modest additional expenditure beginning in the short-term. Our analysis of options is based on four main criteria: impact on completion, risks, flexibility, and feasibility. All policy options could be pursued with limited additional expenditure beginning in the short-term. Appendix II includes a table summarising many elements of the options analysis: the targeted policy problems, key actors, key steps to implementation, key obstacles and possible timeline. We generate three primary recommendations, one for each theme. Chapter 6 also includes nine secondary recommendations, which aim to complement the primary recommendations and/or provide intermediate solutions as other policies are in development. Primary Recommendation 1: Provide targeted institutional funding for services to identify and support students at risk of dropping out Among all our recommendations, we believe this one promises the most significant short-term returns on investment. It could be implemented more or less immediately, subject mainly to constraints on the capacity of institutional data systems and the use of academic student information for purposes of institutional outreach. Funding could help institutions to hire staff who are specifically responsible for tracking, outreach to identified students and coordinating different services as needed. Not all services may be initially adequate to the task. However, having the targeting mechanism in place could help inform and drive their continuous improvement. The programme should aim for students to be contacted directly by the service once identified as being at risk. The Government would need to develop standards and reporting requirements for the use of the funds. Institutions should have track the numbers of students reached, basic characteristics of these students (admissions stream, year of study, gender, etc.), and the reasons why they sought assistance, with suitable privacy protections. This data could be helpful firstly in supporting accountability for the funds provided to support these services, and secondly to further build data on factors in study success. Data collection and service performance should be assessed under institutional accreditation processes.


16

Primary Recommendation 2: Include performance criteria relating to study success within an institutional funding formula The Government of Portugal appears committed to introduce a funding formula, thereby fulfilling requirements under the higher education financing law. This would be a positive step, especially if the new formula includes thoughtful criteria that make institutions accountable for their students’ study success. Tying funding to the number of degrees granted would be the simplest measure and an improvement on enrolment based funding. It implies the risk that institutions simply pass students through and would require that quality assurance structures operate effectively to prevent this. Passing too many students does not appear to have been a systematic problem in Portugal, but it should not become one in the future. To ensure equity, Portugal should ensure that the funding formula recognises where institutions are recruiting more disadvantaged students, or running programming associated with lower rates of study success. The formula should adjust funding accordingly either through its study success measure or by including a separate funding envelope to reward institutions for their performance in terms of access for disadvantaged students. Whatever measures are used should be developed carefully and not in a rush. The Government could begin to work on this during its current mandate, but perhaps take longer to put an effective formula in place, given it should consult higher education institutions and other stakeholders extensively. It would be essential to tie enough funding to the instrument to actually affect institutional behaviour. Introducing the new formula in parallel with an overall increase in funding to institutions could help to mitigate any negative short-run impacts and thereby improve feasibility. Primary Recommendation 3: Tighten rules regarding academic suspension and monitor institutional compliance. The high rates of failure, especially the presence of inactive students, undermine the credibility of commitment to completion system-wide. If the Government of Portugal wants faculty to hold and act upon stronger expectations that their students will be successful, it should take steps to build these expectations more strongly into its own policies. More detailed and stringent requirements for institutional policies on academic suspension should be developed in consultation with higher education institutions, although we make a series of suggestions. Fulfillment of these requirements should be a criterion in institutional accreditation, and the accreditation agency should especially explore measures to audit the shares of students who are serially inactive.


17

There is no question that higher rates of academic failure are a by-product of expanding access to higher education to students with less academic preparation, often because they have faced systematic disadvantage in their earlier education. To preserve equity, offering an appeals mechanism is essential and this recommendation should complement the recommendation on developing services to identify and support students at risk. Further research regarding the relationship between academic failure and study success in Portugal could help to refine this mechanism and the targeting services from Primary Recommendation 1. Raising completion rates is a complex challenge, but it is critical for Portugal to achieve its goals for tertiary attainment and increase the country’s long-term productivity and prosperity. We have identified complex factors that undermine students’ study success, but we believe that the Government of Portugal can take concrete steps over the next three years to address many of these. We recommend that the Government assist students at risk of dropping out by funding targeted support services, begin to create stronger financial incentives for institutional commitment to study success through a funding formula, and establish stronger expectations that students will be successful by adopting and enforcing stronger academic suspension policies.


19

Chapter 2: Measuring and Conceptualizing

Study Success OECD data on completion rates places Portugal somewhat below the average of member countries, notwithstanding challenges in comparing data collected across jurisdictions using different methodologies. At the national level, data on dropout after first year indicates significant differences between institution types, institutions and study programmes. Public institutions, universities, more competitive programmes, and Licenciado degrees tend to have lower dropout rates than private institutions, polytechnic institutes, less competitive programmes and Master’s degrees. Students drop out most in first year. The international literature identifies what we call “intervening factors” that influence study success, i.e. linked with societal trends that are independent from but interact with higher education system policies. These factors are (1) non-modifiable student characteristics (socioeconomic status, gender, and ethnic origin), (2) student academic skills (based on their prior educational experiences), and (3) student motivations (relating to expectations, sense of belonging and labour market goals). International evidence indicates that while study success matters for individuals, families, institutions and society, no policymaker should seek simply to maximise the completion rate, or minimise dropout and time-to-completion. Some very high quality higher education systems have low completion rates by OECD standards, based on the roles they play within their specific societies. Efforts to raise completion must take into account trade-offs with access, student learning, and time-to-completion. Ultimately, Portugal should establish a target for completion rates that corresponds with its own economic and social circumstances and objectives.


20

Chapter Guide

1. How Do We Measure Study Success ................................................................................. 22

2. A Snapshot of Study Success in Portugal ........................................................................... 24 2.1 Completion Rates by International Standards.......................................................................... 24 2.2 Study Success among Portuguese Higher Education Institutions .............................................. 26 2.3 When Do Students Drop Out? ................................................................................................. 31

3. Intervening Factors in Study Success ................................................................................ 33 3.1 Non-Modifiable Student Characteristics: Socioeconomic Status, Gender, and Ethnic Origin ...... 33 3.2 Student Skills at Admission ..................................................................................................... 35

3.2.1 Evidence from the Programme for International Student Assessment 2015 ........................... 36 3.2.2 Pathways to Higher Education ................................................................................................... 39

3.3 Student Motivation ................................................................................................................ 44 3.3.1 Expectations ............................................................................................................................... 45 3.3.2 Sense of Belonging ..................................................................................................................... 45 3.3.3 Career Related Motives ............................................................................................................. 46

4. Why and How Study Success Matters ............................................................................... 48

List of Tables

Table 2.1: Key empirical measures of study success .................................................................. 22 Table 2.2: Comparative national completion rates using cross-cohort methodology .............. 25 Table 2.3: Comparative national completion rates using true-cohort methodology – completed any ISCED level by theoretical duration plus 3 years, 2014 ..................................... 26 Table 2.4: Status of first-time first-year students after one year by institution type and first cycle study programme, 2013 and 2014...................................................................................... 27 Table 2.5: Gap between share of students who discontinued studies and share of entering students at public IPs (percentage point figures in brackets) .................................................... 29 Table 2.6: Gap between share of students who discontinued studies and share of entering students at public universities (percentage point figures in brackets) ...................................... 29 Table 2.7: Gap between share of students who discontinued studies and share of entering students from all higher education institutions, 2014-15 (percentage point figures in brackets)....................................................................................................................................................... 30 Table 2.8: Summary of Portuguese PISA results relative to comparator jurisdictions, 2015 ... 37 Table 2.9: Indicators of equity in Portuguese PISA results, 2015 ............................................... 38 Table 2.10: Characteristics of secondary graduates and secondary dropout rate, 2014-15 ..... 40


21

List of Figures Figure 2.1: Maximum and minimum institutional first-year dropout rates by institution type, 2013-14 ......................................................................................................................................... 28 Figure 2.2: Dropout rates among first-year first-time second-cycle master’s students by institution type, 2012 ................................................................................................................... 31 Figure 2.3: Comparative national completion rates by gender, 2014 ........................................ 35 Figure 2.4: Status one year after graduation of Continental-Portugal secondary graduates by secondary stream, 2013-14 and 2014-15 .................................................................................... 41 Figure 2.5: Status one year after graduation of Continental-Portugal CH secondary graduates by secondary stream, 2013-14 and 2014-15 ............................................................................... 41 Figure 2.6: Characteristics of mature students at Santos et al. case study universities ........... 44


22

1. How Do We Measure Study Success Study success is a broad concept with a variety of interpretations. A first, baseline interpretation is simply completion of a study programme, but it can expand from this point to completion within a given period of time, to completion with good grades, to completion and successful progression into a job or further studies. Our focus in this study is on study success endogenous to a study programme (i.e. not considering the student’s subsequent pathway) without emphasising students’ grades, basically to control scope. This approach implies some important challenges to be considered in detail in this chapter. Within our more constrained focus on student success, there are three key empirical output measures used internationally: Completion Rates, Drop-Out Rates, and Time-to-Degree. Each of these, as well as the Switching Rate, are described in Table 2.1.32 Table 2.1: Key empirical measures of study success

Measure Description

Completion Rate

Measures the proportion of students who begin a study programme at a higher education institution who complete the study programme successfully. Previously called “survival rates”

Drop-Out or Attrition Rate

Number of students in a study programme who fail to re-enrol in subsequent year and did not graduate. Antonyms: Retention, Persistence or Continuation.

Time-to-Degree

Average number of years that students take to complete the study programme among those who do so.

Switching or Transfer Rate

Proportion of students who change their study programme or higher education institution.

Different institutions use very different terminologies for dropout. Within Portugal, however, the most common term used is abandono, which translates to abandonment of studies. Alarcão distinguishes between three types of abandon:33

• Effective abandono (abandono efetivo) which is the number of students who do not re-enroll in the subsequent year.

• Internal abandono (abandon interno) is the number of students who switch study programmes within the same institution.

• Finally, total abandono (abandono total) is equal to the total of effective and internal. This interpretation of abandono is basically equivalent to dropout at the level of specific study programmes, although it can be slightly different in terms of being measured not at the level of a particular year, but across the student population over a series of years. An IP de Setúbal study references three types of dropout as well: temporary, transfer to another institution, and definitive. Finally, a Universidade de Évora study preferred to use the term “inactivation of

32 Hans Vossensteyn et al., “Dropout and Completion in Higher Education in Europe” (European Union, 2015). 33 Madalena Alarcão, “Insucesso Académico e Abandono Escolar” (Seminário Sucesso Académico, Lisbon, May 12, 2015).


23

enrolment” over abandono, recognizing that in many cases students actually feel forced to leave higher education, contrary to their expectations and aspirations, whereas abandono implies voluntary discontinuance.34 For precision, this report will generally opt for the international study success terminology. Conceptually our measures may seem reasonably clear, however they can become quite complex to implement. Perhaps the most significant challenge is the time-frame of analysis. Many students may complete only after a very extended period of time, making the measurement of completion among the most policy relevant recent cohorts especially difficult. Switching between institutions, programmes (including programme levels, for example by transferring from a Master’s into a Doctoral Programme) and full or part-time status can also complicate measurement considerably. Students can also re-enrol without gaining credits and thereby artificially limit the drop-out rate. Finally developing cross-cutting measures is complicated by varying degree programme lengths (e.g. programmes of three or four years may both be classified as first-cycle undergraduate degrees). Measures are generally based on either entrance cohorts (a group of students who enter an institution or study programme in a given year) or exit cohorts (those who leave the institution or study programme in the same year either due to completion, failure, or withdrawal for other reasons). Entrance cohorts are used the most frequently, but are significantly compromised by the time-frame challenge. Exit cohorts can provide a more punctual assessment of completion and failure/withdrawal rates, but still do not escape many of the challenges above and are subject in particular to biases based on variations in the flow of students into study programmes from year to year. Different higher education systems and even institutions have approached these challenges in measurement in different ways. This has produced significant challenges in terms of generating comparable measurements. For completion rates, national data provided to the OECD uses true-cohort and cross-cohort methodologies.35 N represents the theoretical length of a study programme. The true-cohort methodology tracks an entry cohort over a given period of time, which it monitors at N and at N plus three years. This methodology requires a student registry to be able to track the individual students’ progress over the time period, including if they switch programmes or higher education institutions. The cross-cohort method captures the number of new entrants to a programme and the number of graduates N years later. This methodology is slightly less precise given how it does not track individual students and is therefore affected considerably by variations in enrolment over time (it assumes constant student flows, i.e. increases or decreases in entering student numbers, and will provide inaccurate data where this assumption is not met). The cross-cohort method may reflect more the total completion rate however because it does not differentiate

34 Rosalina Pisco Costa et al., “Identificação Das Causas Do Abandono Escolar Na Universidade de Évora” (Évora: Universidade de Évora, 2015). 35 OECD, “Education at a Glance 2016: OECD Indicators” (Paris: OECD Publishing, 2016).


24

between when students began their programmes. The OECD considers the true-cohort methodology to be more reliable for these reasons.36 It is not possible to confidently compare data from the two methodologies, but cross-cohort data should be more comparable with true-cohort data developed based on a longer time-to-completion (e.g. N plus three years more so than simply N). The other two primary study success measures are less complex to calculate than completion. The most common measurement of drop-out rate is the proportion of students who re-enroll in Year Two of a higher education programme. This is based on the view that persistence from Year One to Year Two is a crucial step in the educational pathway. Time-to-completion measurement simply tracks when graduating students began their studies compared with N. These indicators complement each other to provide a more complete understanding of study success. The completion rate is the most comprehensive. The drop-out rate is useful in allowing for more immediate measurement of trends that feed into the completion rate, given completion rates can take time to measure. Meanwhile time-to-completion provides no information on students who do not graduate but indicates the efficiency of an education system for those students who do successfully graduate.

2. A Snapshot of Study Success in Portugal

2.1 Completion Rates by International Standards Our study objective is to recommend policies that the Government of Portugal could implement to begin to bring completion rates to international reference levels. The OECD provides the best and really only comparative data on completion rates in higher education. Reviewing this data, however, indicates that this Portugal’s completion rates currently cannot be compared reliably to an international standard. The details of factors in study success also suggest that international reference levels are not terribly meaningful. Education at a Glance 2016 captured data on completion rates in higher education using cross cohort and true-cohort methodologies. Data on Portugal is provided in Table 2.2 alongside other countries that used this methodology, for three years for which OECD data is available.37 38 39

36 OECD, “Education at a Glance 2016: OECD Indicators - Annex 3: Sources, Methods and Technical Notes” (Paris: OECD Publishing, 2016). 37 OECD, “Education at a Glance 2016: OECD Indicators.” 38 OECD, “Education at a Glance 2016: OECD Indicators - Annex 3: Sources, Methods and Technical Notes.” 39 OECD, “Education at a Glance 2007 - OECD Indicators” (Paris, France: OECD Publishing, 2007).


25

Table 2.2: Comparative national completion rates using cross-cohort methodology

2004 2011 2014

Tertiary Type

5A* Tertiary Type

5A* Licenciatura Equivalent

Integrated-Masters Equivalent

Australia 67 82 - -

Austria 65 65 - - Belgium 74 - - -

Brazil - - 48 -

Czech Republic 65 - - -

Finland 71 - - -

France 79 - - - Germany 73 - - -

Greece 79 - - - Hungary 64 48 - -

Iceland 67 - - -

Ireland 83 - 94 - Japan 90 91 92 -

Korea - - 85 - Mexico 69 67 - -

Netherlands 76 - - -

New Zealand 54 - - - Poland 66 62 - -

Portugal 68 67 65 71

Slovak Republic - 71 - -

Slovenia - - 47 60

Spain 74 78 - - Sweden 60 - - -

Switzerland 69 - - -

Turkey 74 88 94 84

United Kingdom 78 79 - -

United States 54 - - - Average 71 72 75 72

* Tertiary Type 5A is equivalent to ISCED Levels 5 and 6, or a combination of the 2014 figures. Portugal’s completion rates fall below the average among countries recorded at the Licenciatura level for all three years. Portugal also basically falls right between Slovenia and Turkey at the Integrated Master level in 2014. The fact that only seven countries used the cross-cohort methodology at the Licenciatura level in 2014, however, limits the relevance of these findings. Moreover, few of the countries using the methodology are European, which would seem to be the most salient comparators, and just three countries have data at the Integrated-Masters level. The older data includes more countries, so is somewhat stronger in indicating Portugal’s performance falls below average.


26

Table 2.3 presents 2014 completion rates in 15 countries (only six at the Integrated-Masters level) using the true-cohort methodology, adding three years to the theoretical duration of programmes to improve comparability with Portugal’s cross-cohort data.40 Table 2.3: Comparative national completion rates using true-cohort methodology – completed any ISCED level by theoretical duration plus 3 years, 2014

Countries Licenciatura Equivalent Integrated-Masters Equivalent

Australia 70 -

Austria 58 61 Belgium (Flanders) 73 -

Czech Republic 60 65 Denmark 81 -

Estonia 51 54

Finland 68 -

France 70 -

Israel 70 - Netherlands 66 -

Norway 76 68

New Zealand 81 -

Sweden 53 71

United Kingdom 84 88

United States 78 - Average 69 68

Portugal (Cross-Cohort) 65 71

Again, please note again that this comparison with true-cohort data is only approximate and in fact would be more likely to overstate Portugal’s relative completion rate. It reinforces the impression that Portugal significantly falls below the average among OECD countries in completion rates at the Licenciatura level. The coverage of other countries is still too limited at the Integrated-Masters level to derive a meaningful comparative assessment. In closing, Portuguese completion rates do appear to fall somewhat below the national average

2.2 Study Success among Portuguese Higher Education Institutions For further detail in terms of study success across Portugal’s higher education system, we have to rely on students’ status at the end of their first year of study, as tracked by the DGEEC. The only more detailed data on completion rates by institution type within Portugal is more than ten years old, and we were also unable to find Portuguese data on time to completion.41

40 OECD, “Education at a Glance 2016: OECD Indicators.” 41 António Firmino da Costa and João Teixeira Lopes, “Os Estudantes e Os Seus Trajectos No Ensino Superior: Sucesso e Insucesso, Factores e Processos, Promoção de Boas Práticas - Relatório Final” (Lisbon: Centro de Investigação e Estudos de Sociologia, 2008).


27

The measure of students’ status after one year of study can tell us about how study success varies between different institutions and study programmes. However, it is an especially limited measure given student choices to persist in, change or abandon their study programme may occur throughout the programme’s duration. There may be broader institutional data, but interviewees expressed significant concerns about how Portuguese institutions record this data.42 Students are often recorded as having dropped out when they cancel their registration, but many students may not cancel their registration because this would cause them to incur fees. For example, at the University of Coimbra, 22% of freshmen students and more than 39.6% of other students who dropped out in 2012-13 and 2013-14 still owed outstanding fees.43 Failing to dropout formally is characterised as “silent dropout”. Other students may be recorded as dropping out when they miss an evaluation or when they switch their programme or institution. The DGEEC data in contrast uses an identification number to track students between different Portuguese higher education institutions, while protecting students’ privacy Table 2.4 presents Portuguese Government data on dropout after year for students first enrolled in 2012-13 and 2013-14.44 Further findings from this source will be considered throughout this report. Table 2.4: Status of first-time first-year students after one year by institution type and first cycle study programme, 2013 and 2014

Status After One Year Universities IPs

Licenciatura Integrated Masters Licenciatura Public Private Public Private Public Private

Not in Higher Education 9.0% 12.6% 3.7% 9.8% 10.8% 12.3%

Different Institution and Study Programme 5.2% 7.7% 6.2% 9.3% 4.4% 5.1%

Same Institution, Different Study Programme 4.3% 2.4% 5.3% 0.7% 3.1% 1.0%

Same Institution, Same Study Programme 81.5% 76.4% 84.7% 80.1% 81.5% 80.5% Graduated* 0.1% 0.8% 0.1% 0.1% 0.3% 1.1%

* These students must have either been recorded as entering as first-time first-year students erroneously, or else transferred to a short one-year programme (e.g. CET). These data indicate important differences in dropout rates between institution types. For Licenciatura students, those in public IP education dropout at a 1.8 percentage point higher rate than those in public university education, while those in private IPs dropout at a 2.5 percentage point higher rate than those in private universities. Additionally, dropouts at public higher education institutions are significantly lower than in private institutions, with the difference at

42 Personal email correspondence, February 26, 2017. 43 Alarcão, “Insucesso Académico e Abandono Escolar.” 44 DGEEC, “Dados e Estatísticas de Cursos Superiores,” Infocursos, 2017, http://infocursos.mec.pt/.


28

the Licenciatura level 3.6 percentage points in universities, and 1.5 percentage points in IPs. Another analysis based on gaps between shares of entering students and shares of students who dropped out in 2013-14 indicates that public IPs had a 4.6 percentage points larger share of dropouts than entering students, whereas corresponding figures for other institution types were +3.8% for private universities, +1.3% for private IPs, and -9.7% for public universities. There is considerable variation among institutions of a certain type, as shown in Figure 2.1. This variability is dramatically higher among private institutions as compared with public institutions – the Universidade Aberta had the highest dropout rate among public universities at 40.5%, but the next highest rate was just 13.6%. The Universidade Aberta would be expected to have a higher dropout rate because it delivers all its programmes online.45 The lowest dropout rates for public polytechnic education are actually among universities that deliver such programmes, and the three higher education nursing schools.

Figure 2.1: Maximum and minimum institutional first-year dropout rates by institution type, 2013-14

To identify the importance of individual institutions’ contributions to the overall dropout challenge in Portugal, I conducted a gap analysis between each institution’s proportion of the total first-year cohort and its proportion of dropouts, by institution type. To be clear, appearing on these lists should not be taken necessarily as an indictment of an institution, nor a tribute to it. As we will explore in this chapter, but also throughout the report, numerous factors contribute to an institution’s dropout or completion rate, many of which are outside institutional control or inherent to their specific role within the system. There were a range of outcomes among private institutions in 2014-15. Four IPs and four universities were responsible for two percentage points more dropouts than their share of entering enrolment. Meanwhile five IPs and three universities were responsible for two percentage points fewer dropouts than their share of entering students.

45 Eric Bettinger, Lindsay Fox, and Susanna Loeb, “Changing Distributions: How Online College Classes Alter Student and Professor Performance,” Working Paper, CEPA Working Papers (Stanford: Stanford Center for Education Policy Analysis, 2015).

0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50%

University

IP

University

IP

Pri

vate

Pu

blic

Minimum

Max


29

Table 2.5 presents public IPs that had a share of dropout one percentage point above or below their share of the first-year-cohort in 2011-12 and 2014-15. These results sugget dropout in IPs is greater in secondary cities and towns, and lower at urban institutions. The only institution to report higher than expected dropout both years is IP de Setúbal, but the IP’s dropout rate was 3.2 percentage points lower in 2014-15 than in 2011-12 suggesting some progress. Table 2.5: Gap between share of students who discontinued studies and share of entering students at public IPs (percentage point figures in brackets)

2011-12 2014-15

Considerably lower dropout than expected

-IP Lisboa (-2.5) -IP Porto (-1.41) -IP Coimbra (-1.22)

-IP Coimbra (-2.75) -ES Enfermagem de Coimbra (-1.38) -ES Enfermagem de Lisboa (-1.11) -ES Enfermagem do Porto (-1.08)

Considerably higher dropout than expected

-IP Setúbal (+2.95) -IP Tomar (+1.86) -IP Cávado e do Ave (+1.31) -IP Viseu (+1.07)

-IP Setúbal (+2.63) -IP Viseu (+2.15)

Table 2.6 presents the same analysis for public universities, however the figures exclude the Universidade Aberta to facilitate comparison (the Universidade Aberta had a share of dropout 20.2 percentage points over expected in 2011-12 and 15.19 percentage points over expected in 2014-15). Universities differ from IPs in that institutions both from more disadvantaged regions and the central cities count among those with higher and lower than expected dropout rates. The results also show a combination of consistency and inconsistency: three universities had very low dropout relative to expectations in both years, whereas the Universidade de Coimbra had the lowest dropout relative to expectations in 2011-12 and then the highest in 2014-15 (reflecting a 3.1 percentage point increase in the institution’s dropout rate). It is also notable that while the Universidade dos Açores made higher than expected contributions to university dropout both years, at the IP level the institution had dropout rates of 5.3% and 3.9%. Table 2.6: Gap between share of students who discontinued studies and share of entering students at public universities (percentage point figures in brackets)

2011-12 2014-15

Lower dropout than expected

-U. Coimbra (-4.05) -U. Porto (-2.12) -U. Aveiro (-1.82) U. Beira Interior (-1.26)

-U. Porto (-3.37) -U. Beira Interior (-1.56) -U. Aveiro (-1.38)

Higher than expected

-U. Lisboa* (+3.61) -U. Açores (+2.72)

-U. Coimbra (+5.44) -U. Açores (+1.83)

*This figure is for the Universidade de Lisboa prior to its merger with the Universidade Técnica de Lisboa.

An additional analysis based on each institution’s contribution to overall dropout in Portugal in 2014-15 identified six institutions where their difference from the national average for dropout in higher education raised total dropout by one percentage point or more, as shown in Table 2.7.


30

Bringing the performance of these institutions alone to the national average would reduce dropout by 12.2%, almost half from the Universidade Aberta. Table 2.7: Gap between share of students who discontinued studies and share of entering students from all higher education institutions, 2014-15 (percentage point figures in brackets)

Private University Public University Public IP U. Lusófona de Humanidades e Tecnologias (+2.2%) U. Autónoma de Lisboa Luís de Camões (+1.1%)

U. Aberta (+5.7%) IP Setúbal (+1.2%) IP Viseu (+1%) IP Porto (+1%)

Finally, dropout rates vary considerably between degree levels and study programmes. For example, dropout rates in 2012-13 at the IP do Porto varied by school between 43.3% and 6.1%.46 Institutional rates may actually be shaped less by the particular institution type than the types of study programmes that it offers, given wide ranges in dropout rates across different study programmes.47 48 49 Rates are considerably lower in Integrated-Masters programmes than in Licenciatura programmes. Many interviewees also indicated differences across faculties, with special concerns cited about engineering and management schools which is also borne out in the system-level data. There is even variability among study programmes within a single faculty. Among Licenciatura students across institution types, these student dropout numbers are modestly better than for the entering 2010-11 cohort (DGEEC, 2013).50 However, dropout rates are somewhat (two percentage points) worse for Integrated-Masters students at private universities. Other studies suggest students in earlier years of their study programmes are more likely to transfer relative to students in later years who will drop out much more than transfer, while graduate students are also considerably more likely to dropout than to transfer.51 52 In general measures of dropout are little developed and in particular little standardised and centralised for OECD/EU countries. In fact, Portugal has the makings of one of the best dropout tracking systems in Europe.53 Yet, for point of comparison approximately 12.2% of university students in Austria drop out by the end of the first year and the UK rate is approximately 7%, while roughly 16% of Irish students leave their higher education institutions after the first year

46 Gabinete de Planeamento, Projectos e Desnvolvimento, “Estudo de Caso: Abandono Escolar 2012-13 (ESE, ESMAE, ESEIG, ISCAP, ESTGF, ESTSP e APNOR)” (Porto: Politécnico do Porto, 2014). 47 Firmino da Costa and Teixeira Lopes, “Os Estudantes e Os Seus Trajectos No Ensino Superior: Sucesso e Insucesso, Factores e Processos, Promoção de Boas Práticas - Relatório Final.” 48 DGEEC, “Ensino Superior: Situação Em 2012-13 Dos Inscritos Pela Primeira Vez Em 2011-12” (Lisbon: Direção-Geral de Estatísticas da Educação e Ciência, 2013). 49 Firmino da Costa and Teixeira Lopes, “Os Estudantes e Os Seus Trajectos No Ensino Superior: Sucesso e Insucesso, Factores e Processos, Promoção de Boas Práticas - Relatório Final.” 50 DGEEC, “Ensino Superior: Situação Em 2012-13 Dos Inscritos Pela Primeira Vez Em 2011-12.” 51 IP de Setúbal, “O Abandono Nos Cursos de Primeiro Ciclo” (Setúbal: Instituto Politécnico de Setúbal, 2013). 52 IP de Setúbal, “O Abandono Nos Cursos de Segundo Ciclo” (Setúbal: Instituto Politécnico de Setúbal, 2014). 53 DGEEC - Equipa de Estudos de Educação e Ciência, “Draft: Medidas de Promoção Do Sucesso Escolar Nas Instituições de Ensino Superior Públicas Em Portugal, Tal Como São Relevadas Nos Respetivos Sítios” (Lisboa: DGEEC, Unpublished).


31

(either through transfer or drop out) (Vossensteyn et al., 2015). Only the UK figure appears clearly superior to Portugal’s performance. These figures focus overwhelmingly on the first cycle level, which is our main emphasis given our overall focus on helping Portugal achieve its target rate of population higher education attainment and the newness of Higher Technical Professional Programmes (Cursos Técnicos Superiores Profissionais, TESP) programmes. Data on TESP programmes should become available very soon according to the DGEEC. Somewhat older DGEEC data on dropout in second-cycle Master’s degrees however is provided in Figure 2.2.54

Figure 2.2: Dropout rates among first-year first-time second-cycle master’s students by institution type, 2012

Dropout among Master’s students is much higher than across first-cycle programmes. Dropout across all institution types except private IPs is relatively consistent at around 22%. The figures for private IPs are especially high, though these represent just 6% of enrolment; 1 876 students. In closing, based on dropout rates, study success also appears stronger in public higher education institutions than private institutions, in universities relative to IPs, and for Integrated-Masters students relative to Licenciatura students.

2.3 When Do Students Drop Out? To understand the relationship between the delivery of higher education and study success, it is valuable first to identify data on when students drop out. The DGEEC analyses done to date are limited in only tracking dropout by the end of the first year, although data for later years has been collected. Completing this analysis would be very helpful. Even this data, however, is specific only to the year and not more specific time horizons. We have to rely therefore on student surveys and our interviews for greater detail.

54 DGEEC, “Ensino Superior: Situação Em 2012-13 Dos Inscritos Pela Primeira Vez Em 2011-12 - Parte II” (Lisbon: Direção-Geral de Estatísticas da Educação e Ciência, 2013).

0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50%

University

Polytechnic

Private

Public


32

The most detailed evidence on this comes from a recent (2015) study at the Universidade de Évora, which tracked dropout over time from the beginning of students’ studies and found significant differences based on students’ age, phase of entry, admissions stream, and study programme (but not entering grades).55 Other studies also provide information on timing of dropout in Portugal. In general, studies consistently highlight students’ challenges during their first year of higher education as contributing significantly to dropout.56 In this period, students are adapting to considerably greater autonomy and responsibility, which relates to much more than just their prior schooling. Fully 46% of Licenciatura students who dropped out at the Universidade de Évora did so in first year. In a 2015 presentation on the Universidade do Porto, Sarsfield Cabral characterised the first year as critical, with dropout averaging 17%.57 A 2009 study of engineering students at the Universidade do Minho also found dropout was highest in the first year, with the primary factors being difficulty to transition into higher education and difficulty reconciling prior expectations with the reality of their study programme and life in higher education.58. A more recent (2013) study at the IP de Setúbal similarly found half of dropouts occurred in the first year of studies, though more in the second semester than the first, while a survey at the IP do Porto found roughly two-thirds of students dropped out in first year.59 60 At the Universidade de Coimbra a majority of the freshmen who dropped out were women, as compared with a majority of men among all students who dropped out overall.61 Finally, interviewees also highlighted the salience of the first few months of study in terms of dropout. One IP representative in particular noted that three-quarters of their dropouts occur in the first semester. Nevertheless, there is significant evidence of higher dropout in later years. A study at the Universidade de Coimbra actually found 84% of dropouts over a period of two years were not freshman students.62 The Universidade de Évora study found at a more detailed level that the highest risk periods for dropout were 18-21 months and 33-34 months from time of entry. Among Masters students at the IP de Setúbal, most dropouts occurred in the second semester of either first or second year.63 Another study at the Universidade do Porto identified the second year as the highest period of dropout, but argued that this was likely because students would only have

55 Pisco Costa et al., “Identificação Das Causas Do Abandono Escolar Na Universidade de Évora.” 56 Filomena Ferreira and Preciosa Fernandes, “Fatores Que Influenciam o Abandono No Ensino Superior e Iniciativas Para a Sua Prevenção: O Olhar de Estudantes,” Educação, Sociedade e Culturas, no. 45 (2015): 177–97. 57 José António Sarsfield Cabral, “Algumas Notas Sobre o Acesso e Desempenho Dos Estudantes Admitidos Na U.Porto Em 2008/09, 2009/10 e 2010/11 (Pelo Regime Geral de Acesso Em 1s Ciclos e Mestrados Integrados)” (Seminário para o “Sucesso Académico,” Lisbon, May 12, 2015). 58 IP de Setúbal, “O Abandono Nos Cursos de Primeiro Ciclo.” 59 Ibid. 60 Gabinete de Planeamento, Projectos e Desnvolvimento, “Estudo de Caso: Abandono Escolar 2012-13 (ESE, ESMAE, ESEIG, ISCAP, ESTGF, ESTSP e APNOR).” 61 Alarcão, “Insucesso Académico e Abandono Escolar.” 62 Ibid. 63 IP de Setúbal, “O Abandono Nos Cursos de Segundo Ciclo.”


33

their enrolment suspended after failing to complete required credit loads by the end of the first year.64 The Universidade de Évora study also found the differences in dropout by admissions stream occurred at different times during their study programmes (among licenciatura students p<0.001).65 The most notable difference is that Special Competition students are much more likely to drop out in the first three months, though they experience a second even greater risk period towards the end of the second year much like most other students.

3. Intervening Factors in Study Success It is crucial to understand the factors in study success to propose effective policies. In this Chapter, we emphasise factors affecting individual students that are largely independent from the higher education system itself, as it is largely the interaction of these factors with the higher education system and institutions that generates study success outcomes. We will consider system and institutional policy-driven factors in later chapters. The international literature documented principally by Vossensteyn et al., identifies a host of factors in study success at the individual level.66 We can categorize these as: (1) non-modifiable student characteristics (socioeconomic status, gender, and ethnic origin), (2) student academic skills, and (3) student motivations. Does Portugal share many of the same factors in study success documented in the international literature? In many cases evidence within Portugal is limited to case studies on particular public institutions. However, these studies demonstrate strong parallels between Portugal and the wider developed world.

3.1 Non-Modifiable Student Characteristics: Socioeconomic Status, Gender, and Ethnic Origin Massified higher education systems like Portugal’s educate a diverse student population in terms of socioeconomic status, gender, ethnic origins and (dis)ability. These different elements of diversity have been shown to affect individual students’ study success.67

64 Ferreira and Fernandes, “Fatores Que Influenciam o Abandono No Ensino Superior e Iniciativas Para a Sua Prevenção: O Olhar de Estudantes.” 65 Pisco Costa et al., “Identificação Das Causas Do Abandono Escolar Na Universidade de Évora.” 66 Vossensteyn et al., “Dropout and Completion in Higher Education in Europe.” 67 Ibid.


34

Students’ socioeconomic backgrounds are considered the most significant factor in their study success. Essentially, students from disadvantaged backgrounds have access to more limited social and cultural capital, as well as financial resources, to support their academic pursuits. Gender also appears to affect study success. In general, female students obtain better results in terms of study success than male students, and are less likely to switch their study programme. Drop-out and course switching are also more common for students of the minority gender in gender-dominated fields (e.g. male students in nursing programmes or female students in engineering programmes in many countries). The literature indicates that female students report dropping out more frequently due to low interest or motivation for their study programmes, whereas male students report dropping out more often because their aptitude or preparation is inadequate. “Moral support” from families appears to be a more significant factor in success for male students. The extent to which ethnic origin shapes study success generally appears very context specific. However, in general members of underrepresented minority groups tend to do more poorly in terms of study success, but study success increases with more heterogeneous student populations overall. The literature documents strong interactions between these various factors. For example, ethnicity can be closely related to socioeconomic status such that it is unclear to what extent ethnicity is in fact the crucial factor. Of course, these attributes can also shape the elements of student motivation, academic preparation, and time or financial pressures, while these other policies may be considerably more subject to direct policy interventions. It is in perhaps most important to consider non-modifiable student characteristics in the context of these interactions when crafting policy and programme interventions. In Portugal, we can observe many of these trends in higher education study success. OECD data indicates that female students outperform male students in Portugal by 13 percentage points in terms of completion rates. While this pattern applies across the OECD, it is relatively more significant in Portugal as shown in Figure 2.3.68 This likely reflects observed stronger performance of female students in secondary education and higher education.69

68 OECD, “Education at a Glance 2016: OECD Indicators.” 69 Sarsfield Cabral, “Algumas Notas Sobre o Acesso e Desempenho Dos Estudantes Admitidos Na U.Porto Em 2008/09, 2009/10 e 2010/11 (Pelo Regime Geral de Acesso Em 1s Ciclos e Mestrados Integrados).”


35

Figure 2.3: Comparative national completion rates by gender, 2014

* Both true-cohort methodologies use N + 3.

Dropout rates are also lower for students whose mothers are better educated, and whose fathers are better education in some studies.70 71 72 OECD figures indicate that Portugal has the second strongest relationship between parental education and tertiary enrolment among member countries.73 In other words, the Portuguese student body is comprised proportionally more of relatively advantaged students, and less of disadvantaged students. There are also strong indications that socioeconomic status and gender in Portugal especially influence study success through their effects on students’ prior academic preparation. These factors likely also shape students’ choices of institutions and study programmes. For example, short-cycle higher education programmes tend to attract students from lower socioeconomic status backgrounds.74

3.2 Student Skills at Admission It is unsurprising that better prepared and stronger students tend to achieve greater study success. This is reflective in large part of the trade-off noted earlier between access and study success. Higher drop-out rates among students with low high school achievements have been confirmed in numerous international studies, including in Spain. More specifically, one Spanish study found poor secondary preparation especially among vocational students was key to dropout.75 Controlling for low high school achievement even eliminated socioeconomic status as

70 IP de Setúbal, “O Abandono Nos Cursos de Primeiro Ciclo.” 71 João Guerreiro et al., “Relatório Sobre a Avaliação Do Acesso Ao Ensino Superior (Diagnóstico e Questões Para Debate)” (Lisboa: Grupo de Trabalho para a Avaliação do Acesso ao Ensino Superior, 2016). 72 Pisco Costa et al., “Identificação Das Causas Do Abandono Escolar Na Universidade de Évora.” 73 OECD, “OECD Economic Surveys: Portugal 2017” (Paris: OECD Publishing, 2017), http://dx.doi.org/10.1787/eco_surveys-prt-2017-en. 74 Sandra Hasanefendic, Manuel Heitor, and Hugo Horta, “Training Students for New Jobs: The Role of Technical and Vocational Higher Education and Implications for Science Policy in Portugal,” Technological Forecasting and Social Change 113 (2016): 328–40. 75 IP de Setúbal, “O Abandono Nos Cursos de Primeiro Ciclo.”

0%10%20%30%40%50%60%70%80%

Portugal OECD Cross-Cohort

OECD True-Cohort*

Portugal OECD Cross-Cohort

OECD True-Cohort*

Licenciatura Integrated-Masters

Female Students

Male Students

Female-Male


36

a factor in study success in a UK study.76 As a result of this, more selective higher education systems, institutions and study programmes will be expected to have higher completion rates. The international evidence indicates that students who use unconventional education pathways are also more likely to dropout. This includes students who have repeated grades, or use indirect pathways to access higher education. Mature students are particularly noteworthy in this sense, almost by definition following an unconventional pathway. Most mature students also come from lower socioeconomic status (SES) backgrounds.77 These patterns are very consistent in Portugal. To consider the extent to which they may be a factor in Portugal’s overall completion rate and differential completion rates between institutions and programmes, we will first consider Portugal’s results in the most recent Programme for International Student Assessment (PISA), then the different preparatory pathways students can take prior to higher education.

3.2.1 Evidence from the Programme for International Student Assessment 2015 The OECD’s PISA is an assessment of the learning outcomes achieved by 15-year-old students in participating economies, as these students are completing compulsory education.78 79 The assessments focus on the three core areas of science, reading and mathematics, as well as another “innovative domain” which in 2015 was collaborative problem solving. The intention is not simply to measure students’ knowledge, but how they can apply that knowledge, i.e. “what they can do with what they know”.80 PISA is particularly useful because the measures are applied across many different economies. All 35 OECD Member Countries and 37 partner countries and economies participated in PISA 2015. The 2015 assessment was administered as a two-hour test with four 30-minute elements, completed on a computer except where otherwise requested. Students and principals were also asked to complete mandatory questionnaires providing contextual information, which respectively took 35 and 45 minutes to complete. PISA aims to strictly limit the proportion of students excluded from sampling for PISA to less than 5%. Students completing PISA receive a score that situates their proficiency at a level between 1 and 6 for each of the core subjects (Science, Reading and Mathematics). Level 2 is the “baseline level of proficiency” that is considered necessary to participate effectively and productively in society.

76 In Vossensteyn et al., “Dropout and Completion in Higher Education in Europe.” 77 Lucília Santos et al., “Academic Success of Mature Students in Higher Education - A Portugese Case Study,” European Journal for Research on the Education and Learning of Adults 7, no. 1 (2016): 57–73. 78 OECD, “PISA 2015 Results (Volume I): Excellence and Equity in Education,” PISA (Paris, France: OECD Publishing, 2016). 79 Specific ages of students sampled in PISA are between 15 years and 3 months and 16 years and 2 months, with at least 6 years of formal schooling completed. 80 OECD, “PISA 2015 Results (Volume I): Excellence and Equity in Education,” 25.


37

Level 3 represents approximately the median level of performance. Levels 5 and 6 are described as top levels of performance, allowing the application of skills and knowledge “creatively and autonomously” even in unfamiliar situations. PISA also generates mean scores for each country by subject, and various further indicators based on the contextual survey, which generates an index of economic, social and cultural status. Portugal has participated in all five PISA cycles since 2000. In PISA 2015, 7 325 Portuguese students participated representing 97 214 students, with an exclusion rate well below that required. Portugal achieved strong results in PISA 2015, summarized in Table 2.8.81 Portuguese students’ mean results were above the OECD average in Science and Reading, and equal to the OECD average in Mathematics. Table 2.8: Summary of Portuguese PISA results relative to comparator jurisdictions, 2015

Study Area

Portugal OECD Countries whose mean PISA scores are not statistically different from Portugal Mean

Score 3-year trend

Mean Score

3-year trend

Science 501+ +8 493 -1 Switzerland, Ireland, Belgium, Denmark, Poland, Norway, United States, Austria, France, Sweden

Reading 498+ +4 493 -1 Netherlands, Australia, Sweden, Denmark, France, Belgium, United Kingdom, Chinese

Taipei, United States, Spain, Russia, B-S-J-G*, Switzerland

Math 492= +7 490 -1 Austria, New Zealand, Vietnam, Russia, Sweden, Australia, France, United Kingdom, Czech

Republic, Italy, Iceland, Spain +Result significantly indicating higher quality or equity than the OECD average. =Result indicating not significantly different quality or equity from the OECD average. *The Chinese municipalities of Beijing and Shanghai and the provinces of Jiangsu and Guangdong participated in PISA 2015.

As importantly, Portuguese results have been consistently and significantly improving. Portuguese students’ Science performance actually improved more between 2006 and 2015 than in any other OECD country, increasing the share of students performing at or above Level 5 while reducing the share performing below the baseline of proficiency (Level 2). Portugal also increased the share of students at or above Level 5 in Reading. Portuguese students report higher levels of epistemic beliefs, enjoyment of science and science related career expectations (particularly among girls) than the OECD average.

81 OECD, “PISA 2015 Results (Volume I): Excellence and Equity in Education.”


38

Table 2.9 provides indicators of the equity in Portugal’s PISA results.82 Portuguese students’ success is also relatively distributed across SES. The share of top performers in at least one subject is roughly equal to the OECD average, but Portugal has a smaller share of low achievers. Table 2.9: Indicators of equity in Portuguese PISA results, 2015

Indicator Portugal OECD Share of top performers in at least one subject 15.6%= 15.3%

Share of low achievers in all subjects 10.7%+ 13%

Percentage of resilient students* 38.1% 29.2%

Difference in % of resilient students +4.4% +1.5%

% of variation in performance explained by students’ SES (Science) 15%= 13% Score-point difference associated with a one-unit increase in the ESCS (Science)

31+ 38

+Result significantly indicating higher quality or equity than the OECD average. =Result indicating not significantly different quality or equity from the OECD average. *Students in the bottom quarter of PISA’s index of economic, social and cultural status who perform in the top quarter of students in all participating countries/economies, after accounting for socio-economic status.

The OECD classifies as resilient students those who fall in the bottom quarter of the PISA ISCS index but who perform in the top quarter of students from all participating countries and economies, controlling for SES. Portugal has a higher share of resilient students than the OECD average, with the proportion growing relatively rapidly. The percentage of variation in science performance explained by students’ SES in Portugal is roughly equal to the OECD average, but the score-point difference associates with a one-unit increase in the ESCS is smaller than the OECD average. Still, disadvantaged Portuguese students are slightly more likely than the OECD average to have low performance relative to non-disadvantaged students, and PISA captured the important differences in grade repetition by SES in Portugal. Altogether, PISA results suggest that by international standards Portuguese students overall are well prepared for higher education in terms of their core skills and knowledge. There are countries with largely equivalent 2015 PISA results that, notwithstanding measurement challenges, appear to have clearly stronger completion rates in higher education than Portugal, including Denmark (Portugal is equivalent in Science and Reading, weaker in Math) and the United Kingdom (Portugal is equivalent in Reading and Math, weaker in Science). This evidence does not support the view that inadequate academic preparation in primary to secondary critically undermines study success in Portugal relative to other OECD countries, particularly among students currently being recruited directly from secondary. Prior preparation is important for study success in Portugal, but policy interventions at the higher education level could significantly improve study success rates for students currently coming from secondary.

82 Ibid.


39

3.2.2 Pathways to Higher Education It is important, however, to distinguish between different students entering higher education. There are in fact multiple educational pathways students can pursue to prepare for higher education. Consistent with international experience, the more conventional the channel into higher education, the higher the rates of study success in Portugal.

3.2.2.1 Compulsory Education As of 2009-10, Portuguese education is compulsory up to the twelfth year, or approximately age 17 (OECD, 2012). Basic education extends from grades one to nine, for children aged approximately six through 14, followed by secondary education for grades 10-12 (i.e. ages 15-17). Compulsory education can be provided in both public and private schools. Traditional secondary schooling divides into four primary streams:83

1. The scientific-humanistic (científico-humanístico, CH) stream, aims to prepare students for higher education and covers four sub-streams: science and technology (Ciências e Tecnologias, CT), social and economic sciences (Ciências Socioeconómicas, CS), languages and humanities (Línguas e Humanidades, LH), and visual arts (Artes Visuais, AV).

2. The technological stream aims to prepare students who want to enter the job market directly, but also who want to pursue further study through post-secondary technological specialisation programmes (Cursos de Especialização Tecnológica, CETs) or higher education programmes.

3. The vocational (profissional) stream prioritises students aiming to enter the workforce directly.

4. The last stream is a specialised art stream. The technological stream is generally grouped with the CH stream, as both prepare for the same final exams. Nevertheless, each of these streams differs in terms of not only pedagogy, but the tools used for evaluating students at graduation. There are two additional programmes for non-traditional students. Learning Programmes (Cursos de Aprendizagem) are offered by Training Centres of the Employment and Vocational Training Institute (Instituto do Emprego e Formação Profissional, IEFP) and essentially another form of vocational training emphasizing workplace learning. Education and Training Programmes (Cursos de Educação e Formação, CEF) are flexible programmes adjusted to student profiles. All secondary diplomas provided in these programmes correspond to ISCED level 3. However, every stream other than CH is or can be doubly certified as academic and vocational, obtaining a level 4 instead of 3 in the Portuguese qualifications framework (Grupo de Trabalho, 2016).

83 Guerreiro et al., “Relatório Sobre a Avaliação Do Acesso Ao Ensino Superior (Diagnóstico e Questões Para Debate).”


40

Vocational programmes tripled in enrolment from 2005 to 2014 while enrolment in the CH stream fell by almost 14%.84 At 46%, Portugal’s proportion of upper secondary students was slightly above the OECD average (44%) in 2014.85 Gender and socioeconomic status relate closely to students’ participation in the various streams. Characteristics of graduates from secondary programmes (excluding the two adult learner programmes) are presented in Table 2.10.86 87 Table 2.10: Characteristics of secondary graduates and secondary dropout rate, 2014-15

Number of graduates

% of secondary graduates

% female

Average age

Secondary Dropout Rate

CH 41 714 53.6% 58% 17.3 18%

Technological 1 027 1.3% 46% 17.4 7% Vocational 23 051 29.6% 47% 18.3 13%

Specialised artistic 685 0.9% 71% 17.7 18%

Learning Programme* 11 389 14.6% - - - TOTAL 77 866 100% - - -

* Data on this programme is limited because many are administered by the IEFP.

Among the graduates of the different secondary streams, CH graduates are generally the youngest. Vocational graduates are the oldest and approximately one year older than CH graduates. This difference results mostly from higher grade repetition among vocational students, even though dropout rates are higher in CH secondary schooling. At 31.2%, Portugal has the third highest rate of grade repetition in the OECD behind Belgium and Spain, as compared to the OECD average of 12%. After accounting for science and reading performance, disadvantaged students are almost four times more likely to have repeated a grade than advantaged students, again the third highest figure in the OECD. Without accounting for science and reading performance, the odds ratio rises to 10.7 times. Grade repetition usually occurs prior to secondary schooling, in basic education. Portugal also had the highest premature school dropout rate in the EU until recently, in terms of the proportion of residents aged 18-24 who left school without completing upper secondary.88 There are significant differences in higher education entry and completion between graduates of these different programmes, as shown in Figure 2.4.89 For example, among 2013-14 secondary

84 Ibid. 85 OECD, “OECD Economic Surveys: Portugal 2017.” 86 DGEEC, “Transição Entre o Secundário e o Superior: Parte I.” 87 Guerreiro et al., “Relatório Sobre a Avaliação Do Acesso Ao Ensino Superior (Diagnóstico e Questões Para Debate).” 88 Manuel Heitor, Hugo Horta, and Miguel Leocádio, “Enlarging the Social Basis of Higher Education: Lessons Learned from Extending a Social Support System with a Risk-Sharing Loan Scheme in Portugal,” Technological Forecasting and Social Change 113 (2016): 319–27. 89 DGEEC, “Transição Entre o Secundário e o Superior: Parte I.”


41

graduates, approximately 80% of those who completed the CH stream were enrolled in higher education one year later, whereas only 16% of graduates from the vocational secondary stream were in higher education. Moreover, students’ secondary streams are a more important determining factor in higher education participation than socioeconomic status, mother’s educational attainment, or than their prior academic results (average in grade nine finals).

Figure 2.4: Status one year after graduation of Continental-Portugal secondary graduates by secondary stream, 2013-14 and 2014-15

* It is possible that many of these students are pursuing programmes outside of Portugal

The percentage of students who continued in studies after graduation declined with student age at both the CH and vocational levels, reflecting a relationship between educational attainment and grade repetition. Meanwhile, among CH stream graduates, there were also variations between sub-streams as shown in Figure 2.5.90 The CT and CS streams clearly led to greater participation in higher education than the AV and especially the LH stream.

Figure 2.5: Status one year after graduation of Continental-Portugal CH secondary graduates by secondary stream, 2013-14 and 2014-15

90 Ibid.

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

CH Vocational Technological SpecialisedArtistic*

Not Studying

In Secondary Education

In CET Programme Outside HigherEducation

At Higher Education Institution: CET orTESP

At Higher Education Institution:Degree Programme

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

AV CS CT LH

Not Studying

In Secondary Education

In CET Programme Outside HigherEducation

At Higher Education Institution: CET orTESP

At Higher Education Institution: DegreeProgramme


42

Unsurprisingly, multiple studies find relationships between Portuguese students’ secondary school grades and their study success in higher education.91 92 Better performing secondary students generally become better performing higher education students. At the Universidade do Porto, a 2015 study found most students with entering grades of 13 and below (out of 20) did not successfully complete 75% of the expected credit load in their first three years. Dropout rates also appear to be lower in more competitive higher education programmes , and higher in less competitive programmes. These include Integrated-Masters degrees and health and social protection specialties.93 Evidence is limited regarding differences in completion rates among public and private secondary school graduates. At the Universidade do Porto, graduates of private secondary schools have a slightly higher dropout rate (22.9%) than those from public secondary (20.8%).94

3.2.2.2 Adult Education Given the low levels of attainment among the adult population, adult education has been a significant priority in Portugal. In particular, the Nova Oportunidades (New Opportunities) program launched in 2005 sought to massively increase formal qualifications in the Portuguese population, partly though recognition of prior learning, both formal and informal. By 2010, the initiative had provided a credential to 500 000 of the three million Portuguese adults who had not completed secondary education. The program was ended in 2013 in an effort to save money. Today, Portuguese adult learners can pursue two pathways to prepare for higher education. Firstly, they can pursue a secondary credential. Adult Education and Training Programmes (Cursos de Educação e Formação de Adultos, EFA) support adults over age 23 and emphasize workplace learning and placement. Finally, CH Continuing Education Programmes support students 18 and older who did not complete secondary schooling, some students above age 16, and students above the age of 20 who did not complete the CH secondary stream. The EFA also more than doubled in size from 2010-11 to 2014-15, while CEF programmes were essentially phased out.95 The alternative is post-secondary non-tertiary (ISCED-4) training. CET programmes generally are one year in duration. Universities and IPs offered just 24% of such programmes in 2005-06, with the balance delivered by technical schools, technological schools and training colleges. However,

91 IP de Setúbal, “O Abandono Nos Cursos de Primeiro Ciclo.” 92 Sarsfield Cabral, “Algumas Notas Sobre o Acesso e Desempenho Dos Estudantes Admitidos Na U.Porto Em 2008/09, 2009/10 e 2010/11 (Pelo Regime Geral de Acesso Em 1s Ciclos e Mestrados Integrados).” 93 DGEEC, “Ensino Superior: Situação Em 2012-13 Dos Inscritos Pela Primeira Vez Em 2011-12.” 94 Sarsfield Cabral, “Algumas Notas Sobre o Acesso e Desempenho Dos Estudantes Admitidos Na U.Porto Em 2008/09, 2009/10 e 2010/11 (Pelo Regime Geral de Acesso Em 1s Ciclos e Mestrados Integrados).” 95 DGEEC, “Transição Entre o Secundário e o Superior: Parte I” (Lisboa: Direção-Geral de Estatísticas da Educação e Ciência, 2016).


43

subsequently higher education institutions increased their CET programme offerings considerably, from 61 programmes in 2005-06 to 450 programmes in 2013-14, with just over half of these programmes at public IPs. Students who continue from CET programmes into higher education typically attended IPs (Grupo de Trabalho, 2016). Another policy change in 2016-17, however, is taken away the right to deliver CET programmes from higher education institutions, so that CETs will only be delivered by institutions affiliated with the Institute for Employment and Vocational Training, the Tourism Institute of Portugal, and other institutions affiliated with the Ministry of Employment and the Ministry of Economy.96 Santos et al. conducted a study on academic success among mature students (above the age of 23) at two Portuguese universities, profiled in Figure 2.6.97 It illustrates that comparatively large numbers of mature students at the universities were above the age of 40 (22% to 28%), large proportions had less than upper secondary education credentials, and most had interrupted studies before their programme for more than 11 years. The study found older students had better grades at one of the institutions, but surprisingly no significant relationship between students’ prior academic preparation and grades. Students above the age of 50 did report the greatest difficulties with study content. Research indicates that older students in Portugal are less prepared for higher education than traditional age colleagues in general and institutional level studies find higher dropout rates among older students.98 Older students also appear to transfer relatively less than younger students. Yet, an IP de Setúbal survey (2013) of first-cycle students who had dropped out, almost three-quarters of whom were 24 or older, found that they discounted inadequate preparation as a key factor.99 Fully 90% reported feeling sufficiently prepared.

96 Guerreiro et al., “Relatório Sobre a Avaliação Do Acesso Ao Ensino Superior (Diagnóstico e Questões Para Debate).” 97 Santos et al., “Academic Success of Mature Students in Higher Education - A Portugese Case Study.” 98 Alarcão, “Insucesso Académico e Abandono Escolar.” 99 IP de Setúbal, “O Abandono Nos Cursos de Primeiro Ciclo.”


44

Figure 2.6: Characteristics of mature students at Santos et al. case study universities

These observations regarding pathways into higher education provides further depth to our understanding of the secondary preparation of Portugal’s higher education students. It seems certain that the level of competencies among traditional, academically-trained (i.e. CH and CT) secondary students are more than adequate to accomplish high rates of study success in higher education, and these students are, on the whole, already accomplishing relatively high study success rates. The preparation of Portugal’s other higher education students, however, is more clearly a source of difficulty in improving higher education study success. Certainly, this leaves scope for considerable improvements in basic and secondary schooling to ultimately improve study success throughout students’ educational careers. Expanding higher education enrolment among students from these pathways should be done cautiously to avoid recruiting students who simply do not have adequate preparation to complete higher education successfully. However, these findings indicate there is considerable scope for the higher education system to improve its own policies and practices to better support admitted students.

3.3 Student Motivation Student motivation is broadly a function of self-motivation, self-esteem and self-efficacy. This is arguably the most important actionable factor in study success. Students who are interested in and committed to their study programme or their later career are considerably more likely to complete and stay within the same institution. We will discuss here three notable themes in student motivation, which appear very consistent with evidence of Portuguese students’ experiences. Specific data connecting students to these themes, however, will be discussed largely in Chapter 3 on admissions processes, Chapter 4 on quality in the delivery of higher education and Chapter 5 on financing of the higher education system.

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

University 1

University 2

Age ProfileLess than 30 years old

Between 30 and 39 years old

Between 40 and 49 years old

50 years old and older

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

University 1

University 2

Highest Prior Level of Education Upper secondary education

Incomplete upper secondaryeducation

Only primary schooling

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

University 1

University 2

Interruption in education prior to enrolment More than 11 years

Between 6 and 11 years

Less than 6 years


45

3.3.1 Expectations There is considerable evidence of the relationship between student expectations and study success: students are more likely to dropout when their prior expectations are not met. Expectations are built around accurate information up front about the nature of a study programme, including content, the academic culture, and instrumental career-related elements, as well as accurate estimation of students’ own ability to complete the programme successfully. In other words, the quality of students’ decision-making in selecting their study programme is closely related to their likelihood of success. A UK study found students who “used fewer sources of advice in applying” and believed they received poorer quality career guidance were more likely to drop out. A Spanish study found students placed in higher education programmes that were not their preference had higher dropout rates.100 International research also suggests differences in motivation related to expectations between traditional age and mature students. In particular, mature students appear to have more defined goals, and less challenges with motivation more broadly, largely as a result of this and greater maturity overall.101 Santos et al. found that motivation was a bigger issue among the youngest mature students, though the oldest students (over 50 years of age) reported that courses did not meet their expectations.102 Interestingly, while Portuguese students have relatively high dropout rates, they also report planning to continue their studies after graduation relatively often by European standards.103 This was true for 64% of all students and 63% of Licenciatura students in the EuroStudent Survey, with only five and six percent respectively indicating they did not plan to continue studying. Both of these figures were considerably more favourable than the medians among participating European countries, which were 51.5% to continue studies among all students and 53% among Licenciatura-equivalent students. Helping to corroborate this finding, a IP do Porto survey found 76% of students who dropped out intended to return to their studies.104 These observations support an overall view that active students are relatively committed to studying.

3.3.2 Sense of Belonging There is considerable evidence that “students’ social and academic integration” contributes to their likelihood of persisting and completing their study programme successfully. This is based

100 Ibid. 101 Santos et al., “Academic Success of Mature Students in Higher Education - A Portugese Case Study.” 102 Ibid. 103 Dominic Orr, Christoph Gwosc, and Nicolai Netz, “Social and Economic Conditions of Student Life in Europe: Synopsis of Indicators - Final Report - Eurostudent IV 2008-2011” (Bielefield: W. Bertelsmann Verlag, 2011). 104 Gabinete de Planeamento, Projectos e Desnvolvimento, “Estudo de Caso: Abandono Escolar 2012-13 (ESE, ESMAE, ESEIG, ISCAP, ESTGF, ESTSP e APNOR).”


46

largely upon the development of relationships with students’ peers and instructors that encourage the students to fully engage with or participate in not only their courses but in wider activities of the institutional community. Related to this is the empirical observation that smaller institutions tend to better engage students in relationships with their professors and the broader school community. As a result, students at smaller institutions are more likely to complete their degrees. Support from family can be important in helping students overcome challenges to completion. Lower collaboration of spouses contributes to women reporting greater difficulties completing their Masters degrees than men (although completion rates are still lower for men).105 At the IP de Setúbal, married students were less likely to dropout than single students, particularly of their age, though they dropped out much more often than they transferred programmes.106

3.3.3 Career Related Motives This section focuses on long-term career motivations and their effects on completion. We distinguish between long-term career motivations and part-time employment (discussed in later chapters) because in many cases part-time student jobs are not very relevant to students’ post-graduation career plans, even though employment during studies may improve students’ long-run employment prospects.107 This is, of course, not a perfect distinction, and variably true for different students. Santos et al. found in a case study of two Portuguese universities that mature students’ jobs typically took precedence over their classes and other educational activities.108 On the other hand, Alves et al. provide evidence that only 27.1% of Portuguese students have work experiences relevant to their longer-term career aspirations while studying, as compared to 43.2% of students in other European countries.109 In general, Portugal is similar to other countries in that career aspirations appear to be important in students’ decisions to pursue higher education. A 2006-2007 survey of 11 637 students found that gaining access to an appealing career was the most commonly cited motivation for attending higher education and noted as the most important among 41.4% of respondents.110 Men were slightly more likely to emphasise economic motivations than women, while other research suggests labor market goals may be especially important to adult students.111 Santos et al. also

105 IP de Setúbal, “O Abandono Nos Cursos de Segundo Ciclo.” 106 IP de Setúbal, “O Abandono Nos Cursos de Primeiro Ciclo.” 107 Vossensteyn et al., “Dropout and Completion in Higher Education in Europe.” 108 Santos et al., “Academic Success of Mature Students in Higher Education - A Portugese Case Study.” 109 Mariana Gaio Alves, Natália Alves, and Miguel Chaves, “Inserção Profissional e Razões de Ingresso e Reingresso No Ensino Superior: Um Ponto de Partida Para Uma Temática Em Aberto,” Sociologia, Problemas e Práticas 69 (2012): 99–118. 110 António Magalhães, Maria de Lourdes Machado-Taylor, and Maria José Sá, “Satisfação Dos Estudantes Do Ensino Superior Português” (Fundação para a ciência e a tecnologia, 2012). 111 IP de Setúbal, “O Abandono Nos Cursos de Primeiro Ciclo.”


47

found professional motives extremely important among mature students at two Portuguese universities, especially for those aged 30-49.112 Another qualitative study (referenced more in Chapter 3) found that Portuguese students were very conscious of the difficult job market, and interested in obtaining higher education to achieve better economic outcomes.113 The most significant advantages students ascribed to higher education were improved job opportunities, greater expertise, social status and recognition, and improved income. An IP de Setúbal study on first-cycle students for example found that 72% of students who dropped out enrolled for reasons of personal realization, 45% for the job market value of the degree, 31% to pursue deeper knowledge, 23% to progress in their careers, and 6% to ease transitions between jobs.114 Research at the Universidade do Minho indicated that students who believed their programmes were not helping them prepare for the labour market dropped out more, which has been found in other studies as well.115 Career motivations may even be key drivers of withdrawal among students with prevailing epistemic motivations, as was the case among second-cycle students at IP de Setúbal.116 There is no evidence that employment related motivations to attend higher education intrinsically affect study success relative to other motivations. However, there are cases where long-term career-related motivations may encourage dropout, at times for reasons outside of an institution’s control.

Returns to Higher Education Portugal has struggled with high unemployment rates for quite some time, particularly among young people.117 Youth unemployment peaked at 42% in 2013, before falling back to roughly 26% today. Unemployment is considerably lower, however, among tertiary graduates, at 9% across the population as compared with 14% among those with lower formal credentials. For those young people who secure jobs, job quality is generally low in terms of earnings, working environments and job security. Portugal’s labor market is marked by a high share of workers on temporary contracts. Portugal has the third highest proportion in the OECD of employees aged 25-49 on temporary contracts at just over 20%. However, this figure spikes to two-thirds of workers aged 15-24. These trends are relatively common even among tertiary graduates. The primary cause according to the OECD appears to be the creation of a bifurcated labour market due to extremely strong protections for permanent employees.

112 Santos et al., “Academic Success of Mature Students in Higher Education - A Portugese Case Study.” 113 Orlanda Tavares, “Routes towards Portuguese Higher Education: Students’ Preferred or Feasible Choices?,” Educational Research 55, no. 1 (2013): 99–110. 114 IP de Setúbal, “O Abandono Nos Cursos de Primeiro Ciclo.” 115 Ibid. 116 IP de Setúbal, “O Abandono Nos Cursos de Segundo Ciclo.” 117 OECD, “OECD Economic Surveys: Portugal 2017.”


48

Despite or as a result of these circumstances, Portugal has among the highest internal rates of return to tertiary education in the OECD, at 18.9% for women and 18.5% for men (the respective OECD averages by gender are 11.6% and 13.9%). Of course, these returns may vary considerably between graduates of different institutions and study programs. Yet, returns have been declining across all fields. This might be expected given the considerable expansion in supply of educated workers with rising tertiary attainment, but the OECD’s assessment is that this trend weakens incentives to invest in skills development, therefore undermining completion. Interviewees supported the idea that changes in perceived graduate employability affect students’ commitment to their academic programmes

Education as a “parking lot for the labour market” International research also suggests that where job opportunities for youth are limited, students who are not motivated for any particular study programme may enter higher education only to improve their career prospects. They will be expected to leave school if they find employment opportunities. Of course, if employment opportunities for high school graduates expand fewer of these students may also be expected to enter higher education, therefore potentially increasing short-term drop-out rates but reducing them in the medium to long-term. This pattern has been documented in Italy and characterised as “education as a parking lot for the labour market”. There is not systemic evidence of the “parking lot” phenomenon in study success in Portugal. Almeida et al. in 2008 did find that many students dropped out because they found job opportunities prior to completion.118 However, only 2.1% of students in the large, national 2006-2007 survey mentioned earlier indicated that they pursued higher education because they could not find a job, and just 0.3% cited this as their primary motivation for studying.119 Nevertheless, the necessary patterns are present in the labour market, and based on interviewees’ views (including their observation of cultural similarities with Italy), the parking lot motivation seems likely to be affecting many Portuguese higher education students.

4. Why and How Study Success Matters Having reviewed the data on study success internationally and within Portugal, as well as the evidence regarding independent factors in study success, can we confirm whether rates of study success matter for a higher education system, an institution, or a study programme? For students, credentials are meant to communicate that they have obtained valuable skills and knowledge, which they may not achieve if they drop-out. The financial benefits of completing

118 IP de Setúbal, “O Abandono Nos Cursos de Primeiro Ciclo.” 119 Magalhães, Machado-Taylor, and Sá, “Satisfação Dos Estudantes Do Ensino Superior Português.”


49

degrees are also significant. Data cited in Chapter 1 indicates that there are high financial returns to having a higher education degree in Portugal and empirical evidence from the US indicates that these returns are important even for students with marginal skills at admission and during their studies.120 121 122 Dropout also affects self-esteem for students and their families.123 In an institutional survey, almost two-thirds (63.5%) of students who dropped out from the IP do Porto reported being dissatisfied with the decision.124 Low study success can also have significant consequences for institutions and higher education systems. It can affect the strength of institutional communities, their financial sustainability and their successful execution of their academic mission.125 Poor study success can indicate that the system is inefficient, failing to meet students’ needs and recruiting students who do not have sufficient preparation to be successful. Finally, given there are social factors that shape students’ study success and the personal benefits of completion, low rates of study success can reflect and reproduce inequities not only within higher education, but across society. Nevertheless, consider that even though Portugal’s completion rates fall below the OECD average by various measures, they appear to be higher than those in Sweden and Austria, and close to those of the Netherlands.126 Recognising that these countries’ higher education systems are widely admired and these countries are comparatively very equitable, this suggests successful systems can have lower study success outcomes. Suitable levels of study success depend on the role that higher education plays within the particular society and economy. In systems that value lifelong learning, students may not be seeking to complete a degree and earn a credential, but simply to pursue certain courses for personal or professional reasons.127 Students may also be encouraged to study between jobs, or leave their studies to work when employment opportunities are present. In Sweden, for example, completion rates have been lower because there is a pattern of students moving between the higher education system and the labour market based on their interests and the skills they need, which arguably helps promote the country’s competitiveness. To take a Portuguese example, a study found 10% of Masters students who dropped out at the IP de Setúbal reported that they had only intended to complete part of their degree equivalent to a post-graduate programme

120 Joshua Goodman, Michael Hurwitz, and Jonathan Smith, “Access to 4-Year Public Colleges and Degree Completion,” Journal of Labor Economics 35, no. 3 (2017): 829–67. 121 Seth D. Zimmerman, “The Returns to College Admission for Academically Marginal Students,” Journal of Labor Economics 32, no. 4 (2014): 711–54. 122 Ben Ost, Weixiang Pan, and Doug Webber, “The Returns to College Persistence for Marginal Students: Regression Discontinuity Evidence from University Dismissal Policies,” Discussion Paper, IZA Discussion Paper Series (Bonn, Germany: IZA, 2016). 123 Alarcão, “Insucesso Académico e Abandono Escolar.” 124 Gabinete de Planeamento, Projectos e Desnvolvimento, “Estudo de Caso: Abandono Escolar 2012-13 (ESE, ESMAE, ESEIG, ISCAP, ESTGF, ESTSP e APNOR).” 125 Alarcão, “Insucesso Académico e Abandono Escolar.” 126 “QS Higher Education System Strength Rankings (HESS),” Top Universities, April 25, 2016, https://www.topuniversities.com/system-strength-rankings/2016. 127 OECD, “Education at a Glance 2016: OECD Indicators.”


50

(i.e. the in-class part of their program without the thesis).128 In this case, these students arguably have been successful in accomplishing their goals, so this is in a sense a harmless withdrawal that is only negative for the institution’s statistics. The more detailed the data on study success collected is, the clearer it will for policy analysts to interpret and distinguish constructive and destructive factors contributing to the overall picture. There are three important trade-offs inherent to the concept of study success at the system or institution levels, which any study success strategy will have to navigate or break out from:

1. Study Success and Access. We have seen that traditional students recruited into higher education often have the highest rates of study success. Expanding access is a very legitimate policy objective, however, that entails recruiting more students from disadvantaged backgrounds and more unconventional education pathways. This may reduce study success rates as a proportion of students, but make the country better off by generating far more graduates in absolute terms. Zimmerman’s study on access for academically marginal students in Florida provides persuasive evidence to this fact, which would appear equally relevant to Portugal.

2. Study Success and Learning. An obvious option for improving study success based on the measures we have indicated would be to simply pass students through irrespective of what they have learned. Of course, this would be contrary to the purpose of higher education, which is not merely about earning a credential but developing the knowledge and skills that the credential represents.

3. Completion and Time-to-Completion. Slow time-to-completion can reduce the

completion rate, measured as the proportion of students who finish the programmes in a given length of time. Yet, some policy instruments may promote faster time-to-degree at the expense of lower completion, such as policies that dismiss students who do not achieve satisfactory progress in a given year or do not complete their study programmes in a given period of time. Policy-makers may also wish to encourage students to drop out or transfer at the front end of their programmes, so that the time of unsuccessful study is more limited. Completion and drop-out rate objectives are more clearly complementary because it is impossible to attain a high completion rate without a low drop-out rate. Our emphasis in this study is on completion, but not at the expense of great inefficiency.

These trade-offs indicate that Portugal must strike a balance in its approach to study success. Portugal is committed to expanding access to higher education, as it should be, but not to a point where institutions are recruiting students who are overwhelmingly unprepared to succeed. Depending on prioritisation of time-to-completion relative to drop-out, Portugal should look to establish and reasonably enforce timelines for programme completion, but not through measures that force students out of education who require more time to complete their studies

128 IP de Setúbal, “O Abandono Nos Cursos de Segundo Ciclo.”


51

for legitimate reasons (e.g. disability, illness, changes in interests, some part-time employment, etc.).129 Finally, academic standards must be rigorously reflective of expected learning outcomes, but grounded in an understanding of what students really need to learn and can be reasonably expected to accomplish. Most importantly, Portugal should seek to minimize or break through these trade-offs where possible. The OECD’s observation that the best education systems attain both quality and equity can be reinterpreted as arguing that the best education systems escape trade-offs that would compromise quality or equity for one another. Finding ways to escape these trade-offs will be the focus of much of this report. We will therefore consider for example strategies to improve student recruitment and bridging to expand access with minimal study failure, and to deliver education and services so more active students make progress and complete their programmes while accomplishing expected learning outcomes.

129 In Canada, for example, students with certified disabilities can qualify for full-time status at a lower course-load than other students, and have extended student financial aid eligibility (i.e. they have one additional year of eligibility).


53

Chapter 3: Admissions This chapter analyses how Portugal’s higher education admissions system promotes or hinders the effective matching of students, institutions and programmes, by balancing access and study success through smart selectivity, fostering accurate student expectations, and connecting students with programmes that meet their expectations. We first describe the structure of Portugal’s higher education system and its admissions processes, as well as the information available students and research on how they make decisions. Portuguese students gain admission to higher education through a number of different pathways. At public institutions, most traditional-age students use the National Access Competition (CNA), which is based on their grades secondary education and especially from secondary education completion exams. Pathways for students at private institutions, as well as those for non-traditional students, are controlled at the institutional level pursuant to some national standards. The Government controls the total number of spaces in institutions and study programmes through Numerus Clausus (NC). We identify four ways in which Portugal’s admissions system may undermine study success. Alternative admissions streams lead to high dropout to such an extent that they may simply be setting students up to fail. The CNA promotes shallow matching based on grades that are unreliable and that provide a limited indication of student attributes at best. NC restricts access to sought-after programmes, which promotes dissatisfaction among some and undermines competition between institutions. Finally, there is insufficient information on the student experience, in terms of student engagement or student satisfaction survey results. International jurisdictions have adopted many different approaches to promote higher study success through improvements in admissions. These include efforts to expand criteria for admissions and improve information for students upon application and enrolment.


54

Chapter Guide

1. Analytical Framework ...................................................................................................... 55

2. Lay of the Land ................................................................................................................ 55 2.1 Overall System Structure ........................................................................................................ 55 2.2 Factors in Student Choices ...................................................................................................... 57 2.3 Information for Prospective Students...................................................................................... 59 2.4 Admissions Processes ............................................................................................................. 61

2.4.1 The General Access Stream........................................................................................................ 61 2.4.2 Special Admissions Competitions .............................................................................................. 64 2.4.3 Special Admissions Stream ........................................................................................................ 65 2.4.4 Admissions by Admission Pathway ............................................................................................ 65

2.5 Numerus Clausus .................................................................................................................... 66

3. How Could Higher Education Admissions Contribute to Study Success in Portugal? ........... 67 3.1 Alternative Access Streams Lead to High Dropout ................................................................... 68 3.2 The National Access Competition Promotes Shallow Matching ................................................ 69 3.3 Numerus Clausus Restricts Access to Sought-After Programmes .............................................. 71 3.4 Insufficient Information on the Student Experience ................................................................. 75

4. International Experiences ................................................................................................ 76

List of Tables Table 3.1: Motivations for choosing their higher education institution and indications of their importance, 2005-2006 ................................................................................................................ 58 Table 3.2: Subjects of National Access Exams ............................................................................. 62 Table 3.3: Reserved quotas for the first round of the National Access Competition ................ 63

List of Figures

Figure 3.1: Shares of public institutions providing relevant information on activities relating to communication with potential candidates and attraction of new students on their websites 61 Figure 3.3: Newly admitted students by admissions stream and institution type, 2014-15 .... 65 Figure 3.2: National Access Competition applications and seats, 1999-2016 ........................... 66 Figure 3.4: Enrolment and number of dropouts after first year in public higher education by student admissions pathway, 2012 ............................................................................................. 68 Figure 3.5: RGA admissions by option or other pathway, 2014-15 ............................................ 71


55

1. Analytical Framework Higher education admissions processes are identified in the literature as playing an important role in determining student success. Our findings in the previous chapter indicate that there are differences in study success rates based on the admissions processes students use in Portugal. Of course, this is not an adequate measure of the significance of the admissions mechanisms in study success because the different access measures are deliberately tied to students’ prior preparation. Again, students using conventional education pathways generally obtain higher study success, and students who transfer also might be expected to have inferior results because their transfer is partly reflective of them confronting difficulties in higher education. We also observe that the admissions pathways are used differently by different higher education institutions that have varying study success rates themselves. With these caveats noted, International literature provides us with a key touchstone for evaluating the effectiveness of Portugal’s admissions structures in terms of study success: promoting effective matching.130 There is strong international evidence that students should be encouraged to study with the most capable peers in the best environment’s possible, in contrast with some notions of matching. Still, our emphasis is on balancing access and study success through smart selectivity, fostering accurate student expectations, and connecting students with programmes that meet their expectations. A strong higher education system should collect sufficient accurate information from students to judge their suitability for the specific study programme, provide strong information to support students’ choices of study programmes, and connect students and programmes where these two processes align.

2. Lay of the Land

2.1 Overall System Structure Before discussing admissions to higher education in Portugal, we need to understand briefly the structure of the system students are seeking admissions to. Portugal has a binary higher education system with both universities and polytechnic institutes (institutos politécnicos, IPs), which may be public or private. The public system includes 14 universities, 15 IPs and five non-integrated higher education schools (Escolas Superiores Não Integradas, ESs) that are generally categorised among the IPs.131 Most of these institutions were established since the establishment of democracy in 1974, for instance the IP system was launched in 1979-80. The institutions generally have largely

130 Hans Vossensteyn et al., “Dropout and Completion in Higher Education in Europe” (European Union, 2015). 131 Jon File et al., “Policy Challenges for the Portuguese Polytechnic Sector” (Centre for Higher Education Policy Studies, April 2013).


56

comprehensive orientations and therefore limited degrees of specialisation as compared with institutions in many other European countries.132 ESs are an exception and focus on speific areas: three are nursing schools, one is a tourism institute and the other delivers nautical programmes.

Private institutions developed largely in the 1980s and 90s in response to inadequate capacity in the public system to meet demand. Private coverage rose from 10% of total higher education enrolment in 1983-84 to peak at 36% in 1996, but has been falling since. It was just 21% in 2012. No new private universities have been opened since 1996, or private polytechnics since 2001.133 These institutions are generally smaller and much more specialised than public institutions, especially at the polytechnic level. The higher education system has been seriously affected by the country’s demographic transition. The 20-24-year-old population declined 29% from 2000 to 2015 and is expected to fall by a further 7.3% by 2030.134 135 These declines are having the greatest impacts on the disadvantaged interior, Madeira and the Azores. Higher education institutions are largely concentrated in the country’s major cities: 42% of public and 76% of private seats are in the cities of Porto and Lisbon. The more modest representation of public seats in the major cities results largely from a deliberate decision to locate many public IPs especially in areas that did not have universities to promote regional development. Seven of the fifteen public polytechnics are situated in Portugal’s less developed interior, where they are the predominant higher education providers.136 137 In 2009-10, the median public IP recruited 74% of new entrants from their local regions, which was consistent with universities of applied sciences across much of Europe.138 There are also four public universities in the interior, as well as one each in Madeira and the Azores. Portugal follows the Bologna degree structure of first cycle (bachelors or Licenciatura), second cycle (Master) and third cycle (Doctoral) degrees. Universities can offer three-to-four-year Licenciatura degrees, while IPs can offer only three-year Licenciatura degrees with few exceptions. The IP Licenciatura “must value particularly training actions targeted at the practice of a professional activity, ensuring a component of application of the knowledge acquired to the actual activities of the respective professional profile”.139 Both universities and IPs can also offer Master degrees, though universities must "ensure that the student acquires an academic specialisation relating to research, innovation or expansion of professional competences" while

132 Ibid. 133 Ibid. 134 Ibid. 135 European Commission, “Database - Eurostat,” Ec.Europa.Eu, March 5, 2017, http://ec.europa.eu/eurostat/data/database. 136 Jon File, “Higher Education in Portugal,” IHEM Country Reports (Enschede, the Netherlands: Center for Higher Education Policy Studies, 2008). 137 Andrea Blattler et al., “Portuguese Higher Education: A View From the Outside” (European University Association, 2013). 138 File et al., “Policy Challenges for the Portuguese Polytechnic Sector.” 139 Ibid.


57

IPs must "ensure predominantly that the student acquires a professional specialisation".140 Integrated-Masters programs generally integrate the first and second cycles creating programs that are usually five years in length. Finally, universities alone offer doctoral degrees. Additionally, as of 2014-15 Portugal’s higher education system also offers TESP programmes that generally last two years (120 ECTS). The programmes specialise in areas of skilled shortage and are linked to jobs, but these programmes were also framed as an effort to improve completion.141 They are organised into three components: general and scientific training, technical training, and work-place training.142 The OECD reports that enrolment and private sector participation in the new programs has been strong.143

2.2 Factors in Student Choices Understanding the original bases for students’ decision-making about higher education is necessary to inform our analysis of the admissions structure. We can rely on three key studies in particular for information on student decision-making. The first study is a survey of 11 637 students at public (10 908) and private (729) institutions in 2005-2006.144 It measured students’ motivations in attending higher education and choosing their particular institution (as well as student satisfaction which we will consider in Chapter 4). Students’ career motivations to attend higher education were highlighted in the previous chapter, but many students also expressed interest in the intrinsic value of learning. Table 3.1 presents findings on student priorities in choosing their higher education institution.145 Most students cited their institution’s reputation as a factor in choosing that institution, and this is the most important factor for 17% of students. More students cited as their primary motivation their institution as being best for their particular study program (26.3%), and selection based on study program was especially common among students attending private institutions. Being close to home was mentioned by almost half of students and the most important factor for 23%. It was particularly important for students at IPs. The last two factors identified each by more than 8% of students as the most important related to their institution being their only option, in terms of either their study program of interest or admissions offer.

140 Ibid. 141 Vossensteyn et al., “Dropout and Completion in Higher Education in Europe.” 142 João Guerreiro et al., “Relatório Sobre a Avaliação Do Acesso Ao Ensino Superior (Diagnóstico e Questões Para Debate)” (Lisboa: Grupo de Trabalho para a Avaliação do Acesso ao Ensino Superior, 2016). 143 OECD, “OECD Economic Surveys: Portugal 2017” (Paris: OECD Publishing, 2017), http://dx.doi.org/10.1787/eco_surveys-prt-2017-en. 144 António Magalhães, Maria de Lourdes Machado-Taylor, and Maria José Sá, “Satisfação Dos Estudantes Do Ensino Superior Português” (Fundação para a ciência e a tecnologia, 2012). 145 Ibid.


58

Table 3.1: Motivations for choosing their higher education institution and indications of their importance, 2005-2006

Motivation Students citing

Students citing as most important

Good reputation academically 56.5% 16.6%

Best for the study program that I wanted 45.8% 26.3%

Close to home 45.7% 23.3% Good facilities for curricular activities 24.2% negligible

Graduates get good jobs 23.3% negligible

Good reputation in terms of social life 20.9% negligible Graduates of the institution have social prestige 20.2% negligible

Recommended by friends 16% negligible Only institution offering the study program they wanted 14.8% 8.4%

Institutional information and promotional activities 14.3% negligible

Only one they were admitted to 14.3% 8.8% Recommended by family 13.2% negligible

Good facilities for non-curricular activities 7.7% negligible Cost of tuition 7.6% negligible

Friends chose it 3.3% negligible

Sá et al. completed a survey of 13,527 first-time students in 2006-2007.146 They found that students’ higher education choices are strongly related to their income. In particular, lower income students were more likely to study at IPs, while higher income students were more likely to attend universities. The same pattern was consistent with parents’ education levels. The study found that Portuguese students’ choice of a specific institution were informed by the opinions of family members and friends. However, this did not affect their choice as to whether they would attend a university or an IP. University students more often emphasised prestige and location in their choice of institution, while IP students emphasised leisure and employability. Much of the Sá et al. study focused on geographic accessibility of institutions. In particular, students demonstrated a strong preference for higher education institutions close to their family home, notably so that they can continue to live with parents. This related to rent prices. The choice to stay at home was mostly driven by financial considerations, but also related to “family resources, community ties, aspirations and expectations”. Higher income students with better educated parents were more likely to stay at home, as were students attending IPs. There was no significant relationship between the choice to stay at home and student gender, but older students were less likely to leave. Fonseca and Encarnação also cite “emotional, social or

146 Carla Sá et al., “Higher Education (Related) Choices in Portugal: Joint Decisions on Institution Type and Leaving Home,” 2012.


59

psychological factors” in students’ choices of whether to leave home (e.g. desire for change of scenery, emancipation, or conversely emotional dependency and loneliness).147 Orlanda Tavares interviewed 60 students from public and private universities and polytechnics in Porto in 2010-11 to gain a better understanding of the factors in their choices of first cycle study programmes.148 Recognizing the constrains on generalization from such a modest sample size, her paper still offers useful insights into students’ choices of higher education institutions and programmes. Family members represented a strong influence in her interviews in terms of students’ desire to access higher education, but not their choices of institution and only modestly their study programmes. Instead, her research indicated that secondary schools had an important influence on the students’ choice of study programmes, through study choices they had to make starting in eighth grade, the year of streaming into different secondary streams and sub-streams, but also based on their success in particular subjects and positive relationships with the teachers for those subjects. Friends were less significant in driving study choices. Much of Tavares’ work reflects upon students’ perceptions of highly stratified prestige within the higher education system. Public universities were viewed as particularly prestigious, and students associated higher prestige with improved economic prospects. Most students in the study reported choosing a feasible institution over their preferred institution, constrained by finances and their academic scores. Financial constraints related especially to being located close to home, where presumably students received substantial familial support. Still, the majority of the students reported entering their study programmes of choice. Pharmacy students were the main exceptions, and research suggests they might have been pursuing pharmacy in lieu of medical school.149 Interest in a particular vocation was the biggest factor in study programme choice. Many of the students attending their preferred institutions were pursuing study programmes that their institution was the only one to offer.

2.3 Information for Prospective Students The Portuguese Government has various websites to inform prospective students. Most importantly, the site of the Directorate-General for Higher Education (Direção-Geral do Ensino Superior, DGES) operates a public database of higher education programmes, allowing students

147 Madalena Fonseca and Sara Encarnação, “A Massificação Do Ensino Superior Em Portugal: Efeitos Espaciais Na Diversificação Do Sistema” (Lisboa: Agência de Avaliação e Acreditação do Ensino Superior, 2012). 148 Orlanda Tavares, “Routes towards Portuguese Higher Education: Students’ Preferred or Feasible Choices?,” Educational Research 55, no. 1 (2013): 99–110. 149 Madalena Fonseca et al., “Waves of (Dis)Satisfaction: Effects of the Numerus Clausus System in Portugal.,” European Journal of Education 49, no. 1 (2014): 144–58.


60

to search for programmes by type, institution, or district.150 A Choice Assistant (Assistente de Escolha) allows students to search for programmes based on their preferences and the national exams they completed. The site provides general information on each programme (duration, degree level, duration, ECTS), the admissions streams available for the programme, the number of seats for the subsequent year, the required national exams, required minimum classifications, and the method of calculation of students’ application grade (the proportion derived from secondary school grades and the proportion from entrance exams). Information is also provided regarding prior years’ admissions including the numbers of candidates in the first or second phase of the national competition of the RGA, the numbers of students placed in the two phases, breakdown by gender, students’ application grades, and the grade of the last students accepted in each round. The site also provides links to the Agência de Avaliação e Acreditação do Ensino Superior (Higher Education Evaluation and Accreditation Agency, A3ES) website so students can review the accreditation status of a given programme, and the website of the DGEEC for further information. The DGEEC site, Infocursos, provides detailed information on many courses of study as of 2014-2015.151 This information is available not only for the country as a whole, but at the level of individual study programmes at particular institutions. This information may include the following, although some of these data points are missing in some cases:

• Students’ admissions pathways

• Percentiles of students entering the programme in the national entrance exams, relative to all other students who completed the same exams

• An indication of where students are one year after beginning the programme (graduated, studying in the same programme, studying in another programme at the same institution, studying in programme at another institution, or outside of Portuguese higher education)

• Distribution of students by gender compared to the national average

• Distribution of final grades among programme graduates

• Distribution of students by nationality (Portuguese and foreign nationality)

• Percentage of recent graduates who are registered as unemployed

• Distribution of student ages Pie charts, line charts and bar charts present the data to facilitate interpretation, but the DGEEC also provides access to excel spreadsheets. Obviously, this information is backed by a broader system tracking student admissions, progress, completion and post-graduate (un)employment. Institutions also pursue some activities to promote recruitment. A not yet published review of the websites of public institutions indicates that many institutions have prospective student

150 DGES, “Acesso Ao Ensino Superior,” Sítio Da Direção-Geral Do Ensino Superior, 2017, http://www.dges.gov.pt/pt. 151 DGEEC, “Dados e Estatísticas de Cursos Superiores,” Infocursos, 2017, http://infocursos.mec.pt/.


61

portals, host visits by secondary school students or teachers to their facilities, or hold scientific or vocational fairs or exhibitions. Figure 3.1 presents the findings for public universities and IPs.152

Figure 3.1: Shares of public institutions providing relevant information on activities relating to communication with potential candidates and attraction of new students on their websites

Prospective student portals are the most common, followed by campus visit programmes. The websites at nine IPs and six universities gave relevant information on all these activities, while only one IP and two university websites gave no relevant information on any of these activities.

2.4 Admissions Processes General higher education admissions to Portugal’s universities and polytechnic institutes, both public and private, are strictly regulated by the MCTES. There are multiple application streams to enter the institutions, under varying government or institutional control.

2.4.1 The General Access Stream The General Access Stream (Regime Geral de Acesso, RGA) serves recent graduates of upper secondary seeing to enter Licenciatura and Integrated-Masters programmes. Students must have completed their secondary diploma to be eligible for this admissions stream, and generally are the traditional age for entering the first cycle of higher education. Secondary students in the SH or Technological Streams must complete the National Admissions Exams (Provas de Ingresso) to graduate. These exams are based on CH secondary curricula and

152 DGEEC - Equipa de Estudos de Educação e Ciência, “Draft: Medidas de Promoção Do Sucesso Escolar Nas Instituições de Ensino Superior Públicas Em Portugal, Tal Como São Relevadas Nos Respetivos Sítios” (Lisboa: DGEEC, Unpublished).

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Prospective Student Portal

Campus Visits

Scientific or vocational fairs and exhibitions

IPs Universities


62

currently offered in eighteen different subjects (see Table 3.2).153 They are valid for application to higher education in the year they are completed and for two subsequent years. Table 3.2: Subjects of National Access Exams

# Subject # Subject # Subject

1 German 8 French 15 Portuguese Literature

2 Biology and Geology 9 Geography 16 Mathematics

3 Design 10 Descriptive Geometry 17 Applied Mathematics - Social Sciences 4 Economics 11 History

5 Spanish 12 History of Culture and Arts 18 Portuguese 6 Philosophy 13 English 19 Mathematics A

7 Physics and Chemistry 14 Latin

Public and private higher education institutions identify the required exams for each programme, though in general they cannot require completion of more than two exams for a particular study programme. In many cases, students may choose to be assessed based on up to three different combinations of exams to be admitted. For example, students applying to the Basic Education Licenciatura programme at the Universidade do Minho in 2017 may apply based on exam results in: (4) Economics; or (17) Applied Mathematics – Social Sciences; or (17) Applied Mathematics – Social Sciences and (18) Portuguese.154 The MCTES establishes a minimum grade for each exam that students must meet or exceed for admission to all higher education institutions (OECD, 2006). No programme can admit students through the RGA whose results relevant to their programme fall below this grade, even if they have excess capacity. This minimum application grade requirement is the Portuguese government’s most significant tool to directly influence overall access and standards in higher education aside from Numerus Clausus. In some cases, MCTES will also identify the exams required for admission to specific programmes, for example applicants for most engineering programmes must now pass the entrance exams in both Mathematics and Physics/Chemistry. RGA access to public higher education operates principally through the National Access Competition (Concurso Nacional de Acesso, CNA). Students apply for the CAN through a central admissions portal administered by MCTES, selecting up to six different study programmes by order of preference.155 They are subsequently allocated to their programme based only on an application grade derived from their grades in the national examinations and in upper secondary education, which the institutions weight based on their overall priorities and the demands of the specific study programme. Secondary grades must account for at least 50% of the overall application weight, and secondary final exams at least 35% of the assessment, although final

153 DGES, “Provas de Ingresso - Alterações Para 2017, 2018, 2019 - Público/Privado” (Lisboa: Direção-Geral do Ensino Superior, 2016), http://www.dges.gov.pt/sites/default/files/guiapi_altera171819.pdf. 154 Ibid. 155 Fonseca et al., “Waves of (Dis)Satisfaction: Effects of the Numerus Clausus System in Portugal.”


63

exams are responsible for 30% of students’ final secondary grades.156 The overall grade of the last student admitted to each study programme is published. Placement occurs over the course of three phases, with students with different options rankings often placed in the same round. In 2014-15, 55% of students placed through the CNA entered their first-choice programme and 21% their second-choice programme.157 CH and CT graduates took up 90% of CNA spaces in 2014-15, whereas vocational students occupied little more than 4% of CNA spaces.158 Fully 18.5% of spaces in the first round of the are reserved in public institutions for special quotas, as indicated in Table 3.3.159 In recent years, the number of students admitted based on these quotas has averaged 1 168.160 161 Table 3.3: Reserved quotas for the first round of the National Access Competition

Quota Group First-Round Places

Average 2011-15 Admissions

Residents of the Azores Autonomous Region 3.5% 339 Residents of the Madeira Autonomous Region 3.5% 536

Émigré Portuguese citizens and cohabiting family-members 7% 128

Armed forces members under contract with at least two years of service

2.5% 43

Students with physical or sensorial disabilities 2% 122 TOTAL 18.5% 1 168

There are also local competitions (Concursos Locais) for study programmes, where authorised by MCTES and supported by the higher education institution and the National Commission for Access to Higher Education (Comissão Nacional de Acesso ao Ensino Superior, CNAES). In general, these competitions are associated with performance arts (i.e. music, theatre and dance). They still fall under the RGA and rely on scores in the national admissions exams, but institutions have more descretion to allow for specialised examinations in relevant vocational skills.162 Private institutions admit RGA students through institutional competitions. Each institution operates its own portal based on their own criteria, provided they respect the basic MCTES requirements (i.e. minimum grades, high school diploma).

156 Guerreiro et al., “Relatório Sobre a Avaliação Do Acesso Ao Ensino Superior (Diagnóstico e Questões Para Debate).” 157 DGEEC, “Dados e Estatísticas de Cursos Superiores.” 158 Guerreiro et al., “Relatório Sobre a Avaliação Do Acesso Ao Ensino Superior (Diagnóstico e Questões Para Debate).” 159 Ibid. 160 Ibid. 161 DGES, “Contingentes | DGES,” DGES - Direção-Geral Do Ensino Superior, March 27, 2017, /pt/pagina/contingentes. 162 Guerreiro et al., “Relatório Sobre a Avaliação Do Acesso Ao Ensino Superior (Diagnóstico e Questões Para Debate).”


64

2.4.2 Special Admissions Competitions Portugal also operates seven different special admissions competitions for other applicants to Licenciatura and Integrated-Masters programmes in private and public higher education institutions.163,164 Higher education institutions prepare and administer these admissions processes pursuant to national legislation (establishing clear competition requirements, timelines and processes for attributing students to study programmes). They are allowed to use secondary final exams to assess admissions through these pathways provided that the exams fulfill the objectives of the particular pathway. The most significant of these competitions is for candidates over 23 years of age who do not have a secondary education diploma, called the M23 stream. Institutions are required to administer exams to these students to determine their capacity to succeed in higher education, placing value on professional experience and students’ other prior learning. At least 5% of spaces in the RGA must be available to applicants using this admissions stream. In 2005, the Government decentralised admissions decisions to institutions (previously the admissions exams were administered centrally by the government) and shifted the minimum eligibility age from 25 to 23. This contributed to significant increases in the enrolment of mature students from 550 in 2004-05 to 10 900 in 2006-07, before falling back to 5 900 in 2012-13.165 The other special admissions competitions for Licenciatura and Integrated-Masters programmes are for graduates of CET programmes, TESP programmes, and other higher education programmes including one stream for admissions into medicine programmes, as well as international students. CET and TESP graduates must complete the national admissions exams and achieve the minimum scores for the RGA to enter university studies (but not polytechnic studies), and institutions may only admit students to programmes with spaces open for the RGA. Admissions for TESP, Masters programmes and Doctoral programmes also work through special admissions competitions. Prospective TESP students can be eligible for admission if they completed the tenth and eleventh grades in secondary education but did not complete secondary, dependent on other qualifications, and each institution is required to publish further admissions conditions. Applicants to Masters programmes (not Integrated-Masters programmes) must have completed a Licenciatura degree, while doctoral candidates must have master’s degree or equivalent, or a Licenciatura degree with salient scholastic or scientific achievements that demonstrate ability to complete tertiary cycle studies at the particular institution.

163 These programmes exclude military and police affiliated institutions, as well as the Universidade Aberta (Open University) in the case of international students. 164 DGES, “Acesso Ao Ensino Superior.” 165 File et al., “Policy Challenges for the Portuguese Polytechnic Sector.”


65

Student transfers between institution and study programme pairings operate basically the same as special admissions competitions. Students cannot transfer in their first year of enrolment.

2.4.3 Special Admissions Stream Finally, there is an additional special admissions stream (Regime Especial) for admission to Licenciatura and Integrated-Masters programmes. This stream is distinct from a special admissions competition because it is administered by the MCTES. This special admissions stream includes seven different programmes to support diplomats (foreign and Portuguese), other civil servants and armed forces members, citizens of former colonies, and high-performance athletes. Special admissions stream students cannot take up more than 10% of general admissions spaces.

2.4.4 Admissions by Admission Pathway Figure 3.3 presents the 2014-15 entering cohort into higher education based on the admissions pathways that they used.166 Overall, the CNA is the predominant admissions pathway, followed by institutional admissions competitions and then programme changes.

Figure 3.3: Newly admitted students by admissions stream and institution type, 2014-15

166 Guerreiro et al., “Relatório Sobre a Avaliação Do Acesso Ao Ensino Superior (Diagnóstico e Questões Para Debate).”

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

University Polytechnic University Polytechnic

Public Private Total

Other admissions

CET Graduates

Graduates of other higher educationprogrammes

Programme Change

Admissions over age 23

Institutional Competition

Local Competition

National Access Competition


66

These data highlight a general pattern where students use a greater diversity of admissions pathways into polytechnics than universities, but also into private institutions relative to publics. The CNA is predominant because it accounts for over 80% of public university enrolments, as well as almost two-thirds of public polytechnic enrolments. Institutional competitions are similarly important for private universities, but less so for polytechnics.

2.5 Numerus Clausus A key factor in admissions to higher education in Portugal is the Numerus Clausus (NC) policy framework. Under this policy, MCTES determines the number of places public and private higher education institutions may offer in each of their different study programmes, though institutions may determine the number of spaces for international students on their own. The general application of NC in Portugal is somewhat unique among higher education systems. Usually these kinds of restrictions are applied only to high demand programmes like Medicine.167 Portugal introduced NC in 1977 shortly after the Revolution to control the increasing demand for access to the higher education system, given resource constraints and fears of deterioration in quality. The role of the policy has since evolved however, notably because supply of higher education has come to much more closely approximate supply, or even outstrip it.168 Figure 3.2 indicates applications through the CNA relative to seats from 2005 to 2016.169,170 The figure indicates that supply of seats exceeded demand for three years over this eleven-year period, by as much as 2.2%. Under-capacity was highest in 2007-2008 however, well in excess of 20%.

Figure 3.2: National Access Competition applications and seats, 1999-2016

167 Fonseca et al., “Waves of (Dis)Satisfaction: Effects of the Numerus Clausus System in Portugal.” 168 Pedro Teixeira et al., “Competition and Diversity in Higher Education: An Empirical Approach to Specialization Patterns of Portuguese Institutions,” Higher Education 63 (2012): 337–52. 169 DGES, “Concurso Nacional de Acesso 2016 Em Números,” Concurso Nacional de Acesso 2016 Em Números, 2016, http://www.dges.mctes.pt/estatisticasacesso/2016/. 170 Ibid.

0

10000

20000

30000

40000

50000

60000

70000

1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016

Seats Applications 2005-2016 Source 1 Applications 2000-2009 Source 2


67

The total number of seats and applications does not reflect how they are distributed however. Even if the number of applications in a given year exceeds the total number of seats, there will likely be empty seats because students do not apply for the specific programme. Occupation rate figures capture the proportion of seats taken by students, which can vary considerably between different institutions and study programmes although generally no institutions manage to achieve full occupancy given the complexities of filling each program.171,172 Occupation rates are generally lower for polytechnic institutes than for universities, and have been consistently lower for institutions in rural areas.173 This relates to the student preferences noted above, notably based on institutional prestige and location, as well as particular study programs. Most applications for urban institutions come from within the local district, which is not the case for rural institutions. There have been some adjustments in institutional allocations of spaces, with declines in spaces at four rural public universities between 2000 and 2009 averaging 5.7%, and five rural public IPs averaging 8.6%.174 Still occupancy levels fell over the period at all IPs outside Porto and Lisbon (except for IP do Viana do Castelo), while rising at all universities except the Universidade de Lisboa and the Universidade Técnica de Lisboa. Nevertheless, given the important changes in demand for higher education relative to supply, NC is now principally a control on the distribution of students between study programmes and institutions. It limits the supply of high demand programmes and institutions, and directs students into lower demand programmes and institutions, especially through the CNA.

3. How Could Higher Education Admissions Contribute to Study Success in Portugal? There are a number of areas of concern where Portugal’s admissions system may contribute towards reducing study success. We focus on high dropout among students using alternative admissions processes, matching under the CNA, NC restrictions on enrolment, and the lack of information available on the student experience.

171 Miguel Portela et al., “Evaluating Student Allocation in the Portuguese Public Higher Education System,” Higher Education 56 (2008): 185–203. 172 Direcção-Geral do Ensino Superior, “Acesso Ao Ensino Superior - Dez Anos de Concurso Nacional - 2000-2009” (Ministério da Ciência, Tecnologia e Ensino Superior, 2010). 173 Ibid. 174 Ibid.


68

3.1 Alternative Access Streams Lead to High Dropout Dropout rates for students in the other application streams than the RGA are much higher, especially for students using the M23 stream and students who already have higher education credentials. Figure 3.4 shows how more students in absolute numbers drop out from these access schemes after year one than from the RGA, even though far more students are admitted through the RGA.175

Figure 3.4: Enrolment and number of dropouts after first year in public higher education by student admissions pathway, 2012

We should expect the CNA admissions channel, the largest share of RGA students, to have the highest rates of study success since it serves the most traditional students. It is difficult to know how much this pattern in study success relates to admissions processes themselves, or other aspects of educating non-traditional students. We will further discuss concerns respecting education of these students in the next chapter. However, the literature does provide some limited information on institutional admissions processes that indicate areas of concern. Alternative admissions streams may be setting students up to fail. Multiple leaders from the Portuguese higher education system during the 1999-2009 period indicated in interviews with Teixeira and Koryakina that institutions apply lower admissions standards to attract students over the age of 23.176 This was noted especially in terms of institutions that were less competitive for traditional students because of relatively weak reputations. We heard this same observation repeatedly during interviews. Some interviewees noted that at one point the original M23 admissions exams were so easy that they created expectations among students that their courses would be easier than they in fact were, leading to high failure rates. It was suggested that exams have become somewhat more difficult since. Another concern resulting from this policy was that older students admitted based on lax criteria would mix with students admitted through the more rigorous established criteria and weaken the student body and learning outcomes.

175 DGEEC, “Ensino Superior: Situação Em 2012-13 Dos Inscritos Pela Primeira Vez Em 2011-12” (Lisbon: Direção-Geral de Estatísticas da Educação e Ciência, 2013). 176 Pedro Teixeira and Tatyana Koryakina, “Political Instability, Austerity and Wishful Thinking - Analysing Stakeholders Perceptions of HE’s Funding Reforms in Portugal,” European Journal of Education 51, no. 1 (2016): 126–39.

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Enrolment

Dropouts

RGA M23 Programme Change Higher education graduates Transfers CET Graduates Other


69

On a related note, an institutional study at the Universidade de Évora found significantly higher dropout rates among students who began their programmes in less conventional admissions phases (phases three or four, as opposed to the more standard entry times in phases one and two) (among Lienciatura students p<0.001). Research in Holland found that students who applied late generally felt less connected in their study programmes and less confident of graduating.177 Ultimately it is easier to assess the quality of the CNA admissions process than the diversity of institution-level processes. Concerns with the CNA further suggest institutional processes may have difficulty ensuring effective matching if they adopt a similar conceptual framework

3.2 The National Access Competition Promotes Shallow Matching The CNA arguably is less about determining students’ suitability than creating a fair and credible “competition”. The basic premise is that spaces in higher education should be limited and highly competitive, reflecting more an “elite” higher education model than massification. From this perspective, the CNA has achieved important successes. In particular, it is broadly trusted in society because it is perceived as fair whereas there is a perception that institutionally managed admissions would be more inequitable (although this is debated). One interviewee argued that such trust can be very difficult to achieve in Portugal. This common understanding and confidence in the CNA also provides a way for society to assess the quality of institutions and programmes, based on the entering CNA scores of their students. The CNA performs this function likely at a relatively low cost. Student applications can be assessed using an algorithm alone, which does not require the creation of large admissions departments at every higher education institution to make subjective determinations. Yet, there are important reasons to question the fairness of the CNA in practice. One key challenge is bias in secondary grades.178 Nata et al. found through a review of three million high school graduate cases that Portuguese independent private schools inflate students’ school grades in years that contribute towards competition for spaces in higher education.179 Guerreiro et al. also report observable patterns across different regions.180 The admissions system does not currently permit adjustments of admissions scores based on these patterns. The grade inflation is significant enough to make the difference for admissions in many study programmes. One interviewee also reported that their institution finds graduates of private

177 Vossensteyn et al., “Dropout and Completion in Higher Education in Europe.” 178 Guerreiro et al., “Relatório Sobre a Avaliação Do Acesso Ao Ensino Superior (Diagnóstico e Questões Para Debate).” 179 Gil Nata, Maria João Pereira, and Tiago Neves, “Unfairness in Access to Higher Education: A 11 Year Comparison of Grade Inflation by Private and Public Secondary Schools in Portugal,” Higher Education 68 (2014): 851–74. 180 Guerreiro et al., “Relatório Sobre a Avaliação Do Acesso Ao Ensino Superior (Diagnóstico e Questões Para Debate).”


70

schools do poorly relative to what would be predicted based on their admissions scores. The exams are also viewed as disadvantaging students notably from the vocational secondary stream, although again these students may perform worse for other reasons as well.181 The reliability of the final secondary exams is also questionable. Median results have varied wildly from year to year, for example ranging in Mathematics A from a high of 14.6/20 to a low of 8.6/20 between 2007 and 2015, while grades provided by secondary schools were relatively consistent between 13 and 14.182 This speaks to great difficulties in establishing consistent exams from year to year. Most students’ placement is based on their relative performance to others, which will adjust for the difficulty of the year’s exams, however the minimum passing grade required to access higher education is not adjusted, so the difficulty of each year’s exam can be reflected in the skills of the students at the margin of admissibility (i.e. with marks just in excess of the 9.5 minimum passing grade). One interviewee indicated this may be a significant factor in variability in dropout from year to year, as these marginal students have quite elevated dropout rates. Even if exam and secondary school class grades were clearly reliable, it is very unclear that admissions based purely on marks are fair. Students who access better schools or better teachers, who have more intellectually engaging or supportive families, or whose families have financial resources are advantaged to achieve higher grades.183 A broader question is whether the CNA structure for applications and admissions to higher education is effective in supporting matching students to a higher education programme that best fits their interests, and of matching institutions with students who best match their programmes’ academic and other objectives. Again, matching appears to be an incidental or secondary objective of the CNA model that prioritises fair competition. The CNA’s rigidity does not permit institutions to appreciate students’ suitability for study programmes based on more subjective measures than simply their marks in national entrance exams and secondary school. This may be of special concern given the exam applies across all higher education programmes, from business to medicine. Other skills than simply academic performance as reflected in marks may be more essential, not only for success while studying, but in the professions those programmes lead to. Interviewees from one institution also reflected that their university has no say in the content of the exams, which are more suitable in measuring performance at the secondary level than assessing suitability for tertiary education. The model places an emphasis on entry grades alone for students evaluating institutions too, leading them to discount other elements of institutional performance. This connects with the absence of information on the student experience.

181 DGEEC, “Transição Entre o Secundário e o Superior: Parte I” (Lisboa: Direção-Geral de Estatísticas da Educação e Ciência, 2016). 182 Guerreiro et al., “Relatório Sobre a Avaliação Do Acesso Ao Ensino Superior (Diagnóstico e Questões Para Debate).” 183 Paulo Santiago et al., “Tertiary Education for the Knowledge Society - Volume 2 - Special Features: Equity, Innovation, Labour Market, Internationalisation” (Paris: OECD Publishing, 2008).


71

The CNA forces students to make choices about their study programmes and institutions prior to application, whereas other systems allow students to gain admission to multiple institutions and study programmes and then determine what option is the best fit for them, often through direct engagement with the institution itself. Students conceivably may not research their study options as fully in advance of their decision on an application, because they are not certain of admission to the institutions they apply to, whereas once offered admissions they may become more invested in exploring the institution and study programme to better understand if it is a good fit. Interviewees were mixed as to whether this was important or not. Ensuring fairness in admissions is critically important.184 However, matching is also important for preventing dropout. The CNA could better address both of these categories of concerns.

3.3 Numerus Clausus Restricts Access to Sought-After Programmes The NC system has been attributed an important role to low student motivation in Portugal by some researchers. It can limit students’ options so they have to choose programs they would otherwise prefer not to attend.185 The NC system is notably important in its relationship with the CNA, which is the primary access portal for Portugal’s higher education system. Fonseca et al. refer to a “wave of dissatisfaction” through the CNA, as students are shifted down from their first through sixth choices.186 The evidence suggests that NC is not a significant restriction on students’ placement. Figure 3.5 shows the proportion of students using the RGA placed in Round 1 or Round 2 of the CNA, or through a later round or other RGA stream in 2014-15.187 Most RGA students entered their first-choice programmes, and more than three quarters entered their first or second choice.

Figure 3.5: RGA admissions by option or other pathway, 2014-15

184 Ibid. 185 Fonseca et al., “Waves of (Dis)Satisfaction: Effects of the Numerus Clausus System in Portugal.” 186 Ibid. 187 DGEEC, “Dados e Estatísticas de Cursos Superiores.”

55%

21%

24%

First Option

Second Option

Other Option or RGA Modality


72

However, there are programmes where more than 70% of the students enrolled had another preference for their first option.188 Fonseca et al. focus in their study on unsuccessful prospective medical students, who shift in many cases into pharmacy programmes, displacing additional students further down the line. Surely many students consider the competitiveness of programmes when they apply, which is a function of NC, and may not apply for their genuine first preference. Some also pursue private higher education so the implications of NC and the CNA extend beyond the public system.189 Fonseca et al. do not document any causal impacts of lower placement on completion rates. Other sources provide some correlational evidence however. Portuguese national data shows a decline in the proportion of students who remain in the same study programme at the same institution, and the proportion who stay in the same institution, by their placement rank (i.e. first through sixth preference in the CNA), but students’ likelihood of dropping out by the beginning of year two does not increase with students’ placement (i.e. in their first through sixth choice). These data suggest low placement affects study success in their particular programme of study, but not in terms of their overall commitment to higher education. Research on students’ choices of institutions described earlier also highlighted the significant share of students studying at the only institution they could attend, as opposed to necessarily the institution they would prefer. Often students want to locate close to home and a host of studies have found that distance from home is a factor in dropout.190 For example, approximately 69% of Universidade de Coimbra students in one study were from outside the community.191 Fully 53.3% of surveyed students who transferred from the IP de Setúbal in 2011-13 reported that distance from home to school was their primary reason for transferring, followed by disinterest in their particular programme (20%).192 Distance from home could relate to sense of belonging, but also financial and time pressures. Students also show a strong preference for placement in urban institutions, as indicated in occupancy rates.193 This preference is even very strong among students from rural areas. At the institution level, placement preference was identified as a highly statistically significant factor in determining dropout and also timing of dropout at the Universidade de Évora (among Licenciatura students p<0.006).194 Similar results were found at the Universidade do Porto and

188 Fonseca et al., “Waves of (Dis)Satisfaction: Effects of the Numerus Clausus System in Portugal.” 189 Tavares, “Routes towards Portuguese Higher Education: Students’ Preferred or Feasible Choices?” 190 IP de Setúbal, “O Abandono Nos Cursos de Primeiro Ciclo” (Setúbal: Instituto Politécnico de Setúbal, 2013). 191 Madalena Alarcão, “Insucesso Académico e Abandono Escolar” (Seminário Sucesso Académico, Lisbon, May 12, 2015). 192 Ibid. 193 Direcção-Geral do Ensino Superior, “Acesso Ao Ensino Superior - Dez Anos de Concurso Nacional - 2000-2009.” 194 Rosalina Pisco Costa et al., “Identificação Das Causas Do Abandono Escolar Na Universidade de Évora” (Évora: Universidade de Évora, 2015).


73

the Universidade de Lisboa, although in the former case this factor did not significantly affect students’ completion of credits.195,196 Two studies at the Universidade do Minho also found student placement in a lower-choice study programme, or away from their hometown, was associated with dropout notably by aggravating first year students’ difficulties in transitioning to higher education and in reconciling prior expectations with the reality of their study programme and life in higher education.197 Students using the CNA are the most likely to change programme and/or institution relative to the other major admissions streams.198 Almeida’s study (2007) unsurprisingly found disappointment in study programmes to be the top motivator specifically for changes in study programmes.199 Students may switch programmes because they discover that they are a poor fit after studying for a time, but NC and the CNA have also promoting manipulation for admissions purposes. In particular, students can try to gain admissions into less competitive programmes and then transfer into more competitive programmes internally. The Government of Portugal has sought to limit this by restricting student transfers to only programmes for which they have the required entrance grades, but this limits flexibility for students and does not allow for the possibility that students’ performance could improve while in higher education. NC allocations of course still restrict spaces for transfer students. Logically, students who are placed in lower choices may have more limited academic preparation and cognitive skills given they were unsuccessful in access their preferred programmes, which may also affect their completion rate. On the other hand, better students may be reaching for the most competitive programmes, and weaker students applying to less competitive programmes. It is therefore uncertain how best to interpret the relationship between them receiving a lower choice placement and their study success without better data. One IP undertakes extensive efforts to help students overcome problems of NC limitations on programme seats. It has permitted students who did not gain admissions to their study programme of choice to study at a non-subsidized rate and then try to upgrade their exam marks to gain access. The institution has also been very committed to supporting student transfers between programmes, for instance allowing students to pursue compulsory specific courses for programmes they would like to transfer into. The NC model also assumes that the Government of Portugal overall has better information as to what studies students ought to pursue than students themselves. Three key factors can inform the allocation of students to academic programmes: personal intellectual or vocational interest, institutional cost management, and labor market demand. Students clearly are best positioned

195 IP de Setúbal, “O Abandono Nos Cursos de Primeiro Ciclo.” 196 José António Sarsfield Cabral, “Algumas Notas Sobre o Acesso e Desempenho Dos Estudantes Admitidos Na U.Porto Em 2008/09, 2009/10 e 2010/11 (Pelo Regime Geral de Acesso Em 1s Ciclos e Mestrados Integrados)” (Seminário para o “Sucesso Académico,” Lisbon, May 12, 2015). 197 IP de Setúbal, “O Abandono Nos Cursos de Primeiro Ciclo.” 198 DGEEC, “Ensino Superior: Situação Em 2012-13 Dos Inscritos Pela Primeira Vez Em 2011-12.” 199 IP de Setúbal, “O Abandono Nos Cursos de Primeiro Ciclo.”


74

to determine which study programmes most interest them personally, but institutions and government should determine whether they have adequate preparation for those programmes. As noted, these determinations on the part of students and institutions are very important factors in study success, and an area of focus in this report. Governments and institutions, on the other hand, are responsible for management and clearly best positioned to determine how to allocate student seats to manage education costs based on their full scope of priorities. Meeting labour market needs may be the key area for debate. Government regulation of spaces may be best justified for programmes where students train for a particular profession, especially when that profession is largely within the public sector (e.g. medicine). In many other areas though, insufficient information may exist to inform planning, or no serious planning efforts are taking place: the OECD previously observed that NC was not formally supported by a national higher education planning framework.200 Many students may also choose poorly, but they would be expected to respond to higher earnings where workers are scarce. Currently some study programs have vacant places even though unemployment rates for their graduates are in excess of 20% (compared to overall graduate unemployment of 8%).201 Transparency may be an issue: a more recent OECD study found the large numbers of study programs particularly at the IP level make it more difficult for students to make informed choices however.202 IPs currently offer 711 degrees, with a single institution offering 506. Also, labor markets may in fact be shaped by the supply of graduates themselves. American research indicates that even marginally qualified students do best when permitted to enter top quality institutions, so restricting access to these institutions reduces completion.203 Restricting access is precisely the intent of the NC, in the interest of supporting less competitive institutions that are often disadvantaged rural institutions especially in disadvantaged regions.204,205 However, many researchers also find that this approach has not fundamentally shifted the distribution of higher education spaces in Portugal, where the system continues to be concentrated largely in urban areas, partly because private offerings remain concentrated in urban areas and there remains considerable capacity for students who do not wish to leave.206,207 Whether coercive restrictions on enrolment in institutions and study programmes are the best way to support competitively disadvantaged institutions is questionable. The next two chapters further consider how some institutions are competitively disadvantaged.

200 OECD, “Reviews of National Policies for Education: Tertiary Education in Portugal” (Paris: OECD Publishing, 2006). 201 OECD, “OECD Economic Surveys: Portugal 2017.” 202 Ibid. 203 Joshua Goodman, Michael Hurwitz, and Jonathan Smith, “Access to 4-Year Public Colleges and Degree Completion,” Journal of Labor Economics 35, no. 3 (2017): 829–67. 204 Portela et al., “Evaluating Student Allocation in the Portuguese Public Higher Education System.” 205 Ibid. 206 Fonseca and Encarnação, “A Massificação Do Ensino Superior Em Portugal: Efeitos Espaciais Na Diversificação Do Sistema.” 207 Sá et al., “Higher Education (Related) Choices in Portugal: Joint Decisions on Institution Type and Leaving Home.”


75

The extent to which NC especially restricts students’ ability to choose programmes depends on how rigidly it is applied. Institutions and government can adapt programme seat allocations to meet student demand in the medium to long-term. Certainly, NC restricts institutions’ short-term flexibility, because they have to make choices about programmes’ maximum enrolment before receiving applications, not in response to applications. Of course, many of these same challenges could also emerge in higher education systems without NC and Portugal’s particular admissions streams, where enrolment in particular programmes is limited and/or admissions are competitive.

3.4 Insufficient Information on the Student Experience There is no question that information matters in helping students develop accurate expectations. In addition to academic preparation and difficulties, Almeida (2007) for example found that uncertain choice of study programme was a key factor in dropout.208 When it comes to operating online applications to higher education, Portugal appears to be doing a lot right.209 The use of a single unified portal allowing students to compare and apply easily to different programme options is something many different higher education systems are seeking to do.210 Moreover, the application process is highly transparent for students to understand how competitive programmes are (simplified by the CNA process), and to obtain useful information as to how students in the programmes have done. Some minor improvements could be made certainly to facilitate students making comparisons, notably through functionality allowing students to more easily juxtapose results between two or more programmes without needing special facility with excel. One interviewee also noted that unemployment figures for programme graduates are based on registration for unemployment benefits and more reliable figures could be obtained in other ways. Nevertheless, students of course can and should make choices on higher education options based on broader elements than have been captured, including much more subjective aspects of life on campus or in the surrounding community. Information provided can also affect institutional priorities and decision-making, so this system provides relatively limited incentives for institutions to strengthen these elements that are excluded.

208 IP de Setúbal, “O Abandono Nos Cursos de Primeiro Ciclo.” 209 DGES, “Acesso Ao Ensino Superior.” 210 Vossensteyn et al., “Dropout and Completion in Higher Education in Europe.”


76

4. International Experiences The 2015 Vossensteyn et al. review of study success policies across Europe found many countries were seeking to strengthen admissions processes or improve information availability.211 The Netherlands has gone perhaps the furthest in developing its admissions processes to promote study success. Higher education institutions are required to offer procedures to help admitted students evaluate their study choice. The institutions have discretion over the particular steps they would like to take, but instruments can include intake interviews, one-day student experiences, and additional aptitude tests. Austria, the Netherlands and Sweden grant their institutions more flexibility to select students based on motivation and personality traits, and not only test results, with a view to improving completion rates. Shifts to this more broad-based admissions of this kind has been documented as having downsides though, notably in North America, introducing bias based on cultural or social capital and ethnicity, although many North American institutions have made sophisticated efforts to address these problems.212 Vossensteyn et al. found that providing improved information to prospective students was the most commonly adopted national policy to promote study success, attributed to seventeen different jurisdictions. Most jurisdictions emphasise websites, as is the case of Portugal, however France and Denmark also developed secondary school curricula directed towards informing students’ higher education study choices. France’s Bac-3/Bac+3 programme initiated education and career pathways counselling three years before completion of upper secondary. Other countries developed regional centres to provide counselling on education and training pathways. France views its central admissions portal as another tool to promote study success.213 A priority for France has been to strengthen the site to provide even more useful information and guidance to students. Other jurisdictions operate such portals, but view them as less related to success. Some countries further developed information sources to facilitate comparison between institutions. This included league tables in the UK, which one evaluation found improved retention rates.214 In many cases this information is provided less to inform students, however, than to motivate institutions to pursue improved policies and practices.

211 Ibid. 212 Alex Usher, “Admissions Policies: Marks-Based, Broad-Based, or Random? | HESA,” One Thought to Start Your Day, January 11, 2017, http://higheredstrategy.com/admissions-policies-marks-based-broad-based-or-random/. 213 Vossensteyn et al., “Dropout and Completion in Higher Education in Europe.” 214 Bernard Longden, “‘Bearing Down’ on Student Non-Completion: Implications and Consequences for English Higher Education,” Journal of College Student Retention 14, no. 1 (2013 2012): 117–47.


77

Chapter 4: Quality of Education Delivery This chapter analyses how the quality of education delivery affects study success based on the following criteria: institutional commitment; tracking and monitoring of students; learning, teaching and assessment; flexibility; and social integration and support services. Overall, we find that the Portuguese higher education system is characterised by uneven institutional commitment in terms of strategic prioritisation and service offerings. Also evident is a widespread failure to identify and provide targeted support to students at risk of dropping out. The system has a host of more strictly academic challenges. There is insufficient focus on teaching and serving students, for example in terms of strengthening pedagogy and being responsive to student views. Rates of academic failure are high, while many students who fail reenroll but actually do not attend classes or assessments. Rigid programme structures result from extensive required courses starting in first year and very few optional classes, affecting student satisfaction and flexibility. Students appear to have excessive class time, contributing to difficulties balancing schedules and responsibilities. The polytechnic role is undermined by the unclear binary distinction, which is most apparent in the requirement that IP faculty have doctoral degrees. Finally, the system offers inadequate remedial support for students with weaker academic preparation and is poorly adapted to the needs of mature students. On the non-academic side, the system has important weaknesses in first year student integration, but also suffers from limited student engagement more broadly. As well, the quality of student services is uncertain at best, and it is apparent specifically that supports for students with disabilities are underdeveloped compared with other jurisdictions. International jurisdictions have sought to strengthen the delivery of education to promote study success through policies we classify into three themes. The first is reshaping programme options through new degree structures, greater flexibility through credit transfer and recognition, or tighter restrictions on maximum enrolment periods. The second is improving academic and social supports, often focused especially on supporting students’ transitions into higher education. The last theme is improving the collection and use of information in terms of tracking students’ progression, integrating study success metrics in quality assurance processes, and building structures to share best practices.


78

Chapter Guide

1. Analytical Framework ...................................................................................................... 80

2. Lay of the Land ................................................................................................................ 81 2.1 Teaching Faculty ..................................................................................................................... 81 2.2 Quality Assurance................................................................................................................... 82 2.3 Student Services and Engagement .......................................................................................... 83 2.4 Supports for Students with Disabilities .................................................................................... 86

3. How May the Quality of Higher Education Delivery Contribute to Low Completion? ......... 87 3.1 Student (Dis)Satisfaction ........................................................................................................ 87 3.2 Academic Factors ................................................................................................................... 90

3.2.1 Insufficient Focus on Teaching and Serving Students................................................................ 91 3.2.2 High Rates of Academic Failure ................................................................................................. 95 3.2.3 Rigid Program Structures ......................................................................................................... 100 3.2.3 Excessive Time Demands on Students ..................................................................................... 102 3.2.5 The Unclear Binary Distinction................................................................................................. 105 3.2.6 Inadequate Remedial Support for Students with Weaker Academic Preparation.................. 107 3.2.7 Poor Adaptation to the Academic Needs of Mature Students ................................................ 108

3.3 Non-Academic Factors .......................................................................................................... 109 3.3.1 Underdeveloped First Year Student Integration...................................................................... 110 3.3.2 Limited Student Engagement................................................................................................... 112 3.3.3 The Quality of Student Services is Uncertain at Best .............................................................. 115 3.3.4 Supports for Students with Disabilities are Underdeveloped ................................................. 116

3.4 Failure to Assist Students at Risk of Dropping Out ................................................................. 117 3.5 Uneven Institutional Commitment to Study Success .............................................................. 118

4. International Experiences .............................................................................................. 119 4.1 Restructuring Programme Options ........................................................................................ 120 4.2 Improving Academic and Social Supports .............................................................................. 120 4.3 Improving Information Collection and Dissemination ............................................................ 121

List of Tables Table 4.1: Teaching faculty ranks in public higher education institutions in portugal.............. 81 Table 4.2: Statistically significant differences in satisfaction by student group........................ 89 Table 4.3: Student evaluations of importance of and satisfaction with Academic Elements and Academic Supports (scale of 1-10) .............................................................................................. 90 Table 4.4: Average assessment of the importance and implementation of the European Standards and Guidelines by Portuguese university faculty (scale of 1-7) ................................ 92 Table 4.5: Study progress limits for academic suspension required under the higher education financing law ................................................................................................................................ 99 Table 4.6: Reported weekly hours of activity and rank among 22 European countries, 2011 102 Table 4.7: Reported weekly hours of activity among 22 European countries by student age for students not living with parents, 2011 ...................................................................................... 103


79

Table 4.8: Reported weekly hours of activity for Licenciatura students among 22 European countries by study programme type ......................................................................................... 103 Table 4.9: Satisfaction with non-academic processes and services (scale of 1-10) ................. 109

List of Figures Figure 4.1: Proportions of university academic staff expected to retire (base year = 2015) .... 82 Figure 4.2: Proportions of public institutions providing relevant information on student services on their website ............................................................................................................. 84 Figure 4.3: Proportions of public institutions providing relevant information on student engagement activities on their website ...................................................................................... 85 Figure 4.4: The proportion of students who indicate the fulfillment of goals through their studies is (very) high. ................................................................................................................... 88 Figure 4.5: Satisfaction with areas of learning (scale from 1-10) ............................................... 89 Figure 4.6: Proportions of public institutions providing relevant information on pedagogy-related activities on their website ............................................................................................... 94 Figure 4.7: Percentages of mature students having failed at least one class by students’ year of study, in Santos et al. 2016 ...................................................................................................... 96 Figure 4.8: Total failure rate across courses with 10 or more evaluated students at the ISEG 96 Figure 4.9: Rate of non-participation in assessment across courses with 10 or more evaluated students at the ISEG, based on enrolment in the course for the first time or the second or greater time .................................................................................................................................. 97 Figure 4.10: Rate of failure among evaluated students across courses with 10 or more evaluated students at the ISEG, based on enrolment in the course for the first time or the second or greater time ................................................................................................................. 98 Figure 4.11: Dropout after first-year among first-time students at public higher education institutions, 2012 ........................................................................................................................ 107 Figure 4.12: Proportions of public institutions providing relevant information on orientation activities for new students on their website ............................................................................ 111 Figure 4.13: Proportions of public institutions providing relevant information on strategic activities relating to study success on their website ................................................................ 118


80

1. Analytical Framework Work from the US has indicated empirically that when holding other characteristics constant students’ study success rates can fall when they attend weaker institutions.215 There is no discernible reason to believe that attending a weaker institution within Portugal would not reduce study success too. Therefore, there is every reason to believe the quality of education delivery at Portuguese institutions matters in study success. This chapter is informed by the following five key elements of higher education delivery that have been identified in the literature as strong factors in study success:216

1. Institutional commitment: Commitment in this case is about prioritizing students’ sense of belonging and engagement in their higher education institution throughout the length of their studies. It also relates to the monitoring of students’ progress and overall taking a more holistic approach to study success that is student centred. Finally, this is associated with continuous improvement in teaching. The other key elements all relate ultimately to institutional commitment.

2. Tracking and Monitoring of Students: Monitoring is crucial to permit targeted

interventions with students at risk of dropping out. It can also allow the higher education system to adapt based on objective observations of policies’ effectiveness.

3. Learning, Teaching and Assessment: The quality of students’ academic experience affects

their choice of whether to continue in higher education. Pedagogy must be student-centred, which entails active learning approaches that empower students to drive their own learning. Performance in this area relates largely to the culture of academic staff.

4. Flexibility of the Higher Education System: Vossensteyen et al. define flexibility as “the

opportunity to move between programmes and institutions and to transfer credits”.217 This is important where students may begin a study programme at an institution and then realise it is not fulfilling their expectations. Flexibility then may determine whether the student drops out or continues in higher education through a programme that is a better fit. We do not look in detail at credit transfer procedures in this report, instead we focus more closely on the design of study programmes, in terms of whether they have rigid sets of required courses that are very discipline specific, or more flexibility for students to navigate a pathway that best meets their needs.

5. Social integration and support services: This also reflects an emphasis on building

students’ sense of belonging. It can include a focus on the social and academic integration

215 Sarah R. Cohodes and Joshua S. Goodman, “Merit Aid, College Quality, and College Completion: Massachusetts’ Adams Scholarship as an In-Kind Subsidy,” American Economic Journal: Applied Economics 6, no. 4 (2014): 251–85. 216 Hans Vossensteyn et al., “Dropout and Completion in Higher Education in Europe” (European Union, 2015). 217 Ibid., 18.


81

of new students, and building strong relationships among students and between students and faculty. This element also relates to the provision of different services that support students’: study skills; psychological and emotional wellbeing; financial management; academic and career planning; physical health; disability accommodation; etc.

2. Lay of the Land

2.1 Teaching Faculty Academic staff complements are strictly regulated by the Ministry of Science, Technology and Higher Education.218 However, within this complement institutions are responsible for recruiting and appointing staff autonomously.219 There are three primary ranks for instructional faculty in both universities and polytechnic institutes (See Table 4.1).220 Polytechnic institutes can also hire specialists with industrial or professional experience. Table 4.1: Teaching faculty ranks in public higher education institutions in portugal

Universities Polytechnic Institutes First/Highest Rank

Full Professor (Catedrático) Senior Coordinating Professor (Coordenador principal)

Second Rank Associate Professor (Associado) Coordinating Professor (Coordenador) Third Rank Assistant Professor (Auxiliar) Assistant Professor (Adjunto)

Other Specialist (Especialista) Universities and IPs have made considerable efforts to increase the proportion of academic staff with doctoral degrees. Proportions with PhDs rose particularly in the first decade of the century, but has since stagnated. As of 2011, roughly 70% of academic staff at public universities held doctorates, as did just less than half at private universities.221 Proportions of faculty with doctoral degrees vary significantly by discipline. Private institutions generally have fewer doctoral holders because of their greater focus on teaching relative to research, and concentration in the social sciences and humanities.

218 Jon File et al., “Policy Challenges for the Portuguese Polytechnic Sector” (Centre for Higher Education Policy Studies, April 2013). 219 Sónia Cardoso, Orlanda Tavares, and Cristina Sin, “The Quality of Teaching Staff - Higher Education Institutions’ Compliance with the European Standards and Guidelines for QA - the Case of Portugal,” Educational Assessment, Evaluation and Accountability 27 (2015): 205–22. 220 File et al., “Policy Challenges for the Portuguese Polytechnic Sector.” 221 João M. Santos, Hugo Horta, and Manuel Heitor, “Too Many PhDs? An Invalid Argument for Countries Developing Their Scientific and Academic Systems: The Case of Portugal,” Technological Forecasting and Social Change, 2015.


82

The Government of Portugal has set a requirement for members of non-specialist professor ranks at IPs to have doctoral degrees. The deadline for faculty to attain their degrees is now 2019. Just 29% of academic staff had PhDs in 2011, but a further 26% of polytechnic instructors were enrolled in doctoral programmes.222 An IP representative indicated that now more than 50% of teaching staff at most IPs have doctoral degrees. Institutional accreditation is tied to staff quality. Since 2009, higher education institutions have been required to complete appraisal of academic staff members every three years.223 Institutions must define their own related regulations that have an impact on faculty career progression and pay, in compliance with national legislation. Criteria are defined by law and, but weighted differently across the different institutions and institution types. Finally, as of 2015 many university faculty were expected to retire within the next 15 years, as indicated in Figure 4.1.224 Therefore a process of renewal of teaching faculty will need to take place, although not all faculty will be replaced given the system is shrinking.225

Figure 4.1: Proportions of university academic staff expected to retire (base year = 2015)

2.2 Quality Assurance Portugal has established a strong national quality assurance system, built around national legislation and the A3ES.226 This system is relatively new and therefore continues to develop, but has been accredited by the European Quality Assurance Register. A3ES activities are: the accreditation of new study programs, the accreditation of existing study programs in five year cycles, audits of institutional quality assurance systems in six year cycles, and research on higher education quality assurance. Institutional audits became compulsory only in 2016. Institutions are considered most responsible for their own quality, with the A3ES playing a support role. Completion and dropout rates are used as criteria for evaluations, as well as

222 File et al., “Policy Challenges for the Portuguese Polytechnic Sector.” 223 Cardoso, Tavares, and Sin, “The Quality of Teaching Staff - Higher Education Institutions’ Compliance with the European Standards and Guidelines for QA - the Case of Portugal.” 224 Santos, Horta, and Heitor, “Too Many PhDs? An Invalid Argument for Countries Developing Their Scientific and Academic Systems: The Case of Portugal.” 225 Cláudia S. Sarrico, “Re: Two Chapters of Portugal Review,” March 8, 2017. 226 Stephen Jackson et al., “Report of the Panel for the External Review of the Agency for Assessment and Accreditation of Higher Education (A3ES) Portugal” (European Quality Assurance Register, 2014).

0% 5% 10% 15% 20% 25% 30% 35% 40% 45%

Public Universities

Private Universities

Within 5-7 Years

Within 15 Years


83

institutional performance in integrating students.227 All evaluations involve foreign experts. A3ES is also considered one of the best agencies in Europe in conducting policy analysis.228 As of 2014, there were just three areas where A3ES was evaluated as only substantially complying and not fully complying with European criteria.229 The first was the use of internal quality assurance procedures, because not all institutions had fully developed and implemented their internal quality assurance mechanisms. The second element was the fitness for purpose of procedures, which was hindered by the inadequate involvement of students. Finally, the third was the inaccessibility to the public of public reports in terms of their readability in particular. Sarrico et al. indicate that the new quality assurance regime has changed many elements of institutional practice.230 One result has been the proliferation of questionnaires for students, faculty and alumni, including student evaluations of faculty. The website-based review found 85% of universities and 77% of IPs provided relevant information on services responsible for quality management using surveys.231 Many elements of study programs have become more formal also, such as exam dates, course syllabi and degree curricula, and much of this information is now available online which has improved transparency. Institutions have had to develop more advanced information systems. There is a broad sense that these changes have been helpful. The new quality assurance system has also been credited with helping to significantly reduce the number of study programs on offer in Portugal.232 The A3ES has even refused to accredit degrees at some of Portugal’s most prestigious institutions.233

2.3 Student Services and Engagement The recent website-based review of institutional activities related to study success provides the best available evidence regarding student services available at public Portuguese higher education institutions. The review identifies four student services in particular that we will

227 DGEEC - Equipa de Estudos de Educação e Ciência, “Draft: Medidas de Promoção Do Sucesso Escolar Nas Instituições de Ensino Superior Públicas Em Portugal, Tal Como São Relevadas Nos Respetivos Sítios” (Lisboa: DGEEC, Unpublished). 228 Jackson et al., “Report of the Panel for the External Review of the Agency for Assessment and Accreditation of Higher Education (A3ES) Portugal.” 229 Ibid. 230 Cláudia S. Sarrico, Amélia Veiga, and Alberto Amaral, “The Long Road—how Evolving Institutional Governance Mechanisms Are Changing the Face of Quality in Portuguese Higher Education,” Educational Assessment, Evaluation and Accountability 25, no. 4 (2013): 375–91. 231 DGEEC - Equipa de Estudos de Educação e Ciência, “Draft: Medidas de Promoção Do Sucesso Escolar Nas Instituições de Ensino Superior Públicas Em Portugal, Tal Como São Relevadas Nos Respetivos Sítios.” 232 Andrea Blattler et al., “Portuguese Higher Education: A View From the Outside” (European University Association, 2013). 233 Sarrico, Veiga, and Amaral, “The Long Road—how Evolving Institutional Governance Mechanisms Are Changing the Face of Quality in Portuguese Higher Education.”


84

consider here. The presence of these services is summarised in Figure 4.2.234 In reviewing what these services actually do, in many cases we observe significant overlaps.

Figure 4.2: Proportions of public institutions providing relevant information on student services on their website

Counselling services provide psychological appointments, psycho-pedagogical supports including for students with disabilities, social supports, academic advising, career counselling, behavioural training, and monitoring of curricular internships. Staff are generally psychologists or education specialises. For example, the Universidade de Aveiro offers a programme of counselling and advising by psychologists and student peers.235 The private university interviewee indicated that their psychology faculty provide counselling during office hours. The student ombudsperson (Provedor) is a professor appointed by the institution head (rector at universities or president at IPs), in coordination with the student association and pedagogical councils. They are an impartial, independent and confidential resource for students to discuss problems, make complaints or seek conflict mediation. The student ombudsperson is also expected to help clarify procedures and policies by providing feedback and helping update rules for discipline and student conduct each year, partly through the completion of an annual report at most institutions, which is only made publicly available in some cases. Careers centres (Unidades de Inserção na Vida Ativa - UNIVA) support workforce transitions through varying activities depending on the centre. In some cases, they help train faculty as well. Student engagement can include international or workplace connections. Institutions are in fact required legally to support students’ integration into the workplace.236 The proportions of public institutions reporting such activities on their websites are highlighted in Figure 4.3.237

234 DGEEC - Equipa de Estudos de Educação e Ciência, “Draft: Medidas de Promoção Do Sucesso Escolar Nas Instituições de Ensino Superior Públicas Em Portugal, Tal Como São Relevadas Nos Respetivos Sítios.” 235 Ibid. 236 Ibid. 237 Ibid.

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Psychological/pedagogical counselling service

Student ombudsperson

Careers Centre

IPs Universities


85

Figure 4.3: Proportions of public institutions providing relevant information on student engagement activities on their website

Institutional partnerships may be with employers, the national government, municipal governments, foundations, other educational institutions, associations, university networks, non-profit organisations, etc. They can support exchanges, commercial or educational arrangements, or student internships. Universities and polytechnics differ somewhat in terms of their relationships with external market actors. In general universities engage more with larger employers, including corporations, for research collaborations and student work placements, whereas polytechnics work more with small and medium-sized enterprises.238 Employment exchanges are digital platforms that assist employers in recruiting and assist students in finding work. One interviewee indicated their university wanted to set up such an exchange, but only had a volunteer portal organised to date. Finally, employment and employability observatories represent legally required tracking of graduates’ employment outcomes. One IP reported offering an internship option for all finishing students, and is looking to introduce a skills portfolio. In the 2011 Eurostudent survey, 7% of Portuguese students reported having enrolled in a higher education institution abroad and 2% having pursued an overseas internship or language course.239 The figures for enrolling in a foreign institution and pursuing a language were equal to the median among surveyed countries, but the internship figure fell slightly below the median. Other institutional services not discussed in the survey include athletics facilities, student associations, and student housing. Interviewees indicated that student associations must be in place at all higher education institutions by law. Some institutions also have significant programming for student athletes. Meanwhile, only a modest share of Portuguese students live in student halls by European standards, at just 5%.240 The great majority of these students are aged 24 or under, but still just 7% of students in this age cohort live in student halls.

238 Blattler et al., “Portuguese Higher Education: A View From the Outside.” 239 Dominic Orr, Christoph Gwosc, and Nicolai Netz, “Social and Economic Conditions of Student Life in Europe: Synopsis of Indicators - Final Report - Eurostudent IV 2008-2011” (Bielefield: W. Bertelsmann Verlag, 2011). 240 Ibid.

0% 20% 40% 60% 80% 100%

Institutional partnerships

Employment and employability fairs or exhibitions

Employment exchanges

Employment and employability observatory

IPs Universities


86

2.4 Supports for Students with Disabilities Students with disabilities are practically unmentioned in the Vossensteyn et al. review of European policies on study success.241 Yet there is no question that disability can affect students’ study success. We cannot address the needs of these students in depth because we do not have the resources to such an analysis justice. Yet we will provide some consideration because this is almost certainly a very important factor in access and completion in Portugal. Disability can be understood through a few different approaches. Firstly, through the impairment perspective or medical model that “defines disability as a health problem, disease, illness or abnormality within the individual”.242 The ecological perspective views disability “as resulting from the interaction of impairment, activity limitations, or restricted participation, and a specific social or physical environment such as work, home or school”.243 From this view, an “impairment refers to problems in physical or mental functioning”, whereas a “disability” is the interaction between the individual with an impairment and their external environment.244 This is the view adopted in the UN Convention on the Rights of Persons with Disabilities (UNCRDP) to which Portugal is a signatory.245 The UNCRDP defines ‘persons with disabilities’ as: “all persons with disabilities including those who have long-term physical, mental, intellectual or sensory impairments which, in interaction with various attitudinal and environmental barriers, hinder their full and effective participation in society on an equal basis with others”).246 It is on the basis of this view that we do not consider disability as an individual-level factor in study success operating independently from the higher education system. Disabilities are often classified as permanent impairments. Portuguese practice follows this model, with a particular emphasis on students with sensorial or mobility impairments. However, students may also be disabled temporarily due to an injury or illness, or cyclically, when affected by mental illness for example.247

241 Vossensteyn et al., “Dropout and Completion in Higher Education in Europe.” 242 Kayti Baur et al., “Disable the Label: Improving Post-Secondary Policy, Practice, and Academic Culture for Students with Disabilities” (Halifax, NS: Students Nova Scotia, 2014), 3, http://studentsns.ca/wp-content/uploads/2014/12/2014-12-03-disable-the-label-final-report-accessible-3.pdf. 243 Ibid., 4. 244 Ibid. 245 United Nations, “Convention on the Rights of Persons with Disabilities (CRPD),” UN.Org, 2017, https://www.un.org/development/desa/disabilities/convention-on-the-rights-of-persons-with-disabilities.html. 246 United Nations, “United Nations Convention on the Rights of Persons with Disabilities,” United Nations Convention on the Rights of Persons with Disabilities - HTML, 2006, https://www.un.org/development/desa/disabilities/convention-on-the-rights-of-persons-with-disabilities/convention-on-the-rights-of-persons-with-disabilities-2.html. 247 Baur et al., “Disable the Label: Improving Post-Secondary Policy, Practice, and Academic Culture for Students with Disabilities.”


87

Approximately 120-140 students with mobility or sensorial impairments enroll in higher education each year, representing between 12% and 17% of the spaces reserved for them in the CNA.248 Far more students are registered as having disabilities at the secondary level largely, principally because the definition at that level is broader. In the website survey, 85% of universities and 46% of IPs reported having services for students with special education needs.249 Resources may include a personal aid, material or equipment. Some institutions conduct assessments to develop specific accommodations both for class work and assessments, including longer periods of time and separate spaces for exams, and may also offer training programmes for instructors. Services can be specific or non-specific, as in someone working with the students from within another unit, and may may be offered by volunteers. To access these services, students usually must certify their disabilities.

3. How May the Quality of Higher Education Delivery Contribute to Low Completion? We identify a host of elements of delivery of higher education in Portugal that may contribute to reducing completion rates. We will first review findings on student satisfaction in Portugal, before delving into our particular concerns in depth.

3.1 Student (Dis)Satisfaction Dropout is not necessarily a product of dissatisfaction, it can be involuntary due to external pressures. In a survey of students who dropped out from the IP do Porto, 62.8% reported being satisfied and 27.1% reported being very satisfied with their experience at the IP.250 Nevertheless, measures of student satisfaction can provide a proxy for students’ assessment of the quality of higher education, and also relate to student motivation.251,252 The Eurostudent Survey found Portuguese students express low confidence that their studies will help them accomplish their personal development or professional goals relative to students in other European countries, as indicated in Figure 4.4.253 The slightly higher dissatisfaction among

248 Interview Notes: March 14, 2017. 249 DGEEC - Equipa de Estudos de Educação e Ciência, “Draft: Medidas de Promoção Do Sucesso Escolar Nas Instituições de Ensino Superior Públicas Em Portugal, Tal Como São Relevadas Nos Respetivos Sítios.” 250 Gabinete de Planeamento, Projectos e Desnvolvimento, “Estudo de Caso: Abandono Escolar 2012-13 (ESE, ESMAE, ESEIG, ISCAP, ESTGF, ESTSP e APNOR)” (Porto: Politécnico do Porto, 2014). 251 Cláudia S. Sarrico and Maria J Rosa, “Student Satisfaction with Portuguese Higher Education Institutions: The View of Different Types of Students,” Tertiary Education and Management 20, no. 2 (2014): 165–78. 252 IP de Setúbal, “O Abandono Nos Cursos de Primeiro Ciclo” (Setúbal: Instituto Politécnico de Setúbal, 2013). 253 Orr, Gwosc, and Netz, “Social and Economic Conditions of Student Life in Europe: Synopsis of Indicators - Final Report - Eurostudent IV 2008-2011.”


88

Licenciatura students is perhaps to be expected as graduate students are likely greater enthusiasts of higher education, more knowledgeable about how to navigate the system, and higher skilled.254 Among Eurostudent participants, Portugal ranks clear ahead of only three countries in terms of confidence in personal development and two countries in terms of professional development. Portugal’s results are closer to the median however among students for whom these goals are important, which may suggest the system performs well for more motivated students (or that students who have greater confidence are more motivated).

Figure 4.4: The proportion of students who indicate the fulfillment of goals through their studies is (very) high.

A 2006-2007 survey of almost 12 000 students (7.7% of the total student population) at all 30 public institutions and 18 private institutions provided substantial data as to students’ satisfaction with their higher education experiences.255 Overall, 84% of students indicated that if given the choice again they would apply to the same institutions, and 85% indicated they would recommend their institution. Fully 81% of students indicated they would not change study programme and 74% that they would attend the same institution. Satisfaction was relatively consistent by institution type, but highest overall at public universities. Students at private institutions reported the highest likelihood of changing their institution if given the opportunity and public university students the lowest. Students’ impressions of employability were essentially equal across the institution types. Satisfaction varied considerably by study program area. Men were more likely to be satisfied with the employability of their program. Satisfaction with particular different areas of learning is indicated in Figure 4.5.256 It suggests greater satisfaction with elements of more formal education, and less satisfaction with the acquisition of softer skills relating to interpersonal relationships and leadership, although the differences between the areas of learning appear minor.

254 Sarrico, “Re: Two Chapters of Portugal Review.” 255 António Magalhães, Maria de Lourdes Machado-Taylor, and Maria José Sá, “Satisfação Dos Estudantes Do Ensino Superior Português” (Fundação para a ciência e a tecnologia, 2012). 256 Ibid.

0%

10%

20%

30%

40%

50%

60%

70%

80%

General studentbody

Those who considergoal (very)important

General studentbody

Those who considergoal (very)important

All students Licenciatura

To develop personally

To be prepared for work


89

Figure 4.5: Satisfaction with areas of learning (scale from 1-10)

Table 4.2: Statistically significant differences in satisfaction by student group

Difference Measures evaluated significantly differently (out of 45)

Institution Type

Differences in 23 areas, with mostly students at public universities differing from students at the other two types of institutions

Discipline All areas except out-of-class interaction with teaching staff, with lower satisfaction notably among students in Arts and Humanities

Gender Male students more satisfied in 40 areas

Year of Study First year students more satisfied than third year students in all measures

Option of Study Program

Students in their first option program significantly more satisfied than other students in 14 indicators

Living at Home or Away

Students living away from home less satisfied with 11 indicators, students at home are less satisfied with quality of discipline content

Financial Need Students who consider they have sufficient financial resources more satisfied than those who feel they lack support in 42 areas

Working Status*

Working students more satisfied in eight indicators, less satisfied with class size

*The sample size for this indicator was relatively modest compared with the others.

This student satisfaction survey also indicated how important students considered different elements of higher education and how satisfied they were with each, under three categories: academic elements, academic supports, and non-academic processes and services. The gap is the difference between importance and satisfaction, and provides an indication as to whether students’ expectations are met.257 For every element, expectations were significantly higher than perceived quality received at better than 99% confidence. Table 4.2 indicates where there were statistically significant differences between groups of students on different measures, with the most important factors being: discipline, gender, year of study and feelings about their financial situation.258 The following two sections consider these results in greater detail.

257 Sarrico and Rosa, “Student Satisfaction with Portuguese Higher Education Institutions: The View of Different Types of Students.” 258 Ibid.

0 1 2 3 4 5 6 7 8 9 10

Greater knowledge

Better work skills

Better communication skills

Met intellectual expectations

Better skills or possibilities for relating interpersonally

Met personal expectations

Developed leadership skills


90

Another survey completed in 2003 found relatively low student satisfaction in Portugal compared to 18 other advanced countries, with Portugal outperforming just Turkey, Lithuania, Spain and Estonia.259 Just 55.4% of Portuguese students reported they would pursue the same study program at the same institution if they had to choose again, as compared with 59% across the sample countries. Portuguese students reported higher satisfaction with personal development and theoretical training, less satisfaction with connections to professional contexts and practice.

3.2 Academic Factors The findings for satisfaction with academic elements and academic supports (namely relating to facilities) are outlined in Table 4.3, again on a ten-point scale with the gap calculation comparing the importance of an element and students’ levels of satisfaction. 260 Table 4.3: Student evaluations of importance of and satisfaction with Academic Elements and Academic Supports (scale of 1-10)

Category Element Importance Satisfaction Gap

Aca

dem

ic E

lem

ents

Quality of teaching 8.2 7.1 1.1

Knowledge of study disciplines 8 7 1

Quality of discipline content 7.8 6.8 1

Relevance of disciplines 7.4 6.8 0.6

Evaluation of learning 7.2 6.6 0.6

Quality of academic counselling 6.6 6 0.6

Interaction with faculty outside class 6.5 6.3 0.2

Offer of optional disciplines 5.9 5.2 0.7

Aca

dem

ic

Sup

po

rts

Computer resources 7.7 6.6 1.1

Library resources 7.7 6.6 1.1

Conditions of classrooms 7.1 6 1.1

Class-size 7 6.1 0.9

Conditions of buildings and environs 7 6.1 0.9

Conditions of laboratories 6.7 5.8 0.9 These results show a relatively strong correspondence between the importance of different elements and students’ satisfaction, which would appear to indicate that higher education is delivering best where it matters most. Nevertheless, we identify important challenges with the academic side of higher education delivery in numerous areas. Public university students were

259 Mariana Gaio Alves, Natália Alves, and Miguel Chaves, “Inserção Profissional e Razões de Ingresso e Reingresso No Ensino Superior: Um Ponto de Partida Para Uma Temática Em Aberto,” Sociologia, Problemas e Práticas 69 (2012): 99–118. 260 Magalhães, Machado-Taylor, and Sá, “Satisfação Dos Estudantes Do Ensino Superior Português.”


91

relatively less satisfied with some academic elements, while three academic elements were among the four overall where differences by gender were not statistically significant.261

3.2.1 Insufficient Focus on Teaching and Serving Students Portuguese students ranked the quality of teaching as the single most important element of higher education delivery in the survey noted earlier. The evidence that we have on the classroom environment, however, provides considerable grounds for concern. Among surveyed withdrawn-students from the IP do Porto who provided suggestions, 41% indicated that pedagogical measures could have helped prevent them from dropping out.262 This was by far the most popular response, well ahead of administrative or even financial measures (16% and 15% respectively), though the term “pedagogical measures” appears very broad. In the Universidade de Évora study, approximately 38% of students reported that curriculum and functioning of the class as a factor in their dropout (the fourth most important factor cited), while 30% cited thesis/internship/project work, and 28% cited teaching quality.263 Curriculum and program structure problems were most significant for Masters students in first year and integrated-masters students, and more specific issues included a badly organized study plan, too much theoretical content, disconnections between material and application, weak teaching methods, too demanding disciplines and dissatisfaction with evaluation methods. Thesis problems were also most significant for Masters students, with more specific issues including difficulty in choosing topics and defining objectives, lack of supervisor feedback and availability, poor time management to complete the project, and difficulties in data collection and field work. Finally, in terms of performance of faculty issues included communication difficulties, poor faculty effort, faculty being unavailable to address concerns, faculty lacking knowledge, personal conflicts and unequal marking. Studies at the IP de Setúbal cite similar factors at both the Licenciatura and Masters levels.264 265 Masters students who withdrew from the IP de Setúbal indicated that it was through improving the learning and teaching process that the institution could have most effectively altered their decision.266 Specific items were: better support in the preparation of the final dissertation, greater engagement with faculty in general, better achievement of curriculum elements, and lastly greater pedagogical resources, schedule flexibility and tutoring. The findings from the IP de

261 Sarrico and Rosa, “Student Satisfaction with Portuguese Higher Education Institutions: The View of Different Types of Students.” 262 Gabinete de Planeamento, Projectos e Desnvolvimento, “Estudo de Caso: Abandono Escolar 2012-13 (ESE, ESMAE, ESEIG, ISCAP, ESTGF, ESTSP e APNOR).” 263 These figures are among students who attended at least some classes. Rosalina Pisco Costa et al., “Identificação Das Causas Do Abandono Escolar Na Universidade de Évora” (Évora: Universidade de Évora, 2015). 264 IP de Setúbal, “O Abandono Nos Cursos de Primeiro Ciclo.” 265 IP de Setúbal, “O Abandono Nos Cursos de Segundo Ciclo” (Setúbal: Instituto Politécnico de Setúbal, 2014). 266 Ibid.


92

Setúbal also highlight how students in different degree programmes within an institution may have very different experiences. Masters students who dropped out were critical of their study programs, but Licenciatura students who dropped out were mostly positive about the programmatic content of the curriculum, teaching quality, and relationships with teachers.267 Multiple interviewees believed that the quality of instruction can be very poor in many cases. One argued that students particularly learn little in theoretical classes. Some interviewees indicated that changing pedagogical methods and faculty mentality was the most difficult challenge. Faculty were often elite students and are not prepared to teach non-elite students. Addressing the individual needs of a diverse student body, in lieu of treating all students the same way, is also challenging. Examples of this kind of individual accommodation can include avoiding exams on religious holidays, or accommodating students with disabilities. Faculty also may not know how to define and assess learning outcomes, according to one interviewee. Finally, physical infrastructure (e.g. classrooms) is also ill adapted in many cases to different pedagogical approaches. Manatos et al. completed a survey of academics at all of Portuguese universities regarding quality of instruction and received 1 116 responses.268 In particular, they measured faculty perceptions of the importance and implementation of the Standards and Guidelines for Quality Assurance in the European Higher Education Area (ESG) for internal quality assurance. The results showed that academics found the standards were important, but their implementation was relatively weak, as indicated in Table 4.4.269 The gaps were statistically significant across all the standards at better than 99% confidence. The largest gaps were in terms of public information, the quality assurance of teaching staff and learning resources and student support, with the implementation of teaching staff quality assurance the weakest evaluated overall. Table 4.4: Average assessment of the importance and implementation of the European Standards and Guidelines by Portuguese university faculty (scale of 1-7)

Standard or Guideline Importance Implementation Gap*

Policy and procedures for quality assurance 6.3 5.6 -0.6

Approval, monitoring and periodic reviews of programs and awards

6 5.5 -0.4

Assessment of students 5.4 5.7 0.2 Quality assurance of teaching staff 6.3 5.2 -1.1

Learning resources and student support 6.4 5.6 -0.8

Information systems 6 5.4 -0.6

Public information 6.7 5.5 -1.3 *Gap figures may not be equal to importance minus implementation due to rounding.

267 IP de Setúbal, “O Abandono Nos Cursos de Primeiro Ciclo.” 268 Maria J. Manatos, Maria J Rosa, and Cláudia S. Sarrico, “The Importance and Degree of Implementation of the European Standards and Guidelines for Internal Quality Assurance in Universities: The Views of Portuguese Academics,” Tertiary Education and Management 21, no. 3 (2015): 245–61. 269 Ibid.


93

Academics at private institutions generally had the most positive opinions of the importance and implementation of the standards, while differences by discipline were significant and faculty with doctoral degrees generally provided lower evaluations of importance and implementation. It is possible also that this survey was biased to be more positive, given that the response rate was modest (6.2%), most of the respondents were involved in quality management, and faculty involved in quality management were more likely to provide positive evaluations and could logically have been more motivated to participate in the study. Reviews have found that institutional efforts to incentivise and support teaching improvement generally do not meet the European Standards and Guidelines for Quality Assurance or the national legal framework. Relatively few institutions provide strong incentives or rewards for improving teaching quality. An important challenge, which is common to practically all higher education systems, is the weighting of different criteria in faculty evaluation.270 The literature reports that research is systematically weighted more heavily than teaching in Portugal and this was often strongly emphasised in interviews. One interviewee indicated that a recent promotion decision weighted 60% on research, and just 30% on teaching and 10% on service. One institutional representative indicated that their administration has pushed for student evaluations to represent 25% of the weight in faculty promotion, but scientific councils push to weight these evaluations at just 5%. The heavy weighting of research can punish faculty who make a greater effort to be effective in the classroom. Institutions may have little control over this since legally promotions must be decided by faculty committees and many members are from other institutions, according to interviewees. Sarrico et al.’s view was that a quality culture will take longer to develop.271 The established pattern is one of accountability, but not of improvement. They found that most faculty, institutional administrators and students believed little has changed in the classroom. Critically, more information has been made available through surveys and so on, but it seems that this is often not used very widely to drive improvements. Satisfaction questionnaires in particular are considered to have limited consequences. Many activities were pursued by the service-level agents because of requirements from the centre of the institution or the government, but not in a sincere drive for improvement. Many efforts for improvement are still pursued only informally. The website-based survey of services found that although the great majority of institutions had quality management services, their development was broadly uneven.272 It is worth noting that in many cases students themselves may very much have adapted to this teaching model. Interviewees indicated that students are often more comfortable, for example, with an exam centred assessment model akin to what they have seen in compulsory education.

270 Cardoso, Tavares, and Sin, “The Quality of Teaching Staff - Higher Education Institutions’ Compliance with the European Standards and Guidelines for QA - the Case of Portugal.” 271 Sarrico, Veiga, and Amaral, “The Long Road—how Evolving Institutional Governance Mechanisms Are Changing the Face of Quality in Portuguese Higher Education.” 272 DGEEC - Equipa de Estudos de Educação e Ciência, “Draft: Medidas de Promoção Do Sucesso Escolar Nas Instituições de Ensino Superior Públicas Em Portugal, Tal Como São Relevadas Nos Respetivos Sítios.”


94

In general, however, writers and interviewees believe that the situation is improving and will continue to do so with the help of the quality assurance regime. There was a broad sense among many interviewees that central institutional administrations cannot be successful if they are seen as imposing on faculty, but faculty may be feeling less insecure and more comfortable with change since the economic situation has improved somewhat and the threat of budget cuts lessened. Many challenges also may vary between degree programmes, with better performance in more professional schools like nursing. Pedagogical improvement activities are generally non-compulsory and many faculty do not engage, but many institutions are still working hard in these areas. Some institutions are operating teaching and learning laboratories, peer-based pedagogical support initiatives, councils for continuous improvement of teaching, and other pedagogical support initiatives.273 Interviewees also indicate that negative faculty behaviours, such as missing classes or taking long periods of time to publish student assessment results, appear to have declined largely due to student surveys. Institutions are also adjusting and often shortening student surveys to increase participation and undercut opposition that suggests they are not representative. The website-based review tracked the presence of curricular reforms or innovations, teaching of general or soft skills, and the presence of pedagogical observatories to evaluate pedagogy, track teaching and learning results and develop projects to identify and eliminate learning difficulties. Their findings are in Figure 4.6, and indicate stronger performance at universities.274

Figure 4.6: Proportions of public institutions providing relevant information on pedagogy-related activities on their website

Some examples of these activities are detailed in the literature or were explored in interviews. The Universidade do Porto has its Teaching and Learning Improvement Unit (Unidade Melhoria do Ensino e Aprendizagem), which is pursuing a host of activities including regular training of

273 Cardoso, Tavares, and Sin, “The Quality of Teaching Staff - Higher Education Institutions’ Compliance with the European Standards and Guidelines for QA - the Case of Portugal.” 274 DGEEC - Equipa de Estudos de Educação e Ciência, “Draft: Medidas de Promoção Do Sucesso Escolar Nas Instituições de Ensino Superior Públicas Em Portugal, Tal Como São Relevadas Nos Respetivos Sítios.”

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Curricular Innovation or Reforms

General Skills Instruction

Pedagogical Observatory

IPs Universities


95

faculty on pedagogical methods, peer teaching support and a teaching award.275 The programme has gone from being stigmatised to attracting increasing interest and engagement among faculty. Among schools, between and 6% and 44% of faculty participated in workshops in 2015-16. The participation of respected senior professors, including from prestigious institutions outside of Portugal, has helped considerably. The institution has also adopted a leadership role in promoting these activities more broadly in Northern Portugal by inviting faculty from other institutions to participate in the workshops. The Lisbon School of Economics and Management (Instituto Superior de Economia e Gestão, ISEG) at the Universidade de Lisboa operates a Pedagogical Observatory that collects longitudinal data on failure and dropout rates, but also monitors teaching methods, hosts an annual seminar on related to study success, and connects faculty with other institutions.276 One of the IPs we visited has also been implementing activities to train academic staff in English and different pedagogical methods, facing varying challenges across different schools within the institution, and recently introduced a prize to recognize innovation and success in pedagogy. The IP administration is looking to send some faculty to learn pedagogical skills at partner institutions in Northern Europe, and has wanted to place advisors in classes to observe and support pedagogical improvement, but felt this would be strongly opposed by faculty. Finally, they are looking to emphasise practical over theoretical work in first year to better engage students.

3.2.2 High Rates of Academic Failure The difficulties students face in their courses are key factors in their choices of whether to drop out. Students who fail more classes or enroll in fewer courses are more likely to drop out.277 High rates of academic failure are also a testament to the weakness of student-centred instruction.278 We do not have access to comparative data on failure rates in higher education courses, or even national level data within Portugal. However, institutional data and interviews suggest that failure is very common and normalised in Portuguese higher education. At the Universidade de Évora, from 2011-2014, the median licenciatura student who dropped out had failed 55% percent of their classes, whereas the corresponding rate among non-drop-outs was 9%.279 Half of dropouts had completed one course or less, while only 9% had completed half the credits required for their degree, so even students who had been at the institution longer

275 Universidade do Porto Melhoria do Ensino e Aprendizagem, “Participação Em Projetos| Inovação Pedagógica,” UPorto | Inovação Pedagógica, March 23, 2017, https://inovacaopedagogica.up.pt/mea/projetos/. 276 DGEEC - Equipa de Estudos de Educação e Ciência, “Draft: Medidas de Promoção Do Sucesso Escolar Nas Instituições de Ensino Superior Públicas Em Portugal, Tal Como São Relevadas Nos Respetivos Sítios.” 277 Ibid. 278 IP de Setúbal, “O Abandono Nos Cursos de Primeiro Ciclo.” 279 Pisco Costa et al., “Identificação Das Causas Do Abandono Escolar Na Universidade de Évora.”


96

barely advanced.280 A study at the Universidade de Coimbra found most freshmen who dropped out passed fewer than 25% of the credits they enrolled in, while another from the Universidade do Porto indicated 36% of first year students failed to pass 75% of the expected ECTS credits.281,282 The shares of mature students who reported having failed at least one class across two universities in the Santos et al. study are presented in Figure 4.7.283 Close to two-thirds of all students in the study failed at least one class, including over half of students in first year and more than three-quarters of students by year three. At University 2, 14.1% of students failed to pass the year, while the corresponding figure at University 1 was 6.5%.

Figure 4.7: Percentages of mature students having failed at least one class by students’ year of study, in Santos et al. 2016

Finally, data from the ISEG’s pedagogical observatory provides a time-series on failure rates. The overall data is presented in Figure 4.8.284

Figure 4.8: Total failure rate across courses with 10 or more evaluated students at the ISEG

280 Ibid. 281 Madalena Alarcão, “Insucesso Académico e Abandono Escolar” (Seminário Sucesso Académico, Lisbon, May 12, 2015). 282 José António Sarsfield Cabral, “Algumas Notas Sobre o Acesso e Desempenho Dos Estudantes Admitidos Na U.Porto Em 2008/09, 2009/10 e 2010/11 (Pelo Regime Geral de Acesso Em 1s Ciclos e Mestrados Integrados)” (Seminário para o “Sucesso Académico,” Lisbon, May 12, 2015). 283 Lucília Santos et al., “Academic Success of Mature Students in Higher Education - A Portugese Case Study,” European Journal for Research on the Education and Learning of Adults 7, no. 1 (2016): 57–73. 284 Carlos Farinha Rodrigues, “Indicadores de (in)Sucesso Académico” (Sexto Seminário Anual do Observatório Pedagógico do ISEG, Lisboa, February 24, 2016).

0% 10% 20% 30% 40% 50% 60% 70% 80% 90%

First Year Students

Second Year Students

Third Year Students

All Students

University 2

University 1

50.3% 48.2% 49.2% 46.3% 45.1% 43.4% 42.1% 40.5%

0%

10%

20%

30%

40%

50%

60%

2007 2008 2009 2010 2011 2012 2013 2014


97

The proportion of enrolled students who failed declined from just over 50% in 2007-08 to just over 40% in 2014-15, still a high figure. In other words, in a given class half of students on average failed. At the level of specific courses, failure rates were highest in the mathematics department, followed by economics, and lowest in the social sciences. Failure was generally higher in compulsory courses and lower in optional classes. Rates were similar in first and second year, but lower in third year. Interviewees indicate that failure rates vary considerably between different schools within institutions, and between study programmes even within schools. Multiple interviewees referenced high rates of study failure notably in programmes and courses requiring mathematics or physics, which was associated with insufficient prior preparation. Interviewees also indicated that many students who are failing classes are not actually attending classes or even completing assessments. The rates of non-participation in assessment are presented in Figure 4.9.285 Enrolment 1 indicates rates among students taking the course for the first time, and 2+ for students taking the course a second or subsequent time.

Figure 4.9: Rate of non-participation in assessment across courses with 10 or more evaluated students at the ISEG, based on enrolment in the course for the first time or the second or greater time

The ISEG figures indicate that just over a quarter of enrolled students in 2014-15 did not participate in evaluations, a figure that had declined slightly since 2007-08. Among students taking the course for the second or greater time, i.e. students who had previously been unsuccessful, approximately 40% did not participate in evaluations. Moreover, the overall failure rate among these students was in excess of 60%. One interviewee showed the record of a class where one student was enrolled for the eleventh time. Another interviewee indicated that they knew of people enrolling for more than 20 consecutive years to maintain student status, and that this was particularly common in engineering programmes, though they felt this has become less common than in the past.

285 Ibid.

0%

5%

10%

15%

20%

25%

30%

35%

40%

45%

2007 2008 2009 2010 2011 2012 2013 2014

Total

Enrolment 1

Enrolment 2+


98

For the most part, it is unclear why students continue to enroll without trying to pass, paying fees to do so. One motivation identified by interviewees was access to services associated with student status and retaining their place within an institution under the NC in case the students wish to complete their degree later. Interviewees in one case indicated that music students used to skip final auditions and fail because training at the IP was cheaper than anywhere else, and students would not want to pass until they had the excellent grades required to enter an orchestra. Another speculative reason may be that given students do not pay tuition for each course they pursue, they may register for a full course load, but choose to actually pursue only some of the courses. One interviewee noted that inactive students might not be such a problem, given the students provide funding to the institutions without taking on services. This makes them contribuentes líquidos – basically free money. Accounting for students who did not participate in evaluations, we can calculate the failure rate among students who actually are trying to pass. This data is presented in Figure 4.10.286

Figure 4.10: Rate of failure among evaluated students across courses with 10 or more evaluated students at the ISEG, based on enrolment in the course for the first time or the second or greater time

Our figures here indicate that in 2014-15 one-fifth of active students in an LSEM course would be expected to fail. The share of active students failing has been declining however, especially among students who previously failed the course. Multiple interviewees indicated that an older academic culture in Portugal encouraged faculty to fail a certain share of students. While there was a common belief that the situation has improved, it appears that multiple faculty still believe that failing a significant share of students indicates quality of instruction and of a discipline. There was a sense that this is improving and the data from the LSEM appears to bear this out.

286 Ibid.

0%

5%

10%

15%

20%

25%

30%

35%

40%

45%

2007 2008 2009 2010 2011 2012 2013 2014

Total

Enrolment 1

Enrolment 2+


99

According to some interviewees, students may also be very accepting of failure notably in certain exams. Some students may in fact seek to have many different exams for a course and come unprepared to some of the evaluations, approaching them as “practice” and taking only some of their examinations seriously. There may be special allowances for some students to take more exams, such as athletes and student association leaders, which student associations may notably manipulate. Students may also be very disengaged until the lead up to final exams, which were the historical norm. Those faculty who still support this model appear to do so largely to minimise assessment effort. The Government of Portugal has requirements for academic suspension (prescrição) under its higher education financing law. Institutions’ internal policies must meet or exceed the requirements indicated in Table 4.5.287 For whatever year of study a student is in they have usually three spare years to progress before institutions are required to place them on academic suspension. Table 4.5: Study progress limits for academic suspension required under the higher education financing law

Study Progress Maximum registrations

Years ECTS Full-time Part-time

0 <60 3 6

1 60 to < 120 4 8 2 120 to < 180 5 10

3 180 to <240 6 12 4 240 to < 300 8 16

5 300 to < 360 8 16

6 360+ 9 18 Students who exceed the maximum number of registrations for their credits obtained are suspended from enrolling in their current study programme or another study programme for the subsequent two semesters. Institutional regulations may grant students part-time academic suspension schedules based on their particular situation. As of 2004, however, students with Worker-Student Status under another law are exempted from these requirements and may not face academic suspension. This may help explain the high rates of course repetition among working students. One interviewee also suggested that institutions may not be enforcing academic suspension as required, considering the free money. Another view was that institutions may have little incentive to suspend students who fail classes because the students can simply go complete the credits somewhere else and then transfer them back to their original institution, reducing the tuition that the original institution collects. Of course, many of these students do not appear to be passing many courses anyway. It seems there is little monitoring to detect violations of

287 Assembleia da República, “Lei n. 37/2003, de 22 de Agosto: Estabelece as Bases Do Financiamento Do Ensino Superior” (2003).


100

academic suspension requirements, although we could not find evidence to confirm that violations are occurring. Many of the efforts in this area have been discussed already with respect to strengthening pedagogy, but it is worth taking note of the ISEG’s pedagogical observatory again given the institution’s commitment as demonstrated through the collection of the data on study success highlighted above. These figures were used largely to provide an initial evaluation of the programme’s success in supporting students. The ISEG also provides special classes for students who have previously failed, and makes doctoral students available to support students in high failure-rate classes, but there is little take up of the doctoral student support. An interviewee noted also of another university that they require faculty to write an explanatory report and improvement plan when the failure rate in their class exceeds a certain amount. However, there was little follow up as to whether the plans are followed.

3.2.3 Rigid Program Structures This review could not consider in depth Portugal’s system of credit transfer and prior learning assessment and recognition, though Portugal has been considered a leader in these areas in the past. Nevertheless, we do consider two key elements of rigidity in programme curricula and structures: the extent to which required courses within study programmes are relevant to multiple fields firstly, and secondly the extent to which students can select courses to customise their programme. We did not find substantial literature on these topics and so will rely in large part on the comments of interviewees. However, interviewees almost universally agreed that the rigidity of degree programmes is a factor in reducing completion, and indicated this is frequently noted in external reviews. The extent to which there is a problem seems especially dependent on the study programme. Having general study programme courses is relevant for student completion because it can allow students to shift programmes and follow their interests more easily. For example, institutions may offer students a very general first year to provide a broad preparation and in case students’ interests change. Our interviewees had mixed views on this issue, with some indicating there were many flexible core courses and others saying otherwise, although the general impression was that core courses are relatively specialised by international standards, including in year one of Licenciatura degrees. Where there was basically universal consensus among interviewees was in the idea that students have too few optional courses. Interviewees suggest the approximate share of credits open for electives across many programmes is roughly 5%. Dissatisfaction with the lack of optional courses is also noted in student surveys. In many cases, large numbers of optional credits may be listed for students up front, but then they discover very few optional courses are available when they actually go to register, which spurs considerable disappointment. Traditionally students also


101

pursue their optional classes within the same school or department, instead of branching out into other areas of the institution. Faculty also may not be well coordinated. Sarrico et al. note that many students complain that faculty do not consult with one other about their courses so there is significant overlap in material, compounding workloads.288 IP de Setúbal Masters students who dropped out also reported considerable repetition of content notably from their Licenciatura programs, and they found their programs ultimately not very demanding.289 Interviewees noted also that in some cases hours of classes are poorly coordinated, so that if students fail a required class in first year, in second year it may overlap with another required class. This seems to relate to an incomplete transition from the pre-Bologna course structure based on annual courses to the credits system. Interviewees noted that often the course offerings less relate less to strategic thinking about what students need to learn, than meeting faculty expertise and preferences for courses to teach. One interviewee described how faculty proposed a programme structure to the administration after just three hours of discussion, whereas programme design should require more reflection and engagement with external stakeholders to seriously consider students’ needs. Interviewees noted that relatively rigid degree programmes are a historic norm in Portugal, but also a result of partial implementation of the Bologna Process. In particular, when reducing the length of Licenciatura programmes from 5-6 years to 3-4 years, faculty frequently insisted on covering as much of the original material as possible out of fear of watering programmes down. One interviewee indicated that half of first year students’ courses were generalist prior to the Bologna process. In some cases, programmes are also must meet legislative requirements. Financial constraints are another important factor in some cases, inhibiting institutions from hiring faculty to teach additional courses. One interviewee even noted that an institution had adopted a policy of only giving students very limited course choices, more specifically: students needing to fill two optional slots would be able to choose between only three courses. In an effort to counteract this problem, this institution planned to offer different optional classes each year. This challenge generally also appears to be worse at smaller institutions. Some interesting things are happening in this area. The Universidade de Lisboa has created a general studies Licenciatura programmes that has become very popular according to interviewees, though they suggested it also faced considerable difficulties up front including from A3ES, because of questioning as to what career it would prepare students for. Representatives from an IP indicated that they were committed to providing common courses for their programmes so that students could pursue the programme most of interest to them. Many interviewees indicated a preference for establishing more general programmes at the Licenciatura level and then encouraging students to specialise more in the second cycle.

288 Sarrico, Veiga, and Amaral, “The Long Road—how Evolving Institutional Governance Mechanisms Are Changing the Face of Quality in Portuguese Higher Education.” 289 IP de Setúbal, “O Abandono Nos Cursos de Segundo Ciclo.”


102

3.2.3 Excessive Time Demands on Students Evidence from the Eurostudent 2011 study presented in Table 4.6 indicates that Portuguese students at both the Licenciatura and Masters level budgeted more time per week for academic activities or paid employment than students in almost any other country in Europe.290 In particular, Portuguese students reported the longest hours of taught study, with almost three-quarters (74%) of students in Portugal undertaking study activities for more than 30 hours in a typical study week and 92% for 21 hours per week or more. Further data from the same study indicated that working Portuguese students across a range of employment hours study more than students in any other European country working equivalent hours, with students working over 15 hours still averaging almost 35 hours per week of study time. Table 4.6: Reported weekly hours of activity and rank among 22 European countries, 2011

Activity Licenciatura Masters Hours Rank Europe

Median Hours Rank Europe

Median Taught studies 26 1 18.5 20 1 15

Personal study time 15 =11.5 15 18 =9.5 17

Paid employment* 14 4 15 18 7 14 Total Hours of Study and Employment**

55 1 41.5 56 1 46

Where rankings are tied the ranking indicated is noted with an = sign and calculated based on the mean ranking among the tied countries, so if two countries rank ninth the tied ranking recorded is 9.5 (sum of 9+10 divided by 2). *Paid employment in this case does not distinguish between “ordinary students” and “working students” as defined in Portuguese legislation. **Total hours do not necessarily correspond to the sum of the hours spent on the various sub-activities.

When we consider the breakdown of time commitments tied to students’ age, we obtain similar findings, presented in Table 4.7.291 Again, Portuguese students had the longest class hours among students under 25 years of age and students thirty years of age or older. As well, younger students worked considerably fewer hours than older students and less than the median among participating European countries, but older students worked among the most hours at almost 70 hours per week, while also having 60% more than the median hours of class time. Of course, older students also often have dependent family members.292

290 Orr, Gwosc, and Netz, “Social and Economic Conditions of Student Life in Europe: Synopsis of Indicators - Final Report - Eurostudent IV 2008-2011.” 291 Ibid. 292 Santos et al., “Academic Success of Mature Students in Higher Education - A Portugese Case Study.”


103

Table 4.7: Reported weekly hours of activity among 22 European countries by student age for students not living with parents, 2011

Activity Age 24 or less Age 30-plus

Hours Rank Europe Median


Taught studies 27 1 19.5 21 1 13 Personal study time 17 =9.5 16.5 15 11 14.5

Paid employment* 4 15 6 32 =6 27

Total Hours of Study and Employment**

48 1 41 68 1 52.5


Course commitments can vary significantly by study programme. Table 4.8 indicates time commitments for students in Humanities and Arts programmes and for Engineering, Manufacturing and Construction.293 The time commitments among humanities and arts students appeared particularly exceptional, notably in terms of taught studies, although Portuguese students were the busiest still across both programme types. Table 4.8: Reported weekly hours of activity for Licenciatura students among 22 European countries by study programme type

Activity Humanities and Arts Engineering, Manufacturing and Construction



Taught studies 25 1 17.5 27 3 22 Personal study time 17 =9.5 16.5 16 =8.5 15

Paid employment* 11 3 7 14 2 6

Total Hours of Study and Employment**

53 1 39 57 1 43.5


Portuguese students also reported the highest levels of dissatisfaction with their time budget among participating countries in the Eurostudent survey, with almost 40% reportedly dissatisfied or very dissatisfied as compared with an average of 24% across the participating countries. The study suggests satisfaction is especially low among students aged 30 and older.

293 Orr, Gwosc, and Netz, “Social and Economic Conditions of Student Life in Europe: Synopsis of Indicators - Final Report - Eurostudent IV 2008-2011.”


104

Time pressures are compounded for students who have to travel relatively long distances to attend classes. In a 2008 study at the Universidade do Minho, incompatibility of class times with their other responsibilities was referenced as a cause of dropout by 21% of students.294 Portuguese respondents to the EuroStudent survey reported a median daily travel time from home to their higher education institution of approximately 27 minutes, though closer 37 minutes for students living with parents. Travel time can affect students’ ability to arrive on time and consequent academic success and community engagement.295 It may be a particular concern for institutions located in more isolated areas.296 In contrast, Portuguese students are not at the very top in terms of the proportion who consider studies to be a central concern (i.e. of more importance). In fact, at 71% they fall behind four other countries on this measure. This suggests many Portuguese students spend very large amounts of time on study-related activities even though they do not consider their studies as fundamentally central to their lives. Related with this, some interviewees indicated that many students do not view their education as a full-time occupation and do little work until cramming for exams. Another interviewee suggested that students are under so much pressure in upper secondary to gain admission into higher education that once they enter they often relax somewhat. One last perspective was that many students believe they can do all their work in classes partly because the hours are so long, which undermines their learning. Part-time studies can represent a strategy to help balance various commitments. Institutions are required to offer part-time options, but interviewees suggest not all institutions may have actualised this requirement. Students may also be unaware of this possibility given that study part-time is simply not the norm. Reflecting this, the OECD has no data on part-time enrolment in Portugal for example.297 In general, international data indicates that part-time students have lower study success rates than full-time students.298 This could be unrelated to part-time status itself, but instead student characteristics including higher shares of mature students, greater competing time or financial pressures, and lower motivation. The EuroStudent figures are of course relatively dated, especially for a higher education system that has changed considerably in the aftermath of the Bologna process. Unfortunately, more recent statistics are not available. Interviewees indicated, however, that class hours used to be even higher pre-Bologna in many study programmes, even 30+ hours. Most interviewees, though not all, believed that class hours had likely fallen further since the EuroStudent data was collected and were likely around 20-22 hours. If true, this would be closer but still higher than the European median. Of course, this says nothing of hours for other activities. Interviewees also confirmed

294 IP de Setúbal, “O Abandono Nos Cursos de Primeiro Ciclo.” 295 Santos et al., “Academic Success of Mature Students in Higher Education - A Portugese Case Study.” 296 Sandra Hasanefendic, Manuel Heitor, and Hugo Horta, “Training Students for New Jobs: The Role of Technical and Vocational Higher Education and Implications for Science Policy in Portugal,” Technological Forecasting and Social Change 113 (2016): 328–40. 297 OECD, “Education at a Glance 2016: OECD Indicators” (Paris: OECD Publishing, 2016). 298 Ibid.


105

not only that schedules vary across programmes, but also may be heavier in the first year of a study programme than in subsequent years. The institutional or system factors in high class hours appear to be multiple. Institutional representatives cite legislative and programme accreditation requirements, though institutional policies also affect study hours independently, as at one of the case study universities where the maximum formal class hours per week was set at 23.5. Post Bologna there was also pressure to condense programme material into the shorter degrees and fewer courses, particularly at the Licenciatura level. At one of the IPs visited, institutional leaders indicated that internal agents consistently still call for more hours to be added to class requirements. One interviewee indicated that historically contact hours were a major metric for faculty contracts, and the basis for the Government to determine whether institutions could hire more faculty. Now ECTS are connected to workload, but there are still legislated minimum teaching hours for each faculty member. Interviewees indicate that faculty fear that reducing their hours of class-time may create redundancies and thereby threaten unemployment, although since the country’s economic situation has improved somewhat this concern may have diminished. The actual timing of classes also is often not conducive to students’ needs. This is often cited as a particular challenge for mature and working students, and in many cases Portuguese students’ schedules are reportedly incompatible with their paid employment.299 Among Licenciatura students who dropped out of the IP de Setúbal and felt they could have been helped to persist, 41.9% cited improved class hours as a factor that would have helped.300 According to an interviewee, some institutions may also scatter classes throughout the day requiring students to be on campus for longer, and the main driver of class timing may be faculty preferences and availability, as opposed to student needs. Multiple visited institutions were delivering night classes or exploring online platforms. One university, however, had cancelled many night offerings due to low enrolment and high per unit costs. They considered it unrealistic for students to work full time and do lengthy night classes. Students need to learn time management skills to be successful in higher education.301 However, extremely long hours of study, combined with great dissatisfaction with time commitments and modest student prioritisation relative to other activities seems like a recipe for low completion.

3.2.5 The Unclear Binary Distinction Interviews and our literature review suggest that differences in quality between universities and IPs do not necessarily drive differences in study success. Various IPs may be performing

299 IP de Setúbal, “O Abandono Nos Cursos de Primeiro Ciclo.” 300 Ibid. 301 Ibid.


106

particularly well in supporting students who face many challenges that undercut study success. Nevertheless, there are good reasons to be concerned about some policies affecting the IP sector. IPs have been restricted in terms of their hiring, which has translated into more than half of faculty in 2012 being on part-time contracts. Interviewees indicated that at one major urban IP close to 70% of faculty were on part-time contracts. Many interviewees argued that it is desirable for many IP faculty to work part-time, so that they can continue to work professionally and maintain workplace connections. However there appeared to be broad consensus that the extent of part-time work was very likely excessive. Teaching loads are generally heavier in polytechnics than in universities, yet at the same time academic research is unduly emphasised in policy. The push to increase the share of polytechnic faculty with doctoral degrees would only appear to aggravate this challenge. It seems unlikely to efficiently improve teaching quality for students, if it does so at all. In fact, multiple interviewees argued that doctoral training in a university appeared unlikely to train faculty to operate successfully in the polytechnic environment, but instead to deliver the style of research and instruction more appropriate to universities. Notably this approach may undercut the industry engagement and knowledge transfer that can make programmes relevant and generate practical workplace opportunities for students. These IP challenges seem to largely reflect the problematic binary division in Portugal’s higher education system, which a 2013 European University Association Review Team described as “not functional, transparent or strategic”.302 It seemed obvious to many interviewees that many IPs are still seeking to become universities and differential research output is viewed as a key challenge and a major parity of esteem challenge, which could be addressed by recruiting doctoral faculty to do research.303 It appears that the major urban IPs, which are the most similar to universities, are the most motivated to expand doctoral faculty. What is lost is IP motivation to coherently fulfill their own distinct and important mission. As one interviewee put it, it is “better to be a good polytechnic than a bad university” from a policy perspective. The incentives for institutions may not be aligned in this way however. While the interviewees and much of the literature focuses on this type of mission creep, the opposite creep has been as apparent given that universities owned and operated 18 polytechnic institutes in 2013 and more than half of the complement of public universities (seven) offered some polytechnic provision. The restriction of CTESP courses to delivery at IPSs seems to have been a positive step in this area, however some interviewees questioned whether it will be possible for these programmes to overcome the esteem challenge.

302 Blattler et al., “Portuguese Higher Education: A View From the Outside.” 303 Ibid.


107

3.2.6 Inadequate Remedial Support for Students with Weaker Academic Preparation As indicated in Chapter 2, students with lower grades are expected to drop out more often. However, based on 2011 figures for Licenciatura students at public institutions in Figure 4.11, the extent of dropout among those with very low grades appears extreme, particularly in universities.304 This begs the question as to what support universities should be providing to help these students overcome their clear disadvantages, if they are going to admit them at all.

Figure 4.11: Dropout after first-year among first-time students at public higher education institutions, 2012

Figures for students with entering grades of 20 are excluded due to very low sample size.

The institutional website survey found no universities and only two IPs offered remedial courses for students with inadequate academic preparation.305 The seemingly limited provision of such activities is especially remarkable given that institutions are legally required to offer students compensatory classes or pedagogical support where they consider material to be essential.306 However, both the IPs we visited offer courses but were not included on the website list. One IP appeared particularly strong, with various of its schools offering preparatory courses for credit in different areas, including mathematics, and sometimes adjusting regular courses to students’ levels of prior preparation. Often courses aimed at helping students prepare for entrance exams, including the M23 exam or the final secondary exams. Representatives of this IP also indicated that they encourage mature students to enroll part-time and pursue preparatory classes because those who start with full course-loads more often fail. The other visited IP provided some voluntary modules in mathematics and physics, and Portuguese language classes targeted at international students. Finally, the private university representative indicated they were offering students extra assistance in English, mathematics and scientific methods free of charge.

304 DGEEC, “Ensino Superior: Situação Em 2012-13 Dos Inscritos Pela Primeira Vez Em 2011-12” (Lisbon: Direção-Geral de Estatísticas da Educação e Ciência, 2013). 305 DGEEC - Equipa de Estudos de Educação e Ciência, “Draft: Medidas de Promoção Do Sucesso Escolar Nas Instituições de Ensino Superior Públicas Em Portugal, Tal Como São Relevadas Nos Respetivos Sítios.” 306 Assembleia da República, “Lei n. 105/2009 - Regulamenta e Altera o Código Do Trabalho, Aprovado Pela Lei n.o 7/2009, de 12 de Fevereiro, e Procede à Primeira Alteração Da Lei n.o 4/2008, de 7 de Fevereiro” (2009).

0%

5%

10%

15%

20%

25%

30%

35%

40%

10 11 12 13 14 15 16 17 18 19

Universities

ISPs


108

It is unclear to what extent students seek these services out. Just 8.6% of IP de Setúbal Licenciatura students who reported they could have been helped to persist cited pedagogical resources as an important aid.307

3.2.7 Poor Adaptation to the Academic Needs of Mature Students We have noted many challenges that mature students face especially. The time demands placed by the higher education system, as well as the high failure rates, seem indicative of a broader pattern of limited accommodation. The introduction of the M23 admissions stream arguably expanded access at entry for mature students, but not access to study success.308 If mature students believe higher education is not targeted towards their demographic, we would expect this to undercut their sense of belonging and lead many to drop out.309 Mature students at the Santos et al. study universities reported that institutions and faculty were unaware of or unconcerned with their challenges.310 They found assigned tasks excessive, complex and tied to very short deadlines, with little accommodation for their difficulties in meeting deadlines (due to conflicts with their other responsibilities) and certain academic objectives. Some also reported weaker communication, often based on informal word of mouth instead of formal emails, for example. Targeted supports from the institution were lacking. The Universidade de Évora study also cited poor faculty communication as a factor in dropout.311 Barros and Lopes highlight the need for institutions to better adapt to mature students through greater curricular flexibility and accessibility, more responsive pedagogy, the timing of services and classes, as well as training in study practices and of support as students begin their programs.312 The extent to which institutions fail mature students may be indicative that Portugal has informally adopted an approach where initial admissions are essentially open at many institutions, but the constraint is placed on students actually passing. This approach has been taken in other countries in a more formal fashion and is intensely inefficient.313 As noted in the Chapter 3, less competitive institutions might have opened up admissions the most for mature students. Interviewees also suggested, however, that some institutions might not want to recruit mature students at all and only be opening spots to fulfill legal requirements.

307 IP de Setúbal, “O Abandono Nos Cursos de Primeiro Ciclo.” 308 Rosanna Maria Barros and Sara Mónico Lopes, “As Políticas de Ensino Superior Em Portugal e o Caso Dos Estudantes Não Tradicionais: Reflexões Sobre o Direito Dos Adultos a Educação (Superior),” Poiésis, Tubarão 9, no. 16 (2015): 364–83. 309 IP de Setúbal, “O Abandono Nos Cursos de Primeiro Ciclo.” 310 Santos et al., “Academic Success of Mature Students in Higher Education - A Portugese Case Study.” 311 Pisco Costa et al., “Identificação Das Causas Do Abandono Escolar Na Universidade de Évora.” 312 Barros and Lopes, “As Políticas de Ensino Superior Em Portugal e o Caso Dos Estudantes Não Tradicionais: Reflexões Sobre o Direito Dos Adultos a Educação (Superior).” 313 Alex Usher, “Admissions Policies: Marks-Based, Broad-Based, or Random? | HESA,” One Thought to Start Your Day, January 11, 2017, http://higheredstrategy.com/admissions-policies-marks-based-broad-based-or-random/.


109

Almost all interviewees agreed that the system is not serving mature students well enough. The one exception argued that the distinction was less between mature students and traditional age students than between full and part-time students. It did seem that most institutional efforts mentioned to support mature students related to class scheduling or online delivery. Research on dropout from online courses and night classes indicate however that such courses have very high dropout rates, as observed at the Universidade Aberta in Chapter 2, and will not be a silver bullet for improving mature students’ success rates.314

3.3 Non-Academic Factors Reviewing our findings early on satisfaction, it was clear that students were relatively satisfied with many academic aspects of higher education, notwithstanding the challenges we have identified. Table 4.9 presents satisfaction with non-academic processes and services.315 Table 4.9: Satisfaction with non-academic processes and services (scale of 1-10)

Element Importance Satisfaction Gap

Study spaces 7.6 6 1.6

Support for students with special needs 7.5 5.5 2

Website 7.5 6.5 1

Food services - bar(s) 7.2 6 1.2

Food services - Canteen(s) 7.1 5.6 1.5

Care in service delivery 7.1 5.7 1.4

Non-teaching staff attitudes with students 6.9 6.2 0.7

Orientation of new students 6.9 5.7 1.2

Bookstore 6.9 5.7 1.2

Ease of enrolment process 6.8 6 0.8

Financial aid services 6.7 5.3 1.4

Leisure spaces 6.7 5.4 1.3

Student association 6.6 5.5 1.1

Information and promotional activities 6.6 5.9 0.7

Cultural programs 6.5 5.5 1

Student health services 6.4 5 1.4

Student housing 6.3 5.3 1

Student organisations and clubs 6.2 5.5 0.7

Extra-curricular activities 6 4.9 1.1

Sport facilities 5.9 4.6 1.3

314 See, for example, DGEEC, “Ensino Superior: Situação Em 2012-13 Dos Inscritos Pela Primeira Vez Em 2011-12 - Parte II” (Lisbon: Direção-Geral de Estatísticas da Educação e Ciência, 2013). 315 Magalhães, Machado-Taylor, and Sá, “Satisfação Dos Estudantes Do Ensino Superior Português.”


110

Whereas for academic services and academic supports, gaps attained no more than 1.1 and were as low as 0.2, shortfalls relative to students’ expectations are much more consistent and in many cases, much more significant. Moreover, there appears to be little relationship between students’ assessment of elements’ importance, and their satisfaction. Public university students provided significantly higher evaluations of non-academic processes and services than students at the other types of institutions.316 Taking into account also high study hours, Portuguese higher education appears to suffer systematic weaknesses in areas of student life and student engagement, which research indicates can be particularly important for study success.317

3.3.1 Underdeveloped First Year Student Integration We previously highlighted the importance of the first year of studies when it comes to preventing dropout in higher education. Given that it can take students time to fall behind on credit requirements and therefore have their registration suspended, even later dropouts again may reflect the students’ difficulties in transitioning successfully into higher education. The orientation process is partly about academic adaptation, but also psychosocial adaptation. The stresses of entering higher education can lead to disinterest, depression and dropout.318 At the Universidade de Évora, about 10% of students who dropped out cited their transition or integration in the university as a factor in their decision to withdraw, with the issue more salient to Licenciatura students.319 However, the transition is thought to be particularly difficult at larger higher education institutions, and among students who are leaving home for the first time to attend and/or who are the first in their families to attend higher education.320 Institutions need to support students’ adaptation to overcome this challenge.321 Evidence from the institutional website survey, presented in Figure 4.12 indicates low coverage of the various integration activities considered.322 Barely more than half of the universities had information on just four of the six activities. Only welcoming activities were implemented by a majority of the IPs, while much fewer institutions engaged in the other relevant activities.

316 Sarrico and Rosa, “Student Satisfaction with Portuguese Higher Education Institutions: The View of Different Types of Students.” 317 IP de Setúbal, “O Abandono Nos Cursos de Primeiro Ciclo.” 318 Ibid. 319 Pisco Costa et al., “Identificação Das Causas Do Abandono Escolar Na Universidade de Évora.” 320 Ana Mouraz and Armando Sousa, “An Institutional Approach to First-Year Adjustment: The ‘Projeto FEUP’ Case Study of a Portuguese University,” Journal of HIPanic Higher Education, 2015, 1–19. 321 IP de Setúbal, “O Abandono Nos Cursos de Primeiro Ciclo.” 322 DGEEC - Equipa de Estudos de Educação e Ciência, “Draft: Medidas de Promoção Do Sucesso Escolar Nas Instituições de Ensino Superior Públicas Em Portugal, Tal Como São Relevadas Nos Respetivos Sítios.”


111

The activities welcoming new students documented in the survey also range from specific programmes like the PFC described earlier, to sessions or receptions, to leaflets or online information, making this measure fairly superficial. There is no evidence as to how many orientation activities include steps to introduce incoming students to the extracurricular opportunities available on campus.323 One key issue identified by an interviewee is that student may arrive at different times of year or choose not to participate in activities.

Figure 4.12: Proportions of public institutions providing relevant information on orientation activities for new students on their website

Formal student integration or orientation may also be handled by student liaison offices and/or formal student associations.324 Interviewees noted the importance of student associations in facilitating student orientation. The control of formal student groups over integration is a potential challenge, but while still at times mismanaged remains broadly preferable to informal orientation. The longstanding and entrenched practices of informal integration (called praxe in local parlance) have been a serious problem in Portugal that has received considerable media attention.325 Activities can include degrading hazing and be physically dangerous to the point where students have died. Senior students who have not been strong performers in higher education often lead these activities and can establish norms that discourage students from studying hard to achieve better results, encourage absenteeism, and undermine student well-being. It seems that these leaders may include some of the inactive students identified earlier.

323 Mouraz and Sousa, “An Institutional Approach to First-Year Adjustment: The ‘Projeto FEUP’ Case Study of a Portuguese University.” 324 Ibid. 325 Ibid.

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Welcoming of new students

Tutoring programmes by faculty

Mentoring Programmes by older students

Training sessions in study methods and time or resourcemanagement

Student guide (paper or online, with information on thecampus, study programmes, faculty members,

regulations)

Statement of rights and responsibilities

IPs Universities


112

The Government of Portugal has sought to reinforce institutional responsibility, concentrated with rectors, over student orientation and the hazing challenge. In the fall of 2016, the MCTES Minister sent a strong letter to higher education institutions indicating how various praxe activities are contrary to Portuguese values and the basic respect for personal rights.326 There may be relatively little that institutions can do at a certain point, however, when praxe activities take place in public spaces. One institutional representative noted that they encourage faculty to give students work in the first week to make them busier. The Minister is supporting scientific and cultural institutions to engage in orientation activities, but these efforts may reflect a somewhat naïve understanding of students’ psychosocial transition entering higher education. There is little question that orientation activities involving humiliation and degradation of students, let alone endangerment, are unethical and contrary to objectives of increasing student motivation and study success. More respectful traditions of student social integration, however, can help build a sense of belonging for students as they transition into higher education and some interviewees noted that some praxe activities help in this respect. One interviewee also noted he does not observe academic differences between students who do and do not participate in praxe. Notwithstanding these challenges, some institutions operate very developed orientation programs that appear likely to strongly support students’ successful integration into student life and subsequent study success. The Projeto-Faculdade de Engenharia da Universidade do Porto course is an example of a strong integration program that integrates social and academic elements, and is described in Box 1.327 Interviewees at one IP indicated that their institution is seeking to strengthen vocational elements in student integration so that students are aware from the beginning of their programmes about the careers they are being prepared for. The other activity of particular note is study and life skills training programmes, such as the Universidade de Aveiro’s “strategies for promoting academic success” class. Interviewees indicated that their efforts with such programming have confronted important challenges. In one case, reaching the students most in need was difficult as participants in the programme were typically more motivated and successful students. Representatives from another institution also reported low uptake. Multiple institutions expressed interest in providing these supports online.

3.3.2 Limited Student Engagement In surveys, students report relatively positive relationships with their peers. For example, at the IP de Setúbal, Licenciatura students who dropped out were positive (90%) about relationships

326 Manuel Heitor, “O Ingresso No Ensino Superior Tem Sido Sistematicamente Marcado Por Práticas Contrárias Aos Ideais de Liberdade, Crítica e Emancipação Dos Jovens,” September 2, 2016, http://www.esepf.pt/a_nots_16/docs_nots/carta_mctes_2016.pdf. 327 Mouraz and Sousa, “An Institutional Approach to First-Year Adjustment: The ‘Projeto FEUP’ Case Study of a Portuguese University.”


113

with other students, as were Masters students.328 329 Another study found that mature students reported a “fear of rejection” by younger students, but this fear generally faded over time as students were more exposed to one another and became socially integrated.330 Nevertheless, there are reasons to be concerned about whether Portuguese higher education institutions are doing enough to foster social connectedness effectively.

Box 1: Projeto-Faculdade de Engenharia da Universidade do Porto course (PFC) The PFC is a compulsory programme organised centrally by the faculty and applied across its programmes. It is comprised of an initial orientation week that students spend exclusively engaged with the course, followed by a half semester of weekly two-hour sessions in parallel with other engineering coursework. First week activities include receptions, training sessions building skills required for the study programme, visits to campus facilities, and some engineering-related technical work. Later programming focuses on the preparation of three group projects that include an array of tasks that will be repeated over the course of the study programme: a written report, a poster, and an oral presentation. In recognition of the time required and importance of the programme, students receive credits for successful completion. Professors supervise the programme, developing the project assignments and attending roughly half the programme session time, with the balance overseen by more senior students. The programme also once included an element of faculty mentoring, but this was dropped due to faculty disinterest. The Universidade do Porto PFC programme clearly is providing some academic integration, but it also aims to support social integration. Groups are assigned to connect students who previously did not know each other. Perhaps even more importantly, the course is mainly facilitated by monitors who are older students hired by the university. These older students should provide information from their recent experience beginning a study programme and also serve as positive role models. The programme is also connected to the university’s counselling services and the student association through a formal partnership, as well as a clubs fair that aims to introduce newcomer students to the extracurricular opportunities on campus. In surveys with over 500 students, students report very high satisfaction in particular with the monitors component of the programme, with positive responses on a series of questions ranging between 79% and 92%. Fully 80% of the students indicated monitors became someone students could rely upon. The surveys indicated that the programme was more important to female and younger students. Over 90% of students participating in the programme were enrolling in university directly from secondary school.

328 IP de Setúbal, “O Abandono Nos Cursos de Primeiro Ciclo.” 329 Ibid. 330 Santos et al., “Academic Success of Mature Students in Higher Education - A Portugese Case Study.”


114

In our findings on student satisfaction with different non-academic processes and services, among the lowest figures were reported for sport facilities, extra-curricular activities, student housing, leisure spaces, student organisations and clubs, and cultural programs. Moreover, roughly half (53%) of students in student halls reported being satisfied or very satisfied with their living situation in the 2011 Eurostudent survey, well below the median figure of 59%.331 Many of these areas showed modest gaps, however, because students also judged them to be less important. For students’ sense of belonging, however, these elements could be very important, which means the low levels of satisfaction really should be concerning. There are strong further indications of weaknesses in extra-curricular activities for example. Only 14.2% of students who had dropped out from the IP de Setúbal had participated in extra-curricular activities.332 Mouraz and Sousa note that at large institutions there may be insufficient extracurricular opportunities for all newcomer students.333 Institutions can also offer jobs to engage students. Interviewees from one institution indicated that it provides internships for students to pursue “research activities” related to their study programme to similarly get a sense of their career, with modest compensation.334 For example, students in library sciences operate the library for some of the time. This institution also hires recent graduates to deliver sign language interpretation to students who need it. Students conducted the phone interviews for the IP do Porto survey of students who had dropped out. However, another institution indicated that considerable legal ambiguity undermines their ability to offer these kinds of jobs on campus. Tuition fees collected are technically state revenue, and so providing a tuition waiver of any kind could be interpreted as public officials withholding state revenues. Technically institutions are permitted only to hire students for “research” projects. This is highly unfortunate in undermining student engagement on campus and students’ financial positions, given also that higher education institutions would likely be relatively accommodating of students’ academic commitments compared to other employers. There may also be many engagement related jobs that students could more effectively fill than anyone else, such as outreach activities with prospective students. The low share of students living in student dorms in Portugal also presents a challenge for student engagement. Student housing can create a community for students, especially helpful for those attending away from their family home. Finally, small institutions may be expected to have advantages in student engagement. However, one interviewee indicated that this was not so much the case because these institutions in

331 Orr, Gwosc, and Netz, “Social and Economic Conditions of Student Life in Europe: Synopsis of Indicators - Final Report - Eurostudent IV 2008-2011.” 332 IP de Setúbal, “O Abandono Nos Cursos de Primeiro Ciclo.” 333 Mouraz and Sousa, “An Institutional Approach to First-Year Adjustment: The ‘Projeto FEUP’ Case Study of a Portuguese University.” 334 Gabinete de Planeamento, Projectos e Desnvolvimento, “Estudo de Caso: Abandono Escolar 2012-13 (ESE,

ESMAE, ESEIG, ISCAP, ESTGF, ESTSP e APNOR).”


115

Portugal mostly recruit students from away, and many of these students return home very frequently. Consequently, the students are living a “parallel life” in the university town and their home town. Weekends notably can be very inactive around the university. Some institutional representatives indicated that they were pursuing interesting initiatives to address these challenges. In one case, they are taking multiple steps to try to develop more housing and work spaces on campus so that students’ lives would be more centred at the institution. In another case, the institution was looking to develop a diploma supplement that could recognise student engagement in sports activities and volunteering, as well as participation in workshops on time management, study skills and other life skills.

3.3.3 The Quality of Student Services is Uncertain at Best The institutional website survey found among the strongest coverage of different student services among the activities that it considered. However, this provides little if any indication as to the quality of these services. Some of the largest gaps in student satisfaction relative to importance from the system student satisfaction survey related to student services. There were important gaps in terms of care in service delivery, financial aid services, and student health services. Of course, these findings are relatively old. Surveys of students who dropped out provide some more recent indications of service quality as assessed by students. Both Licenciatura and Masters students who dropped out from the IP de Setúbal were most negative about support and administrative services, with the proportions of positive and negative evaluations almost even among Licenciatura students. However just 3.2% of the Licenciatura students indicated psychological supports could have helped them to persist. At the Universidade de Évora, roughly 20% of students who dropped out cited support and counselling services as a factor in their decision to drop out and.335 More specifically, students felt supports were insufficient and procedures were unclear. Among those who indicated they could have been helped to stay, 23% indicated various university services would have helped and 10% psychological counselling. Oliveira and Temudo also found that services generally are not adequate for the specific needs of working students.336 These studies suggest that student services could be considerably improved, but not necessarily that their inadequacies are a major contributor to dropout. One step that may help in this area is the progressive advancement of quality assurance into this area. A3ES has naturally focused on degree programmes in its activities to date, but could adopt activities addressing student services moving forward. Visited institutions were pursuing a number of different activities to strengthen services. One institution was concerned that students and even staff may not be aware of many of the services available on campus and had created an email account that students can contact to seek support

335 Pisco Costa et al., “Identificação Das Causas Do Abandono Escolar Na Universidade de Évora.” 336 IP de Setúbal, “O Abandono Nos Cursos de Primeiro Ciclo.”


116

with whatever challenge they face (the institution is tracking the subjects of these emails too), and placed an emphasis on connecting students with services during orientation.

3.3.4 Supports for Students with Disabilities are Underdeveloped Data relating students with disabilities to dropout is even more limited than data on the numbers of students with disabilities overall. At the IP de Setúbal, 17% of Licenciatura students who dropped out cited health or personal problems as key reasons, which provides a somewhat vague measure.337 Regardless, there is little reason to believe disability would not present at least a similar challenge for study success in Portugal as in other countries. It seems almost certain that difficulties in accommodating students with invisible disabilities (learning disabilities and mental health challenges) is an underappreciated factor in dropout. In the student satisfaction survey mentioned earlier, the largest gap was respecting disability services, which students considered to among the most important elements of higher education delivery. The website survey suggested that disability services are absent at most IPs, even using a loose definition of disability services. This is particularly noteworthy given evidence from other countries would suggest that IPs likely have higher shares of students with disabilities than universities, in part because they may face greater academic difficulties.338 Our interviews indicated that in higher education the Portuguese government has focused narrowly on disabilities as mobility or sensorial impairments. Some institutions appear to be further ahead in addressing a wider range of disabilities, including learning disabilities. However institutional efforts are highly uneven. Strong urban institutions are well ahead in supporting these students, in fact a large share of students with disabilities change regions to attend higher education likely to access these institutions. According to an interviewee, this is a positive outcome because these institutions are best positioned to help these students to be successful, but accepting the students also takes the pressure off other institutions to become accessible. Other challenges include little advanced notice of students and their specific needs at admission and requirements for students to repeatedly certify their disability to access different services. The Government of Portugal has launched a working group bringing together people from across the system to seek strategies to enhance accessibility for students with disabilities. There are also many institutional actors working hard to support students with disabilities. There appears to be considerable promise of progress to come.

337 Ibid. 338 Baur et al., “Disable the Label: Improving Post-Secondary Policy, Practice, and Academic Culture for Students with Disabilities.”


117

3.4 Failure to Assist Students at Risk of Dropping Out We have cited the Universidade do Porto Engineering Faculty’s student integration programme as in many ways a model.339 The key weakness of this programme though was that students who did not pass the academic course were only likely to be asked informal questions. There was no formal or mandatory counselling, nor would the students receive any other targeted supports or even have their access to further courses limited. The very weak approach to students who fail the course is very much contrary to the purpose of strengthening students’ study success, but seems reflective of a broader pattern in Portuguese education even at the compulsory levels, where excessive rates of student failure have been an accepted norm. Many students dropping out may not ever communicate with their institution. Half of the Universidade de Évora students who dropped out did not speak with anyone about their decision – just one-fifth spoke with professors and 6.3% engaged with a university service. Surely not all students at risk can be helped by their institution, but it seems that many could be. At the Universidade de Évora, more than half (56%) the students who dropped out indicated their institution could have helped them to continue, as did 34% of Licenciatura drop-outs and 38% of Masters dropouts at the IP de Setúbal.340,341 Many of our findings, such as study failure statistics, indicate institutions could identify and contact students who are at risk of dropping out to identify their challenges and provide support. Some interviewees supported this view. The only current examples we have found in the literature of institutions reaching out to students at risk of dropping out were an informatics system at the Universidade de Lisboa Instituto Superior Técnico (based on low academic performance) and a commitment to building a permanent observatory to coordinate actions at the Escola Superior de Desporto de Rio Maior of the IP de Santarém.342 In interviews, most institutions did report centralised processing of dropout so that they could reach out to students and encourage them to change their minds, but of course many students do not dropout formally. The IP de Setúbal has a programme of outreach to contact students who only have a few credits left in their degrees regarding their particular needs.343 Only the private institution representative indicated they were consistently reaching out to students at risk, although this appeared to be more informal than systematic.

339 Mouraz and Sousa, “An Institutional Approach to First-Year Adjustment: The ‘Projeto FEUP’ Case Study of a Portuguese University.” 340 Pisco Costa et al., “Identificação Das Causas Do Abandono Escolar Na Universidade de Évora.” 341 IP de Setúbal, “O Abandono Nos Cursos de Primeiro Ciclo.” 342 DGEEC - Equipa de Estudos de Educação e Ciência, “Draft: Medidas de Promoção Do Sucesso Escolar Nas Instituições de Ensino Superior Públicas Em Portugal, Tal Como São Relevadas Nos Respetivos Sítios.” 343 Ibid.


118

3.5 Uneven Institutional Commitment to Study Success Obviously, this report is considering completion in the higher education system in general. Our evidence in Chapter 2 indicated that study success rates differ between institutions, but this could result in large part from the different student populations that these institutions are recruiting from, and the difference in study programmes that they offer. We find indications of differences between the strategic initiatives pursued by different institutions however. The website-based review of study success related activities mentioned throughout this and the previous chapter indicates important differences in services among higher education institutions (Figure 4.13).344 It indicates that most institutions cite study success as a priority in their strategic plan and/or are planning or reporting on activities related to the issue. An even larger share of universities are explicitly pursuing programmes or projects on the issue, whereas only a small minority of IPs are doing so. This similar pattern of higher emphasis at universities than IPs is apparent in various indicators of monitoring and research.

Figure 4.13: Proportions of public institutions providing relevant information on strategic activities relating to study success on their website

Driven largely by a recognition that the institution was serving many students at particular risk of dropping out, including mature and working students, one of the IPs we visited had prioritised study success in its strategic plan and appointed an administrative leader responsible for addressing the challenge. The strategy revolved around: (1) reflection and discussion of the issue; (2) collecting and publishing data on study success which is used to inform institutional decision-making; (3) strengthening training of academic staff; and (4) improving pedagogy through problem-based learning, case studies and internships.

344 Ibid.

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Strategic plan with relevant items

Plan/Report of activities with relevant items

Programmes and projects on study success and dropout

Indications of monitoring of study success/dropout

Participation in networks concerned with study successand dropout

Documents or studies on study success or dropout

Survey of entering students to understand socioeconomiccircumstances and goals

IPs Universities


119

We observe significant differences by institution in terms of the 30 services tracked in the website survey. The strongest five IPs provided information on their website regarding more than two-thirds of the thirty services, and twelve of the 26 institutions had information on at least half. On the other hand, four institutions provided information on one-third or fewer of the indicators. Six of the 13 universities provided information on at least two-thirds of the activities, and the Instituto Universitário de Lisboa had relevant information on 27 activities. Of particular concern, some of the activities or services absent at certain institutions are required by law. It is important to reiterate that these findings are based on institutional websites. They may therefore reflect as much website quality as institutional activity in some cases. More broadly, interviewees suggested institutional quality varies more based on their location than their status as universities or IPs. This perception seems to be reflected in the higher student demand for programs in cities as described in the previous chapter, but no such pattern is clear in the study success activities survey. Some large institutions have lower dropout rates however when studies in other countries have indicated that students at smaller institutions are more likely to complete due to stronger engagement, which suggests Portugal’s smaller institutions may be falling particularly short of their potential.345 We do not have data on private institutions’ services. Our dropout data indicated wide differences in performance, and this was the impression of interviewees. Some institutions are clearly demonstrating leadership for the whole system. The Universidade do Porto is a key driver of some network activities, having created a Facebook group aiming to support the integration autonomy and learning of first year students in particular, which at least 14 institutions now participate in. The Universidade de Aveiro and the Instituto Superior de Engenharia de Lisboa (Universidade de Lisboa) also participated from 2011-2014 in a network of Latin American and European institutions dedicated to reducing dropout. Many faculty participate annually in the National Congress on Pedagogical Practices in Higher Education (Congresso Nacional de Práticas Pedagógicas no Ensino Superior, CNAPPES). The Universidade de Évora, the IP do Porto and the IP de Setúbal have confronted the problem with in-depth studies.

4. International Experiences Vossensteyn et al. document a host of different policies pursued across Europe that have sought to strengthen the quality of education delivery to promote study success.346

345 Vossensteyn et al., “Dropout and Completion in Higher Education in Europe.” 346 Ibid.


120

4.1 Restructuring Programme Options Much like Portugal with its introduction of TESP programs, France, the Netherlands, Norway and Serbia have sought to introduce new degree structures that better meet students’ preferences. In particular, many of these programs are more professionally oriented, as with Portugal. Other measures have emphasised granting students greater flexibility to design a study program that best meets their needs, through fewer compulsory courses particularly in first year, and later declaration of majors. France has introduced bachelor programs that allow students to choose their specialisation in second year or even later, aimed at students who are uncertain what study program to pursue. A host of other countries347 have placed a similar emphasis on the flexibility of study programs especially through enhancing credit transfer and recognition. Italy, for example, views its National Qualifications Framework as consistent with its effort to raise completion. In terms of evaluated efforts, Flanders’ Decree Flexibilité has meant that students can take courses at different institutions without discontinuing at their primary institution. However, evaluations have reported slightly negative results for study success (time-to-degree and dropout) because the program is complex, it makes students’ learning less transparent, and it makes targeted interventions to support struggling students more difficult, while this program has also raised administration costs and teacher workload. Greece and Denmark actually emphasise restrictions on students’ program choices in their efforts to accelerate completion. Greece has established a maximum time of enrolment, targeting in particular large numbers of inactive students. Denmark requires that students enrol for 60 credits per year and does not permit students to withdraw from examinations.

4.2 Improving Academic and Social Supports Strengthening student support services is important for a large number of countries.348 These services include psychological and health supports, career counselling, housing and activities around social integration. In most cases the services are driven by the institutional level, but France and Denmark have national level agencies. In France, the Centre regional des oeuvres universitaires et scolaires (CROUS) administers bursaries, housing, food services and cultural activities across a number of institutions. France, Hungary and the Netherlands have policies that require institutions to deliver certain student services, with institutions retaining discretion over how these services are organized. Aligned with the introduction of binding study advice, the Netherlands promoted the improvement of academic counselling and other services for first year students. Hungary has reported improvements in study success since introducing student

347 Bulgaria, Denmark, Flanders, Germany, Hungary, Italy, Lithuania, Switzerland and Macedonia. 348 Flanders, France, Germany, Greece, Hungary, Iceland, Ireland, Italy, Latvia, Lithuania, Macedonia, Montenegro, Netherlands, Norway, Poland, Romania and Serbia.


121

services. Research from the US finds that expenditures on student services in particular contribute to improved study success.349 Another set of countries350 have introduced new curricular measures that include changes in assessment procedures, compulsory extra study courses, and even an additional compulsory semester or bridging year in advance of the first year of study. Many of these programs are anchored on continuous assessment, to help students gain feedback and make early choices to switch or even drop out. As of 2009, Austrian universities are required to deliver a compulsory study phase for entering students to provide an introduction to study and research methods, and make students reflect on their study choice. Evaluations did not find that the program reduced dropout, but that it helped dropouts make better decisions subsequently. In the Netherlands, the detailed admissions processes described in the previous chapter are accompanied by a process of “binding study advice” at the end of first year, through which institutions will dismiss students who are judged as very unlikely to complete their programs successfully.

4.3 Improving Information Collection and Dissemination Clearly Portugal monitors students’ educational progression and later careers, and other countries.351 Other countries also emphasize this in their completion strategies, notably to facilitate the identification of students at risk and to adjust study programs to meet labour market demands. Some countries352 have sought to integrate study success metrics into their quality assurance system. Institutions have been required to report on standard measures of study success and measures seeking to promote study success. The data is published in some cases, in some cases the indicators inform quality assurance accreditation for institutions or programs. Finally, four countries353 have created structures to disseminate good practices in promoting study success. In some cases, these measures include competitions, and there is a perception that they have strengthened the focus of institutional leadership on teaching and learning.

349 Vossensteyn et al., “Dropout and Completion in Higher Education in Europe.” 350 Austria, Estonia, France, Germany, the Netherlands, Romania, Serbia and Sweden. 351 Finland, Flanders, Hungary, Ireland, the Netherlands, Norway and Sweden. 352 Croatia, Flanders, France, Hungary, Ireland, Italy and Montenegro. 353 England, Finland, Germany and Ireland.


123

Chapter 5: Financing This chapter analyses the adequacy of financial support to Portuguese higher education institutions and students and how the Government of Portugal’s funding and financial aid programmes affect institutional commitment to study success and student motivation. Portuguese higher education institutions are funded through a combination of government grants and private funds – namely tuition fees. The Government of Portugal also offers needs based grants to students and provides backing for private student loans, but take-up of loans is minimal. In addition, close to half of students pursue paid employment, and legislation and policies provide for many accommodations for these students. There is no question that institutional funding in Portugal is limited by international standards. However, there are also weak incentives in the institutional funding structure, as core funding is not tied to performance. The Government has provided only limited targeted funding for key initiatives to promote study success. On the student side, many find it a struggle to balance studies and employment. This likely relates in part to the insufficiency of student financial aid. Financial aid eligibility criteria may also undermine completion, including by making it very difficult to regain eligibility once it is lost for academic reasons. Finally, Portugal provides negligible disability-related financial aid, such that it is certain that most students with disabilities do not receive any targeted support. International jurisdictions have sought to promote study success through institutional funding and financial aid measures. Governments have provided targeted institutional funding for support services, or linked funding with performance through a funding formula or performance agreements. Similarly, governments have expanded financial aid for students with need, but also introduced stronger conditions on funding amounts or repayment often based on study progress.


124

Chapter Guide

1. Analytical Framework .................................................................................................... 126

2. Lay of the Land .............................................................................................................. 126 2.1 How does Portugal fund higher education? ........................................................................... 126

2.1.1 Public Funding .......................................................................................................................... 127 2.1.2 Tuition Fees .............................................................................................................................. 127

2.2 How do students afford to study? ......................................................................................... 128 2.2.1 Expenditures ............................................................................................................................ 128 2.2.2 Income...................................................................................................................................... 129 2.2.3 Public or Government-Backed Financial Aid............................................................................ 130 2.2.4 Institutional Financial Aid ........................................................................................................ 135 2.2.5 Student Employment ............................................................................................................... 135

3. How May Higher Education Financing in Portugal Contribute to Lower Study Success? ... 136 3.1 Weaknesses in Institutional Funding ..................................................................................... 136

3.1.1 Low Institutional Funding by International Standards ............................................................ 136 3.1.2 Weak Incentives in the Institutional Funding Structure .......................................................... 139 3.1.3 Limited Targeted Funding for Key Initiatives ........................................................................... 140

3.2 Student Financial Resources ................................................................................................. 140 3.2.1 Students Struggle to Balance Studies and Employment .......................................................... 141 3.2.2 Insufficient Student Financial Aid ............................................................................................ 142 3.2.3 Financial Aid Eligibility Criteria May Undermine Completion.................................................. 144 3.2.4 Negligible Disability-Related Financial Aid ............................................................................... 146

4. International Experiences .............................................................................................. 146 4.1 Institutional Funding ............................................................................................................ 146 4.2 Student Funding ................................................................................................................... 148

List of Tables

Table 5.1: Sources of income among students not living with parents, by education background, 2011 ....................................................................................................................... 130 Table 5.2: Exceptions to eligibility criteria d), e) and f) for the Bolsa de Estudo ..................... 132 Table 5.3: Ranking of Portugal’s Higher Education Spending among OECD Member Countries for which Data is Available, 2013 .............................................................................................. 137 Table 5.4: Expenditure on educational institutions relative to Portugal’s national wealth (all tertiary), 2013 ............................................................................................................................. 138 Table 5.5: Public expenditure on tertiary education as a percentage of total public expenditure in Portugal, 2013 ................................................................................................... 138 Table 5.6: Summary of results of conditional funding envelopes in Austria, the Netherlands and Norway ................................................................................................................................ 147


125

List of Figures Figure 5.1: Student housing situations, 2011 ............................................................................ 129 Figure 5.2: Sources of Portuguese students’ income, 2011 ...................................................... 130 Figure 5.3: Proportion of student loan recipients who receive funding from other sources, 2009 ............................................................................................................................................ 135 Figure 5.4: Annual Tertiary Expenditure per Student by Educational Institutions in Portugal, 2013 ............................................................................................................................................ 137 Figure 5.5: Dropout of first-time first-year students based on receipt of financial aid and access stream, 2012 ................................................................................................................... 143 Figure 5.6: Causes of rejection of applications for the Bolsa de Estudo at the IP do Porto, 2011-2015 ............................................................................................................................................ 144


126

1. Analytical Framework The financial resources made available to institutions and to students are important. These can affect both these actors’ ability to make choices that support or do not study success, and their incentives to do so. From Vossensteyn et al.’s international literature review, we identify four key elements that inform the analysis in this chapter:354

1. Institutional commitment to study success: Institutional commitment is explained in greater detail in the previous chapter and related to all the challenges in effective delivery of higher education that we discussed. This chapter focuses on how institutional funding shapes incentives for institutional commitment to study success.

2. Adequacy of funding to institutions: Institutions must have sufficient resources or else they will be unable to invest in the various elements of effective delivery described in the previous chapter.

3. Adequacy of funding to students: Commentators often attribute critical importance in

determining higher education study success to student financial support and tuition fees. While little empirical evidence bears out the view internationally that student fees are overwhelming contributors to higher education study success rates, it is reasonable to assume that students may be unable to complete higher education if they simply cannot pull together the money to pay for their fees and other associated costs. Students also may only be able to study if they work for long hours in paid employment, which undercuts their ability to be successful. This study is not able to explore in depth the adequacy of student financial aid in meeting Portuguese students’ financial need, which could be the subject of a review all in itself. Nevertheless, we can provide general impressions regarding the adequacy of Portugal’s current student financing regime.

4. Impacts on student motivation: Policies on fees and financial aid include (re)payment

terms or eligibility conditions that relate to students’ progression in their studies.

2. Lay of the Land

2.1 How does Portugal fund higher education? Portuguese higher education institutions are funded primarily through government grants, and secondarily through student tuition fees.

354 Hans Vossensteyn et al., “Dropout and Completion in Higher Education in Europe” (European Union, 2015).


127

2.1.1 Public Funding Government grants to institutions can be generally divided between grants for teaching and grants for research.355 Core teaching grants this year amounted to EUR 1.002 billion, with roughly 70% of those funds going to universities. An additional EUR 75 million in European funding was provided to IPs on a competitive basis to support vocational courses and practice based research. Institutions also collected approximately EUR 372 million in research grants. Legally the Government of Portugal is required to operate a funding formula, which should include criteria relating to inputs (staffing levels, staff composition, and personnel qualifications), outputs (efficiency of teaching, science and administration), and external quality assurance.356 Such a formula was in place from 2006 to 2008, but then suspended. Portugal now operates a historical funding model, where the distribution of funding between institutions is constant and the amounts are adjusted by an across-the-board increment (say increased or reduced by 2%). The Government of Portugal is also required to establish programme contracts and institutional development contracts with funded institutions. Programme contract elements would include elements to promote academic staff, to support teaching staff development, to strengthen online learning and other information tools, etc. Institutional development programmes relate more to institutional management and assets and should be five years in duration. To our knowledge, such agreements are not currently in place either. Private higher education institutions generally do not receive public funding. Under the law, however, they may receive funds for social support to students (social support is explained later in the chapter), projects consistent with government priorities, and faculty training.357 The Government of Portugal has provided modest one-time grants in recent years to support study success.

2.1.2 Tuition Fees Portuguese higher education institutions also collect tuition fees from students. System-wide fee data is remarkably difficult to collect, given notably how strictly they are regulated by government, but maximum fees in 2016 were approximately EUR 1063.47.358 Under the law they must exceed 1.3 times the national minimum salary, but the maximum amount is set annually. International students may be charged fees to cover their full cost of training, institutions are

355 Interview notes: March 17, 2017. 356 Assembleia da República, “Lei n. 37/2003, de 22 de Agosto: Estabelece as Bases Do Financiamento Do Ensino Superior” (2003). 357 Ibid. 358 Samuel Silva, “Estudantes Pagam 300 Milhões Por Ano Em Propinas,” PÚBLICO, March 10, 2017, https://www.publico.pt/2017/03/10/sociedade/noticia/estudantes-pagam-300-milhoes-por-ano-em-propinas-1764675.


128

free to establish graduate tuition at rates they deem appropriate.359 Private higher education institutions have full discretion over their tuition fees. In 2013, one third (32%) of funding to tertiary education in Portugal came from private sources.360 Of this, 32 percentage points was provided from household expenditures, and 10 percentage points from other private entities. In contrast, private sources accounted for just 22% of funding to tertiary education in the EU-22. The extent of private funding is partly a result of Portugal’s relatively sizable private system, public institutions are much more dependent on public funding.

2.2 How do students afford to study? Our observation that Portuguese higher education institutions are relatively reliant on student fees poses the question of how students afford to study in Portugal. This is especially true given that students’ costs extent well beyond tuition. We can answer this question first by considering students’ expenses and resources. The 2011 EuroStudent survey provides the best available data, but it is important to note that of course these data are now somewhat dated and also that costs in particular almost certainly vary considerably between different parts of the country. 361

2.2.1 Expenditures Aside from fees, the major higher education student expenditure is housing. As indicated in Figure 5.1, almost half of all Portuguese students live with parents, and this is especially common among those with higher education backgrounds.362 The proportion living at home is relatively high by European standards, though lower than Italy (73%) and neighbouring Spain (51%). Portuguese students living at home spent the lowest proportion of their monthly expenditures on living costs among participating countries in the EuroStudent Survey (54%).363 The balance (46%) was spent on study-related costs. For students living away from home, Portugal had the second lowest proportion of spending on living costs (66%) and the third highest proportion for study-related costs (34%). These results reflect in part Portuguese Licenciatura students spending 14% of their monthly income on fees, a figure well above the median among countries in the study but well below some other countries where students nevertheless spent relatively more on living costs.

359 Assembleia da República, Lei n. 37/2003, de 22 de Agosto: Estabelece as bases do financiamento do ensino Superior. 360 OECD, “Education at a Glance 2016: OECD Indicators” (Paris: OECD Publishing, 2016). 361 Dominic Orr, Christoph Gwosc, and Nicolai Netz, “Social and Economic Conditions of Student Life in Europe: Synopsis of Indicators - Final Report - Eurostudent IV 2008-2011” (Bielefield: W. Bertelsmann Verlag, 2011). 362 Ibid. 363 Ibid.


129

Figure 5.1: Student housing situations, 2011

Among students not living at home, parents/partners/others paid a relatively low proportion of monthly expenditure at 23% (including transfers in kind), compared to the median among participating countries of 25.5%.

2.2.2 Income Having addressed costs, we can now turn to students’ income. In 2011, without accounting for in kind support, students living with their parents enjoyed average monthly incomes of 641 euros, equal to the fifth highest figure in Europe and well above the median figure of 426 euros in Germany. Portuguese students not living with parents (accounting for in kind transfers) had median incomes of 1 034 euros, above the median of 850 euros again in Germany. Income was unequally distributed (Gini of 0.32) roughly to the same extent as the European median (Gini of 0.31). Income differences by gender were negligible, however, though male students reported a slightly higher proportion of their income from employment (45%) than female students (41%). Figure 5.2 presents 2011 data on the sources of Portuguese students’ income.364 Almost 90% of income comes from students’ families or partners or is self-earned (presumably from employment), relatively very high figures among the European countries participating in the survey. Meanwhile the shares of Portuguese students’ income from public sources was among the lowest: the median country figures were 14% for students living with parents and 12% for students not living with parents. These patterns seem to be consistent over time: the figures for all higher education students are essentially identical for Eurostudent data in 2005 and 2011.365

364 Ibid. 365 Manuel Heitor, Hugo Horta, and Miguel Leocádio, “Enlarging the Social Basis of Higher Education: Lessons Learned from Extending a Social Support System with a Risk-Sharing Loan Scheme in Portugal,” Technological Forecasting and Social Change 113 (2016): 319–27.

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

All Students

High education background (ISCED 5-6)

Low education background (ISCED 0-2)

Master’s students

With Parents With (an)other person(s) Alone With partner/children


130

Figure 5.2: Sources of Portuguese students’ income, 2011

Income excludes in-kind transfers for students living with parents, but includes these transfers for students not

living with parents.

Table 5.1 shows that students from different socioeconomic backgrounds have very different sources of income.366 Students from higher education backgrounds receive more support from parents or partners. Fewer students from lower education backgrounds receive support from families or partners and this makes up a smaller share of their income. Portugal also has among the largest gaps in the share of students working based on the education background of parents among countries in the Eurostudent survey, after just Romania and Italy.367 Finally, our figures suggest public support is progressive, favouring students from less educated families. Table 5.1: Sources of income among students not living with parents, by education background, 2011

Source of income

Proportion of students who receive income

Income source as a share of total Income

Low Education Background (ISCED 0-2)

Family/Partner 59% 43%

Public 25% 6% Self-Earned 53%* 46%

High Education Background (ISCED 5-6)

Family/Partner 81% 63% Public 14% 3%

Self-Earned 32%* 36% *These figures represent the proportion of students employed during term time, a slightly different measure than that used through the rest of the table.

2.2.3 Public or Government-Backed Financial Aid Portugal has two key public or government-backed sources of financial assistance to students: a needs-based “social support” programme of mostly non-repayable grants, and partially

366 Orr, Gwosc, and Netz, “Social and Economic Conditions of Student Life in Europe: Synopsis of Indicators - Final Report - Eurostudent IV 2008-2011.” 367 Ibid.

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Living with Parents

Not Living with Parents

Family/Partner Self-Earned Income Public Sources Other


131

government-guaranteed student loans.368 The Government greatly expanded the first programme and created the second following the OECD’s 2006 Review of the Tertiary Education System. The grant programme is considerably more important. Fully 20% of higher education students benefit from the social support system overall.369 Of those who receive financial support, 81% do so through non-repayable grants and just 19% receive repayable loans.

2.2.3.1 Social Support

Portugal’s social support programmes offer a package of grants, as well as some indirect supports, aiming to ensure students can access higher education regardless of social class and financial means. The Study Bursary (Bolsa de Estudo, BE) is the primary financial aid mechanism. The BE is delivered based on financial need, and can vary between EUR 1063 and EUR 5676.370 According to the MCTES, roughly 40% of students receive the minimum scholarship, which is enough to cover tuition alone. The funds are available to support students completing their first CET, Licenciado, Masters or CTESP degree at a public or private higher education institution. Money is deposited electronically every month into students’ bank accounts. From a study success perspective, eligibility criteria for the BE tied to study progression are key. The following are the most relevant general criteria:

d) The student must be enrolled in a minimum of 30 ECTS, except i. where fewer courses are required to complete their degree program, or

ii. they cannot enroll in 30 ECTS because this would violate rules for their thesis, dissertation, project or internship.

e) If they were enrolled in a higher education institution in a prior year, the student must have passed at minimum:

i. 60% of their credits, if they studied for more than 60 ECTS; ii. 36 ECTS, if they took between 36 and 60 ECTS; or,

iii. all their credits, if they took fewer than 36 ECTS. f) Accounting for prior registrations at their degree-level in higher education, the student

must be able to complete their study programme with a total number of annual registrations no greater than:

i. n+1, for programs with three years or less of theoretical duration; or, ii. n+2, for programs of more than three years of theoretical duration;

iii. where n refers to the theoretical duration of the program.

368 Heitor, Horta, and Leocádio, “Enlarging the Social Basis of Higher Education: Lessons Learned from Extending a Social Support System with a Risk-Sharing Loan Scheme in Portugal.” 369 Ibid. 370 OECD, “OECD Economic Surveys: Portugal 2017” (Paris: OECD Publishing, 2017), http://dx.doi.org/10.1787/eco_surveys-prt-2017-en.


132

Table 5.2 outlines exceptions to these criteria. As well, in special cases program eligibility will not consider registrations for years of prolonged serious illness that is duly proven, or other especially serious circumstances that are also proven (maternity/paternity, need to assist family member, physical or sensorial challenge reducing capacity by 60% or more). This can only apply in one year except in exceptional circumstances or where students have legal entitlements. Table 5.2: Exceptions to eligibility criteria d), e) and f) for the Bolsa de Estudo

Student characteristic

Clauses

d) e) f)

In CET and TESP Programs

Students have not previously received a bursary to attend a CET or TESP program they did not complete.

- Replaced by: Students are able to complete in the theoretical program duration

Change Study Programme

Waived, provided the student did not receive a bursary for the previous academic year

Years to complete increased by 1

Student Workers - - Years to complete increased by 1

Part-time students

- - Allowed time to complete the study programme under f) are doubled.

Interns (under certain laws)

Does not apply Does not apply

Does not apply

Students with physical, sensorial or other disabilities also have special eligibility for grants, conditional on a doctor certifying that they are impaired by “60% or more”, without regard to income.371 These students can receive supplementary funding of up to three times the maximum financial aid to acquire academically indispensable products or services. The social support system also provides indirect aid.372 Under the higher education financing law, this support can promote access to food services and housing, access to health services, support for cultural and sport activities, and access to other educational supports. Housing and food services supports are targeted to students who must travel to attend higher education, prioritising those with greater financial need. Rent in residence is approximately EUR 80 per month, and students with sufficient need can fully offset this subsidised rate through grants (students who do not live in residence are eligible for up to EUR 120 per month for housing).373

371 Ministério da Educação e Ciência, “Regulamento de Atribuição de Bolsas de Estudo a Estudantes Do Ensino Superior” (Diário da República, June 24, 2015), https://dre.pt/application/file/67591094. 372 Assembleia da República, Lei n. 37/2003, de 22 de Agosto: Estabelece as bases do financiamento do ensino Superior. 373 Interview notes, March 17, 2017.


133

2.2.3.2 Other Government Financial Aid Programmes The Government of Portugal also funds merit scholarships for distribution by each institution to one student of every 500 enrolled, worth five times the minimum monthly remuneration for the given year.374 These are not based on a needs assessment. Two further grant programs were introduced in 2014. The first was the Retomar (“Retake”) Program, which specifically aimed to raise completion rates. The program provided for EUR 1 200 bursaries for up to 3 000 students under the age of 30 who had dropped out but wished to recommence their study program, without a calculation of financial need. Uptake of the program was well below expectations, with only 482 applications and 195 scholarships provided in 2014-15, and even fewer applications (333) and scholarships (133) in 2015-16.375 Given this, the Government decided to cancel the Retomar Program and reallocate funds to support students to study specifically in digital themes in 2016. The second program was +Superior, which provided grants of up to EUR 1 500 to up to 1 000 students who were moving to study in regions with lower demographic growth and excess tertiary capacity.376 The scholarships were renewed automatically.377 Bursaries were allocated based on merit (students’ high school grades), without a financial needs assessment. Fully 991 scholarships were allocated in 2014-15, and 1 732 in 2015-16 (798 through renewal).378 Just 57% of bursaries were provided to economically disadvantaged families. The Government decided to change this program in 2016 as well. The +Superior programme is not being cancelled but redesigned, notably to provide more targeted support to disadvantaged students. Eligibility has been reduced to only students from economically disadvantaged families, but has also been extended to students in TESP programs, to students using other access streams than the CNA, to additional regions (Algarve, the Azores Autonomous Region and the Madeira Autonomous Region), and to students travelling between eligible regions. The program will also become open to students who have dropped out or who are transferring between institutions and study programs, and will prioritise supporting students using the M23 admissions stream and those accessing TESP programs. Finally, the total number of new scholarships to be introduced in the current year will be increased 29% to 1 320, and TESP and M23 students will be eligible for a 15% supplement to the base bursary.

374 Ministério da Ciência, Tecnologia e Ensino Superior, “Despacho n. 13531/2009” (2009). 375 Gabinete da Secretária de Estado da Ciência, Tecnologia e Ensino Superior, “Ciência, Tecnologia e Ensino Superior - Despacho Normativo N. 1-E/2016” (2016). 376 OECD, “OECD Economic Surveys: Portugal 2017.” 377 Secretário De Estado Do Ensino Superior, “Despacho 11306-C/2014, de 8 de Setembro,” accessed March 7, 2017, https://dre.tretas.org/dre/319144/despacho-11306-C-2014-de-8-de-setembro. 378 Gabinete do Ministro Ciência, Tecnologia e Ensino Superior, “Despacho 14447-A/2016, de 29 de Novembro” (2016), https://dre.tretas.org/dre/2806898/despacho-14447-A-2016-de-29-de-novembro.


134

2.2.3.3 Student Loans Portugal’s loan programme was created to complement the government’s grant programme, especially to serve middle income students who might not be eligible for grants. Loans are provided by private institutions, but are guaranteed by the Portuguese Government for up to a 10% default rate. By reducing institutions’ risk, the government guarantee allows institutions to provide the loans through a simplified process without requiring collateral from students and their families, at a relatively advantageous interest rate and term. This programme design permitted the government to introduce the programme despite very significant fiscal constraints. It was modelled after prior experiences providing similar guarantees to boost lending to small and medium enterprises. Initial public investment was minimal, amounting primarily to starting capital of 2% to 3% of the total fund.379 The loans are available to support all types of tertiary and post-secondary degree programmes, as well as post-doctoral training and participation in international mobility programmes. Loans values can range from 1000 to 5000 euros per year of study, up to a maximum of 25 000 euros over five years (depending on the length of the degree programme). The maximum amount aims to provide adequate support to allow students to cover their costs, without generating excessive indebtedness. The funds are provided in monthly installments deposited directly in recipients’ bank accounts. The loans are of the same general type as a mortgage. Repayment is collected in fixed installments over a fixed period of time, and not income-contingent. The repayment period is generally capped at two times the length of the student’s degree programme. The interest rate spread is capped at 1%. From 2007-2014, just over 5% of loan values per year went into default, which is very low considering one study of 44 student loan schemes found almost half had repayment rates below 40%.380 This is particularly noteworthy given this covers a period of financial and economic crisis in Portugal. Figure 5.3 presents data indicating that loans are generally a complement to other sources of financial support, including most exceptionally grants.381 This suggests that while students who take up loans generally come from middle income families, as was originally intended, these are generally still families reliant on needs-based financial aid.

379 Heitor, Horta, and Leocádio, “Enlarging the Social Basis of Higher Education: Lessons Learned from Extending a Social Support System with a Risk-Sharing Loan Scheme in Portugal.” 380 Hua Shen and Adrian Ziderman, “Student Loans Repayment and Recovery: International Comparisons,” Higher Education 57, no. 3 (2009): 315–33. 381 António Firmino da Costa et al., “Estudantes Do Ensino Superior e Empréstimos Com Garantia Mútua - Inquérito de 2009” (Lisboa: Centro de Investigação e Estudos de Sociologia, 2009).


135

Figure 5.3: Proportion of student loan recipients who receive funding from other sources, 2009

2.2.4 Institutional Financial Aid Institutions indicated that they also provide financial aid. Some of this aid is financed through external foundations. They target students based on merit or financial need, but indicate that they do not have enough funding to fully meet financial needs. Multiple institutional representatives also indicated that they have funding to support students in financial emergencies, as required under current legislation. In many cases, funding is tied to “research projects” as indicated in the previous Chapter.

2.2.5 Student Employment As noted earlier, Portugal is near the middle among European countries in terms of the share of students who work during term time at approximately 46%.382 As indicated, these financial resources are also very important to many students. The Government of Portugal has approved a Worker-Student Statute within the Labour Code to support these students and try to promote completion of education credentials.383 Workers who are attending any level of education in a programme of six months or greater are eligible. To claim the status, the worker-student must provide proof of student status to their employer, along with the schedule of their academic activities. The statute requires employers to adjust work hours to permit employees to attend courses, or allow students to reduce their hours of work without losing their basic terms of employment where possible, with additional flexibility for academic evaluation events.

382 Orr, Gwosc, and Netz, “Social and Economic Conditions of Student Life in Europe: Synopsis of Indicators - Final Report - Eurostudent IV 2008-2011.” 383 Assembleia da República, “Código Do Trabalho,” Lei 28/2016 § (2016), http://www.pgdlisboa.pt/leis/lei_mostra_articulado.php?nid=1047&tabela=leis&so_miolo=.

0%

10%

20%

30%

40%

50%

60%

70%

80%

Students with loans only Students with loans and grants

Family/Partner

Work

Grants/subsidies

Other sources


136

Worker-students must pass their year to retain their status. Passing the year is defined as passing at least half the courses in which they enroll. Exemptions to this apply where the student does not pass because of a work accident, work-related illness, or other significant health or parental event. All worker-student rights relating to hours of work and holidays end when a student has not passed their year. The student can regain these rights at the beginning of the subsequent year of enrolment, but if they fail again all worker-student rights are nullified. Worker-students must prove their study progress to their employer and the employer may request proof of the student’s attendance and progress. With respect to higher education institutions, worker students are not subject to requirements for a minimum course enrolment or academic suspension, nor any regulations that require minimal attendance of class sessions to pass.384 These students are also not subject to limits on the number of exams they can complete during the make-up evaluation period (época de recurso), and where institutions do not have a make-up evaluation period, worker-students have the right to a special time period for exams in all time periods where legally permissible.

3. How May Higher Education Financing in Portugal Contribute to Lower Study Success?

3.1 Weaknesses in Institutional Funding

3.1.1 Low Institutional Funding by International Standards Portuguese higher education institutions have relatively limited financial resources compared to their counterparts across the OECD and the EU-22. This is true in terms of per student spending, but also spending as a share of the country’s wealth and total government spending. Comparative data on Portuguese per-student higher education spending are provided in Figure 5.4, and Portugal’s rank within the OECD in Table 5.3.385 In 2013, Portuguese higher education institutions spent $11 106 per student as compared with $15 772 and $15 663 for the respective comparator groups. Funding classified for research and development (R&D), however, was higher than the median, leaving core education spending ahead of only Slovakia among OECD and EU-22 countries for which data are available.386 We cannot distinguish whether this reflects spending priorities or simply differences in accounting.

384 Assembleia da República, “Lei n. 105/2009 - Regulamenta e Altera o Código Do Trabalho, Aprovado Pela Lei n.o 7/2009, de 12 de Fevereiro, e Procede à Primeira Alteração Da Lei n.o 4/2008, de 7 de Fevereiro” (2009). 385 OECD, “Education at a Glance 2016: OECD Indicators.” 386 Among the nine countries that have lower expenditure per student for all services than Portugal, four countries did not provide data on expenditure for core educational services: Chile, Latvia, Mexico and Turkey.


137

Figure 5.4: Annual Tertiary Expenditure per Student by Educational Institutions in Portugal, 2013

* Portugal’s figures for tertiary expenditure for all services include some spending for post-secondary non-tertiary,

which means we may be understanding Portuguese higher education institutions’ funding disadvantage. If spending on post-secondary non-tertiary is included in tertiary spending for all services then Portugal’s rank falls to

31/34 in spending. This comparison would exaggerate Portugal’s funding disadvantage, however, given some of Portugal’s spending on post-secondary non-tertiary is also calculated under secondary spending, in fact this is

more the norm in the OECD statistics.

Table 5.3: Ranking of Portugal’s Higher Education Spending among OECD Member Countries for which Data is Available, 2013

Indicator Rank Number of Comparator Countries Expenditure for all services* 25 34

Expenditure for core educational services 24 25

Expenditure for R&D 14 32 Public expenditure, all institutions 27 31

Public expenditure, public institutions 28 31 Public expenditure, private institutions 16 29

* See note in previous figure regarding expenditure for all services

The Portuguese economy is smaller than the OECD and EU-22 average, which largely but not fully explains the more limited resources available to Portuguese higher education. Table 5.4 indicates spending on higher education as a share of GDP.387 Portugal’s spending on higher education as a proportion of its total wealth is equal to the EU-22 average at 1.4% of GDP, but slightly below the OECD average of 1.6% of GDP. Compared to the EU-22, lower public spending is fully offset by private expenditures that are equal to the OECD average. Spending per student relative to GDP per capita shows a similar pattern.

387 OECD, “Education at a Glance 2016: OECD Indicators.”

$0$2,000$4,000$6,000$8,000

$10,000$12,000$14,000$16,000

Expenditurefor all

services*

Expenditurefor core

educationalservices

Expenditurefor R&D

Publicexpenditure,

all institutions

Publicexpenditure,

publicinstitutions

Publicexpenditure,

privateinstitutions

Portugal

OECD

EU-22


138

Table 5.4: Expenditure on educational institutions relative to Portugal’s national wealth (all tertiary), 2013

Indicator Portugal OECD EU-22 OECD Rank

Total expenditure as a percentage of GDP* 1.4% 1.6% 1.4% 20/34 Public expenditure as a percentage of GDP* 0.9% 1.1% 1.2% 28/33

Private expenditure as a percentage of GDP* 0.5% 0.5% 0.3% 10/32 For all services, per student relative to per capita GDP* 40% 41% 40% 17/34

For all services excluding R&D activities, per student relative to per capita GDP 23% 29% 27% 25/32

*Again, these figures include some spending on post-secondary non-tertiary. We cannot calculate the effects of this on public expenditure, private expenditure, or spending per student relative to per capita GDP. However, if spending on post-secondary non-tertiary is included in tertiary spending for all services then Portugal’s rank falls to 25/34. Once again, this comparison would exaggerate Portugal’s funding disadvantage given some or most of Portugal’s spending on post-secondary non-tertiary is also calculated under secondary spending.

Portugal falls even closer to the bottom among OECD countries when it comes to public funding, at $5 883 per student.388 Public sources provide less (58%) of funding for higher education in Portugal than the average for the OECD (70%) or EU-22 (78%). Whereas the national government is the source of all public funding for higher education in Portugal, regional and local governments provide 14%-15% on average across the EU-22 and the OECD. Having a small government could explain these trends. However, as indicated in Table 5.5, at 1.8% Portugal is ahead of only Italy in terms of spending on higher education as a share of total public expenditure among comparator countries.389 Excluding R&D, Portugal ranks last. Table 5.5: Public expenditure on tertiary education as a percentage of total public expenditure in Portugal, 2013

Indicator Portugal OECD EU-22 OECD Rank

All tertiary including R&D* 1.8 3.1 2.7 31/32 All tertiary excluding R&D 0.8 2.3 1.9 26/26

Primary to tertiary including R&D 9.6 11.3 9.9 15/33

Primary 3.1 3.5 2.8 19/33 Secondary* 4.6 4.5 4.2 15/33

*Portugal figures for all tertiary including R&D and secondary include spending on post-secondary non-tertiary that is considered separately by other countries and averaged 0.2% across the OECD and the EU-22. Accounting for this spending for other countries as if it were all included in all tertiary including R&D or in secondary changes only Portugal’s rank for secondary spending, from 15th to 16th out of 33 countries.

Portugal’s below average spending on higher education is not reflective of low spending on education overall. In fact, Portugal is sixth in the OECD in terms of expenditure on primary to tertiary educational institutions as a percentage of GDP at 6.1% (averages across the OECD and the EU-22 are 5.2% and 5% respectively). In terms of public expenditure on all primary to tertiary

388 Ibid. 389 It falls behind Italy. Among countries with similar though higher results in terms of all tertiary spending including R&D, the Czech Republic and Japan are the only ones for which data excluding R&D are unavailable.


139

educational institutions as a share of GDP, Portugal ranks eighth at 5.1% (compared to 4.5% across both the OECD and EU-22). Finally, Portugal is only slightly below average in terms of public spending on education as a share of total public expenditure at 9.6% (compared to 11.3% across the OECD and 9.9% across the EU-22). Spending appears to be especially focused on secondary schooling, where Portugal ranks third in the OECD in spending as a share of GDP.390 In 2011, an OECD report calculated a measure of the extra tax revenues collected as a result of higher education.391 This suggested that Portugal had the highest returns to higher education expenditures in the study. The Government of Portugal’s position is that funding is inadequate, notably since institutions faced considerable restraint during the economic crisis.

3.1.2 Weak Incentives in the Institutional Funding Structure Very little about Portugal’s current approach to higher education funding encourages institutions to strengthen their performance. In fact, many elements do the opposite. The historical funding approach means that for the foreseeable future institutional performance will not determine the funding that institutions receive. Institutions are instead locked in to growth or cuts based on circumstances that fade further into history every year. Consequently, there is less imminent financial pressure on institutions to change and improve. In the absence of conditional government funding, student tuition is the only other year-to-year financial flow that could relate to performance. There are multiple reasons why tuition is unlikely to drive improved institutional performance. Firstly, and most importantly, it still represents a modest share of revenues in most public institutions. Secondly, students’ choices of institutions in Portugal appear to be largely driven by prestige – which relates closely to institutions’ age and therefore is developed over decades – and location, which is even more difficult to change. Thirdly, NC almost guarantees enrolment at least in institutions’ most desirable programmes. As well, as described in the previous chapter, interviewees believe that tuition fees lead institutions to maintain the status of students who continuously fail, given there is no loss of public funding. Government representatives indicated that there is interest in adopting a funding formula once again, as required in legislation. However, the Government would aim to raise overall funding before introducing conditions that affect its distribution, recognizing that some institutions could be severely harmed if a formula took funding away.

390 Portugal ranks fourth when you count secondary education and post-secondary non-tertiary education together. 391 OECD, “OECD Economic Surveys: Portugal 2017.”


140

3.1.3 Limited Targeted Funding for Key Initiatives It is worth noting that the Government of Portugal has repeatedly introduced funding targeted specifically at promoting study success. This includes funding for 22 projects on the issue between 2005 and 2009, and at least five or six projects between 2012 and 2015. This support helped to generate much of the research we rely on in this report. Institutional interviewees also cited these grants as important boosts to a number of the positive initiatives we listed in the previous chapter, although other interviewees questioned the effectiveness of these envelopes. A key area where funding appears to be lacking is regarding supports for students with disabilities. Interviewees acknowledged that there are significant costs associated with accommodating these students, which are unevenly distributed between institutions as the best prepared accept the most students in need. Some institutions are securing donor support to help, but there is basically no special support from government to address this challenge.

3.2 Student Financial Resources The economic difficulties facing many students and their families are the explanation for dropout rates most often cited in the Portuguese popular press.392 Students also identify these pressures as important contributors to dropout in surveys and interviews.393 394 In the Universidade de Évora study, finances, personal life and their professional situation were cited by most students (60%, 55% and 52% respectively) as the most important factors for dropout, and Licenciatura students emphasized finances and their professional situation especially in first year.395 These financial difficulties included difficulty paying fees (78%) and difficulty being away from home due principally to the cost of living (51%). In the 2013 IP de Setúbal study, economic difficulties (48% of respondents), workplace demands (27%) or inconvenient class times (18%) were the top reasons Licenciado students identified for dropping out.396 At the IP do Porto, 73% of students cited personal problems as the leading cause of their withdrawal from studies, including 26 percentage points who more specifically cited balancing studies and their work or families, and 24 percentage points who cited insufficient economic and financial resources.397

392 Rosalina Pisco Costa et al., “Identificação Das Causas Do Abandono Escolar Na Universidade de Évora” (Évora: Universidade de Évora, 2015). 393 IP de Setúbal, “O Abandono Nos Cursos de Primeiro Ciclo” (Setúbal: Instituto Politécnico de Setúbal, 2013). 394 IP de Setúbal, “O Abandono Nos Cursos de Segundo Ciclo” (Setúbal: Instituto Politécnico de Setúbal, 2014). 395 Pisco Costa et al., “Identificação Das Causas Do Abandono Escolar Na Universidade de Évora.” 396 IP de Setúbal, “O Abandono Nos Cursos de Primeiro Ciclo.” 397 Gabinete de Planeamento, Projectos e Desnvolvimento, “Estudo de Caso: Abandono Escolar 2012-13 (ESE, ESMAE, ESEIG, ISCAP, ESTGF, ESTSP e APNOR)” (Porto: Politécnico do Porto, 2014).


141

3.2.1 Students Struggle to Balance Studies and Employment Most students who had dropped out in the IP de Setúbal study were working students.398 Among students who dropped out in the Universidade de Évora study, at least 58% were working at least part-time, 22.3 percentage points of whom had legal status as student workers.399 As well, 10% of these students were the main provider of family income and 18% were the only contributors to family income, reflecting the relatively high representation of older students. Guerreiro and Abrantes (2007) found limited financial aid often explains why students are working so many hours.400 Working students appear more likely to drop out than transfer study programmes. A Universidade do Porto study, among others, also found employers were often unsupportive of students’ academic pursuits. Given how few Portuguese student-workers appear to work in career-related jobs and the low rates of employer investment in training this lack of support is perhaps somewhat unsurprising. On the other hand, Santos et al. record many mature students being reluctant to exercise special student-worker rights, though most reported that their employers at least acknowledged their efforts to balance their various commitments.401

Working takes time and energy that can affect students’ ability to study successfully, perhaps especially given Portuguese students’ busy schedules. Studies in Norway, the US and Canada, indicate that working over approximately 20 hours per week increases likelihood of dropout, while fewer hours has little effect.402 Research in the UK also found working affects grades. Given these concerns, academic accommodations for worker-students seem excessive. Granting longer timelines to complete degrees and retain financial aid, exemptions from rules on academic suspension, allowance for extra make-up assessments, and even exemption from compulsory attendance requirements would appear to deemphasise students’ studies relative to their employment. The primary element contrary to this trend is the monitoring of study success by employers, but this seems to be mostly about employers ensuring their workers are actually studying and not about employer investment in their employee’s education. The Worker-Student Statute necessarily had to strike a delicate balance between empowering and motivating worker-students to study successfully. It is unclear that it did so successfully.

398 Madalena Alarcão, “Insucesso Académico e Abandono Escolar” (Seminário Sucesso Académico, Lisbon, May 12, 2015). 399 Pisco Costa et al., “Identificação Das Causas Do Abandono Escolar Na Universidade de Évora.” 400 IP de Setúbal, “O Abandono Nos Cursos de Primeiro Ciclo.” 401 Lucília Santos et al., “Academic Success of Mature Students in Higher Education - A Portugese Case Study,” European Journal for Research on the Education and Learning of Adults 7, no. 1 (2016): 57–73. 402 Vossensteyn et al., “Dropout and Completion in Higher Education in Europe.”


142

3.2.2 Insufficient Student Financial Aid Even though Eurostudent data indicates their incomes are relatively high compared to other European countries, Portuguese students are quite clear in their views of the inadequacy of student financial aid. Portugal had the highest proportion of students (71%) reporting agreement or strong agreement that their funding situation is inadequate to cover their costs among participating countries in the Eurostudent study.403 This was especially true among students who are not living with their parents, with over 40% reporting strong disagreement with a statement that funding is sufficient to cover their monthly costs. Students who had delayed transition into higher education, had a low education background (ISCED 0-2), had children or were dependent on paid employment were also less satisfied than other students. Some interviewees questioned whether perceptions of financial need reflect reality so much as the politicisation of the tuition issue. However, the EuroStudent survey also placed Portugal at the bottom among participating countries in terms of its share of students receiving public support and public support as a share of recipients’ total monthly income, ahead of just one country in each category (the Czech Republic and Romania respectively).404 This is more consistent with another interviewee’s critique of concerns regarding financial aid and study success, that basically funding is inadequate for low income students but very few of these students are attending higher education anyway given the other challenges they face. Among students who dropped out and felt they could have been helped, almost half (49.5%) from licenciado programs at the IP Setúbal and 42% at the Universidade de Évora indicated greater financial aid could have helped them stay in school, by far the most significant factor.405,406 Multiple studies find higher completion rates among students who secure grants, and even the number of years students receive grants at the Universidade de Évora.407 408 409 Perhaps the most persuasive data is from the DGEEC, presented in Figure 5.5.410 These figures appear remarkable. Students who received grants were basically half as likely to dropout as students whose applications failed. The contrast in dropout between students from other access streams, i.e. less traditional students, who received grants and those who did not apply is even sharper, basically at one third.

403 Orr, Gwosc, and Netz, “Social and Economic Conditions of Student Life in Europe: Synopsis of Indicators - Final Report - Eurostudent IV 2008-2011.” 404 Ibid. 405 Pisco Costa et al., “Identificação Das Causas Do Abandono Escolar Na Universidade de Évora.” 406 IP de Setúbal, “O Abandono Nos Cursos de Primeiro Ciclo.” 407 José António Sarsfield Cabral, “Algumas Notas Sobre o Acesso e Desempenho Dos Estudantes Admitidos Na U.Porto Em 2008/09, 2009/10 e 2010/11 (Pelo Regime Geral de Acesso Em 1s Ciclos e Mestrados Integrados)” (Seminário para o “Sucesso Académico,” Lisbon, May 12, 2015). 408 DGEEC, “Ensino Superior: Situação Em 2012-13 Dos Inscritos Pela Primeira Vez Em 2011-12 - Parte II” (Lisbon: Direção-Geral de Estatísticas da Educação e Ciência, 2013). 409 Pisco Costa et al., “Identificação Das Causas Do Abandono Escolar Na Universidade de Évora.” 410 DGEEC, “Ensino Superior: Situação Em 2012-13 Dos Inscritos Pela Primeira Vez Em 2011-12” (Lisbon: Direção-Geral de Estatísticas da Educação e Ciência, 2013).


143

Figure 5.5: Dropout of first-time first-year students based on receipt of financial aid and access stream, 2012

A priori we might have anticipated students who did not apply for need-based grants to have lower dropout rates because greater financial resources could explain their non-application. The absence of this trend may indicate that applications reflect higher levels of motivation, organization or support in preparing an application successfully, among those students who apply and successfully obtain aid. Consistent with this, the principal reason students’ applications for financial aid failed in a 2015 study at the Universidade de Évora was because they were incomplete.411 These observations almost certainly apply, the question is to what extent. This view that the greater performance of financial aid recipients relates overwhelmingly to their motivation is undermined by the observation that dropout rates among students who receive grants only on appeal are almost as high as among rejected applicants. Given they are pursuing an appeal process even after being rejected, these students must be highly motivated and may also have the background and social capital that would predispose them towards challenging administrative barriers. Interviewees cited the considerable delays in successfully processing appeals as the explanation for these students’ lower study success. Further iterations of this data would be very valuable to confirm these trends and build stronger, more nuanced evidence as to the effects of greater financial aid investments. In terms of Portugal’s student loan system, its effects on completion rates appear very limited because it seem to be providing very few students with meaningful financial aid. Take up of the loan system is equivalent to just 2% of the student body.412 From 2007 to 2014, less than 21 000 loans were issued, whereas 70 000 social support grants were provided in 2010 alone.

411 Pisco Costa et al., “Identificação Das Causas Do Abandono Escolar Na Universidade de Évora.” 412 Heitor, Horta, and Leocádio, “Enlarging the Social Basis of Higher Education: Lessons Learned from Extending a Social Support System with a Risk-Sharing Loan Scheme in Portugal.”

0%

5%

10%

15%

20%

25%

30%

Universities ISPs Universities ISPs

General Access Stream Other Access Streams

Accepted applicants Accepted on appeal Rejected Did not apply


144

Heitor et al. attribute low take up to three key factors.413 Firstly, the relatively more generous and expansive social support grant system. Secondly, student and family concerns about defaults, especially in times of economic uncertainty when many households are already heavily in debt for other purposes, and also given Portugal does not have a history of using loans to finance education and. Finally, banks restricted lending during the financial crisis – the number of loans provided fell from 2011 to 2014. All of these challenges seem to especially affect prospective low-income borrowers. The very low default rates may be viewed as a success, but they also suggest loans may not be supporting many people whose circumstances and characteristics indicate they would most need support, even if they would be most at risk of default post-graduation. Some argue that the loan programme has been an important achievement despite low uptake because it has begun to challenge Portuguese families’ reluctance to make private investments in higher education and take on debt. There may be some truth in this statement, but this does little to address the immediate financial challenges facing Portuguese students.

3.2.3 Financial Aid Eligibility Criteria May Undermine Completion Related to the relationship between student social support grants and dropout noted above, many interviewees argued that the loss of eligibility for grants when students were academically unsuccessful feeds dropout. Student representatives argued that students should be permitted to fail classes in first year without losing eligibility given the difficulties of adaptation. Figure 5.6 presents data from the IP do Porto indicating the proportion of rejected applications to the institution for BE grants between 2011-12 and 2015-16 that were due to clauses e) (academic performance in the prior year) and f) (time to completion eligibility requirements), as well as excessive income (clause g).414 A key caveat to these data is of course that many students likely due not apply when they predict or assume that they will be ineligible.

Figure 5.6: Causes of rejection of applications for the Bolsa de Estudo at the IP do Porto, 2011-2015

413 Ibid. 414 IP do Porto, “Apreciação Parlamentar Na Comissão de Educação e Ciência” (Porto: IP do Porto, 2016).

0%

5%

10%

15%

20%

25%

30%

35%

40%

45%

2011 2012 2013 2014 2015

Clause e)

Clause d)

Excessive Income


145

The data indicate that poor academic performance was the primary cause for rejection of applications, even more so than income-based ineligibility. However, they also show that a non-negligible number of students are rejected each year due to time eligibility requirements. The time to completion eligibility requirements (under clause f) are of special concern, notwithstanding the higher share of rejections due to poor academic performance, because they mean once students lose their eligibility for poor performance, it is very difficult to restore it by earning additional credits without grant support. To regain eligibility, students must earn credits at a faster pace than the minimum degree progression required under the programme to make up ground, and their prior failure suggests few of these students are positioned to do this. The BE’s allowance for students facing extreme circumstances is appropriate, but there is no mechanism for determining whether students’ circumstances are sufficiently extreme and the accommodations may be unduly restricted given the multiplicity of factors that can affect students’ circumstances and threaten their ability to progress academically. These concerns apply equally to the +Superior programme now that it is conditional on receipt of the BE. The +Superior revamp appears to be a very positive step since it will better target support to students at greater risk of failing to complete their study programmes. In principle, student loans should have some advantages in promoting study success relative to grants, because students must pay more back if they study for longer, and completing their degree raises the student’s earnings and therefore the ease of paying back the loans. There are two further design elements of Portugal’s student loan programme that seek to promote completion.415 Firstly, there is the limit on eligibility for loans to the length of the degree programme plus one additional year. Providers may also except one failure per study programme, justified to the financial provider. Providers, are however, required to foreclose loans and initiate an early repayment if students fail a second time. Secondly, students who secure high grades are given reductions in the allowable spread: after students’ first year the maximum spread is reduced from 1% to thirty-five basis points for students who obtain a yearly average grade equal to or above 14/20 and 80 basis points for students whose average grades are equal to or above 16/20. The basic principle that responsibility for completion should be jointly shared between higher education institutions and students is sound. The Government of Portugal, and higher education institutions should limit their exposure to paying for students to be unsuccessful. However, it is uncertain that penalizing students who do not succeed will promote completion. Financially, the much higher potential earnings among degree holders should alone provide sufficient incentive for completion and faster time-to-degree without the prospect of greater difficulties with loan

415 Heitor, Horta, and Leocádio, “Enlarging the Social Basis of Higher Education: Lessons Learned from Extending a Social Support System with a Risk-Sharing Loan Scheme in Portugal.”


146

payments. What is more, many of the same challenges that could cause a student to struggle to complete their degree would also likely undercut their success in post-graduate employment and consequently raise their likelihood of default. Punishing these students with higher debt payments can therefore compound the disadvantages that they face, worsening social inequality.

3.2.4 Negligible Disability-Related Financial Aid Support for students with disabilities appears to be an area of critical weakness in Portugal’s financial aid regime. The concern is not the amounts of money that can be made available, which are a priori considerable, so much as eligibility and coverage. The primary source of concern is the note from an interviewee that just 34 students with special needs are receiving the targeted financial aid. Conservatively, this represents roughly 0.055% of all students receiving grants.416 There must be many more students with disabilities who are not receiving targeted support, so the definition of disability used for eligibility is much too narrow. There is a similar absence of disability accommodations in Portugal’s student loans programme. A second concern is that students with disabilities are not granted an extended timeline for completion, unlike Student-Workers. Evidence from other contexts indicates disabilities often raise students’ workloads and make them need more time to complete their degrees.417

4. International Experiences Vossensteyn et al. identified a number of funding approaches to promote study success adopted by other European countries.418 These target both institutional funding, and student aid.

4.1 Institutional Funding Institutional funding can be adjusted to promote study success in two main ways. The first is funding to promote particular activities or account for certain inputs. Five European countries419 have provided additional funding to institutions to strengthen teaching and learning. France’s Plan pour la réussite en licence provided universities funding to

416 Ibid. 417 Kayti Baur et al., “Disable the Label: Improving Post-Secondary Policy, Practice, and Academic Culture for Students with Disabilities” (Halifax, NS: Students Nova Scotia, 2014), http://studentsns.ca/wp-content/uploads/2014/12/2014-12-03-disable-the-label-final-report-accessible-3.pdf. 418 Vossensteyn et al., “Dropout and Completion in Higher Education in Europe.” 419 The Czech Republic, Flanders, France, Germany and Macedonia.


147

pursue innovations in teaching and learning and develop other instruments assisting first cycle students in particular. An evaluation found the funding motivated institutions to better adapt to student needs and invest in student supports, but did not find any changes in completion rates. France also funded an expansion of faculty and other staff. Germany’s 2015 Quality Pact aimed to improve staff-student ratios, improve staff qualifications and training, and improve teaching quality, and is currently under evaluation. Overall, these types of initiative seem to encourage greater attention to teaching and learning, but one concern is that the temporary status of funds may undercut the durability of changes. The Czech Republic, England and Flanders provide additional funding to institutions that recruit more students at risk of early withdrawal, including students from lower SES families, justified partly by the notion that these students will likely require more resources to support. In England, the Student Opportunity Allocation is based on a formula accounting for higher dropout rates among students of particular ages and with certain prior qualifications, as well as an application based mechanism where institutions can sign performance agreements for funding to support specific study success initiatives. An evaluation found no causal impact on student outcomes, though the rate of “non-continuation” (i.e. basically dropout) fell from 9.1% in 2003-04 to 5.7% in 2012-13 and the funding encouraged institutional efforts to strengthen study success. The second main way to adjust institutional funding is to introduce performance based funding envelopes. This approach is increasingly common, with fifteen countries reported as using it in the Vossensteyn et al. report, including most German Länder. The idea is to create direct incentives for institutions to improve study success and ultimately shift resources and potentially enrolment from institutions that perform poorly to those that perform better. Evaluation results of three programs are summarised in Table 5.6.420 Table 5.6: Summary of results of conditional funding envelopes in Austria, the Netherlands and Norway

Country Evaluation

Austria Performance funding failed to achieve goals due to excessive complexity and failure to reward institutional achievements. The conditional envelope was too small as a share of revenues to motivate adequate action on study success, such as student monitoring.

The Netherlands

Successful degree completion the basis for allocating 50% of teaching funds. Together with other instruments, policy led to slight increase in completion and decrease in time-to-degree from 6.5 to 5.8 years for typical 5-year Integrated-Masters program.

Norway Performance based funding increased institutional focus on completion and dropout, but only moderately because the conditionality was relatively open-ended. Institutions encouraged students to take more credits, but graduate numbers did not rise significantly.

420 Vossensteyn et al., “Dropout and Completion in Higher Education in Europe.”


148

4.2 Student Funding Again, with respect to student funding, we can distinguish between two different categories of policy efforts, which in fact line up with the institutional funding approaches. The first represents the provision or expansion of general financial aid. Many countries421 identify their student financial aid programs with their study success objectives. The basic notion is that students who are better financially supported will be more focused on their studies and less likely to pursue distracting part-time work. France also introduced scholarships in 2013 specifically to finance student living costs and accommodations, with a special focus on students from non-traditional backgrounds. There is no evidence as to the effectiveness of these European programs in terms of completion. Empirical research in the United States has found that financial aid can increase completion in some cases, but decrease it in others. A study of largescale merit scholarships in Arkansas and Georgia, which were available to most students in both states and eliminated the direct the costs of schooling, increased completion rates by three percentage points, with strongest effects among women.422 On the other hand, a merit-based scholarship in Massachusetts for the top 25% of students in each district was found to reduce completion rates because its design shifted students into weaker higher education institutions.423 The second major approach is to provide contingent financial aid. Eleven European countries424 have financial aid contingent most often on student progress (in terms of credits achieved or time-to-completion), or in some cases based on students not working in paid employment more than a set number of hours. Some of these programs limit the amount of funding available to students, but very often the incentive is embedded in repayment conditions. For example, Finland has extended access to loans so students do not have to find paid employment, and those who complete on time have their repayment amount reduced. Norway converts its loans to grants when students complete on time, creating an incentive worth up to EUR 4 800 per year. Evaluations of the Norway program have found however that this policy has not significantly affected delays in student completion, in fact time-to-degree has increased. Finally, in the Netherlands from 1996-2015 all financial aid was provided in the form of loans that could be converted to grants if students completed their degrees within ten years of enrolment.

421 Austria, Bulgaria, Czech Republic, England, Estonia, Germany, Ireland, Italy, Macedonia, Norway, Romania, Serbia and Turkey. 422 Susan Dynarski, “Building the Stock of College-Educated Labor,” Journal of Human Resources 43, no. 3 (2008): 576–610. 423 Sarah R. Cohodes and Joshua S. Goodman, “Merit Aid, College Quality, and College Completion: Massachusetts’ Adams Scholarship as an In-Kind Subsidy,” American Economic Journal: Applied Economics 6, no. 4 (2014): 251–85. 424 Croatia, Denmark, Finland, Hungary, Iceland, Lithuania, Montenegro, the Netherlands, Norway, Sweden and Spain.


149

Evaluations found the policy contributed to reducing time-to-degree but did not improve completion. The conversion to grants was cancelled in 2015. Similarly, ten mostly Eastern European countries425 have introduced conditionality in tuition fees. Again, students who achieve targets for study progress or even grades have their tuition reduced relative to other students (even eliminated), or tuition fees are charged to or raised students who take too long to complete or enroll in too few courses. Many of these conditional financial aid policies are more oriented towards time-to-degree as compared with completion, and they may feed into the time-to-completion/completion trade-off described in Chapter 2. Overall evaluations completed have found little impact of such financial reward schemes on study success. To have the possibility of being successful the amounts of funding must be significant and accompanied by wider conditions that support study success. More generally, financial aid has not been found to be a key factor in study success internationally. Even in the empirical study on Arkansas and Georgia referenced earlier, the author noted the gains were relatively modest, and direct costs of education might not even be the “central impediment to degree completion.”426 Students emphasize financial aid and report that it helps, but there is limited empirical support for this view.427

425 Croatia, Estonia, Hungary, Lithuania, Poland, Serbia, Slovakia, Slovenia, Spain and Turkey. 426 Dynarski, “Building the Stock of College-Educated Labor,” 579. 427 Vossensteyn et al., “Dropout and Completion in Higher Education in Europe.”


151

Chapter 6: Options and Recommendations

This chapter seeks first to prioritise factors in study success from Chapters 2, 3 and 4, using a heuristic based on each factor’s importance in determining overall completion rates, our confidence in related research findings, and our assessment of how susceptible the factor is to policy interventions by government. This assessment clearly emphasises the following concerns: failure to assist students at risk of dropping out; weak incentives in the institutional funding structure; Uneven institutional commitment to study success; and high rates of academic failure. A second heuristic helps us categorise which factors the Portuguese Government can address directly (first-level), or through influencing the behaviour of institutions (second-level), or even institutional units or individual stakeholders (third-level). The Government has considerable direct control over financial factors and the admissions systems, but in general cannot address quality challenges directly. Institutional leaders, faculty, administrative staff, student representatives and quality assurance agency staff all must play a role in raising completion. We review 17 policy options under three themes: (1) Targeted Supports for Students at the Margins, (2) System Steering, and (3) Shaping Expectations of Student Success. Our analysis relies on four main criteria: impact on completion, risks, flexibility, and feasibility. We do not estimate costs of implementing our proposals, however we only consider policy options that we believe could be pursued with modest additional expenditure beginning in the short-term. Our three primary recommendations for the Government of Portugal are:

1. Provide targeted institutional funding for services to identify and support students at risk of dropping out

2. Include performance criteria relating to study success within an institutional funding formula

3. Tighten rules regarding academic suspension and monitor institutional compliance Ten secondary recommendations aim to complement the primary recommendations and/or provide intermediate solutions as other policies are in development.


152

Chapter Guide

1. Prioritising Factors in Non-Completion ........................................................................... 153

2. Policy Options ................................................................................................................ 155 2.1 Targeted Supports for Students at the Margin ....................................................................... 156 2.2 System Steering.................................................................................................................... 160 2.3 Shaping Expectations of Study Success .................................................................................. 166

3. Recommendations ......................................................................................................... 170 3.1 Support Students at Risk of Dropping Out ............................................................................. 170 3.2 Steer the System for Study Success ....................................................................................... 172 3.3 Entrench Expectations that Students Succeed ....................................................................... 175

List of Tables Table 6.1: Analytical tool to prioritise factors in completion based on importance, confidence in assessments and susceptibility to policy interventions, on a scale of 1-10......................... 153 Table 6.2: Analytical tool to identify distance of completion factors from the Government of Portugal ...................................................................................................................................... 155


153

1. Prioritising Factors in Non-Completion We use a heuristic to assess the appropriate prioritisation of each factor, based on our view of its importance in determining overall completion rates across the system, our confidence in our research findings regarding the factor’s importance, and our assessment of how susceptible the factor is to a policy intervention by government. This analysis is presented in Table 6.1. Table 6.1: Analytical tool to prioritise factors in completion based on importance, confidence in assessments and susceptibility to policy interventions, on a scale of 1-10

Concerns Chapter

Imp

ort

ance

Co

nfi

den

ce

Susc

epti

bili

ty

Pri

ori

tisa

tio

n

Failure to assist students at risk of dropping out Quality 9 10 9 9.2

Weak incentives in the institutional funding structure Financing 9 8 9 8.8

Uneven institutional commitment to study success Quality 9 8 8 8.4 High rates of academic failure Quality 9 8 8 8.4

Students struggle to balance studies and employment Financing 8 9 8 8.2 Insufficient student financial aid Financing 9 7 8 8.2

Insufficient focus on teaching and serving students Quality 9 9 7 8.2

Financial Aid eligibility criteria may undermine completion Financing 7 8 9 8 Limited targeted funding for key initiatives Financing 7 8 9 8

Underdeveloped first year student integration Quality 7 8 9 8

Rigid programme structures Quality 9 8 7 8

Negligible disability-related financial aid Financing 5 10 9 7.6

Low institutional funding by international standards Financing 7 8 8 7.6 Poor adaptation to the needs of mature students Quality 8 8 7 7.6

Excessive time demands on students Quality 9 6 7 7.6

The unclear binary distinction Quality 7 8 8 7.6

Numerus Clausus restricts access to sought-after programmes

Admissions 8 8 6 7.2

Supports for students with disabilities are underdeveloped Quality 6 7 8 7

The National Access Competition promotes shallow matching

Admissions 7 8 6 6.8

Alternative access streams lead to high dropout Admissions 6 8 7 6.8

Insufficient information on the student experience Admissions 4 6 10 6.8 Limited student engagement Quality 6 6 8 6.8

The quality of student services is uncertain at best Quality 7 4 8 6.8 Inadequate remedial support for students with weaker academic preparation

Quality 6 6 8 6.8

* Prioritisationi = (2*Importancei + Confidencei + 2*Susceptibilityi)/5


154

Our assessment of the importance of different factors is with respect to the total population of students who drop out in a given year. It is not intended to be an evaluation of the importance of particular students, such as students with disabilities or mature students, except with respect to their proportion of the total student population. There are strong reasons to provide targeted support to students with disabilities grounded in Portugal’s commitment to the UNCRDP, regardless of how much this will contribute to the population-level higher education completion rate. With regards to confidence, the research we outlined in Chapters 3 through 5 suggests that the factors we have identified contribute to low completion, but the data are rarely conclusive. There is a total absence of empirical research establishing causal relationships with completion in Portugal. Much of the descriptive data that we use has special deeper limitations too. Noting where our confidence is weak can help to indicate which areas should become priorities as soon as updated data is collected. Our assessment of susceptibility is grounded in reflection on the financial, administrative, and political feasibility of possible government interventions to address each policy problem. More expensive problems require not only resources, but that government eschew other expenditures in a context of fiscal constraint. Many of the factors in non-completion would not require expenditures to address, but it can often be much easier to simply spend money than to change controversial policies and practices. Some non-financial challenges we have identified appear very politically difficult. Finally, in many areas it is unclear what tools the Government of Portugal can put to use. For example, the Government has made student loans available but it cannot force students and their families to enroll and take on debt for education. Of course, many of the factors from across the three chapters are very closely interrelated. My susceptibility assessment focuses on direct susceptibility, not means to affect one of the challenges via another. We can also organise the factors in completion into three levels. The Government of Portugal can address first level factors directly through spending or changes in legislation, for example increasing funding to institutions or changing eligibility criteria for financial aid. The Government must address second level factors by using the instruments at its disposal to influence other parties, namely higher education institutions. Finally, to address third level problems the Government must influence one party, like a higher education institution, to influence another party, say units within institutions or individual faculty members. These levels are not mutually exclusive. This analysis is presented in Table 6.2. The Portuguese Government has considerable direct control on the financial factors because higher education is mostly publicly funded, but also over admissions systems through its statutory powers over NC and the CNA. The Government is much more limited in its ability to directly address quality related challenges, although many of these factors are very important determinants of study success. Institutional leaders, faculty, administrative staff, student representatives and quality assurance agency staff all have a role to play in raising completion.


155

Table 6.2: Analytical tool to identify distance of completion factors from the Government of Portugal

Concerns Chapter Suscep-tibility

Levels

1 2 3

Insufficient information on the student experience Admissions 9 √ √ √

Weak incentives in the institutional funding structure Financing 9 √

Financial Aid eligibility criteria may undermine completion Financing 9 √ Limited targeted funding for key initiatives Financing 9 √

Negligible disability-related financial aid Financing 9 √

Students struggle to balance studies and employment Financing 8 √ √ √

The unclear binary distinction Quality 8 √ √ √

Insufficient student financial aid Financing 8 √ Low institutional funding by international standards Financing 8 √

Alternative access streams lead to high dropout Admissions 7 √ √ The National Access Competition promotes shallow matching

Admissions 7 √

Numerus Clausus restricts access to sought-after programmes

Admissions 6 √ √

Failure to assist students at risk of dropping out Quality 9 √ √ Underdeveloped first year student integration Quality 9 √ √

High rates of academic failure Quality 8 √ √

Supports for students with disabilities are underdeveloped Quality 8 √ √ The quality of student services is uncertain at best Quality 8 √ √

Uneven institutional commitment to study success Quality 8 √ Rigid programme structures Quality 7 √ √

Poor adaptation to the needs of mature students Quality 7 √ √

Excessive time demands on students Quality 7 √ √ Limited student engagement Quality 7 √ √

Inadequate remedial support for students with weaker academic preparation

Quality 7 √ √

Insufficient focus on teaching and serving students Quality 7 √ √

2. Policy Options We used the following basic criteria for reviewing policy options:

1. Impact on completion – assessed primarily by the prioritisation analysis. 2. Associated risks – may relate to unforeseen expenditures, possible negative side-effects

for the higher education system and its stakeholders, and popular opposition. 3. Flexibility – to be adjusted over time or meet different possible policy goals. 4. Feasibility – administrative, legal, technical or political capacity to implement the policy.


156

We decided ultimately to focus on three areas, each of which directly addresses multiple factors in study success. We were not in a position to estimate costs of implementing our proposals, however we only consider policy options that could be pursued with modest additional expenditure beginning in the short-term. Appendix II includes a table summarising many elements of the options analysis: the targeted policy problems, key actors, key steps to implementation, key obstacles and possible timeline. To control scope, we do not address some factors that appear nevertheless very important. In particular, our analysis suggests that the low funding of Portuguese higher education and limited financial aid for students contribute to reducing completion, but it would require more extended analysis to provide an informative review of the Government of Portugal’s approach to either issue in line with the Government of Portugal’s wider strategy for higher education, workforce development, research and innovation. I also do not provide recommendations specifically targeting the private higher education system. However, the wide range of dropout levels from private institutions, particularly at the university level, suggests this sub-sector would be particularly suitable for strategies that focus on securing improvements from specific institutions with higher dropout rates.

2.1 Targeted Supports for Students at the Margin Policies in this area would aim to address the top priority concern from our analysis: Failure to assist students at risk of dropping out. Additional related concerns include:

➢ Limited targeted funding for key initiatives; ➢ The unclear binary distinction; ➢ Underdeveloped first year student integration; and, ➢ Limited student engagement.

Option 1.1: Provide targeted institutional funding for services to identify and support students at risk of dropping out. Funding could help institutions to hire staff who are specifically responsible for tracking, outreach to identified students and coordinating different services as needed. The Government would need to develop standards and reporting requirements for the use of the funds. Impact on completion: The absence of targeted supports for students at risk of dropping out is likely one of our most significant factors affecting student completion. A study completed at the Escola Superior de Tecnologia de Setúbal showed an early intervention with students at risk of dropping out based on grades and course registration was effective in reducing dropout.428 Interviewees suggested that such a programme could have considerable impact on the completion challenge.

428 IP de Setúbal, “O Abandono Nos Cursos de Primeiro Ciclo” (Setúbal: Instituto Politécnico de Setúbal, 2013).


157

Risks: The primary risk of such a programme is that institutions do not spend the funds effectively. Hence standards and reporting requirements are important, though they must be balanced so as not to create excessive administrative demands. Flexibility: This programme would support institutions to provide tailored support to students based on their particular needs. Feasibility: Such a programme could be rolled out relatively rapidly likely at a modest cost. The quality of higher education institutions’ information systems could be the key challenge for implementation, however, requiring investment to develop these systems, though this could have numerous additional benefits. Navigating legal restrictions to protect students’ privacy could be the other main feasibility constraint. Option 1.2: Cancel and replace the requirement that tenured IP faculty have doctoral degrees. Impact on completion: Redirecting the institutional energy and resources that are supporting recruitment and training of doctoral degree holders at IPs towards higher quality teaching and workplace engagement could help to improve outcomes. In particular, faculty better connected with workplaces could deliver more relevant instructions and facilitate professional experiences for students to strengthen their engagement. Risks: The Government of Portugal and polytechnics would have to ensure that the new recruitment approach would align with goals to improve study success. There may also be a risk that in eliminating the doctoral requirement the Government simply relaxes standards for faculty hiring, instead of making the standards more sophisticated and strategic. Flexibility: This policy should expand flexibility for polytechnics to hire faculty that best support their institutional mission. Feasibility: Suddenly reversing the policy of incentivising polytechnic faculty to obtain doctoral degrees could be very unpopular and demotivating among these professors. The Government and institutions might need to ensure faculty with doctoral degrees at least earn higher salaries, to recognise their investment in their own human capital. Option 1.3: Adjust legislation to allow public higher education institutions to hire or otherwise compensate students for non-research activities. Institutions clearly have an interest in being able to hire students for different activities, many of which could be directly relevant to their study programmes or the study success of others. Students may also be interested in employment. Impact on completion: The students employed themselves could benefit from greater engagement, and earning income with an employer that should be particularly understanding of


158

their academic responsibilities. These students in turn could improve services and engagement for their peers and prospective students. Risks: One interviewee suggested a possible concern would be that institutions would exploit student employees. Student associations do not appear overly concerned about this possibility, given that they have endorsed hiring of students by institutions as a measure to improve completion.429 Flexibility: The current legislation is in principle very restrictive, though it does not appear to be fully applied. Changing the legislation would grant institutions greater de jure flexibility. Feasibility: The Government can alter its legislation in this area and it does not appear that such a policy change would be very controversial, especially with institutional and student support. Option 1.4: Develop partnerships with student associations to allow them to coordinate formal student orientation activities, with strict requirements to protect safety and social inclusion. This option is inspired by student orientation weeks that take place at universities in other jurisdictions, including Canada. Students develop the activities with some institutional oversight, including treasure hunts that expose students to their campus and community, visits to amusement parks, large concerts and fundraisers. The student leaders often receive significant training on issues of alcohol abuse, sexual violence, etc. Negative or dangerous events can certainly occur, but many orientation weeks are largely positive. The Portuguese Government could empower student associations to facilitate these kinds of positive student orientation activities through partnership agreements that include their higher education institutions. Agreements could require student associations to: pursue activities to familiarise students with institutional services and practices; strictly uphold respect for diversity (of gender, ethnicity, class, sexual orientation, nationality, etc.) and human rights; maintain high standards in protecting student safety; share information and consult with institutional representatives; facilitate social inclusion and student safety training for orientation leaders; and, engage in responsible financial management. Institutions would be responsible for monitoring. In exchange for fulfilling these conditions, student associations access funding (from students, institutions or the Government) and institutional facilities. Impact on completion: The likely direct impacts of such a policy are difficult to project. However, this style of orientation could help promote positive student engagement not only at the beginning of their studies but throughout the length of their degrees. It could also help to stamp out negative praxe activities by offering students a better alternative.

429 Movimento Associativo Estudantil, “Não Desistas! Combate Ao Abandono Escolar - O Guia de Boas Práticas No Ensino Superior” (Porto: Movimento Associativo Estudantil, 2016).


159

Risks: The key risk is that student associations will facilitate dangerous or degrading activities. Funds could also be misspent. The requirements outlined above could help mitigate these risks. Flexibility: Flexibility would depend on how strict requirements are for student associations. It would be necessary to grant some meaningful control in particular if this policy were to be successful in replacing praxe activities. Partnerships agreements could include pilot projects with particular associations/institutions to determine if the approach is positive. The Government could also cancel or adjust agreements in the event of serious violations. Feasibility: There is currently considerable public opposition to praxe, which may provide an opportunity to approach student orientation differently. Funding the orientation activities could be a challenge, and modest participation fees may be the most reasonable approach. In other jurisdictions student associations have also secured private sector sponsorships. Option 1.5: Introduce legislation requiring institutions to provide annual student progression data to the MCTES. The DGEEC collects data from institutions on students’ enrolment status at the end of each year, as described in Chapter 2. This data includes a unique student identifier that protects students’ privacy, but allows the DGEEC to track students as they transfer between institutions. Use of this data is strictly restricted by privacy legislation because it is gathered under the auspices of the Sistema Estatístico Nacional, according to an interviewee. The DGEEC cannot allow external researchers to use the data, and so whatever analysis is done must be completed by DGEEC staff, who have many other responsibilities. If this data was collected by the MCTES, however, it could be made available to researchers and other agencies to better inform policy-making on completion. It could also be more easily linked with other data collected by MCTES, such as data on financial aid and students’ admissions pathways. This would require a legal change. Impact on completion: Allowing the Government of Portugal’s data on student progression in higher education to be used by researchers and institutions could connect Portugal’s excellent data with analysts who can put it to use, and thereby inform policy interventions to promote study success. Risks: The Government of Portugal would have to carefully address risks related to the protection of privacy. Flexibility: This policy would allow considerably more flexibility in the use of data on student progression. Feasibility: Any change would have to respect legal privacy protections.


160

Option 1.6: Maintain current policies. Impact on completion: We would not expect the status quo to further support understanding and activities regarding the completion challenge. Risks: Expanding IP faculty with doctoral degrees would likely worsen difficulties in distinguishing Portugal’s binary higher education structure. The current policy on hiring of students means that institutions may not be respecting the letter of the law, so if there is a legal challenge they and their student employees could be greatly affected. The most serious risk in not developing a positive student-driven orientation and focusing on very academic activities is the persistence of informal praxe, because students’ interest in some form of social induction that they control is unlikely to disappear. Finally, the Portuguese Government may pursue misguided policies or investments in higher education that would have been avoided if the Government had access to the dropout data. Flexibility: Polytechnics are significantly constrained in their hiring, potentially missing out on many faculty members who would excel in educating polytechnic students. Legislative restrictions on student hiring are formally restraining. Lastly, the Government would continue to have little flexibility in the use of data collected on student progression. Feasibility: Again, maintaining the restriction on student hiring may be infeasible in the event that it is challenged legally.

2.2 System Steering Steering policies aim to systematically reshape higher education institutions’ incentives to align with the Government’s policy goals, while recognising that institutions are autonomous. Many of these system-steering mechanisms could have broader implications beyond study success. Therefore, the Government of Portugal must therefore consider them within the context of its other system objectives as well. The top priority factors in completion for government steering are:

➢ Uneven institutional commitment to study success; and, ➢ Weak incentives in the institutional funding structure.

Other concerns that the policy options would also seek to address are:

➢ Limited targeted funding for key initiatives; ➢ Numerus Clausus restricts access to sought-after programmes; ➢ Insufficient information on the student experience; and, ➢ The quality of student services is uncertain at best.


161

Option 2.1: Develop performance agreements with institutions to establish expectations of improvement in study success. Through performance agreements, Governments work with higher education institutions to establish institutionally relevant goals that are consistent with public policy priorities.430 Usually the agreements are tied to funding, formally or informally, but ex-ante and not ex-post unlike funding formulae. Impact on completion: Performance agreements could permit the Government of Portugal to establish expectations for institutional outcomes, and collaborate on particular activities that support study success. They could address any number of the quality-related factors that I identified as second or third level, such as failures to assist students at risk of dropping out, uneven commitment to student completion, and high rates of academic failure. The knowledge that an agreement prioritising study success is in place could also help advocates within institutions to build momentum for more student-centred practices. Of course, the quality of the agreements themselves would be decisive in terms of their impacts on completion. Risks: Agreements could be poorly designed and consequently promote misguided objectives, or have other unintended consequences. Flexibility: Performance agreements have the advantage of being institution specific, unlike most funding formulae. In fact, they are frequently used to promote institutional differentiation. This would be particularly relevant for Portugal given its diversity of institutions. Feasibility: Portugal’s higher education financing law already envisions similar agreements. However, if the Government rushes to reach performance agreements without adequate strategic planning and preparation they may be relatively ineffective, so system-wide performance agreements may be infeasible under the current parliament except perhaps at a subset of institutions. Portugal could prioritise agreements with larger institutions that have relatively low completion rates, such as the larger IPs, or institutions that have already taken the lead in addressing study success to reinforce strong expectations. Option 2.2: Tie institutional funding to study success related activities, outputs or outcomes through a funding formula. Formulas help determine the amount and/or distribution of public funds to higher education institutions, often using some form of ex-post conditionality on institutional activities, outputs or outcomes. Chapter 5 provides multiple notes on funding formulas, relating notably to their adoption in other countries to pursue study success objectives. Study success metrics that have

430 Harry de Boer et al., “Performance-Based Funding and Performance Agreements in Fourteen Higher Education Systems” (Enschede, the Netherlands: Center for Higher Education Policy Studies, 2015), http://doc.utwente.nl/93619/7/jongbloed%20ea%20performance-based-funding-and-performance-agreements-in-fourteen-higher-education-systems.pdf.


162

been used or discussed in other jurisdictions include degrees granted, exams passed or credits earned, time-to-completion, and results from student surveys.431 Impact on completion: Evidence cited in Chapter 5 provided mixed evidence on the effectiveness of funding formulae from elsewhere in Europe. It is apparent that formula design is crucial to create strategic incentives. Risks: Funding formulae redistribute funds between institutions. This creates winners and losers, which is partly the point of formulas, to reward good performance and punish poor performance, but the system must be able to accept that some institutions will lose. If the losers do poorly enough, institutional quality and even viability may be threatened. Interviewees indicated this would be the case if Portugal’s 2008 formula was introduced immediately at current funding levels, and many of the institutions expected to be worst affected play a crucial role in maintaining the economic and cultural vitality of rural communities. Funding formulas can also be difficult because they increase revenue variability whereas many costs are fixed.432 One additional risk of funding formulas is poor design. A bad formula could lose out on some of the study success trade-offs I described in Chapter 2, say by encouraging institutions to simply pass students through who have not learned required material. This point highlights how funding formulae also often are insufficient on their own, but for example need the backing of a strong quality assurance system. Flexibility: Funding formulae are relatively inflexible. This is actually their point in a sense, because their inflexibility generates credibility in motivating institutions. It is possible however to design a formula to account for some diversity among institutional missions and circumstances. Feasibility: Portugal’s higher education financing law specifically requires the use of a funding formula, so introducing one would not require legislative changes. Again, however, the instrument would affect multiple dimensions of the higher education system and should be designed with care. Multiple interviewees argued in favour of introducing a formula, but only once the overall funding envelope has increased, to insulate institutions that could be too adversely affected. The Government seems to endorse this approach. Option 2.3: Introduce Funding Envelopes to Support Study Success Initiatives. Portugal has previously provided targeted funding to support study success, while Option 1.1 was to introduce targeted funding for institutional services to identify and support students at risk of dropping out. This option is a broader proposal to create a larger fund for initiatives to support study success. This funding could be competitive, with institutions required to submit proposals for support and then report on outcomes.

431 Ibid. 432 Bob Parker and Jonathan Williams, “Getting the Most from Our Universities: A New Approach to System Planning and Funding in Nova Scotia” (Halifax, NS: Students Nova Scotia, January 15, 2012).


163

Impact on completion: The effectiveness of this approach will depend on the quality of projects supported, as well as the amount of funds delivered. It could permit the Government of Portugal to facilitate innovations in study success, however, and then build monitoring and reporting on these initiatives to help disseminate successes across the system. Risks: The funds would require careful administration to identify and support the most promising projects. The Government would also need to establish appropriate reporting mechanisms and ideally evaluation structures to ensure accountability and promote learning from experiences. Flexibility: This approach would be relatively flexible. It could allow the Government of Portugal to cut off funding for programmes that appear unsuccessful and support more promising programmes. Feasibility: The Government of Portugal would need to make sufficient funds available to support the programme and have a meaningful impact. We have already noted that Portugal has significant funding constraints that it would need to overcome. Option 2.4: Remove Numerus Clausus restrictions for non-professional programmes, or introduce conditionality related to study success. Changes to the NC regime would in many ways have similar implications for institutions to the introduction of a funding formula. Abolishing NC could be akin to tying funding to enrolment, but NC allocations could also be made conditional on institutional or programme performance. Impact on completion: I considered the contribution of NC to student dissatisfaction as considerable, and this is a problem the government could address directly through a policy change. This perspective accounts only for effects on students’ motivation, and not additional potential benefits in better incentivising institutions to support their students. Risks: Abolishing NC could disadvantage some institutions more so even than the adoption of an strict funding formula. A 2008 study found that 10 out of 29 public higher education institutions would lose half of more of their students if the NC policy was removed.433 Some interviewees suggested that today the ramifications could be even greater given years of limited public funding and ongoing demographic transition. Private institutions also rely on NC to create unmet demand in urban areas. Even those institutions who gained students could struggle to adapt to their new situation, with implications for their performance. Risks of an incentive-driven NC would again depend on the details of the incentive design.

433 Miguel Portela et al., “Evaluating Student Allocation in the Portuguese Public Higher Education System,” Higher Education 56 (2008): 185–203.


164

Flexibility: The abolition of NC would permit institutions to grow to meet student demand pursuant to their strategic goals. The Government of Portugal would of course lose a significant policy tool it uses to control the structure of the system. Feasibility: The major constraint on feasibility is tolerance for impacts on less competitive institutions and their communities. Cutting off these institutions could be very unpopular. Option 2.5: Develop, administer and publish the results of a national survey of student engagement. Student engagement surveys measure student satisfaction and the quality of the learning environment.434 The EuroStudent survey has many elements of a student engagement survey, but at the system level. The United Kingdom (UK) operates two student engagement surveys. The first is the National Student Survey (NSS), which is compulsory for all publicly funding higher education institutions across the UK.435 Survey questions focus largely on academic elements, although not exclusively. The second is the UK Engagement Survey (UKES), created by the Higher Education Academy.436 It is a more detailed non compulsory survey relating to more diverse elements of student learning and engagement. Students complete the NSS in fourth year, whereas the UKES is targeted to first and second year students. NSS results are published, UKES results are confidential but allow institutions to do some benchmarking. In Canada and the US, many universities participate in the National Survey of Student Engagement (NSSE), while many community colleges (polytechnic-equivalent) participate in the Community College Survey of Student Engagement (CCSSE). The NSSE for example focuses on five themes: academic challenges, learning with peers, experiences with faculty, the campus environment and high-impact practices. Both surveys are annual. CCSSE results are published, while universities have discretion regarding NSSE result dissemination. Impact on completion: We judge limited information for students on the student experience among the least important of all the factors in study success identified in this study because we did not believe this information would significantly influence many students’ decisions on enrolment. It is possible, however, that the collection and publication of student engagement data could create internal momentum and pressure on institutions to strengthen their performance. This would seem particularly important if the government maintains NC and therefore is protecting some institutions from the effects of student demand. More information

434 Danielle Andres and Jonathan Williams, “Focus on Learning: A Student Vision for Improving Post-Secondary Education in Nova Scotia” (Halifax, NS: Students Nova Scotia, May 21, 2014). 435 Ipsos MORI and HEFCE, “The National Student Survey 2017,” National Student Survey, March 31, 2017, http://www.thestudentsurvey.com/about.php. 436 Higher Education Academy, “UK Engagement Survey | Higher Education Academy,” Higher Education Academy, March 31, 2017, https://www.heacademy.ac.uk/institutions/surveys/uk-engagement-survey.


165

could also greatly improve our understanding of many factors in study success identified in Chapter 4, and thereby facilitate improvements in policies and programmes. Risks: Institutions may perceive risks in having greater information available about their performance in student engagement. One fear is that such information can generate simplistic and damaging comparisons. However, the current overreliance on the publication of students’ entering grades for judging quality is arguably much more problematic. It may not be possible to completely displace the emphasis on this information, but information on the student experience could help to improve individual decision-making and public policy debates. Flexibility: Implementing a student engagement survey would not commit the Government or institutions to a broader policy direction of any major significance. Feasibility: Portugal could likely develop such a survey quickly and relatively affordably by obtaining and translating a tool from another jurisdiction. The Government would have to work out the details of survey administration and data storage with institutions. The NSSE and the CCSSE are administered by institutions independently from government – students complete the former online and the latter during class time. Portugal could build upon its experience with the EuroStudent survey as well as the many institutional student satisfaction surveys. It is possible that a common survey could actually help reduce the workload associated with extensive surveys administered at the institutional level. Aligning questions with the EuroStudent Survey could also raise efficiency. Option 2.6: Maintain Portugal’s current higher education steering structure. Impact on completion: Maintaining Portugal’s current steering structure would not be expected to contribute to improvements in completion rates. Risks: There are winners and losers without a funding formula too because funding must still be distributed between institutions. Successful institutions that grow find that they have more limited resources on a per student basis, and institutions that shrink the opposite. This challenge is only aggravated in Portugal’s circumstances, where there is little prospect of enrolment growth, what growth potential exists is unevenly distributed, and the country almost certainly has more institutions than would be viable if all funding accompanied students to wherever they chose to attend. Flexibility: Historical funding models are highly inflexible. The Government can allocate supplementary funds differently, but most funds are already allocated to the system in a frozen distribution. Feasibility: The current funding model may not be legally sustainable because it does not comply with the higher education financing law.


166

2.3 Shaping Expectations of Study Success The policy options in this area would directly address the following factors:

➢ High rates of academic failure; ➢ Difficulties balancing studies and employment; ➢ Grant eligibility criteria may undermine completion; and, ➢ Alternative access streams lead to high failure rates.

Part of the logic of these policies is also their potential to support a greater focus on teaching and serving students, which is largely about faculty. Option 3.1: Tighten rules regarding academic suspension and monitor institutional compliance. Within this policy options, steps could include further extending the period of the second suspension, requiring that students reapply to the study programme after their third suspension, and doubling all calculations of the time to academic suspension and length of suspension for students who receive grades of zero, except in exceptional circumstances. Impact on completion: This policy intent is to indirectly affect student completion, instead it is about creating more authentic norms around evaluations and study success and ending the enrolment of basically fraudulent students. This could help build expectations of enrolled students are there to learn, in turn motivating faculty. This could also help to improve tracking of study success. Risks: Increasing academic suspensions to promote completion also may seem counter-intuitive and based on empirical research from the US, increasing academic suspensions inappropriately could reduce completion, while the returns from staying enrolled could be high for students at the margin of suspension.437 Conversely, it may be that few of the students who would be directly suspended on a path to completion. This policy would cause institutions to lose tuition fees, mostly from inactive students. Flexibility: Compliance monitoring would obviously reduce flexibility by ensuring requirements are respected. However, assuming the Government continued to only set minimum requirements for institutional academic suspension policies, institutions could still adopt stricter policies. Feasibility: The Government might have to change its legislation to implement a new policy. For monitoring compliance, it would also have to determine whether to use the quality assurance system or another form of auditing. Weaknesses in institutional information systems may be a constraint for monitoring.

437 Ben Ost, Weixiang Pan, and Doug Webber, “The Returns to College Persistence for Marginal Students: Regression Discontinuity Evidence from University Dismissal Policies,” Discussion Paper, IZA Discussion Paper Series (Bonn, Germany: IZA, 2016).


167

Option 3.2: Adjust Worker-Student rules to more strongly emphasise study success. This proposal is firstly to eliminate policies that allow Worker-Students longer timelines to complete degrees while retaining financial aid, provide extra allowance for make-up assessments, and exempt these students from compulsory attendance requirements. This elimination could occur wholesale or be targeted based on students’ age, admissions channel, status as having dependents or not, or some other means. Limiting the eligibility for the BE or even Worker-Student Status based on the number of hours work could be another possibility. More specifically, students who work more than say 20-30 hours weekly could be eligible for funding to study only part-time. Impact on completion: These policies would aim to ensure Worker-Student rules attract workers to study, but do not encourage students to work counterproductively long hours. Again, international evidence suggests that students are more likely to drop out if they work for more than 20 hours per week. On this basis, many countries have adopted policies to discourage students from pursuing paid employment as outlined in Chapter 5. This is also consistent with ensuring for faculty that their students can be committed and motivated. One last note is that targeting the youngest students may not be effective because relatively few of these students are working. Risks: This policy could cause some students to lose financial aid and find it more difficult to complete. However, many of these students could again be very unlikely to complete anyway. Flexibility: In restricting eligibility, the Government would need to consider carefully potential targeting and exceptions consistent with its broader policy goals and values. Feasibility: It is difficult to restrict financial aid based on employment is very challenging where financial support is inadequate for many students. A broader expansion of financial aid could complement and improve the feasibility of these policies on working students. Any targeting measures would also need to be vetted to ensure they could not be challenged on grounds of discrimination. Connecting eligibility criteria with students’ hours of employment may be administratively challenging. Finally, it may be difficult politically to amend the Labour Code to change the Worker-Student Statute. The Government of Portugal may be wasting money supporting full-time workers to study full-time when relatively few of these students complete. Changing this could therefore generate cost useful savings. Option 3.3: Increase the flexibility of BE eligibility criteria for academically struggling students. This policy would target the concern about grant eligibility criteria undermining completion. The first possible eligibility change would be to extend the period during which a student may be


168

unsuccessful before losing eligibility, from two semesters to three semesters, with the aim of becoming more consistent with academic suspension regulations. The second change would be to allow students to regain eligibility based on the time-to-completion requirement (clause f of the legislation outlined in Chapter 5) by discounting semesters of study with suspended BE support from the calculation of the student’s pathway. So, for example, if a 3-year Licenciatura programme student loses their eligibility because after two years they fall off the path to complete in four years (3 years +1), allow them to regain eligibility if they study for a year with the BE and end up where they were expected to be when they lost eligibility. Impact on completion: Extending BE eligibility for students who fail classes would expand access to financial aid and notably recognise that even motivated students may face difficulties, especially in first year. The possibility of regaining eligibility could also motivate students who have been unsuccessful to try to regain their grant. On the other hand, the basic eligibility rules for the BE appear important to create some accountability for academic progression that appears to be lacking elsewhere in the higher education system. Risks: Dramatically relaxing these rules could aggravate the tolerance for academic failure that appears to be a significant factor in the overall study success problem. The Government of Portugal could also end up funding unsuccessful students for longer. Flexibility: These policies would create more flexibility with respect to students who are struggling academically. Feasibility: There may be administrative difficulties in processing students’ status after three semesters, if this is processed on a yearly basis. These policies would imply some additional costs from higher eligibility, but these costs would likely be modest. Option 3.4: Review Institutional Admissions Processes under alternative access streams. The Government of Portugal and the CNAES could launch an independent review to determine whether institutional admissions processes under alternative access stream are systematically accepting students who do not have minimal qualifications, and/or giving prospective students a false understanding of their capacity to successfully complete their higher education programme. This review could inform policies to raise standards for admissions to Licenciado programmes, perhaps allowing non-traditional students to more easily access TESP programmes. Impact on completion: Reducing the enrolment of students who are only marginally qualified for higher education would increase the completion rate. Risks: The trade off from this way of improving study success rates is reducing access, whereas evidence from other jurisdictions, cited at multiple points in the report, indicates that students who qualify for higher education at the margin can still secure considerable returns from completing degrees.


169

Flexibility: In principle, the Government of Portugal would retain flexibility to respond to the review as it deems appropriate. Feasibility: There may be some costs associated with the review project, which would also require some policy attention from the MCTES that might otherwise be dedicated elsewhere. Option 3.5: Support an annual Study Success Summit The Government of Portugal could invest in bringing together institutional leaders, faculty, service providers, student representatives, civil servants and international experts to share lessons and best practices in supporting study success. The event should be co-organised with external organisations such as CCISP, CRUP, and A3ES. Themes of focus could change from year to year, to address different factors in study success. Impact on completion: International experiences suggest similar initiatives have reinforced institutional focus on improving teaching and learning. Sharing of best practices could also strengthen efforts across institutions. Risks: A poor quality event could offer limited value to participants and fail to sustain interest or momentum. A key challenge could be to ensure the event is inclusive of different types of institutions and does not become focused on the needs only of some. The event would also have to coordinate with the CNAPPES to ensure it is complementary. Flexibility: Flexibility will depend on how the event is planned and designed. Feasibility: There appear to be few barriers to feasibility. The initiative should be relatively low cost, especially given costs could be shared with higher education institutions and possibly participants. Option 3.6: Maintain the current relevant policies. Impact on completion: Would not be expected to contribute to improvements in completion. Risks: At some point, it may become apparent that inactive “students” are taking advantage of benefits that the Government of Portugal had targeted towards actual students. The reasons why inactive students choose to enroll are very unclear. Flexibility: The Worker-Student Statute and BE eligibility policies restrict the flexibility of many affected parties. Feasibility: It would appear feasible to maintain current policies in these areas.


170

Recommendations We have identified a number of policy options that would help the Government of Portugal initiate a durable improvement in the study success of Portuguese higher education students. We have identified a primary recommendation that would promise the greatest impact under each of the three themes, as well as secondary recommendations that would be valuable complimentary measures.

Support Students at Risk of Dropping Out Primary Recommendation 1: Provide targeted institutional funding for services to identify and support students at risk of dropping out Among all our recommendations, we believe this one promises the most significant short-term returns on investment. It could be implemented more or less immediately, subject mainly to constraints on the capacity of institutional data systems and the use of academic student information for purposes of institutional outreach. Clear bases for identifying students as being at risk of dropping out include:

1. Admission through pathways that are not part of the RGA; 2. Admission through the RGA with a relatively low admissions score; 3. Consistent absences from classes; and, 4. Failed evaluations starting from the beginning of the first semester (i.e. including in

evaluations during the teaching term). It should be possible to target students based on failed evaluations and absences even in the absence of developed data systems, provided that services coordinate effectively with faculty. Developing data systems will allow this targeting to be more efficient however. The programme should aim for students to be contacted directly by the service once identified as being at risk. Supports for the at-risk students could include the following, where needed and available:

• Academic advising on the demands of their study programme, its suitability for the particular student based on their skills and aspirations, and the possibility of changing study programme if relevant;

• Financial aid resources on campus or assistance in applying to other financial aid sources;

• Employment services that assist students in securing part-time employment or planning their career;

• Study and life skills development workshops;

• The student ombudsperson;


171

• Disability supports;

• Psychological counselling; and,

• Health services. Not all services may be initially adequate to the task. However, having the targeting mechanism in place could help inform and drive their continuous improvement. The Government would be advised to require that the institutions track the numbers of students reached, basic characteristics of these students (admissions stream, year of study, gender, etc.), and the reasons why they sought assistance, with suitable privacy protections. This data could be helpful firstly in supporting accountability for the funds provided to support these services, and secondly to further build data on factors in study success. Data collection and service performance should be assessed under A3ES’ institutional accreditation processes. We are not in a position to identify the specific funding requirements for this initiative, and these would almost certainly vary by institution. However, we anticipate this effort could have among the highest possible returns on investment if pursued appropriately. Secondary Recommendation 1: Cancel and replace the requirement that tenured polytechnic faculty have doctoral degrees. This would require polytechnics and the Government of Portugal to develop new professional requirements for polytechnic faculty. It is beyond the scope of this report to speak to these, except to say that they should be firmly grounded in the polytechnic system’s distinct mission by emphasising instructional skills and professional relevance. Recognising that many faculty are currently completing doctoral degrees or have done so already in response to this requirement, the Government should find some way to compensate faculty for the training they have completed even as the sought-after qualifications shift moving forward, without continuing to overly incentivise doctoral degree completion. Secondary Recommendation 2: Introduce legislation requiring institutions to provide annual student progression data to the MCTES Establishing privacy protections will of course be essential, however annual reporting of currently collected data to the MCTES would allow Portugal to inform decision-making on study success with among the best data available anywhere in the world. MCTES could make the data available in appropriate ways to consultants, institutions and even academic researchers. Academic use in particular could improve understanding of the system at essentially no additional cost. This study for example, could have developed much greater confidence with access to the full data on student progression, especially if this were then connected with other ministry data on student grants and admissions.


172

This policy would help to reinforce Primary Recommendation 1 on targeting of students at greatest risk of dropout. It could also help to strengthen A3ES quality assurance processes and help MCTES to determine where institutions or programmes should be prioritised for broader study success measures. The MCTES should also consider steps to communicate the improved data to prospective students. Improved information sharing could create pressure on institutions to improve their performance. Secondary Recommendation 3: Adjust legislation to allow public higher education institutions to hire or otherwise compensate students for non-research activities. Institutions should be permitted to hire students to complete tasks that contribute to the institution fulfilling its mission. There is no reason to further restrict the purposes of student employment.

Other Observations

We believe that responsible student-association-driven student orientation activities could promote improved student integration and engagement. It seems apparent that social integration will remain highly important to students themselves, and if formal, well-meaning student-driven activities are not in place then praxe is likely to fill the void. We are unprepared, however, to endorse a wholesale policy based on the information collected as of this point. Possible intermediate steps direction could include participation in seminars or other activities with institutions and students from other jurisdictions where such activities are well established (e.g. Canada), and pilot projects to test the approach within Portugal.

Steer the System for Study Success Primary Recommendation 2: Include performance criteria relating to study success within an institutional funding formula The Government of Portugal appears committed to introduce a funding formula, thereby fulfilling requirements under the higher education financing law. This would be a positive step, especially if the new formula includes thoughtful criteria that make institutions accountable for their students’ study success. Tying funding to the number of degrees granted would be the simplest measure and an improvement on enrolment based funding. It implies the risk that institutions simply pass students through and would require that quality assurance structures operate effectively to


173

prevent this. Passing too many students does not appear to have been a systematic problem in Portugal, but it should not become one in the future. To ensure equity, Portugal should ensure that the funding formula recognises where institutions are recruiting more disadvantaged students, or running programming associated with lower rates of study success. The formula should adjust funding accordingly either through its study success measure or by including a separate funding envelope to reward institutions for their performance in terms of access for disadvantaged students. Whatever measures are used should be developed carefully and not in a rush. The Government could begin to work on this during its current mandate, but perhaps take longer to put an effective formula in place, given it should consult higher education institutions and other stakeholders extensively. It would be essential to tie enough funding to the instrument to actually affect institutional behaviour. Introducing the new formula in parallel with an overall increase in funding to institutions could help to mitigate any negative short-run impacts and thereby improve feasibility. Secondary Recommendation 4: Develop performance agreements with institutions to promote study success The Government should pursue performance agreements that prioritise study success, as a complement and/or precursor to a funding formula. These performance agreements could focus on the study success issue alone or as one item within a broader higher education system strategy. Study success components should prioritise factors in quality of education delivery identified in Chapter 4, including in particular programme structure, student-centred instruction, and better serving mature students and students with disabilities. Given it will take time to develop effective agreements, the Government almost certainly should not try to reach agreements with all institutions under the current parliament. Focusing first on institutions most aligned with its study success agenda could help the government to learn how to develop these agreements collaboratively and also establish expectations for other institutions. If the government greatly increases institutional funding it would likely help to link this with the performance agreements. The five-year timeline envisioned by the higher education financing law for institutional development agreements would appear to be a suitable maximum length of the agreements. Three years would be a reasonable minimum length if the Government and institutions would prefer shorter agreements. Secondary Recommendation 5: Introduce Funding Envelopes to Support Study Success Initiatives At least as a short-term measure while a funding formula and performance agreements are in development, and as a complement to the targeted funding for services to identify and support


174

students at risk of dropping out, the Government of Portugal would do well to introduce competitive grants for initiatives to support study success. The competition could be overseen by an expert committee independent from the MCTES and should be connected with strong monitoring and evaluation to identify which programmes succeed and fail. Again, providing more funds to such an initiative would heighten its potential impact, but the targeting initiative in Primary Recommendation 1 should be the priority. Secondary Recommendation 6: Launch a student engagement survey, implemented on an annual or biannual basis The Government should work with institutions to implement a survey of student engagement on an annual or biannual basis. The survey would help to help track performance for performance agreements and could even be used within a funding formula. The Government and institutions would have to sort out the details of administration. This should not be too difficult or time consuming, so it seems very possible to have the survey in place during the Government’s current mandate. We would recommend that key indicators be published online as a tool to foster a more holistic understanding of the system than is generated by the current emphasis on entry grades.

Other Observations

We do not recommend significant changes to the NC regime at this time. Almost all interviewees seemed to believe changes were infeasible in the short-to-medium term, largely because the risks of impacts on numerous institutions are simply too great. Greatly destabilising the higher education system in the short-run might lower completion. Nevertheless, Portugal at some point must allow institutions to fail financially if they are failing academically to serve their students, or else it will waste large amounts of money and obtain poor results. For a funding formula to fulfill its potential, the Government will have to show a credible intention to maintain the mechanism even when institutions are losing out. In a similar way, greatly reducing the scope of the NC could be an important step to improve the system in the future, again tied to ensuring students are not forced to attend institutions that are failing to provide a high-quality experience.

Entrench Expectations that Students Succeed The high rates of failure, especially the presence of inactive students, undermine the credibility of commitment to completion system-wide. Nevertheless, multiple policies and processes within the higher education system seem to normalise longstanding patterns where large numbers of students fail academically or otherwise drop out. If the Government of Portugal wants faculty to


175

hold and act upon stronger expectations that their students will be successful, it should take steps to build these expectations more strongly into its own policies. Primary Recommendation 3: Tighten rules regarding academic suspension and monitor institutional compliance. More detailed and stringent requirements for institutional policies on academic suspension should be developed in consultation with higher education institutions. Fulfillment of these requirements should be a criterion in institutional accreditation by A3ES, and A3ES should especially explore measures to audit the shares of students who are serially inactive. The following modifications to the basic requirements for academic suspension are merely suggestions:

a. The maximum registrations per year of study progress prior to academic suspension should be reduced by 1 across all study years.

b. When students return from academic suspension they should be required to pass at least 50% of their courses on a yearly basis or be suspended a second time.

c. Students should be required to reapply for admission after their third academic suspension.

d. Academic failures where students did not participate in any evaluations should have double the weight for the purposes of determining academic suspension.

e. Exemptions from these conditions should be permitted for students facing exceptional attenuating circumstances, as determined through an institutional appeals mechanism.

There is no question that higher rates of academic failure are a by-product of expanding access to higher education to students with less academic preparation, often because they have faced systematic disadvantage in their earlier education. To preserve equity, the proposed appeals mechanism is essential and this recommendation should complement the recommendation on developing services to identify and support students at risk. Further research regarding the relationship between academic failure and study success in Portugal could help to refine this mechanism and the targeting services from Primary Recommendation 1. Institutions should be collecting revenues to actually deliver education, not simply to provide students access to incidental privileges. Hence the loss of tuition revenues that institutions collect from serially unsuccessful/inactive students should not be a bar to adopting this policy. Secondary Recommendation 7: Adjust Worker-Student rules to more strongly emphasise study success. For the Government of Portugal should rebalance its legislation and policies to place a greater emphasis on study success among employed students. More specifically, we recommend the following changes to the Worker-Student Statute:


176

a. Eliminate exemptions for student-workers with respect to timelines for completion of degrees.

b. Remove the requirement for institutions to provide Student-Workers make-up assessments.

c. Eliminate the wholesale exemptions from compulsory attendance requirements for Student-Workers (more modest exemptions could be adopted instead).

d. Limit eligibility for Worker-Student Status to part-time students when workers work over a threshold set of hours.

Furthermore, we recommend the following adjustments to BE eligibility policies:

a. Allow students with more than 20 hours per week of paid employment only to receive the BE to pursue part-time studies; and,

b. Eliminate the 1-year extension of the time-to-completion requirement (clause f) for Worker-Students.

The hours-of-work based limits on eligibility would aim to promote part-time studies among students who work many hours. The 20-hours threshold seems very appropriate for BE grants based on international evidence, but with respect to Worker-Student Status a somewhat higher limit could be reasonable given that the policy is not funding these students. Any limit could also use a different method from the single step of full and part-time status, such as limits tied to the specific number of courses in which the student is registered. Whatever approach taken needs to be administratively feasible, including in terms of tracking students’ hours of work. These policies would not be expected to increase the cost of government programmes since they will constrain eligibility. However, given that financial aid is likely inadequate for many students it may be difficult to justify restricting funds based on hours worked, even if these funds likely would not have efficiently promoted these students’ success. Implementing these policies alongside wider expansion of financial assistance would help mitigate these problems. Secondary Recommendation 8: Adjust the time-to-completion related eligibility criterion for the BE to allow students to regain eligibility through successful study without BE support In approximate language, our recommendation would be to institute the following: Where a student loses eligibility due to clause f), the subsequent semesters of study pursued by the student while not receiving the BE will not be counted in calculations for eligibility based on clause f). The student may regain eligibility under clause f) based on their study period prior to the loss of eligibility and total credits completed before and after the loss of eligibility, provided they fulfil all other eligibility criteria. We are uncertain how many students lose eligibility based on the time-to-completion requirement. The cost of this policy would be determined by this number and the number of students who would work to regain eligibility as a result of this policy change. In other words, the more effective the policy is, the higher its cost.


177

Secondary Recommendation 9: Review Institutional Admissions Processes under Alternative Access Streams A guiding principle for Portuguese higher education should be that students should not be admitted to programmes if they almost certainly do not have the skills to succeed. A student admissions letter should indicate that the institution believes the student can succeed, and moreover represent a commitment by that institution to endeavour to help the student succeed by all reasonable means. The Government of Portugal and the CNAES should review the quality of institutional admissions processes for alternative access streams. The Review should not seek automatically to make admissions requirements more demanding, but instead to review whether minimal requirements reflect a credible commitment to students’ success and fulfil a minimum level of rigour. Improved information collected pursuant to other recommendations could help highlight institutions for special attention. If Portugal determines that admissions processes should be stricter for Licenciatura programmes, it could retain more open admissions for TESP programmes. TESP programmes may better target the needs of many students accessing alternative admissions processes, and could serve as a suitable stepping stone to Licenciatura degrees. The effectiveness of such an approach will depend on how effectively institutions implement the TESP model over the coming years. Secondary Recommendation 10: Support an annual Study Success Summit The Government of Portugal should collaborate with CCISP and CRUP to create an annual summit for stakeholders and experts to explore the challenges facing students and institutions, and learn from local and international experiences. The Summit could rotate between institutions that have demonstrated strong commitment to study success, as a way to recognise their efforts and provide them the opportunity to exercise leadership for the whole system.

Other Observations

We do not recommend relaxing requirements for BE recipients to pass classes (clause e), to maintain expectations that students will be academically successful. Assuming Portugal implements Primary Recommendation 1, students should receive considerable support if they are beginning to fail classes. If Primary Recommendation 1 is not implemented, then it may be justifiable to grant students an additional semester of eligibility for financial aid (from the time of first enrolment in first year) based on the proportion of classes passed (clause e).

June 2017 Addressing the Completion Challenge in Portuguese Higher Education Appendix I

179

Appendix I: List of Interviewees

➢ Fernando Almeida, Pro-President, Instituto Politécnico de Setúbal

➢ Alberto Amaral, President, A3ES

➢ João Baptista, General Sub-Director, DGEEC

➢ Pedro Barrias, Adjunto, MCTES

➢ António Firmino da Costa, Professor, University Institute of Lisbon (ISCTE – IUL)

➢ Pedro Dominguinhos, President, Instituto Politécnico de Setúbal and representative of

the Conselho Coordenador dos Institutos Superiores Politécnicos (CCISP)

➢ António Feijó, Vice-Rector, Universidade de Lisboa

➢ Paulo Ferraz, Social Services Director, Instituto Politécnico do Porto

➢ Daniel Freitas, Student Representative, Conselho Coordenador do Ensino Superior

➢ Nuno Lima, Técnico Especialista, MCTES

➢ Delminda Lopes, Vice-President, Instituto Politécnico do Porto

➢ Susana Paula Veiga da Rocha Oliveira, Professor, Universidade de Lisboa

➢ Eduardo Pereira, Vice-Rector, Universidade de Lisboa and representative of the

Conselho de Reitores das Universidades Portuguesas (CRUP)

➢ José Pimentel, student representative, Conselho Coordenador do Ensino Superior

➢ Rosalina Pisco Costa, Pro-Rector, Universidade de Évora

➢ Fernando Remião, Pro-Rector, Universidade do Porto

➢ Carlos Farinha Rodrigues, Assistant Professor, ISEG

➢ Cláudia Sarrico, Higher Education Policy Analyst, OECD Directorate for Education and

Skills

➢ Felipe Silva, Chief of Staff for the Secretary of State, MCTES

➢ José Amado da Silva, Rector, Universidade Autónoma de Lisboa and representative of

the Associação Portuguesa do Ensino Superior Privado (APESP)

➢ Pedro Teixeira, Vice-Rector, Universidade do Porto

➢ Thomas Weko, Senior Analyst, OECD Directorate for Education and Skills

June 2017 Addressing the Completion Challenge in Portuguese Higher Education Appendix II

181

Appendix II: Summary of Analysis of Policy Options

Policy Option Targeted Policy

Problem Key Actors

Key Steps to Implement

Key Obstacles Timeline

1.1 Provide targeted institutional funding for services to identify and support students at risk of dropping out

Failure to assist students at risk of

dropping out

MCTES, HEIs, HEI staff and

faculty

Secure funding allocation, negotiate

with institutions

Funding requirements, administrative

capacity

1-2 years

1.2 Cancel the requirement that tenured IP faculty have doctoral degrees

The unclear binary distinction

MCTES, HEIs, faculty, A3ES

Change MCTES requirements, develop new guiding standards,

explore measures to compensate faculty

with PhDs

Clarifying system vision, compensation for faculty and HEIs

who have responded to current incentives

1-2 years

1.3 Adjust legislation to allow public higher education institutions to hire or otherwise compensate students for non-research activities

Limited student engagement;

Insufficient student financial aid

Assembly of the Republic,

MCTES

Bring forward legislation, secure

stakeholder backing (HEIs, student associations)

Legislative process, possible political

controversy 1-2 years

1.4 Develop partnerships with student associations to allow them to coordinate formal student orientation activities, with strict requirements to protect safety and social inclusion

Underdeveloped first year student

integration; Limited student engagement

MCTES, HEIs, student

associations

Reach agreement with stakeholders, create

instrument for funding orientation activities,

establish standards for activities in

consultation

Political controversy, securing funds,

establishing new responsibilities for

student associations

1-2 years


182


Problem Key Actors



1.5 Introduce legislation requiring institutions to provide annual student progression data to the MCTES

Failure to assist students at risk of

dropping out; Insufficient information

on the student experience

Assembly of the Republic, MCTES, HEIs

Investigate appropriate privacy

protections, introduce legislation

Legislative process, addressing privacy

concerns, developing suitable information

system

1-2 years

2.1 Develop performance agreements with institutions to establish expectations of improvement in study success.

Uneven institutional commitment to study

success; Weak incentives in the

institutional funding structure

MCTES, HEIs

Create plan for performance

agreements, negotiate with HEIs, secure

associated funding allocation if necessary

Negotiation, securing funding if necessary

1-5 years

2.2 Tie institutional funding to study success related activities, outputs or outcomes through a funding formula

Uneven institutional commitment to study

success; Weak incentives in the

institutional funding structure

MCTES, HEIs

Consult with stakeholders, design

funding formula consistent with system

goals, phase-in(?)

Risk of distributing funds between HEIs, political controversy

3-5 years

2.3 Introduce Funding Envelopes to Support Study Success Initiatives

Limited targeted funding for key

initiatives MCTES, HEIs

Secure funding allocation, negotiate

standards with institutions

Securing funds, effective

administration 1-2 years

2.4 Remove Numerus Clausus restrictions for non-professional programmes, or introduce conditionality related to study success

Numerus Clausus restricts access to

sought-after programmes; Uneven

institutional commitment to study

success;

MCTES

Consult with stakeholders, design

new approach consistent with system

goals, phase-in(?)

Risk of distributing funds between HEIs, political controversy

3-5 years


183


Problem Key Actors



2.5 Develop, administer and publish the results of a national survey of student engagement

Uneven institutional commitment to study success; Insufficient information on the student experience

MCTES, HEIs Develop plan for survey with HEIs, develop survey

Securing funds, reaching agreement

on use of information, design and

implementation of survey

2-3 years

3.1 Tighten rules regarding academic suspension and monitor institutional compliance

High rates of academic failure; Insufficient

focus on teaching and serving students

MCTES, possibly the Assembly of the Republic

Consult with HEIs and other stakeholders,

establish new standards

Political controversy, potential loss of

institutional revenues, requires careful

design

1-2 years

3.2 Adjust Worker-Student rules to more strongly emphasise study success

Students have difficulties balancing

studies and employment

Assembly of the Republic,

MCTES

Consult with stakeholders,

introduce legislation and adjust MCTES

policies

Political controversy, legislative process

2-3 years

3.3 Increase the flexibility of BE eligibility criteria for academically struggling students

Financial Aid eligibility criteria may undermine

completion MCTES Revise current policies

Risk of increasing costs

1-2 years

3.4 Review Institutional Admissions Processes under alternative access streams

Alternative access streams lead to high

failure rates

MCTES, HEIs, CNAES

Establish review team and terms of

reference, secure funding

Securing funds, risk of limiting access

1-2 years

3.5 Support an annual Study Success Summit

Uneven institutional commitment to study success; Insufficient

focus on teaching and serving students

MCTES, HEIs, A3ES

Secure funding, consult with stakeholders

Secure buy-in from stakeholders, secure

funds 1-2 years


184