towards better literacy achievements - unesco · to mariano palamidessi, paula scalliter and...
Post on 11-Oct-2018
215 Views
Preview:
TRANSCRIPT
Towards better literacy achievements in the 2nd
grade of primary schools
http://www.iipe-buenosaires.org.ar
Towards better literacy achievements in the 2nd
grade of primary schools
http://www.iipe-buenosaires.org.ar
To the Director of the IIEP-UNESCO, Mr. Khalil Mahshi, and to the Head of IIPE-Buenos Aires, Ms. Margarita Poggi, who enabled a mechanism to support my participation in the
specialization phase of Advanced Training Programme (ATP).
To my supervisors, Muriel Poisson and Mariano Palamidessi, for the time spent in the attentive revisions and valuable guidance, and also for their personal support, which made me move
forward and complete this memoir.
To the Training Unit of the IIEP Paris and the team of professionals highly committed with education, especially Gabriele Göttelmann-Duret, Yasmin Haq, Patricia Dias da Graça,
Véronique Quénehen, Ludivine Gargam, Natalie Frederic and Jimena Pereyra.
To Mariano Palamidessi, Paula Scalliter and Cecilia Mata, from the IIEP-Buenos Aires, who offered all their support and guidance during the Regional Course in 2009, and more recently
helped me with the preparation of my participation in this second phase in Paris.
To Mauricio Holanda Maia, Marcia Campos, Lucidalva Pereira and Carmilva Flores, from the Ceará State Secretariat for Basic Education, for their availability and support, especially for the
interviews phase.
To my colleagues from the ATP 2011/2012, and those who were with me in the Regional Course in 2009, who taught me new ways of seeing and reading the world, and helped me to become more aware of the wonderful differences and similarities that characterize people around the world and the importance of education for all in our countries. To you all I offer my respect and affection. To the Board of the House of Representatives, who authorized my absence from my labour activities, for its commitment with the enhancement of the skills and competences of its employees as a form of strengthening the legislative branch in Brazil.
To my colleagues from the Legislative Consultancy of the Brazilian House of Representatives who were generous to encourage my participation in the ATP while they were overburdened in my absence. Finally, I would have never been able to have this professional and personal enriching experience without the support and love of my dear family, especially my husband Daniel, a companion in all dreams, and my son Gustavo and daughter Beatriz, who once again showed
how wonderful children they are.
Towards better literacy achievements in the 2nd
grade of primary schools
http://www.iipe-buenosaires.org.ar
AL-CE Legislative Assembly of Ceará
ANEB National Basic Education Assessment
ANRESC National Assessment of Educational Achievement
APRECE Association of Ceará‟s Municipalities and Mayors
CCEAE Ceará State Committee for Eradication of Illiteracy at School
CTT Classical Test Theory
CEC/CD Education Committee of House of Representatives
CNE National Education Council
COPEM Coordinating Body for Cooperation with the Municipalities
CREDES Regional Coordinating Bodies for Education Development
EGRA Early Grade Reading Assessment
FUNDEF Fund for Maintenance and Development of Primary Education
FUNDEB Fund for Maintenance and Development of Basic Education
HDI-M Human Development Index-Municipal
ICMS Value-Added Tax
IDE-Alfa Literacy Performance Index
INEP National Institute for Educational Studies and Research
IRT Item Response Theory
MEC Ministry of Education
NPE National Plan of Education
NRCOM Regional Centre for Cooperation with the Municipalities
PAIC Literacy Program at the Right Age
PCN National Curricular Parameters
PNLD National Program of Textbooks
SAEB National Evaluation System for Basic Education
SARESP São Paulo Education System of Learning Assessment
SEDUC Ceará State Secretariat for Basic Education
SME Municipal Education Secretariat
SPAECE Ceará‟s Permanent Evaluation System for Basic Education
SPAECE-Alfa Literacy Assessment System of Ceará
UFC Federal University of Ceará
UNDIME/CE National Union of Municipal Education Officer - Ceará
UNDP United Nations Development Programme in Brazil
UNESCO United Nations Educational Scientific and Cultural Organization
UNICEF The United Nations Children‟s Fund
WCEFA World Conference on Education for All
Towards better literacy achievements in the 2nd
grade of primary schools
http://www.iipe-buenosaires.org.ar
Towards better literacy achievements in the 2nd
grade of primary schools
http://www.iipe-buenosaires.org.ar
Towards better literacy achievements in the 2nd
grade of primary schools
http://www.iipe-buenosaires.org.ar
This paper describes and analyses the Programa Alfabetização na Idade Certa, (PAIC)
Literacy Program at the Right Age, which has been conducted by the Ceará State Secretariat
for Basic Education since 2007, aimed at better literacy achievements in the 2nd grade of
primary schools. The paper is focused on the program‟s design and implementation and its
highlights: a) the facts that influenced the decision-making process to formulate PAIC and
framed its design; b) the main features of the collaborative action structured by the State of
Ceará and its municipalities to implement the program; c) the role of the two kinds of external
learning assessments implemented under PAIC.
The program is based on two main messages - focus on learning and accountability for results.
It has managed to turn these messages into well-defined targets and goals and effectively
communicated them to all stakeholders. The state of Ceará and its municipalities forged a
model of intergovernmental cooperation that distinguishes itself from past experiences by
establishing a new institutional arrangement; creating a systemic approach and a
comprehensive set of actions to all municipalities, regardless of political bias; and improving
technical capacities in SME management and monitoring, among other aspects.
The experience of PAIC underlines that external and large scale learning assessments may
serve a variety of purposes if they are properly designed and have findings communicated to
the relevant audience. In Ceará, the findings have been disseminated in a way that system
and school managers are able to grasp and to use them to improve management practices
and pedagogical routines. The dissemination of a “culture of monitoring and evaluation” in the
education sector has been reinforced by financial incentives to the local government (by tax
sharing) and to schools (by awards and financial assistance).
The paper concludes with some lessons learned from this experience that might be useful for
other Brazilian states and municipalities which face the same challenge of fighting illiteracy at
primary schools.
Towards better literacy achievements in the 2nd
grade of primary schools
http://www.iipe-buenosaires.org.ar
1.1 Socio-economic and educational setting in Ceará1
Ceará, one of the 27 Brazilian states, is 148,825.6 km² and corresponds to almost 10% of the
Northeast Region. The population was estimated at 8.4 million in 2010 and is spread over 184
municipalities. Apart from the capital Fortaleza, other major cities are Juazeiro, Crato and
Sobral2. Most of Ceará‟s territory (93%) is in the semi-arid region, with a predominantly
tropical climate. Given the aridity, droughts are periodic and sometimes the population,
especially in the area of caatinga, faces problems of water scarcity.
Like most of the country, the state went through an accelerated process of urbanization and
paid a high price for that, facing many problems to provide adequate public services. In 1950,
the urbanization rate in Ceará was 25%, reaching 65.4% in 1991 and 77% in 2010, still under
the 84% of Brazil.
From the 1960‟s to the 1980‟s, prolonged droughts and low growth led large numbers to
migrate to the Southeast Region which concentrated the industrialization of the economy. This
migration flow is a highlight in the Brazilian demography dynamics. Besides the phenomenon
of droughts, agriculture with low productivity, huge income disparities and large land
concentration also contributed to intense migration. However from the 1990s on economic
crises and transformation of labor markets reversed this trend and many north easterners
made their way back.
Economy
Since 2000s, a process of economic growth has been taking place in much of the Northeast.
The first cycle of economic development was heavily based on tax incentives to attract large
companies. In the last decade, Ceará chose different strategies by investing in activities that
have helped to create a new perspective of economic development, such as: i) better
organization of the tourism sector, which gives rise to more employment opportunities and
income generation, not only in the coast, but also in the back lands of the state; ii)
implementation of new economic arrangements that seek to maximize local productive
vocations; iii) activities related to the exportation sector both with the production of goods and
provision of docking facilities, such as the port of Pecém.
Currently, the economy of Ceará has developed a good profile with respect to the distribution
of productive activities, with predominance of the service sector (69.3%) compared to the
1 All data in this section was retrieved on February,10 2012 from Instituto Brasileiro de Geografia e
Estatística (IBGE) and Instituto de Pesquisa e Estratégia Econômica do Ceará (IPECE), available on www.ibge.gov.br and www.ipece.ce.gov.br/categoria5/ceara-em-numeros, respectively. 2 There are at least two interesting historical facts about Ceará: it was the first state to abolish slavery in
Brazil, four years before the national act determining the end of slavery in 1888; and it also pioneered in the first democratic election in 1985, after twenty years of military dictatorship, electing a woman from the left-wing worker‟s party to be mayor of the capital.
Towards better literacy achievements in the 2nd
grade of primary schools
http://www.iipe-buenosaires.org.ar
agricultural sector (7.1%) and the industrial sector (23.6%). In 2008, the State held the 12th
GDP of the country, but it is noteworthy that its relative share in the national GDP was only
2%, with the Brazilian economic activity still concentrated in the south and southeast (relative
shares of São Paulo 33.9%, Rio de Janeiro 11.2% and Minas Gerais 9.1%).
In line with the national growth wave, the state goes through a period of economic prosperity.
The Gross Domestic Product, at current values, increased 32.9% from 2002-2007. The budget
revenues in the same period grew by 88.4% and the expenditure by 84.2%. GDP per capita
grew on average 3.6% per year in the period 2000-2008, above the national average of 3.2%.
Nevertheless, it is still a poor state considering the GDP per capita in Brazil. In 2008, GDP per
capita of Ceará was R$3,681 less than half of R$8,277, the national average.
Social Indicators
The improvement in some social indicators is quite significant. The Human Development
Index-Municipal (HDI-M), produced by the United Nations Development Programme in Brazil
(UNDP), shows that the country has evolved from 0.696 to 0.766 between 1991 and 2000.
Ceará has evolved in a higher range from 0.593 to 0.700 in the same period.
The infant mortality rate3 in the state was 54.2% in 2000 and reached 27.6% in 2009. In turn,
the illiteracy rate among those aged 15 or older went down from 31% in 1996 to 18.6% in
2009. Despite these improvements, regional disparities still persist. The infant mortality rate in
Ceará is more than twice of that identified in Rio Grande do Sul (12.7%) in 2009, and the
illiteracy rate is over five times of that recorded in the Federal District in 2009 (3.5%).
Educational Setting4
a) Access and efficiency indicators
The state has made significant progress in its access indicators, especially in the last decade.
In 2000, the gross enrolment rate in pre-school and secondary was 48.7% and 55.9%,
respectively. In 2009, both were around 80%. In primary, gross enrolment rate went down from
123.2% to 107.6% in the same period. Ceará presents enrolment rates in pre-school above the
regional and national rates; however, it faces problems related to quality5. In primary school,
gross and net enrolment rates are quite aligned with the national rates. In secondary, net
enrolment rate is substantially lower than the gross rate. (Table 1)
3 Mortality rate at age one per thousand live births.
4 See appendix I for information on Brazilian Education System – Structure and Organization.
5 For additional information see: Maria Malta Campos and Jodete Füllgraf. “A qualidade da educação
infantil brasileira: alguns resultados de pesquisa”. In Cadernos de Pesquisa, v.36, n.127, p.87-128, jan-abr. 2006. Available on: http://www.fcc.org.br/pesquisa/eixostematicos/educacaoinfantil/pdf/aqualidadedaeducacaoinfantilbrasileira.pdf Fundação Carlos Chagas. “Educação Infantil no Brasil: avaliação qualitativa e quantitativa”. 2010. Available on: http://www.fcc.org.br/pesquisa/eixostematicos/educacaoinfantil/pdf/sumario_executivo.pdf
Towards better literacy achievements in the 2nd
grade of primary schools
http://www.iipe-buenosaires.org.ar
Table 1 - Enrolment rates in basic education-2009
Level
Gross
Rate
Pre-
school
Net Rate
Pre-school
Gross
Rate
Primary
Net Rate
Primary
Gross Rate
Secondary
Net Rate
Secondary
Ceará 80.0% 68.9% 107.6% 93.5% 79.2% 49.6%
Northeast 67.3% 59.5% 110,.% 89.4% 77.4% 39.2%
Brazil 62.2% 50.6% 105.7% 91.1% 83.0% 50.9%
Source: PNAD/IBGE; Todos pela Educação.
The state deals with a history of high repetition rates and over aged students. In Ceará, over
one third of the students enrolled in secondary are older (at least two years more) than the
recommended age, but in the Northeast Region the rate is even higher. In the period 1995-
2005, after the strong expansion of the education system, some efficiency indicators got
worse. (See appendix II).
Nevertheless, the state has managed to progress in recent years. For example: in the second
cycle of primary education6, over aged students rate went down from 73.7 to 31% between
2001 and 2010. In 2005, in the first cycle of primary, 79% of students had been approved, 15%
had failed and 6% had not completed the school year. In 2009, the rates improved
significantly: 87%, 10% and 2%, respectively.
b) Learning outcomes
The states in the Northeast Region, and Ceará is no exception, have performed learning
outcomes below the national average since the Sistema de Avaliação da Educação Básica,
SAEB (National Evaluation System for Basic Education) was implemented in 1995.
In 2003, the Instituto Nacional de Estudos e Pesquisas Educacionais, INEP (National Institute
for Educational Studies and Research) released a study on the performance of students in 4 th
grade, stating that 22.2% of them had performed very poorly in Portuguese. Those students
were considered illiterate or poorly literate, unable to respond to the simpler items in the test.
The results were bad throughout the country, but significantly worse in less developed regions,
North and Northeast. In the Northeast, the total number of students placed in the critical and
very critical stages of performance was around 70% (Table 2). (INEP, 2003)
6 Currently, the enrolment of all children aged 6-14 in primary education is mandatory. A constitutional
amendment passed in 2009 extends this requirement for all the population aged 4-17 years from 2016 on. In Brazil, primary education is divided in two cycles: 1st/5th grades and 6th/9th grades.
Towards better literacy achievements in the 2nd
grade of primary schools
http://www.iipe-buenosaires.org.ar
Table 2 - Percentage of 4th graders according to the competences developed in Portuguese – 2001
Levels Brazil
Regions
Northeast North Southeast South Midwest
Very Critical 22.2 22.6 33.4 15.8 13.5 20.5
Critical 36.8 44.9 41.8 30.8 35.7 39.2
Intermediate 36.2 31 22.9 45.2 45.8 36.6
Adequate 4.4 1.4 1.8 7.4 4.8 3.3
Advanced 0.4 0.1 0.1 0.8 0.3 0.4
TOTAL 100.0 100.0 100.0 100.0 100.1 100.0
Source: (INEP, 2003)
According to the scales of competences defined by SAEB, the minimum scores to be achieved
by students in grades assessed by the system are:
5th grade of Primary – 200 points in Portuguese and 225 in Math;
9th grade of Primary – 275 points in Portuguese and 300 in Math;
3rd grade of Secondary – 300 points in Portuguese and 350 in Math.
Based on these learning expectations, it can be said that the whole system – state, regional
and national levels perform far below what is considered minimum achievements, but still
some progress have been made in recent years. Table 3 shows that Ceará has got better
results in terms of average proficiency in Portuguese in the first cycle of primary education. It
raised the measure in 23 points between 2005 and 2009, almost the double compared with
national and regional levels. There is a relevant and positive change in the second cycle too.
The progress is lower in secondary education.
Table 3 – Average scoring in Portuguese according to SAEB – 2005-20097
Levels
Primary 4th/5th grade Primary 8th/9th grade Secondary 3rd grade
2005 2009 range 2005 2009 range 2005 2009 range
Ceará 157.0 180.1 23.1 217.9 238.0 20.1 257.8 263.1 5.3
Northeast 155.2 167.3 12.1 219.5 232.4 12.9 246.2 258.4 12.2
Brazil 172.3 184.3 12.0 231.8 244.0 12.2 257.6 268.8 11.2
Source: Todos pela Educação; INEP/MEC
7 Brazilian education system undergoes a transition to compulsory basic education from eight to nine
years. This explains why the data is presented to 4th/5th grades and 8th/9th grades.
Towards better literacy achievements in the 2nd
grade of primary schools
http://www.iipe-buenosaires.org.ar
Ceará has been especially successful in improving literacy achievements in the 2nd grade of
primary education. Since 2007 the State has implemented the Programa Alfabetização na
Idade Certa, PAIC (Literacy Program at the Right Age), with pedagogical and managerial
actions focused on better literacy achievements in the two first years of schooling.
According to the Sistema Permanente de Avaliação da Educação Básica do Ceará –
Alfabetização, SPAECE-Alfa (Literacy Assessment System of Ceará), in 2007 only 26% of the
municipalities had average scores corresponding to the first two levels (sufficient and
recommended) of the scale used to assess proficiency in reading for second graders. In 2010
this percentage came to 99%.
Table 4 shows the evolution of second graders performance in terms of average proficiency in
reading after the program‟s adoption which reached its target in 20108. In this period, the
percentage of students considered illiterate dropped from 33 to 7% and the percentage of
students grouped in the recommended stage increased from 30 to 55%.
Table 4 - PAIC results: average proficiency and percentage of students in each level of the scale of
reading proficiency 2007-2010
Year Average
Proficiency Illiterate
Incomplete
Literacy Intermediate Sufficient Recommended
2007 118.9 32.8 14.6 12.7 9.9 29.9
2008 127.7 20.0 16.2 17.3 15.7 30.8
2009 142.5 14.4 14.0 15.7 14.5 41.5
2010 162.7 6.9 9.4 13.0 15.8 54.9
Source: SEDUC/SPAECE-Alfa
In short, Ceará used to be depicted as a poor state, with low learning outcomes and disparities
in the last decades, as well as the Brazilian Northeast. More recently, it has been going
through a period of economic development and of improvement of some social indicators. It is
expected that some structural policies such as education may pave the way for future
sustainability of this process. As seen above, the latest results from SAEB and SPAECE-Alfa
have shown that the state has been successful in its efforts to raise learning outcomes in
language for students in primary schools.
1.2 The topic and its rationale
This study will focus on the design and implementation of the Literacy Program at the Right
Age (PAIC) that has been conducted by the Secretaria de Educação Básica do Ceará, SEDUC
(Ceará State Secretariat for Basic Education). The study will cover the period from 2007 to
8 Ceará state education system has established the recommended proficiency over 150 points.
Towards better literacy achievements in the 2nd
grade of primary schools
http://www.iipe-buenosaires.org.ar
2010 when the program was successful in raising learning achievements in reading for 2nd
graders.
The changes that occurred in Ceará are highly interesting for the whole country. Brazil has
managed to enroll 97% of school-age children (from 6 to 14 years old) in primary education,
but it is not surprising that after eight or nine years of schooling a young boy or girl still
performs reading and writing at insufficient levels. This is one of the most serious quality
problems because the learning gaps begin in the early years of schooling, deepen with time
and tend to become unmanageable towards the end of basic education. It is a great loss to
children, particularly the poor ones.
This poor learning performance is especially serious for the poorest states. By failing to ensure
effective early learning, school also fails in its role of contributing to reduce inequalities. This
program is particularly interesting because Ceará is located in the Northeast of Brazil. The
region is historically marked by higher levels of poverty, low economic development and
educational indicators are frequently below national average, as seen in the previous section.
Brazil is a federation with three government levels that has faced difficulties in implementing
collaborative actions to induce changes in quality and equity education issues. Furthermore,
there are difficulties at all government levels to use the data generated by the system of
learning assessment to guide system management, policy options and decision making.
In Ceará, the governments of state and municipality levels are working in a collaborative action
for implementing PAIC and have introduced a set of initiatives to address the problem of
illiteracy at school, among them specific learning assessment for literacy outcomes as a tool to
guide planning and monitoring overall program results. This study is based on the premise that
intergovernmental collaborative action and external assessment are the main aspects of PAIC
and that the context where the decision-making process to implement the program took place
forged this collaborative action and framed the program design.
A number of national academic studies have tackled primary education quality issues. Most of
them are focused on the evolution of learning outcomes at the end of the primary cycles (5 th
and 9th grades) based on national data. Little effort has been dedicated to this specific issue
(better literacy outcomes in early child literacy and programs designed to achieve them), and
even less attention has been driven to how these programs have been designed and
implemented.
Currently, there is a bill on setting up a new National Plan of Education (NPE) at the Brazilian
National Congress that includes a goal on children‟s literacy. Given the momentum, it is an
opportunity for new learning on the subject. The new plan is pointing to the need for
government action towards a more effective literacy process in the first years of primary
education.
Therefore, the aim is to contribute to the design and implementation of other institutional
interventions in states and municipalities where children have also shown low levels of reading
and writing.
A better and deeper understanding of this program can also be useful to the National
Congress in fulfilling its responsibilities for following up and evaluating the NPE‟s
Towards better literacy achievements in the 2nd
grade of primary schools
http://www.iipe-buenosaires.org.ar
implementation and for supporting the accomplishment of its goals by approving laws and
regulations that can induce it.
1.3 Problematic
Considering that child‟s literacy at appropriate age will gain even more prominence when the
new NPE is approved, and in order to assist the states and municipalities with some
challenges they may face in fulfilling the goal, this study hopes to shed light on how the State
of Ceará has managed to turn literacy achievements into a priority for the municipal education
systems during the period 2007-2010. What are the main features of the collaborative action
and the external learning assessment put in place by the state? In the light of this main guiding
question, there are selected critical issues that will be described and analyzed in this study.
They are presented in the following questions:
1. What are the main policies and practices related to child‟s literacy in Brazil? What is the
legal framework for intergovernmental collaborative action and what kind of cooperation is
developed in this field? How child‟s literacy has become the focus of learning assessments?
2. What was the context for the emergence of PAIC in Ceará and what are the relevant actors
in this process? Why and how was a collaborative action between State and its
Municipalities structured to address the problem?
3. What is the role of external evaluation in the design of PAIC?
4. What lessons can be drawn from the experience implemented in Ceará?
1.4 Objectives of the study
In order to respond to the questions posed above, the study has identified the following
objectives:
- To identify key concepts, relevant issues and policies related to literacy in Brazil. To
understand the legal framework for intergovernmental collaborative action in the
Brazilian federalism. To identify the main characteristics of learning assessments and
to analyze how they were linked to literacy achievement.
- To describe relevant facts and the role of main stakeholders involved in the events that
preceded the implementation of PAIC and to analyze how they have influenced the
design of the program. To describe PAIC main features and to identify the framework
for the collaborative action that was established.
- To describe and analyze the design and the uses that external assessment has taken
in PAIC‟s implementation.
- To identify lessons that can be drawn from the experience implemented in Ceará.
1.5 Methodology Aspects
The study utilizes descriptive and explanatory design and is based on primary and secondary
sources of information. In order to meet the objectives stipulated by the study, different types
of data were collected and analyzed. A brief literature review on child‟s literacy in Brazil and
the legal framework related to intergovernmental collaborative action will help to place the
program in the Brazilian context and hopefully make it easier to understand its conception and
Towards better literacy achievements in the 2nd
grade of primary schools
http://www.iipe-buenosaires.org.ar
relevance. The literature review also tries to deal with the characteristics and expansion of
learning assessments since this is an important dimension of PAIC.
Data on the context of PAIC‟s emergence, design and implementation was obtained from
institutional documents review, relevant reports and records available on the Internet. The
statistical data is gathered on PAIC‟s webpage. National statistics on learning outcomes used
as a means of comparison are obtained from the INEP webpage.
Besides that, data was also collected through six interviews, conducted on the 14 th, 15th and
16th May 20129. At state level, the interviews were with two key agents, both responsible for
managing the program in SEDUC: the Coordinator of Coordenadoria de Cooperação com os
Municípios, COPEM (Coordinating Body for Cooperation with the Municipalities) and the
leader of the Management of State Programs Cell. At municipal level, they were with two
Municipal Secretaries of Education and two PAIC City Managers. The interviews intended to
collect information on the structure established, constraints and challenges that program
managers have faced. They should also complete, refine or correct the findings derived from
the document analysis.
Two municipalities (São Luiz do Curu and Caucaia) were chosen based on the different scale
of its education system (one is considered small and the other large by Ceará patterns). The
other selection criterion was the system performance through SPAECE-Alfa results in 2007-
2010. One managed to take a great leap (more than 100 points in average proficiency) and the
other has not advanced so much (less than 10 points). The point, of course, is not to explain
the difference between the cities, but just to add a different perspective to the issues
addressed by this study.
This seemed to be the best approach considering the access constraints to data, time and
changes and relocations the author faced to write this paper.
1.6 Scope and limitations of the study
Attention will be drawn to literacy achievement in public schools of Ceará, so the private sector
will not be covered. Curriculum contents and pedagogical methods for literacy are discussed
very briefly, with the sole intention of contextualizing choices (or lack of them) in public
policies. Due to space constraints, adult illiteracy is not treated in this study.
Moreover, the discussion on learning assessment does not intend to focus on the diverse
influences on pupil achievement, but rather on how the results generated by the system can be
articulated with governmental actions aimed at a better quality of education and at ensuring
children‟s learning.
Finally, regarding the experiences of intergovernmental cooperation, only the major actions
taken by the Union were included in the study. They are the most frequent model due to more
availability of funding and trained personnel at federal level.
9 The author also attended a meeting held in at CREDE in Itapipoca City on 15
th May,2012, with PAIC
city managers and municipal secretaries from that zone, when SPAECE-Alfa results from 2011 as well as other operational topics were presented and discussed.
Towards better literacy achievements in the 2nd
grade of primary schools
http://www.iipe-buenosaires.org.ar
1.7 Structure of the paper
This initial chapter presents a general perspective of socio-economic and educational setting in
Ceará and introduces PAIC and its main achievements from 2007-2010. The objectives of the
study, the methodology applied to carry it out and its limitations are also presented.
Chapter two presents the debate on concepts of partial and full literacy that have influenced
practices and policies in Brazil. It also describes the role of intergovernmental collaborative
action established by the Constitution for the education sector as a means of balancing
autonomy and interdependence, and thus, of reducing disparities among federation entities.
Lastly, it tries to identify the main issues in learning assessments, its growth in the developing
countries and how they were linked to child‟s literacy.
Chapter three is dedicated to the facts and the role of main stakeholders involved in the events
that preceded PAIC implementation and to analyze how they influence the design of the
program. This section also describes PAIC‟s main features and analyses the framework for the
collaborative action that was established between the state government of Ceará and its
municipalities.
Chapter four explores the two kinds of external learning assessments design and uses
implemented by Ceará under PAIC. It analyses the rationale for their implementation and
describes their core aspects in order to understand how they became relevant instruments in
fighting illiteracy at Ceará‟s public schools. Finally, chapter five brings some lessons drawn
from PAIC‟s experience that can be useful for the three government levels in Brazil.
Towards better literacy achievements in the 2nd
grade of primary schools
http://www.iipe-buenosaires.org.ar
Introduction
This chapter presents some key concepts, relevant issues and national policies related to
child‟s literacy in Brazil, as well as the role of government collaborative action in this scenario.
The chapter also tries to identify the main issues of large-scale learning assessment, how they
have been disseminated in developing countries and recently linked to child‟s literacy.
2.1 Literacy policies and practices in Brazil
2.1.1 Literacy in the first cycle of primary school and academic failure
As presented in chapter one, Learning outcomes in the Brazilian education system haven‟t met
the adequate government standards. Low levels of student performance in Portuguese
assessed by SAEB are often associated to a lack of effective literacy in the early years of
primary school. (Oliveira, n.d.; MEC, 1997; Soares, s.d.)
In the analyzed literature, effective literacy in early years (1st ,2nd and 3rd grades) of primary
education is correlated with the development of the necessary skills for students to read and
write autonomously. (Oliveira, n.d.; Soares, n.d.; CD, 2003)
In Brazil, there is no national curriculum10, but the Conselho Nacional de Educação, CNE
(National Education Council) is responsible for releasing guidelines on the organization of the
systems and on the approach to the areas of common core. These guidelines are far from
being specific about competences and skills to be developed in each grade of primary
education. Thus, with regard to child‟s literacy, some states such as Ceará and Minas Gerais
have opted to develop a reference matrix of competences and skills to be pursued in the early
years of schooling in order to provide more specific guidance to their teachers.
It is difficult to address literacy policies and practices in Brazil without at least mentioning the
controversies over the competencies to be developed in the literacy process; as well as the
methodological debate. Although the scope of this study does not allow for further discussion
of these issues, they are relevant to be cited because apparently they were able to curb the
government action for fighting failure in early schooling.
When it comes to what kind of skills children must perform and when (at which grade) there is
no consensus. Partial and full literacy11 are frequently assumed as distinct concepts. While
partial literacy is more identified with decoding skills (related to alphabet knowledge, letter
names and sounds, concepts about print, phonemic awareness), full literacy is more
connected with early language skills (with emphasis on developing early readers‟ vocabulary
and text comprehension). Priority placed in one or another dimension gives rise to a lot of
10
Federal act nº 9.394 of 1996 sets up some areas to be included in the curriculum as a common core. There is also a diversified part to be defined by each education system. 11
Free translation adopted by the author to distinguish two different Portuguese words for literacy: alfabetização and letramento.
Towards better literacy achievements in the 2nd
grade of primary schools
http://www.iipe-buenosaires.org.ar
debate. The concern is that these latter skills are the ones to really indicate long-term
difference in individual literacy proficiency, while the first ones must be interpreted with caution
because they indicate a relative development in this field. On the other hand, some argue this
process is misunderstood by teachers leading them to fail in developing basic skills in child‟s
literacy.
What seems to be important in terms of public policies is that neither legislation nor national
guidelines have been able to clearly define education objectives for the first three years of
schooling. By doing so, it would help teachers to understand what skills should be pursued and
which criteria should be used to evaluate students, and determine to what degree they
succeeded at each stage.
The Parâmetros Curriculares Nacionais, PCN (National Curricular Parameters), released by
the Ministry of Education (MEC) in 1997, suggest that the practices widely disseminated
among Brazilian teachers to address literacy in two stages of development must be overcome.
Moreover, according to this document, the analysis of most recent theories on literacy would
recommend a review of traditional practices adopted in primary schools such as the use of
cartilhas12, which should be replaced with a wide variety of texts. They should be used as a
basic unit in the teaching and learning process. This strategy would replace the quite
widespread literacy practice of instruction using an incremental sequence of contents: syllables
(or letters), words, sentences and texts.
This approach is in line with the “whole language13” paradigm, a literacy theory which
emphasizes that the literacy process should comprise a holistic model. It emphasizes that the
acquisition of reading and writing should not be seen as a cumulative process. It is usually
associated with a constructivist approach and it states that the language should be presented
as a complete meaning unit to early readers rather than as small chunks. Therefore, whole
language is often contrasted with phonics-based methods of instruction which emphasize
decoding and spelling skills.
In Brazil, it has drawn criticism by those who plead "back to basics" pedagogy because whole
language is based on a limited body of scientific research. Oliveira (2004) says that the
paradigm became hegemonic in undergraduate teachers training courses and in public policies
and it has done a great disservice to education by diverting Brazilian teachers from literacy
methods which had been proved more effective.
This is obviously a very synthetic summary of these ideas and polemics14. The debate on
these issues is endless. The trend of school ineffectiveness identified by SAEB since the
1990‟s and the methodological/ideological debate led the Comissão de Educação da Câmara
dos Deputados, CEC/CD (The Education Committee of the House of Representatives) in 2003
to organize a work group on child‟s literacy in Brazil. The main concern was with the future
12
Cartilha is a kind of Spelling Book to teach children how to read by following a particular letter and syllable method, which was used by most schools in Brazil, but are not so popular nowadays. 13
See the ideas of Ken Goodman and Noah Chomsky. 14
See Ferreiro, E. 1992. Com todas as letras; Oliveira, J.B.A. 2004. Alfabetização de crianças e adultos: novos parâmetros; Kramer, S. (ed.). 1986. Alfabetização: dilemas da prática; Soares, M. 1985. As muitas facetas da alfabetização; CEC/CD. 2003. Relatório final do grupo de trabalho sobre alfabetização infantil: os novos caminhos.
Towards better literacy achievements in the 2nd
grade of primary schools
http://www.iipe-buenosaires.org.ar
performance and retention of these children within the education system and the need to break
the cycle of illiteracy in the country15.
This work group concluded that there is indeed a wide variety of interpretations among the
education systems in states and municipalities16 about what it means to be literate and at what
point (grade) of schooling it should be achieved. This fact stems from the lack of more precise
definitions released by the national level. Moreover, the uncertainties about performance
expectations make proper evaluation a difficult task for teachers. The work group concluded
that the scenario of vague definitions is also affected by poor training of teachers. According to
the study, specialists who select the material to be used in training courses for teachers
working in initial grades tend to prefer books related to theories and ignore the publications
geared to literacy methods and techniques. (CEC/CD, 2003)
The debate on the literacy concept and methods is far from being exclusive to Brazilian
education. According to UNESCO (2006), academics from a wide-range of disciplines such as
psychology, economics, linguistics, sociology, anthropology, philosophy and history have
engaged in this ongoing debate. UNESCO states that:
The most common understanding of literacy is that it is a set of tangible skills –
particularly the cognitive skills of reading and writing – that are independent of the
context in which they are acquired and the background of the person who acquires them.
Scholars continue to disagree on the best way to acquire literacy, with some advocating
the „phonetic‟ approach and others „reading for meaning‟, resulting in what has
sometimes been called the „reading wars‟. (UNESCO, 2006, p.149)
2.1.2 Collaborative action: framework and federal initiatives aimed at improving learning in the
first years of primary school
The Constitution defines that each federation body is responsible for the offer of priority
educational levels17, but it determines that the responsibility for primary education should be
shared between the states and the municipalities. It also provides autonomy for each
government level as regards the organization and management of their education systems.
However, they must operate in a collaborative way to ensure compulsory education for all, in a
way that balances the principles of autonomy and interdependence, and thus, reduce possible
inequalities. Unfortunately, there is no concept or features associated with the legal
requirement of collaborative action.
15
In 2009, the illiteracy rate among people aged 15 and over was about 10%, representing 14 million people. The Paulo Montenegro Institute measures the levels of functional literacy of the population aged 15 to 64 years, all over the country. The Indicator of Functional Literacy (INAF) generated by the Institute tries to reflect the skills of reading, writing and numeracy of this group. In 2009, according to INAF‟s sample, 52% of Brazilians who have studied up to 4
th grade reached only a rudimentary level of
literacy. This means they could only find explicit information in short and familiar texts (like a small ad or letter). UNESCO adopted the concept of functional literacy in 1978: „A person is functionally literate when he/she can engage in all those activities in which literacy is required for effective functioning of his group and community and also for enabling him to continue to use reading, writing and calculation for his own and the community‟s development‟. (UNESCO, 2006, p.154) 16
In Brazil there are 5.561 municipalities. 17
The Union is responsible for the funding and the management of the federal educational system, mainly higher education institutions; states must provide secondary education and municipalities are responsible for early childhood education.
Towards better literacy achievements in the 2nd
grade of primary schools
http://www.iipe-buenosaires.org.ar
As Abrucio (2010) says, this dualism in Brazilian federalism deeply affects educational public
policies. The institutional configuration in the education sector adopted by the Constitution has
caused acting gaps and intergovernmental tensions. Most municipalities claim that the
decentralization of educational responsibilities was not accompanied by the corresponding
financial resources and that the collaborative principle was rarely put in place to help building
the new model envisioned.
The decentralization was deepened, from 1996 onwards, with the setting up of the Fundo de
Manutenção e Desenvolvimento do Ensino Fundamental, FUNDEF (Fund for the Maintenance
and Development of Primary Education), which sought to balance public funding distribution
based on enrolments. However, municipalities were unprepared to take over their new tasks.
Most had and still have very low technical and administrative capacity.
Much of the aid to make up for institutional weakness in Brazilian municipalities has originated
in the federal government, which has a redistributive and supplementary role to ensure equal
opportunities for students and a minimum standard of quality. According to the Constitution,
this should be put in place through technical and financial support. As a consequence, the
collaborative action is generally associated with these two lines of action.
The transfer of financial and technical resources from Union to states and municipalities is the
most frequent model of cooperation due to more availability of funding and trained personnel at
federal level. Despite that, the collaborative principle around those two lines of action covers
both vertical and horizontal models of cooperation within the Brazilian federation as shown in
Figure 1.
Figure 1 – Types and levels of collaborative action in education within Brazilian federalism
Source: Author
Actions taken by the federal government
In order to fulfill the role of national coordinator of the education sector and responsible for
releasing general guidelines, the federal government has promoted some important changes in
the legislation related to the issue addressed by this study18. Since 2005, the Brazilian
education system has undergone a transition due to the adoption of Act no. 11,114 of 2005
which establishes compulsory primary school at age six and Act no. 11,274 of 2006 which
establishes a nine-year primary education.
In 2007, the National Congress approved the extension of FUNDEF, an enrolment based
formula funding mechanism, to consider the students of early childhood education (and also
18
There are also other objectives which will not be detailed in this study.
Collaborative Action
Technical
Resources
Financial Resources
Horizontal: Between states Between municipalities
Vertical: Union-State /Municipality
State-Municipalities
Towards better literacy achievements in the 2nd
grade of primary schools
http://www.iipe-buenosaires.org.ar
secondary school) through the creation of the Fundo de Manutenção e Desenvolvimento da
Educação Básica, FUNDEB (Fund for the Maintenance and Development of Basic
Education19). In 2009, constitutional amendment no. 59 extended the compulsory and free
education to all population aged 4 to 17 years. It should happen gradually and be completed
by 2016.
In practice, this set of changes allowed, in a first stage, the enrolment of 6-year old children in
public schools and the reorganization of the literacy cycle. Then, with FUNDEB‟s granted
funding, it was possible to expand access to preschool for children aged 4 and 5. The target is
to assist the children who live in low-income households since this is already a reality for
wealthier families. In summary, the new legislation aims to ensure earlier access to the
education system and thus to provide greater opportunities for learning, as well as reduce
disparities. Behind it there is the expectation that more years at school will result in more and
better learning.
Besides these changes in the legal framework, the Ministry of Education has implemented
national programs towards better literacy achievements in primary schools. The actions are
currently focused on three major areas: in-service teacher training, textbooks and assessment
through Pro-Letramento (Pro-Literacy); Programa Nacional do Livro Didático, PNLD (National
Program of Textbooks); and, Provinha Brasil, respectively20.
Pro-Literacy
It is aimed at improving Portuguese and Mathematics teaching in lower grades of primary
school. It offers a distance course with some face sessions and it is based on a tutorial model
operated by trainers21. Its implementation depends on states and municipalities adherence to
the program. Silva and Cafiero (2010) evaluated the initiative and considered that the program
has positive aspects such as more practical approach, but faces high levels of dropouts.
Textbooks
Since 2007 the PNLD has been revised to meet the needs of new legislation and the
enrolment of children aged six in primary schools. The previous single volume of literacy
textbook has been replaced by a collection of two volumes, which considers that the literacy
process should be consolidated at the end of the second grade. Here again there is a lot of
debate on which skills should be focused on these materials22. Other disagreement in the field
of literacy policies can be easily identified by contrasting the PNLD collection and the guidance
issued by CNE. The Council states that the literacy cycle should cover the first three years of
schooling23.
19
Act no. 11,494 of 2007 20
The federal government also implements the National Programme for Restructuring Schools for Early Childhood (ProInfância) which supports new schools for children up to five years (building and equipments). 21
The word used for trainers in Portuguese is multiplicadores, to imply the idea they are responsible for „multiplying‟ knowledge. 22
See Oliveira, s.d.; Silva and Cafiero, 2010 23
See CNE Res. CEB nº 7 of 2010
Towards better literacy achievements in the 2nd
grade of primary schools
http://www.iipe-buenosaires.org.ar
Provinha Brasil
The Provinha Brasil is a diagnostic test designed to assess reading in students enrolled in the
2nd grade of primary schools. Since 2008 the Ministry of Education develops the instrument
and makes it available on the Internet. School systems are autonomous to decide whether to
use it or not. The test can be applied by the teacher or by an external agent at the discretion of
each manager. The analysis, use and dissemination of results are also defined by each
system.
2.2 The growth of learning assessments in developing countries
In the educational reforms of the 1990s, learning assessment became a key concept in
developing countries. Several have put in place national systems of assessment, generally in
the context of broader processes of changes such as devolution of responsibilities, curriculum
reform, training for teachers without proper qualifications etc. These national systems of
assessment are a systematic, regular and external instrument to measure student learning
achievement. They are frequently identified in the literature as large-scale learning
assessments.
Ferrer (2003) points out that those assessment trends started in the 1990s covered other
modes of action, like institutional evaluation, evaluation of teachers‟ professional performance
and impact assessment of educational policies and programs. Nevertheless, the evaluation of
educational outcomes measured by student learning achievement became the most prominent
mode and faced great development over the following decade. Here, we will be concentrated
only on this latter mode24.
Assessment may refer to “any procedure or activity that is designed to collect information
about the knowledge, attitudes, or skills of a learner or group of learners” (Kellaghan &
Greaney, 2001 p.19). Thus, learning assessment is an integral part of the educational process.
In the past, it was just an internal school issue. The adoption of learning assessments carried
out by agents external to the school as a performance appraisal of national education systems
is associated with a number of factors, with different emphases depending on the author
analysis (Kellaghan and Greaney, 2001; Lockheed, 1996; Pigozzi, 2008; Saus, 2003;
Kellaghan, 2008).
In 1990, the World Conference on Education for All (WCEFA) that took place in Jomtiem
emphasized the countries‟ commitment in assuring learning acquisition rather than exclusively
in expanding enrolments. The WCEFA framework for action states clearly that basic education
is more than an end in itself and should provide foundation for lifelong learning and human
development. Regarding these needs, two activities were identified as relevant: i) defining
acceptable levels of learning acquisition for education systems, and ii) improving and applying
24
A different approach for external learning assessment is the national exams, which intend to assure that a certain level of knowledge was acquired to issue diplomas or to guarantee the access to higher levels of education. It will not be covered by this paper.
Towards better literacy achievements in the 2nd
grade of primary schools
http://www.iipe-buenosaires.org.ar
systems to assess learning achievement. This fact is pointed out as one of the reasons for the
growth of national assessment systems in developing countries.
Since then, the reforms have proposed the use of assessment to improve educational quality
and have three main characteristics: focus on outcomes; the assessment is conducted by an
agency outside the school; and the expectation is that the assessment will act as a lever of
change. In this sense, learning achievements are assumed as a proxy for evaluating the
quality25 of education offered by a national system.
The systems used to measure inputs, such as textbooks and facilities, as a proxy for quality.
The shift in focus from inputs to learning outcomes in national evaluations is also seen as an
attempt to ascribe some technical rationality to the education sector. Concepts such as
management by results, targets, use of indicators and efficiency (the ratio of inputs to outputs)
were introduced to public sector management as part of lessons learned in private enterprises
and began to influence educational policies.
Moreover, the quantitative expansion of the education system in terms of enrolments and the
rising costs associated with it would reinforce the growing demand for more accountability on
outcomes. In recent years, accountability has achieved an increasing importance in public
administration in many countries.
Finally, concerns related to the economic development of countries and linked to the
globalization of markets, the needs of increasing levels of economic competitiveness and the
so-called new competences that will be demanded for future workers have also played a role
in this process.
Postlethwaite and Kellaghan (2008) explain that the information obtained in a national
assessment can supplement information on inputs and on educational processes. Together,
they provide policy-makers and managers with evidence about their education system‟s
achievements, operational constraints, and the problems it may be experiencing. Since it is
difficult to plan without such information, national assessments can be considered as an
essential component of the professional administration of any education system.
That is why learning assessments have become the subject of much attention for both
industrialized and developing countries. They also got interested in working in international
initiatives such as the Program for International Student Assessment (PISA), Southern and
Eastern Africa Consortium for Monitoring Educational Quality (SACMEQ) and Latin American
Laboratory for Assessment of the Quality of Education (LLECE)26.
SACMEQ was created as a „hands-on‟ capacity-building programme in the skills of
assessment research. It selected grade 6 as the target population. SACMEQ differs from other
studies in that it takes a great deal of trouble to discover the major policy concerns and the
research questions its ministries wish to answer.
25
Quality in education is an issue on the policy agenda of nearly every country and it has of course different and complex meanings. One of EFA goals is “improving all aspects of the quality of education and ensuring excellence of all so that recognized and measurable learning outcomes are achieved by all, especially in literacy, numeracy and essential life skills”. Quality is seen by UNESCO as a determinant element of enrolment, retention and achievement and its expanded vision comprehends characteristics of learners, processes, contents and systems. (UNESCO, 2005, p.2) 26
See Postlethwaite (2004a).
Towards better literacy achievements in the 2nd
grade of primary schools
http://www.iipe-buenosaires.org.ar
PISA opted for an age group (15-year-olds), regardless of the stage they were at in the school
system. It aims to provide policy-relevant information on the cumulative yield of education
systems, measured in terms of performance of students in applying knowledge and skills they
have acquired.
LLECE forms the network of the National Educational Assessment Systems of 16 countries in
the region. It is co-ordinated by the UNESCO Regional Office for Latin America and the
Caribbean (OREALC).
According to Postlethwaite (2004a, p. 10) of more interest than the relative positions of the
countries and regions in achievement standard has been the development of a more
accessible and meaningful approach for displaying results reporting student performance in
terms of a hierarchy of skills levels. “This is a more useful approach than the rank order of
countries because it can inform the Ministry of Education about what percentage of pupils in
each country can do what.” The international studies have been conducted with a very high
level of technical expertise, therefore they also can help countries to acquire technical
background to undertake good national studies.
The main steps involved in the initial planning for a national assessment are: a) decide on the
purposes of the assessment; b) construct the test; c) decide on educational indicators; d)
define the target population; e) ensure that a good sampling frame exists; f) decide one where
„over-sampling‟ is required and then draw the sample; g) try test items, questionnaires, and
procedures; h) plan the data collection and train data collectors. There is a sequence of
events: field work, data preparation, data analysis and reporting results. (Postlethwaite and
Kellaghan, 2008)
In short, when it comes to learning assessment, three questions are pointed out as the most
relevant ones: What is the purpose of it? What and how to evaluate? How might the
information derived from learning assessments be used?
2.2.1 What are the purposes of a learning assessment?
Generally speaking, the main purposes of a learning assessment are to judge the
effectiveness of an education system or to check the adequacy of the performance of a school
or even of a student. However, learning assessments can have a wide variety of purposes,
which sometimes are blurred and are used to provide information for different levels. The most
common ones are:
to inform policies by providing information that can guide decision-making process or
allocation of funds;
to monitor achievement trends over time;
to raise or to maintain standards;
to establish a basis for holding managers and schools accountable for student‟s
performance;
to promote changes in instruction;
to identify disparities between subgroups (by gender, language, regions etc);
to balance control in a decentralized education system;
to ascertain the degree to which children have acquired what is established in the
national curriculum or guidelines.
Towards better literacy achievements in the 2nd
grade of primary schools
http://www.iipe-buenosaires.org.ar
to identify correlates of learning (the causes and effects of learning in classroom);
to diagnose individual learning needs.
The purposes should be expressed in the research questions to be answered. Officials and
researchers should agree about this aspect from the beginning so that utmost care will be
taken to ensure that the study will collect the data required. As said by Postlethwaite and
Kellaghan (2008, p.18): "This may sound like common sense, but many national assessments
have been undertaken in which ministry of education officials were not involved in stating the
research questions, and many others have collected large amounts of data that were not
focused on the concerns of officials”.
2.2.2 What, when and how should it be assessed?
This question is intrinsically related to the information needs generated by the purposes of an
evaluation. By and large the purpose of the assessment determines its overall format or
design.
The assessment is summative when it occurs after a certain period of time such as a school
year or a cycle of education. It intends to summarize learning up to that point or to verify if the
system has met the educational objectives set. Another approach is the formative assessment.
The findings of this type of evaluation are supposed to be used to improve the instruction and
to ensure that the objectives are more likely to be accomplished.
The decision on evaluating in a census or sample basis also depends on the objective
pursued. A sample-based assessment cannot identify problems in a particular school.
Therefore some national assessments are applied in a census-based. On the other hand, if the
goal is to monitor learning outcomes for a general picture of system performance, then a
sample-based evaluation will be adequate and more cost-effective. As stated by Postlethwaite
and Kellaghan (2008) “when all students in an education system are tested (…) the costs can
be very high and therefore such „census-based‟ approaches may not be affordable in poorer
countries”27.
In national or international learning assessments, the comparability of data collected is a major
concern, so tests are standardized as well as administration procedures and scoring. Utmost
care is taken to ensure the credibility and accuracy of the findings, so measures of validity and
reliability are taken. Validity has to do with consistency and accuracy. Reliability is usually
established by using pre-tests, but there are other equivalent methods.
What to evaluate and when to evaluate are also relevant issues to guide instrument design.
Considering the difficulty of obtaining consensus on what competences and skills should be
acquired in the schooling process, most of the learning assessments focuses on basic
cognitive skills, such as literacy and numeracy. Therefore, the fields mostly assessed are
language and math.
27
Postlethwaite (2004b, p.87) explains that the more numerous the differences among schools, the more schools are needed in the sample. In poor countries, the differences among schools tend to be greater and therefore it is those countries that need more schools in the sample.
Towards better literacy achievements in the 2nd
grade of primary schools
http://www.iipe-buenosaires.org.ar
Given the general objective of getting information on the performance of an educational
system through learning achievements of its students, many assessments focus on the main
school exit points or on the end of cycles depending on the organization of each education
system.
The idea of measuring quality by learning outcomes often implies establishing standards28.
Standards are mainly presented as a measure of what is considered adequate for students to
have acquired at particular levels of schooling. Setting standards, as said by Wagner (2011),
involves the exercise of choice and the application of values.
Most large scale assessments utilize a statistical technique called Item Response Theory
(IRT). This approach allows covering a number of competences/skills in the instruments, but
each student is tested in only a few of them. It involves scaling of student responses and
allows comparability of results over time as some items are repeated. The analysis of data
requires more sophisticated skills from the personnel involved in the assessment. IRT
contrasts to the traditional approach of Classical Test Theory (CTT) with regard to raw scores
or percentage of items correctly answered by students.
According to Postlethwaite and Kellaghan (2008), some countries experienced to test the
teachers of the students in national assessments. This was undertaken in the Vietnam survey
in 2001 and also in the international surveys of SACMEQ. Testing teachers can be used to
identify gaps in their knowledge about school subject matter – which, in turn, can help to point
to areas that can be addressed in in-service teacher training.
Finally, large-scale assessments often make use of questionnaires to collect data related to
the socioeconomic background of students, school conditions and teachers, among others.
They are seen as complementary data to the analysis of learning achievements. This data
allow more meaningful comparisons since there is evidence in the literature that student's
academic performance is mostly explained by background and socioeconomic status29.
Postlethwaite (2004a, p.14) explains that great efforts have been made in international
assessments to identify the factors in and out of school that could have influenced
achievement. It is this kind of information that is needed by countries. In nearly all systems of
education the home background of pupils was important, but this is an issue to be carefully
analyzed. The author summarizes some conclusions from the studies conducted by the
International Association for the Evaluation of Educational Achievement (IEA) in the last 30
years. One of the points highlighted: “although the effects of home background variables were
similar across subject areas, the effects of the learning conditions in the schools differed
between subject areas, and in some subject areas were equivalent to or greater in the size of
their influence than the effects of the home”.
28
Standard-based assessments (or criterion-referenced) are, according to Wagner, “those that allow scores to be judged against some level of expected result. Norm-referenced assessments are those that provide a level of skill that is matched against the learners‟ relative position among peers taking the same test”. (Wagner, 2011, p.30) 29
The Coleman Report, commissioned by the Department of Education of the United States in the 1960s, is the most widely cited study about school effects.
Towards better literacy achievements in the 2nd
grade of primary schools
http://www.iipe-buenosaires.org.ar
2.2.3 How might the information derived from an assessment be used?
Here the main aspect is how the information generated is used. In theory, the more
appropriate the assessment model is to its purposes, the greater the chances that it addresses
the needs are, and therefore, the more likely their results are used. However, the use made of
large-scale assessments is also related to the analysis produced and the communication
procedures adopted to inform and disseminate findings. Wagner states that “learning
assessments are only as good as the uses that are made of them”. (Wagner, 2011, p.14)
Most widely used procedures of data dissemination include the preparation of reports with
variable coverage (international, regional, national, by sector) and aimed at different audiences
(education authorities, researchers, media, school managers, teachers, and families). Here
again, appropriate formats and language features are very relevant and should consider the
audience they are targeting. A wider dissemination strategy is designed when an important
social and media impact is intended.
There are also cases in which data is not disclosed. This approach takes into consideration
that the best use to be given to some assessment results is to offer the institution and its
professionals the opportunity to compare and reflect on their own performance. At the other
extreme, there are results that are published as league tables, ranking systems or schools.
This approach has received many criticisms because it tends to encourage comparisons and
interpretations out of context and based only on ranking position. This type of data
presentation has also been accused of producing competition between schools. Nevertheless,
as Kellaghan and Greaney (2001) have highlighted, it seems to arouse more interest in the
media. This interest is useful to put education in the spotlight and to generate mobilization
around issues of quality.
Postlethwaite (2004b, p.89) claims that “there is general agreement that research results have
a „drip effect‟ on public (not the general public, but the public most interested in the results)
opinion and that over time the results affect how planners think about education”. In some
cases direct actions are taken, in others the effects take a generation to induce improvements
in the schools. The author affirms that one important feature of repeat surveys is that it is
possible to check if action has been taken.
One of the drawbacks of external assessments already identified is the teaching-to-the test. It
tends to be more present in systems where schools are placed under great pressure to
improve their results. Regarding the use of assessments for funding, some negative aspects
have been identified, such as hierarchies among schools or rejection of teachers to be
deployed to institutions with poor results. It is noteworthy that they are strongly linked to the
reality of the United States, where such initiatives are more common.
Performance pay programs
The United States have adopted some experiences with the so-called bonus programs. The
government pays bonus for the best results, considering either school-wide performance or
teacher‟s group performance separately. This use for external learning assessments is very
controversial in the education sector.
Towards better literacy achievements in the 2nd
grade of primary schools
http://www.iipe-buenosaires.org.ar
In 2011, New York City Department of Education announced that its Schoolwide Performance
Bonus Program would be discontinued30. The program lasted for three school years, and its
broad objective was to improve student performance through school-based financial
incentives. A study commissioned by the city “found that the program did not, by itself, improve
student achievement, perhaps in part because conditions needed to motivate staff were not
achieved.” (Marsh, 2011, p.14)
The evaluation concluded that the program had no effects on teacher reported attitudes,
perceptions and behaviors. It was hypothesized by the researchers that lack of results might
be due to the limited motivational power of bonus – resulting from school staff viewing the
award as reward or recognition rather than an incentive.
Another hypothesis is that New York schools faced a lot of pressure to increase the grades of
their students, under the threat of severe penalties, such as closing the institution and in this
environment the bonus, which could reach $ 1,500 per teacher after taxes, but could still not
motivate them. On the other hand, Goodman (2011) found some positive findings and
suggested that a heavy reliance on this concept may hamper the effectiveness of it:
(…) what this experiment makes frustratingly clear for merit pay proponents is that the
structure of the payment scheme can make a large difference. For merit pay to improve
student outcomes, teachers must face strong incentives to improve their performance.
Our study indicates that school-wide bonus programs may be able to provide those
incentives in schools with relatively small teaching staffs. They may also be appropriate
for schools characterized by a high degree of staff cohesion, in which teachers work
collaboratively to improve student learning and it is difficult to isolate the performance of
a single teacher. (Goodman, 2011, p.71)
New York City Program inspired a similar initiative in the São Paulo State. The program
awarded a bonus to teachers and non-teaching staff based on the students‟ performance. The
school should meet targets designed by the State Secretariat of Education based on the Índice
de Desenvolvimento da Educação do Estado de São Paulo, IDESP (Index of Education
Development of the São Paulo State). The index is generated from the performance of
students in the Sistema de Avaliação de Rendimento Escolar do Estado de São Paulo,
SARESP (São Paulo Education System of Learning Assessment). In addition, teachers must
attend work at least two thirds of school calendar. Unlike New York, the initiative faced
problems with unions from the beginning, since it was seen as a means of taking the
responsibility on education problems out of public authorities and putting them on the teacher‟s
back. In 2011, facing negative variation in learning outcomes identified by SARESP, the state
decided to promote some changes in the proposal.
Old demands and new trends in learning assessments
Several authors have indicated that the use of large-scale assessments is now the weakest
part of assessment systems in developing countries, with little effort made to understand the
30
See “New York City Abandons Teacher Bonus Program”, published by The New York Times in 17 October 2011. Retrieved from: http://www.nytimes.com/2011/07/18/education/18rand.html in 12 April 2012.
Towards better literacy achievements in the 2nd
grade of primary schools
http://www.iipe-buenosaires.org.ar
reasons for that. They agree that more efforts have been made in structuring models and
collecting data than in analyzing it and in communicating it properly to stakeholders. (Wagner,
2011; Ferrer, 2003; Iaies, 2003; Kellaghan and Greaney, 2001)
The poor use is due to a variety of factors: lack of, or poor analysis, insufficient dissemination
or exclusive use of mass media to do so, low ownership of the findings by local actors, long
delay between field work and dissemination of results which hampers the use of reports as a
planning tool.
In reviewing the evaluations implemented in the 1990s in Latin America, many authors
recognized their importance to increase public awareness and to inform political debate,
especially regarding inequalities within the education systems. Broadly, they agree that
information obtained from national assessments has been used in the discussions about public
policies, but stress their low impact on school practices. Even ministry teams responsible for
training and curriculum areas rarely use them as input for their work.
This is a critical issue for the region because, as argued by Iaies, no significant variations in
the results have been observed in the period assessments have been conducted (Iaies et al,
2003).
According to this review, the teams responsible for building learning assessment systems in
Latin America were clear about the demand of organizing devices, but have little guidance
regarding their purposes or the impact that was intended to generate by their results.
The advances generated since the first experiences were implemented seem to have
been built with autonomy over purposes. There are few innovations in instruments.
Those already produced have to do with statistical or technical-pedagogical adjustments.
The innovative element appeared in some international devices that incorporated new
models and contents to the instruments. (Iaies, 2003, p.29-30) (in translation)
The review suggests that it would be interesting to reflect upon the current assessment
strategies in order to embed the advances made and to reorient these tools. But it
recommends that these changes should be consistent with policy objectives defined by the
governments.
Indeed, different approaches of assessing learning in developing countries have been tried
recently. One of them is the Early Grade Reading Assessment (EGRA) which focuses on early
readers and considers local contexts rather than national or international comparability. It tries
to deal with policy constraints such as ethnic diversity or linguistic variation that are sacrificed
in large-scale assessments to ensure comparability of results. It also pays close attention to
individual differences in learning.
Wagner (2011) states that EGRA is part of a new trend to put the focus of assessment design
on less developed countries context and needs. Its concept is based on the paradigm of
smaller, cheaper and faster assessments. Its methodology is simple and allows wide
„shareability‟ which reduces costs. The timeliness of analysis is more suitable to local
decisions.
Because the test is applied by teachers, EGRA cannot be considered an external learning
assessment. Nevertheless, it is interesting to note that its emergence meets one of the most
Towards better literacy achievements in the 2nd
grade of primary schools
http://www.iipe-buenosaires.org.ar
urgent needs of education systems in developing countries: to identify reading problems much
earlier than the national or international tests taken by the countries.
Another trend is the dissemination of experiences with large-scale assessments through lower
levels of government. This trend has been possible due to the advances in technology, which
hugely improved the capacity for processing data, making data collection and analysis easier
and faster. The dissemination of technical expertise on learning assessment and training of
teams also played an important role in this process.
2.2.4 National experience in learning assessments
Brazil started implementing the SAEB31 in 1990, but it was only after 1995 that the Item
Response Theory was adopted, opening the comparability of results over time. SAEB is an
external large-scale learning assessment system. It aims to provide information about the
quality of education and to inform the design, redesign and monitoring of educational policies
at national, state and municipal levels.
Currently SAEB comprises two complementary learning assessments. The first, called
Avaliação Nacional da Educação Básica, ANEB (National Basic Education Assessment), is
based on a sample of students from public and private schools, in rural and urban areas.
Groups must have a minimum of ten students enrolled in each of the 5 th and 9th grades of
primary school and also in the 3rd year of secondary. Polled results are provided by national,
regional and state level.
The second, called Avaliação Nacional do Rendimento Escolar, ANRESC (National
Assessment of Educational Achievement), implies a similar format to ANEB‟s. The difference
is that it is implemented, at a census basis, in some 5 th and 9th grades in public school groups.
Each school must have a minimum of 20 students enrolled in each grade assessed. Results
are provided by national, state, municipal and school level. Both tests include Portuguese and
Mathematics, focusing respectively on reading and problem solving, and are held every two
years.
ANRESC is well known as Prova Brasil and is a very recent initiative. It has been implemented
since 2005. It was designed to meet the demands of public authorities, educators, researchers
and society in general for information about the education offered in each municipality and
school. The objective, as said by MEC, is to assist local governments in the allocation of
technical and financial resources, as well as the school community in setting goals and
implementing actions aimed at improving the quality of education.
Long before Prova Brasil had been created, several state governments32 adopted their own
learning assessments to complement or to address deficiencies related to SAEB. They are
often used by state governments as a mechanism to meet the legal requirement necessary to
evaluate their educational systems.33. Legislation requires evaluation of educational institutions
under state responsibility, there is no specific instruction as regards students‟ achievements.
31
To access data or to get more information about SAEB see www.inep.gov.br 32
Rio Grande do Sul, Ceará, São Paulo and Minas Gerais, among others. 33
See article 10 of Act nº 9.394 of 1996 which establishes guidelines and objectives for national education.
Towards better literacy achievements in the 2nd
grade of primary schools
http://www.iipe-buenosaires.org.ar
More recently, some large-scale learning assessments are also being held at the municipal
level, especially in state capitals such as Recife or cities with greater economic development
as Campinas, which can afford the financial costs and have trained personnel to implement
them. Municipalities do not face the legal requirement to perform evaluations in their education
system.
The need for identifying reading problems in earlier stages and the goal of all children being
literate by the age of eight, as stated by the NPE, have introduced new demands for learning
assessments in Brazil. Besides Provinha Brasil, provided by MEC to states and municipalities
and already mentioned in the previous section, the NGO Todos pela Educação (All for
Education) conducted the first large-scale learning assessment in 2011 (Prova ABC) at a
sample basis to diagnose 2nd graders‟ learning in writing and reading. However, in Brazil, there
is still no systematic assessment on this topic at a national scale. There are experiences
conducted at state level, such as those occurred in Ceará and Minas Gerais.
Conclusion
Learning assessments were widely spread out in developing countries due to a broad
combination of factors which have had many purposes. In general, they have been conducted
at national levels through large-scale models. Recently, their drawbacks and more specific
needs – to monitor literacy achievements in the first years of schooling – have lead to some
innovations.
In Brazil, the issue of literacy outcomes does not concern only to learning assessment
innovations. There is a wide variety of interpretations among the education systems in states
and municipalities about what it means to be literate and at what point (grade) of schooling it
should be achieved. This fact stems from the lack of more precise definitions, so better
guidelines should be the focus of federal government action. Although the responsibility for
literacy outcomes is posed mostly on municipalities, the more fragile part of the federation in
terms of financial and institutional resources, the path is also linked to more intense and
effective forms of intergovernmental collaboration.
Towards better literacy achievements in the 2nd
grade of primary schools
http://www.iipe-buenosaires.org.ar
Introduction
This chapter describes relevant facts that influenced the decision-making process to formulate
and implement PAIC and the main features of the collaborative action structured by the State
of Ceará and its municipalities. It analyses some political and technical aspects that
contributed to identify the issue the program intends to address and that underlies PAIC
design. The program is understood as an output of this process and it is described regarding
the actions carried out and the responsibilities of each government level.
3.1 The Committee for Eradication of Illiteracy at School (2004-2006)
3.1.1 How and why was it set up?
In 25 March 2004, the Assembléia Legislativa do Ceará, AL-CE (Legislative Assembly of
Ceará) set up the Comitê Cearense para a Eliminação do Analfabetismo Escolar, CCEAE
(The Ceará State Committee for Eradication of Illiteracy at School). This initiative was a
response to the impact of the 2001 SAEB results, which were released in 2003.
It is important to highlight that the year 2001 marks the lowest inflection point reached by the
learning outcomes of the Brazilian educational system. After increasing the number of students
enrolled in primary schools in the 1990s, the system indicated that it had not managed to keep
quality levels34.
Originally, the idea of investigating low learning levels in Ceará‟s education system was
proposed by the then State Representative, Cid Ferreira Gomes, as a Parliamentary
Commission of Inquiry35, but it was transformed into a Committee led by the state legislature,
with plural participation of stakeholders from the three government levels and an international
34
Neri and Carvalho associate this drop in students‟ mean performance with other public policies, such as the setting-up of Programa Bolsa-Escola* and the adoption of the automatic grade progress policy, which stimulated students at critical socio-economic conditions, with very poor school performance to remain in school. (Nery & Carvalho, 2002, apud Todos pela Educação, 2007, p.13). *Bolsa-Escola was a cash transfer program to poor families that guaranteed their children (maximum three) minimum school attendance. Later, it was redesigned and named Programa Bolsa Família (Direct Cash Transfer Program). 35
It is a largely used tool in the Brazilian Parliament in order to investigate and include relevant topics in the political agenda.
Towards better literacy achievements in the 2nd
grade of primary schools
http://www.iipe-buenosaires.org.ar
institution, during its tenure36: União Nacional dos Dirigentes Municipais de Educação,
UNDIME/CE (National Union of Municipal Education Officers), INEP, SEDUC and The United
Nations Children‟s Fund – (UNICEF)37.
Over an 18-month period of work, the Committee‟s agenda was divided into two main lines of
action: i) to carry out a set of diagnostic studies to provide more accurate data on the actual
literacy level of Ceará children, considering that „information on children illiteracy and on the
quality of education is scattered or from samples, insufficient to mobilize forces towards
more strategic actions and to influence resulting policies in the field‟; and ii) to mobilize
government and society and mainly public municipal power to acknowledge and face the
problem. (AL-CE/CCEAE, 2006, p. 26, emphasis added)
3.1.2 Diagnosis of illiteracy in Ceará schools
By means of technical and financial support from both federal and state governments, the
Committee put five universities38 in charge of carrying out three studies39:
i. Diagnosis of 2nd graders literacy levels (reading and writing) in public schools40;
ii. Identification of ways and conditions of organizing school work and teaching work in
child literacy.
iii. Identification of both initial and continuing teacher training mechanisms for child literacy
education41.
Some of the main findings were:
a) Assessment of the students‟ literacy levels as regards three dimensions – text production,
reading and text comprehension – indicated that two thirds of the students were not literate at
least in two of the dimensions assessed.
b) The school‟s textbooks and the children‟s literature books were insufficient and of poor
quality. Little or inadequate use was made of the existing material.
c) School calendar and lessons timetable were not being followed; besides, the actual lesson
time was being wasted with irrelevant activities for children‟s learning.
d) There was low student school attendance and high teacher absenteeism.
36
The Committee also had the systematic collaboration of other stakeholders: main public and private universities, state and municipal councils of education, bodies representing mayors and the economic sector, and also non-governmental entities linked to children‟s rights defense. 37
UNICEF assists several Brazilian municipalities technically in the development of programs that lead to fulfilling the right to learn. 38
Universidade Federal do Ceará (UFC), Universidade Estadual do Ceará (UECE), Universidade Estadual Vale do Acaraú (UVA), Universidade Regional do Cariri (URCA) and Universidade de Fortaleza (UNIFOR). 39
The Committee considered that the academic performance of students under literacy process is a result of both intra and extra-school factors; thus it was also useful to investigate the supply conditions and the ways of organizing school work. 40
The study involved 8 thousand 2nd graders from public schools in 48 Municipalities. 41
The core objective of this study was to assess knowledge about child literacy mastered by students and teachers in Pedagogy, Child Education and School Teaching Courses, in all public teacher education institutions in the State of Ceará.
Towards better literacy achievements in the 2nd
grade of primary schools
http://www.iipe-buenosaires.org.ar
e) Teacher profile had changed regarding training; most of them had a degree, but they were
still poorly qualified for the literacy teaching task.
f) Most teachers sought explanation for the phenomenon of illiteracy at school in outside-
school-related factors.
3.1.3 The importance of the Committee’s work
As the final report itself highlights, the CCEAE research findings were already well known and
the school organization problems were not new. Thus, what was the relevance of the
Committee for the elaboration of a program, such as PAIC, to fight illiteracy at school?
Roughly speaking, it can be said that the Committee‟s work represented a source of change
for the formulation process of PAIC. One of the great achievements could be the capacity to
translate insights and indirect indicators (produced by SAEB) into evidence-based diagnosis
which was disseminated and quite discussed with key stakeholders within the State. They
were fully involved in the Committee and helped to carry out the work from the beginning. At
the end of the process, they were all quite aware of the diagnosis and what goals should be
pursued to address the problem of illiteracy at school.
In the CCEAE final report, some key issues in the policy-formulation process are clearly stated,
such as the vision and goal setting, priorities and some strategies to reach them. All the
speeches made by these stakeholders, included in the final report, stress two basic elements
that framed PAIC design: i) focus on learning (effective literacy in the first years of primary
education) as a sine qua non condition to fulfill the right to education; and ii) accountability for
results. Generally speaking, these key actors presented political will to tackle the problem.
Apart from involving representatives of the three levels of government during its administration,
the Committee was successful in the task of mobilizing and including the illiteracy at school
issue in the agenda of municipalities, civil society and the media. Several events and public
hearings were held in the state capital city of Fortaleza and in small towns of the State, which
turned into discussion forums on the issue.
The process of mobilization and sensitization of stakeholders (providers and beneficiaries) to
the need for a change was made both within the Committee‟s activities and by media
coverage. The final report offers a collection of news published in mainstream State media,
particularly with respect to research findings.
The activities undertaken by the Committee may also have played an important role in
generating legitimacy for its findings and a sense of ownership by all groups when PAIC
started to be implemented.
The final recommendations made by CCEAE highlighted the importance of focus on learning,
but also suggested in-service teacher training, municipal management strengthening, and
better monitoring of learning results at school and system level. These recommendations, as
will be seen, were broadly incorporated by PAIC.
Finally, the local legislative power leadership on this process was unusual, reversing the sense
of political power and setting up of agendas that normally stem from the Executive Branch, at
Towards better literacy achievements in the 2nd
grade of primary schools
http://www.iipe-buenosaires.org.ar
all levels of the Brazilian federation. It is worth highlighting this fact because it indicates new
directions for other legislative forums and collaborates with the consolidation of the democratic
institutions in the country.
3.2 Sobral’s experience in fighting illiteracy at school (2001-2004)
The Committee organized by the Ceará state parliament and later the team in charge of PAIC
benefited from a previous experience of fighting illiteracy at school that had already taken
place in the same state. The city of Sobral42 had implemented from 2001 onwards an
educational policy that prioritized literacy of all students in the first cycle of primary school.
Similarly to the process that will occur within state scope, the decision of implementing actions
addressing effective children literacy stems from a diagnostic assessment with students who
finished the 2nd grade. The results showed extremely low reading skills. In Sobral, the
diagnostic was used to mobilize and sensitize the educational community to the issue.
(Coelho, 2006)
3.2.1 Lessons learned from Sobral’s experience
The core aspects of Sobral‟s experience are presented in the PAIC set-up, such as: focus on
learning in the early school years, defining clear and well-known targets43, accountability for
results and specific external assessment design to guide and monitor the changing process44.
Carried out twice a year, the external assessment provided indicators of both reading and
writing development levels of 6 and 7 year olds enrolled in the first two primary school years.
The results gathered in the first semester allowed schools to adopt new strategies to meet their
learning targets. At the end of the second semester, the second assessment found out the
learning outcomes and enabled adjustments for the following years. Apparently, one of the
most interesting lessons learned from Sobral and incorporated in PAIC was the use of the
external assessment results as a managerial instrument by the Secretaria Municipal de
Educação, SME (Municipal Education Secretariat) and by the schools. Learning outcomes
were scrutinized and were used to guide planning and subsequent adjustments in actions
implemented by these teams.
42
Sobral is one of the 184 municipalities in Ceará and it is located in the backlands of the state, in the Northeastern semi-arid region. According to the 2010 census, its population totals 190 thousand people, being the 5th more populated town in the state. It is an industrial hub and one of the biggest economies of Ceará. Its GDP was R$1.9 million in 2009, when its per capita GDP of R$10,769.79 was higher than Ceará‟s (R$7,686.62) and the Northeast region‟s (R$8,167.75). Data source: Instituto Brasileiro de Geografia e Estatística, IBGE (Brazilian Institute of Geography and Statistics), “Gross Domestic Product by Municipalities 2005-2009”. Date of Access: 15.02.2012 http://www.ibge.gov.br/home/estatistica/economia/pibmunicipios/2005_2009/ 43
During the 2001-2004 tenure, seven targets were defined for Sobral‟s educational system. Two of them were prioritized: i) literacy of all 6 and 7-year-old children; and ii) under a compensation basis, literacy of all students from the 2nd to the 6th grades who cannot read. 44
Sobral‟s experience was registered by INEP in: Vencendo o desafio da aprendizagem nas séries iniciais: a experiência de Sobral/CE. Série Projeto Boas Práticas na Educação nº1. Brasília: INEP, 2005.
Towards better literacy achievements in the 2nd
grade of primary schools
http://www.iipe-buenosaires.org.ar
Another relevant point gained is the search for accountability for students‟ learning results.
External assessment allowed tracking the competences developed by students individually.
Each group and each teacher could be correlated with their respective results. This is a crucial
issue in the Brazilian educational system – also pointed out by the Committee – because,
generally, students‟ low learning is attributed to only outside school related factors45. INEP‟s
report informs that:
Interviews with school principals and coordinators certify that vague phrases such as
“most can read”, or “some improved”, which little contributed to checking learning
success, are not part of the school‟s discourse anymore. The teacher started to know
every student by their names, as well as each one‟s advances and difficulties, therefore
getting more prepared to deal with their main difficulties. Principals and coordinators also
started to learn the difficulties each teacher had, what enabled them to design new
training strategies and guidance. Thus, external assessment enabled the consolidation of
a systematic culture of teacher‟s assessment . (INEP, 2005, p. 44)
One more lesson learned and incorporated in PAIC was the design of a set of comprehensive
actions to fight illiteracy at school, shaping it as a public policy rather than as an isolated
program: expanding access to pre-school, improving school supervision, strengthening
technical support to teachers, complying with school calendar and timetables, and availability
of children‟s books and learning games, among others.
In order to achieve the desired changes in the literacy teaching practice and face teachers‟
poor qualification, well structured teaching materials were introduced, describing contents and
step-by-step daily activities to be developed with students. Teachers attended specific training
sessions to use these materials and also attended meetings with the support team from the
Secretariat to discuss learning targets.
As regards the reorganization of Sobral‟s education system, the actions were on creating rural
school clusters and on selecting school principals through public exams and technical profile
analysis, aiming at appreciating professional merit rather than political nominations for the
occupation of these positions. These were ideas that had a great impact for a positive change
and are also incorporated in PAIC.
Finally, Sobral‟s experience resorted to mechanisms of financially rewarding schools and
teachers that met the learning targets, creating the “Prêmio Escola Alfabetizadora” (Literacy
School Award), and a bonus, equivalent to 30% of their monthly basis salaries, to the literacy-
promoter teachers who met the learning targets for their groups. This measure also resulted in
unwanted effects, such as forced school transfer of students with low performance. As a
lesson learned, this school award strategy is later redesigned by the State.
In 2004, when CCEAE unveiled the outcomes of its data collection on the literacy level of
students in Ceará, Sobral students‟ performance proved to be well above the one achieved by
45
In 2011, the NGO Todos pela Educação (All for Education) analysed the questionnaires completed by teachers for Prova Brasil, the exam that assesses 5th and 9th graders‟ performance and collects information from principals and teachers in order to identify the main causes of learning problems, according to these professionals‟ view. The analysis of that data indicated that teachers believe low learning levels are mainly due to outside school related factors, such as lack of family assistance and support (88.1%), students‟ lack of interest and effort (88.1%), and socio-economic levels (79.5%). The analysis results are available at http://www.todospelaeducacao.org.br .
Towards better literacy achievements in the 2nd
grade of primary schools
http://www.iipe-buenosaires.org.ar
the State (Table 5). There was also a reduction in school dropout rate of students enrolled in
the first cycle of primary school. Between 2001 and 2004 it fell from 9.6% to 0.7%. In
summary, the experience seemed to be quite successful to inspire and legitimate an action to
be taken within the scope of the whole State46.
Table 5 - Percentage of students who finished 2nd grade of primary school with adequate performance
according to CCEAE assessment standards, by dimension assessed, Ceará and Sobral, 2004.
Level Text Writing Text Comprehension Reading
Ceará 42% 15% 34%
Sobral 80% 31% 59%
Source: Data retrieved from INEP, 2005 and AL-CE, 2005. Note: In reading, the adequate performance
refers to students who read above the speed of 50 words per minute. The two other dimensions refer,
respectively, to those who managed to produce a text and to understand the issue required.
3.2.2 Policy priorities and technical expertise transfer from the city of Sobral to Ceará’s
Government
Government decisions are not made by a monolithic state based on rational choice, but rather
are negotiated by various leaders who sit on top of the organizations involved in that particular
decision-making process. Each leader is compelled by his own conception of the problem as
well as by the imperatives of his organization and his own personal goals. (Haddad, 1995)
In PAIC‟s case, it should be registered that there were links among some core actors and this
could somehow have influenced the agenda and, consequently, policy options. CCEAE‟s
president, representative Ivo Gomes, who led the liaising process among several entities, was
Secretary of Education in Sobral. PAIC‟s piloting stage was implemented during the mandate
of the then Ceará‟s governor, Lúcio Alcântara, but the program gained more scope from the
moment Cid Gomes, the new governor and Sobral‟s former mayor, was first elected and took
up office for the 2007-2010 mandate, and recently re-elected for another 4-year mandate
(2011-2014)47. Additionally, the managers who were placed in SEDUC in 2007 were the same
leaders of the changes that occurred in Sobral.
Accurate diagnoses of issues and mobilization around them are not enough to induce
consistent changes in the education sector. Setting priorities and simultaneously making it
clear that all efforts should be put in place to implement them depend on the commitment of
political leaders and on the technical capacity of the managers in charge. Apparently PAIC
may be benefited from a combination of political leadership and technical expertise, both
considered vital elements for the formulation and implementation of effective policies.
46
In 2007, the same year of PAIC‟s expansion to 100% of Ceará‟s municipalities, the Ministry of Education created the Índice de Desenvolvimento da Educação, IDEB (Education Development Index), an indicator aimed at measuring the advances in the quality of public education combining learning outcomes with flow data. The target set to all federative entities is to reach value 6 in year 2021. IDEB registered an index of 4.5 for the first years of primary school in Sobral, in 2005. The national and state mean was, respectively, 3.8 and 3.2. In 2009, IDEB‟s results were: Sobral (6.6), Ceará (4.1), Brazil (4.4) 47
Ivo Gomes and Cid Gomes are affiliated to the Partido Socialista Brasileiro, PSB (Brazilian Socialist Party).
Towards better literacy achievements in the 2nd
grade of primary schools
http://www.iipe-buenosaires.org.ar
3.3 PAIC and government collaborative action in its implementation
“Once a problem within the educational system is recognized then a solution is frequently
forced upon the system. This is especially likely to occur when there is a public debate
about a problem”. (Haddad, 1995, p.31)
The mobilization generated by the Committee and the media48 repercussion of the children‟s
low literacy level diagnosis led to a consensus and a commitment to action. However,
transforming consensus and commitment into action would be a complex task as learnt from
CCEAE‟s and Sobral‟s experience. They proved that fighting illiteracy at school would demand
acting in several spheres, mainly in the management of municipal education systems, and not
only in classrooms. Even if the State took over the matter as a priority in its agenda, it would
be necessary that municipalities acted as the changing agents.
The first stage of the Program (2005-2006), not by chance, was headed by the Associação de
Municípios e Prefeitos do Ceará, APRECE (Association of Ceará‟s Municipalities and Mayors)
and by UNICEF, with the support of the state government. Timing in this case was a crucial
element because it was necessary to raise mayors‟ awareness (the ones elected in 2004 for
the 2005-2008 term of office) towards “their commitment for action” as regards the setting up
of the program within their mandate. Between 2005 and 2006, PAIC was set up in 56
municipalities. Although it had not been institutionally defined as such, in practice and
experience it worked as a pilot, especially regarding external assessment mechanism.
In 2007, the new governor of Ceará, Cid Gomes, decided to turn the program into a state
public policy, which implied a more systemic approach to fight the problem of illiteracy at
school and to design a model for specific cooperation with the municipalities that complied with
the priority taken. In the following pages, it is described how PAIC was structured and why it
presents innovative features49.
3.3.1 Institutional arrangements for PAIC implementation
Institutionalizing the proposal was done in two ways: i) The State Act nº 14,026 / 2007, which
institutes PAIC; and ii) a Memorandum of Understanding among 100% of the town halls. The
Memorandum called the mayors to take on responsibility for the issue as a priority in their
mandate and committed to act under seven lines of actions: relocation of financial resources to
all programs in the area; review of career plan, prioritizing the literacy teacher position;
encouragement of teacher‟s commitment with child learning by means of training; technical
criteria setting for selecting school principals; setting up municipal systems for learning
assessment; and adoption of policies for encouraging reading and writing.
As regards its structure, PAIC has a managing committee with consulting and supervisory
functions, composed of institutions that had been involved in CCEAE, which have accumulated
48
The final CCEAE report brings the set of articles published in the media. During 2004, there was intensive coverage of the Committee‟s work, totalling 42 articles published. 49
In the same year, the state of Minas Gerais also started a program dedicated to help 3rd graders develop better literacy skills. Nevertheless, it was designed for schools under the state system responsibility.
Towards better literacy achievements in the 2nd
grade of primary schools
http://www.iipe-buenosaires.org.ar
experience in mobilizing and giving technical support to the municipalities50. Possibly this
networking allows a wider follow-up of the program with different approaches and
contributions, depending on the view of each institution involved.
Thus, a specific organizational arrangement was set up: SEDUC created the Coordenadoria
de Cooperação com os Municípios, COPEM (Coordinating Body for Cooperation with the
Municipalities)51, with its own personnel and budget. Each of the 20 Coordenadorias Regionais
de Desenvolvimento da Educação, CREDES (Regional Coordinating Bodies for Education
Development), the decentralized branches of SEDUC, created a Núcleo Regional de
Cooperação com os Municípios, NRCOM (Regional Centre for Cooperation with the
Municipalities) to mirror COPEM‟s structure and to operate at local levels. This structure is
responsible for giving technical support to the municipalities in the setting up of PAIC, but it
also manages other state government actions, such as school transport.
At COPEM, around 30 people run the program grouped by the acting axes and count on ad
hoc advice from Ceará universities‟ experts, whose work is paid by the state. Each Secretaria
Municipal de Educação, SME (Municipal Secretariat for Education) also organized its own
team of around three people, depending on the size of the education system, for the setting up
of PAIC. The State gives a financial bonus to the members of PAIC‟s municipal teams. This
bonus policy has caused some dissatisfaction among SME personnel not involved in PAIC, but
SEDUC has seen it as a strategic measure to signal for literacy priority, team organization and
for encouraging engagement. Figure 2 summarizes the management structure of PAIC at the
state, regional and municipal levels, identifying the main responsible for the actions and the
agency/staff the manager reports to within the administrative structure.
Figure 2 – Synthesis of PAIC Management Structure
Source: Author
50
Participating Institutions: Associação para o Desenvolvimento dos Municípios do Estado do Ceará, APDM-CE (Association for the Development of Ceará State‟s Municipalities), experienced in managing programs for competent reader training; Aprece and Undime, already mentioned in the text, are municipal entities; Fórum de Educação Infantil (Preschool Education Forum), a civil society entity which works to promote preschool education and has a vast experience in the field; Secretaria de Cultura do Estado do Ceará (Ceará State Secretariat for Culture), whose role is to strengthen the links among reading, writing and culture; and Unicef, also already mentioned, which implements the program “Selo Unicef Município Aprovado” (Unicef Passing Stamp for Municipalities) in Brazil. 51
See appendix III.
Towards better literacy achievements in the 2nd
grade of primary schools
http://www.iipe-buenosaires.org.ar
3.3.2 PAIC Design
Clear target and a comprehensive set of actions
The core target is literacy of 100% of the students at the end of the 2nd grade of primary
school, but PAIC is organized around five acting axes: Literacy, Municipal Management,
Preschool Education, Competent Reader Training, and External Assessment. Each one of
these axes encompasses a specific set of goals52, for instance full compliance with the 200-
day academic calendar in each municipality, universal access to preschool and availability of
reading corners in all 1st and 2nd grade classrooms. The State led the municipalities to commit
themselves to a set of actions considered relevant to ensure the literacy of a child at the “right
age”.
Municipal policies should incorporate these goals and have predetermined indicators to
monitor its implementation. Thus, for universal access to preschool, for instance, the indicator
will be the enrolment rate of 4-5 year-old children. Each municipality is responsible for
preparing a work plan, including targets and projections for four years, timetable and the
responsible parties involved. It should consider political support, baseline indicators and the
availability of financial resources to be allocated for the execution of each goal.
Improving municipal management and inducing policies at local level
The poor technical capacity of SME and the plurality of action fronts started by PAIC led
SEDUC to help with the process of improving municipal planning and management. A
thorough operation took place in order to guide each town to carry out a diagnosis of its
educational system, identifying action priorities, and defining targets and strategies related to
PAIC.
In this process, the state does not transfer financial resources to municipalities but gives
technical assistance. It also provides statistical data, planning tools and information systems
and suggests actions and indicators to be monitored. For example, it offers guidance to create
rural school clusters, to establish a selecting process for school principals through public
exams and technical profile analysis, or to define criteria to review career plan and teacher‟s
pay.
This is an extremely interesting process if it is compared to the traditional model of cooperation
between state and municipalities, which used to operate mainly through isolated programs and
transfer of financial resources. SEDUC decided to support the capacity building of SME in
diagnoses and planning to improve the effectiveness of municipal policies and thereby to be
able to meet the targets set by PAIC.
According to the information collected, the municipalities had initial difficulties in taking in the
idea that diagnosis is an essential step before setting priorities and planning actions. They
were not used to dealing with large quantities of data to be systematized and considered it a
waste of time for the manager and his team. Currently, these processes have been
incorporated into the dynamic functioning of the secretariats.
52
See Appendix IV for a detailed list of objectives and targets for each acting axis.
Towards better literacy achievements in the 2nd
grade of primary schools
http://www.iipe-buenosaires.org.ar
This does not mean that all the recommended processes were implemented. Sometimes the
creation of clusters is necessary - about 70% of the 6,524 primary public schools are in rural
areas – but runs into the opposition of communities. The appointment of principals based on
political nominations rather than professional merit is still the rule. The state finds it difficult to
discontinue this traditional practice in municipal policy. These issues have effects on the
program‟s implementation. Municipal managers said that multi-grade classrooms of rural
schools hinder the adoption of PAIC teaching materials. Furthermore, they say that some
principals little committed with results are, in many cases, a major cause of school slow
progress towards its learning outcomes, despite all efforts.
SEDUC designed other strategy of cooperation to ensure school transport in rural areas and to
expand education access to early childhood/preschool. Both involve financial requirement hard
to be taken only by municipalities. In these cases, for each vehicle purchased by the cities,
another is state funded, up to five units. In addition, each school built by the municipality to
assist children up to 5 years old adds another one with state funds. This is, of course, a way of
giving financial aid and inducing policies simultaneously.
Capacity building to manage the cycle of public policy – from diagnosis and planning to
monitoring and evaluation – can contribute to the sustainability of PAIC, and additionally may
help to turn the local leadership in education into actual professionals in the field. First
evidence of this can be inferred from the reported cases where the PAIC city manager has
been appointed to the position of municipal secretary of education.
At school level: improving management and offering in-service training
At school level, there are two main lines of action. The first is the provision of in-service
training for teachers working in the first two years of primary education. This training is linked
to the use of structured teaching materials. This material provides the contents and step-by-
step daily activities to be developed with students. The option for this type of teaching material
is due to the low theoretical and methodological teachers‟ background identified by CCEAE,
which have resulted in ineffective teaching practices in the classroom.
In Brazil, these kinds of materials face some criticism claiming that they are incompatible with
the autonomy and creativity expected from a teacher. It seems that the state decided to avoid
the controversy over the emphasis given (or not) to partial or full literacy53. The option was to
face the well-known weaknesses in initial teacher education54 by offering materials that
enable them to work and obtain better learning outcomes. On the other hand, schools still
receive textbooks from PNLD (they are to be used as supplementary material), which seems to
be a waste of public resources.
SEDUC states that textbooks selected by PNLD are incompatible with PAIC objectives and
that the kits Ceará purchased for 2nd grades includes teacher manuals, student textbooks,
classroom pedagogical materials and 80 hours of in-service training. Teachers can choose
53
According to SEDUC, PAIC materials consider full and partial literacy as interrelated processes. The approach combines the development of decoding skills with the constant interaction with different types of text and activities oriented to facilitate the construction of comprehension skills in reading activities. (SEDUC, 2011, p.86) 54
See Gatti, B. A. and Barreto, E.S. 2009. “Professores do Brasil: impasses e desafios”. UNESCO: Brasília.
Towards better literacy achievements in the 2nd
grade of primary schools
http://www.iipe-buenosaires.org.ar
from four different publishers, who will carry out monthly training sessions in each city55. For 1st
grade teachers, one specific teaching material was developed by a team of consultants from
Ceará. They attend training sessions conducted by PAIC‟s local team who receives guidance
from the consultants that had produced the material. In priority municipalities - with lower levels
of literacy proficiency - training is carried out directly by the consultants. Training is provided by
the state, but it is each city‟s responsibility to enable teachers‟ attendance (providing transport,
food and appointment of substitute teachers when it occurs in school days).
The selection, acquisition and distribution of the materials were bureaucratic and took long
time. In the first year, they were made available only after the beginning of the school year
because each municipality had to collect them in the capital. Later, an agreement with
publishers allowed them to be sent to CREDES, which streamlined the distribution process.
Another ongoing challenge is to ensure teacher attendance to training sessions. SEDUC
proposes these sessions are held on Saturdays, but this suggestion is not always welcome by
teachers.
The lessons from Sobral and the Committee grounded the decision that offering material and
training was not enough. It was essential to guide the attention and efforts of schools and SME
teams towards the relevance of monitoring some indicators: attendance rate of students,
teachers and principals; number of titles read by students; number of school days complied;
use (or not) of PAIC materials; attendance rate of teachers in training sessions and reading
status of each student, among others. SEDUC made available an online information system56
to gather and to process data collected from schools and SME. Bimonthly, reports are
analyzed at different levels of aggregation and by different agents: school managers, PAIC city
manager, PAIC regional teams (CREDES) and finally by COPEM team.
These strategies were designed to ensure “focus on student‟s learning” and to disseminate a
“culture of monitoring”. Both state and municipal agents who were interviewed said that the
merged processes of in-service training, pedagogical support and ongoing monitoring are
critical to the success of PAIC. According to the interviewees, cities that carry them out more
carefully are those which progress more.
The most important thing we have is the consistent support from the state, maintained
over the years. Training and monitoring actions are done every month. It is a strong
support, but much depends on the city to play its role. Here we do not let teachers miss
the training sessions. We seek to provide all the necessary logistics to facilitate such
participation. (…) When you are involved in this program you have to look at all parts of
the educational process. I am fully aware of what improved in my city, what is getting
better and what can improve in the future. (Municipal Agent)
Promoting education changes does not mean giving money to the municipalities. What
changes education is teacher training, provision of materials, follow-up of what is
happening in the classroom, and focusing on students‟ learning. The State provides
training, assistance in monitoring, but the municipality should play its role of ensuring
teacher‟s attendance and assuring the use of the structured material. If teachers are not
using the material in classroom, what are they doing instead? Are students learning? (…)
The more successful cities are, the greater their commitment is. The municipalities that
55
PAIC city manager and coordinators from schools also join these meetings. 56
Sistema de Acompanhamento das Ações do PAIC, SAAP (System for Monitoring PAIC Actions).
Towards better literacy achievements in the 2nd
grade of primary schools
http://www.iipe-buenosaires.org.ar
have participated in the training, which have kept their best teachers in the 1st and 2nd
grades, already have a good number of literacy-promoter teachers. They went forward.
They were concerned about placing teachers in the right places. Deployment is no longer
just a bureaucratic task. (State Agent)
Training and systematic monitoring are essential. Every month, our regional teams meet
with municipal teams to carry out follow–up work. And we (COPEM) meet monthly with
regional teams and bimonthly with municipal teams in order to monitor and to detect
problems and the need for visits. The municipal team never feels unattended, never feels
alone in the process. A network was built. There is a sense of collective responsibility.
(State Agent)
The training gives them (the teachers) the tools, but pedagogical support strengthens the
commitment. They become more cooperative when they are understood. (…) And then
the monitoring gives us information on how things run. (Municipal Agent)
There are other actions undertaken by the state: acquisition of children‟s literature books to
start “reading corners” for 1st and 2nd grade groups; production and distribution of children's
literature books based on local themes, aimed at linking regional culture and education;
meetings among schools and SME teams to exchange best practices and provision of training
for staff and preschool teachers. This latest action will be a great step forward in improving
preschool quality, a shortcoming already identified by research in Ceará supply57.
As a final stage, an external assessment was created to evaluate PAIC‟s achievements in
terms of literacy outcomes. Learning assessment gained its own dimension within this
educational policy and was designed as a means of refining municipal and school managers‟
accountability. Considering its relevance, this dimension will be further analyzed in the next
chapter.
In short, the target was set, several actions have been carried out and monitored and, at the
end of the process, the state evaluated how much the system progressed. Based on the
documents and on the interviews, it can be said that the state was successful in
communicating the main goal of PAIC and the need for acting and for monitoring progress in
different areas to reach it. Obviously the process does not always run smoothly.
It is expected that local teams use the monitoring reports to quickly identify which schools are
not doing well. In general, coordinators are the first to be contacted to understand the reasons.
Then, the schools are visited, and school managers and teachers receive specific guidelines
57
The CCEAE report highlights that 76% of the children who participated in the research for assessing literacy levels affirmed they had attended preschool, an index considered high by the researchers for the Ceará context and certainly meaningful for the national context (AL-CE/CCEAE p. 51). Coelho, in reporting Sobral‟s experience, states that “6-year-old children, assisted at preschool, finished the academic year without performing the first acts of reading and writing autonomously (...)” (Coelho, 2006, p. 158). The obvious inference is that if the early years of the first cycle of primary school are not successful in teaching a child to become literate, preschool education is not fulfilling its role of triggering the process either. This interpretation seems to be consistent with the studies of Maria Malta (2009), evidencing that higher accessibility to preschool education in the Northeast Region had frequently been associated to low quality education services. The issue becomes especially serious now with universal access to 4 and 5-year-old children until year 2016, as stated in the Constitutional Amendment 59/2009, and deserves attention for future studies. Preschool access has been considered by recent studies a relevant factor for success in subsequent school grades (Felício et. al., 2009; Calderini & Souza, 2009).
Towards better literacy achievements in the 2nd
grade of primary schools
http://www.iipe-buenosaires.org.ar
based on the problems identified. Considering the territorial extension – with many schools in
rural areas - the accomplishment of this assistance in situ depends on the availability of
transport. This is a permanent issue for PAIC city teams who depend on the support of
municipal leaders in SME and town halls. They depend on it to attend meetings at CREDES
and in the capital (COPEM) too. Logistical aspects are supposed to be tackled locally, however
if it is not possible due to technical or political reasons, CREDES and COPEM should be
involved in order to keep the prioritization of literacy in the agenda of the municipality.
The state agents who were interviewed consider that most municipalities have indeed
prioritized literacy and have been able to successfully implement the set of actions proposed.
Nevertheless, larger education systems face more difficulties to give pedagogical support in
situ and to monitor in a continuous way. It is also in the larger cities, such as the capital and
some CREDES headquarters, where there is a greater challenge of building a view of
partnership and non-competition among government levels.
We have 220 schools and 60 thousand students. Many are in rural areas. We have
multigraded classrooms. It is difficult to organize the logistics to make visits. We have to
group people responsible for different actions. When they go, sometimes they spend two
or three days. In rural schools everything is more difficult because the professionals are
less qualified. People do not want to work there. (Municipal Agent)
There is a historical duality between state and capital teams. It has to be with a
competition between the technical teams that are very similar and sometimes change
positions. It is a territorial issue. We got some improvements, but still have a long way.
The state schools were always the best, people from the state secretariat felt better than
the others... At CREDES the same thing happened. The regional coordinator was called
delegate in the past. There was a relationship of dependency and subordination between
the regional and the municipal teams. This is more evident in the largest cities. There,
the struggle is more evident. They used to think that everything that comes from SEDUC,
comes from the top down. But we have begun to improve this relationship. We are
improving little by little. (State Agent)
The work done in the first two grades have given rise to some discomfort among teachers
working with other groups. Another problem identified was that some municipalities were able
to get good results in SPAECE-Alfa, but 5th graders‟ learning outcomes remain below the
expected level. Upon the promotion, the 2nd grader dropped in the "real" condition of supply
and quality of municipal systems, said one municipal agent. In 2010, SEDUC considered that
the first step was taken in terms of literacy achievements and that it was time to meet the
demands to expand the program to 3rd, 4th and 5th grades. From 2011, it began to implement
PAIC +5.
Table 6 summarizes the role of the state and its municipalities in the development of PAIC
main lines of actions discussed in the last sections: organizational structure, SME
management, school management and in-service training/structured material.
Towards better literacy achievements in the 2nd
grade of primary schools
http://www.iipe-buenosaires.org.ar
Table 6 - Synthesis of PAIC main lines of actions and level of responsibilities
Topic State Municipality
organizational
structure
pays bonus to PAIC local teams identifies and selects PAIC teams
SME management makes technical assistance and
planning tools available
makes diagnosis, identifies priorities
and prepares work plan
school management suggests indicators to monitor and
make information system available;
aids in report analyses and in
shortcomings in terms of technical
and political aspects
collects indicators; analyses
reports; visits schools and helps in
solving the shortcomings identified
by the monitoring; logistics support
in-service training +
structured material
identifies options, selects, in
consultation with the cities, and
purchases the materials associated
with training sessions and other
pedagogical materials
Provides logistics to ensure
teachers‟ attendance to training
sessions and monitors the use of
materials in classrooms.
Source: Author
3.3.3 Inter-government collaborative action: what has changed?
It is important to note that cooperation between the state and its municipalities is, according to
Vieira (2010), a remarkable feature of the educational policy in Ceará, which began in the
1970s and deepened during the 1990s and 2000s, under the recent educational legislation58.
The long process towards the decentralization of education supply in the first cycle of primary
education occurred in Ceará in an intensive and dramatic manner in terms of impact on the
quality, as well as in most Northeast Region. Nevertheless, the staff interviewed reported that
the previous cooperation was more focused on the transfer of financial resources, assignment
of state personnel to work at local institutions, funding for building schools, and on keeping
school transportation. Actions aimed at improving learning outcomes and management at SME
and school level were rather feeble.
What did we have? The state used to have a good relationship with the municipal
entities. By 2006, SEDUC had a very small unit to deal with municipalities. The core of
this work was financial cooperation, personnel, cooperation for construction of schools or
for school transportation. Practically, actions oriented to support the quality of teaching
were rare. (State Agent)
In the past, the state government was much more focused on the problems of its own school
system. With PAIC, it has started to be a partner and co-responsible for improving quality in
municipal education. Moreover, the state has taken over the role of enhancer of municipalities‟
competences, a task that none of them would be able to lead.
58
The 1996 Act nº 9,394, which establishes the guidelines for national education, and the 2001 Act nº 10,172, which approves the National Plan for Education. They are norms aimed at guiding the organization and planning of educational policies in Brazil.
Towards better literacy achievements in the 2nd
grade of primary schools
http://www.iipe-buenosaires.org.ar
It was easy to get adherence of the municipalities to PAIC because of the committee's
work and the mobilization around the issue. What we realized was that the proposal was
well received by the municipalities. It was as if they were expecting the state to take over
this leadership. The pact signed by the mayors just formalized this option. (State Agent)
Another point is the systemic approach, from the induction of more effective and relevant
municipal policies to the organization of teachers' pedagogical routines. One of those
interviewed, who has worked for 35 years in the Ceará‟s educational system, stated that it is
the first time that state government is able to "get inside" the classrooms of municipal schools.
“At most, we arrived at the municipal office.” The idea of a systemic approach also appears in
the speech of municipal staff and leaders when they say that the “same opportunities” are
offered by the state to all municipalities, regardless of political bias. They add, however, that
prioritizing the issue and making the best use of the resources made available depend on local
leaders. All of them agreed that previously the intergovernmental collaborative action relied
heavily on demands of each city or political relations between mayors and the state secretary.
As we could see, the actions foreseen in PAIC to be properly implemented depend on the
prioritization of the issue on the agendas of the municipal secretary of education and the
mayor. There is consensus on this point. Rather than rely solely on the memorandum of
understanding, signed by the mayors when joining PAIC, regional and state teams work
together to demand this support whenever local teams or indicators show the need. In
addition, the state uses the external assessment as a means of reinforcing this priority, as will
be seen in Chapter 4.
These measures are associated with others, such as institutionalizing the program in state
legislation, to try to avoid the administrative discontinuity which commonly occurs after mayors‟
elections. The disruption of educational policies and programs is a widespread feature of the
education sector in Brazil. Much less present in recent years in state and federal governments
and still very frequent in municipalities where the processes are less institutionalized and
social control is weaker. PAIC successfully went through a first change in town halls in 2008
and will face another one this year 2012. The news is not only this accomplishment, but the
concern in structuring the continuity.
Lastly, the institutional arrangement that was set is also a unique point regarding the small
organizational unit that previously was in charge of articulating the two government levels. This
is another important signal to the municipalities about the priority given by the state to child´s
literacy. Moreover, this institutional arrangement will enable the continuous monitoring
planned in PAIC‟s design.
It can be said that PAIC‟s design and implementation is an attempt to respond to a set of
issues faced by most Brazilian states and municipalities: i) the difficulties resulting from the
dualism of Brazilian federalism in the educational field, in which one sought to define priorities,
but at the same time to generate shared competences that forced an interdependence among
the entities and reinforced the idea of a federation; ii) the process of turning the educational
sector into a complex realm, in which the solution to a problem demands several actions in
various fields which are interconnected and influence each other; and iii) the need to face the
issue of municipality dependence and frailty to occupy its own scope of action granted by
decentralization and effectively take responsibility over public education policy.
Towards better literacy achievements in the 2nd
grade of primary schools
http://www.iipe-buenosaires.org.ar
When taking on the coordination of this intergovernmental cooperation mechanism, Ceará
government clearly tries to induce the municipality to take over tasks that it considers relevant
for the development of the state as a whole, mainly to grant better learning levels in the
subsequent grades – these ones under state responsibility – but also with a view on sustaining
the socio-economic development that benefited this traditionally poor region more recently.
In order to achieve that, the state makes tools, systems and human resources available to
enable task execution within the municipal scope. Few Brazilian states took on the task of
liaising and coordinating educational policies along with its municipalities whereas this mission
is stated in the Constitution. This role used to be played mainly by MEC. This is good news
and if it is well applied, it can lead to better intergovernmental balance and may reduce
educational disparities.
Conclusion
In Ceará, the mobilization of different stakeholders, together with strong political and technical
leadership, was able to introduce the issue of illiteracy at school in the public policy agenda. In
the design and implementation of PAIC, the state government has taken a central role with
respect to: a) definition of focus; b) effective communication of priorities and alignment of
policies to achieve them; c) coordination of processes and actions in terms of management
and training, tasks that few municipalities have technical or financial conditions to accomplish
by themselves. The cooperation established also seeks to induce in the municipalities the
same priority that the state government has given to literacy outcomes by: a) creating a new
institutional arrangement; b) fostering more accountability through monitoring and evaluation;
c) implementing a systemic approach and a comprehensive set of actions to all municipalities,
regardless of political bias.
Towards better literacy achievements in the 2nd
grade of primary schools
http://www.iipe-buenosaires.org.ar
Introduction
This chapter explores the two kinds of external learning assessments implemented by Ceará
under PAIC. It begins with its rationale and describes its approach within the program. Then,
their main aspects are analyzed in order to understand how they became relevant instruments
for diagnosis and pedagogical interventions at school level and monitoring progress, promoting
mobilization (increasing public awareness) and accountability at system level. It also provides
a brief view on how the learning outcomes assessed are used for funding the municipal
systems and reward schools.
4.1 Rationale for external assessment on literacy achievement
As highlighted in Chapter 2, the institution of SAEB in the 1990s and later the addition of Prova
Brasil framed a national system for assessment in order to address the needs for describing
students‟ learning achievements, identifying shortcomings and informing public policies
throughout the country. In fact, it has been fulfilling these purposes over the years;
nevertheless, the centralization of this process results in at least two drawbacks: i) there is no
ownership over its results since most of the process takes place away from the municipalities;
ii) there is a long delay between fieldwork and communication/dissemination of reports.
Gradually, as learning outcomes got more attention from political actors and attracted
headlines in the media, the idea spread out was that since those evaluations were designed to
meet national system needs, they would not meet the needs of states and municipalities,
especially to reorient local policies and programs and to have impact on teacher practices and
school management. The reduction of costs and dissemination of technical knowledge in this
field are also aspects to be considered59.
This has led some states to develop their own assessment system, which in general seeks to
be complementary to the national one. Ceará is one of the pioneers in this process; it has
implemented the Ceará‟s Permanent Evaluation System for Basic Education (SPAECE) since
1992. Thus, when the state decided to implement PAIC and to give great relevance to external
assessment making it one of the structuring axes of the program, it already had a sort of
tradition and knowhow established at state level.
As seen in the previous chapter, PAIC is focused on learning achievement and on assigning
and raising accountability over it. In order to achieve this, it tries to promote changes in the
management of municipal systems of education, but also in teacher practices at classroom
level. Still in the pilot phase, the initial strategy of implementation was to build an external
assessment model which should set forth the reality at municipal and school levels and also for
each group and each student. The idea behind it is that evaluation findings should be truly
59
Expenditures on learning assessments are around 10% of PAIC total budget.
Towards better literacy achievements in the 2nd
grade of primary schools
http://www.iipe-buenosaires.org.ar
understood by managers, principals, coordinators and teachers, so that they can know
strengths and weaknesses of students and produce tailor-made interventions. It is obvious that
program designers see external assessment as a lever for change.
Under PAIC, two annual, census-based external assessments are conducted . The first one is
called PAIC Test and is carried out early in the school year with 2nd graders. It has a diagnostic
role and their results are not published. It is coordinated by SEDUC, but implemented
separately by each municipality of Ceará. The other is conducted by SEDUC only and it is a
branch of the state assessment system (SPAECE) which was expanded to incorporate the
assessment of literacy in public schools. SPAECE-Alfa, as it is known, is carried out at the end
of the school year with second graders and should provide information on the literacy process
as well.
If focus is on literacy outcomes, a serious problem arises: what should be considered an
adequate level of literacy for children in the first years of schooling? When is the time to test
them? As presented in chapter two, these are relevant questions still in the center of many
studies, however, they are not to be addressed in this paper. The important point here is that
Ceará State and its municipalities were able to reach a consensus and to set a standard about
this issue: children at the end of second grade must perform some competences in reading.
This is a very specific path in Ceará public policy since the CNE (National Council for
Education) indicates that the first three years of basic education constitute a literacy cycle,
which should be completed at eight years old (see CNE, 2010). Complying with this guideline,
the PNE states that the national target is all children become literate at age eight. Hence,
some Brazilian states, such as Rio Grande do Sul, Mato Grosso, Minas Gerais and Espírito
Santo, carried out tests in the 3rd grade.
Ceará does not ignore the guideline for a three-year literacy cycle in primary, but made a
choice of having children literate at age seven and designed its external assessment to be
adequate for second graders. Once again it seems that the context of PAIC formulation and
design helped to shape that consensus and to give it legitimacy. The Committee promoted
open sessions to discuss these concepts and the experience of Sobral is focused on literacy at
the end of the first two years of schooling too.
4.2 PAIC Test - external learning assessment as a tool for diagnosis, guidance to
pedagogical interventions and for monitoring of learning achievements.
PAIC Test resembles a hybrid assessment such as presented by Wagner (2011) in two
aspects: i) it focuses on local needs; and ii) there is a shorter timeline through the phases of
data collection, analysis and communication of reports. However, there are also differences.
The characteristic of potential "shareability" is less evident, since it was designed to cater for
the state context. It does not present the informal characterization described by Wagner for
hybrid assessment either.
The state established a thorough assessment protocol to be applied by municipalities so as to
guarantee the credibility of the assessment and reliability of its results. The fact that it is
applied to all students in a standardized pattern and carried out by an agent outside the school
approximates it to a large-scale model. So, it could be said that it is actually in between these
two models, but SEDUC identifies it as a large-scale assessment.
Towards better literacy achievements in the 2nd
grade of primary schools
http://www.iipe-buenosaires.org.ar
4.2.1 Design and implementation
PAIC Test aims to improve the quality of the literacy process providing a diagnosis of learning
achievement at disaggregated levels – city, school, group and student. The idea is to provide
information on early reading and writing skills of each student in time for the school to correct
paths and redesign pedagogical strategies for those who present deficiencies, as well as to
gradually respond in the direction of the target of all pupils‟ literacy at the end of 2nd grade.
Thus, the assessment has a formative intention and it is seen as an instrument for learning by
the government. Results are not to be published.
In order to plan this new model of assessment, SEDUC established partnership with
Universidade Federal do Ceará - UFC (Federal University of Ceará), so as to organize a
reference matrix of literacy competences and skills60 which should subsidize the design of the
assessment instrument (See Table 7). This initiative also responded to the initial obstacle that
there was no clarity in what the benchmarks should be for skills in literacy. UFC is responsible
for the three stages: developing the test items, carrying out their statistical analysis and
conducting pre-tests.
Table 7 – Reference matrix of competences and skills for children literacy (synthesis)
Domains Competences Descriptors
Alphabet acquisition
Recognize letters, differentiating them from other
graphic signs. D1 and D2
Recognize graphic conventions. D3, D4 and D5
Decode words. D6, D7, D8, D9, D10,
D11, D12, D13
Reading procedures
Find information. D14 and D15
Make inferences and/or meaning. D16
Support implications Indentify genre, function and target in a text
range. D17 and D18
Source: Manual for Reading Findings of PAIC Test.
The evaluation instrument is simple, with 20 multiple choice questions and 4 activities that
assess writing and reading skills outlined by the 18 descriptors of the reference matrix61. The
60
In 2007, UFC received technical assistance of Centro de Alfabetização, Leitura e Escrita,CEALE (Center for Literacy, Reading and Writing) , a body of the Faculdade de Educação da Universidade Federal de Minas Gerais, FE-UFMG (Federal University of Minas Gerais School of Education). 61
In 2009, PAIC Test consisted of 20 multiple choice items with a single correct alternative, which covered the acquisition of reading. There were 13 items that assessed the most basic skills, which were read aloud by the person in charge. The remaining seven items contained the comprehension skills of sentences and texts, and therefore, students were asked to read alone. Skills related to the writing system (it involves some skills for identification and recognition of the Portuguese code) were evaluated by other specific four activities: write his/her own name, some words, one sentence and a small text. (Manual de Leitura dos Resultados da Provinha PAIC, 2009, p.21)
Towards better literacy achievements in the 2nd
grade of primary schools
http://www.iipe-buenosaires.org.ar
results are computed by percentages of correct and wrong answers and do not refer to a
proficiency scale as adopted by SPAECE-Alfa and SAEB. The descriptors span from the most
basic skills such as identification of letters among other symbols (D1) to more advanced skills
such as inferring information in texts (D18).
PAIC local teams are responsible for conducting the tests in their cities, correcting them and
analyzing data, under state supervision and collaboration of experts from UFC. They were
trained to do that. Moreover, the evaluation routines were documented in a video and there is
a protocol that works as a roadmap with criteria for correction and detailed guidance for all
steps. SEDUC provides an on line information system, named SISPAIC, where the local team
processes the data collected. Reports are available immediately. The CREDES (the regional
branches of SEDUC) and COPEM are in charge of checking data consistency and doing
aggregated levels of analysis.
According to the local actors that were interviewed, this process runs smoothly. COPEM staff
confirms that the assessment axis is one of the best mastered by municipalities, although in
the past as in the present there are operational problems such as limited availability or lack of
paper, printers and computers. Due to this fact, SEDUC decided to provide one computer to
each SME to be used idealy in external assessment activities, but the agreement implies that
each level should fund its activities. It is important to highlight that local assessment teams
have been nurtured by SEDUC in view of an explicit target of PAIC.
The assessment goes smoothly. The whole team is mobilized to complete the process
quickly. When necessary, we have someone from outside SME. Last year, we called a
pedagogical coordinator with a literacy-promoter teacher profile to assist us in the test
scoring phase. (Municipal Agent)
If there is one thing that municipalities have seized is the assessment process. To put it
into the hands of municipalities was not just a matter of methodology, but a decision to
make them able to develop monitoring and evaluation. Municipalities have all kind of
difficulties: to make copies, to meet deadlines, to have people available, but they are
making continuous progress. Currently, the assessment goes without great problems. A
culture of evaluation is already part of their work. (State Agent)
4.2.2 Communication and uses of data
If PAIC Test is designed to monitor learning achievement and to impact directly on SME,
schools and teachers, then it is necessary to understand how the data collected is
communicated and used by them. That is how PAIC manages to close or at least minimize the
gap between what the assessment tools make available in terms of findings and how they are
used by stakeholders. As seen in chapter two, this issue has received very little attention in
external learning assessments.
This study did not intend to discuss whether or not there were effective changes in the
instructional practices of teachers based on PAIC Test findings. Nevertheless, this is an
interesting field that should be targeted for further investigation. The approach here is to
understand the procedures adopted in order to promote a better analysis and ownership of the
data generated. In PAIC‟s context, this trial of disseminating a better understanding of
numbers and graphs generated by SISPAIC can be divided into two phases.
Towards better literacy achievements in the 2nd
grade of primary schools
http://www.iipe-buenosaires.org.ar
The first phase is in charge of local actors. Findings are analyzed by SME teams and
discussed with the Secretary of Education and principals, as well as coordinators and
teachers. Pedagogical meetings are held to shed new light on and guide teachers‟ practice,
when there are insufficient results. The reports of each school, group and student performance
are the main material for these meetings and workshops. The focus is in the awareness of
general performance and problematic findings, but also in guiding pedagogical interventions if
specific deficiencies are identified.
SEDUC developed a user-friendly manual for guiding data analysis on PAIC‟s Tests at local
level. So it brings practical examples such as the meaning of the average of correct answers
for each item test or even what it reveals about the pupils‟ competence in accordance with the
ability of writing their own name. The manual also suggests exercises to be practiced with local
data and a set of questions to prepare its own diagnosis of learning achievement.
There was an intense training process for local teams. The 20 CREDES organized workshops
with SME managers and their teams to work on this analysis, and also visited each town to
assist them individually. Over time, both SME and school teams have become more
acquainted with the indicators, the process has become more dynamic and quick and the
teachers have started to improve their own strategies for the daily monitoring of their students‟
learning. In a parallel second phase, CREDES and COPEM also analyze the data and carry
out meetings with publishers responsible for in-service training to discuss over the topic. The
purpose is to enhance activities towards the skills that are less developed in students‟ groups,
according to SEDUC.
This is a very interesting process in Brazil and in the Northeast Region context, since one of
the criticisms made on the national assessment is the low ownership generated by its results in
local institutions. Other relevant point is that Ceará government does not expect that teachers
will be the only ones accountable for pupils‟ performance. All the main local education
managers and leaders are seen as responsible parties and should support teachers if changes
are needed. Moreover, SEDUC also plays its role in this field. Specific deficiencies in reading
or writing that arise in a systematic way and very poorly performing schools are the focus of
more technical assistance and more frequent visits organized by state and regional PAIC
teams.
More recently, as part of the capacity building strategy for shaping assessment competences
in municipalities, SEDUC has invited teachers from primary schools to take part in workshops
so as to prepare test items to assess skills and competences related to literacy. This action is
justified by SEDUC as a way to develop teachers‟ better understanding of PAIC Test and also
to induce better use of its findings. By performing these tasks it is expected that they will be
more aware of learning objectives and more capable of monitoring and assessing pupils in
class. The best items produced by them in these workshops are in fact incorporated into the
test.
Kellaghan and Greaney (2001) say that efforts to involve teachers in learning assessments
have been more frequent, and some try to involve teachers in the developing assessment
instruments and/or in scoring student responses as a strategy to impact their practices. They
acknowledge that these strategies can be useful if the proposed actions to address problems
in an education system require the dissemination of new knowledge and skills or changes in
teaching. In any case, “problems associated with changing the behavior of teachers should be
recognized” (Kellaghan and Greaney, 2001, p. 61).
Towards better literacy achievements in the 2nd
grade of primary schools
http://www.iipe-buenosaires.org.ar
Clearly, there is a good point in helping teachers to develop good assessment practices and
hopefully influencing their teaching through more effective strategies. However, considering
that one of the drawbacks of external learning assessments is the malpractice of teaching-to-
the-test, care must be taken to promote teacher‟s autonomy and to keep his/her focus broad
enough to encompass the development of all skills the CNE established to the initial phase of
schooling in Brazil:
School experiences include all the aspects of school environment: those comprising the
explicit part of the curriculum and those that also implicitly contribute to the acquisition of
socially relevant knowledge. Values, attitudes and guidance are conveyed not only by
formal knowledge, but also by means of routine, rituals, social rules, festivities, time and
organization of the educational space, learning materials and breaks, and finally by
means of all experiences offered by the school (CNE, 2010, Res. 7 of 2010, art. 9º, § 2º,
p.3). (in translation)
As the municipalities have met the challenge of implementing PAIC Test for 2nd grade, the
initiative is now spread out at the request of SME. It has been expanded gradually to the other
grades of the first cycle of primary school. In 2009, the assessment was expanded to 3rd, 4th
and 5th graders in Portuguese. In 2010, it reached literacy level of first graders and included
Math for 3rd to 5th graders.
4.3 SPAECE-Alfa - external learning assessment as a tool for monitoring achievements,
accountability and mobilization.
4.3.1 Design and implementation
By the end of each school year, Ceará carries out SPAECE-Alfa62, an external and census-
based assessment in reading for all public schools with 2nd grade groups. It provides a picture
of system performance on literacy skills after the two first years of schooling, therefore it is a
summative assessment. It considers the needs for monitoring, informing policy-makers and
raising standards. Since it provides data from state/regional/municipal levels to more
disaggregated ones (per institution, per group and per student), it is used as a feedback
parameter for the efforts made in a given school year.
Additionally, there are also other important roles this learning assessment plays in Ceará.
Perhaps, they are just as important as those mentioned above since they improve
accountability and mobilize educational actors to go on with the efforts and actions within
PAIC, as we will see ahead.
SPAECE-Alfa is designed in a more, say, „orthodox pattern‟ for large-scale assessments.
Instead of using classical test theory (CTT) like PAIC Test does, this assessment uses a more
sophisticated instrument involving the scaling of student responses and is based on item
response theory (IRT). As an “orthodox” large-scale assessment, SPAECE-Alfa uses an
62
The whole SPAECE system also comprehends learning assessment tests in a census-based approach in Portuguese and Math for 5
th and 9
th graders of primary school and 1st, 2nd and 3rd graders
of secondary school. The tests are carried out every two years, interspersed with the national assessment ones.
Towards better literacy achievements in the 2nd
grade of primary schools
http://www.iipe-buenosaires.org.ar
instrument that enables comparisons of results over the years. It focuses on municipal system
performance rather than on individual student's achievement. It is carried out for the state
government only; municipalities do not take part in data collection, processing and analysis.
Some items used by the assessment in 2009 were included in appendix V.
SPAECE-Alfa uses a continuum proficiency scale which goes from 0 to 200 points and it is
aligned with SAEB scale which goes from zero to 350 points for primary grades63. This
alignment enables comparisons between the results of state assessment and the ones
obtained from SAEB. There are two ways of interpreting learning outcomes using the scale:
the first one is linking them with levels of proficiency, and the other one is making the
connection with performance standards. From their position in the scale, a municipality or a
school knows which skills students have already developed.
4.3.2 Communication and uses
Data communication is carefully prepared. One of the features used by SEDUC to
communicate external assessment findings to all stakeholders is very simple, but with great
visual impact. Since 2007, after each evaluation round, the state map is painted in accordance
with the average performance cities have reached. Colors fade a gradient from dark green
(recommended standard) to white (illiterate) (see table 8).
Besides being a simple and quite objective tool to inform stakeholders, this measure of
comparison also seems to play a role in mobilization aspects, apparently helping to push the
process of change that the state government has planned. There are many references coming
from politicians, SME leaders and school managing teams in reports, speeches, newspaper
articles and blogs, resorting to ideas like "painting our town green" or highlighting that the city
already got its "green seal”.
The data dissemination process is very wide. Meetings are organized to inform municipal
managers, PAIC local teams, principals, teachers and mass media. SME and schools receive
a user-friendly report with SPAECE-Alfa findings about their students. The state produces
detailed reports which seek to guide managers of municipalities and schools to understand
them, to optimize the use of data available and to rethink strategies designed by SME and
schools for a more effective teaching-and-learning process.
Reports come with instructions on how to read the results. They describe the meanings of the
present average proficiency and the evolution over the years, percentage of pup ils‟
participation in the evaluation, students in each level of proficiency, general correct answers
and the individual percentage for each descriptor. They can get more details per student in the
information system64.
Both the most simplified information such as the total percentage of success in tests and the
more sophisticated one such as the results according to the standards of performance levels
are available by state, CREDES and municipalities. They enable comparisons among the
different management levels. The government does not produce ranking lists, but the press
does.
63
The scale is divided into intervals of 25 points. 64
See http://www.spaece.caedufjf.net/spaece-inst/
Towards better literacy achievements in the 2nd
grade of primary schools
http://www.iipe-buenosaires.org.ar
Table 8 - Scale for interpretation of SPAECE-Alfa
Performance Standards
No literate Literacy Incomplete Intermediate Sufficient Recommended
Proficiency
Levels
0-75 points 75-100 points 100-1250 points 125 a 150 points Over 150 points
How to
interpret
(summary)
Indicates
that the
student
has neither
learnt the
writing
system
code yet,
nor
developed
basic skills
for
reading.
Beginning of
learning process and
systematization of
basic literacy skills.
Indicates that
the student
acquires
autonomy in
reading syllable
words in (or
through)
various patterns
and begins to
understand
simple
sentences.
The student can
find information
in about 50
word- long texts;
the student can
also identify the
topic and
purpose of a
short narrative
text.
Early
development of
skills for
proficient
reading.
color in the
map
white red orange light green dark green
Source: Manual for Reading Findings of SPAECE-Alfa, 2009.
The use of league tables by the media has played an important role because it constrains
those mayors of cities with the worst outcomes, puts them under pressure and has helped to
keep the issue as a priority on the municipal policy agenda, according to the interviews. Cases
of teachers who are reluctant to be placed in schools with low results in the assessment have
already been identified by the municipalities. Nevertheless, they do not consider it as an issue.
One must admit that league tables are a way to forge accountability, but the media is only
interested in headlines. As highlighted in Brunner (2003), a social phenomenon to be treated
as news by the media needs to be pulled from the net of events in which it is produced.
Context factors are not relevant but only the results produced. This is the reason why this
approach has received criticism, as we can see in chapter two. It tends to encourage
comparisons and interpretations based only on ranking position, reducing the complexity of
academic achievement to a table of winners and losers. Maybe the most undesirable effect
could be schools focusing on what is best for their league table position and not on what is
best for their students.
The combination of the two assessments, PAIC Test and SPAECE-Alfa, allow a mini-
longitudinal look over the early literacy skills of students. They are seen as complementary by
the state. Indeed, the simplicity of PAIC Test produces rapid results that enable changes in
instruction and more effective literacy strategies. Teachers and students can benefit from their
formative role. In turn, the comparability and greater methodological rigor applied in SPAECE-
Towards better literacy achievements in the 2nd
grade of primary schools
http://www.iipe-buenosaires.org.ar
Alfa supply the means for monitoring the progress of cities over the years and for adopting
financial incentives which require extreme reliability to work well.
The state has set the level of difficulty in SPAECE-Alfa according to the improvements in
proficiency levels. The progress already achieved prevents the use of the same matrix of
Provinha Brazil, considered too easy for the students of Ceará.
Table 9 summarizes the main characteristics of the two learning assessments in terms of
theory used for designing the instrument, the approach adopted and its main purpose. It also
brings information on who carries out the assessment, the scope and the period of evaluation.
Table 9 - Synthesis of main characteristics of PAIC learning assessments
Assessment
Theory used for
instrument
design
Approach Main
purpose
Who
carries out When Basis
PAIC Test CCT formative diagnosis municipality
1st semester
of school
year
census
SPAECE-
Alfa IRT summative
monitoring ;
accountability state
end of school
year census
Source: Author. Note: Both instruments are based on the matrix of skills for children literacy in Ceará
4.3.3 Financial incentive – ICMS Sharing
Apart from being used for increasing public awareness on education issues and accountability
purposes, SPAECE-Alfa is also used for resource allocation and school rewarding. They
constitute financial incentives, intentionally designed to lead the system in the “right” direction.
In December 2007, the state government of Ceará revised the sharing of the Imposto sobre
Comércio de Mercadorias e Serviços, ICMS (a Value-Added Tax – VAT - equivalent), through
State Act nº 14.023. ICMS is the most important tax in the set of state revenues and,
consequently, a major source of revenue for municipalities65. A quarter of total ICMS is
distributed by the state to city governments through the so called Quota Share.
This share has two parts, the first one is calculated according to the city participation in the tax
generation66 (75%), and the second one (25%) is calculated based on three indexes
(education, health and quality of the environment)67. For the education sector there is the Index
of Education Quality, a mathematical equation that measures educational progress through
learning outcomes in the large scale assessments for students in grade 5 and grade 2 with a
greater weight on the latter, as shown in Figure 3.
65
The Federation of Industries of the State of Rio de Janeiro conducted a study on the fiscal management of municipalities. The study pointed to the great dependence of municipalities on transfers of federal and state funds. The vast majority of municipalities (83%) generated less than 20% of its revenue in 2010. See http://www.firjan.org.br/IFGF/ifgf_release.html 66
Valor Adicional Fiscal, VAF, (Fiscal Value Added). 67
Before the amendment, the smallest share was distributed considering population criteria and percentage of education spending over municipal revenues. There was also a percentage evenly distributed to all municipalities.
Towards better literacy achievements in the 2nd
grade of primary schools
http://www.iipe-buenosaires.org.ar
Figure 3 - ICMS Quota Share
Source: Author (adapted from Cartilha Cota Parte do ICMS)
The state government has established some measures to adjust the average scores of
municipalities. The index considers the extent of student participation in the test. It seeks to
curb unintended behaviors in the system, such as the willful exclusion of children with low
levels of proficiency, to raise the average scores. Cities are also penalized if they present a
high level of inequality in the performance of their students. Thus, the change promoted in
ICMS sharing has made the students participation in learning assessments an issue of great
relevance to municipal finances.
The engagement would be different without the ICMS and the School Award. The
assessment works as a lever. It's a chain. The school wants to win the award, teachers
naturally want to see their work recognized, the SME leaves the meetings convinced that
the results need to be improved to make a good impression, the mayor wants to receive
the funds. And his support is a critical point. With his support anything can happen (or
not). (Municipal Agent)
This decision seems to be crucial to make mayors more aware of children‟s illiteracy problems
and to induce practical actions at local level. ICMS Act certainly did not influence the mayors‟
decision of implementing PAIC, since it was taken after all cities started the program
implementation. Nevertheless, it is interesting to note that the participation of students in
SPAECE-Alfa has indeed increased, reaching nearly 100% in 2010. (Figure 4)
Figure 4 - Evolution of Students Participation Rates in SPAECE-Alfa from 2007 to 2010
Source: Author. Data from SPAECE-Alfa Reports
A) 18% based on education results: A.1) 12% based on 2
nd graders literacy evaluation;
A.2) 6% based on the quality index education of 5
th graders.
B) 5% based on health results. C) 2% based on quality of environment.
Towards better literacy achievements in the 2nd
grade of primary schools
http://www.iipe-buenosaires.org.ar
It is not possible to make considerations on potential equity problems derived from this
measure. More comprehensive data would be necessary to give an idea about municipalities‟
reality. However, it is interesting to notice that cities that got better and faster results at the
beginning of PAIC are the ones to receive less resources recently because the sharing
considers the index variation between the last two years. As stated by SEDUC (2011), some
small towns were able to triple its revenue from ICMS, while part of the cities with the largest
population had shrunk due to a slower index growth. Some mayors became quite dissatisfied
with this change. This is good news since small towns in Ceará are generally the poorest ones
without significant economic activity. And paradoxically, as seen previously in chapter three,
larger education systems face much more difficulties to fully implement PAIC strategies such
as in situ assistance.
In Brazil, the state of Minas Gerais adopted the most well known case of redistribution of ICMS
from the richest municipalities to the poorest ones, the so-called Robin Hood Act. However,
there are substantial differences from the Ceará initiative. The Robin Hood Act (Acts no.
13,803 of 2000 and no. 18,030 of 2009) considers a wide set of variables: ranging from
population size to cultural activities, from health to environment. In regard to education, it does
not consider learning outcomes, but aspects related to access. The priority given to the area is
also different as can be inferred from the weight dedicated to it in the formula.
4.3.4 Financial incentive – School Award
The state government also established the Prêmio Escola Nota Dez, (Mark Ten School
Award). The objective is to reward schools and to strengthen teacher efforts regarding the
children‟s literacy process. The state Act no. 14,371 of 2009 determines two conditions for the
awards: i) up to 150 schools can be awarded if they score between 8.5 and 10 on the Índice de
Desempenho Escolar – Alfabetização, IDE-Alfa (Literacy Performance Index); and ii) the
school must have a minimum of 20 students enrolled in the 2nd grade. The latter aims to induce
school clusters in rural areas, which is one of PAIC‟s targets.
The index is based on the results of SPAECE-Alfa. In the award design, the state has adopted
the same measure considered in the tax sharing. The score of each school is adjusted by the
percentage of participation of the students assessed in relation to the ones enrolled. It intends
to avoid the exclusion of children with low levels of proficiency in the tests. Furthermore, the
amount of money the school will receive takes into account the number of students assessed.
This is an attempt to have a real picture of school work and requires professionals to pay
attention to all students regardless of their strengths or weaknesses. On the other hand, it is
advisable to be aware of adopting non-explicit selection mechanisms in future enrollments.
The most interesting part is that the same number of schools (150) with lowest ratings in IDE-
Alfa also gets money from the state as a financial assistance to implement an action plan so as
to improve performance. The amount represents only half of what the „best‟ ones get per
student assessed. In both cases (awarded schools and schools supported), part of the prize
depends on increasing (or maintaining for the top ones) performance in the following year.
Another condition is that each of the top schools should „adopt‟ and give pedagogical support
to one at the bottom of the ranking during a school-year. The strategy intends to promote
cooperation among schools to disseminate best practices and to inspire more schools to
improve their learning results.
Towards better literacy achievements in the 2nd
grade of primary schools
http://www.iipe-buenosaires.org.ar
The results of this strategy are too inconsistent to be interpreted without a study of school
contexts and constraints. In 2009 56% of the schools awarded fail to maintain its index score
and in 2010 this percentage dropped to 24%. On the other hand, among the schools
supported, 72% achieved the target, at minimum score 5 in IDE-Alfa, in 2009. In 2010, this
percentage reached 90%. SEDUC says that it is hard for the best schools to keep the same
high results from one year to another. Another piece of data that seems interesting and could
be further investigated by the state is related to reducing disparities. The lower limit of IDE-
Alfa, where schools at the end of the ranking are placed, evolved from the interval (1.6 - 3.4) in
2009 to (3.5 - 5.7) in 2011, while the IDE-Alfa of top schools ranged from the interval (8.5 -
10.0) to 10.0 in the same years.
Figure 5 synthesizes the structure of the two financial incentives adopted, based on the
SPAECE-Alfa results. Tax sharing with the municipalities is based on IDE. Another index (IDE-
Alfa) is used to select schools that will be awarded or given financial aid. These schools must
be also involved in giving or receiving pedagogical support and must present an action plan to
get part of the financial resources. Both initiatives are regulated by state laws.
Figure 5 - Financial Incentives based on SPAECE-Alfa
Source: Author
Although this initiative has specific features, as noted above, proposals like this are very close
to the so-called bonus programs adopted in some parts of the United States. Bonus-pay
programs, considering either school-wide performance or teacher‟s group performance
separately, are a controversial issue in the education sector. The experiences of New York and
São Paulo, as it was seen in chapter 2, were not successful. Actually, it seems that there are
more uncertainties than certainties in this field.
In Ceará‟s case, it should be considered that some aspects can reinforce it positively. The
bonus initiative was implemented as a means of rewarding and recognizing efforts. It is also
relevant to consider that the context where the idea is implemented apparently has not been
considered as a way to push responsibility to the weakest link in the education system, the
teacher. All in all, it has motivated schools agents towards better learning outcomes so far, as
can be seen from documents and the interviews.
Moreover, the proposal is included in a structured set of actions that seek to improve learning
outcomes, and provide effective support for schools to perform them. Therefore, to add one
more aspect to Goodman‟s hypothesis, it is not only “the structure of the payment scheme”
that may make a difference in terms of effectiveness, but also the policy structure in which the
proposal is inserted. Anyway, the experience is very recent and should be carefully evaluated
by the state in the future.
Spaece Alfa and Financial
Incentives
Schools
Local Govern-
ment
Level
Tax Sharing
Awards and Financial Aid
Instruments
Index of Education
Quality (IDE)
IDE-Alfa / Pedagogical Support / Plan
Criteria Legal
Framework State Act nº
14.023 of 2007
State Act nº 14.371 of 2009
Towards better literacy achievements in the 2nd
grade of primary schools
http://www.iipe-buenosaires.org.ar
In 2011, the legislation related to the financial incentives was revised to be adjusted to the
goals of PAIC +5. The award now includes over 150 schools with better learning outcomes in
the 5th grade. The ICMS sharing now has a better balance between the results of 2nd and 5th
grades. Both regulations included changes in favour of the education systems that reach the
highest percentage of students in adequate levels of proficiency. “We want to drive the who le
system to universal learning. Now many cities present adequate averages, but have only 50%
of the students at the recommended level”, explained the coordinator of COPEM. This is good
news because Ceará shows that is able to adapt PAIC to the current stage, when the
education system has responded to the interventions planned. The state is concerned about
keeping pushing it to improve quality and to reduce disparities.
As reported by COPEM‟s coordinator, a survey conducted by the state indicated that 40% of
municipalities have already set up their own assessment systems. Many implement bonus
programs based on performance in these assessments. Municipalities do not want to wait for
the results of SPAECE-Alfa. In fact, one secretary of education interviewed stated his (and the
mayor‟s) intention of starting such initiative as a way of recognizing merit. Moreover, it is
common to have simulated testing in the cities, which aims to familiarize children with the
instrument used in SPAECE-Alfa.
Many cities have implemented the 14th salary, 15th salary for teachers and they cannot
(or do not want) to wait for the state assessment. Then they establish other
assessments. Many municipalities make their own assessments now. This is not a good
thing, it is too much, but weighing up everything, I prefer their concern for ceasing
illiteracy among their students. I prefer they cease to find it normal. In many places
children aren‟t prepared for the objective test, because they are not used to it. They do
not know what is to set the correct alternative. We do not agree with this practice, but I
understand it. Let me emphasize that I do not blame them for wanting the child to get
acquainted with the instrument. (State Agent)
Whereas the students‟ achievement is "cash value" for the municipalities, it seems that they do
not want to run the risk of receiving less money because children do not understand well the
multiple-choice test or for any other reason. States and municipalities should be aware of the
risk of overtesting practices. They are not welcome in any stage, but are especially
inappropriate in a very early stage of schooling.
Raising expectations
The state of Ceará seems to have put into practice by the book lessons about the role of
external assessment in education. The government targeted the effective literacy of all 2nd
graders, set the standards to measure it and has been monitoring students‟ performance by
way of external assessment since then. It has sought to disseminate a „culture of evaluation‟,
to communicate better the results and to make effective use of findings. These lessons are
easy to be learnt, but not so simple to be implemented in a complex federation like Brazil.
What seems to be an important step forward in this process is that both municipal
leaders/managers and school personnel raised their expectations over children‟s learning
outcomes. This has special meaning in an education system that got used to great proportions
of students with low levels of achievement as SAEB has showed since 1990‟s. This is also a
Towards better literacy achievements in the 2nd
grade of primary schools
http://www.iipe-buenosaires.org.ar
good path to start motivating all school personnel in performing better their activities, and to
make teachers more aware of students‟ needs. In this sense, the intended ownership over
external assessment findings may be interpreted as a natural consequence of it.
Another interesting point is that the external assessment is put in a central role for forging
accountability and prioritization, but it is also aligned with a set of actions as presented in the
last chapter: instructional materials, teacher training, capacity building, pedagogical support,
etc. This perspective reinforces the idea that measurement can be an important source of
change, but it is not enough. Between pros and cons, learning assessment has been used in
Ceará as a management tool to improve the quality of education. One more aspect that should
be highlighted is the action of giving further support to schools that have presented low levels
of learning achievement in the assessments. This is a relevant strategy if one considers that
one of the main objectives of intergovernmental cooperation is to reduce disparities.
On the other hand, returning to the by the book guidelines and considering the financial
incentives that were put into practice, it might be interesting to pay more attention and even to
commission studies on overtesting to ensure that the program remains on track.
Conclusion
Regarding the learning assessment within PAIC, three points stand out:
a) The state government has indeed used its results as a management tool to monitor the
education system, measuring progress and reorienting actions.
b) In order to induce this practice at local level, SME and school teams should be guided to
grasp and to use evaluation results to improve management practices and pedagogical
routines. This process is unlikely to occur spontaneously given the institutional frailties in
Brazil, and especially in the Northeast region. If the focus set by educational policy is clear and
well communicated, this process tends to be much more effective.
c) The dissemination of a “culture of monitoring and evaluation” in the education sector was
reinforced by financial incentives that seek to lead prioritization and mobilization. This
combination has proven to be virtuous and has resulted in greater accountability and political
support, which are critical issues for PAIC. It is worth noting that the system has been put
under pressure for results and should be aware of over testing practices in the long run.
Towards better literacy achievements in the 2nd
grade of primary schools
http://www.iipe-buenosaires.org.ar
This brief chapter tries to summarize some lessons learned from Ceará regarding the design
and implementation of PAIC, a recent experience of fighting illiteracy at schools.
Context analysis
The state government took into account relevant facts while formulating its policy towards
better literacy achievements at the right age, which have enhanced the success of its
implementation:
a) The mobilization generated by the Committee headed by the State Assembly represented a
source of change and made it easier to get the adherence of all mayors from 184 cities to the
proposal. It also provided the necessary legitimacy to some complex decisions, such as setting
the target of all children literate at age seven and promoting changes in ICMS sharing.
b) The state learned it from an initiative already implemented in one of its municipalities, the
city of Sobral, and perfected it, in a rare case of bottom-up policy in Brazilian federalism.
Unfortunately, most of the state legislatures perform a very weak role in setting agendas and in
discussing and contributing to public policies. The Ceará experience shows how these
institutions can have a more vigorous role in this field.
Intergovernmental collaborative action
PAIC is based on two main messages: focus on learning and accountability for results. The
program has managed to turn these messages into well-defined targets and goals and
effectively communicated them to all stakeholders and also has managed to align policies to
achieve them.
In the design and implementation of PAIC, the state government has taken a leadership with
respect to the actions implemented. The State has chosen to tackle the most common issues
for the cooperation between states and municipalities in the education sector rather than get
around them:
a) The issue of municipality dependence and frailty to occupy its own scope of action granted
by decentralization and effectively take responsibility over public education policy.
b) The difficulties resulting from the dualism of Brazilian federalism in the education sector, in
which one sought to define priorities, but at the same time to generate shared competences
that forced interdependence among the entities. It caused gaps and tensions among
government levels.
Towards better literacy achievements in the 2nd
grade of primary schools
http://www.iipe-buenosaires.org.ar
c) The growing complexity of education sector, in which the solution to a problem demands
several actions in various fields which are interconnected and influence each other.
The state of Ceará and its municipalities are overcoming these challenges through a model of
intergovernmental cooperation that distinguishes itself from past experiences, with emphasis
on:
a) creating a new institutional arrangement;
b) implementing a systemic approach and comprehensive set of actions to all municipalities,
regardless of political bias;
c) fostering more accountability through monitoring and evaluation;
d) improving technical capacities in SME management (in terms of diagnosis, planning,
monitoring and evaluation);
e) improving monitoring of relevant indicators at school level concerning the objective of having
greater levels of literacy in 2nd graders;
f) supply of in-service training focused on what happens in the classroom and on the
shortcomings of teacher‟s qualification.
Few municipalities would have technical or financial conditions to lead this process of changes.
Under the coordination and ongoing support of the state government, most of the
municipalities got motivated to implement PAIC actions and achieved great progress, as
shown by the results of the program.
And yet the game is not won. The challenge of keeping the municipal education systems
focused on student‟s learning remains. There are great difficulties in changing certain
management practices, such as the appointment of principals by political nomination. In spite
of all the information disseminated on the principal's role in the quality of education, there is
resistance in abolishing it from the system. Either in Ceará or in any other state, simple
aspects such as the full compliance with the 200-day academic calendar should not be taken
for granted. That is why monitoring is so important. Even if the in-service training and the
meetings are considered important, there is the constant issue of assuring transport for
teachers and coordinators. The issue can be even more difficult to deal with in large states
such as Amazonas and Pará or in states with lots of municipalities such as Bahia and Minas
Gerais.
External learning assessment
The experience of PAIC underlines that external and large scale learning assessments can
indeed serve to a variety of purposes: from monitoring systems‟ performance to fostering
greater levels of accountability and awareness of student´s achievements at school; from
diagnosis to funding and mobilization.
Nevertheless, to meet these objectives they should be properly designed and should have
findings communicated to the relevant audience. Ceará has been successful in these tasks: i)
Towards better literacy achievements in the 2nd
grade of primary schools
http://www.iipe-buenosaires.org.ar
it has tailored PAIC assessments according to their needs, goals, availability of human
resources and local experience; ii) it has also disseminated its results in a way that system
and school managers are able to grasp and to use them to improve management practices
and pedagogical routines. Through this strategy it overcame one of the great drawbacks of
learning assessments generated so far, the low ownership over the data by local staff.
Since the focus set by the educational policy is clear and well communicated, this process has
run smoothly.
The experience of Ceará indicates that political support from mayors and municipal education
secretariats is needed if one expects to keep literacy as a priority in local agendas in the long
run. Therefore, the dissemination of a “culture of monitoring and evaluation” in the education
sector has been reinforced by financial incentives. Tax sharing and the school awards have a
role in PAIC results as have led to more prioritization and mobilization around the issue. It is
worth noting that the system has been put under pressure for being more accountable for
learning results, but it has been the focus of a set of support actions to move forward into the
planned direction.
In Ceará, the challenge of keeping PAIC actions as a priority to the municipalities is
permanent, but deepened significantly in government elections. The threat of administrative
discontinuity is always lurking around the cities of Brazil. That is why it is so essential to
institutionalize the actions as much as possible and to get local staff ready to do their
functions. On the other hand, the adoption of financial incentives is always controversial. It
seems that these measures can be adopted with minor restrictions when there is great
legitimacy involved in the decision.
Whereas the students‟ achievement is "cash value" for the municipalities or local performance
pay initiatives start to be implemented, states and municipalities should be aware of a risk of
overtesting practices. They are not welcome in any stage, but are especially inappropriate in a
very early stage of schooling.
The results from external learning assessment have been used not only to monitor the
program, but to guide new actions when the system responded to the first interventions
planned. Nowadays the state is concerned about keeping pushing it to improve quality and to
reduce disparities in the next years. The more recent changes in tax sharing seek to ensure
that higher percentages of students reach the appropriate stage of literacy and to reduce the
relevance of average proficiencies. Clearly, there will also be new and greater challenges to
achieve better learning results at the end of the first cycle of primary schools as proposed by
PAIC+5. This is either because the approach is more complex (for instance, the inclusion of
new disciplines such as math) or because the expenditures on in-service training, monitoring
and logistics among others will increase.
Finally, Ceará shows that the assessment can be used as a tool to raise expectations about
student‟s learning, which is definitely an achievement in Brazil where education systems
became very used to low levels of performance in the last two decades.
Towards better literacy achievements in the 2nd
grade of primary schools
http://www.iipe-buenosaires.org.ar
Abrucio, F. L. 2010. “A dinâmica federativa da educação brasileira: diagnóstico e propostas de
aperfeiçoamento”. In R. P. Oliveira and Wagner Santana (ed.) Educação e federalismo
no Brasil: combater as desigualdades, garantir a diversidade. Brasília: UNESCO -
Representação Brasil. pp. 39-67
AL-CE/CCEAE (Assembléia Legislativa do Estado do Ceará/Comitê Cearense para a
Eliminação do Analfabetismo Escolar). 2006. Final Report. Fortaleza: AL-CE.
Bruner, J. J. 2003. “Límites de la lectura periodística de resultados educacionales”. In G. Iaes
et al Evaluar las evaluaciones. Una mirada política acerca de las evaluaciones de la
calidad educativa. Buenos Aires: UNESCO/Instituto Internacional de Planeamiento de la
Educación. pp. 67-80
Calderini, R. and Souza, A.P., 2009. Pré-escola no Brasil: seu impacto na qualidade da
educação fundamental. Article presented in XXXVII Encontro Nacional de Economia
(ANPEC). Retrieved in 29 February 2012, in
http://www.anpec.org.br/encontro_2009.htm#trabalhos
CEC/CD (Comissão de Educação da Câmara dos Deputados). 2003. Relatório final do grupo
de trabalho – alfabetização infantil: novos caminhos. Brasília: Câmara dos Deputados.
Coelho, I. C. de A. 2006. “Sistema Municipal para Avaliação da Alfabetização de Crianças”. In
AL-CE/CCEAE (Assembleia Legislativa do Estado do Ceará/Comitê Cearense para a
Eliminação do Analfabetismo Escolar). 2006. Final Report. p.156-163. Fortaleza: AL-CE.
Felicio, F. and others. 2009. The effects of early child education of literacy scores using data
from a new Brazilian assessment tool. Article presented in XXXVII Encontro Nacional de
Economia (ANPEC). Retrieved in 29 February 2012, in
http://www.anpec.org.br/encontro_2009.htm#trabalhos
Ferrer, A. T. 2003. “Qué pretendemos evaluar, qué evaluamos y qué conclusiones podemos
extraer de la evaluación?” In G. Iaes et al Evaluar las evaluaciones. Una mirada política
acerca de las evaluaciones de la calidad educativa. Buenos Aires: UNESCO/Instituto
Internacional de Planeamiento de la Educación. P.111-132.
Goodman, S. and Turner, L. 2011. Does whole-school performance pay improve student
learning? In Education Next. Spring. pp. 66-71. Retrieved from: www.educationnext.org
in April 12, 2012.
Haddad, W. D., 1995. Education policy-planning process: an applied framework.
Fundamentals of Educational Planning nº 51. Paris: UNESCO/International Institute for
Educational Planning.
Iaies, G. 2003. “Evaluar las evaluaciones”. In G. Iaes et al Evaluar las evaluaciones. Una
mirada política acerca de las evaluaciones de la calidad educativa. Buenos Aires:
UNESCO/Instituto Internacional de Planeamiento de la Educación. pp. 15-35
Towards better literacy achievements in the 2nd
grade of primary schools
http://www.iipe-buenosaires.org.ar
Iaies, G. et al 2003. Evaluar las evaluaciones: una mirada política acerca de las evaluaciones
de la calidad educativa. Buenos Aires: UNESCO/Instituto Internacional de Planeamiento
de la Educación.
INEP (Instituto Nacional de Estudos e Pesquisas Educacionais). 2003. Qualidade da
Educação: uma nova leitura do desempenho dos estudantes da 4ª série do ensino
fundamental. Brasília: MEC/INEP.
_________. 2005. Vencendo o desafio da aprendizagem nas séries iniciais: a experiência de
Sobral/CE. Série Projeto Boas Práticas na Educação nº1. Brasília: INEP.
IPECE (Instituto de Pesquisa e Estratégia Econômica do Ceará). 2010. Ceará em Números
2010. Retrieved in 12.02.2012, in www.ipece.ce.gov.br/
Kellaghan, T. 2008. “Qué mecanismos de monitoreo se pueden utilizar para efectuar estudios
internacionales comparados y estudios nacionales?” In Kenneth N. Ross & Ilona J.
Genevois (ed.). Estudios internacionales sobre la calidad de la educación: La
planificación de su diseño y la gestión de su impacto. Paris: UNESCO/ Instituto
Internacional de Planeamiento de la Educación. (pp. 54-69)
Kellaghan, T. and Greaney, V. 2001. Using assessment to improve the quality of education.
Fundamentals of Educational Planning nº 71. Paris: UNESCO/International Institute for
Educational Planning.
Lockheed, M. E. 1996. “International Context for Assessments”. In Paud Murphy et al
(ed.).National assessments: testing the system. Washington D.C.: Economic
Development Institute of the World Bank (pp. 9-19).
Marsh, J. and others (2011). A Big Apple for Educators - New York City's Experiment with
Schoolwide Performance Bonuses: Final Evaluation Report. New York: Rand
Corporation and Vanderbilt University.
Ministério da Educação, Brasil (MEC). 1997. Parâmetros Curriculares Nacionais: Língua
Portuguesa. Brasília: MEC
Oliveira, J.B.A. n.d. Alfabetização, fracasso escolar e políticas públicas. Instituto Alfa e Beto.
April 11, 2012 retrieved from:
http://www.alfaebeto.org.br/Arquivos/Documentos/Artigos/Cap%C3%ADtulo%207.%20J
o%C3%A3o%20Batista%20Ara%C3%BAjo%20e%20Oliveira.pdf
Oliveira, J.B.A. 2004. Alfabetização de crianças e adultos: novos parâmetros. Belo Horizonte:
Alfa Educativa.
Pigozzi, M.J. 2008. “Que és la „calidad de la educación‟?(Desde la perspectiva de la
UNESCO)”. In Kenneth N. Ross & Ilona J. Genevois (ed.). Estudios internacionales
sobre la calidad de la educación: La planificación de su diseño y la gestión de su
impacto. Paris: UNESCO/ Instituto Internacional de Planeamiento de la Educación. (pp.
41-53)
Towards better literacy achievements in the 2nd
grade of primary schools
http://www.iipe-buenosaires.org.ar
Postlethwaite, T. N. 2004(a). What do international assessment studies tell us about the quality
of school systems? Background paper prepared for the Education for All Global
Monitoring Report 2005, The Quality Imperative. Paris: UNESCO.
Postlethwaite, T. N. 2004(b). Monitoring educational achievement. Fundamentals of
educational planning n. 81. Paris: UNESCO/International Institute for Educational
Planning.
Postlethwaite, T.N. and Kellaghan, T. 2008. National assessments of educational
achievement. Education Policy Series n. 9. Paris: UNESCO/International Institute for
Educational Planning.
Saus, J. B. 2003. “Encuentros y desencuentros con los procesos de evaluación de la calidad
educativa en América Latina”. In G. Iaes et al Evaluar las evaluaciones: una mirada
política acerca de las evaluaciones de la calidad educativa. Buenos Aires:
UNESCO/Instituto Internacional de Planeamiento de la Educación. (pp.37-65)
SEDUC (Secretaria de Educação do Estado do Ceará). 2011. Regime de Colaboração para a
garantia do direito à aprendizagem: o Programa Alfabetização na Idade Certa (PAIC) no
Ceará. Fortaleza: Governo do Estado do Ceará. Mimeo.
Silva, C. S. R. and Cafiero, D. 2010. “Alfabetização e letramento de crianças de seis anos: as
políticas para a implantação do ensino fundamental de nove anos”. In InterMeio. Revista
do Programa de Pós-Graduação em Educação, v.16, n.32, p.27-48, jul./dez. 2010.
Campo Grande: Universidade Federal do Mato Grosso do Sul.
Soares, M. n.d. Por uma alfabetização até os oito anos de idade. Todos pela Educação. April
2, 2012 retrieved from http://www.todospelaeducacao.org.br/biblioteca/1387/por-uma-
alfabetizacao-ate-os-oito-anos-de-idade/
Todos pela Educação. 2007. Nota Técnica Preliminar Metodologia para a Obtenção das
Metas Finais e Parciais. Comissão Técnica do Compromisso Todos pela Educação. São
Paulo, 10 de setembro de 2007.
UNESCO (United Nations Educational Scientific and Cultural Organization). 2005. Education
for all: the quality imperative. Global Monitoring Report. Paris: UNESCO.
_______. 2006. Education for all: literacy for life. Global Monitoring Report. Paris: UNESCO.
Vieira, Sofia L. 2010. “Educação básica no Ceará: construindo um pacto colaborativo”. In R. P.
Oliveira and Wagner Santana (ed.) Educação e federalismo no Brasil: combater as
desigualdades, garantir a diversidade. Brasília: UNESCO - Representação Brasil. pp
271-286
Wagner, D. 2011. Smaller, quicker and cheaper: improving learning assessments for
developing countries. Paris: UNESCO/International Institute for Educational Planning.
Towards better literacy achievements in the 2nd
grade of primary schools
http://www.iipe-buenosaires.org.ar
Act no 9.394 of 1996: establishes the basis and guidelines of national education.
Federal Constitution
CNE (National Education Council). 2010. Regulation no 7 of 14th December 2010:
establishes national curriculum guidelines of the 9-year primary education.
Decree no. 29,896 of 16/09/2009: regulates the Act no. 14,371/2009.
Decree no. 30,796 of 29/12/2011: makes amendments to Decree no. 29,881/2009 in
order to improve the calculations of the Municipal Index of
Educational Quality.
Decree no. 30,797, of 29/12/2011: regulates Act no. 15.052, of 06/12/2011.
State Act no. 14,026, of 17/12/2007: institutes PAIC – The Literacy Program at the
Right Age. State Act no. 14,023 of 2007: distributes ICMS resources, according to the
external assessment results.
State Act no. 14,371, of 19/06/2009: institutes the Mark Ten Award.
Act no. 14,580, of 21/12/2009: make amendments to Act 14,371/2009.
Act no. 14,949, of 27/06/2011: make amendments to Act 14,371/2009, including the
award to the 5th grade of primary school.
Act no. 15,052, of 06/12/2011: intitutes the Mark Ten Award to the Best 2nd and 5th
grade schools.
Boletim do Sistema de Avaliação SPAECE – Volume I
Boletim do Sistema de Avaliação SPAECE – Língua Portuguesa – 5º ano – Volume II
Boletim de Resultados da Escola – SPAECE-Alfa 2010 – Língua Portuguesa – 2º ano do Ensino Fundamental – Volume III
Boletim de Resultados da Escola – SPAECE-Alfa 2009 – Língua Portuguesa – 2º Ano do Ensino Fundamental – Volume III
Boletim de Resultados Gerais – SPAECE-Alfa 2009 – Língua Portuguesa – 2º Ano do Ensino Fundamental – Volume II
Cartilha Cota Parte do ICMS – Governo do Estado do Ceará e IPECE
Manual de Leitura dos Resultados – Provinha PAIC 2º Ano – 2009 – Eixo Avaliação Externa (SEDUC e Universidade Federal do Ceará)
Manual de Orientação para o Acompanhamento das Ações do PAIC – Regional – 2009
Manual de Orientação para o Acompanhamento das Ações do PAIC – Município – 2009
Matriz de Referência da Alfabetização
Relatório Final do CCEAE
Relatório Geral de Resultados do Estado do Ceará – Atividade Avaliativa PAIC-Alfa 1º ano – 2010.
Relatório Geral de Resultados do Estado do Ceará – Provinha PAIC 3º Ano e Provinha PAIC 4º e 5º Anos - 2010
Oficina de elaboração de itens – Material de apoio didático – 2009
Orientações Curriculares para a Educação Infantil – Secretaria de Educação do Estado do Ceará – 2011
Towards better literacy achievements in the 2nd
grade of primary schools
http://www.iipe-buenosaires.org.ar
Orientações para elaboração do plano de aplicação dos recursos referentes às escolas premiadas e apoiadas, para o exercício financeiro de 2011,segundo a Lei nº. 14.371, de 19 de junho de 2009.
Notas taquigráficas de audiência pública realizada, em 08/06/2011, pela Comissão
Especial do Plano Nacional de Educação (Projeto de Lei nº 8035 de 2010) na Câmara
dos Deputados, com a presença da Secretária de Educação do Ceará, Izolda Coelho,
e o governador do Estado do Ceará, Cid Gomes.
Notas taquigráficas de audiência pública, em 07/04/2010, realizada pela Comissão de
Educação, Cultura e Desporto do Senado Federal, com a presença da Secretária de
Educação do Ceará, Izolda Coelho.
http://www.spaece.caedufjf.net/spaece-inst/
http://www.idadecerta.seduc.ce.gov.br/
http://www.inep.gov.br
http://www.ibge.gov.br
Towards better literacy achievements in the 2nd
grade of primary schools
http://www.iipe-buenosaires.org.ar
I) Brazilian Education System – Structure and Organization
A) Correspondence between the International Standard Classification of Education (ISCED 97)
and Brazilian educational levels.
ISCED 97 levels Brazilian educational levels
ISCED 0
(Pre-primary level of education)
Crèche: 0 up to 3 years old
Pre-school: age 4 and 5
ISCED 1
(Primary level of education)
Ensino Fundamental, Fundamental (or Primary)
Education
(1st to 5
th grade)
(age 6 to 10 years)
ISCED 2
(Lower secondary level of education)
Ensino Fundamental, Fundamental (or Primary)
Education
(6th to 9
th grade)
(age 11 to 14 years)
ISCED 3
(Upper secondary level of education)
Ensino Médio, Secondary Education
(1st to 3
rd grade)
(age 15 to 17 years)
ISCED 4
(Post-secondary non-tertiary)
Not applicable to Brazil
ISCED 5
(First stage of tertiary education)
Tertiary education Type B (Technological Higher
Education)
Tertiary education Type A
(undergraduate, master‟s degree and professional
master‟s degree courses)
Source: Based on information available in “Education at a Glance 2011 OCDE Country Note - Brazil”.
Retrieved in July 21, 2012 from: http://www.oecd.org/dataoecd/44/63/48657313.pdf
B) All members of the Brazilian federation held education responsibilities. The Union is
responsible for the funding and the management of the federal educational system, mainly
higher education institutions; states must provide secondary education and early childhood
education (crèche and pre-school) is in charge of municipalities. States and Municipalities
share responsibilities over primary education.
Towards better literacy achievements in the 2nd
grade of primary schools
http://www.iipe-buenosaires.org.ar
II) Efficiency indicators for primary and secondary levels in Ceará, Northeast Region and Brazil
Levels/Rates Primary - 1º/5º Primary - 6º/9º Secondary
Overaged Rate*
2001 2010 2001 2010 2001 2010
Ceará 46,5% 21,0% 73,7% 31,0% 60,2% 34,7%
Northeast 59,8% 26,6% 66,6% 40,4% 67,4% 46,6%
Brazil 39,4% 18,5% 45,7% 29,6% 50,0% 34,5%
Drop out Rate
1995 2005 1995 2005 1995 2005
Ceará 6,8% 6,9% 11,8% 18,5% 1,4% 2,6%
Northeast 12% 11,4% 14,1% 15,0% 3,5% 3,3%
Brazil 9% 7,4% 13,1% 14,1% 2,8% 3,4%
Promotion Rate
1995 2005 1995 2005 1995 2005
Ceará 73,5% 76,6% 73,5% 64,1% 89,2% 82,3%
Northeast 63,4% 68,7% 64,5% 66,3% 82,6% 80,5%
Brazil 71,9% 78,3% 69,0% 68,5% 83,7% 81,8%
Repetition Rate
1995 2005 1995 2005 1995 2005
Ceará 19,7% 16,5% 14,7% 17,4% 9,4% 15,1%
Northeast 24,6% 19,9% 21,4% 18,7% 13,9% 16,2%
Brazil 19,1% 14,3% 17,9% 17,4% 13,5% 14,8%
Note: *Expresses % of students at least two years more than the recommended age.
Source: Todos pela Educação/INEP/MEC
Towards better literacy achievements in the 2nd
grade of primary schools
http://www.iipe-buenosaires.org.ar
III) Institutional arrangement for COPEM in Ceará State Secretariat for Basic Education
(SEDUC)
Towards better literacy achievements in the 2nd
grade of primary schools
http://www.iipe-buenosaires.org.ar
IV) Objectives and targets of PAIC’s five acting axes
Management Axis
Objective: to promote institutional strengthening of municipal teaching systems by providing
technical assistance for the setting up of a management model focused on learning results.
Targets:
1. Raising IDEB to 6.0 in the early years of Primary Education.
2. Assistance for 100% of children in the 6 – 14 age group.
3. Reduction of the dropout rate to 0% in the first cycle of Primary Education.
4. Reduction of the dropout rate to 0% in the second cycle of Primary Education.
5. Correction of school flow.
6. Creation of school clusters.
7. Compliance of 100% of the 200 days of the school calendar.
8. Strengthening of school autonomy.
9. Policy making on assistance to municipal schools.
10. Setting up technical criteria for the selection process of school managers prioritizing
merit.
Literacy Axis
Objective: to provide pedagogical and technical assistance to municipalities in setting up
didactic literacy proposals which guarantee effective children‟s literacy.
Targets:
1. Literacy of 100% of children up to the end of the 2nd grade of Primary Education.
2. Literacy of pupils who have not achieved full literacy between the 3rd and 9th grades of
Primary Education.
Preschool Education Axis
Objectives: To contribute to improve quality in the service offered by Preschool Institutions and
to the implementation of both pedagogical proposals and in-service training of Preschool
teachers.
Targets:
1. Broaden the offer of preschool places to children aged 0 to 3;
2. Have all children between 4 and 5 years old enrolled in preschool.
Children Literature and Competent Reader Training Axis
Objective: To fulfill both children‟s and teachers‟ right to human development, to cultural
instruction and to social inclusion through access to literature, promoting acquisition,
distribution and effective use of book collections, as well as permanent in-service training,
focusing on the importance of Children Literature during the full literacy process.
Towards better literacy achievements in the 2nd
grade of primary schools
http://www.iipe-buenosaires.org.ar
Targets:
1. 100% of children in Ceará‟s municipalities with access to quality book collections and
having read a minimum of 5 (five) children‟s books per year, from the 2008 school
calendar onwards.
2. All educators working in preschool and in the early years of Primary school have
attended a Continuing and Permanent Training Program of at least 120 hours per year,
from the 2008 school calendar onwards.
3. All teachers have participated in the Effective Children Literature Workshops and have
acquired the Effective Children Literature Kit.
4. Creation of a reading collection in the classrooms, with a minimum of two books per
children, from the 2008 school calendar onwards.
5. All pupils mastering reading fluency and comprehension, checked by regular
assessment.
Assessment Axis
Objectives: i) To disseminate an educational assessment culture in Ceará‟s municipalities, so
that they have a team in the Municipal Education Secretariat. ii) To diagnose reading, writing
and text comprehension learning status of pupils in the early grades of all municipal schools,
communicating assessment results by municipality, school, class and student. iii) To provide
subsidies so that the Education Secretariats develop a management focused on students‟
learning and implement policies to improve the quality of the teaching offered in the early
grades.
Targets:
1. Carrying out diagnostic assessments with children enrolled in the early grades of
primary school (2nd to 5th grades) in all Ceará‟s municipalities at least once a year.
2. Availability of an information system to input external assessment data collected by the
municipalities.
3. Publishing assessment results to directors, supervisors and teachers, aiming to
improve the quality of education in the early years of primary education.
4. Ownership of the assessment results by the Municipal Education Secretariats through
the various teams in charge of management, assessment, pedagogical management,
preschool education and children‟s literature, in order to suggest actions that aim to
improve pupils‟ learning process.
5. Use of external assessment results by the municipalities for local educational policy
making.
6. Setting up of an external assessment team in the Municipal Education Secretariat.
Towards better literacy achievements in the 2nd
grade of primary schools
http://www.iipe-buenosaires.org.ar
V) Items from SPAECE-Alfa 2009
A) Item P020170A9 - skill assessed: letter identification
Tick the box containing the two letters I am going to say appear:
The letters are: D Z.
% Scores: A = 1.2% B = 1.4% C = 93.5%
D = 2.0% Blank and null = 1.9%
B) Item P020194A9 – skill assessed: writing direction identification
Tick the box containing the last word of this verse:
How can a living fish live out of cold water?
% Scores:
A = 4.7%
B = 4.5%
C = 77.1%
D = 12.2%
Blank and null = 1.5%
G S
O C
D Z
T J
HOW
COLD
WATER
FISH
Towards better literacy achievements in the 2nd
grade of primary schools
http://www.iipe-buenosaires.org.ar
C) Item AL0163MG – skill assessed: different letter spelling recognition.
Read the words: BARATA - TOMADA
Tick the box containing the words you have just read.
% Scores: A = 8.2% B = 4.9% C = 81.2%
D = 2.9% Blank and null = 2.8%
D) Item D15N3 – skill assessed: sentence comprehension.
Tick the sign indicating two products for sale.
balada - topada
banana - tocada
barata - tomada
batata - torrada
ROAST CHICKEN
FOR SALE
FARM CHICKEN
FOR SALE
CHICKEN AND EGGS
FOR SALE
FREE RANGE CHICKEN
FOR SALE
Towards better literacy achievements in the 2nd
grade of primary schools
http://www.iipe-buenosaires.org.ar
% Scores: A = 10.2% B = 10.2% C = 69.2%
D = 7.5% Blank and null = 2.9% E) Item PA2.0032 – skill assessed: explicit information finding in texts.
Read the text:
Tick the box that shows what Goldilocks found.
% Scores: A = 72.9%
B = 11.2% C = 9.2% D = 4.6% Blank and null = 2.1%
Source: Boletim de Resultados da Escola – SPAECE-Alfa 2009 Volume III – Alfabetização 2º
ano do ensino fundamental.
Note: Free translation by the author.
Once upon a time there were three bears living in a cottage in the woods. Papa Bear, Mama Bear and Baby Bear. They loved oatmeal. In the same forest also lived a girl named Goldilocks because of her long blond hair, which she loved to comb. She loved to find new places and one day she found the cottage of three
bears.
THE COTTAGE OF THREE BEARS
THE WOODS OF THREE BEARS
OATMEAL
BABY BEAR
top related