ARTICLE IN PRESS

0738-0593/$ - se

doi:10.1016/j.ije

�Correspond

Fax: +852 2362

E-mail addr

International Journal of Educational Development 25 (2005) 145–155

www.elsevier.com/locate/ijedudev

Performance evaluation or standardized testing of aptitudes?Innovations at the margins Mexico’s of school system

Chris Martin�, Elsa Guzman

Ford Foundation, Mexico

Abstract

This paper argues the need for broader criteria than are customarily used, for measuring pupil achievement, school

effectiveness and impact on educational policy in the plural and highly unequal societies of Mexico and Central

America. We proceed by first presenting the context of highly centralized mainstream educational provision,

concentrated in urban areas contrasting with the extremely limited services available in rural communities. Second, we

describe grass-roots educational solutions to this latter condition. Finally, we will situate these solutions within the

framework of international scholarship on assessment, evaluation and impact measurement in order to permit the

achievements of these experiences to be credited and lessons to be drawn from them. We conclude that comprehensive,

contest-sensitive evaluation establishes a two way communication between the educational system and local school

experience.

r 2004 Elsevier Ltd. All rights reserved.

1. Introduction

Starting from John Holt’s comment that ‘‘Ourconcern should not be to improve testing but toeliminate it.’’ (Holt, 1972, p. 58), we note thatstandard aptitude tests (SATs) are pervasive in theUSA and used increasingly in Mexico. Criticsargue SATs measure a narrow range of abilitiesand display strong cultural biases (O’ Freedle,2003). They preclude other forms of evaluationand risk widespread injustice, particularly to

e front matter r 2004 Elsevier Ltd. All rights reserve

dudev.2004.11.018

ing author. Tel.: +852 2766 6347;

9362.

ess: c.martin@fordfound.or (C. Martin).

underprivileged and culturally marginal studentswho approach learning and knowledge in ways notvalued in SAT repertoires (OCE, 2002, pp. 29–31;Baker and Robert, 1997, pp. 10ff, and Smagor-insky, undated).

Differentiating simplistic assessment of aptitudeand attainment from more holistic evaluations ofperformance, Holt argues that educationally validevaluation springs from learner desires to monitorprogress and teacher roles in assisting it. Evalua-tion as an administrative impulse may be counter-productive, for educators and learners alike.Learning occurs when the volition of the indivi-dual is freely released, not at another’s behest. Besteducational practice enhances learning-centered

d.

ARTICLE IN PRESS

C. Martin, E. Guzman / International Journal of Educational Development 25 (2005) 145–155146

evaluation and benefits from it, in a two-wayprocess.

The most successful learning innovations inMexico tend to be learner-centered, particularlywhen located at the margins of the formaleducational system, in unpropitious social condi-tions. Community-based, they prioritize localknowledge over mainstream culture and learning.Their pyscho-pedagogical and philosophical as-sumptions inspire bridge-building between the twoto enable children to move from the familiartoward an understanding of their place in thewider world (Cole and Wertsch, 1996, p. 4;Arendt, 1993, Chapter 5; Caballero, 2002, pp.161–171; Gramsci, 1976 pp. 74ff). Achieving thisrequires considerable skill and school wide policiesand conditions grounded in a democratic, tolerantand socially committed school-ethos (Crick 2000,Chapter 9). Some statistical indicators of learningoutcomes of these initiatives are available, butaccounts of the process and its intrinsic value areignored in official records.

2. The Mexican setting

Mexico has seen a doubling of school atten-dance across the population, from 3.7 years in1970 to 7.8 years in 2000 (SEP, 2000). Theproportion of the age cohort in primary andsecondary school has increased (SEP, 2000), butmore than a million children are not enrolled(Presidencia de la Republica, 2001). Primaryattainment has improved and the recent dissemi-nation of performance statistics, nationally andinternationally, has seen renewed public interest inschools.

The new statistics also reveal continuing pro-blems. Young people do poorly in internationaleducational league tables, most recently in themathematics tests of the PISA1 evaluation (Re-forma 1.6.03, 10A). Nationally, poor performancereflects wide attainment disparities and the un-equal distribution of educational services, asso-ciated with wide social inequality (CONAPO,2000, Fig. 14, p. 28). The overall picture is a cause

1Py Iy Sy Ay.

for public concern (Reforma ibid, Financiero2.6.03, La Jornada 4.6.03), particularly given thelevel of educational investment over the past 10years. Reforms include major administrative in-novations (decentralization); changes in teachertraining and performance expectations (Martinand Solorzano, 2003); new compensatory pro-grams in marginal areas (Ornelas, 2001); and thedevelopment of evaluation systems. The reformshave seen modest improvements in enrolment,retention and attainment, but persistent inequal-ities remain, with rural, indigenous studentsremaining underprivileged.

Compensatory programs for marginal groupswere introduced with the disappointing effect oftop-down reforms on scholastic attainment inpoor areas, but do not usually receive governmentsponsorship. Run by popular education activistgroups, they offer low-cost basic services to poorpeople. Participants taking official SATs scorewell, but evaluations of how this is achieved arelimited. This is in spite of the interest of widernational and international organizations in theirsuccess, among them the World Bank andUNESCO. The cultural bias and restricted con-ceptual reach of SATs cannot account for thecommunity effects of these educational initiativesand the value they add to achievement, particu-larly in indigenous communities. Mindful of theseminal study of the Escuela Nueva in Colombia(McEwan, 1999), there is a case for examiningcontemporary Mexican innovations more closely.Selected for their success, we present below anumber of case studies and discuss their implica-tions for approaches to evaluation.

3. Five innovations

The vignettes below derive from research andexternal evaluation reports of five educationalinnovations introduced in marginal areas ofMexico in the last 10 years (inter alia, seeSolorzano, 1999, Corona, 2002; MacDonald,2001; Zorrilla, 2003a, b, De Andraca, 2003). Thereports were prepared over 3 years, with FordFoundation support, and presented in November2003 to the biennial conference of COMIE, the

ARTICLE IN PRESS

Table 1

Classification of case study educational innovations

Projects: project

features

CIS NRS Tatutsi REI CC

Students served All primary

students from

locality and those

from neighboring

communities

requesting it

All local

secondary

students

All local

secondary

students

All local

secondary

students

All primary and

secondary students

from schools

volunteering to enter

the program

Pedagogy Mix of teacher

and tutorial

learning enriched

with hands-on

practice

Active,

constructivist

learning with

strong

engagement with

local context

Active,

constructivist

learning with

strong

engagement with

local context

Traditional

learning style of

guided student-

centered learning

Active, constructivist

learning, with strong

emphasis on group

work

Relations with ed.

authorities

Relative

autonomy within

mainstream

system

Relative

autonomy within

mainstream

system

Independence

from mainstream

except for teacher

salaries and

official

recognition of

curriculum

Complete

independence

from mainstream,

complementing

students’

mainstream

schooling

Independent program

with official

recognition and close

collaboration with the

authorities to upgrade

mainstream learning

Relations with

community

Close relationship

of mutual services

between school

and community

Engagement with

community

development

needs

Complete identity

between school

and community

Complete identity

between school

and community

Indirect relationship

with the community

through satisfying

general demand for

educational

improvement

Overall mission Provide general

but enriched

education to

underprivileged

students

Adapt standard

curriculum to

local

circumstances in

order to arrest

emigration

Fulfill community

educational

aspirations for an

indigenous

version of

mainstream

education

Complement

mainstream

education with

indigenous

knowledge, values

and skills

Upgrade mainstream

education with

enriched learning via

teacher training

C. Martin, E. Guzman / International Journal of Educational Development 25 (2005) 145–155 147

Mexican Congress of Educational Research. Datafor the same period include official and indepen-dent statistics of student and teacher performance,ethnographic interviews and notes of participantobservation.

Small-scale and geographically isolated, each ofthe projects has achieved recognition well beyondtheir operational areas, with some listed innational and international registers of educationalinnovations. They have several features in com-mon and each has acted as a reference point forsimilar younger projects. All are located in remote,mainly indigenous areas. They have strong links

with local communities, which support them inways rarely seen in mainstream rural schools. Allhave some ties to the formal system, but these varyin degree and kind. Common to each, even themost independent, is a desire for official financialsupport and recognition of learner attainment. Allencourage studies that complement and enrich thenational curriculum, often through artistic andincome-generating activities. Table 1 summarizesthe five projects in terms of their social catchment,pedagogy, links to the formal educationalsystem, community relations and their overarchingpurpose.

ARTICLE IN PRESS

C. Martin, E. Guzman / International Journal of Educational Development 25 (2005) 145–155148

3.1. Integrated community schooling in Oaxaca

(CIS)

The oldest project was Zoogocho in Oaxaca. Itwas founded in the early 1940s by the charismatic,socialist ex-president Cardenas. It looks like anordinary school, but is one of the few remainingprimary boarding institutions, most of whichdisappeared as population centers developed theirown primary schools. The live-in community atZoogocho, including students, teaching and ser-vice staff, gives the school its unique character. Itintegrates formal studies with contributions to theupkeep and social life of the school and service tothe community. Music is the focal point of itsintegrated approach. Taught as a curriculumsubject, students learn to play instruments andjoin the school band to enliven school events.Some become music teachers and help establishbands in neighboring villages, performing in localcultural festivals. Providing music has become acargo, a designated community responsibility inthe local area, enabling students to serve theircommunities in culturally valued ways. A con-ducive influence was the desire of nationalPresidents Juarez and Diaz to create a traditionof art music in the region of the Juarez highlandswhere Zoogocho is situated, such that today localcomposers are recognized as contributing toOaxacan identity.

Restricting the evaluation of these achievementsto SAT scores says nothing about the effect ofsuch holistic learning or the influence of the widerschool ethos on the high academic performancesachieved. Typically, poor communities in Oaxacaattain a literacy rate of about 80%. In one of thepoorest parts of the state, Zoogocho has a99%literacy rate, among the highest levels in thecountry (Lira, 2003, referring to 2001 census data).The surrounding schools with which Zoogochohas close contact exhibit similar success.

3.2. The new rural secondary school in Puebla

(NRS): television secondary schools-plus

In 1968, the Mexican government launched itstelevision-based secondary school (TVS). Broad-casts were to supplement scarce human and

physical resources in remote areas, where studentperformance was well below the level of thoseelsewhere attending complete secondary schools.In most rural areas today, the TVS model is theonly secondary provision available. Very giftedand politically radical, the former priest whoinitiated the scheme has worked for over 30 yearsto improve the model, based in a remote highlandarea of northern Puebla.

The NRS project includes all the 14 schools ofthe district. Its aim is to activate learning withprinted texts and broadcasts providing the basis ofa curriculum, which encourages learning about thelocal area and involvement in community devel-opment schemes. With student participation, theproject seeks to revitalize the local economy andsociety, threatened by economic depression andout-migration. In humanities, NRS SAT scores areabove average for schools in such areas, but belowaverage in mathematics and science. The externalevaluation conducted by MacDonald (2001) sug-gested that sustaining active learning demands toomuch of the teachers with their current levels oftraining. To redress the deficiency, the project isinvesting in intensive in-service training in activescience teaching, with a mobile laboratory, a widerange of web materials and close engagement withresearch and development projects in the locality.MacDonald’s evaluation reveals greater achieve-ments in the retention of students and in participa-tion in school-community projects. There isconcern that opting for locally relevant knowledgemay give insufficient exposure to the mainstreamcurriculum, to the eventual disadvantage ofstudents seeking their future outside a localitywhere only 10% will remain.

3.3. The Tatutsi Waxarari educational centre in

Jalisco

If the NRS seeks to transform the standardTVS, the Tatusti project wants to replace it.Tatutsi is the only independent, complete second-ary institution in the Huichol highlands. Almostall the others are TVSs, without the enrichment ofthe Puebla model. Inspired by popular demandand built by the community, Tatutsi stands out inthe region and beyond as a UNESCO, 2003

ARTICLE IN PRESS

C. Martin, E. Guzman / International Journal of Educational Development 25 (2005) 145–155 149

educational innovation. Situated in the northernhighlands of Jalisco, Tatutsi is an expression ofindigenous educational aspirations, enjoying thesupport of indigenous associations, a private Jesuituniversity, a private high school and internationalfoundations. Its aim is to provide high-qualityeducation from a Huichol perspective, at anequivalent standard to mainstream secondaryeducation. In practice, this has meant coveringthe main subject areas of the official curriculum,eliciting discussion and activities engaging with theHuichol context, and promoting values of leader-ship, negotiating-skills in cross-cultural contextsand a commitment to service, inside and outsidethe school. Complementing the full mainstreamcurriculum, there are courses in music, crafts, foodpreparation, construction and agriculture. There isofficial recognition of Tatutsi courses and somesalaries are paid by the state. Students have notparticipated in the official SAT evaluations, butCorona assessed standard aptitudes in 2002.Although entry levels are low, Tatutsi studentsperform better overall than equivalent students inurban schools, with the exception of mathematics.This creates a strong argument for more holisticmeasures of value-added. More significant, socialstudies work on a topic of local relevance,indigenous rights and land appropriation by drugtraffickers, assured Huichol students of a higherlevel of conceptual development than their urbancounterparts in the region (Corona, 2002). Themore holistic evaluation was able to measureoutputs in terms of aptitudes and also describethe processes likely to have produced them.

3.4. Raramuri educational initiative (REI)

The REI complements rather than enriches poorquality official schooling. It is outside the school,meeting in the afternoons and evenings at aspecially designated venue. As such, its program,still being developed at the time this paper waswritten, will not have to conform to officialregulations. The REI involves expert researchers(working locally in the sciences, arts and linguis-tics) documenting the region’s bio- and culturaldiversity and then incorporating this informationinto the curriculum. The aim is to reaffirm

Raramuri identity through studies that valueindigenous culture in national and internationalcontexts, to the mutual enrichment of each.Evaluation will need to assess student progress inthe light of the local expectations that informedthe project and also of SAT scores achieved in theofficial school to determine the REI effect onmainstream school progress.

3.5. Casa de la Ciencia (CC): upgrading teachers

in multicultural, multigrade schools

This project differs from the four above in thatit stands at one remove from the school and itscommunity. Its aim is to strengthen education inpoor, multicultural schools through teacher up-grading in Chiapas. Pre-service teacher training ishighly prescriptive and standardized, not lendingitself to work in culturally diverse areas, and in-service training is limited. The CC innovation isfor an adviser to accompany teachers in theclassroom and demonstrate alternative approachesto teaching. The CC claims that it does go beyondteaching currently esteemed best practice. Itssuccess lies inculcating best practice in localschools, through their voluntary adoption of theCC program, the maintenance of the commitmentrequired and the long-term relationship createdbetween them and the CC.

An example is the reading program. The CClibrary holds multiple copies of a large number oflively texts lent to schools on request. The CCgives guidance on how to make best use of thebooks and assists the teachers in using them inclassroom activities. The schools may lend thebooks to students, who read and use them in wayssuggested. The schools may also lend books toother schools, monitoring and reporting back ontheir use. In this way the CC promotes readingand, keeping track of books, establishes a bondbetween itself, client schools and other schoolspotentially interested in CC support. Besidespedagogical support, the CC provides staff devel-opment on broader aspects of school life, such asmanaging multicultural classrooms, leadershipand school development for directors and super-visors.

ARTICLE IN PRESS

C. Martin, E. Guzman / International Journal of Educational Development 25 (2005) 145–155150

Improved attainment in the subject areassupported by the CC and in the working climateof the school would confirm the value of the CCapproach. As the program is still new, the CC doesnot think that its achievements will be fairlyreflected in official SATs, so it is working to moreimmediate benchmarks of progress. It has readingprogram statistics for the number of books lent,read and used in classroom exercises. It alsocollects quantitative and qualitative data on leveland type of participation of students and teachers.Through SAT-type data, the CC monitors studentprogress in attaining key reading indicators,remembering titles, summarizing stories and con-ducting exercises similar to those in official text-books.

4. Discussion

4.1. The raison d0etre of the projects

Mexico has three major programs to provideeducational services in poor and remote areas, andamong migrants as well. They include the CON-AFE system of low-cost basic education usingvolunteer teachers, the TVS and its equivalent atprimary school level. Finally, there is the multi-grade school program. In spite of all this,educational services do not reach all young people.Projects like the five above represent an innovativepopular response to otherwise unrealizable educa-tional aspirations in underprivileged communities,but the value they add to student achievement isnot necessarily made apparent in evaluations. Theprojects also aim to enhance the quality of existingprovision, challenging its insensitivity and preju-dice toward local people.

Evaluators should assess their success in inte-grating local culture, knowledge and learningprocesses and official national educational pur-poses. Where users feel that such innovativelearning provides an equivalent or above averageservice to users, evaluations accounting for theprocess would be more informative than thoserestricted to input/output comparisons. Processindicators would measure how desired outputswere reached, given the very limited inputs typical

in marginal areas. In time, such data may improveunderstanding of mainstream deficiencies, whichsupplementary projects such as these set out toredress.

4.2. SAT limitations

The points raised above in relation to the fiveMexican case studies have been central to manydiscussions of educational evaluation, while every-where preoccupation with the rigor of SATs hasoccasioned considerable technical research (see,for example, the US National Center for Researchon Evaluation, Standards and Student testing,CRESST, http://www.cse.ucla.edu/index4.htm).In contrast, there are many critiques which high-light the shortcomings and dangers of SATs (inter

alia see discussions in Noam and Smith, 1996).They stress the following:

1.

The discrimination of SATs against aptitudes ofcultural minorities.

2.

The negative effects of a testing culture onlearning.

3.

Attempts to collect and classify comparableperformance data encourage a narrow view oflearning.

4.

SATs privilege low levels of conceptual abilityand the ability to guess correctly in multiple-choice tests.

5.

Their tendency to measure inert competencies,rather than active applications of learningthrough performance-based assessments.

4.3. The option for the learner: constructivism

Critical research on SATs argues for equitablesystems of educational opportunity and theegalitarian and comprehensive representation oflearner achievement. These concerns are transpar-ent in the evaluations of the projects presentedabove, all of which serve marginal communitiesand espouse strong commitment to equity.

Learner-centered pedagogies construct under-standing and conceptual ability on the basis of thelearner’s existing knowledge, independently ofteacher transmission of prescribed information.

ARTICLE IN PRESS

C. Martin, E. Guzman / International Journal of Educational Development 25 (2005) 145–155 151

Their scientific base lies in cognitive psychology(see Cole and Wertsch, 2000). Two schools ofthought accept that children learn by buildingstepwise on their developing abilities (Piaget’sstages of development) and upon their socio-cultural reference points (progressing throughVigotsky’s zones of proximal development, ZPDs).Learning abilities common to all humans (thecentral tenet of Piagetians), including communica-tive competencies, calculation, productive skillsand artistic endeavor, are not acquired in theabstract, but through and in specific culturalsettings. These provide the symbolic tools enablingthe development of high-level conceptualizations.In respect of language, contemporary thinking onbi-lingual education stresses the importance ofearly learning in the mother tongue (L1) (Lopez,1998; Ferreiro, 1997), as the template acquiringother skills, such as navigating place and time andlearning a second language (L2), as requirementsfor fuller participation in society. Constructingknowledge on the pre-exciting knowledge of thelearner stimulates learner involvement in thelearning process, rather than the passive receptionof new information. Using the familiar becomesthe approach to the unfamiliar, including moreabstract ideas (again, see Piaget’s developmentalstages and Vigotsky’s ZPDs) (Stone, 1977). Con-ceptual development enables the learner to com-municate in wider society and to access a stillwider range of skills, such as those of contempor-ary science and technology. In sum, learner-centered pedagogy is based (as Piaget stresses) onnature (biology and psychology), but occurs in aspecific socio-cultural milieu (central to the Vig-otskian view) and enables the individual tocommunicate between milieus.

4.4. Learner-centeredness in the Latin American

context: three challenges

4.4.1. Empowerment

Bridging cultural milieus, those of the commu-nity and the state, is a common purpose of the fiveprojects discussed above. It may be argued thatlearning of itself does this, with schooling the linkbetween private and public spheres (Parsons,1959), and that in socio-economically and cultu-

rally diverse settings such links may be especiallyimportant. At the same time, differing perspectiveson learner-centeredness may result in contradic-tory understandings of bridging. With populareducation in Latin America (Schmelkes, 1988, pp.44–72), the pragmatic view is of a gateway to newsocial participation. In Mexico, this view isespoused in the main by younger learners and isde facto learner-centered. It jostles uncomfortablywith the more likely radical thinking of olderpeople, project leaders, teachers and local autho-rities. Influenced by Paolo Freire and the Catholicliberation theology of the 1960s and 1970s, peopleof this older generation see education empoweringthe poor to transcend the culture of silence andaffirm their cultural identity to the wider world(Guzman and Martin, 1996, 1997). Combined withlearner-centeredness, this view may justify thecontent of programs being scaled down to whatis locally relevant, so as to inhibit migration to thecities and contamination with urban life styles(One Country, 1996). The model is of a draw-bridge being raised to keep learners from the widerworld, but students today may not be inspired byvisions of learning for cultural affirmation inidealized indigenous communities. Their desire isfor freedom to choose where to live their lives andhow (Friedlander, 1975; Steele, 2002).

4.4.2. Corruption of learner-centered evaluation

If learner-centeredness risks reducing learning topre-set notions of empowerment and knowledgerelevant to existing learner circumstances, theopposite danger lies in importing SAT-typeevaluation into avowedly learner-centered pro-grams (for the Mexican case, see Tatto, 1999).Performance-based evaluation may sidestep thedangers of achievement-based evaluation, as mea-sured by SATs, by focusing on process instead ofresults. Yet measures of performance can involvemuch second guessing and have the same negativebackwash on learning as achievement-based eva-luation. Indeed, the effect may be more sinister,since performance involves more of the self thananswering a set of multiple-choice questions. Thelearner thus not only learns how to second guess aset of questions but how to assume an appropriatedemeanor when doing so (Muller, 1988). Once

ARTICLE IN PRESS

C. Martin, E. Guzman / International Journal of Educational Development 25 (2005) 145–155152

again this danger arises in community educationfor the poor. Where community learning projectscontain a strong sense of mission, students maywell conform to project ideology, submerging theirindividual aspirations and experiences under ex-pected modes of performance.

4.4.3. Demands on teachers

Such challenges to learner-centered educationare compounded by the fact that constructivistapproaches demand more from teachers thanteacher-centered pedagogy. The new role requiresthe stimulation of active responses from learners,cultural perspectives different from those assumedin mainstream curricula and the developmentof appropriate approaches to facilitate under-standing of new material (Windschitl, 2002). Thepreferred performance-based evaluation for lear-ner-centeredness assesses not just what the studentknows, but the ability to use that knowledge. Thisrequires of the teacher a profound knowledge ofprinciples and practice and of interpreting studentresponses, more than is the case with competency-based assessment. In Mexico, teachers are notprepared for this type of work, even those fullyqualified. Unqualified teachers are even lessprepared.

5. A way forward

Running through the innovations discussed hereis the search for education sensitive to culturaldiversity and offering universal knowledge to aninternational standard. The challenge is how to dothis in a way that frees the individual from thecultural assimilationalism of formal educationalsystems and from the overreaction of populistcultural parochialism, tying individual destiny toparticular cultural and geographical space. Indivi-dual freedom and capacity to chose, constructingbridges between the local and the national,between origins and futures, is perhaps the mostdesirable educational endowment. Two examplesfrom the projects presented demonstrate how thelearner-centered bridge building can be accom-plished, the first in an innovative curriculum andthe second in pedagogical practice.

The ethno mathematics curriculum project atthe National Pedagogical University of Oaxaca(Aldaz, 1994) has studied Mayan mathematics, inorder to systematize its complexity, as the basis ofteacher and learner guides for schools. The projectaims to promote learning of Mayan mathematicsin Mayan schools, in part, since students hadfound the discipline especially difficult whentaught according to modern Indo-Arabian numbersystems. The plan would allow Mayan students toexcel by learning a culturally familiar mathema-tical system. The priority innovation was not therescue of an ancient system of measurement of aMesoamerican cultural minority. It was the con-nection of one of the world’s most ancientmathematical systems to current users by placingit in the comparative context of other worldmathematical systems. The course covers a widerange of mathematical fundamentals, comparingthe Mayan binary system of counting in relation tomodern computational systems (also binary) andto the dominant Indo-Arabian–European decimalsystem. In this way, student grasp of the disciplineis enriched, facilitating higher levels of conceptualdevelopment than occur when only one system islearnt.

The pedagogical example is exemplified inZoogocho’s music education program. Musiceducation is integrated into the mainstreamthrough the system of compulsory subject areaoptions, outside the core subjects of mathematics,languages and social studies. Options are accordedthe same importance as core subjects. This isparticularly true of music education at Zoogocho,the most popular of them all. The teaching-learning process in music is not explicitly con-structivist, but is learner-centered. The coursebegins with the study of basic theories of theory.This is mainly teacher-centered, in that studentsare required to learn the principles of rhythm,melody and reading music, but even at this stage,students are invited to reproduce sounds andcreate simple tunes. Mastering basic theory,students select an instrument and begin to playmusic by local and international composers, underthe tutelage of the teacher. Theories of westernmusic are brought to bear on playing prized pieces.Local ceremonies link the two musical traditions in

ARTICLE IN PRESS

C. Martin, E. Guzman / International Journal of Educational Development 25 (2005) 145–155 153

the moment of performance. Students are judged(in what is certainly a performance evaluation)according to their technical mastery of the instru-ment (requiring many hours of practice, mostlyout of school hours), the expressiveness of theirinterpretation, especially in solos and their com-mitment to the festivals, expressed in punctuality,tidiness and their general disposition towards theevent. This may involve untimely hours in the caseof funerals and where 2 and 3 days festivals arecommon, where traditional skills (cognitive, affec-tive and psychomotor) are brought into play.Students are assessed on all these and severeshortcomings in any one could lead to their beingdismissed from the option. The evaluation issusceptible to input (materials, spaces, teachers,sheet music, repairs, etc); processes (the waylearning is conducted at each stage and perhapstoo, how it may relate to other disciplines); andoutputs in music and other subjects, as justsuggested.

6. Conclusion: integrating education and evaluation

The music example illustrates Vigotskian stresson the socio-cultural dimension of learning. It alsopulls together most of the points of this article.Apart from involving cognitive and psychomotorskills, musical performance requires learning tocollaborate and coordinate playing with otherplayers. It establishes a relationship with listenersand other performers, in this case, other partici-pants in a festival, festival stewards (holders ofcargos), singers and dancers. These socio-culturalabilities in turn depend on the spirit or ethos of theschool and on its being respected and cherished aspart of the community, which it was in Zoogocho.Conditions for good interaction are crucial forlearning and for proceeding to the ZPD. Theyrequire an open, democratic spirit in school andpreferably around it as well. However, thesepreconditions and processes are rarely consideredand even more rarely measured in school evalua-tions. This may explain why students and teachersdo not take them seriously. They fall outside theSAT, which consider such things extraneous to itsnarrow perspective on learning.

Transversals are important for cognitive learn-ing, but also for propitiating the integrateddevelopment of the learner. They echo Crick’sargument for civic education being taught in dailyschool practice, rather than as a discipline, whereit becomes a cool, cognitive object of study andexamination. Transversals underpin the schoolingprocess: how teachers and learners approachteaching and learning. Understanding process iscrucial for explaining school success. To learnfrom evaluation, we need processes to be identi-fied, every bit as much as the outcomes theyproduce from the inputs they employ. Protagonistsof SATs will argue that this is beyond their remit,and that their responsibility to learners is attainedthrough the endless refinements undertaken tocorrect for statistical and cultural bias. Given thehigh stakes they imply for students, this is notgood enough. Such a stance represents an admin-istrative, bureaucratic and doctrinaire view ofeducation. It refuses to see the whole picture. Incontrast, the story revealed through grass rootseducational innovations and cognitive psychologi-cal research tells us that learning is a socialexperience. It involves many actors, not juststudents and teachers, but also the governmentsresponsible for funding, communities responsibleschools and parents supporting their children’slearning. Unless this is taken seriously, we need todo away with evaluation as not enhancing learn-ing, as not worth the effort it takes.

References

Aldaz, I., 1994. Cultura y Educacion Matematica, Revista

Guchaci’Reza (Iguana Rajada), Octubre, pp. 10–16.

Arendt, H., 1993. Between Past and Future, Penguin USA.

Baker, E.L., Robert, L.L., 1997. Emerging educational

standards of performance in the united state, CSE Technical

Report 437, National Center for Research on Evaluation,

University of California, Los Angeles.

Caballero, J.J., 2002, Educacion y cultura. Formacion comu-

nitaria en Tlazoyaltepec y Huitepec, Oaxaca, (Mexico D.F.,

Centro de Investigaciones y Estudios en Antropologıa

Social (CIESAS).

Cole, M., Wertsch, J.V., 1996. Beyond the Individual-Social

Antimony in Discussions of Piaget and Vygotsky, http://

www.massey.ac.nz/�alock/virtual/colevyg.htm

Corona B.S., 2002. Evaluacion externa de Tatu’utsi Maxakwaxi.

Evaluacion 2001–2002, External, unpublished evaluation for

ARTICLE IN PRESS

C. Martin, E. Guzman / International Journal of Educational Development 25 (2005) 145–155154

the Ford Foundation in Mexico City, sponsored by the

University of Guadalajara, Mexico, p. 32.

Consejo Nacional de Poblacion, (CONAPO) 2000, Indices

de Migracion, www.conapo.gob.mx/publicacions/indices/

pdfs/001.

Crick, B., 2000. Essays on Citizenship. Continuum, London.

De Andraca, A.M., 2003. Buenas Practicas para mejora la

educacion en America Latina, 2003 PREAL. Santiago,

Chile.

Ferreiro, E., 1997. Alfabetizacion. Teoria y Practica. Siglo XXI,

Mexico City.

Freedle, R.O., 2003. Correcting the SAT’s ethnic and social-

class bias: a method for reestimating SAT scores. Harvard

Educational Review 73 (1), 1–43.

Friedlander, J., 1975. Being Indian in Hueyapan: A Study of

Forced Identity in Contemporary Mexico, Bedford/St.

Martins.

Gramsci, A., 1976. Selections from the Prison Notebooks.

Lawrence and Wishart, London.

Guzman, E., Martin, C.J., 1996. Pump-priming post-16 school-

ing: city challenge funds in a secondary school consortium.

In: Turner, D. (Ed.), Education Beyond the State, Proceed-

ings of the 29th Annual Conference of the British

Comparative and International Education Society, London,

9–11 September, 1994, pp. 109–126.

Guzman, E., Martin, C.J., 1997. Back to basic Mexican style:

radical catholicism and survival on the margins. Bulletin of

Latin American Research 16 (3), 351–366.

Holt, J., 1972. The Under-Achieving School. First Laurel

Printing, New York.

Lira, S., 2003. En Mexico, enormes brechas en ındices de

desarrollo humano, La Jornada, 17.02.03, www.jornada.u-

nam.mx/2003/feb03/.

Lopez, L.E., 1998. La Eficacia y validez de lo obvio: lecciones

aprendidas desde la evaluacion de procesos educativos

biligues, Revista Iberoamericana de Educacion, Educacion,

Lenguas Culturas, No. 17, pp. 51–89.

MacDonald, J., 2001. The professional competences for teacher

education in Scotland and their impact on classroom

observation, independent, Evaluation for the Ford Founda-

tion, Mexico, of the New Television Secondary Schools of

CESDER (social development), unpublished.

Martin, C.J., Solorzano, C., 2003. Mass education, Privatisa-

tion, compensation and divesification: issues on the

future of public education in Mexico. Compare 33 (1),

15–30.

McEwan, P., 1999. Evaluating rural education reform. The

Case of Colombia’s Escuela Nueva, La Educacion, http://

www.iacd.oas.org/LaEduca132/mcewan/mcewan132-

134.htm, accessed through PREAL grade site: http://

www.grade.org.pe/gtee-preal/gteepreal.htm

Muller, J., 1998. The well-tempered learner: self-regulation,

pedagogical models and teacher education policy. Com-

parative Education 34 (2), 177–193.

Noam, G., Smith, R.B., 1996. Assessment at a crossroad:

Conversations. Harvard Educational Review 66 (3),

631–649.

Observatorio Ciudadano de la Educacion, 2002. Comunicados,

vol. II, pp. 31–60.

One Country, 1996. Rural learning helps stem urban migration

One country, 7(4) p.1 http://www.onecountry.org/

fundaec.html.

Ornelas, C., 2001. Equidad: Educacion comunitaria y progra-

mas compensatorios, in Carlos Ornelas, 2001, Investigacion

y polıticas educativas: Ensayos en honor de Pablo Latapı,

Mexico Aula XXI, Santillana, pp. 391–423.

Parsons, T., 1959. The school class as a social system. Harvard

Educational Review 29, 297–313.

Presidencia de la Republica, 2001. Plan Nacional de Desarrollo

2001–2006, Mexico City.

Schmelkes, S., 1988. El Centro de Estudios Educativos y la

educacion en el medio rural. Revista Latinoamericana de

Estudios Educativos, XVIII( 3–4), 35–80.

Smagorinsky, P., (undated), The Social Construction of Data:

Methodological Problems of Investigating Learning in the

Zone of Proximal Development http://psych.hanover.edu/

vygotsky/smagor.html, consulted 18.08 03.

Steele, S., 2002. The age of white guilt and the disappearance of

the black individual. Harper’s Magazine 305 (1830), 33–42.

Stone, E., 1977. An Introduction to Educational Psychology.

Methuen & Co, London.

Tatto, M.T., 1999. Improving teacher education in rural

Mexico: the challenges and tensions of constructivist

reform. Teaching and Teacher Education 15 (1), 15–35.

UNESCO, 2003. Posprimaria comunitaria rural http://innove-

mos.unesco.cl/medios/Documentos/Innovaciones/eyt/Pos-

primariacomunitariarural

Windschitl, M., 2002. Framing constructivism in practice as the

negotiation of dilemmas: An analysis of the conceptual,

pedagogical, cultural and political challenges facing tea-

chers. Review of Educational Research 72 (2), 131–175.

Zorrilla, M.M., 2003a. For innovation in basic-education pre-

service and in-service teacher training, to improve the

quality of education in Chiapas, Mexico: the work of the

‘‘Casa de la Ciencia’’ in Chipas, External Unpublished

Evaluation for the Ford Foundation, Mexico. pp. 1–20.

Zorrilla, F., 2003b. Margarita. Evaluacion de la Educacion

Basica en Mexico 1990–2000, Universidad Autonoma de

Aguascalientes, (Autonomous Uiniversity of Aguasca-

lientes, Mexico.

Further reading

Baker, S.W., 2001. Vygotskian perspectives on literacy

research, constructing meaning through collaborative in-

quiry/inside city schools: investigating literacy in multi-

cultural classrooms, Harvard Educational Review, 71(2),

296–309

Bertely, M., 1998. Educacion indıgena del siglo XX en Mexico.

In: Latapi, P. (Ed.), Un Siglo de Educacion en Mexico, vol.

II. Fondo de Cultura Economica, Mexico City, pp. 74–101.

Brennan, R.T., Jimmy, K., Melodie, W., Gary, N.S., 2001. the

relative equitability of high-stakes testing versus teacher-

ARTICLE IN PRESS

C. Martin, E. Guzman / International Journal of Educational Development 25 (2005) 145–155 155

assigned grades: an analysis of the massachusetts compre-

hensive assessment system (MCAS). Harvard Educational

Review 71 (2), 173–216.

Camara, G., 2002. Learning for Life in Mexican Rural

Communities. The Conafe Post-primary centers, Mexico

City, Conafe.

Cowen, M.P., Shenton, R.W., 1996. Doctrines of Development.

Routledge, London.

Desmi, A.C., 2001. Si Uno Come, Que Coman Todos,

Economia Solidaria. San Cristobal, Desmi, A.C., Mexico.

Ehrenfeld, D., 2002. Unethical Contexts for Ethical Question.

Harper’s Magazine 305 (1829), 13–20.

Falk, B., 2002. Standards-based reforms: problems and

possibilities. Phi Delta Kappan 83 (8), 612–621.

Freire, P., 1970. Pedagogy of the Oppressed. Penguin,

Harmondsworth.

Fuller, B., 1991. Growing-Up Modern. The Western State

Builds Third-World Schools. Routledge, London.

Hirsch, E.D., 2002. Classroom Research and Cargo Cult,

Policy Review, No. 115, www.policyreview.org/OCT02/

hirsch_print.htlm.

Larrauri, R., 2003. El Discurso Educativo de la Reforma

Curricular de la Educacion Normal: Su Impacto en el

Estado de Mexico, paper presented at Latin American

Studies Association in Dallas, Texas, USA, March.

pp. 1–25.

Levin, H.M., 2001. Pedagogical challenges for educational

futures in industrializing countries. Comparative Education

Review 45 (4), 537–561.

Martinez, R.F., 2002. Nueva visita al pais de la desigualdad. La

distribucion de la escolaridad en Mexico, 1970–2000.

Revista mexicana de investigacion educativa, septiembre-

diciembre 7 (16), 415–443.

Munoz, H., 2002. La Diversidad en las Reformas Educativas

Interculturales, Revista Electronica de Investigacion Edu-

cativa, noviembre. Universidad Autonoma de Baja Cali-

fornia 4 (2), 1–22.

O’Day, J., 2002, Complexity, accountability, and school

improvement, Harvard Educational Review

Observatorio Ciudadano de la Educacion, 2003a. Comunica-

dos, vol. III, pp. 61–90.

Observatorio Ciudadano de la Educacion, 2003b. Comunica-

dos, vol. I, pp. 1–30.

Olson, L., 2002. Education scholars finding new ‘value’ in

student test data. Education Week, November 20, 2002.

Ornelas, C., 1995. El sistema educativo mexicano. La transicion

de fin de siglo, Mexico City, CIDE, Nacional Financiera y

Fondo de Cultura Economica.

Reimers, F. (Ed.), 2000, Unequal Schools, Unequal Chances.

The Challenges to Equal Opportunity in the Americas.

Harvard University Press, Cambridge.

Secretarıa de Educacion Publica, 1992a. Acuerdo Nacional

para la Modernizacion de la Educacion Basica, Mexico pp.

32 This is a full, complete, official document and really does

not need page numbers.

Secretarıa de Educacion Publica (SEP), 1992b. Programa de

formacion y actualizacion de docentes Mexico City SEP.

Secretarıa de Educacion Publica (no date). Carrera Magisterial,

Mexico City, http://www.sep.gob.mx/wb2/sep/sep_617_car-

rera_magisterial

Secretarıa de Educacion Publica, 1999. Perfil de la Educacion

en Mexico. SEP, Mexico City.

Sennett, R., 1993. Authority. Norton, New York.

The Plowden Report: Children and their Primary Schools

(HMSO 1967).

Vazquez, M., 2001. La Voz y la Palabra del Pueblo Ayuujk.

Universidad Pedagogica Nacional and M.A. Porrua,

Mexico City.

Weiss, E., 2000. La situacion de la ensenanza multigrado en

Mexico. Perfiles Educativos 22, 57–76.