SCALING-UP REFORMS ACROSS A SCHOOLDISTRICT

Linda TaylorLos Angeles Unied School District

University of California Los A ngeles California USA

Perry Nelson and Howard S AdelmanUniversity of California Los A ngeles California USA

Each pendulum swing in the debate over how best to teach reading and writingcalls for large-scale systemic changes For the most part however the eld ofeducation has paid little attention to the full array of complexities involved inlarge-scale replication of curricular changes and other new directions forschool-based interventions Such neglect has contributed to the failure of manyreforms This article highlights a framework of general phases and specicsteps for di usion of major new approaches across a school district The over-lapping phases are seen as encompassing (a) creating readiness (b) initialimplementation (c) institutionalization and (d) ongoing evolution The dis-cussion includes lessons learned in applying the framework

E orts to reform schools require much more than implementingdemonstrations at a few sites Improved approaches are only as goodas a school districtrsquos ability to develop and institutionalize them on alarge scale This process often is called di usion replication roll outor scale-up

For the most part education researchers and reformers have paidlittle attention to the complexities of large-scale di usion This isevident from the fact that the nationrsquos research agenda does notinclude major initiatives to delineate and test models for widespreadreplication of education reforms (see Replication and Program Ser-vices Inc 1993 Schorr 1997 Slavin 1996) Furthermore leadershiptraining has given short shrift to the topic of scale-up Thus it is notsurprising that the pendulum swings that characterize shifts in thedebate over how best to teach reading are not accompanied with the

This article was prepared in conjunction with work done by the Center for MentalHealth in Schools at UCLA which is partially supported by funds from the USDepartment of Health and Human Services Public Health Services Health Resourcesand Services Administration Bureau of Maternal and Child Health Office of Adole-scent Health

Address correspondence to Howard Adelman Department of Psychology UCLABox 95163 Los Angeles CA 90095-1563 USA

Reading amp Writing Quarterly 15 303ndash325 1999Copyright 1999 Taylor amp FrancisOacute

1057-356999 $1200 1 00 303

304 L Taylor et al

resources necessary to accomplish prescribed changes throughout aschool district in an e ective manner Common deciencies includeinadequate strategies for creating motivational readiness among acritical mass of stakeholders especially principals and teachersassignment of change agents with relatively little specic training infacilitating large-scale systemic change and scheduling unrealisti-cally short time frames for building capacity to accomplish desiredinstitutional changes

For many years our work revolved mainly around developingdemonstration programs Major examples include the Early Assist-ance for Students and Families project (funded by the US Depart-ment of Education see Adelman amp Taylor 1993a) the restructuringof education support services in a large school district (see Adelman1996a 1996b Adelman amp Taylor 1997a) and the development of theUrban Learning Centersrsquo model for comprehensive school reform(supported by the New American Schools Development Corporation[NASDC] see Urban Learning Center Model 1995) Over the last fewyears we have moved into the world of replicating new approaches toschooling on a large-scale Confronted with the problems and pro-cesses of scale-up we analyzed a broad range of psychological andorganizational literature and delineated a working framework forscale-up (Adelman amp Taylor 1997b) The following presentation high-lights that framework and discusses some major lessons learned fromour recent e orts

OVERVIEW OF PHASES AND MAJOR TASKS OFSCALING-UP

In reading the following think about the best model around for howschools can improve student literacy Assuming the model is reason-ably cost-e ective and that a school district wants to adoptadapt itthe problem becomes one of how to replicate it at every school Forwidespread school change to occur a complex set of interventions isrequired For this to happen e ectively and efficiently the interven-tions must be guided by a sophisticated scale-up model that addressessubstantive organizational changes at multiple levels

A scale-up model is a tool for systemic change It addresses thequestion lsquolsquoHow do we get from here to therersquorsquo Such a model is guidedby a vision of organizational aims and is oriented toward results Weconceive scale-up as encompassing four overlapping phases (a) cre-ating readiness by enhancing a climateculture for change (b) initialimplementation whereby replication is carried out in stages using a

Scaling up Reforms 305

FIGURE 1 Scale-up phases and major tasks

well-designed guidance and support infrastructure (c) institutional-ization by ensuring there is an infrastructure to maintain andenhance productive changes and (d) ongoing evolution through useof mechanisms to improve quality and provide continuing support

To initiate and guide prototype replication a scale-up mechanismis needed One way to conceive such a mechanism is in terms of a

306 L Taylor et al

scale-up project Such a project provides a necessary organizationalbase and skilled personnel for disseminating a prototype negotiatingdecisions about replication and dispensing the expertise to facilitatescale-up A scale-up project can dispense expertise by sending out ascale-up team consisting of project sta who for designated periodsof time travel to replication sites A core team of perhaps two-to-fourproject sta works closely with a site throughout the replicationprocess The team is augmented whenever a specialist is needed toassist with a specic element such as new curricula use of advancedtechnology or restructuring of education support programs Scaling-up a comprehensive prototype almost always requires phased-inchange and the addition of temporary infrastructure mechanisms tofacilitate changes

Figure 1 briey highlights specic tasks related to the four phasesof scale-up (For more on each phase see Adelman and Taylor 1997b)Each task requires careful planning based on sound intervention fun-damentals (see Adelman amp Taylor 1994) This means paying specialattention to the problem of the match as discussed in the rst articlein this issue

PHASE I CREATING READINESSETH ENHANCING THECLIMATE FOR CHANGE

In most organizations mandated changes often lead to change inform rather than substance Substantive systemic change requirespatience and perseverance E orts to alter an organizationrsquos cultureevolve slowly in transaction with the specic organizational and pro-grammatic changes Early in the process the emphasis is on creatingan official and psychological climate for change including over-coming institutionalized resistance negative attitudes and barriersto change New attitudes new working relationships new skills allmust be engendered and negative reactions and dynamics must beaddressed

Creating readiness for reforms involves tasks designed to producefundamental changes in the culture that characterizes schools Sub-stantive reform is most likely when high levels of positive energyamong stakeholders can be mobilized and appropriately directed overextended periods of time Thus one of the rst concerns is how tomobilize and direct the energy of a critical mass of participants toensure readiness and commitment This calls for proceeding in waysthat establish and maintain an e ective match with the motivationand capabilities of involved parties In this respect a review of the

Scaling up Reforms 307

literature claries the value of (a) a high level of policy and lead-ership commitment that is translated into an inspiring vision andappropriate resources (leadership space budget time) (b) incentivesfor change such as intrinsically valued outcomes expectations forsuccess recognitions rewards (c) procedural options that reectstakeholder strengths and from which those expected to implementchange can select strategies they see as workable (d) a willingness toestablish an infrastructure and processes that facilitate changee orts such as a governance mechanism that adopts strategies forimproving organizational health including one that enhances a senseof community (e) use of change agents who are perceived as pragma-tic by maintaining ideals while embracing practical solutions (f)accomplishing change in stages and with realistic timelines (g) pro-viding feedback on progress and (h) institutionalizing supportmechanisms to maintain and evolve changes and to generate periodicrenewal There is an extensive literature in this area (eg Argyris1993 Barth 1990 Bass 1997 Bass amp Avolio 1994 Connor amp Lake1988 Cunningham amp Gresso 1993 Donahoe 1993 Elmore amp Associ-ates 1990 Fullan amp Stiegelbauer 1991 Hatch 1998 Heller 1990Hollander amp O ermann 1990 House 1996 Lewis 1989 Lieberman ampMiller 1990 Maton amp Salem 1995 Miles amp Louis 1990 Murphy1991 Newmann 1993 Peterson McCarthey amp Elmore 1996 Repli-cation and Program Services Inc 1993 Sarason 1990 1996 Schle-chty 1990 Schmuck amp Runkel 1985 Smith amp OrsquoDay 1991 Spillane1998 Waterman 1987 Wehlage Smith amp Lipman 1992)

In terms of specic tasks associated with creating readiness therst involves disseminating the prototype and pursuing activities tobuild interest and consensus for change Decisions follow about spe-cic sites for replication Then steps are taken to negotiate a policyframework and agreements for engagement This is followed by activ-ity to modify the institutional infrastructure at chosen sites to t theprototype and address replication needs All these tasks should beaccomplished with a process that reects understanding of the natureof the organization and its stakeholders involves stakeholders inmaking substantive decisions and redesigning those mechanisms thatconstitute the organizational and programmatic infrastructure clari-es personal relevance when identifying the potential benets ofchange elicits genuine public statements of commitment and empo-wers and creates a sense of community

Creating a climate for change requires appreciation of the rolesplayed by vision and leadership for change policy direction supportsafeguards for risk-taking and infrastructure redesign Each of thesetopics is discussed briey below

308 L Taylor et al

Vision and Leadership

Any major reform begins with a vision of what a desired newapproach would look like and an understanding of how to facilitatenecessary changes One without the other is insufficient Leadershave a triple burden as they attempt to improve literacy the rst isto ensure that substantive organizational and programmaticrestructuring are considered the second is to build consensus forchange nally they must pursue e ective implementation includingspecic strategies for nancing establishing maintaining andenhancing productive changes

Examples of key objectives at this stage include clarifying poten-tial gains without creating unrealistic expectations delineating costswithout seriously dampening expectations about benets o eringincentives that mesh with intrinsic motives and conveying thedegree to which a prototype can be adapted while emphasizing thatcertain facets are essential and nonnegotiable A thread runningthrough all this is the need to stimulate increasing interest or motiva-tional readiness among a sufficient number of stakeholders Toclarify the point successful change at any level of educationrestructuring requires the committed involvement of a critical massof policy makers sta and parents Almost any promising idea orpractice for improving studentsrsquo reading and writing performancemay nd a receptive audience among a small group Many more indi-viduals however are likely to remain politely unresponsive and rel-uctant to make changes and some will be actively resistant Thusleaders are confronted with the task of shifting the attitudes of asignicant proportion of those who appear reluctant and resistant

The next step involves deciding about which sites to begin withCriteria for making such decisions try to balance immediate concernsabout a sitersquos current level of readiness (including analyses of poten-tial barriers) and the likelihood of success over the long run Forinstance in making initial judgements about the appropriateness of apotential site we gather information about How likely is it that acritical mass of decision makers will commit to allocating sufficientnances personnel time and space How likely is it that a criticalmass of stakeholders will develop sufficient motivational readinessand appropriate levels of competence With respect to the most inu-ential stakeholders will enough be supportive or at least sufficientlycommitted not to undermine the process Do enough youngsters at asite t the prole of students for whom the program model wasdesigned As these questions illustrate most initial selection criteriareect general considerations related to any di usion process More

Scaling up Reforms 309

specic criteria emerge during the negotiation process For examplea principal may be attracted by the idea of establishing a programthat brings in volunteer reading tutors but in subsequent discussionswith teachers union concerns may arise that require arbitration

Policy

Substantive restructuring is unlikely without the adoption of newpolicies at all relevant jurisdictional levels (Spillane 1998) More-over such policies must elevate desired reforms so that they are notseen simply as demonstrations pilot projects passing fads or supple-mentary e orts When reforms are not assigned a high priority theytend to be treated in a marginalized manner (Center for MentalHealth in Schools 1998) This continues to be the fate of programssuch as Head Start Even Start and many other approaches toenhancing school readiness and literacy Relatedly e orts must bemade to revoke policies that preserve an unsatisfactory status quo(see critique of remedial reading programs by Dudley-Marling ampMurphy 1997)

Lasting reform requires processes that ensure informed com-mitment ownership and on-going support on the part of policymakers This involves strategies to create interest and formalizeagreements about fundamental changes Local ownership is estab-lished through solid policy commitments well-designed infrastruc-ture mechanisms allocation of adequate resources (eg nancespersonnel space equipment) to operationalize the policy andrestructuring of time to ensure sta involvement in adapting theprototype to the setting We nd three steps are essential (a) build-ing on introductory presentations to provide indepth information andunderstanding as a basis for establishing consensus (b) negotiationof a policy framework and a set of agreements for engagementincluding a realistic budget and (c) informed and voluntary rati-cation of agreements by legitimate representatives of all major stake-holders

For any program there are principles components elements andstandards that dene its essence and thus must be agreed to as a rstcondition for engagement Equally important are fundamental scale-up considerations o -line nonnegotiable such as the need for tempo-rary mechanisms to facilitate change Once essentials are agreed onall other matters are negotiable

Informed commitment is strengthened and operationalized throughnegotiating formal agreements at each jurisdictional level and amongvarious stakeholders Policy statements articulate the commitment to

310 L Taylor et al

a programrsquos essence Memoranda of understanding and contractsspecify agreements about such matters as funding sources resourceappropriations personnel functions incentives and safeguards forrisk-taking stakeholder development immediate and long-term com-mitments and timelines accountability procedures and so forth

Scale-up is aided when the decision to proceed is ratied by sanc-tioned representatives of stakeholder groups Developing and nego-tiating policies contracts and other formal agreements is a complexbusiness We nd that addressing the many logistics and legalitiesrequires extensive involvement of a small number of authorized andwell-informed stakeholder representatives Thus in pursuing thesetasks our commitment to include everyone moves from a town hallapproach to a representative democratic process with enfranchisedrepresentatives reporting back frequently to their constituencies Atrst endorsement is in principle over time it is manifested throughsustained support When ratication reects e ective consensusbuilding scale-up e orts benet from a broad base of informed com-mitment ownership and active sponsorship These attributes areessential in ensuring requisite support and protections for those whomust bear the burden of learning new ways and who risk dips in per-formance and productivity while doing so

Redesigning Infrastructure

After agreements are ratied a scale-up team can begin its work(again see Figure 1) A central challenge at every jurisdictional levelis redesign of regular mechanisms and processes used to make andimplement decisions These modications ensure ownership supportparticipation and address specic concerns associated with scale-up

Five fundamental facets of the ongoing infrastructure of schoolsthat are the focus of redesign are (a) governance (b) planning andimplementation associated with specic organizational and programobjectives (c) coordination and integration to ensure cohesive func-tioning (d) daily leadership and (e) communication and informationmanagement A common example of the need for infrastructure modi-cation is seen in the trend to increase school stakeholdersrsquo collabo-ration participation and inuence One implication is thatgovernance mechanisms will be altered to redistribute power Amajor problem of course is how to empower additional stakeholdergroups without disempowering those who have essentialresponsibilities and abilities related to the educational enterprise Inaddition it is one thing to o er lsquolsquopartnershipsrsquorsquo to stakeholders suchas parents students sta and community agency representatives it

Scaling up Reforms 311

is another to create conditions that allow for e ective participationOne such condition involves translating capacity building activityinto comprehensive programs for stakeholder development

The necessity of all this can be appreciated by thinking aboutintroducing a comprehensive approach for improving student literacy(Stringeld Ross amp Smith 1996) Such approaches involve major sys-temic changes that encompass intensive partnerships with parents (ortheir surrogates) and with various entities in the community such aslibraries youth development programs businesses the faith com-munity and so forth Substantive partnerships require a true sharingof leadership blending of resources and leadership training for pro-fessionals and nonprofessionals alike In communities where manyparents have little or no connection to the school major outreache orts are inevitable prerequisites to increasing home involvement inschool reform Parent outreach of course has not been very suc-cessful in many neighborhoods Our experience suggests that a neces-sary rst step in most cases is to o er programs and services thatassist the family in meeting its most pressing needs Furthermorethere is the matter of building parent competence to deal with plan-ning reforms and restructuring schools and for low income familiesthere is a need to nd ways to pay parents for the time they devote toserving on governance and other committees

Time is one of the most critical elements determining the successof scale-up Even if a prototype doesnrsquot call for restructuring theschool day the scale-up process does Substantial blocks of time areneeded for stakeholder capacity building and for individual and col-lective planning (National Education Commission on Time andLearning 1994) Particularly critical is the need for freeing-up tea-chers to learn new approaches For example e orts to make impor-tant revisions in literacy programs seem consistently undermined bynot providing enough time during the school day for the mentoring ofteachers and by the difficulty of carving out sufficient time to teachparents how to help their children Clearly a nonnegotiable condi-tion for engagement is a realistic plan for ensuring time to plan andbuild capacity

Lessons Learned

Complex interventions of course are seldom implemented in a com-pletely planned and linear manner The many practical and unfore-seen events that arise require exible problem-solving Articulationof a scale-up model can guide planning but those facilitating the

312 L Taylor et al

process must be prepared to capitalize on every opportunity that canmove the process ahead

Among the most fundamental lessons learned in carrying outPhase 1 has been the tendency of all parties to set actions into motionwithout taking sufficient time to lay the foundation needed for sub-stantive change In marketing new ideas it is tempting to accentuatetheir promising attributes and minimize complications In negotiatingagreements policy makers at a school site frequently are askedsimply for a go-ahead rather than for their informed commitmentSometimes they assent mainly to get extra resources sometimes theyare motivated by a desire to be seen by constituents as doing some-thing to improve the school This all tends to produce pressures forpremature implementation that results in the form rather than thesubstance of change especially when administrators are under thegun of political accountability measures that make unrealisticdemands for quick and dramatic results in studentsrsquo reading scores

Although formulation of policy and related agreements take con-siderable time and other resources their importance cannot be over-emphasized Failure to establish and successfully maintainsubstantive reforms in schools probably is attributable in greatmeasure to proceeding without strong and clear policy support

Another unfortunate trend we have found is the omission of in-depth planning for ongoing capacity building for change agents andteam members Mechanisms function only as well as the personnelwho operate them Such personnel must be recruited and developed inways that ensure appropriate motivation and capability and suffi-cient time must be redeployed so they can learn and carry out newfunctions e ectively (Peterson McCarthey amp Elmore 1996) Allchanges require constant care and feeding Those who steer theprocess must be motivated and competent not just initially but overtime The complexity of systemic change requires close monitoring ofmechanisms and immediate follow-up to address problems In particu-lar it means providing continuous personalized guidance andsupport to enhance knowledge and skills and counter anxiety frus-tration and other stressors To these ends adequate resource supportmust be provided (time space materials equipment) opportunitiesmust be available for increasing ability and generating a sense ofrenewed mission and personnel turnover must be addressed quicklyAll stakeholders can benet from e orts designed to increase levelsof competence and enhance motivation for working together Suche orts encompass four stages of stakeholder development orienta-tion foundation-building capacity-building and continuing educa-tion

Scaling up Reforms 313

There is no simple solution to the chronic problem of providingtime for creating readiness building capacity and planning Indeedrestructuring time represents one of the most difficult scale-up prob-lems Examples of how the problem might be addressed includefreeing up sta by establishing opportunities for students to spendtime pursuing activities such as music art and sports with special-ists or supervised by aides and community volunteers Alternativelyschool might start later or end earlier on a given day As these exam-ples suggest any approach will be controversial but if the problem isnot addressed satisfactorily successful replication of comprehensiveprototypes is unlikely

PHASE II INITIAL IMPLEMENTATION OF APROTOTYPE

Initial implementation involves adapting and phasing-in a programwith well-designed guidance and support If there is anything certainabout e orts to replicate a prototype it is that the process is stress-ful Some of the stress arises from the nature of the program some isinherent in the process of organizational change Coalitions must bedeveloped new working relationships established disruptive rumorsand information overload countered and interpersonal conictsresolved Short-term frustrations must be kept in perspective vis a visthe reform vision To help deal with all this temporary mechanismsare added to the organizational infrastructure They include (a) asite-based steering mechanism to guide and support replication (b) achange agent from the scale-up team working with site stakeholderson a change team to facilitate coalition building problem solvingand conict resolution and (c) mentors and coaches to model andteach elements of the prototype These structures are created tofacilitate replication and some are assimilated into a sitersquos infra-structure at the end of the initial implementation phase to supportinstitutionalization and ongoing evolution

A scale-up team and steering group work at a site with the schoolrsquosleadership specic planning groups and other stakeholders to formu-late phase-in plans steer program development and generallyprovide guidance and support for change Two major facets of thiswork are delineating a sequence for introducing major program ele-ments and outlining strategies to facilitate implementation Particu-lar attention is given to how to start with special emphasis onspecifying structures and resources for guidance and support For

314 L Taylor et al

instance in restructuring to better address barriers to learning therst steps at a school site involve creating processes to map analyzecoordinate and redeploy existing resources Special change mecha-nisms such as an organization facilitator and a resource coordinatingteam are created to guide and support the activity (Adelman 19931996a 1996b Adelman amp Taylor 1997a 1997b 1998)

Throughout this phase formative evaluation procedures are estab-lished to provide feedback for program development As noted abovee ective e orts to lsquolsquoreinventrsquorsquo schools require ensuring that allinvolved have the time to develop and institutionalize a soundprogram and that they are not penalized for unavoidable missteps Asa prototype is phased-in evaluation must not be thought of in termsof accountability Major systemic changes to improve literacy cantake years to develop Outcome e ectiveness is demonstrated afterthe program is in place The purpose of evaluation at this stage is toguide revision and ne-tuning of processes Formative evaluationsgather and analyze information relevant to changes in planning pro-cesses governance structures and policies and resources they alsofocus on implementation strategies and barriers program organiz-ation and staffing and initial outcomes Are teachers mastering thenew curricula for literacy Is there increased student engagement inreading Are parents involved in supporting reading at home Ifthings are not progressing satisfactorily why not Whatrsquos the down-side of the new approach

Well-designed organizational support and guidance is needed toenhance productivity minimize problems and accommodate individ-ual di erences This involves various forms of capacity building andpersonalized day-by-day facilitation Intensive coaching with somefollow-up consultation for instance are key processes so are mentor-ship and technical assistance Continuing education provides a criti-cal vehicle for enhancing productive changes generating renewaland countering burnout As new stakeholders arrive technologicaltools can be particularly useful in helping them catch up All thisactivity not only builds capacity but can foster networking and otherforms of task-related social and personal support as well as providea wide range of enrichment opportunities that enhance morale

If the steps discussed to this point are done well a sound founda-tion for initial implementation should be in place This initial phase-in period can however consume considerable e ort create specialproblems and may yield a temporary drop in some performance indi-cators Good day-by-day facilitation aims at minimizing such negativeimpact by e ectively addressing stakeholder motivation and capabil-ity and overcoming barriers to productive working relationships

Scaling up Reforms 315

Lessons Learned

Failure to take sufficient time to create readiness (Phase 1) can resultin implementing the form rather than the substance of a prototypeFor example we nd that change agents frequently are sent intoschools before essential policy support is enacted and before schoolleaders have assimilated and decided to support reforms Teams areconvened to assist with reforms (plan coordinate develop newapproaches) but the absence of supportive policy means substantivechanges are not accomplished As a result the initial motivation ofmany key team members wanes and other counterproductivedynamics arise All of this seems inevitable when initial implementa-tion proceeds without adequate policy support

Even in situations where sufficient readiness is created difficultiesfrequently arise due to a failure to keep enough stakeholders consis-tently moving in the direction of desired outcomes Comprehensivechange is usually achieved only when fairly high levels of positiveenergy can be mobilized over extended periods of time among a criti-cal mass of stakeholders sustained energy is appropriately directedthe process is supported with ongoing and well-conceived capacitybuilding and individuals are not pushed beyond their capabilitiesAnd because low and negative motivation are related to resistance tochange and poor functioning matching motivation is a rst-orderconsideration That is scale-up e orts must use strategies designed tomobilize and maintain proactive e ort and overcome barriers toworking relationships As in personalizing instruction approx-imating a good motivational t also requires matching capabilitiessuch as starting with fewer elements at sites at which resources arelimited and accounting for variability in stakeholdersrsquo competenceOver and over we nd too little attention is paid to these mattersThe result is failure to create an lsquolsquoenvironmentrsquorsquo that mobilizesdirects and then maintains stakeholder involvement

As with students the problem can be conceived as that of main-taining an appropriate match between the demands of the situationand individual motivation and capabilities In this respect we thinkthe construct of personalization o ers a concept around which toorganize thinking about facilitating change As stressed in the rstarticle in this issue personalization calls for systematically planningand implementing processes focused not only on knowledge andskills but on attitudes In particular it emphasizes the importance ofa primary and constant focus on ensuring positive attitudes Mobi-lization probably is best facilitated when procedures are perceived byindividuals as good ways to reach desired outcomes This requires

316 L Taylor et al

processes that can instigate and enhance an individualrsquos perceptionsof valued opportunities choice and control accomplishment andrelatedness to others Even if a task isnrsquot enjoyable expectation offeeling some sense of satisfaction related to process or outcome canbe a powerful intrinsic factor motivating individual behavior Taskpersistence for example can be facilitated by the expectation thatone will feel competent self-determining or more closely connectedto others From this perspective ensuring that individuals havevalued options a meaningful role in decision making feedback thatemphasizes progress toward desired outcomes and positive workingrelationships are among the most basic facilitation strategies(Adelman amp Taylor 1993b 1994 Deci amp Ryan 1985)

One other initial implementation problem that often arises is diffi-culty in establishing mechanisms to facilitate productive workingrelationships and identify and deal with problems quickly Forexample it is expected that change agents will encounter manyinstances of individual resistance and apathy interpersonal conictsand resentments (including lsquolsquous vs themrsquorsquo dynamics) rumors thatoveremphasize the negative and underestimate the positive and indi-viduals who are frequent faultnders Such problems seriouslyimpede e ective replication The roots of some of these problemsoften are present at a site prior to scale-up change simply o ers anew focus and perhaps magnies troubling matters Other problemsare a direct product of the activities and relationships that the scale-up process engenders Given the inevitability of such problems build-ing and maintaining working relationships need to be among themost basic concerns for those who have responsibility for scale-up Inparticular considerable attention must be paid to enhancing themotivational readiness and capability of those who are to worktogether and ensure there is an appropriate infrastructure to guideand support working relationships Proactively this requires problemprevention mechanisms that help create an atmosphere where defen-siveness is curtailed and positive rapport is engendered The point isto enhance attitudes knowledge and skills that foster interpersonalconnections and a sense of community Reactively the emphasis is onproblem solving resolving conict and providing ongoing support torebuild relationships Policies must encourage problem-solving-oriented critiques safeguards that protect those making changesappreciation for e ort and celebration of progress We nd thateveryone understands such matters but the culture at many schoolsites is more attuned to problem naming and analyzing than to antici-pating preventing and solving problems that arise around workingrelationships

Scaling up Reforms 317

Those responsible for systemic change need to spend as much timeas necessary ensuring that a schoolrsquos infrastructure is ready toprevent and ameliorate problems Special attention must be paid toensuring that problem solving mechanisms and communication pro-cesses are in place and properly sta ed and that stakeholders arewell informed about how to use the procedures Furthermore somestakeholders may have to be encouraged to interact in ways thatconvey genuine empathy warmth and mutual regard and respectwith a view to creating and maintaining a positive working climateand a psychological sense of community

At times we nd it necessary to target a specic problem anddesignated persons In some instances rather simple strategies aree ective For example most motivated individuals can be directlytaught ways to improve understanding and communication and avoidor resolve conicts that interfere with working relationships Inother instances however signicant remedial action is necessary aswhen overcoming barriers to a working relationship involvescountering negative attitudes Helpful in this regard are analysessuch as that by Sue and Zane (1987) that suggest how to demonstratethat something of value can be gained from individuals workingtogether and how to establish each participantrsquos credibility (eg bymaximizing task-focus and positive outcomes)

PHASE III INSTITUTIONALIZING THE PROTOTYPE

Maintaining and enhancing changes can be as difficult as makingthem in the rst place The history of education reform is one offailure to foster promising prototypes in substantive ways and overan extended period of times (Tyack amp Cuban 1995) Institutionalizinga prototype entails ensuring that the organization assumes long-termownership and that there is a blueprint for countering forces that canerode the changes Moreover institutionalization is more than atechnical process It requires assimilation of and ongoing adherenceto the values inherent in the prototypersquos underlying rationale Thefocus of course is not just on maintenance the point is to moveforward by enhancing productive changes and generating a sense ofrenewal as needed Critical in all this are specic plans that guar-antee ongoing and enhanced leadership and delineate ways in whichplanning implementation coordination and continuing educationmechanisms are maintained

Some Major Tasks

Whose responsibility is it to advocate for maintaining and evolving a

318 L Taylor et al

replicated prototype for improving literacy As problems arise whoseresponsibility is it to lead the way in resolving them Leadership isthe key here by both official leaders such as administrators mentorsta union chapter chairs and elected parent representatives aswell as natural leaders such as reading and writing teachers Obvi-ously official and natural leaders are not mutually exclusive groupsAt this phase both types of leadership are essential to ensure a broadenough base for ongoing advocacy problem solving enhancementand renewal Official leaders provide a legitimate power base asvarious interests compete for the organizationrsquos limited resourcesand they play a key role in ensuring that the contributions of naturalleaders are recognized and rewarded

Maintenance and enhancement require that the organizationrsquosgovernance body assumes ownership and program advocacy such astaking over the temporary steering grouprsquos functions addressingongoing policy and long-range planning concerns and maintainingnancial support The foundation for such ownership is laid duringthe readiness phase Each element becomes the organizationrsquos pro-perty as it is established during initial implementation The officialdeed of ownership is transferred as soon as the prototype is in place

Ownership however is no guarantee of institutionalizationVarious forces that can erode reforms are always at work Forinstance teams at a site experience turnover problems with commu-nication and sharing of resources are chronic competing interestsand the attractiveness of moving on to something new pull attentionand resources to other activity To minimize such problems stepsmust be taken to identify and solve them as quickly as is feasibleThis requires someone who has the time energy and expertise tomeet periodically with stakeholders to anticipate and amelioratethreats to a prototypersquos integrity

Over time mechanisms for planning implementation and coordi-nation are maintained by ensuring the activity is an official part ofthe infrastructure has appropriate leadership and is e ectively sup-ported Anyone who has worked on a school-based team to improve aliteracy program knows there must be a critical mass of teammembers so that the work load is manageable and a broad base ofinvolvement is ensured Also essential are adequate resources includ-ing time to learn the role and time to perform the functions reason-ably interesting tasks technical support for problem solvingrecognition and rewards for contributions immediate replacementwhen someone leaves continuing education to enhance team func-tioning and so forth Without serious attention to such matters theteamsrsquo morale and motivation will wane

Scaling up Reforms 319

Lessons Learned

Newly institutionalized approaches are seriously jeopardized in theabsence of dedicated ongoing capacity-building Of particular impor-tance are ways to rapidly and e ectively assimilate new arrivals at aschool (sta students families) This is a major concern at sites withconsiderable turnover or growth At such sites the majority of thoseinitially involved in implementing a new approach may be gonewithin a period of two to three years Whatever the mobility rate it isessential to design and maintain transition programs for newarrivals Initial welcoming and introductory orientations of coursemust be followed-up with ongoing support systems and intensivecapacity building related to understanding and valuing theapproaches the school has adopted We nd that all this is essentialnot only to maintain what has been adopted but also can contributeto establishing schools as caring environments

PHASE IV ONGOING EVOLUTION

Ongoing evolution of organizations and programs is the product ofe orts to account for accomplishments deal with changing times andconditions incorporate new knowledge and create a sense of renewalas the excitement of newness wears o and the demands of changesap energy As suggested already in part vigor and direction can bemaintained through continuing education especially exposure toideas that suggest a range of ways for evolving a program As thefollowing discussion indicates ongoing evolution also is fostered byevaluation designed to document accomplishments and provide feed-back designed to improve quality

Increased concern over accountability has advanced the wayevaluation is conceived (Posavac amp Carey 1989 Rossi amp Freeman1989 Scriven 1993 Sechrest amp Figueredo 1993 Shadish Jr Cook ampLeviton 1991 Stake 1967 1976 Stufflebeam amp Webster 1983 Weiss1995) At the same time social and political forces have literallyshaped the whole enterprise and in the process have narrowed theway professionals clients policymakers underwriters and thegeneral public think about program evaluation A prevailing cry isfor specic evidence of e ectiveness For schools this means imme-diate gains in literacy as measured by achievement tests Althoughunderstandable in light of the unlled promise of so many programsand the insatiable demands on limited public nances such simplisti-cally conceived accountability demands ignore the complexities ofdeveloping and scaling-up major reforms

320 L Taylor et al

Formative and Summative Evaluation

Evaluation of a prototype involves more than determining efficacyfor students Broadly stated it encompasses concerns about how toexpand the focus of evaluative research not only to contribute toimproving practice but also to aid in evolving practice and policy(General Accounting Office 1989 Lyon amp Moats 1997) To facilitateprogram development and organizational change the primary orien-tation for evaluation in the early phases is formative It is especiallyfocused on data gathering and analyses that can help improve pro-cedures Most of what is written about educational and psychosocialintervention however is oriented to summative evaluation and mea-suring outcomes for individuals such as improved reading achieve-ment scores Replicating approaches to improve literacy involve notonly changing individuals but changing organizations and systemsThus both individuals and systems must be evaluated

All this presumes appropriate mechanisms to provide and analyzeessential information To these ends a scale-up sta can help estab-lish an evaluation team and capacity building that prepares a schoolto conduct evaluation that enhances reforms The immediate focus ison successful program replication ultimately of course the empha-sis must be on student outcomes

Pursuing Results

Due to the increased interest in accountability many complex aimsare broken down into specic objectives Indeed short-range objec-tives stated in measurable terms generally assume a central role inplanning However short-range objectives are not ends in them-selves They are a small part of a particular goal and aim and some-times are prerequisites for moving on to a goal It is essential not tolose sight of the fact that many specic objectives are relativelysmall unrepresentative and often unimportant segments of the mostvalued aims that society has for its citizens and citizens have forthemselves

The problem is well exemplied by the narrow focus found inreviews analyses and reanalyses of data on early education (eg seeAlbee amp Gullotta 1997 Bond amp Compas 1989 Dryfoos 1990 Durlak1995 Elias 1997 Mitchell Seligson amp Marx 1989 Schorr 1988Slavin Karweit amp Madden 1989 Weissberg Gullotta HamptomRyan amp Adams 1997) As such work demonstrates overemphasis onevaluating the efficacy of underdeveloped prototypes draws resourcesaway from formative evaluation

Scaling up Reforms 321

With specic respect to scale-up the rst accomplishment is thereplication itself Have all facets been implemented How completelyhas each been implemented At how many locations The next set ofresults are any indications of progress for students such as improve-ments in attitudes toward school health attendance behavior andacademic achievement A nal set of evaluation concerns is thedegree to which student outcomes approximate societal standards

Lessons Learned

The process of evaluating results is costly in terms of nancial invest-ment the negative psychological impact on those evaluated and theways it can inappropriately reshape new approaches Cost-e ectiveoutcomes cannot be achieved in the absence of e ective prototypedevelopment and research Premature e orts to carry out com-prehensive summative evaluations clearly are not cost-e ective Anyreading and writing program will show poor results if it is evaluatedbefore teachers have mastered its application None of this of courseis an argument against evaluating results Rather it is meant tounderscore concerns and encourage greater attention to addressingthem

Once a prototype is established care must be taken to avoiddeveloping outcome evaluation as an adversarial process Due to thepolitical realities related to accountability one of the most per-plexing facets to negotiate is the time frame for summative evalu-ation The more complex the prototype the longer it takes and thecostlier it is to implement and evaluate Schools usually want quickprocesses and results and of course rarely can a ord costly innova-tions or lengthy di usion activity Compromises are inevitable butmust be arrived at with great care so as not to undermine the sub-stance of proposed changes

The psychology of evaluation suggests that an overemphasis onaccountability tends to produce negative reactions One possible wayto counter this may be to conceive evaluation as a way for everystakeholder to self-evaluate as a basis for quality improvement and asa way of getting credit for all that is accomplished Unfortunately asaccountability pressures increase we nd that replication of proto-types are guided more by what can be measured than by long-rangeaims That is demands for immediate accountability reshape prac-tices so that the emphasis shifts to immediate and readily measuredobjectives and away from fundamental purposes Over time this inap-propriately leads to radical revision of the underlying rationale for aprototype

322 L Taylor et al

CONCLUDING COMMENTS

Those who set out to change schools and schooling are confrontedwith two enormous tasks The rst is to develop prototypes Thesecond involves large-scale replication One without the other isinsufficient Yet considerably more attention is paid to developingand validating prototypes than to delineating and testing scale-upprocesses Clearly it is time to correct this deciency The ideas pre-sented in this article are meant to stimulate work on the problem andthereby to advance the cause of educational reform

Finally in fairness to those who labor for educational reform weall must remember that the quality of schooling family life and com-munity functioning spirals up or down as a function of the quality ofthe ongoing transactions among each Thus scale-up e orts relatedto educational reform must take place within the context of a poli-tical agenda that addresses ways to strengthen the family and com-munity infrastructure through strategies that enhance economicopportunity adult literacy and so forth What we need are policiesthat develop demonstrate and scale-up comprehensive multifacetedintegrated approaches that can e ectively address barriers to devel-opment learning and teaching

REFERENCES

Adelman H S (1993) School-linked mental health interventions Toward mechanismsfor service coordination and integration Journal of Community Psychology 21 309ndash319

Adelman H S (1996a) Restructuring education support services and integrating com-munity resources Beyond the full service school model School Psychology Review25 431ndash445

Adelman H S (1996b) Restructuring support services Toward a comprehensiveapproach Kent OH American School Health Association

Adelman H S amp Taylor L (1993a) Early Assistance for Students and FamiliesProgram Guidebook Los Angeles Early Assistance for Students and FamiliesProject Copies available from the School Mental Health Project Dept of Psychol-ogy UCLA

Adelman H S amp Taylor L (1993b) Learning problems and learning disabilities Moving forward Pacic Grove CA BrooksCole

Adelman H S amp Taylor L (1994) On understanding intervention in psychology andeducation Westport CT Praeger

Adelman H S amp Taylor L (1997a) Addressing barriers to learning Beyond school-linked services and full service schools American Journal of Orthopsychiatry 67408ndash421

Adelman H S amp Taylor L (1997b) Toward a scale-up model for replicating newapproaches to schooling Journal of Educational and Psychological Consultation 8197ndash230

304 L Taylor et al

resources necessary to accomplish prescribed changes throughout aschool district in an e ective manner Common deciencies includeinadequate strategies for creating motivational readiness among acritical mass of stakeholders especially principals and teachersassignment of change agents with relatively little specic training infacilitating large-scale systemic change and scheduling unrealisti-cally short time frames for building capacity to accomplish desiredinstitutional changes

For many years our work revolved mainly around developingdemonstration programs Major examples include the Early Assist-ance for Students and Families project (funded by the US Depart-ment of Education see Adelman amp Taylor 1993a) the restructuringof education support services in a large school district (see Adelman1996a 1996b Adelman amp Taylor 1997a) and the development of theUrban Learning Centersrsquo model for comprehensive school reform(supported by the New American Schools Development Corporation[NASDC] see Urban Learning Center Model 1995) Over the last fewyears we have moved into the world of replicating new approaches toschooling on a large-scale Confronted with the problems and pro-cesses of scale-up we analyzed a broad range of psychological andorganizational literature and delineated a working framework forscale-up (Adelman amp Taylor 1997b) The following presentation high-lights that framework and discusses some major lessons learned fromour recent e orts

OVERVIEW OF PHASES AND MAJOR TASKS OFSCALING-UP

In reading the following think about the best model around for howschools can improve student literacy Assuming the model is reason-ably cost-e ective and that a school district wants to adoptadapt itthe problem becomes one of how to replicate it at every school Forwidespread school change to occur a complex set of interventions isrequired For this to happen e ectively and efficiently the interven-tions must be guided by a sophisticated scale-up model that addressessubstantive organizational changes at multiple levels

A scale-up model is a tool for systemic change It addresses thequestion lsquolsquoHow do we get from here to therersquorsquo Such a model is guidedby a vision of organizational aims and is oriented toward results Weconceive scale-up as encompassing four overlapping phases (a) cre-ating readiness by enhancing a climateculture for change (b) initialimplementation whereby replication is carried out in stages using a

Scaling up Reforms 305

FIGURE 1 Scale-up phases and major tasks

well-designed guidance and support infrastructure (c) institutional-ization by ensuring there is an infrastructure to maintain andenhance productive changes and (d) ongoing evolution through useof mechanisms to improve quality and provide continuing support

To initiate and guide prototype replication a scale-up mechanismis needed One way to conceive such a mechanism is in terms of a

306 L Taylor et al

scale-up project Such a project provides a necessary organizationalbase and skilled personnel for disseminating a prototype negotiatingdecisions about replication and dispensing the expertise to facilitatescale-up A scale-up project can dispense expertise by sending out ascale-up team consisting of project sta who for designated periodsof time travel to replication sites A core team of perhaps two-to-fourproject sta works closely with a site throughout the replicationprocess The team is augmented whenever a specialist is needed toassist with a specic element such as new curricula use of advancedtechnology or restructuring of education support programs Scaling-up a comprehensive prototype almost always requires phased-inchange and the addition of temporary infrastructure mechanisms tofacilitate changes

Figure 1 briey highlights specic tasks related to the four phasesof scale-up (For more on each phase see Adelman and Taylor 1997b)Each task requires careful planning based on sound intervention fun-damentals (see Adelman amp Taylor 1994) This means paying specialattention to the problem of the match as discussed in the rst articlein this issue

PHASE I CREATING READINESSETH ENHANCING THECLIMATE FOR CHANGE

In most organizations mandated changes often lead to change inform rather than substance Substantive systemic change requirespatience and perseverance E orts to alter an organizationrsquos cultureevolve slowly in transaction with the specic organizational and pro-grammatic changes Early in the process the emphasis is on creatingan official and psychological climate for change including over-coming institutionalized resistance negative attitudes and barriersto change New attitudes new working relationships new skills allmust be engendered and negative reactions and dynamics must beaddressed

Creating readiness for reforms involves tasks designed to producefundamental changes in the culture that characterizes schools Sub-stantive reform is most likely when high levels of positive energyamong stakeholders can be mobilized and appropriately directed overextended periods of time Thus one of the rst concerns is how tomobilize and direct the energy of a critical mass of participants toensure readiness and commitment This calls for proceeding in waysthat establish and maintain an e ective match with the motivationand capabilities of involved parties In this respect a review of the

Scaling up Reforms 307

literature claries the value of (a) a high level of policy and lead-ership commitment that is translated into an inspiring vision andappropriate resources (leadership space budget time) (b) incentivesfor change such as intrinsically valued outcomes expectations forsuccess recognitions rewards (c) procedural options that reectstakeholder strengths and from which those expected to implementchange can select strategies they see as workable (d) a willingness toestablish an infrastructure and processes that facilitate changee orts such as a governance mechanism that adopts strategies forimproving organizational health including one that enhances a senseof community (e) use of change agents who are perceived as pragma-tic by maintaining ideals while embracing practical solutions (f)accomplishing change in stages and with realistic timelines (g) pro-viding feedback on progress and (h) institutionalizing supportmechanisms to maintain and evolve changes and to generate periodicrenewal There is an extensive literature in this area (eg Argyris1993 Barth 1990 Bass 1997 Bass amp Avolio 1994 Connor amp Lake1988 Cunningham amp Gresso 1993 Donahoe 1993 Elmore amp Associ-ates 1990 Fullan amp Stiegelbauer 1991 Hatch 1998 Heller 1990Hollander amp O ermann 1990 House 1996 Lewis 1989 Lieberman ampMiller 1990 Maton amp Salem 1995 Miles amp Louis 1990 Murphy1991 Newmann 1993 Peterson McCarthey amp Elmore 1996 Repli-cation and Program Services Inc 1993 Sarason 1990 1996 Schle-chty 1990 Schmuck amp Runkel 1985 Smith amp OrsquoDay 1991 Spillane1998 Waterman 1987 Wehlage Smith amp Lipman 1992)

In terms of specic tasks associated with creating readiness therst involves disseminating the prototype and pursuing activities tobuild interest and consensus for change Decisions follow about spe-cic sites for replication Then steps are taken to negotiate a policyframework and agreements for engagement This is followed by activ-ity to modify the institutional infrastructure at chosen sites to t theprototype and address replication needs All these tasks should beaccomplished with a process that reects understanding of the natureof the organization and its stakeholders involves stakeholders inmaking substantive decisions and redesigning those mechanisms thatconstitute the organizational and programmatic infrastructure clari-es personal relevance when identifying the potential benets ofchange elicits genuine public statements of commitment and empo-wers and creates a sense of community

Creating a climate for change requires appreciation of the rolesplayed by vision and leadership for change policy direction supportsafeguards for risk-taking and infrastructure redesign Each of thesetopics is discussed briey below

308 L Taylor et al

Vision and Leadership

Any major reform begins with a vision of what a desired newapproach would look like and an understanding of how to facilitatenecessary changes One without the other is insufficient Leadershave a triple burden as they attempt to improve literacy the rst isto ensure that substantive organizational and programmaticrestructuring are considered the second is to build consensus forchange nally they must pursue e ective implementation includingspecic strategies for nancing establishing maintaining andenhancing productive changes

Examples of key objectives at this stage include clarifying poten-tial gains without creating unrealistic expectations delineating costswithout seriously dampening expectations about benets o eringincentives that mesh with intrinsic motives and conveying thedegree to which a prototype can be adapted while emphasizing thatcertain facets are essential and nonnegotiable A thread runningthrough all this is the need to stimulate increasing interest or motiva-tional readiness among a sufficient number of stakeholders Toclarify the point successful change at any level of educationrestructuring requires the committed involvement of a critical massof policy makers sta and parents Almost any promising idea orpractice for improving studentsrsquo reading and writing performancemay nd a receptive audience among a small group Many more indi-viduals however are likely to remain politely unresponsive and rel-uctant to make changes and some will be actively resistant Thusleaders are confronted with the task of shifting the attitudes of asignicant proportion of those who appear reluctant and resistant

The next step involves deciding about which sites to begin withCriteria for making such decisions try to balance immediate concernsabout a sitersquos current level of readiness (including analyses of poten-tial barriers) and the likelihood of success over the long run Forinstance in making initial judgements about the appropriateness of apotential site we gather information about How likely is it that acritical mass of decision makers will commit to allocating sufficientnances personnel time and space How likely is it that a criticalmass of stakeholders will develop sufficient motivational readinessand appropriate levels of competence With respect to the most inu-ential stakeholders will enough be supportive or at least sufficientlycommitted not to undermine the process Do enough youngsters at asite t the prole of students for whom the program model wasdesigned As these questions illustrate most initial selection criteriareect general considerations related to any di usion process More

Scaling up Reforms 309

specic criteria emerge during the negotiation process For examplea principal may be attracted by the idea of establishing a programthat brings in volunteer reading tutors but in subsequent discussionswith teachers union concerns may arise that require arbitration

Policy

Substantive restructuring is unlikely without the adoption of newpolicies at all relevant jurisdictional levels (Spillane 1998) More-over such policies must elevate desired reforms so that they are notseen simply as demonstrations pilot projects passing fads or supple-mentary e orts When reforms are not assigned a high priority theytend to be treated in a marginalized manner (Center for MentalHealth in Schools 1998) This continues to be the fate of programssuch as Head Start Even Start and many other approaches toenhancing school readiness and literacy Relatedly e orts must bemade to revoke policies that preserve an unsatisfactory status quo(see critique of remedial reading programs by Dudley-Marling ampMurphy 1997)

Lasting reform requires processes that ensure informed com-mitment ownership and on-going support on the part of policymakers This involves strategies to create interest and formalizeagreements about fundamental changes Local ownership is estab-lished through solid policy commitments well-designed infrastruc-ture mechanisms allocation of adequate resources (eg nancespersonnel space equipment) to operationalize the policy andrestructuring of time to ensure sta involvement in adapting theprototype to the setting We nd three steps are essential (a) build-ing on introductory presentations to provide indepth information andunderstanding as a basis for establishing consensus (b) negotiationof a policy framework and a set of agreements for engagementincluding a realistic budget and (c) informed and voluntary rati-cation of agreements by legitimate representatives of all major stake-holders

For any program there are principles components elements andstandards that dene its essence and thus must be agreed to as a rstcondition for engagement Equally important are fundamental scale-up considerations o -line nonnegotiable such as the need for tempo-rary mechanisms to facilitate change Once essentials are agreed onall other matters are negotiable

Informed commitment is strengthened and operationalized throughnegotiating formal agreements at each jurisdictional level and amongvarious stakeholders Policy statements articulate the commitment to

310 L Taylor et al

a programrsquos essence Memoranda of understanding and contractsspecify agreements about such matters as funding sources resourceappropriations personnel functions incentives and safeguards forrisk-taking stakeholder development immediate and long-term com-mitments and timelines accountability procedures and so forth

Scale-up is aided when the decision to proceed is ratied by sanc-tioned representatives of stakeholder groups Developing and nego-tiating policies contracts and other formal agreements is a complexbusiness We nd that addressing the many logistics and legalitiesrequires extensive involvement of a small number of authorized andwell-informed stakeholder representatives Thus in pursuing thesetasks our commitment to include everyone moves from a town hallapproach to a representative democratic process with enfranchisedrepresentatives reporting back frequently to their constituencies Atrst endorsement is in principle over time it is manifested throughsustained support When ratication reects e ective consensusbuilding scale-up e orts benet from a broad base of informed com-mitment ownership and active sponsorship These attributes areessential in ensuring requisite support and protections for those whomust bear the burden of learning new ways and who risk dips in per-formance and productivity while doing so

Redesigning Infrastructure

After agreements are ratied a scale-up team can begin its work(again see Figure 1) A central challenge at every jurisdictional levelis redesign of regular mechanisms and processes used to make andimplement decisions These modications ensure ownership supportparticipation and address specic concerns associated with scale-up

Five fundamental facets of the ongoing infrastructure of schoolsthat are the focus of redesign are (a) governance (b) planning andimplementation associated with specic organizational and programobjectives (c) coordination and integration to ensure cohesive func-tioning (d) daily leadership and (e) communication and informationmanagement A common example of the need for infrastructure modi-cation is seen in the trend to increase school stakeholdersrsquo collabo-ration participation and inuence One implication is thatgovernance mechanisms will be altered to redistribute power Amajor problem of course is how to empower additional stakeholdergroups without disempowering those who have essentialresponsibilities and abilities related to the educational enterprise Inaddition it is one thing to o er lsquolsquopartnershipsrsquorsquo to stakeholders suchas parents students sta and community agency representatives it

Scaling up Reforms 311

is another to create conditions that allow for e ective participationOne such condition involves translating capacity building activityinto comprehensive programs for stakeholder development

The necessity of all this can be appreciated by thinking aboutintroducing a comprehensive approach for improving student literacy(Stringeld Ross amp Smith 1996) Such approaches involve major sys-temic changes that encompass intensive partnerships with parents (ortheir surrogates) and with various entities in the community such aslibraries youth development programs businesses the faith com-munity and so forth Substantive partnerships require a true sharingof leadership blending of resources and leadership training for pro-fessionals and nonprofessionals alike In communities where manyparents have little or no connection to the school major outreache orts are inevitable prerequisites to increasing home involvement inschool reform Parent outreach of course has not been very suc-cessful in many neighborhoods Our experience suggests that a neces-sary rst step in most cases is to o er programs and services thatassist the family in meeting its most pressing needs Furthermorethere is the matter of building parent competence to deal with plan-ning reforms and restructuring schools and for low income familiesthere is a need to nd ways to pay parents for the time they devote toserving on governance and other committees

Time is one of the most critical elements determining the successof scale-up Even if a prototype doesnrsquot call for restructuring theschool day the scale-up process does Substantial blocks of time areneeded for stakeholder capacity building and for individual and col-lective planning (National Education Commission on Time andLearning 1994) Particularly critical is the need for freeing-up tea-chers to learn new approaches For example e orts to make impor-tant revisions in literacy programs seem consistently undermined bynot providing enough time during the school day for the mentoring ofteachers and by the difficulty of carving out sufficient time to teachparents how to help their children Clearly a nonnegotiable condi-tion for engagement is a realistic plan for ensuring time to plan andbuild capacity

Lessons Learned

Complex interventions of course are seldom implemented in a com-pletely planned and linear manner The many practical and unfore-seen events that arise require exible problem-solving Articulationof a scale-up model can guide planning but those facilitating the

312 L Taylor et al

process must be prepared to capitalize on every opportunity that canmove the process ahead

Among the most fundamental lessons learned in carrying outPhase 1 has been the tendency of all parties to set actions into motionwithout taking sufficient time to lay the foundation needed for sub-stantive change In marketing new ideas it is tempting to accentuatetheir promising attributes and minimize complications In negotiatingagreements policy makers at a school site frequently are askedsimply for a go-ahead rather than for their informed commitmentSometimes they assent mainly to get extra resources sometimes theyare motivated by a desire to be seen by constituents as doing some-thing to improve the school This all tends to produce pressures forpremature implementation that results in the form rather than thesubstance of change especially when administrators are under thegun of political accountability measures that make unrealisticdemands for quick and dramatic results in studentsrsquo reading scores

Although formulation of policy and related agreements take con-siderable time and other resources their importance cannot be over-emphasized Failure to establish and successfully maintainsubstantive reforms in schools probably is attributable in greatmeasure to proceeding without strong and clear policy support

Another unfortunate trend we have found is the omission of in-depth planning for ongoing capacity building for change agents andteam members Mechanisms function only as well as the personnelwho operate them Such personnel must be recruited and developed inways that ensure appropriate motivation and capability and suffi-cient time must be redeployed so they can learn and carry out newfunctions e ectively (Peterson McCarthey amp Elmore 1996) Allchanges require constant care and feeding Those who steer theprocess must be motivated and competent not just initially but overtime The complexity of systemic change requires close monitoring ofmechanisms and immediate follow-up to address problems In particu-lar it means providing continuous personalized guidance andsupport to enhance knowledge and skills and counter anxiety frus-tration and other stressors To these ends adequate resource supportmust be provided (time space materials equipment) opportunitiesmust be available for increasing ability and generating a sense ofrenewed mission and personnel turnover must be addressed quicklyAll stakeholders can benet from e orts designed to increase levelsof competence and enhance motivation for working together Suche orts encompass four stages of stakeholder development orienta-tion foundation-building capacity-building and continuing educa-tion

Scaling up Reforms 313

There is no simple solution to the chronic problem of providingtime for creating readiness building capacity and planning Indeedrestructuring time represents one of the most difficult scale-up prob-lems Examples of how the problem might be addressed includefreeing up sta by establishing opportunities for students to spendtime pursuing activities such as music art and sports with special-ists or supervised by aides and community volunteers Alternativelyschool might start later or end earlier on a given day As these exam-ples suggest any approach will be controversial but if the problem isnot addressed satisfactorily successful replication of comprehensiveprototypes is unlikely

PHASE II INITIAL IMPLEMENTATION OF APROTOTYPE

Initial implementation involves adapting and phasing-in a programwith well-designed guidance and support If there is anything certainabout e orts to replicate a prototype it is that the process is stress-ful Some of the stress arises from the nature of the program some isinherent in the process of organizational change Coalitions must bedeveloped new working relationships established disruptive rumorsand information overload countered and interpersonal conictsresolved Short-term frustrations must be kept in perspective vis a visthe reform vision To help deal with all this temporary mechanismsare added to the organizational infrastructure They include (a) asite-based steering mechanism to guide and support replication (b) achange agent from the scale-up team working with site stakeholderson a change team to facilitate coalition building problem solvingand conict resolution and (c) mentors and coaches to model andteach elements of the prototype These structures are created tofacilitate replication and some are assimilated into a sitersquos infra-structure at the end of the initial implementation phase to supportinstitutionalization and ongoing evolution

A scale-up team and steering group work at a site with the schoolrsquosleadership specic planning groups and other stakeholders to formu-late phase-in plans steer program development and generallyprovide guidance and support for change Two major facets of thiswork are delineating a sequence for introducing major program ele-ments and outlining strategies to facilitate implementation Particu-lar attention is given to how to start with special emphasis onspecifying structures and resources for guidance and support For

314 L Taylor et al

instance in restructuring to better address barriers to learning therst steps at a school site involve creating processes to map analyzecoordinate and redeploy existing resources Special change mecha-nisms such as an organization facilitator and a resource coordinatingteam are created to guide and support the activity (Adelman 19931996a 1996b Adelman amp Taylor 1997a 1997b 1998)

Throughout this phase formative evaluation procedures are estab-lished to provide feedback for program development As noted abovee ective e orts to lsquolsquoreinventrsquorsquo schools require ensuring that allinvolved have the time to develop and institutionalize a soundprogram and that they are not penalized for unavoidable missteps Asa prototype is phased-in evaluation must not be thought of in termsof accountability Major systemic changes to improve literacy cantake years to develop Outcome e ectiveness is demonstrated afterthe program is in place The purpose of evaluation at this stage is toguide revision and ne-tuning of processes Formative evaluationsgather and analyze information relevant to changes in planning pro-cesses governance structures and policies and resources they alsofocus on implementation strategies and barriers program organiz-ation and staffing and initial outcomes Are teachers mastering thenew curricula for literacy Is there increased student engagement inreading Are parents involved in supporting reading at home Ifthings are not progressing satisfactorily why not Whatrsquos the down-side of the new approach

Well-designed organizational support and guidance is needed toenhance productivity minimize problems and accommodate individ-ual di erences This involves various forms of capacity building andpersonalized day-by-day facilitation Intensive coaching with somefollow-up consultation for instance are key processes so are mentor-ship and technical assistance Continuing education provides a criti-cal vehicle for enhancing productive changes generating renewaland countering burnout As new stakeholders arrive technologicaltools can be particularly useful in helping them catch up All thisactivity not only builds capacity but can foster networking and otherforms of task-related social and personal support as well as providea wide range of enrichment opportunities that enhance morale

If the steps discussed to this point are done well a sound founda-tion for initial implementation should be in place This initial phase-in period can however consume considerable e ort create specialproblems and may yield a temporary drop in some performance indi-cators Good day-by-day facilitation aims at minimizing such negativeimpact by e ectively addressing stakeholder motivation and capabil-ity and overcoming barriers to productive working relationships

Scaling up Reforms 315

Lessons Learned

Failure to take sufficient time to create readiness (Phase 1) can resultin implementing the form rather than the substance of a prototypeFor example we nd that change agents frequently are sent intoschools before essential policy support is enacted and before schoolleaders have assimilated and decided to support reforms Teams areconvened to assist with reforms (plan coordinate develop newapproaches) but the absence of supportive policy means substantivechanges are not accomplished As a result the initial motivation ofmany key team members wanes and other counterproductivedynamics arise All of this seems inevitable when initial implementa-tion proceeds without adequate policy support

Even in situations where sufficient readiness is created difficultiesfrequently arise due to a failure to keep enough stakeholders consis-tently moving in the direction of desired outcomes Comprehensivechange is usually achieved only when fairly high levels of positiveenergy can be mobilized over extended periods of time among a criti-cal mass of stakeholders sustained energy is appropriately directedthe process is supported with ongoing and well-conceived capacitybuilding and individuals are not pushed beyond their capabilitiesAnd because low and negative motivation are related to resistance tochange and poor functioning matching motivation is a rst-orderconsideration That is scale-up e orts must use strategies designed tomobilize and maintain proactive e ort and overcome barriers toworking relationships As in personalizing instruction approx-imating a good motivational t also requires matching capabilitiessuch as starting with fewer elements at sites at which resources arelimited and accounting for variability in stakeholdersrsquo competenceOver and over we nd too little attention is paid to these mattersThe result is failure to create an lsquolsquoenvironmentrsquorsquo that mobilizesdirects and then maintains stakeholder involvement

As with students the problem can be conceived as that of main-taining an appropriate match between the demands of the situationand individual motivation and capabilities In this respect we thinkthe construct of personalization o ers a concept around which toorganize thinking about facilitating change As stressed in the rstarticle in this issue personalization calls for systematically planningand implementing processes focused not only on knowledge andskills but on attitudes In particular it emphasizes the importance ofa primary and constant focus on ensuring positive attitudes Mobi-lization probably is best facilitated when procedures are perceived byindividuals as good ways to reach desired outcomes This requires

316 L Taylor et al

processes that can instigate and enhance an individualrsquos perceptionsof valued opportunities choice and control accomplishment andrelatedness to others Even if a task isnrsquot enjoyable expectation offeeling some sense of satisfaction related to process or outcome canbe a powerful intrinsic factor motivating individual behavior Taskpersistence for example can be facilitated by the expectation thatone will feel competent self-determining or more closely connectedto others From this perspective ensuring that individuals havevalued options a meaningful role in decision making feedback thatemphasizes progress toward desired outcomes and positive workingrelationships are among the most basic facilitation strategies(Adelman amp Taylor 1993b 1994 Deci amp Ryan 1985)

One other initial implementation problem that often arises is diffi-culty in establishing mechanisms to facilitate productive workingrelationships and identify and deal with problems quickly Forexample it is expected that change agents will encounter manyinstances of individual resistance and apathy interpersonal conictsand resentments (including lsquolsquous vs themrsquorsquo dynamics) rumors thatoveremphasize the negative and underestimate the positive and indi-viduals who are frequent faultnders Such problems seriouslyimpede e ective replication The roots of some of these problemsoften are present at a site prior to scale-up change simply o ers anew focus and perhaps magnies troubling matters Other problemsare a direct product of the activities and relationships that the scale-up process engenders Given the inevitability of such problems build-ing and maintaining working relationships need to be among themost basic concerns for those who have responsibility for scale-up Inparticular considerable attention must be paid to enhancing themotivational readiness and capability of those who are to worktogether and ensure there is an appropriate infrastructure to guideand support working relationships Proactively this requires problemprevention mechanisms that help create an atmosphere where defen-siveness is curtailed and positive rapport is engendered The point isto enhance attitudes knowledge and skills that foster interpersonalconnections and a sense of community Reactively the emphasis is onproblem solving resolving conict and providing ongoing support torebuild relationships Policies must encourage problem-solving-oriented critiques safeguards that protect those making changesappreciation for e ort and celebration of progress We nd thateveryone understands such matters but the culture at many schoolsites is more attuned to problem naming and analyzing than to antici-pating preventing and solving problems that arise around workingrelationships

Scaling up Reforms 317

Those responsible for systemic change need to spend as much timeas necessary ensuring that a schoolrsquos infrastructure is ready toprevent and ameliorate problems Special attention must be paid toensuring that problem solving mechanisms and communication pro-cesses are in place and properly sta ed and that stakeholders arewell informed about how to use the procedures Furthermore somestakeholders may have to be encouraged to interact in ways thatconvey genuine empathy warmth and mutual regard and respectwith a view to creating and maintaining a positive working climateand a psychological sense of community

At times we nd it necessary to target a specic problem anddesignated persons In some instances rather simple strategies aree ective For example most motivated individuals can be directlytaught ways to improve understanding and communication and avoidor resolve conicts that interfere with working relationships Inother instances however signicant remedial action is necessary aswhen overcoming barriers to a working relationship involvescountering negative attitudes Helpful in this regard are analysessuch as that by Sue and Zane (1987) that suggest how to demonstratethat something of value can be gained from individuals workingtogether and how to establish each participantrsquos credibility (eg bymaximizing task-focus and positive outcomes)

PHASE III INSTITUTIONALIZING THE PROTOTYPE

Maintaining and enhancing changes can be as difficult as makingthem in the rst place The history of education reform is one offailure to foster promising prototypes in substantive ways and overan extended period of times (Tyack amp Cuban 1995) Institutionalizinga prototype entails ensuring that the organization assumes long-termownership and that there is a blueprint for countering forces that canerode the changes Moreover institutionalization is more than atechnical process It requires assimilation of and ongoing adherenceto the values inherent in the prototypersquos underlying rationale Thefocus of course is not just on maintenance the point is to moveforward by enhancing productive changes and generating a sense ofrenewal as needed Critical in all this are specic plans that guar-antee ongoing and enhanced leadership and delineate ways in whichplanning implementation coordination and continuing educationmechanisms are maintained

Some Major Tasks

Whose responsibility is it to advocate for maintaining and evolving a

318 L Taylor et al

replicated prototype for improving literacy As problems arise whoseresponsibility is it to lead the way in resolving them Leadership isthe key here by both official leaders such as administrators mentorsta union chapter chairs and elected parent representatives aswell as natural leaders such as reading and writing teachers Obvi-ously official and natural leaders are not mutually exclusive groupsAt this phase both types of leadership are essential to ensure a broadenough base for ongoing advocacy problem solving enhancementand renewal Official leaders provide a legitimate power base asvarious interests compete for the organizationrsquos limited resourcesand they play a key role in ensuring that the contributions of naturalleaders are recognized and rewarded

Maintenance and enhancement require that the organizationrsquosgovernance body assumes ownership and program advocacy such astaking over the temporary steering grouprsquos functions addressingongoing policy and long-range planning concerns and maintainingnancial support The foundation for such ownership is laid duringthe readiness phase Each element becomes the organizationrsquos pro-perty as it is established during initial implementation The officialdeed of ownership is transferred as soon as the prototype is in place

Ownership however is no guarantee of institutionalizationVarious forces that can erode reforms are always at work Forinstance teams at a site experience turnover problems with commu-nication and sharing of resources are chronic competing interestsand the attractiveness of moving on to something new pull attentionand resources to other activity To minimize such problems stepsmust be taken to identify and solve them as quickly as is feasibleThis requires someone who has the time energy and expertise tomeet periodically with stakeholders to anticipate and amelioratethreats to a prototypersquos integrity

Over time mechanisms for planning implementation and coordi-nation are maintained by ensuring the activity is an official part ofthe infrastructure has appropriate leadership and is e ectively sup-ported Anyone who has worked on a school-based team to improve aliteracy program knows there must be a critical mass of teammembers so that the work load is manageable and a broad base ofinvolvement is ensured Also essential are adequate resources includ-ing time to learn the role and time to perform the functions reason-ably interesting tasks technical support for problem solvingrecognition and rewards for contributions immediate replacementwhen someone leaves continuing education to enhance team func-tioning and so forth Without serious attention to such matters theteamsrsquo morale and motivation will wane

Scaling up Reforms 319

Lessons Learned

Newly institutionalized approaches are seriously jeopardized in theabsence of dedicated ongoing capacity-building Of particular impor-tance are ways to rapidly and e ectively assimilate new arrivals at aschool (sta students families) This is a major concern at sites withconsiderable turnover or growth At such sites the majority of thoseinitially involved in implementing a new approach may be gonewithin a period of two to three years Whatever the mobility rate it isessential to design and maintain transition programs for newarrivals Initial welcoming and introductory orientations of coursemust be followed-up with ongoing support systems and intensivecapacity building related to understanding and valuing theapproaches the school has adopted We nd that all this is essentialnot only to maintain what has been adopted but also can contributeto establishing schools as caring environments

PHASE IV ONGOING EVOLUTION

Ongoing evolution of organizations and programs is the product ofe orts to account for accomplishments deal with changing times andconditions incorporate new knowledge and create a sense of renewalas the excitement of newness wears o and the demands of changesap energy As suggested already in part vigor and direction can bemaintained through continuing education especially exposure toideas that suggest a range of ways for evolving a program As thefollowing discussion indicates ongoing evolution also is fostered byevaluation designed to document accomplishments and provide feed-back designed to improve quality

Increased concern over accountability has advanced the wayevaluation is conceived (Posavac amp Carey 1989 Rossi amp Freeman1989 Scriven 1993 Sechrest amp Figueredo 1993 Shadish Jr Cook ampLeviton 1991 Stake 1967 1976 Stufflebeam amp Webster 1983 Weiss1995) At the same time social and political forces have literallyshaped the whole enterprise and in the process have narrowed theway professionals clients policymakers underwriters and thegeneral public think about program evaluation A prevailing cry isfor specic evidence of e ectiveness For schools this means imme-diate gains in literacy as measured by achievement tests Althoughunderstandable in light of the unlled promise of so many programsand the insatiable demands on limited public nances such simplisti-cally conceived accountability demands ignore the complexities ofdeveloping and scaling-up major reforms

320 L Taylor et al

Formative and Summative Evaluation

Evaluation of a prototype involves more than determining efficacyfor students Broadly stated it encompasses concerns about how toexpand the focus of evaluative research not only to contribute toimproving practice but also to aid in evolving practice and policy(General Accounting Office 1989 Lyon amp Moats 1997) To facilitateprogram development and organizational change the primary orien-tation for evaluation in the early phases is formative It is especiallyfocused on data gathering and analyses that can help improve pro-cedures Most of what is written about educational and psychosocialintervention however is oriented to summative evaluation and mea-suring outcomes for individuals such as improved reading achieve-ment scores Replicating approaches to improve literacy involve notonly changing individuals but changing organizations and systemsThus both individuals and systems must be evaluated

All this presumes appropriate mechanisms to provide and analyzeessential information To these ends a scale-up sta can help estab-lish an evaluation team and capacity building that prepares a schoolto conduct evaluation that enhances reforms The immediate focus ison successful program replication ultimately of course the empha-sis must be on student outcomes

Pursuing Results

Due to the increased interest in accountability many complex aimsare broken down into specic objectives Indeed short-range objec-tives stated in measurable terms generally assume a central role inplanning However short-range objectives are not ends in them-selves They are a small part of a particular goal and aim and some-times are prerequisites for moving on to a goal It is essential not tolose sight of the fact that many specic objectives are relativelysmall unrepresentative and often unimportant segments of the mostvalued aims that society has for its citizens and citizens have forthemselves

The problem is well exemplied by the narrow focus found inreviews analyses and reanalyses of data on early education (eg seeAlbee amp Gullotta 1997 Bond amp Compas 1989 Dryfoos 1990 Durlak1995 Elias 1997 Mitchell Seligson amp Marx 1989 Schorr 1988Slavin Karweit amp Madden 1989 Weissberg Gullotta HamptomRyan amp Adams 1997) As such work demonstrates overemphasis onevaluating the efficacy of underdeveloped prototypes draws resourcesaway from formative evaluation

Scaling up Reforms 321

With specic respect to scale-up the rst accomplishment is thereplication itself Have all facets been implemented How completelyhas each been implemented At how many locations The next set ofresults are any indications of progress for students such as improve-ments in attitudes toward school health attendance behavior andacademic achievement A nal set of evaluation concerns is thedegree to which student outcomes approximate societal standards

Lessons Learned

The process of evaluating results is costly in terms of nancial invest-ment the negative psychological impact on those evaluated and theways it can inappropriately reshape new approaches Cost-e ectiveoutcomes cannot be achieved in the absence of e ective prototypedevelopment and research Premature e orts to carry out com-prehensive summative evaluations clearly are not cost-e ective Anyreading and writing program will show poor results if it is evaluatedbefore teachers have mastered its application None of this of courseis an argument against evaluating results Rather it is meant tounderscore concerns and encourage greater attention to addressingthem

Once a prototype is established care must be taken to avoiddeveloping outcome evaluation as an adversarial process Due to thepolitical realities related to accountability one of the most per-plexing facets to negotiate is the time frame for summative evalu-ation The more complex the prototype the longer it takes and thecostlier it is to implement and evaluate Schools usually want quickprocesses and results and of course rarely can a ord costly innova-tions or lengthy di usion activity Compromises are inevitable butmust be arrived at with great care so as not to undermine the sub-stance of proposed changes

The psychology of evaluation suggests that an overemphasis onaccountability tends to produce negative reactions One possible wayto counter this may be to conceive evaluation as a way for everystakeholder to self-evaluate as a basis for quality improvement and asa way of getting credit for all that is accomplished Unfortunately asaccountability pressures increase we nd that replication of proto-types are guided more by what can be measured than by long-rangeaims That is demands for immediate accountability reshape prac-tices so that the emphasis shifts to immediate and readily measuredobjectives and away from fundamental purposes Over time this inap-propriately leads to radical revision of the underlying rationale for aprototype

322 L Taylor et al

CONCLUDING COMMENTS

Those who set out to change schools and schooling are confrontedwith two enormous tasks The rst is to develop prototypes Thesecond involves large-scale replication One without the other isinsufficient Yet considerably more attention is paid to developingand validating prototypes than to delineating and testing scale-upprocesses Clearly it is time to correct this deciency The ideas pre-sented in this article are meant to stimulate work on the problem andthereby to advance the cause of educational reform

Finally in fairness to those who labor for educational reform weall must remember that the quality of schooling family life and com-munity functioning spirals up or down as a function of the quality ofthe ongoing transactions among each Thus scale-up e orts relatedto educational reform must take place within the context of a poli-tical agenda that addresses ways to strengthen the family and com-munity infrastructure through strategies that enhance economicopportunity adult literacy and so forth What we need are policiesthat develop demonstrate and scale-up comprehensive multifacetedintegrated approaches that can e ectively address barriers to devel-opment learning and teaching

REFERENCES

Adelman H S (1993) School-linked mental health interventions Toward mechanismsfor service coordination and integration Journal of Community Psychology 21 309ndash319

Adelman H S (1996a) Restructuring education support services and integrating com-munity resources Beyond the full service school model School Psychology Review25 431ndash445

Adelman H S (1996b) Restructuring support services Toward a comprehensiveapproach Kent OH American School Health Association

Adelman H S amp Taylor L (1993a) Early Assistance for Students and FamiliesProgram Guidebook Los Angeles Early Assistance for Students and FamiliesProject Copies available from the School Mental Health Project Dept of Psychol-ogy UCLA

Adelman H S amp Taylor L (1993b) Learning problems and learning disabilities Moving forward Pacic Grove CA BrooksCole

Adelman H S amp Taylor L (1994) On understanding intervention in psychology andeducation Westport CT Praeger

Adelman H S amp Taylor L (1997a) Addressing barriers to learning Beyond school-linked services and full service schools American Journal of Orthopsychiatry 67408ndash421

Adelman H S amp Taylor L (1997b) Toward a scale-up model for replicating newapproaches to schooling Journal of Educational and Psychological Consultation 8197ndash230

Scaling up Reforms 305

FIGURE 1 Scale-up phases and major tasks

well-designed guidance and support infrastructure (c) institutional-ization by ensuring there is an infrastructure to maintain andenhance productive changes and (d) ongoing evolution through useof mechanisms to improve quality and provide continuing support

To initiate and guide prototype replication a scale-up mechanismis needed One way to conceive such a mechanism is in terms of a

306 L Taylor et al

scale-up project Such a project provides a necessary organizationalbase and skilled personnel for disseminating a prototype negotiatingdecisions about replication and dispensing the expertise to facilitatescale-up A scale-up project can dispense expertise by sending out ascale-up team consisting of project sta who for designated periodsof time travel to replication sites A core team of perhaps two-to-fourproject sta works closely with a site throughout the replicationprocess The team is augmented whenever a specialist is needed toassist with a specic element such as new curricula use of advancedtechnology or restructuring of education support programs Scaling-up a comprehensive prototype almost always requires phased-inchange and the addition of temporary infrastructure mechanisms tofacilitate changes

Figure 1 briey highlights specic tasks related to the four phasesof scale-up (For more on each phase see Adelman and Taylor 1997b)Each task requires careful planning based on sound intervention fun-damentals (see Adelman amp Taylor 1994) This means paying specialattention to the problem of the match as discussed in the rst articlein this issue

PHASE I CREATING READINESSETH ENHANCING THECLIMATE FOR CHANGE

In most organizations mandated changes often lead to change inform rather than substance Substantive systemic change requirespatience and perseverance E orts to alter an organizationrsquos cultureevolve slowly in transaction with the specic organizational and pro-grammatic changes Early in the process the emphasis is on creatingan official and psychological climate for change including over-coming institutionalized resistance negative attitudes and barriersto change New attitudes new working relationships new skills allmust be engendered and negative reactions and dynamics must beaddressed

Creating readiness for reforms involves tasks designed to producefundamental changes in the culture that characterizes schools Sub-stantive reform is most likely when high levels of positive energyamong stakeholders can be mobilized and appropriately directed overextended periods of time Thus one of the rst concerns is how tomobilize and direct the energy of a critical mass of participants toensure readiness and commitment This calls for proceeding in waysthat establish and maintain an e ective match with the motivationand capabilities of involved parties In this respect a review of the

Scaling up Reforms 307

literature claries the value of (a) a high level of policy and lead-ership commitment that is translated into an inspiring vision andappropriate resources (leadership space budget time) (b) incentivesfor change such as intrinsically valued outcomes expectations forsuccess recognitions rewards (c) procedural options that reectstakeholder strengths and from which those expected to implementchange can select strategies they see as workable (d) a willingness toestablish an infrastructure and processes that facilitate changee orts such as a governance mechanism that adopts strategies forimproving organizational health including one that enhances a senseof community (e) use of change agents who are perceived as pragma-tic by maintaining ideals while embracing practical solutions (f)accomplishing change in stages and with realistic timelines (g) pro-viding feedback on progress and (h) institutionalizing supportmechanisms to maintain and evolve changes and to generate periodicrenewal There is an extensive literature in this area (eg Argyris1993 Barth 1990 Bass 1997 Bass amp Avolio 1994 Connor amp Lake1988 Cunningham amp Gresso 1993 Donahoe 1993 Elmore amp Associ-ates 1990 Fullan amp Stiegelbauer 1991 Hatch 1998 Heller 1990Hollander amp O ermann 1990 House 1996 Lewis 1989 Lieberman ampMiller 1990 Maton amp Salem 1995 Miles amp Louis 1990 Murphy1991 Newmann 1993 Peterson McCarthey amp Elmore 1996 Repli-cation and Program Services Inc 1993 Sarason 1990 1996 Schle-chty 1990 Schmuck amp Runkel 1985 Smith amp OrsquoDay 1991 Spillane1998 Waterman 1987 Wehlage Smith amp Lipman 1992)

In terms of specic tasks associated with creating readiness therst involves disseminating the prototype and pursuing activities tobuild interest and consensus for change Decisions follow about spe-cic sites for replication Then steps are taken to negotiate a policyframework and agreements for engagement This is followed by activ-ity to modify the institutional infrastructure at chosen sites to t theprototype and address replication needs All these tasks should beaccomplished with a process that reects understanding of the natureof the organization and its stakeholders involves stakeholders inmaking substantive decisions and redesigning those mechanisms thatconstitute the organizational and programmatic infrastructure clari-es personal relevance when identifying the potential benets ofchange elicits genuine public statements of commitment and empo-wers and creates a sense of community

Creating a climate for change requires appreciation of the rolesplayed by vision and leadership for change policy direction supportsafeguards for risk-taking and infrastructure redesign Each of thesetopics is discussed briey below

308 L Taylor et al

Vision and Leadership

Any major reform begins with a vision of what a desired newapproach would look like and an understanding of how to facilitatenecessary changes One without the other is insufficient Leadershave a triple burden as they attempt to improve literacy the rst isto ensure that substantive organizational and programmaticrestructuring are considered the second is to build consensus forchange nally they must pursue e ective implementation includingspecic strategies for nancing establishing maintaining andenhancing productive changes

Examples of key objectives at this stage include clarifying poten-tial gains without creating unrealistic expectations delineating costswithout seriously dampening expectations about benets o eringincentives that mesh with intrinsic motives and conveying thedegree to which a prototype can be adapted while emphasizing thatcertain facets are essential and nonnegotiable A thread runningthrough all this is the need to stimulate increasing interest or motiva-tional readiness among a sufficient number of stakeholders Toclarify the point successful change at any level of educationrestructuring requires the committed involvement of a critical massof policy makers sta and parents Almost any promising idea orpractice for improving studentsrsquo reading and writing performancemay nd a receptive audience among a small group Many more indi-viduals however are likely to remain politely unresponsive and rel-uctant to make changes and some will be actively resistant Thusleaders are confronted with the task of shifting the attitudes of asignicant proportion of those who appear reluctant and resistant

The next step involves deciding about which sites to begin withCriteria for making such decisions try to balance immediate concernsabout a sitersquos current level of readiness (including analyses of poten-tial barriers) and the likelihood of success over the long run Forinstance in making initial judgements about the appropriateness of apotential site we gather information about How likely is it that acritical mass of decision makers will commit to allocating sufficientnances personnel time and space How likely is it that a criticalmass of stakeholders will develop sufficient motivational readinessand appropriate levels of competence With respect to the most inu-ential stakeholders will enough be supportive or at least sufficientlycommitted not to undermine the process Do enough youngsters at asite t the prole of students for whom the program model wasdesigned As these questions illustrate most initial selection criteriareect general considerations related to any di usion process More

Scaling up Reforms 309

specic criteria emerge during the negotiation process For examplea principal may be attracted by the idea of establishing a programthat brings in volunteer reading tutors but in subsequent discussionswith teachers union concerns may arise that require arbitration

Policy

Substantive restructuring is unlikely without the adoption of newpolicies at all relevant jurisdictional levels (Spillane 1998) More-over such policies must elevate desired reforms so that they are notseen simply as demonstrations pilot projects passing fads or supple-mentary e orts When reforms are not assigned a high priority theytend to be treated in a marginalized manner (Center for MentalHealth in Schools 1998) This continues to be the fate of programssuch as Head Start Even Start and many other approaches toenhancing school readiness and literacy Relatedly e orts must bemade to revoke policies that preserve an unsatisfactory status quo(see critique of remedial reading programs by Dudley-Marling ampMurphy 1997)

Lasting reform requires processes that ensure informed com-mitment ownership and on-going support on the part of policymakers This involves strategies to create interest and formalizeagreements about fundamental changes Local ownership is estab-lished through solid policy commitments well-designed infrastruc-ture mechanisms allocation of adequate resources (eg nancespersonnel space equipment) to operationalize the policy andrestructuring of time to ensure sta involvement in adapting theprototype to the setting We nd three steps are essential (a) build-ing on introductory presentations to provide indepth information andunderstanding as a basis for establishing consensus (b) negotiationof a policy framework and a set of agreements for engagementincluding a realistic budget and (c) informed and voluntary rati-cation of agreements by legitimate representatives of all major stake-holders

For any program there are principles components elements andstandards that dene its essence and thus must be agreed to as a rstcondition for engagement Equally important are fundamental scale-up considerations o -line nonnegotiable such as the need for tempo-rary mechanisms to facilitate change Once essentials are agreed onall other matters are negotiable

Informed commitment is strengthened and operationalized throughnegotiating formal agreements at each jurisdictional level and amongvarious stakeholders Policy statements articulate the commitment to

310 L Taylor et al

a programrsquos essence Memoranda of understanding and contractsspecify agreements about such matters as funding sources resourceappropriations personnel functions incentives and safeguards forrisk-taking stakeholder development immediate and long-term com-mitments and timelines accountability procedures and so forth

Scale-up is aided when the decision to proceed is ratied by sanc-tioned representatives of stakeholder groups Developing and nego-tiating policies contracts and other formal agreements is a complexbusiness We nd that addressing the many logistics and legalitiesrequires extensive involvement of a small number of authorized andwell-informed stakeholder representatives Thus in pursuing thesetasks our commitment to include everyone moves from a town hallapproach to a representative democratic process with enfranchisedrepresentatives reporting back frequently to their constituencies Atrst endorsement is in principle over time it is manifested throughsustained support When ratication reects e ective consensusbuilding scale-up e orts benet from a broad base of informed com-mitment ownership and active sponsorship These attributes areessential in ensuring requisite support and protections for those whomust bear the burden of learning new ways and who risk dips in per-formance and productivity while doing so

Redesigning Infrastructure

After agreements are ratied a scale-up team can begin its work(again see Figure 1) A central challenge at every jurisdictional levelis redesign of regular mechanisms and processes used to make andimplement decisions These modications ensure ownership supportparticipation and address specic concerns associated with scale-up

Five fundamental facets of the ongoing infrastructure of schoolsthat are the focus of redesign are (a) governance (b) planning andimplementation associated with specic organizational and programobjectives (c) coordination and integration to ensure cohesive func-tioning (d) daily leadership and (e) communication and informationmanagement A common example of the need for infrastructure modi-cation is seen in the trend to increase school stakeholdersrsquo collabo-ration participation and inuence One implication is thatgovernance mechanisms will be altered to redistribute power Amajor problem of course is how to empower additional stakeholdergroups without disempowering those who have essentialresponsibilities and abilities related to the educational enterprise Inaddition it is one thing to o er lsquolsquopartnershipsrsquorsquo to stakeholders suchas parents students sta and community agency representatives it

Scaling up Reforms 311

is another to create conditions that allow for e ective participationOne such condition involves translating capacity building activityinto comprehensive programs for stakeholder development

The necessity of all this can be appreciated by thinking aboutintroducing a comprehensive approach for improving student literacy(Stringeld Ross amp Smith 1996) Such approaches involve major sys-temic changes that encompass intensive partnerships with parents (ortheir surrogates) and with various entities in the community such aslibraries youth development programs businesses the faith com-munity and so forth Substantive partnerships require a true sharingof leadership blending of resources and leadership training for pro-fessionals and nonprofessionals alike In communities where manyparents have little or no connection to the school major outreache orts are inevitable prerequisites to increasing home involvement inschool reform Parent outreach of course has not been very suc-cessful in many neighborhoods Our experience suggests that a neces-sary rst step in most cases is to o er programs and services thatassist the family in meeting its most pressing needs Furthermorethere is the matter of building parent competence to deal with plan-ning reforms and restructuring schools and for low income familiesthere is a need to nd ways to pay parents for the time they devote toserving on governance and other committees

Time is one of the most critical elements determining the successof scale-up Even if a prototype doesnrsquot call for restructuring theschool day the scale-up process does Substantial blocks of time areneeded for stakeholder capacity building and for individual and col-lective planning (National Education Commission on Time andLearning 1994) Particularly critical is the need for freeing-up tea-chers to learn new approaches For example e orts to make impor-tant revisions in literacy programs seem consistently undermined bynot providing enough time during the school day for the mentoring ofteachers and by the difficulty of carving out sufficient time to teachparents how to help their children Clearly a nonnegotiable condi-tion for engagement is a realistic plan for ensuring time to plan andbuild capacity

Lessons Learned

Complex interventions of course are seldom implemented in a com-pletely planned and linear manner The many practical and unfore-seen events that arise require exible problem-solving Articulationof a scale-up model can guide planning but those facilitating the

312 L Taylor et al

process must be prepared to capitalize on every opportunity that canmove the process ahead

Among the most fundamental lessons learned in carrying outPhase 1 has been the tendency of all parties to set actions into motionwithout taking sufficient time to lay the foundation needed for sub-stantive change In marketing new ideas it is tempting to accentuatetheir promising attributes and minimize complications In negotiatingagreements policy makers at a school site frequently are askedsimply for a go-ahead rather than for their informed commitmentSometimes they assent mainly to get extra resources sometimes theyare motivated by a desire to be seen by constituents as doing some-thing to improve the school This all tends to produce pressures forpremature implementation that results in the form rather than thesubstance of change especially when administrators are under thegun of political accountability measures that make unrealisticdemands for quick and dramatic results in studentsrsquo reading scores

Although formulation of policy and related agreements take con-siderable time and other resources their importance cannot be over-emphasized Failure to establish and successfully maintainsubstantive reforms in schools probably is attributable in greatmeasure to proceeding without strong and clear policy support

Another unfortunate trend we have found is the omission of in-depth planning for ongoing capacity building for change agents andteam members Mechanisms function only as well as the personnelwho operate them Such personnel must be recruited and developed inways that ensure appropriate motivation and capability and suffi-cient time must be redeployed so they can learn and carry out newfunctions e ectively (Peterson McCarthey amp Elmore 1996) Allchanges require constant care and feeding Those who steer theprocess must be motivated and competent not just initially but overtime The complexity of systemic change requires close monitoring ofmechanisms and immediate follow-up to address problems In particu-lar it means providing continuous personalized guidance andsupport to enhance knowledge and skills and counter anxiety frus-tration and other stressors To these ends adequate resource supportmust be provided (time space materials equipment) opportunitiesmust be available for increasing ability and generating a sense ofrenewed mission and personnel turnover must be addressed quicklyAll stakeholders can benet from e orts designed to increase levelsof competence and enhance motivation for working together Suche orts encompass four stages of stakeholder development orienta-tion foundation-building capacity-building and continuing educa-tion

Scaling up Reforms 313

There is no simple solution to the chronic problem of providingtime for creating readiness building capacity and planning Indeedrestructuring time represents one of the most difficult scale-up prob-lems Examples of how the problem might be addressed includefreeing up sta by establishing opportunities for students to spendtime pursuing activities such as music art and sports with special-ists or supervised by aides and community volunteers Alternativelyschool might start later or end earlier on a given day As these exam-ples suggest any approach will be controversial but if the problem isnot addressed satisfactorily successful replication of comprehensiveprototypes is unlikely

PHASE II INITIAL IMPLEMENTATION OF APROTOTYPE

Initial implementation involves adapting and phasing-in a programwith well-designed guidance and support If there is anything certainabout e orts to replicate a prototype it is that the process is stress-ful Some of the stress arises from the nature of the program some isinherent in the process of organizational change Coalitions must bedeveloped new working relationships established disruptive rumorsand information overload countered and interpersonal conictsresolved Short-term frustrations must be kept in perspective vis a visthe reform vision To help deal with all this temporary mechanismsare added to the organizational infrastructure They include (a) asite-based steering mechanism to guide and support replication (b) achange agent from the scale-up team working with site stakeholderson a change team to facilitate coalition building problem solvingand conict resolution and (c) mentors and coaches to model andteach elements of the prototype These structures are created tofacilitate replication and some are assimilated into a sitersquos infra-structure at the end of the initial implementation phase to supportinstitutionalization and ongoing evolution

A scale-up team and steering group work at a site with the schoolrsquosleadership specic planning groups and other stakeholders to formu-late phase-in plans steer program development and generallyprovide guidance and support for change Two major facets of thiswork are delineating a sequence for introducing major program ele-ments and outlining strategies to facilitate implementation Particu-lar attention is given to how to start with special emphasis onspecifying structures and resources for guidance and support For

314 L Taylor et al

instance in restructuring to better address barriers to learning therst steps at a school site involve creating processes to map analyzecoordinate and redeploy existing resources Special change mecha-nisms such as an organization facilitator and a resource coordinatingteam are created to guide and support the activity (Adelman 19931996a 1996b Adelman amp Taylor 1997a 1997b 1998)

Throughout this phase formative evaluation procedures are estab-lished to provide feedback for program development As noted abovee ective e orts to lsquolsquoreinventrsquorsquo schools require ensuring that allinvolved have the time to develop and institutionalize a soundprogram and that they are not penalized for unavoidable missteps Asa prototype is phased-in evaluation must not be thought of in termsof accountability Major systemic changes to improve literacy cantake years to develop Outcome e ectiveness is demonstrated afterthe program is in place The purpose of evaluation at this stage is toguide revision and ne-tuning of processes Formative evaluationsgather and analyze information relevant to changes in planning pro-cesses governance structures and policies and resources they alsofocus on implementation strategies and barriers program organiz-ation and staffing and initial outcomes Are teachers mastering thenew curricula for literacy Is there increased student engagement inreading Are parents involved in supporting reading at home Ifthings are not progressing satisfactorily why not Whatrsquos the down-side of the new approach

Well-designed organizational support and guidance is needed toenhance productivity minimize problems and accommodate individ-ual di erences This involves various forms of capacity building andpersonalized day-by-day facilitation Intensive coaching with somefollow-up consultation for instance are key processes so are mentor-ship and technical assistance Continuing education provides a criti-cal vehicle for enhancing productive changes generating renewaland countering burnout As new stakeholders arrive technologicaltools can be particularly useful in helping them catch up All thisactivity not only builds capacity but can foster networking and otherforms of task-related social and personal support as well as providea wide range of enrichment opportunities that enhance morale

If the steps discussed to this point are done well a sound founda-tion for initial implementation should be in place This initial phase-in period can however consume considerable e ort create specialproblems and may yield a temporary drop in some performance indi-cators Good day-by-day facilitation aims at minimizing such negativeimpact by e ectively addressing stakeholder motivation and capabil-ity and overcoming barriers to productive working relationships

Scaling up Reforms 315

Lessons Learned

Failure to take sufficient time to create readiness (Phase 1) can resultin implementing the form rather than the substance of a prototypeFor example we nd that change agents frequently are sent intoschools before essential policy support is enacted and before schoolleaders have assimilated and decided to support reforms Teams areconvened to assist with reforms (plan coordinate develop newapproaches) but the absence of supportive policy means substantivechanges are not accomplished As a result the initial motivation ofmany key team members wanes and other counterproductivedynamics arise All of this seems inevitable when initial implementa-tion proceeds without adequate policy support

Even in situations where sufficient readiness is created difficultiesfrequently arise due to a failure to keep enough stakeholders consis-tently moving in the direction of desired outcomes Comprehensivechange is usually achieved only when fairly high levels of positiveenergy can be mobilized over extended periods of time among a criti-cal mass of stakeholders sustained energy is appropriately directedthe process is supported with ongoing and well-conceived capacitybuilding and individuals are not pushed beyond their capabilitiesAnd because low and negative motivation are related to resistance tochange and poor functioning matching motivation is a rst-orderconsideration That is scale-up e orts must use strategies designed tomobilize and maintain proactive e ort and overcome barriers toworking relationships As in personalizing instruction approx-imating a good motivational t also requires matching capabilitiessuch as starting with fewer elements at sites at which resources arelimited and accounting for variability in stakeholdersrsquo competenceOver and over we nd too little attention is paid to these mattersThe result is failure to create an lsquolsquoenvironmentrsquorsquo that mobilizesdirects and then maintains stakeholder involvement

As with students the problem can be conceived as that of main-taining an appropriate match between the demands of the situationand individual motivation and capabilities In this respect we thinkthe construct of personalization o ers a concept around which toorganize thinking about facilitating change As stressed in the rstarticle in this issue personalization calls for systematically planningand implementing processes focused not only on knowledge andskills but on attitudes In particular it emphasizes the importance ofa primary and constant focus on ensuring positive attitudes Mobi-lization probably is best facilitated when procedures are perceived byindividuals as good ways to reach desired outcomes This requires

316 L Taylor et al

processes that can instigate and enhance an individualrsquos perceptionsof valued opportunities choice and control accomplishment andrelatedness to others Even if a task isnrsquot enjoyable expectation offeeling some sense of satisfaction related to process or outcome canbe a powerful intrinsic factor motivating individual behavior Taskpersistence for example can be facilitated by the expectation thatone will feel competent self-determining or more closely connectedto others From this perspective ensuring that individuals havevalued options a meaningful role in decision making feedback thatemphasizes progress toward desired outcomes and positive workingrelationships are among the most basic facilitation strategies(Adelman amp Taylor 1993b 1994 Deci amp Ryan 1985)

One other initial implementation problem that often arises is diffi-culty in establishing mechanisms to facilitate productive workingrelationships and identify and deal with problems quickly Forexample it is expected that change agents will encounter manyinstances of individual resistance and apathy interpersonal conictsand resentments (including lsquolsquous vs themrsquorsquo dynamics) rumors thatoveremphasize the negative and underestimate the positive and indi-viduals who are frequent faultnders Such problems seriouslyimpede e ective replication The roots of some of these problemsoften are present at a site prior to scale-up change simply o ers anew focus and perhaps magnies troubling matters Other problemsare a direct product of the activities and relationships that the scale-up process engenders Given the inevitability of such problems build-ing and maintaining working relationships need to be among themost basic concerns for those who have responsibility for scale-up Inparticular considerable attention must be paid to enhancing themotivational readiness and capability of those who are to worktogether and ensure there is an appropriate infrastructure to guideand support working relationships Proactively this requires problemprevention mechanisms that help create an atmosphere where defen-siveness is curtailed and positive rapport is engendered The point isto enhance attitudes knowledge and skills that foster interpersonalconnections and a sense of community Reactively the emphasis is onproblem solving resolving conict and providing ongoing support torebuild relationships Policies must encourage problem-solving-oriented critiques safeguards that protect those making changesappreciation for e ort and celebration of progress We nd thateveryone understands such matters but the culture at many schoolsites is more attuned to problem naming and analyzing than to antici-pating preventing and solving problems that arise around workingrelationships

Scaling up Reforms 317

Those responsible for systemic change need to spend as much timeas necessary ensuring that a schoolrsquos infrastructure is ready toprevent and ameliorate problems Special attention must be paid toensuring that problem solving mechanisms and communication pro-cesses are in place and properly sta ed and that stakeholders arewell informed about how to use the procedures Furthermore somestakeholders may have to be encouraged to interact in ways thatconvey genuine empathy warmth and mutual regard and respectwith a view to creating and maintaining a positive working climateand a psychological sense of community

At times we nd it necessary to target a specic problem anddesignated persons In some instances rather simple strategies aree ective For example most motivated individuals can be directlytaught ways to improve understanding and communication and avoidor resolve conicts that interfere with working relationships Inother instances however signicant remedial action is necessary aswhen overcoming barriers to a working relationship involvescountering negative attitudes Helpful in this regard are analysessuch as that by Sue and Zane (1987) that suggest how to demonstratethat something of value can be gained from individuals workingtogether and how to establish each participantrsquos credibility (eg bymaximizing task-focus and positive outcomes)

PHASE III INSTITUTIONALIZING THE PROTOTYPE

Maintaining and enhancing changes can be as difficult as makingthem in the rst place The history of education reform is one offailure to foster promising prototypes in substantive ways and overan extended period of times (Tyack amp Cuban 1995) Institutionalizinga prototype entails ensuring that the organization assumes long-termownership and that there is a blueprint for countering forces that canerode the changes Moreover institutionalization is more than atechnical process It requires assimilation of and ongoing adherenceto the values inherent in the prototypersquos underlying rationale Thefocus of course is not just on maintenance the point is to moveforward by enhancing productive changes and generating a sense ofrenewal as needed Critical in all this are specic plans that guar-antee ongoing and enhanced leadership and delineate ways in whichplanning implementation coordination and continuing educationmechanisms are maintained

Some Major Tasks

Whose responsibility is it to advocate for maintaining and evolving a

318 L Taylor et al

replicated prototype for improving literacy As problems arise whoseresponsibility is it to lead the way in resolving them Leadership isthe key here by both official leaders such as administrators mentorsta union chapter chairs and elected parent representatives aswell as natural leaders such as reading and writing teachers Obvi-ously official and natural leaders are not mutually exclusive groupsAt this phase both types of leadership are essential to ensure a broadenough base for ongoing advocacy problem solving enhancementand renewal Official leaders provide a legitimate power base asvarious interests compete for the organizationrsquos limited resourcesand they play a key role in ensuring that the contributions of naturalleaders are recognized and rewarded

Maintenance and enhancement require that the organizationrsquosgovernance body assumes ownership and program advocacy such astaking over the temporary steering grouprsquos functions addressingongoing policy and long-range planning concerns and maintainingnancial support The foundation for such ownership is laid duringthe readiness phase Each element becomes the organizationrsquos pro-perty as it is established during initial implementation The officialdeed of ownership is transferred as soon as the prototype is in place

Ownership however is no guarantee of institutionalizationVarious forces that can erode reforms are always at work Forinstance teams at a site experience turnover problems with commu-nication and sharing of resources are chronic competing interestsand the attractiveness of moving on to something new pull attentionand resources to other activity To minimize such problems stepsmust be taken to identify and solve them as quickly as is feasibleThis requires someone who has the time energy and expertise tomeet periodically with stakeholders to anticipate and amelioratethreats to a prototypersquos integrity

Over time mechanisms for planning implementation and coordi-nation are maintained by ensuring the activity is an official part ofthe infrastructure has appropriate leadership and is e ectively sup-ported Anyone who has worked on a school-based team to improve aliteracy program knows there must be a critical mass of teammembers so that the work load is manageable and a broad base ofinvolvement is ensured Also essential are adequate resources includ-ing time to learn the role and time to perform the functions reason-ably interesting tasks technical support for problem solvingrecognition and rewards for contributions immediate replacementwhen someone leaves continuing education to enhance team func-tioning and so forth Without serious attention to such matters theteamsrsquo morale and motivation will wane

Scaling up Reforms 319

Lessons Learned

Newly institutionalized approaches are seriously jeopardized in theabsence of dedicated ongoing capacity-building Of particular impor-tance are ways to rapidly and e ectively assimilate new arrivals at aschool (sta students families) This is a major concern at sites withconsiderable turnover or growth At such sites the majority of thoseinitially involved in implementing a new approach may be gonewithin a period of two to three years Whatever the mobility rate it isessential to design and maintain transition programs for newarrivals Initial welcoming and introductory orientations of coursemust be followed-up with ongoing support systems and intensivecapacity building related to understanding and valuing theapproaches the school has adopted We nd that all this is essentialnot only to maintain what has been adopted but also can contributeto establishing schools as caring environments

PHASE IV ONGOING EVOLUTION

Ongoing evolution of organizations and programs is the product ofe orts to account for accomplishments deal with changing times andconditions incorporate new knowledge and create a sense of renewalas the excitement of newness wears o and the demands of changesap energy As suggested already in part vigor and direction can bemaintained through continuing education especially exposure toideas that suggest a range of ways for evolving a program As thefollowing discussion indicates ongoing evolution also is fostered byevaluation designed to document accomplishments and provide feed-back designed to improve quality

Increased concern over accountability has advanced the wayevaluation is conceived (Posavac amp Carey 1989 Rossi amp Freeman1989 Scriven 1993 Sechrest amp Figueredo 1993 Shadish Jr Cook ampLeviton 1991 Stake 1967 1976 Stufflebeam amp Webster 1983 Weiss1995) At the same time social and political forces have literallyshaped the whole enterprise and in the process have narrowed theway professionals clients policymakers underwriters and thegeneral public think about program evaluation A prevailing cry isfor specic evidence of e ectiveness For schools this means imme-diate gains in literacy as measured by achievement tests Althoughunderstandable in light of the unlled promise of so many programsand the insatiable demands on limited public nances such simplisti-cally conceived accountability demands ignore the complexities ofdeveloping and scaling-up major reforms

320 L Taylor et al

Formative and Summative Evaluation

Evaluation of a prototype involves more than determining efficacyfor students Broadly stated it encompasses concerns about how toexpand the focus of evaluative research not only to contribute toimproving practice but also to aid in evolving practice and policy(General Accounting Office 1989 Lyon amp Moats 1997) To facilitateprogram development and organizational change the primary orien-tation for evaluation in the early phases is formative It is especiallyfocused on data gathering and analyses that can help improve pro-cedures Most of what is written about educational and psychosocialintervention however is oriented to summative evaluation and mea-suring outcomes for individuals such as improved reading achieve-ment scores Replicating approaches to improve literacy involve notonly changing individuals but changing organizations and systemsThus both individuals and systems must be evaluated

All this presumes appropriate mechanisms to provide and analyzeessential information To these ends a scale-up sta can help estab-lish an evaluation team and capacity building that prepares a schoolto conduct evaluation that enhances reforms The immediate focus ison successful program replication ultimately of course the empha-sis must be on student outcomes

Pursuing Results

Due to the increased interest in accountability many complex aimsare broken down into specic objectives Indeed short-range objec-tives stated in measurable terms generally assume a central role inplanning However short-range objectives are not ends in them-selves They are a small part of a particular goal and aim and some-times are prerequisites for moving on to a goal It is essential not tolose sight of the fact that many specic objectives are relativelysmall unrepresentative and often unimportant segments of the mostvalued aims that society has for its citizens and citizens have forthemselves

The problem is well exemplied by the narrow focus found inreviews analyses and reanalyses of data on early education (eg seeAlbee amp Gullotta 1997 Bond amp Compas 1989 Dryfoos 1990 Durlak1995 Elias 1997 Mitchell Seligson amp Marx 1989 Schorr 1988Slavin Karweit amp Madden 1989 Weissberg Gullotta HamptomRyan amp Adams 1997) As such work demonstrates overemphasis onevaluating the efficacy of underdeveloped prototypes draws resourcesaway from formative evaluation

Scaling up Reforms 321

With specic respect to scale-up the rst accomplishment is thereplication itself Have all facets been implemented How completelyhas each been implemented At how many locations The next set ofresults are any indications of progress for students such as improve-ments in attitudes toward school health attendance behavior andacademic achievement A nal set of evaluation concerns is thedegree to which student outcomes approximate societal standards

Lessons Learned

The process of evaluating results is costly in terms of nancial invest-ment the negative psychological impact on those evaluated and theways it can inappropriately reshape new approaches Cost-e ectiveoutcomes cannot be achieved in the absence of e ective prototypedevelopment and research Premature e orts to carry out com-prehensive summative evaluations clearly are not cost-e ective Anyreading and writing program will show poor results if it is evaluatedbefore teachers have mastered its application None of this of courseis an argument against evaluating results Rather it is meant tounderscore concerns and encourage greater attention to addressingthem

Once a prototype is established care must be taken to avoiddeveloping outcome evaluation as an adversarial process Due to thepolitical realities related to accountability one of the most per-plexing facets to negotiate is the time frame for summative evalu-ation The more complex the prototype the longer it takes and thecostlier it is to implement and evaluate Schools usually want quickprocesses and results and of course rarely can a ord costly innova-tions or lengthy di usion activity Compromises are inevitable butmust be arrived at with great care so as not to undermine the sub-stance of proposed changes

The psychology of evaluation suggests that an overemphasis onaccountability tends to produce negative reactions One possible wayto counter this may be to conceive evaluation as a way for everystakeholder to self-evaluate as a basis for quality improvement and asa way of getting credit for all that is accomplished Unfortunately asaccountability pressures increase we nd that replication of proto-types are guided more by what can be measured than by long-rangeaims That is demands for immediate accountability reshape prac-tices so that the emphasis shifts to immediate and readily measuredobjectives and away from fundamental purposes Over time this inap-propriately leads to radical revision of the underlying rationale for aprototype

322 L Taylor et al

CONCLUDING COMMENTS

Those who set out to change schools and schooling are confrontedwith two enormous tasks The rst is to develop prototypes Thesecond involves large-scale replication One without the other isinsufficient Yet considerably more attention is paid to developingand validating prototypes than to delineating and testing scale-upprocesses Clearly it is time to correct this deciency The ideas pre-sented in this article are meant to stimulate work on the problem andthereby to advance the cause of educational reform

Finally in fairness to those who labor for educational reform weall must remember that the quality of schooling family life and com-munity functioning spirals up or down as a function of the quality ofthe ongoing transactions among each Thus scale-up e orts relatedto educational reform must take place within the context of a poli-tical agenda that addresses ways to strengthen the family and com-munity infrastructure through strategies that enhance economicopportunity adult literacy and so forth What we need are policiesthat develop demonstrate and scale-up comprehensive multifacetedintegrated approaches that can e ectively address barriers to devel-opment learning and teaching

REFERENCES

Adelman H S (1993) School-linked mental health interventions Toward mechanismsfor service coordination and integration Journal of Community Psychology 21 309ndash319

Adelman H S (1996a) Restructuring education support services and integrating com-munity resources Beyond the full service school model School Psychology Review25 431ndash445

Adelman H S (1996b) Restructuring support services Toward a comprehensiveapproach Kent OH American School Health Association

Adelman H S amp Taylor L (1993a) Early Assistance for Students and FamiliesProgram Guidebook Los Angeles Early Assistance for Students and FamiliesProject Copies available from the School Mental Health Project Dept of Psychol-ogy UCLA

Adelman H S amp Taylor L (1993b) Learning problems and learning disabilities Moving forward Pacic Grove CA BrooksCole

Adelman H S amp Taylor L (1994) On understanding intervention in psychology andeducation Westport CT Praeger

Adelman H S amp Taylor L (1997a) Addressing barriers to learning Beyond school-linked services and full service schools American Journal of Orthopsychiatry 67408ndash421

Adelman H S amp Taylor L (1997b) Toward a scale-up model for replicating newapproaches to schooling Journal of Educational and Psychological Consultation 8197ndash230

306 L Taylor et al

scale-up project Such a project provides a necessary organizationalbase and skilled personnel for disseminating a prototype negotiatingdecisions about replication and dispensing the expertise to facilitatescale-up A scale-up project can dispense expertise by sending out ascale-up team consisting of project sta who for designated periodsof time travel to replication sites A core team of perhaps two-to-fourproject sta works closely with a site throughout the replicationprocess The team is augmented whenever a specialist is needed toassist with a specic element such as new curricula use of advancedtechnology or restructuring of education support programs Scaling-up a comprehensive prototype almost always requires phased-inchange and the addition of temporary infrastructure mechanisms tofacilitate changes

Figure 1 briey highlights specic tasks related to the four phasesof scale-up (For more on each phase see Adelman and Taylor 1997b)Each task requires careful planning based on sound intervention fun-damentals (see Adelman amp Taylor 1994) This means paying specialattention to the problem of the match as discussed in the rst articlein this issue

PHASE I CREATING READINESSETH ENHANCING THECLIMATE FOR CHANGE

In most organizations mandated changes often lead to change inform rather than substance Substantive systemic change requirespatience and perseverance E orts to alter an organizationrsquos cultureevolve slowly in transaction with the specic organizational and pro-grammatic changes Early in the process the emphasis is on creatingan official and psychological climate for change including over-coming institutionalized resistance negative attitudes and barriersto change New attitudes new working relationships new skills allmust be engendered and negative reactions and dynamics must beaddressed

Creating readiness for reforms involves tasks designed to producefundamental changes in the culture that characterizes schools Sub-stantive reform is most likely when high levels of positive energyamong stakeholders can be mobilized and appropriately directed overextended periods of time Thus one of the rst concerns is how tomobilize and direct the energy of a critical mass of participants toensure readiness and commitment This calls for proceeding in waysthat establish and maintain an e ective match with the motivationand capabilities of involved parties In this respect a review of the

Scaling up Reforms 307

literature claries the value of (a) a high level of policy and lead-ership commitment that is translated into an inspiring vision andappropriate resources (leadership space budget time) (b) incentivesfor change such as intrinsically valued outcomes expectations forsuccess recognitions rewards (c) procedural options that reectstakeholder strengths and from which those expected to implementchange can select strategies they see as workable (d) a willingness toestablish an infrastructure and processes that facilitate changee orts such as a governance mechanism that adopts strategies forimproving organizational health including one that enhances a senseof community (e) use of change agents who are perceived as pragma-tic by maintaining ideals while embracing practical solutions (f)accomplishing change in stages and with realistic timelines (g) pro-viding feedback on progress and (h) institutionalizing supportmechanisms to maintain and evolve changes and to generate periodicrenewal There is an extensive literature in this area (eg Argyris1993 Barth 1990 Bass 1997 Bass amp Avolio 1994 Connor amp Lake1988 Cunningham amp Gresso 1993 Donahoe 1993 Elmore amp Associ-ates 1990 Fullan amp Stiegelbauer 1991 Hatch 1998 Heller 1990Hollander amp O ermann 1990 House 1996 Lewis 1989 Lieberman ampMiller 1990 Maton amp Salem 1995 Miles amp Louis 1990 Murphy1991 Newmann 1993 Peterson McCarthey amp Elmore 1996 Repli-cation and Program Services Inc 1993 Sarason 1990 1996 Schle-chty 1990 Schmuck amp Runkel 1985 Smith amp OrsquoDay 1991 Spillane1998 Waterman 1987 Wehlage Smith amp Lipman 1992)

In terms of specic tasks associated with creating readiness therst involves disseminating the prototype and pursuing activities tobuild interest and consensus for change Decisions follow about spe-cic sites for replication Then steps are taken to negotiate a policyframework and agreements for engagement This is followed by activ-ity to modify the institutional infrastructure at chosen sites to t theprototype and address replication needs All these tasks should beaccomplished with a process that reects understanding of the natureof the organization and its stakeholders involves stakeholders inmaking substantive decisions and redesigning those mechanisms thatconstitute the organizational and programmatic infrastructure clari-es personal relevance when identifying the potential benets ofchange elicits genuine public statements of commitment and empo-wers and creates a sense of community

Creating a climate for change requires appreciation of the rolesplayed by vision and leadership for change policy direction supportsafeguards for risk-taking and infrastructure redesign Each of thesetopics is discussed briey below

308 L Taylor et al

Vision and Leadership

Any major reform begins with a vision of what a desired newapproach would look like and an understanding of how to facilitatenecessary changes One without the other is insufficient Leadershave a triple burden as they attempt to improve literacy the rst isto ensure that substantive organizational and programmaticrestructuring are considered the second is to build consensus forchange nally they must pursue e ective implementation includingspecic strategies for nancing establishing maintaining andenhancing productive changes

Examples of key objectives at this stage include clarifying poten-tial gains without creating unrealistic expectations delineating costswithout seriously dampening expectations about benets o eringincentives that mesh with intrinsic motives and conveying thedegree to which a prototype can be adapted while emphasizing thatcertain facets are essential and nonnegotiable A thread runningthrough all this is the need to stimulate increasing interest or motiva-tional readiness among a sufficient number of stakeholders Toclarify the point successful change at any level of educationrestructuring requires the committed involvement of a critical massof policy makers sta and parents Almost any promising idea orpractice for improving studentsrsquo reading and writing performancemay nd a receptive audience among a small group Many more indi-viduals however are likely to remain politely unresponsive and rel-uctant to make changes and some will be actively resistant Thusleaders are confronted with the task of shifting the attitudes of asignicant proportion of those who appear reluctant and resistant

The next step involves deciding about which sites to begin withCriteria for making such decisions try to balance immediate concernsabout a sitersquos current level of readiness (including analyses of poten-tial barriers) and the likelihood of success over the long run Forinstance in making initial judgements about the appropriateness of apotential site we gather information about How likely is it that acritical mass of decision makers will commit to allocating sufficientnances personnel time and space How likely is it that a criticalmass of stakeholders will develop sufficient motivational readinessand appropriate levels of competence With respect to the most inu-ential stakeholders will enough be supportive or at least sufficientlycommitted not to undermine the process Do enough youngsters at asite t the prole of students for whom the program model wasdesigned As these questions illustrate most initial selection criteriareect general considerations related to any di usion process More

Scaling up Reforms 309

specic criteria emerge during the negotiation process For examplea principal may be attracted by the idea of establishing a programthat brings in volunteer reading tutors but in subsequent discussionswith teachers union concerns may arise that require arbitration

Policy

Substantive restructuring is unlikely without the adoption of newpolicies at all relevant jurisdictional levels (Spillane 1998) More-over such policies must elevate desired reforms so that they are notseen simply as demonstrations pilot projects passing fads or supple-mentary e orts When reforms are not assigned a high priority theytend to be treated in a marginalized manner (Center for MentalHealth in Schools 1998) This continues to be the fate of programssuch as Head Start Even Start and many other approaches toenhancing school readiness and literacy Relatedly e orts must bemade to revoke policies that preserve an unsatisfactory status quo(see critique of remedial reading programs by Dudley-Marling ampMurphy 1997)

Lasting reform requires processes that ensure informed com-mitment ownership and on-going support on the part of policymakers This involves strategies to create interest and formalizeagreements about fundamental changes Local ownership is estab-lished through solid policy commitments well-designed infrastruc-ture mechanisms allocation of adequate resources (eg nancespersonnel space equipment) to operationalize the policy andrestructuring of time to ensure sta involvement in adapting theprototype to the setting We nd three steps are essential (a) build-ing on introductory presentations to provide indepth information andunderstanding as a basis for establishing consensus (b) negotiationof a policy framework and a set of agreements for engagementincluding a realistic budget and (c) informed and voluntary rati-cation of agreements by legitimate representatives of all major stake-holders

For any program there are principles components elements andstandards that dene its essence and thus must be agreed to as a rstcondition for engagement Equally important are fundamental scale-up considerations o -line nonnegotiable such as the need for tempo-rary mechanisms to facilitate change Once essentials are agreed onall other matters are negotiable

Informed commitment is strengthened and operationalized throughnegotiating formal agreements at each jurisdictional level and amongvarious stakeholders Policy statements articulate the commitment to

310 L Taylor et al

a programrsquos essence Memoranda of understanding and contractsspecify agreements about such matters as funding sources resourceappropriations personnel functions incentives and safeguards forrisk-taking stakeholder development immediate and long-term com-mitments and timelines accountability procedures and so forth

Scale-up is aided when the decision to proceed is ratied by sanc-tioned representatives of stakeholder groups Developing and nego-tiating policies contracts and other formal agreements is a complexbusiness We nd that addressing the many logistics and legalitiesrequires extensive involvement of a small number of authorized andwell-informed stakeholder representatives Thus in pursuing thesetasks our commitment to include everyone moves from a town hallapproach to a representative democratic process with enfranchisedrepresentatives reporting back frequently to their constituencies Atrst endorsement is in principle over time it is manifested throughsustained support When ratication reects e ective consensusbuilding scale-up e orts benet from a broad base of informed com-mitment ownership and active sponsorship These attributes areessential in ensuring requisite support and protections for those whomust bear the burden of learning new ways and who risk dips in per-formance and productivity while doing so

Redesigning Infrastructure

After agreements are ratied a scale-up team can begin its work(again see Figure 1) A central challenge at every jurisdictional levelis redesign of regular mechanisms and processes used to make andimplement decisions These modications ensure ownership supportparticipation and address specic concerns associated with scale-up

Five fundamental facets of the ongoing infrastructure of schoolsthat are the focus of redesign are (a) governance (b) planning andimplementation associated with specic organizational and programobjectives (c) coordination and integration to ensure cohesive func-tioning (d) daily leadership and (e) communication and informationmanagement A common example of the need for infrastructure modi-cation is seen in the trend to increase school stakeholdersrsquo collabo-ration participation and inuence One implication is thatgovernance mechanisms will be altered to redistribute power Amajor problem of course is how to empower additional stakeholdergroups without disempowering those who have essentialresponsibilities and abilities related to the educational enterprise Inaddition it is one thing to o er lsquolsquopartnershipsrsquorsquo to stakeholders suchas parents students sta and community agency representatives it

Scaling up Reforms 311

is another to create conditions that allow for e ective participationOne such condition involves translating capacity building activityinto comprehensive programs for stakeholder development

The necessity of all this can be appreciated by thinking aboutintroducing a comprehensive approach for improving student literacy(Stringeld Ross amp Smith 1996) Such approaches involve major sys-temic changes that encompass intensive partnerships with parents (ortheir surrogates) and with various entities in the community such aslibraries youth development programs businesses the faith com-munity and so forth Substantive partnerships require a true sharingof leadership blending of resources and leadership training for pro-fessionals and nonprofessionals alike In communities where manyparents have little or no connection to the school major outreache orts are inevitable prerequisites to increasing home involvement inschool reform Parent outreach of course has not been very suc-cessful in many neighborhoods Our experience suggests that a neces-sary rst step in most cases is to o er programs and services thatassist the family in meeting its most pressing needs Furthermorethere is the matter of building parent competence to deal with plan-ning reforms and restructuring schools and for low income familiesthere is a need to nd ways to pay parents for the time they devote toserving on governance and other committees

Time is one of the most critical elements determining the successof scale-up Even if a prototype doesnrsquot call for restructuring theschool day the scale-up process does Substantial blocks of time areneeded for stakeholder capacity building and for individual and col-lective planning (National Education Commission on Time andLearning 1994) Particularly critical is the need for freeing-up tea-chers to learn new approaches For example e orts to make impor-tant revisions in literacy programs seem consistently undermined bynot providing enough time during the school day for the mentoring ofteachers and by the difficulty of carving out sufficient time to teachparents how to help their children Clearly a nonnegotiable condi-tion for engagement is a realistic plan for ensuring time to plan andbuild capacity

Lessons Learned

Complex interventions of course are seldom implemented in a com-pletely planned and linear manner The many practical and unfore-seen events that arise require exible problem-solving Articulationof a scale-up model can guide planning but those facilitating the

312 L Taylor et al

process must be prepared to capitalize on every opportunity that canmove the process ahead

Among the most fundamental lessons learned in carrying outPhase 1 has been the tendency of all parties to set actions into motionwithout taking sufficient time to lay the foundation needed for sub-stantive change In marketing new ideas it is tempting to accentuatetheir promising attributes and minimize complications In negotiatingagreements policy makers at a school site frequently are askedsimply for a go-ahead rather than for their informed commitmentSometimes they assent mainly to get extra resources sometimes theyare motivated by a desire to be seen by constituents as doing some-thing to improve the school This all tends to produce pressures forpremature implementation that results in the form rather than thesubstance of change especially when administrators are under thegun of political accountability measures that make unrealisticdemands for quick and dramatic results in studentsrsquo reading scores

Although formulation of policy and related agreements take con-siderable time and other resources their importance cannot be over-emphasized Failure to establish and successfully maintainsubstantive reforms in schools probably is attributable in greatmeasure to proceeding without strong and clear policy support

Another unfortunate trend we have found is the omission of in-depth planning for ongoing capacity building for change agents andteam members Mechanisms function only as well as the personnelwho operate them Such personnel must be recruited and developed inways that ensure appropriate motivation and capability and suffi-cient time must be redeployed so they can learn and carry out newfunctions e ectively (Peterson McCarthey amp Elmore 1996) Allchanges require constant care and feeding Those who steer theprocess must be motivated and competent not just initially but overtime The complexity of systemic change requires close monitoring ofmechanisms and immediate follow-up to address problems In particu-lar it means providing continuous personalized guidance andsupport to enhance knowledge and skills and counter anxiety frus-tration and other stressors To these ends adequate resource supportmust be provided (time space materials equipment) opportunitiesmust be available for increasing ability and generating a sense ofrenewed mission and personnel turnover must be addressed quicklyAll stakeholders can benet from e orts designed to increase levelsof competence and enhance motivation for working together Suche orts encompass four stages of stakeholder development orienta-tion foundation-building capacity-building and continuing educa-tion

Scaling up Reforms 313

There is no simple solution to the chronic problem of providingtime for creating readiness building capacity and planning Indeedrestructuring time represents one of the most difficult scale-up prob-lems Examples of how the problem might be addressed includefreeing up sta by establishing opportunities for students to spendtime pursuing activities such as music art and sports with special-ists or supervised by aides and community volunteers Alternativelyschool might start later or end earlier on a given day As these exam-ples suggest any approach will be controversial but if the problem isnot addressed satisfactorily successful replication of comprehensiveprototypes is unlikely

PHASE II INITIAL IMPLEMENTATION OF APROTOTYPE

Initial implementation involves adapting and phasing-in a programwith well-designed guidance and support If there is anything certainabout e orts to replicate a prototype it is that the process is stress-ful Some of the stress arises from the nature of the program some isinherent in the process of organizational change Coalitions must bedeveloped new working relationships established disruptive rumorsand information overload countered and interpersonal conictsresolved Short-term frustrations must be kept in perspective vis a visthe reform vision To help deal with all this temporary mechanismsare added to the organizational infrastructure They include (a) asite-based steering mechanism to guide and support replication (b) achange agent from the scale-up team working with site stakeholderson a change team to facilitate coalition building problem solvingand conict resolution and (c) mentors and coaches to model andteach elements of the prototype These structures are created tofacilitate replication and some are assimilated into a sitersquos infra-structure at the end of the initial implementation phase to supportinstitutionalization and ongoing evolution

A scale-up team and steering group work at a site with the schoolrsquosleadership specic planning groups and other stakeholders to formu-late phase-in plans steer program development and generallyprovide guidance and support for change Two major facets of thiswork are delineating a sequence for introducing major program ele-ments and outlining strategies to facilitate implementation Particu-lar attention is given to how to start with special emphasis onspecifying structures and resources for guidance and support For

314 L Taylor et al

instance in restructuring to better address barriers to learning therst steps at a school site involve creating processes to map analyzecoordinate and redeploy existing resources Special change mecha-nisms such as an organization facilitator and a resource coordinatingteam are created to guide and support the activity (Adelman 19931996a 1996b Adelman amp Taylor 1997a 1997b 1998)

Throughout this phase formative evaluation procedures are estab-lished to provide feedback for program development As noted abovee ective e orts to lsquolsquoreinventrsquorsquo schools require ensuring that allinvolved have the time to develop and institutionalize a soundprogram and that they are not penalized for unavoidable missteps Asa prototype is phased-in evaluation must not be thought of in termsof accountability Major systemic changes to improve literacy cantake years to develop Outcome e ectiveness is demonstrated afterthe program is in place The purpose of evaluation at this stage is toguide revision and ne-tuning of processes Formative evaluationsgather and analyze information relevant to changes in planning pro-cesses governance structures and policies and resources they alsofocus on implementation strategies and barriers program organiz-ation and staffing and initial outcomes Are teachers mastering thenew curricula for literacy Is there increased student engagement inreading Are parents involved in supporting reading at home Ifthings are not progressing satisfactorily why not Whatrsquos the down-side of the new approach

Well-designed organizational support and guidance is needed toenhance productivity minimize problems and accommodate individ-ual di erences This involves various forms of capacity building andpersonalized day-by-day facilitation Intensive coaching with somefollow-up consultation for instance are key processes so are mentor-ship and technical assistance Continuing education provides a criti-cal vehicle for enhancing productive changes generating renewaland countering burnout As new stakeholders arrive technologicaltools can be particularly useful in helping them catch up All thisactivity not only builds capacity but can foster networking and otherforms of task-related social and personal support as well as providea wide range of enrichment opportunities that enhance morale

If the steps discussed to this point are done well a sound founda-tion for initial implementation should be in place This initial phase-in period can however consume considerable e ort create specialproblems and may yield a temporary drop in some performance indi-cators Good day-by-day facilitation aims at minimizing such negativeimpact by e ectively addressing stakeholder motivation and capabil-ity and overcoming barriers to productive working relationships

Scaling up Reforms 315

Lessons Learned

Failure to take sufficient time to create readiness (Phase 1) can resultin implementing the form rather than the substance of a prototypeFor example we nd that change agents frequently are sent intoschools before essential policy support is enacted and before schoolleaders have assimilated and decided to support reforms Teams areconvened to assist with reforms (plan coordinate develop newapproaches) but the absence of supportive policy means substantivechanges are not accomplished As a result the initial motivation ofmany key team members wanes and other counterproductivedynamics arise All of this seems inevitable when initial implementa-tion proceeds without adequate policy support

Even in situations where sufficient readiness is created difficultiesfrequently arise due to a failure to keep enough stakeholders consis-tently moving in the direction of desired outcomes Comprehensivechange is usually achieved only when fairly high levels of positiveenergy can be mobilized over extended periods of time among a criti-cal mass of stakeholders sustained energy is appropriately directedthe process is supported with ongoing and well-conceived capacitybuilding and individuals are not pushed beyond their capabilitiesAnd because low and negative motivation are related to resistance tochange and poor functioning matching motivation is a rst-orderconsideration That is scale-up e orts must use strategies designed tomobilize and maintain proactive e ort and overcome barriers toworking relationships As in personalizing instruction approx-imating a good motivational t also requires matching capabilitiessuch as starting with fewer elements at sites at which resources arelimited and accounting for variability in stakeholdersrsquo competenceOver and over we nd too little attention is paid to these mattersThe result is failure to create an lsquolsquoenvironmentrsquorsquo that mobilizesdirects and then maintains stakeholder involvement

As with students the problem can be conceived as that of main-taining an appropriate match between the demands of the situationand individual motivation and capabilities In this respect we thinkthe construct of personalization o ers a concept around which toorganize thinking about facilitating change As stressed in the rstarticle in this issue personalization calls for systematically planningand implementing processes focused not only on knowledge andskills but on attitudes In particular it emphasizes the importance ofa primary and constant focus on ensuring positive attitudes Mobi-lization probably is best facilitated when procedures are perceived byindividuals as good ways to reach desired outcomes This requires

316 L Taylor et al

processes that can instigate and enhance an individualrsquos perceptionsof valued opportunities choice and control accomplishment andrelatedness to others Even if a task isnrsquot enjoyable expectation offeeling some sense of satisfaction related to process or outcome canbe a powerful intrinsic factor motivating individual behavior Taskpersistence for example can be facilitated by the expectation thatone will feel competent self-determining or more closely connectedto others From this perspective ensuring that individuals havevalued options a meaningful role in decision making feedback thatemphasizes progress toward desired outcomes and positive workingrelationships are among the most basic facilitation strategies(Adelman amp Taylor 1993b 1994 Deci amp Ryan 1985)

One other initial implementation problem that often arises is diffi-culty in establishing mechanisms to facilitate productive workingrelationships and identify and deal with problems quickly Forexample it is expected that change agents will encounter manyinstances of individual resistance and apathy interpersonal conictsand resentments (including lsquolsquous vs themrsquorsquo dynamics) rumors thatoveremphasize the negative and underestimate the positive and indi-viduals who are frequent faultnders Such problems seriouslyimpede e ective replication The roots of some of these problemsoften are present at a site prior to scale-up change simply o ers anew focus and perhaps magnies troubling matters Other problemsare a direct product of the activities and relationships that the scale-up process engenders Given the inevitability of such problems build-ing and maintaining working relationships need to be among themost basic concerns for those who have responsibility for scale-up Inparticular considerable attention must be paid to enhancing themotivational readiness and capability of those who are to worktogether and ensure there is an appropriate infrastructure to guideand support working relationships Proactively this requires problemprevention mechanisms that help create an atmosphere where defen-siveness is curtailed and positive rapport is engendered The point isto enhance attitudes knowledge and skills that foster interpersonalconnections and a sense of community Reactively the emphasis is onproblem solving resolving conict and providing ongoing support torebuild relationships Policies must encourage problem-solving-oriented critiques safeguards that protect those making changesappreciation for e ort and celebration of progress We nd thateveryone understands such matters but the culture at many schoolsites is more attuned to problem naming and analyzing than to antici-pating preventing and solving problems that arise around workingrelationships

Scaling up Reforms 317

Those responsible for systemic change need to spend as much timeas necessary ensuring that a schoolrsquos infrastructure is ready toprevent and ameliorate problems Special attention must be paid toensuring that problem solving mechanisms and communication pro-cesses are in place and properly sta ed and that stakeholders arewell informed about how to use the procedures Furthermore somestakeholders may have to be encouraged to interact in ways thatconvey genuine empathy warmth and mutual regard and respectwith a view to creating and maintaining a positive working climateand a psychological sense of community

At times we nd it necessary to target a specic problem anddesignated persons In some instances rather simple strategies aree ective For example most motivated individuals can be directlytaught ways to improve understanding and communication and avoidor resolve conicts that interfere with working relationships Inother instances however signicant remedial action is necessary aswhen overcoming barriers to a working relationship involvescountering negative attitudes Helpful in this regard are analysessuch as that by Sue and Zane (1987) that suggest how to demonstratethat something of value can be gained from individuals workingtogether and how to establish each participantrsquos credibility (eg bymaximizing task-focus and positive outcomes)

PHASE III INSTITUTIONALIZING THE PROTOTYPE

Maintaining and enhancing changes can be as difficult as makingthem in the rst place The history of education reform is one offailure to foster promising prototypes in substantive ways and overan extended period of times (Tyack amp Cuban 1995) Institutionalizinga prototype entails ensuring that the organization assumes long-termownership and that there is a blueprint for countering forces that canerode the changes Moreover institutionalization is more than atechnical process It requires assimilation of and ongoing adherenceto the values inherent in the prototypersquos underlying rationale Thefocus of course is not just on maintenance the point is to moveforward by enhancing productive changes and generating a sense ofrenewal as needed Critical in all this are specic plans that guar-antee ongoing and enhanced leadership and delineate ways in whichplanning implementation coordination and continuing educationmechanisms are maintained

Some Major Tasks

Whose responsibility is it to advocate for maintaining and evolving a

318 L Taylor et al

replicated prototype for improving literacy As problems arise whoseresponsibility is it to lead the way in resolving them Leadership isthe key here by both official leaders such as administrators mentorsta union chapter chairs and elected parent representatives aswell as natural leaders such as reading and writing teachers Obvi-ously official and natural leaders are not mutually exclusive groupsAt this phase both types of leadership are essential to ensure a broadenough base for ongoing advocacy problem solving enhancementand renewal Official leaders provide a legitimate power base asvarious interests compete for the organizationrsquos limited resourcesand they play a key role in ensuring that the contributions of naturalleaders are recognized and rewarded

Maintenance and enhancement require that the organizationrsquosgovernance body assumes ownership and program advocacy such astaking over the temporary steering grouprsquos functions addressingongoing policy and long-range planning concerns and maintainingnancial support The foundation for such ownership is laid duringthe readiness phase Each element becomes the organizationrsquos pro-perty as it is established during initial implementation The officialdeed of ownership is transferred as soon as the prototype is in place

Ownership however is no guarantee of institutionalizationVarious forces that can erode reforms are always at work Forinstance teams at a site experience turnover problems with commu-nication and sharing of resources are chronic competing interestsand the attractiveness of moving on to something new pull attentionand resources to other activity To minimize such problems stepsmust be taken to identify and solve them as quickly as is feasibleThis requires someone who has the time energy and expertise tomeet periodically with stakeholders to anticipate and amelioratethreats to a prototypersquos integrity

Over time mechanisms for planning implementation and coordi-nation are maintained by ensuring the activity is an official part ofthe infrastructure has appropriate leadership and is e ectively sup-ported Anyone who has worked on a school-based team to improve aliteracy program knows there must be a critical mass of teammembers so that the work load is manageable and a broad base ofinvolvement is ensured Also essential are adequate resources includ-ing time to learn the role and time to perform the functions reason-ably interesting tasks technical support for problem solvingrecognition and rewards for contributions immediate replacementwhen someone leaves continuing education to enhance team func-tioning and so forth Without serious attention to such matters theteamsrsquo morale and motivation will wane

Scaling up Reforms 319

Lessons Learned

Newly institutionalized approaches are seriously jeopardized in theabsence of dedicated ongoing capacity-building Of particular impor-tance are ways to rapidly and e ectively assimilate new arrivals at aschool (sta students families) This is a major concern at sites withconsiderable turnover or growth At such sites the majority of thoseinitially involved in implementing a new approach may be gonewithin a period of two to three years Whatever the mobility rate it isessential to design and maintain transition programs for newarrivals Initial welcoming and introductory orientations of coursemust be followed-up with ongoing support systems and intensivecapacity building related to understanding and valuing theapproaches the school has adopted We nd that all this is essentialnot only to maintain what has been adopted but also can contributeto establishing schools as caring environments

PHASE IV ONGOING EVOLUTION

Ongoing evolution of organizations and programs is the product ofe orts to account for accomplishments deal with changing times andconditions incorporate new knowledge and create a sense of renewalas the excitement of newness wears o and the demands of changesap energy As suggested already in part vigor and direction can bemaintained through continuing education especially exposure toideas that suggest a range of ways for evolving a program As thefollowing discussion indicates ongoing evolution also is fostered byevaluation designed to document accomplishments and provide feed-back designed to improve quality

Increased concern over accountability has advanced the wayevaluation is conceived (Posavac amp Carey 1989 Rossi amp Freeman1989 Scriven 1993 Sechrest amp Figueredo 1993 Shadish Jr Cook ampLeviton 1991 Stake 1967 1976 Stufflebeam amp Webster 1983 Weiss1995) At the same time social and political forces have literallyshaped the whole enterprise and in the process have narrowed theway professionals clients policymakers underwriters and thegeneral public think about program evaluation A prevailing cry isfor specic evidence of e ectiveness For schools this means imme-diate gains in literacy as measured by achievement tests Althoughunderstandable in light of the unlled promise of so many programsand the insatiable demands on limited public nances such simplisti-cally conceived accountability demands ignore the complexities ofdeveloping and scaling-up major reforms

320 L Taylor et al

Formative and Summative Evaluation

Evaluation of a prototype involves more than determining efficacyfor students Broadly stated it encompasses concerns about how toexpand the focus of evaluative research not only to contribute toimproving practice but also to aid in evolving practice and policy(General Accounting Office 1989 Lyon amp Moats 1997) To facilitateprogram development and organizational change the primary orien-tation for evaluation in the early phases is formative It is especiallyfocused on data gathering and analyses that can help improve pro-cedures Most of what is written about educational and psychosocialintervention however is oriented to summative evaluation and mea-suring outcomes for individuals such as improved reading achieve-ment scores Replicating approaches to improve literacy involve notonly changing individuals but changing organizations and systemsThus both individuals and systems must be evaluated

All this presumes appropriate mechanisms to provide and analyzeessential information To these ends a scale-up sta can help estab-lish an evaluation team and capacity building that prepares a schoolto conduct evaluation that enhances reforms The immediate focus ison successful program replication ultimately of course the empha-sis must be on student outcomes

Pursuing Results

Due to the increased interest in accountability many complex aimsare broken down into specic objectives Indeed short-range objec-tives stated in measurable terms generally assume a central role inplanning However short-range objectives are not ends in them-selves They are a small part of a particular goal and aim and some-times are prerequisites for moving on to a goal It is essential not tolose sight of the fact that many specic objectives are relativelysmall unrepresentative and often unimportant segments of the mostvalued aims that society has for its citizens and citizens have forthemselves

The problem is well exemplied by the narrow focus found inreviews analyses and reanalyses of data on early education (eg seeAlbee amp Gullotta 1997 Bond amp Compas 1989 Dryfoos 1990 Durlak1995 Elias 1997 Mitchell Seligson amp Marx 1989 Schorr 1988Slavin Karweit amp Madden 1989 Weissberg Gullotta HamptomRyan amp Adams 1997) As such work demonstrates overemphasis onevaluating the efficacy of underdeveloped prototypes draws resourcesaway from formative evaluation

Scaling up Reforms 321

With specic respect to scale-up the rst accomplishment is thereplication itself Have all facets been implemented How completelyhas each been implemented At how many locations The next set ofresults are any indications of progress for students such as improve-ments in attitudes toward school health attendance behavior andacademic achievement A nal set of evaluation concerns is thedegree to which student outcomes approximate societal standards

Lessons Learned

The process of evaluating results is costly in terms of nancial invest-ment the negative psychological impact on those evaluated and theways it can inappropriately reshape new approaches Cost-e ectiveoutcomes cannot be achieved in the absence of e ective prototypedevelopment and research Premature e orts to carry out com-prehensive summative evaluations clearly are not cost-e ective Anyreading and writing program will show poor results if it is evaluatedbefore teachers have mastered its application None of this of courseis an argument against evaluating results Rather it is meant tounderscore concerns and encourage greater attention to addressingthem

Once a prototype is established care must be taken to avoiddeveloping outcome evaluation as an adversarial process Due to thepolitical realities related to accountability one of the most per-plexing facets to negotiate is the time frame for summative evalu-ation The more complex the prototype the longer it takes and thecostlier it is to implement and evaluate Schools usually want quickprocesses and results and of course rarely can a ord costly innova-tions or lengthy di usion activity Compromises are inevitable butmust be arrived at with great care so as not to undermine the sub-stance of proposed changes

The psychology of evaluation suggests that an overemphasis onaccountability tends to produce negative reactions One possible wayto counter this may be to conceive evaluation as a way for everystakeholder to self-evaluate as a basis for quality improvement and asa way of getting credit for all that is accomplished Unfortunately asaccountability pressures increase we nd that replication of proto-types are guided more by what can be measured than by long-rangeaims That is demands for immediate accountability reshape prac-tices so that the emphasis shifts to immediate and readily measuredobjectives and away from fundamental purposes Over time this inap-propriately leads to radical revision of the underlying rationale for aprototype

322 L Taylor et al

CONCLUDING COMMENTS

Those who set out to change schools and schooling are confrontedwith two enormous tasks The rst is to develop prototypes Thesecond involves large-scale replication One without the other isinsufficient Yet considerably more attention is paid to developingand validating prototypes than to delineating and testing scale-upprocesses Clearly it is time to correct this deciency The ideas pre-sented in this article are meant to stimulate work on the problem andthereby to advance the cause of educational reform

Finally in fairness to those who labor for educational reform weall must remember that the quality of schooling family life and com-munity functioning spirals up or down as a function of the quality ofthe ongoing transactions among each Thus scale-up e orts relatedto educational reform must take place within the context of a poli-tical agenda that addresses ways to strengthen the family and com-munity infrastructure through strategies that enhance economicopportunity adult literacy and so forth What we need are policiesthat develop demonstrate and scale-up comprehensive multifacetedintegrated approaches that can e ectively address barriers to devel-opment learning and teaching

REFERENCES

Adelman H S (1993) School-linked mental health interventions Toward mechanismsfor service coordination and integration Journal of Community Psychology 21 309ndash319

Adelman H S (1996a) Restructuring education support services and integrating com-munity resources Beyond the full service school model School Psychology Review25 431ndash445

Adelman H S (1996b) Restructuring support services Toward a comprehensiveapproach Kent OH American School Health Association

Adelman H S amp Taylor L (1993a) Early Assistance for Students and FamiliesProgram Guidebook Los Angeles Early Assistance for Students and FamiliesProject Copies available from the School Mental Health Project Dept of Psychol-ogy UCLA

Adelman H S amp Taylor L (1993b) Learning problems and learning disabilities Moving forward Pacic Grove CA BrooksCole

Adelman H S amp Taylor L (1994) On understanding intervention in psychology andeducation Westport CT Praeger

Adelman H S amp Taylor L (1997a) Addressing barriers to learning Beyond school-linked services and full service schools American Journal of Orthopsychiatry 67408ndash421

Adelman H S amp Taylor L (1997b) Toward a scale-up model for replicating newapproaches to schooling Journal of Educational and Psychological Consultation 8197ndash230

Scaling up Reforms 307

literature claries the value of (a) a high level of policy and lead-ership commitment that is translated into an inspiring vision andappropriate resources (leadership space budget time) (b) incentivesfor change such as intrinsically valued outcomes expectations forsuccess recognitions rewards (c) procedural options that reectstakeholder strengths and from which those expected to implementchange can select strategies they see as workable (d) a willingness toestablish an infrastructure and processes that facilitate changee orts such as a governance mechanism that adopts strategies forimproving organizational health including one that enhances a senseof community (e) use of change agents who are perceived as pragma-tic by maintaining ideals while embracing practical solutions (f)accomplishing change in stages and with realistic timelines (g) pro-viding feedback on progress and (h) institutionalizing supportmechanisms to maintain and evolve changes and to generate periodicrenewal There is an extensive literature in this area (eg Argyris1993 Barth 1990 Bass 1997 Bass amp Avolio 1994 Connor amp Lake1988 Cunningham amp Gresso 1993 Donahoe 1993 Elmore amp Associ-ates 1990 Fullan amp Stiegelbauer 1991 Hatch 1998 Heller 1990Hollander amp O ermann 1990 House 1996 Lewis 1989 Lieberman ampMiller 1990 Maton amp Salem 1995 Miles amp Louis 1990 Murphy1991 Newmann 1993 Peterson McCarthey amp Elmore 1996 Repli-cation and Program Services Inc 1993 Sarason 1990 1996 Schle-chty 1990 Schmuck amp Runkel 1985 Smith amp OrsquoDay 1991 Spillane1998 Waterman 1987 Wehlage Smith amp Lipman 1992)

In terms of specic tasks associated with creating readiness therst involves disseminating the prototype and pursuing activities tobuild interest and consensus for change Decisions follow about spe-cic sites for replication Then steps are taken to negotiate a policyframework and agreements for engagement This is followed by activ-ity to modify the institutional infrastructure at chosen sites to t theprototype and address replication needs All these tasks should beaccomplished with a process that reects understanding of the natureof the organization and its stakeholders involves stakeholders inmaking substantive decisions and redesigning those mechanisms thatconstitute the organizational and programmatic infrastructure clari-es personal relevance when identifying the potential benets ofchange elicits genuine public statements of commitment and empo-wers and creates a sense of community

Creating a climate for change requires appreciation of the rolesplayed by vision and leadership for change policy direction supportsafeguards for risk-taking and infrastructure redesign Each of thesetopics is discussed briey below

308 L Taylor et al

Vision and Leadership

Any major reform begins with a vision of what a desired newapproach would look like and an understanding of how to facilitatenecessary changes One without the other is insufficient Leadershave a triple burden as they attempt to improve literacy the rst isto ensure that substantive organizational and programmaticrestructuring are considered the second is to build consensus forchange nally they must pursue e ective implementation includingspecic strategies for nancing establishing maintaining andenhancing productive changes

Examples of key objectives at this stage include clarifying poten-tial gains without creating unrealistic expectations delineating costswithout seriously dampening expectations about benets o eringincentives that mesh with intrinsic motives and conveying thedegree to which a prototype can be adapted while emphasizing thatcertain facets are essential and nonnegotiable A thread runningthrough all this is the need to stimulate increasing interest or motiva-tional readiness among a sufficient number of stakeholders Toclarify the point successful change at any level of educationrestructuring requires the committed involvement of a critical massof policy makers sta and parents Almost any promising idea orpractice for improving studentsrsquo reading and writing performancemay nd a receptive audience among a small group Many more indi-viduals however are likely to remain politely unresponsive and rel-uctant to make changes and some will be actively resistant Thusleaders are confronted with the task of shifting the attitudes of asignicant proportion of those who appear reluctant and resistant

The next step involves deciding about which sites to begin withCriteria for making such decisions try to balance immediate concernsabout a sitersquos current level of readiness (including analyses of poten-tial barriers) and the likelihood of success over the long run Forinstance in making initial judgements about the appropriateness of apotential site we gather information about How likely is it that acritical mass of decision makers will commit to allocating sufficientnances personnel time and space How likely is it that a criticalmass of stakeholders will develop sufficient motivational readinessand appropriate levels of competence With respect to the most inu-ential stakeholders will enough be supportive or at least sufficientlycommitted not to undermine the process Do enough youngsters at asite t the prole of students for whom the program model wasdesigned As these questions illustrate most initial selection criteriareect general considerations related to any di usion process More

Scaling up Reforms 309

specic criteria emerge during the negotiation process For examplea principal may be attracted by the idea of establishing a programthat brings in volunteer reading tutors but in subsequent discussionswith teachers union concerns may arise that require arbitration

Policy

Substantive restructuring is unlikely without the adoption of newpolicies at all relevant jurisdictional levels (Spillane 1998) More-over such policies must elevate desired reforms so that they are notseen simply as demonstrations pilot projects passing fads or supple-mentary e orts When reforms are not assigned a high priority theytend to be treated in a marginalized manner (Center for MentalHealth in Schools 1998) This continues to be the fate of programssuch as Head Start Even Start and many other approaches toenhancing school readiness and literacy Relatedly e orts must bemade to revoke policies that preserve an unsatisfactory status quo(see critique of remedial reading programs by Dudley-Marling ampMurphy 1997)

Lasting reform requires processes that ensure informed com-mitment ownership and on-going support on the part of policymakers This involves strategies to create interest and formalizeagreements about fundamental changes Local ownership is estab-lished through solid policy commitments well-designed infrastruc-ture mechanisms allocation of adequate resources (eg nancespersonnel space equipment) to operationalize the policy andrestructuring of time to ensure sta involvement in adapting theprototype to the setting We nd three steps are essential (a) build-ing on introductory presentations to provide indepth information andunderstanding as a basis for establishing consensus (b) negotiationof a policy framework and a set of agreements for engagementincluding a realistic budget and (c) informed and voluntary rati-cation of agreements by legitimate representatives of all major stake-holders

For any program there are principles components elements andstandards that dene its essence and thus must be agreed to as a rstcondition for engagement Equally important are fundamental scale-up considerations o -line nonnegotiable such as the need for tempo-rary mechanisms to facilitate change Once essentials are agreed onall other matters are negotiable

Informed commitment is strengthened and operationalized throughnegotiating formal agreements at each jurisdictional level and amongvarious stakeholders Policy statements articulate the commitment to

310 L Taylor et al

a programrsquos essence Memoranda of understanding and contractsspecify agreements about such matters as funding sources resourceappropriations personnel functions incentives and safeguards forrisk-taking stakeholder development immediate and long-term com-mitments and timelines accountability procedures and so forth

Scale-up is aided when the decision to proceed is ratied by sanc-tioned representatives of stakeholder groups Developing and nego-tiating policies contracts and other formal agreements is a complexbusiness We nd that addressing the many logistics and legalitiesrequires extensive involvement of a small number of authorized andwell-informed stakeholder representatives Thus in pursuing thesetasks our commitment to include everyone moves from a town hallapproach to a representative democratic process with enfranchisedrepresentatives reporting back frequently to their constituencies Atrst endorsement is in principle over time it is manifested throughsustained support When ratication reects e ective consensusbuilding scale-up e orts benet from a broad base of informed com-mitment ownership and active sponsorship These attributes areessential in ensuring requisite support and protections for those whomust bear the burden of learning new ways and who risk dips in per-formance and productivity while doing so

Redesigning Infrastructure

After agreements are ratied a scale-up team can begin its work(again see Figure 1) A central challenge at every jurisdictional levelis redesign of regular mechanisms and processes used to make andimplement decisions These modications ensure ownership supportparticipation and address specic concerns associated with scale-up

Five fundamental facets of the ongoing infrastructure of schoolsthat are the focus of redesign are (a) governance (b) planning andimplementation associated with specic organizational and programobjectives (c) coordination and integration to ensure cohesive func-tioning (d) daily leadership and (e) communication and informationmanagement A common example of the need for infrastructure modi-cation is seen in the trend to increase school stakeholdersrsquo collabo-ration participation and inuence One implication is thatgovernance mechanisms will be altered to redistribute power Amajor problem of course is how to empower additional stakeholdergroups without disempowering those who have essentialresponsibilities and abilities related to the educational enterprise Inaddition it is one thing to o er lsquolsquopartnershipsrsquorsquo to stakeholders suchas parents students sta and community agency representatives it

Scaling up Reforms 311

is another to create conditions that allow for e ective participationOne such condition involves translating capacity building activityinto comprehensive programs for stakeholder development

The necessity of all this can be appreciated by thinking aboutintroducing a comprehensive approach for improving student literacy(Stringeld Ross amp Smith 1996) Such approaches involve major sys-temic changes that encompass intensive partnerships with parents (ortheir surrogates) and with various entities in the community such aslibraries youth development programs businesses the faith com-munity and so forth Substantive partnerships require a true sharingof leadership blending of resources and leadership training for pro-fessionals and nonprofessionals alike In communities where manyparents have little or no connection to the school major outreache orts are inevitable prerequisites to increasing home involvement inschool reform Parent outreach of course has not been very suc-cessful in many neighborhoods Our experience suggests that a neces-sary rst step in most cases is to o er programs and services thatassist the family in meeting its most pressing needs Furthermorethere is the matter of building parent competence to deal with plan-ning reforms and restructuring schools and for low income familiesthere is a need to nd ways to pay parents for the time they devote toserving on governance and other committees

Time is one of the most critical elements determining the successof scale-up Even if a prototype doesnrsquot call for restructuring theschool day the scale-up process does Substantial blocks of time areneeded for stakeholder capacity building and for individual and col-lective planning (National Education Commission on Time andLearning 1994) Particularly critical is the need for freeing-up tea-chers to learn new approaches For example e orts to make impor-tant revisions in literacy programs seem consistently undermined bynot providing enough time during the school day for the mentoring ofteachers and by the difficulty of carving out sufficient time to teachparents how to help their children Clearly a nonnegotiable condi-tion for engagement is a realistic plan for ensuring time to plan andbuild capacity

Lessons Learned

Complex interventions of course are seldom implemented in a com-pletely planned and linear manner The many practical and unfore-seen events that arise require exible problem-solving Articulationof a scale-up model can guide planning but those facilitating the

312 L Taylor et al

process must be prepared to capitalize on every opportunity that canmove the process ahead

Among the most fundamental lessons learned in carrying outPhase 1 has been the tendency of all parties to set actions into motionwithout taking sufficient time to lay the foundation needed for sub-stantive change In marketing new ideas it is tempting to accentuatetheir promising attributes and minimize complications In negotiatingagreements policy makers at a school site frequently are askedsimply for a go-ahead rather than for their informed commitmentSometimes they assent mainly to get extra resources sometimes theyare motivated by a desire to be seen by constituents as doing some-thing to improve the school This all tends to produce pressures forpremature implementation that results in the form rather than thesubstance of change especially when administrators are under thegun of political accountability measures that make unrealisticdemands for quick and dramatic results in studentsrsquo reading scores

Although formulation of policy and related agreements take con-siderable time and other resources their importance cannot be over-emphasized Failure to establish and successfully maintainsubstantive reforms in schools probably is attributable in greatmeasure to proceeding without strong and clear policy support

Another unfortunate trend we have found is the omission of in-depth planning for ongoing capacity building for change agents andteam members Mechanisms function only as well as the personnelwho operate them Such personnel must be recruited and developed inways that ensure appropriate motivation and capability and suffi-cient time must be redeployed so they can learn and carry out newfunctions e ectively (Peterson McCarthey amp Elmore 1996) Allchanges require constant care and feeding Those who steer theprocess must be motivated and competent not just initially but overtime The complexity of systemic change requires close monitoring ofmechanisms and immediate follow-up to address problems In particu-lar it means providing continuous personalized guidance andsupport to enhance knowledge and skills and counter anxiety frus-tration and other stressors To these ends adequate resource supportmust be provided (time space materials equipment) opportunitiesmust be available for increasing ability and generating a sense ofrenewed mission and personnel turnover must be addressed quicklyAll stakeholders can benet from e orts designed to increase levelsof competence and enhance motivation for working together Suche orts encompass four stages of stakeholder development orienta-tion foundation-building capacity-building and continuing educa-tion

Scaling up Reforms 313

There is no simple solution to the chronic problem of providingtime for creating readiness building capacity and planning Indeedrestructuring time represents one of the most difficult scale-up prob-lems Examples of how the problem might be addressed includefreeing up sta by establishing opportunities for students to spendtime pursuing activities such as music art and sports with special-ists or supervised by aides and community volunteers Alternativelyschool might start later or end earlier on a given day As these exam-ples suggest any approach will be controversial but if the problem isnot addressed satisfactorily successful replication of comprehensiveprototypes is unlikely

PHASE II INITIAL IMPLEMENTATION OF APROTOTYPE

Initial implementation involves adapting and phasing-in a programwith well-designed guidance and support If there is anything certainabout e orts to replicate a prototype it is that the process is stress-ful Some of the stress arises from the nature of the program some isinherent in the process of organizational change Coalitions must bedeveloped new working relationships established disruptive rumorsand information overload countered and interpersonal conictsresolved Short-term frustrations must be kept in perspective vis a visthe reform vision To help deal with all this temporary mechanismsare added to the organizational infrastructure They include (a) asite-based steering mechanism to guide and support replication (b) achange agent from the scale-up team working with site stakeholderson a change team to facilitate coalition building problem solvingand conict resolution and (c) mentors and coaches to model andteach elements of the prototype These structures are created tofacilitate replication and some are assimilated into a sitersquos infra-structure at the end of the initial implementation phase to supportinstitutionalization and ongoing evolution

A scale-up team and steering group work at a site with the schoolrsquosleadership specic planning groups and other stakeholders to formu-late phase-in plans steer program development and generallyprovide guidance and support for change Two major facets of thiswork are delineating a sequence for introducing major program ele-ments and outlining strategies to facilitate implementation Particu-lar attention is given to how to start with special emphasis onspecifying structures and resources for guidance and support For

314 L Taylor et al

instance in restructuring to better address barriers to learning therst steps at a school site involve creating processes to map analyzecoordinate and redeploy existing resources Special change mecha-nisms such as an organization facilitator and a resource coordinatingteam are created to guide and support the activity (Adelman 19931996a 1996b Adelman amp Taylor 1997a 1997b 1998)

Throughout this phase formative evaluation procedures are estab-lished to provide feedback for program development As noted abovee ective e orts to lsquolsquoreinventrsquorsquo schools require ensuring that allinvolved have the time to develop and institutionalize a soundprogram and that they are not penalized for unavoidable missteps Asa prototype is phased-in evaluation must not be thought of in termsof accountability Major systemic changes to improve literacy cantake years to develop Outcome e ectiveness is demonstrated afterthe program is in place The purpose of evaluation at this stage is toguide revision and ne-tuning of processes Formative evaluationsgather and analyze information relevant to changes in planning pro-cesses governance structures and policies and resources they alsofocus on implementation strategies and barriers program organiz-ation and staffing and initial outcomes Are teachers mastering thenew curricula for literacy Is there increased student engagement inreading Are parents involved in supporting reading at home Ifthings are not progressing satisfactorily why not Whatrsquos the down-side of the new approach

Well-designed organizational support and guidance is needed toenhance productivity minimize problems and accommodate individ-ual di erences This involves various forms of capacity building andpersonalized day-by-day facilitation Intensive coaching with somefollow-up consultation for instance are key processes so are mentor-ship and technical assistance Continuing education provides a criti-cal vehicle for enhancing productive changes generating renewaland countering burnout As new stakeholders arrive technologicaltools can be particularly useful in helping them catch up All thisactivity not only builds capacity but can foster networking and otherforms of task-related social and personal support as well as providea wide range of enrichment opportunities that enhance morale

If the steps discussed to this point are done well a sound founda-tion for initial implementation should be in place This initial phase-in period can however consume considerable e ort create specialproblems and may yield a temporary drop in some performance indi-cators Good day-by-day facilitation aims at minimizing such negativeimpact by e ectively addressing stakeholder motivation and capabil-ity and overcoming barriers to productive working relationships

Scaling up Reforms 315

Lessons Learned

Failure to take sufficient time to create readiness (Phase 1) can resultin implementing the form rather than the substance of a prototypeFor example we nd that change agents frequently are sent intoschools before essential policy support is enacted and before schoolleaders have assimilated and decided to support reforms Teams areconvened to assist with reforms (plan coordinate develop newapproaches) but the absence of supportive policy means substantivechanges are not accomplished As a result the initial motivation ofmany key team members wanes and other counterproductivedynamics arise All of this seems inevitable when initial implementa-tion proceeds without adequate policy support

Even in situations where sufficient readiness is created difficultiesfrequently arise due to a failure to keep enough stakeholders consis-tently moving in the direction of desired outcomes Comprehensivechange is usually achieved only when fairly high levels of positiveenergy can be mobilized over extended periods of time among a criti-cal mass of stakeholders sustained energy is appropriately directedthe process is supported with ongoing and well-conceived capacitybuilding and individuals are not pushed beyond their capabilitiesAnd because low and negative motivation are related to resistance tochange and poor functioning matching motivation is a rst-orderconsideration That is scale-up e orts must use strategies designed tomobilize and maintain proactive e ort and overcome barriers toworking relationships As in personalizing instruction approx-imating a good motivational t also requires matching capabilitiessuch as starting with fewer elements at sites at which resources arelimited and accounting for variability in stakeholdersrsquo competenceOver and over we nd too little attention is paid to these mattersThe result is failure to create an lsquolsquoenvironmentrsquorsquo that mobilizesdirects and then maintains stakeholder involvement

As with students the problem can be conceived as that of main-taining an appropriate match between the demands of the situationand individual motivation and capabilities In this respect we thinkthe construct of personalization o ers a concept around which toorganize thinking about facilitating change As stressed in the rstarticle in this issue personalization calls for systematically planningand implementing processes focused not only on knowledge andskills but on attitudes In particular it emphasizes the importance ofa primary and constant focus on ensuring positive attitudes Mobi-lization probably is best facilitated when procedures are perceived byindividuals as good ways to reach desired outcomes This requires

316 L Taylor et al

processes that can instigate and enhance an individualrsquos perceptionsof valued opportunities choice and control accomplishment andrelatedness to others Even if a task isnrsquot enjoyable expectation offeeling some sense of satisfaction related to process or outcome canbe a powerful intrinsic factor motivating individual behavior Taskpersistence for example can be facilitated by the expectation thatone will feel competent self-determining or more closely connectedto others From this perspective ensuring that individuals havevalued options a meaningful role in decision making feedback thatemphasizes progress toward desired outcomes and positive workingrelationships are among the most basic facilitation strategies(Adelman amp Taylor 1993b 1994 Deci amp Ryan 1985)

One other initial implementation problem that often arises is diffi-culty in establishing mechanisms to facilitate productive workingrelationships and identify and deal with problems quickly Forexample it is expected that change agents will encounter manyinstances of individual resistance and apathy interpersonal conictsand resentments (including lsquolsquous vs themrsquorsquo dynamics) rumors thatoveremphasize the negative and underestimate the positive and indi-viduals who are frequent faultnders Such problems seriouslyimpede e ective replication The roots of some of these problemsoften are present at a site prior to scale-up change simply o ers anew focus and perhaps magnies troubling matters Other problemsare a direct product of the activities and relationships that the scale-up process engenders Given the inevitability of such problems build-ing and maintaining working relationships need to be among themost basic concerns for those who have responsibility for scale-up Inparticular considerable attention must be paid to enhancing themotivational readiness and capability of those who are to worktogether and ensure there is an appropriate infrastructure to guideand support working relationships Proactively this requires problemprevention mechanisms that help create an atmosphere where defen-siveness is curtailed and positive rapport is engendered The point isto enhance attitudes knowledge and skills that foster interpersonalconnections and a sense of community Reactively the emphasis is onproblem solving resolving conict and providing ongoing support torebuild relationships Policies must encourage problem-solving-oriented critiques safeguards that protect those making changesappreciation for e ort and celebration of progress We nd thateveryone understands such matters but the culture at many schoolsites is more attuned to problem naming and analyzing than to antici-pating preventing and solving problems that arise around workingrelationships

Scaling up Reforms 317

Those responsible for systemic change need to spend as much timeas necessary ensuring that a schoolrsquos infrastructure is ready toprevent and ameliorate problems Special attention must be paid toensuring that problem solving mechanisms and communication pro-cesses are in place and properly sta ed and that stakeholders arewell informed about how to use the procedures Furthermore somestakeholders may have to be encouraged to interact in ways thatconvey genuine empathy warmth and mutual regard and respectwith a view to creating and maintaining a positive working climateand a psychological sense of community

At times we nd it necessary to target a specic problem anddesignated persons In some instances rather simple strategies aree ective For example most motivated individuals can be directlytaught ways to improve understanding and communication and avoidor resolve conicts that interfere with working relationships Inother instances however signicant remedial action is necessary aswhen overcoming barriers to a working relationship involvescountering negative attitudes Helpful in this regard are analysessuch as that by Sue and Zane (1987) that suggest how to demonstratethat something of value can be gained from individuals workingtogether and how to establish each participantrsquos credibility (eg bymaximizing task-focus and positive outcomes)

PHASE III INSTITUTIONALIZING THE PROTOTYPE

Maintaining and enhancing changes can be as difficult as makingthem in the rst place The history of education reform is one offailure to foster promising prototypes in substantive ways and overan extended period of times (Tyack amp Cuban 1995) Institutionalizinga prototype entails ensuring that the organization assumes long-termownership and that there is a blueprint for countering forces that canerode the changes Moreover institutionalization is more than atechnical process It requires assimilation of and ongoing adherenceto the values inherent in the prototypersquos underlying rationale Thefocus of course is not just on maintenance the point is to moveforward by enhancing productive changes and generating a sense ofrenewal as needed Critical in all this are specic plans that guar-antee ongoing and enhanced leadership and delineate ways in whichplanning implementation coordination and continuing educationmechanisms are maintained

Some Major Tasks

Whose responsibility is it to advocate for maintaining and evolving a

318 L Taylor et al

replicated prototype for improving literacy As problems arise whoseresponsibility is it to lead the way in resolving them Leadership isthe key here by both official leaders such as administrators mentorsta union chapter chairs and elected parent representatives aswell as natural leaders such as reading and writing teachers Obvi-ously official and natural leaders are not mutually exclusive groupsAt this phase both types of leadership are essential to ensure a broadenough base for ongoing advocacy problem solving enhancementand renewal Official leaders provide a legitimate power base asvarious interests compete for the organizationrsquos limited resourcesand they play a key role in ensuring that the contributions of naturalleaders are recognized and rewarded

Maintenance and enhancement require that the organizationrsquosgovernance body assumes ownership and program advocacy such astaking over the temporary steering grouprsquos functions addressingongoing policy and long-range planning concerns and maintainingnancial support The foundation for such ownership is laid duringthe readiness phase Each element becomes the organizationrsquos pro-perty as it is established during initial implementation The officialdeed of ownership is transferred as soon as the prototype is in place

Ownership however is no guarantee of institutionalizationVarious forces that can erode reforms are always at work Forinstance teams at a site experience turnover problems with commu-nication and sharing of resources are chronic competing interestsand the attractiveness of moving on to something new pull attentionand resources to other activity To minimize such problems stepsmust be taken to identify and solve them as quickly as is feasibleThis requires someone who has the time energy and expertise tomeet periodically with stakeholders to anticipate and amelioratethreats to a prototypersquos integrity

Over time mechanisms for planning implementation and coordi-nation are maintained by ensuring the activity is an official part ofthe infrastructure has appropriate leadership and is e ectively sup-ported Anyone who has worked on a school-based team to improve aliteracy program knows there must be a critical mass of teammembers so that the work load is manageable and a broad base ofinvolvement is ensured Also essential are adequate resources includ-ing time to learn the role and time to perform the functions reason-ably interesting tasks technical support for problem solvingrecognition and rewards for contributions immediate replacementwhen someone leaves continuing education to enhance team func-tioning and so forth Without serious attention to such matters theteamsrsquo morale and motivation will wane

Scaling up Reforms 319

Lessons Learned

Newly institutionalized approaches are seriously jeopardized in theabsence of dedicated ongoing capacity-building Of particular impor-tance are ways to rapidly and e ectively assimilate new arrivals at aschool (sta students families) This is a major concern at sites withconsiderable turnover or growth At such sites the majority of thoseinitially involved in implementing a new approach may be gonewithin a period of two to three years Whatever the mobility rate it isessential to design and maintain transition programs for newarrivals Initial welcoming and introductory orientations of coursemust be followed-up with ongoing support systems and intensivecapacity building related to understanding and valuing theapproaches the school has adopted We nd that all this is essentialnot only to maintain what has been adopted but also can contributeto establishing schools as caring environments

PHASE IV ONGOING EVOLUTION

Ongoing evolution of organizations and programs is the product ofe orts to account for accomplishments deal with changing times andconditions incorporate new knowledge and create a sense of renewalas the excitement of newness wears o and the demands of changesap energy As suggested already in part vigor and direction can bemaintained through continuing education especially exposure toideas that suggest a range of ways for evolving a program As thefollowing discussion indicates ongoing evolution also is fostered byevaluation designed to document accomplishments and provide feed-back designed to improve quality

Increased concern over accountability has advanced the wayevaluation is conceived (Posavac amp Carey 1989 Rossi amp Freeman1989 Scriven 1993 Sechrest amp Figueredo 1993 Shadish Jr Cook ampLeviton 1991 Stake 1967 1976 Stufflebeam amp Webster 1983 Weiss1995) At the same time social and political forces have literallyshaped the whole enterprise and in the process have narrowed theway professionals clients policymakers underwriters and thegeneral public think about program evaluation A prevailing cry isfor specic evidence of e ectiveness For schools this means imme-diate gains in literacy as measured by achievement tests Althoughunderstandable in light of the unlled promise of so many programsand the insatiable demands on limited public nances such simplisti-cally conceived accountability demands ignore the complexities ofdeveloping and scaling-up major reforms

320 L Taylor et al

Formative and Summative Evaluation

Evaluation of a prototype involves more than determining efficacyfor students Broadly stated it encompasses concerns about how toexpand the focus of evaluative research not only to contribute toimproving practice but also to aid in evolving practice and policy(General Accounting Office 1989 Lyon amp Moats 1997) To facilitateprogram development and organizational change the primary orien-tation for evaluation in the early phases is formative It is especiallyfocused on data gathering and analyses that can help improve pro-cedures Most of what is written about educational and psychosocialintervention however is oriented to summative evaluation and mea-suring outcomes for individuals such as improved reading achieve-ment scores Replicating approaches to improve literacy involve notonly changing individuals but changing organizations and systemsThus both individuals and systems must be evaluated

All this presumes appropriate mechanisms to provide and analyzeessential information To these ends a scale-up sta can help estab-lish an evaluation team and capacity building that prepares a schoolto conduct evaluation that enhances reforms The immediate focus ison successful program replication ultimately of course the empha-sis must be on student outcomes

Pursuing Results

Due to the increased interest in accountability many complex aimsare broken down into specic objectives Indeed short-range objec-tives stated in measurable terms generally assume a central role inplanning However short-range objectives are not ends in them-selves They are a small part of a particular goal and aim and some-times are prerequisites for moving on to a goal It is essential not tolose sight of the fact that many specic objectives are relativelysmall unrepresentative and often unimportant segments of the mostvalued aims that society has for its citizens and citizens have forthemselves

The problem is well exemplied by the narrow focus found inreviews analyses and reanalyses of data on early education (eg seeAlbee amp Gullotta 1997 Bond amp Compas 1989 Dryfoos 1990 Durlak1995 Elias 1997 Mitchell Seligson amp Marx 1989 Schorr 1988Slavin Karweit amp Madden 1989 Weissberg Gullotta HamptomRyan amp Adams 1997) As such work demonstrates overemphasis onevaluating the efficacy of underdeveloped prototypes draws resourcesaway from formative evaluation

Scaling up Reforms 321

With specic respect to scale-up the rst accomplishment is thereplication itself Have all facets been implemented How completelyhas each been implemented At how many locations The next set ofresults are any indications of progress for students such as improve-ments in attitudes toward school health attendance behavior andacademic achievement A nal set of evaluation concerns is thedegree to which student outcomes approximate societal standards

Lessons Learned

The process of evaluating results is costly in terms of nancial invest-ment the negative psychological impact on those evaluated and theways it can inappropriately reshape new approaches Cost-e ectiveoutcomes cannot be achieved in the absence of e ective prototypedevelopment and research Premature e orts to carry out com-prehensive summative evaluations clearly are not cost-e ective Anyreading and writing program will show poor results if it is evaluatedbefore teachers have mastered its application None of this of courseis an argument against evaluating results Rather it is meant tounderscore concerns and encourage greater attention to addressingthem

Once a prototype is established care must be taken to avoiddeveloping outcome evaluation as an adversarial process Due to thepolitical realities related to accountability one of the most per-plexing facets to negotiate is the time frame for summative evalu-ation The more complex the prototype the longer it takes and thecostlier it is to implement and evaluate Schools usually want quickprocesses and results and of course rarely can a ord costly innova-tions or lengthy di usion activity Compromises are inevitable butmust be arrived at with great care so as not to undermine the sub-stance of proposed changes

The psychology of evaluation suggests that an overemphasis onaccountability tends to produce negative reactions One possible wayto counter this may be to conceive evaluation as a way for everystakeholder to self-evaluate as a basis for quality improvement and asa way of getting credit for all that is accomplished Unfortunately asaccountability pressures increase we nd that replication of proto-types are guided more by what can be measured than by long-rangeaims That is demands for immediate accountability reshape prac-tices so that the emphasis shifts to immediate and readily measuredobjectives and away from fundamental purposes Over time this inap-propriately leads to radical revision of the underlying rationale for aprototype

322 L Taylor et al

CONCLUDING COMMENTS

Those who set out to change schools and schooling are confrontedwith two enormous tasks The rst is to develop prototypes Thesecond involves large-scale replication One without the other isinsufficient Yet considerably more attention is paid to developingand validating prototypes than to delineating and testing scale-upprocesses Clearly it is time to correct this deciency The ideas pre-sented in this article are meant to stimulate work on the problem andthereby to advance the cause of educational reform

Finally in fairness to those who labor for educational reform weall must remember that the quality of schooling family life and com-munity functioning spirals up or down as a function of the quality ofthe ongoing transactions among each Thus scale-up e orts relatedto educational reform must take place within the context of a poli-tical agenda that addresses ways to strengthen the family and com-munity infrastructure through strategies that enhance economicopportunity adult literacy and so forth What we need are policiesthat develop demonstrate and scale-up comprehensive multifacetedintegrated approaches that can e ectively address barriers to devel-opment learning and teaching

REFERENCES

Adelman H S (1993) School-linked mental health interventions Toward mechanismsfor service coordination and integration Journal of Community Psychology 21 309ndash319

Adelman H S (1996a) Restructuring education support services and integrating com-munity resources Beyond the full service school model School Psychology Review25 431ndash445

Adelman H S (1996b) Restructuring support services Toward a comprehensiveapproach Kent OH American School Health Association

Adelman H S amp Taylor L (1993a) Early Assistance for Students and FamiliesProgram Guidebook Los Angeles Early Assistance for Students and FamiliesProject Copies available from the School Mental Health Project Dept of Psychol-ogy UCLA

Adelman H S amp Taylor L (1993b) Learning problems and learning disabilities Moving forward Pacic Grove CA BrooksCole

Adelman H S amp Taylor L (1994) On understanding intervention in psychology andeducation Westport CT Praeger

Adelman H S amp Taylor L (1997a) Addressing barriers to learning Beyond school-linked services and full service schools American Journal of Orthopsychiatry 67408ndash421

Adelman H S amp Taylor L (1997b) Toward a scale-up model for replicating newapproaches to schooling Journal of Educational and Psychological Consultation 8197ndash230

308 L Taylor et al

Vision and Leadership

Any major reform begins with a vision of what a desired newapproach would look like and an understanding of how to facilitatenecessary changes One without the other is insufficient Leadershave a triple burden as they attempt to improve literacy the rst isto ensure that substantive organizational and programmaticrestructuring are considered the second is to build consensus forchange nally they must pursue e ective implementation includingspecic strategies for nancing establishing maintaining andenhancing productive changes

Examples of key objectives at this stage include clarifying poten-tial gains without creating unrealistic expectations delineating costswithout seriously dampening expectations about benets o eringincentives that mesh with intrinsic motives and conveying thedegree to which a prototype can be adapted while emphasizing thatcertain facets are essential and nonnegotiable A thread runningthrough all this is the need to stimulate increasing interest or motiva-tional readiness among a sufficient number of stakeholders Toclarify the point successful change at any level of educationrestructuring requires the committed involvement of a critical massof policy makers sta and parents Almost any promising idea orpractice for improving studentsrsquo reading and writing performancemay nd a receptive audience among a small group Many more indi-viduals however are likely to remain politely unresponsive and rel-uctant to make changes and some will be actively resistant Thusleaders are confronted with the task of shifting the attitudes of asignicant proportion of those who appear reluctant and resistant

The next step involves deciding about which sites to begin withCriteria for making such decisions try to balance immediate concernsabout a sitersquos current level of readiness (including analyses of poten-tial barriers) and the likelihood of success over the long run Forinstance in making initial judgements about the appropriateness of apotential site we gather information about How likely is it that acritical mass of decision makers will commit to allocating sufficientnances personnel time and space How likely is it that a criticalmass of stakeholders will develop sufficient motivational readinessand appropriate levels of competence With respect to the most inu-ential stakeholders will enough be supportive or at least sufficientlycommitted not to undermine the process Do enough youngsters at asite t the prole of students for whom the program model wasdesigned As these questions illustrate most initial selection criteriareect general considerations related to any di usion process More

Scaling up Reforms 309

specic criteria emerge during the negotiation process For examplea principal may be attracted by the idea of establishing a programthat brings in volunteer reading tutors but in subsequent discussionswith teachers union concerns may arise that require arbitration

Policy

Substantive restructuring is unlikely without the adoption of newpolicies at all relevant jurisdictional levels (Spillane 1998) More-over such policies must elevate desired reforms so that they are notseen simply as demonstrations pilot projects passing fads or supple-mentary e orts When reforms are not assigned a high priority theytend to be treated in a marginalized manner (Center for MentalHealth in Schools 1998) This continues to be the fate of programssuch as Head Start Even Start and many other approaches toenhancing school readiness and literacy Relatedly e orts must bemade to revoke policies that preserve an unsatisfactory status quo(see critique of remedial reading programs by Dudley-Marling ampMurphy 1997)

Lasting reform requires processes that ensure informed com-mitment ownership and on-going support on the part of policymakers This involves strategies to create interest and formalizeagreements about fundamental changes Local ownership is estab-lished through solid policy commitments well-designed infrastruc-ture mechanisms allocation of adequate resources (eg nancespersonnel space equipment) to operationalize the policy andrestructuring of time to ensure sta involvement in adapting theprototype to the setting We nd three steps are essential (a) build-ing on introductory presentations to provide indepth information andunderstanding as a basis for establishing consensus (b) negotiationof a policy framework and a set of agreements for engagementincluding a realistic budget and (c) informed and voluntary rati-cation of agreements by legitimate representatives of all major stake-holders

For any program there are principles components elements andstandards that dene its essence and thus must be agreed to as a rstcondition for engagement Equally important are fundamental scale-up considerations o -line nonnegotiable such as the need for tempo-rary mechanisms to facilitate change Once essentials are agreed onall other matters are negotiable

Informed commitment is strengthened and operationalized throughnegotiating formal agreements at each jurisdictional level and amongvarious stakeholders Policy statements articulate the commitment to

310 L Taylor et al

a programrsquos essence Memoranda of understanding and contractsspecify agreements about such matters as funding sources resourceappropriations personnel functions incentives and safeguards forrisk-taking stakeholder development immediate and long-term com-mitments and timelines accountability procedures and so forth

Scale-up is aided when the decision to proceed is ratied by sanc-tioned representatives of stakeholder groups Developing and nego-tiating policies contracts and other formal agreements is a complexbusiness We nd that addressing the many logistics and legalitiesrequires extensive involvement of a small number of authorized andwell-informed stakeholder representatives Thus in pursuing thesetasks our commitment to include everyone moves from a town hallapproach to a representative democratic process with enfranchisedrepresentatives reporting back frequently to their constituencies Atrst endorsement is in principle over time it is manifested throughsustained support When ratication reects e ective consensusbuilding scale-up e orts benet from a broad base of informed com-mitment ownership and active sponsorship These attributes areessential in ensuring requisite support and protections for those whomust bear the burden of learning new ways and who risk dips in per-formance and productivity while doing so

Redesigning Infrastructure

After agreements are ratied a scale-up team can begin its work(again see Figure 1) A central challenge at every jurisdictional levelis redesign of regular mechanisms and processes used to make andimplement decisions These modications ensure ownership supportparticipation and address specic concerns associated with scale-up

Five fundamental facets of the ongoing infrastructure of schoolsthat are the focus of redesign are (a) governance (b) planning andimplementation associated with specic organizational and programobjectives (c) coordination and integration to ensure cohesive func-tioning (d) daily leadership and (e) communication and informationmanagement A common example of the need for infrastructure modi-cation is seen in the trend to increase school stakeholdersrsquo collabo-ration participation and inuence One implication is thatgovernance mechanisms will be altered to redistribute power Amajor problem of course is how to empower additional stakeholdergroups without disempowering those who have essentialresponsibilities and abilities related to the educational enterprise Inaddition it is one thing to o er lsquolsquopartnershipsrsquorsquo to stakeholders suchas parents students sta and community agency representatives it

Scaling up Reforms 311

is another to create conditions that allow for e ective participationOne such condition involves translating capacity building activityinto comprehensive programs for stakeholder development

The necessity of all this can be appreciated by thinking aboutintroducing a comprehensive approach for improving student literacy(Stringeld Ross amp Smith 1996) Such approaches involve major sys-temic changes that encompass intensive partnerships with parents (ortheir surrogates) and with various entities in the community such aslibraries youth development programs businesses the faith com-munity and so forth Substantive partnerships require a true sharingof leadership blending of resources and leadership training for pro-fessionals and nonprofessionals alike In communities where manyparents have little or no connection to the school major outreache orts are inevitable prerequisites to increasing home involvement inschool reform Parent outreach of course has not been very suc-cessful in many neighborhoods Our experience suggests that a neces-sary rst step in most cases is to o er programs and services thatassist the family in meeting its most pressing needs Furthermorethere is the matter of building parent competence to deal with plan-ning reforms and restructuring schools and for low income familiesthere is a need to nd ways to pay parents for the time they devote toserving on governance and other committees

Time is one of the most critical elements determining the successof scale-up Even if a prototype doesnrsquot call for restructuring theschool day the scale-up process does Substantial blocks of time areneeded for stakeholder capacity building and for individual and col-lective planning (National Education Commission on Time andLearning 1994) Particularly critical is the need for freeing-up tea-chers to learn new approaches For example e orts to make impor-tant revisions in literacy programs seem consistently undermined bynot providing enough time during the school day for the mentoring ofteachers and by the difficulty of carving out sufficient time to teachparents how to help their children Clearly a nonnegotiable condi-tion for engagement is a realistic plan for ensuring time to plan andbuild capacity

Lessons Learned

Complex interventions of course are seldom implemented in a com-pletely planned and linear manner The many practical and unfore-seen events that arise require exible problem-solving Articulationof a scale-up model can guide planning but those facilitating the

312 L Taylor et al

process must be prepared to capitalize on every opportunity that canmove the process ahead

Among the most fundamental lessons learned in carrying outPhase 1 has been the tendency of all parties to set actions into motionwithout taking sufficient time to lay the foundation needed for sub-stantive change In marketing new ideas it is tempting to accentuatetheir promising attributes and minimize complications In negotiatingagreements policy makers at a school site frequently are askedsimply for a go-ahead rather than for their informed commitmentSometimes they assent mainly to get extra resources sometimes theyare motivated by a desire to be seen by constituents as doing some-thing to improve the school This all tends to produce pressures forpremature implementation that results in the form rather than thesubstance of change especially when administrators are under thegun of political accountability measures that make unrealisticdemands for quick and dramatic results in studentsrsquo reading scores

Although formulation of policy and related agreements take con-siderable time and other resources their importance cannot be over-emphasized Failure to establish and successfully maintainsubstantive reforms in schools probably is attributable in greatmeasure to proceeding without strong and clear policy support

Another unfortunate trend we have found is the omission of in-depth planning for ongoing capacity building for change agents andteam members Mechanisms function only as well as the personnelwho operate them Such personnel must be recruited and developed inways that ensure appropriate motivation and capability and suffi-cient time must be redeployed so they can learn and carry out newfunctions e ectively (Peterson McCarthey amp Elmore 1996) Allchanges require constant care and feeding Those who steer theprocess must be motivated and competent not just initially but overtime The complexity of systemic change requires close monitoring ofmechanisms and immediate follow-up to address problems In particu-lar it means providing continuous personalized guidance andsupport to enhance knowledge and skills and counter anxiety frus-tration and other stressors To these ends adequate resource supportmust be provided (time space materials equipment) opportunitiesmust be available for increasing ability and generating a sense ofrenewed mission and personnel turnover must be addressed quicklyAll stakeholders can benet from e orts designed to increase levelsof competence and enhance motivation for working together Suche orts encompass four stages of stakeholder development orienta-tion foundation-building capacity-building and continuing educa-tion

Scaling up Reforms 313

There is no simple solution to the chronic problem of providingtime for creating readiness building capacity and planning Indeedrestructuring time represents one of the most difficult scale-up prob-lems Examples of how the problem might be addressed includefreeing up sta by establishing opportunities for students to spendtime pursuing activities such as music art and sports with special-ists or supervised by aides and community volunteers Alternativelyschool might start later or end earlier on a given day As these exam-ples suggest any approach will be controversial but if the problem isnot addressed satisfactorily successful replication of comprehensiveprototypes is unlikely

PHASE II INITIAL IMPLEMENTATION OF APROTOTYPE

Initial implementation involves adapting and phasing-in a programwith well-designed guidance and support If there is anything certainabout e orts to replicate a prototype it is that the process is stress-ful Some of the stress arises from the nature of the program some isinherent in the process of organizational change Coalitions must bedeveloped new working relationships established disruptive rumorsand information overload countered and interpersonal conictsresolved Short-term frustrations must be kept in perspective vis a visthe reform vision To help deal with all this temporary mechanismsare added to the organizational infrastructure They include (a) asite-based steering mechanism to guide and support replication (b) achange agent from the scale-up team working with site stakeholderson a change team to facilitate coalition building problem solvingand conict resolution and (c) mentors and coaches to model andteach elements of the prototype These structures are created tofacilitate replication and some are assimilated into a sitersquos infra-structure at the end of the initial implementation phase to supportinstitutionalization and ongoing evolution

A scale-up team and steering group work at a site with the schoolrsquosleadership specic planning groups and other stakeholders to formu-late phase-in plans steer program development and generallyprovide guidance and support for change Two major facets of thiswork are delineating a sequence for introducing major program ele-ments and outlining strategies to facilitate implementation Particu-lar attention is given to how to start with special emphasis onspecifying structures and resources for guidance and support For

314 L Taylor et al

instance in restructuring to better address barriers to learning therst steps at a school site involve creating processes to map analyzecoordinate and redeploy existing resources Special change mecha-nisms such as an organization facilitator and a resource coordinatingteam are created to guide and support the activity (Adelman 19931996a 1996b Adelman amp Taylor 1997a 1997b 1998)

Throughout this phase formative evaluation procedures are estab-lished to provide feedback for program development As noted abovee ective e orts to lsquolsquoreinventrsquorsquo schools require ensuring that allinvolved have the time to develop and institutionalize a soundprogram and that they are not penalized for unavoidable missteps Asa prototype is phased-in evaluation must not be thought of in termsof accountability Major systemic changes to improve literacy cantake years to develop Outcome e ectiveness is demonstrated afterthe program is in place The purpose of evaluation at this stage is toguide revision and ne-tuning of processes Formative evaluationsgather and analyze information relevant to changes in planning pro-cesses governance structures and policies and resources they alsofocus on implementation strategies and barriers program organiz-ation and staffing and initial outcomes Are teachers mastering thenew curricula for literacy Is there increased student engagement inreading Are parents involved in supporting reading at home Ifthings are not progressing satisfactorily why not Whatrsquos the down-side of the new approach

Well-designed organizational support and guidance is needed toenhance productivity minimize problems and accommodate individ-ual di erences This involves various forms of capacity building andpersonalized day-by-day facilitation Intensive coaching with somefollow-up consultation for instance are key processes so are mentor-ship and technical assistance Continuing education provides a criti-cal vehicle for enhancing productive changes generating renewaland countering burnout As new stakeholders arrive technologicaltools can be particularly useful in helping them catch up All thisactivity not only builds capacity but can foster networking and otherforms of task-related social and personal support as well as providea wide range of enrichment opportunities that enhance morale

If the steps discussed to this point are done well a sound founda-tion for initial implementation should be in place This initial phase-in period can however consume considerable e ort create specialproblems and may yield a temporary drop in some performance indi-cators Good day-by-day facilitation aims at minimizing such negativeimpact by e ectively addressing stakeholder motivation and capabil-ity and overcoming barriers to productive working relationships

Scaling up Reforms 315

Lessons Learned

Failure to take sufficient time to create readiness (Phase 1) can resultin implementing the form rather than the substance of a prototypeFor example we nd that change agents frequently are sent intoschools before essential policy support is enacted and before schoolleaders have assimilated and decided to support reforms Teams areconvened to assist with reforms (plan coordinate develop newapproaches) but the absence of supportive policy means substantivechanges are not accomplished As a result the initial motivation ofmany key team members wanes and other counterproductivedynamics arise All of this seems inevitable when initial implementa-tion proceeds without adequate policy support

Even in situations where sufficient readiness is created difficultiesfrequently arise due to a failure to keep enough stakeholders consis-tently moving in the direction of desired outcomes Comprehensivechange is usually achieved only when fairly high levels of positiveenergy can be mobilized over extended periods of time among a criti-cal mass of stakeholders sustained energy is appropriately directedthe process is supported with ongoing and well-conceived capacitybuilding and individuals are not pushed beyond their capabilitiesAnd because low and negative motivation are related to resistance tochange and poor functioning matching motivation is a rst-orderconsideration That is scale-up e orts must use strategies designed tomobilize and maintain proactive e ort and overcome barriers toworking relationships As in personalizing instruction approx-imating a good motivational t also requires matching capabilitiessuch as starting with fewer elements at sites at which resources arelimited and accounting for variability in stakeholdersrsquo competenceOver and over we nd too little attention is paid to these mattersThe result is failure to create an lsquolsquoenvironmentrsquorsquo that mobilizesdirects and then maintains stakeholder involvement

As with students the problem can be conceived as that of main-taining an appropriate match between the demands of the situationand individual motivation and capabilities In this respect we thinkthe construct of personalization o ers a concept around which toorganize thinking about facilitating change As stressed in the rstarticle in this issue personalization calls for systematically planningand implementing processes focused not only on knowledge andskills but on attitudes In particular it emphasizes the importance ofa primary and constant focus on ensuring positive attitudes Mobi-lization probably is best facilitated when procedures are perceived byindividuals as good ways to reach desired outcomes This requires

316 L Taylor et al

processes that can instigate and enhance an individualrsquos perceptionsof valued opportunities choice and control accomplishment andrelatedness to others Even if a task isnrsquot enjoyable expectation offeeling some sense of satisfaction related to process or outcome canbe a powerful intrinsic factor motivating individual behavior Taskpersistence for example can be facilitated by the expectation thatone will feel competent self-determining or more closely connectedto others From this perspective ensuring that individuals havevalued options a meaningful role in decision making feedback thatemphasizes progress toward desired outcomes and positive workingrelationships are among the most basic facilitation strategies(Adelman amp Taylor 1993b 1994 Deci amp Ryan 1985)

One other initial implementation problem that often arises is diffi-culty in establishing mechanisms to facilitate productive workingrelationships and identify and deal with problems quickly Forexample it is expected that change agents will encounter manyinstances of individual resistance and apathy interpersonal conictsand resentments (including lsquolsquous vs themrsquorsquo dynamics) rumors thatoveremphasize the negative and underestimate the positive and indi-viduals who are frequent faultnders Such problems seriouslyimpede e ective replication The roots of some of these problemsoften are present at a site prior to scale-up change simply o ers anew focus and perhaps magnies troubling matters Other problemsare a direct product of the activities and relationships that the scale-up process engenders Given the inevitability of such problems build-ing and maintaining working relationships need to be among themost basic concerns for those who have responsibility for scale-up Inparticular considerable attention must be paid to enhancing themotivational readiness and capability of those who are to worktogether and ensure there is an appropriate infrastructure to guideand support working relationships Proactively this requires problemprevention mechanisms that help create an atmosphere where defen-siveness is curtailed and positive rapport is engendered The point isto enhance attitudes knowledge and skills that foster interpersonalconnections and a sense of community Reactively the emphasis is onproblem solving resolving conict and providing ongoing support torebuild relationships Policies must encourage problem-solving-oriented critiques safeguards that protect those making changesappreciation for e ort and celebration of progress We nd thateveryone understands such matters but the culture at many schoolsites is more attuned to problem naming and analyzing than to antici-pating preventing and solving problems that arise around workingrelationships

Scaling up Reforms 317

Those responsible for systemic change need to spend as much timeas necessary ensuring that a schoolrsquos infrastructure is ready toprevent and ameliorate problems Special attention must be paid toensuring that problem solving mechanisms and communication pro-cesses are in place and properly sta ed and that stakeholders arewell informed about how to use the procedures Furthermore somestakeholders may have to be encouraged to interact in ways thatconvey genuine empathy warmth and mutual regard and respectwith a view to creating and maintaining a positive working climateand a psychological sense of community

At times we nd it necessary to target a specic problem anddesignated persons In some instances rather simple strategies aree ective For example most motivated individuals can be directlytaught ways to improve understanding and communication and avoidor resolve conicts that interfere with working relationships Inother instances however signicant remedial action is necessary aswhen overcoming barriers to a working relationship involvescountering negative attitudes Helpful in this regard are analysessuch as that by Sue and Zane (1987) that suggest how to demonstratethat something of value can be gained from individuals workingtogether and how to establish each participantrsquos credibility (eg bymaximizing task-focus and positive outcomes)

PHASE III INSTITUTIONALIZING THE PROTOTYPE

Maintaining and enhancing changes can be as difficult as makingthem in the rst place The history of education reform is one offailure to foster promising prototypes in substantive ways and overan extended period of times (Tyack amp Cuban 1995) Institutionalizinga prototype entails ensuring that the organization assumes long-termownership and that there is a blueprint for countering forces that canerode the changes Moreover institutionalization is more than atechnical process It requires assimilation of and ongoing adherenceto the values inherent in the prototypersquos underlying rationale Thefocus of course is not just on maintenance the point is to moveforward by enhancing productive changes and generating a sense ofrenewal as needed Critical in all this are specic plans that guar-antee ongoing and enhanced leadership and delineate ways in whichplanning implementation coordination and continuing educationmechanisms are maintained

Some Major Tasks

Whose responsibility is it to advocate for maintaining and evolving a

318 L Taylor et al

replicated prototype for improving literacy As problems arise whoseresponsibility is it to lead the way in resolving them Leadership isthe key here by both official leaders such as administrators mentorsta union chapter chairs and elected parent representatives aswell as natural leaders such as reading and writing teachers Obvi-ously official and natural leaders are not mutually exclusive groupsAt this phase both types of leadership are essential to ensure a broadenough base for ongoing advocacy problem solving enhancementand renewal Official leaders provide a legitimate power base asvarious interests compete for the organizationrsquos limited resourcesand they play a key role in ensuring that the contributions of naturalleaders are recognized and rewarded

Maintenance and enhancement require that the organizationrsquosgovernance body assumes ownership and program advocacy such astaking over the temporary steering grouprsquos functions addressingongoing policy and long-range planning concerns and maintainingnancial support The foundation for such ownership is laid duringthe readiness phase Each element becomes the organizationrsquos pro-perty as it is established during initial implementation The officialdeed of ownership is transferred as soon as the prototype is in place

Ownership however is no guarantee of institutionalizationVarious forces that can erode reforms are always at work Forinstance teams at a site experience turnover problems with commu-nication and sharing of resources are chronic competing interestsand the attractiveness of moving on to something new pull attentionand resources to other activity To minimize such problems stepsmust be taken to identify and solve them as quickly as is feasibleThis requires someone who has the time energy and expertise tomeet periodically with stakeholders to anticipate and amelioratethreats to a prototypersquos integrity

Over time mechanisms for planning implementation and coordi-nation are maintained by ensuring the activity is an official part ofthe infrastructure has appropriate leadership and is e ectively sup-ported Anyone who has worked on a school-based team to improve aliteracy program knows there must be a critical mass of teammembers so that the work load is manageable and a broad base ofinvolvement is ensured Also essential are adequate resources includ-ing time to learn the role and time to perform the functions reason-ably interesting tasks technical support for problem solvingrecognition and rewards for contributions immediate replacementwhen someone leaves continuing education to enhance team func-tioning and so forth Without serious attention to such matters theteamsrsquo morale and motivation will wane

Scaling up Reforms 319

Lessons Learned

Newly institutionalized approaches are seriously jeopardized in theabsence of dedicated ongoing capacity-building Of particular impor-tance are ways to rapidly and e ectively assimilate new arrivals at aschool (sta students families) This is a major concern at sites withconsiderable turnover or growth At such sites the majority of thoseinitially involved in implementing a new approach may be gonewithin a period of two to three years Whatever the mobility rate it isessential to design and maintain transition programs for newarrivals Initial welcoming and introductory orientations of coursemust be followed-up with ongoing support systems and intensivecapacity building related to understanding and valuing theapproaches the school has adopted We nd that all this is essentialnot only to maintain what has been adopted but also can contributeto establishing schools as caring environments

PHASE IV ONGOING EVOLUTION

Ongoing evolution of organizations and programs is the product ofe orts to account for accomplishments deal with changing times andconditions incorporate new knowledge and create a sense of renewalas the excitement of newness wears o and the demands of changesap energy As suggested already in part vigor and direction can bemaintained through continuing education especially exposure toideas that suggest a range of ways for evolving a program As thefollowing discussion indicates ongoing evolution also is fostered byevaluation designed to document accomplishments and provide feed-back designed to improve quality

Increased concern over accountability has advanced the wayevaluation is conceived (Posavac amp Carey 1989 Rossi amp Freeman1989 Scriven 1993 Sechrest amp Figueredo 1993 Shadish Jr Cook ampLeviton 1991 Stake 1967 1976 Stufflebeam amp Webster 1983 Weiss1995) At the same time social and political forces have literallyshaped the whole enterprise and in the process have narrowed theway professionals clients policymakers underwriters and thegeneral public think about program evaluation A prevailing cry isfor specic evidence of e ectiveness For schools this means imme-diate gains in literacy as measured by achievement tests Althoughunderstandable in light of the unlled promise of so many programsand the insatiable demands on limited public nances such simplisti-cally conceived accountability demands ignore the complexities ofdeveloping and scaling-up major reforms

320 L Taylor et al

Formative and Summative Evaluation

Evaluation of a prototype involves more than determining efficacyfor students Broadly stated it encompasses concerns about how toexpand the focus of evaluative research not only to contribute toimproving practice but also to aid in evolving practice and policy(General Accounting Office 1989 Lyon amp Moats 1997) To facilitateprogram development and organizational change the primary orien-tation for evaluation in the early phases is formative It is especiallyfocused on data gathering and analyses that can help improve pro-cedures Most of what is written about educational and psychosocialintervention however is oriented to summative evaluation and mea-suring outcomes for individuals such as improved reading achieve-ment scores Replicating approaches to improve literacy involve notonly changing individuals but changing organizations and systemsThus both individuals and systems must be evaluated

All this presumes appropriate mechanisms to provide and analyzeessential information To these ends a scale-up sta can help estab-lish an evaluation team and capacity building that prepares a schoolto conduct evaluation that enhances reforms The immediate focus ison successful program replication ultimately of course the empha-sis must be on student outcomes

Pursuing Results

Due to the increased interest in accountability many complex aimsare broken down into specic objectives Indeed short-range objec-tives stated in measurable terms generally assume a central role inplanning However short-range objectives are not ends in them-selves They are a small part of a particular goal and aim and some-times are prerequisites for moving on to a goal It is essential not tolose sight of the fact that many specic objectives are relativelysmall unrepresentative and often unimportant segments of the mostvalued aims that society has for its citizens and citizens have forthemselves

The problem is well exemplied by the narrow focus found inreviews analyses and reanalyses of data on early education (eg seeAlbee amp Gullotta 1997 Bond amp Compas 1989 Dryfoos 1990 Durlak1995 Elias 1997 Mitchell Seligson amp Marx 1989 Schorr 1988Slavin Karweit amp Madden 1989 Weissberg Gullotta HamptomRyan amp Adams 1997) As such work demonstrates overemphasis onevaluating the efficacy of underdeveloped prototypes draws resourcesaway from formative evaluation

Scaling up Reforms 321

With specic respect to scale-up the rst accomplishment is thereplication itself Have all facets been implemented How completelyhas each been implemented At how many locations The next set ofresults are any indications of progress for students such as improve-ments in attitudes toward school health attendance behavior andacademic achievement A nal set of evaluation concerns is thedegree to which student outcomes approximate societal standards

Lessons Learned

The process of evaluating results is costly in terms of nancial invest-ment the negative psychological impact on those evaluated and theways it can inappropriately reshape new approaches Cost-e ectiveoutcomes cannot be achieved in the absence of e ective prototypedevelopment and research Premature e orts to carry out com-prehensive summative evaluations clearly are not cost-e ective Anyreading and writing program will show poor results if it is evaluatedbefore teachers have mastered its application None of this of courseis an argument against evaluating results Rather it is meant tounderscore concerns and encourage greater attention to addressingthem

Once a prototype is established care must be taken to avoiddeveloping outcome evaluation as an adversarial process Due to thepolitical realities related to accountability one of the most per-plexing facets to negotiate is the time frame for summative evalu-ation The more complex the prototype the longer it takes and thecostlier it is to implement and evaluate Schools usually want quickprocesses and results and of course rarely can a ord costly innova-tions or lengthy di usion activity Compromises are inevitable butmust be arrived at with great care so as not to undermine the sub-stance of proposed changes

The psychology of evaluation suggests that an overemphasis onaccountability tends to produce negative reactions One possible wayto counter this may be to conceive evaluation as a way for everystakeholder to self-evaluate as a basis for quality improvement and asa way of getting credit for all that is accomplished Unfortunately asaccountability pressures increase we nd that replication of proto-types are guided more by what can be measured than by long-rangeaims That is demands for immediate accountability reshape prac-tices so that the emphasis shifts to immediate and readily measuredobjectives and away from fundamental purposes Over time this inap-propriately leads to radical revision of the underlying rationale for aprototype

322 L Taylor et al

CONCLUDING COMMENTS

Those who set out to change schools and schooling are confrontedwith two enormous tasks The rst is to develop prototypes Thesecond involves large-scale replication One without the other isinsufficient Yet considerably more attention is paid to developingand validating prototypes than to delineating and testing scale-upprocesses Clearly it is time to correct this deciency The ideas pre-sented in this article are meant to stimulate work on the problem andthereby to advance the cause of educational reform

Finally in fairness to those who labor for educational reform weall must remember that the quality of schooling family life and com-munity functioning spirals up or down as a function of the quality ofthe ongoing transactions among each Thus scale-up e orts relatedto educational reform must take place within the context of a poli-tical agenda that addresses ways to strengthen the family and com-munity infrastructure through strategies that enhance economicopportunity adult literacy and so forth What we need are policiesthat develop demonstrate and scale-up comprehensive multifacetedintegrated approaches that can e ectively address barriers to devel-opment learning and teaching

REFERENCES

Adelman H S (1993) School-linked mental health interventions Toward mechanismsfor service coordination and integration Journal of Community Psychology 21 309ndash319

Adelman H S (1996a) Restructuring education support services and integrating com-munity resources Beyond the full service school model School Psychology Review25 431ndash445

Adelman H S (1996b) Restructuring support services Toward a comprehensiveapproach Kent OH American School Health Association

Adelman H S amp Taylor L (1993a) Early Assistance for Students and FamiliesProgram Guidebook Los Angeles Early Assistance for Students and FamiliesProject Copies available from the School Mental Health Project Dept of Psychol-ogy UCLA

Adelman H S amp Taylor L (1993b) Learning problems and learning disabilities Moving forward Pacic Grove CA BrooksCole

Adelman H S amp Taylor L (1994) On understanding intervention in psychology andeducation Westport CT Praeger

Adelman H S amp Taylor L (1997a) Addressing barriers to learning Beyond school-linked services and full service schools American Journal of Orthopsychiatry 67408ndash421

Adelman H S amp Taylor L (1997b) Toward a scale-up model for replicating newapproaches to schooling Journal of Educational and Psychological Consultation 8197ndash230

Scaling up Reforms 309

specic criteria emerge during the negotiation process For examplea principal may be attracted by the idea of establishing a programthat brings in volunteer reading tutors but in subsequent discussionswith teachers union concerns may arise that require arbitration

Policy

Substantive restructuring is unlikely without the adoption of newpolicies at all relevant jurisdictional levels (Spillane 1998) More-over such policies must elevate desired reforms so that they are notseen simply as demonstrations pilot projects passing fads or supple-mentary e orts When reforms are not assigned a high priority theytend to be treated in a marginalized manner (Center for MentalHealth in Schools 1998) This continues to be the fate of programssuch as Head Start Even Start and many other approaches toenhancing school readiness and literacy Relatedly e orts must bemade to revoke policies that preserve an unsatisfactory status quo(see critique of remedial reading programs by Dudley-Marling ampMurphy 1997)

Lasting reform requires processes that ensure informed com-mitment ownership and on-going support on the part of policymakers This involves strategies to create interest and formalizeagreements about fundamental changes Local ownership is estab-lished through solid policy commitments well-designed infrastruc-ture mechanisms allocation of adequate resources (eg nancespersonnel space equipment) to operationalize the policy andrestructuring of time to ensure sta involvement in adapting theprototype to the setting We nd three steps are essential (a) build-ing on introductory presentations to provide indepth information andunderstanding as a basis for establishing consensus (b) negotiationof a policy framework and a set of agreements for engagementincluding a realistic budget and (c) informed and voluntary rati-cation of agreements by legitimate representatives of all major stake-holders

For any program there are principles components elements andstandards that dene its essence and thus must be agreed to as a rstcondition for engagement Equally important are fundamental scale-up considerations o -line nonnegotiable such as the need for tempo-rary mechanisms to facilitate change Once essentials are agreed onall other matters are negotiable

Informed commitment is strengthened and operationalized throughnegotiating formal agreements at each jurisdictional level and amongvarious stakeholders Policy statements articulate the commitment to

310 L Taylor et al

a programrsquos essence Memoranda of understanding and contractsspecify agreements about such matters as funding sources resourceappropriations personnel functions incentives and safeguards forrisk-taking stakeholder development immediate and long-term com-mitments and timelines accountability procedures and so forth

Scale-up is aided when the decision to proceed is ratied by sanc-tioned representatives of stakeholder groups Developing and nego-tiating policies contracts and other formal agreements is a complexbusiness We nd that addressing the many logistics and legalitiesrequires extensive involvement of a small number of authorized andwell-informed stakeholder representatives Thus in pursuing thesetasks our commitment to include everyone moves from a town hallapproach to a representative democratic process with enfranchisedrepresentatives reporting back frequently to their constituencies Atrst endorsement is in principle over time it is manifested throughsustained support When ratication reects e ective consensusbuilding scale-up e orts benet from a broad base of informed com-mitment ownership and active sponsorship These attributes areessential in ensuring requisite support and protections for those whomust bear the burden of learning new ways and who risk dips in per-formance and productivity while doing so

Redesigning Infrastructure

After agreements are ratied a scale-up team can begin its work(again see Figure 1) A central challenge at every jurisdictional levelis redesign of regular mechanisms and processes used to make andimplement decisions These modications ensure ownership supportparticipation and address specic concerns associated with scale-up

Five fundamental facets of the ongoing infrastructure of schoolsthat are the focus of redesign are (a) governance (b) planning andimplementation associated with specic organizational and programobjectives (c) coordination and integration to ensure cohesive func-tioning (d) daily leadership and (e) communication and informationmanagement A common example of the need for infrastructure modi-cation is seen in the trend to increase school stakeholdersrsquo collabo-ration participation and inuence One implication is thatgovernance mechanisms will be altered to redistribute power Amajor problem of course is how to empower additional stakeholdergroups without disempowering those who have essentialresponsibilities and abilities related to the educational enterprise Inaddition it is one thing to o er lsquolsquopartnershipsrsquorsquo to stakeholders suchas parents students sta and community agency representatives it

Scaling up Reforms 311

is another to create conditions that allow for e ective participationOne such condition involves translating capacity building activityinto comprehensive programs for stakeholder development

The necessity of all this can be appreciated by thinking aboutintroducing a comprehensive approach for improving student literacy(Stringeld Ross amp Smith 1996) Such approaches involve major sys-temic changes that encompass intensive partnerships with parents (ortheir surrogates) and with various entities in the community such aslibraries youth development programs businesses the faith com-munity and so forth Substantive partnerships require a true sharingof leadership blending of resources and leadership training for pro-fessionals and nonprofessionals alike In communities where manyparents have little or no connection to the school major outreache orts are inevitable prerequisites to increasing home involvement inschool reform Parent outreach of course has not been very suc-cessful in many neighborhoods Our experience suggests that a neces-sary rst step in most cases is to o er programs and services thatassist the family in meeting its most pressing needs Furthermorethere is the matter of building parent competence to deal with plan-ning reforms and restructuring schools and for low income familiesthere is a need to nd ways to pay parents for the time they devote toserving on governance and other committees

Time is one of the most critical elements determining the successof scale-up Even if a prototype doesnrsquot call for restructuring theschool day the scale-up process does Substantial blocks of time areneeded for stakeholder capacity building and for individual and col-lective planning (National Education Commission on Time andLearning 1994) Particularly critical is the need for freeing-up tea-chers to learn new approaches For example e orts to make impor-tant revisions in literacy programs seem consistently undermined bynot providing enough time during the school day for the mentoring ofteachers and by the difficulty of carving out sufficient time to teachparents how to help their children Clearly a nonnegotiable condi-tion for engagement is a realistic plan for ensuring time to plan andbuild capacity

Lessons Learned

Complex interventions of course are seldom implemented in a com-pletely planned and linear manner The many practical and unfore-seen events that arise require exible problem-solving Articulationof a scale-up model can guide planning but those facilitating the

312 L Taylor et al

process must be prepared to capitalize on every opportunity that canmove the process ahead

Among the most fundamental lessons learned in carrying outPhase 1 has been the tendency of all parties to set actions into motionwithout taking sufficient time to lay the foundation needed for sub-stantive change In marketing new ideas it is tempting to accentuatetheir promising attributes and minimize complications In negotiatingagreements policy makers at a school site frequently are askedsimply for a go-ahead rather than for their informed commitmentSometimes they assent mainly to get extra resources sometimes theyare motivated by a desire to be seen by constituents as doing some-thing to improve the school This all tends to produce pressures forpremature implementation that results in the form rather than thesubstance of change especially when administrators are under thegun of political accountability measures that make unrealisticdemands for quick and dramatic results in studentsrsquo reading scores

Although formulation of policy and related agreements take con-siderable time and other resources their importance cannot be over-emphasized Failure to establish and successfully maintainsubstantive reforms in schools probably is attributable in greatmeasure to proceeding without strong and clear policy support

Another unfortunate trend we have found is the omission of in-depth planning for ongoing capacity building for change agents andteam members Mechanisms function only as well as the personnelwho operate them Such personnel must be recruited and developed inways that ensure appropriate motivation and capability and suffi-cient time must be redeployed so they can learn and carry out newfunctions e ectively (Peterson McCarthey amp Elmore 1996) Allchanges require constant care and feeding Those who steer theprocess must be motivated and competent not just initially but overtime The complexity of systemic change requires close monitoring ofmechanisms and immediate follow-up to address problems In particu-lar it means providing continuous personalized guidance andsupport to enhance knowledge and skills and counter anxiety frus-tration and other stressors To these ends adequate resource supportmust be provided (time space materials equipment) opportunitiesmust be available for increasing ability and generating a sense ofrenewed mission and personnel turnover must be addressed quicklyAll stakeholders can benet from e orts designed to increase levelsof competence and enhance motivation for working together Suche orts encompass four stages of stakeholder development orienta-tion foundation-building capacity-building and continuing educa-tion

Scaling up Reforms 313

There is no simple solution to the chronic problem of providingtime for creating readiness building capacity and planning Indeedrestructuring time represents one of the most difficult scale-up prob-lems Examples of how the problem might be addressed includefreeing up sta by establishing opportunities for students to spendtime pursuing activities such as music art and sports with special-ists or supervised by aides and community volunteers Alternativelyschool might start later or end earlier on a given day As these exam-ples suggest any approach will be controversial but if the problem isnot addressed satisfactorily successful replication of comprehensiveprototypes is unlikely

PHASE II INITIAL IMPLEMENTATION OF APROTOTYPE

Initial implementation involves adapting and phasing-in a programwith well-designed guidance and support If there is anything certainabout e orts to replicate a prototype it is that the process is stress-ful Some of the stress arises from the nature of the program some isinherent in the process of organizational change Coalitions must bedeveloped new working relationships established disruptive rumorsand information overload countered and interpersonal conictsresolved Short-term frustrations must be kept in perspective vis a visthe reform vision To help deal with all this temporary mechanismsare added to the organizational infrastructure They include (a) asite-based steering mechanism to guide and support replication (b) achange agent from the scale-up team working with site stakeholderson a change team to facilitate coalition building problem solvingand conict resolution and (c) mentors and coaches to model andteach elements of the prototype These structures are created tofacilitate replication and some are assimilated into a sitersquos infra-structure at the end of the initial implementation phase to supportinstitutionalization and ongoing evolution

A scale-up team and steering group work at a site with the schoolrsquosleadership specic planning groups and other stakeholders to formu-late phase-in plans steer program development and generallyprovide guidance and support for change Two major facets of thiswork are delineating a sequence for introducing major program ele-ments and outlining strategies to facilitate implementation Particu-lar attention is given to how to start with special emphasis onspecifying structures and resources for guidance and support For

314 L Taylor et al

instance in restructuring to better address barriers to learning therst steps at a school site involve creating processes to map analyzecoordinate and redeploy existing resources Special change mecha-nisms such as an organization facilitator and a resource coordinatingteam are created to guide and support the activity (Adelman 19931996a 1996b Adelman amp Taylor 1997a 1997b 1998)

Throughout this phase formative evaluation procedures are estab-lished to provide feedback for program development As noted abovee ective e orts to lsquolsquoreinventrsquorsquo schools require ensuring that allinvolved have the time to develop and institutionalize a soundprogram and that they are not penalized for unavoidable missteps Asa prototype is phased-in evaluation must not be thought of in termsof accountability Major systemic changes to improve literacy cantake years to develop Outcome e ectiveness is demonstrated afterthe program is in place The purpose of evaluation at this stage is toguide revision and ne-tuning of processes Formative evaluationsgather and analyze information relevant to changes in planning pro-cesses governance structures and policies and resources they alsofocus on implementation strategies and barriers program organiz-ation and staffing and initial outcomes Are teachers mastering thenew curricula for literacy Is there increased student engagement inreading Are parents involved in supporting reading at home Ifthings are not progressing satisfactorily why not Whatrsquos the down-side of the new approach

Well-designed organizational support and guidance is needed toenhance productivity minimize problems and accommodate individ-ual di erences This involves various forms of capacity building andpersonalized day-by-day facilitation Intensive coaching with somefollow-up consultation for instance are key processes so are mentor-ship and technical assistance Continuing education provides a criti-cal vehicle for enhancing productive changes generating renewaland countering burnout As new stakeholders arrive technologicaltools can be particularly useful in helping them catch up All thisactivity not only builds capacity but can foster networking and otherforms of task-related social and personal support as well as providea wide range of enrichment opportunities that enhance morale

If the steps discussed to this point are done well a sound founda-tion for initial implementation should be in place This initial phase-in period can however consume considerable e ort create specialproblems and may yield a temporary drop in some performance indi-cators Good day-by-day facilitation aims at minimizing such negativeimpact by e ectively addressing stakeholder motivation and capabil-ity and overcoming barriers to productive working relationships

Scaling up Reforms 315

Lessons Learned

Failure to take sufficient time to create readiness (Phase 1) can resultin implementing the form rather than the substance of a prototypeFor example we nd that change agents frequently are sent intoschools before essential policy support is enacted and before schoolleaders have assimilated and decided to support reforms Teams areconvened to assist with reforms (plan coordinate develop newapproaches) but the absence of supportive policy means substantivechanges are not accomplished As a result the initial motivation ofmany key team members wanes and other counterproductivedynamics arise All of this seems inevitable when initial implementa-tion proceeds without adequate policy support

Even in situations where sufficient readiness is created difficultiesfrequently arise due to a failure to keep enough stakeholders consis-tently moving in the direction of desired outcomes Comprehensivechange is usually achieved only when fairly high levels of positiveenergy can be mobilized over extended periods of time among a criti-cal mass of stakeholders sustained energy is appropriately directedthe process is supported with ongoing and well-conceived capacitybuilding and individuals are not pushed beyond their capabilitiesAnd because low and negative motivation are related to resistance tochange and poor functioning matching motivation is a rst-orderconsideration That is scale-up e orts must use strategies designed tomobilize and maintain proactive e ort and overcome barriers toworking relationships As in personalizing instruction approx-imating a good motivational t also requires matching capabilitiessuch as starting with fewer elements at sites at which resources arelimited and accounting for variability in stakeholdersrsquo competenceOver and over we nd too little attention is paid to these mattersThe result is failure to create an lsquolsquoenvironmentrsquorsquo that mobilizesdirects and then maintains stakeholder involvement

As with students the problem can be conceived as that of main-taining an appropriate match between the demands of the situationand individual motivation and capabilities In this respect we thinkthe construct of personalization o ers a concept around which toorganize thinking about facilitating change As stressed in the rstarticle in this issue personalization calls for systematically planningand implementing processes focused not only on knowledge andskills but on attitudes In particular it emphasizes the importance ofa primary and constant focus on ensuring positive attitudes Mobi-lization probably is best facilitated when procedures are perceived byindividuals as good ways to reach desired outcomes This requires

316 L Taylor et al

processes that can instigate and enhance an individualrsquos perceptionsof valued opportunities choice and control accomplishment andrelatedness to others Even if a task isnrsquot enjoyable expectation offeeling some sense of satisfaction related to process or outcome canbe a powerful intrinsic factor motivating individual behavior Taskpersistence for example can be facilitated by the expectation thatone will feel competent self-determining or more closely connectedto others From this perspective ensuring that individuals havevalued options a meaningful role in decision making feedback thatemphasizes progress toward desired outcomes and positive workingrelationships are among the most basic facilitation strategies(Adelman amp Taylor 1993b 1994 Deci amp Ryan 1985)

One other initial implementation problem that often arises is diffi-culty in establishing mechanisms to facilitate productive workingrelationships and identify and deal with problems quickly Forexample it is expected that change agents will encounter manyinstances of individual resistance and apathy interpersonal conictsand resentments (including lsquolsquous vs themrsquorsquo dynamics) rumors thatoveremphasize the negative and underestimate the positive and indi-viduals who are frequent faultnders Such problems seriouslyimpede e ective replication The roots of some of these problemsoften are present at a site prior to scale-up change simply o ers anew focus and perhaps magnies troubling matters Other problemsare a direct product of the activities and relationships that the scale-up process engenders Given the inevitability of such problems build-ing and maintaining working relationships need to be among themost basic concerns for those who have responsibility for scale-up Inparticular considerable attention must be paid to enhancing themotivational readiness and capability of those who are to worktogether and ensure there is an appropriate infrastructure to guideand support working relationships Proactively this requires problemprevention mechanisms that help create an atmosphere where defen-siveness is curtailed and positive rapport is engendered The point isto enhance attitudes knowledge and skills that foster interpersonalconnections and a sense of community Reactively the emphasis is onproblem solving resolving conict and providing ongoing support torebuild relationships Policies must encourage problem-solving-oriented critiques safeguards that protect those making changesappreciation for e ort and celebration of progress We nd thateveryone understands such matters but the culture at many schoolsites is more attuned to problem naming and analyzing than to antici-pating preventing and solving problems that arise around workingrelationships

Scaling up Reforms 317

Those responsible for systemic change need to spend as much timeas necessary ensuring that a schoolrsquos infrastructure is ready toprevent and ameliorate problems Special attention must be paid toensuring that problem solving mechanisms and communication pro-cesses are in place and properly sta ed and that stakeholders arewell informed about how to use the procedures Furthermore somestakeholders may have to be encouraged to interact in ways thatconvey genuine empathy warmth and mutual regard and respectwith a view to creating and maintaining a positive working climateand a psychological sense of community

At times we nd it necessary to target a specic problem anddesignated persons In some instances rather simple strategies aree ective For example most motivated individuals can be directlytaught ways to improve understanding and communication and avoidor resolve conicts that interfere with working relationships Inother instances however signicant remedial action is necessary aswhen overcoming barriers to a working relationship involvescountering negative attitudes Helpful in this regard are analysessuch as that by Sue and Zane (1987) that suggest how to demonstratethat something of value can be gained from individuals workingtogether and how to establish each participantrsquos credibility (eg bymaximizing task-focus and positive outcomes)

PHASE III INSTITUTIONALIZING THE PROTOTYPE

Maintaining and enhancing changes can be as difficult as makingthem in the rst place The history of education reform is one offailure to foster promising prototypes in substantive ways and overan extended period of times (Tyack amp Cuban 1995) Institutionalizinga prototype entails ensuring that the organization assumes long-termownership and that there is a blueprint for countering forces that canerode the changes Moreover institutionalization is more than atechnical process It requires assimilation of and ongoing adherenceto the values inherent in the prototypersquos underlying rationale Thefocus of course is not just on maintenance the point is to moveforward by enhancing productive changes and generating a sense ofrenewal as needed Critical in all this are specic plans that guar-antee ongoing and enhanced leadership and delineate ways in whichplanning implementation coordination and continuing educationmechanisms are maintained

Some Major Tasks

Whose responsibility is it to advocate for maintaining and evolving a

318 L Taylor et al

replicated prototype for improving literacy As problems arise whoseresponsibility is it to lead the way in resolving them Leadership isthe key here by both official leaders such as administrators mentorsta union chapter chairs and elected parent representatives aswell as natural leaders such as reading and writing teachers Obvi-ously official and natural leaders are not mutually exclusive groupsAt this phase both types of leadership are essential to ensure a broadenough base for ongoing advocacy problem solving enhancementand renewal Official leaders provide a legitimate power base asvarious interests compete for the organizationrsquos limited resourcesand they play a key role in ensuring that the contributions of naturalleaders are recognized and rewarded

Maintenance and enhancement require that the organizationrsquosgovernance body assumes ownership and program advocacy such astaking over the temporary steering grouprsquos functions addressingongoing policy and long-range planning concerns and maintainingnancial support The foundation for such ownership is laid duringthe readiness phase Each element becomes the organizationrsquos pro-perty as it is established during initial implementation The officialdeed of ownership is transferred as soon as the prototype is in place

Ownership however is no guarantee of institutionalizationVarious forces that can erode reforms are always at work Forinstance teams at a site experience turnover problems with commu-nication and sharing of resources are chronic competing interestsand the attractiveness of moving on to something new pull attentionand resources to other activity To minimize such problems stepsmust be taken to identify and solve them as quickly as is feasibleThis requires someone who has the time energy and expertise tomeet periodically with stakeholders to anticipate and amelioratethreats to a prototypersquos integrity

Over time mechanisms for planning implementation and coordi-nation are maintained by ensuring the activity is an official part ofthe infrastructure has appropriate leadership and is e ectively sup-ported Anyone who has worked on a school-based team to improve aliteracy program knows there must be a critical mass of teammembers so that the work load is manageable and a broad base ofinvolvement is ensured Also essential are adequate resources includ-ing time to learn the role and time to perform the functions reason-ably interesting tasks technical support for problem solvingrecognition and rewards for contributions immediate replacementwhen someone leaves continuing education to enhance team func-tioning and so forth Without serious attention to such matters theteamsrsquo morale and motivation will wane

Scaling up Reforms 319

Lessons Learned

Newly institutionalized approaches are seriously jeopardized in theabsence of dedicated ongoing capacity-building Of particular impor-tance are ways to rapidly and e ectively assimilate new arrivals at aschool (sta students families) This is a major concern at sites withconsiderable turnover or growth At such sites the majority of thoseinitially involved in implementing a new approach may be gonewithin a period of two to three years Whatever the mobility rate it isessential to design and maintain transition programs for newarrivals Initial welcoming and introductory orientations of coursemust be followed-up with ongoing support systems and intensivecapacity building related to understanding and valuing theapproaches the school has adopted We nd that all this is essentialnot only to maintain what has been adopted but also can contributeto establishing schools as caring environments

PHASE IV ONGOING EVOLUTION

Ongoing evolution of organizations and programs is the product ofe orts to account for accomplishments deal with changing times andconditions incorporate new knowledge and create a sense of renewalas the excitement of newness wears o and the demands of changesap energy As suggested already in part vigor and direction can bemaintained through continuing education especially exposure toideas that suggest a range of ways for evolving a program As thefollowing discussion indicates ongoing evolution also is fostered byevaluation designed to document accomplishments and provide feed-back designed to improve quality

Increased concern over accountability has advanced the wayevaluation is conceived (Posavac amp Carey 1989 Rossi amp Freeman1989 Scriven 1993 Sechrest amp Figueredo 1993 Shadish Jr Cook ampLeviton 1991 Stake 1967 1976 Stufflebeam amp Webster 1983 Weiss1995) At the same time social and political forces have literallyshaped the whole enterprise and in the process have narrowed theway professionals clients policymakers underwriters and thegeneral public think about program evaluation A prevailing cry isfor specic evidence of e ectiveness For schools this means imme-diate gains in literacy as measured by achievement tests Althoughunderstandable in light of the unlled promise of so many programsand the insatiable demands on limited public nances such simplisti-cally conceived accountability demands ignore the complexities ofdeveloping and scaling-up major reforms

320 L Taylor et al

Formative and Summative Evaluation

Evaluation of a prototype involves more than determining efficacyfor students Broadly stated it encompasses concerns about how toexpand the focus of evaluative research not only to contribute toimproving practice but also to aid in evolving practice and policy(General Accounting Office 1989 Lyon amp Moats 1997) To facilitateprogram development and organizational change the primary orien-tation for evaluation in the early phases is formative It is especiallyfocused on data gathering and analyses that can help improve pro-cedures Most of what is written about educational and psychosocialintervention however is oriented to summative evaluation and mea-suring outcomes for individuals such as improved reading achieve-ment scores Replicating approaches to improve literacy involve notonly changing individuals but changing organizations and systemsThus both individuals and systems must be evaluated

All this presumes appropriate mechanisms to provide and analyzeessential information To these ends a scale-up sta can help estab-lish an evaluation team and capacity building that prepares a schoolto conduct evaluation that enhances reforms The immediate focus ison successful program replication ultimately of course the empha-sis must be on student outcomes

Pursuing Results

Due to the increased interest in accountability many complex aimsare broken down into specic objectives Indeed short-range objec-tives stated in measurable terms generally assume a central role inplanning However short-range objectives are not ends in them-selves They are a small part of a particular goal and aim and some-times are prerequisites for moving on to a goal It is essential not tolose sight of the fact that many specic objectives are relativelysmall unrepresentative and often unimportant segments of the mostvalued aims that society has for its citizens and citizens have forthemselves

The problem is well exemplied by the narrow focus found inreviews analyses and reanalyses of data on early education (eg seeAlbee amp Gullotta 1997 Bond amp Compas 1989 Dryfoos 1990 Durlak1995 Elias 1997 Mitchell Seligson amp Marx 1989 Schorr 1988Slavin Karweit amp Madden 1989 Weissberg Gullotta HamptomRyan amp Adams 1997) As such work demonstrates overemphasis onevaluating the efficacy of underdeveloped prototypes draws resourcesaway from formative evaluation

Scaling up Reforms 321

With specic respect to scale-up the rst accomplishment is thereplication itself Have all facets been implemented How completelyhas each been implemented At how many locations The next set ofresults are any indications of progress for students such as improve-ments in attitudes toward school health attendance behavior andacademic achievement A nal set of evaluation concerns is thedegree to which student outcomes approximate societal standards

Lessons Learned

The process of evaluating results is costly in terms of nancial invest-ment the negative psychological impact on those evaluated and theways it can inappropriately reshape new approaches Cost-e ectiveoutcomes cannot be achieved in the absence of e ective prototypedevelopment and research Premature e orts to carry out com-prehensive summative evaluations clearly are not cost-e ective Anyreading and writing program will show poor results if it is evaluatedbefore teachers have mastered its application None of this of courseis an argument against evaluating results Rather it is meant tounderscore concerns and encourage greater attention to addressingthem

Once a prototype is established care must be taken to avoiddeveloping outcome evaluation as an adversarial process Due to thepolitical realities related to accountability one of the most per-plexing facets to negotiate is the time frame for summative evalu-ation The more complex the prototype the longer it takes and thecostlier it is to implement and evaluate Schools usually want quickprocesses and results and of course rarely can a ord costly innova-tions or lengthy di usion activity Compromises are inevitable butmust be arrived at with great care so as not to undermine the sub-stance of proposed changes

The psychology of evaluation suggests that an overemphasis onaccountability tends to produce negative reactions One possible wayto counter this may be to conceive evaluation as a way for everystakeholder to self-evaluate as a basis for quality improvement and asa way of getting credit for all that is accomplished Unfortunately asaccountability pressures increase we nd that replication of proto-types are guided more by what can be measured than by long-rangeaims That is demands for immediate accountability reshape prac-tices so that the emphasis shifts to immediate and readily measuredobjectives and away from fundamental purposes Over time this inap-propriately leads to radical revision of the underlying rationale for aprototype

322 L Taylor et al

CONCLUDING COMMENTS

Those who set out to change schools and schooling are confrontedwith two enormous tasks The rst is to develop prototypes Thesecond involves large-scale replication One without the other isinsufficient Yet considerably more attention is paid to developingand validating prototypes than to delineating and testing scale-upprocesses Clearly it is time to correct this deciency The ideas pre-sented in this article are meant to stimulate work on the problem andthereby to advance the cause of educational reform

Finally in fairness to those who labor for educational reform weall must remember that the quality of schooling family life and com-munity functioning spirals up or down as a function of the quality ofthe ongoing transactions among each Thus scale-up e orts relatedto educational reform must take place within the context of a poli-tical agenda that addresses ways to strengthen the family and com-munity infrastructure through strategies that enhance economicopportunity adult literacy and so forth What we need are policiesthat develop demonstrate and scale-up comprehensive multifacetedintegrated approaches that can e ectively address barriers to devel-opment learning and teaching

REFERENCES

Adelman H S (1993) School-linked mental health interventions Toward mechanismsfor service coordination and integration Journal of Community Psychology 21 309ndash319

Adelman H S (1996a) Restructuring education support services and integrating com-munity resources Beyond the full service school model School Psychology Review25 431ndash445

Adelman H S (1996b) Restructuring support services Toward a comprehensiveapproach Kent OH American School Health Association

Adelman H S amp Taylor L (1993a) Early Assistance for Students and FamiliesProgram Guidebook Los Angeles Early Assistance for Students and FamiliesProject Copies available from the School Mental Health Project Dept of Psychol-ogy UCLA

Adelman H S amp Taylor L (1993b) Learning problems and learning disabilities Moving forward Pacic Grove CA BrooksCole

Adelman H S amp Taylor L (1994) On understanding intervention in psychology andeducation Westport CT Praeger

Adelman H S amp Taylor L (1997a) Addressing barriers to learning Beyond school-linked services and full service schools American Journal of Orthopsychiatry 67408ndash421

Adelman H S amp Taylor L (1997b) Toward a scale-up model for replicating newapproaches to schooling Journal of Educational and Psychological Consultation 8197ndash230

310 L Taylor et al

a programrsquos essence Memoranda of understanding and contractsspecify agreements about such matters as funding sources resourceappropriations personnel functions incentives and safeguards forrisk-taking stakeholder development immediate and long-term com-mitments and timelines accountability procedures and so forth

Scale-up is aided when the decision to proceed is ratied by sanc-tioned representatives of stakeholder groups Developing and nego-tiating policies contracts and other formal agreements is a complexbusiness We nd that addressing the many logistics and legalitiesrequires extensive involvement of a small number of authorized andwell-informed stakeholder representatives Thus in pursuing thesetasks our commitment to include everyone moves from a town hallapproach to a representative democratic process with enfranchisedrepresentatives reporting back frequently to their constituencies Atrst endorsement is in principle over time it is manifested throughsustained support When ratication reects e ective consensusbuilding scale-up e orts benet from a broad base of informed com-mitment ownership and active sponsorship These attributes areessential in ensuring requisite support and protections for those whomust bear the burden of learning new ways and who risk dips in per-formance and productivity while doing so

Redesigning Infrastructure

After agreements are ratied a scale-up team can begin its work(again see Figure 1) A central challenge at every jurisdictional levelis redesign of regular mechanisms and processes used to make andimplement decisions These modications ensure ownership supportparticipation and address specic concerns associated with scale-up

Five fundamental facets of the ongoing infrastructure of schoolsthat are the focus of redesign are (a) governance (b) planning andimplementation associated with specic organizational and programobjectives (c) coordination and integration to ensure cohesive func-tioning (d) daily leadership and (e) communication and informationmanagement A common example of the need for infrastructure modi-cation is seen in the trend to increase school stakeholdersrsquo collabo-ration participation and inuence One implication is thatgovernance mechanisms will be altered to redistribute power Amajor problem of course is how to empower additional stakeholdergroups without disempowering those who have essentialresponsibilities and abilities related to the educational enterprise Inaddition it is one thing to o er lsquolsquopartnershipsrsquorsquo to stakeholders suchas parents students sta and community agency representatives it

Scaling up Reforms 311

is another to create conditions that allow for e ective participationOne such condition involves translating capacity building activityinto comprehensive programs for stakeholder development

The necessity of all this can be appreciated by thinking aboutintroducing a comprehensive approach for improving student literacy(Stringeld Ross amp Smith 1996) Such approaches involve major sys-temic changes that encompass intensive partnerships with parents (ortheir surrogates) and with various entities in the community such aslibraries youth development programs businesses the faith com-munity and so forth Substantive partnerships require a true sharingof leadership blending of resources and leadership training for pro-fessionals and nonprofessionals alike In communities where manyparents have little or no connection to the school major outreache orts are inevitable prerequisites to increasing home involvement inschool reform Parent outreach of course has not been very suc-cessful in many neighborhoods Our experience suggests that a neces-sary rst step in most cases is to o er programs and services thatassist the family in meeting its most pressing needs Furthermorethere is the matter of building parent competence to deal with plan-ning reforms and restructuring schools and for low income familiesthere is a need to nd ways to pay parents for the time they devote toserving on governance and other committees

Time is one of the most critical elements determining the successof scale-up Even if a prototype doesnrsquot call for restructuring theschool day the scale-up process does Substantial blocks of time areneeded for stakeholder capacity building and for individual and col-lective planning (National Education Commission on Time andLearning 1994) Particularly critical is the need for freeing-up tea-chers to learn new approaches For example e orts to make impor-tant revisions in literacy programs seem consistently undermined bynot providing enough time during the school day for the mentoring ofteachers and by the difficulty of carving out sufficient time to teachparents how to help their children Clearly a nonnegotiable condi-tion for engagement is a realistic plan for ensuring time to plan andbuild capacity

Lessons Learned

Complex interventions of course are seldom implemented in a com-pletely planned and linear manner The many practical and unfore-seen events that arise require exible problem-solving Articulationof a scale-up model can guide planning but those facilitating the

312 L Taylor et al

process must be prepared to capitalize on every opportunity that canmove the process ahead

Among the most fundamental lessons learned in carrying outPhase 1 has been the tendency of all parties to set actions into motionwithout taking sufficient time to lay the foundation needed for sub-stantive change In marketing new ideas it is tempting to accentuatetheir promising attributes and minimize complications In negotiatingagreements policy makers at a school site frequently are askedsimply for a go-ahead rather than for their informed commitmentSometimes they assent mainly to get extra resources sometimes theyare motivated by a desire to be seen by constituents as doing some-thing to improve the school This all tends to produce pressures forpremature implementation that results in the form rather than thesubstance of change especially when administrators are under thegun of political accountability measures that make unrealisticdemands for quick and dramatic results in studentsrsquo reading scores

Although formulation of policy and related agreements take con-siderable time and other resources their importance cannot be over-emphasized Failure to establish and successfully maintainsubstantive reforms in schools probably is attributable in greatmeasure to proceeding without strong and clear policy support

Another unfortunate trend we have found is the omission of in-depth planning for ongoing capacity building for change agents andteam members Mechanisms function only as well as the personnelwho operate them Such personnel must be recruited and developed inways that ensure appropriate motivation and capability and suffi-cient time must be redeployed so they can learn and carry out newfunctions e ectively (Peterson McCarthey amp Elmore 1996) Allchanges require constant care and feeding Those who steer theprocess must be motivated and competent not just initially but overtime The complexity of systemic change requires close monitoring ofmechanisms and immediate follow-up to address problems In particu-lar it means providing continuous personalized guidance andsupport to enhance knowledge and skills and counter anxiety frus-tration and other stressors To these ends adequate resource supportmust be provided (time space materials equipment) opportunitiesmust be available for increasing ability and generating a sense ofrenewed mission and personnel turnover must be addressed quicklyAll stakeholders can benet from e orts designed to increase levelsof competence and enhance motivation for working together Suche orts encompass four stages of stakeholder development orienta-tion foundation-building capacity-building and continuing educa-tion

Scaling up Reforms 313

There is no simple solution to the chronic problem of providingtime for creating readiness building capacity and planning Indeedrestructuring time represents one of the most difficult scale-up prob-lems Examples of how the problem might be addressed includefreeing up sta by establishing opportunities for students to spendtime pursuing activities such as music art and sports with special-ists or supervised by aides and community volunteers Alternativelyschool might start later or end earlier on a given day As these exam-ples suggest any approach will be controversial but if the problem isnot addressed satisfactorily successful replication of comprehensiveprototypes is unlikely

PHASE II INITIAL IMPLEMENTATION OF APROTOTYPE

Initial implementation involves adapting and phasing-in a programwith well-designed guidance and support If there is anything certainabout e orts to replicate a prototype it is that the process is stress-ful Some of the stress arises from the nature of the program some isinherent in the process of organizational change Coalitions must bedeveloped new working relationships established disruptive rumorsand information overload countered and interpersonal conictsresolved Short-term frustrations must be kept in perspective vis a visthe reform vision To help deal with all this temporary mechanismsare added to the organizational infrastructure They include (a) asite-based steering mechanism to guide and support replication (b) achange agent from the scale-up team working with site stakeholderson a change team to facilitate coalition building problem solvingand conict resolution and (c) mentors and coaches to model andteach elements of the prototype These structures are created tofacilitate replication and some are assimilated into a sitersquos infra-structure at the end of the initial implementation phase to supportinstitutionalization and ongoing evolution

A scale-up team and steering group work at a site with the schoolrsquosleadership specic planning groups and other stakeholders to formu-late phase-in plans steer program development and generallyprovide guidance and support for change Two major facets of thiswork are delineating a sequence for introducing major program ele-ments and outlining strategies to facilitate implementation Particu-lar attention is given to how to start with special emphasis onspecifying structures and resources for guidance and support For

314 L Taylor et al

instance in restructuring to better address barriers to learning therst steps at a school site involve creating processes to map analyzecoordinate and redeploy existing resources Special change mecha-nisms such as an organization facilitator and a resource coordinatingteam are created to guide and support the activity (Adelman 19931996a 1996b Adelman amp Taylor 1997a 1997b 1998)

Throughout this phase formative evaluation procedures are estab-lished to provide feedback for program development As noted abovee ective e orts to lsquolsquoreinventrsquorsquo schools require ensuring that allinvolved have the time to develop and institutionalize a soundprogram and that they are not penalized for unavoidable missteps Asa prototype is phased-in evaluation must not be thought of in termsof accountability Major systemic changes to improve literacy cantake years to develop Outcome e ectiveness is demonstrated afterthe program is in place The purpose of evaluation at this stage is toguide revision and ne-tuning of processes Formative evaluationsgather and analyze information relevant to changes in planning pro-cesses governance structures and policies and resources they alsofocus on implementation strategies and barriers program organiz-ation and staffing and initial outcomes Are teachers mastering thenew curricula for literacy Is there increased student engagement inreading Are parents involved in supporting reading at home Ifthings are not progressing satisfactorily why not Whatrsquos the down-side of the new approach

Well-designed organizational support and guidance is needed toenhance productivity minimize problems and accommodate individ-ual di erences This involves various forms of capacity building andpersonalized day-by-day facilitation Intensive coaching with somefollow-up consultation for instance are key processes so are mentor-ship and technical assistance Continuing education provides a criti-cal vehicle for enhancing productive changes generating renewaland countering burnout As new stakeholders arrive technologicaltools can be particularly useful in helping them catch up All thisactivity not only builds capacity but can foster networking and otherforms of task-related social and personal support as well as providea wide range of enrichment opportunities that enhance morale

If the steps discussed to this point are done well a sound founda-tion for initial implementation should be in place This initial phase-in period can however consume considerable e ort create specialproblems and may yield a temporary drop in some performance indi-cators Good day-by-day facilitation aims at minimizing such negativeimpact by e ectively addressing stakeholder motivation and capabil-ity and overcoming barriers to productive working relationships

Scaling up Reforms 315

Lessons Learned

Failure to take sufficient time to create readiness (Phase 1) can resultin implementing the form rather than the substance of a prototypeFor example we nd that change agents frequently are sent intoschools before essential policy support is enacted and before schoolleaders have assimilated and decided to support reforms Teams areconvened to assist with reforms (plan coordinate develop newapproaches) but the absence of supportive policy means substantivechanges are not accomplished As a result the initial motivation ofmany key team members wanes and other counterproductivedynamics arise All of this seems inevitable when initial implementa-tion proceeds without adequate policy support

Even in situations where sufficient readiness is created difficultiesfrequently arise due to a failure to keep enough stakeholders consis-tently moving in the direction of desired outcomes Comprehensivechange is usually achieved only when fairly high levels of positiveenergy can be mobilized over extended periods of time among a criti-cal mass of stakeholders sustained energy is appropriately directedthe process is supported with ongoing and well-conceived capacitybuilding and individuals are not pushed beyond their capabilitiesAnd because low and negative motivation are related to resistance tochange and poor functioning matching motivation is a rst-orderconsideration That is scale-up e orts must use strategies designed tomobilize and maintain proactive e ort and overcome barriers toworking relationships As in personalizing instruction approx-imating a good motivational t also requires matching capabilitiessuch as starting with fewer elements at sites at which resources arelimited and accounting for variability in stakeholdersrsquo competenceOver and over we nd too little attention is paid to these mattersThe result is failure to create an lsquolsquoenvironmentrsquorsquo that mobilizesdirects and then maintains stakeholder involvement

As with students the problem can be conceived as that of main-taining an appropriate match between the demands of the situationand individual motivation and capabilities In this respect we thinkthe construct of personalization o ers a concept around which toorganize thinking about facilitating change As stressed in the rstarticle in this issue personalization calls for systematically planningand implementing processes focused not only on knowledge andskills but on attitudes In particular it emphasizes the importance ofa primary and constant focus on ensuring positive attitudes Mobi-lization probably is best facilitated when procedures are perceived byindividuals as good ways to reach desired outcomes This requires

316 L Taylor et al

processes that can instigate and enhance an individualrsquos perceptionsof valued opportunities choice and control accomplishment andrelatedness to others Even if a task isnrsquot enjoyable expectation offeeling some sense of satisfaction related to process or outcome canbe a powerful intrinsic factor motivating individual behavior Taskpersistence for example can be facilitated by the expectation thatone will feel competent self-determining or more closely connectedto others From this perspective ensuring that individuals havevalued options a meaningful role in decision making feedback thatemphasizes progress toward desired outcomes and positive workingrelationships are among the most basic facilitation strategies(Adelman amp Taylor 1993b 1994 Deci amp Ryan 1985)

One other initial implementation problem that often arises is diffi-culty in establishing mechanisms to facilitate productive workingrelationships and identify and deal with problems quickly Forexample it is expected that change agents will encounter manyinstances of individual resistance and apathy interpersonal conictsand resentments (including lsquolsquous vs themrsquorsquo dynamics) rumors thatoveremphasize the negative and underestimate the positive and indi-viduals who are frequent faultnders Such problems seriouslyimpede e ective replication The roots of some of these problemsoften are present at a site prior to scale-up change simply o ers anew focus and perhaps magnies troubling matters Other problemsare a direct product of the activities and relationships that the scale-up process engenders Given the inevitability of such problems build-ing and maintaining working relationships need to be among themost basic concerns for those who have responsibility for scale-up Inparticular considerable attention must be paid to enhancing themotivational readiness and capability of those who are to worktogether and ensure there is an appropriate infrastructure to guideand support working relationships Proactively this requires problemprevention mechanisms that help create an atmosphere where defen-siveness is curtailed and positive rapport is engendered The point isto enhance attitudes knowledge and skills that foster interpersonalconnections and a sense of community Reactively the emphasis is onproblem solving resolving conict and providing ongoing support torebuild relationships Policies must encourage problem-solving-oriented critiques safeguards that protect those making changesappreciation for e ort and celebration of progress We nd thateveryone understands such matters but the culture at many schoolsites is more attuned to problem naming and analyzing than to antici-pating preventing and solving problems that arise around workingrelationships

Scaling up Reforms 317

Those responsible for systemic change need to spend as much timeas necessary ensuring that a schoolrsquos infrastructure is ready toprevent and ameliorate problems Special attention must be paid toensuring that problem solving mechanisms and communication pro-cesses are in place and properly sta ed and that stakeholders arewell informed about how to use the procedures Furthermore somestakeholders may have to be encouraged to interact in ways thatconvey genuine empathy warmth and mutual regard and respectwith a view to creating and maintaining a positive working climateand a psychological sense of community

At times we nd it necessary to target a specic problem anddesignated persons In some instances rather simple strategies aree ective For example most motivated individuals can be directlytaught ways to improve understanding and communication and avoidor resolve conicts that interfere with working relationships Inother instances however signicant remedial action is necessary aswhen overcoming barriers to a working relationship involvescountering negative attitudes Helpful in this regard are analysessuch as that by Sue and Zane (1987) that suggest how to demonstratethat something of value can be gained from individuals workingtogether and how to establish each participantrsquos credibility (eg bymaximizing task-focus and positive outcomes)

PHASE III INSTITUTIONALIZING THE PROTOTYPE

Maintaining and enhancing changes can be as difficult as makingthem in the rst place The history of education reform is one offailure to foster promising prototypes in substantive ways and overan extended period of times (Tyack amp Cuban 1995) Institutionalizinga prototype entails ensuring that the organization assumes long-termownership and that there is a blueprint for countering forces that canerode the changes Moreover institutionalization is more than atechnical process It requires assimilation of and ongoing adherenceto the values inherent in the prototypersquos underlying rationale Thefocus of course is not just on maintenance the point is to moveforward by enhancing productive changes and generating a sense ofrenewal as needed Critical in all this are specic plans that guar-antee ongoing and enhanced leadership and delineate ways in whichplanning implementation coordination and continuing educationmechanisms are maintained

Some Major Tasks

Whose responsibility is it to advocate for maintaining and evolving a

318 L Taylor et al

replicated prototype for improving literacy As problems arise whoseresponsibility is it to lead the way in resolving them Leadership isthe key here by both official leaders such as administrators mentorsta union chapter chairs and elected parent representatives aswell as natural leaders such as reading and writing teachers Obvi-ously official and natural leaders are not mutually exclusive groupsAt this phase both types of leadership are essential to ensure a broadenough base for ongoing advocacy problem solving enhancementand renewal Official leaders provide a legitimate power base asvarious interests compete for the organizationrsquos limited resourcesand they play a key role in ensuring that the contributions of naturalleaders are recognized and rewarded

Maintenance and enhancement require that the organizationrsquosgovernance body assumes ownership and program advocacy such astaking over the temporary steering grouprsquos functions addressingongoing policy and long-range planning concerns and maintainingnancial support The foundation for such ownership is laid duringthe readiness phase Each element becomes the organizationrsquos pro-perty as it is established during initial implementation The officialdeed of ownership is transferred as soon as the prototype is in place

Ownership however is no guarantee of institutionalizationVarious forces that can erode reforms are always at work Forinstance teams at a site experience turnover problems with commu-nication and sharing of resources are chronic competing interestsand the attractiveness of moving on to something new pull attentionand resources to other activity To minimize such problems stepsmust be taken to identify and solve them as quickly as is feasibleThis requires someone who has the time energy and expertise tomeet periodically with stakeholders to anticipate and amelioratethreats to a prototypersquos integrity

Over time mechanisms for planning implementation and coordi-nation are maintained by ensuring the activity is an official part ofthe infrastructure has appropriate leadership and is e ectively sup-ported Anyone who has worked on a school-based team to improve aliteracy program knows there must be a critical mass of teammembers so that the work load is manageable and a broad base ofinvolvement is ensured Also essential are adequate resources includ-ing time to learn the role and time to perform the functions reason-ably interesting tasks technical support for problem solvingrecognition and rewards for contributions immediate replacementwhen someone leaves continuing education to enhance team func-tioning and so forth Without serious attention to such matters theteamsrsquo morale and motivation will wane

Scaling up Reforms 319

Lessons Learned

Newly institutionalized approaches are seriously jeopardized in theabsence of dedicated ongoing capacity-building Of particular impor-tance are ways to rapidly and e ectively assimilate new arrivals at aschool (sta students families) This is a major concern at sites withconsiderable turnover or growth At such sites the majority of thoseinitially involved in implementing a new approach may be gonewithin a period of two to three years Whatever the mobility rate it isessential to design and maintain transition programs for newarrivals Initial welcoming and introductory orientations of coursemust be followed-up with ongoing support systems and intensivecapacity building related to understanding and valuing theapproaches the school has adopted We nd that all this is essentialnot only to maintain what has been adopted but also can contributeto establishing schools as caring environments

PHASE IV ONGOING EVOLUTION

Ongoing evolution of organizations and programs is the product ofe orts to account for accomplishments deal with changing times andconditions incorporate new knowledge and create a sense of renewalas the excitement of newness wears o and the demands of changesap energy As suggested already in part vigor and direction can bemaintained through continuing education especially exposure toideas that suggest a range of ways for evolving a program As thefollowing discussion indicates ongoing evolution also is fostered byevaluation designed to document accomplishments and provide feed-back designed to improve quality

Increased concern over accountability has advanced the wayevaluation is conceived (Posavac amp Carey 1989 Rossi amp Freeman1989 Scriven 1993 Sechrest amp Figueredo 1993 Shadish Jr Cook ampLeviton 1991 Stake 1967 1976 Stufflebeam amp Webster 1983 Weiss1995) At the same time social and political forces have literallyshaped the whole enterprise and in the process have narrowed theway professionals clients policymakers underwriters and thegeneral public think about program evaluation A prevailing cry isfor specic evidence of e ectiveness For schools this means imme-diate gains in literacy as measured by achievement tests Althoughunderstandable in light of the unlled promise of so many programsand the insatiable demands on limited public nances such simplisti-cally conceived accountability demands ignore the complexities ofdeveloping and scaling-up major reforms

320 L Taylor et al

Formative and Summative Evaluation

Evaluation of a prototype involves more than determining efficacyfor students Broadly stated it encompasses concerns about how toexpand the focus of evaluative research not only to contribute toimproving practice but also to aid in evolving practice and policy(General Accounting Office 1989 Lyon amp Moats 1997) To facilitateprogram development and organizational change the primary orien-tation for evaluation in the early phases is formative It is especiallyfocused on data gathering and analyses that can help improve pro-cedures Most of what is written about educational and psychosocialintervention however is oriented to summative evaluation and mea-suring outcomes for individuals such as improved reading achieve-ment scores Replicating approaches to improve literacy involve notonly changing individuals but changing organizations and systemsThus both individuals and systems must be evaluated

All this presumes appropriate mechanisms to provide and analyzeessential information To these ends a scale-up sta can help estab-lish an evaluation team and capacity building that prepares a schoolto conduct evaluation that enhances reforms The immediate focus ison successful program replication ultimately of course the empha-sis must be on student outcomes

Pursuing Results

Due to the increased interest in accountability many complex aimsare broken down into specic objectives Indeed short-range objec-tives stated in measurable terms generally assume a central role inplanning However short-range objectives are not ends in them-selves They are a small part of a particular goal and aim and some-times are prerequisites for moving on to a goal It is essential not tolose sight of the fact that many specic objectives are relativelysmall unrepresentative and often unimportant segments of the mostvalued aims that society has for its citizens and citizens have forthemselves

The problem is well exemplied by the narrow focus found inreviews analyses and reanalyses of data on early education (eg seeAlbee amp Gullotta 1997 Bond amp Compas 1989 Dryfoos 1990 Durlak1995 Elias 1997 Mitchell Seligson amp Marx 1989 Schorr 1988Slavin Karweit amp Madden 1989 Weissberg Gullotta HamptomRyan amp Adams 1997) As such work demonstrates overemphasis onevaluating the efficacy of underdeveloped prototypes draws resourcesaway from formative evaluation

Scaling up Reforms 321

With specic respect to scale-up the rst accomplishment is thereplication itself Have all facets been implemented How completelyhas each been implemented At how many locations The next set ofresults are any indications of progress for students such as improve-ments in attitudes toward school health attendance behavior andacademic achievement A nal set of evaluation concerns is thedegree to which student outcomes approximate societal standards

Lessons Learned

The process of evaluating results is costly in terms of nancial invest-ment the negative psychological impact on those evaluated and theways it can inappropriately reshape new approaches Cost-e ectiveoutcomes cannot be achieved in the absence of e ective prototypedevelopment and research Premature e orts to carry out com-prehensive summative evaluations clearly are not cost-e ective Anyreading and writing program will show poor results if it is evaluatedbefore teachers have mastered its application None of this of courseis an argument against evaluating results Rather it is meant tounderscore concerns and encourage greater attention to addressingthem

Once a prototype is established care must be taken to avoiddeveloping outcome evaluation as an adversarial process Due to thepolitical realities related to accountability one of the most per-plexing facets to negotiate is the time frame for summative evalu-ation The more complex the prototype the longer it takes and thecostlier it is to implement and evaluate Schools usually want quickprocesses and results and of course rarely can a ord costly innova-tions or lengthy di usion activity Compromises are inevitable butmust be arrived at with great care so as not to undermine the sub-stance of proposed changes

The psychology of evaluation suggests that an overemphasis onaccountability tends to produce negative reactions One possible wayto counter this may be to conceive evaluation as a way for everystakeholder to self-evaluate as a basis for quality improvement and asa way of getting credit for all that is accomplished Unfortunately asaccountability pressures increase we nd that replication of proto-types are guided more by what can be measured than by long-rangeaims That is demands for immediate accountability reshape prac-tices so that the emphasis shifts to immediate and readily measuredobjectives and away from fundamental purposes Over time this inap-propriately leads to radical revision of the underlying rationale for aprototype

322 L Taylor et al

CONCLUDING COMMENTS

Those who set out to change schools and schooling are confrontedwith two enormous tasks The rst is to develop prototypes Thesecond involves large-scale replication One without the other isinsufficient Yet considerably more attention is paid to developingand validating prototypes than to delineating and testing scale-upprocesses Clearly it is time to correct this deciency The ideas pre-sented in this article are meant to stimulate work on the problem andthereby to advance the cause of educational reform

Finally in fairness to those who labor for educational reform weall must remember that the quality of schooling family life and com-munity functioning spirals up or down as a function of the quality ofthe ongoing transactions among each Thus scale-up e orts relatedto educational reform must take place within the context of a poli-tical agenda that addresses ways to strengthen the family and com-munity infrastructure through strategies that enhance economicopportunity adult literacy and so forth What we need are policiesthat develop demonstrate and scale-up comprehensive multifacetedintegrated approaches that can e ectively address barriers to devel-opment learning and teaching

REFERENCES

Adelman H S (1993) School-linked mental health interventions Toward mechanismsfor service coordination and integration Journal of Community Psychology 21 309ndash319

Adelman H S (1996a) Restructuring education support services and integrating com-munity resources Beyond the full service school model School Psychology Review25 431ndash445

Adelman H S (1996b) Restructuring support services Toward a comprehensiveapproach Kent OH American School Health Association

Adelman H S amp Taylor L (1993a) Early Assistance for Students and FamiliesProgram Guidebook Los Angeles Early Assistance for Students and FamiliesProject Copies available from the School Mental Health Project Dept of Psychol-ogy UCLA

Adelman H S amp Taylor L (1993b) Learning problems and learning disabilities Moving forward Pacic Grove CA BrooksCole

Adelman H S amp Taylor L (1994) On understanding intervention in psychology andeducation Westport CT Praeger

Adelman H S amp Taylor L (1997a) Addressing barriers to learning Beyond school-linked services and full service schools American Journal of Orthopsychiatry 67408ndash421

Adelman H S amp Taylor L (1997b) Toward a scale-up model for replicating newapproaches to schooling Journal of Educational and Psychological Consultation 8197ndash230

Scaling up Reforms 311

is another to create conditions that allow for e ective participationOne such condition involves translating capacity building activityinto comprehensive programs for stakeholder development

The necessity of all this can be appreciated by thinking aboutintroducing a comprehensive approach for improving student literacy(Stringeld Ross amp Smith 1996) Such approaches involve major sys-temic changes that encompass intensive partnerships with parents (ortheir surrogates) and with various entities in the community such aslibraries youth development programs businesses the faith com-munity and so forth Substantive partnerships require a true sharingof leadership blending of resources and leadership training for pro-fessionals and nonprofessionals alike In communities where manyparents have little or no connection to the school major outreache orts are inevitable prerequisites to increasing home involvement inschool reform Parent outreach of course has not been very suc-cessful in many neighborhoods Our experience suggests that a neces-sary rst step in most cases is to o er programs and services thatassist the family in meeting its most pressing needs Furthermorethere is the matter of building parent competence to deal with plan-ning reforms and restructuring schools and for low income familiesthere is a need to nd ways to pay parents for the time they devote toserving on governance and other committees

Time is one of the most critical elements determining the successof scale-up Even if a prototype doesnrsquot call for restructuring theschool day the scale-up process does Substantial blocks of time areneeded for stakeholder capacity building and for individual and col-lective planning (National Education Commission on Time andLearning 1994) Particularly critical is the need for freeing-up tea-chers to learn new approaches For example e orts to make impor-tant revisions in literacy programs seem consistently undermined bynot providing enough time during the school day for the mentoring ofteachers and by the difficulty of carving out sufficient time to teachparents how to help their children Clearly a nonnegotiable condi-tion for engagement is a realistic plan for ensuring time to plan andbuild capacity

Lessons Learned

Complex interventions of course are seldom implemented in a com-pletely planned and linear manner The many practical and unfore-seen events that arise require exible problem-solving Articulationof a scale-up model can guide planning but those facilitating the

312 L Taylor et al

process must be prepared to capitalize on every opportunity that canmove the process ahead

Among the most fundamental lessons learned in carrying outPhase 1 has been the tendency of all parties to set actions into motionwithout taking sufficient time to lay the foundation needed for sub-stantive change In marketing new ideas it is tempting to accentuatetheir promising attributes and minimize complications In negotiatingagreements policy makers at a school site frequently are askedsimply for a go-ahead rather than for their informed commitmentSometimes they assent mainly to get extra resources sometimes theyare motivated by a desire to be seen by constituents as doing some-thing to improve the school This all tends to produce pressures forpremature implementation that results in the form rather than thesubstance of change especially when administrators are under thegun of political accountability measures that make unrealisticdemands for quick and dramatic results in studentsrsquo reading scores

Although formulation of policy and related agreements take con-siderable time and other resources their importance cannot be over-emphasized Failure to establish and successfully maintainsubstantive reforms in schools probably is attributable in greatmeasure to proceeding without strong and clear policy support

Another unfortunate trend we have found is the omission of in-depth planning for ongoing capacity building for change agents andteam members Mechanisms function only as well as the personnelwho operate them Such personnel must be recruited and developed inways that ensure appropriate motivation and capability and suffi-cient time must be redeployed so they can learn and carry out newfunctions e ectively (Peterson McCarthey amp Elmore 1996) Allchanges require constant care and feeding Those who steer theprocess must be motivated and competent not just initially but overtime The complexity of systemic change requires close monitoring ofmechanisms and immediate follow-up to address problems In particu-lar it means providing continuous personalized guidance andsupport to enhance knowledge and skills and counter anxiety frus-tration and other stressors To these ends adequate resource supportmust be provided (time space materials equipment) opportunitiesmust be available for increasing ability and generating a sense ofrenewed mission and personnel turnover must be addressed quicklyAll stakeholders can benet from e orts designed to increase levelsof competence and enhance motivation for working together Suche orts encompass four stages of stakeholder development orienta-tion foundation-building capacity-building and continuing educa-tion

Scaling up Reforms 313

There is no simple solution to the chronic problem of providingtime for creating readiness building capacity and planning Indeedrestructuring time represents one of the most difficult scale-up prob-lems Examples of how the problem might be addressed includefreeing up sta by establishing opportunities for students to spendtime pursuing activities such as music art and sports with special-ists or supervised by aides and community volunteers Alternativelyschool might start later or end earlier on a given day As these exam-ples suggest any approach will be controversial but if the problem isnot addressed satisfactorily successful replication of comprehensiveprototypes is unlikely

PHASE II INITIAL IMPLEMENTATION OF APROTOTYPE

Initial implementation involves adapting and phasing-in a programwith well-designed guidance and support If there is anything certainabout e orts to replicate a prototype it is that the process is stress-ful Some of the stress arises from the nature of the program some isinherent in the process of organizational change Coalitions must bedeveloped new working relationships established disruptive rumorsand information overload countered and interpersonal conictsresolved Short-term frustrations must be kept in perspective vis a visthe reform vision To help deal with all this temporary mechanismsare added to the organizational infrastructure They include (a) asite-based steering mechanism to guide and support replication (b) achange agent from the scale-up team working with site stakeholderson a change team to facilitate coalition building problem solvingand conict resolution and (c) mentors and coaches to model andteach elements of the prototype These structures are created tofacilitate replication and some are assimilated into a sitersquos infra-structure at the end of the initial implementation phase to supportinstitutionalization and ongoing evolution

A scale-up team and steering group work at a site with the schoolrsquosleadership specic planning groups and other stakeholders to formu-late phase-in plans steer program development and generallyprovide guidance and support for change Two major facets of thiswork are delineating a sequence for introducing major program ele-ments and outlining strategies to facilitate implementation Particu-lar attention is given to how to start with special emphasis onspecifying structures and resources for guidance and support For

314 L Taylor et al

instance in restructuring to better address barriers to learning therst steps at a school site involve creating processes to map analyzecoordinate and redeploy existing resources Special change mecha-nisms such as an organization facilitator and a resource coordinatingteam are created to guide and support the activity (Adelman 19931996a 1996b Adelman amp Taylor 1997a 1997b 1998)

Throughout this phase formative evaluation procedures are estab-lished to provide feedback for program development As noted abovee ective e orts to lsquolsquoreinventrsquorsquo schools require ensuring that allinvolved have the time to develop and institutionalize a soundprogram and that they are not penalized for unavoidable missteps Asa prototype is phased-in evaluation must not be thought of in termsof accountability Major systemic changes to improve literacy cantake years to develop Outcome e ectiveness is demonstrated afterthe program is in place The purpose of evaluation at this stage is toguide revision and ne-tuning of processes Formative evaluationsgather and analyze information relevant to changes in planning pro-cesses governance structures and policies and resources they alsofocus on implementation strategies and barriers program organiz-ation and staffing and initial outcomes Are teachers mastering thenew curricula for literacy Is there increased student engagement inreading Are parents involved in supporting reading at home Ifthings are not progressing satisfactorily why not Whatrsquos the down-side of the new approach

Well-designed organizational support and guidance is needed toenhance productivity minimize problems and accommodate individ-ual di erences This involves various forms of capacity building andpersonalized day-by-day facilitation Intensive coaching with somefollow-up consultation for instance are key processes so are mentor-ship and technical assistance Continuing education provides a criti-cal vehicle for enhancing productive changes generating renewaland countering burnout As new stakeholders arrive technologicaltools can be particularly useful in helping them catch up All thisactivity not only builds capacity but can foster networking and otherforms of task-related social and personal support as well as providea wide range of enrichment opportunities that enhance morale

If the steps discussed to this point are done well a sound founda-tion for initial implementation should be in place This initial phase-in period can however consume considerable e ort create specialproblems and may yield a temporary drop in some performance indi-cators Good day-by-day facilitation aims at minimizing such negativeimpact by e ectively addressing stakeholder motivation and capabil-ity and overcoming barriers to productive working relationships

Scaling up Reforms 315

Lessons Learned

Failure to take sufficient time to create readiness (Phase 1) can resultin implementing the form rather than the substance of a prototypeFor example we nd that change agents frequently are sent intoschools before essential policy support is enacted and before schoolleaders have assimilated and decided to support reforms Teams areconvened to assist with reforms (plan coordinate develop newapproaches) but the absence of supportive policy means substantivechanges are not accomplished As a result the initial motivation ofmany key team members wanes and other counterproductivedynamics arise All of this seems inevitable when initial implementa-tion proceeds without adequate policy support

Even in situations where sufficient readiness is created difficultiesfrequently arise due to a failure to keep enough stakeholders consis-tently moving in the direction of desired outcomes Comprehensivechange is usually achieved only when fairly high levels of positiveenergy can be mobilized over extended periods of time among a criti-cal mass of stakeholders sustained energy is appropriately directedthe process is supported with ongoing and well-conceived capacitybuilding and individuals are not pushed beyond their capabilitiesAnd because low and negative motivation are related to resistance tochange and poor functioning matching motivation is a rst-orderconsideration That is scale-up e orts must use strategies designed tomobilize and maintain proactive e ort and overcome barriers toworking relationships As in personalizing instruction approx-imating a good motivational t also requires matching capabilitiessuch as starting with fewer elements at sites at which resources arelimited and accounting for variability in stakeholdersrsquo competenceOver and over we nd too little attention is paid to these mattersThe result is failure to create an lsquolsquoenvironmentrsquorsquo that mobilizesdirects and then maintains stakeholder involvement

As with students the problem can be conceived as that of main-taining an appropriate match between the demands of the situationand individual motivation and capabilities In this respect we thinkthe construct of personalization o ers a concept around which toorganize thinking about facilitating change As stressed in the rstarticle in this issue personalization calls for systematically planningand implementing processes focused not only on knowledge andskills but on attitudes In particular it emphasizes the importance ofa primary and constant focus on ensuring positive attitudes Mobi-lization probably is best facilitated when procedures are perceived byindividuals as good ways to reach desired outcomes This requires

316 L Taylor et al

processes that can instigate and enhance an individualrsquos perceptionsof valued opportunities choice and control accomplishment andrelatedness to others Even if a task isnrsquot enjoyable expectation offeeling some sense of satisfaction related to process or outcome canbe a powerful intrinsic factor motivating individual behavior Taskpersistence for example can be facilitated by the expectation thatone will feel competent self-determining or more closely connectedto others From this perspective ensuring that individuals havevalued options a meaningful role in decision making feedback thatemphasizes progress toward desired outcomes and positive workingrelationships are among the most basic facilitation strategies(Adelman amp Taylor 1993b 1994 Deci amp Ryan 1985)

One other initial implementation problem that often arises is diffi-culty in establishing mechanisms to facilitate productive workingrelationships and identify and deal with problems quickly Forexample it is expected that change agents will encounter manyinstances of individual resistance and apathy interpersonal conictsand resentments (including lsquolsquous vs themrsquorsquo dynamics) rumors thatoveremphasize the negative and underestimate the positive and indi-viduals who are frequent faultnders Such problems seriouslyimpede e ective replication The roots of some of these problemsoften are present at a site prior to scale-up change simply o ers anew focus and perhaps magnies troubling matters Other problemsare a direct product of the activities and relationships that the scale-up process engenders Given the inevitability of such problems build-ing and maintaining working relationships need to be among themost basic concerns for those who have responsibility for scale-up Inparticular considerable attention must be paid to enhancing themotivational readiness and capability of those who are to worktogether and ensure there is an appropriate infrastructure to guideand support working relationships Proactively this requires problemprevention mechanisms that help create an atmosphere where defen-siveness is curtailed and positive rapport is engendered The point isto enhance attitudes knowledge and skills that foster interpersonalconnections and a sense of community Reactively the emphasis is onproblem solving resolving conict and providing ongoing support torebuild relationships Policies must encourage problem-solving-oriented critiques safeguards that protect those making changesappreciation for e ort and celebration of progress We nd thateveryone understands such matters but the culture at many schoolsites is more attuned to problem naming and analyzing than to antici-pating preventing and solving problems that arise around workingrelationships

Scaling up Reforms 317

Those responsible for systemic change need to spend as much timeas necessary ensuring that a schoolrsquos infrastructure is ready toprevent and ameliorate problems Special attention must be paid toensuring that problem solving mechanisms and communication pro-cesses are in place and properly sta ed and that stakeholders arewell informed about how to use the procedures Furthermore somestakeholders may have to be encouraged to interact in ways thatconvey genuine empathy warmth and mutual regard and respectwith a view to creating and maintaining a positive working climateand a psychological sense of community

At times we nd it necessary to target a specic problem anddesignated persons In some instances rather simple strategies aree ective For example most motivated individuals can be directlytaught ways to improve understanding and communication and avoidor resolve conicts that interfere with working relationships Inother instances however signicant remedial action is necessary aswhen overcoming barriers to a working relationship involvescountering negative attitudes Helpful in this regard are analysessuch as that by Sue and Zane (1987) that suggest how to demonstratethat something of value can be gained from individuals workingtogether and how to establish each participantrsquos credibility (eg bymaximizing task-focus and positive outcomes)

PHASE III INSTITUTIONALIZING THE PROTOTYPE

Maintaining and enhancing changes can be as difficult as makingthem in the rst place The history of education reform is one offailure to foster promising prototypes in substantive ways and overan extended period of times (Tyack amp Cuban 1995) Institutionalizinga prototype entails ensuring that the organization assumes long-termownership and that there is a blueprint for countering forces that canerode the changes Moreover institutionalization is more than atechnical process It requires assimilation of and ongoing adherenceto the values inherent in the prototypersquos underlying rationale Thefocus of course is not just on maintenance the point is to moveforward by enhancing productive changes and generating a sense ofrenewal as needed Critical in all this are specic plans that guar-antee ongoing and enhanced leadership and delineate ways in whichplanning implementation coordination and continuing educationmechanisms are maintained

Some Major Tasks

Whose responsibility is it to advocate for maintaining and evolving a

318 L Taylor et al

replicated prototype for improving literacy As problems arise whoseresponsibility is it to lead the way in resolving them Leadership isthe key here by both official leaders such as administrators mentorsta union chapter chairs and elected parent representatives aswell as natural leaders such as reading and writing teachers Obvi-ously official and natural leaders are not mutually exclusive groupsAt this phase both types of leadership are essential to ensure a broadenough base for ongoing advocacy problem solving enhancementand renewal Official leaders provide a legitimate power base asvarious interests compete for the organizationrsquos limited resourcesand they play a key role in ensuring that the contributions of naturalleaders are recognized and rewarded

Maintenance and enhancement require that the organizationrsquosgovernance body assumes ownership and program advocacy such astaking over the temporary steering grouprsquos functions addressingongoing policy and long-range planning concerns and maintainingnancial support The foundation for such ownership is laid duringthe readiness phase Each element becomes the organizationrsquos pro-perty as it is established during initial implementation The officialdeed of ownership is transferred as soon as the prototype is in place

Ownership however is no guarantee of institutionalizationVarious forces that can erode reforms are always at work Forinstance teams at a site experience turnover problems with commu-nication and sharing of resources are chronic competing interestsand the attractiveness of moving on to something new pull attentionand resources to other activity To minimize such problems stepsmust be taken to identify and solve them as quickly as is feasibleThis requires someone who has the time energy and expertise tomeet periodically with stakeholders to anticipate and amelioratethreats to a prototypersquos integrity

Over time mechanisms for planning implementation and coordi-nation are maintained by ensuring the activity is an official part ofthe infrastructure has appropriate leadership and is e ectively sup-ported Anyone who has worked on a school-based team to improve aliteracy program knows there must be a critical mass of teammembers so that the work load is manageable and a broad base ofinvolvement is ensured Also essential are adequate resources includ-ing time to learn the role and time to perform the functions reason-ably interesting tasks technical support for problem solvingrecognition and rewards for contributions immediate replacementwhen someone leaves continuing education to enhance team func-tioning and so forth Without serious attention to such matters theteamsrsquo morale and motivation will wane

Scaling up Reforms 319

Lessons Learned

Newly institutionalized approaches are seriously jeopardized in theabsence of dedicated ongoing capacity-building Of particular impor-tance are ways to rapidly and e ectively assimilate new arrivals at aschool (sta students families) This is a major concern at sites withconsiderable turnover or growth At such sites the majority of thoseinitially involved in implementing a new approach may be gonewithin a period of two to three years Whatever the mobility rate it isessential to design and maintain transition programs for newarrivals Initial welcoming and introductory orientations of coursemust be followed-up with ongoing support systems and intensivecapacity building related to understanding and valuing theapproaches the school has adopted We nd that all this is essentialnot only to maintain what has been adopted but also can contributeto establishing schools as caring environments

PHASE IV ONGOING EVOLUTION

Ongoing evolution of organizations and programs is the product ofe orts to account for accomplishments deal with changing times andconditions incorporate new knowledge and create a sense of renewalas the excitement of newness wears o and the demands of changesap energy As suggested already in part vigor and direction can bemaintained through continuing education especially exposure toideas that suggest a range of ways for evolving a program As thefollowing discussion indicates ongoing evolution also is fostered byevaluation designed to document accomplishments and provide feed-back designed to improve quality

Increased concern over accountability has advanced the wayevaluation is conceived (Posavac amp Carey 1989 Rossi amp Freeman1989 Scriven 1993 Sechrest amp Figueredo 1993 Shadish Jr Cook ampLeviton 1991 Stake 1967 1976 Stufflebeam amp Webster 1983 Weiss1995) At the same time social and political forces have literallyshaped the whole enterprise and in the process have narrowed theway professionals clients policymakers underwriters and thegeneral public think about program evaluation A prevailing cry isfor specic evidence of e ectiveness For schools this means imme-diate gains in literacy as measured by achievement tests Althoughunderstandable in light of the unlled promise of so many programsand the insatiable demands on limited public nances such simplisti-cally conceived accountability demands ignore the complexities ofdeveloping and scaling-up major reforms

320 L Taylor et al

Formative and Summative Evaluation

Evaluation of a prototype involves more than determining efficacyfor students Broadly stated it encompasses concerns about how toexpand the focus of evaluative research not only to contribute toimproving practice but also to aid in evolving practice and policy(General Accounting Office 1989 Lyon amp Moats 1997) To facilitateprogram development and organizational change the primary orien-tation for evaluation in the early phases is formative It is especiallyfocused on data gathering and analyses that can help improve pro-cedures Most of what is written about educational and psychosocialintervention however is oriented to summative evaluation and mea-suring outcomes for individuals such as improved reading achieve-ment scores Replicating approaches to improve literacy involve notonly changing individuals but changing organizations and systemsThus both individuals and systems must be evaluated

All this presumes appropriate mechanisms to provide and analyzeessential information To these ends a scale-up sta can help estab-lish an evaluation team and capacity building that prepares a schoolto conduct evaluation that enhances reforms The immediate focus ison successful program replication ultimately of course the empha-sis must be on student outcomes

Pursuing Results

Due to the increased interest in accountability many complex aimsare broken down into specic objectives Indeed short-range objec-tives stated in measurable terms generally assume a central role inplanning However short-range objectives are not ends in them-selves They are a small part of a particular goal and aim and some-times are prerequisites for moving on to a goal It is essential not tolose sight of the fact that many specic objectives are relativelysmall unrepresentative and often unimportant segments of the mostvalued aims that society has for its citizens and citizens have forthemselves

The problem is well exemplied by the narrow focus found inreviews analyses and reanalyses of data on early education (eg seeAlbee amp Gullotta 1997 Bond amp Compas 1989 Dryfoos 1990 Durlak1995 Elias 1997 Mitchell Seligson amp Marx 1989 Schorr 1988Slavin Karweit amp Madden 1989 Weissberg Gullotta HamptomRyan amp Adams 1997) As such work demonstrates overemphasis onevaluating the efficacy of underdeveloped prototypes draws resourcesaway from formative evaluation

Scaling up Reforms 321

With specic respect to scale-up the rst accomplishment is thereplication itself Have all facets been implemented How completelyhas each been implemented At how many locations The next set ofresults are any indications of progress for students such as improve-ments in attitudes toward school health attendance behavior andacademic achievement A nal set of evaluation concerns is thedegree to which student outcomes approximate societal standards

Lessons Learned

The process of evaluating results is costly in terms of nancial invest-ment the negative psychological impact on those evaluated and theways it can inappropriately reshape new approaches Cost-e ectiveoutcomes cannot be achieved in the absence of e ective prototypedevelopment and research Premature e orts to carry out com-prehensive summative evaluations clearly are not cost-e ective Anyreading and writing program will show poor results if it is evaluatedbefore teachers have mastered its application None of this of courseis an argument against evaluating results Rather it is meant tounderscore concerns and encourage greater attention to addressingthem

Once a prototype is established care must be taken to avoiddeveloping outcome evaluation as an adversarial process Due to thepolitical realities related to accountability one of the most per-plexing facets to negotiate is the time frame for summative evalu-ation The more complex the prototype the longer it takes and thecostlier it is to implement and evaluate Schools usually want quickprocesses and results and of course rarely can a ord costly innova-tions or lengthy di usion activity Compromises are inevitable butmust be arrived at with great care so as not to undermine the sub-stance of proposed changes

The psychology of evaluation suggests that an overemphasis onaccountability tends to produce negative reactions One possible wayto counter this may be to conceive evaluation as a way for everystakeholder to self-evaluate as a basis for quality improvement and asa way of getting credit for all that is accomplished Unfortunately asaccountability pressures increase we nd that replication of proto-types are guided more by what can be measured than by long-rangeaims That is demands for immediate accountability reshape prac-tices so that the emphasis shifts to immediate and readily measuredobjectives and away from fundamental purposes Over time this inap-propriately leads to radical revision of the underlying rationale for aprototype

322 L Taylor et al

CONCLUDING COMMENTS

Those who set out to change schools and schooling are confrontedwith two enormous tasks The rst is to develop prototypes Thesecond involves large-scale replication One without the other isinsufficient Yet considerably more attention is paid to developingand validating prototypes than to delineating and testing scale-upprocesses Clearly it is time to correct this deciency The ideas pre-sented in this article are meant to stimulate work on the problem andthereby to advance the cause of educational reform

Finally in fairness to those who labor for educational reform weall must remember that the quality of schooling family life and com-munity functioning spirals up or down as a function of the quality ofthe ongoing transactions among each Thus scale-up e orts relatedto educational reform must take place within the context of a poli-tical agenda that addresses ways to strengthen the family and com-munity infrastructure through strategies that enhance economicopportunity adult literacy and so forth What we need are policiesthat develop demonstrate and scale-up comprehensive multifacetedintegrated approaches that can e ectively address barriers to devel-opment learning and teaching

REFERENCES

Adelman H S (1993) School-linked mental health interventions Toward mechanismsfor service coordination and integration Journal of Community Psychology 21 309ndash319

Adelman H S (1996a) Restructuring education support services and integrating com-munity resources Beyond the full service school model School Psychology Review25 431ndash445

Adelman H S (1996b) Restructuring support services Toward a comprehensiveapproach Kent OH American School Health Association

Adelman H S amp Taylor L (1993a) Early Assistance for Students and FamiliesProgram Guidebook Los Angeles Early Assistance for Students and FamiliesProject Copies available from the School Mental Health Project Dept of Psychol-ogy UCLA

Adelman H S amp Taylor L (1993b) Learning problems and learning disabilities Moving forward Pacic Grove CA BrooksCole

Adelman H S amp Taylor L (1994) On understanding intervention in psychology andeducation Westport CT Praeger

Adelman H S amp Taylor L (1997a) Addressing barriers to learning Beyond school-linked services and full service schools American Journal of Orthopsychiatry 67408ndash421

Adelman H S amp Taylor L (1997b) Toward a scale-up model for replicating newapproaches to schooling Journal of Educational and Psychological Consultation 8197ndash230

312 L Taylor et al

process must be prepared to capitalize on every opportunity that canmove the process ahead

Among the most fundamental lessons learned in carrying outPhase 1 has been the tendency of all parties to set actions into motionwithout taking sufficient time to lay the foundation needed for sub-stantive change In marketing new ideas it is tempting to accentuatetheir promising attributes and minimize complications In negotiatingagreements policy makers at a school site frequently are askedsimply for a go-ahead rather than for their informed commitmentSometimes they assent mainly to get extra resources sometimes theyare motivated by a desire to be seen by constituents as doing some-thing to improve the school This all tends to produce pressures forpremature implementation that results in the form rather than thesubstance of change especially when administrators are under thegun of political accountability measures that make unrealisticdemands for quick and dramatic results in studentsrsquo reading scores

Although formulation of policy and related agreements take con-siderable time and other resources their importance cannot be over-emphasized Failure to establish and successfully maintainsubstantive reforms in schools probably is attributable in greatmeasure to proceeding without strong and clear policy support

Another unfortunate trend we have found is the omission of in-depth planning for ongoing capacity building for change agents andteam members Mechanisms function only as well as the personnelwho operate them Such personnel must be recruited and developed inways that ensure appropriate motivation and capability and suffi-cient time must be redeployed so they can learn and carry out newfunctions e ectively (Peterson McCarthey amp Elmore 1996) Allchanges require constant care and feeding Those who steer theprocess must be motivated and competent not just initially but overtime The complexity of systemic change requires close monitoring ofmechanisms and immediate follow-up to address problems In particu-lar it means providing continuous personalized guidance andsupport to enhance knowledge and skills and counter anxiety frus-tration and other stressors To these ends adequate resource supportmust be provided (time space materials equipment) opportunitiesmust be available for increasing ability and generating a sense ofrenewed mission and personnel turnover must be addressed quicklyAll stakeholders can benet from e orts designed to increase levelsof competence and enhance motivation for working together Suche orts encompass four stages of stakeholder development orienta-tion foundation-building capacity-building and continuing educa-tion

Scaling up Reforms 313

There is no simple solution to the chronic problem of providingtime for creating readiness building capacity and planning Indeedrestructuring time represents one of the most difficult scale-up prob-lems Examples of how the problem might be addressed includefreeing up sta by establishing opportunities for students to spendtime pursuing activities such as music art and sports with special-ists or supervised by aides and community volunteers Alternativelyschool might start later or end earlier on a given day As these exam-ples suggest any approach will be controversial but if the problem isnot addressed satisfactorily successful replication of comprehensiveprototypes is unlikely

PHASE II INITIAL IMPLEMENTATION OF APROTOTYPE

Initial implementation involves adapting and phasing-in a programwith well-designed guidance and support If there is anything certainabout e orts to replicate a prototype it is that the process is stress-ful Some of the stress arises from the nature of the program some isinherent in the process of organizational change Coalitions must bedeveloped new working relationships established disruptive rumorsand information overload countered and interpersonal conictsresolved Short-term frustrations must be kept in perspective vis a visthe reform vision To help deal with all this temporary mechanismsare added to the organizational infrastructure They include (a) asite-based steering mechanism to guide and support replication (b) achange agent from the scale-up team working with site stakeholderson a change team to facilitate coalition building problem solvingand conict resolution and (c) mentors and coaches to model andteach elements of the prototype These structures are created tofacilitate replication and some are assimilated into a sitersquos infra-structure at the end of the initial implementation phase to supportinstitutionalization and ongoing evolution

A scale-up team and steering group work at a site with the schoolrsquosleadership specic planning groups and other stakeholders to formu-late phase-in plans steer program development and generallyprovide guidance and support for change Two major facets of thiswork are delineating a sequence for introducing major program ele-ments and outlining strategies to facilitate implementation Particu-lar attention is given to how to start with special emphasis onspecifying structures and resources for guidance and support For

314 L Taylor et al

instance in restructuring to better address barriers to learning therst steps at a school site involve creating processes to map analyzecoordinate and redeploy existing resources Special change mecha-nisms such as an organization facilitator and a resource coordinatingteam are created to guide and support the activity (Adelman 19931996a 1996b Adelman amp Taylor 1997a 1997b 1998)

Throughout this phase formative evaluation procedures are estab-lished to provide feedback for program development As noted abovee ective e orts to lsquolsquoreinventrsquorsquo schools require ensuring that allinvolved have the time to develop and institutionalize a soundprogram and that they are not penalized for unavoidable missteps Asa prototype is phased-in evaluation must not be thought of in termsof accountability Major systemic changes to improve literacy cantake years to develop Outcome e ectiveness is demonstrated afterthe program is in place The purpose of evaluation at this stage is toguide revision and ne-tuning of processes Formative evaluationsgather and analyze information relevant to changes in planning pro-cesses governance structures and policies and resources they alsofocus on implementation strategies and barriers program organiz-ation and staffing and initial outcomes Are teachers mastering thenew curricula for literacy Is there increased student engagement inreading Are parents involved in supporting reading at home Ifthings are not progressing satisfactorily why not Whatrsquos the down-side of the new approach

Well-designed organizational support and guidance is needed toenhance productivity minimize problems and accommodate individ-ual di erences This involves various forms of capacity building andpersonalized day-by-day facilitation Intensive coaching with somefollow-up consultation for instance are key processes so are mentor-ship and technical assistance Continuing education provides a criti-cal vehicle for enhancing productive changes generating renewaland countering burnout As new stakeholders arrive technologicaltools can be particularly useful in helping them catch up All thisactivity not only builds capacity but can foster networking and otherforms of task-related social and personal support as well as providea wide range of enrichment opportunities that enhance morale

If the steps discussed to this point are done well a sound founda-tion for initial implementation should be in place This initial phase-in period can however consume considerable e ort create specialproblems and may yield a temporary drop in some performance indi-cators Good day-by-day facilitation aims at minimizing such negativeimpact by e ectively addressing stakeholder motivation and capabil-ity and overcoming barriers to productive working relationships

Scaling up Reforms 315

Lessons Learned

Failure to take sufficient time to create readiness (Phase 1) can resultin implementing the form rather than the substance of a prototypeFor example we nd that change agents frequently are sent intoschools before essential policy support is enacted and before schoolleaders have assimilated and decided to support reforms Teams areconvened to assist with reforms (plan coordinate develop newapproaches) but the absence of supportive policy means substantivechanges are not accomplished As a result the initial motivation ofmany key team members wanes and other counterproductivedynamics arise All of this seems inevitable when initial implementa-tion proceeds without adequate policy support

Even in situations where sufficient readiness is created difficultiesfrequently arise due to a failure to keep enough stakeholders consis-tently moving in the direction of desired outcomes Comprehensivechange is usually achieved only when fairly high levels of positiveenergy can be mobilized over extended periods of time among a criti-cal mass of stakeholders sustained energy is appropriately directedthe process is supported with ongoing and well-conceived capacitybuilding and individuals are not pushed beyond their capabilitiesAnd because low and negative motivation are related to resistance tochange and poor functioning matching motivation is a rst-orderconsideration That is scale-up e orts must use strategies designed tomobilize and maintain proactive e ort and overcome barriers toworking relationships As in personalizing instruction approx-imating a good motivational t also requires matching capabilitiessuch as starting with fewer elements at sites at which resources arelimited and accounting for variability in stakeholdersrsquo competenceOver and over we nd too little attention is paid to these mattersThe result is failure to create an lsquolsquoenvironmentrsquorsquo that mobilizesdirects and then maintains stakeholder involvement

As with students the problem can be conceived as that of main-taining an appropriate match between the demands of the situationand individual motivation and capabilities In this respect we thinkthe construct of personalization o ers a concept around which toorganize thinking about facilitating change As stressed in the rstarticle in this issue personalization calls for systematically planningand implementing processes focused not only on knowledge andskills but on attitudes In particular it emphasizes the importance ofa primary and constant focus on ensuring positive attitudes Mobi-lization probably is best facilitated when procedures are perceived byindividuals as good ways to reach desired outcomes This requires

316 L Taylor et al

processes that can instigate and enhance an individualrsquos perceptionsof valued opportunities choice and control accomplishment andrelatedness to others Even if a task isnrsquot enjoyable expectation offeeling some sense of satisfaction related to process or outcome canbe a powerful intrinsic factor motivating individual behavior Taskpersistence for example can be facilitated by the expectation thatone will feel competent self-determining or more closely connectedto others From this perspective ensuring that individuals havevalued options a meaningful role in decision making feedback thatemphasizes progress toward desired outcomes and positive workingrelationships are among the most basic facilitation strategies(Adelman amp Taylor 1993b 1994 Deci amp Ryan 1985)

One other initial implementation problem that often arises is diffi-culty in establishing mechanisms to facilitate productive workingrelationships and identify and deal with problems quickly Forexample it is expected that change agents will encounter manyinstances of individual resistance and apathy interpersonal conictsand resentments (including lsquolsquous vs themrsquorsquo dynamics) rumors thatoveremphasize the negative and underestimate the positive and indi-viduals who are frequent faultnders Such problems seriouslyimpede e ective replication The roots of some of these problemsoften are present at a site prior to scale-up change simply o ers anew focus and perhaps magnies troubling matters Other problemsare a direct product of the activities and relationships that the scale-up process engenders Given the inevitability of such problems build-ing and maintaining working relationships need to be among themost basic concerns for those who have responsibility for scale-up Inparticular considerable attention must be paid to enhancing themotivational readiness and capability of those who are to worktogether and ensure there is an appropriate infrastructure to guideand support working relationships Proactively this requires problemprevention mechanisms that help create an atmosphere where defen-siveness is curtailed and positive rapport is engendered The point isto enhance attitudes knowledge and skills that foster interpersonalconnections and a sense of community Reactively the emphasis is onproblem solving resolving conict and providing ongoing support torebuild relationships Policies must encourage problem-solving-oriented critiques safeguards that protect those making changesappreciation for e ort and celebration of progress We nd thateveryone understands such matters but the culture at many schoolsites is more attuned to problem naming and analyzing than to antici-pating preventing and solving problems that arise around workingrelationships

Scaling up Reforms 317

Those responsible for systemic change need to spend as much timeas necessary ensuring that a schoolrsquos infrastructure is ready toprevent and ameliorate problems Special attention must be paid toensuring that problem solving mechanisms and communication pro-cesses are in place and properly sta ed and that stakeholders arewell informed about how to use the procedures Furthermore somestakeholders may have to be encouraged to interact in ways thatconvey genuine empathy warmth and mutual regard and respectwith a view to creating and maintaining a positive working climateand a psychological sense of community

At times we nd it necessary to target a specic problem anddesignated persons In some instances rather simple strategies aree ective For example most motivated individuals can be directlytaught ways to improve understanding and communication and avoidor resolve conicts that interfere with working relationships Inother instances however signicant remedial action is necessary aswhen overcoming barriers to a working relationship involvescountering negative attitudes Helpful in this regard are analysessuch as that by Sue and Zane (1987) that suggest how to demonstratethat something of value can be gained from individuals workingtogether and how to establish each participantrsquos credibility (eg bymaximizing task-focus and positive outcomes)

PHASE III INSTITUTIONALIZING THE PROTOTYPE

Maintaining and enhancing changes can be as difficult as makingthem in the rst place The history of education reform is one offailure to foster promising prototypes in substantive ways and overan extended period of times (Tyack amp Cuban 1995) Institutionalizinga prototype entails ensuring that the organization assumes long-termownership and that there is a blueprint for countering forces that canerode the changes Moreover institutionalization is more than atechnical process It requires assimilation of and ongoing adherenceto the values inherent in the prototypersquos underlying rationale Thefocus of course is not just on maintenance the point is to moveforward by enhancing productive changes and generating a sense ofrenewal as needed Critical in all this are specic plans that guar-antee ongoing and enhanced leadership and delineate ways in whichplanning implementation coordination and continuing educationmechanisms are maintained

Some Major Tasks

Whose responsibility is it to advocate for maintaining and evolving a

318 L Taylor et al

replicated prototype for improving literacy As problems arise whoseresponsibility is it to lead the way in resolving them Leadership isthe key here by both official leaders such as administrators mentorsta union chapter chairs and elected parent representatives aswell as natural leaders such as reading and writing teachers Obvi-ously official and natural leaders are not mutually exclusive groupsAt this phase both types of leadership are essential to ensure a broadenough base for ongoing advocacy problem solving enhancementand renewal Official leaders provide a legitimate power base asvarious interests compete for the organizationrsquos limited resourcesand they play a key role in ensuring that the contributions of naturalleaders are recognized and rewarded

Maintenance and enhancement require that the organizationrsquosgovernance body assumes ownership and program advocacy such astaking over the temporary steering grouprsquos functions addressingongoing policy and long-range planning concerns and maintainingnancial support The foundation for such ownership is laid duringthe readiness phase Each element becomes the organizationrsquos pro-perty as it is established during initial implementation The officialdeed of ownership is transferred as soon as the prototype is in place

Ownership however is no guarantee of institutionalizationVarious forces that can erode reforms are always at work Forinstance teams at a site experience turnover problems with commu-nication and sharing of resources are chronic competing interestsand the attractiveness of moving on to something new pull attentionand resources to other activity To minimize such problems stepsmust be taken to identify and solve them as quickly as is feasibleThis requires someone who has the time energy and expertise tomeet periodically with stakeholders to anticipate and amelioratethreats to a prototypersquos integrity

Over time mechanisms for planning implementation and coordi-nation are maintained by ensuring the activity is an official part ofthe infrastructure has appropriate leadership and is e ectively sup-ported Anyone who has worked on a school-based team to improve aliteracy program knows there must be a critical mass of teammembers so that the work load is manageable and a broad base ofinvolvement is ensured Also essential are adequate resources includ-ing time to learn the role and time to perform the functions reason-ably interesting tasks technical support for problem solvingrecognition and rewards for contributions immediate replacementwhen someone leaves continuing education to enhance team func-tioning and so forth Without serious attention to such matters theteamsrsquo morale and motivation will wane

Scaling up Reforms 319

Lessons Learned

Newly institutionalized approaches are seriously jeopardized in theabsence of dedicated ongoing capacity-building Of particular impor-tance are ways to rapidly and e ectively assimilate new arrivals at aschool (sta students families) This is a major concern at sites withconsiderable turnover or growth At such sites the majority of thoseinitially involved in implementing a new approach may be gonewithin a period of two to three years Whatever the mobility rate it isessential to design and maintain transition programs for newarrivals Initial welcoming and introductory orientations of coursemust be followed-up with ongoing support systems and intensivecapacity building related to understanding and valuing theapproaches the school has adopted We nd that all this is essentialnot only to maintain what has been adopted but also can contributeto establishing schools as caring environments

PHASE IV ONGOING EVOLUTION

Ongoing evolution of organizations and programs is the product ofe orts to account for accomplishments deal with changing times andconditions incorporate new knowledge and create a sense of renewalas the excitement of newness wears o and the demands of changesap energy As suggested already in part vigor and direction can bemaintained through continuing education especially exposure toideas that suggest a range of ways for evolving a program As thefollowing discussion indicates ongoing evolution also is fostered byevaluation designed to document accomplishments and provide feed-back designed to improve quality

Increased concern over accountability has advanced the wayevaluation is conceived (Posavac amp Carey 1989 Rossi amp Freeman1989 Scriven 1993 Sechrest amp Figueredo 1993 Shadish Jr Cook ampLeviton 1991 Stake 1967 1976 Stufflebeam amp Webster 1983 Weiss1995) At the same time social and political forces have literallyshaped the whole enterprise and in the process have narrowed theway professionals clients policymakers underwriters and thegeneral public think about program evaluation A prevailing cry isfor specic evidence of e ectiveness For schools this means imme-diate gains in literacy as measured by achievement tests Althoughunderstandable in light of the unlled promise of so many programsand the insatiable demands on limited public nances such simplisti-cally conceived accountability demands ignore the complexities ofdeveloping and scaling-up major reforms

320 L Taylor et al

Formative and Summative Evaluation

Evaluation of a prototype involves more than determining efficacyfor students Broadly stated it encompasses concerns about how toexpand the focus of evaluative research not only to contribute toimproving practice but also to aid in evolving practice and policy(General Accounting Office 1989 Lyon amp Moats 1997) To facilitateprogram development and organizational change the primary orien-tation for evaluation in the early phases is formative It is especiallyfocused on data gathering and analyses that can help improve pro-cedures Most of what is written about educational and psychosocialintervention however is oriented to summative evaluation and mea-suring outcomes for individuals such as improved reading achieve-ment scores Replicating approaches to improve literacy involve notonly changing individuals but changing organizations and systemsThus both individuals and systems must be evaluated

All this presumes appropriate mechanisms to provide and analyzeessential information To these ends a scale-up sta can help estab-lish an evaluation team and capacity building that prepares a schoolto conduct evaluation that enhances reforms The immediate focus ison successful program replication ultimately of course the empha-sis must be on student outcomes

Pursuing Results

Due to the increased interest in accountability many complex aimsare broken down into specic objectives Indeed short-range objec-tives stated in measurable terms generally assume a central role inplanning However short-range objectives are not ends in them-selves They are a small part of a particular goal and aim and some-times are prerequisites for moving on to a goal It is essential not tolose sight of the fact that many specic objectives are relativelysmall unrepresentative and often unimportant segments of the mostvalued aims that society has for its citizens and citizens have forthemselves

The problem is well exemplied by the narrow focus found inreviews analyses and reanalyses of data on early education (eg seeAlbee amp Gullotta 1997 Bond amp Compas 1989 Dryfoos 1990 Durlak1995 Elias 1997 Mitchell Seligson amp Marx 1989 Schorr 1988Slavin Karweit amp Madden 1989 Weissberg Gullotta HamptomRyan amp Adams 1997) As such work demonstrates overemphasis onevaluating the efficacy of underdeveloped prototypes draws resourcesaway from formative evaluation

Scaling up Reforms 321

With specic respect to scale-up the rst accomplishment is thereplication itself Have all facets been implemented How completelyhas each been implemented At how many locations The next set ofresults are any indications of progress for students such as improve-ments in attitudes toward school health attendance behavior andacademic achievement A nal set of evaluation concerns is thedegree to which student outcomes approximate societal standards

Lessons Learned

The process of evaluating results is costly in terms of nancial invest-ment the negative psychological impact on those evaluated and theways it can inappropriately reshape new approaches Cost-e ectiveoutcomes cannot be achieved in the absence of e ective prototypedevelopment and research Premature e orts to carry out com-prehensive summative evaluations clearly are not cost-e ective Anyreading and writing program will show poor results if it is evaluatedbefore teachers have mastered its application None of this of courseis an argument against evaluating results Rather it is meant tounderscore concerns and encourage greater attention to addressingthem

Once a prototype is established care must be taken to avoiddeveloping outcome evaluation as an adversarial process Due to thepolitical realities related to accountability one of the most per-plexing facets to negotiate is the time frame for summative evalu-ation The more complex the prototype the longer it takes and thecostlier it is to implement and evaluate Schools usually want quickprocesses and results and of course rarely can a ord costly innova-tions or lengthy di usion activity Compromises are inevitable butmust be arrived at with great care so as not to undermine the sub-stance of proposed changes

The psychology of evaluation suggests that an overemphasis onaccountability tends to produce negative reactions One possible wayto counter this may be to conceive evaluation as a way for everystakeholder to self-evaluate as a basis for quality improvement and asa way of getting credit for all that is accomplished Unfortunately asaccountability pressures increase we nd that replication of proto-types are guided more by what can be measured than by long-rangeaims That is demands for immediate accountability reshape prac-tices so that the emphasis shifts to immediate and readily measuredobjectives and away from fundamental purposes Over time this inap-propriately leads to radical revision of the underlying rationale for aprototype

322 L Taylor et al

CONCLUDING COMMENTS

Those who set out to change schools and schooling are confrontedwith two enormous tasks The rst is to develop prototypes Thesecond involves large-scale replication One without the other isinsufficient Yet considerably more attention is paid to developingand validating prototypes than to delineating and testing scale-upprocesses Clearly it is time to correct this deciency The ideas pre-sented in this article are meant to stimulate work on the problem andthereby to advance the cause of educational reform

Finally in fairness to those who labor for educational reform weall must remember that the quality of schooling family life and com-munity functioning spirals up or down as a function of the quality ofthe ongoing transactions among each Thus scale-up e orts relatedto educational reform must take place within the context of a poli-tical agenda that addresses ways to strengthen the family and com-munity infrastructure through strategies that enhance economicopportunity adult literacy and so forth What we need are policiesthat develop demonstrate and scale-up comprehensive multifacetedintegrated approaches that can e ectively address barriers to devel-opment learning and teaching

REFERENCES

Adelman H S (1993) School-linked mental health interventions Toward mechanismsfor service coordination and integration Journal of Community Psychology 21 309ndash319

Adelman H S (1996a) Restructuring education support services and integrating com-munity resources Beyond the full service school model School Psychology Review25 431ndash445

Adelman H S (1996b) Restructuring support services Toward a comprehensiveapproach Kent OH American School Health Association

Adelman H S amp Taylor L (1993a) Early Assistance for Students and FamiliesProgram Guidebook Los Angeles Early Assistance for Students and FamiliesProject Copies available from the School Mental Health Project Dept of Psychol-ogy UCLA

Adelman H S amp Taylor L (1993b) Learning problems and learning disabilities Moving forward Pacic Grove CA BrooksCole

Adelman H S amp Taylor L (1994) On understanding intervention in psychology andeducation Westport CT Praeger

Adelman H S amp Taylor L (1997a) Addressing barriers to learning Beyond school-linked services and full service schools American Journal of Orthopsychiatry 67408ndash421

Adelman H S amp Taylor L (1997b) Toward a scale-up model for replicating newapproaches to schooling Journal of Educational and Psychological Consultation 8197ndash230

Scaling up Reforms 313

There is no simple solution to the chronic problem of providingtime for creating readiness building capacity and planning Indeedrestructuring time represents one of the most difficult scale-up prob-lems Examples of how the problem might be addressed includefreeing up sta by establishing opportunities for students to spendtime pursuing activities such as music art and sports with special-ists or supervised by aides and community volunteers Alternativelyschool might start later or end earlier on a given day As these exam-ples suggest any approach will be controversial but if the problem isnot addressed satisfactorily successful replication of comprehensiveprototypes is unlikely

PHASE II INITIAL IMPLEMENTATION OF APROTOTYPE

Initial implementation involves adapting and phasing-in a programwith well-designed guidance and support If there is anything certainabout e orts to replicate a prototype it is that the process is stress-ful Some of the stress arises from the nature of the program some isinherent in the process of organizational change Coalitions must bedeveloped new working relationships established disruptive rumorsand information overload countered and interpersonal conictsresolved Short-term frustrations must be kept in perspective vis a visthe reform vision To help deal with all this temporary mechanismsare added to the organizational infrastructure They include (a) asite-based steering mechanism to guide and support replication (b) achange agent from the scale-up team working with site stakeholderson a change team to facilitate coalition building problem solvingand conict resolution and (c) mentors and coaches to model andteach elements of the prototype These structures are created tofacilitate replication and some are assimilated into a sitersquos infra-structure at the end of the initial implementation phase to supportinstitutionalization and ongoing evolution

A scale-up team and steering group work at a site with the schoolrsquosleadership specic planning groups and other stakeholders to formu-late phase-in plans steer program development and generallyprovide guidance and support for change Two major facets of thiswork are delineating a sequence for introducing major program ele-ments and outlining strategies to facilitate implementation Particu-lar attention is given to how to start with special emphasis onspecifying structures and resources for guidance and support For

314 L Taylor et al

instance in restructuring to better address barriers to learning therst steps at a school site involve creating processes to map analyzecoordinate and redeploy existing resources Special change mecha-nisms such as an organization facilitator and a resource coordinatingteam are created to guide and support the activity (Adelman 19931996a 1996b Adelman amp Taylor 1997a 1997b 1998)

Throughout this phase formative evaluation procedures are estab-lished to provide feedback for program development As noted abovee ective e orts to lsquolsquoreinventrsquorsquo schools require ensuring that allinvolved have the time to develop and institutionalize a soundprogram and that they are not penalized for unavoidable missteps Asa prototype is phased-in evaluation must not be thought of in termsof accountability Major systemic changes to improve literacy cantake years to develop Outcome e ectiveness is demonstrated afterthe program is in place The purpose of evaluation at this stage is toguide revision and ne-tuning of processes Formative evaluationsgather and analyze information relevant to changes in planning pro-cesses governance structures and policies and resources they alsofocus on implementation strategies and barriers program organiz-ation and staffing and initial outcomes Are teachers mastering thenew curricula for literacy Is there increased student engagement inreading Are parents involved in supporting reading at home Ifthings are not progressing satisfactorily why not Whatrsquos the down-side of the new approach

Well-designed organizational support and guidance is needed toenhance productivity minimize problems and accommodate individ-ual di erences This involves various forms of capacity building andpersonalized day-by-day facilitation Intensive coaching with somefollow-up consultation for instance are key processes so are mentor-ship and technical assistance Continuing education provides a criti-cal vehicle for enhancing productive changes generating renewaland countering burnout As new stakeholders arrive technologicaltools can be particularly useful in helping them catch up All thisactivity not only builds capacity but can foster networking and otherforms of task-related social and personal support as well as providea wide range of enrichment opportunities that enhance morale

If the steps discussed to this point are done well a sound founda-tion for initial implementation should be in place This initial phase-in period can however consume considerable e ort create specialproblems and may yield a temporary drop in some performance indi-cators Good day-by-day facilitation aims at minimizing such negativeimpact by e ectively addressing stakeholder motivation and capabil-ity and overcoming barriers to productive working relationships

Scaling up Reforms 315

Lessons Learned

Failure to take sufficient time to create readiness (Phase 1) can resultin implementing the form rather than the substance of a prototypeFor example we nd that change agents frequently are sent intoschools before essential policy support is enacted and before schoolleaders have assimilated and decided to support reforms Teams areconvened to assist with reforms (plan coordinate develop newapproaches) but the absence of supportive policy means substantivechanges are not accomplished As a result the initial motivation ofmany key team members wanes and other counterproductivedynamics arise All of this seems inevitable when initial implementa-tion proceeds without adequate policy support

Even in situations where sufficient readiness is created difficultiesfrequently arise due to a failure to keep enough stakeholders consis-tently moving in the direction of desired outcomes Comprehensivechange is usually achieved only when fairly high levels of positiveenergy can be mobilized over extended periods of time among a criti-cal mass of stakeholders sustained energy is appropriately directedthe process is supported with ongoing and well-conceived capacitybuilding and individuals are not pushed beyond their capabilitiesAnd because low and negative motivation are related to resistance tochange and poor functioning matching motivation is a rst-orderconsideration That is scale-up e orts must use strategies designed tomobilize and maintain proactive e ort and overcome barriers toworking relationships As in personalizing instruction approx-imating a good motivational t also requires matching capabilitiessuch as starting with fewer elements at sites at which resources arelimited and accounting for variability in stakeholdersrsquo competenceOver and over we nd too little attention is paid to these mattersThe result is failure to create an lsquolsquoenvironmentrsquorsquo that mobilizesdirects and then maintains stakeholder involvement

As with students the problem can be conceived as that of main-taining an appropriate match between the demands of the situationand individual motivation and capabilities In this respect we thinkthe construct of personalization o ers a concept around which toorganize thinking about facilitating change As stressed in the rstarticle in this issue personalization calls for systematically planningand implementing processes focused not only on knowledge andskills but on attitudes In particular it emphasizes the importance ofa primary and constant focus on ensuring positive attitudes Mobi-lization probably is best facilitated when procedures are perceived byindividuals as good ways to reach desired outcomes This requires

316 L Taylor et al

processes that can instigate and enhance an individualrsquos perceptionsof valued opportunities choice and control accomplishment andrelatedness to others Even if a task isnrsquot enjoyable expectation offeeling some sense of satisfaction related to process or outcome canbe a powerful intrinsic factor motivating individual behavior Taskpersistence for example can be facilitated by the expectation thatone will feel competent self-determining or more closely connectedto others From this perspective ensuring that individuals havevalued options a meaningful role in decision making feedback thatemphasizes progress toward desired outcomes and positive workingrelationships are among the most basic facilitation strategies(Adelman amp Taylor 1993b 1994 Deci amp Ryan 1985)

One other initial implementation problem that often arises is diffi-culty in establishing mechanisms to facilitate productive workingrelationships and identify and deal with problems quickly Forexample it is expected that change agents will encounter manyinstances of individual resistance and apathy interpersonal conictsand resentments (including lsquolsquous vs themrsquorsquo dynamics) rumors thatoveremphasize the negative and underestimate the positive and indi-viduals who are frequent faultnders Such problems seriouslyimpede e ective replication The roots of some of these problemsoften are present at a site prior to scale-up change simply o ers anew focus and perhaps magnies troubling matters Other problemsare a direct product of the activities and relationships that the scale-up process engenders Given the inevitability of such problems build-ing and maintaining working relationships need to be among themost basic concerns for those who have responsibility for scale-up Inparticular considerable attention must be paid to enhancing themotivational readiness and capability of those who are to worktogether and ensure there is an appropriate infrastructure to guideand support working relationships Proactively this requires problemprevention mechanisms that help create an atmosphere where defen-siveness is curtailed and positive rapport is engendered The point isto enhance attitudes knowledge and skills that foster interpersonalconnections and a sense of community Reactively the emphasis is onproblem solving resolving conict and providing ongoing support torebuild relationships Policies must encourage problem-solving-oriented critiques safeguards that protect those making changesappreciation for e ort and celebration of progress We nd thateveryone understands such matters but the culture at many schoolsites is more attuned to problem naming and analyzing than to antici-pating preventing and solving problems that arise around workingrelationships

Scaling up Reforms 317

Those responsible for systemic change need to spend as much timeas necessary ensuring that a schoolrsquos infrastructure is ready toprevent and ameliorate problems Special attention must be paid toensuring that problem solving mechanisms and communication pro-cesses are in place and properly sta ed and that stakeholders arewell informed about how to use the procedures Furthermore somestakeholders may have to be encouraged to interact in ways thatconvey genuine empathy warmth and mutual regard and respectwith a view to creating and maintaining a positive working climateand a psychological sense of community

At times we nd it necessary to target a specic problem anddesignated persons In some instances rather simple strategies aree ective For example most motivated individuals can be directlytaught ways to improve understanding and communication and avoidor resolve conicts that interfere with working relationships Inother instances however signicant remedial action is necessary aswhen overcoming barriers to a working relationship involvescountering negative attitudes Helpful in this regard are analysessuch as that by Sue and Zane (1987) that suggest how to demonstratethat something of value can be gained from individuals workingtogether and how to establish each participantrsquos credibility (eg bymaximizing task-focus and positive outcomes)

PHASE III INSTITUTIONALIZING THE PROTOTYPE

Maintaining and enhancing changes can be as difficult as makingthem in the rst place The history of education reform is one offailure to foster promising prototypes in substantive ways and overan extended period of times (Tyack amp Cuban 1995) Institutionalizinga prototype entails ensuring that the organization assumes long-termownership and that there is a blueprint for countering forces that canerode the changes Moreover institutionalization is more than atechnical process It requires assimilation of and ongoing adherenceto the values inherent in the prototypersquos underlying rationale Thefocus of course is not just on maintenance the point is to moveforward by enhancing productive changes and generating a sense ofrenewal as needed Critical in all this are specic plans that guar-antee ongoing and enhanced leadership and delineate ways in whichplanning implementation coordination and continuing educationmechanisms are maintained

Some Major Tasks

Whose responsibility is it to advocate for maintaining and evolving a

318 L Taylor et al

replicated prototype for improving literacy As problems arise whoseresponsibility is it to lead the way in resolving them Leadership isthe key here by both official leaders such as administrators mentorsta union chapter chairs and elected parent representatives aswell as natural leaders such as reading and writing teachers Obvi-ously official and natural leaders are not mutually exclusive groupsAt this phase both types of leadership are essential to ensure a broadenough base for ongoing advocacy problem solving enhancementand renewal Official leaders provide a legitimate power base asvarious interests compete for the organizationrsquos limited resourcesand they play a key role in ensuring that the contributions of naturalleaders are recognized and rewarded

Maintenance and enhancement require that the organizationrsquosgovernance body assumes ownership and program advocacy such astaking over the temporary steering grouprsquos functions addressingongoing policy and long-range planning concerns and maintainingnancial support The foundation for such ownership is laid duringthe readiness phase Each element becomes the organizationrsquos pro-perty as it is established during initial implementation The officialdeed of ownership is transferred as soon as the prototype is in place

Ownership however is no guarantee of institutionalizationVarious forces that can erode reforms are always at work Forinstance teams at a site experience turnover problems with commu-nication and sharing of resources are chronic competing interestsand the attractiveness of moving on to something new pull attentionand resources to other activity To minimize such problems stepsmust be taken to identify and solve them as quickly as is feasibleThis requires someone who has the time energy and expertise tomeet periodically with stakeholders to anticipate and amelioratethreats to a prototypersquos integrity

Over time mechanisms for planning implementation and coordi-nation are maintained by ensuring the activity is an official part ofthe infrastructure has appropriate leadership and is e ectively sup-ported Anyone who has worked on a school-based team to improve aliteracy program knows there must be a critical mass of teammembers so that the work load is manageable and a broad base ofinvolvement is ensured Also essential are adequate resources includ-ing time to learn the role and time to perform the functions reason-ably interesting tasks technical support for problem solvingrecognition and rewards for contributions immediate replacementwhen someone leaves continuing education to enhance team func-tioning and so forth Without serious attention to such matters theteamsrsquo morale and motivation will wane

Scaling up Reforms 319

Lessons Learned

Newly institutionalized approaches are seriously jeopardized in theabsence of dedicated ongoing capacity-building Of particular impor-tance are ways to rapidly and e ectively assimilate new arrivals at aschool (sta students families) This is a major concern at sites withconsiderable turnover or growth At such sites the majority of thoseinitially involved in implementing a new approach may be gonewithin a period of two to three years Whatever the mobility rate it isessential to design and maintain transition programs for newarrivals Initial welcoming and introductory orientations of coursemust be followed-up with ongoing support systems and intensivecapacity building related to understanding and valuing theapproaches the school has adopted We nd that all this is essentialnot only to maintain what has been adopted but also can contributeto establishing schools as caring environments

PHASE IV ONGOING EVOLUTION

Ongoing evolution of organizations and programs is the product ofe orts to account for accomplishments deal with changing times andconditions incorporate new knowledge and create a sense of renewalas the excitement of newness wears o and the demands of changesap energy As suggested already in part vigor and direction can bemaintained through continuing education especially exposure toideas that suggest a range of ways for evolving a program As thefollowing discussion indicates ongoing evolution also is fostered byevaluation designed to document accomplishments and provide feed-back designed to improve quality

Increased concern over accountability has advanced the wayevaluation is conceived (Posavac amp Carey 1989 Rossi amp Freeman1989 Scriven 1993 Sechrest amp Figueredo 1993 Shadish Jr Cook ampLeviton 1991 Stake 1967 1976 Stufflebeam amp Webster 1983 Weiss1995) At the same time social and political forces have literallyshaped the whole enterprise and in the process have narrowed theway professionals clients policymakers underwriters and thegeneral public think about program evaluation A prevailing cry isfor specic evidence of e ectiveness For schools this means imme-diate gains in literacy as measured by achievement tests Althoughunderstandable in light of the unlled promise of so many programsand the insatiable demands on limited public nances such simplisti-cally conceived accountability demands ignore the complexities ofdeveloping and scaling-up major reforms

320 L Taylor et al

Formative and Summative Evaluation

Evaluation of a prototype involves more than determining efficacyfor students Broadly stated it encompasses concerns about how toexpand the focus of evaluative research not only to contribute toimproving practice but also to aid in evolving practice and policy(General Accounting Office 1989 Lyon amp Moats 1997) To facilitateprogram development and organizational change the primary orien-tation for evaluation in the early phases is formative It is especiallyfocused on data gathering and analyses that can help improve pro-cedures Most of what is written about educational and psychosocialintervention however is oriented to summative evaluation and mea-suring outcomes for individuals such as improved reading achieve-ment scores Replicating approaches to improve literacy involve notonly changing individuals but changing organizations and systemsThus both individuals and systems must be evaluated

All this presumes appropriate mechanisms to provide and analyzeessential information To these ends a scale-up sta can help estab-lish an evaluation team and capacity building that prepares a schoolto conduct evaluation that enhances reforms The immediate focus ison successful program replication ultimately of course the empha-sis must be on student outcomes

Pursuing Results

Due to the increased interest in accountability many complex aimsare broken down into specic objectives Indeed short-range objec-tives stated in measurable terms generally assume a central role inplanning However short-range objectives are not ends in them-selves They are a small part of a particular goal and aim and some-times are prerequisites for moving on to a goal It is essential not tolose sight of the fact that many specic objectives are relativelysmall unrepresentative and often unimportant segments of the mostvalued aims that society has for its citizens and citizens have forthemselves

The problem is well exemplied by the narrow focus found inreviews analyses and reanalyses of data on early education (eg seeAlbee amp Gullotta 1997 Bond amp Compas 1989 Dryfoos 1990 Durlak1995 Elias 1997 Mitchell Seligson amp Marx 1989 Schorr 1988Slavin Karweit amp Madden 1989 Weissberg Gullotta HamptomRyan amp Adams 1997) As such work demonstrates overemphasis onevaluating the efficacy of underdeveloped prototypes draws resourcesaway from formative evaluation

Scaling up Reforms 321

With specic respect to scale-up the rst accomplishment is thereplication itself Have all facets been implemented How completelyhas each been implemented At how many locations The next set ofresults are any indications of progress for students such as improve-ments in attitudes toward school health attendance behavior andacademic achievement A nal set of evaluation concerns is thedegree to which student outcomes approximate societal standards

Lessons Learned

The process of evaluating results is costly in terms of nancial invest-ment the negative psychological impact on those evaluated and theways it can inappropriately reshape new approaches Cost-e ectiveoutcomes cannot be achieved in the absence of e ective prototypedevelopment and research Premature e orts to carry out com-prehensive summative evaluations clearly are not cost-e ective Anyreading and writing program will show poor results if it is evaluatedbefore teachers have mastered its application None of this of courseis an argument against evaluating results Rather it is meant tounderscore concerns and encourage greater attention to addressingthem

Once a prototype is established care must be taken to avoiddeveloping outcome evaluation as an adversarial process Due to thepolitical realities related to accountability one of the most per-plexing facets to negotiate is the time frame for summative evalu-ation The more complex the prototype the longer it takes and thecostlier it is to implement and evaluate Schools usually want quickprocesses and results and of course rarely can a ord costly innova-tions or lengthy di usion activity Compromises are inevitable butmust be arrived at with great care so as not to undermine the sub-stance of proposed changes

The psychology of evaluation suggests that an overemphasis onaccountability tends to produce negative reactions One possible wayto counter this may be to conceive evaluation as a way for everystakeholder to self-evaluate as a basis for quality improvement and asa way of getting credit for all that is accomplished Unfortunately asaccountability pressures increase we nd that replication of proto-types are guided more by what can be measured than by long-rangeaims That is demands for immediate accountability reshape prac-tices so that the emphasis shifts to immediate and readily measuredobjectives and away from fundamental purposes Over time this inap-propriately leads to radical revision of the underlying rationale for aprototype

322 L Taylor et al

CONCLUDING COMMENTS

Those who set out to change schools and schooling are confrontedwith two enormous tasks The rst is to develop prototypes Thesecond involves large-scale replication One without the other isinsufficient Yet considerably more attention is paid to developingand validating prototypes than to delineating and testing scale-upprocesses Clearly it is time to correct this deciency The ideas pre-sented in this article are meant to stimulate work on the problem andthereby to advance the cause of educational reform

Finally in fairness to those who labor for educational reform weall must remember that the quality of schooling family life and com-munity functioning spirals up or down as a function of the quality ofthe ongoing transactions among each Thus scale-up e orts relatedto educational reform must take place within the context of a poli-tical agenda that addresses ways to strengthen the family and com-munity infrastructure through strategies that enhance economicopportunity adult literacy and so forth What we need are policiesthat develop demonstrate and scale-up comprehensive multifacetedintegrated approaches that can e ectively address barriers to devel-opment learning and teaching

REFERENCES

Adelman H S (1993) School-linked mental health interventions Toward mechanismsfor service coordination and integration Journal of Community Psychology 21 309ndash319

Adelman H S (1996a) Restructuring education support services and integrating com-munity resources Beyond the full service school model School Psychology Review25 431ndash445

Adelman H S (1996b) Restructuring support services Toward a comprehensiveapproach Kent OH American School Health Association

Adelman H S amp Taylor L (1993a) Early Assistance for Students and FamiliesProgram Guidebook Los Angeles Early Assistance for Students and FamiliesProject Copies available from the School Mental Health Project Dept of Psychol-ogy UCLA

Adelman H S amp Taylor L (1993b) Learning problems and learning disabilities Moving forward Pacic Grove CA BrooksCole

Adelman H S amp Taylor L (1994) On understanding intervention in psychology andeducation Westport CT Praeger

Adelman H S amp Taylor L (1997a) Addressing barriers to learning Beyond school-linked services and full service schools American Journal of Orthopsychiatry 67408ndash421

Adelman H S amp Taylor L (1997b) Toward a scale-up model for replicating newapproaches to schooling Journal of Educational and Psychological Consultation 8197ndash230

314 L Taylor et al

instance in restructuring to better address barriers to learning therst steps at a school site involve creating processes to map analyzecoordinate and redeploy existing resources Special change mecha-nisms such as an organization facilitator and a resource coordinatingteam are created to guide and support the activity (Adelman 19931996a 1996b Adelman amp Taylor 1997a 1997b 1998)

Throughout this phase formative evaluation procedures are estab-lished to provide feedback for program development As noted abovee ective e orts to lsquolsquoreinventrsquorsquo schools require ensuring that allinvolved have the time to develop and institutionalize a soundprogram and that they are not penalized for unavoidable missteps Asa prototype is phased-in evaluation must not be thought of in termsof accountability Major systemic changes to improve literacy cantake years to develop Outcome e ectiveness is demonstrated afterthe program is in place The purpose of evaluation at this stage is toguide revision and ne-tuning of processes Formative evaluationsgather and analyze information relevant to changes in planning pro-cesses governance structures and policies and resources they alsofocus on implementation strategies and barriers program organiz-ation and staffing and initial outcomes Are teachers mastering thenew curricula for literacy Is there increased student engagement inreading Are parents involved in supporting reading at home Ifthings are not progressing satisfactorily why not Whatrsquos the down-side of the new approach

Well-designed organizational support and guidance is needed toenhance productivity minimize problems and accommodate individ-ual di erences This involves various forms of capacity building andpersonalized day-by-day facilitation Intensive coaching with somefollow-up consultation for instance are key processes so are mentor-ship and technical assistance Continuing education provides a criti-cal vehicle for enhancing productive changes generating renewaland countering burnout As new stakeholders arrive technologicaltools can be particularly useful in helping them catch up All thisactivity not only builds capacity but can foster networking and otherforms of task-related social and personal support as well as providea wide range of enrichment opportunities that enhance morale

If the steps discussed to this point are done well a sound founda-tion for initial implementation should be in place This initial phase-in period can however consume considerable e ort create specialproblems and may yield a temporary drop in some performance indi-cators Good day-by-day facilitation aims at minimizing such negativeimpact by e ectively addressing stakeholder motivation and capabil-ity and overcoming barriers to productive working relationships

Scaling up Reforms 315

Lessons Learned

Failure to take sufficient time to create readiness (Phase 1) can resultin implementing the form rather than the substance of a prototypeFor example we nd that change agents frequently are sent intoschools before essential policy support is enacted and before schoolleaders have assimilated and decided to support reforms Teams areconvened to assist with reforms (plan coordinate develop newapproaches) but the absence of supportive policy means substantivechanges are not accomplished As a result the initial motivation ofmany key team members wanes and other counterproductivedynamics arise All of this seems inevitable when initial implementa-tion proceeds without adequate policy support

Even in situations where sufficient readiness is created difficultiesfrequently arise due to a failure to keep enough stakeholders consis-tently moving in the direction of desired outcomes Comprehensivechange is usually achieved only when fairly high levels of positiveenergy can be mobilized over extended periods of time among a criti-cal mass of stakeholders sustained energy is appropriately directedthe process is supported with ongoing and well-conceived capacitybuilding and individuals are not pushed beyond their capabilitiesAnd because low and negative motivation are related to resistance tochange and poor functioning matching motivation is a rst-orderconsideration That is scale-up e orts must use strategies designed tomobilize and maintain proactive e ort and overcome barriers toworking relationships As in personalizing instruction approx-imating a good motivational t also requires matching capabilitiessuch as starting with fewer elements at sites at which resources arelimited and accounting for variability in stakeholdersrsquo competenceOver and over we nd too little attention is paid to these mattersThe result is failure to create an lsquolsquoenvironmentrsquorsquo that mobilizesdirects and then maintains stakeholder involvement

As with students the problem can be conceived as that of main-taining an appropriate match between the demands of the situationand individual motivation and capabilities In this respect we thinkthe construct of personalization o ers a concept around which toorganize thinking about facilitating change As stressed in the rstarticle in this issue personalization calls for systematically planningand implementing processes focused not only on knowledge andskills but on attitudes In particular it emphasizes the importance ofa primary and constant focus on ensuring positive attitudes Mobi-lization probably is best facilitated when procedures are perceived byindividuals as good ways to reach desired outcomes This requires

316 L Taylor et al

processes that can instigate and enhance an individualrsquos perceptionsof valued opportunities choice and control accomplishment andrelatedness to others Even if a task isnrsquot enjoyable expectation offeeling some sense of satisfaction related to process or outcome canbe a powerful intrinsic factor motivating individual behavior Taskpersistence for example can be facilitated by the expectation thatone will feel competent self-determining or more closely connectedto others From this perspective ensuring that individuals havevalued options a meaningful role in decision making feedback thatemphasizes progress toward desired outcomes and positive workingrelationships are among the most basic facilitation strategies(Adelman amp Taylor 1993b 1994 Deci amp Ryan 1985)

One other initial implementation problem that often arises is diffi-culty in establishing mechanisms to facilitate productive workingrelationships and identify and deal with problems quickly Forexample it is expected that change agents will encounter manyinstances of individual resistance and apathy interpersonal conictsand resentments (including lsquolsquous vs themrsquorsquo dynamics) rumors thatoveremphasize the negative and underestimate the positive and indi-viduals who are frequent faultnders Such problems seriouslyimpede e ective replication The roots of some of these problemsoften are present at a site prior to scale-up change simply o ers anew focus and perhaps magnies troubling matters Other problemsare a direct product of the activities and relationships that the scale-up process engenders Given the inevitability of such problems build-ing and maintaining working relationships need to be among themost basic concerns for those who have responsibility for scale-up Inparticular considerable attention must be paid to enhancing themotivational readiness and capability of those who are to worktogether and ensure there is an appropriate infrastructure to guideand support working relationships Proactively this requires problemprevention mechanisms that help create an atmosphere where defen-siveness is curtailed and positive rapport is engendered The point isto enhance attitudes knowledge and skills that foster interpersonalconnections and a sense of community Reactively the emphasis is onproblem solving resolving conict and providing ongoing support torebuild relationships Policies must encourage problem-solving-oriented critiques safeguards that protect those making changesappreciation for e ort and celebration of progress We nd thateveryone understands such matters but the culture at many schoolsites is more attuned to problem naming and analyzing than to antici-pating preventing and solving problems that arise around workingrelationships

Scaling up Reforms 317

Those responsible for systemic change need to spend as much timeas necessary ensuring that a schoolrsquos infrastructure is ready toprevent and ameliorate problems Special attention must be paid toensuring that problem solving mechanisms and communication pro-cesses are in place and properly sta ed and that stakeholders arewell informed about how to use the procedures Furthermore somestakeholders may have to be encouraged to interact in ways thatconvey genuine empathy warmth and mutual regard and respectwith a view to creating and maintaining a positive working climateand a psychological sense of community

At times we nd it necessary to target a specic problem anddesignated persons In some instances rather simple strategies aree ective For example most motivated individuals can be directlytaught ways to improve understanding and communication and avoidor resolve conicts that interfere with working relationships Inother instances however signicant remedial action is necessary aswhen overcoming barriers to a working relationship involvescountering negative attitudes Helpful in this regard are analysessuch as that by Sue and Zane (1987) that suggest how to demonstratethat something of value can be gained from individuals workingtogether and how to establish each participantrsquos credibility (eg bymaximizing task-focus and positive outcomes)

PHASE III INSTITUTIONALIZING THE PROTOTYPE

Maintaining and enhancing changes can be as difficult as makingthem in the rst place The history of education reform is one offailure to foster promising prototypes in substantive ways and overan extended period of times (Tyack amp Cuban 1995) Institutionalizinga prototype entails ensuring that the organization assumes long-termownership and that there is a blueprint for countering forces that canerode the changes Moreover institutionalization is more than atechnical process It requires assimilation of and ongoing adherenceto the values inherent in the prototypersquos underlying rationale Thefocus of course is not just on maintenance the point is to moveforward by enhancing productive changes and generating a sense ofrenewal as needed Critical in all this are specic plans that guar-antee ongoing and enhanced leadership and delineate ways in whichplanning implementation coordination and continuing educationmechanisms are maintained

Some Major Tasks

Whose responsibility is it to advocate for maintaining and evolving a

318 L Taylor et al

replicated prototype for improving literacy As problems arise whoseresponsibility is it to lead the way in resolving them Leadership isthe key here by both official leaders such as administrators mentorsta union chapter chairs and elected parent representatives aswell as natural leaders such as reading and writing teachers Obvi-ously official and natural leaders are not mutually exclusive groupsAt this phase both types of leadership are essential to ensure a broadenough base for ongoing advocacy problem solving enhancementand renewal Official leaders provide a legitimate power base asvarious interests compete for the organizationrsquos limited resourcesand they play a key role in ensuring that the contributions of naturalleaders are recognized and rewarded

Maintenance and enhancement require that the organizationrsquosgovernance body assumes ownership and program advocacy such astaking over the temporary steering grouprsquos functions addressingongoing policy and long-range planning concerns and maintainingnancial support The foundation for such ownership is laid duringthe readiness phase Each element becomes the organizationrsquos pro-perty as it is established during initial implementation The officialdeed of ownership is transferred as soon as the prototype is in place

Ownership however is no guarantee of institutionalizationVarious forces that can erode reforms are always at work Forinstance teams at a site experience turnover problems with commu-nication and sharing of resources are chronic competing interestsand the attractiveness of moving on to something new pull attentionand resources to other activity To minimize such problems stepsmust be taken to identify and solve them as quickly as is feasibleThis requires someone who has the time energy and expertise tomeet periodically with stakeholders to anticipate and amelioratethreats to a prototypersquos integrity

Over time mechanisms for planning implementation and coordi-nation are maintained by ensuring the activity is an official part ofthe infrastructure has appropriate leadership and is e ectively sup-ported Anyone who has worked on a school-based team to improve aliteracy program knows there must be a critical mass of teammembers so that the work load is manageable and a broad base ofinvolvement is ensured Also essential are adequate resources includ-ing time to learn the role and time to perform the functions reason-ably interesting tasks technical support for problem solvingrecognition and rewards for contributions immediate replacementwhen someone leaves continuing education to enhance team func-tioning and so forth Without serious attention to such matters theteamsrsquo morale and motivation will wane

Scaling up Reforms 319

Lessons Learned

Newly institutionalized approaches are seriously jeopardized in theabsence of dedicated ongoing capacity-building Of particular impor-tance are ways to rapidly and e ectively assimilate new arrivals at aschool (sta students families) This is a major concern at sites withconsiderable turnover or growth At such sites the majority of thoseinitially involved in implementing a new approach may be gonewithin a period of two to three years Whatever the mobility rate it isessential to design and maintain transition programs for newarrivals Initial welcoming and introductory orientations of coursemust be followed-up with ongoing support systems and intensivecapacity building related to understanding and valuing theapproaches the school has adopted We nd that all this is essentialnot only to maintain what has been adopted but also can contributeto establishing schools as caring environments

PHASE IV ONGOING EVOLUTION

Ongoing evolution of organizations and programs is the product ofe orts to account for accomplishments deal with changing times andconditions incorporate new knowledge and create a sense of renewalas the excitement of newness wears o and the demands of changesap energy As suggested already in part vigor and direction can bemaintained through continuing education especially exposure toideas that suggest a range of ways for evolving a program As thefollowing discussion indicates ongoing evolution also is fostered byevaluation designed to document accomplishments and provide feed-back designed to improve quality

Increased concern over accountability has advanced the wayevaluation is conceived (Posavac amp Carey 1989 Rossi amp Freeman1989 Scriven 1993 Sechrest amp Figueredo 1993 Shadish Jr Cook ampLeviton 1991 Stake 1967 1976 Stufflebeam amp Webster 1983 Weiss1995) At the same time social and political forces have literallyshaped the whole enterprise and in the process have narrowed theway professionals clients policymakers underwriters and thegeneral public think about program evaluation A prevailing cry isfor specic evidence of e ectiveness For schools this means imme-diate gains in literacy as measured by achievement tests Althoughunderstandable in light of the unlled promise of so many programsand the insatiable demands on limited public nances such simplisti-cally conceived accountability demands ignore the complexities ofdeveloping and scaling-up major reforms

320 L Taylor et al

Formative and Summative Evaluation

Evaluation of a prototype involves more than determining efficacyfor students Broadly stated it encompasses concerns about how toexpand the focus of evaluative research not only to contribute toimproving practice but also to aid in evolving practice and policy(General Accounting Office 1989 Lyon amp Moats 1997) To facilitateprogram development and organizational change the primary orien-tation for evaluation in the early phases is formative It is especiallyfocused on data gathering and analyses that can help improve pro-cedures Most of what is written about educational and psychosocialintervention however is oriented to summative evaluation and mea-suring outcomes for individuals such as improved reading achieve-ment scores Replicating approaches to improve literacy involve notonly changing individuals but changing organizations and systemsThus both individuals and systems must be evaluated

All this presumes appropriate mechanisms to provide and analyzeessential information To these ends a scale-up sta can help estab-lish an evaluation team and capacity building that prepares a schoolto conduct evaluation that enhances reforms The immediate focus ison successful program replication ultimately of course the empha-sis must be on student outcomes

Pursuing Results

Due to the increased interest in accountability many complex aimsare broken down into specic objectives Indeed short-range objec-tives stated in measurable terms generally assume a central role inplanning However short-range objectives are not ends in them-selves They are a small part of a particular goal and aim and some-times are prerequisites for moving on to a goal It is essential not tolose sight of the fact that many specic objectives are relativelysmall unrepresentative and often unimportant segments of the mostvalued aims that society has for its citizens and citizens have forthemselves

The problem is well exemplied by the narrow focus found inreviews analyses and reanalyses of data on early education (eg seeAlbee amp Gullotta 1997 Bond amp Compas 1989 Dryfoos 1990 Durlak1995 Elias 1997 Mitchell Seligson amp Marx 1989 Schorr 1988Slavin Karweit amp Madden 1989 Weissberg Gullotta HamptomRyan amp Adams 1997) As such work demonstrates overemphasis onevaluating the efficacy of underdeveloped prototypes draws resourcesaway from formative evaluation

Scaling up Reforms 321

With specic respect to scale-up the rst accomplishment is thereplication itself Have all facets been implemented How completelyhas each been implemented At how many locations The next set ofresults are any indications of progress for students such as improve-ments in attitudes toward school health attendance behavior andacademic achievement A nal set of evaluation concerns is thedegree to which student outcomes approximate societal standards

Lessons Learned

The process of evaluating results is costly in terms of nancial invest-ment the negative psychological impact on those evaluated and theways it can inappropriately reshape new approaches Cost-e ectiveoutcomes cannot be achieved in the absence of e ective prototypedevelopment and research Premature e orts to carry out com-prehensive summative evaluations clearly are not cost-e ective Anyreading and writing program will show poor results if it is evaluatedbefore teachers have mastered its application None of this of courseis an argument against evaluating results Rather it is meant tounderscore concerns and encourage greater attention to addressingthem

Once a prototype is established care must be taken to avoiddeveloping outcome evaluation as an adversarial process Due to thepolitical realities related to accountability one of the most per-plexing facets to negotiate is the time frame for summative evalu-ation The more complex the prototype the longer it takes and thecostlier it is to implement and evaluate Schools usually want quickprocesses and results and of course rarely can a ord costly innova-tions or lengthy di usion activity Compromises are inevitable butmust be arrived at with great care so as not to undermine the sub-stance of proposed changes

The psychology of evaluation suggests that an overemphasis onaccountability tends to produce negative reactions One possible wayto counter this may be to conceive evaluation as a way for everystakeholder to self-evaluate as a basis for quality improvement and asa way of getting credit for all that is accomplished Unfortunately asaccountability pressures increase we nd that replication of proto-types are guided more by what can be measured than by long-rangeaims That is demands for immediate accountability reshape prac-tices so that the emphasis shifts to immediate and readily measuredobjectives and away from fundamental purposes Over time this inap-propriately leads to radical revision of the underlying rationale for aprototype

322 L Taylor et al

CONCLUDING COMMENTS

Those who set out to change schools and schooling are confrontedwith two enormous tasks The rst is to develop prototypes Thesecond involves large-scale replication One without the other isinsufficient Yet considerably more attention is paid to developingand validating prototypes than to delineating and testing scale-upprocesses Clearly it is time to correct this deciency The ideas pre-sented in this article are meant to stimulate work on the problem andthereby to advance the cause of educational reform

Finally in fairness to those who labor for educational reform weall must remember that the quality of schooling family life and com-munity functioning spirals up or down as a function of the quality ofthe ongoing transactions among each Thus scale-up e orts relatedto educational reform must take place within the context of a poli-tical agenda that addresses ways to strengthen the family and com-munity infrastructure through strategies that enhance economicopportunity adult literacy and so forth What we need are policiesthat develop demonstrate and scale-up comprehensive multifacetedintegrated approaches that can e ectively address barriers to devel-opment learning and teaching

REFERENCES

Adelman H S (1993) School-linked mental health interventions Toward mechanismsfor service coordination and integration Journal of Community Psychology 21 309ndash319

Adelman H S (1996a) Restructuring education support services and integrating com-munity resources Beyond the full service school model School Psychology Review25 431ndash445

Adelman H S (1996b) Restructuring support services Toward a comprehensiveapproach Kent OH American School Health Association

Adelman H S amp Taylor L (1993a) Early Assistance for Students and FamiliesProgram Guidebook Los Angeles Early Assistance for Students and FamiliesProject Copies available from the School Mental Health Project Dept of Psychol-ogy UCLA

Adelman H S amp Taylor L (1993b) Learning problems and learning disabilities Moving forward Pacic Grove CA BrooksCole

Adelman H S amp Taylor L (1994) On understanding intervention in psychology andeducation Westport CT Praeger

Adelman H S amp Taylor L (1997a) Addressing barriers to learning Beyond school-linked services and full service schools American Journal of Orthopsychiatry 67408ndash421

Adelman H S amp Taylor L (1997b) Toward a scale-up model for replicating newapproaches to schooling Journal of Educational and Psychological Consultation 8197ndash230

Scaling up Reforms 315

Lessons Learned

Failure to take sufficient time to create readiness (Phase 1) can resultin implementing the form rather than the substance of a prototypeFor example we nd that change agents frequently are sent intoschools before essential policy support is enacted and before schoolleaders have assimilated and decided to support reforms Teams areconvened to assist with reforms (plan coordinate develop newapproaches) but the absence of supportive policy means substantivechanges are not accomplished As a result the initial motivation ofmany key team members wanes and other counterproductivedynamics arise All of this seems inevitable when initial implementa-tion proceeds without adequate policy support

Even in situations where sufficient readiness is created difficultiesfrequently arise due to a failure to keep enough stakeholders consis-tently moving in the direction of desired outcomes Comprehensivechange is usually achieved only when fairly high levels of positiveenergy can be mobilized over extended periods of time among a criti-cal mass of stakeholders sustained energy is appropriately directedthe process is supported with ongoing and well-conceived capacitybuilding and individuals are not pushed beyond their capabilitiesAnd because low and negative motivation are related to resistance tochange and poor functioning matching motivation is a rst-orderconsideration That is scale-up e orts must use strategies designed tomobilize and maintain proactive e ort and overcome barriers toworking relationships As in personalizing instruction approx-imating a good motivational t also requires matching capabilitiessuch as starting with fewer elements at sites at which resources arelimited and accounting for variability in stakeholdersrsquo competenceOver and over we nd too little attention is paid to these mattersThe result is failure to create an lsquolsquoenvironmentrsquorsquo that mobilizesdirects and then maintains stakeholder involvement

As with students the problem can be conceived as that of main-taining an appropriate match between the demands of the situationand individual motivation and capabilities In this respect we thinkthe construct of personalization o ers a concept around which toorganize thinking about facilitating change As stressed in the rstarticle in this issue personalization calls for systematically planningand implementing processes focused not only on knowledge andskills but on attitudes In particular it emphasizes the importance ofa primary and constant focus on ensuring positive attitudes Mobi-lization probably is best facilitated when procedures are perceived byindividuals as good ways to reach desired outcomes This requires

316 L Taylor et al

processes that can instigate and enhance an individualrsquos perceptionsof valued opportunities choice and control accomplishment andrelatedness to others Even if a task isnrsquot enjoyable expectation offeeling some sense of satisfaction related to process or outcome canbe a powerful intrinsic factor motivating individual behavior Taskpersistence for example can be facilitated by the expectation thatone will feel competent self-determining or more closely connectedto others From this perspective ensuring that individuals havevalued options a meaningful role in decision making feedback thatemphasizes progress toward desired outcomes and positive workingrelationships are among the most basic facilitation strategies(Adelman amp Taylor 1993b 1994 Deci amp Ryan 1985)

One other initial implementation problem that often arises is diffi-culty in establishing mechanisms to facilitate productive workingrelationships and identify and deal with problems quickly Forexample it is expected that change agents will encounter manyinstances of individual resistance and apathy interpersonal conictsand resentments (including lsquolsquous vs themrsquorsquo dynamics) rumors thatoveremphasize the negative and underestimate the positive and indi-viduals who are frequent faultnders Such problems seriouslyimpede e ective replication The roots of some of these problemsoften are present at a site prior to scale-up change simply o ers anew focus and perhaps magnies troubling matters Other problemsare a direct product of the activities and relationships that the scale-up process engenders Given the inevitability of such problems build-ing and maintaining working relationships need to be among themost basic concerns for those who have responsibility for scale-up Inparticular considerable attention must be paid to enhancing themotivational readiness and capability of those who are to worktogether and ensure there is an appropriate infrastructure to guideand support working relationships Proactively this requires problemprevention mechanisms that help create an atmosphere where defen-siveness is curtailed and positive rapport is engendered The point isto enhance attitudes knowledge and skills that foster interpersonalconnections and a sense of community Reactively the emphasis is onproblem solving resolving conict and providing ongoing support torebuild relationships Policies must encourage problem-solving-oriented critiques safeguards that protect those making changesappreciation for e ort and celebration of progress We nd thateveryone understands such matters but the culture at many schoolsites is more attuned to problem naming and analyzing than to antici-pating preventing and solving problems that arise around workingrelationships

Scaling up Reforms 317

Those responsible for systemic change need to spend as much timeas necessary ensuring that a schoolrsquos infrastructure is ready toprevent and ameliorate problems Special attention must be paid toensuring that problem solving mechanisms and communication pro-cesses are in place and properly sta ed and that stakeholders arewell informed about how to use the procedures Furthermore somestakeholders may have to be encouraged to interact in ways thatconvey genuine empathy warmth and mutual regard and respectwith a view to creating and maintaining a positive working climateand a psychological sense of community

At times we nd it necessary to target a specic problem anddesignated persons In some instances rather simple strategies aree ective For example most motivated individuals can be directlytaught ways to improve understanding and communication and avoidor resolve conicts that interfere with working relationships Inother instances however signicant remedial action is necessary aswhen overcoming barriers to a working relationship involvescountering negative attitudes Helpful in this regard are analysessuch as that by Sue and Zane (1987) that suggest how to demonstratethat something of value can be gained from individuals workingtogether and how to establish each participantrsquos credibility (eg bymaximizing task-focus and positive outcomes)

PHASE III INSTITUTIONALIZING THE PROTOTYPE

Maintaining and enhancing changes can be as difficult as makingthem in the rst place The history of education reform is one offailure to foster promising prototypes in substantive ways and overan extended period of times (Tyack amp Cuban 1995) Institutionalizinga prototype entails ensuring that the organization assumes long-termownership and that there is a blueprint for countering forces that canerode the changes Moreover institutionalization is more than atechnical process It requires assimilation of and ongoing adherenceto the values inherent in the prototypersquos underlying rationale Thefocus of course is not just on maintenance the point is to moveforward by enhancing productive changes and generating a sense ofrenewal as needed Critical in all this are specic plans that guar-antee ongoing and enhanced leadership and delineate ways in whichplanning implementation coordination and continuing educationmechanisms are maintained

Some Major Tasks

Whose responsibility is it to advocate for maintaining and evolving a

318 L Taylor et al

replicated prototype for improving literacy As problems arise whoseresponsibility is it to lead the way in resolving them Leadership isthe key here by both official leaders such as administrators mentorsta union chapter chairs and elected parent representatives aswell as natural leaders such as reading and writing teachers Obvi-ously official and natural leaders are not mutually exclusive groupsAt this phase both types of leadership are essential to ensure a broadenough base for ongoing advocacy problem solving enhancementand renewal Official leaders provide a legitimate power base asvarious interests compete for the organizationrsquos limited resourcesand they play a key role in ensuring that the contributions of naturalleaders are recognized and rewarded

Maintenance and enhancement require that the organizationrsquosgovernance body assumes ownership and program advocacy such astaking over the temporary steering grouprsquos functions addressingongoing policy and long-range planning concerns and maintainingnancial support The foundation for such ownership is laid duringthe readiness phase Each element becomes the organizationrsquos pro-perty as it is established during initial implementation The officialdeed of ownership is transferred as soon as the prototype is in place

Ownership however is no guarantee of institutionalizationVarious forces that can erode reforms are always at work Forinstance teams at a site experience turnover problems with commu-nication and sharing of resources are chronic competing interestsand the attractiveness of moving on to something new pull attentionand resources to other activity To minimize such problems stepsmust be taken to identify and solve them as quickly as is feasibleThis requires someone who has the time energy and expertise tomeet periodically with stakeholders to anticipate and amelioratethreats to a prototypersquos integrity

Over time mechanisms for planning implementation and coordi-nation are maintained by ensuring the activity is an official part ofthe infrastructure has appropriate leadership and is e ectively sup-ported Anyone who has worked on a school-based team to improve aliteracy program knows there must be a critical mass of teammembers so that the work load is manageable and a broad base ofinvolvement is ensured Also essential are adequate resources includ-ing time to learn the role and time to perform the functions reason-ably interesting tasks technical support for problem solvingrecognition and rewards for contributions immediate replacementwhen someone leaves continuing education to enhance team func-tioning and so forth Without serious attention to such matters theteamsrsquo morale and motivation will wane

Scaling up Reforms 319

Lessons Learned

Newly institutionalized approaches are seriously jeopardized in theabsence of dedicated ongoing capacity-building Of particular impor-tance are ways to rapidly and e ectively assimilate new arrivals at aschool (sta students families) This is a major concern at sites withconsiderable turnover or growth At such sites the majority of thoseinitially involved in implementing a new approach may be gonewithin a period of two to three years Whatever the mobility rate it isessential to design and maintain transition programs for newarrivals Initial welcoming and introductory orientations of coursemust be followed-up with ongoing support systems and intensivecapacity building related to understanding and valuing theapproaches the school has adopted We nd that all this is essentialnot only to maintain what has been adopted but also can contributeto establishing schools as caring environments

PHASE IV ONGOING EVOLUTION

Ongoing evolution of organizations and programs is the product ofe orts to account for accomplishments deal with changing times andconditions incorporate new knowledge and create a sense of renewalas the excitement of newness wears o and the demands of changesap energy As suggested already in part vigor and direction can bemaintained through continuing education especially exposure toideas that suggest a range of ways for evolving a program As thefollowing discussion indicates ongoing evolution also is fostered byevaluation designed to document accomplishments and provide feed-back designed to improve quality

Increased concern over accountability has advanced the wayevaluation is conceived (Posavac amp Carey 1989 Rossi amp Freeman1989 Scriven 1993 Sechrest amp Figueredo 1993 Shadish Jr Cook ampLeviton 1991 Stake 1967 1976 Stufflebeam amp Webster 1983 Weiss1995) At the same time social and political forces have literallyshaped the whole enterprise and in the process have narrowed theway professionals clients policymakers underwriters and thegeneral public think about program evaluation A prevailing cry isfor specic evidence of e ectiveness For schools this means imme-diate gains in literacy as measured by achievement tests Althoughunderstandable in light of the unlled promise of so many programsand the insatiable demands on limited public nances such simplisti-cally conceived accountability demands ignore the complexities ofdeveloping and scaling-up major reforms

320 L Taylor et al

Formative and Summative Evaluation

Evaluation of a prototype involves more than determining efficacyfor students Broadly stated it encompasses concerns about how toexpand the focus of evaluative research not only to contribute toimproving practice but also to aid in evolving practice and policy(General Accounting Office 1989 Lyon amp Moats 1997) To facilitateprogram development and organizational change the primary orien-tation for evaluation in the early phases is formative It is especiallyfocused on data gathering and analyses that can help improve pro-cedures Most of what is written about educational and psychosocialintervention however is oriented to summative evaluation and mea-suring outcomes for individuals such as improved reading achieve-ment scores Replicating approaches to improve literacy involve notonly changing individuals but changing organizations and systemsThus both individuals and systems must be evaluated

All this presumes appropriate mechanisms to provide and analyzeessential information To these ends a scale-up sta can help estab-lish an evaluation team and capacity building that prepares a schoolto conduct evaluation that enhances reforms The immediate focus ison successful program replication ultimately of course the empha-sis must be on student outcomes

Pursuing Results

Due to the increased interest in accountability many complex aimsare broken down into specic objectives Indeed short-range objec-tives stated in measurable terms generally assume a central role inplanning However short-range objectives are not ends in them-selves They are a small part of a particular goal and aim and some-times are prerequisites for moving on to a goal It is essential not tolose sight of the fact that many specic objectives are relativelysmall unrepresentative and often unimportant segments of the mostvalued aims that society has for its citizens and citizens have forthemselves

The problem is well exemplied by the narrow focus found inreviews analyses and reanalyses of data on early education (eg seeAlbee amp Gullotta 1997 Bond amp Compas 1989 Dryfoos 1990 Durlak1995 Elias 1997 Mitchell Seligson amp Marx 1989 Schorr 1988Slavin Karweit amp Madden 1989 Weissberg Gullotta HamptomRyan amp Adams 1997) As such work demonstrates overemphasis onevaluating the efficacy of underdeveloped prototypes draws resourcesaway from formative evaluation

Scaling up Reforms 321

With specic respect to scale-up the rst accomplishment is thereplication itself Have all facets been implemented How completelyhas each been implemented At how many locations The next set ofresults are any indications of progress for students such as improve-ments in attitudes toward school health attendance behavior andacademic achievement A nal set of evaluation concerns is thedegree to which student outcomes approximate societal standards

Lessons Learned

The process of evaluating results is costly in terms of nancial invest-ment the negative psychological impact on those evaluated and theways it can inappropriately reshape new approaches Cost-e ectiveoutcomes cannot be achieved in the absence of e ective prototypedevelopment and research Premature e orts to carry out com-prehensive summative evaluations clearly are not cost-e ective Anyreading and writing program will show poor results if it is evaluatedbefore teachers have mastered its application None of this of courseis an argument against evaluating results Rather it is meant tounderscore concerns and encourage greater attention to addressingthem

Once a prototype is established care must be taken to avoiddeveloping outcome evaluation as an adversarial process Due to thepolitical realities related to accountability one of the most per-plexing facets to negotiate is the time frame for summative evalu-ation The more complex the prototype the longer it takes and thecostlier it is to implement and evaluate Schools usually want quickprocesses and results and of course rarely can a ord costly innova-tions or lengthy di usion activity Compromises are inevitable butmust be arrived at with great care so as not to undermine the sub-stance of proposed changes

The psychology of evaluation suggests that an overemphasis onaccountability tends to produce negative reactions One possible wayto counter this may be to conceive evaluation as a way for everystakeholder to self-evaluate as a basis for quality improvement and asa way of getting credit for all that is accomplished Unfortunately asaccountability pressures increase we nd that replication of proto-types are guided more by what can be measured than by long-rangeaims That is demands for immediate accountability reshape prac-tices so that the emphasis shifts to immediate and readily measuredobjectives and away from fundamental purposes Over time this inap-propriately leads to radical revision of the underlying rationale for aprototype

322 L Taylor et al

CONCLUDING COMMENTS

Those who set out to change schools and schooling are confrontedwith two enormous tasks The rst is to develop prototypes Thesecond involves large-scale replication One without the other isinsufficient Yet considerably more attention is paid to developingand validating prototypes than to delineating and testing scale-upprocesses Clearly it is time to correct this deciency The ideas pre-sented in this article are meant to stimulate work on the problem andthereby to advance the cause of educational reform

Finally in fairness to those who labor for educational reform weall must remember that the quality of schooling family life and com-munity functioning spirals up or down as a function of the quality ofthe ongoing transactions among each Thus scale-up e orts relatedto educational reform must take place within the context of a poli-tical agenda that addresses ways to strengthen the family and com-munity infrastructure through strategies that enhance economicopportunity adult literacy and so forth What we need are policiesthat develop demonstrate and scale-up comprehensive multifacetedintegrated approaches that can e ectively address barriers to devel-opment learning and teaching

REFERENCES

Adelman H S (1993) School-linked mental health interventions Toward mechanismsfor service coordination and integration Journal of Community Psychology 21 309ndash319

Adelman H S (1996a) Restructuring education support services and integrating com-munity resources Beyond the full service school model School Psychology Review25 431ndash445

Adelman H S (1996b) Restructuring support services Toward a comprehensiveapproach Kent OH American School Health Association

Adelman H S amp Taylor L (1993a) Early Assistance for Students and FamiliesProgram Guidebook Los Angeles Early Assistance for Students and FamiliesProject Copies available from the School Mental Health Project Dept of Psychol-ogy UCLA

Adelman H S amp Taylor L (1993b) Learning problems and learning disabilities Moving forward Pacic Grove CA BrooksCole

Adelman H S amp Taylor L (1994) On understanding intervention in psychology andeducation Westport CT Praeger

Adelman H S amp Taylor L (1997a) Addressing barriers to learning Beyond school-linked services and full service schools American Journal of Orthopsychiatry 67408ndash421

Adelman H S amp Taylor L (1997b) Toward a scale-up model for replicating newapproaches to schooling Journal of Educational and Psychological Consultation 8197ndash230

316 L Taylor et al

processes that can instigate and enhance an individualrsquos perceptionsof valued opportunities choice and control accomplishment andrelatedness to others Even if a task isnrsquot enjoyable expectation offeeling some sense of satisfaction related to process or outcome canbe a powerful intrinsic factor motivating individual behavior Taskpersistence for example can be facilitated by the expectation thatone will feel competent self-determining or more closely connectedto others From this perspective ensuring that individuals havevalued options a meaningful role in decision making feedback thatemphasizes progress toward desired outcomes and positive workingrelationships are among the most basic facilitation strategies(Adelman amp Taylor 1993b 1994 Deci amp Ryan 1985)

One other initial implementation problem that often arises is diffi-culty in establishing mechanisms to facilitate productive workingrelationships and identify and deal with problems quickly Forexample it is expected that change agents will encounter manyinstances of individual resistance and apathy interpersonal conictsand resentments (including lsquolsquous vs themrsquorsquo dynamics) rumors thatoveremphasize the negative and underestimate the positive and indi-viduals who are frequent faultnders Such problems seriouslyimpede e ective replication The roots of some of these problemsoften are present at a site prior to scale-up change simply o ers anew focus and perhaps magnies troubling matters Other problemsare a direct product of the activities and relationships that the scale-up process engenders Given the inevitability of such problems build-ing and maintaining working relationships need to be among themost basic concerns for those who have responsibility for scale-up Inparticular considerable attention must be paid to enhancing themotivational readiness and capability of those who are to worktogether and ensure there is an appropriate infrastructure to guideand support working relationships Proactively this requires problemprevention mechanisms that help create an atmosphere where defen-siveness is curtailed and positive rapport is engendered The point isto enhance attitudes knowledge and skills that foster interpersonalconnections and a sense of community Reactively the emphasis is onproblem solving resolving conict and providing ongoing support torebuild relationships Policies must encourage problem-solving-oriented critiques safeguards that protect those making changesappreciation for e ort and celebration of progress We nd thateveryone understands such matters but the culture at many schoolsites is more attuned to problem naming and analyzing than to antici-pating preventing and solving problems that arise around workingrelationships

Scaling up Reforms 317

Those responsible for systemic change need to spend as much timeas necessary ensuring that a schoolrsquos infrastructure is ready toprevent and ameliorate problems Special attention must be paid toensuring that problem solving mechanisms and communication pro-cesses are in place and properly sta ed and that stakeholders arewell informed about how to use the procedures Furthermore somestakeholders may have to be encouraged to interact in ways thatconvey genuine empathy warmth and mutual regard and respectwith a view to creating and maintaining a positive working climateand a psychological sense of community

At times we nd it necessary to target a specic problem anddesignated persons In some instances rather simple strategies aree ective For example most motivated individuals can be directlytaught ways to improve understanding and communication and avoidor resolve conicts that interfere with working relationships Inother instances however signicant remedial action is necessary aswhen overcoming barriers to a working relationship involvescountering negative attitudes Helpful in this regard are analysessuch as that by Sue and Zane (1987) that suggest how to demonstratethat something of value can be gained from individuals workingtogether and how to establish each participantrsquos credibility (eg bymaximizing task-focus and positive outcomes)

PHASE III INSTITUTIONALIZING THE PROTOTYPE

Maintaining and enhancing changes can be as difficult as makingthem in the rst place The history of education reform is one offailure to foster promising prototypes in substantive ways and overan extended period of times (Tyack amp Cuban 1995) Institutionalizinga prototype entails ensuring that the organization assumes long-termownership and that there is a blueprint for countering forces that canerode the changes Moreover institutionalization is more than atechnical process It requires assimilation of and ongoing adherenceto the values inherent in the prototypersquos underlying rationale Thefocus of course is not just on maintenance the point is to moveforward by enhancing productive changes and generating a sense ofrenewal as needed Critical in all this are specic plans that guar-antee ongoing and enhanced leadership and delineate ways in whichplanning implementation coordination and continuing educationmechanisms are maintained

Some Major Tasks

Whose responsibility is it to advocate for maintaining and evolving a

318 L Taylor et al

replicated prototype for improving literacy As problems arise whoseresponsibility is it to lead the way in resolving them Leadership isthe key here by both official leaders such as administrators mentorsta union chapter chairs and elected parent representatives aswell as natural leaders such as reading and writing teachers Obvi-ously official and natural leaders are not mutually exclusive groupsAt this phase both types of leadership are essential to ensure a broadenough base for ongoing advocacy problem solving enhancementand renewal Official leaders provide a legitimate power base asvarious interests compete for the organizationrsquos limited resourcesand they play a key role in ensuring that the contributions of naturalleaders are recognized and rewarded

Maintenance and enhancement require that the organizationrsquosgovernance body assumes ownership and program advocacy such astaking over the temporary steering grouprsquos functions addressingongoing policy and long-range planning concerns and maintainingnancial support The foundation for such ownership is laid duringthe readiness phase Each element becomes the organizationrsquos pro-perty as it is established during initial implementation The officialdeed of ownership is transferred as soon as the prototype is in place

Ownership however is no guarantee of institutionalizationVarious forces that can erode reforms are always at work Forinstance teams at a site experience turnover problems with commu-nication and sharing of resources are chronic competing interestsand the attractiveness of moving on to something new pull attentionand resources to other activity To minimize such problems stepsmust be taken to identify and solve them as quickly as is feasibleThis requires someone who has the time energy and expertise tomeet periodically with stakeholders to anticipate and amelioratethreats to a prototypersquos integrity

Over time mechanisms for planning implementation and coordi-nation are maintained by ensuring the activity is an official part ofthe infrastructure has appropriate leadership and is e ectively sup-ported Anyone who has worked on a school-based team to improve aliteracy program knows there must be a critical mass of teammembers so that the work load is manageable and a broad base ofinvolvement is ensured Also essential are adequate resources includ-ing time to learn the role and time to perform the functions reason-ably interesting tasks technical support for problem solvingrecognition and rewards for contributions immediate replacementwhen someone leaves continuing education to enhance team func-tioning and so forth Without serious attention to such matters theteamsrsquo morale and motivation will wane

Scaling up Reforms 319

Lessons Learned

Newly institutionalized approaches are seriously jeopardized in theabsence of dedicated ongoing capacity-building Of particular impor-tance are ways to rapidly and e ectively assimilate new arrivals at aschool (sta students families) This is a major concern at sites withconsiderable turnover or growth At such sites the majority of thoseinitially involved in implementing a new approach may be gonewithin a period of two to three years Whatever the mobility rate it isessential to design and maintain transition programs for newarrivals Initial welcoming and introductory orientations of coursemust be followed-up with ongoing support systems and intensivecapacity building related to understanding and valuing theapproaches the school has adopted We nd that all this is essentialnot only to maintain what has been adopted but also can contributeto establishing schools as caring environments

PHASE IV ONGOING EVOLUTION

Ongoing evolution of organizations and programs is the product ofe orts to account for accomplishments deal with changing times andconditions incorporate new knowledge and create a sense of renewalas the excitement of newness wears o and the demands of changesap energy As suggested already in part vigor and direction can bemaintained through continuing education especially exposure toideas that suggest a range of ways for evolving a program As thefollowing discussion indicates ongoing evolution also is fostered byevaluation designed to document accomplishments and provide feed-back designed to improve quality

Increased concern over accountability has advanced the wayevaluation is conceived (Posavac amp Carey 1989 Rossi amp Freeman1989 Scriven 1993 Sechrest amp Figueredo 1993 Shadish Jr Cook ampLeviton 1991 Stake 1967 1976 Stufflebeam amp Webster 1983 Weiss1995) At the same time social and political forces have literallyshaped the whole enterprise and in the process have narrowed theway professionals clients policymakers underwriters and thegeneral public think about program evaluation A prevailing cry isfor specic evidence of e ectiveness For schools this means imme-diate gains in literacy as measured by achievement tests Althoughunderstandable in light of the unlled promise of so many programsand the insatiable demands on limited public nances such simplisti-cally conceived accountability demands ignore the complexities ofdeveloping and scaling-up major reforms

320 L Taylor et al

Formative and Summative Evaluation

Evaluation of a prototype involves more than determining efficacyfor students Broadly stated it encompasses concerns about how toexpand the focus of evaluative research not only to contribute toimproving practice but also to aid in evolving practice and policy(General Accounting Office 1989 Lyon amp Moats 1997) To facilitateprogram development and organizational change the primary orien-tation for evaluation in the early phases is formative It is especiallyfocused on data gathering and analyses that can help improve pro-cedures Most of what is written about educational and psychosocialintervention however is oriented to summative evaluation and mea-suring outcomes for individuals such as improved reading achieve-ment scores Replicating approaches to improve literacy involve notonly changing individuals but changing organizations and systemsThus both individuals and systems must be evaluated

All this presumes appropriate mechanisms to provide and analyzeessential information To these ends a scale-up sta can help estab-lish an evaluation team and capacity building that prepares a schoolto conduct evaluation that enhances reforms The immediate focus ison successful program replication ultimately of course the empha-sis must be on student outcomes

Pursuing Results

Due to the increased interest in accountability many complex aimsare broken down into specic objectives Indeed short-range objec-tives stated in measurable terms generally assume a central role inplanning However short-range objectives are not ends in them-selves They are a small part of a particular goal and aim and some-times are prerequisites for moving on to a goal It is essential not tolose sight of the fact that many specic objectives are relativelysmall unrepresentative and often unimportant segments of the mostvalued aims that society has for its citizens and citizens have forthemselves

The problem is well exemplied by the narrow focus found inreviews analyses and reanalyses of data on early education (eg seeAlbee amp Gullotta 1997 Bond amp Compas 1989 Dryfoos 1990 Durlak1995 Elias 1997 Mitchell Seligson amp Marx 1989 Schorr 1988Slavin Karweit amp Madden 1989 Weissberg Gullotta HamptomRyan amp Adams 1997) As such work demonstrates overemphasis onevaluating the efficacy of underdeveloped prototypes draws resourcesaway from formative evaluation

Scaling up Reforms 321

With specic respect to scale-up the rst accomplishment is thereplication itself Have all facets been implemented How completelyhas each been implemented At how many locations The next set ofresults are any indications of progress for students such as improve-ments in attitudes toward school health attendance behavior andacademic achievement A nal set of evaluation concerns is thedegree to which student outcomes approximate societal standards

Lessons Learned

The process of evaluating results is costly in terms of nancial invest-ment the negative psychological impact on those evaluated and theways it can inappropriately reshape new approaches Cost-e ectiveoutcomes cannot be achieved in the absence of e ective prototypedevelopment and research Premature e orts to carry out com-prehensive summative evaluations clearly are not cost-e ective Anyreading and writing program will show poor results if it is evaluatedbefore teachers have mastered its application None of this of courseis an argument against evaluating results Rather it is meant tounderscore concerns and encourage greater attention to addressingthem

Once a prototype is established care must be taken to avoiddeveloping outcome evaluation as an adversarial process Due to thepolitical realities related to accountability one of the most per-plexing facets to negotiate is the time frame for summative evalu-ation The more complex the prototype the longer it takes and thecostlier it is to implement and evaluate Schools usually want quickprocesses and results and of course rarely can a ord costly innova-tions or lengthy di usion activity Compromises are inevitable butmust be arrived at with great care so as not to undermine the sub-stance of proposed changes

The psychology of evaluation suggests that an overemphasis onaccountability tends to produce negative reactions One possible wayto counter this may be to conceive evaluation as a way for everystakeholder to self-evaluate as a basis for quality improvement and asa way of getting credit for all that is accomplished Unfortunately asaccountability pressures increase we nd that replication of proto-types are guided more by what can be measured than by long-rangeaims That is demands for immediate accountability reshape prac-tices so that the emphasis shifts to immediate and readily measuredobjectives and away from fundamental purposes Over time this inap-propriately leads to radical revision of the underlying rationale for aprototype

322 L Taylor et al

CONCLUDING COMMENTS

Those who set out to change schools and schooling are confrontedwith two enormous tasks The rst is to develop prototypes Thesecond involves large-scale replication One without the other isinsufficient Yet considerably more attention is paid to developingand validating prototypes than to delineating and testing scale-upprocesses Clearly it is time to correct this deciency The ideas pre-sented in this article are meant to stimulate work on the problem andthereby to advance the cause of educational reform

Finally in fairness to those who labor for educational reform weall must remember that the quality of schooling family life and com-munity functioning spirals up or down as a function of the quality ofthe ongoing transactions among each Thus scale-up e orts relatedto educational reform must take place within the context of a poli-tical agenda that addresses ways to strengthen the family and com-munity infrastructure through strategies that enhance economicopportunity adult literacy and so forth What we need are policiesthat develop demonstrate and scale-up comprehensive multifacetedintegrated approaches that can e ectively address barriers to devel-opment learning and teaching

REFERENCES

Adelman H S (1993) School-linked mental health interventions Toward mechanismsfor service coordination and integration Journal of Community Psychology 21 309ndash319

Adelman H S (1996a) Restructuring education support services and integrating com-munity resources Beyond the full service school model School Psychology Review25 431ndash445

Adelman H S (1996b) Restructuring support services Toward a comprehensiveapproach Kent OH American School Health Association

Adelman H S amp Taylor L (1993a) Early Assistance for Students and FamiliesProgram Guidebook Los Angeles Early Assistance for Students and FamiliesProject Copies available from the School Mental Health Project Dept of Psychol-ogy UCLA

Adelman H S amp Taylor L (1993b) Learning problems and learning disabilities Moving forward Pacic Grove CA BrooksCole

Adelman H S amp Taylor L (1994) On understanding intervention in psychology andeducation Westport CT Praeger

Adelman H S amp Taylor L (1997a) Addressing barriers to learning Beyond school-linked services and full service schools American Journal of Orthopsychiatry 67408ndash421

Adelman H S amp Taylor L (1997b) Toward a scale-up model for replicating newapproaches to schooling Journal of Educational and Psychological Consultation 8197ndash230

Scaling up Reforms 317

Those responsible for systemic change need to spend as much timeas necessary ensuring that a schoolrsquos infrastructure is ready toprevent and ameliorate problems Special attention must be paid toensuring that problem solving mechanisms and communication pro-cesses are in place and properly sta ed and that stakeholders arewell informed about how to use the procedures Furthermore somestakeholders may have to be encouraged to interact in ways thatconvey genuine empathy warmth and mutual regard and respectwith a view to creating and maintaining a positive working climateand a psychological sense of community

At times we nd it necessary to target a specic problem anddesignated persons In some instances rather simple strategies aree ective For example most motivated individuals can be directlytaught ways to improve understanding and communication and avoidor resolve conicts that interfere with working relationships Inother instances however signicant remedial action is necessary aswhen overcoming barriers to a working relationship involvescountering negative attitudes Helpful in this regard are analysessuch as that by Sue and Zane (1987) that suggest how to demonstratethat something of value can be gained from individuals workingtogether and how to establish each participantrsquos credibility (eg bymaximizing task-focus and positive outcomes)

PHASE III INSTITUTIONALIZING THE PROTOTYPE

Maintaining and enhancing changes can be as difficult as makingthem in the rst place The history of education reform is one offailure to foster promising prototypes in substantive ways and overan extended period of times (Tyack amp Cuban 1995) Institutionalizinga prototype entails ensuring that the organization assumes long-termownership and that there is a blueprint for countering forces that canerode the changes Moreover institutionalization is more than atechnical process It requires assimilation of and ongoing adherenceto the values inherent in the prototypersquos underlying rationale Thefocus of course is not just on maintenance the point is to moveforward by enhancing productive changes and generating a sense ofrenewal as needed Critical in all this are specic plans that guar-antee ongoing and enhanced leadership and delineate ways in whichplanning implementation coordination and continuing educationmechanisms are maintained

Some Major Tasks

Whose responsibility is it to advocate for maintaining and evolving a

318 L Taylor et al

replicated prototype for improving literacy As problems arise whoseresponsibility is it to lead the way in resolving them Leadership isthe key here by both official leaders such as administrators mentorsta union chapter chairs and elected parent representatives aswell as natural leaders such as reading and writing teachers Obvi-ously official and natural leaders are not mutually exclusive groupsAt this phase both types of leadership are essential to ensure a broadenough base for ongoing advocacy problem solving enhancementand renewal Official leaders provide a legitimate power base asvarious interests compete for the organizationrsquos limited resourcesand they play a key role in ensuring that the contributions of naturalleaders are recognized and rewarded

Maintenance and enhancement require that the organizationrsquosgovernance body assumes ownership and program advocacy such astaking over the temporary steering grouprsquos functions addressingongoing policy and long-range planning concerns and maintainingnancial support The foundation for such ownership is laid duringthe readiness phase Each element becomes the organizationrsquos pro-perty as it is established during initial implementation The officialdeed of ownership is transferred as soon as the prototype is in place

Ownership however is no guarantee of institutionalizationVarious forces that can erode reforms are always at work Forinstance teams at a site experience turnover problems with commu-nication and sharing of resources are chronic competing interestsand the attractiveness of moving on to something new pull attentionand resources to other activity To minimize such problems stepsmust be taken to identify and solve them as quickly as is feasibleThis requires someone who has the time energy and expertise tomeet periodically with stakeholders to anticipate and amelioratethreats to a prototypersquos integrity

Over time mechanisms for planning implementation and coordi-nation are maintained by ensuring the activity is an official part ofthe infrastructure has appropriate leadership and is e ectively sup-ported Anyone who has worked on a school-based team to improve aliteracy program knows there must be a critical mass of teammembers so that the work load is manageable and a broad base ofinvolvement is ensured Also essential are adequate resources includ-ing time to learn the role and time to perform the functions reason-ably interesting tasks technical support for problem solvingrecognition and rewards for contributions immediate replacementwhen someone leaves continuing education to enhance team func-tioning and so forth Without serious attention to such matters theteamsrsquo morale and motivation will wane

Scaling up Reforms 319

Lessons Learned

Newly institutionalized approaches are seriously jeopardized in theabsence of dedicated ongoing capacity-building Of particular impor-tance are ways to rapidly and e ectively assimilate new arrivals at aschool (sta students families) This is a major concern at sites withconsiderable turnover or growth At such sites the majority of thoseinitially involved in implementing a new approach may be gonewithin a period of two to three years Whatever the mobility rate it isessential to design and maintain transition programs for newarrivals Initial welcoming and introductory orientations of coursemust be followed-up with ongoing support systems and intensivecapacity building related to understanding and valuing theapproaches the school has adopted We nd that all this is essentialnot only to maintain what has been adopted but also can contributeto establishing schools as caring environments

PHASE IV ONGOING EVOLUTION

Ongoing evolution of organizations and programs is the product ofe orts to account for accomplishments deal with changing times andconditions incorporate new knowledge and create a sense of renewalas the excitement of newness wears o and the demands of changesap energy As suggested already in part vigor and direction can bemaintained through continuing education especially exposure toideas that suggest a range of ways for evolving a program As thefollowing discussion indicates ongoing evolution also is fostered byevaluation designed to document accomplishments and provide feed-back designed to improve quality

Increased concern over accountability has advanced the wayevaluation is conceived (Posavac amp Carey 1989 Rossi amp Freeman1989 Scriven 1993 Sechrest amp Figueredo 1993 Shadish Jr Cook ampLeviton 1991 Stake 1967 1976 Stufflebeam amp Webster 1983 Weiss1995) At the same time social and political forces have literallyshaped the whole enterprise and in the process have narrowed theway professionals clients policymakers underwriters and thegeneral public think about program evaluation A prevailing cry isfor specic evidence of e ectiveness For schools this means imme-diate gains in literacy as measured by achievement tests Althoughunderstandable in light of the unlled promise of so many programsand the insatiable demands on limited public nances such simplisti-cally conceived accountability demands ignore the complexities ofdeveloping and scaling-up major reforms

320 L Taylor et al

Formative and Summative Evaluation

Evaluation of a prototype involves more than determining efficacyfor students Broadly stated it encompasses concerns about how toexpand the focus of evaluative research not only to contribute toimproving practice but also to aid in evolving practice and policy(General Accounting Office 1989 Lyon amp Moats 1997) To facilitateprogram development and organizational change the primary orien-tation for evaluation in the early phases is formative It is especiallyfocused on data gathering and analyses that can help improve pro-cedures Most of what is written about educational and psychosocialintervention however is oriented to summative evaluation and mea-suring outcomes for individuals such as improved reading achieve-ment scores Replicating approaches to improve literacy involve notonly changing individuals but changing organizations and systemsThus both individuals and systems must be evaluated

All this presumes appropriate mechanisms to provide and analyzeessential information To these ends a scale-up sta can help estab-lish an evaluation team and capacity building that prepares a schoolto conduct evaluation that enhances reforms The immediate focus ison successful program replication ultimately of course the empha-sis must be on student outcomes

Pursuing Results

Due to the increased interest in accountability many complex aimsare broken down into specic objectives Indeed short-range objec-tives stated in measurable terms generally assume a central role inplanning However short-range objectives are not ends in them-selves They are a small part of a particular goal and aim and some-times are prerequisites for moving on to a goal It is essential not tolose sight of the fact that many specic objectives are relativelysmall unrepresentative and often unimportant segments of the mostvalued aims that society has for its citizens and citizens have forthemselves

The problem is well exemplied by the narrow focus found inreviews analyses and reanalyses of data on early education (eg seeAlbee amp Gullotta 1997 Bond amp Compas 1989 Dryfoos 1990 Durlak1995 Elias 1997 Mitchell Seligson amp Marx 1989 Schorr 1988Slavin Karweit amp Madden 1989 Weissberg Gullotta HamptomRyan amp Adams 1997) As such work demonstrates overemphasis onevaluating the efficacy of underdeveloped prototypes draws resourcesaway from formative evaluation

Scaling up Reforms 321

With specic respect to scale-up the rst accomplishment is thereplication itself Have all facets been implemented How completelyhas each been implemented At how many locations The next set ofresults are any indications of progress for students such as improve-ments in attitudes toward school health attendance behavior andacademic achievement A nal set of evaluation concerns is thedegree to which student outcomes approximate societal standards

Lessons Learned

The process of evaluating results is costly in terms of nancial invest-ment the negative psychological impact on those evaluated and theways it can inappropriately reshape new approaches Cost-e ectiveoutcomes cannot be achieved in the absence of e ective prototypedevelopment and research Premature e orts to carry out com-prehensive summative evaluations clearly are not cost-e ective Anyreading and writing program will show poor results if it is evaluatedbefore teachers have mastered its application None of this of courseis an argument against evaluating results Rather it is meant tounderscore concerns and encourage greater attention to addressingthem

Once a prototype is established care must be taken to avoiddeveloping outcome evaluation as an adversarial process Due to thepolitical realities related to accountability one of the most per-plexing facets to negotiate is the time frame for summative evalu-ation The more complex the prototype the longer it takes and thecostlier it is to implement and evaluate Schools usually want quickprocesses and results and of course rarely can a ord costly innova-tions or lengthy di usion activity Compromises are inevitable butmust be arrived at with great care so as not to undermine the sub-stance of proposed changes

The psychology of evaluation suggests that an overemphasis onaccountability tends to produce negative reactions One possible wayto counter this may be to conceive evaluation as a way for everystakeholder to self-evaluate as a basis for quality improvement and asa way of getting credit for all that is accomplished Unfortunately asaccountability pressures increase we nd that replication of proto-types are guided more by what can be measured than by long-rangeaims That is demands for immediate accountability reshape prac-tices so that the emphasis shifts to immediate and readily measuredobjectives and away from fundamental purposes Over time this inap-propriately leads to radical revision of the underlying rationale for aprototype

322 L Taylor et al

CONCLUDING COMMENTS

Those who set out to change schools and schooling are confrontedwith two enormous tasks The rst is to develop prototypes Thesecond involves large-scale replication One without the other isinsufficient Yet considerably more attention is paid to developingand validating prototypes than to delineating and testing scale-upprocesses Clearly it is time to correct this deciency The ideas pre-sented in this article are meant to stimulate work on the problem andthereby to advance the cause of educational reform

Finally in fairness to those who labor for educational reform weall must remember that the quality of schooling family life and com-munity functioning spirals up or down as a function of the quality ofthe ongoing transactions among each Thus scale-up e orts relatedto educational reform must take place within the context of a poli-tical agenda that addresses ways to strengthen the family and com-munity infrastructure through strategies that enhance economicopportunity adult literacy and so forth What we need are policiesthat develop demonstrate and scale-up comprehensive multifacetedintegrated approaches that can e ectively address barriers to devel-opment learning and teaching

REFERENCES

Adelman H S (1993) School-linked mental health interventions Toward mechanismsfor service coordination and integration Journal of Community Psychology 21 309ndash319

Adelman H S (1996a) Restructuring education support services and integrating com-munity resources Beyond the full service school model School Psychology Review25 431ndash445

Adelman H S (1996b) Restructuring support services Toward a comprehensiveapproach Kent OH American School Health Association

Adelman H S amp Taylor L (1993a) Early Assistance for Students and FamiliesProgram Guidebook Los Angeles Early Assistance for Students and FamiliesProject Copies available from the School Mental Health Project Dept of Psychol-ogy UCLA

Adelman H S amp Taylor L (1993b) Learning problems and learning disabilities Moving forward Pacic Grove CA BrooksCole

Adelman H S amp Taylor L (1994) On understanding intervention in psychology andeducation Westport CT Praeger

Adelman H S amp Taylor L (1997a) Addressing barriers to learning Beyond school-linked services and full service schools American Journal of Orthopsychiatry 67408ndash421

Adelman H S amp Taylor L (1997b) Toward a scale-up model for replicating newapproaches to schooling Journal of Educational and Psychological Consultation 8197ndash230

318 L Taylor et al

replicated prototype for improving literacy As problems arise whoseresponsibility is it to lead the way in resolving them Leadership isthe key here by both official leaders such as administrators mentorsta union chapter chairs and elected parent representatives aswell as natural leaders such as reading and writing teachers Obvi-ously official and natural leaders are not mutually exclusive groupsAt this phase both types of leadership are essential to ensure a broadenough base for ongoing advocacy problem solving enhancementand renewal Official leaders provide a legitimate power base asvarious interests compete for the organizationrsquos limited resourcesand they play a key role in ensuring that the contributions of naturalleaders are recognized and rewarded

Maintenance and enhancement require that the organizationrsquosgovernance body assumes ownership and program advocacy such astaking over the temporary steering grouprsquos functions addressingongoing policy and long-range planning concerns and maintainingnancial support The foundation for such ownership is laid duringthe readiness phase Each element becomes the organizationrsquos pro-perty as it is established during initial implementation The officialdeed of ownership is transferred as soon as the prototype is in place

Ownership however is no guarantee of institutionalizationVarious forces that can erode reforms are always at work Forinstance teams at a site experience turnover problems with commu-nication and sharing of resources are chronic competing interestsand the attractiveness of moving on to something new pull attentionand resources to other activity To minimize such problems stepsmust be taken to identify and solve them as quickly as is feasibleThis requires someone who has the time energy and expertise tomeet periodically with stakeholders to anticipate and amelioratethreats to a prototypersquos integrity

Over time mechanisms for planning implementation and coordi-nation are maintained by ensuring the activity is an official part ofthe infrastructure has appropriate leadership and is e ectively sup-ported Anyone who has worked on a school-based team to improve aliteracy program knows there must be a critical mass of teammembers so that the work load is manageable and a broad base ofinvolvement is ensured Also essential are adequate resources includ-ing time to learn the role and time to perform the functions reason-ably interesting tasks technical support for problem solvingrecognition and rewards for contributions immediate replacementwhen someone leaves continuing education to enhance team func-tioning and so forth Without serious attention to such matters theteamsrsquo morale and motivation will wane

Scaling up Reforms 319

Lessons Learned

Newly institutionalized approaches are seriously jeopardized in theabsence of dedicated ongoing capacity-building Of particular impor-tance are ways to rapidly and e ectively assimilate new arrivals at aschool (sta students families) This is a major concern at sites withconsiderable turnover or growth At such sites the majority of thoseinitially involved in implementing a new approach may be gonewithin a period of two to three years Whatever the mobility rate it isessential to design and maintain transition programs for newarrivals Initial welcoming and introductory orientations of coursemust be followed-up with ongoing support systems and intensivecapacity building related to understanding and valuing theapproaches the school has adopted We nd that all this is essentialnot only to maintain what has been adopted but also can contributeto establishing schools as caring environments

PHASE IV ONGOING EVOLUTION

Ongoing evolution of organizations and programs is the product ofe orts to account for accomplishments deal with changing times andconditions incorporate new knowledge and create a sense of renewalas the excitement of newness wears o and the demands of changesap energy As suggested already in part vigor and direction can bemaintained through continuing education especially exposure toideas that suggest a range of ways for evolving a program As thefollowing discussion indicates ongoing evolution also is fostered byevaluation designed to document accomplishments and provide feed-back designed to improve quality

Increased concern over accountability has advanced the wayevaluation is conceived (Posavac amp Carey 1989 Rossi amp Freeman1989 Scriven 1993 Sechrest amp Figueredo 1993 Shadish Jr Cook ampLeviton 1991 Stake 1967 1976 Stufflebeam amp Webster 1983 Weiss1995) At the same time social and political forces have literallyshaped the whole enterprise and in the process have narrowed theway professionals clients policymakers underwriters and thegeneral public think about program evaluation A prevailing cry isfor specic evidence of e ectiveness For schools this means imme-diate gains in literacy as measured by achievement tests Althoughunderstandable in light of the unlled promise of so many programsand the insatiable demands on limited public nances such simplisti-cally conceived accountability demands ignore the complexities ofdeveloping and scaling-up major reforms

320 L Taylor et al

Formative and Summative Evaluation

Evaluation of a prototype involves more than determining efficacyfor students Broadly stated it encompasses concerns about how toexpand the focus of evaluative research not only to contribute toimproving practice but also to aid in evolving practice and policy(General Accounting Office 1989 Lyon amp Moats 1997) To facilitateprogram development and organizational change the primary orien-tation for evaluation in the early phases is formative It is especiallyfocused on data gathering and analyses that can help improve pro-cedures Most of what is written about educational and psychosocialintervention however is oriented to summative evaluation and mea-suring outcomes for individuals such as improved reading achieve-ment scores Replicating approaches to improve literacy involve notonly changing individuals but changing organizations and systemsThus both individuals and systems must be evaluated

All this presumes appropriate mechanisms to provide and analyzeessential information To these ends a scale-up sta can help estab-lish an evaluation team and capacity building that prepares a schoolto conduct evaluation that enhances reforms The immediate focus ison successful program replication ultimately of course the empha-sis must be on student outcomes

Pursuing Results

Due to the increased interest in accountability many complex aimsare broken down into specic objectives Indeed short-range objec-tives stated in measurable terms generally assume a central role inplanning However short-range objectives are not ends in them-selves They are a small part of a particular goal and aim and some-times are prerequisites for moving on to a goal It is essential not tolose sight of the fact that many specic objectives are relativelysmall unrepresentative and often unimportant segments of the mostvalued aims that society has for its citizens and citizens have forthemselves

The problem is well exemplied by the narrow focus found inreviews analyses and reanalyses of data on early education (eg seeAlbee amp Gullotta 1997 Bond amp Compas 1989 Dryfoos 1990 Durlak1995 Elias 1997 Mitchell Seligson amp Marx 1989 Schorr 1988Slavin Karweit amp Madden 1989 Weissberg Gullotta HamptomRyan amp Adams 1997) As such work demonstrates overemphasis onevaluating the efficacy of underdeveloped prototypes draws resourcesaway from formative evaluation

Scaling up Reforms 321

With specic respect to scale-up the rst accomplishment is thereplication itself Have all facets been implemented How completelyhas each been implemented At how many locations The next set ofresults are any indications of progress for students such as improve-ments in attitudes toward school health attendance behavior andacademic achievement A nal set of evaluation concerns is thedegree to which student outcomes approximate societal standards

Lessons Learned

The process of evaluating results is costly in terms of nancial invest-ment the negative psychological impact on those evaluated and theways it can inappropriately reshape new approaches Cost-e ectiveoutcomes cannot be achieved in the absence of e ective prototypedevelopment and research Premature e orts to carry out com-prehensive summative evaluations clearly are not cost-e ective Anyreading and writing program will show poor results if it is evaluatedbefore teachers have mastered its application None of this of courseis an argument against evaluating results Rather it is meant tounderscore concerns and encourage greater attention to addressingthem

Once a prototype is established care must be taken to avoiddeveloping outcome evaluation as an adversarial process Due to thepolitical realities related to accountability one of the most per-plexing facets to negotiate is the time frame for summative evalu-ation The more complex the prototype the longer it takes and thecostlier it is to implement and evaluate Schools usually want quickprocesses and results and of course rarely can a ord costly innova-tions or lengthy di usion activity Compromises are inevitable butmust be arrived at with great care so as not to undermine the sub-stance of proposed changes

The psychology of evaluation suggests that an overemphasis onaccountability tends to produce negative reactions One possible wayto counter this may be to conceive evaluation as a way for everystakeholder to self-evaluate as a basis for quality improvement and asa way of getting credit for all that is accomplished Unfortunately asaccountability pressures increase we nd that replication of proto-types are guided more by what can be measured than by long-rangeaims That is demands for immediate accountability reshape prac-tices so that the emphasis shifts to immediate and readily measuredobjectives and away from fundamental purposes Over time this inap-propriately leads to radical revision of the underlying rationale for aprototype

322 L Taylor et al

CONCLUDING COMMENTS

Those who set out to change schools and schooling are confrontedwith two enormous tasks The rst is to develop prototypes Thesecond involves large-scale replication One without the other isinsufficient Yet considerably more attention is paid to developingand validating prototypes than to delineating and testing scale-upprocesses Clearly it is time to correct this deciency The ideas pre-sented in this article are meant to stimulate work on the problem andthereby to advance the cause of educational reform

Finally in fairness to those who labor for educational reform weall must remember that the quality of schooling family life and com-munity functioning spirals up or down as a function of the quality ofthe ongoing transactions among each Thus scale-up e orts relatedto educational reform must take place within the context of a poli-tical agenda that addresses ways to strengthen the family and com-munity infrastructure through strategies that enhance economicopportunity adult literacy and so forth What we need are policiesthat develop demonstrate and scale-up comprehensive multifacetedintegrated approaches that can e ectively address barriers to devel-opment learning and teaching

REFERENCES

Adelman H S (1993) School-linked mental health interventions Toward mechanismsfor service coordination and integration Journal of Community Psychology 21 309ndash319

Adelman H S (1996a) Restructuring education support services and integrating com-munity resources Beyond the full service school model School Psychology Review25 431ndash445

Adelman H S (1996b) Restructuring support services Toward a comprehensiveapproach Kent OH American School Health Association

Adelman H S amp Taylor L (1993a) Early Assistance for Students and FamiliesProgram Guidebook Los Angeles Early Assistance for Students and FamiliesProject Copies available from the School Mental Health Project Dept of Psychol-ogy UCLA

Adelman H S amp Taylor L (1993b) Learning problems and learning disabilities Moving forward Pacic Grove CA BrooksCole

Adelman H S amp Taylor L (1994) On understanding intervention in psychology andeducation Westport CT Praeger

Adelman H S amp Taylor L (1997a) Addressing barriers to learning Beyond school-linked services and full service schools American Journal of Orthopsychiatry 67408ndash421

Adelman H S amp Taylor L (1997b) Toward a scale-up model for replicating newapproaches to schooling Journal of Educational and Psychological Consultation 8197ndash230

Scaling up Reforms 319

Lessons Learned

Newly institutionalized approaches are seriously jeopardized in theabsence of dedicated ongoing capacity-building Of particular impor-tance are ways to rapidly and e ectively assimilate new arrivals at aschool (sta students families) This is a major concern at sites withconsiderable turnover or growth At such sites the majority of thoseinitially involved in implementing a new approach may be gonewithin a period of two to three years Whatever the mobility rate it isessential to design and maintain transition programs for newarrivals Initial welcoming and introductory orientations of coursemust be followed-up with ongoing support systems and intensivecapacity building related to understanding and valuing theapproaches the school has adopted We nd that all this is essentialnot only to maintain what has been adopted but also can contributeto establishing schools as caring environments

PHASE IV ONGOING EVOLUTION

Ongoing evolution of organizations and programs is the product ofe orts to account for accomplishments deal with changing times andconditions incorporate new knowledge and create a sense of renewalas the excitement of newness wears o and the demands of changesap energy As suggested already in part vigor and direction can bemaintained through continuing education especially exposure toideas that suggest a range of ways for evolving a program As thefollowing discussion indicates ongoing evolution also is fostered byevaluation designed to document accomplishments and provide feed-back designed to improve quality

Increased concern over accountability has advanced the wayevaluation is conceived (Posavac amp Carey 1989 Rossi amp Freeman1989 Scriven 1993 Sechrest amp Figueredo 1993 Shadish Jr Cook ampLeviton 1991 Stake 1967 1976 Stufflebeam amp Webster 1983 Weiss1995) At the same time social and political forces have literallyshaped the whole enterprise and in the process have narrowed theway professionals clients policymakers underwriters and thegeneral public think about program evaluation A prevailing cry isfor specic evidence of e ectiveness For schools this means imme-diate gains in literacy as measured by achievement tests Althoughunderstandable in light of the unlled promise of so many programsand the insatiable demands on limited public nances such simplisti-cally conceived accountability demands ignore the complexities ofdeveloping and scaling-up major reforms

320 L Taylor et al

Formative and Summative Evaluation

Evaluation of a prototype involves more than determining efficacyfor students Broadly stated it encompasses concerns about how toexpand the focus of evaluative research not only to contribute toimproving practice but also to aid in evolving practice and policy(General Accounting Office 1989 Lyon amp Moats 1997) To facilitateprogram development and organizational change the primary orien-tation for evaluation in the early phases is formative It is especiallyfocused on data gathering and analyses that can help improve pro-cedures Most of what is written about educational and psychosocialintervention however is oriented to summative evaluation and mea-suring outcomes for individuals such as improved reading achieve-ment scores Replicating approaches to improve literacy involve notonly changing individuals but changing organizations and systemsThus both individuals and systems must be evaluated

All this presumes appropriate mechanisms to provide and analyzeessential information To these ends a scale-up sta can help estab-lish an evaluation team and capacity building that prepares a schoolto conduct evaluation that enhances reforms The immediate focus ison successful program replication ultimately of course the empha-sis must be on student outcomes

Pursuing Results

Due to the increased interest in accountability many complex aimsare broken down into specic objectives Indeed short-range objec-tives stated in measurable terms generally assume a central role inplanning However short-range objectives are not ends in them-selves They are a small part of a particular goal and aim and some-times are prerequisites for moving on to a goal It is essential not tolose sight of the fact that many specic objectives are relativelysmall unrepresentative and often unimportant segments of the mostvalued aims that society has for its citizens and citizens have forthemselves

The problem is well exemplied by the narrow focus found inreviews analyses and reanalyses of data on early education (eg seeAlbee amp Gullotta 1997 Bond amp Compas 1989 Dryfoos 1990 Durlak1995 Elias 1997 Mitchell Seligson amp Marx 1989 Schorr 1988Slavin Karweit amp Madden 1989 Weissberg Gullotta HamptomRyan amp Adams 1997) As such work demonstrates overemphasis onevaluating the efficacy of underdeveloped prototypes draws resourcesaway from formative evaluation

Scaling up Reforms 321

With specic respect to scale-up the rst accomplishment is thereplication itself Have all facets been implemented How completelyhas each been implemented At how many locations The next set ofresults are any indications of progress for students such as improve-ments in attitudes toward school health attendance behavior andacademic achievement A nal set of evaluation concerns is thedegree to which student outcomes approximate societal standards

Lessons Learned

The process of evaluating results is costly in terms of nancial invest-ment the negative psychological impact on those evaluated and theways it can inappropriately reshape new approaches Cost-e ectiveoutcomes cannot be achieved in the absence of e ective prototypedevelopment and research Premature e orts to carry out com-prehensive summative evaluations clearly are not cost-e ective Anyreading and writing program will show poor results if it is evaluatedbefore teachers have mastered its application None of this of courseis an argument against evaluating results Rather it is meant tounderscore concerns and encourage greater attention to addressingthem

Once a prototype is established care must be taken to avoiddeveloping outcome evaluation as an adversarial process Due to thepolitical realities related to accountability one of the most per-plexing facets to negotiate is the time frame for summative evalu-ation The more complex the prototype the longer it takes and thecostlier it is to implement and evaluate Schools usually want quickprocesses and results and of course rarely can a ord costly innova-tions or lengthy di usion activity Compromises are inevitable butmust be arrived at with great care so as not to undermine the sub-stance of proposed changes

The psychology of evaluation suggests that an overemphasis onaccountability tends to produce negative reactions One possible wayto counter this may be to conceive evaluation as a way for everystakeholder to self-evaluate as a basis for quality improvement and asa way of getting credit for all that is accomplished Unfortunately asaccountability pressures increase we nd that replication of proto-types are guided more by what can be measured than by long-rangeaims That is demands for immediate accountability reshape prac-tices so that the emphasis shifts to immediate and readily measuredobjectives and away from fundamental purposes Over time this inap-propriately leads to radical revision of the underlying rationale for aprototype

322 L Taylor et al

CONCLUDING COMMENTS

Those who set out to change schools and schooling are confrontedwith two enormous tasks The rst is to develop prototypes Thesecond involves large-scale replication One without the other isinsufficient Yet considerably more attention is paid to developingand validating prototypes than to delineating and testing scale-upprocesses Clearly it is time to correct this deciency The ideas pre-sented in this article are meant to stimulate work on the problem andthereby to advance the cause of educational reform

Finally in fairness to those who labor for educational reform weall must remember that the quality of schooling family life and com-munity functioning spirals up or down as a function of the quality ofthe ongoing transactions among each Thus scale-up e orts relatedto educational reform must take place within the context of a poli-tical agenda that addresses ways to strengthen the family and com-munity infrastructure through strategies that enhance economicopportunity adult literacy and so forth What we need are policiesthat develop demonstrate and scale-up comprehensive multifacetedintegrated approaches that can e ectively address barriers to devel-opment learning and teaching

REFERENCES

Adelman H S (1993) School-linked mental health interventions Toward mechanismsfor service coordination and integration Journal of Community Psychology 21 309ndash319

Adelman H S (1996a) Restructuring education support services and integrating com-munity resources Beyond the full service school model School Psychology Review25 431ndash445

Adelman H S (1996b) Restructuring support services Toward a comprehensiveapproach Kent OH American School Health Association

Adelman H S amp Taylor L (1993a) Early Assistance for Students and FamiliesProgram Guidebook Los Angeles Early Assistance for Students and FamiliesProject Copies available from the School Mental Health Project Dept of Psychol-ogy UCLA

Adelman H S amp Taylor L (1993b) Learning problems and learning disabilities Moving forward Pacic Grove CA BrooksCole

Adelman H S amp Taylor L (1994) On understanding intervention in psychology andeducation Westport CT Praeger

Adelman H S amp Taylor L (1997a) Addressing barriers to learning Beyond school-linked services and full service schools American Journal of Orthopsychiatry 67408ndash421

Adelman H S amp Taylor L (1997b) Toward a scale-up model for replicating newapproaches to schooling Journal of Educational and Psychological Consultation 8197ndash230

320 L Taylor et al

Formative and Summative Evaluation

Evaluation of a prototype involves more than determining efficacyfor students Broadly stated it encompasses concerns about how toexpand the focus of evaluative research not only to contribute toimproving practice but also to aid in evolving practice and policy(General Accounting Office 1989 Lyon amp Moats 1997) To facilitateprogram development and organizational change the primary orien-tation for evaluation in the early phases is formative It is especiallyfocused on data gathering and analyses that can help improve pro-cedures Most of what is written about educational and psychosocialintervention however is oriented to summative evaluation and mea-suring outcomes for individuals such as improved reading achieve-ment scores Replicating approaches to improve literacy involve notonly changing individuals but changing organizations and systemsThus both individuals and systems must be evaluated

All this presumes appropriate mechanisms to provide and analyzeessential information To these ends a scale-up sta can help estab-lish an evaluation team and capacity building that prepares a schoolto conduct evaluation that enhances reforms The immediate focus ison successful program replication ultimately of course the empha-sis must be on student outcomes

Pursuing Results

Due to the increased interest in accountability many complex aimsare broken down into specic objectives Indeed short-range objec-tives stated in measurable terms generally assume a central role inplanning However short-range objectives are not ends in them-selves They are a small part of a particular goal and aim and some-times are prerequisites for moving on to a goal It is essential not tolose sight of the fact that many specic objectives are relativelysmall unrepresentative and often unimportant segments of the mostvalued aims that society has for its citizens and citizens have forthemselves

The problem is well exemplied by the narrow focus found inreviews analyses and reanalyses of data on early education (eg seeAlbee amp Gullotta 1997 Bond amp Compas 1989 Dryfoos 1990 Durlak1995 Elias 1997 Mitchell Seligson amp Marx 1989 Schorr 1988Slavin Karweit amp Madden 1989 Weissberg Gullotta HamptomRyan amp Adams 1997) As such work demonstrates overemphasis onevaluating the efficacy of underdeveloped prototypes draws resourcesaway from formative evaluation

Scaling up Reforms 321

With specic respect to scale-up the rst accomplishment is thereplication itself Have all facets been implemented How completelyhas each been implemented At how many locations The next set ofresults are any indications of progress for students such as improve-ments in attitudes toward school health attendance behavior andacademic achievement A nal set of evaluation concerns is thedegree to which student outcomes approximate societal standards

Lessons Learned

The process of evaluating results is costly in terms of nancial invest-ment the negative psychological impact on those evaluated and theways it can inappropriately reshape new approaches Cost-e ectiveoutcomes cannot be achieved in the absence of e ective prototypedevelopment and research Premature e orts to carry out com-prehensive summative evaluations clearly are not cost-e ective Anyreading and writing program will show poor results if it is evaluatedbefore teachers have mastered its application None of this of courseis an argument against evaluating results Rather it is meant tounderscore concerns and encourage greater attention to addressingthem

Once a prototype is established care must be taken to avoiddeveloping outcome evaluation as an adversarial process Due to thepolitical realities related to accountability one of the most per-plexing facets to negotiate is the time frame for summative evalu-ation The more complex the prototype the longer it takes and thecostlier it is to implement and evaluate Schools usually want quickprocesses and results and of course rarely can a ord costly innova-tions or lengthy di usion activity Compromises are inevitable butmust be arrived at with great care so as not to undermine the sub-stance of proposed changes

The psychology of evaluation suggests that an overemphasis onaccountability tends to produce negative reactions One possible wayto counter this may be to conceive evaluation as a way for everystakeholder to self-evaluate as a basis for quality improvement and asa way of getting credit for all that is accomplished Unfortunately asaccountability pressures increase we nd that replication of proto-types are guided more by what can be measured than by long-rangeaims That is demands for immediate accountability reshape prac-tices so that the emphasis shifts to immediate and readily measuredobjectives and away from fundamental purposes Over time this inap-propriately leads to radical revision of the underlying rationale for aprototype

322 L Taylor et al

CONCLUDING COMMENTS

Those who set out to change schools and schooling are confrontedwith two enormous tasks The rst is to develop prototypes Thesecond involves large-scale replication One without the other isinsufficient Yet considerably more attention is paid to developingand validating prototypes than to delineating and testing scale-upprocesses Clearly it is time to correct this deciency The ideas pre-sented in this article are meant to stimulate work on the problem andthereby to advance the cause of educational reform

Finally in fairness to those who labor for educational reform weall must remember that the quality of schooling family life and com-munity functioning spirals up or down as a function of the quality ofthe ongoing transactions among each Thus scale-up e orts relatedto educational reform must take place within the context of a poli-tical agenda that addresses ways to strengthen the family and com-munity infrastructure through strategies that enhance economicopportunity adult literacy and so forth What we need are policiesthat develop demonstrate and scale-up comprehensive multifacetedintegrated approaches that can e ectively address barriers to devel-opment learning and teaching

REFERENCES

Adelman H S (1993) School-linked mental health interventions Toward mechanismsfor service coordination and integration Journal of Community Psychology 21 309ndash319

Adelman H S (1996a) Restructuring education support services and integrating com-munity resources Beyond the full service school model School Psychology Review25 431ndash445

Adelman H S (1996b) Restructuring support services Toward a comprehensiveapproach Kent OH American School Health Association

Adelman H S amp Taylor L (1993a) Early Assistance for Students and FamiliesProgram Guidebook Los Angeles Early Assistance for Students and FamiliesProject Copies available from the School Mental Health Project Dept of Psychol-ogy UCLA

Adelman H S amp Taylor L (1993b) Learning problems and learning disabilities Moving forward Pacic Grove CA BrooksCole

Adelman H S amp Taylor L (1994) On understanding intervention in psychology andeducation Westport CT Praeger

Adelman H S amp Taylor L (1997a) Addressing barriers to learning Beyond school-linked services and full service schools American Journal of Orthopsychiatry 67408ndash421

Adelman H S amp Taylor L (1997b) Toward a scale-up model for replicating newapproaches to schooling Journal of Educational and Psychological Consultation 8197ndash230

Scaling up Reforms 321

With specic respect to scale-up the rst accomplishment is thereplication itself Have all facets been implemented How completelyhas each been implemented At how many locations The next set ofresults are any indications of progress for students such as improve-ments in attitudes toward school health attendance behavior andacademic achievement A nal set of evaluation concerns is thedegree to which student outcomes approximate societal standards

Lessons Learned

The process of evaluating results is costly in terms of nancial invest-ment the negative psychological impact on those evaluated and theways it can inappropriately reshape new approaches Cost-e ectiveoutcomes cannot be achieved in the absence of e ective prototypedevelopment and research Premature e orts to carry out com-prehensive summative evaluations clearly are not cost-e ective Anyreading and writing program will show poor results if it is evaluatedbefore teachers have mastered its application None of this of courseis an argument against evaluating results Rather it is meant tounderscore concerns and encourage greater attention to addressingthem

Once a prototype is established care must be taken to avoiddeveloping outcome evaluation as an adversarial process Due to thepolitical realities related to accountability one of the most per-plexing facets to negotiate is the time frame for summative evalu-ation The more complex the prototype the longer it takes and thecostlier it is to implement and evaluate Schools usually want quickprocesses and results and of course rarely can a ord costly innova-tions or lengthy di usion activity Compromises are inevitable butmust be arrived at with great care so as not to undermine the sub-stance of proposed changes

The psychology of evaluation suggests that an overemphasis onaccountability tends to produce negative reactions One possible wayto counter this may be to conceive evaluation as a way for everystakeholder to self-evaluate as a basis for quality improvement and asa way of getting credit for all that is accomplished Unfortunately asaccountability pressures increase we nd that replication of proto-types are guided more by what can be measured than by long-rangeaims That is demands for immediate accountability reshape prac-tices so that the emphasis shifts to immediate and readily measuredobjectives and away from fundamental purposes Over time this inap-propriately leads to radical revision of the underlying rationale for aprototype

322 L Taylor et al

CONCLUDING COMMENTS

Those who set out to change schools and schooling are confrontedwith two enormous tasks The rst is to develop prototypes Thesecond involves large-scale replication One without the other isinsufficient Yet considerably more attention is paid to developingand validating prototypes than to delineating and testing scale-upprocesses Clearly it is time to correct this deciency The ideas pre-sented in this article are meant to stimulate work on the problem andthereby to advance the cause of educational reform

Finally in fairness to those who labor for educational reform weall must remember that the quality of schooling family life and com-munity functioning spirals up or down as a function of the quality ofthe ongoing transactions among each Thus scale-up e orts relatedto educational reform must take place within the context of a poli-tical agenda that addresses ways to strengthen the family and com-munity infrastructure through strategies that enhance economicopportunity adult literacy and so forth What we need are policiesthat develop demonstrate and scale-up comprehensive multifacetedintegrated approaches that can e ectively address barriers to devel-opment learning and teaching

REFERENCES

Adelman H S (1993) School-linked mental health interventions Toward mechanismsfor service coordination and integration Journal of Community Psychology 21 309ndash319

Adelman H S (1996a) Restructuring education support services and integrating com-munity resources Beyond the full service school model School Psychology Review25 431ndash445

Adelman H S (1996b) Restructuring support services Toward a comprehensiveapproach Kent OH American School Health Association

Adelman H S amp Taylor L (1993a) Early Assistance for Students and FamiliesProgram Guidebook Los Angeles Early Assistance for Students and FamiliesProject Copies available from the School Mental Health Project Dept of Psychol-ogy UCLA

Adelman H S amp Taylor L (1993b) Learning problems and learning disabilities Moving forward Pacic Grove CA BrooksCole

Adelman H S amp Taylor L (1994) On understanding intervention in psychology andeducation Westport CT Praeger

Adelman H S amp Taylor L (1997a) Addressing barriers to learning Beyond school-linked services and full service schools American Journal of Orthopsychiatry 67408ndash421

Adelman H S amp Taylor L (1997b) Toward a scale-up model for replicating newapproaches to schooling Journal of Educational and Psychological Consultation 8197ndash230

322 L Taylor et al

CONCLUDING COMMENTS

Those who set out to change schools and schooling are confrontedwith two enormous tasks The rst is to develop prototypes Thesecond involves large-scale replication One without the other isinsufficient Yet considerably more attention is paid to developingand validating prototypes than to delineating and testing scale-upprocesses Clearly it is time to correct this deciency The ideas pre-sented in this article are meant to stimulate work on the problem andthereby to advance the cause of educational reform

Finally in fairness to those who labor for educational reform weall must remember that the quality of schooling family life and com-munity functioning spirals up or down as a function of the quality ofthe ongoing transactions among each Thus scale-up e orts relatedto educational reform must take place within the context of a poli-tical agenda that addresses ways to strengthen the family and com-munity infrastructure through strategies that enhance economicopportunity adult literacy and so forth What we need are policiesthat develop demonstrate and scale-up comprehensive multifacetedintegrated approaches that can e ectively address barriers to devel-opment learning and teaching

REFERENCES

Adelman H S (1993) School-linked mental health interventions Toward mechanismsfor service coordination and integration Journal of Community Psychology 21 309ndash319

Adelman H S (1996a) Restructuring education support services and integrating com-munity resources Beyond the full service school model School Psychology Review25 431ndash445

Adelman H S (1996b) Restructuring support services Toward a comprehensiveapproach Kent OH American School Health Association

Adelman H S amp Taylor L (1993a) Early Assistance for Students and FamiliesProgram Guidebook Los Angeles Early Assistance for Students and FamiliesProject Copies available from the School Mental Health Project Dept of Psychol-ogy UCLA

Adelman H S amp Taylor L (1993b) Learning problems and learning disabilities Moving forward Pacic Grove CA BrooksCole

Adelman H S amp Taylor L (1994) On understanding intervention in psychology andeducation Westport CT Praeger

Adelman H S amp Taylor L (1997a) Addressing barriers to learning Beyond school-linked services and full service schools American Journal of Orthopsychiatry 67408ndash421

Adelman H S amp Taylor L (1997b) Toward a scale-up model for replicating newapproaches to schooling Journal of Educational and Psychological Consultation 8197ndash230

Scaling up Reforms 323

Adelman H S amp Taylor L (1998) Involving teachers in collaborative e orts to betteraddress the barriers to student learning Preventing School Failure 42 55ndash60

Albee G W amp Gullotta T P (Eds) (1997) Primary prevention works ThousandOaks CA Sage

Argyris C (1993) Knowledge for action A guide to overcoming barriers to organiz-ational change San Francisco Jossey-Bass

Barth R S (1990) Improving schools from within Teachers parents and principles canmake a di erence San Francisco Jossey-Bass

Bass B M (1997) Transformational leadership Industrial military and educationalimpact Mahwah NJ Erlbaum

Bass B M amp Avolio B J (Eds) (1994) Improving organizational e ectivenessthrough transformational leadership Newbury Park CA Sage

Bond L A amp Compas B E (Eds) (1989) Primary prevention and promotion in theschools Newbury Park CA Sage

Center for Mental Health in Schools (1998) Restructuring boards of education toenhance schoolsrsquo e ectiveness in addressing barriers to student learning Los AngelesCA Author

Connor P E amp Lake L K (1988) Managing organization change New York Praeger

Cunningham W G amp Gresso D W (1993) Cultural leadership The culture of excel-lence in education Boston Allyn and Bacon

Deci E L amp Ryan R M (1985) Intrinsic motivation and self-determination in humanbehavior New York Plenum Press

Donahoe T (1993) Finding the way Structure time and culture in school improve-ment Phi Delta Kappan December 75 298ndash305

Dryfoos J G (1990) Adolescents at risk Prevalence and prevention New York OxfordUniversity Press

Dudley-Marling C amp Murphy S (1997) A political critique of remedial reading pro-grams The example of reading recovery Reading Teacher 50 460ndash468

Durlak J A (1995) School-based prevention programs for children and adolescentsThousand Oaks CA Sage

Elias M J (1997) Reinterpreting dissemination of prevention programs as widespreadimplementation with e ectiveness and delity In R P Weissberg T P Gullotta RL Hampton B A Ryan amp G R Adams (Eds) Establishing preventive services pp253ndash289 Thousand Oaks CA Sage

Elmore R F amp Associates (1990) Restructuring schools The next generation of educa-tional reform San Francisco Jossey-Bass

Fullan M G amp Stiegelbauer S (1991) The new meaning of educational changes (2nded) New York Teachers College Press

General Accounting Office (1989) Prospective evaluation methods The prospectiveevaluation synthesis GAOPEMD-89-10 Washington DC Author

Hatch T (1998) The di erences in theory that matter in the practice of schoolimprovement American Educational Research Journal 35 3ndash32

Heller K (1990) Social and community intervention A nnual Review of Psychology 41141ndash168

Hollander E P amp O ermann L R (1990) Power and leadership in organizationsRelationships in transition American Psychologist 45 179ndash189

House E R (1996) A framework for appraising educational reforms EducationalResearcher 25 6ndash14

Lewis A C (1989) Restructuring Americarsquos schools Arlington VA American Associ-ation of School Administrators

324 L Taylor et al

Lieberman A amp Miller L (1990) Restructuring schools What matters and whatworks Phi Delta Kappan 71 759ndash764

Lyon G R amp Moats L C (1997) Critical concepts and methodological considerationsin reading intervention research Journal of Learning Disabilities 30 578ndash588

Maton K I amp Salem D A (1995) Organizational characteristics of empowering com-munity settings A multiple case study approach American Journal of CommunityPsychology 23 631ndash656

Miles M B amp Louis K S (1990) Mustering the will and skill for change The nd-ings from a four-year study of high schools that are experiencing real improvemento er insights into successful change Educational Leadership 47 57ndash61

Mitchell A Seligson M amp Marx F (1989) Early childhood programs and the publicschools Promise and practice Dover MA Auburn House

Murphy J (1991) Restructuring schools Capturing and assessing the phenomena NewYork Teachers College Press

National Education Commission on Time and Learning (1994) Prisoners of time Wash-ington DC US Government Printing Office

Newmann F M (1993) Beyond common sense in educational restructuring The issuesof content and linkage Educational Reviewer 22 4ndash13 22

Peterson P L McCarthey S J amp Elmore R F (1996) Learning from schoolrestructuring A merican Educational Research Journal 33 119ndash153

Posavac E J amp Carey R G (1989) Program evaluation Methods and case studies(3rd ed) Englewood Cli s NJ Prentice Hall

Replication and Program Services Inc (1993) Building from strength Replication as astrategy for expanding social programs that work Philadelphia Author

Rossi P H amp Freeman H E (1989) Evaluation A systematic approach (4th ed)Newbury Park CA Sage

Sarason S B (1990) The predictable failure of educational reform Can we changecourse before its too late San Francisco Jossey-Bass

Sarason S B (1996) Revisiting lsquolsquoThe culture of school and the problem of changersquorsquo NewYork Teachers College Press

Schlechty P C (1990) Schools for the twenty-rst century Leadership imperatives foreducational reform San Francisco Jossey-Bass

Schorr L B (1988) Within our reach Breaking the cycle of disadvantage New York Doubleday

Schorr L B (1997) Common purpose Strengthening families and neighborhoods torebuild America New York Anchor Press

Schmuck R A amp Runkel P J (1985) The handbook of organizational development inschools (3rd ed) Palo Alto CA Mayeld Publishing

Scriven M (1993) Hard-won lessons in program evaluation San Francisco Jossey-Bass

Sechrest L amp Figueredo A J (1993) Program evaluation Annual Review of Psychol-ogy 44 645ndash674

Shadish Jr W R Cook T D amp Leviton L C (1991) Foundations of program evalu-ation Theories of practice Newbury Park CA Sage

Slavin R E (1996) Reforming state and federal policies to support adoption of provenpractices Educational Researcher 25 4ndash5

Slavin R Karweit B J amp Madden N (Eds) (1989) E ective programs for students atrisk Boston Allyn amp Bacon

Smith M S amp OrsquoDay J (1991) Systemic school reform In S H Fuhrman amp B Malen(Eds) The politics of curriculum and testing The 1990 Yearbook of the Politics ofEducation Association (pp 233ndash267) Philadelphia Falmer

Scaling up Reforms 325

Spillane J P (1998) State policy and the non-monolithic nature of the local schooldistrict Organizational and professional considerations American EducationalResearch Journal 35 33ndash63

Stake R E (1967) The countenance of educational evaluation Teachers CollegeRecord 68 523ndash540

Stake R E (1976) Evaluating educational programs The need and the response ParisOrganization for Economic Cooperation and Development

Stringeld S Ross S amp Smith L (Eds) (1996) Bold plans for school restructuring Mahwah NJ Erlbaum

Stufflebeam D L amp Webster W J (1983) An analysis of alternative approaches toevaluation In G F Madaus M S Scriven amp D L Stufflebeam (Eds) Evaluationmodels Boston Kluwer-Nijho

Sue S amp Zane N (1987) The role of culture and cultural techniques A critique andreformulation American Psychologist 42 37ndash45

Tyack D amp Cuban L (1995) Tinkering toward Utopia A century of public schoolreform Cambridge MA Harvard Educational Press

Urban Learning Center Model (1995) A design for a new learning community LosAngeles Los Angeles Educational Partnership

Waterman R H (1987) The renewal factor New York Bantam BooksWehlage G Smith G amp Lipman P (1992) Restructuring urban schools The New

Futures experience American Educational Research Journal 29 51ndash93Weiss C H (1995) Nothing as practical as a good theory Exploring theory-based

evaluation for comprehensive community initiatives for children and families In JB Connell A C Kubisch L Schorr amp C H Weiss (Eds) New approaches to evalu-ating community initiatives Concepts methods and concepts Washington DCAspen Institute

Weissberg R P Gullotta T P Hamptom R L Ryan B A amp Adams G R (Eds)(1997) Establishing preventive services Thousand Oaks CA Sage