thesis

INFORMATION TO USERS

This manuscript has been reproduced from the microfilm master. UMI

films the text directly from the original or copy submitted. Thus, some

thesis and dissertation copies are in typewriter face, while others may be

from any type of computer printer.

The quality of this reproduction is dependent upon the quality of the

copy submitted. Broken or indistinct print, colored or poor quality

illustrations and photographs, print bleedthrough, substandard margins,

and improper alignment can adversely affect reproduction.

In the unlikely event that the author did not send UMI a complete

manuscript and there are missing pages, these will be noted. Also, if

unauthorized copyright material had to be removed, a note will indicate

the deletion.

Oversize materials (e.g., maps, drawings, charts) are reproduced by

sectioning the original, beginning at the upper left-hand comer and

continuing from left to right in equal sections with small overlaps. Each

original is also photographed in one exposure and is included in reduced

form at the back of the book.

Photographs included in the original manuscript have been reproduced

xerographically in this copy. Higher quality 6” x 9” black and white

photographic prints are available for any photographs or illustrations

appearing in this copy for an additional charge. Contact UMI directly to

order.

UMIA Bell & Howell Information Company

300 North Zeeb Road, Ann Arbor MI 48106-1346 USA 313/761-4700 800/521-0600

R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.

R e p ro d u c e d with perm iss ion of th e copyright ow ner. F u r th e r reproduction prohibited without perm iss ion .

NOTE TO USERS

The original manuscript received by UMI contains indistinct, slanted and or light print. All efforts were made to acquire the highest quality manuscript from the author or school.

Microfilmed as received.

This reproduction is the best copy available

UMI


R e p ro d u c e d with perm iss ion of th e copyright ow ner. F u r th e r reproduction prohibited without perm iss ion .

Using the Malcolm Baldrige National Quality Award Education Pilot Criteria for Self-

Assessment o f School Districts

Presented in Partial Fulfillment o f the Requirements for the

Degree o f Doctor o f Philosophy

with a

Major in Education

in the

College o f Graduate Studies

University o f Idaho

By

Sally Anderson

December, 1997

Major Professor: Dr. Cleve Taylor


UMI Number: 9827869

Copyright 1998 by Anderson, Sally C.

AH rights reserved.

UMI Microform 9827869 Copyright 1998, by UMI Company. AH rights reserved.

This microform edition is protected against unauthorized copying under Title 17, United States Code.

UMI300 North Zeeb Road Ann Arbor, MI 48103


Authorization to Submit Dissertation

This dissertation o f Sally Anderson, submitted for the degree o f Doctor o f Education with a

major in Education and titled, “Using the Malcom Baldrige National Quality Award

Education Pilot Criteria for Self-Assessment o f School Districts” has been reviewed in final

form, as indicated by the signatures and dates given below. Permission is now granted to

submit final copies to the College o f Graduate Studies for approval.

Major Professor Date*:

■1Date:Committee Members

Dr. Michael Tomlin

r Penny SchweibertDate: #

Date."2

DepartmentAdministrator

Dean, College o f Education

Dr. Roger Reyno bison

/ — Date: V 9 $Dr. .ferry T/lbhscherer

DateDr. Dale Gentry

Final Approval and Acceptance by the College o f Graduate Studies

V I A - — Date: _ _ 5 V / 3 / ^ /Jeanme M. Shreeve


Abstract

The demand for improvements in education continues. Failed reform attempts,

educational fads, and poor planning designs have been cited as variables affecting the

approach to improvements in public schools. This study examines the literature o f failed

reforms, current approaches to determine the performance o f schools, and models o f

improvement based on quality theory and practices.

This study investigated the perceptions o f three types o f educators— superintendents,

principals, and teachers— regarding the performance o f their school district in seven

categories o f organizational performance. The size o f the district based on student

enrollment was used as the second independent variable to determine if there were any

significant differences in perceptions based on size o f district. An instrument was developed

using the criteria in the Malcolm Baldrige National Quality Award, 1997 version; the

Education Pilot criteria; curriculum audit standards; and accreditation standards. The study

used a proportional stratified random sampling procedure by size o f district and type of

educator. The findings were analyzed using a two-way analysis o f variance for each o f the

seven categories.

The study found the reliability o f the instrument to be a low o f .74 for School District

Results to a high o f .85 for Leadership. Significant differences in the perceptions o f

performance o f the school districts in each o f the seven categories were found to exist

between superintendents and teachers, as well as principals and teachers. No significant

differences were found between superintendents and principals or in any category by the size

o f district. The study discusses the implications o f the findings for a framework for school

improvement.


Acknowledgements

The ideas, design, and completion o f this project were the result o f many dialogues

and the collective knowledge o f many. I wish to express my sincere appreciation for the

support and guidance o f my advisor, Dr. Cleve Taylor, and the mentoring o f my committee,

Dr. Roger Reynoldson, Dr. Penny Schweibert, and Dr. Mike Tomlin.

I wish to thank Dr. Mike Friend o f the Idaho School Administrators Association and

Jim Shackleford o f the Idaho Teachers Association for their contributions o f resources and

support for this study. My sincere appreciation goes to Dr. Carolyn Keeler, Dr. Del Siegle,

and Dr. Bill Parrett for their technical expertise and recommendations. I also wish to thank

Eleanor Fisk for her assistance with the laborious task o f scanning the returned instruments

and Alice Gould, Stephanie Fox , Dawn Davis, and Chris Latter for their assistance in the

details and preparation o f this document and the defense.

My most sincere appreciation is to my husband, Mike, and our boys, A. J. and Jon,

for countless sacrifices they made so that my goals could be accomplished.


Dedication

This effort is dedicated to the three men who have taught me the most important

lessons in my life. To the memory o f my father who gave me the thirst for new knowledge

and the potential to seek it; to my husband, whose love is the greatest gift o f my life and

whose commitment, support, and patience are true models for all; and to my son, A. J., who

inspires me to grow and who will always be a continual source o f pride and enlightenment.


vi

Table o f Contents

Page

Authorization to Submit Dissertation...................................................................................................... ii

Abstract.........................................................................................................................................................iii

Acknowledgements.................................................................................................................................... iv

Dedication.......................................................................................................................................................v

List o f Tables.............................................................................................................................................viii

List o f Figures.............................................................................................................................................. ix

Chapter 1: Introduction............................................................................................................................. 1

Background o f the Problem....................................................................................................... 1

The Effectiveness o f School Reform .......................................................................... 2

Statement o f the Problem..............................................................................................................5

Significance o f the Problem.........................................................................................................5

Traditional Methods for Determining School Performance............................................... 6

Quality Models for Organizational Effectiveness..................................................................7

Research Questions........................................................................................................................ 8

Hypotheses.......................................................................................................................................9

Limitations...................................................................................................................................... 9

Delimitations................................................................................................................................. 10

Definitions...................................................................................................................................... 10

Summary.........................................................................................................................................13

Chapter 2: Literature Review...................................................................................................................14

Introduction....................................................................................................................................14


Education as a System ................................................................................................................ 14

Organizational E ffectiveness....................................................................................................17

Determining Effectiveness in School Systems.......................................................................21

Quality Theory................................................................................................................34

Continuous Improvement in B usiness..................................................................... 40

Summary.........................................................................................................................................59

Chapter 3: M ethodology...........................................................................................................................60

Introduction....................................................................................................................................60

The Research M odel....................................................................................................................60

Instrumentation............................................................................................................................. 61

Subjects and Settings.................................................................................................................. 63

Collection o f Data........................................................................................................................ 64

Data .Analysis................................................................................................................................65

Summary.........................................................................................................................................65

Chapter 4: Findings....................................................................................................................................6 6

Introduction....................................................................................................................................6 6

Rate o f Return...............................................................................................................................67

Characteristics o f Sam ple......................................................................................................... 6 8

Reliability o f Performance Analysis for School D istricts................................................ 71

Descriptive A nalysis................................................................................................................... 72

Inferential Statistical A nalysis............................................................................................... 115

Analysis o f “Do Not Know” Responses.............................................................................. 121

Usefulness o f the Instrument as a Tool.................................................................................122


viii

Summary......................................................................................................................................124

Chapter 5: Summary, Conclusions, and Recommendations.........................................................125

Summary......................................................................................................................................125

Conclusions.................................................................................................................................126

Recommendations.....................................................................................................................128

References................................................................................................................................................ 131

Appendix A: Instrument........................................................................................................................ 140

Appendix B: Panel o f Experts Used in Content Validation.........................................................175

Appendix C: Letter to Panel o f Experts.............................................................................................176

Appendix D: Matrix o f Population Sample.......................................................................................177

Appendix E: Codes on the Instrument............................................................................................... 178

Appendix F: Cover Letter and Directions.........................................................................................179

Appendix G: Postcard Reminder.........................................................................................................181

Appendix H: Letters o f Support..........................................................................................................182

Appendix I: Districts by Enrollment S iz e .........................................................................................184


ix

List o f Tables

Page

Table 1. Factors o f Organizational Effectiveness............................................................................22

Table 2. Meta-analysis Findings o f School System Evaluation Components a

Reported in the Literature.................................................................................................... 23

Table 3. Curriculum Audit Findings o f 67 School Districts

Between 1988 and 199 4 ...................................................................................................... 29

Table 4. A Comparison Betweeen Teaching Theories o f Quality Experts...............................40

Table 5. Baldrige National Quality Award Criteria 1997............................................................ 47

Table 6 . Validity o f the MBNQA M odel...........................................................................................48

Table 7. Accuracy o f the MBNQA W eights................................................................................... 49

Table 8 . Core Values/Concepts o f MBNQA Education Pilot 1995............................................. 54

Table 9. 1995 MBNQA Educational Pilot Criteria......................................................................... 58

Table 10. Stratified Random Sample Matrix....................................................................................64

Table 11. Total Return Rates by Educator Position........................................................................67

Table 12. Frequencies and Percentages o f Returns Received by Educator Position

And District S ize.................................................................................................................... 6 8

Table 13. Percentage o f Highest Degree and Time in Position by Size and Position...........70

Table 14. Rank Order o f Combined Sam ple....................................................................................69

Table 15. Reliability o f Instrument..................................................................................................... 71

Table 16. Means by District Size and Positions Combined......................................................... 72

Table 17. Means by District Size for Districts With More Than 5,000 Students Enrolled.. 73

Table 18. Means by District Size for Districts With 4, 999 to 2,500 Students Enrolled......73


Table 19. Means by District Size for Districts With 2, 499 to 1, 000 Students Enrolled 74

Table 20. Means by District Size for Districts With 999 to 500 Students Enrolled...............74

Table 21. Means by District Size for Districts With Less Than 499 Students Enrolled....... 75

Table 22. Means by Educator Position for Superintendents.........................................................75

Table 23. Means by Education Position for Principals.................................................................. 76

Table 24. Means by Educator Position for Teachers......................................................................76

Table 25. Item Frequency and Percentage o f Response by Position and Size o f District

For Items in the Leadership Category............................................................................... 80


For Items in the Strategic Planning Category..................................................................87


For Items in the Student Focus and Satisfaction/Stakeholder Categories............... 90


For Items in the Information and Analysis Category.................................................... 94


For Items in the Human Resource Development Category......................................... 97


For Items in the Educational Process Management Category.................................. 103


For Items in the School Districts Results Category.....................................................109

Table 32. Two-Way ANOVA Leadership Construct....................................................................116

Table 33. Two-Way ANOVA Strategic Planning Construct...................................................... 117


xi

Table 34. Two-Way ANOVA Student/Stakeholder Construct..................................................118

Table 35. Two-Way ANOVA Information and Analysis Construct........................................119

Table 36. Two-Way ANOVA Human Resource/Management Construct.............................. 119

Table 37. Two-Way ANOVA Educational Process/Operational Management

Construct..................................................................................................................................120

Table 38. Two-Way ANOVA School District Results Construct.............................................121

Table 39. Chi Square for “Do Not Know” Responses..................................................................122

Table 40. Combined Percentage for Usefulness o f Instrumentation........................................ 123


xii

List o f Figures

Page

Figure 1. An Educational System as an Open System.....................................................................16

Figure 2. A Quality Systems Model for Performance Improvement....................................... 129


1

Chapter 1

Introduction

Background o f the Problem

The performance o f schools is currently determined by a multitude o f indicators

based on political, traditional, and institutional influences. Public opinion for the

performance o f the complete system is often based on one or more o f these indicators

(Bracey, 1997; Bushweller, 1996; Elam, Lowell & Gallup, 1996). Although improvements

are occurring in many o f the nation’s schools, results are still anecdotal, isolated, and far

from replicable (Fullan, 1993). Public criticism still abounds and the perception o f inferior

quality and poor performance remains (Bushweller, 1996; Hodgkinson, 1996; Houston,

1996; Huelskamp, 1993). The demands for greater accountability for publicly funded

institutions have not diminished. The lack o f evidence o f improved performance, effective

planning, and the increase spending o f public funds without discernible measures o f tangible

results have led to the demand for more business-like strategies (DeMont, 1973; Gerstner,

1995; Kearns & Doyle, 1988).

School improvement and how to achieve it continues to inspire public, political, and

professional dialogue and debate. The approach to improving public schools is as varied as

the prophets and their doctrines. Little to no sustainable improvements, public hostility, and

disenfranchised teachers are left in the wake o f such well-intentioned efforts (English & Hill,

1994). When teachers from the high performing Willamette Primary School in Oregon were

asked why they thought so many schools were failing, they blamed the pursuit o f “it” (Sagor,

1995). Solving the problems in education with a one-solution approach perpetuates the

notion that “it” will remedy the problem and things will be better once we find “it.” These


2

types o f solutions are often at a visible, obvious level denying the complexity o f other

interdependent relationships and root causes (Bernhardt, 1994; Deming, 1994; Scholtes,

1995). Non-systemic interventions to improve organizations merely shift problems from one

part o f the organization to another (Senge, 1990).

The Effectiveness o f School Reform

Upon review o f current literature, factors affecting the efficacy o f reform strategies

appear to fall into three categories: (a) selection o f reform initiatives, (b) implementation o f

reform initiatives, and (c) the improvement or change strategy selected.

The 1960s saw a multitude o f reform initiatives influenced significantly by a national

concern that American education was falling behind foreign accomplishments and the civil

rights movement (Fullan, 1993). Solutions were often superficial, quick-fix remedies made

impatiently as a result o f various pressures facing the decision makers or educational fads

(Fields, 1994; Fullan, 1992). The result was often— and still continues to be— an abundance

o f disjointed, incomplete improvement initiatives (Fullan, 1997). The presumption that

developing innovations on a national scale would lead to widespread adoption appeared to be

flawed (Fullan, 1993).

Flawed implementation is another source o f much discussion in the literature o f

educational reform. Berman and McLaughlin (1977) did a comprehensive study o f programs

that were federally funded. They found many examples o f failed implementation which

included failure to take into account local nuances and capacity, desire for additional funds

for political reasons rather than educational reasons, and the presumption that innovations are

implemented one at a time contrary to the reality o f schools. Another perspective offered by

Fullan and Miles (1992) is the misunderstanding o f resistance. They argue that issues o f


3

effective implementation and communication strategy are often at the heart o f the reason why

reform fails rather than personal attitudes and resistance (Fullan et al, 1992).

A third reason for failed reform reflected in the literature is related to the absence o f a

specific change strategy. It has been found that often each stakeholder o f education has a

different, and usually faulty, belief about how change occurs (Fullan, 1993; Fullan et al.

1992; Hargreaves, 1997). This results in confusion and conflict during both design and

implementation phases. Denial o f the complexity o f problems and solutions is often

observed. Critics o f past reform efforts advocate for three things: (a) greater recognition o f

the complexity o f the educational system; (b) deeper second order changes in the

organization; and (c) the need to be created, designed, and implemented by those

knowledgeable o f the institution (Fields, 1994; Hargreaves, 1997; O ’Neil, 1995; Sarason,

1990; Wagner, 1993). Reform efforts, particularly those driven through mandated practices

contingent on state or federal dollars, often result in symbols o f improvement over substance

(Berman, 1977; Wagner, 1993).

During the past decade, most people involved in the reform o f education have come

to advocate a systemic perspective (Fullan, 1992; Timpane & Reich, 1997). The resurgence

o f interest in systems theory applications is currently resulting in a heightened attention to

and recognition o f the complexity o f organizations, particularly educational institutions. A

central principle underlying systems thinking is that structure influences behavior (Deming,

1986; Patterson, Purkey & Parker, 1986; Senge, 1990). The structure, therefore, used to

initiate, conduct, and evaluate an improvement process is related to the potential

effectiveness o f each specific solution deployed.


4

Sarason (1990) proposes two basic premises to influence the design and

implementation o f solutions in school reform efforts. The first is the presence o f a

conceptual framework that recognizes the relatedness o f human behavior. The second is a

thorough understanding o f the context o f that improvement. Reformers to date have been

criticized for not having an implicit theory about how to achieve change, and they do not

always recognize the influence o f the intractability o f the system (Fullan & Miles, 1992;

Sarason, 1990). Ignoring these two factors can result in an approach that seeks the cure and

ignores the diagnoses. Focusing on doing the right thing, over doing in the right way, can

result in using the means as the end (Bennis, 1976).

In K-12 educational systems, the development o f school improvement plans often

becomes a substitute for results (Sergiovanni, 1992). A focus on the outcomes or results o f

education have rarely been operationalized (Schmoker, 1996). Educators often resist

confronting the results and using them to make decisions for school improvement (Bernhardt,

1994; Schmoker, 1996). Schools are traditionally limited in their use o f information and

have little need to depend on systematic feedback from a variety o f their customers

(Bernhardt, 1994; Consortium on Productivity in Schools, 1995; Schmoker. 1996). The

capacity o f data and information to reveal strengths, weaknesses, successes, and failures are

threatening to educators particularly in a political context (Schmoker, 1996). Schmoker

(1996) further states that schools are too poorly organized to see the connection between

effort and outcomes.

The theoretical base upon which improvements are determined and made in the total

organization, or any part o f the total organization, is critically important in demonstrating

outcomes (Deming, 1991). The framework which follows from the theory results in the


5

models, methods, and tools (Skrtic, 1991). This study investigates a theory and the

development and use o f a framework for a process o f organizational analysis o f its

performance.

Statement o f the Problem

The process o f improving school performance often lacks an organized strategy,

processes for decision-making, deployment o f those decisions, and a mechanism for

evaluating results o f those strategies (Bernhardt, 1994; Fullan & M iles, 1992). The manner

in which schools think about the school improvement process determines their ability to

deploy it successfully (Bernhardt, 1994). The current methods o f determining organizational

performance in schools, identifying the areas o f improvement, and implementing these

changes lack a conceptual framework which recognizes the relatedness o f human behavior

(Sarason, 1990). The accreditation process, once intended to be a mechanism for self-study,

has become a political formality which focuses on the surface indicators with no mechanism

for deeper improvements leading to results (Portner, 1997). Education is lacking a useful

comprehensive framework for systemic analysis o f its performance and its approach to

improvement.

Significance o f the Problem

Goodlad (1984) remarked that in order to survive, an institution must have the faith o f

its clients in its usefulness and a measure o f satisfaction o f its performance. More than 10

years later, public education continues to be challenged by the many constituencies who have

similar criticisms (Bushweller, 1996; Hodgkinson, 1996; Houston, 1996; Huelskamp, 1993;

Gerstner, 1995; Kearns et al, 1988). The use o f measures o f satisfaction from the customers

o f education is limited. Subsequent approaches to improving performance that have the


6

potential o f increasing satisfaction are diverse, scattered, and often politically motivated

(Consortium on Productivity in the Schools, 1995). This research explores the existing

approaches used to determine the performance o f a school district. The arguments germane

to this research have been organized into three categories: (a) the investigation o f the

traditional methods for currently determining the performance o f a school district, (b) the

concept o f organizational effectiveness, and (c) the potential use o f models emerging from

quality theory to analyze organizational effectiveness in school districts.

Traditional Methods For Determining School Performance

Determining the performance o f education is an undertaking worthy o f

in-depth analysis o f its own. It has not been established in the literature that the measures

currently used and analyzed are the appropriate indicators o f the performance o f the

educational system (Huelskamp, 1993). Traditional methods o f assessing the successes and

failures o f public education include, most typically, multiple constituency models. These

models are designed to meet standards or criteria set by various stakeholders for various

purposes (Brassard, 1993).

Traditional models include financial, management, and curricular audit procedures;

program evaluation studies; federal or state compliance reviews; and specific indicators o f

student performance. Accreditation is currently the most comprehensive practice which

purports to determine the performance o f a school (Portner, 1997). The accreditation

process, once a status symbol for schools, now is viewed as a routine examine with little

relevance to school improvement (Portner, 1997). It does not, however, look

comprehensively at the entire school district since schools are accredited as singular units.

The performance o f schools, and, therefore, school systems, is often inferred by the general


7

public based on the performance o f student scores on annual measures as reported in the

media (Rotberg, 1996). Specific units or functions o f the school district are often reviewed,

as required by state and federal laws, such as financial audits, regulations o f Title I, or the

Individuals with Disabilities Act. The focus o f these processes is to determine compliance

with regulations and the need for any corrective action. Non-compliance in some cases can

mean financial penalties to the district. Curriculum audits offer a thorough process for

assessing the organization, delivery, support, and results o f the instructional process.

Specific standards have been developed and criteria are used to determine the degree o f

effectiveness. Professionally trained auditors, external to the district, conduct the process

and prepare a final report. Peer reviews, such as management audits, also occur. They are

often designed by administrators to analyze specific parameters o f management.

Quality Models For Organizational Effectiveness

According to Field (1994), the National Education Association and the American

Association o f School Administrators suggest that few innovations or educational changes

stimulated from outside o f education will occur without educator commitment. Management

is responsible for the design and approach to improving the performance o f the system

(Crosby, 1984). The people who understand the processes and their outcomes well enough

are school education leaders within school organizations (Fields, 1994; Sarason, 1990). In

the absence o f a foundational theory upon which to base practices and an organized approach

to accomplish the improvements, the educational leader is vulnerable to public criticism and

negligent o f their duties. There is also a need to bring the practitioner into the creation and

design o f the practice (Deming, 1993, Glasser, 1992; Imai, 1986). There are ever-increasing

examples o f classroom teachers who are feeling helpless against a barrage o f public criticism


8

and increasing, uncoordinated demands o f federal, state, and local officials (Bracey, 1997;

Fullan, 1997; Hargreaves, 1997).

Quality theory, practices, and tools are being used increasingly by educational and

service organizations. Educators are applying quality principles by defining the needs and

perceptions o f internal and external customers, using information to make decisions, and

designing results-oriented strategies for systemic improvement involving people in all parts

o f the organization. The application o f the Malcolm Baldrige National Quality Award,

originally designed for business, has been extended to educational institutions and offers a

framework for analysis and recognition (National Institute o f Standards and Technology,

1995).

Research Questions

The research questions posed in this study are:

1. How do educators perceive their own school district’s performance based on an

instrument designed using the Malcolm Baldrige National Quality Award Education Criteria?

2. Are there differences in these ratings based on type o f educator or size o f district?

3. Do educators find this instrument a useful tool to study these areas o f a school

district?

4. Do educators believe this instrument could be useful in determining school

improvement needs?

Hypotheses

The study will test the following null hypotheses:

Ho i: There are no significant differences in the Leadership category o f the

Performance Analysis for School Districts by type or size.


9

Hen: There are no significant differences in the Strategic Planning category o f the

Performance Analysis for School Districts by type or size.

H0 3 : There are no significant differences in the Student Focus and Student and

Stakeholder Satisfaction category o f the Performance Analysis for School Districts by type or

size.

H04 : There are no significant differences in the Information and Analysis category o f

the Performance Analysis for School Districts by type or size.

H0 5 : There are no significant differences in the Human Resource Development and

Management category o f the Performance Analysis for School Districts by type or size.

H o 6: There are no significant differences in the Educational and Operational Process

Management category o f the Performance Analysis for School Districts by type or size.

H07 : There are no significant differences in the School D istrict Results category o f

the Performance Analysis for School Districts by type or size.

Limitations

The study is subject to the following limitations:

1. The study presumes a truthful response and that respondents will understand

items.

2. Responses to items are subject to personal biases, motivations, perspectives, and

experience o f the respondents.

3. Responses are presumed to be independently made.

4. Respondents’ prior knowledge o f theoretical constructs behind the instrument is

unknown.


10

5. The design o f the study is not experimental; therefore no causal relationship can

be inferred.

6 . Responses will be collected through a mailed survey that decreases the probability

of 1 0 0 % participation.

7. The study presumes that meaningful analyses can be made on less than 100% o f

returned responses.

Delimitations

1. The study is limited to superintendents, principals, and classroom teachers within

Idaho, which affects generalizability o f the findings to other educators outside o f Idaho.

2. The entire population will not be used. A proportional stratified random sample

will be drawn from the population o f interest. Therefore, the data realized is subject to the

limitations o f the sample.

Definitions

The following terms are used in the study or in the Performance Analysis for School

Districts instrument:

1. Approach refers to the systems in place to improve quality and customer

satisfaction (Brown, 1994).

2. Collaborative and participatory approach to management is defined as jointly

working to identify problems and determining improvements with others in the organization

who are knowledgeable, involved, and affected by any decisions made.

3. Communication processes refer to methods used to inform and seek opinions from

others.


11

4. Comparative data or benchmarking refer to an improvement process in which an

organization compares its performance against best-in-class organizations, determines how

these organizations achieved their performance levels, and uses the information to improve

its own performance (Shipley & Collins, 1997).

5. Conventional information refers to standardized and state test scores, enrollment,

attendance, dropout, discipline, and operating budget.

6. Deployment refers to the extent to which an approach has been implemented

across an organization (Brown, 1994).

7. Educational program s and services refer to all programs and services provided to

students and conducted by professional, certified personnel or by non-certified personnel

supervised by certified personnel.

8. Educational support services refer to all programs and services which support

educational programs, such as business operations, transportation, public relations,

purchasing, clerical services, legal services, volunteers, food service, records, buildings, and

grounds.

9. Expectations refer to clearly defined statements describing specific academic,

behavioral, or social criterion to measure achievement.

10. Data and information processes include the collection, management, and

dissemination o f data on enrollment, achievement, operations, and stakeholder satisfaction

that are used in evaluation and planning processes.

11. Internal communication processes refer to personnel and students within the

school district.

12. External communication processes refer to parents and community stakeholders.


12

13. Human resource area includes employee well-being, satisfaction, professional

development, work system performance, and effectiveness.

14. Human resource indicators include employee well-being, labor relations,

satisfaction, professional development, work system performance, and effectiveness.

15. Leadership refers to district-level senior administrators and board o f trustees.

16. Organizational effectiveness is a social construct referring to the quality o f an

organization that achieves the performance expected o f it (Brassard, 1993).

17. Performance refers to the results produced by the school district as illustrated by

multiple indicators.

18. Performance data includes data or information from all aspects o f the

organization, including student performance measures, enrollment, discipline, human

resources, business operations, and community.

19. Results refer to data on the performance o f the organization (Brown, 1994).

20. School district units refer to the specific schools, departments, or services o f that

school district.

21. Stakeholder refers to individuals or groups, both internal to the school (students,

all personnel) and external (parents, community members, business) which are affected by

the conditions and quality o f education and the preparedness o f graduates.

22. Student conduct indicators refer to measures o f student behavior such as

disciplinary infractions, suspensions, expulsions, arrests, etc.

23. Strategic developm ent refers to the process by which members o f an organization

clarify the purpose and develop the necessary procedures and operations to achieve a purpose

and design a strategic plan.


13

24. Suppliers refer to those businesses or individuals with which the district contracts

for specific services such as training, consulting, transportation, legal, etc.

25. Partnering processes refer to relationships between organizations, community

agencies, and businesses through which services for students and stakeholders are designed,

implemented, and provided.

26. System refers to a complex o f elements in mutual interaction (Owens, 1970).

27. Total quality or continuous improvement refer to a system that elicits

organization-wide participation in planning and improving processes in order to meet or

exceed customer expectations.

28. Work system s are defined as the way in which jobs, work, and decision-making

are designed at all levels within the organization.

Summary

There are few organized approaches to the assessment o f performance in school

districts that apply an integrated analysis o f the subsystems. The lack o f use o f information

to make strategic improvement decisions and a systems-based approach to assessing the

current effectiveness contributes to unsuccessful reform initiatives in education. An initial

step in any process o f examination is to determine what now exists (Goodlad. 1984). This

study seeks to determine the usefulness o f an assessment process to a acquire baseline

perception o f the organization’s performance as it exists today using three constituencies of

the organization.


14

Chapter 2

Literature Review

Introduction

Areas o f literature previously cited in Chapter 1 are investigated in depth in this

chapter. First, it is critical to understand the nature o f the educational institution as a system.

Defining the nature o f educational subsystems and their relationship to each other in the

larger system is fundamental to understanding how to study and improve its effectiveness.

Second, organizational effectiveness theory and practice is discussed. Third, current

approaches to determining school effectiveness and issues o f accountability are reviewed.

Fourth, quality theory is discussed as foundational to understanding the emerging

applications in both business and education. Finally, the applications o f the Malcolm

Baldrige National Quality Award in business and education are described.

Education as a System

During the past decade there has been an increased attention to systems thinking. The

field o f systems thinking includes cybernetics, chaos theory, and Gestalt theory, and is

reflected in the works o f Ludwig von Bertallanfy, Russell Ackoff, Gregory Bateson, and Eric

Trist (Senge et al, 1994). Ludwig von Bertalanfy recognized the relationships among several

important concepts current in biology in the 1930s (Levine & Fitzgerald, 1992). He named

the integration o f these ideas general systems theory, incorporating cybernetic concepts such

as feedback. Miller (1993) describes the theory as a philosophy o f science that studyies

natural phenomena o f all sorts as heterogeneous wholes composed o f multiple different but

interrelated parts rather than studying each part in isolation. Three types o f systems are

described in the literature o f the biological sciences. Isolated systems are described by


15

Nicolis and Prigogine (1977) as those which do not exchange matter or energy with their

environment. Closed systems do exchange energy with their environment while open

systems exchange both energy and matter with their environment (N icolis & Prigogine,

1977).

Systems thinking is a discipline for seeing the wholes and the pattern o f

interrelationships among key components o f a system (Eisenberg & Goodall, 1993; Owens,

1970; Senge, 1990; Senge et al, 1994). A system is a collection o f parts that interact to

function purposefully as a whole (Deming, 1986; Patterson, 1993; Senge. 1990). Inter

dependence is the primary quality o f a system. It refers both to the completeness o f the

workings o f a system in its environment and the interrelationships o f individuals that fall

within the system. These interdependent relationships between people give the organization

its culture. Process, feedback, and contingency are also components o f systems (Eisenberg &

Goodall, 1993).

From a systems perspective, a school district, like other organizations, does not exist

as an entity unto itself, yet it often behaves as one (Eisenberg & Goodall, 1993). School

districts are both open and social systems (Hoy & Miskell, 1991; Owens, 1970). A social

system is defined as an interactive, interrelated, and interdependent network o f components

and unique organizational properties that form an organized whole and function to serve

common goals (MacLellan, 1994). An open system depends on the external environment for

their continued existence, requiring resources from external inputs to the systems

(Consortium on Productivity in the Schools, 1995; Deming, 1986; Eisenberg & Goodall,

1993; Owens, 1970; Senge, 19903). Figure 1 presents a school district as an open system.


16

People []"" Money Support Resources

INPUTS OUTPUT

EducationSystem

$$ CUSTOMERSHigher EducationTaxpayersEmployers

Educated students who can continue to learn

Figure 1. An Educational System as an Open System.

In open systems, groups outside the system affect the system’s survival and their

ability to change. School districts take in political, financial, and human resources and use

them to create a service. This service results in a product to the surrounding environment o f

the workplace, higher education, and community. Open systems theory emphasizes the

dynamic aspects o f organization; that is, the movement in one part leads in predictable

fashion to movement in other parts. They are in a constant state o f flux because they are

open to inputs from the environment (Katz & Kahn, 1978).

School districts have also been described as loosely coupled systems (Weick, 1976).

Weick (1976) explains that the use o f the term intends to convey the image that coupled

events are responsive but that each event also preserves its own identity and some evidence

o f its physical separateness. There is usually lack o f clarity, coordination, and articulation

between and among subsystems within the larger system, despite their interdependence.

Such systems often are organizations in which accountability and interdependence between

subsystems are low and autonomy is high (Deer, 1976; Fullan, 1980). Subsystems are

purposely not closely connected and do little to control each other’s activities. They tend to


17

respond by shutting out environmental threats and increase the sense o f efficacy and

autonomy o f its members.

Theories o f bureaucracy from which schools continue to be organized, have paid little

attention to the organization’s dependency on internal and external environments (Deming,

1986; Eisenberg & Goodall, 1993). The task o f teaching was viewed as clearly understood,

routine, and predictable (Katz & Kahn, 1978; Owens, 1970). This mechanistic approach to

organizing work in schools is considered to be efficient. This structure does not work when

the environment is uncertain and, in fact, interferes with the organization’s ability to be

adaptive to its inputs (Consortium o f Productivity in Schools, 1995). The influence o f

systems theory results in emphasis on inputs, processes, how the processes interact,

information flow and feedback, management o f relationships, and outputs (Deming, 1986;

Eisenberg & Goodall, 1993).

Organizational Effectiveness

The body o f knowledge o f organizational theory, behavior, and the pursuit o f a model

to determine organizational effectiveness is substantial, confusing, and often in conflict

(Brassard, 1993; Georgopoulos, 1957; Zammuto, 1982). Organizational effectiveness has

been defined in the literature in a variety o f ways. Attempting to define effectiveness,

develop criteria, and apply them to a variety o f organizations continues to be noted in the

literature (Brassard, 1993; Cameron, 1980; Georgopoulos, 1957; Zammuto, 1982).

Yuchtman (1967) points out two assumptions that are either implicitly or explicitly made: (a)

Complex organizations have an ultimate goal or function, and (b) the ultimate goal can be

identified empirically and progress toward it measured. How organizational effectiveness is


18

defined is related to the theoretical model from which it was developed. Three models

emerge in the literature (Brassard, 1993; Zammuto, 1982).

As early as the 1930s the emergence o f a goal-based approaches to organizational

effectiveness can be seen (Brassard, 1993; Cameron, 1980; Zammuto, 1982). These models

are often referred to as rational models and are functional rather than conceptual

(Georgopoulos, 1957). Organizational effectiveness is seen as the degree o f achievement o f

multiple goals or the degree o f congruence between organizational goals and observable

outcomes (Zammuto, 1982). The focus o f the rational organization is goal orientation

(Cameron, 1980; Patterson, Purkey & Parker, 1986). The design, articulation, and

achievement o f goals are emphasized in the organizations applying this model. The

assumptions in this model are that goals remain stable over time, goals are determined by the

leaders o f the organization, and goals become translated into objectives within the sub-units

o f the organization (Patterson et al, 1986). The organization is seen as an entity rationally

structured in order to achieve the goals to which it ascribes. The goals are typically created

to help the organization achieve its expected performance.

The focus o f evaluating effectiveness from this model is on the outputs produced by

the attainment o f goals (Cameron, 1980). The development o f efficiency-related criteria to

insure the accomplishment o f goals is often designed to influence the use o f resources to

achieve optimal performance, productivity, and profits for the organization (Bressard, 1993).

This model led to such practices as management by objectives that remained popular through

the 1970s. The focus o f management in this model is the accomplishment o f the

organization’s goals (Hersey & Blanchard, 1982). The emphasis o f management within this

model is setting goals and objectives that are accomplished by motivating and controlling


19

others in the organization to carry them out. In practice, however, organizations often do not

reflect these goals in their daily activities, either at a micro or macro level (Brassard 1993).

There are limitations to a goals approach pointed out in the literature (Cameron, 1980;

Etizoni, 1960; Katz & Kahn, 1978). Success may be overlooked if there is no goal to

measure it. Goals may be too low, misplaced, or harmful long term. Goals are usually

expressed as idealized states and are often not realistically assessed. The nature and effects

o f the social systems from which goals emerge often are not considered in the attainment o f

goals. The goals model may be useful when organizational goals are clear, consensual, and

measurable (Cameron, 1980; Patterson, Purkey & Parker, 1986). The criteria for determining

effectiveness then become unique to that organization and its goals.

Systems-based approaches emerged during the 1950s, according to Owens (1970) and

Zammuto (1982). These models draw on the emerging body o f general systems theory

discussed initially in this chapter. Applying this theory, organizational effectiveness is then

viewed as the extent to which an organization as a social system fulfills its objectives without

incapacitating its means and resources and without placing a strain upon its members

(Zammuto, 1982). W. Edwards Deming believed that systems are developed to perform

repetitive tasks (Deming, 1982). Most problems within organizations, he believed, came

from sub-optimization o f that system, meaning the system was performing these tasks below

their capability. The inconsistencies and contradictions that become apparent upon analysis

o f the system can be used to detect and isolate the flaws o f that system (Bradley, 1993).

Other models appearing during the 1970s are referred to as the multiple constituent

definitions o f effectiveness. The organization is effective insofar as it meets the expectations

o f actors associated with it in one way or another and who try to promote their objectives and


20

interests (Brassard, 1993; Cameron, 1980). An organization is effective insofar as the

majority o f those participating in it perceive that they can use it to satisfy their interests. This

model stresses the importance o f satisfying the expectations o f the actors who agree to

support the organization and who influence its ability to obtain the resources it needs and

conserve its legitimacy (Brassard, 1993). This model acknowledges up front the influence

that these constituents have on the organization. Related to these models are those

approaches driven by the requirements o f external organizations such as accreditation

agencies, laws, and regulations (Brassard, 1993).

Dubin (1976) pointed out that organizational effectiveness has a different meaning,

depending on whether the organization is viewed from the outside or inside. The inside

perspective o f an organization tends to be a traditional managerial viewpoint which

emphasizes return on investment and efficient use o f resources. The perspective from the

outside evaluates the output o f the organization relative to its contribution to the environment

or the context outside the organization. Dubin (1976) further points out that there is no

correlation relationship between these two perspectives. In fact, he says, they are worlds

apart and cannot be reconciled. “We must face squarely the fact that organizations live under

conflicting demands regarding their effectiveness” (Dubin, 1976, p. 8). Bass (1952)

suggested the criterion o f organizational success needed to be expanded to include measures

relevant to employees, society as a whole, and the organization’s management. He suggested

organizational performance should be assessed based on: (a) the degree to which an

organization’s performance was profitable and productive, (b) the degree to which an

organization was o f value to its employees, (c) the degree to which an organization and its

members were o f value to society. Campbell et al (1974) found over 25 different variables


21

that were used as measures o f effectiveness in organizations prior to 1973 (see Table 1). The

most commonly recurring criteria were adaptability/flexibility, productivity, and satisfaction.

Determining Effectiveness in School Systems

MacLellan (1994) did a meta-analysis o f organizational effectiveness criteria used in

school systems. He found fifteen criteria: goals, environment, leadership, structure, work

force, interaction, process, decision-making, workplace, culture, change, communication,

curriculum, accountability and politics. The studies used ranged from 1967 through 1991.

Determining the effectiveness o f schools is even more elusive than for other

organizations. Schools, universities, and colleges have been referred to as organized anarchy

in the literature o f organizational study (Cameron, 1980). Some typical characteristics are:

1. Goals are ill-defined, complex, changing, and often contradictory. Goals o f some

sub-units may be unrelated to the broader organizational goals.

2. There is often no connection in the way work is done and the outcome.

3. More than one strategy can produce the same outcome.

4. There is little or no feedback from the output to the input.

5. Sub-units are not tightly connected, so it is easier to ignore outside influences.

6. Widely differing criteria o f success may be operating simultaneously in various

parts o f the organization.

There is often an ambiguous connection between the organizational structure and the

activities o f the organization. It is typical to find rigid structures and hierarchies imposed

upon loose, fuzzy processes.


22

Table 1. Factors o f Organizational Effectiveness

Overall effectiveness o f the organization Productivity

Efficiency Profit

Quality Accidents

Growth Absenteeism

Turnover

Motivation

Control

Flexibility/adaptation

Role and norm compliance

Readiness

Utilization o f environment

Evaluations by external entities

Internalization o f organizational goals

Satisfaction

Morale

Conflict/cohesion

Goal consensus

Managerial task skills

Managerial interpersonal skills

Managerial management communication

Stability

Value o f human resources

Note: From The M easurement o f Organizational Effectiveness: A Review o f Relevant Research atid Opinion (pages 39-40), by J. P. Campbell, E. A. Brownas, N. G. Peterson and M. D. Dunnette, 1974, San Diego: Naval Personnel Research.

Cameron (1980) makes the point that none o f the described models o f organizational

effectiveness will work for organized anarchy. Criteria o f effectiveness are usually vague

and ambiguous, making organizational goals difficult to measure and not necessarily agreed

upon by all sub-units. There is often no feedback loop between outputs and inputs, making

the systems model an unnatural fit. Cameron (1980) suggests that the multiple constituencies

model may be the most appropriate for the organized anarchy. The demands o f the

constituencies, once defined, can be assessed on the degree to which they are met.

R eproduced with perm ission o f the copyright owner. Further reproduction prohibited without perm ission.

23

Although there is substantial information regarding evaluation o f schools and school

programs in the literature, there is little consistency in the approaches and parameters

evaluated. Eight components are reported by Nowaiowski (1985). They include business

and finance, curriculum and instruction, policy, planning and evaluation, pupil personnel

services, personnel, school-community relations, and school management. MacLellan (1994)

ranked components o f school systems evaluated in the literature. Table 2 illustrates his

findings.

Table 2. Meta-analysis Findings o f School System Evaluation Components by Rank as Reported in the Literature.

1. Goals 8. Decision-making

2. Environment 9. Work place

3. Leadership 10. Culture

4. Structure 11. Change

5. Workforce 12. Communication

6. Interaction 13. Curriculum

7. Process

Note: From “Towards a New Approach for School System Evaluation,” (Page 159), by David MacLellan, 1994. (Doctoral dissertation, Dalhousie University, Nova Scotia).DisseNational Abstracts International.

There are three methodologies that appear to be represented in a substantial manner in

the literature: (a) evaluation research, (b) curriculum audits, and (c) effective schools

research. The researcher has also included a discussion o f the accreditation process in Idaho.

The discussion o f the effectiveness o f schools is often linked with discussions o f

accountability or to performance o f student learning. Although both factors are relevant to


24

this study, the researcher focused on models that were using multiple indicators o f

effectiveness.

A key deficit in most educational systems, which is all to frequently pointed out by

the critics o f public education, is the lack o f effective evaluation (Worthen, 1987). The

public demand for accountability has made many educators fearful o f the concept. DeMont

and DeMont (1973) suggested three improvements to the process o f demonstrating

accountability: (a) an increased focus on the outputs o f education, (b) the production o f more

effective evaluative and research models, and (c) the inclusion o f non-educators in the

decision-making process. They suggest that an accountability model be a comprehensive

plan for problem solving aimed at improving educational practice. The requirements o f this

model include: (a) the designation o f the persons responsible for the program operation.

(b) conducting an internal program review, (c) conducting an external program review, and

(d) use o f the results to diagnose needs and prescribe action.

Evaluation research has been a tool used frequently in public schools to make

judgments about the merit, value, or worth o f educational programs (Gall, Borg & Gall.

1996). They are most often used to determine the effectiveness o f specific programs,

benefits to cost ratios, or areas for improvement. Formal evaluation consists o f systematic

efforts, using qualitative and/or quantitative designs to define criteria and obtain accurate

information (Worthen, 1987). Formal evaluation studies are often done as a basis for

decision-making and policy formation, to evaluate curricula, monitor expenditure o f public

funds, or improve educational programs (Worthen, 1987). Worthen (1987) notes that many

evaluation studies do not lead to significant improvements in school programs. He cites

several reasons including inadequacies o f research design, the use o f evaluation information.


25

and the view o f evaluation as a discrete study rather than a system o f self-renewal (Worthen,

1987/ The Joint Committee on Standards for Educational Evaluation (1994) developed

standards designed for use in judging quality o f educational evaluation. These standards

cover criteria involving the utility o f the evaluation, its usefulness to the persons involved,

feasibility o f the design to the setting, legal and ethical factors, and the extent to which the

study yields valid, reliable, and comprehensive information for making judgements.

Guba (1981) outlined four major models o f educational evaluation: (a) objectives,

(b) outcomes, (c) effects, and (d) audience concerns. Evaluation approaches that are based

on specific goals and objectives assess the congruence between the standard or the goal and

the performance (Provus, 1971). In a discrepancy-based model o f evaluation, standards are

defined and developed, the performance is assessed, the discrepancy is determined, there is

feedback to the decision-makers, and there is a decision. The critical point in this model is

the establishment o f a standard and assessment against that standard. The C. I. P. P.

(Context, Input, Process, Product) model is a decision-making approach relying on

generation o f information to be used in making decisions (Stufflebeam, 1983). The context

provides an illustration o f the needs and goals, input on how resources and procedures are

used to reach goals. The process focuses on any defects in the implementation o f those goals

and products and the measurement o f the outcomes. Scriven (1973) proposed the consumer-

oriented model that included establishing standards or indicators; comparing effects to

benefits and costs; and making judgements about change, use, and choice. The focus in this

model is the judgement o f merit or worth. The countenance model— later called the

responsive mode— distinguishes three phases: (a) antecedents, (b) transactions, and


26

(c) outcomes (Stake, 1967). This model relies on an informal framework in which

observation, judgement, and data matrices are emphasized.

Curriculum audits provide another source o f evaluating organizational effectiveness

for schools. Curriculum management audits were first offered by the accounting firm o f

Peat, Marwick, Mitchell and Company where a partner, Fenwick English, adapted it from the

financial audit process (Vertiz, 1995). He brought the service to the American Association o f

School Administrators which created the National Curriculum Audit Center. The Center

trains curriculum auditors and contracts with school districts. The first audit was done in

1979 in the Columbus Public Schools in Ohio. As o f April 1995, curriculum audits had been

performed in nearly 100 school districts in the United States and two foreign countries

(Vertiz & Bates, 1995). According to Vertiz (1995), the audit became an important data

source in state take-over o f school systems in New Jersey and Kentucky. It is based upon the

concepts o f effective instruction, curricular design, and delivery. The audit is designed to

determine the extent to which a sound, valid, and operational system o f curriculum

management is implemented (Vertiz & Bates, 1995). According to Vertiz and Bates,

curricular quality control requires: (a) a written curriculum in a clear, translatable form for

application by teachers in classrooms or related instructional settings; (b) a taught curriculum

which is shaped by, and is interactive with, written curriculum; and (c) a tested curriculum

which includes the tasks, concepts, and skills o f pupil learning that are linked to both the

taught and written curricula.


27

English (1988) described the five standards he created for the auditing process:

1. The school district is able to demonstrate its control o f resources, programs, and

personnel. Control refers to the system’s ability to channel and focus its resources toward the

achievement o f its goals and its mission (Kemen, 1993). Auditors look for indicators that

demonstrate linkages between the board, central management, and the instructional process

(English, 1988).

2. The school district has established clear and valid objectives for students.

Auditors examine board policy, administrative procedures, courses o f study, and scope and

sequence o f curriculum (English, 1988).

3. The school district has documentation explaining how its programs have been

developed, implemented, and conducted. The district must demonstrate clear and operational

linkages between all layers o f the system. Auditors look for alignment between policy,

curriculum, instruction, materials, and assessment (English, 1988).

4. The school district uses the results from the district designed or adopted

assessments to adjust, improve, or terminate ineffective practices. Auditors evaluate the

extent to which the district collects data to evaluate its performance. The data should reflect

its goals, provide usable information that should be used to adjust, or improve district goals

(English, 1988).

5. The school district has been able to improve productivity. Productivity is the

relationship between the inputs and the cost o f obtaining any given level o f outputs (English,

1988).

Each standard encompasses numerous criteria. These criteria are evaluated through

document reviews, interviews with the board and professionals, and observations by trained


28

auditors who are school administrators external to the organization. The data is then

triangulated according to agreed upon conditions. Table 3 illustrates specific criteria and the

findings o f a study o f audits conducted between 1988 and 1994 by Vertiz and Bates (1995).

The authors conclude that the majority o f school districts who participated in the audit

process were deficient in major management structures and functions that pertain to the

design and delivery o f curriculum. The investigators found that 90% or more o f the findings

were deficient in 80% o f the areas investigated.

Kamen (1993) found that the extent o f implementation o f the audit recommendations

is dramatically affected by the nature o f the audit selection method. When the audit is

voluntarily selected by a district, there is a high level o f implementation. There are positive

effects generally as demonstrated by greater empowerment o f all personnel and a tendency

towards a systems perspective. Management processes appeared to improve. Results

suggest that there is significantly less implementation o f recommendations when the process

is mandated. Under some conditions, it can become a political battleground with resistance,

denial, and defensiveness.


29

Table 3. Curriculum Audit Findings o f 67 School Districts Between 1988 and 1994.

Standard number: Criteria Strong rating Deficient rating

1 : Policy design 6% 94%

I : Policy implementation 0% 100%

1 : Planning design 10% 90%

1 : Planning implementation 10% 90%

I : Organizational structure 5% 95%

1 : Organizational implementation 27% 73%

1 : Personnel practices 0% 100%

1 : Personnel supervision and supervision 14% 86%

2 : Instructional goals and objectives 6% 94%

2 : Curriculum scope 22% 78%

2 : Curriculum guide: design 2% 98%

2 : Curriculum guide: delivery 0% 100%

2 : Curriculum management structure 3% 97%

3 : Internal consistency 3% 97%

3 : Equity: design 5% 95%

3 : Equity: implementation 7% 93%

3 : Monitoring practices 4% 96%

3 : Staff Development: design 2% 98%

3 : Staff development: delivery 0% 100%

3 : Articulation and coordination 2% 98%

(table continues)


30

Table 3, cont’d. Curriculum Audit Findings o f 67 School Districts Between 1988 and 1994.

Standard number: Criteria Strong rating Deficient rating

Testing program: scope 2% 98%

Testing program: quality 3% 97%

Use o f assessment data 2% 98%

Use o f program evaluation data 0% 100%

Curriculum-driven budget 0% 100%

Cost effectiveness 4% 96%

Organizational improvement 0% 100%

Facilities 39% 61%

School climate 83% 17%

Support system functioning 30% 70%

Note: From The Curriculum Management Audit: Revelations About Our School (1995), by Virginia Vertiz and Glynn Bates. Paper delivered to the American Education Research Association, Division B.

Effective schools research offers a set o f criteria for determining organizational

performance. Effective schools have been described by several parameters. Effective

schools add value through their services, high evaluations from students, high expectations

and high norms o f achievement, strong leadership, collaborative decision-making, clear

goals, system wide culture, safe environment, and a dedicated workforce (Mann, 1976,

Purkey & Smith, 1982). Seven characteristics emerged from the body o f literature known as

effective schools research (Edmonds, 1980). They include (a) strong, instructional

leadership; (b) a safe, orderly climate; (c) high expectations for achievement; (d) emphasis on


31

basic skills; (e) continual monitoring o f progress; (f) goals that are clear and understood; and

(g) culture.

Research began in the mid-1970s to determine why some schools were effective and

others were not. The research o f Ron Edmonds and Lawrence Lezotte began in large urban

schools. Their study resulted in the identification o f correlates present in effective schools

(Edmonds, 1979; Lezotte & Bancroft, 1985). Strong administrative leadership was found to

exist, particularly focused on planning, supporting and monitoring the instructional process.

There were high expectations for all students and the staff o f the building. A positive school

climate existed in the building as evidenced by a sense o f pride and community. There was a

focus on the instructional program in the total school with emphasis on training in teaching

practices. Finally, there was a thorough assessment process allowing for continual

monitoring o f student progress. Numerous improvement strategies followed that focused on

developing the specific correlates in schools. Stefanich (1983) pointed out that much o f the

impetus for these applications was based on intuitive rationale rather than hard data. Lezotte

(1989) has since integrated quality theory into his approach to school improvement, citing

such precepts as the importance o f an attitude o f continuous improvement, a deliberate

change strategy, and attention to all parts o f the system. Some independent contractors using

the correlates o f effective schools as the standard have created an audit-type process.

Each state has an accreditation process usually affiliated with a regional accrediting

organization (Portner, 1997). In Idaho the purpose o f accreditation is to help schools achieve

the required Standards for Idaho Schools and enhance school improvement (Idaho State

Department o f Education, 1996). There are four options for how Idaho schools seek

accreditation. They may choose one o f the following options:


32

1. The Idaho Elementary/Secondary Accreditation Standards.

2. The Northwest Accreditation Standards.

3. The Idaho School Accreditation School Improvement Model.

4. An alternative school improvement plan.

Regardless o f the option selected, the school must demonstrate the standards defined

and the components o f thoroughness as specified by Idaho Code 33-119. A thorough system

o f education has been defined in Idaho code as one in which:

1. A safe environment conducive to learning is provided.

2. Educators are empowered to maintain classroom discipline.

3. The basic values o f honesty, self-discipline, unselfishness, respect for authority,

and the central importance o f work are emphasized.

4. The skills necessary to communicate effectively are taught.

5. A basic curriculum necessary to enable students to enter academic or vocational

post-secondary educational programs is provided.

6. The skills necessary for students to enter the workforce are taught.

7. The students are introduced to current technology.

8. The importance o f student’s acquiring the skills to enable them to be responsible

citizens o f their home, schools, communities, state, and nation is emphasized.

Regardless o f the option selected, schools must demonstrate on an annual basis the

five required standards:

Standard I: Philosophy/Mission, Vision, Polices: School philosophy and policies

need to be aligned with thoroughness legislation.


33

Standard II: Personnel and Certification: All educators o f Idaho students must be

certified as specified in the State Board o f Education Rules for the Public Schools o f Idaho.

Standard III: Curriculum/Instruction/School Improvement: This standard is defined

in the thoroughness legislation.

Standard IV: Accountability/Assessments/Measures: Schools must establish

standards for all grade levels and high school exiting standards for graduation, participate in

statewide testing programs, have written plans to reduce dropouts, and report on student

attendance.

Standard V: Safe Learning Environment: Schools must have safe facilities. Each

school must have a comprehensive, district-wide policy and procedure in place encompassing

safe environment and discipline.

There are specific additional standards for each level— elementary, middle, and

secondary. Each standard has specific criteria to which deviation points are assigned.

Schools are accredited annually according to ratings determined by points. If schools receive

a status o f not approved for more than one consecutive year, state funds can be withheld and

a report to the public is made. The Northwest accreditation process involves a self-study for

initial accreditation involving staff, students, and community (NASC, 1996). The

accreditation process runs on a ten-year cycle involving a self-study during the ninth year o f

the process.

What is unclear in the literature is how the information from any method o f

determining organizational effectiveness is used. Brassard (1993) cautions against the need

to compare the performance o f organizations or to identify characteristics o f those that are

effective. Having criteria o f effectiveness reinforces the notion that: (a) organizations


34

possess an inherent rationality and (b) criteria become requirements which are imposed often

independent o f their purpose. He makes the point that criteria adopted must define the

performance that the organization must achieve if it is to be useful. Hannan and Freeman

(1977) argue that inter-organizational comparisons can not be accomplished because there is

no way for scientific analysis o f comparative organizational effectiveness.

The above review o f approaches to the overwhelming task o f determining

organizational effectiveness illustrates the varied strategies used in the past and present.

Such performance assessments are done for different reason. There is little discussion in the

literature regarding the process o f evaluation o f organizational self-study for the ultimate

purpose o f improvement. There is an increasing interest in action research or practitioner

based research done by the practitioners within their own site as a reflective process o f

investigation (Anderson et al, 1994). Practitioner research is best done as a collaborative

effort to accomplish multiple perspectives for the purposes o f taking actions in a specific

situation. Zammuto (1982) points out that it is useful to remember that organizations are

social inventions created to satisfy human needs. These needs influence how people evaluate

the effectiveness o f organizational performance based on their experience with organizations

and the impact o f that performance on them or their preferences. The purpose o f assessment

in anything is to determine the performance and then improve it.

Quality Theory

The importance o f theory as it relates to the areas cited above is explored since from

theory, assumptions, models, practices, and tools emerge (Skrtic, 1991). Many companies

today are using total quality theory or continuous improvement theory as both a conceptual

framework and operationally (Walton, 1990). Total quality or continuous improvement is an


35

approach to organizational development that has both historic roots and evolving tenets. It

involves both reflective and active components for organizational development. It has been

defined as a people-focused management system that aims at continual increase o f customer

satisfaction at continually lower real cost (Crosby, 1984; Deming, 1986; Imai, 1986). This

theoretical model is a systems approach to organizational improvement, meaning that

improvements should be made with the whole organization in mind (Deming, 1986, 1994;

Senge, 1990). The terms to ta l quality control and total quality management were coined by

Fiegenbaum (1983). He defined total quality control as “an effective system for integrating

quality development, quality maintenance, and quality improvement efforts o f various groups

in an organization so as to enable marketing, engineering, production, and service at the most

economical levels to allow for full customer satisfaction” (Fiegenbaum, 1983, p. 823). He

used the term total to mean a systems approach to achieve excellence. He defined quality in

terms o f the specific requirements o f the customer.

Japanese management theory has influenced quality theory in the Western world

significantly. Referred to as Kaizen in Japan, it is the single most important concept

influencing Japanese management (Imai, 1986). The Kaizen philosophy means on-going

improvement through the involvement o f everyone, in all aspects o f life. Imai remarks, “I

came to the conclusion that the key difference between how change is understood in Japan

and how it is viewed in the West lies in the Kaizen concept; a concept that is so natural and

obvious to many Japanese managers that they often do not realize that they possess it!”

(Imai, 1986, p. 3). He concludes that this concept is either very weak or non-existent in

American and European business based on his many years o f studying the differences.


36

There are consistent fundamental principles in quality theory, which emerge upon

review o f the literature by those who are credited as being the major experts in the quality.

The researcher will focus on these principles rather than an in-depth analysis o f the historical

perspectives o f quality theory. Upon review o f the literature, it is clear that quality theory

has emerged from a variety o f historical management approaches, economic contexts o f the

times, cross-cultural influences o f both East and West, and an ever-increasing body o f

knowledge that is evolving through practice (Crosby, 1984; Danne, 1991; Deming, 1986).

W. Edwards Deming, often considered the “father o f quality,” developed a theory o f

profound knowledge that incorporates the major tenets o f quality theory (Deming, 1986,

1989, 1994). He believed that not only skills, but also knowledge about management was

paramount. Deming (1989) stated, “hard work and best efforts, put forth without guidance o f

profound knowledge, leads to ruin in the world that we are in today. There is no substitute

for knowledge” (Deming, 1984, p. 10). The system o f profound knowledge includes four

principles, each related and interacting with the other.

The first principle is appreciation for a system, which Deming defined as “a network

o f interdependent components that work together to try to accomplish the aim o f the system”

(Deming, 1984, p. 50). He stressed the interdependencies within a system and the necessity

o f cooperation among the parts. The greater the independence between the components, the

greater the need for communication and cooperation between them. The system needs to

have an aim that is clear to all in the organization. Without this clear purpose, says Deming.

the aim becomes a value-judgement made on individual bases (Deming, 1984). Deming

often used the example o f a good orchestra to illustrate a well-optimized system. “The

players are not there to play solos as prima donnas, to catch the ear o f the listener. They are


37

there to support each other. They need not be the best players in the country.” (Deming,

1984, p. 15). According to Deming, management o f a system is action based on prediction.

The prediction needs to be rational and based on the information that the system teaches

people in the organization. Therefore, the performance o f any part o f the system must be

judged in relationship to other parts and the aim o f the system.

A second element is knowledge o f variation or statistical theory. Deming believed

that without statistical analysis methods, attempts to improve a process would be hit or miss.

Understanding that variation will always exist in all components o f a system— people,

processes, results— is fundamental. He called for an understanding o f the capability o f a

process. Developing stable processes— which means the process is in a state o f statistical

control— is the goal in determining a system ’s capability. He makes the distinction between

the types o f variation, special cause, and common cause. Common cause he defines as the

variations that occur by chance and can be attributable to a system Special causes, on the

other hand, are caused by events outside o f a system. Deming felt that these were important

to know before one attempted to work on a system (Deming, 1986, 1989). If these

distinctions are not understood, he suggested, mistakes can be made that are costly and

ineffective.

The prevention o f errors and nonconformance to specifications are key principles

resulting from the knowledge o f variation (Crosby, 1984; Deming, 1986). Philip Crosby, a

recognized quality expert, invented the term zero defects which he defined as no acceptable

rate o f defects for products or services that do not meet customer’s requirements (Crosby,

1984). The emphasis is on prevention, rather inspection or the process o f detecting the good

and the bad.


38

Deming’s third principle builds on the need to use the system and its variation to

generate what he called the theory o f knowledge (Deming, 1986, 1989, 1994). He believed

that good intentions were not enough for management. Managers, he insisted, need to

continually build knowledge and theories based on that knowledge. Deming believed that

this was the only basis for management’s ability to predict. When processes are in the state

o f statistical control, statistical theory can assist in prediction. Theories then emerge from

this knowledge. Theories, professed Deming, are necessary to generate questions. Without

questions, there may only be examples o f successes, and if these are duplicated under the

pretense o f a solution, they can lead to failure. Continual approach o f narrow solutions can

lead to more and more o f the solution (Senge, 1990). Theory is critical in optimizing a

system which can meet the customer’s expectations the first time (Deming, 1986, 1989;

Crosby, 1984).

Joseph Juran, another quality expert, extended Deming’s beliefs as they pertained to

knowledge-based decisions to the role o f management (Juran, 1988). He believed that it was

the responsibility o f top management to lead the company through massive training in

quality. Juran placed an emphasis on planning, customer satisfaction, and the use o f data

collection and analysis and has been credited with being the first to address the broader

issues o f management as they relate to quality (Danne, 1991; Miller, 1993).

The fourth principle o f profound knowledge is psychology. Deming felt that this

body o f knowledge was critical in the interaction between people and circumstances, the

interaction between people, and the interaction between people and the system (Deming,

1986, 1989). He emphasized the importance o f leaders in recognizing the differences in

people and using these differences to optimize the system. Recognition o f differences in how


39

people leam, how they work, and how they relate to each other is an additional factor that a

manager should understand. Leaders are obligated to make changes in the system which will

result in improvement (Crosby, 1984; Deming, 1986).

Critical to this system o f profound knowledge are the following:

1. People have an innate need for self-esteem and respect.

2. Circumstances can either provide or deny people opportunities for dignity and

self-esteem.

3. Management that denies opportunities for dignity and self-esteem will smother

intrinsic motivation.

4. Some extrinsic motivators rob employees o f dignity and self-esteem.

5. Management should recognize the innate inclination o f people to leam and invent.

Deming believed that new systems o f rewards needed to be established to restore respect for

the individual and release the potential o f human resources (Deming, 1986, 1989, 1994).

Organizational behavior can affects the quality o f services, products, and, in the case o f

schools, the quality o f instruction (Deming, 1994; Patterson et al, 1986).

What has emerged from the Deming system o f profound knowledge is an evolving

body o f knowledge that incorporates systems theory, scientific method, management by fact,

and participation o f everyone within the system. Each o f the quality experts mentioned have

similar messages emphasizing different concepts. Table 4 provides a matrix o f key quality

principles and the interpretation o f each offered by Deming, Juran, and Crosby.


40

Table 4. A Comparison Among Teaching Theories o f Quality Experts.

Concept Deming Juran Crosby

Definition Predictable degree o f Fitness for use Conformance

o f quality dependability suitable to requirements

to market

Performance Use o f statistics to Avoidance o f Zero defects

Standards measure performance campaigns to do

in all areas perfect work

Approach to Optimization o f system; Management must Prevention;

Improvement elimination o f goals consider human process

without methods side o f quality development

Statistical Use SPC for Use could lead Rejection o f

Process quality control to “tool-driven” statistically accepta

Control approach levels o f quality

Employee Employee Use o f teams; Quality

Participation participation in quality circles improvement

decision-making teams

Continuous Improvement in Business

The history o f the recent movement to improve performance in the private sector is

relevant to current and future applications in other settings. The origins o f the quality


41

movement can be traced to the 1940s and the context o f World War II (Pines, 1990). The

United States War Department established a Quality Control section in 1942 as a response to

an increased demand for mass production o f weapons and other war materials. Staff from

Bell Telephone Laboratories were used— primarily two statisticians, Walter A. Shewhart and

W. Edwards Deming. Their approach was to predict the performance o f production by

measuring manufacturing processes and stabilizing their performance. When these statistical

techniques were applied, America’s defense was exemplary. At that time, the progressive

approach to manufacturing was referred to as acceptable quality levels (AQL), which

assumed that there was an acceptable level o f allowable failures. The approach offered by

Shewhart and Deming suggested that this approach was one o f the reasons why the United

States was seeing a decline in productivity compared to other countries.

Garvin ( I98S) reports four major quality eras. Prior to and during the 1930s, the

emphasis was on inspection. Processes for detecting defects such as grading, counting, and

repairing were common in American businesses. From the 1930s to the 1950s, statistical

quality control became popular. This strategy assumed that the principles o f probability and

statistics would allow managers to control the variation in a production process to determine

if the cause o f the variation was inherent in the process or the result o f a special cause.

During the 1950s and 1960s, the quality assurance movement emphasized the planning

function, and the concept o f continuous process improvement was originated. The linkage

between quality and controlling costs was made. Beginning in the 1980s, the quality

management period was significantly influenced by W. Edwards Deming. An NBC-TV

documentary that aired on June 24, 1980, I f Japan Can, Why Can't We? explored how

Japanese products came to be perceived as far superior to those o f the United States (Walton,


42

1990). During an interview, Deming, age 79 at that time, shared how he taught Japanese

management and engineers how to use quality as a system. These techniques enabled them

to detect and eliminate defects, cut down on waste, reduce costs, and increase productivity.

He used methods referred to as statistical p rocess control (SPC). Although they had been

used in America after World War II, their use faded when volume overruled quality.

Since 1980, many companies have adopted quality principles and practices. Curt

Reimann, recently retired Director o f the Malcolm Baldrige National Quality Award,

reported in a telephone interview that all o f the high performing companies today are, in one

way or another, applying quality principles and practices. He further related that there were

many examples o f failed attempts, but companies that have successfully applied these

principles and became learning organizations are realizing results. Brown (1994) found that

some executives felt quality peaked in 1992 and many companies have abandoned quality to

resume a back-to-basics approach emphasizing results. Reimann pointed out in the interview

that the only reason to attend to processes was to improve results. This unfortunate but

common misunderstanding in the application o f quality has been substantiated by the

literature (Brown, 1994).

To help encourage United States companies and reward them for providing high

quality products and services, the Malcolm Baldrige National Quality Award was created in

1987 under President Reagan. The award was named after Malcolm Baldrige, the Secretary

o f Commerce credited for his managerial approach to long-term improvement in economy,

efficiency, and effectiveness in government (National Institute o f Standards and Technology,

1994). By enacting the Stevenson-Wydler Technology Innovation Act o f 1980, Congress

established the Baldrige Award which created a public-private partnership designed to


43

encourage quality in American companies (Brown, 1994). Gavin (1991) stated, “In just four

years, the Malcolm Baldrige National Quality Award has become the most important catalyst

for transforming American business. More than any other initiative, public or private, it has

reshaped managers’ thinking and behavior” (Gavin, 1991, p. 80).

The MBNQA Award (1994) was designed to promote:

1. Awareness o f quality as an increasingly important element in competitiveness.

2. Understanding o f the requirements for quality excellence.

3. Sharing o f information on successful quality strategies and the benefits derived

from the implementation o f those strategies.

The Council on Competitiveness (1995) compiled a report after studying the Baldrige

Award and quality in the United States. Their finding were as follows:

1. The quality o f American goods and services is getting better. Unfortunately, this

progress has led to the perception that extending quality management principles and practices

is no longer a high national priority. Our competitors are continuously improving their

quality, and the United States cannot afford to be complacent.

2. The Baldrige National Quality Award and its state and local offshoots have been

key in the effort to strengthen United States competitiveness. The annual government

investment o f $3.4 million in this program is leveraged by over $100 million in private sector

contributions. The impact o f the Baldrige Award on the competitiveness o f United States

industry and the dividends it pays to the United States economy far exceed these investments.

3. The United States quality movement faces a new set o f challenges. We need to

overcome the confusion o f terms and apparently competing approaches (TQM, ISO 9000,

reengineering). New ways to extend quality to more large companies, as well as to small-


44

and medium-sized enterprises, are needed, and new sectors, such as government, education,

and healthcare, should be included.

4. Although a number o f vehicles are available to advance the process o f promoting

quality management— including state and local quality award programs, colleges and

universities, and the Manufacturing Extension Partnership— there has been inadequate

coordination among them and with the National Baldrige Award Program.

5. The Baldrige Award Program, having galvanized United States quality efforts, is

now positioned to become the vehicle for stimulating and coordinating efforts to expand

quality as a national priority.

In a telephone interview with Curt Reimann, this researcher inquired about the

development o f the specific criteria used. He related that the National Institute o f Standards

and Technology (NIST) began analyzing organizations that were currently succeeding, and

iisolated characteristics that were present in these organizations. A model was developed

consistent with the prevailing quality theory at that time. In order to ensure that the criteria

and processes remained relevant and reflected current thinking, the designers o f the MBNQA

developed a two-year revision cycle (Bemowski, 1996). The process allows for continuous

improvement reflecting what has been learned both in theory and in practice. Reimann

indicated that there has not been any effort on the part o f NIST to empirically validate the

criteria. The approach, however, has been one o f accumulating the information qualitatively

and drawing inferences. The intent o f the criteria and award process is not to be

prescriptive.


45

A survey was conducted to determine how the criteria are being used, specifically by

those companies who requested applications but who had not applied for the award

(Bemowski & Stratton, 1995). Three findings were reported:

1. The criteria were overwhelmingly used as a source o f information on how to

achieve business excellence. Over 48% were using the criteria to improve processes in their

companies while less than 25% o f the respondents used the criteria to apply for the award.

About 50% o f the respondents indicated that they used the criteria to promote a common

language within the company.

2. The majority o f respondents found that the criteria’s usefulness met or exceeded

their expectations.

3. There was great diversity in the enterprises using the criteria. They were

predominately used by managers o f a broad range o f industries.

The researchers concluded that the stated purposes o f the award were being

accomplished. The difficulties in interpreting the MBNQA criteria are well known

(Bemowski, 1996). That factor was a consideration during the latest revision o f the Award

criteria, according to Reimann in his interview.

The 1997 MBNQA criteria categories are as follows:

1. Leadership: Refers to how well senior managers provide leadership and sustain

clear values, directions, performance expectations, customer focus, and a leadership system

throughout the company.

2. Strategic Planning: Examines how the company sets and determines strategic

directions and key action plans by translating them into an effective performance system.


46

3. Customer and Market Focus: Examines how well a company determines their

customers’ expectations and then satisfies customer needs.

4. Information and Analysis: Examines managing and effectively using data and

information to support key company processes and the performance measurement system.

5. Human Resource Development and Management: Examines requirements to

develop full work force potential, i.e. an environment conducive to full participation, quality

leadership, and personal and organizational growth.

6 . Business Results: Examines performance and improvement made by the

organization, including customer satisfaction, financial and market performance, human

resource results, supplier and partner performance, and operational performance.

The framework from which the criteria are designed is based on a systems

perspective as illustrated in Figure 1. Refer to Table 5 for the organization o f the categories.

Pannirselvam (1995) conducted a study to validate the MBNQA model and

evaluation process. Results from data in the 1993 version o f state awards following the same

criteria revealed that the model is internally consistent and a reliable measure o f quality.

Tables 5 through 7 summarize the findings o f that study.


Table 5. Baldrige National Quality Award Criteria, 1997.

47

Category Criteria

1. Leadership 1.1 Leadership system1.2 Company responsibility and citizenship

2. Strategic Planning 2.1 Strategy development process2.2 Company strategy

3. Customer and Market Focus

3.1 Customer and market knowledge3.2 Customer satisfaction and relationship

enhancement

4. Information and Analysis

5. Human Resource Development and Management

6 . Process Management

4.1 Selection and use o f information and data4.2 Selection and use o f comparative information and data4.3 Analysis and review o f company performance

5.1 Work systems5.2 Employee education, training, and development5.3 Employee well-being and satisfaction

6 .1 Management o f Product and Service Processes6.2 Management o f Support Processes6.3 Management o f Supplier and Partnering Processes

7. Business Results 7.1 Customer Satisfaction Results7.2 Financial and Market Results7.3 Human Resource Results7.4 Supplier and Partner Results7.5 Company Specific Results

Note: From Malcolm Baldrige National Quality Award Criteria, 1997, National Institute o f Standards and Technology. (Gaithersburg, MD: United States Department o f Commerce and Technology Administration)


48

Table 6. Validity o f the MBNQA Model.

Research Question Finding

1. Are the items under each criterion a reliable measure o f the trait they attempt to measure?

Yes

2 . Is the MBNQA model a good and complete measure o f quality management practices?

Yes

3. Do the MBNQA criteria represent an accurate measure o f an organization’s quality management practices?

Yes

4. Do all the items under each o f the seven categories represent a single construct?

Yes

5. Is variability in the assessment o f total quality systems? Yes

6 . Is there assessment o f some elements more variable than others?

Yes

7. Is the variability in assessment related to the type o f organization evaluated?

Yes

8. Is the variability in assessment related to the characteristics o f the evaluator?

Yes

Note: From Statistical Validation o f the Malcolm Baldrige National Quality Award Model and Evaluation Process, 1995, by Pannirselvam. (Doctoral dissertation, Arizona State University) D octoral Dissertations.


49

Table 7. Accuracy o f the MBNQA Weights.

Research Question Finding

1. Do the weights assigned to the examination items Noaccurately reflect the importance o f each o f these items to agood management system?

2. Should the weights assigned to the seven criteria be different Yesfor different sizes and types o f businesses?

Note: From Statistical Validation o f the Malcolm Baldrige National Quality Award Model and Evaluation Process, 1995, by Pannirselvam. (Doctoral dissertation, Arizona State University) D octoral Dissertations.

There is debate in the literature and in the field regarding the effectiveness o f the

M BNQA criteria and the award process. Some criticisms focus around the notion that

companies spend too much time and money on the application process and are distracted

from the work o f the company (Crosby, 1991). There is also concern expressed that there

was no clear definition o f quality. The advocates for the criteria stand firmly on the belief

that since it is not intended to be prescriptive, there should not be a common definition o f

quality. Criticism has also centered on the belief that the criteria support the selection o f

companies that produce high quality results and are financially successful.

Today quality is seen as a field unto itself, with its own theory, models, practices, and

tools. Although its early applications were predominately in manufacturing, the applications

have quickly spread to service industries and the public sector.

Continuous Improvement in Education

Deming (1994) described America 2000: An Education Study as a “ . . . horrible

example o f numerical goals, tests, rewards but no method.” Can the theories, principles, and


50

practices o f quality or continuous improvement, developed in industry, help in the

transformation o f schools (Bradley, 1993; Fields, 1994; Glasser, 1992; Langford, 1993;

Rhodes, 1990; Schmoker, 1996; Tribus, 1993)?

Our current system o f education has been influenced historically by industry as well.

Educational administration has its roots in the theory o f scientific management, spawned by

Fredrick Taylor during the period o f 1910-35 (Bradley, 1993; Stempen, 1987). Max Weber

also had considerable influence during that period on the management and administration o f

organizations (Owens, 1970). He characterized the ideal bureaucracy as having the

following characteristics:

1. A division o f labor based on functional specialization.

2. A well-defined hierarchy o f authority.

3. A system o f rules covering the rights and duties o f employees.

4. A system o f procedures for dealing with work situations.

5. Impersonality o f interpersonal relations.

6 . Selection and promotion based on technical competence.

However, Weber also warned that massive, uncontrollable bureaucracy could be a threat to

free enterprise capitalism (Owens, 1970).

In the 1950s, systems theory was applied to schools as a social system with a

hierarchical role structure (Owens, 1970). These theories were attempting to understand the

organization as a place o f greater productivity and efficiency. Ernest Hartwell, a

superintendent o f three different large city school systems in the early 1900s, held that if

administrators applied business principles to progressive educational ideas, schools would

become efficient, stable organizations and would be more profitable to the students and the


51

community (Thomas & Moran, 1992). Today’s organization o f teaching, testing, and the

judging by grades has its roots in the industrial revolution’s theories o f mass production,

inspection, and re-work (Deming, 1994).

Deming (1986, 1994) had much to say about the system o f education and the type o f

changes that need to occur. He believed that not only did business management ignore

psychology, but so did the managers o f education. His beliefs about the individual

differences o f people, their need for self-esteem, dignity, and intrinsic motivation provided a

basis for his criticism o f the educational system. His theory, practices, and tools have been

very appealing to educators who are trying not to lose hope in the on-going battle to improve

schools and reverse the tide o f public criticism.

In 1991, the Association o f Quality Control conducted its first Quality in Education

Survey (Klaus, 1996). At that time, 133 K-12 and higher education institutions responded,

indicating that quality had been implemented. Five years later, the study indicated that 451

educational institutions had implemented quality (Klaus, 1996). Educators have wrestled

with the theories o f quality and the wisdom o f experts in industry and have made applications

in a meaningful way (Bernhardt, 1994; Bonstingl, 1996; English et al, 1994; Fields, 1994;

Glasser, 1992; McClanahan& Wicks, 1994; Rubin, 1994; Tribus, Langford & Cleary, 1995).

There are increasingly more efforts being made to conduct research in the application o f

quality in the field o f education (Chapell, 1993; Danne, 1991; Fritz, 1993; Louer, 1993;

Miller, 1993; Partin, 1992; Regauld, 1993; Smith, 1996).

The results in organizational improvement can be seen in several schools and school

districts across the country. Improvements on disciplinary action have been reported by Mt.

Edgecumbe in Sitka, Alaska (Danne, 1991; Langford et al, 1995). Decline in drop-out rates


52

have been cited by George Washington Vocational/ Technical High School in N ew York

City (Danne, 1991). Redesign o f programs to prevent students from dropping out and to

increase their success is a focus for some schools (Danne, 1991). Improved systems o f data

collection, analysis, and benchmarking have been developed in a number o f school districts

(Langford et al, 1995; Seigal et al, 1994). Motivated by high failure rates, staff at Parkview

School District identified root causes and implemented multiple systemic solutions resulting

in a decrease o f the failure rate by 50% in just one year (Seigel et al, 1994). The emphasis at

the Christa McAuliffe Elementary School in Prince W illiam County, Virginia, has been on

teaching students quality practices and tools to assist them in working together, being

responsible for their own learning and progress, and involving the larger community (Seigal

et al, 1994). There are 70 elementary schools in the United States and abroad where the

Koalaty Kid model o f continuous improvement has been implemented (Green, 1996).

Students use quality tools to monitor their own progress and improvement in mastering new

skills and content.

In 1993 the decision was made to launch the Malcolm Baldridge National Quality

Award program using the Education Criteria Pilot in 1994-95. A pilot approach was taken to

address the many issues involved in extending eligibility to education (National Institute o f

Standards and Technology, 1995). During the pilot year, schools who applied were not

eligible for the award.

The objectives o f the Education Pilot Program were to:

1. Determine the interest and readiness o f educational organizations to participate in

a national-level recognition program based on the ability to demonstrate overall performance

improvement.


53

2. Evaluate the Pilot Criteria.

3. Determine the capability o f the evaluation system, including volunteer experience,

availability, and time commitment

4. Determine the value o f the feedback given to Pilot Program participants.

5. Determine whether or not there should be subcategories o f eligibility, taking into

account school type and size.

6 . Determine the likely influence o f the award on: (a) sharing o f best practices

information, (b) cross-sector cooperation, (c) elevation o f educational standards.

The criteria for the Education Pilot are based on core values and concepts. These are

summarized Table 8.


54

Table 8. Core Values/Concepts o f MBNQA, Education Pilot, 1995.

Core Value Core Concepts

I. Learning-Centered A. Focus on learning and real needs o f learnersEducation B. High developmental expectations/standards for all students

C. Understanding that student learning rates/styles varyD. Major emphasis on active learningE. Regular, extensive formative assessment early in

learning processF. Periodic use o f summative assessment to measure

progress against key relevant external standards/normsG. Assist students/families chart progress using self-

assessmentH. Focus on key transitions such as school-to-school and

school-to-career

II. Leadership A. Clear, visible directions and high expectationsB. Modeling o f strategies for continuous improvement

methods and processes by senior administratorsC. School polices that reinforce learning/improvement

climate and encourage self-directed responsibilitythroughout the school

D. Building community support and aligning business andcommunity leaders with their aims

III. Continuous A. Clearly established goalsImprovement/ B. Fact-based measures/indicatorsOrganizational C. Systematic cycles o f planning/execution/evaluationLearning D. Focus on improving processes for improved results

E. Embedded approach that involves students

IV. Faculty/Staff A. Increased knowledge o f faculty/staff about studentParticipation/ learning and assessment strategiesDevelopment B. Improved performance o f faculty/staff

C. Organization tailored to a more diverse workforce andmore flexible, high-performance work practices

V. Partnership A. Internal and external partnerships to better accomplishDevelopment overall goals

B. Partnerships that seek to develop long-term objectives,strategies for evaluating progress, and means forchanging conditions

(table continues)


55

Table 8, cont’d. Core Values/Concepts o f MBNQA, Education Pilot, 1995.

Core Value Core Concepts

VI. Management A. Improvement system based on cause-effect thinking,by Fact measurement, information, data, and analysis

B. Measurements that support school’s mission/strategyC. Focus on student learning through a comprehensive and

integrated fact-based system

VII . Long-range A. Strong future orientation with long-term commitmentView to students and stakeholders

B. Investment in creating and sustaining assessmentsystem focused on student learning

C. School leadership familiar with research findings andpractical applications o f assessment/learning

D. School serves as role model in its operations

VIII. Public A. Protection o f public health, safety, and environment inResponsibility all practicesAnd Citizenship B. Ethical and non-discriminatory in all practices

D. Support of, and leadership in, purposes important to public

IX. Fast Response A. Faster, more flexible response to customer needsB. Simultaneous improvement in quality and productivityC. Strong customer focus

X. Results A. School performance system focused on resultsOriented B. Balanced needs and interests o f all stakeholders

C. Student performance demonstrated throughout theircareer in a variety o f ways

D. Effective and efficient use o f school resources

There are emerging initiatives in states and school districts in which the MBNQA

Education Pilot Criteria provides the framework for school improvement. The researcher is

aware o f several efforts. Pinellas County Schools in Florida has implemented the

Superintendent’s Quality Challenge, a model based on the Education Pilot criteria. The state

o f New M exico has initiated a joint private and public sector project, Strengthening Quality

in Schools, which incorporates the criteria and their state award process as a component. The


56

Pacific Bell Foundation has sponsored the Education for the Future Initiative in several

schools in California in which a school portfolio process o f organizational improvement was

developed based on components o f the MBNQA Education Pilot Criteria (Bernhardt, 1994).

The specific criteria are listed in Table 9. Revisions are currently being conducted by

the National Institute on Standards and Technology to align the framework to the 1997

MBNQA for businesses. Since 1995 there has not been funding from the legislature to

continue the development o f the Education Pilot, and efforts are actively being made to raise

the capital needed to continue to develop the process. Several states have now included K-12

education in their state quality award process. New Mexico, Florida, New York, and

Minnesota are four which have done so. Idaho is currently initiating those discussions.

The MBNQA Education Pilot Criteria (Table 9) are described below:

1. Leadership: Examines the personal leadership o f senior administrators and their

involvement in creating and sustaining student focus, clear goals, high expectations, and a

leadership system that promotes performance excellence. Also examines how these

objectives and expectations are integrated into the school’s management system.

2. Strategic and Operational Planning: Examines how the school sets strategic

directions and determines key plan requirements and how plan requirements are translated

into an effective performance management system with primary focus on student

performance.

3. Student Focus and Student and Stakeholder Satisfaction: Examines how the

school determines student and stakeholder needs and expectations by defining levels and

trends in key measures o f satisfaction relative to comparable schools and/or appropriately

selected organizations.


57

4. Information and Analysis: Examines the management and effectiveness o f data

and information used to support overall mission-related performance excellence.

5. Human Resource Development and Management: Examines how faculty and staff

development are aligned with the school’s performance objectives. Also examined are the

school’s efforts to build and maintain a climate conducive to performance excellence, full

participation, and personal and organizational growth.

6. Educational and Business Process Management: Examines the key aspects o f

process management, including learning-focused education design, education delivery,

school services, and business operations. Examines how key processes are designed,

effectively managed, and improved to achieve higher performance.

7. School Performance Results: Examines improvement o f student performance; the

school’s educational climate, services, and business operations at performance levels relative

to comparable schools; and/or appropriately selected organizations.


Table 9. 1995 M BNQA Education Pilot Criteria.

58

Category Criteria

1. Leadership 1.1 Senior Administration Leadership1.2 System and Organization1.3 Public Responsibility and Citizenship

2. Information and Analysis 2.1 Management o f Information and Data2.2 Comparisons and Benchmarking2.3 Analysis and Use o f School Level Data

3. Strategic and Operational Planning

3.1 Strategy Development3.2 Strategy Deployment

4. Human Resource Development and

4.1 Human Resource Planning and Evaluation4.2 Faculty/Staff Work Systems4.3 Facuity/Staff Develop ment4.4 Faculty/Staff Well-being and Satisfaction

5. Educational and Business Process Management

5.1 Education Design5.2 Education Delivery5.3 Education Support Service Design/Delivery5.4 Research, Scholarship, and Service5.5 Enrollment Management5.6 Business Operations Management

6 . School Performance Results 6.1 Student Performance Results6.2 School Climate Improvement Results6.3 Research, Scholarship, and Service6.4 School Business Performance Results

7. Student Focus and Student/ Stakeholder Satisfaction

7.1 Current Student Needs and Expectations7.2 Future Student Needs and Expectations3.3 Stakeholder Relationship Management3.4 Student and Stakeholder Satisfaction

Determination

Note: From M alcolm Baldrige National Quality Award, 1997, National Institute o f Standards and Technology. (Gaithersburg, MD: United States Department o f Commerce and Technology Administration)


59

Summary

The call for increased productivity, efficiency, and effectiveness remains constant

over the decades o f school reform. This chapter reviewed current literature on organizational

effectiveness, effectiveness in schools, and emerging models o f quality applications. The

literature reviewed establishes the background and current practices both in business and

education for measuring and improving organizational performance. Quality theory has been

explored as a framework for approaching school improvement. Current traditional practices

for determining comprehensive organizational performance were described. The application

o f the Malcolm Baldrige National Quality Award Education Pilot to school improvement is

an emerging area o f research and application. School improvement and reform strategies are

now recognizing the need for a systemic change strategy that recognizes the comprehensive

and complex nature o f school districts (Anderson, 1993; O’Neil, 1993; Wagner, 1993).

Increasingly, businesses are using the criteria o f the Malcolm Baldrige National Quality

Award to assess their status and guide them towards improvements that produces sustained

results through an aligned system which illustrates the core values o f the MBNQA.


60

Chapter 3

Methodology

Introduction

This research study examined a theory and a framework on which the operation and

performance o f a school district can be assessed to determine where improvements might be

necessary. There are three purposes:

1. To determine how participants rate the performance o f their district currently in

each o f the seven categories o f the Performance Analysis fo r School Districts.

2. To determine if these scores differ by type o f educator or size o f school district.

3. To determine how participants perceive the usefulness o f the instrument as a

framework for self-analysis by a school district in school improvement.

The study involved the development o f an instrument to collect information regarding

school district performance. It investigated differences by type o f educator and size o f

district. It measured the perception o f participants about the instrument’s usefulness in

approaching school improvement.

The Research Model

The model for the research study was as follows: Y t]k = u + a, ~ bj+ (ab),j ~ e ljk The value

of the response variable is the sum of:

u = the effect o f the overall mean,

a, = the effect o f the district size.

bj = the effect o f the position type.

(ab),j = the effect o f the interaction o f district size and position.

e ,jk = random error in model.


61

Instrumentation

The framework for determining organizational performance was adapted from the

Malcolm Baldrige National Quality Award Education Pilot (1995). The researcher

constructed an instrument based on the Malcolm Baldrige National Quality Award (1997)

and the Education Pilot Criteria (1995) as the primary framework. Since the 1995 Education

Pilot is being revised, the researcher was advised by Curt Reimann, retired Director o f NIST,

to consider the current 1997 changes. The researcher also integrated components from the

Northwest Accreditation Standards and the curriculum audit process. The intent o f such an

instrument was to reflect the comprehensive system o f a school district. Therefore, the

researcher felt that there were elements in both the accreditation process and the curriculum

audit process that could potentially be overlooked by an exclusive approach using only the

MBNQA. There are seven categories o f organizational performance used in this instrument:

1.0 Leadership

2.0 Strategic Planning

3.0 Information and Analysis

4.0 Student Focus and Student and Stakeholder Satisfaction

5.0 Human Resource Development and Management

6.0 Educational and Operational Process Management

7.0 School Performance Results

The descriptions in each category are based on the following Likert-type scale used to

construct the language in each o f the seven subcategories. The scale for the subcategories o f

Leadership, Strategic Planning, Information and Analysis, Student Focus and Student and


62

Stakeholder Satisfaction, Human Resource Development and Management, and Educational

and Operational Process Management were:

1. No systematic approach evident.

2. Awareness stages o f systematic approach but minimal requirements.

3. Developing a system that emphasizes prevention o f problems meets expectations.

4. A refined, well-developed approach that is deployed with broad applications.

5. A thorough, systematic approach that is fully deployed, institutionalized, and

idealized.

The scale for the subcategory o f School Performance Results was:

1. No results or results below expectations.

2. Some improvements; early stages o f developing trends.

3. Improvement trends or good performance in some areas.

4. Current performance is good to excellent with trends over time.

5. Superior performance with sustained results; state or national benchmark.

The instrument used to collect data was designed to yield continuous data reflecting

the ordered nature o f the items in each category and a weighting for each category. (See

Appendix A). The rationale was designed to: (a) more closely align it with the scoring

design o f the M BNQA in which additional points are awarded for more fully developed

quality practices and performance, and (b) yield continuous data weightings that more

appropriately answer the research questions put forward in the study.

The instrument was reviewed for content validity to insure it would answer the

research questions. Selected experts in the use o f the Baldrige criteria in business and/or

education, both in-state and out-of-state, were used. To qualify as a content-area expert, the


63

individual must be, or have been, a state or national quality award examiner using the criteria

from the Malcolm Baldrige National Quality Award and must be experienced in applying

quality applications in educational institutions. Appendix B contains a list o f the individuals

consulted and their qualifications. A cover letter (Appendix C) addressed the specific

research questions and the nature o f the feedback that the researcher was requesting.

Comments were received and items were revised.

The revised instrument was tested on twelve Idaho educators— two superintendents,

five principals, and five teachers. Minor revisions were made and the category o f “I do not

know” was added to each subcategory.

Subjects and Setting

The population used was educators working in Idaho public schools. A proportional

stratified random sample was selected using the 1996-97 database from the Idaho State

Department o f Education. The population was stratified by size o f student enrollment using

the classifications as outlined by the Idaho State Department o f Education as follows:

Classification 1 = 5,000+

Classification 2 = 2,500 - 4,999

Classification 3 = 1,000 - 2,499

Classification 4 = 500 - 999

Classification 5 = 1 - 499

A proportional allocation for sample size was used based on the ratio o f the number o f

districts in each category to the total number o f school districts in the state (Weirsma, 1995).

The sample size was determined using a table o f recommended sample size (Krejcie, 1970).

Table 10 and Appendix D provide the matrix for the sample design.


64

Table 10. Stratified Random Sample Matrix by Classification.

Superintendents Principals Teachers

Classification N % s** N % s** N % s**

#1 5,000+ 13 13.4 10 220 44 96 6,736 51.5 191

#2 2,500-4,999 13 13.4 10 84 16.8 37 2,358 18 67

#3 1,000-2,499 27 27.8 21 106 21.2 46 2,374 18 67

#4• 500-999 22 22.6 18 55 11 24 987 8 29

#5 1-499 21 21.6 17 31 6 4 617 5 19

State Totals 97 — 76 500 — 217 13,076 — 373

Collection o f Data

The instrument was prepared for electronic scanning. Each sheet was coded by size

o f district and type o f educator (Appendix E). The instrument was mailed to individuals

selected in the sample. This method was selected for convenience and to insure anonymity

o f the respondents. Appendix F contains cover letters and directions. The researcher secured

letters o f support from the Idaho Association o f School Administrators and the Idaho

Education Association to help ensure return (Appendix H). Each person selected in the

sample was also sent a pen printed with the message, “Thank you for participating in the

Performance Analysis for School Districts,” and a self-addressed, stamped envelopes was

enclosed. Mailing was conducted in October o f 1997, and respondents were given two

weeks to respond. A reminder post card was sent immediately following the deadline

(Appendix G). Random phone calls were also made asking for a response.


65

Data Analysis

There were seven dependent variables, i.e. scores from each category. There were

two independent variables: (a) type o f educator, with three levels (superintendents,

principals, teachers), and (b) size o f district, with five levels o f enrollment (over 5000, 4999-

2500, 2499-1000, 999-500, 499-1 (Appendix I). The SAS computer software program was

used to compile the data and generate the statistical analysis. Descriptive data was collected

to determine the characteristics o f the sample, the highest degree earned and the number o f

years o f experience in their current position, and the attitudes o f the respondents regarding

the instrument. Frequencies o f responses is also illustrated. A two-way factorial analysis o f

variance for each category was used to compare two independent variables (Huck, Cormier

& Bounds, 1974). Cronbach’s alpha was done to test reliability o f the instrument. The final

research question regarding the potential usefulness o f the instrument was answered with

descriptive statistics. Qualitative analysis o f the comments about the instrument was done

using a constant comparative model (Patton, 1983).

Summary

Chapter 3 established the procedural design o f the study that investigated how

educators in Idaho perceive their school districts in seven different categories. The sample

and instrument development were described. The statistical analysis used in Chapter 4 was

specified.


Chapter 4

Findings

Introduction

This study was designed to investigate three areas:

1. The perceptions o f superintendents, principals, and teachers in Idaho

regarding the performance o f their school district in seven areas.

2. The differences o f perceptions based on type o f position and/or size o f school

district.

3. The perceived usefulness o f the instrument constructed by the researcher for

self-study.

The dependent variables were the scores for each o f the seven constructs. There

were two independent variables, size o f district and position o f educator. District size

was divided into five levels depending on student enrollment: (a) over 5,000; (b) 4,999-

2,500; (c) 2,499-1,000; (d) 999-500; and (e) 499-1. The position o f the educators were

separated into three types: (a) superintendents, (b) principals, and (c) teachers. The

instrument, the Performance Analysis o f School Districts, was designed based on the

1997 Malcolm Baldrige National Quality Award criteria, the 1995 Education Criteria,

curriculum audit standards, and the Northwest Accreditation Standards. The instrument

was piloted, tested, and reviewed for content validity. Revisions were made based on

results. The sample was selected from the population o f educators in Idaho public

schools. The data was collected through a mailed survey. The results were scanned from

returned individual instruments. The data was analyzed using the Statistical Analysis

System (SAS). Descriptive analysis was done for characteristics o f central tendency and


67

a factorial analysis o f variance was used to test hypotheses. Cronbach’s alpha was used to

determine reliability. Qualitative analysis was done using a constant comparative model

for the comments o f participants.

Rate o f Return

The total sample size was 666. The total number sent was 656. Adjustments

were made for instances in which several participants had responsibility for more than

one o f the targeted positions or because participants were no longer in the position. A

total o f 258 surveys were returned for a 36% rate o f return. Nine (9) were eliminated due

to participant error, and eleven (11) were not used because they were received too late.

Wiersma (1995) reports 70% as a minimal acceptable rate o f return when surveying

professional samples. Table 11 illustrates the return rates by educator position for the

total number o f surveys sent. Table 12 illustrates the frequencies and percentages o f the

returns received by educator position and district size. The highest percentage returned

from the total sample by position was for teachers, and the highest percentage returned

from the total sample by size was for districts with over 5,000 students enrolled.

Table 11. Total Return Rates by Educator Position.

NumberSent

NumberReceived

PercentageReceived

Superintendents 76 49 64%

Principals 211 88 42%

Teachers 369 101 27%

Total 656 238 36%


68

Table 12. Frequencies and Percentages o f Returns Received by Educator Position and District Size.

5000+ 2500-4999 1000-2499 500-999 1-499 Total

Superintendents:

Frequency 8 6 13 13 9 49

Percent o f total 3.4 2.5 5.5 5.5 3.8

Percent by size 7.5 14.6 30.2 52.0 40.9

Percent by position 16.3 12.2 26.5 26.5 18.3 20.6

Principals:

Frequency 44 17 14 8 5 88

Percent o f total 18.5 7.1 5.9 3.4 2.1

Percent by size 41.1 41.5 32.6 32.0 22.7


Teachers:

Frequency 55 18 16 4 8 101

Percent o f total 23.1 7.6 6.7 1.7 3.6

Percent by size 51.4 43.9 37.2 16.0 36.6


Total:

Frequency 107 41 43 25 22 238

Percent by size 45 17.2 18.1 10.5 9.2

Characteristics o f Sample

A proportional, stratified random sample was selected from the state-wide data

base o f certified educators employed in Idaho public schools during the 1996-97 school


year. Participants were asked two demographic questions: (a) their highest terminal

degree, and (b) the length o f time in their current position. Table 13 illustrates the

highest degree by size o f district and position, and length o f time in current position

district size and by position. Table 14 illustrates the rank order by percent o f time in

position and highest degree. The most frequent terminal degree in the sample was a

Masters with the most frequently occurring range o f experience being twelve or more

years. Percentages o f terminal degrees varied by size o f district as illustrated in Table


70

Table 13. Percentage o f Highest Degree and Time in Position by Size and Position.

Variables

Degrees Years

B M S D 1 1-3 4-7 8-11 12+

5,000+:

Superintendents 37.5 62.5 12.5 25.0 50.0 12.5

Principals 52.3 27.3 13.6 9.1 9.1 29.5 18.2 34.1

Teachers 45.5 50.9 3.6 5.5 12.7 18.2 12.7 50.9

4,999-2,500:

Superintendents 33.3 33.3 33.3 16.7 50.0 16.7 16.7

Principals 47.1 47.1 5.9 23.5 29.4 29.4

Teachers 55.6 38.9 5.6 5.6 22.2 11.1 16.7 44.4

2499-1000:

Superintendents 15.4 69.2 15.4 15.4 7.7 53.8 15.4 7.7

Principals 78.6 14.3 7.1 7.1 21.4 21.4 28.6 21.4

Teachers 87.5 12.5 25.0 12.5 12.5 50.0

999-500:


Principals 87.5 12.5 25.0 62.5 12.5

Teachers 75.0 25.0 25.0 25.0 50.0

499-1:


Principals 100 40.0 60.0

Teachers 50.0 37.5 12.5 12.5 37.5 25.0 12.5 12.5


71

Table 14. Rank Order o f Combined Sample.

Percentage Highest Degree Percentage Years in Position

44.1 Bachelors 32.8 12+

23.5 Masters 27.7 4-7

22.3 Specialist 15.5 1-3

8-11

8.4 Doctorate 7.6 >1

Reliability o f Performance Analysis for School Districts

Cronbach correlation coefficient was used to determine the consistency o f the

instrument in measuring the seven constructs. Reliability coefficients (Table 15)

suggested internal test consistency existed in each construct, with the Leadership

construct having the highest reliability and the School District Results the lowest.

Table 15. Reliability o f Instrument.

Category Construct Cronbach’s Alpha

Leadership

Strategic Planning

Student and Stakeholder Satisfaction

Information and Analysis

Human Resources

Educational Process Management

School District Results

.851446

.831169

.768750

.803754

.842505

.828683

.741474


72

Descriptive Analysis

Descriptive statistics were used to illustrate central tendency and variability

among the dependent variables. Tables 16 through 24 illustrate the means and standard

deviations by district size and position o f educator. The lowest overall mean occurred in

the Information and Analysis category when combining district size and educator

position, while the highest mean occurred in the Leadership construct.

Table 16. Means For District Size and Positions Combined.

Construct N Mean SD

Leadership 233 3.15 .98

Strategic Planning 233 3.06 1.11

Student /Stakeholder Satisfaction 233 2.82 .90

Information & Analysis 237 2.62 1.13

Human Resources 233 2.71 .99

Educational Process 228 2.77 .93

School District Results 229 2.94 .91


73

Table 17. Means by District Size for Districts With 5000 or More Students Enrolled.

Construct N Mean SD

Leadership 106 3.15 1.02




Human Resources 107 2.62 1.02

Educational Process 104 2.84 1.02


Table 18. Means by District Size for Districts With Between 4,999 and 2,500 Students Enrolled.

Construct N Mean SD









74

Table 19. Means by District Size for Districts With Between 2,499 and 1,000 StudentEnrolled.

Construct N Mean SD




Information & Analysis 43 2.51 .95




Table 20. Means by District Size for Districts With Between 999 and 500 Students Enrolled.

Construct N Mean SD







School District Results 25 3.06 1.10


75

Table 21. Means by District Size for Districts With Less Than 499 Students Enrolled.

Construct N Mean SD





Human Resources 19 3.02 1.07



Table 22. Means by Position for Superintendents.

Construct N Mean SD









76

Table 23. Means by Position for Principals.

Construct N Mean SD








Table 24. Means by Position for Teachers.

Construct N Mean SD








The frequencies and percentages for each item in the seven constructs are charted

in Tables 25 through 31. Teacher responses tended to be distributed across all six choices

more frequently than those o f superintendents and principals. This was consistent across

all constructs. Greater frequencies occurred in item #3 through #5, signifying a more


77

all constructs. Greater frequencies occurred in item #3 through #5, signifying a more

developed and refined quality approach on the part o f superintendents and principals,

while greater frequencies occurred in item #1 through #3, indicating a less developed and

a more arbitrary approach by teachers. Teachers also tended to select the “do not know”

response more frequently.

In Table 25 data collected for the Leadership category suggested that for the

districts with 5,000 or more students there was a higher ratio o f superintendents and

principals who saw a clearly communicated, fully deployed direction in their district than

the ratio o f teachers. In the largest districts, 63% o f superintendents responded to item #3

or #5, compared to 59% o f the principals and 42% o f the teachers. In districts with 499

or less students, 88% o f the superintendents responded to item #4 or #5, compared to

60% o f the principals and 25% o f the teachers. Teacher responses occurred more

frequently in item #1 and #2 regarding the existence o f a systematic study process o f the

performance o f their school district than did superintendents or principals. Across all

sizes o f districts, 15% o f the superintendents suggested there were only minimal school

improvement efforts on the part o f district leaders, compared 75% to 44% o f teachers.

There also appeared to be a greater perception on the part o f superintendents and

principals that a participatory approach to management exists in their district. More

teachers than superintendents or principals report that there is little involvement o f

stakeholders in policy development before the local board o f trustees. There was less

variability among the groups when responding to items on Responsibility to Public or

Legal, Ethical Conduct, with most responses across all positions and district sizes

occurring in item #4 and #5.


78

In the Strategic Planning construct in Table 26, more superintendents and

principals responded to item #3 through #5 than did teachers regarding both Strategic

Development and Focus o f the Plan. However, the majority o f all three groups perceived

that their district did not have a well-deployed system for implementing or assessing their

strategic plan. In the Student and Stakeholder Satisfaction construct in Table 27,

teachers in all districts responded to item #1 or #2 describing standardized tests scores as

the primary means o f determining student needs. Superintendents and principals in the

same districts responded with greater frequency to choices #3 through #5. Teachers

reported that there were minimal attempts to determine student and stakeholder

satisfaction, while superintendents reported that more refined attempts existed.

Increased frequencies o f “do not know” responses occurred among teachers in the

Information and Analysis construct in Table 28. Teacher responses occurred with greater

frequency in item #1 and #2, compared to superintendents and principals who replied

with greater frequency to items #3 through #5 regarding the collection, use, and analysis

o f information. In the Human Resources construct in Table 29, teacher perceptions o f the

learning and working climates, work systems, and employee satisfaction were less

positive than the perceptions o f their superintendents and principals. Table 30 illustrates

frequencies and percentages for Educational and Operational Process Management.

Teachers, more often than administrators, perceived that educational programs and

services were primarily designed and delivered based on federal and state regulations,

traditional practices, or test results. Teachers selected the “do not know” response with

greater frequency than superintendents or principals regarding supply and partnering


79

processes. Table 31 illustrates frequencies for the School District Results construct with

similar patterns o f responses.


Reproduced

with perm

ission of the

copyright ow

ner. Further

reproduction prohibited

without

permission.

Table 25. Item Frequency and Percentage o f Response by Position and Size o f District For Items in the Leadership Category.

Supervisors Principals Teachers

ItemsDistrict Size*

1 2 3 4 5District Size*


1 2 3 4 5

1. Clearly communicated direction:

Scale: 1 Frequency 1 2 1 3 1 1 1 2 9 3 4 1 2Percentage 16.7 15.4 7.7 6.8 5.9 7.1 12.5 40.0 16.4 16.7 25,0 25.0 25.0

Scale: 2 Frequency 1 2 4 1 3 2 1 1 7 4 2 2 1Percentage 12.5 15.4 30.8 11.1 6.8 11.8 7.1 12.5 12.7 22.2 12.5 50.0 12.5

Scale: 3 Frequency 2 1 3 3 11 2 2 15 3 2 3Percentage 25.0 16.7 23.1 23.1 25.0 11.8 14.3 27.3 16.7 12.5 37.5

Scale: 4 Frequency 4 3 4 3 4 11 8 4 5 3 17 5 7 1 1Percentage 50.0 50.0 30.8 23.1 44.4 25.0 47.1 28,6 62.5 60,0 30.9 27.8 43.8 25.0 12.5


“Do Not Know” Frequency 1 1 1Percentage 7.1 1.8 5.6

(table continues)

* 1 = 5,000 + enrolled; 2 = 2,500 - 4,999 enrolled; 3 = 1,000 - 2,499 enrolled; 4 = 500 - 999 enrolled; 5 = 1 - 499 enrolled

00o

Reproduced

with perm

ission of the

copyright ow

ner. Further


without

permission.

Table 25, cont’d. Item Frequency and Percentage o f Response by Position and Size o f District For Items in the Leadership Category.

Items


1District Size* 2 3 4 5 1

District Size* 2 3 4 5 1

District Size*2 3 4 5

2. Process to study performance:

Scale: 1 Frequency 3 1 6 3 1 2 12 3 4 1 3Percentage 23.1 11.1 13.6 17.6 7.1 25.0 21.8 16.7 25,0 25.0 37.5

Scale: 2 Frequency 1 1 3 4 1 5 6 5 1 3 17 4 4 2 2Percentage 12.5 16.7 23.1 30.8 11.1 11.4 35.3 35.7 12.5 60.0 30,9 22.2 43.8 50.0 25.0

Scale: 3 Frequency 2 3 7 4 3 12 4 5 1 1 4 4 2 1Percentage 25.0 50.0 53,8 30,8 33.3 27.3 23.5 35.7 12.5 20,0 7.3 22.2 12.5 25.0

Scale: 4 Frequency 5 2 5 2 8 2 1 2 1 10 2 2 1Percentage 62.5 33.3 38.5 22.2 18.2 11.8 7.1 25.0 20,0 18.2 111 12.5 12.5

Scale: 5 Frequency 2 12 2 2 2 7 2 1 2Percentage 22.2 27.3 11.8 14.3 25.0 12.7 11.1 6.3 25.0

“Do Not Know” Frequency 5 3Percentage 9.1 16.7

(table continues)

*1 = 5,000 + enrolled; 2 = 2,500 - 4,999 enrolled; 3 = 1,000 - 2,499 enrolled; 4 = 500 - 999 enrolled; 5 = 1 - 499 enrolled

00

Reproduced

with perm

ission of the

copyright ow

ner. Further


without

permission.



ItemsDistrict Size*



1 2 3 4 5

3. Leadership role in improvement:

Scale. 1 Frequency 6 1 14 6 5 1 3Percentage 13.6 7.1 25,5 33.3 31.3 25.0 37.5

Scale: 2 Frequency 2 5 2 2 1 2 12 2 5 2 2Percentage 15.4 11.4 11.8 14.3 12.5 40.0 21.8 11.1 31.3 50,0 25,0

Scale: 3 Frequency 1 1 6 3 10 6 5 2 1 9 4 2 1 1Percentage 12.5 16.7 46.2 23.1 22.7 35.3 35.7 25.0 20.0 16.4 22.2 12.5 25.0 12.5

Scale: 4 Frequency 4 1 5 4 3 9 3 3 2 1 15 1 2 2Percentage 50,0 16.7 38.5 30.8 33.3 20.5 17.6 21.4 25.0 20.0 27.3 5.6 12.5 25.0

Scale: 5 Frequency 3 4 2 4 6 13 6 3 2 1 4 4 2Percentage 37.5 66.7 15.4 30.8 66.7 29,5 35.3 21.4 25.0 20.0 7.3 22.2 12.5


(table continues)


00K>

73CD

■ o- 5oQ.

Q.s

g> Table 25, cont’d. Item Frequency and Percentage o f Response by Position and Size o f District For Items in the Leadership Category.

(/)■ <f )

o 'oo

CDo Items





o■O-5

cq '17

4. Participatory management:

o

o Scale: 1 Frequency 1 3 1 1 13 3 4 2 2CD—s Percentage 12.5 6.8 5.9 12.5 23.6 16.7 25.0 50.0 25.0~n c

Scale: 2 Frequency 1 3 11 3 3 1 18 6 6 1 4CD

CDPercentage 7.7 23.1 25 17.6 21.3 20.0 32.7 33,3 37.5 25.0 50.0

■o- 5OQ . Scale: 3 Frequency 6 3 8 6 3 7 5 8 4 2 11 4 2C&o

Percentage 75.0 50.0 61,5 46.2 33.3 15.9 29.4 57.1 50.0 40.0 20.0 22.2 12.53

■oo Scale: 4 Frequency 1 4 2 3 11 8 3 3 1 7 2 2 1 1o ;l-H

Percentage 16.7 30.8 15.4 33.3 25.0 47.1 21.4 37.5 20,0 12.7 11.1 12.5 25.0 12.5CDQ .

Scale: 5 Frequency 1 2 2 3 11 1 4 2 1 1O Percentage 12.5 33.3 15.4 33.3 25.0 20,0 7.3 111 6.3 12.5

■OCD “Do Not Know” Frequency 2 1 1oC/)'(/)

Percentage 3.6 5.6 6.3oo

(table continues)


00u>

Reproduced

with perm

ission of the

copyright ow

ner. Further


without

permission.



District Size* District Size* District Size*Items 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5

Board policy:

Scale: 1 Frequency 1 1 1 4 3 1 1 3 10 2 5 3 1Percentage 12.5 7.7 111 9.1 17.6 7.7 12.5 60.0 18.2 111 31.3 75.0 12.5






(table continues)


OO4̂

73CD■o-5oQ.

Q.s

to Table 25, cont’d. Item Frequency and Percentage o f Response by Position and Size o f District For Items in the Leadership Category.3 (/)■ <f )

o 'oo

O’CDO Items





O■o

cq 'O’

6. Responsibility to public:

o

3 Scale: 1 Frequency 1 4 3 1CD—s

T |

Percentage 5.9 7.3 18.8 25.0

C

O ’ Scale: 2 Frequency 1 1 2 2 9 4 2 4 27 6 9 2 4CD—iCD

Percentage 12.5 7.7 15.4 22.2 20,5 28.6 25.0 80.0 49.1 33.3 56,3 50.0 50.0"OoQ . Scale: 3 Frequency 1 1 4 5 1 8 2 3 2 10 4 2 1 1ao

Percentage 12.5 16.7 30.8 38.5 11.1 18.2 11.8 21.4 25.0 18.2 22.2 12.5 25.0 12.5

"Oo Scale: 4 Frequency 6 5 6 5 5 15 12 4 4 4 3 2 1o ;l-HCD

Percentage 75.0 83.3 46.2 38,5 55.6 34.1 70,6 28.6 50.0 7.3 16.7 12.5 12.5Q .

l-HScale: 5 Frequency 2 1 11 1 5 1 9 3

O’ocl-H

Percentage 15.4 11.1 25.0 5.9 21.4 20.0 16,4 16.7■OCDg “Do Not Know” Frequency 1 2c/5'wo

Percentage 1.8 111

o(fable continues)


00KJ\

73CD■o- 5oQ.

Q.s

g> Table 25, cont’d. Item Frequency and Percentage o f Response by Position and Size o f District For Items in the Leadership Category.

(/)■ <f )

o '3O

CDo Items




District Size* 2 3 4 5

O■ocq'17

7. Legal, ethical conduct:

o

o Scale: 1 Frequency 5 2 1CD—s —n

Percentage 9.1 12.5 25.011 C

Scale: 2 Frequency 1 2 5CD

CDPercentage 12.5 4.5 9.1

"O- 5oQ. Scale: 3 Frequency 2 2 1 8 4 1 1 19 5 7 1 2C&o

Percentage 25.0 15.4 11.1 18.2 23.5 7.1 12.5 34.5 27.8 43.8 25.0 25.02■oo Scale: 4 Frequency 2 4 10 8 7 17 7 8 5 4 10 9 3 1 42 ^o;l-H

Percentage 25.0 50.0 76.9 61.5 77.8 38,6 41.2 57.1 62.5 80,0 18.2 50.0 18.8 25.0 50,0CDQ .| Scale: 5 Frequency 3 4 2 3 1 15 5 5 5 1 7 1 4 12 ^O Percentage 37.5 50.0 15.4 23.1 11.1 34.1 29.4 35.7 25.0 20.0 12.7 5.6 25.0 25.0

■OCD3 “Do Not Know” Frequency 1 9 32C/)'C/)o '2

Percentage 2.3 16.4 16.7


00ON

Reproduced

with perm

ission of the

copyright ow

ner. Further


without

permission.

Table 26. Item Frequency and Percentage o f Response by Position and Size o f District For Items in the Strategic Planning Category.



1. Strategic development:

Scale: 1 Frequency 1 1 1 1 1Percentage 7.7 1.8 5.6 6.3 25,0

Scale: 2 Frequency 2 2 1 4 3 11 12.5 2 16 5 6 2 3Percentage 15.4 15.4 11.1 9.1 17.6 78.6 40.0 29.1 27.8 37.5 50.0 37.5

Scale: 3 Frequency 3 3 9 7 4 12 5 2 2 2 20 8 2 1 4Percentage 37.5 50.0 69.2 53.8 44.4 27.3 29.4 14.3 25.0 40,0 36.4 44.4 12.5 25.0 50,0

Scale: 4 Frequency 5 4 2 14 5 1 4 6 2 3Percentage 62.5 30,8 22.2 31.8 29.4 7.1 50.0 10.9 11.1 18.8


“Do Not Know” Frequency 1 1 1 1Percentage 1.8 5.6 6.3 12.5

(table continues)


00

Reproduced

with perm

ission of the

copyright ow

ner. Further


without

permission.

Table 26, cont’d. Item Frequency and Percentage o f Response by Position and Size o f District For Items in the Strategic PlanningCategory.



2. Focus o f plan:

Scale: 1 Frequency 1 1 1 3 2 1 7 2 2 2 3Percentage 7.7 7.7 11.1 6.8 11.8 7.1 12.7 111 12.5 50.0 37.5

Scale: 2 Frequency 1 11 4 3 1 8 4 4 2 2 17 6 5 1Percentage 12.5 16.7 30.8 23.1 111 18.2 23.5 28.6 25.0 40.0 30.9 33.3 31.3 25.0

Scale: 3 Frequency 1 4 4 2 19 2 2 1 2 12 3 5 1Percentage 12.5 30.8 30,8 22.2 22.7 11.8 14.3 12.5 40.0 21.8 16.7 31.3 25.0




(table continues)


0000

Reproduced

with perm

ission of the

copyright ow

ner. Further


without

permission.

Table 26, cont’d. Item Frequency and Percentage o f Response by Position and Size o f District For Items in the Strategic PlanningCategory.



3. Implementation and assessment o f plan:

Scale: 1 Frequency 2 2 3 5 2 6 2 2 2 3 8 5 5 3 1Percentage 25.0 33.3 23.1 38.5 22.2 13.6 11.8 14.3 25.0 60.0 14.5 27.8 31.3 75,0 12.5

Scale: 2 Frequency 1 3 1 5 5 4 1 16 3 3 3Percentage 12.5 23.1 11.1 1 1.4 29.4 28.6 12.5 29.1 16.7 31.3 37.5

Scale. 3 Frequency 3 2 4 3 3 15 4 3 3 2 13 4 3 1 2Percentage 37.5 33.3 30.8 23.1 33.3 34.1 23.5 21.4 37.5 40.0 23.6 22.2 18.8 25,0 25.0

Scale: 4 Frequency 1 1 1 8 3 3 1 11 1 1Percentage 16.7 7.7 11.1 20.5 17.6 21.4 12.5 20.0 6.3 12.5

Scale: 5 Frequency 2 1 1 3 2 9 1 2 1 6 2 1 1Percentage 25.0 16.7 7.7 23.1 22.2 20.5 5.9 14.3 12.5 10.9 111 6.3 12.5

“Do Not Know” Frequency 1 1 1 4 1Percentage 7.7 2.3 1.8 22.2 6.3


oo

Reproduced

with perm

ission of the

copyright ow

ner. Further


without

permission.

Table 27. Item Frequency and Percentage o f Response by Position and Size o f District For Items in the Student Focus andSatisfaction/Stakeholder Categories.



How student needs and expectations are determined:

Scale: 1 FrequencyPercentage

1 2 12.5 15.4

17.7

111.1

36.8

225.0

1629.1

844.4

531.3

375.0

450.0


1 1 3 12.5 16.7 23.1

646.2

222.2

1329.5

1058.8

857.1

450.0

240.0

2240.0

422.2

531.3

337.5


1 3 12.5 23.1

215.4

111.1

1022.7

214.3

225.0

120.0

35.5

2111

212.5


5 5 5 62.5 83.3 38.5

430.8

333.3

1227.3

635.3

428.6

240.0

712.7

211.1

318.8

125.0

112.5


222.2

613.6

47.3

211.1

16.3

“Do Not Know” FrequencyPercentage

35.5

(table continues)


vOO

Reproduced

with perm

ission of the

copyright ow

ner. Further


without

permission.

Table 27, cont’d. Item Frequency and Percentage o f Response by Position and Size o f District For Items in the Student Focus andSatisfaction/Stakeholder Categories.



2. High expectations for performance o f students:

Scale: 1 Frequency 1 3 1 5 2 2 1Percentage 7.7 6.8 5.9 9.1 11.1 12.5 25.0



Scale: 4 Frequency 2 1 2 1 2 7 3 2 16 2 4 2Percentage 25.0 16.7 15.4 7.7 22.2 15.9 17.6 14.3 29.1 11.1 25.0 25.0

Scale: 5 Frequency 1 1 6 1 1 3 2Percentage 7.7 111 13.6 5.9 7.1 5.5 111


(table continues)


Reproduced

with perm

ission of the

copyright ow

ner. Further


without

permission.




3. Student and stakeholder satisfaction:

Scale: 1 Frequency 2 2 1 2 3 2 2 1 1 13.6 9 5 3Percentage 25.0 15.4 7.7 22.2 6.8 11.8 14.3 12.5 20.0 23.6 50.0 31.3 75.0

Scale: 2 Frequency 1 2 4 3 1 17 6 5 2 3 21 4 7 1 3Percentage 12.5 33.3 30.8 23.1 11.1 38.6 35.3 35.7 25.0 60,0 38.2 22.2 43.8 25.0 37.5

Scale: 3 Frequency 2 2 4 5 1 8 3 5 1 7 1 2 1Percentage 25.0 33.3 30.8 38.5 111 18.2 17.6 35.7 12.5 12.7 5.6 12.5 12.5

Scale: 4 Frequency 3 2 3 4 4 14 5 2 2 1 9 1 1Percentage 37.5 33.3 23.1 30.8 44.4 31.8 29,4 14.3 25.0 20.0 16.4 5.6 12.5

Scale: 5 Frequency 1 2 2 1 2 1Percentage 111 4.5 25.0 1.8 11.1 6.3


(table continuesj

*1 = 5,000 + enrolled; 2 = 2,500 - 4,999 enrolled; 3 = 1,000 - 2,499 enrolled; 4 = 500 - 999 enrolled; 5 = 1 - 499 enrolledvOto

Reproduced

with perm

ission of the

copyright ow

ner. Further


without

permission.




4. Future needs o f students and stakeholders:


Scale: 2 Frequency 1 1 1 6 5 2 8 5 6 1 2Percentage 12.5 7.7 111 13.6 29.4 40.0 14.5 27.8 37.5 25.0 25.0



Scale: 5 Frequency 1 2 3 6 1 1 1 5 2 1Percentage 7.7 15.4 33,3 13.6 5.9 7.1 12.5 9.1 11.1 12.5

“Do Not Know” Frequency 1 1 4 2Percentage 111 2.3 7.3 111


v£>U)

Reproduced

with perm

ission of the

copyright ow

ner. Further


without

permission.

Table 28. Item Frequency and Percentage o f Response by Position and Size o f District For Items in the Information and AnalysisCategory.

Items

Supervisors Principals T eachers




1. Selection and use;



Scale: 3 Frequency 3 2 7 5 10 5 2 1 6 2 2 1 1Percentage 37.5 33.3 53.8 38,5 22.7 29.4 14.3 20.0 10.9 111 12.5 25.0 12.5


Scale: 5 Frequency 1 1 1 1 1 6 1 3 1Percentage 12.5 16.7 7.7 7.7 11.1 13.6 7.1 5.5 6.3

“Do Not Know” Frequency 1 5 3 1Percentage 2.3 9.1 16.7 12.5

0able continues)

* 1 = 5,000 + enrolled; 2 = 2,500 - 4,999 enrolled; 3 = 1,000 - 2,499 enrolled; 4 = 500 - 999 enrolled; 5 = 1 - 499 enrolledvO-fc.

Reproduced

with perm

ission of the

copyright ow

ner. Further


without

permission.

Table 28, cont’d. Item Frequency and Percentage o f Response by Position and Size o f District For Items in the Information andAnalysis Category.

Supervisors Principals T eachers

ItemsDistrict Size*



1 2 3 4 5

2. Selection and use o f comparative data:

Scale: 1 Frequency 1 1 2 3 2 6 1 1 7 3 3 3 2Percentage 12.5 16.7 15.4 23.1 22.2 13.6 7.1 20,0 12.7 16.7 18.8 75.0 25.0

Scale: 2 Frequency 1 5 5 2 9 10 5 3 2 15 5 8 1Percentage 12.5 38,5 38.5 22.2 20.5 58.8 35.7 37.5 40.0 27.3 27.8 50.0 12.5


Scale: 4 Frequency 3 1 1 1 6 3 2 3 1 8 2 1 1 1Percentage 37.5 16.7 7.7 11.1 13.6 17.6 14.3 37.5 20.0 14.5 111 6.3 25.0 12.5

Scale: 5 Frequency 2 1 1 8 5 1Percentage 33.3 7.7 111 18.2 9.1 6.3

“Do Not Know” Frequency 2 1 11 4 2 1Percentage 4.5 7.1 20.0 22.2 12.5 12.5

(table continues)

* 1 = 5,000 + enrolled; 2 = 2,500 - 4,999 enrolled; 3 = 1,000 - 2,499 enrolled; 4 = 500 - 999 enrolled; 5 = 1 - 499 enrolledvO

Reproduced

with perm

ission of the

copyright ow

ner. Further


without

permission.

Table 28, cont’d. Item Frequency and Percentage o f Response by Position and Size o f District For Items in the Information andAnalysis Category.


ItemsDistrict Size*



1 2 3 4 5

3. Analysis and use o f school performance data:



Scale: 3 Frequency 3 1 2 4 2 9 6 3 2 1 7 4 1Percentage 25.0 16.7 15.4 30.8 22.2 20.5 35.3 21.4 25,0 20.0 12.7 25.0 12.5

Scale. 4 Frequency 4 2 3 9 4 2 3 1 10 3 1 2Percentage 50.0 15.4 23.1 20.5 23.5 14.3 37.5 20.0 18.2 16.7 6.3 25,0

Scale: 5 Frequency 3 1 1 4 8 1 2 7 2 1Percentage 50.0 7.7 7.7 44.4 18.2 5.9 14.3 12.7 11.1 6.3

“Do Not Know” Frequency 2 1 2 7 3 3Percentage 4.5 7.1 40.0 12.7 16.7 18.8


vOON

Reproduced

with perm

ission of the

copyright ow

ner. Further


without

permission.

Table 29. Item Frequency and Percentage o f Response by Position and Size o f District For Items in the Human Resource Developmentand Management Category.


Items 1District Size* 2 3 4 5 1



1. Learning and working climate:

Scale. 1 FrequencyPercentage

49.1

112.5

1425.5

5 4 2 2 27.8 25.0 50.0 25.0


112.5

2 1 15.4 7.7

1125.0

3 2 17.6 14.3

120.0

1120.0

4 5 1 22.2 31.3 12.5


225.0

1 6 4 16.7 46.2 30.8

111.1

1431.8

6 4 2 35.3 28.6 25.0

240.0

1934.5

4 2 1 4 22.2 12.5 25.0 50.0


562.5

3 4 6 50.0 30.8 46.2

444.4

920.5

4 5 5 23.5 35,7 62.5

240.0

814.5

4 5 1 1 22.2 31.3 25.0 12.5


2 1 2 33.3 7.7 15.4

444.4

613.6

3 3 17.6 21.4

35.5

15.6


(table continues)

*1 = 5,000 + enrolled; 2 = 2,500 - 4,999 enrolled; 3 = 1,000 - 2,499 enrolled; 4 = 500 - 999 enrolled; 5 = 1 - 499 enrolledvO

Reproduced

with perm

ission of the

copyright ow

ner. Further


without

permission.

Table 29, cont’d. Item Frequency and Percentage o f Response by Position and Size o f District For Items in the Human ResourceDevelopment and Management Category.



Work systems;

Scale; 1 Frequency 1 6 1 2 1 1 18 5 5 2 5Percentage 7.7 13.6 5.9 14.3 12.5 20,0 32.7 27.8 31.3 50,0 62.5


Scale: 3 Frequency 5 3 6 3 1 10 4 3 1 1 7 5 4Percentage 62.5 50.0 46.2 23.1 11.1 22.7 23.5 21.4 12.5 20,0 12.7 27.8 25.0

Scale: 4 Frequency 3 1 1 5 7 15 7 5 2 2 11 2 2 1 1Percentage 37.5 16.7 7.7 38.5 77.8 34.1 41.2 35.7 25.0 40.0 20,0 111 12.5 25.0 12.5

Scale: 5 Frequency 2 1 1 4 1 1Percentage 33.3 7.7 11.1 9.1 5.9 12.5


2 1 43.6 5.6 25.0

(table continues)

*1 = 5,000 + enrolled; 2 = 2,500 - 4,999 enrolled; 3 = 1,000 - 2,499 enrolled; 4 = 500 - 999 enrolled; 5 = 1 - 499 enrolled\o00

Reproduced

with perm

ission of the

copyright ow

ner. Further


without

permission.




2. Work systems:

Scale. 1 Frequency 1 6 1 2 1 1 18 5 5 2 5Percentage 7.7 13.6 5.9 14.3 12.5 20.0 32.7 27.8 31.3 50.0 62.5

Scale: 2 Frequency 4 4 9 3 4 3 1 17 5 5 1 2Percentage 30.8 30.8 20.5 17.6 28.6 37.5 20,0 30.9 27.8 31.3 25.0 25.0

Scale. 3 Frequency 5 3 6 3 1 10 4 3 1 1 7 5 4Percentage 62.5 50.0 46.2 23.1 11.1 22.7 23.5 21.4 12.5 20,0 12.7 27.8 25.0

Scale: 4 Frequency 3 1 1 5 7 15 7 5 2 2 11 2 2 1 1Percentage 37.5 16.7 7.7 38.5 77.8 34.1 41.2 35.7 25.0 40.0 20.0 111 12.5 25.0 12.5

Scale: 5 Frequency 2 1 1 4 1 1Percentage 33.3 7.7 11.1 9.1 5.9 12.5


(table continues)

* 1 = 5,000 + enrolled; 2 = 2,500 - 4,999 enrolled; 3 = 1,000 - 2,499 enrolled; 4 = 500 - 999 enrolled; 5 = 1 - 499 enrolledvO\o

Reproduced

with perm

ission of the

copyright ow

ner. Further


without

permission.



ItemsDistrict Size*



1 2 3 4 5

Personal training and development:


17.7

920.5

214.3

112.5

1730.9

63.3

8 3 50.0 75.0

337.5


112.5

116.7

323.1

646.2

1111

511.4

317.6

214.3

225.0

240.0

916.4

316.7

16.3

337.5


450.0

116.7

538.5

215.4

111.1

1022.7

635.3

321.4

112.5

240.0

1221.8

422.2

16.3


225.0

116.7

430.8

323.1

666.7

1329.5

635,3

535.7

450.0

120.0

814.5

422.2

125.0

112.5

Scale; 5 FrequencyPercentage

112.5

350.0

215.4

111.1

715.9

15.9

214.3

712.7

15.6


23.6

(table continues)


oo

Reproduced

with perm

ission of the

copyright ow

ner. Further


without

permission.




4. Performance appraisal:


Scale: 2 Frequency 5 1 3 6 1 21 6 5 3 3 20 5 9Percentage 62.5 16.7 23.1 46.2 11.1 47.7 35.3 35.7 37.5 60,0 36.4 27.8 56.3


Scale: 4 Frequency 1 3 215. 1 1 11 2 3 1 8 1 1Percentage 12.5 50.0 4 7.7 11.1 25.0 11.8 21.4 12.5 14.5 5.6 6.3

Scale: 5 Frequency 1 2 3 2 1 3 1Percentage 7.7 22.2 6.8 11.8 12.5 5.5 5.6

“Do Not Know” Frequency 2 2Percentage 3.6 11.1

(table continues)


o

Reproduced

with perm

ission of the

copyright ow

ner. Further


without

permission.




5. Employee satisfaction:

Scale: 1 Frequency 6 3 21 8 7 3 3Percentage 13,6 21.4 38.2 44.4 43.8 75.0 37.5




Scale: 5 Frequency 1 1 1 3 4 1 3 3Percentage 12.5 7.7 7.7 33.3 9.1 12.5 5.5 16.7

“Do Not Know” Frequency Percentage


oto

Reproduced

with perm

ission of the

copyright ow

ner. Further


without

permission.

Table 30. Item Frequency and Percentage o f Response by Position and Size o f District For Items in the Educational and OperationalProcess Management Category.





1. Design o f educational programs:


613.6

1 15.9 7.1

610.9

633.3

3 3 4 18.8 75.0 50.0


1 5 3 16.7 38.5 23.1

49.1

4 2 3 23.5 14.3 37.5

120.0

1629.1

15.6

7 1 3 43.8 25.0 37.5


562.5

2 5 7 33,3 38.5 53,8

111.1

1534.1

7 4 1 41.2 28.6 12.5

240.0

1730.9

422.2


225.0

3 3 50.0 23.1

777.8

1738.6

3 6 3 17.6 42.9 37.5

240.0

47.3

527.8

4 1 25.0 12.5


112.5

323.1

111.1

24.5

1 17.1 12.5

712.7

15.6

16.3


59.1

15.6

16.3

(table continues)


oU>

Reproduced

with perm

ission of the

copyright ow

ner. Further


without

permission.

Table 30. cont’d. Item Frequency and Percentage o f Response by Position and Size o f District For Items in the Educational andOperational Process Management Category.


ItemsDistrict Size*



1 2 3 4 5

2. Delivery o f educational programs:


Scale: 2 Frequency 2 5 3 6 6 3 3 2 2 0 3 7 2 2Percentage 33.3 38.5 23.1 13.6 35.3 21.4 37.5 40.0 36.4 16.7 43.8 50,0 25.0

Scale: 3 Frequency 3 1 5 4 1 15 3 4 2 1 10 2 1

Percentage 37.5 16.7 38,5 30.8 1 1 1 34.1 17.6 28.6 25.0 2 0 . 0 18.2 11.1 6.3

Scale: 4 Frequency 3 2 2 3 6 14 5 5 1 2 6 5 4 2

Percentage 37.5 33,3 15.4 23.1 66.7 31.8 29.4 35.7 12.5 40.0 10.9 27.8 25.0 25.0

Scale: 5 Frequency 2 1 2 2 4 1 1 6 1

Percentage 25.0 16.7 15.4 22.2 9.1 7.1 12.5 10.9 5.6

“Do Not Know” Frequency 4 2Percentage 7.3 1 1 1

(table continues)


oa

Reproduced

with perm

ission of the

copyright ow

ner. Further


without

permission.

Table 30, cont’d. Item Frequency and Percentage o f Response by Position and Size o f District For Items in the Educational andOperational Process Management Category.



3. Design and delivery o f educational support services:


2

15.47

15.91 1

12.5 2 0 . 0

13 223.6

41 1 1

2

25.01

50.0 12.5


337.5

233.3

753.8

538.5

333.3

715.9

635.3

642.9

337.5

12 0 . 0

1730.9

633.3

743.8

250.0

450,0


337.5

116.7

323.1

430.8

555.6

920,5

529.4

2

14.33

37.51

2 0 , 0

35.5

316.7

16.3


112.5

350,0

323.1

17.7

1I I I

1227.3

529.4

642.9

112.5

112 0 . 0

316.7

212.5

225.0


1

12.51

7.78

18.22

40.035.5

1

5.6


12.3

8

14.53

16.72

12.5

(table continues)


o

Reproduced

with perm

ission of the

copyright ow

ner. Further


without

permission.




4. Data and information processes:

Scale: 1 Frequency 1 1 1 2 4 1 2 2 1 16 5 2 3 2Percentage 12.5 16.7 7.7 15.4 9.1 5.9 14.3 25.0 2 0 . 0 29.1 27.8 12.5 75,0 25.0

Scale: 2 Frequency 2 4 7 2 8 7 4 1 3 18 6 10 1 3Percentage 25.0 30.8 53.8 2 2 . 2 18.2 41.2 28.6 12.5 60,0 32.7 33.3 62.5 25.0 37.5

Scale: 3 Frequency 3 3 7 2 2 14 4 1 1 5 5 1Percentage 37.5 50.0 53.8 15.4 2 2 . 2 31.8 23.5 7.1 12.5 9.1 27.8 6.3

Scale: 4 Frequency 1 2 1 2 3 15 2 6 3 1 8 1 1Percentage 12.5 33.3 7.7 15.4 33.3 34.1 11 .8 42.9 37.5 2 0 . 0 14.5 6.3 12.5

Scale: 5 Frequency 1 1 1 2 1 1 3Percentage 12.5 16.7 1 1 1 4.5 7.1 12.5 5.5

“Do Not Know” Frequency 1 1 5 2 2Percentage 11.1 2.3 9.1 11.1 12.5

(table continues)


oON

Reproduced

with perm

ission of the

copyright ow

ner. Further


without

permission.




Communication processes:

Scale: 1 Frequency 1 1 4 2 1 9 5 3 2 1Percentage 16.7 7.7 9.1 14.3 2 0 . 0 16.4 27.8 18.8 50.0 12.5

Scale: 2 Frequency 1 1 3 8 2 2 2 16 4 7 1Percentage 12.5 7.7 23.1 18.2 11.8 25.0 40.0 29.1 2 2 . 2 43.8 25.0


Scale: 4 Frequency 2 1 3 1 4 13 5 4 1 2 6 3 3 1 2Percentage 25.0 16.7 23.1 7.7 44.4 29.5 29.4 28.6 12.5 40.0 10,9 16.7 18.8 25.0 25.0

Scale: 5 Frequency 1 4 3 2 2Percentage 11.1 9.1 21.4 25.0 3.6

‘Do Not Know” Frequency Percentage 12.5

(table continues)


o

Reproduced

with perm

ission of the

copyright ow

ner. Further


without

permission.




6 . Supplier and partnering processes:

Scale: 1 Frequency 3 1 3 1 1 6 2 2Percentage 23.1 7.7 6 . 8 7.1 12.5 10.9 12.5 50.0

Scale: 2 Frequency 2 3 4 5 4 7 8 4 1 3 10 3 11 1 3Percentage 25,0 50.0 30,8 38.5 44.4 15.9 47.1 28.6 12.5 60.0 18.2 16.7 6 8 . 8 25.0 37.5

Scale: 3 Frequency 3 1 5 6 1 12 6 5 2 5 4 1 2

Percentage 37.5 16.7 38.5 46.2 1 1 1 27.3 35.3 35.7 25.0 9.1 2 2 . 2 6.3 25.0

Scale. 4 Frequency 2 1 1 1 1 9 4 2 2 1 6 1Percentage 25,0 16.7 7.7 7.7 11.1 20.5 11.8 14.3 25.0 2 0 . 0 10.9 6.3

Scale: 5 Frequency 1 1 2 9 1 1 3Percentage 12.5 16.7 2 2 . 2 20.5 12.5 2 0 . 0 5.5

“Do Not Know” Frequency 1 4 2 1 25 11 1 1Percentage 11.1 9.1 14.3 12.5 45.5 61.1 6.3 25.0


o00

Reproduced

with perm

ission of the

copyright ow

ner. Further


without

permission.

Table 31. Item Frequency and Percentage o f Response by Position and Size o f District For Items in the School District PerformanceResults Category.



1. Student performance results:


112.5

1

7.717.7

35.5

15.6

318.8

125.0

112.5


233.3

321.1

430,8

22 2 . 2

715.9

6

35.35

35.72

25.01

2 0 . 0

1934.5

8

44.45

31.32

50.05

62.5


225,0

466.7

538.5

753.8

555.6

1431.8

6

35.35

35.73

37.52

40.012

2 1 . 84

2 2 . 2

425,0

2

25.0


562.5

323.1

1

11.1

2352.3

529.4

321.4

225.0

12 0 . 0

1832.7

316.7

425.0

125.0


1

7.71

7.71

1 1 1

1

12.51

2 0 , 0

2

3.6


17.1

11.8

2

11.1

(table continues)


oVO

Reproduced

with perm

ission of the

copyright ow

ner. Further


without

permission.

Table 31, cont’d. Item Frequency and Percentage o f Response by Position and Size o f District For Items in the School DistrictPerformance Results Category.


ItemsDistrict Size*



1 2 3 4 5

2 . Student conduct results:


1

12.52

15.41

7.72

4.515.9

1527.3

2

11.1

2

12.51

25.01

12.5


1

12.51

16.74

30.817.7

1

11.1

818.2

317.6

916.4

42 2 . 2

531.3

225.0


1

12.53

50.04

30.83

23.12

2 2 . 2

8

18.23

17.65

35.71

12.51

2 0 . 0

11

2 0 . 0

2

11.1

531.3

1

25.02

25,0


450.0

2

33.32

15.44

30.84

44.41431.8

6

35.34

28.64

50.03

60.07

12.72

11.1

16.3

112.5


1

7.73

23.12

2 2 . 2

36 .8

2

14.32

25.01

2 0 . 0

1

1.815.6

125.0

1

12.5


112.5

17.7

920.5

317.6

321.4

112.5

122 1 . 8

738,9

318.8

125.0

112.5

(table continues)


o

Reproduced

with perm

ission of the

copyright ow

ner. Further


without

permission.






3. Student and stakeholder satisfaction results:


1

16.71

6.349.1

5 4 9.1

1

2 2 . 2

1

6.31

25.0 12.5


16.3

112.5

712.7

1

6.31 2

25.0 25.0


337.5

350.0

538.5

425.0

555.6

1431.8

741.2

428.6

225.0

240.0

1323.6

316.7

425.0

225,0


450.0

233.3

430.8

16.3

22 2 . 2

1022.7

87.1

642,9

337.5

360.0

814.5

633.3

16.3

125.0

225.0


1

12.51

7.79

56.32

2 2 . 2

511.4

2

14.31

12.535.5


323.1

10

22.71

5.91

7.11

12.51934.5

527.8

956.3

1

25.01

12.5

(table continues)


Reproduced

with perm

ission of the

copyright ow

ner. Further


without

permission.



ItemsDistrict Size*



1 2 3 4 5

4. Human resource results:

Scale: 1 Frequency 1 3 1 6 1 1 2 1Percentage 6.3 6 . 8 5.9 10.9 5.6 6.3 50.0 12.5

Scale: 2 Frequency 3 4 6 12 2 18 5 7 4 3 25 7 12 1 2

Percentage 37.5 66.7 46.2 75.0 22.2 40.9 29.4 50,0 50.0 60.0 45.5 38.9 75.0 25.0 25.0

Scale: 3 Frequency 2 1 5 2 2 1 6 2 2 1

Percentage 25.0 7.7 11.4 11.8 14.3 2 0 . 0 10.9 11.1 12.5 12.5

Scale: 4 Frequency 1 4 2 3 8 5 1 1 7 3 1Percentage 12.5 30.8 12.5 33.3 18.2 29.4 12.5 2 0 . 0 12,7 16.7 12.5

Scale: 5 Frequency 2 2 2 4 6 4 2 5 1 1 1

Percentage 25,0 33.3 15.4 44.4 13.6 28.6 25.0 9.1 5.6 25.0 12.5

“Do Not Know” Frequency 1 4 1 1 6 4 1 1Percentage 6.3 9.1 7.1 12.5 10.9 2 2 . 2 6.3 12.5

(table continues)


Reproduced

with perm

ission of the

copyright ow

ner. Further


without

permission.


Items





5. Educational program and service results:

Scale: 1 Frequency 2 1 5 2 1 2 2 12 5 4 2 2

Percentage 25.0 16.7 11.4 11.8 7.1 25.0 40.0 2 1 . 8 27.8 25.0 50.0 25.0

Scale: 2 Frequency 1 3 1 1 8 4 1 1 2

Percentage 12.5 23.1 2.3 5.9 14.5 2 2 . 2 6.3 25.0 25.0

Scale: 3 Frequency 3 2 4 3 10 7 5 1 10 3 4 1

Percentage 37.5 33.3 30.8 33.3 22.7 41.2 35.7 2 0 . 0 18.2 16.7 25.0 12.5

Scale: 4 Frequency 2 3 5 4 18 4 4 4 1 10 3 3 1 2

Percentage 25.0 50.0 38.5 44.4 40.9 23.5 28.6 50.0 2 0 . 0 18.2 16.7 18.8 25.0 25.0

Scale: 5 Frequency 2 2 4 2 2 1 1 1

Percentage 15.4 2 2 . 2 9.1 14.3 25.0 2 0 , 0 1.8 12.5

“Do Not Know” Frequency 1 6 1 2 13 3 4Percentage 7.7 13.6 5.9 14.3 23.6 16.7 25.0

(table continues)


Reproduced

with perm

ission of the

copyright ow

ner. Further


without

permission.






6 . Educational support services results:


2

33.31

2.32

11.8

1

12.547.3

1

5.61

25.01

12.5


112.5

4 1 66.7 7.7

1

11.1

715.9

11 3 64.7 21.4

59.1

2

11.1

3 1 18.8 25.0

450.0


450.0

430.8

333,3

511.4

3 1 17.6 7.1

1

2 0 . 0

10

18.24

2 2 . 2

6

37.53

37.5


337.5

861.5

2

2 2 . 21022.7

428.6

112.5

12 0 . 0

1018.2

316.7


1i l l

715.9

535.7

450.0

2

40.01

25.0


2

2 2 . 2

1431.8

1

7.12

25.01

2 0 . 0

2545.5

8

44.47 1

43.8 25.0


115

Inferential Statistical Analysis

A general linear models procedure was applied, and a 3 x 5, two-way factorial

analysis o f variance was used to test the hypotheses. The assumptions o f homogeneity

o f variance, independence o f variables, and continuous dependent variables were met.

Type III sum o f squares for each construct was used since there were different numbers

in the cells.

The null hypothesis for the Leadership construct was, “Hoi: There are no

significant differences in the Leadership category o f the Performance Analysis for School

Districts by type or size.” Table 32 illustrates a significant difference was found with the

variable position at .001. No significant differences for p values at .05 were found for

district size or interaction. Scheffe’s post hoc test for multiple comparisons was done.

This test is the most conservative measure and controls for Type 1 error rate. The alpha

level o f .05 determined that there were no significant differences when comparing

superintendents and principals. However, there was a significant difference at .05 when

teachers were compared to both superintendents and principals. Therefore, the null

hypothesis is rejected.


116

Table 32. Two-Way ANOVA Leadership Construct.

Source d f SS MS F value Pr > F

Model:Size 4 1.94558390 0.48639598 0.59 0.6736

Type 2 25.06159973 12.53079986 15.08 0 . 0 0 1

Size x Type 8 2.71734755 0.33966844 0.41 0.9148

Error 218 181.14322972 0.83093225

Total 232 221.53565721

The null hypothesis for the Strategic Planning construct was, “H0 2 : There are no

significant differences in the strategic planning category o f the Performance Analysis for

School Districts by type or size.” Table 33 illustrates a significant difference was found

with the variable position at .001. No significant differences for p values at .05 were

found for district size or interaction. Scheffe’s post hoc test for multiple comparisons

was done. The alpha level o f .05 determined that there were no significant differences

when comparing superintendents and principals. There was a significant difference,

however, at .05 when teachers were compared to both superintendents and principals.

Therefore, the null hypothesis is rejected.


117

Table 33. Two-Way ANOVA Strategic Planning Construct.

Source df SS MS F value Pr > F

Model:

Size 4 8.04564589 2.01141147 1.75 0.1397

Type 2 22.17052545 11.08526273 9.66 0 . 0 0 1

Size x Type 8 7.73834769 0.96729346 0.84 0.5661

Error 218 250.27735473 1.14806126

Total 232 287.11826419

The null hypothesis for the Student and Stakeholder Satisfaction construct was,

“H03 : There are no significant differences in Student and Stakeholder Satisfaction

category o f the Performance Analysis for School Districts by type or size.” Table 34

illustrates a significant difference was found with the variable position at .001. No

significant differences for p values at .05 were found for district size or interaction.

Scheffe’s post hoc test for multiple comparisons was done. The alpha level o f .05

determined that there was no significant difference when comparing superintendents and

principals. However, the difference between the mean o f the teachers and principals and

the mean o f teachers and superintendents was large enough to be significant at .05.


The null hypothesis for the Information and Analysis construct was, “Ho4: There

are no significant differences in the Information and Analysis category o f the

Performance Analysis for School Districts by type or size.” Table 35 illustrates a

significant difference was found for position at .001. No significant differences for the p


118

values at .05 were found for district size or interaction. Scheffe’s post hoc test for

multiple comparisons was done. The alpha level o f .05 determined that there was no

significant difference when comparing superintendents and principals. However, the

difference between the mean o f the teachers and principals and the teachers and

superintendents was large enough to be significant at .05. Therefore, the null hypothesis

is rejected.

Table 34. Two-Way ANOVA Student/Stakeholder Satisfaction Construct.


Model:

Size 4 3.81446595 0.95361649 1.27 0.2820

Type 2 18.42615628 9.21307814 12.29 0.001

Size x Type 8 5.01819420 0.62727427 0.84 0.571

Error 219 164.20135256 0.74977787

Total 233 188.16503443

The null hypothesis for the Human Resource Development and Management

construct was, “H0 5 : There are no significant differences in the Human Resource

Development and Managemen/ category o f the Performance Analysis for School Districts

by type or size.” Table 36 illustrates a significant difference was found for factor

position at .001. No significant differences for the p values at .05 were found for district

size or interaction. Scheffe’s post hoc test for multiple comparisons was done. The alpha

level o f .05 determined that there was no significant difference when comparing


119

superintendents and principals. The difference between the mean o f the teachers and

principals and the teachers and superintendents was large enough to be significant at .05.


Table 35. Two-Way ANOVA Information and Analysis Construct.


Model:

Size 4 5.10247757 1.27561939 1.05 0.3822

Type 2 22.98733438 11.49366719 9.46 0.001

Size x Type 8 5.87610935 0.73451367 0.60 0.7735

Error 222 269.64388447 1.21461209

Total 236 301.32301922

Table 36. Two-Way ANOVA Human Resource Development / Management Construct.


Model:

Size 4 3.30666687 0.82666672 1.02 0.3975

Type 2 36.47055806 18.23527903 22.51 0.001

Size x Type 8 6.18997506 0.77374688 0.96 0.4721

Error 218 176.57388647 0.80997196

Total 232 227.56105150


120

The null hypothesis for the Educational and Operational Process Management

construct was, “Ho6: There are no significant differences in the Educational and

Operational Process Management category o f the Performance Analysis for School

Districts by type or size.” Table 37 illustrates a significant difference was found for

position at .001. No significant difference for the p values at .05 was found for district

size or interaction. Scheffe’s post hoc test for multiple comparisons was done. The alpha

level o f .05 determined that there were no significant differences when comparing

superintendents and principals. The difference between the means o f the teachers and

principals and the teachers and superintendents was large enough to be significant at .05.


Table 37. Two-Way ANOVA Educational / Operational Process Management Construct.


Model:

Size 4 4.86845737 1.21711434 1.59 0.1772

Type 2 21.22474914 10.61237457 13.89 0.001

Size x Type 8 7.513 14414 0.93914302 1.23 0.2830

Error 213 162.69105458 0.76380777

Total 227 196.23901925

The null hypothesis for the School District Results construct was, “H0 7 : There are

no significant differences in the school district results category o f the Performance

Analysis for School Districts by type or size.” Table 38 illustrates a significant


121

difference was found for factor position at .001. No significant difference for the p

values at .05 was found for factor district size or interaction o f the factors. Scheffe’s post

hoc test for multiple comparisons was done. The alpha level o f .05 determined that there

were no significant differences when comparing superintendents and principals. The

difference between the means o f the teachers and principals and the teachers and

superintendents was large enough to be significant at .05. Therefore, the null hypothesis

is rejected.

Table 38. Two-Way ANOVA School District Performance Results Construct.


Model:

Size 4 4.02605474 1.00651369 1.37 0.2444

Type 2 19.01940018 9.50970009 12.97 0.001

Size x Type 8 4.73059064 0.59132383 0.81 0.5974

Error 214 156.88729215 0.73311819

Total 228 189.84039301

Analysis o f “Do Not Know” Responses

Chi square analysis was done to analyze the “Do Not Know” responses (See

Table 39). There were no significant differences when positions were combined and

district sizes were compared for responses to the “Do Not Know” choice. When districts

were combined and positions were compared, significant differences were found at the

.05 level. Significant differences were found in all but the Strategic Planning category.

Teachers were more likely to select the “Do Not Know” category than superintendents or


122

Teachers were more likely to select the “Do Not Know” category than superintendents or

principals. In the Strategic Planning category, the p values was .061, which could be

considered significant if alpha was set at .10.

Table 39. Chi Square Analysis for “Do Not Know” Responses.

Construct DF Value Prob


Strategic Planning 4 8.989 .061

Student Stakeholder Satisfaction 4 16.521 .002


Human Resource 4 9.783 .044



Usefulness o f the Instrument as a Tool

All educator types and district sizes were combined to analyze the perceived

usefulness o f the instrument for self-study o f their school district (See Table 40). A total

o f 81% o f the sample found the instrument to have som e use. Thirteen percent found it to

have little or no use. Seventy-seven percent o f the sample responded positively to the

potential use o f the instrument for school improvement.


123

Table 40. Combined Percentages for Usefulness o f Instrument.

Extremely Somewhat Little No

As a study tool 24.4 61.8 13.0 .4

As a tool for school improvement 25.2 52.1 12.6 5.9

Comments

Qualitative analysis was done using a constant comparative approach for

emerging categories o f focus in the written comments o f the respondents. Two categories

o f comment emerged :

1. Usefulness o f instrument in school improvement process.

2. Perceptions o f current climate and approach to school improvement.

For the first category o f written comments, respondents who felt the instrument to

have potential utility commented on its thoroughness, scope, and organization. Some

suggested it could assist in focusing attention to specific components o f the organization

and could force reflective thought about school improvement. Some indicated that it

could be a “rubric” for analysis, while others indicated it could show “growth” and

development in school improvement. Those who felt that the instrument had little or no

use commented that it was too long, complicated, and time consuming. Several

comments were related to use o f “jargon.” Two respondents suggested that since each

district was different, one instrument would not be appropriate. The responses o f

participants from the smaller size districts indicated that this approach would not be

necessary in their district.


124

For the second category o f written comments dealing with perceptions o f the

current climate and approach to school improvement, the majority o f comments had to do

with respondent’s opinions regarding school improvement and how it is designed in their

district. There were several comments, mostly by teachers, which communicated a lack

o f confidence that their opinions would be seriously considered in decision-making or

that their involvement would be desired by the administration. The teachers were also

more likely to comment on their lack o f access to information needed to answer some o f

the items on the instrument. There were several comments which reflected frustration

with school change and the lack o f resources to do anything about it. Teachers also

commented that they were unaware o f how school improvement and decisions were made

currently.

Summary

The instrument was determined to be a reliable tool for internal consistency. The

results o f this study found a significant difference between the perceptions o f teachers

and those o f superintendents and principals related to seven constructs in a school district

organization. No significant differences were found between superintendents and

principals or among the five sizes o f school districts in Idaho. No significant interaction

between the factors o f position and size o f district was found. The majority o f

respondents felt the instrument could be potentially useful for self-study o f their school

district.


125

Chapter 5

Summary, Conclusions, and Recommendations

Summary

The evolution o f school reform calls for an increased emphasis on a clear sense o f

focus, clear expectations from stakeholders, analysis o f root causes o f the problems, and

the realization that the call for increased evidence is not subsiding. School reform suffers

from too much scatter and the absence o f a framework, the implementation o f solutions,

and the measurement o f the effectiveness o f programs (Consortium on Productivity in

Schools, 1995). The lessons learned from reform to date indicate poorly defined

problems, lack o f process for improvement, the absence o f measurement and the

intractability o f the educational system are significant factors for the lack o f impact

(Bernhardt, 1994; Deming, 1994; English & Hill, 1994; Fields, 1994; Fullan, 1992;

Fullan, 1993; Fullan, 1997; Fullan and Miles, 1992; Sagor, 1995; Scholtes, 1995; Senge,

1990). The emerging emphasis on improving the performance o f organizations is to

adopt a systems model. Increasingly, employees and consumers are looking for the

interconnectedness o f work function, process, and productivity (Senge, 1990).

Continuous improvement is not a new approach for some businesses or service

agencies such as hospitals. Quality theory and practice have gone through an evolution,

much like school reform. Early stages involved costly and inefficient inspection-based

practices (Wu, 1996). Quality control systems emerged that identified defects upon

inspection. As quality theory and application developed, a focus on prevention emerged.

This emphasis led to an emphasis on the proactive assessment o f the organization in order

to improve management systems, service, and product quality. What emerged was a need


126

to learn from our work organization and understand the problem before the solution was

prescribed.

The Malcolm Baldrige National Quality Award was originally developed to

motivate businesses to excel in productivity and operations as well as to recognize their

performance. Award winners were obligated to share their lessons with other companies

who wished to achieve the same high standard o f performance. An unanticipated result

o f the award process was the development o f criteria for the organizational self-study o f

performance for the purpose o f improvement (Caravatta, 1997).

Models o f school district performance have traditionally been inspection-based

and have reflected a minimal standard o f compliance. Effective models for

demonstrating performance were needed, with clearly defined indicators o f excellence

that reflected the comprehensive scope o f the school district as an interconnected system.

This study examined the current literature on organizational improvement and traditional

approaches used by schools to assess their effectiveness, and proposed a model for

improvement grounded in quality theory. The research also investigated the perceptions

o f Idaho educators about the performance o f their school districts, using an instrument

adapted from the Malcolm Baldrige National Quality Award, and based on a systems

model and quality theory.

Conclusions

The instrument developed— Performance Analysis for School Districts— was

found to be reliable and demonstrate internal consistency. The study examined

perceptual differences o f current performance all seven areas and the influence o f two

variables— educator position and size o f district— were explored. Hypotheses for the


127

categories o f Leadership, Strategic Planning, Student and Stakeholder Satisfaction,

Information and Analysis, Human Resource Development, Educational Process

Management, and School District Results were tested using a factorial two-way analysis

o f variance. Each hypothesis was rejected based on the significance o f one o f the main

effects. A significant difference was found in the responses o f teachers when compared

to both superintendents and principals in each o f the seven areas. No significant

difference was found between the responses o f superintendents and principals. The size

o f the district was not found to be a factor in responses in any category. There was no

significant interaction between size o f district and educator position.

Teachers responded at a significantly higher rate with the “Do Not Know”

selection. The majority o f responses were answered positively regarding the potential

usefulness o f the instrument for organizational analysis and school improvement.

However, comments made by respondents suggested that the length, complexity, and

language might be obstacles. Lack o f knowledge about the scope o f operations was

apparent on the part o f respondents and they commented on the fact consistently.

Teacher comments also indicated doubts that administrators desired their sincere

involvement or opinion.

The Performance Analysis for School Districts is a reliable tool for consistently

assessing perception o f the current situation in each construct examined. The reliability

o f the School District Results construct could be improved to increase reliability, but

findings were still positive.. Although the response rate was only 36%, the total number

was large enough to positively influence the power o f the statistical analysis. The use o f a

stratified random sample increased the generalization o f the findings to the larger


128

population o f Idaho educators. The findings only measured perceptions o f current

performance and did not account for varying levels o f prior knowledge o f quality or

organizational performance. The differences in the perceptions o f teachers and

administrators were statistically highly significant. The instrument has the potential to be

useful in discovering differences in perception among various components o f operation

and performance. Teachers clearly do not have comprehensive knowledge o f the system

in which they work. Administrators consistently perceived the current situation to be

better than the teachers did. Comments suggested that educators desire a simple, quick

approach to school district performance and generally feel cynical toward their role in

impacting school improvement.

The findings o f this study are o f practical significance for leadership and teachers

as school improvement strategies and management systems are designed. The need for a

model to be used as a framework for improving school district performance is apparent.

Figure 3 suggests such a framework to improve school productivity based on the theory

o f quality within a systems perspective.

Recommendations

The Malcolm Baldrige National Quality Award takes into account the systems

and multiple constituencies and goal attainment models for organizational effectiveness

described in this study. Increasingly studies are examining the applications o f these

models in a variety o f settings (Danne, 1992; Fritz, 1993; Miller, 1993; Smith, 1995;

Thompson, 1996; Wu, 1996). While most studies measured perception and opinion, they

often neglect to examine objective measures o f actual improvements in organizations that

have applied a self-study process, quality applications, and a systems approach.


129

Leadership

K-12Educational System

Process Management

Stakeholder and Student Expectations and Satisfaction

Strategic Planning

Information & Analysis Human Resource Development & Management

Results

Figure 2. A Quality Systems Model for Performance Improvement.

approach to improving organizational performance. Continued research is needed to

study the application and the outcomes achieved as a result o f this approach.

Consideration might be given to a causal-comparative design for such investigations


130

The findings related to differences in the perceptions o f educators should be

further investigated, and the relationship between how administrators lead and the teacher

perceptions o f the system as a whole are also important issues. Exemplary companies

that have been recognized as benchmarks for human resource practices could be

investigated to determine possible applications for schools o f the 21st century. Studies

could also explore how teacher performance is effected by teacher attitudes and levels o f

pride in the organization. Larger samples selected from other states could further

increase the generalization potential o f the findings. Other strategies, besides the mail

survey approach, should be considered to simulate actual use o f the instrument.

The instrument should be applied in a school district(s) to determine its potential

usefulness as a starting point for organizational improvement. A case study model which

simulates the MBNQA preparation and review process may be an effective research

design. Case studies for in-depth analysis could contribute significantly to the body o f

knowledge about using quality approaches in schools.

The necessity for an agreed upon framework and criteria to measure productivity

and accountability is apparent. Fullan (1980) suggested that, “because it is so difficult to

evaluate the effectiveness o f organizations with anarchistic characteristics, managers tend

to seek, simple, uncomplicated indicators to justify their effectiveness. Decision theorists

have found that when complex and ambiguous situations are encountered, overly

simplistic decisions are applied. Overly simplistic indicators are frequently relied on.

The results o f this study and the comments made by the respondents support this notion.


131

References

Anderson, G., Herr, K. & Nihlen, A. (1994). Studying your own school. Thousand Oaks, CA: Corwin Press, Inc.

Bass, B. M. (1952). Ultimate criteria o f organizational worth. Personnel Psychology, 5, 57- 73.

Bemowski, K. (1996). Baldrige Award celebrates its 10th birthday with a new look. Quality Progress, 29, 49-54.

Bemowski, K. & Stratton, B. (1995). How do people use the Baldrige Award criteria? Quality Progress, 28, 43-47.

Bennis, W. (1976). The planning o f change, (3rd ed.). New York: Holt, Rinehart and Winston.

Bernhardt, V. (1994). The school portfolio. Princeton, NJ: Eye on Education.

Bonstingl, J. (1996). Schools o f quality. Alexandria, VA: Association for Supervision and Curriculum Development.

Bracey, G. (1997). Setting the record straight. Alexandria, VA: Association for Supervision and Curriculum Development.

Bradley, L. (1993). Total quality management fo r schools. Lancaster, PA: Technomic Publishing.

Brassard, A. (1993). Conceptions o f organizational effectiveness revisited. The Alberta Journal o f Educational Research, 39, 143-62.

Berman, McLaughlin. (1977).

Brown, M. (1994). Baldrige Award Winning Quality. Milwaukee, WI: ASQC Quality Press.

Bushweller, K., Ed. (1996). Education vital signs. The American School Board, 183, A l- A31.

Cameron, K. Critical questions in assessing organizational effectivness. Organizational Dynamics, 9, 66-80.

Campbell, J. P., Brownas, E. A., Peterson, N. G. & Dunnetee, M. D. (1974). Themeasurement o f organizational effectiveness: A review o f relevant research and opinion. San Diego: Naval Personnel Research Center.


132

Caravatta, M. (1997). Conducting an organizational self-assessment using the Baldrige Award criteria. Quality Progress, 30, 87-91.

Chappell, R. (1993). Effects o f the implementations o f total quality management on the Rappahannock County, Virginia public schools. (Doctoral dissertation, Virginia Polytechnic Institute and State University, 1993). Dissertation Abstracts International.

Collins, B. & Huge, E. (1993). M anagement by policy. Milwaukee, WI: ASQC Quality Press.

Consortium on Productivity in Schools. (1995). Using what we have to g e t the schools weneed. New York: Institute on Education and the Economy.

Council on Competitiveness. (1995). Building on the Baldrige: American quality fo r the 21st century. Washington, DC: Council on Competitiveness.

Crosby, P. (1984). Quality without tears. N ew York: Plume Press.

Crosby, P. & Reimann, C. (1991). Criticism and support for the Baldrige Award. Quality Progress, M ay , 41-44.

Danne, D. J. (1991). Total quality management and its implications for secondaryeducation. (Doctoral dissertation, Pepperdine University, 1991). Dissertation Abstracts International.

Deer, C. (1976). ‘O. D .’ won’t work in schools. Education and Urban Society, 8, 227.

Deming, W. E. (1986). Out o f the crisis. Cambridgem MA: Massachusetts Institute o f Technology, Center for Advanced Engineering Study.

Deming, W. E. (1989). Foundation fo r management o f quality in the western world. Paper presented at the meeting o f the Institute o f Management Science, Osaka, Japan.

Deming, W. E. (1994). The new econom ics fo r industry, government and education. Cambridge, MA: Massachusetts Institute o f Technology, Center for Advanced Engineering Study.

DeMont, B. & DeMont, R. (1973). A practical approach to accountability. Education Technology, 40-45.

Dubin, R. (1976). Organizational effectiveness: Some dilemmas o f perspective. O rganizational and Administrative Sciences, 7, 7-14.

Edmonds, R. R. (1979). Some schools work and more can. Social Policy, 9, 28-32.


133

Edmonds, R. (1980). Effective schools for the urban poor. Educational Leadership, 40, 15- 23.

Eisenberg, E. & Goodall, H. (1993). Organizational communication. New York: St. Martin’s Press.

Elam, S., Rose, L., & Gallup, A. (1996). The 28th annual Phi Delta Kappa/Gallup poll o f the public’s attitudes toward the public schools. Phi D elta Kappan, 78, 41-59.

English, F. (1988). Curriculum auditing. Lancaster, PA: Technomic Publishing Co.

English, F. & Hill, J. (1994). Total quality education. Thousand Oaks, CA: Corwin Press.

Etzioni, A. (1960). Two approaches to organizational analysis: A critique and a suggestion. Administrative Science Journal, 5, 257-278.

Feigenbaum, A. V. (1983). Total quality control. New York: McGraw-Hill.

Fields, J. (1994). Total quality fo r schools a guide fo r implementation. Milwaukee, WI: ASQC Quality Press.

Fritz, S. (1993). A quality assessment using the Baldrige criteria: Non-academic serviceunits in a large university. (Doctoral dissertation, University o f Nebraska— Lincoln). Dissertation Abstract International.

Fullan, M. (1980). Organizational development in schools: The state o f the art Review o f Educational Research, 50, 121-183.

Fullan, M. & Miles, M. (1992). Getting reform right: What works, what doesn’t. Phi Delta K appan , 73, 744-52.

Fullan, M. (1993). Innovation, reform, and restructuring strategies. In Cawelti, G. (Ed.), Challenges and achievements o f American education innovation, reform, and restructuring strategies (pp. 116-133). Alexandria, VA: Association for Supervision and Curriculum Development.

Fullan, M. (1997). Emotion and hope: Constructive concepts for complex times. InHargreaves, A. (Ed). Rethinking educational change with heart and mind (pp. 216- 233). Alexandria, VA: Association for Supervision and Curriculum Development.

Gall, M., Borg, W. & Gall, J. (1996). Educational Research An Introduction (6th ed.).White Plains, NY: Longman.

Garvin, D. (1988). M anaging quality: The strategic and competitive advantage. New York: Free Press.


134

Georgopoulos, B. & Tannenbaum, A. (1957). The study o f organizational effectiveness. American Sociological Review, 22, 534-550.

Gerstner, L., Semerad, R., Doyle, D. & Johnston, W. (1995). Reinventing education entepreneurship in A m erica’s public schools. New York: Plume Press.

Glasser, W. (1992). The quality school. New York: Harper Collins.

Goodlad, J. (1984). A place called school. New York: McGraw-Hill.

Green, D. (1996). A case for the Koalaty Kid. Quality Progress, 29, 97-99.

Guba, E. (1981). Investigative journalism. In Smith, N. L. (Ed.). New techniques fo r evaluation. Beverly Hills, CA: Saga Publications.

Hannan, M. & Freeman, J. (1977). Obstacles to the comparative study and organizational effectiveness. In Goodman, P. & Pennings, J. (Eds.), New perspectives on organizational effectiveness. San Francisco: Jossey-Bass.

Hargreaves, A. (1997). Rethinking educational change: going deeper and wider in the quest for success. In Hargreaves, A. (Ed.), Rethinking educational change with heart and mind (pp. 1-26). Alexandria, VA: Association for Supervision and Curriculum Development.

Hersey, P. & Blanchard, K. (1982). Management o f organizational behavior: Utilizing human resources. Englewood Cliffs, NJ: Prentice-Hall.

Hodgkinson, H. (1996). Why have Americans never admired their own schools? The School Administrator, 53, 18-22.

Hoy, W. & Ferguson, J. (1985). A theoretical framework and exploration o f organizational effectiveness o f schools. Educational Administration Quarterly, 21, 117-134.

Houston, P. (1996). From Horace Mann to the contrarians. The School Administrator, 53, 10-13.

Huelskamp, R. (1993). Perspectives on education in America. Phi D ella Kappan, 7 4 , 1 18- 721.

Idaho State Department o f Education. (1996). Accreditation Standards and Procedures For Idaho Schools. Boise: Author.

Idaho State Department o f Education. (1996). Idaho School Profiles, 1995-96. Boise: Author.


135

Imai, M. (1986). Kaizen: The key to Japan's competitive success. N ew York: McGraw-Hill Publishing.

Jaeger, R. & Hattie, J. (1996). Artifact and artifice in education policy analysis: It’s not all in the data. The School Administrator, 53, 24-29.

Joint Committee on Standards For Educational Evaluation. (1994). The program evaluation standards, (2nd ed.). Thousand Oaks, CA: Sage Publications.

Juran, J. (1988). Juran on planning fo r quality. Cambridge, MA: Productivity Press.

Kamen, H. (1993). A study o f the impact o f the curriculum audit process in three school systems. (Doctoral dissertation, University o f Cincinnati, 1993). Dissertation Abstracts International.

Katz, D. & Kahn, R. (1978). The socia l psychology o f organizations, (2nd ed.). New York: Wiley.

Kearns, D. & Doyle, D. (1988). Winning the brain race. San Francisco, CA: Institute for Contemporary Studies Press.

Kennedy, M. (1995). An analysis and comparison o f school improvement planning models. (Doctoral Dissertation, University o f Central Florida, 1995/ Dissertation Abstracts International.

Klaus, L. (1996). Quality Progress sixth quality in education listing. Quality Progress, 29, 29-45.

Krejcie, R. & Margan, D. (1970). Determining sample size for research activities. Educational and Psychological Measurement, 30, 607-610.

Langford, D. & Cleary, B. (1995). Orchestrating learning with quality. Milwaukee, WI: ASQC Quality Press.

Levine, R. & Fitzgerald, H. Living systems, dynamical systems, and cybernetics. In Levine, R. & Fitgerald, H. (Eds.). (1 9 9 2 / Analysis o f Dynamic Psychological Systems, Volume I. New York: Plume Press.

Lezotte, L. (1989). Base school improvement on what we know about effective schools. The American School B oard Journal, 176, 18-20.

Lezotte, L. W. & Bancroft, B. A. (1985). School improvement based on effective schools research: A promising approach for economically disadvantaged and minority students. Journal o f Negro Education, 54, 301-312.


136

MacLellan, D. (1994). Towards a new approach for school system evaluation. (Doctoral dissertation, Dalhousie University, Nova Scotia). D issertational Abstracts International.

Mann, D. (1976 / Policy decision-making in education: An introduction to calculation and control. New York: Teachers College Press.

McCIanahan, E. & Wicks, C. (1994). Future force. Glendale, CA: Griffen Publishing.

Miller, S. (1993). The applicability o f the Malcolm Baldrige National Quality Awardcriteria to assessing the quality o f student affairs in colleges. (Doctoral dissertation, Ohio University— Athens, 1993). Dissertation A bstracts International.

National Institute o f Standards and Technology. (1994). M alcolm Baldrige national Quality award. Gaithersburg, MD: United State Department o f Commerce and Technology Administration.

National Institute o f Standards and Technology. (1995). M alcolm Baldrige national quality aw ard education pilot. Gaithersburg, MD: United State Department o f Commerce and Technology Administration.

Nicolis, G. & Prigogine, I. (1977). Self-organization in nonequilibrium systems: From dissipative structures to order through fluctuations. N ew York: Wiley.

Northwest Association o f Schools and Colleges. (1996). Setting World Standards For Accreditation.

Nowakowski, J., Bunda, M., Working, B. & Harrington, P. (1985). A handbook o f educational variables. Boston, MA: Kluover-Nijhoff.

O’Neil, John. (1995). On schools as learning organizations: A conversation with Peter Senge. Educational Leadership, 52 , 20-23.

Owens, R. (1970). Organizational Behavior in Schools. Englewood Cliffs, NJ: Prentice- Hall.

Pannirselvam, G. (1995). Statistical validation o f the M alcolm Baldrige National Quality Award m odel and evaluation process. Unpublished doctoral dissertation, Arizona State University, Tempe.

Partin, J. (1992). A measurement o f total quality management in the two-year collegedistricts o f Texas. (Doctoral dissertation, East Texas State University). Dissertation Abstracts International.

Patterson, J. (1993). Leadership fo r tom orrow’s schools. Alexandria, V A: Association for Supervision and Curriculum Development.


137

Patterson, J., Purkey, S. & Parker, J. (1986). Productive school systems fo r a rational world. Alexandria, VA: Association for Supervision and Curriculum Developnment.

Patton, M. (1983). Qualitative evaluation methods. Beverly Hills, C A: Sage Publications.

Pines, E. (1990). From top secret to top priority: The story o f TQM. Aviation Week &Space Technology, S5-S24.

Portner, J. (1997, March). Once a status symbol for schools, accreditation becomes rote drill, Education Week, xvi (1), 30-31.

Provus, M. (1971). Discrepancy evaluation fo r education program: Improvement and assessment. Berkley, CA: McCutchan.

Purkey, S. & Smith, M. (1982). Too soon to cheer?: Synthesis o f research on effective schools. Educational Leadership, 43, 64-69.

Regeuld, M. (1993). A study o f continuous improvement processes based on total quality management principles as applied to the educational environment. (Doctoral dissertation, Pennsylvania State University). Dissertation Abstracts International.

Rhodes, L. (1990). Beyond your beliefs: Quantum leaps toward quality schools. The School Administrator, 47, 23-26.

Rotberg, Iris. (1996). Five myths about test score comparison. The School Administrator,53, 30-35.

Rubin, S. (1994). Public schools should learn to ski. Milwaukee, WI: ASQC Press.

Sagor, R. (1995). Overcoming the one-solution syndrome. Educational Leadership, 52, 24- 27.

Sarason, S. (1990). The predictable failure o f educational reform. San Francisco, CA: Jossey-Bass Publishers.

Schmoker, M. (1996). Results the key to continuous improvement. Alexandria, VA: ASCD.

Scholtes, P. (1993, February). Quality Learning Series. Paper presented by the QualityLearning Services Division o f the U. S. Chamber o f Commerce. Madison, WI: Joiner Associates.

Scriven, M. (1973). Goal-free evaluation. In House, E. R. (Ed.), School evaluation: The politics and process. Berkley, CA: McCutchan.

Seigal, P. & Byrne, S. (1994). Using quality to redesign school systems. San Francisco,CA: Jossey-Bass Publishers.


138

Senge, P. (1990). The Fifth Discipline. New York: Currency Doubleday.

Senge, P., Kleiner, A., Roberts, C., Ross, R. & Smith, B. (1994). The Fifth Discipline Handbook. New York: Doubleday.

Sergiovanni, T. (1992). M oral Leadership. San Francisco, CA: Jossey -Bass Publishers.

Shipley, J. & Collins, C. (1997). Going to scale with TOM. Tallahassee, FL: Southeastern Regional Vision for Education.

Skrtic, T. (1991). Behind Special Education. Denver, CO: Love Publishing.

Smith, R. (1996). The im pact o f training based on the M alcolm Baldrige National Quality Award on employee perceptions. Unpublished doctoral dissertation, University o f Idaho, Moscow.

Stake, R. (1967). Toward a technology for the evaluation o f educational programs. In Tyler, R., Gagne, R. & Scriven, M. (Eds.). Perspectives o f curriculum evaluation (pp. 1-12). Chicago: Randy McNally.

Stampen, J. (1987). Improving the quality o f education: W. Edwards Deming and effective schools. Contemporary Education Review, 3 ,423-433.

Stefanich, G. (1983). The relationship o f effective schools research to school evaluation, North Central Association Quarterly, 88, 343-349.

Stufflebeam, D. (1983). The CEPP model for program evaluation. In Maciaus, G., Scriven, M. & Stufflebeam, D. (Eds.). Evaluation models: Viewpoints in educational and human services evaluation. Boston, MA: Kluver-Nijhoff Publishing.

Thomas, W. & Moran, K. (1992). Reconsidering the power o f the superintendent in the progressive period. American Educational Research Journal, 29, 22-50.

Timpane, M. & Reich, R. (1997). Revitalizing the ecosystem for youth. Phi Delta Kappan, 464-470.

Tribus, M. (n.d.). The transformation o f American education to a system fo r continuously im proved learning. Hayward, CA: Exergy, Inc.

Vertiz, V. (1995). What the curriculum audit reveals about schools. The School Administrator, 25-27.

Vertiz , V. & Bates, G. (1995, April). The Curriculum Audit: Revelations About Our Schools. Paper presented at the meeting o f the American Educational Research Association, Division B.


139

Wagner, T. (1996). Bringing school reform back down to Earth. Phi D elta Kappan, 78, 145-149.

Wagner, T. (1993). Systemic change: Rethinking the purpose o f school Educational Leadership, 51, 24-28.

Walton, Mary. (1990). Deming management at work. New York: Putnam Publishing Co.

Weick, E. (1976). Educational organizations as loosely couple systems. Administrative Science Quarterly, 21, 1-19.

Wiersma, W. (1995). Research M ethods in Education. Needham Heights, MA: Allyn and Bacon.

Worthen, B. & Sanders, J (1987). Educational Evaluation. White Plains, NY: Longman.

Wu, Hung-Yi. (1996). Development o f a self-evaluation system fo r to ta l quality management using the Baldrige criteria. Unpublished doctoral dissertation, University o f Missouri, Rolla.

Yuchtman, E. & Seashore, S. (1967). A system resource approach to organizational effectiveness. American Sociological Review, 32, 891-903.

Zammuto, R. (1982). Assessing organizational effectiveness. Albany, NY: State University o f New York Press.


Reproduced

with perm

ission of the

copyright ow

ner. Further


without

permission.

Appendix A: Instrument

1.0 LeadershipThis category examines the school district’s leadership (defined as the board o f trustees and senior

administrators, at the district level) personal leadership and involvement in creating and sustaining a student focus, clear goals, high expectations, and a leadership system that promotes performance excellence.

Please select I from the five choices in each box which most closely describes your school district in that area. Fill in the circle completely next to the statement selected with pencil or black ink.__________________________________________________________

Item # Description1.1 The extent to which

there is clear direction throughout the district

Definitions:*Stakeholder- Individuals or groups, both internal to the school (students, a ll personnel) and external (parents, community members, business) which are affected by the conditions and quality o f education and the preparedness o f graduates.

1.1.1

1.1.2

1.1.3

1.1.4

1.1.5

1. 1.6

A clearly communicated and consistent direction o f the district focus based on stakeholder* needs and expectations does not exist.

A clearly communicated and consistent direction o f the district focus based on stakeholder* needs and expectations exists. It is not widely disseminated or used by personnel.

A clearly communicated and consistent direction o f the district focus based on stakeholder* needs and expectations exist and is known throughout the district. It is does not appear to be used consistently in district decision-making.

A clearly communicated and consistent direction o f the district focus based on stakeholder* needs and expectations exists and appears to be considered to some degree when making planning decisions. This information is communicated to parents.

A clear direction o f district focus is broadly communicated throughout the community. It guides all major decisions throughout the entire system. The district direction is systematically re-evaluated through an improvement cycle, involving multiple stakeholders * and sources o f information.

I do not know.

Reproduced

with perm

ission of the

copyright ow

ner. Further


without

permission.

Item # Description1.2 The extent to which a

review process exists to study the performance** of the district

Definitions:*Stakeholder- Individuals or groups, both internal to the school (students, all personnel) and external (parents, community members, business) which are affected by the conditions and quality o f education and the preparedness o f graduates.

**Performance refers to the results produced by the school district as illustrated by multiple sources o f information.

1.2.1

1.2.2

1.2.3

1.2.4

1.2.5

1.2.6

A review process is done to meet state accreditation standards as often as required. There does not appear to be any other structured review process to determine how the school district is meeting or exceeding the needs and expectations o f its stakeholders*.

Leaders study performance** information on an annual basis, using standardized test scores, attendance, enrollment and financial information.

A systematic process is being developed district wide to establish multiple reliable and valid indicators o f student and district performance* *.

A study process exists to regularly assess the school district performance based on multiple sources o f information. All personnel regularly assess the performance o f their programs which is aligned to the district study process. This information is used to create district direction and areas for improvement.

A systematic process for review o f the district’s performance exists which exceeds the requirements for state accreditation. Multiple stakeholders are involved. Decisions are based on multiple indicators o f performance information, community demographics, and forecasting future needs. The review process is systematically evaluated and improved as necessary.

I do not know.1.3 Leadership’s role in

improvement efforts

Definition:*Stakeholder- Individuals or groups, both internal to the school (students, personnel)

1.3.1

1.3.2

Leaders** initiate few district-wide improvements. Most improvements are initiated at the building level.

Leaders** initiate district-wide improvements and allocate existing resources to support those improvements.

Reproduced

with perm

ission of the

copyright ow

ner. Further


without

permission.

Item # Descriptionand external (parents, community members, business) which are affected by the conditions and quality o f education or the preparedness o f graduates

**Leaders defined as Board o f Trustees and senior administrators a t the district level.

1.3.3

1.3.4

1.3.5

1.3.6

Leaders** personally encourage and advocate for school or district wide program improvements that are consistent with district directions.

Leaders** are committed to continuous improvement o f district performance based on a thorough understanding o f the needs and expectations o f stakeholders**. District resources are allocated to accomplish the targeted improvement areas.

Leaders** are visibly involved and facilitate improvements throughout the system. Leaders** monitor the system for progress and view themselves as accountable for the performance o f the district. There is a systematic review o f the role o f leadership in improvement efforts.

1 do not know.1.4 The extent to which

there exists a district- wide collaborative and participatory approach to management*

Definitions.*Defined as jo in tly working to identify problems and determining improvements with others in the organization who are knowledgeable, involved and affected by any decisions made.

1.4.1

1.4.2

1.4.3

1.4.4

There appears to be little collaboration or substantive involvement o f appropriate personnel in decision-making. Important decisions are made at a district level.

There are vehicles in school district units** to make some decisions. District-level decision making and approval is still the primary vehicle for significant decisions.

Leaders*** value participatory decision-making and are actively working to improve and develop these processes in all school district units**.

Effective processes to involve personnel, parents and community stakeholders are in place in all school district units**. Decision-making responsibilities and expectations are clearly defined for the school district unit**.

to

Reproduced

with perm

ission of the

copyright ow

ner. Further


without

permission.

Item # Description** School district units are defined as specific schools, departments, or services o f that school district.

**Leaders defined as Board o f Trustees and senior administrators at the district level

1.4.5 Collaborative and participatory management is institutionalized, valued and a part o f the culture. All personnel o f the school community and significant stakeholders are included. Collaborative processes are systematically evaluated to determine effectiveness and improvements are made as necessary.

1.5. Board policy

Definitions:* Stakeholder- Individuals or groups, both internal to the school (students, personnel) and external (parents, community members, business) which are affected by the conditions and quality o f education or the preparedness o f graduates

1.5.1

1.5.2

1.5.3

1.5.4

1.5.5

1.5.6

Board policies are developed by the Board o f Trustees with minimal involvement of stakeholders* and reviewed as becomes necessary.

There is some involvement o f stakeholders in the development and review o f Board Policies.

The district is developing a systematic process to review and design board policy involving stakeholders to support the district direction and strategic plan.

Board policies have been revised to support the district direction and strategic plan. Polices support the delivery o f curriculum and instruction. Processes exist to communicate this information to personnel who implement policy.

A systematic review process o f board policy exists to insure alignment to district direction and strategic plan. A review process o f district performance is aligned to strategic planning processes. Board polices are clearly communicated to all personnel.

I do not know.

U>

Reproduced

with perm

ission of the

copyright ow

ner. Further


without

permission.

Item # Description1.6. School District

Responsibility to the Public

Definition:*Leaders defined as Board o f Trustees and senior administrators at the district level

1.6.1

1.6.2

1.6.3

1.6.4

No defined processes exist to anticipate public concerns or expectations.

Some activities have been done to involve the community, but are usually reactive to an expressed public concern. There is some involvement o f the leadership in activities to strengthen and support their communities.

Leaders* recognize the importance o f anticipating public needs and expectations and are developing processes to do so. Leaders* are actively involved in the community.

Leaders* invite stakeholders into operations o f the district and are actively involved in community groups to solve community problems.

1.6.5 The district serves as a role model o f outreach and public service to the community through effective processes to anticipate the public’s interests and needs. The district systematically evaluates its own effectiveness in involving the public and being involved in the community.

1.6.6 I do not know.

1.7 State, Legal and Ethical Conduct in Operations

1.7.1 Citations for non-compliance to required federal, state regulations have frequently been found. Expectations for ethical conduct are not clarified.

Definition:* School district units are defined as specific schools, departments, or services o f that school district.

1.7.2

1.7.3

Citations for non-compliance are being addressed. Ethical conduct is demonstrated by leadership.

Efforts are made to achieve compliance throughout all school district units*.

£

Reproduced

with perm

ission of the

copyright ow

ner. Further


without

permission.

Item # Description1.7.4

1.7.5

1.7.6

Leaders demonstrate high commitment to legal and ethical conduct in all aspects o f the school district through existing policies and practices. Prevention processes are in place to ensure performance above minimal requirements.

The school district exceeds minimal requirements for compliance to federal, state regulations. Leaders are role models for ethical conduct in all activities within the district and the community. There is a systematic process in place to review policies and practices related to ethical standards and compliance to legal requirements.

1 do not know.

Reproduced

with perm

ission of the

copyright ow

ner. Further


without

permission.

2.0 Strategic PlanningThis category examines how the school district sets strategic directions and how it determines

stakeholders* expectations o f the school district. Please consider how these stakeholder expectations and requirements are translated into an effective performance management system, with a primary focus on student performance.(*Stakeholder- Individuals or groups, both internal to the school (students, personnel) and external (parents, community members, business) which are affected by the conditions and quality o f education or the preparedness o f graduates; also includes state agencies and their requirements)Please select 1 from the five choices in each box which most closely describes your school district in that area. Fill in the circle

Item # Description2.1 Strategic Development*

Definitions:

2.1.1 No school district improvement plan exists. There appears to be little organized effort to examine performance o f the school district, including student performance and district operations.

* Defined as the process by which members o f an organization clarify

2.1.2 The school district does have some goals and objectives, usually associated with state or federal requirements. Traditional indicators including standardized tests scores, compliance reviews and accreditation ratings are used to determine district improvements.

the purpose and develop the necessary procedures and operations to achieve that purpose. A

2.1.3 District is developing a more comprehensive approach to strategic planning by examining needs o f students, analyzing the current performance o f students, focusing on the needs and expectations o f stakeholders through the collaborative involvement o f all school district units** and community.

strategic plan is designed.

2.1.4 District considers extensive sources o f information both internal to the district and external, such as community demographic information. Representatives from a variety o f stakeholder groups participate in strategic planning. Clear long and short term goals and objectives exist. Measures are defined, with clearly specified timelines and responsibility. School district units** develop plans consistent with district strategic plan.

O n

Reproduced

with perm

ission of the

copyright ow

ner. Further


without

permission.

Item # Description* *SchooI district units are defined as specific schools, departments, or services o f that school district.

2.1.5

2.1.6

District has specific short and long term goals and objectives in place established as a result o f a comprehensive assessment o f student performance and district operations, stakeholder needs and expectations through the collaboration o f stakeholders. All aspects o f the school district are examined to support the implementation and accomplishment o f the goals and objectives. School district units** have plans aligned to the district strategic plan. A systematic process is in place to review the strategic plan to make necessary improvements.

1 do not know.2.2 Focus o f strategic

plan

Definition:*School district units are defined as specific schools, departments, or services o f that school district.

2.2.1

2.2.2

2.2.3

2.2.4

2.2.5

2.2.6

There appears to be a fragmented focus on improved student performance in the district.

Emphasis exists on improving student performance on standardized and state tests.

Improved student performance is the focus o f strategic plan. Improvements are being identified throughout school district units* to support high student performance.

District plan reflects integrated efforts o f involved school district units* to support and accomplish improved performance for all students.

District focus on improved student performance is evident throughout the school district strategic plan. The district regularly assesses its plan for its focus on improved student performance.

1 do not know.2 .3 Implementation

and assessment of strategic plans

2.3.1

2.3.2

District strategic plan may exist but is not disseminated and reviewed with all school district units*. Responsibility for implementation is unclear.

District plan is disseminated to all personnel in school district. Responsibility for implementation is noted. Processes to accomplish the goals are unclear with no evaluation measures specified.

-p*- j

Reproduced

with perm

ission of the

copyright ow

ner. Further


without

permission.

Item # DescriptionDefinition:* School units are defined as specific schools, departments, or services o f that school district.

2.3.3

2.3.4

District strategic plan is disseminated and reviewed with all school district units* and with all personnel throughout the school district. Plan is published and disseminated to the community.

A clear implementation process exists, with responsibilities and timelines clearly delineated. Work teams, involving personnel in appropriate school district unit* and external stakeholder, are established. Evaluation measures are specified. Monitoring processes support successful implementation.

2.3.5 Implementation o f district strategic plan is clear, and aligned throughout all school district units*. District and building plans are implemented and progress is assessed continually throughout the district. A systematic process exists to evaluate the implementation o f strategic plans throughout the school district for continual improvement.

2.3.6 1 do not know.

4̂00

Reproduced

with perm

ission of the

copyright ow

ner. Further


without

permission.

3.0 Student Focus and Satisfaction/Stakeholder SatisfactionThis category examine how the school district determines

student and stakeholder* needs and expectations. Please consider how the school district enhances stakeholder relationships and determines their satisfaction.* {Stakeholder- Individuals or groups, both internal to the school (students, personnel) and external (parents, community members, business) which are affected by the conditions and quality o f education or the preparedness o f graduates; also includes state agencies and their requirements)Please select 1 from the five choices in each box which most closely describes your school district in that area. Fill in the circle completely next to the statement selected with pencil or black ink._____________________________________________

Item Description3.1 The ways in which the

school district determines students’ needs from and expectations o f educational programs

Definition:*School district units are defined as specific schools, departments, or services o f that school district.

3.1.1

3.1.2

3.1.3

3.1.4

Standardized test scores are the primary source o f information to determine student needs. Little or no analyses o f the data are done nor disseminated to appropriate school district units*. Students are not asked in any formal way for their needs or expectations.

Besides state test scores, the district uses other sources o f locally generated and collected information to determine student needs and expectations Some analysis is done as determined by individual school district unit *.

District is developing comprehensive strategies to determine all student needs.

District is in early stages o f using multiple strategies, both inside and outside the school district, to determine students’ comprehensive needs throughout all grade levels and at specific times. Information is used in the district’s strategic planning process.

149

Reproduced

with perm

ission of the

copyright ow

ner. Further


without

permission.


3.1.6

District has a well developed and extensive system o f determining student needs and expectations. The emphasis is on prevention o f school failure and the analysis o f multiple sources o f information to determine trends. Information is analyzed, compared and used in all district decision-making processes. The process to determine needs and expectations o f students is systematically evaluated.

I do not know.3.2 High expectations for the

performance o f students

Definitions:*Performance expectations refer to clearly defined statements describing specific academic, behavioral or social criteria to measure achievement, often referred to as standards.

* *Stakeholder-Individuals or groups, both internal to the school (students, personnel) and external (parents, community members, business) which are affected by the conditions and quality o f education or the preparedness o f graduates; includes state agencies and requirements

3.2.1

3.2.2

3.2.3

3.2.4

3.2.5

3.2.6

Performance expectations* for students do not exist at any level.

Performance expectations* are being developed for some areas.

Performance expectations* exist for some grades/subject areas.

Performance expectations*, including exit standards for graduation, exist throughout the K-12 system in all grade levels, courses and social conduct. Strong stakeholder support exists for these criteria.

Performance expectations* are systematically evaluated and adjusted to reflect stakeholder** expectations.

1 do not know.

o

Reproduced

with perm

ission of the

copyright ow

ner. Further


without

permission.

Item # Description3.3 Student and stakeholder

satisfaction

Definitions.*School district units are defined as specific schools, departments, or services o f that school district.

**Satisfaction with educational programs and performance o f school district

3.3.1

3.3.2

3.3.3

3.3.4

3.3.5

3.3.6

There are no processes district-wide to determine student or stakeholder satisfaction**. Anecdotal information often used to determine student or stakeholder satisfaction*.

Some attempts to determine student or stakeholder satisfaction are made by individual school district units*.

District is developing a process to determine student and stakeholder satisfaction.

District is making deliberate and consistent efforts to utilize a variety o f strategies to determine student and stakeholder satisfaction throughout the district for all school district units*. This information is disseminated throughout the district to all school district units*.

District has clearly defined processes in place to measure, analyze and compare student and stakeholder satisfaction** results at specific points and in all school district units*. Information is used in strategic planning process. Processes are systematically evaluated for effectiveness on a regular basis.

I do not know3.4 Identifying future needs

and expectations o f students and stakeholders*.

Definition:*Stakeholder- Individuals or groups, both internal to the school (students, personnel) and external (parents,

3.4.1

3.4.2

3.4.3

District focuses on immediate needs o f students as they occur.

District considers local demographic factors and trends which affect enrollment and student needs and stakeholder* expectations.

District considers local, state, and national trend data and demographics to determine future needs o f students and expectations o f stakeholders*.

Reproduced

with perm

ission of the

copyright ow

ner. Further


without

permission.

Item # Descriptioncommunity members, business) which are affected by the conditions and quality o f education or the preparedness o f graduates; also includes state agencies and their requirements

3.4.4

3.4.5

3.4.6

District is in early stages o f using demographic factors, changing state, federal requirements or trends, changing expectations and needs o f higher education and the workplace as part o f the strategic planning process.

Multiple strategies to consider future needs/expectations o f students and stakeholders* from multiple sources are systematically used in all strategic planning processes. These processes are evaluated on a regular basis to determine need for improvement. Processes used are compared to other organizations who have demonstrated exemplary ways to forecast student and stakeholder* needs.

I do not know.

LAN>

Reproduced

with perm

ission of the

copyright ow

ner. Further


without

permission.

4.0 Information and Analysis This category examines the management o f and effective use o f data and information to drive mission-related performance excellence in a school district. Please consider how your school district collects and uses information and data to make decisions.Please select / from the five choices in each box which most closely describes your school district in that area. Fill in the circle completely next to the statement selected with pencil or black ink.___________________________________________________

Item # Description4.1 Selection and Use o f Information and Data

Definitions:*Conventional information is defined as standardized and state test scores, enrollment, attendance, dropout, discipline, operating budget

* *School district units are defined as specific schools, departments, or services o f that school district.

4.1.1

4.1.2

4.1.3

4.1.4

4.1.5

4.1.6

Conventional information* is used primarily by district office and board in planning. This information is not widely or regularly disseminated throughout the district.

Conventional information* is disseminated at least annually to all school district units**.This information is used by school district units to assess student or operational performance.

A process is being developed to collect, manage, and use specific information and data which are needed to meet the mission and develop key district goals which focus on improving student performance.

District has a systematic process in place to collect, analyze, and disseminate critical information, beyond conventional information*, to all school units** related to district key goals and objectives. Information used comes from a variety o f sources used to determine student performance, student needs and satisfaction, stakeholder needs and satisfaction.

A systematic process for collecting, analyzing, and using comprehensive information and data from multiple sources is fully deployed throughout all school district units**. All personnel have easy access to the information and use it to evaluate, adjust, and create key goals for school district unit** plans. The information and data process is systematically evaluated and improvements are made as necessary.

1 do not know.

u>

Reproduced

with perm

ission of the

copyright ow

ner. Further


without

permission.

Item U Description

4.2 Selection and Use 4.2.1 No process is in place currently to seek or use comparative data*.of ComparativeInformation and Data 4.2.2 Some practices exist to compare conventional data*. Practice is limited to specific school

district units** or personnel.Definitions:*Comparative data, or 4.2.3 District is developing a process to determine needs and priorities, criteria, and use ofbenchmarking, is an comparative data*.improvement process inwhich a school district 4.2.4 A systematic process exists to use comparative data* and is fully implemented in all schoolcompares its district units**. A process for benchmarking is used to set improvement targets and integrateperformance against best practices.against best-in-classschool districts and uses 4.2.5 The use o f comparative data* is widely used throughout all school district units** to setthe information to targets, integrate new practices, and evaluate performance. The benchmarking* process isimprove its own systematically evaluated and improved as necessary.performance.

4.2.6 1 do not know.* * School district unitsare defined as specificschools, departments, orservices o f that schooldistrict.4 .3 Analysis and Use o f 4.3.1 Analysis o f conventional information** is done at the district level.

School DistrictPerformance* Data 4.3.2 Analysis o f conventional information** is done by the school district unit.

Reproduced

with perm

ission of the

copyright ow

ner. Further


without

permission.

Item # DescriptionDefinitions: 4.3.3 Information and data are collected, disaggregated, analyzed and disseminated district wide.*Performance data Information is used to gain understanding o f student and student group performance andincludes data or school district unit performance.information from allaspects o f the 4.3.4 Performance data* from all school district units are integrated and analyzed to assess overallorganization including district performance. Comparative data is used in the analysis.student performancemeasures, enrollment, 4.3.5 Comprehensive performance data* is analyzed, disseminated throughout the district anddiscipline, human accessible to all school district units. This information is an integral part o f planning processresources, business and used to adjust and establish key objectives essential to all decisions by all school districtoperations and units.community.

4.3.6 I do not know.** Conventionalinformation is definedas standardized andstate test scores,enrollment, attendance.dropout, discipline,operating budget

Reproduced

with perm

ission of the

copyright ow

ner. Further


without

permission.

5.0 Human Resource Development and Management This category examines how administrative, faculty and support personnel (all non-certified, business and operations staff throughout the district) are empowered to develop and utilize their full potential in order to enable the school district to realize its mission through the accomplishment o f key performance goals. Please consider the district driven efforts to build and maintain an environment conducive to performance excellence, full participation, and personal and organizational growth.Please select 1 from the five choices in each box which most closely describes your school district in that area. Fill in the circle

Item # Description5.1 Learning and 5.1.1 Insufficient attention is given to the learning and/or working climate by the leadership*.

Working Climate5.1.2 Some attention is given to creating a high performance environment for both students and

Definitions: personnel, but usually driven by school district unit ** administrator.*Leadership is defined asdistrict level senior 5.1.3 Efforts to create a positive, productive, safe environment are evident throughout theadministrators and board district.o f trustees.

5.1.4 A strong positive, productive and safe environment is visible throughout the district. High**School district units performance is fostered among personnel. Morale is high among personnel district-wide.are defined as specificschools, departments, or 5.1.5 Feedback from personnel is systematically obtained and used to guide decisions regardingservices o f that school the working and learning environment. This area is systematically reviewed and aligneddistrict. with the strategic planning process in the district.


ON

Reproduced

with perm

ission of the

copyright ow

ner. Further


without

permission.

Item # Description5.2 Work Systems* 5.2.1 Positions and work tasks are organized within traditional positions, roles, and

responsibilities with most o f the authority for decision-making with district level leadership.

5.2.2 Efforts are being made to move decision-making to teams within school district units**. There have been some decisions to de-centralize specific functions.

Definitions: 5.2.3 There are deliberate efforts underway to improve the district’s work systems through*Work systems are increased opportunities for self-directed responsibility o f personnel in all school units** indefined as how jobs, work designing, managing, and improving the district’s operations in order to accomplish theand decision-making are designed at a ll levels

district’s mission and key goals.

within the organization. 5.2.4 Work processes exist which allow for all personnel to contribute optimally in their school unit** through self-directed teams which foster flexibility, communication among school

**School district units units, and accomplishment o f key goals. Labor and management have collaborativelyare defined as specific schools, departments, or

designed personnel practices to support this.

services o f that school 5.2.5 Work and job functions are designed to accomplish district goals. Effectivedistrict. communication and collaboration exists across work functions. Processes for evaluation,

compensation, promotion, and recognition is exemplary. Work system processes are systematically evaluated on a regular basis.


157

Reproduced

with perm

ission of the

copyright ow

ner. Further


without

permission.

Item # Description5.3 Personnel Education,

Training, and Development

Definitions:* School district units are defined as specific schools, departments, or services o f that school district.

5.3.1

5.3.2

5.3.3

Opportunities for staff development are limited to in-service days. Personnel have some input into topics or design.

Needs assessments are conducted for faculty and support personnel. Staff development is based on the needs identified. Topics may or may not be aligned with the district’s strategic plan.

Processes are being designed to align education and training decisions to the district’s mission and key performance goals. Orientation for new personnel provides training on the district’s mission, goals, and work systems.

5.3.4 A process for determining, designing, and evaluating education and training is established and implemented across all school district units*. Education and training are provided in a variety o f ways. Application o f knowledge and skills is expected and supported through specifically designed strategies. Reaction to training is regularly assessed and evaluated for necessary improvements.

5.3.5 Processes for education and training are aligned to strategic planning processes. Decisions for education and training are made on the basis o f key performance goals and district competencies needed to achieve performance expectations. Staff development activities are measured for impact on learning, performance o f staff and effect on students’ performance. The design and delivery processes for staff development are evaluated on a regular basis.


L /l00

Reproduced

with perm

ission of the

copyright ow

ner. Further


without

permission.

Item # Description5.4 Performance 5.4.1 The performance appraisal* process is completed on an annual basis involving the

Appraisal Systems individual and the immediate supervisor. Little information is generated from the process to promote further development. Personnel find little value in the process.

Definition:* Performance appraisal 5.4.2 Performance appraisals* are done annually and result in the development o f specific goalsrefers to the regular established cooperatively by the individual and supervisor to promote further development.evaluation procedures o f Personnel find some value in the process, but agree it could be improved.(he performance o fpersonnel. 5.4.3 Personnel are engaged in developing performance appraisal* processes to support high

performance, stakeholder satisfaction, continuous improvement and collaboration between management and personnel.

5.4.4 Each personnel unit, including leadership, uses a performance appraisal* process which involves feedback from key identified stakeholders with whom they work closely. Results o f the appraisal process are linked to district’s professional development plans and continual development o f performance.

5.4.5 Performance appraisal* processes meet the professional needs o f all personnel and provide useful feedback from identified key stakeholders. Personnel work collaboratively with management to identify areas o f growth to further the district’s mission. Performance appraisal* processes are systematically reviewed to identify improvement areas.


sO

Reproduced

with perm

ission of the

copyright ow

ner. Further


without

permission.

Item # Description5.5 Employee Well- Being and Satisfaction

5.5.1

5.5.2

Employee motivation and satisfaction are given little attention.

Employee motivation and satisfaction are addressed through employee recognition events, or individual administrator’s efforts.

5.5.3 Processes are being developed to determine employee needs and expectations. The work environment is being improved to accommodate needs to the extent possible.

5.5.4 Employee satisfaction, well-being, and motivation are seen as key requirements to develop capabilities to realize its mission and reach its performance goals. Satisfaction surveys are administered on a regular basis. Personnel are able to identify, recommend, and make improvements. Multiple strategies for reward and recognition exist.

5.5.5 The accomplishment o f performance goals is recognized as fundamental to employee satisfaction. Processes to measure employee needs and satisfaction are integrated into strategic planning process. A variety o f opportunities are available to promote well-being, satisfaction and motivation throughout the district. These processes are systematically evaluated to make improvements as necessary.


O no

Reproduced

with perm

ission of the

copyright ow

ner. Further


without

permission.

6.0 Educational and Operational Process* ManagementThis category examines how work is designed, managed,

accomplished, and evaluated for effectiveness. Please think about the extent to which all work processes are designed, managed, accomplished and evaluated with a focus on stakeholder satisfaction in all work units. Also consider how processes are designed, effectively managed, and improved to achieve better performance.(*Process refers to the steps and sequence o f steps done in a specific work activity; e.g. enrolling a new student in school; requisition o f instructional supplies; hiring new personnel.)Please select 1 from the five choices in each box which most closely describes your school district in that area. Fill in the circle completely next to the statement selected with pencil or black ink.__________________________________________________

Item Description6.1 The design and managemento f educational programs*

Definitions:*Educational programs are defined as a ll programs and services provided to students and conducted by professional, certified personnel or non-certified personnel under the supervision o f certified personnel

6.1.1

6.1.2

6.1.3

6.1.4

Design and management o f programs are based on federal or state regulations, traditional practices and individual preferences or opinions.

Design and management o f some programs are based on student performance on statewide testing, perceived student needs, and/or preferences o f management.

Processes are being designed and developed to base decisions for educational programs and services on student needs, performance results, stakeholder satisfaction and research- proven practices. Student performance expectations and curricula are being developed to align with these needs, expectations, and practices.

Many programs and services are designed and managed through established processes based on a review o f student needs, performance results, stakeholder satisfaction and research-based best practices. Curricula are aligned to performance standards in most units and levels.

O n

Reproduced

with perm

ission of the

copyright ow

ner. Further


without

permission.


6.1.6

A systematic process exists for the design and management o f all educational programs.All educational programs and services meet established standards to assure high quality o f programs and services. Programs and services are based on a systematic review o f student needs, performance results, community demographics, stakeholder expectations and cost analyses. All curricula are aligned to performance expectations for all grades and courses. This process is systematically evaluated and improvements are made as determined necessary.

1 do not know.6.2 Delivery o f educational

programs and services

Definition:*Educational Programs and services are defined as all programs and services provided to students and conducted by professional, certified personnel or noncertified personnel under the supervision o f certified personnel

6.2.1

6.2.2

6.2.3

6.2.4.

Delivery o f educational programs and services* are based on federal or state regulations, traditional practices and individual preferences, opinions. Local curricula are inconsistently delivered.

Delivery o f some educational programs and services* are based on student performance information and in some cases, research-proven effective practices. Curricula are delivered consistently in some school district units.

Processes are being developed to improve the delivery o f educational programs* through a systematic review o f performance results, research-based best practices and stakeholder expectations.

Educational programs and services* are delivered to meet student needs, prevent school failure, optimize student achievement, integrate research-based practices and respond to stakeholder expectations. Delivery o f services is evaluated based on performance information and improvements are made as necessary.

Onto


6.2.6

A systematic process exists to insure delivery o f educational programs and services* that optimally meet student needs, insure student success and exceed stakeholder expectations. Processes are proactive to prevent student failure. The process for program review is systematically evaluated as an integral part o f the strategic planning process and improvements made as necessary.

1 do not know.6.3 Design, management and delivery o f educational support services

* Educational support services include all programs and services which support the educational programs, such as business operations, transportation, public relations, purchasing, clerk, legal, volunteers, fo o d service, records, buildings and grounds.

* *Stakeholder- Individuals or groups, both internal to the school (students, personnel) and external (parents, community members, business) which

6.3.1

6.3.2

6.3.3

6.3.4

Educational support services are designed, managed and delivered based on traditional practices and individual preferences. Frequent complaints from internal and external stakeholders occur with no defined process to address them. Decisions are made with little to no input or involvement from stakeholders**.

Educational support services* are designed with some input from stakeholders**. Cost analyses o f operations are routinely done. Some stakeholder** satisfaction information has been acquired. Required audits are performed for district finances.

Processes are being developed to include problem identification, analyses o f performance results, cost analyses, and stakeholder** requirements. Stakeholders o f services are included in the design. Information is being acquired to determine stakeholder satisfaction.

Most services have processes*** which support the mission o f the district through a focus on improved productivity, efficiency and quality to support optimal student performance. Processes include early identification o f problems, corrective action processes, and comparing processes to other external organizations. Stakeholder** requirements and satisfaction are regularly assessed and used to make improvements.

Reproduced

with perm

ission of the

copyright ow

ner. Further


without

permission.

Item # Descriptionare affected by the conditions and quality o f education or the preparedness o f graduates; also includes state agencies and their requirements

***Process refers to the steps and sequence o f steps done in a specific work activity; e.g. enrolling a new student in school; requisition o f instructional supplies; hiring new personnel

6.3.5

6.3.6

A systematic process exists to design, manage and deliver all support services v/hich involve stakeholders** and is based on the educational needs o f students, performance results o f that service, cost analyses, productivity measures, and stakeholder** satisfaction. This process is systematically reviewed and improvements are made as necessary.

1 do not know.

6.4 Data and Information Processes*

Definition:* Includes the collection, management and dissemination o f data on enrollment, achievement, operations, stakeholder satisfaction and other pertinent information which are used in evaluation and planning processes

6.4.1

6.4.2

6.4.3

Data collected includes district, school, grade-level enrollment, attendance, dropout, and student performance on statewide testing. Information is not widely disseminated throughout the district. Some decisions made by leadership are based on this information.

Additional data regarding enrollment, district demographics and utilization o f specific educational programs and services is collected. Data collected is determined by program managers or building principals. Some analyses occur and are used to make recommendations and decisions for improvement.

Processes are being developed to collect, analyze, disseminate and use more comprehensive information necessary to determine improvement areas. Additional measures o f student performance are being developed for frequent indicators o f learning. Stakeholder needs and satisfaction data are included.

CT\4̂

Item # Description6.4.4 Decision-making processes are based on information on student and operational

performance. Information includes all measures o f student achievement by grade level, assessment o f educational programs and services, assessment o f support services, and comprehensive building level data including school, grade and course enrollment, disciplinary, graduation, drop-out, parent involvement, and stakeholder satisfaction. Processes exist to collect, analyze and disseminate specific data which may be required to make improvements.

6.4.5 A systematic process exists for the collection, analysis, dissemination and use o f data and information. Strategic decisions affecting the direction o f the district and target goals are made using this information. Comprehensive assessments exist for monitoring student performance, evaluating effectiveness o f all programs and services, and for stakeholder satisfaction. Data and information processes are systematically evaluated and necessary improvements are made.


6 .5 Internal and external* communication process**

Definition:* Internal refers to personnel and students within the school district and external refers to parents and community stakeholders

6.5.1

6.5.2

6.5.3

There is unclear and inconsistent communication regarding the direction and goals within the district. Some efforts made to communicate with parents and community.

Communication regarding the direction and goals within the district is usually done on an annual basis, but not frequently. School newsletters or district newsletters are sent to parents.

Internal communication is improving but still inconsistent. Efforts are being made to improve communication with external stakeholders.

Reproduced

with perm

ission of the

copyright ow

ner. Further


without

permission.

Item # Description* Communication process refers to methods used to inform and seek opinion form others.

6.5.4

6.5.5

6.5.6

Communication is consistent, timely and thorough, both with internal and external stakeholders.

A clear and understood process for thorough communication both internally and externally exists. District personnel, students, parent and community are fully informed in a timely manner. Communication process** is systematically evaluated for improvement opportunities.

I do not know.

6.6 Management o f supplier and partnering* processes

Definitions:* Suppliers are those businesses or individuals with which the district contracts for specific services such as training, consulting, transportation, legal, etc. Partnering processes are defined as those relationships with other organizations, community agencies, businesses to design, implement, provide services

6.6.1

6.6.2

6.6.3

6.6.4

No specifications are established by school district for expectations from suppliers.There is little effort to develop collaborative relationships with stakeholders.

Some specifications and expectations exist for some supplier areas. Problems are addressed as they occur. The district has developed collaborative relationships with parent, local businesses and some community organizations.

Efforts are being made to develop a proactive approach to problem identification and prevention with suppliers*. Requirements are being developed for some suppliers and communicated to achieve improved quality o f supplies and materials. District is making increasing efforts to develop partnerships with stakeholder groups to support mission and goals. Efforts are being made to involve them in decision-making processes.

Established processes exist to determine supplier and partner* requirements for most areas. Requirements are communicated to assure expected performance by supplier and partner. Stakeholders** are asked to evaluate effectiveness.

O nO n

Reproduced

with perm

ission of the

copyright ow

ner. Further


without

permission.

Item # Descriptionfor the students or stakeholders o f the district

**Stakeholder- Individuals or groups, both internal to the school (students, personnel) and external (parents, community members, business) which are affected by the conditions and quality o f education or the preparedness o f graduates; also includes state agencies and their requirements

6.6.5

6.6.6

A systematic process exists to establish and communicate requirements to suppliers and partners*. Processes include regular evaluation o f their effectiveness, quality and costs. Suppliers and partners* share in the district’s goals. Processes are regularly evaluated to determine improvements necessary.

1 do not know.

o\

Reproduced

with perm

ission of the

copyright ow

ner. Further


without

permission.

7.0 School District Performance Results This category examines the district’s results in student achievement, quality o f programs and services, and operations. Please think about the current results that your school district can demonstrate in student achievement, human resources and stakeholder satisfaction. Please consider how your school district performance results compare with comparable districts.Please select one of the five choices in each item which most closely describes your school district by filling in the circle next to it with a pencil or black pen.

# Description7.1 Student performance

results

*Stakeholder- Individuals or groups, both internal to the school (students, all personnel) and external (parents, community members, business) which are affected by the conditions and quality o f education and preparedness o f graduates.

* * Benchmarking is an improvement process in which an organization compares its performance against best-in-class organizations, determines how these organizations

7.1.1

7.1.2

7.1.3

7.1.4

7.1.5

7.1.6

Student performance, as revealed by national, state or local measures, reveal significant deficiencies and are below expectations o f stakeholders* and other comparable districts

Performance by some students on national, state or local measures reveals satisfactory results when compared to state or national results. Below average performance results exists for other students.

Student performance on national, state or local measures is improving in specific areas and by increasing numbers o f students. Data is being accumulated from past few years to determine if this is a trend.

Student performance on national, state or local measures is showing consistent upward trends over time. Performance indicates improvement over the past few years compared to other comparable districts, both in and out o f state.

Student performance results are sustained on all measures and by all student groups. All student groups recognize school district for outstanding performance. Other school districts use performance results for benchmarking**. Graduate follow-up information reveals successful employment or completion o f higher education.

1 do not know.

O n00

Reproduced

with perm

ission of the

copyright ow

ner. Further


without

permission.

# Description

7.2 Student Conduct Results

7.2.1 Unacceptable rates o f student absenteeism, tardies, suspensions, expulsions for students exist.

*Indicators involving student behavior such as disciplinary infractions,

7.2.2 Recent improvement in at least one student conduct indicator*. Results are consistently reviewed for progress.

suspensions, expulsions, arrests, etc.

**Benchmarking is an

7.2.3 Improvement trends are beginning to emerge in 3 or more indicators*. Improvements are emerging over time when compared to other comparable school districts. Some student groups show little improvement.

improvement process in which an organization compares its performance

7.2.4 Significant gains are noted in student conduct areas over a three year period among most student groups, compared to in state and out o f state comparable districts.

against best-in-class organizations, determines how these organizations achieved their performance levels and uses the

7.2.5 Sustained results are demonstrated in all areas o f student conduct and among all student groups. District performance in this area is used for a state or national benchmarking**. Graduate follow-up information reveals successful employment or completion o f higher education.

information to improve its own performance.


ONNO

Reproduced

with perm

ission of the

copyright ow

ner. Further


without

permission.

# Description

7.3 Student and Stakeholder Satisfaction Results

* Stakeholder-Individuals or groups, both internal to the school (students, personnel ) and external (parents, community members, business) which are affected by the conditions and quality o f education or the preparedness o f graduates; also includes state agencies and their requirements

**Benchmarking is an improvement process in which an organization compares its performance against best-in-class organizations, determines how these organizations achieved their performance levels and uses the information to improve its own performance_________

7.3.1

7.3.2

7.3.3

7.3.4

7.3.5

7.3.6

Anecdotal information suggests low satisfaction among students and stakeholders*.

Results collected through a systematic process reveal significant low satisfaction among students and stakeholders with school district performance and operations.

Some areas reveal improvement in stakeholder* and student satisfaction when compared to baseline data.

Many areas reveal significant gains in stakeholder * and student satisfaction. Long term trends are emerging. District is comparing trends with other comparable school districts.

Sustained results over several years reveal high satisfaction o f all student and stakeholder* groups for the performance o f student achievement, quality o f educational programs and services, and support services. The school district is used for benchmarking** in and out o f state.

I do not know.

Reproduced

with perm

ission of the

copyright ow

ner. Further


without

permission.

Description7 .4 Human Resource*

Results

* Human resource indicators include employee well-being, labor relations, satisfaction, professional de velopment, work system performance and effectiveness.

**Benchmarking is an improvement process in which an organization compares its performance against hest-in-class organizations, determines how these organizations achieved their performance levels and use the information to improve its own performance.

7.4.1

7.4.2

7.4.3

7.4.4

7.4.5

7.4.6

Anecdotal information reveals low satisfaction o f personnel and poor employee-employer relationships. High absence and turnover rates by personnel exist. High rate o f grievances exist.

No formal measurement exists but anecdotal information suggests that most personnel appear satisfied. Personnel accept current terms o f employment. Personnel absenteeism, turnover and recruitment are not considered to be problems. Reactions to staff development activities are generally positive.

Systematic attempts to measure human resource indicators* reveal some emerging improvement patterns.

Results indicate improvements in many human resource indicators*. Absenteeism rates for personnel and turnover show steady decline. Employee satisfaction measures reveal consistent upward trends. Staff development activities reveal impact on instructional and work practices.

Employee satisfaction is high among all personnel classifications. District is recognized as an organization to benchmark**. Personnel absenteeism, turnover is low , with sustained employment pool o f high quality qualified applicants. Staff development opportunities indicate sustained results on improved student performance.

1 do not know.

Reproduced

with perm

ission of the

copyright ow

ner. Further


without

permission.

Description7.5 Educational Program and Service Results

Definitions:* Educational programs and services are defined as all programs and services provided to students and conducted by professional, certified personnel or non- certified personnel under the supervision o f certified personnel

**Benchmarking is an improvement process in which an organization compares its performance against best-in-class organizations, determines how these organizations achieved their performance levels and uses the information to improve its own performance_________

7.5.1

7.5.2

7.5.3

7.5.4

7.5.5

7.5.6

No results, other than state, national or local student achievement measures, are available to determine effectiveness o f educational programs and services*. Enrollment data is used as indicator o f effectiveness for some educational programs or services*.

Some educational programs and services * evaluations exist but demonstrate poor performance results. Compliance with federal, state, local requirements exists.

Some educational program and service* evaluations are beginning to show improvements on several indicators compared to baseline data.

Steady gains are being made in enrollment, attendance and student graduation rates when compared to comparable school districts. Educational programs and services* are demonstrating improvements in performance results.

Sustained results exist indicating outstanding performance for educational programs and services. High student satisfaction exists. High attendance, high graduation rates exist with graduate follow-up information revealing successful employment or higher education after graduation. District is used as a benchmark** both state and nationally for excellence in educational programs and services.

I do not know.

-jto

Reproduced

with perm

ission of the

copyright ow

ner. Further


without

permission.

Description7.6 Educational support services

Definitions:*Educational support services include all programs and services which support the educational programs, such as business operations, transportation, public relations, purchasing, clerk, legal, volunteers, food service, records, buildings and grounds.

**Benchmarking is an improvement process in which an organization compares its performance against best-in-class organizations, determines how these organizations achieved their performance levels and use the information to improve its own performance_________

7.6.1

7.6.2

7.6.3

7.6.4

7.6.5

7.6.6

Audits o f district reveal some areas o f non-compliance with regulatory or legal requirements. Little or no information is available from educational support services*.

Some performance data exists for some services. Results reveal significant gaps in stakeholder satisfaction. Regulatory and legal compliance is improving.

Performance results o f support services reveal some recent improvement

Improvements are evident in some educational support service areas. Benchmarks are being used from other comparable organizations and reveal upward trend.

Sustained improvement results exist with high customer satisfaction, cost effectiveness, efficiency and productivity. District educational support services are used as benchmarks** for other comparable districts.

1 do not know.

U>

Reproduced

with perm

ission of the

copyright ow

ner. Further


without

permission.

1. Could this instrument be useful as a tool to study the seven areas o f your school district?

Extremely Usefi.il Somewhat Useful Little Use No Use______

Please explain why or why n ot._________________________________________________________________________________

2. Could this instrument be useful as a tool to help district personnel determine areas for future school district improvement?

Extremely Useful Somewhat Useful Little Use No Use______

Please explain why or why not._________________________________________________________________________________

My most sincere appreciation for your responses

174

175

Appendix B:

Panel o f Experts Used in Content Validation

Dr. Susan Leddick— Trainer, consultant, practitioner o f quality and the Malcolm Baldrige National Quality Award.

Dr. Roland Smith— Consultant in quality applications and Malcolm Baldrige Examiner at the state and national level.

Dr, Jim Shipley— Trainer in Pinellas County, Florida schools. Designed a similar instrument for use in those schools. Malcolm Baldrige Examiner at the national level.


176

Appendix C:

Letter to Panel o f Experts

August, 1997

Dear

Thank you for agreeing to be a content area expert for my dissertation study. I have enclosed the current draft o f the instrument for your review and critique. The feedback that I am requesting is in the following areas.

• Alignments o f the intent, design and content o f the instrument with the Malcolm Baldrige National Quality Award criteria.

• Clarity o f items.• Specific suggestions for improving the instrument• Extent to which this instrument could be a guide for self-study in a school district.

The research questions posed in this study are:1. How do educators perceive their own school district’s performance based on an

instrument designed using the Malcolm Baldrige National Quality Award Education Criteria?

2. Are there differences in these ratings based on type o f educators or size o f district?

3. Do educators find this instrument a useful tool to study these areas o f a school district?

4. Do educators believe this instrument could be useful in determining school improvement needs?

Please feel free to give your comments to me in the easiest way for you. You can reach me through these numbers, fax, or e-mail.

I so appreciate your willingness to advise me and share your expertise.

Sincerely,

Sally Anderson


177

Appendix D:

Matrix o f Population Sample


Grp. Population N % S** N % S** N % S**

1 5000 + 13 13.4 10 220 44 96 6736 51.5 191

2 2500-4999 13 13.4 10 84 16.8 37 2358 18 67

1000-2499 27 27.8 21 106 21.2 46 2374 18 67

4 500- 999 22 22.6 18 55 11 24 987 8 29

5 1- 499 21 21.6 17 31 6 14 617 5 19

Total 26 217 373


N S N S N S

State Totals 97 76 500 217 13,076 373

*Note: Group classifications by size are based on the 1996-97 Annual Statistical Report prepared by the Idaho State Department o f Education.

**Note: Proportional sample number from total sample.


178

Appendix E:

Codes on the Instrument

Size o f District Type o f Educator:

1 S1 P1 T

2 S2 P2 T

-> S3 PJ T

4 S4 P4 T

5 S5 P5 T


179

Appendix F:

Cover Letter and Directions

October 6, 1997

Dear Colleague,

As educators, w e work diligently to improve student performance and optimize the effectiveness o f our schools. It is on this shared responsibility that I ask you to participate in my dissertation study.

The enclosed instrument is not a survey. It is an analysis tool designed to assist school districts in the process o f self-study for the purpose o f continual improvement o f school systems. Selections you make on this instrument are neither right nor wrong, good nor bad. They should reflect your perception o f where your school district is at this point in time in that area. It is your honest and realistic professional opinion that will provide me the information for my study.

This analysis instrument is adapted from the Malcolm Baldrige National Quality Award - Education Pilot, the curriculum audit process, and the 1996 accreditation standards from the Northwest Association o f Schools and Colleges. The instrument asks you to reflect on seven categories common to every school district. Within each category are specific items to which you will respond.

You, specifically, are in a unique position o f responsibility to influence and implement school improvement efforts in your district. I realize that I am asking very busy professionals to use some o f your already limited time towards this endeavor. Since it is not a survey, it may take you a little longer to thoughtfully respond. I am appealing to your commitment to the improvement o f education in Idaho, your curiosity about the findings o f this research, and your willingness to assist a colleague who shares your interest and desire for continued excellence in education.

Your returned response is critical to the findings. R ead the directions thoroughly to assist you with this un ique project. Please return the completed instrument to me in the self- addressed stamped envelope provided. I am asking that you send it to me by October 27, 1997.

Please accept the pen as a small token o f my appreciation for participation in this study. Please feel free to use it when completing the enclosed analysis. I would welcome the opportunity to discuss my results with you personally and the implications for how school improvement might be approached. You may reach me at the J.A. & Kathryn Albertson Foundation, 208-342-7931.

My sincere appreciation,

Sally Anderson, Doctoral Student University o f Idaho


180

Directions

Please Read Carefully! Thank You! Please Read Carefully! Thank You!

• The Performance Analysis for School Districts has been prepared on paper that will be scanned. Therefore, please fill in the circle next to the item you select completely with pencil or black ink. Do not fold or tear. Do not staple. Use clip.

• This instrument is an analysis tool to assist you in a reflective self-study o f the major components o f a school district. The questions do NOT ask about a specific school. Rather, they focus on the whole school district.

• Unlike a survey, this is not intended to be responded to quickly. It is intended to be a reflective tool for your thoughtful consideration regarding the complete scope o f your school district operations.

• As the sole researcher, I can assure you that all information is confidential and anonymous. Results will be reported in grouped data only. No individual responses are reported.

• It is not necessary to do any research or consult with others to help you answer the items, unless you wish to. I am interested in your perspective, your knowledge, your perceptionbased on what you know from your view in your school district.

• There is the opportunity to select an “I do not know category” i f you truly feel you can not make a ju dgem en t on that item.

• Some terms used are defined in each sub-category. Please refer to the definitions under each sub-category in the upper left-hand corner for added clarity.

• If you feel that you do not have information about an area, answer based on what it appears to be from your point o f view.

• Please do not write in any comments unless they are asked for.

• Please respond to every section to the best o f your ability.

• Select only one o f the five choices in each section that most accurately reflects your perception o f your school district at this point in time. I am interested in knowing your perception.

• Return the completed Performance Analysis for Schools to me in the self-addressed stamped envelope provided by October 27, 1997.

• If you have any questions or would like the results o f the study, please call me at the J. A. & Kathryn Albertson Foundation (208-342-793 1).


181

Appendix G:

Postcard Reminder

October 25, 1997

Dear Colleague,

Recently, I sent you a school district self-study instrument in the mail to collect information for my dissertation research study. If you have returned it to me already, I am most grateful. If you have not, I ask that you take the time to do so.

Assessing the performance o f organizations is becoming an extremely important practice and as school districts, we will need to consider such a process. The instrument serves only to find out your perceptions o f the current performance o f your school district using the Malcolm Baldrige National Quality Award as the criteria. Please send me the completed instrument by November 5, 1997.

Thank you for your contribution to my study and your gift o f time.

Sally Anderson, Doctoral Candidate (208-342-7931)University o f Idaho


182

Appendix H:

Letters o f Support from IASA and IEA

IDAHO ASSOCIATION OF SCHOOL ADMINISTRATORS777 South Latah Boise. ID 83705

PHONE: (208) 345-1171 FAX: 345-1172M ichael L . F rien d , Executive D irec to r

E-MAIL: [email protected] WEB SITE: httpi//www.lewiston.icl2.id.us.oiganizations/iasa/"Leadership For Tomorrow's Leaders"

October 1, 1997 Dear Participant:The Idaho Association of School Administrators is pleased to endorse this research study and provide the opportunity for you to participate in the project. As the key leaders of school improvement efforts, our perspective is vitally important; this study recognizes that fact. The IASA has great interest in the findings of this particular research study designed by Sally Anderson, a University of Idaho doctoral student. The results of the research will be published in our journal, Perspectives.

Your completion of this survey instrument is of critical importance to the study. The responses will be kept confidential and handled only by Sally. Results will be reported in the aggregate.We recognize that this task will require valuable time and considerable thought on your part. Consider the use of your time as an investment in our future approach to school improvement and the development of high performance districts. Please take the necessary time to support your colleague in this project!Thanking you in advance for your assistance.

Mike Friend Executive Director

Affiliated DivMoaf: Idaho School Superintendent!' Auocietioo: Idaho Ataociaiioo oT Secondary School Principals: Idaho Aisociarion of Betncntaiy School Principal!; Idaho Association of Special Education Administrators AUkd Aamdatec Idaho School District Council: Northwest Women for Educational Action:

Idaho School Business Officials; Idaho Rural Schools Association; Idaho Middle Level Aisociarion; Idaho School Public Relations Association;Idaho Association of Educational Office Professionals; Idaho Association for Supervision and Curriculum Development

Sincerely,


mailto:[email protected]

http://www.lewiston.icl2.id.us.oiganizations/iasa/

IDAHO EDUCATION ASSOCIATIONP.O. BOX 2638, BOISE. IDAHO 83701,620 NORTH SIXTH STREET. 83702INTERNET: http://www.idahoea.org

208/344-1341 'FAX 208/336-6967

ROBIN NETTINGAPresidente-mail: [email protected]

JAMES A. SHACKELFORDExecutive Director e-mail: [email protected]

October 1, 1997

Dear Participant:

The Idaho Education Association is pleased to have the opportunity to encourage you to assist with this important study. As the cornerstone of all school improvement efforts, your perspective is vitally important and this study recognizes that. The IEA is interested in the findings of this research study that has been designed by Sally Anderson, doctoral student with the University of Idaho.

You have been randomly selected to be a participant in this study. Your completion of this instrument is both critically important to the study and could be of practical significance to you and your district. The results of this study are entirely confidential and handled only by Sally Anderson. Results will only be reported in the aggregate.

We recognize that it will require some of your time and considerable thought. Consider the use of your time as an investment towards our future approach to school improvement and high performance school districts. Please take the necessary time to support your colleagues in this study by completing this instrument.

Thank you for your assistance.

Sincerely,

Executive DirectorJAS/jh


http://www.idahoea.org



184

Appendix I:

Districts by Enrollment Size

District DistrictNumber Name

Classification 1— Districts o f Over 5,000:

001 Boise

002 Meridian

025 Pocatello

091 Idaho Falls

131 Nampa

271 Coeur D ’Alene

093 Bonneville

411 Twin Falls

082 Bonner County

151 Cassia County

331 Minidoka County

340 Lewiston

132 Caldwell

13 districts to ta lfo r Classification I

Classification 2— Districts o f 2,500 to 4,999:

055 Blackfoot

193 Mountain Home

(table continues)


185


Classification 2, continued:

321 Madison

273 Post Falls

251 Jefferson County

272 Lakeland

139 Vallivue

261 Jerome

221 Emmett

061 Blaine County

281 Moscow

215 Fremont County

03 Kuna

13 districts total fo r Classification 2

Classification 3— Districts o f 1,000 to 2,499:

52 Snake River

201 Preston

60 Shelley

241 Grangeville

134 Middleton

371 Payette

(table continues)


186

DistrictNumber

DistrictName

(Classification 3, continued)

33 Bear Lake County

101 Boundary County

381 American Falls

171 Orofino

21 Marsh Valley

431 Weiser

391 Kellogg

412 Buhl

322 Sugar-Salem

291 Salmon

41 St. Maries

372 Fruitland

413 Filer

231 Gooding

150 Soda Springs

414 Kimberly

370 Homedale

401 Teton County

421 McCall-Donnelly

232 Wendell

(table continues)


187



59

351


Classification 4— Districts o f from 500 to 999:

137 Parma

58 Aberdeen

371 N ew Plymouth

392 Wallace

252 Ririe

253 West Jefferson

262 Valley

192 Glenns Ferry

363 Marsing

181 Challis

286 Whitepine

304 Kamiah

136 Melba

285 Potlatch

111 Butte County

(table continues)

Firth

Oneida County


188

DistrictNumber

DistrictName

(Classification 4, continues)

365 Bruneau-Grand View

148 Grace

202 West Side

044 Plummer-Worley

242 Cottonwood

341 Lapwai

133 Wilder

072 Basin

t districts total fo r Classification 4

Classification 5— Districts o f from 1 to 499:

312 Shoshone

422 Cascade

233 Hagerman

013 Council

283 Kendrick

135 Notus

415 Hansen

417 Castleford

(table continues)


189

District District

Number Name


282 Genesee

071 Garden Valley

274 Kootenai

418 Murtaugh

073 Horseshoe Bend

432 Cambridge

305 Highland

182 Mackay

161 Clark County

342 Culdesac

O il Meadows Valley

302 NezPerce

149 North Gem

314 Dietrich

121 Camas County

316 Richfield

234 Bliss

392 Mullan

292 South Lemhi

(table continues)


190

District

Number

District

Name


382 Rockland

433 Midvale

*092 Swan Valley

*394 Avery School

*364 Pleasant Valley

*383 Arbon

*191 Prairie

*416 Three Creek

*Note: Eliminated due to size— less than 100 and no superintendent.



IMAGE EVALUATION TEST TARGET (QA-3)

✓

' v

A

1.0

1.1

L i ■ 2.8 L£ _Ui K£If U£

■4° lllll 2.0

2 3

2.2

1.8

1.25 1.4 1.6

150mm

IIWlGE.Inc1653 East Main Street Rochester. NY 14609 USA Phone: 716/482-0300 Fax: 716/288-5989

O 1993. Applied Imago. Inc.. All Rignts Reserved


thesis

Documents

perm ission

copyright owner

umi company

umi microform

original manuscript

umi films

umi number

contact umi