early detection of dropout risk: measuring … · discussion and implications of these findings are...
TRANSCRIPT
EARLY DETECTION OF DROPOUT RISK:
MEASURING STUDENT ENGAGEMENT AT THE
ELEMENTARY SCHOOL LEVEL
by
CHANDRA P. CARTER
(Under the Direction of Amy L. Reschly)
ABSTRACT
Early school withdrawal, commonly referred to as dropout, is associated with a plethora of
negative outcomes for students, schools, and society. Student engagement, however, presents a
promising theoretical model and cornerstone of school completion interventions. The purpose of
the present study was to validate an elementary version (SEI-E) of a student engagement survey,
the SEI. The psychometric properties of this measure were assessed based on the responses of an
ethnically diverse sample of 2,504 students from an urban locale. Exploratory factor analyses
indicated that the six-factor model of engagement obtained with the original SEI (validated on
students in grades 6-12) was replicated with the present sample of early elementary school
students (grades 3-5). Discussion and implications of these findings are presented in the context
of student engagement and dropout prevention.
INDEX WORDS: Student engagement, Early detection, Dropout, SEI
EARLY DETECTION OF DROPOUT RISK:
MEASURING STUDENT ENGAGEMENT AT THE
ELEMENTARY SCHOOL LEVEL
by
CHANDRA P. CARTER
B.S., Xavier University of Louisiana, 2007
A Thesis Submitted to the Graduate Faculty of the University of Georgia in Partial Fulfillment of the
Requirements for the Degree
MASTER OF ARTS
ATHENS, GEORGIA
2010
© 2010
Chandra P. Carter
All Rights Reserved
EARLY DETECTION OF DROPOUT RISK:
MEASURING STUDENT ENGAGEMENT AT THE
ELEMENTARY SCHOOL LEVEL
by
CHANDRA P. CARTER
Major Professor: Amy L. Reschly
Committee: Michele Lease Stacey Neuharth-Pritchett Electronic Version Approved: Maureen Grasso Dean of the Graduate School The University of Georgia August 2010
iv
ACKNOWLEDGEMENTS
I would like to extend sincere gratitude to my family who has provided unwavering support. I
would especially like to thank my mother, Catrina Weston, for her unfailing love, guidance, and
wisdom. To my life partner, Rodney L. Brown, thank you for your endless encouragement. I would
also like to thank my major professor, Amy Reschly, for her support and academic guidance
throughout this process and Drs. Michele Lease and Stacey Neuharth-Pritchett for their useful
feedback. Finally, I would like to thank my friends and program members for their assistance and
helpful insight.
v
TABLE OF CONTENTS
Page
ACKNOWLEDGEMENTS .................................................................................................................. iv
LIST OF TABLES ............................................................................................................................... vii
LIST OF FIGURES ............................................................................................................................ viii
CHAPTER
ONE INTRODUCTION ............................................................................................................ 1
Consequences and Costs of Dropout ............................................................................ 1
Dropout Research ........................................................................................................... 3
Alternative Conceptualizations of Dropout ................................................................... 4
Student Engagement and Dropout ................................................................................. 6
Student Engagement Models ........................................................................................ 8
Measuring Student Engagement .................................................................................... 9
Purpose of the Present Study ....................................................................................... 13
TWO METHOD ....................................................................................................................... 17
Participants .................................................................................................................. 17
Instrument Construction .............................................................................................. 17
Data Collection Procedures ......................................................................................... 18
Analysis Logic and Procedures ................................................................................... 19
THREE RESULTS ...................................................................................................................... 23
Descriptive Statistics ................................................................................................... 23
Exploratory Factor Analyses ....................................................................................... 24
FOUR DISCUSSION AND CONCLUSIONS ......................................................................... 42
vi
Limitations and Directions for Future Research .......................................................... 43
Implications for School Psychologists ....................................................................... 45
REFERENCES ...................................................................................................................... 46
vii
LIST OF TABLES
Page
Table 1: Alterable Variables Associated with Dropout .................................................................16
Table 2: Comparison of item wording between the Student Engagement Instrument (SEI)
and the Elementary Version (SEI-I) ................................................................................21
Table 3: Descriptive Statistics for Items within the SEI-E ............................................................28
Table 4: Five-factor Solution Pattern Matrix .................................................................................30
Table 5: Six-factor Solution Pattern Matrix ...................................................................................32
Table 6: Seven-factor Solution Pattern Matrix ..............................................................................34
Table 7: Items Comprising the Six-factor Model ..........................................................................36
Table 8: Six-factor Solution Cronbach’s Alpha Values ................................................................38 Table 9: Variable Communalities using a Six-factor Model .........................................................39
viii
LIST OF FIGURES
Page
Figure 1: EFA Scree Plot ....................................................................................................................... 41
1
Chapter One
Introduction
There is little contention about the negative individual and societal costs of early school
withdrawal, commonly referred to as dropout. Therefore, it is only logical that the dropout crisis
is currently in the forefront of media, public policy, and educational research. Researchers in a
multitude of disciplines have examined student dropout and associated factors in an effort to
explicate the reasons that students leave school before graduating. The construct of student
engagement has arisen as a principal framework when conceptualizing dropout (Christenson et
al., 2008; Finn, 2006) and formulating dropout interventions; however, further research is needed
to inform prevention efforts, particularly with elementary-age students. Efficacious dropout
prevention and intervention programs may prevent students at-risk for dropout from enduring the
deleterious consequences of leaving school prior to earning a diploma.
Consequences and Costs of Dropout
The negative prognoses for high school dropouts are well established. Students who drop
out of school are at exponentially greater risk for negative outcomes than are their counterparts
who graduate or earn a General Equivalency Diploma (GED) (Kirsh, Braun, Yamamoto, & Sum,
2007; Reschly & Christenson, 2006b). For example, students who do not complete high school
are more likely to require government assistance such as welfare or Medicaid, are more likely to
live in poverty, experience higher levels of unemployment or underemployment, and are more
likely to engage in deviant or criminal behavior and experience incarceration [Alliance for
Excellent Education (AEE), 2009]. Additionally, high school dropouts compose approximately
2
80% of the general prison population and juveniles in court (AEE, 2009), perhaps indicating that
dropouts may sometimes resort to criminal activity when employment opportunities or legitimate
income are insufficient.
The verity that the United States economy is currently in a state of economic turmoil is
widely accepted. Persons who are not adequately prepared for work may face particularly
difficult challenges as they may be ill-equipped to compete in the advancing global marketplace
(Kirsh et al., 2007). According to most recent national estimates, approximately 71% of students
graduate on time with a regular high school diploma (AEE, 2009). While this estimate seems
promising at first glance, further examination reveals that of the four million students who enter
ninth grade each fall, one-third of these students will dropout before earning a high school
diploma (AEE, 2008). The students who leave school prematurely may be placed at-risk for
enduring the aforementioned injurious consequences.
Dropping out of high school is not only costly to the individual, but also to society. Each
year, the United States government spends approximately $76 billion dollars providing for
dropouts and their families (AEE, 2008). According to most recent estimates, each dropout costs
the nation approximately $260,000 in lost earnings, taxes, and productivity over the course of his
or her lifetime (AEE, 2009). If graduation rates remain stable over the next decade, the national
cost of dropout will be a staggering three trillion dollars (AEE, 2008). High dropout rates also
negatively affect the nation’s ability to compete in the ever-changing global economy, as dropouts
represent uncultivated human potential and decrease the overall productivity of the United States’
citizenry (Kirsch et al., 2007). It is clear that student dropout is detrimental to both individuals and
society, and efforts to prevent dropout are of utmost importance.
3
Whereas educational researchers have historically examined early school withdrawal,
recent political goals and legislation put forth by the U.S. Congress and other governing bodies
have resulted in a major upsurge in the interest of high school dropout (Reschly, Appleton, &
Christenson, 2010). The last twenty years of legislation (e.g., Title I, No Child Left Behind) has
bolstered the need for educational research regarding best practices for increasing student
competencies. Now more than ever, students need immense preparation to become productive
citizens and capable competitors in an increasingly aggressive global economy (McDonnell,
2005; Reschly et al., 2010). Consequently, focus on alleviating dropout and promoting school
completion has increased exponentially.
Dropout Research
With the fundamental aim of preventing early withdrawal, researchers have paid
consistent attention to predictors of dropout. Various descriptive studies have delineated the
demographic characteristics of noncompleters and described nonacademic precursors to dropout
(Barclay & Doll, 2001). Numerous predictors of dropout have been identified throughout the
literature and educational researchers commonly use indicators of student engagement to predict
which students will drop out of school. Student background characteristics (e.g., socioeconomic
status, ethnicity), family context (e.g., level of parent education, family stressors, parental
involvement in schooling), early school experiences (e.g., reading level, academic achievement),
and engagement behaviors (e.g., attendance, participation) are but a few indicators that have
been used to predict which students will drop out (Alexander, Entwisle, Horsey, 1997).
Several prospective (e.g., Cairns, Cairns, & Neckerman, 1989) and retrospective studies
(e.g., Barrington & Hendricks, 2001) have demonstrated the efficacy of using predictors of
dropout such as school attendance, assignment completion, and disruptive behavior to identify
4
students who are either at risk for dropping out, or have already done so. Using third grade
school attendance records and achievement test scores, Barrington and Hendricks identified
students who eventually dropped out, with 70% accuracy and considering additional predictors
such as cognitive ability (i.e., intelligence test scores) may increase correct classification of
dropouts and completers to 75% (Lloyd, 1978).
In addition to predicting who will drop out, much of the current dropout literature is
descriptive in nature and focuses on status variables such as race, gender, ethnicity, and
socioeconomic status (Reschly & Christenson, 2006a). Alternatively, a consensus is emerging
that sociodemographic factors and status variables alone do not fully account for dropout risk
(Christenson, Sinclair, Lehr, & Godber, 2001). Additional factors, such as student behavior (e.g.,
school absences, tardies, classroom conduct), personal resources (e.g., locus of control, academic
self-efficacy, satisfaction with school) and early school experiences (e.g., grade retention, special
education status, reading ability) are also significant predictors of early school withdrawal
(Alexander et al., 1997).
Due to the unalterable nature of status characteristics, solely focusing on identifying the
demographic predictors of noncompleters confers little benefit to parents, interventionists, and
other parties concerned with preventing dropout (Reschly & Christenson, 2006a). In contrast,
several alternative conceptualizations of dropout risk have emerged which may be more useful
when conceptualizing dropout and formulating interventions.
Alternative Conceptualizations of Dropout
The concept of functional risk has surfaced as a useful distinction when conceptualizing
dropout and identifying at-risk students (Christenson & Thurlow, 2004; Reschly, 2010) as it
provides a possible explanation regarding why certain subsets of “at-risk” students graduate
5
whereas others drop out. Functional risk includes factors such as attendance, student behavior,
and engagement (Christenson et al., 2008), and guides researchers and interventionists in
differentiating students who are truly at risk from those who are not. Functional risk and other
alterable dropout variables can be used to inform intervention by providing useful distinctions
that better aid in identifying students who are at risk for dropout.
A number of other distinctions among variables predictive of dropout and completion
have also been described in the literature (Reschly et al., 2010). For example, a distinction can be
made between risk and protective factors (Reschly & Christenson, 2006b). Risk factors are
behaviors or experiences that exacerbate the likelihood that a student will drop out of school,
whereas protective factors provide a buffer against the trajectory of early school withdrawal
(Reschly & Christenson, 2006a). These indicators of functional risk may be further categorized
according to context (i.e., school, family, or peer related; see Table 1). School, family, and peer
related factors are further divided into proximal variables, such as attendance and homework
completion, and distal variables such as early educational experiences. Proximal variables have a
direct relation between the student and school dropout, whereas distal variables are further
removed in their relationship to dropout (Rumberger, 1987). Targeting proximal variables when
evaluating student behavior and considering intervention strategies is often most effective
because these variables require low levels of inference and conjecture, allowing for ease of
measurement and observation.
The distinction between push and pull factors (Jordan, McPartland, & Lara, 1996) is
another pivotal component when conceptualizing dropout. Educators, policy makers, and
laypersons often assume that most students drop out due to external factors (e.g., teen pregnancy,
financially assisting family members, chronic illness) that prevent them from further attending
6
(pull factors). Although this is sometimes the case, research suggests that inhospitable school-
related factors, such as not getting along well with teachers, peers who dropped out, and harsh
discipline policies (push factors) are more often the catalysts for drop out (Reschly &
Christenson, 2006a). The alterable nature of school-level push factors provides the opportunity
for system-wide reform to change the educational policies and practices that so often push
students to drop out.
Student Engagement and Dropout
Complementary to the aforementioned aim of focusing on alterable variables, the
construct of student engagement has arisen as a promising theoretical model for understanding
dropout and promoting high school completion (Appleton, Christenson, Kim, & Reschly, 2006;
Christenson et al., 2008; Reschly & Christenson, 2006a). The presumably alterable nature of
engagement has resulted in a many high school reform efforts and dropout prevention programs
targeting the engagement construct as an attractive intervention option (Christenson et al., 2008).
Notably, student engagement interventions do not focus solely on the student as engagement is
often viewed as a mediator between important contexts (Table 1) – home, school, and peers –
and outcomes, such as achievement and graduation (Appleton et al., 2006; Christenson et al.,
2008).
As a seminal foundation of contemporary conceptualizations of student engagement,
Finn’s (1989) Participation-Identification model, often serves as a guiding heuristic for
understanding the relationship between engagement and dropout. The Participation-Identification
model describes a developmental sequence that begins in the early primary grades with student
participation in the classroom setting (Finn, 1989). When participatory behaviors are
accompanied by positive reinforcement, such as academic success, students develop and
7
internalize a sense of belonging or identification. This sense of identification with school, in turn,
further perpetuates active student participation in the classroom as well as the general school
environment.
In the context of dropout, Finn (1989; 1993) defined disengagement as the antithesis of
the relationship between engagement and participation. Disengagement is described as the
gradual process by which students become disaffected with the school environment (Finn, 1989).
A cycle of engagement emerges early during the primary grades and differentially affects
students with high and low levels of engagement (Finn, 1993; Finn & Rock, 1997). Furthermore,
the literature has proposed there are reciprocal, spiraling effects among contexts, individuals, and
developmental outcomes (Furrer, Skinner, Marchand, & Kindermann, 2006), such that
individuals who are engaged receive greater support, which further reinforces their engagement.
The converse is true of those demonstrating low levels of engagement, resulting in
disengagement and withdrawal.
In light of the long-lasting effects early educational experiences have on student
engagement and dropout, implementing early intervention with students at risk for
disengagement and dropout may be the most effective option. The term early intervention refers
to a wide array of services and activities designed to enhance young children’s cognitive, social,
and academic experiences; and the efficacy of early intervention for students at-risk for negative
outcomes is well- established (Ramey & Ramey, 1998). Longitudinal studies of intensive early
interventions indicate reductions in special education placement, grade retention and increased
likelihood of high school graduation (e.g., Hart & Risley, 1995; Ramey & Ramey, 1998).
Although there is an extensive body of information regarding dropout interventions,
notable limitations exist within the dropout literature. In their comprehensive review of dropout
8
intervention programs, Prevatt and Kelly (2003) concluded that many programs are implemented
system-wide, prohibiting analysis of intervention effects at the individual level. Many studies
also included samples that were based on convenience, teacher nomination, ethnic/minority
membership, or socioeconomic status (Prevatt & Kelly, 2003). Selecting samples based on such
information may provide useful descriptive information and between group comparisons;
however, lack of random assignment and control groups limits inferences regarding intervention
effects (Prevatt & Kelly, 2003).
In a similar vein, Lehr, Hansen, Sinclair, and Christenson (2003) noted that many studies
included poor design quality (e.g., lack of treatment integrity procedures) and lack of rigorous
methodology (e.g., evidence of adequate sample size, randomized control groups). An
overwhelming majority (71%) of interventions included within the Lehr et al. (2003) review
were conducted at the high school level, whereas 32% of interventions were implemented at the
middle school level. In spite of implications suggesting that early intervention may be
efficacious in ameliorating dropout risk, only 20% of reviewed interventions targeted elementary
school students at risk for dropout (Lehr et al., 2003). Perhaps most troubling is the lack of
research grounded in theory (Prevatt & Kelly, 2003; Lehr et al., 2003), considering the
availability of promising models that conceptualize dropout such as student engagement.
Student Engagement Models
Student engagement is theoretically described as having multiple dimensions (Appleton
et al., 2006). Although various two, three, and four part typologies have been posited; consensus
has been reached that there are at least two types of engagement: behavioral engagement and
affective engagement. Behavioral engagement refers to involvement in academic and
extracurricular activities and includes indicators such as time on task, school attendance, and
9
homework completion (Appleton et al, 2006; Finn, 1989; Fredericks, Blumefield, & Paris, 2004).
Affective engagement is comprised of positive and negative interactions between classmates,
teachers, and school as indicated by factors such as interest in learning and valuing education
(Christenson et al., 2008; Finn, 1989; Fredericks et al., 2004).
Other researchers (e.g., Fredericks, et al., 2004) have incorporated a three-part typology,
adding the cognitive engagement component, drawing upon the idea of cognitive investment, or
the willingness to exert the necessary effort to master difficult skills and comprehend complex
ideas (Fredericks et al., 2004). Students demonstrating high levels of cognitive engagement
employ self-regulation strategies, use effective problem-solving techniques, and seek mastery of
skills and concepts. Four-part theoretical models of engagement have conceptualized
engagement as consisting of academic, behavioral, cognitive, and affective subtypes (Appleton et
al., 2006).
Measuring Student Engagement
Although behavioral (e.g., voluntary class participation, attendance, effort) and academic
(e.g., homework completion, time on-task) engagement are readily observable and relatively
easy to measure, considerable limitations have been noted when measuring cognitive (e.g., self-
regulation, value of learning, relevance of schoolwork) and affective engagement (e.g., feelings
of belonging, relationships with teachers and peers). Students internally represent indicators of
cognitive and affective engagement (Appleton et al., 2006) and observers may be unable to
accurately report on such internally represented constructs. However, overemphasizing
observable indicators, while ignoring cognitive and affective aspects of engagement, may be
hazardous because it results in an incomplete representation of the engagement construct
10
(Appleton et al., 2006). Student self-report, however, is one possible option to circumvent this
issue.
The four-part typology is most prevalent student engagement model within the school
psychology field (Appleton et al., 2006; Christenson et al., 2008) and is the basis for one
prominent measure of student engagement, the Student Engagement Instrument (SEI). The SEI
was developed to measure cognitive and affective engagement from the student perspective
(Appleton et al., 2006; Betts, Appleton, Reschly, Christenson, & Huebner, in press) thereby
avoiding possible erroneous third-party inferences about the students’ sense of belonging, desire
to achieve goals, value of education, and so forth (Appleton et al., 2006).
Regarding the theoretical orientation of the instrument, the SEI draws from the seminal
work of Finn (1989) and Connell and Wellborn’s (1991) self-systems processes model of human
development. Included within the SEI are specifically worded items intended to measure student
levels of identification with school, an aspect of affective engagement addressed in Finn’s
participation-identification model (Appleton et al., 2006). Connell and Wellborn’s (1991) self-
systems processes model of human development guided the creation of items intended to
measure cognitive engagement (Appleton et al., 2006). As suggested by Connell and Wellborn
(1991), intrapersonal characteristics such as the ability to self-regulate, motivation to learn, and a
persistent response to failure contribute to student academic engagement or disaffection with
school. Cognitive engagement items of the SEI were designed to capture student levels of the
aforementioned constructs (e.g. self-regulation ability, response to failure).
The SEI was initially constructed and validated on an ethnically and economically
diverse sample of 1,931 ninth-graders from an urban school district. A later study (Betts et al., in
press) extended the age range of respondents and found evidence suggesting that the SEI is a
11
reliable and valid indicator of student engagement for students in grades 6-12. Sample
demographics of the SEI follow-up study (Betts et al., in press) were also expanded to include
students from the Southeast and Upper Midwest in rural and suburban school districts.
Depending on whether five-factor or six-factors are expected, the SEI may consist of 33
or 35 items. The revised 33-item SEI (Betts et al., in press) consists of five subscales, each
measuring a specific aspect of student engagement: Teacher-Student Relationships (TSR),
Control and Relevance of School Work (CRSW), Peer Support for Learning (PSL), Future
Aspirations and Goals (FG), and Family Support for Learning (FSL) (Betts et al., in press). The
original 35-item SEI (Appleton et al., 2006), however, included two additional items intended to
measure a sixth factor, Extrinsic Motivation (EM). The CRSW, FG, and EM subscales measure
student levels of cognitive engagement, whereas TSR, PSL, and FSL are factors related to
affective engagement (Appleton et al., 2006; Betts et al., in press). The Extrinsic Motivation
subscale, which was on the original version of the SEI, has sometimes been dropped in
subsequent research (Betts et al., in press; Reschly, Betts, & Appleton, 2010) due to questionable
validity and reliability.
All items on the SEI are intended to measure student levels of cognitive engagement
(e.g., perceived value of learning) and affective engagement (e.g., feelings of identification with
school) (Appleton et al., 2006; Betts et al., in press). Items are scored using a four point Likert-
type rating scale (1= strongly disagree, 2= disagree, 3= agree, and 4= strongly agree) and coded
so that higher scores indicate higher levels of engagement. In order to control for reading
difficulties, the SEI is often orally administered to students and standardized administration
instructions and prompts are included within the survey.
12
While promising, there are still a number of theoretical and measurement issues to be
addressed with respect to student engagement (Appleton et al., 2006; Betts et al, in press).
Subsequent research should include further examination of the external validity of the SEI
relative to other engagement measures, evaluation of the convergent and divergent validity of
hypothesized constructs, and investigating the extent to which relevant demographic variables
affect the equivalency in measurement of the student engagement construct (Appleton et al.,
2006; Betts et al., in press). Additionally, the current version of the SEI measures student
engagement at the middle and high school level which is less than ideal as disengagement is
commonly conceptualized as a long-term process that commences early in a student’s
educational career (Finn, 1989).
Extending the age range of the SEI (i.e., creating an elementary version) may confer
additional benefits to the field of education research. First, an elementary version of the SEI
would engender a more inclusive representation of the student engagement construct across
multiple age ranges. Researchers would be able to determine if the construct of student
engagement changes as a function of age and development; or whether it remains stable
throughout elementary, middle, and high school. The relative importance of engagement
constructs may change over time as a function of the fluidity in students’ school related attitudes
and behaviors (Alexander et al., 1997). This information is pertinent when conceptualizing
dropout prevention and intervention methods.
Second, there is currently a modicum of longitudinal examinations of the student
engagement construct (Appleton et al., 2006). A developmentally appropriate elementary school
version of the SEI could arm researchers and educators with the resources to monitor
engagement as early as possible, thereby increasing feasibility of longitudinal research on
13
engagement. Third, the elementary version of the SEI can further assist researchers in
determining whether the correlations between indicators of engagement and engagement
facilitators differ throughout students’ academic careers, which is important to ensure that
progress monitoring and interventions are valid across grade levels. Fourth, an elementary
version of the SEI may aid in the development of a more complete framework to alleviate
dropout, which is the foremost purpose of student engagement research.
Last, and perhaps most importantly, developing an elementary version of the SEI may
allow for early detection of students at risk for dropout and consequently aid in early prevention
efforts. Currently, attempts to prevent dropout often occur at the secondary-level (Lehr et al.,
2003) thereby allowing students to continue their dropout trajectory for an extended period of
time (Reschly & Christenson, 2006a). Identifying at-risk students earlier, before they become
irreparably disengaged and disaffected with school, may be more effective. Based on the well-
established efficacy of early intervention (Hart & Risley, 1995), the focus of this paper is on
developing an elementary school version of the SEI to use with younger students in order to
foster earlier identification of students at risk for disengagement and dropout.
Purpose of the Present Study
The purpose of the present study is to construct and validate an elementary version
(Student Engagement Instrument – Elementary Version, SEI-E) of the SEI to assist in early
identification and intervention efforts for students at risk for dropout. Instrument construction
and validation is an ongoing process. Typical scale development consists of several phases
during which the researcher attempts to provide evidence toward the reliability and validity of
the measure. Validity refers to the accuracy of inferences obtained from measurement scores.
The process of scale validation is conducted to provide a logically sound rationale to support the
14
interpretation of test scores and relevance of the measure (American Educational Research
Association [AERA], 1999). The validation process begins by positing a proposed interpretation
of test scores, often based on hypothetical constructs or underlying latent variables (AERA); the
present study serves to fulfill this purpose.
Validity evidence takes various forms depending on the purpose of the measure (Huck,
2004). Considering the intended purpose of the SEI-E (identifying students at-risk for
disengaging from school and later dropping out), establishing evidence towards construct and
criterion-related validity is most relevant to the current study. Frequently, validity is
conceptualized as consisting of two subtypes: internal and external. Internal validity refers to the
extent to which relationships are inferential within the internal structure of the measure (AERA,
1999). Alternatively, external validity examines the relationship of test scores to variables
outside of, or external to, the measure (AERA). Examining external validity may include
determining whether the current measure correlates as expected with some criterion or other
hypothesized construct (e.g., motivation).
Reliability, or consistency in measurement across time or populations, is a prerequisite to
validity, as a measure must demonstrate consistency before accurate inferences are possible.
Evidence towards reliability may be established via several methods such as demonstrating
internal consistency of items, examining equivalence in measure over time, and demonstrating
equivalence across parallel forms (versions) of the same measure. The current study assesses the
reliability and the internal validity of the SEI-E; however, researchers will investigate the
external validity in subsequent studies by identifying concurrent and predictive relationships
between SEI-E data and other indicators of student engagement (e.g., standardized test scores,
attendance).
15
This study is unique in that, to date, there are no published measures that reliably and
validly measure student cognitive and affective engagement during the formative years of early
education. An accurate depiction of student engagement across an extended range of grade levels
allows for further examination of student engagement and indicators of school success or failure
such as grade retention, passing high stakes assessments, behavior, and dropout trajectories. Such
information may be important for dropout prevention and intervention efforts. The current study
seeks to answer the following research questions:
(1) Is the SEI-E a reliable and valid measure of third through fifth graders’ cognitive and
affective engagement with school?
(2) What is the factor structure of the SEI with third through fifth graders?
16
Table 1
Alterable Variables Associated with School Dropout
Protective Risk Students • Complete homework
• Come to class prepared • High locus of control • Good self-concept • Expectations for school
completion
• High rate of absences • Behavior problems • Poor academic
performance • Grade retention • Working
Families • Academic support (e.g., help with homework) and motivational support (e.g., high expectations, talk to children about school) for learning
• Parental monitoring
• Low educational expectations
• Mobility • Permissive parenting
styles
Schools • Orderly school environments
• Committed, caring
teachers • Fair discipline policies
• Weak adult authority • Large school size
(>1,000 students) • High pupil-teacher
ratios • Few caring
relationships between staff and students
• Poor or uninteresting
curricula
• Low expectations and high rates of truancy
Source: Reschly & Christenson, 2006a
17
Chapter Two
Method
Participants
Participants were third through fifth graders in a large, diverse, urban school district in
the Southeast. The sample was comprised of an approximately equal number of males (50.1%)
and females (49.9%). An ethnically diverse sample was obtained as participant ethnicities were
29.8% African American (N= 745), 28.9% Hispanic (N= 724), 28.6% White (N= 715), 8.5%
Asian/Pacific Islander (N= 214), 4% Multiracial (N=102), and less than 1 % Native
American/Alaskan Native (N= 4). In order to obtain a sample representative of the school
district, four elementary schools were chosen based on the following demographic
characteristics: ethnicity, percentage of students receiving free and reduced lunch, special
education eligibility, and English Language Learner (ELL) status, yielding a total of 2,504
students. Equivalent samples of students were obtained across third, fourth, and fifth grades
(32.9%, 34.4%, and 32.6%, respectively). Available data indicated that 58.9% of these students
were eligible for free or reduced lunch, 13.7% received special education services, and 16.2%
were designated as ELL.
Instrument Construction
The principal investigator created a pilot elementary version (henceforth referred to as the
SEI-E) of the SEI, adapted from the original instrument (Appleton et al., 2006). An expert panel
was then consulted to ensure that item wording of modified survey items was developmentally
appropriate and addressed appropriate engagement constructs. This feedback resulted in
semantic and structural changes including several completely reworded items (Table 2).
18
The final version of the SEI-E consisted of 19 items intended to measure student levels of
cognitive engagement (e.g., perceived value of learning) and 14 items intended to examine
affective engagement (e.g., feelings of identification with school). All items are scored using a
four-point Likert-type rating scale (1= strongly disagree, 2= disagree, 3= agree, and 4= strongly
agree). Items were coded so that higher scores indicate higher levels of engagement.
Congruent with the factor structure obtained with the original SEI, the SEI-E was
designed to consist of six subscales, each measuring a specific aspect of student engagement:
Teacher-Student Relationships (TSR), Control and Relevance of School Work (CRSW), Peer
Support for Learning (PSL), Future Aspirations and Goals (FG), Family Support for Learning,
and Extrinsic Motivation (EM) (Appleton et al., 2006). Evidence from previous studies
(Appleton et al., 2006; Betts et al., in press) indicated that factors were well saturated with only
one item (item 13) resulting in complex loadings across two factors. In line with theoretical
rationale of the original version, the SEI-E includes specifically worded items that are intended
to measure affective engagement as described in Finn’s participation-identification model
(Appleton et al., 2006) and cognitive engagement as conceptualized within Connell and
Wellborn’s (1991) self-systems processes model of human development.
Data Collection Procedures
The data in this study were obtained from a de-identified data set collected as part of the
school district’s ongoing efforts to monitor student engagement with school and ameliorate
dropout. The district collects student engagement information regularly from all students in
grades 6-12 via passive consent. The SEI-E was administered to whole classes of students in
each selected elementary school.
19
To ensure integrity of SEI administration, on-site monitors were utilized as a resource for
teachers and other survey administrators. Like the original version of the SEI, the SEI-E was
orally administered to students in order to control for reading difficulties. Students were closely
monitored by survey administrators who made efforts to control careless responding and
response acquiescence. Following scale administration, a school district representative scanned
the completed scales into an SPSS data file and entered applicable data. Irreconcilable answers
(e.g., “agree” and “disagree”) were coded as missing. To verify proper scanning, a subsample of
surveys was checked against the data set; no errors were found.
Analysis Logic and Procedures
Congruent with standard scale development methods, initial scale items were constructed
according to emerging theory (e.g., Appleton et al., 2006; Christenson et al., 2008; Reschly &
Christenson, 2006a). Exploratory factor analyses (EFAs) were then conducted with a randomly
selected half of the dataset (n= 1,267) to explore the underlying factor structure. The composition
(i.e., demographic characteristics) of the split-half sample was similar to that of the full sample.
A common factor analysis procedure, as opposed to component analysis, was deemed
appropriate given that the goal of the analysis was to model the relations among the SEI-E items
as functions of latent constructs. Likewise, the researchers hypothesized a six-factor structure a
priori based on previous factor analyses of the original SEI which determined that a six-factor
structure was most appropriate (Appleton et al., 2006). The SPSS 17.0 software package was
utilized to conduct Principal Axis Factoring (PAF). Direct oblimin, an oblique rotation, was
chosen given the likelihood that the factors would be correlated as all items were constructed to
represent related subtypes of the student engagement construct. A delta value of zero was
assigned to represent the suspected moderate correlation between the variables. Missing value
20
analysis revealed that all items had a missing case value of at least 20% (mean=22.01%, range=
21.70%- 23.00%). In other words, at least 20% of respondents did not provide a response to each
question. Notably, this statistic includes students who did not participate in the study (i.e., were
absent during survey administration). Missing cases were excluded using listwise deletion, as
this is the default setting for SPSS.
21
Table 2 Comparison of item wording between the Student Engagement Instrument (SEI) and the
Elementary Version (SEI-I)
SEI SEI-E Overall, adults at my school treat students fairly. Adults at my school are fair towards students
most of the time. Adults at my school listen to the students. Adults at my school listen to the students. At my school, teachers care about students. Teachers at my school care about the
students. My teachers are there for me when I need them. My teachers are there for me when I need
them. The school rules are fair. The rules at my school are fair. Overall, my teachers are open and honest with me.
My teachers are honest with me.
I enjoy talking to the teachers here. I like talking to the teachers here. I feel safe at school. I feel safe at school. Most teachers at my school are interested in me as a person, not just a student.
Most teachers care about me as a person, not just a student.
The tests in my classes do a good job of measuring what I’m able to do.
The tests in my class do a good job of showing what I learned.
Most of what is important to know you learn in school.
School is where I learn important things.
The grades in my classes do a good job of measuring what I’m able to do.
The grades in my classes do a good job of showing what I am able to do.
What I’m learning in my classes will be important in my future.
What I learn in my classes will be important in my future.
After finishing my schoolwork I check it over to see if it’s correct.
After I finish my schoolwork, I check it to see if it is correct.
When I do schoolwork, I check to see whether I understand what I’m doing.
I make sure that I understand what I am doing when I do schoolwork.
Learning is fun because I get better at something.
Learning is fun because I get better at something.
When I do well in school it’s because I work hard.
When I do well in school it is because I work hard.
I feel like I have a say about what happens to me at school.
I have a say about what happens to me at school.
Other students at school care about me. Other students care about me. Students at my school are there for me when I need them.
Students at my school are there for me when I need them.
Other students here like me the way I am. Other students here like me the way I am. I enjoy talking to the students here. I enjoy talking to the students here. Students here respect what I have to say. Students here respect what I have to say. I have some friends at school. I have friends at school.
22
I plan to continue my education following high school.
I plan to go to college after I graduate high school.
Going to school after high school is important. Continuing to learn after high school is important.
School is important for achieving my future goals.
School is important for reaching my future goals.
My education will create many future opportunities for me.
My education will create many chances for me to reach my future goals.
I am hopeful about my future. I am hopeful about my future. My family/guardian(s) are there for me when I need them.
My family/guardian(s) are there for me when I need them.
When I have problems at my school my family/guardian(s) are willing to help me.
When I have problems at my school my family/guardian(s) are ready to help me.
When something good happens at school, my family/guardians want to know about it.
My family/ guardian(s) want to know when something good happens at school.
My family/guardian(s) want me to keep trying when things are tough at school.
My family/guardian(s) want me to keep trying when things are tough at school.
I’ll learn, but only if my family/guardian(s) give me a reward. (Reversed)
I will learn only if my parent/ guardian(s) give me a reward. (Reversed)
I’ll learn, but only if the teacher gives me a reward. (Reversed).
I will learn only if my teacher(s) give(s) me a reward. (Reversed)
23
Chapter Three
Results
Descriptive Statistics
To simultaneously test for multivariate normality and screen for multivariate outliers, an
adapted version of DeCarlo’s SPSS macro was utilized. DeCarlo’s macro calculates
Mahalanobi’s distance (Mahal D sq) for each case, as well as critical values (Bonferroni), in
order to identify significantly different outliers. Seventy-two cases (0.057% of all cases) were
identified as significant (p<.05) multivariate outliers using this program. There were also several
items (e.g., item 1, item 17) that demonstrated kurtosis values greater than two, which suggests a
non-normal distribution. Upon examination, the researchers determined that these cases should
remain in the data set given the large size of the sample and lack of theoretical justification
suggesting otherwise. The matrix reporting correlations between scales, for which the subsequent
factor analysis was conducted and results of the Kaiser-Meyer-Olkin procedure and Bartlett’s
test of sphericity (p = .000) indicated that the correlation matrix was highly amenable to
factoring.
Descriptive statistics for each item are presented below (Table 3). Several items (e.g.,
item 8, item 17) demonstrated considerably non-normal distributions as evidenced by skewness
values greater than absolute value of two. All items were skewed in the negative direction,
indicating that students tended to endorse higher rates of engagement across these items. Further
investigation of these items indicated that the items demonstrating the highest levels of skew are
items which require conjecture about future events (e.g., item 17: I plan to go to college after I
graduate high school). The tendency for young students to self-report high abilities while
24
maintaining optimistic views of future-oriented events presents one plausible explanation for this
pattern.
Exploratory Factor Analyses
A combination of procedures was employed to determine the appropriate number of
factors to extract. The eigenvalues greater than 1.0 criterion was used in addition to an
examination of scree plots to determine the number of factors to retain. Examination of the scree
plots suggested extracting six factors (Figure 1), and the first seven factors had eigenvalues
above 1.0 before extraction. Given that the eigenvalues greater than 1.0 rule tends to result in
overextraction of factors (Floyd & Widaman, 1995), a range of factor models was examined.
Separate EFAs were conducted that forced the items onto five-, six-, and seven-factor models.
The procedure was iterative, employing a direct oblimin rotation with a delta value of zero for
each iteration.
Examination of the five-factor solution (Table 4) revealed that when five factors were
retained several of the items which comprised the Control and Relevance of Schoolwork scale on
the original SEI failed to exhibit salient loadings on any factor. The five-factor model also failed
to demonstrate clear factor structure as indicated by several nonsalient items with loadings
greater than .20. The aforementioned findings suggest that the five-factor model extracted too
few factors. Retaining six-factors resulted in a model that demonstrated clearer factor structure
(Table 5). Greater distinctions in loading values were more apparent in the six-factor solution
with nonsalient loadings being closer to zero; a characteristic of clear factor structure. Analysis
of the seven-factor model (Table 6) indicated that there were several factors with less than three
salient loadings. It is desirable that factors have at least three items to warrant extraction;
therefore, the seven-factor model demonstrated overextraction. Furthermore, given that the
25
additional seventh factor explained less than 1% of the shared variance, a six-factor solution
(accounting for 39.07% of the variance among measured variables) was taken as the final
solution).
This solution was chosen based on both theoretical and practical considerations. First, it
provided the best direct comparison with previous findings regarding the factor structure of the
SEI. The six-factor solution also provided the most easily interpretable solution among the three
possible factor structures and provided a clear simple structure characterized by strong loadings
across factors and discernable patterns of loadings between factors. For clarity of presentation,
items are grouped according to the retained six-factor structure (Table 7). Salient pattern
coefficients were deemed to be those with values of .30 or higher. Factors were moderately
correlated with the highest correlation (-.460) between Factors 1 and 5; therefore, both the
pattern and structure matrices were considered during the interpretation process. Distinct factors
were interpreted based on the resulting pattern matrix.
Further investigation of the six-factor model revealed that 33 of the 35 SEI-E items
demonstrated high pattern coefficients (>.30) on a single factor, while one item (item 29)
exhibited equally salient loadings on two factors using the six-factor solution. Two items (items
9 and 28) failed to demonstrate significant loadings on any factor. Factor loadings, though not as
high as in previous analyses of the SEI, were moderately high, ranging from .316 to .592 for
Factor 1, from .544 to .764 for Factor 2, from .378 to .636 for Factor 3, .794 and .906 for
Factor 4, from -.312 to -.665 for Factor 5, and from -.340 to .401 for Factor 6.
Cronbach’s alpha values were calculated to provide further evidence towards the internal
consistency of the proposed six-factor model (Table 8). All factors demonstrated acceptable
internal consistency as demonstrated by Cronbach’s alpha values ranging from .639 to .820. The
26
high values of the obtained alpha coefficients indicate that individual items within each proposed
factor “hang together” well or measure the same construct (Huck, 2004). Collectively, factor
loadings, pattern matrices, and internal validity coefficients suggest that a six-factor model most
appropriately captures the common variance among the variables of the SEI-E. Variable
communality values for this solution, which were estimated using common iterative procedures,
are presented in Table 9.
In general, evidence indicated that the six-factor structure obtained with the original SEI
was replicated with the SEI-E. In contrast to the initial validation of the SEI (Appleton et al.,
2006), one item on the SEI-E demonstrated a significant cross-loading greater than the criterion
of 0.30 across two factors. A single item cross-loading was also found in a later validation study
of the original SEI; however, the hyperplaning item was not the same as in the current
examination (Betts et al., in press). The order of the factors in the current analysis also differed
from previous studies; however, variability in factor order across studies is a relatively common
finding (Huck, 2004). Factors were assigned labels in accordance with the loading pattern
exhibited for each factor and based on previous factor labels from validation studies of the
original SEI.
Factor 1, which accounted for 19.73% of the variance among the measured variables,
was labeled Future Aspirations and Goals and consisted of items 8, 11, 17, 19, 29, 30, and 34.
Factor 2, accounting for 5.87% of the variance, was comprised of items 4, 6, 7, 14, 23, and 24
and was labeled Peer Support for Learning. Items 3, 5, 10, 13, 16, 21, 22, 27, and 31
demonstrated salient loadings on Factor 3, which was labeled Teacher-Student Relationships and
accounted for 4.80% of the variance among measured variables. Factor 4 accounted for 3.30%
of the variance and was labeled Extrinsic Motivation. This factor included items 18 and 32. Of
27
note, this factor was dropped from the final factor solution of the SEI (Appleton et al., 2006).
Factor 5 accounted for 1.91% of the variance and consisted of items 1, 12 20, and 29 and was
named Family Support for Learning. Items 2, 15, 25, 26, 33, and 35 comprised Factor 6, which
was labeled Control and Relevance of Schoolwork and accounted for 1.49% of the variance
among the measured variables.
28
Table 3 Descriptive Statistics for Items within the SEI-E
Item Mean Statistic
Std. Deviation Statistic
Skewness
Kurtosis
Statistic Std.
Deviation Statistic Std.
Deviation
1 3.67 .565 -1.837 .078 3.889 .155
2 3.00 .702 -.541 .078 .578 .155
3 3.49 .678 -1.291 .078 1.558 .155
4 3.03 .880 -.764 .078 .003 .156
5 3.25 .793 -.996 .078 .714 .156
6 2.89 .861 -.656 .078 -.037 .155
7 2.99 .872 -.696 .078 -.074 .156
8 3.69 .606 -2.201 .078 5.306 .155
9 3.70 .533 -1.762 .078 3.329 .155
10 3.09 .859 -.845 .078 .223 .155
11 3.71 .550 -2.060 .078 4.931 .155
12 3.55 .696 -1.618 .078 2.432 .155
13 3.25 .838 -1.027 .078 .499 .155
14 2.66 .895 -.343 .078 -.601 .156
15 3.50 .615 -1.022 .078 .902 .155
16 3.57 .669 -1.762 .078 3.382 .155
17 3.76 .556 -2.792 .078 8.763 .155
18 1.61 .944 1.452 .078 .959 .156
19 3.75 .536 -2.500 .078 7.364 .156
20 3.52 .733 -1.615 .078 2.343 .155
21 3.05 .805 -.751 .078 .354 .155
22 3.17 .812 -.842 .078 .321 .155
23 3.51 .729 -1.592 .078 2.388 .155
24 3.66 .643 -2.234 .078 5.368 .156
25 3.67 .550 -1.665 .078 3.020 .156
29
26 3.39 .718 -1.155 .078 1.359 .155
27 3.30 .817 -1.067 .078 .574 .155
28 3.10 .896 -.841 .078 .007 .156
29 3.78 .500 -2.637 .078 8.497 .155
30 3.68 .598 -2.115 .078 5.056 .155
31 3.50 .704 -1.461 .078 2.113 .156
32 1.63 .956 1.402 .078 .779 .156
33 3.37 .768 -1.211 .078 1.182 .155
34 3.67 .604 -2.069 .078 4.786 .155
35 3.52 .706 -1.607 .078 2.612 .155
30
Table 4 Five-factor Solution Pattern Matrix
Item Factor
1 2 3 4 5
1 -.051 .017 -.033 -.051 -.6732 .202 .060 .203 -.002 .0193 -.048 .051 .588 -.013 .0564 .031 .703 -.060 -.010 .0255 -.061 .025 .529 .035 -.0656 .001 .766 .036 -.014 .0507 -.115 .724 .049 -.009 -.0668 .580 .086 .012 -.001 .0369 .299 -.048 .311 .045 -.07910 .009 -.049 .546 -.041 .00211 .398 -.051 .011 -.082 -.07512 .047 .005 .099 .022 -.40513 .034 .070 .541 -.026 -.00514 -.035 .546 .211 .037 .02815 .277 .152 .039 -.061 -.04416 .016 .027 .582 -.065 -.02317 .464 .036 -.156 -.054 -.04218 -.030 -.043 -.024 .792 -.05519 .631 .023 -.026 .058 .01720 -.013 .035 -.010 .011 -.67121 -.019 .052 .519 .064 -.07122 .064 .085 .515 -.061 .01823 .002 .572 -.015 -.035 -.08824 .156 .570 -.020 .005 -.03825 .486 .000 .078 -.031 -.02426 .315 -.083 .294 -.025 -.02227 .119 .158 .374 -.011 -.089
31
28 .021 .020 .302 -.023 -.18029 .231 .063 -.019 -.010 -.32030 .427 .050 -.051 -.045 -.19231 -.021 .016 .743 -.018 .04632 .057 .037 .001 .902 .03233 .290 -.060 .340 -.032 -.12234 .530 -.020 .139 -.117 -.01535 .300 -.028 .131 -.012 -.066
*Items determined to load on a factor (based on a cut-off of .3) are shown in bold. Principal Axis Factoring with direct oblimin rotation.
32
Table 5
Six-factor Solution Pattern Matrix
Item Factor
1 2 3 4 5 6
1 -.054 .004 -.034 -.040 .011 .6762 .015 .070 -.127 .004 .551 -.0453 .599 .030 -.102 -.004 .046 -.0694 -.107 .717 -.015 -.008 .091 -.0345 .548 .002 -.088 .044 .010 .0546 .044 .771 .037 -.008 -.043 -.0657 .047 .729 -.101 -.009 -.025 .0658 -.027 .070 .550 .031 .137 -.0949 .251 -.067 .179 .065 .215 .04710 .545 -.071 -.055 -.030 .075 -.01711 .030 -.073 .470 -.054 -.035 .03412 .093 -.012 .055 .035 .011 .39713 .629 .036 .114 -.007 -.143 -.02314 .205 .544 -.056 .041 .024 -.03915 -.108 .160 .064 -.051 .391 .01916 .677 -.011 .096 -.046 -.152 .00017 -.140 .022 .564 -.024 -.052 -.00718 -.015 -.046 -.044 .780 -.018 .05319 -.038 .002 .663 .096 .059 -.08320 -.036 .018 -.010 .023 .030 .67421 .511 .031 -.090 .073 .082 .05822 .496 .065 -.022 -.048 .120 -.03723 -.016 .573 .028 -.029 -.025 .08224 -.051 .570 .134 .016 .068 .01825 -.079 -.004 .262 -.008 .444 -.02326 .182 -.097 .107 -.009 .359 -.00727 .386 .135 .114 .008 .015 .066
33
28 .239 .009 -.114 -.017 .203 .177
29 .037 .037 .376 .017 -.166 .30230 -.029 .027 .530 -.011 -.065 .14931 .808 -.021 -.012 -.002 -.060 -.06932 .011 .032 .038 .895 -.008 -.04433 .210 -.077 .047 -.014 .410 .09934 .141 -.048 .561 -.081 .041 -.04235 .000 -.033 .087 .001 .378 .043
*Items determined to load on a factor (based on a cut-off of .3) are shown in bold. Principal Axis Factoring with direct oblimin rotation.
34
Table 6
Seven- factor Solution Pattern Matrix
Item Factor
1 2 3 4 5 6 7
1 -.160 .008 -.036 -.041 -.670 .003 .075 2 -.035 .064 .044 -.009 .022 .476 -.052 3 -.047 .041 .566 -.007 .059 .048 -.023 4 -.097 .693 -.056 -.004 .028 .071 .068 5 -.113 .009 .531 .049 -.054 .018 .040 6 -.043 .750 .080 -.009 .051 -.040 .071 7 -.064 .710 .065 -.006 -.062 -.042 -.040 8 .217 .088 .033 -.018 .012 .195 .336 9 .111 -.044 .259 .033 -.093 .231 .098 10 -.016 -.055 .516 -.038 .005 .077 -.005 11 .012 -.069 .099 -.071 -.068 .015 .458 12 .011 .008 .094 .020 -.398 .024 .042 13 .024 .057 .601 -.019 -.002 -.092 .107 14 -.087 .531 .224 .046 .034 .016 .036 15 -.105 .147 -.049 -.059 -.039 .368 .161 16 .153 .026 .619 -.075 -.032 -.101 -.023 17 -.009 .021 -.044 -.032 -.018 -.008 .595 18 -.047 -.046 .001 .821 -.050 -.005 .040 19 .397 .039 -.003 .029 -.019 .151 .282 20 .056 .044 -.053 -.003 -.688 .036 -.090 21 -.036 .045 .486 .067 -.068 .082 -.023 22 .040 .082 .469 -.067 .009 .123 -.029 23 .134 .586 -.022 -.053 -.105 -.022 -.100 24 .201 .589 -.046 -.018 -.060 .080 -.050 25 .046 -.001 -.014 -.039 -.031 .438 .227 26 .133 -.076 .186 -.046 -.043 .350 .010 27 .075 .155 .374 -.016 -.097 .046 .057 28 -.059 .017 .234 -.023 -.179 .175 -.035
35
29 .191 .069 .044 -.021 -.339 -.092 .171 30 .188 .052 .013 -.055 -.208 .019 .318 31 .060 .010 .748 -.018 .044 -.025 -.027 32 .126 .045 .010 .895 .019 .020 -.035 33 .133 -.052 .206 -.055 -.145 .393 -.044 34 .440 -.002 .141 -.157 -.047 .120 .166 35 .104 -.020 .016 -.031 -.086 .362 .006
*Items determined to load on a factor (based on a cut-off of .3) are shown in bold. Principal Axis Factoring with direct oblimin rotation.
36
Table 7 Items Comprising the Six-factor Model
Item Factor
1 2 3 4 5 6 Item Text 8 .499 .084 .031 -.003 .042 .151 My education will create many chances for me
to reach my future goals. 11 .411 -.056 .063 -.080 -.064 .000 Continuing to learn after high school is
important. 17 .486 .033 -.089 -.050 -.025 -.016 I plan to go to college after I graduate high
school. 19 .592 .020 .023 .061 .029 .093 School is important for reaching my future
goals. 29 .316 .057 .053 -.003 -.312 -.117 My family/guardian(s) want me to keep trying
when things are tough at school. 30 .459 .046 .012 -.040 -.179 -.023 I am hopeful about my future. 34
.502 -.026 .181 -.116 -.005 .074 What I learn in my classes will be important in
my future. 4 -.008 .706 -.080 -.012 .024 .067 Other students here like me the way I am. 6 .030 .764 .060 -.011 .056 -.043 Other students care about me. 7 -.090 .723 .050 -.008 -.063 -.034 Students at my school are there for me when I
need them. 14 -.038 .544 .203 .038 .029 .018 Students here respect what I have to say 23 .021 .570 -.002 -.034 -.085 -.026 I enjoy talking to the students here. 24 .124 .568 -.020 .004 -.037 .061 I have friends at school. 3 -.062 .048 .561 -.011 .052 .051 My teachers are there for me when I need them.
5 -.054 .022 .510 .037 -.065 .022 Adults at my school listen to the students. 10 -.020 -.051 .514 -.041 -.002 .079 The rules at school are fair. 13 .110 .061 .595 -.018 .006 -.099 Most teachers care about me as a person, not just
a student. 16 .094 .016 .636 -.058 -.015 -.106 My teachers are honest with me. 21 -.049 .050 .481 .063 -.075 .083 Adults at my school are fair towards students
most of the time. 22 .011 .083 .476 -.062 .012 .117 I like talking to the teachers here.
37
27 .117 .154 .378 -.009 -.088 .031 I feel safe at school. 31 .015 .007 .758 -.012 .049 -.028 Teachers at my school care about the students. 18 -.021 -.041 -.022 .794 -.054 -.005 I will learn only if my teacher(s) give(s) me a
reward. 32 .057 .038 .013 .906 .032 .009 I will learn only if my parent/guardian(s) give me a
reward.
1 -.033 .022 -.057 -.054 -.662 .012 My gamily/guardian(s) are there for me when I need them.
12 .054 .006 .090 .021 -.399 .021 My family/guardian(s) want to know when something good happens at school.
20 -.008 .038 -.037 .008 -.665 .032 When I have problems at school, my family/guardian(s) are ready to help me.
29 .316 .057 .053 -.003 -.312 -.117 My family/guardian(s) want me to keep trying when things are tough at school.
2 -.054 .071 .042 -.014 .006 .472 After I finish my schoolwork, I check it to see if it is correct.
15 .091 .161
-.067 -.072 -.054 .340 I make sure that I understand what I am doing when I do schoolwork.
25 .274 .006 -.023 -.040 -.031 .401 When I do well in school it is because I work hard.
26 .138 -.082 .203 -.034 -.036 .326 The tests in my classes do a good job of showing what I learned.
33 .092 -.059 .228 -.042 -.141 .369 Learning is fun because I get better at doing something.
35
9
.116 .192
-.024 -.047
.031 .264
-.022 .041
-.080
-.085
.336
.209
The grades in my classes do a good job of showing what I am able to do. School is where I learn important things.
28 -.071 .022 .229 -.029 -.191 .179 I have a say about what happens to me at school.*Items determined to load on a factor (based on a cut-off of .3) are shown in bold. Principal Axis Factoring with direct oblimin rotation.
38
Table 8
Six-factor Solution Cronbach’s Alpha Values
Factor Cronbach’s alpha
Factor Title
1 .731 Future Aspirations and Goals
2 .820 Peer Support for Learning
3 .812 Teacher Student Relationships
4 .816 Extrinsic Motivation
5 .639 Family Support for Learning
6 .670 Control and Relevance of Schoolwork
39
Table 9
Variable Communalities using a Six-factor Model
Item Initial Extraction
1 .304 .427 2 .189 .245 3 .321 .330 4 .418 .477 5 .274 .286 6 .510 .590 7 .497 .550 8 .323 .351 9 .280 .280 10 .282 .292 11 .256 .238 12 .200 .223 13 .361 .383 14 .382 .391 15 .201 .222 16 .378 .422 17 .240 .248 18 .534 .642 19 .334 .378 20 .323 .450 21 .299 .304 22 .349 .345 23 .370 .368 24 .409 .399 25 .294 .343 26 .280 .282 27 .328 .321 28 .202 .197 29 .258 .288 30 .310 .345 31 .460 .547 32 .533 .789 33 .369 .389
40
34 .408 .437 35 .217 .204
Principle Axis Factoring with direct oblimin rotation.
41
Figure 1. EFA Scree Plot. Principal Axis Factoring with direct oblimin rotation.
42
Chapter Four
Discussion and Conclusions
The phenomenon of early school withdrawal has reached epidemic proportions within the
United States public school system. Students who drop out are placed at an educational
disadvantage relative to their peers who graduate, as dropouts are often ill-equipped to compete
in the national marketplace (Kirsch et al., 2007). However, the prognosis for students placed at-
risk for early school withdrawal is not as bleak as it once was. The construct of student
engagement has emerged as a promising heuristic when conceptualizing and formulating dropout
prevention and intervention practices (Reschly & Christenson, 2006a). As a mediator between
context and environment, engagement provides researchers and interventionists with a link
between environmental contexts (e.g., schools, families, communities) and outcomes (e.g.,
academic achievement). In addition, engagement can potentially facilitate accurate identification
of students who are truly placed at risk via the delineation of predictive variables (e.g., push and
pull factors) and alternative conceptualization of risk (e.g., functional risk) that are inherent
within the construct (Reschly & Christenson, 2006a).
The purpose of this study was to examine the psychometric properties of the SEI-E, a
measure designed to assess cognitive and affective engagement with school. Consequently,
researchers aimed to determine if the currently accepted taxonomy of student engagement (i.e.,
six-factor model) was applicable to students of early elementary school age. The six-factor
structure proposed in the current study suggests that Future Aspirations and Goals, Peer Support
for Learning Teacher, Student Relationships, Family Support for Learning, Control and
Relevance of Schoolwork, and Extrinsic Motivation are relevant constructs when
43
conceptualizing student engagement at the early elementary school level. It therefore follows that
these six constructs provide likely intervention paths for school psychologists and
interventionists.
In contrast to the conclusions reached by previous research (i.e., Betts et al., in press), the
authors of the present study decided to retain the Extrinsic Motivation factor. The significant
contribution of this factor is plausible with a younger cohort of students because children may be
more sensitive to external sources of reinforcement and punishment (relative to older students),
which may support the conclusion that extrinsic motivation plays an important role in
engagement among younger children. Of note, the scale only contains two variables, which may
suggest over-factoring occurred when six factors were retained. Additional research is needed in
order to determine the validity of retaining the Extrinsic Motivation factor as used in the current
engagement model. There were also two items (items 9 and 28) that did not load onto any factor.
Further research should investigate the effects of rewording these items or deleting these items
from the scale. In addition, subtle differences were found between the factor loadings in previous
studies (e.g., Appleton et al., 2006; Betts et al., in press) which were also somewhat discrepant
from those found in the current study. Further research of the SEI is warranted to examine the
extent to which the factor structure varies across samples.
Limitations and Directions for Future Research
Although the six-factor model demonstrated the best fit to the sample in the current
study, further replication and cross-validation is needed before the validity of this model can be
generalized to other populations. The current study was conducted in an urban school district;
therefore further research should be conducted with students from rural locales and other regions
of the U.S. In addition, polychoric correlations were not used as input to principal axis factoring.
44
Discrepant findings may have been obtained regarding model fit had these analyses been
conducted.
Regarding future research, the investigators of the current study intend to determine if
correlations between sample characteristics (i.e. ethnicity, socioeconomic status) and levels of
student engagement are consistent with those previously identified in older cohorts. The
demographic data collected during the current study will also be utilized to examine concurrent
and predictive relationships between SEI-E data and other indicators of engagement (e.g.,
attendance, behavior incidents) and school performance (e.g., CRCT scores). Furthermore,
confirmatory factor analysis (CFA) will be conducted on the remaining half of the data set to
further examine the validity of the six-factor model.
Although the present study adds much needed information to the student engagement
literature, there are still many avenues for future research. Previous measurement invariance
studies (Betts et al, in press) suggest that the SEI is a reliable measure of student engagement;
however, future research should examine the external validity of the instrument with other
measures of engagement (e.g., attendance, academic achievement). Further research is also
needed regarding the extent to which indicators of cognitive and affective engagement predict
student outcomes. Considering indicators of cognitive and affective engagement in addition to
focusing on behavioral and academic indicators provides additional information, which may
prove useful dropout interventionists and researchers. Therefore, student engagement
instruments such as the SEI and SEI-E may be beneficial. In addition, providing a full
conceptualization of how engagement changes over a student’s educational career is of utmost
importance. These results suggest that currently the SEI and SEI-E may be used with students in
grades 3-12, but research is needed to examine engagement at first and second grade, which may
45
prove difficult. As suggested in Betts et al. (in press), future research should also examine the
extent to which the SEI provides equivalent measurement across relevant demographic variables
such as ethnicity, special education and disability status, and gender. The process of establishing
the validity of an instrument is both tentative and iterative; however, the aforementioned studies
are the first strides towards constructing a means to accurately measure engagement during early
education.
Implications for School Psychologists
The SEI-E and SEI may help provide school psychologists and researchers with a tool to
quickly identify students at risk for disengagement and dropout. Although decisions should never
be so based on such a brief measure, early identification and intervention methods will likely be
more efficacious if students are identified based on alterable variables such as functional risk
rather than demographic risk and other status variables. The SEI-E and SEI may aid in this
endeavor. The six-factor model also provides school psychologists and other school personnel
with a starting point when consulting the literature regarding remediation efforts. In sum, school
psychologists play an important role in the mission to curb the dropout epidemic. As active
members of school-family partnerships, school psychologists are now armed with a possible
means to connect the contextual nature of engagement with improving outcomes for all students.
46
References
Alliance for Excellent Education, The high costs of high school dropouts: What the nation pays
for inadequate high schools. Issue Brief, August 2009
http://all4ed.org/files/HighCost.pdf (accessed January 29, 2010).
Alliance for Excellent Education, Dropouts, Diplomas, and Dollars. U.S. High Schools and the
Nation’s Economy. August 2008 (accessed March 21, 2010).
Appleton, J.J., Christenson, S.L., & Furlong, M.L. (2008). Student engagement with school:
Critical conceptual and methodological issues of the construct. Psychology in the
Schools, 45 (5), 369-384.
Appleton, J.J., Christenson, S.L., Kim, D., & Reschly, A.L. (2006). Measuring cognitive and
psychological engagement: Validation of the Student Engagement Instrument. Journal
of School Psychology, 44, 427-455.
Alexander, K., Entwistle, D., & Horsey, C. (1997). From first grade forward: Early foundations
of high school dropout. Sociology of Education, 70, 87-107.
Barclay, J.R., & Doll, B. (2001). Early prospective studies of the high school dropout. School
Psychology Quarterly, 16, 357-369.
Barrington, B.L., & Hendricks, B.H. (1989). Differentiating characteristics of high school
graduates, dropouts, and nongraduates. Journal of Educational Research, 82 (6), 309-
319.
Cairns, R.B., Cairns, B.D., & Neckerman, H. (1989). Early school dropout: Configurations and
determinants. Child Development, 60, 1437-1452.
47
Christenson, S.L., & Thurlow, M.L. (2004). School dropouts: Prevention considerations,
interventions, and challenges. Current Directions in Psychological Science, 13, 36-39
Christenson, S.L., Reschly, A.L., Appleton, J.J., Berman, S., Spanjers, D., & Varro, P. (2008).
Best practices in fostering student engagement. In A. Thomas & J. Grimes (Eds). Best
practices in school psychology (5th ed.). Bethesda, MD: National Association of School
Psychologists.
Christenson, S.L., Sinclair, M.F., Lehr, C.A., & Godber, Y (2001). Promoting successful school
completion: Critical conceptual and methodological guidelines. School Psychology
Quarterly, 16, 468-484.
Connell, J.P., & Wellborn, J.G. (1991). Competence, autonomy, and relatedness: A motivational
analysis of self-system processes. In M.R. Gunnar & L. A. Sroufe (Eds.), Self processes
and development. Hillsdale, N.J.: Lawrence Erlbaum Associates.
Floyd, F.J. & Widaman, K.F. (1995). Factor analysis in the development and refinement of
clinical assessment instruments. Psychological Assessment, 7 (3), 286-299.
Finn, J. D. (1989). Withdrawing from school. Review of Educational Research, 59 (2), 117-142.
Finn, J.D., & Rock, D.A. (1997). Academic success among students at risk for school failure.
Journal of Applied Psychology, 82 (2), 221-234.
Finn, J. D. (1993). School engagement and students at risk. Washington, DC: National Center for
Education Statistics.
Finn, J.D. (2006). The adult lives of at-risk students. Washington, D.C. National Center for
Education Statistics.
Fredericks, J.A., Blumenfield, P.C., & Paris, A.H. (2004). School engagement: Potential of the
concept, state of the evidence. Review of Educational Research, 74 (1), 59-109.
48
Furrer, C. J., Skinner, E., Marchand, G., & Kindermann, T. A. (2006, March). Engagement vs.
disaffection as central constructs in the dynamics of motivational development. Paper
presented at the Annual Meeting of the Society for Research on Adolescence, San
Francisco, CA.
Jordan, W., McPartland, J., & Lara, J. (1996). Exploring the causes of early dropout among
race-ethnic and gender groups. Youth & Society, 28 (1), 62-94.
Hart, B., & Risley, T.R. (1995). Meaningful differences in the everyday experience of young
American children. Baltimore, MD: Brookes Publishing Co., Inc.
Huck, S. (2004). Reliability and Validity, Reading stastistics and research. New York, NY:
Pearson Education, Inc.
Kirsh, I., Braun, H., Yamamoto, K., & Sum, A. (2007). America’s perfect storm: Three forces
changing our nation’s future, 1-34, Princeton, N.J.: Educational Testing Service.
McDonnell, L. M. (2005). No child left behind and the role in federal education: Evolution or
revolution? Peabody Journal of Education, 80, 19-38.
Prevatt, F. & Kelly, F.D. (2003). Dropping out of school: A review of intervention programs.
Journal of School Psychology, 41, 377-395.
Ramey, C.T., & Ramey, S.L. (1998). Early intervention and early experience. American
Psychologist, 53 (2), 109-120.
Reschly, A. L. (2010). Reading and school completion: Critical connections and Matthew
effects. Reading & Writing Quarterly, 26, 1-23.
Reschly, A., & Christenson, S. L. (2006a). Promoting School Completion. In G. Bear & K.
Minke (Eds), Children’s Needs III: Understanding and addressing the developmental
49
needs of children (pp. 103- 113). Bethesda, MD: National Association of School
Psychologists.
Reschly, A. & Christenson, S.L. (2006b). Prediction of dropout among students with mild
disabilities: A case for the inclusion of student engagement variables. Remedial and
Special Education, 27, 276-292.
Reschly, A.L., Appleton, J. J., & Christenson, S. L (2010). Dropout prevention, student
engagement, and preparing students for the 21st century. New York: Springer.
Rumberger, R.W. (1987). High school dropouts: A review of issues and evidence.
Review of Educational Research, 57, 101-121.
Schweinhart, L.J. (2005). The High/Scope Perry Preschool Study Through Age 40: Summaries,
Conclusions, and Frequently Asked Questions.