STUDENT SUCCESS MODELAssessment Based Interventions Impacting

Psycho-Social Variables

International Assessment and Retention ConferenceScottsdale, Arizona

June 13, 2008

Fred B. Newton & Eunhee Kim Kansas State University

DecisionProcess

PersonalHistory

Emotional/Personal Concerns

HealthBehaviors

Study Behaviors

OUTCOMEOUTCOME Student Success

PSYCHO-SOCIAL FACTORS IMPACTING STUDENT SUCCESS OUTCOMES

THE SPECTRUM OF INFLUENCE VARIABLES

Three Step ModelAssessment - Intervention - Outcome

(1) Identification: Use of three inventories to comprehensively assess student needs

(2) Implication: Providing interpretation and interactive consultation

(3) Improvement: Implementing change strategies and measuring outcomes

PRACTICE TO IMPACT POSITIVE CHANGE

What?

Assessment – Accurate and meaningful measurement of individual attitudes, behaviors and personal/emotional impact

So What? Integration – Understanding and making personalized and meaningful connection to the individual within the system context

Now What? Application – Appropriate intervention to create new learning strategies, better habits, problem-solution, and self-confidence

MEASUREMENT TO OUTCOME

ImplicationImplication(Goal Activity)(Goal Activity)

ImprovementImprovement(Outcome)(Outcome)

CLEICLEI

K-PIRSK-PIRSHBAHBA

Baseline

ReadinessStandard

IdentificationIdentification(Assessment)(Assessment)

STEP 1: I D E N T I F I C A T I O N

Use of three inventories to comprehensively assess student needs

CLEICLEI

K-PIRSK-PIRSHBAHBA

Baseline

ReadinessStandard

IdentificationIdentification(Assessment)(Assessment)

The Assessment Instruments

CLEI (College Learning Effectiveness Inventory)

HBA (Health Behavior Assessment)

K-PIRS (K-State Personal Identification Scales)

* KCAT is the non-profit intellectual properties corporation of Kansas State University

C L E I(College Learning Effectiveness Inventory)

The CLEI is an inventory of six scales with 50 questions representing a continuum of individual behaviors, attitudes, and dispositions related to academic activity.

The CLEI was developed over the past 10 years starting from over 300 generated items following theoretical assumptions on factors that impact college student learning (Russell & Petrie, 1992 – Handbook of Counseling Psychology, NY: Wiley).

The purpose of the CLEI is to organize the self-reported student responses into thematic domains or categories that have been shown to contribute to academic success…

To provide immediate feedback to the student completing the inventory by showing a pattern of strengths and weakness on an individualized profile

To provide information for advising and counseling a student, making it a tool for discussion of goals, selection of interventions, referral to relevant student services, and a measurement of progress and involvement in the change process

C L E I(College Learning Effectiveness Inventory)

CLEI Six Scales(1) Academic Self-Efficacy (ASE Scale):

Expressing confidence in academic ability, awareness of effort toward study, and expectation for success in college attainment.

(2) Organization and Attention to Study (OAS Scale):The organization of tasks and structuring of time to set goals, plan, and carry out necessary academic activity.

(3) Stress and Time Press (STP Scale):Dealing with pressures of time, environmental concerns, and the academic demands that impact academic study.

(4) Involvement with College Activity (ICA Scale):Belonging to organizations and participating in activities, including formal or informal gatherings of friends and classmates, within the campus environment.

(5) Emotional Satisfaction (ES Scale):Degree of interest and emotional response to academic life including people and the campus educational environment.

(6) Class Communication (CC Scale):Both verbal and non-verbal effort to engage in class activity.

C L E I Profile

T-Scores are based on the mean scores from a normative sample of college undergraduates (N=879)

H B A(Health Behaviors Assessment)

The HBA is a self-reporting instrument that measures health behaviors in the areas of physical activity, eating behavior, and personal management skill.

The HBA also includes a readiness measure that indicates student awareness and readiness to make personal improvements.

Upon completion of the inventory, the student receives immediate feedback on the individual profile via the Internet.

Example of Assessment

Assessing Baseline Health Behaviors of College Freshmen Using the HBA

70% Meeting physical activity standard

40% Meeting fruit & vegetable standard

32% BMI = overweight or obese 6% Females underweight 19% High consumption alcohol 41% Deficient sleep 32% Feel stress impact

K - P I R S(K-State Problem Identification Rating Scales)

K-PIRS is an instrument that identifies college-student client concerns at the beginning of treatment and, when used with K-PIRS Form-B, assesses behavior change over time.

50-item Client Concern Inventory Presenting Symptoms on 7-Clinical Scales Level of Interference with Academic and Social Function12 items Life Perspective Scales (Personal and Social) Readiness to Change Follow-up Measure of Change

K- P I R S Profile

7-Clinical Scales

MD: Mood DifficultiesLP: Learning ProblemsFC: Food ConcernsIC: Interpersonal ConflictS-HI: Self-Harm IndicatorsS/AI: Substance/ Addiction Issues

AI: Academic InterferenceSI: Social Interference

K - P I R SResearch & Development

Clinical norms developed on college student-clients using samples from counseling centers of nine institutions (N=4,703)

Data includes student readiness for change and also level of interference to academic and social functioning.

Follow up research demonstrates level of change that occurs from intake to session 3 and session 6.

Special forms of the K-PIRS are being adapted to serve as a general measure of student personal/emotional concerns.

L P I (Life Perspective Inventory)

Developed to gain information concerning student’s life perspective and personal resilience.

Factor 1: Personal Effectiveness includes outlook, problem solving, overall health, emotional stability, and personal meaning.

Factor 2: Social Support includes personal and family interactions, available resources, and opportunities.

* Presently, used as an experimental inventory until validity and normative studies completed.

STEP 2: I M P L I C A T I O N

ImplicationImplication(Goal Activity)(Goal Activity)

Providing interpretation and interactive consultation

Designed for self, peer mentor, and professional consultations

N O W W H A T? Interventions That Follow Assessment

Follow-up Measurement of Student Health Behaviors

Advisor Assessment of Student Learning Behaviors

The purpose of the PAC-CATS grant program is to provide an intervention for first-year K-State students that raises awareness and promotes healthy lifestyle behaviors in the areas of physical activity, eating behavior, and personal management skill.

Helping students to develop these lifestyle behaviors can help reduce future risk of overweight and obesity.

PAC-CATS Program

Example Intervention Program

Baseline Behaviors

Readiness for Change

Educational Intervention

Intensive Change Intervention

Outcomes

How Fit are KSU Freshmen Students?

Objectives for Freshman

Gain Awareness of Health Behaviors

Have Knowledge of Health Guidelines

Become Motivated to Change

Set Personal Goals

Learn Processes to Self Regulate

Elements of PAC-CATS Program

1. Health Behaviors Assessment

Individual assessment and personalized feedback on:

Physical Activity Eating Behavior Personal Management Skill

Pre & Post-Tests: Participants take the assessment at the beginning and at the end of the program

2. Behavior Change Process

Establish goals for increasing healthy behaviors, based on health assessment feedback:

• I want to jog or use the aerobics machines at the Rec Center 3 times a week for 30 minutes a session.

Establish action steps to reach goals:• Block off 1 hour for exercise on Mondays, Tuesdays,

and Thursdays of each week in weekly planner.• Arrange to meet a friend at the Rec every Monday at a

certain time.

Design system to monitor behaviors and progress toward goals - which includes designing ways to reinforce your progress.

Elements of PAC-CATS Program

3. Peer Mentors

Participants have their own personal mentor who are students with knowledge in the areas of physical activity, nutrition, and personal management skill.

Mentors help students set goals, design action steps, and monitor progress, and provide motivation.

Mentors provide support (Students Helping Students)

Elements of PAC-CATS Program

MENTORING

Helping Students, Help Themselves

4. Program Resources

The PAC-CATS Website contains information, self-help tips and links to helpful resources in the areas of physical activity, nutrition, and personal management skill. It is a valuable tool for developing the knowledge and skills that help students be successful in their personal programs.

Healthy Behaviors Workbookprovides a systematic process for the student to move from assessment, organize a plan, and monitor and adapt based upon results.

Elements of PAC-CATS Program

HBA

Profile Variables

Positive Change from Pre to Post

Group Difference

Peer Mentored (N=128)

Education Only (N=132)

Physical Activities

General Physical Activity ** **

Stretching/Flexibility ** ** **

Strength Training ** ** **

Eating Behaviors

Fruits & Vegetables ** ** *

Whole Grains ** *

Low-fat or Fat-free Dairy **

High-fat Foods ** *

Caffeinated Beverages ** **

Regular Pop/Soda ** **

Sweetened Beverages ** **

Alcoholic Beverages * *

Personal Management Skill

Time Management

Relaxation Techniques ** * *

Positive Thinking ** **

Creative Problem Solving ** **

Stress Impact ** **

Sleep

Health Behavior Changes: Year 2005 & 2006

* p < .05, ** p < .01

What did we learn about K-State students?

Weight and health issues were present with a significant number of students.

Awareness building using methods of assessment & education were steps that increased readiness for change.

Involvement in programs of change & support produced significantly positive changes in health behaviors: 15 of 17 areas for two cohort freshmen (Fall 05, Fall 06)

Participant reports indicated that learning to self-regulate (set goals and carry through) increased confidence and transferred to other behaviors applied to many areas of their life.

Example Applied to Student Advising

Advisor meets through individual appointment with student(or group seminar)

Assigns a set of Assessments to provide data on student individualized learning process (CLEI, HBA, K-PIRS)

Conducts a developmental interview (some structured process to review key points in student learning history

Synthesizes with the student strengths and weaknesses of approach to learning

Facilitates suggestions for improvement or change with student input

Observations Concordia College Academic Counselors

• “My students really seem to like it. They seem to talk more when we have the CLEI in front of us.”

• “This (the CLEI) is a great tool to use with non-verbal clients.”

• “I feel like our interventions are more on target and the student’s take more ownership in their academic success plans.”

• “The CLEI is easy to implement and the online scoring is great.”

• “Use of the CLEI by counselors increased student engagements.”

 

C L E I Example Profile “A”

• Female, Sophomore, Majoring Business/Marketing• GPA 3.25 on a 4.0 Scale

Dx Hypotheses Brainstorming Possible Explanations

from a CLEI Profile

1.

2.

3.

4.

5.

Possible InterventionsIdeas for Intervention Strategies to be used

based on the Dx

1.

2.

3.

4.

5.

C L E I Example Profile “B”

• Male, Sophomore, Major Undecided• GPA below 1.9 on a 4.0 Scale

Dx Hypotheses Brainstorming Possible Explanations

from a CLEI Profile

1.

2.

3.

4.

5.

Possible InterventionsIdeas for Intervention Strategies to be used

based on the Dx

1.

2.

3.

4.

5.

CLEI Online Follow-upExamples of possible strategies will be included

with the profile in the future.

Keep track of your goals Prepare an academic calendar Join an activity club Establish behavior change goals Practice self reflection Set up an appointment with your academic adviser Set up an appointment with your institutions counseling

Services office

STEP 3: I M P R O V E M E N T

Implementing changes strategies and measuring outcomes

ImprovementImprovement(Outcome)(Outcome)

OUTCOME EVALUATION

Psycho-social Variables Student Success Outcome Variables Relationship of Psycho-social Variables to Success Outcomes

DecisionProcess

PersonalHistory

Emotional/Personal Concerns

HealthBehaviors

Study Behaviors

OUTCOMEOUTCOME Student Success

Relationship of Study Behavior with GPA

1

2

3

4

5

Academic Self-Efficacy

Organization & Attention

Stress & Time Press

Involvement with College Activity

Emtional Satisfaction

Class Communication

CL

EI M

ean

Sco

re (r

aw)

2.9-below 3.0-3.4 3.5-above

Higher GPA significantly associated with higher scores in the six-CLEI scales (N=592).

a bc

a bc

a bc

a bc

a a b

a a b

Relationship of Emotional/Personal Concerns with Academic & Social Functioning

I n t e r f e r e n c e s

K-PIRS Scales Academic

Functioning

Social

Functioning

1. Mood Difficulties .34** .52**

2. Learning Problems .56** .15**

3. Food Concerns .12** .25**

4. Interpersonal Conflicts .11** .40**

5. Career Uncertainties .24** .06**

6. Self-Harm Indicators .17** .23**

7. Substance/Addiction Issues

.09** .13**

N=872**, Correlation Coefficients are significant with p<.01

Relationship of Health Behavior with Student Success Outcome

AlcoholConsumption G P A

Unhealthy Eating

PersonalManagement Skill

Healthy Eating

Vigorous Exercise- .175

- .184

.147

.090

.104

.167

ModeratePhysical Activity

LifeSatisfaction

BMI

.497

- .136

Regression coefficients of health behavior variables for each predictors (GPA, Life Satisfaction, BMI); Significant at alpha=.05; N=347.

.115

- .216

Demographical Factors Impacting Psycho-social Variables

Gender Year in College Ethnicity Fraternity/ Sorority Affiliation Housing Type Other Factors

Demographical Factor in Health Behaviors- Gender -

Female students are doing better with eating behaviors, specifically in the areas of foods to use in moderation or sparingly (caffeinated, sweetened, alcoholic beverages, and regular pop/soda).

Male students do more physical activities, specifically in

strength training.

Male students have less stress impacts, and use relaxation techniques more frequently.

Lower BMI Greater fruits & vegetables consumption Greater physical activity (stretching/flexibility, strength training) Greater alcoholic beverages consumption

Demographical Factor in Health Behaviors- Fraternity/Sorority members -

Demographical Factor in Study Behaviors

- Lower Class vs. Upper Class -

1

2

3

4

5

Academic Self-Efficacy

Organization & Attention

Stress & Time Press

Involvement with College Activity

Emtional Satisfaction

Class Communication

CL

EI M

ean

Sco

re)

Lower Class (n=381) Upper Class (n=216)

Lower class students had significantly lower scores in Academic Self-Efficacy, Involvement with College Activity, and Class Communication of the six-CLEI Scales

**

**

**

Demographical Factor in Emotional/Personal Concerns

- Majority vs. Minority -

30

40

50

60

70

MD LP FC IC CU SH SA ACD SOC

K-P

IRS

T-S

core

Minority (N=1,038) Majority (N=3,502)

**

**

**

Minority clients: Higher levels in Mood Difficulties, Learning Problems, Interpersonal Conflicts, Career Uncertainties, Self-harm Indicators, & Academic interference

**

**

**

D I S C U S S I O N

Future Research

Utilizing a complete battery of the three K-CAT inventories to measure a broad picture of college students’ health behavior, mental health, learning attitudes and behaviors providing a comprehensive view of psycho-social variables that may impact their overall functioning in college.

Additional Tools

Life Perspective Inventory Career Deciding Stress & Resilience Inventory Problem Identification – Students in General

MEASUREMENT TO OUTCOME

CLEICLEI

K-PIRSK-PIRSHBAHBA

ImplicationImplication(Goal Activity)(Goal Activity)

ImprovementImprovement(Outcome)(Outcome)

Baseline

ReadinessStandard

IdentificationIdentification(Assessment)(Assessment)

End of Slide Show