using the college persistence questionnaire to enhance commitment among ngcsu cadets
TRANSCRIPT
Using The College Persistence Questionnaire To Enhance Commitment Among NGCSU
Cadets
Back In The Antediluvian Days
Why the Increased Emphasis on Retention
1. Financial and legislative pressures on colleges.2. Global economy and automation: Heighten the need for
skilled versus semi- and unskilled labor3. Schools are becoming more sensitive to students’ needs.4. Death of a dream: Failure to graduate often leads to
reductions in students’ income and quality of life
Some Students Have Special Requirements
What Do We Need To Know About Our Students?
It Depends Upon Your Responsibilities
Working With Individual Students: What Counselors, Advisors and Faculty Need To Know
1. Which cadets are most at-risk?2. What factor(s) are causing a particular cadet to leave the
Corps?3. We need scale scores and responses to individual items.
CPQ Advisor Portal
https://www.beckdavidson.com/welcome/welcome.aspx
Working With Groups of Students: What Policy-Makers and Instructors Of
Persistence-Oriented Courses Need To Know
1. What factor(s) distinguish students at their schools who will persist and will not persist in their educations?
2. Are our retention programs effective?
Six Revelations On The Path To An Effective Retention Program (Part 1)
Revelation What That Means1. We have a retention problem.
Yes, every school has a retention problem. Some schools have a greater problem than others. Nevertheless, every school includes students that do not graduate who have the capacity to graduate.
2. Someone needs to do something.
Retention problems will not just disappear; they cannot be ignored except at great cost, they must be addressed. Following this revelation, a policy maker often tasks someone or a group to deal with the retention issue.
3. We need to know what works.
You know there is a problem. Now you need to determine what it is and what to do about it. A common practice is for schools to model their initial retention activities after a sister school that claims success or a program reported in the empirical literature.
Six Revelations On The Path To An Effective Retention Program (Part 2)
Revelation What That Means4. We need to know if our activities decrease attrition.
Many schools believe that their retention-reducing activities are effective. Few schools, however, could provide a skeptic with information demonstrating that their activities are attenuating attrition or having beneficial effects. Realizing this is a big step forward.
5. We need to focus our resources, time, and energy on those activities that will have the greatest impact on our students.
Knowing what affects retention at your school is not enough. We must prioritize. There are more variables that affect persistence decisions than any school can address. Resources are limited so we must concentrate our energies on those variables that most strongly influence retention at our institution.
6. We need to establish an ongoing data collection system to evaluate our progress, augment our strengths, and overcome our weaknesses.
Students change, schools change, programs change. An in-place assessment system will allow us to demonstrate our achievements, make steady improvements, and accommodate changes to our educational systems.
Strategies For Improving Retentioin
Rely On Someone’s Professional Expertise
Subject matter experts (SMEs) have invaluable experience. Most of us were employed because we are military and/or academic SMEs.
Rely On Someone’s Professional Expertise
Problems With SMEs: SMEs can make mistakes. Some SMEs may not fully appreciate the consequences of their decisions.
Rely On Someone’s Professional Expertise
King MacArthur MarshallProblems With SMEs: Even the most expert of SMEs sometimes disagree. How do you determine whose position to support?
Look To The Literature For Guidance
In comparison to continuing-generation (CG) students first-generation (FG) students tend to be:1.Female 2. Older3.Have dependents 4. From a lower SES
FG students also tend to have:1.Lower standardized test scores2.Poor math, reading and critical thinking skills3.Low self-images4.Higher attrition rates5.Poor grades6.Low levels of academic and social integration
Problem With Relying On The Literature
The Golden Path To Effective Intervention: Attitude
Aiming for 100% Retention is an unrealistic and counterproductive goal. Not everyone should be a military officer.
Can You Imagine Someone’s
Photograph In This Box?
The Golden Path To Effective Intervention: Attitude
Take the offensive! We have nothing to be defensive about regarding retention. Our mission is to clear a path for others to follow.
The Golden Path To Effective Intervention: Empirical Evidence
Data Driven Decision Making
Goals In Developing the CPQ
1. Greatest Unmet Need: A better way of determining how students respond to the first 6 or 8 weeks of college
2. We need an instrument of great breadth that is applicable at many schools and many types of students.
3. The instrument must provide information pertinent to personnel working with individuals as well as groups of students.
4. The instrument must identify the factors that most strongly govern retention at a particular school or for a particular individual.
5. The instrument must not be too long.6. Data must be immediately available; we can’t wait.7. The instrument must have a mechanism for addressing
issues pertinent to a particular school.
Development of the CPQ-1: Procedure
1. Conducted an extensive review of the retention literature, at least 150 studies
2. Identified at least 85 variables (WOW) that had been associated with retention
3. Wrote items addressing these 85 variables4. Automated the administration and data recording
system5. Administered the questionnaire to more than 2000
students
Simplify, Simplify, Simplify: Why Thoreau Would Like Factor Analysis
1. Reduced 85 variables to 6 factors2. Scales tend to be more reliable than single items3. Findings reported in Journal of College Student
Retention
Reasons For Developing CPQ-2
1. Several CPQ-1 scales could benefit from additional items.
2. Version 1 did not assess financial pressures, the motivation to perform academic work, or the student’s belief that he or she was capable of performing successfully.
CPQ Components: Student Background Form
1. Demographic and Family: (e.g., sex, ethnicity, marital status, parent’s education)
2. Work and Financial Resources: (e.g., hours working, sources of income)
3. Reasons For Attending: (e.g., reputation, location, friends)
The Student Background Form consists of three types of items:
Principal Components Analysis: Participants and Procedure
1. Two thousand eight hundred and thirty-eight undergraduates from Angelo State University, Appalachian State University, Greenville Technical College, and Troy University-Montgomery served.
2. Students responded online, most taking less than 40 minutes to complete the 82-item questionnaire.
Results
1. Responses were converted to 5-point “favorability” scores, depending on whether the answer indicated something positive or negative about the student’s college experience. The solution produced ten factors with eigenvalues greater than 1.0.
2. A principal components analysis was performed on the favorability scores of the 82 items using a direct oblimin rotation.
3. All items with pattern coefficients of .40 or higher were retained for further analysis.
Results Just Keep On Coming
5. A second principal components analysis with a direct oblimin rotation was conducted on the resultant 54 items to ensure that the deletion of questions did not cause substantial changes in the pattern coefficients. 6. The findings of the two analyses were similar. Item deletion did not have a pronounced effect on the coefficients. 7. All correlations between components were less than .32. 8. Alpha levels were in excess of .70 for all factors.
CPQ Components: Student Experience Form
1. Institutional Commitment: Loyalty, intention to reenroll, confidence in school choice
2. Degree Commitment: Personal importance, students’ supportive network, value of degree
3. Academic Integration: Positive views of instruction, instructors, and own intellectual growth; awareness of connections between academics and career.
4. Social Integration: Sense of belonging, shared values, and similarity to others; positive involvement behaviors.
5. Collegiate Stress: Feelings of distress, pressure, and sacrifice.
CPQ Components: Student Experience Form
6. Academic Motivation: Interest and enjoyment in academic tasks; willingness to spend extra time.
7. Scholastic Conscientiousness: Timely performance of academic responsibilities.
8. Academic Efficacy: Confidence in academic skills and outcomes.
9. Financial Strain: Financial worries and difficulties; sense of disadvantage relative to others.
10. Advising Effectiveness: Positive views of advising and school communication processes.
Did Anyone Notice?
That The Ten Student Experience Scales Of The CPQ Assess Many Of The Variables
Composing Institutional Effectiveness
CPQ Components: Institution Specific Form
This form addresses questions not covered by the CPQ but are important to the Corps of Cadets.
1. How confident are you that the Corps of Cadets is right for you?
2. How do you feel about the Corps’ PT requirement?
3. How likely is it that you will participate in the Corps next semester?
4. How do you feel about the policy of requiring cadets to wear a military uniform while on campus?
CPQ Components: Institution Specific Form
5. How do you feel about mandatory quarters for beginning Cadets?
6. During your last year in high school, on the average how much time did you spend in exercise or sports?
7. How likely is it that you will complete eight semesters (or equivalent) of military science?
8. How do you feel about the residency policy where you must withdraw from the university for one academic year if you drop out of the Corps of Cadets?
How Well Does The Field Predict Retention?ACT Meta-Analysis
13 academic and non-Academic variables explain 17% of the variability of college retention across students.
Validation Study:Participants and Procedure
1. Seven hundred and forty-nine freshmen from three comprehensive universities responded online to the CPQ.
2. CPQ was administered six to eight weeks into the first semester of the freshman year.3. Most students took less than 40 minutes to
complete the questionnaire. 4. Return for next Fall semester was the criterion
variable.
Does the CPQ Predict Retention? Identifying At-Risk Students (Individual Items)
Measures Nagelkerke R2
HSR + SAT + Ethnicity +
Sex + Age + Class Size (df = 6).08
CPQ – Background Variables, not usually in student database (df = 12)
.18
Student Experience Items (df = 54)
.56
Using The CPQ To Understand An At-Risk Group: Individual Items
Compared to other freshmen Student Support Services students reported that:1. They had a bigger influence on such matters as course offerings, rule and regulations, and registration procedures. 2. They were more satisfied with the academic advisement they received.3. They more strongly preferred tasks that make extensive use of their own ideas and allowed expression of their individuality.4. Fewer of their close friends were at Appalachian State.5. It was more difficult for their families to handle the cost of college.
CPQ Items Predicting Institutional Commitment: Community College Data Argues For Targeting
Item Business Nursing Arts & Sciences
1 Degree Commitment Degree Commitment Academic Integration
2 Degree Commitment Degree Commitment Academic Integration
3 Degree Commitment Degree Commitment Support Services
4 Academic Integration Academic Integration Acad. Conscientiousness
5 Academic Integration Academic Integration Support Services
6 Academic Integration Academic Integration Social Integration
The Importance Of Student Experiences: Participants and Procedure
1. 701 freshmen from three comprehensive universities. 2. CPQ was administered six to eight weeks into the first
semester of the freshman year.3. Most students took less than 40 minutes to complete the
questionnaire.
How Well Does The CPQ Predict Retention?
Predictor Variables Nagelkerke R Square
Variables typically in student database (df=15).
.13
Variables from Background Form not typically in database (df=12)
.23
CPQ Student Experience Scales .38
Regress retention (return the following Fall semester) on the following predictor variables
Support Services Satisfaction
Institutional Commitment
.11
-.07
.30
.38
.24
.22
.05
.33.23
.19
.10
.14
Social Integration
Academic Integration
AcademicConscientiousness
DegreeCommitment Where We Are
Going
Summary and Implications
1. CPQ is a valid predictor of whether freshmen will return for their sophomore years.
2. CPQ identified why individual students dropped out.
Summary and Implications
3. CPQ identified those variables that best distinguished those students who would from those students who would not continue their educations at these three institutions.
4. This information would be very helpful to policy makers deciding what retention programs to develop.
What Retention Activities At Your School Do You Believe Have Been Successful? A Sample
Mentoring programs have been successful.
Small class sizes, instructors taking personal interest in students-increasing engagement. Low student to advisor ratios in many programs.
Reaching more students by e-mail rather than US mail.
Tutoring
The Student Solutions Center has shown some success.
Meeting 1:1 with students to discuss their method of study, getting them to set up study plans, teaching them to underline or otherwise ID key terms in test questions, eliminating wrong answers by crossing them out, dealing with test anxiety.
All efforts have had some impact, however, all could be improved.
The Summer Bridge Program.
Academic support in freshman dorms has been very beneficial for at-risk or students – We have increased the number of dorm activities to foster the growth of community and sense of home for the students.
What Retention Activities At Your School Do You Believe Have Been Successful? A Sample
Setting The Stage For Tomorrow
Let Me Invite You To List Three Retention Activities That You Believe Have Been Successful at NGCSU
College Persistence Questionnaire Identifies Factors Influencing Commitment Of Military Cadets
Goals Of This Investigation
1) Determine if the CPQ predicts commitment to the Corps2) Identify those factors that are most strongly associated
with commitment3) Offer guidelines for increasing the retention of Cadets at
NGCSU4) Identify those cadets most at-risk for discontinuing their
educations
CPQ Components
1. Student Background Form: Consists of three types of questions: Demographic and Family items (sex, ethnicity, marital status, parent’s education), Work and Financial Resources items (hours working, sources of income), and Reasons for Attending Higher Education items (reputation, location).
2. Student Experience Form: Ten psychometrically validated scales that assess students’ interactions with the academic and social environments.
More About CPQ Components
3. Ten Student Experience Form Scales: Institutional Commitment, Degree Commitment, Academic Integration, Collegiate Stress, Social Integration, Academic Motivation, Scholastic Conscientiousness, Academic Efficacy, Financial Strain, and Advising Effectiveness.
4. Institutional Specific Form: Additional questions pertaining to the Corps of Cadets.
Participants and Procedure
1. 582 Cadets from NGCSU responded in groups online to the CPQ.
2. CPQ was administered six to eight weeks into the first semester of the freshman year.
3. Most students took less than 40 minutes to complete the questionnaire.
Commitment To Corps And NGCSU Overlap
1. Created Commitment To Corps scale from Institution specific items.
2. Correlation of Commitment to Corps and Institutional Commitment scale was .71.
3. Created single scale, Commitment Scale (CS), from corps and institutional items.
4. Cronbach’s alpha of CS was .78.
Does The CPQ Predict Commitment To The Corps?
Concurrent Validity: Regress CS scores on items from the Student Background and Student Experience Forms as well as three items from the Institution Specific Form, excluding items composing the CS scale.
F(68, 513) = 6.75, p < .001, R = .69, R2 = .47
Student Background Form
Cadets who chose NGCSU because of the following factors were more committed than cadets not endorsing these attributes.
1.Academics2.Overall reputation3.Appealing location
Student Background Form
Although these correlations were not strong, they may: 1. Aid in recruiting and identifying the types of prospective students who are likely to stick.2. Indicate that cadets with certain background characteristics may need additional services once they matriculate.
Student Experience Form
1. Eight of nine Student Experience scales were statistically significant, p < .01, two-tailed.
2. The most robust correlations with CS scores were the Academic Integration, Degree Commitment, and Social Integration scales. This result provides a guide to those seeking to augment commitment to the Corps. Improvements in the qualities underlying these scales will be particularly effective in enhancing commitment.
3. A complementary approach is to develop an intervention for a variable that is “networked” with other variables (e.g., advising).
Support Services Satisfaction
Institutional Commitment
.11
-.07
.30
.38
.24
.22
.05
.33.23
.19
.10
.14
Social Integration
Academic Integration
AcademicConscientiousness
DegreeCommitment Where We Are
Going
Student Background Form, Student Experience Form, Institution Specific Form: Comparisons
1. Student Background Form: Questions entered in isolation accounted for 16% of the variance.
2. Student Experience Form: Questions entered in isolation accounted for 43% of the variance.
3. Institution Specific Items: Questions assessing attitudes about requiring cadets to wear their uniform on campus, their views on PT, and whether they exercised in high school accounted for 15% of the variance when entered in isolation.
Two Important Takeaways
1. The Student Experience Form was a much better predictor of commitment than any items on the Student Background Form. This finding indicates that experiences during the first six to eight weeks have a profound impact on commitment.
2. It also argues against attempting to determine commitment during orientation or at an earlier date.
Limitations And Recommendations
1. This study needs to be followed by an analogous investigation in which retention (return for the sophomore year) is the outcome measure.
2. The sample size was small for those regressions employing individual items as predictors. Nevertheless, given the strength of the findings, we can be confident of the reliability of the outcomes.
3. It is encouraging that Student Experience items tend to be better predictors of commitment than Student Background items. Student Experience items are more subject to change.
Limitations And Recommendations
4. Efforts to augment cadet retention should focus on increasing Academic Integration, Social Integration, and Degree Commitment.
5. An alternative strategy is to network these variables and commitment by improving Advising Effectiveness.
6. The CPQ was administered only once in the Fall semester. Many cadets drop out after the Spring semester. Administering the CPQ in the Spring might identify students who were now at-risk but were not at-risk in the Fall. Furthermore, changes in scales scores could prove diagnostic.
What Does This Tell You About These Cadets?
Intervention Strategies: Transforming Cadets Into Officers
Working With Individual Students: What Counselors, Advisors and Faculty Need To Know
1. Which cadets are most at-risk?2. What factor(s) are causing a particular cadet to leave the
Corps?3. We need scale scores and responses to individual items.
Two Measures of Retention
Institutional Commitment ScoreAdvantages: Provides a means of identifying at-risk students after 6 weeks of college. The Student Experience variables that are correlated with Institutional Commitment are subject to change.Disadvantage: Is not perfectly correlated with who actually returns
Returning SophomoresAdvantage: Indicates who actually returns.Disadvantage: It is too late to help at-risk students when this data becomes available.
Beck’s List of Invariably Violated Principles
1. The earlier the diagnosis the better the prognosis2. More information generally enhances the prediction
of at-risk students3. The more time you have to gather information the
more useful information you can acquire
What Do You Need To Do To Reduce Attrition? Not An Easy Task
1. Identify at-risk students within first 6 to 8 weeks of college
2. Determine why a student is at-risk
3. Craft a powerful message that addresses that student’s issues
4. Quickly alter the direction of that student’s life
A Tale Of Two Portals
The Student and Advisor Portals were developed because early users of the CPQ requested an online system.
Student Portal: Administers the CPQ to the student and saves the information to a database.
Advisor Portal: Retrieves information from the database and uses this information to identify at-risk students and indicate why that individual is at-risk.
Advisor Portal: Identifying At-Risk Students
The Advisor Portal provides school personnel with a tool for making one-on-one sessions with students helpful, effective, and fruitful in augmenting retention rates. It identifies individual students who are most at-risk and supplies an in-depth analysis of the problems each one is experiencing. Testimonials of advisors and counselors who have used it are uniformly and overwhelmingly positive in describing its value in their sessions with individual students.
Working With Groups of Students: What Policy-Makers and Instructors Of
Persistence-Oriented Courses Need To Know
1. What factor(s) distinguish students at their schools who will persist and will not persist in their educations?
2. Are our retention programs effective?
Meta Analysis: Intervention Effects On College Performance And Retention
1. Robbins, Oh, Le, and Button (2009)2. Examined the effects of interventions (academic skill,
self-management, socialization, and first-year experience) on academic performance and retention.
Results: Intervention Effects On College Performance And Retention
1. Academic skill training (e.g., study habits, learning strategies) had the strongest effects on academic performance. Self-management skill training (e.g., self-regulatory, emotional control) had a lesser effect.
2. Self-management training had the strongest effect on retention. Socialization interventions also showed a significant effect on retention.
3. First-year experience programs had “weak to negligible effects on academic performance and retention.”
Why Do First Year Courses Have Such Modest Effects On Retention?
Interpreting The Data
They may:1.Be ineffective.2.Have very broad goals which are difficult to measure.3.Be very diverse. Programs may very different goals.4. Have very little structure or guidance as to objectives. Instructors may simply do what they want to do and the institution may hope that this has some benefit.
What Works In Student Retention? Student And Institutional Characteristics
Item Mean
Level of student preparation for college-level work 3.90
Adequacy of personal financial resources 3.90
Student study skills 3.80
Amount of financial aid available to students 3.65
Level of student motivation to succeed 3.64
Level of student commitment to earning a degree 3.56
Level of job demands on students 3.52
Student low socio-economic status 3.49
Student educational aspirations and goals 3.36
Student personal coping skills 3.34
What Works In Student Retention? Retention Activities
Item Mean
Academic advising center 3.98
Increased number of academic advisors 3.98
Advising interventions with selected populations 3.93
Comprehensive learning assistance center/lab 3.92
Supplemental instruction 3.91
Programs for first-generation students 3.90
Required on-campus housing for freshmen 3.86
Reading center/lab 3.86
Tutoring 3.84
Summer bridge program 3.83
What Activities Differentiate Schools With High Versus Low Retention Rates?
Item Top Bottom
Increased number of academic advisors 43% 30%
Advising with selected populations 96% 82%
Supplemental instruction 75% 64%
Summer bridge program 75% 49%
Program for honors students 91% 73%
Integration advising and first-year programs 72% 48%
Training for non-faculty academic advisors 84% 67%
Faculty mentoring 74% 45%
Living/learning communities (residential) 87% 60%
Staff mentoring 57% 31%
What Are The Three Most Important Retention Activities?
Item % Selected
Freshman seminar/university 101 (credit) 24%
Supplemental instruction 16%
Tutoring 15%
Living/learning communities (residential) 14%
Advising with selected populations 14%
Placement in courses based on test scores 13%
Academic advising center 12%
Summer orientation 11%
Early warning system 10%
Which Attrition-Reducing Activities Should We Use?
That depends upon:1. What variables differentiate cadets who become military officers from those who dropout. 2. What the empirical literature indicates about the effectiveness of various attrition-reducing practices.3. What resources and personnel we can direct towards retention.
How Do We Know If Our Actions Are Hitting The Mark?
That Brings Us To Measurement Issues
Your Main Outcome Measures
1. Grades 2. Institutional Commitment
3. Sophomore Retention
Your Palette Of Measures
There must be a logical relationship between your objectives and your measures. Your available indices include:
MeasuresRetention Academic Integration
Graduation Rates Financial Strain
GPA Social Integration
Institutional Commitment Degree Commitment
High School Rank, Standardized Tests Collegiate Stress
Background (e.g., sex, parental education) Academic Motivation
Work and Finances (e.g., hours working) Scholastic Conscientiousness
Reasons To Attend (e.g., friends, location) Academic Efficacy
Institution Specific Items Advising Effectiveness
Activities With Multiple Objectives
Activities
Training for non-faculty academic advisors
Faculty mentoring
Staff mentoring
Freshman seminar/university 101 (credit)
Early warning system
Measures: These activities are very generally stated. Therefore, they should have very diverse effects. If successful, a broad range of indices will be impacted. The measures taken should depend upon the particular objectives of the activities.
Some Attributes, Activities And Their Measures
Activities Measures
Financial resources Financial Strain
Motivation to succeed Academic Integration, Academic Motivation, Scholastic Conscientiousness
Commitment to earning a degree
Degree Commitment, Academic Integration
Job demands Work and Finances Section, Financial Strain
Educational aspirations
Degree Commitment, Academic Integration
Coping skills Collegiate Stress, Scholastic Conscientiousness, Academic Efficacy
Preparation for college-level work
High School Rank, Standardized Tests, Multiple Student Experience Scales
Assessing At-Risk Or Special Populations
Activities
Students of low socio-economic status
Advising interventions with selected student populations
Programs for first-generation students
Programs for honors students
Programs for minority students
Measures: These activities attempt to address problems common to at-risk or special populations. Measures on the Student Background Form (e.g., parents’ education, ethnicity) and student database could help identify these groups. The goals of these programs vary considerably, therefore, the measures used to assess their effectiveness varies accordingly.
Academic Enhancement Activities
Activities
Student study skills
Comprehensive learning assistance center/lab
Supplemental instruction
Reading center/lab
Tutoring
Placement of students in courses based on test scores
Program for honors students
Measures: Academic Integration, Academic Efficacy, Academic Motivation, Scholastic Conscientiousness, Collegiate Stress.
Social Integration Activities
Activities
Required on-campus housing for freshmen
Living/learning communities (residential)
Program for international students
Summer bridge program
Integration of advising with first-year transition programs
Summer orientation programs
Measures: A major objective of these activities appears to be to enhance the extent that students identify with and feel part of the academic community. Other objectives of these programs should be measured, in addition to the Social Integration scale of the CPQ.
Advising Activities
Activities
Academic advising center
Increased number of academic advisors
Advising interventions with selected student populations
Measures: Advising is critical because it networks with so many other variables. The Advising Effectiveness scale of the CPQ should be used to assess these endeavors. Other variables, depending on the goals of the program should also be measured..
What Is A Reasonable Assessment Strategy?
1. Gather attrition-related information from your Institutional Research Office.
2. Administer the CPQ to obtain Student Background and Experience variables.
3. Combine the Institutional Research and CPQ variables into a dataset.
4. Determine who returns the following Fall semester.5. Annually prepare Institutional Commitment and
Retention reports.6. Examine data across years to anticipate trends.
For Your Attention. Do Not Be A Stranger. This Is A beginning,
Not An Ending!