first year findings from the fsmp program · gloria crisp. kayleen eng. nathan petitti. matthew...

First Year Findings from the FSMP Program

College of Education

OSU FSMP Pilot Program (2018-20)

Caitlin McVayRoy HaggertyDan LarsonJen HumphreysKevin DoughertyGabrielle JamesMaureen CochranGloria CrispKayleen EngNathan PetittiMatthew Tradewell

College of Educationeducation.oregonstate.edu

FSMP Team

• Need for program• Program structure • Program goal

• Create a sense of belonging for first-year students by developing a semi-structured platform to increase a first-year student’s opportunity to develop meaningful relationships with faculty and experienced peers

• Evaluation purpose/focus• Assess the extent to which the pilot program is meeting its intended

outcomes


FSMP Program Overview

Related Mentoring Research


• Benefits of mentoring• Definitions• Mentoring framework

1. Emotional and Psychological Support

2. Degree and Career Support3. Academic Subject Knowledge

Support4. Role Model (Crisp, 2009)

• Ways evaluation contributes to current knowledge

• Program resources• Administrative and SSI support• FSMP team • 124 faculty and 124 peer mentors• Peer mentors paid for 10 hours of work per term • Canvas site and mentoring guides

• Program activities • Recruitment and matching process• Mentor training• Learning communities• Mentoring meetings


Method

• Evaluation data1. Student level IR data obtained at end of fall, winter, spring, and

beginning of fall year 2

2. Focus groups with faculty and peer mentors in fall 2018

3. Qualtrics survey administered to faculty and peer mentors in fall

4. Qualtrics survey administered to students in fall and spring


Method cont.

• Design and analysis plan• Impact evaluation designed to understand how well the program is working

(Mertens & Wilson, 2012)• Formative Student and mentors’ perspectives and experiences with the program

• Summative Propensity score matching minimized selection bias Conditioning variables (HS GPA, honors college, gender, race/ethnicity,

first gen status) Treatment effects using matched data and covariates


Method cont.

First Year (2018-19) Findings


• 124 mentoring groups• 173 students participated in one or more mentoring

sessions• Students who chose to participate were systematically

different than population of OSU students (e.g., students of color, females, first gen, honors students were overrepresented)

• Matching used to reduce selection bias and identify a control group of students who did not participate but who were similar to participants

• Treatment Effects• Grade Point Average

• Sig. treatment effects were found for fall 2018 and spring 2019• Participants had higher fall (p < .05) and spring (p < .001) GPAs when compared to

a matched group of non-participants

• Retention• Sig. treatment effects were found for winter, spring and fall 2019• Participants were more likely to be retained to winter (p < .001), spring (p < .001),

and fall of yr. 2 (p < .01) when compared to a matched group of non-participants*Note: All analyses controlled for HS GPA, honors status, gender, race/ethnicity, and first gen


Findings cont.

• Qualitative data suggests the program is supporting students as intended

• Quotes representing students’ experience: “My faculty mentor was very integral in my successes for this year. She gave me great advice on how to navigate the school year and my school work, while also making me feel at home on campus and giving me someone to talk to when needed.”

“My peer mentor was great! She was so easy to talk to and she was really accommodating. I felt like she was a great resource for both me and the other student in my group because she was in our position when she was a freshmen and understood the challenges to moving out of state, making new friends, sticking with school work. I really liked my mentor group.”


Findings cont.

• Student data suggest students needed and appreciated being mentored by both their faculty and peer mentor

• Mentors provided emotional and psychological support“I came into this project thinking I would be like a friendly guide to college for my mentees, but lately I've felt more like a facilitator of a supportive environment, as my mentees get a better handle on college and can offer each other advice.” (peer mentor)

“My mentor was always supportive of the decisions I made. She helped me overcome a couple of struggles that I had, and I enjoyed every minute that I got to work with her.” (student)


Findings cont.

• Mentors provided degree and career supportBoth faculty and peer mentors observed students needed help in learning more about major options and requirements, career opportunities and experiential learning opportunities

“My peer mentor has given me so much information about opportunities on campus when it comes to class scheduling, job opportunities and degree options.” (student)

• Peer mentors served as role models“My peer mentor was a joy to have. She was positive and very charismatic. Her positivity was always contagious and was the main driving force of the conversations. She also provided excellent resources for me to explore, such as clubs and activities that she participates in, acting as a role model and telling me about the wide variety of ways to get involved in school.” (student)


Findings cont.

• Mentors provided academic support and helped uncover the ”hidden curriculum” of higher education“My faculty mentor's challenges have been effective. Our mentees have taken them and ventured outside of their comfort zones, visiting office hours and studying with other students.” (peer mentor)

• Many mentors went out of their way to help students adjust to the college environment and the town

“She [peer mentor] was very helpful and good to talk to. She provided past experiences to help with current issues. It was also helpful to talk with her about stuff outside of school. For example, we discussed apartment hunting and the difficulties in finding a good place.” (student)


Findings cont.

• Strategies used to make mentoring teams successful 1. Be open, honest and vulnerable (e.g., tell about their struggles in college)

2. Limit the power differential (e.g., don’t meet in faculty office)

3. Listen more than talk (don’t make assumptions – actively listen)

4. Develop activities that meet students’ needs and interests (e.g., tour campus, attend campus activities)

5. Faculty and peers work together – partnership of mutual respect

6. Peer and faculty mentor both contribute their perspective


Findings cont.

• Scheduling!!! Groups had a hard time finding a time to meet (looking at purchasing technology to help with communication and scheduling)

• Faculty suggested the program add a beginning of the year kick off event to give mentors and students a time to meet (implemented fall 2019)

• Mentors wanted more information about mentoring (added info to training in fall 2019)

• Group size – both mentors and students offered recommendations to increase the group size (3 seems to be the preferred size)


Challenges

• Group mentoring that includes several students and a faculty and peer mentor can be an effective way to support student outcomes at OSU

• Findings identified types of support students need and strategies for success to improve mentor training and program activities

• Program should include a kick off event or other ways to connect mentors and students at the beginning of the year

• In scaling the program we will need technology (e.g., Mentor Collective) and/or other ways to help with scheduling

• Scaling will require a full-time program director and continued support from administration and College deans and faculty


Lessons Learned

• Present findings at the 13th annual Mentoring Institute at The University of New Mexico in Albuquerque, NM in October

• Submit first year findings to the Evaluation and Program Planning Journal with Adult and Higher Education (AHE) Ph.D. students, Christene James and Patrick McEarchern

• Continue collecting data this year• Hope to be able to share findings regarding group differences at next year’s

summit


Evaluation Next Steps

Questions?

first year findings from the fsmp program · gloria crisp. kayleen eng. nathan petitti. matthew...

Documents