Paper ID #19499

S-STEM Scholarship Program at UNC Pembroke: a COMPASS for ScienceMajors

Dr. Maria Soledad Santisteban, University of North Carolina at Pembroke

Dr. Santisteban is an Associate Professor in the Biology Department at the University of North Carolinaat Pembroke, a position she has held since 2013. She was an Assistant Professor at the same institutionfrom 2007. Dr. Santisteban teaches Microbiology, Cell Biology, and Molecular Biology. Dr. Santiste-ban, a native of Spain, earned her B.S. in Biochemistry from the University of the Basque Country inSpain. She then went to the Universite Joseph Fourier in France where she completed her DEA (Diplomed’Etudes Approfondies, the equivalent of a Masters) and her PhD in Cellular Biology. Under the supervi-sion of Gerald Brugal she worked on the chromatin structure/function relationship using image cytometrymethods. For her postdoctoral studies, Dr. Santisteban went to Dr. Mitchell Smith’s laboratory in theMicrobiology Department at the University of Virginia, working in the field of Yeast Molecular Genet-ics. She became an Assistant Professor of Research at the same institution and remained there till herappointment at UNC Pembroke. Dr. Santisteban continues to work with the yeast Saccharomyces cere-visiae as a model organism. Her research focuses on the role the histone H2A.Z variant in regulatinggene expression and she involves undergraduate students in her research. Dr. Santisteban is the PI in aNSF funded S-STEM program at UNC Pembroke, title COMPASS (Creating Opportunities for Studentsin Science). The current funding period is 2014-2019. She is also involved in K-12 education through herrole as district director of the NC Student Academy of Science and as a facilitator of teacher developmentworkshops for the Moore county schools through an MSP grant. Dr. Santisteban has been very involvedwith the North Carolina Academy of Science that she joined in 2008, and has served under various roles.She initially served two years as an elected member of the Board. Then she served in the 2011 AnnualMeeting Committee. She was vice-president in 2011-2012. In 2013 she organized the 110th North Car-olina Academy of Science (NCAS) Annual Meeting that was held at UNCP and she was chair of the localarrangements committee. She was president-elect in 2014-2015, and president of the Academy in 2016-2017. She is currently serving as past-president. Dr. Santisteban is a member of the editorial board of theKBM Journal of Biology and the advisory board of the KBM Scientific Publishing, LP. She is a memberof the GCAT (Genome Consortium for Active Teaching) and served in the years 2012-2014 as a memberof the GCAT advisory board, whose mission was to recruit faculty from minority serving institutions forthe synthetic biology NSF funded summer workshops. She has served as a reviewer in four NSF panels(three REUs, one PRFB). She is also a member of the Genomics Education Partnership (GEP), a collab-orative between a growing number (about 100 currently) of primarily undergraduate institutions, and theBiology Department and the McDonnell Genome Institute of Washington University in St. Louis. Thegoal of the GEP is to provide opportunities for undergraduate students to participate in genomics research.

Dr. Jill E. Thomley, Appalachian State University

Dr. Thomley is a Professor of Statistics in the Department of Mathematical Sciences at Appalachian StateUniversity, a member institution of the University of North Carolina system. She joined the faculty in2000 after earning her Ph.D. in Decision Sciences (with a focus in statistics) from Rensselaer Polytech-nic Institute. While at Rensselaer, she was awarded a Founders Award of Excellence and the Del andRuth Karger Dissertation Prize for her work on multivariate analysis of rank order data. Her previousdegrees include an M.S. in Industrial and Organizational Psychology from Rensselaer (1992) and an A.B.in Psychology from Harvard University (1990). Dr. Thomley has pursued diverse scholarly interests fromhigh school through the present day, which currently center around applications of statistics and the for-mal scholarship of teaching, learning, and engagement. She frequently consults on statistical design andanalysis of scientific research projects, and she has been involved in the formative and summative eval-uation of federally-funded STEM education grants since 2002. A particular focus of this work has beencomputational science, a discipline arising from the intersection of science, mathematics, and computerscience, as well as the diffusion and adoption of STEM education innovations. Additional scholarly inter-ests include the history of statistics and representations of mathematics and statistics in popular culture,

c©American Society for Engineering Education, 2017

Paper ID #19499

particularly mathematically talented women. Her co-edited work The Encyclopedia of Mathematics andSociety (3 volumes) was named a ”Best Reference 2011” by the editors of the well-respected LibraryJournal. She is also the proud co-creator of the Girl Scout Women in Mathematics Merit Badge program,which introduces middle school girls to the beautiful diversity of mathematics and women who work inthe discipline, as well as the opportunities that exist today for girls and women who want to pursue careersin mathematics.

Dr. Rebecca Bullard-Dillard,

Dr. Rebecca Bullard Dillard is the current Associate Vice Chancellor for Research and Sponsored Pro-grams at the University of North Carolina Pembroke (UNCP), having been recruited in 2013. Dr. Bullard-Dillard earned her B.S. degree in Biochemistry at North Carolina State University in 1990, then her Ph.D.from the University of South Carolina Department of Chemistry in 1996. Dr. Bullard-Dillard, a proteinbiochemist, previously served as the biology department chair, then in increasing levels of responsibilityin research administration, at Claflin University from 1999 – 2013. In 2007, she helped formed a workinggroup whose purpose was to bring alternative energy research and infrastructure to South Carolina. Col-laborative efforts with several energy sector private industry partners were established as well as workinggroups with city, state and federal agencies. These projects resulted in a patent in bioenergy (US Patent:US20146657327). Now, in her role as chief research officer, Dr. Bullard-Dillard has brought that resourcenetwork to the task of research infrastructure building for UNCP. Dr. Bullard-Dillard has been recognizedthrough her Outstanding Alumni Award from North Carolina State University College of Agricultureand Life Sciences, her South Carolina Governor’s Award for Excellence in Scientific Awareness and herSouth Carolina Independent Colleges and University’s Excellence in Teaching Award among others. Sheis deeply committed to her work in education and research to assure that the United States’ scientificcommunity continues to lead the world in solving the complex problems that challenge our future.

c©American Society for Engineering Education, 2017

S-STEM Scholarship Program at UNC Pembroke: A COMPASS for Science Majors

Introduction In the United Stated, educating students and creating leaders in science, technology, engineering and mathematics (STEM) is a growing priority and viewed by many as the key to 21st century global competitiveness. In 2013, the White House National Science and Technology Council released an ambitious five-year STEM Strategic Plan to coordinate federal investments in STEM education targeting five priority areas.1 One of these is graduating one million additional students with STEM degrees over the next ten years. Another is increasing the number of students from groups currently underrepresented in STEM disciplines. Universities are rising to meet the challenge of producing more STEM-educated students. The University of North Carolina at Pembroke (UNCP) is poised to contribute positively to the need for diverse STEM professionals. UNCP already has high quality STEM programs, its STEM enrollment has steadily increased over the last five years, and the university has maintained an overall enrollment of 54% underrepresented minorities (URM). However, despite its advantages, the STEM graduation rate has been low. Attrition is a concern; just under half (48%) of freshmen who declare a STEM major persist to their senior year. These students also take more time to finish. Only 18.7% of STEM majors graduate in four years. Spurred by these issues, STEM faculty designed the “Creating Opportunities for Students in Science (COMPASS) Scholarship Program” to improve STEM retention and align with the STEM Strategic Plan. A grant from the National Science Foundation (NSF) provided funds to recruit 27 talented but financially needy undergraduates in Biology, Biotechnology, Chemistry, and Environmental Science. COMPASS supports these students through a combination of mentoring, financial support, tutoring, research, internships, and professional development in order to improve their educational experiences. This should in turn produce better retention and graduation rates. At the time of the COMPASS program’s conception, the UNCP student body of about 6400 students, mostly undergraduates, had 54% URM and 50% were first generation students.2 The program planned to target these groups preferentially. UNCP also had a large pool of high-talent, financially needy students who would benefit significantly. For example, 58% received Pell grants. COMPASS enrolled students in three cohorts over three years, beginning in Fall 2014. The goal was for 100% of scholars to go on to graduate school or STEM careers within six years. The program creators predicted that COMPASS would increase the number of STEM professionals in this rural region of NC, whose diverse background should result in enhanced innovation in STEM fields. Two and a half year into the program, we report strong progress towards our goals and some valuable lessons learned toward improving our successes and the efforts of other who might plan a similar innovation. The Need for Programs like COMPASS: Preparation and Financial Burden Economic projections estimate that, in the next decade, the U.S. will need about one million additional STEM professionals, beyond current graduation rates.3-5 Employment projections for 2018 state that 68% of STEM jobs will require bachelor’s degree or higher.6 In North Carolina, STEM jobs are expected to increase by the national average of 17%.7 However, the U.S. is failing to keep pace in producing a qualified STEM workforce. Only 1 in 5 high school graduates who score in the top quartile in mathematics pursues a STEM profession6 and 19% of bachelor’s degrees are earned in STEM fields, compared to 50% in China.8 Poor retention and graduation rates among undergraduate STEM majors are major concerns.1 Of students entering as a STEM

major in a four-year public college or university, only 43% persist to graduation in those fields. Retention is even more alarming for underrepresented minorities (URM), who are about 40% of U.S. K-12 students but earn only 27% of all associate’s degrees, 17% of bachelor’s degrees in natural sciences and engineering, and 6.6% of doctorates in those fields.9, 10 The University of North Carolina at Pembroke (UNCP), located in rural southeastern North Carolina, serves a primarily undergraduate (87.8%) student body. The vast majority of UNCP students (96.4%) come from within North Carolina. It was established as Croatan Normal School in 1887, with the goal of training Native American teachers. At the time the COMPASS program was conceived, 16.2 % of student were Native American, 32% African-American, 4% Hispanic, 1.5% Asian, and 2% two or more races (54% total URM). Today those figures are very similar, with 59% total URM. In addition, 88% of all students at UNCP received some form of financial aid and 58% qualified for Pell grants in 2010-2011. The student body demographics of UNCP’s put the institution at risk for poor retention and graduation of STEM majors. At the same time, there is a rich pool of students available to increase the diversity of the STEM community. Table 1 UNCP STEM Enrollment and Degrees Conferred in 2008 and 2012, with % of URM Enrollment in

Fall 2008 Enrollment in

Fall 2012 Degrees in AY 2008

Degrees in AY 2012

Biology 375 (55%) 458 (58%) 49 (53%) 74 (51%) Biotechnology 15 (73%) 16 (53%) 2 (0%) 0 (0%) Environmental Science 23 (30%) 37 (29%) 6 (17%) 6 (0%) Chemistry 143 (52%) 188 (60%) 25 (40%) 25 (44 %) Total 556 (54%) 699 (57%) 82 (45%) 105 (46%) Growth + 26% Enrollment + 28% Degrees Conferred Table 1 shows STEM enrollment and degrees conferred in 2008 and 2012 at UNCP for the four majors that are the focus of the COMPASS program. These four majors share substantial course content and accounted for 83% of all STEM students in 2012. The other STEM majors at UNCP include computer science, mathematics, and physics. The proportion of URM students in these majors is consistent with representation on campus in enrollment and degrees conferred. Total STEM enrollment increased 20% over this five-year period and graduation increased 17%. Among UNCP STEM freshmen, fewer than half (48%) remain by senior year and only 18.7% graduate in four years. Nationally, lack of college readiness is a major culprit in low graduation rates.11, 12 At UNCP, a 2011 survey also identified poor preparation as one important obstacle to choosing or graduating in a STEM field.13 Other challenges cited were perceptions that STEM content is too difficult, lack of understanding of career opportunities, and financial hardship. The US Census Bureau (2010) ranked UNCP’s Robeson County as the poorest county in NC and one of the 250 poorest in the nation. About 34% of UNCP students come from Robeson County, and in 2011, 81% of them reported a family income of less than $75,000 (with 30.5% less than $30,000)2. Thus, UNCP’s students are severely financially disadvantaged. Low-income students are more likely to leave college.14 UNCP recommends that full-time students not work more than 20 hours per week. In 2011 survey, 14.1% of freshman students planned to work more than 20 hours/week and 25.8% planned to work 20 hours or less.2 Even so, UNCP dropped 511 students

in fall 2011 for non-payment. About 38% owed $2001-$3000, while 9% owed over $5,000. In a recent exit questionnaire for withdrawing from school, 27% cited financial reasons. Of the 699 STEM students in Table 1, 379 had Pell grants and 487 had loans. Despite assistance, there was a severe unmet financial need. In 2013, the in-state cost of attendance at UNCP was $16,397 and average financial aid received was $10,910, resulting in an average unmet financial need of $5,487. COMPASS planners projected that an average of $6,000/year per student would be required to bridge that gap. COMPASS Project Objectives and Philosophy The goals of the COMPASS program are: 1.   Recruit 27 motivated, academically talented, and financially needy Biology, Biotechnology,

Environmental Science, and Chemistry majors at UNCP, including underrepresented minorities (URM) and first generation students, and provide them with financial assistance.

2.   Improve STEM education experiences for all of the recruited students and prepare them for the STEM workforce or graduate/professional programs. The intention was that 100% of scholars would graduate with a STEM degree in less than six years.

Program Recruitment and Demographics The first year of recruitment was conducted in spring semester only because of the award date from the National Science Foundation. The principle investigator posted flyers on campus and visited introductory science classes, while other professors made announcements in their classes. The Biology Department hosted a COMPASS web page, with additional links in Chemistry and Admissions/Financial Aid. Rising sophomores in the four targeted majors with a 3.0-4.0 GPA after one semester at UNCP received a direct email. Analysis of the student body had previously indicated that the potential pool should be large enough to recruit nine students per year. For example, of 178 freshmen in the four COMPASS majors in fall 2012, 45 had a 3.0-4.0 GPA. Of this group, 51% were on Pell grants, 51% had federal loans, 73% had personal loans, and 89% had some other financial aid. Only one student received no financial support. E-mails and flyers also were sent to local community colleges. After the first year, COMPASS scholars participated in recruitment efforts and visited freshman seminar sections, which all freshmen and any transfer student with less than 30 hours must attend in the fall semester. A promotional video produced in the fall of the first year was shown during spring visits to introductory science classes, or by course professors in some cases. The COMPASS program currently has three cohorts of 8, 9, and 10 students. One student from Cohort 1 left in the first semester due to family relocation to another state. He was replaced with a student of the same class year, who has been thriving. A second student from Cohort 1 left for personal reasons after two years. A new student of the same class year would have had only one year in the program rather than three. We instead enrolled 10 students in Cohort 3 in Fall 2016. Unfortunately, one student decided to change to a non-STEM major. A replacement was enrolled

at the beginning of Spring 2017. Tables 2 and 3 show scholar demographics and ethnicities. The program has successfully recruited women and URM. Table 2. Demographics of COMPASS Scholars.

Gender Major Female Male Biology Biotechnology Chemistry Double Major

22 (81%) 5 (19%) 14 (52%) 3 (11%) 8 (30%) 2 (7%) Table 3. Ethnicity of COMPASS scholars.

African American

Native American

Hawaiian/ Filipino Hispanic URM

Total White Total

Cohort 1 3 2 1 0 6 (75%) 2 (25%) 8 Cohort 2 2 3 0 1 6 (67%) 3 (33%) 9 Cohort 3 2 5 0 1 8 (80%) 2 (20%) 10

Total 7 10 1 2 20 (74%) 7 (26%) 27 Student Activities, Services and Programs Financial support is one of the pillars of the program. COMPASS was designed to support three cohorts of 9 students from their sophomore year through graduation by awarding a cumulative 81 annual scholarships totaling $486,000. In each year of the project, up to four students can receive an additional $2,100 of summer support if they participate in a research experience or internship and the site they attended does not provide a stipend. The other three program pillars are academic support, career support, and community support (including creating a community of scholars). The COMPASS program uses several strategies, balanced on these pillars, to ensure success and sustainability. First, it leverages existing UNCP programs such as career services, recruitment, and financial aid. Second, it explicitly builds a community of scholars who receive continuous support. Third, it develops career connections with regional industry. The key to accomplishing the critical support and community building outcomes is a coordinated program of intensive mentoring and tutoring, research opportunities and experiences, internships, professional development, and both academic and career advising. Building an inclusive learning community of STEM scholars also strengthens the institutional culture for other students at UNCP to want to excel and be part of this community. Success relies on a strong team approach, including the two principle investigators and nine STEM faculty. Many elements, such as tutoring, are already available to all UNCP STEM majors. However, the COMPASS program has expanded on these offerings to provide unique added benefits. Financial Support Many UNCP students must work or drop out for financial reasons. Research14 has shown that the effect of financial aid on retention in the first year may have a threshold, whereas any amount of financial aid in years two and three reduced the risk of departure of recipients relative to students without financial aid. Thus, the creators believed that offering financial support from sophomore year to graduation should be most effective for improving retention by allowing scholars to focus on academic excellence and career success. It is too early to fully evaluate the impact of financial support on persistence, but we did compare persistence of students COMPASS Cohorts 1 and 2

to students who applied to the program (GPA over 3.0) but did not receive the scholarship. One student from Cohort 1’s entry year left from each group. The COMPASS student persisted for two years and temporarily left school, whereas the student who did not receive the COMPASS scholarship abandoned school the next semester. However, COMPASS group is not performing better academically. Mean student GPA at application was 3.679 for COMPASS and 3.460 for non-COMPASS. In spring 2016 they were 3.477 and 3.336. COMPASS GPAs were 5.5% lower versus 3.6% lower for comparison students. Additional factors make this comparison somewhat biased in favor of the other group. The non-COMPASS group is smaller (12 students versus 18) and no information is available about factors such as their use of tutoring services or number of work hours. Self-reports show that financial support has alleviated stress for COMPASS students (e.g., what is the best thing about the program: “To focus on my studies without working to pay for my tuition”, “Financial support. It has helped me put my studies first and earn a 3.9 GPA”, “The financial, emotional, and academic support”). Yet, many continue to work for personal money. Perhaps more importantly, many non-COMPASS students did not demonstrate financial need. The important take-away may be that COMPASS scholars are not performing significantly less well than their counterparts who have greater financial resources. Academic Support COMPASS scholars all had at least a 3.0 GPA, so we did not expect severe unpreparedness. Even so, faculty believed that scholars would benefit from a math “conditioning” mini-course and academic support in the form of tutoring, supplemental instruction sessions, peer teaching, and science seminars, both with regard to academic success and self-confidence. Math Boot Camp: “Math Boot Camp” in an intensive mini-course designed to refresh students’ mathematics knowledge and increase fluency so students feel equipped and confident in future courses. Mathematics preparedness is the single best predictor of college performance in science, and rigorous high school math courses benefits college biology, chemistry, and physics.15, 16 Students whose coursework includes advanced mathematics such as Trigonometry or Calculus are the most likely to be college ready. This is true of students at all levels of achievement, not just high achievers.17 Science courses are becoming increasingly quantitative and professors notice the math anxiety of students who do not feel ready, especially in biology, sometimes considered the major for the “mathematically challenged” science student. Students attended Math Boot Camp for several days before the beginning of their first semester in COMPASS. There they practiced topics that were often problematic for them, like scientific notation, percentages, fractions, inverse functions, and logarithms. The instructor developed the curriculum with the input of professors at UNCP, who pointed to weaknesses they observed in their classes. After the first year, the instructor modified the curriculum to the needs of scholars. Week 1 instruction focused on algebraic concepts and dimensional analysis. Week 2 addressed calculus concepts. The course received excellent evaluations from students and data analysis showed measurable gains in knowledge as assessed by pre- and post-tests. All but one student in Cohort 1 achieved growth during Math Boot Camp. The mean post-test score across all three cohorts was 81/100 (SD = 15), versus a mean pre-test mean of 52/100 (SD = 29). Using a paired t-­‐test, we found that growth was statistically significant, t(26) = 6.376, p < 0.0001. Besides making virtually all students feel like they had improved their mathematics skills, students also

reported that they had increased confidence in other science subjects. Tutoring: COMPASS scholars are required to perform at an above-average level and keep grades in the A-B range. Tutoring or supplemental instruction (SI) sessions are critical to success. Even when students are comfortable in a subject, there are benefits to participation. Students may discover that they have misconceptions and that they cannot relate concepts learned at different times in a course or across different courses. Tutoring also challenges them to push themselves to a higher order of thinking, analysis, or evaluation. In a group setting, students learn by teaching other students. Indeed, research shows that teaching each other in small groups improves student learning when compared to traditional lecture-based courses in biology and chemistry.18, 19 Even fast learners can derive benefits from peer teaching. This activity helps develop their ability to express ideas and values such as sharing, sense of responsibility, self-confidence, self-discipline, and time management.20 For both tutoring and SI, COMPASS uses existing university resources. The SI program, funded by a UNCP grant, was terminated after our first year. As compared to tutoring, SI was the better mode because the SI leader attended the classes for which students needed help. Tutors have resulted in less consistent success; for upper level classes, tutors are not always available. However, the COMPASS students have developed a peer teaching system and study groups. Following recommendations of the PI and mentors, they made more use of office hours when tutoring was not offered or when the hours conflicted with their schedules. Overall, student ratings for tutoring and SI improved from Fall 14 (the first semester in which COMPASS students provided feedback) until Fall 15 (see Figure 1). Ratings declined somewhat in Spring16, which appears to correlated with the fact that fewer students were attending tutoring sessions. It also likely reflects the loss of the SI program after the first year. Figure 1. Student Ratings of Tutoring/Supplemental Instruction

Guest Speakers: COMPASS creators believe it is very important to broaden students’ knowledge beyond classroom instruction, including current research and applicability of textbook concepts. Scholars are advised to attend the spring semester Biology Seminar series. They can enroll for a one credit hour class or attend the one-hour-per-week sessions without enrolling. We strongly

0%10%20%30%40%50%60%70%80%90%100%

Fall14 Spring15 Fall15 Spring16

Tutoring/SI

Extremely  important

Very  important

Somewhat  important

Somewhat  unimportant

Very  unimportant  

Not  at  all  important

encourage enrollment for as many semesters as possible. All scholars have enrolled at least once. As anticipated, several topics have greatly sparked COMPASS students’ interest. In at least one instance, a student contacted the speaker to explore summer research and beyond. The student considered a career in a field where they might not have before. Figure 2 (Scientific Seminars bar) shows student ratings for the seminar; 66% found seminars either useful or very useful. Figure 2. Student Ratings of Academic and Career Support Activities

Career Support COMPASS offers proactive assistance to place graduates in STEM careers or help them apply to and gain acceptance into graduate programs. Workshops/Seminars: Networking and professional development are important for all careers. Seminars offered to COMPASS scholars contribute to the development of the kind of “soft/life” skills that they also build during cohort meetings. Speakers from the career center held seminars on writing resumes and finding internships to every cohort during their first program semester. They also offered a service called “interview stream” for students to practice interviewing skills. Invited experts spoke on topics such as personal finance and ethical issues. Local speakers with STEM industry experience talked to students about careers in forensics and clinical laboratories. COMPASS scholars attended two presentation and discussion sessions with Dr. Theresa Duello of the University of Wisconsin-Madison, who has significant experience advising students and planning for graduate school. The Collegial Academy of the North Carolina Academy of Science (CANCAS) workshop held off-campus at Pfeiffer University offered plentiful opportunities for COMPASS scholars to learn more about disciplinary research and study abroad opportunities, graduate school programs, interviewing skills, and resume building. Two annual hands-on workshops targeted the needs of COMPASS scholars at different stages. The first focused on strategies for improving academic skills such as studying, reading, test taking, etc. Most attendees were students in the incoming cohort. The second workshop was an intensive GRE preparation mini-course, held as a series of four two-hour sessions. The focus was

0%10%20%30%40%50%60%70%80%90%100%

COMPASS  Activities  -­‐‑ Students  Ratings

Very  useful

Useful

Somehow  useful

Neutral

Somewhat  useless

Useless

Very  useless

mostly junior and senior students. Career development workshops and seminars support the goal of improving the academic experience of students and preparing them for a STEM career. The hands-on workshops received very good ratings, despite requiring students to pay a fee, probably because of perceived immediate applicability. The response to career development seminars was variable, although largely positive, reflecting the variability of participants (Figure 2 above). Site Visits: Off-campus visits provide COMPASS scholars with a clearer picture of what their working life in different STEM occupations might be like. During the first two and a half years of the program, students had nine site visit opportunities, such as University of North Carolina Nutrition Research Institute, Wake Forest Institute for Regenerative Medicine, and Golden LEAF Biomanufacturing Training and Education Center (BTEC), and local production facilities for companies like Novozymes (biotechnology) and Novo-Nordisk (pharmaceuticals). Scholars attended an average of two visits per school year and site visits have received very good reviews in surveys (Figure 2 above). Research Support Understanding the work done by STEM professionals is important but not sufficient to prepare for a career in science. There must be a personal development dimension, which works jointly with professional development to steer students toward science.21 A large body of literature has demonstrated the benefits of student participation in undergraduate research (UR).22-30 Pervasive changes in student behavior required in the future STEM workforce include the ability to think independently, formulate one’s own ideas, become more intrinsically motivated to learn, and become a more active learner. There is no absolute consensus on how UR influences career choices,22, 26, 31 but the chance to conduct independent research is often cited as the compelling experience that launches a scientific career.21, 32 “Introduce the scientific process to students early” is also one of the changes advocated by the American Association for the Advancement of Science's (AAAS) Vision and Change in Undergraduate Biology Education.33 Such experiences can be particularly beneficial for first-generation students, URM, and at-risk students, to improve retention.34-37 The second recommendation in the report “Engage to Excel” (President’s Council of Advisors on Science and Technology38) is to “advocate and provide support for replacing standard laboratory courses with discovery-based research courses.” COMPASS strongly encourages students to seek out summer internships, preferably away from UNCP, so that they can immerse themselves in a peer program and realize that other students share the same aspirations, and perhaps the same insecurities. Living and working with students from all kinds of schools is an empowering feeling for our rural students. It helps mitigate the “impostor syndrome” prevalent among minority groups. For UNCP students, many of whom are not used to traveling outside the small confines of their county or the state of NC, it is important to break out of this comfort zone in this kind of safe and welcoming community. Not only will the research be top-notch, but they will discover whether research is truly an option for them. In particular, we have emphasized NSF Research Experience for Undergraduates (REU) programs. To date, two scholars have participated in REUs. Four students the first summer and three the second summer received the $2100 COMPASS summer stipend.

Scholars have been involved in research during summers and/or the academic year (see Table 4). For summer 2017, 13 of the 18 students remaining after Cohort 1’s graduation applied or plan to apply for a summer research program. Notably, a Cohort 3 student received a 2017 EXceptional Research Opportunities Program (EXROP) award. This prestigious program from the Howard Hughes Medical Institute (HHMI) provides outstanding summer research experiences to talented undergraduates in order to encourage them to pursue careers in academic science. The long-term goal is to ensure that a diverse and highly trained workforce is available to assume leadership roles in science. A Cohort 1 scholar received an EXROP award in 2015, but had to relocate due to a critical family illness. This 2015 scholar received a second EXROP summer research award in 2016 to continue his work with Dr. David Ginsburg at the University of Michigan. Table 4. Numbers and % of Scholars Involved in Research

Summer 15 Cohort 1

AY 15-16 Cohorts 1 and 2

Summer 16 Cohorts 1 and 2*

5 of 9 (56%) 13 of 18 (72%) 13 of 19 (68%) *includes one student from Cohort 3 who was already involved in research when recruited Three Cohort 1 students presented research results at a professional conference in 2015. Nine students from Cohorts 1, 2, and 3 presented in 2016. One student placed third at the Derieux Awards for Excellence in Undergraduate Research for her poster at the NC Academy of Science meeting in 2016. When asked about the accomplishments they were proud of during their time in COMPASS, some students have made comments such as, “Presenting my Research at PURC. I got to talk about my research and explain it to others. I felt I understood” and “I was also proud to present my research at the annual NCAS meeting.” Community Support (Building a Community of Scholars) According to an ACT study, student retention can be improved by integrating both academic and non-academic factors, such as academic self-confidence, achievement, motivation, institutional commitment, and social support.39 In many instances and despite poor academic performance, students persist because they feel like they fit within the institution and that they are socially integrated.40 Studies suggest that programs that explicitly include mentoring and support groups improve student involvement, motivation, and academic self-confidence and, in turn, increase institutional commitment and engagement.41, 42 Inclusive, welcoming institutional environments and the connections students feel have been linked to persistence.43-45 Other research shows that learning communities have positive effects on persistence.46, 47 Through intense mentoring and activities that bring scholars together, COMPASS is working to develop a social and learning community, as well as foster a shared consensus of institutional commitment to retention. Mentoring: Nine faculty members from the Biology and Chemistry departments at UNCP serve as COMPASS mentors. They are the core connections than enable the academic, career support, and community building activities to work as planned. Each mentor had one student in the first year, two in the second year, and are currently mentoring three students now that all cohorts have been recruited. This will be reduced to two and one students in years four and five respectively, assuming on-time graduation by COMPASS scholars. We use input from mentors and students to match them. Mentors meet monthly with each student and promptly address any decline in academic performance. Mentors also help students plan registration, identify internships and/or

research opportunities, and discuss career options. Each mentor brings his or her own particular style. Some of them meet in their offices, while others take students off-campus to lunch. The belief that one is able to accomplish a task is an important motivational construct and a good predictor of course performance.48 The more grounded students are in their traditions, the more likely they are to graduate.49 Research indicates that contextual, cognitive, and cultural factors are particularly significant for academic achievement and retention of ALANA students (African Americans, Latino/as, Southeast Asians, and Native Americans).50 Historically, UNCP offers a comfortable atmosphere to minority students, particularly Native Americans, and demonstrates a high degree of institutional engagement. COMPASS also has diversity among mentors with respect to gender (6 females, 3 males), ethnicity (Native American, Hispanic, Caucasian), and career status (lecturer to full professor), which helps to better fit our diverse group of students. Examples of contextual issues addressed by mentors include:

•   How do COMPASS scholars perceive the climate and faculty support in the department? •   What are the academic barriers to success?

Examples of cognitive issues addressed by mentors include:

•   Are COMPASS scholars confident they can keep the academic performance? •   Are they confident that they can cope with the challenges of degree completion? •   Do they believe their major will help them find a job? •   Do they feel capable of succeeding in the major?

Examples of cultural issues addressed by mentors include:

•   How do COMPASS scholars feel they are fitting into the program? •   Does race or ethnic identity play a role in how comfortable students feel in the learning?

Figure 3. Role of Mentors in Emotional Support and Motivation to Succeed

0%10%20%30%40%50%60%70%80%90%100%

Fall14 Spring15 Fall15 Spring16

Emotional  support

Very  useful

Useful

Somehow  useful

Neutral

Somewhat  useless

Useless

Very  useless

Mentoring is one of the resources students value most (Figure 3, above). Beyond typical advising activities such as help with course planning, mentor emotional support and motivation were particularly important to COMPASS students. Between fall 2014 and spring 2016, more than half rated these mentor functions as “very useful” and the majority found them useful to some degree. Cohort 1 scholars were particularly appreciative. These findings are in line with studies showing the importance of contextual and cultural aspects for retention. COMPASS Community: The third recommendation of AAAS’s Vision and Change report33 is to foster communities of scholars consisting of undergraduates, graduate students, faculty, and administrators. These communities should develop, use, assess, and disseminate best practices in the field. Student success requires institutional investment in structured and aligned activities directed toward success. Deep commitment of all stakeholders and excellent coordination of resources helps COMPASS drive and contribute to a cultural shift to build a strong community. COMPASS promotes interactions among students as well as with the larger community through various activities and resources intended to create personal and professional bonding.

•   Site visits are opportunities for students to interact informally and get to know each other; we always make a point of stopping for dinner and this is a favorite time for students.

•   Joint travel and attendance to seminars, workshops, and conferences with students from other scholarly programs fellows is an opportunity for further interaction.

•   The COMPASS lounge is a dedicated space for students to study with one another or to “hang out” together on campus.

•   Social gatherings for the scholars and mentors facilitate bonding outside the academic environment. There is an annual “welcome and beginning of the year” cookout party at the COMPASS PI’s house, plus other smaller socials.

Student comments at focus groups and in surveys reflect their appreciation for the community that is growing together and beyond the COMPASS core: “[one of the best things is] The sense of community and the overall cohesiveness of the group including the students and the faculty” and “We are sort of a small family that is passionate about science.”

0%10%20%30%40%50%60%70%80%90%100%

Fall14 Spring15 Fall15 Spring16

Motivation  to  succeed

Very  useful

Useful

Somehow  useful

Neutral

Somewhat  useless

Useless

Very  useless

Weekly Meetings: Weekly meetings fulfill the dual goal of maintaining group cohesiveness and provide a venue for some professional development activities. In the first semester of the first year, the emphasis was on introducing students to what graduate school is about and the benefits of getting into graduate school. The student body at UNC Pembroke is predominantly composed of local students from this rural area. The academically successful science students are mostly bound for health careers and are largely unaware of career opportunities outside the health fields. Many never thought about graduate school, now knew about stipends for advanced degrees. We used videocasts produced by the Office of Intramural Training and Education at the National Institutes of Health, which has talks by program admissions officers of top institutions, as well as experts in professional development and various alternative careers in science. The rationale was that these speakers were truly an authority in their fields and it would have been difficult to bring such an array of high caliber speakers to UNCP. We watched and discussed pertinent points to offer clarification from personal experience as needed. We repeated the experience with Cohort 2 the second year while the cohort 1 students concentrated on preparing effective presentations and Networking. Some of the Cohort 1 students had, at that point, undergone their first research experiences and were ready to present at meetings. During the fall semester of their second year we watched talks about preparing posters, speaking to different audiences, improving spoken English, and using LinkedIn for networking. Meetings are the one program aspect where there are divided opinions among students. When asked if the meetings helped them stay focused on schoolwork and strive to keep their grades up, 37.5% agreed or strongly agreed, 37.5% were neutral, and 25% disagreed or strongly disagreed. After the second year in the program, the feedback we received about meetings indicated that students did not favor watching videos during that time. We have slowly moved away from the videos toward free discussion for students to learn about each other, to express their concerns, aspirations, plans, etc., and practice soft skills such as communication. We are further planning to include skills such as work-life balance, surviving as a woman in STEM, time management, etc. Other changes implemented based on student feedback were the distribution of agendas and minutes to students and mentors to keep them informed. Successes and Challenges Through surveys, observations, and focus groups with students and mentors, we have determined where we believe the COMPASS program is meeting its goals and producing positive outcomes, as well as areas where improvement and revisions are needed. When students were asked for best and worst aspects of the program, the main words in their responses were opportunities or help (best) and meetings or time or nothing (worst). The questions were phrased somewhat differently over time. For example, the students initially were asked, “What do you consider to be the best aspect of this scholarship program?” In fall 2016 this was phrased as, “If you could congratulate the organizers of the COMPASS program on ONE thing, what would it be and why?” Figure 4 and Figure 5 show the results of a SIMPLE frequency analysis performed on student responses using the online tool “Wordle”. The top 25 words (including ties) are shown.

Figure 4. 25 Most Frequent Words in Response to Best Aspects of the Program

Figure 5. 25 Most Frequent Words in Response to Worst Aspects of the Program

Successes

1-   Building a learning community of scholars. One of the most valued aspects of the program is the sense of community. Belonging to a support group to which they can turned to for advice, study, and simply friendship is a major plus for students. Our model of cohorts with students in different stages of their college years: sophomore, junior, senior, lends itself to develop and practice soft skills such as leadership, collaboration/team work, absorbing constructive criticism, adaptability. Key to this success was securing a COMPASS lounge with microwave, coffee maker, table chairs, whiteboard and couch where students go regularly to study in groups, tutor each other, reheat a quick meal, or take a short nap during exam crunch. Another critical pillar of this community is the figure of the mentor. When asked about the best aspects of the program, mentors and community are some of the most cited. “The fact that we each get a mentor. The mentor understands us and understands how we are planning our life.” “We have mentors, who guide us and help us out with our plans.’

“The sense of community and the overall cohesiveness of the group including the students and the faculty. We are sort of a small family that is passionate about science.” “A sense of community with goal oriented individuals similar to myself.” “You find people to study with and form relationships with who are taking the same classes and looking for the same careers.” “The sense of community and how close we have become as a group. We have our own little support system in each other and the faculty that assist us.” “Having each other to talk to for support.”

2-   Leveraging with other programs and existing resources. Getting a program started “from scratch” is a pretty formidable effort. The COMPASS program sought to utilize resources and team up with other programs. Seminars and hands-on workshops (study skills, GRE prep) have been often organized in conjunction with the RISE program. COMPASS scholars have benefited from the tutoring and SI programs provided by the University.

3-   Attracting a large percentage of URMs. Currently at the peak of the program with 27 students enrolled, 74% are URMs. In particular, the percentage of Native American students has increased from cohort 1 (20%) to cohort 3 (60%) and now they make over 40% of the total number of students in the program. COMPASS is serving our community and fulfilling the goal of increasing diversity of STEM graduates.

4-   Recruiting sophomore students. This may appear counterintuitive since recruiting raising

freshmen could potentially get more students interested to come to UNCP and increase the number of applicants. However, we don’t believe that this would make a big difference because UNCP students make this institution their school of choice even before the scholarship for various reasons: i) the proximity; indeed most students are from the surrounding region and many are commuters since that allows them to live with their families, ii) a large percentage are Native American (mostly Lumbee) and UNCP historically has been serving this population and was created as a Normal School for Indians; iii) they are from low income households, or are supporting themselves to come to college since a large percentage are non-traditional students, and UNCP offers the lowest tuition rates in the UNC system. Recruiting sophomores has the added advantage that we have access to their transcripts, can request letters of recommendation from college professors, and they have proven to be able to handle the high school-college transition in some degree. This scholarship still constitutes a retention tool because students are informed in their freshman seminar classes and introductory science classes of the existence of the scholarship and the requirements for eligibility. It also constitutes a recruitment tool despite not recruiting raising freshmen directly, because the information about the scholarship is spread out through open houses, the admission website, financial aid office, and honors college, and directly contacting surrounding community colleges.

5-   Keeping the students focused and motivated. Overall, ALL participants report that the COMPASS program has kept them focused and motivated on their school work (Figure 6). The two comments below sum-up well the general sentiment among students: “Staying focused. It was hard but I tried to stay focused on my goals and stay motivated. I wouldn’t have done it without the amazing people around me though.”

“The compass program was one of the best things that has happened to me.” As a direct consequence of feeling motivating to keep their grades up, five of the eight cohort 1 students are graduating with honors this Spring 17 (Summa Cum Laude, Magna Cum Laude and Cum Laude). Figure 6. Students’ response to the statement: I think overall the program has helped me stay focused in my studies and strive to keep my grades up

6-   The COMPASS program is offering a good quantity and variety of quality professional

development opportunities. Students are engaging in professional development, including summer research and work relevant to STEM careers. The research activity has yielded 12 presentations at professional conference forums this far. Students are being accepted into summer programs; for three consecutive years, the prestigious HHMI EXROP program has selected a COMPASS scholar.

Challenges

1-   Recruiting enough students interested in STEM careers or graduate school that have GPAs of 3.0 and above and demonstrated financial aid. Part of this challenge is just to get all the potential candidates to apply. These were raising sophomores in the four targeted majors of the program who had a cumulative 3-4 GPA after one semester at UNCP. These, as well as undecided students, who ranged from 24-40 depending on the year, received direct emails in their school accounts in the spring semester for entry in the next fall. The number of students applying were 16 (out of 60 potential applicants), 18 (of 56), and 20 (of 58) respectively for cohorts 1, 2, and 3, showing a clear increase in the percentage of applicants, which is an indication that the program has become better known over time. However, it is still just above 1/3 of the potential pool. As mentioned above, UNCP academically strong science students tend to have a goal of going into health careers. Perhaps as a combination of lack of information –they have been told by parents that if they are good students they have to be lawyers or doctors- and

0%10%20%30%40%50%60%70%80%90%100%

Fall14 Spring15 Fall15 Spring16

Focus  &  Motivation

Strongly  agree

Agree

Neither  agree  or  disagree

Disagree

Strongly  disagree

because they see health occupations as a way of coming back to work in their community and have a good living. Our strategy has been to expose those students to as much information as possible about alternative careers through site visits, talks, and discussions. Also, encouraging at least one summer research experience, preferably away from UNCP, where they can experience first-hand what life as graduate student might be, and decide if they would like to consider graduate school.

2-   One difficulty has been the lack of an administrative assistant/program coordinator figure. While the proposal was crafted, UNCP had a figure called the “Hawk Assistant”. These are students that are paid by a university grant to work on projects with faculty members. The PI had put two applications for two Hawk Assistant positions. One would be in charge of clerical, administrative tasks such as gathering information from students to enter in data bases, obtaining permissions for travel, making sure students register for conferences and other events, consolidating students’ schedules on files, making hotel reservations, requesting travel reimbursement expenses when needed, etc. The other would be to maintain and update our website and post pictures, events etc. on our Facebook page. Because the Hawk assistant program was terminated right before our program started, we found ourselves without this valuable help and the PI of the grant has been carrying out all these time-consuming tasks. Some administrative tasks that require more privacy are already performed by the PI, such as gathering information on GPAs, communicating with the financial aid office that verifies the unmet need of students based on their FAFSA, and occasionally making sure students provide all document for verification, would this be necessary. Fortunately, the new S-STEM RFA contemplates a higher allocation for administrative support and, if we are awarded a second round of funding, we will be able to request funds for a program coordinator.

3-   We have found that occasionally it has been more difficult than expected to get students to respond in a timely manner to surveys, requests for attendance to events, etc. when using email. When the PI used text messages students were much quicker to respond. However, email remains our preferred method because it is traceable and leaves a record. To improve communication, we are using the “Remind” app for smart phones, which can be used to send a quick message to all students to remind them to complete a task.

4-   Although many students attend the programmed activities and view them as opportunities to learn and grow, some appear to see them as time consuming events or simply they don't prioritize them. In order to encourage accountability, we have implemented a “contract” listing of the activities offered and the minimum participation that is signed by students. Also, instead of working to enforce attendance, which may exacerbate the feeling of obligation, we point out the missed benefits; whatever these might be, from improving public speaking, to learning how to be financially educated, to being ethically responsible, to discovering job opportunities, to finding out about graduate school and navigating the application process, to knowing how to network, etc.

At our weekly meetings, we have been practicing a form of “show and tell” where students having attended an event/activity/research program tell others in their own

words about their experience (good and bad), what they got out of it and why they recommend it. References

1. Council, C.o.S.E.N.S.a.T., Federal Science, Technology, Engineering, and Mathematics (STEM) Education. 5-Year Strategic Plan. 2013.

2. King, B. and M. Osburn. A Snapshot of Students. The University of North Carolina at Pembroke. 2012 [cited 2013 July 13]; Available from: http://www.uncp.edu/ie/resources/UNCPStudents_updated2012.pdf.

3. Carnevale, A.P., N. Smith, and J. Strohl, Help Wanted: Projections of Jobs and Education Requirements through 2018. Executive Summary. 2010, Georgetown University Center on Education and the Workforce. 3300 Whitehaven Street NW Suite 5000 Box 571444, Washington, DC 20057. Tel: 202-687-4922; Fax: 202-687-3110; e-mail: cewgeorgetown@georgetown.edu; Web site: http://cew.georgetown.edu/.

4. Langdon, D., et al., STEM: Good Jobs Now and for the Future. ESA Issue Brief #03-11. 2011, US Department of Commerce. 1401 Constitution Avenue NW, Washington, DC 20230. Tel: 202-482-2000; Web site: http://www.commerce.gov/.

5. Lacey, T.A. and B. Wright, Occupational employment projections to 2018. Monthly Labor Review, 2009. 132(11): p. 82-123.

6. Carnevale, A.P., et al., STEM: Science Technology Engineering Mathematics. 2011, Georgetown University Center on Education and the Workforce.

7. Carnevale, A.P., et al., STEM: Science Technology Engineering Mathematics. State-Level Analysis. 2011, Georgetown University Center on Education and the Workforce.

8. (PCAST), P.s.C.o.A.o.S.T., Engage to Excel: Producing One Million Additional College Graduates with Degrees in Science, Technology, Engineering, and Mathematics. Report to the President. 2012, Executive Office of the President. 1600 Pennsylvania Avenue NW, Washington, DC 20500. Tel: 202-456-1111; Fax: 202-456-2461; e-mail: comments@whitehouse.gov; Web site: http://www.whitehouse.gov/.

9. Engineering, N.R.C.N.A.o., Community Colleges in the Evolving STEM Education Landscape: Summary of a Summit. 2012: The National Academies Press.

10. Statistics, N.C.f.E. Digest of Education Statistics. Table 9. Number of persons age 18 and over, by highest level of educational attainment, sex, race/ethnicity, and age: 2011. 2011; Available from: http://nces.ed.gov/programs/digest/d11/tables/dt11_009.asp.

11. Vasay, E.T., The effects of peer teaching in the performance of students in mathematics E-International Scientific Research Journal 2010. 2(2): p. 161-171.

12. Interactive, H., Stem Perceptions: Study & Parent Study. Parents and Students Weigh in on How to Inspire the Next Generation of Doctors, Scientists, Software Developers and Engineers. Study Commissioned by Microsoft Corp. . 2011.

13. UNCP Survey of Freshman and STEM Faculty. Available from: http://www2.uncp.edu/chem_phys/step.

14. Ishitani, T.T. and S.L. DesJardins, A Longitudinal Investigation of Dropout from College in the United States. Journal of College Student Retention, 2003. 4(2): p. 173-201.

15. Sadler, P.M. and R.H. Tai, Transitions. The two high-school pillars supporting college science. Science, 2007. 317(5837): p. 457-8.

16. Tai, R.H., P.M. Sadler, and J.J. Mintzes, Factors Influencing College Science Success. Journal of College Science Teaching, 2006. 36(1): p. 52-56.

17. Crisis at the Core: Preparing All Students for College and Work. 2005, ACT, Inc. 500 ACT Drive, P.O. Box 168, Iowa City, IA 52243-0168. Web site: http://www.act.org/.

18. Tessier, J., Small-Group Peer Teaching in an Introductory Biology Classroom. Journal of College Science Teaching, 2007. 36(4): p. 64-69.

19. Tien, L.T., V. Roth, and J.A. Kampmeier, Implementation of a Peer-Led Team Learning Instructional Approach in an Undergraduate Organic Chemistry Course. Journal of Research in Science Teaching, 2002. 39(7): p. 606-32.

20. Beyond the Rhetoric: Improving College Readiness through Coherent State Policy. A Special Report by the National Center for Public Policy and Higher Education and the Southern Regional Education Board. 2010, Southern Regional Education Board. 592 10th Street NW, Atlanta, GA 30318-5790. Tel: 404-875-9211; Fax: 404-872-1477; e-mail: publications@sreb.org; Web site: http://www.sreb.org/.

21. Lopatto, D., Science in Solution: The Impact of Undergraduate Research on Student Learning. 2010, Tucson, AZ: Research Corporation for Science Advancement.

22. Bauer, K.W. and J.S. Bennett, Alumni Perceptions Used To Assess Undergraduate Research Experience. Journal of Higher Education, 2003. 74(2): p. 210-30.

23. Hunter, A.-B., S.L. Laursen, and E. Seymour, Becoming a scientist: The role of undergraduate research in students' cognitive, personal, and professional development. Science Education, 2007. 91(1): p. 36-74.

24. Junge, B., et al., Promoting Undergraduate Interest, Preparedness, and Professional Pursuit in the Sciences: An Outcomes Evaluation of the SURE Program at Emory University. CBE-Life Sciences Education, 2010. 9(2): p. 119-132.

25. Kardash, C.M., Evaluation of an Undergraduate Research Experience: Perceptions of Undergraduate Interns and Their Faculty Mentors. Journal of Educational Psychology, 2000. 92(1): p. 191-201.

26. Kremer, J.F. and R.G. Bringle, The Effects of an Intensive Research Experience on the Careers of Talented Undergraduates. Journal of Research and Development in Education, 1990. 24(1): p. 1-5.

27. Laursen, S., et al., Undergraduate Research in the Sciences: Engaging Students in Real Science. 2010: Wiley.

28. Lopatto, D., Undergraduate research as a high-impact student experience. Peer Review, 2010. 12: p. 27+.

29. Russell, S.H., M.P. Hancock, and J. McCullough, The pipeline. Benefits of undergraduate research experiences. Science, 2007. 316(5824): p. 548-9.

30. Seymour, E., et al., Establishing the benefits of research experiences for undergraduates in the sciences: First findings from a three-year study. Science Education, 2004. 88(4): p. 493-534.

31. Thiry, H., S.L. Laursen, and A.-B. Hunter, What Experiences Help Students Become Scientists? A Comparative Study of Research and Other Sources of Personal and Professional Gains for STEM Undergraduates. Journal of Higher Education, 2011. 82(4): p. 357-388.

32. Council, N.R., BIO2010:Transforming Undergraduate Education for Future Research Biologists. 2003: The National Academies Press.

33. (AAAS), A.A.f.t.A.o.S., Vision and change in undergraduate biology education: A call to action. 2011.

34. Lopatto, D., Undergraduate research as a catalyst for liberal learning. Peer Review, 2006. 8: p. 22+.

35. Elgren, T. and N. Hensel, Undergraduate research experiences: synergies between scholarship and teaching. Peer Review, 2006. 8: p. 4+.

36. Goins, G.D., et al., HBCUs model for success: Successful models for effectively retaining and graduating students at North Carolina A&T State University 2009, Thurgood Marshall College Fund: New York, NY.

37. Lopatto, D., Undergraduate Research Experiences Support Science Career Decisions and Active Learning. CBE - Life Sciences Education, 2007. 6(4): p. 297-306.

38. Olson, S. and J.B. Labov, Community Colleges in the Evolving STEM Education Landscape: Summary of a Summit. 2012: National Academies Press. 500 Fifth Street NW, Washington, DC 20001. Tel: 888-624-8373; Tel: 202-334-2000; Fax: 202-334-2793; e-mail: Customer_Service@nap.edu; Web site: http://www.nap.edu/.

39. Lotkowski, V.A., S.B. Robbins, and R.J. Noeth, The Role of Academic and Non-Academic Factors in Improving College Retention. ACT Policy Report. 2004.

40. Kennedy, P.W., B.G. Sheckley, and M.T. Kehrhahn. The Dynamic Nature of Student Persistence: Influence of Interactions between Student Attachment, Academic Adaptation, and Social Adaptation. 2000.

41. Mangold, W.D., et al., Who Goes Who Stays: An Assessment of the Effect of a Freshman Mentoring and Unit Registration Program on College Persistence. Journal of College Student Retention, 2003. 4(2): p. 95-122.

42. Padgett, V.R. and J.F. Reid, Jr., Five Year Evaluation of the Student Diversity Program: A Retrospective Quasi-Experiment. Journal of College Student Retention, 2003. 4(2): p. 135-45.

43. Hurtado, S., et al., Enhancing Campus Climates for Racial/Ethnic Diversity: Educational Policy and Practice. Review of Higher Education, 1998. 21(3): p. 279-302.

44. Kuh, G.D., The Other Curriculum: Out-of-Class Experiences Associated with Student Learning and Personal Development. Journal of Higher Education, 1995. 66(2): p. 123-55.

45. Oseguera, L. and B.S. Rhee, The Influence of Institutional Retention Climates on Student Persistence to Degree Completion: A Multilevel Approach. Research in Higher Education, 2009. 50(6): p. 546-569.

46. Astin, A.W., What Matters in College? Four Critical Years Revisited. Jossey-Bass Higher and Adult Education Series. 1993: Jossey-Bass Inc., 350 Sansome Street, San Francisco, CA 94104 ($30.95).

47. Tinto, V., Leaving College: Rethinking the Causes and Cures of Student Attrition. Second Edition. 1993: University of Chicago Press, 5801 South Ellis Avenue, Chicago, IL 60637 ($24.95).

48. Wheeler, E.R., A theoretical lens on a biology intensive orientation program: A study of self-efficacy and self-regulation of freshman biology majors. 2012, Southern University and Agricultural and Mechanical College: Ann Arbor. p. 170.

49. Rodriguez, R., Learning to live a warrior's life: institute seeks to improve Native American education, in Black Issues in Higher Education. 1997. p. 38+.

50. Wisconsin, C.o.E.a.W.U.o., Increasing STEM retention for underrepresented students: Factors that matter. 2008, University of Wisconsin: Wisconsin-Madison.