three studies to drive campus conversations on student success

An initiative of the Research & Planning Group

for California Community Colleges

Three Studies to Drive Campus Conversations on Student Success

Rob Johnstone, RP Group



Today we’ll cover:

Working toward a “Culture of Inquiry” – selected findings from statewide projects

Grades & Success in Sequenced CoursesLevels of Preparedness in GE Courses –

success of simultaneously enrolled students in Math / English courses

Cohort Tracking in Basic Skills



Ground Rules

Ask lots of questions – especially nagging and tough ones

If I’m talking too fast, throw something at me (preferably not something sharp)

Data doesn’t make meaning; we make meaning from data

Spontaneous outbursts of applause are encouraged



Data, Evidence, College Decision-making, Student Success, and You…



How do colleges make good decisions ?

“Learning emerges from an ongoing conversation about things that matter with both passion and discipline”

- Parker PalmerThe Courage to Teach

If you want to learn you have to take part in the conversation

Learning is founded on evidence and inquiry



Framing Thought #1

“Data do not speak for themselves. What is also needed are occasions that bring educators together to examine evidence about student learning, reflect on its meaning and identify approaches that yield better results.” Carnegie Foundation (2009)



Framing Thought #2

Decision making in complex systems is a highly social process, not solely an intellectual exercise.

The process is composed of many small acts, carried out by different people at different points in time.



Institutional Research (IR) & You

IR is about helping the college and its practitioners make better decisions

Institutional research is fundamentally an interventionist discipline.

We are using non-experimental data to examine behavior in complex human systems

We are not seeking absolute truths; rather we are looking for patterns of evidence that inform action-oriented decisions



Linear Model of Institutional Research

Faculty/Staff

Researcher

Primary Responsibility

Key Features:

• Request-driven• Highly delineated tasks• Emphasis is on generating

metrics



Collaborative Model of Inst. Research

Faculty/Staff

Researcher

Joint Activity

Primary Responsibility

Key Features:• Dialogue-rich• Jointly-driven processes



Colleges may be focusing on the wrong data.

Finding from RP Research



0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

1989 1990 1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008

Retention Rate

Success Rate

Consider the multitude of changes over this period:

- faculty/staff turnover- program successes/failures- changing student demographics - budget contractions/expansion- evolving state & accreditation mandates- leadership turnover

and yet these performance metrics remained amazingly stable.

What does that tell us about the usefulness of these metrics in setting institutional strategies?

Twenty Year Trend for Common Institutional Outcome Measures



Much of what is needed to support good decision making lies beyond the current grasp of our institutional databases.

An Observation



What is Our College’s Transfer Rate?



Insight from the Field: A Look at Three Studies



Section 2: Grades & Success in Sequenced

Courses



82%

67%

49%

37%42%

59%

18%

33%

51%

63%58%

41%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

A in English 826 B in English 826 C in English 826 Other Grade in English 826

English 836/846 Non-Success

English 836/846 -First English Course

Success Rates of English 836/846 Students by Entry Group

Success Rate Non-Success Rate



88%

80%

58%

47% 45%

65%

12%

20%

42%

53% 55%

35%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

A in English 836/846 B in English 836/846 C in English 836/846 Other Grade in English 836/846

English 100/105 Non-Success

English 100/105 -First English Course

Success Rates of English 100 Students by Entry Group




60%

51%

38%

19%

43%

54%

40%

49%

62%

81%

57%

46%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

A in Math 811 B in Math 811 C in Math 811 Other Grade in Math 811

Math 110/111 Non-Success

Math 110/111 - First Math Course

Success Rates of Math 110 Students by Entry Group




77%

55%

35%

21%

45%

57%

23%

45%

65%

79%

55%

43%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

A in Math 110/112 B in Math 110/112 C in Math 110/112 Other Grade in Math 110/112

Math 120/122 Non-Success

Math 120/122 - First Math Course

Success Rates of Math 120 Students by Entry Group




96%

79%

40%

71%

61%

77%

4%

21%

60%

29%

39%

23%

0%

20%

40%

60%

80%

100%

120%

A in Chemistry 210 B in Chemistry 210 C in Chemistry 210 Other Grade in Chemistry 210

Chemistry 220 Non-Success

Chemistry 220 - First Chemistry Course

Success Rates of Chemistry 220 Students by Entry Group

Success-Rate Non-Success Rate



94%91%

76%

64%

81%85%

89%

6%9%

24%

36%

19%15%

11%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

A in Biology 250 B in Biology 250 C in Biology 250 Other Grade in Biology 250

Biology 260 Non-Success

Biology 260 - First Biology Course

Simultaneous / Other Group for

Biology 260

Success Rates of Biology 260 Students by Entry Group




91%

81%

68%

29%

66%71%

9%

19%

32%

71%

34%29%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

A in Economics 100 B in Economics 100 C in Economics 100 Other Grade in Economics 100

Economics 102 Non-Success

Economics 102 - First Economics Course

Success Rates of Economics 102 Students by Entry Group




Section 3: Success in GE Courses by Simultaneously

Enrolled English / Math Students



Psychology 100 Success of Students Simultaneously Enrolled in English

Simultaneously Enrolled English Course

Count in Psyc 100

% of Psyc 100 Enr.

Success in Psyc 100

English 826 139 2% 31%

English 836 / 846 730 9% 48%

English 100 / 105 1,209 15% 68%

English 110 530 7% 75%

Not in English Course 5,207 67% 64%

Total 7,815 100% 63%

Note: Enrollments from Summer 2000 to Spring 2009; Success is defined as A/B/C/CR grade



Psychology 100 Success of Students Simultaneously Enrolled in Math


Simultaneously Enrolled Math Course

Count in Psyc 100

% of Psyc 100 Enr.

Success in Psyc 100

Math 811 157 2% 43%

Math 110 / 111 / 112 970 12% 51%

Math 120 / 122 /123 773 10% 63%

Transfer-Level Math 997 13% 76%

Not in Math Course 4,918 63% 64%

Total 7,815 100% 63%



Success in 5 Highly Enrolled GE Courses by English Enrollment Level



Success in Psyc 100

Success in SPCH 100

Success in ECON 100

Success in HIST 201

Success in HSCI130

English 826 31% 54% 27% 48% 60%

English 836 / 846 48% 57% 56% 43% 74%

English 100 / 105 68% 73% 73% 51% 82%

English 110 75% 82% 79% 64% 88%

Not in English Course 64% 74% 69% 59% 73%

Total 63% 73% 69% 58% 75%

Total Enrollments 7,815 5,793 5,643 4,603 4,475



Success in 5 More Highly Enrolled GE Courses by English Enrollment Level



Success in FITN 334

Success in ACTG 100

Success in MUS 202

Success in BIOL 250

Success in SOCI 100

English 826 48% 37% 31% N/A 40%

English 836 / 846 58% 50% 64% 41% 50%

English 100 / 105 68% 66% 74% 60% 59%

English 110 76% 73% 80% 65% 74%

Not in English Course 66% 69% 70% 70% 69%

Total 66% 68% 71% 68% 68%




Success in 5 Highly Enrolled GE Courses by Math Enrollment Level



Success in PSYC 100

Success in SPCH 100

Success in ECON 100

Success in HIST 201

Success in HSCI 130

Math 811 43% 54% 26% 38% 49%

Math 110 / 111 / 112 51% 63% 46% 48% 71%

Math 120 / 122 /123 63% 67% 62% 51% 79%

Transfer-Level Math 76% 78% 80% 63% 86%

Not in Math Course 64% 75% 69% 59% 74%

Total 63% 73% 69% 58% 75%




Success in 5 More Highly Enrolled GE Courses by Math Enrollment Level



Success in FITN 334

Success in ACTG 100

Success in MUS 202

Success in BIOL 250

Success in SOCI 100

Math 811 59% 43% 50% N/A N/A

Math 110 / 111 / 112 61% 46% 58% 51% 46%

Math 120 / 122 /123 68% 55% 73% 54% 55%

Transfer-Level Math 77% 77% 82% 69% 75%

Not in Math Course 66% 70% 70% 70% 69%

Total 66% 68% 71% 68% 68%




Section 4: Cohort Tracking in Basic Skills Sequences



Cohort Tracking

Cohorts tracked include students whose first math course at Skyline was between 2002-03 and 2006-07. Students were tracked for three academic years.

Students were tracked based on the first math/English/ESL course that they took at Skyline (not on their placement).



Math Cohort Tracking – Starting in Fundamentals

Starting Level Att. Fund

Pass Fund

Att. Beg.

Pass Beg.

Att. Inter.

Pass Inter.

Att.Tran.

Pass Tran.

Fundamentals

1,863

1,215 939 624 331 212 134 99

% of starting cohort

100% 50% 18% 7%

% from previous

step--- 65% 77% 66% 53% 64% 63% 74%

*Note: Fundamentals = Math 811, Beginning Algebra = 110 or 111+112, Intermediate Algebra = 120 or 122+123, Transfer = 130,

150, 200-level

11%33% 5%65%



Math Cohort Tracking – Starting in Beginning Algebra

Starting Level Att. Beg.

Pass Beg.

Att. Inter.

Pass Inter.

Att.Tran.

Pass Tran.

Beginning Algebra

3,449

2,306

1,363 934 577 440


100% 40% 27% 17%

% from previous

step--- 67% 59% 69% 62% 76%

*Note: Beginning Algebra = 110 or 111+112, Intermediate Algebra = 120 or 122+123, Transfer =

130, 150, 200-level

27% 13%67%



Math Cohort Tracking – Starting in Intermediate Algebra

Starting Level Att. Inter.

Pass Inter.

Att.Tran.

Pass Tran.

Intermediate Algebra

2,086

1,494 891 647


100% 43%

% from previous

step--- 72% 60% 73%

*Note: Intermediate Algebra = 120 or 122+123, Transfer = 130, 150, 200-level

31%72%



English Tracking – Starting in Two Levels Below Transfer

Starting LevelAtt. Two

Below

Pass Two

Below

Att. One

Below

Pass One

Below

Att. Trans.

A

Pass Trans.

A

Att. Trans.

B

Pass Trans.

B

Two Levels Below

Transfer

2,328

1,566

1,179 924 742 591 440 362


100% 51% 32% 19%

% from previous

step--- 67% 75% 78% 80% 80% 74% 72%

*Note: Two Levels Below = 826, One Level Below = 836 or 846, Transfer A = English 100 or 105, Transfer B = 110 or 165

25%40% 16%67%



English Tracking – Starting in One Level Below Transfer

Starting LevelAtt. One

Below

Pass One

Below

Att. Trans.

A

Pass Trans.

A

Att. Trans.

B

Pass Trans.

B

One Level Below

Transfer3,017 2,246 1,666 1,391 1,038 879

% of starting cohort 100% 55% 27% 34%

% from previous step --- 74% 74% 83% 75% 85%

*Note: One Level Below = 836 or 846, Transfer A = English 100 or 105, Transfer B = 110 or 165

46% 29%74%



English Tracking – Starting in Transfer A

Starting LevelAtt.

Trans. A

Pass Trans.

A

Att. Trans.

B

Pass Trans.

B

Transfer A 3,320 2,615 1,641 1,406

% of starting cohort 100% 49%

% from previous step --- 79% 63% 86%

*Note: Transfer A = English 100 or 105, Transfer B = 110 or 165

42%79%



ESL Tracking – Starting in Pre Intermediate

Starting Level

Att. Pre

Inter.

Pass Pre

Inter.

Att. Inter.

Pass Inter.

Att. High Inter.

Pass High Inter/

Att. Trans

.

Pass Trans

.

Pre Intermediate 259 177 123 91 78 56 44 32


100% 68% 47% 35% 30% 22% 17% 12%

% from previous

step--- 68% 69% 74% 86% 72% 79% 73%

*Note: Pre Intermediate = 820 / 821-822 / 852-862-872; Intermediate = 830 / 831-832 / 853-863-873; High Intermediate = 840 / 841-842;

Transfer = 400 or English 100/105

22%35% 11%68%



ESL Tracking –Starting in Intermediate

Starting Level

Att. Inter.

Pass Inter.

Att. High Inter.

Pass High Inter/

Att. Trans.

Pass Trans.

Intermediate 361 250 177 133 99 79

% of starting cohort 100% 69% 49% 37% 27% 22%

% from previous step --- 69% 71% 75% 74% 80%

*Note: Intermediate = 830 / 831-832 / 853-863-873; High Intermediate = 840 / 841-842; Transfer = 400 or

English 100/105

37% 22%69%



ESL Tracking – Starting in High Intermediate

Starting LevelAtt.

High Inter.

Pass High Inter/

Att. Trans.

Pass Trans.

High Intermediate 366 243 158 123

% of starting cohort 100% 66% 43% 34%

% from previous step --- 66% 65% 78%

*Note: High Intermediate = 840 / 841-842; Transfer = 400 or English 100/105

34%66%



Something to consider…

25%40% 16%67%

46% 29%74%

42%79%

English Progression

34%

37% 22%69%

22%35% 11%68%

ESL Progression

66%11%33% 5%65%

27% 13%67%

31%72%

Math Progression



25%40% 16%67%

46% 29%74%

42%79%

English Progression

34%

37% 22%69%

22%35% 11%68%

ESL Progression

66%11%33% 5%65%

27% 13%67%

31%72%

Math Progression


25%40% 16%67%

46% 29%74%

42%79%

English Progression

34%

37% 22%69%

22%35% 11%68%

ESL Progression

66%11%33% 5%65%

27% 13%67%

31%72%

Math Progression



25%40% 16%67%

46% 29%74%

42%79%

English Progression

34%

37% 22%69%

22%35% 11%68%

ESL Progression

66%11%33% 5%65%

27% 13%67%

31%72%

Math Progression




Rob Johnstone, Project [email protected]

Priyadarshini Chaplot, Project [email protected]

Lead Technical CoachesDarla CooperJanet FulksBob GabrinerCathy HassonKen MeehanBob PachecoGreg StoupIan Walton

BRIC TAP Core Team

mailto:[email protected]

mailto:[email protected]



Find Out More!RP Group Websitewww.rpgroup.org

BRIC TAPhttp://www.rpgroup.org/projects/BRIC.html

Rob Johnstone, BRIC Project [email protected]

Kathy Booth, RP Group Executive [email protected]

Priya Chaplot, BRIC Project [email protected]

three studies to drive campus conversations on student success

Documents