columbia college english assessment/placement update

Columbia College English Assessment/Placement Update

Preliminary Data Summary

Fall 2008 – Spring 2011

Per AB3 and good practice, it is the responsibility of the College to monitor and review the accuracy of

its placements (cut score ranges) for placing students into coursework. Presumably, the more accurately

a student is placed, the greater chance he or she will have of being successful in the course. Thus, it is

also the responsibility of the College to make considered changes to those cut score ranges when there is

evidence that there is a need to do so. That is commonly called “cut score validation.”

The last cut score validation project for Columbia College English course placement was conducted and

reported in the 2008 “Accuplacer College Placement Validation Project.” This study also included an

“English Placement Consequential Validity” review whereby students and faculty evaluated the accuracy

of the placements via performance in the course. At that time, the cut scores were deemed reasonable

given the students’ performance and the feedback received from faculty. This report begins where the

2008 report ended analyzing the student placement and grade outcome data collected from Fall 2008 to

Spring 2011.

Background

Columbia College requires new students, or students who have been on an extended hiatus from college

coursework, to complete a placement exam in order to enroll in the appropriate English course for their

skill level. Testing is completed prior to the student’s orientation and advisement and before they register

in courses. The Datatel IT registration system blocks students from enrolling above their assessed course

level unless there is a manual override entered by a professional counselor or administrator who deems a

higher placement is appropriate for the student (multiple measures). Students may, however, self-select

to enroll in any English course that is below their assessed level.

Columbia uses the College Board’s online “Accuplacer” assessment system. For placement into English

courses, Accuplacer offers two test modules—sentence or writing skills, and reading. The College places

students into English via a score received on the sentence skills module only. Though the reading skills

module is available and students may choose to test in it, it is not required. Both test modules, sentence

and reading, have score ranges from between 0-120.

There are three levels of English courses: English 650 which is considered entry level or basic skills,

English 151 which is one level below transfer level English, and English 1A. The levels are determined

by the College and input into the Accuplacer system via a detailed set of decision rules. These decision

rules include several “background” questions that are weighted and add or subtract small increments to

the students total placement score. The background questions and decision rules can be updated or

altered by the College at any time. (Note: Though the background questions are currently in use, they

have never been statistically analyzed separately from the total scores.)

Data are taken from the Accuplacer test printouts and input into the district Datatel student database.

They are then extracted for analysis via the TSUM tables from a YCCD Crystal report. Likewise,

semester grades are obtained and downloaded through the Datatel system via the YCCD “Student

Demographic Detail” report from the same system. These term-driven downloads are then combined,

refined to isolate only the English scores and courses, and matched to each other via student IDs. When

the files are ready, they are then imported into the SPSS statistical application and Excel for analysis.

This study includes extracted data from Fall 2008 to the last term that had been graded and reported to

the California State Chancellor’s Office MIS system, Spring 2011.

We begin with summaries taken from the initial downloaded file contents, and then proceed through the

aggregating process to the final filtered and SPSS-ready numbers. They are included here as a means to

understand the data refinement process and to explain why the final Ns appear to be much lower than

would be expected given the number of terms included in the report.

Placement Distributions: Fall 2008 - Spring 2011

All Columbia College Sentence and Reading Skills TSUM test records,

July 15, 2008 to February 15, 2011:

All English Scores via the Accuplacer Assessment Battery

Sentence Skills Reading

N Valid 2301 1220

Mean 83.03 78.65

Median 84.00 81.00

Mode 84.04 81.35

Std. Deviation 20.274 20.903

Variance 411.022 436.954

Range 99 113

Percentiles 25 69 65

50 84 81

75 98 94

Included in the TSUM records were students’ retest data and as expected, average higher.

The following were removed:

All English Retests via the Accuplacer Assessment Battery

Sentence Skills Reading

N Valid 67 28

Mean 92.21 82.93

Median 93.00 85.00

Mode 93.33 84.00

Std. Deviation 16.052 18.599

Variance 257.653 345.921

Range 68 70

Percentiles 25 82 73

50 93 85

75 104 98

General Average Placement Levels1

Columbia College

Accuplacer Tests Taken

Fall 2008 – Spring 2011

Sentence

Skills

Reading

Skills Total

Test Score Mean: 83.03 78.65 81.51

Average Accuplacer Placement: English 151 English 151 Below

Transfer

Accuplacer Tests Completed: 2,301 1,220 3,521

Note: Average mean test scores did not contain records annotated as “retest,” previous course completion

advancements, and/or placements derived through outside institutions’ placement reports.

The next step was to further refine the data to include only those students who tested and enrolled in the

course of placement. After eliminating the retests, placements accepted from outside institutions, etc.,

the data were matched by the closest course completion date to the date of the Sentence Skills placement.

By matching to test and academic years, duplicate course completion records were reduced as much as

possible. (Many students had grades from two and three English courses during the study period.) This

reduced the data set to from 2,301 to 1,070. (See: Appendices, Table A1.)

1 Refer to the “Columbia College Placement Test Score Interpretation”, 2011-12 Columbia College Catalog.

Data were filtered again to include only the three English courses listed on the Columbia College

“Placement Test Score Interpretation” guide. (See: Footnote 1 and Appendices, Table A2.) The records

that remained numbered 712. (See: Appendices, Table A3.)

Because there were reading test scores in the TSUM data, the decision was made to conduct an similar

evaluation and a like process was conducted in matching and filtering for them. The result was 408

records which was sufficient for statistical analysis.

Cut Score Analysis – English

Because the primary Accuplacer instrument used for English placement is the Sentence Skills the first

analysis begins with the filtered 842 records derived from the original 4,301 records found in the TSUM.

English course grades included those from English 649, 650, 249, 250, 151, and 1A.

The Research Assumption:

Given an assessment through the Accuplacer Sentence Skills placement instrument and a review and

interview by a professional, the College would place students into appropriate levels of English

coursework for their skill levels. Students would then be more “likely” to succeed in those courses given

no external or uncontrolled circumstances, e.g., illness, life events, etc., that might interfere with their

performance.

An Overview of All Average English Grades Received (4pt_Grade Scale)

Sorted by the Accuplacer Writing Placement Received (the primary module used for English placement)

Fall 2008- Spring 2011

Note: The following data are highlighted to reflect Accuplacer Sentence Skills tests taken during 2008-

2011 where students enrolled in, and completed courses at the placement levels.

Accuplacer

Test Taken

CC Course

Completed:

English Placement Levels

ENGL-650 ENGL-151 ENGL-1A Total

N Mean N Mean N Mean N Mean

CACC-SENT ENGL 249,

649, 250, 650 37 2.14 72 1.60 4 a 1.50 113 1.77

ENGL 50, 151 4 a 1.25 402 2.16 14 2.21 420 2.16

ENGL 1A 36 1.69 273 2.34 309 2.27

Total 41 2.05 510 2.05 291 2.32 842 2.14

CACC-READ ENGL 249,

649, 250, 650 21 1.71 21 2.29 1 a 2.00 43 2.00

ENGL 50, 151 23 2.48 235 2.29 31 2.65 289 2.34

ENGL 1A 97 2.55 142 2.63 239 2.59

Total 44 2.11 353 2.36 174 2.63 571 2.42 a Cells with < 10 records - provided as information only.

Data were filtered to include only those students who tested and enrolled in their placement course, and

who received a grade during the study period of Fall 2008 to Spring 2010.

Average English Grades Received (4pt_Grade Scale)

by Accuplacer Placement Levels and Courses Completed at Placement Level

Fall 2008- Spring 2011

Accuplacer Sentence Skills Placements

Descriptives of Sentence Placements and Grade Outcomes after Filtering (N: 712)

Sentence Skills Placement Level

Mean Grade Averages (based on 4 pt. grade scale) Statistic Std. Error

Placed into

ENGL 650 level and

received a grade in

ENGL 650 level course 2

N: 37

Mean 1.77 .119

95% Confidence Interval for Mean

Lower Bound 1.53

Upper Bound 2.00

5% Trimmed Mean 1.74

Median 2.00

Variance 1.589

Std. Deviation 1.261

Minimum 0

Maximum 4

Range 4

Interquartile Range 2

Skewness -.071 .227

Kurtosis -.738 .451

Placed into

ENGL 50, 151

and

received a grade

N: 402

Mean 2.16 .066

95% Confidence Interval for Mean Lower Bound 2.03

Upper Bound 2.29


Median 2.00

Variance 1.823


Minimum 0

Maximum 4

Range 4


Skewness -.352 .119

Kurtosis -1.019 .238

2 The “ENGL 650 level” includes the courses ENGL 249-250 and ENGL 649-650. (See: Appendices, Table A3.)

Descriptives of Sentence Placements and Grade Outcomes after Filtering (N: 712)

Placed into

ENGL 1A and

received a grade

N: 273

Mean 2.27 .086


Lower Bound 2.10

Upper Bound 2.43


Median 3.00

Variance 2.280


Minimum 0

Maximum 4

Range 4


Skewness -.441 .139


Correlations for Sentence Skills Assessment

To grades in

courses:

Predictive Correlations

Accuplacer Sentence Skills Score

Pearson’s Kendall’s Spearman’s

N R p Tau_b p Rho p

English 649-650 37 .633* .000 .532* .000 .656* .000

English 151 402 .095 .057 .080* .033 .104* .036

English 1A 273 .078 .201 .066 .148 ,089 .143

* Indicates a statistically significant correlation that is not due to chance error (p < .05).

Accuplacer Reading Placements

Descriptives for Reading Placements and Grade Outcomes after Filtering (N: 398)

Reading Placement Level

Mean Grade Averages (based on 4 pt. grade scale)

Statistic Std. Error

Placed into

ENGL 650 level and

received a grade in

ENGL 650 level

N: 21

Mean 1.71 .302


Lower Bound 1.08

Upper Bound 2.34


Median 2.00

Variance 1.914


Minimum 0

Maximum 4

Range 4


Skewness -.056 .501


Placed into

ENGL 151 level

and received a grade in

ENGL 151

N: 235

Mean 2.29 .088


Lower Bound 2.11

Upper Bound 2.46


Median 3.00

Variance 1.837


Minimum 0

Maximum 4

Range 4


Skewness -.478 .159

Kurtosis -.933 .316

Reading Placement Level

Mean Grade Averages (based on 4 pt. grade scale)

N: 398 Statistic Std. Error

(continued)

Placed into

ENGL 1A level

and received a grade

in ENGL 1A

N: 142

Mean 2.63 .122


Lower Bound 2.39

Upper Bound 2.87


Median 3.00

Variance 2.122


Minimum 0

Maximum 4

Range 4


Skewness -.806 .203

Kurtosis -.691 .404

Correlations for Reading Skills Assessment

To grades in

courses:

Predictive Correlations

Accuplacer Reading Score

Pearson’s Kendall’s Spearman’s

N R p Tau_b p Rho p

English 649-650 21 -.170 .460 -.155 .385 -.171 ,458

English 151 235 .136* .037 .110* .025 .145* ,027

English 1A 142 -.006 .944 .003 .962 -.006 .942

* Indicates a statistically significant correlation that is not due to chance error (p < .05).

Disproportionate Impact

Highlighted areas indicate percentages where placements are disproportionately higher in lower level

courses as compared to the overall and majority population. The rule of thumb: The percent distribution

of any student category should be within 20% of the majority group’s placements (80% of the placement

level). If the proportion is more than that, it has disproportional impact and requires that a plan to

monitor and address the situation be implemented and rectified if possible.

Note: We return to the earlier dataset of 1,070 students with an English placemen, and any English

course grade between the dates of July 15, 2008 and February 15, 2011.

Gender by Sentence Skills Placement Level

Placement Level

Gender Engl-650 Engl-151 Engl-1A Transfer Gender Total

Count % Count % Count % Count %

Female 18 3.0% 349 58.3% 232 38.7% 599 100.0%

Male 25 5.4% 220 47.4% 219 47.2% 464 100.0%

Unk/Unrpt 0 .0% 4 57.1% 3 42.9% 7 100.0%

Placement

Subtotal 43 4.0% 573 53.6% 454 42.4% 1,070 100.0%

Though no gender reached the threshold of 20% higher proportions in their placements compared to the

total, or would trigger a plan for action, it is noteworthy that a higher proportion of females placed in

English 151 than did males. Though the numbers of unknown/unreported are too small to determine

impact, they are included here to explain the missing 7 records and to bring the table’s total to 1,070

records.

Data Regarding Disproportionate Impact (continued)

Ethnicity by Sentence Skills Placement Level

Placement Level

Engl-650/250 Engl-151 Engl-1A Transfer Ethnicity Total

Ethnicity Count % Count % Count % Count %

1. Asian 0 .0% 6 66.7% 3 33.3% 9 100.0%

2. African Am. 1 7.7% 10 76.9% 2 15.4% 13 100.0%

3. Filipino 0 .0% 4 80.0% 1 20.0% 5 100.0%

4. Hispanic 6 5.5% 68 61.8% 36 32.7% 110 100.0%

5. Native Am. 3 14.3% 11 52.4% 7 33.3% 21 100.0%

6. Pacific Isl. 0 .0% 3 100.0% 0 .0% 3 100.0%

7. Other Non-White 1 3.8% 15 57.7% 10 38.5% 26 100.0%

8. White 23 3.1% 371 50.1% 346 46.8% 740 100.0%

9. Unk-Unrpt 9 6.3% 85 59.4% 49 34.3% 143 100.0%

Placement Total 43 4.0% 573 53.6% 454 42.4% 1,070 100.0%

The proportion of Hispanic students who placed into English 151 are noticeably higher than that of the

overall and the majority group.

While the numbers and proportions of African Americans, Native Americans, and Pacific Islanders had

larger proportions of students who placed into lower than transfer level coursework, the numbers

underlying those percentages are very small and therefore not highlighted. That does not mean that the

information is not worthy of notice or further exploration.

Disproportionate Impact (continued)

Age Levels by Sentence Skills Placement

Placement Level

Ages Engl-650 Engl-151 Engl-1A Age Group Total

Count % Count % Count % Count %

00-17 1 1.1% 42 45.7% 49 53.3% 92 100.0%

18-24 31 4.1% 407 54.1% 314 41.8% 752 100.0%

25-34 6 4.7% 71 55.0% 52 40.3% 129 100.0%

35-44 3 6.5% 20 43.5% 23 50.0% 46 100.0%

45-54 1 2.4% 29 70.7% 11 26.8% 41 100.0%

55-64 1 12.5% 3 37.5% 4 50.0% 8 100.0%

65-00 0 .0% 1 50.0% 1 50.0% 2 100.0%

Placement Subtotal 43 4.0% 573 53.6% 454 42.4% 1,070 100.0%

The students aged 24 and younger had the highest proportion of placements into transfer level English

1A compared to other age groups. The 45-54 year old group had the highest proportion overall of

students placing into English 151.

Primary Language by Sentence Skills Placement Levels

Placement Level

Engl-650 Engl-151 Engl-1A Language Total

Language Count % Count % Count % Count %

English 35 3.4% 552 53.5% 445 43.1% 1032 100.0%

Not English 8 28.6% 16 57.1% 4 14.3% 28 100.0%

Unk/Unrpt 0 .0% 5 50.0% 5 50.0% 10 100.0%

Placement Subtotal 43 4.0% 573 53.6% 455 42.5% 1,070 100.0%

With only 28 students indicating primary languages other than English, the percentages are questionable

as a true representation for this population; however, it may be worth noting that over half of the 28, or

57.1%, placed into English 151. This is contrary to the assumption that this group’s placements would be

lower than the English 151 level.

Appendices

Table A1. Descriptive Mean Score Statistics by Placement Level 3

Descriptives – English Placement Sentence Skills Mean Scores by Placement Level

N: 712 Placement Level Statistic Std. Error

MEAN SCORE

Engl-650 Mean 41.63 .905

95% Confidence

Interval for Mean

Lower Bound 39.80

Upper Bound 43.45


Median 42.00

Variance 35.192


Minimum 29

Maximum 50

Range 21


Skewness -.452 .361

Kurtosis -.464 .709

Engl-151 Mean 75.36 .452

95% Confidence

Interval for Mean

Lower Bound 74.47

Upper Bound 76.25


Median 77.00

Variance 117.140


Minimum 51

Maximum 91

Range 40


Skewness -.508 .102

3 Table A1 represents only average (mean) scores by placement level given a Sentence Skills placement test taken between

Fall 2008-Spring 2011. It is provided as a means to evaluation the average score within each placement level, and to highlight

any clusters with possible anomalies.

Descriptives – English Placement Sentence Skills Mean Scores by Placement Level

N: 712 Placement Level Statistic Std. Error

Kurtosis -.752 .204

Engl-1A Mean 104.44 .380

95% Confidence

Interval for Mean

Lower Bound 103.69

Upper Bound 105.19


Median 104.00

Variance 65.611


Minimum 92

Maximum 120

Range 28


Skewness .179 .115


Descriptives – English Placement Reading Skills Mean Scores by Placement Level Descriptives

Placement Level Statistic

Std.

Error

English 650 Mean 37.32 1.057

95% Confidence

Interval for Mean

Lower Bound 35.19

Upper Bound 39.45


Median 37.00

Variance 49.152


Minimum 27

Maximum 50

Range 23


Skewness .046 .357


English 151 Mean 75.47 .581

95% Confidence

Interval for Mean

Lower Bound 74.33

Upper Bound 76.62


Median 77.00

Variance 119.017


Minimum 51

Maximum 91

Range 40


Skewness -.461 .130

Kurtosis -.772 .259

English 1A Mean 101.86 .522

95% Confidence

Interval for Mean

Lower Bound 100.83

Upper Bound 102.89


Median 101.00

Variance 47.449


Minimum 92

Maximum 120

Range 28


Skewness .610 .184

Kurtosis -.223 .366

Table A2. Distribution of all English courses completed by placed students, 2008-2011.

YSTU_COURSE_NAME

Frequency Percent Valid Percent Cumulative Percent

Valid CENGL-125 5 .5 .5 3.3

CENGL-249 3 .3 .3 88.1

CENGL-250 8 .7 .7 88.9

CENGL-606 2 .2 .2 89.7

CENGL-649 68 6.4 6.4 96.1

CENGL-650 34 3.2 3.2 99.3

CENGL-47 6 .6 .6 89.4

CENGL-50 1 .1 .1 89.5

CENGL-151 419 39.2 39.2 42.4

CENGL-1A 309 28.9 28.9 71.6

CENGL-1B 142 13.3 13.3 84.9

CENGL-1C 32 3.0 3.0 87.9

CENGL-81 8 .7 .7 100.0

CENGL-10 19 1.8 1.8 1.8

CENGL-11 11 1.0 1.0 2.8

CENGL-17 1 .1 .1 42.5

CENGL-18 2 .2 .2 42.7

Total 1,070 100.0 100.0

Table A3. English courses completed by students with placement scores and enrollments in

placement courses during the study period of 2008-2011.

Distribution of Students Placements after Filtering

Sentence

Skills Percent

Reading

Skills Percent

Valid CENGL-249, 250, 649, 650 37 5.2 21 5.3

CENGL-151 402 56.5 235 59.0

CENGL-1A 273 38.3 142 35.7

Total 712 100.0 398 100.0

* N records for courses ENGL 249, 250, and 50 were insufficient for data analysis but are provided here

as a means to account for the student course completions given placements. Summaries were aggregated

to include these courses in the ENGL 650 placement level.

columbia college english assessment/placement update

Documents