program slo assessment for english as a second … · program slo assessment for english as a...
Post on 31-Aug-2018
214 Views
Preview:
TRANSCRIPT
Program SLO Assessment for English as a Second Language Upon completion of the credit ESL program, students will have adequate listening, speaking, reading, writing and academic skills to successfully complete transfer-level classes. PLANNING Assessment Describe the Planned Program SLO Assessment
[from the original Program-level SLO Proposal]: “Collect data from [English] 1AX classes as a sample of transfer-level courses, to determine the preparation students have had. Collect data on the retention and success [rates] in these 1AX classes. Compare student retention and success in 1AX (ESL) classes with the retention and success in 1A [non-ESL] classes.”
1. In Spring 2009, students in all sections of English 1AX will complete a survey eliciting information about their prior preparation for the course, including ESL and English classes, and classes taken through the El Camino Language Academy (ECLA). The purpose of the survey will be to identify ESL students and to track the progress of students who will go on to take IGETC critical thinking courses in Fall ’09:
English 1AX Survey
Spring, 2009
Instructor’s Name:__________________________________________ Section #:________
Your Student ID #:____________________
1. Is English your first language? Yes No
1. How long have you lived in an English-speaking country? _____years, _____months
1. 3. Which high school did you attend and where is it located?
________________________________________________________________________
1. 4. Please put a check mark (ü) next to all of the classes that you’ve completed OR that you are taking this semester at ECC:
ESL writing classes:
_____English AX
_____ESL 53B
_____ESL 53A
English writing classes:
_____English A
_____English B
ESL reading classes:
_____ESL 52C
_____ESL 52B
_____ESL 52A
ESL listening/speaking classes:
_____ESL 51C
_____ESL 51B
_____ESL 51A
English reading classes:
_____English 84 (formerly English 2R)
_____English 82 (formerly English R)
_____English 80
English grammar class:
ESL multi-skills class:
_____ESL 55
ECLA classes:
_____College Preparation—Level 3
_____College Preparation—Level 2
_____College Preparation—Level 1
_____English 4
Other ECC English classes (please list):
1. 5. Please list your other classes this semester:
Thank you for your participation.
2. Some of the survey data will be collected and entered into a spreadsheet. This spreadsheet will include information that is not currently available through the Institutional Research (IR) department (i.e. Survey Items 1-3 and ECLA courses taken before entering the ECC ESL program)
ESL Program-level SLO Survey
Excel Spreadsheet Items
Spring, 2009
Col:
Original
Survey Item:
Column Heading and Description of Item:
A
Course Section number / Student ID number
Section Number (bold); ECC Student ID Number
B
1. Is English your first language (LI)?
“Engl. is L2” =”English is my second language (L2)”
Y = Yes
C
D
2. How long have you lived in an English-speaking country?
“Years in Engl Spkg Country?” = “Number of years
living in an English-speaking country?”
“Months?” = “Number of months”
3. Which high school did you attend and where is it located?
E
F
“US HS” = “Did you attend High School in the U.S.?
Y = Yes
“HS Country” = “In which country is your
High School located?”
Refer to “ESL Program-level SLO: English 1AX
Survey: Country Codes” list
Attendance in ECC Language Academy (ECLA)
“Attend ECLA” = “Did you attend the ECC Language
G
Academy?”
Y = Yes
H
I
J
Which ECC Language Academy courses did you take?
“Did you take…”
“…Level 3?”
“…Level 2?”
“…Level 1?”
Y= Yes
3. Country codes will be developed by D. Mochidome for determining students’ first language background (based on the location of these students’ high schools):
ESL Program-level SLO
English 1AX Survey
Country Codes
Spring 2009
Code:
Country:
?
ALG
BG
BZ
CB
CHN
CHL
COL
No notation
Algeria
Bulgaria
Brazil
Cuba
China
Chile
Colombia
ESV
ETH
IDO
IND
IRN
JA
KO
ML
MX
NI
PRU
El Salvador
Ethiopia
Indonesia
India
Iran
Japan
South Korea
Malaysia
Mexico
Nigeria
Peru
SA
SNG
SRB
SRL
TH
TK
TW
VN
Saudi Arabia
Singapore
Serbia
Sri Lanka
Thailand
Turkey
Taiwan
Vietnam
4. The spreadsheet items will be tabulated separately from the information provided by IR. [See Attachment]
5. Subsequently, IR will track improvement and retention rates for Spring ‘09 English 1AX students enrolled in Fall ‘09 IGETC critical thinking courses (i. e. English 1C, Philosophy 5, and Psychology 3)
Standards or Rubric Describe the standards you will use to determine success in your assessment
• Success and retention rates for English 1AX courses
• Comparison of these success and retention rates with those of non-ESL English 1A courses
• Correlation, if any, between success in English 1AX and success in ESL 51ABC (listening/speaking) course series
• Improvement and retention rates for former English 1AX students in IGETC critical thinking courses taken the semester after completing English 1AX
Assessment Leaders Mochidome, Debra Uyemura, Evelyn Uyemura, Evelyn Cortez-Perez, Aurora Location and Semester Semester of Current Assessment: Campus of Assessment: El Camino College Semester of Next Planned Assessment: Spring 2013 Linking Course Assessments No Course SLOs have been matched to this Program SLO REPORTING Assessment Data Provide the survey or assessment data.
IR completed the first 4 of the 5 items in the Research Plan; Item 5 was not completed as IR felt it to be “rather nebulous.” It was suggested, however, that it would be possible to collect this data in the future if the department would like to have it.
Research Plan, Item 1:
With regard to their previous ESL courses (excluding ECLA courses), the average success and retention rates for the Spring 2009 English 1AX students surveyed were:
Average Success Rate for previous ESL Courses:
Average Retention Rate for previous ESL Courses:
Successful Spring 2009 English 1AX students
85.9%
92.5%
Unsuccessful Spring 2009 English 1AX students
51.9%
78.2%
Difference between Successful and Unsuccessful Students
34%
14.3%
These data indicate that:
• The majority of successful English 1AX students (85.9%) had also been successful in their previous ESL courses and, to a lesser extent, the converse was also true; unsuccessful English 1AX students (51.9%) tended to have been unsuccessful in their previous ESL courses. The difference between the success rates for successful and unsuccessful English 1AX students was 34%.
• The retention rate for successful English 1AX students in their previous ESL courses was 92% whereas the retention rate for unsuccessful English 1AX students in their previous ESL classes was 78.2%. The difference in these retention rates was 14.3%.
Item 2:
IR analyzed the Spring ‘09 success and retention rates of English 1AX classes and compared them with those of English 1A classes.
Because students must now register for classes online, and the “X” is not shown in the online version of the Schedule of Classes, some English 1AX students are native English
speakers (i.e. non-ESL students)
Also, because a clear separation was not made between the ESL students and the non-ESL students before the English 1AX Survey data was submitted to IR, the data IR provided below reflects all students, ESL and non-ESL, who were enrolled in English 1AX:
Spring 2009
Course:
No. of Students
(initially enrolled in all sections of course):
Success:
Retention:
English 1AX
161
65.8%
84.5%
English 1A
2140
58.5%
77.6%
Differences between English 1AX sections and other English 1A sections
7.3%
6.5%
The corresponding English 1AX and English 1A grade distributions were:
English 1AX Sections—Spring ’09:
Grade:
Count:
Success:
Retention:
A
43
65.8%
85.4%
B
48
C
15
D
9
DR
5
F
19
2
I
W
20
Total:
161
All Other English 1A Sections—Spring ’09:
Grade:
Count:
Success:
Retention:
A
385
58.5%
77.6%
B
476
C
391
D
125
DR
74
F
241
I
43
405
W
Total:
2140
Difference between English 1AX sections
and all other English 1A sections:
Success:
Retention:
7.3%
6.9%
• IR’s finding here was that students coming through the ESL program were considerably more successful (7.3%) than non-ESL students who attempted English 1A. In addition, the overall retention rate for English 1AX students was 6.5% higher than that of English 1A students
Item 3:
Eligibility for English 1AX requires credit in English AX and in ESL 52B (i.e. completion of the ESL program’s pre-transfer-level writing series and its intermediate-level reading course).
In an attempt to see whether completion of the 51 series (listening/speaking) courses had an impact on the success and retention of English 1AX students, the correlation between successful Fall ‘08 1AX students and their completion of 51 series courses was examined.
Of the 161 students enrolled in Spring ‘09 English 1AX sections, 110 students completed the English 1AX Survey, and of those students, 13 (11.82%) identified themselves as native-English speakers (i.e. non-ESL students). The remaining 97 students (88.18%) identified themselves as non-native English speakers (i.e. ESL students). These ESL students were educated in high schools located in 27 countries (including the U.S.). The following data were collected from the ESL students’ responses to the survey:
English-1AX Success and Retention Rates by Placement/Enrollment in ESL-51 Series
Course/Placement
Group
Count
% of Grp
Success
Retention
ESL-51A
Took
12
67%
58%
67%
Did not take
6
33%
67%
67%
Difference (% pts)
6
33
-8
0
ESL-51B
Took
27
43%
85%
89%
Did not take
36
57%
78%
89%
Difference (% pts)
-9
-14
7
0%
ESL-51C
Took
6
9%
83%
83%
Did not take
58
91%
88%
93%
Difference (% pts)
-52
-81
-5
-10
Overall
Took
45
31%
78%
89%
Did not take
100
69%
83%
95%
Difference (% pts)
-55
-38
-5
-6
Of the students who placed into ESL 51A, 66% enrolled in the course. Forty-three percent of students placed into 51B enrolled in 51B, and 9% of students who placed into 51C enrolled in 51C.
IR’s report indicated “mixed” results in terms of student success and retention:
• Students who had “…enrolled in ESL 51B were [7%] more likely to be successful” in English 1AX than “those who did not”
• However, “the outcome reverses” for those who enrolled in ESL 51A and 51C. The lower counts in these courses may be a factor here
• IR’s conclusion for this portion of the SLO assessment was, “While no clear conclusions can yet be drawn as to whether the oral series provides an added benefit to transfer-level writing, results from the 51B level are promising, given its larger sample size”
Item 4:
After the Fall ‘09 semester, data on students who took IGETC critical thinking courses (English 1C, Philosophy 5, and Psychology 3) were collected. The results for Fall ‘09 courses were:
Success and Retention in Critical Thinking Courses by English 1A Course Status
Course
Engl-1A Status
Total
Success
Retention
ENGL-1C
ESL
26
77%
81%
Not ESL
340
75%
89%
Difference (% pts)
2
-8
PHIL-5
ESL
*
100%
100%
Not ESL
11
64%
73%
Difference (% pts)
36
27
PSYC-3
ESL
*
50%
50%
Not ESL
55
58%
75%
Difference (% pts)
-8
-25
Overall
ESL
*
77%
80%
Not ESL
406
72%
86%
Difference (% pts)
4
-6
*Data suppressed due to small sample size.
Overall, the results of the IGETC course analysis were mixed.
• Twenty-six English 1AX students (roughly 26.8% of the original 97 ESL students surveyed) successfully completed English 1C, representing roughly 6.4% of the total number of all successful 1C students. These 26 students also represented 23.64% of the original 110 respondents to the Spring ‘09 survey.
• Although the number of successful ESL students was small, in comparison to the total number of successful 1C students, the ESL students’ improvement rate was 2% higher than that of their non-ESL peers. However, the retention rate of ESL students in 1C classes was 8% lower than that of their non-ESL peers.
Observable Patterns Summarize the patterns observed in the data.
• ESL students who had come through the ESL program were more successful in English 1AX than non-ESL students who attempted English 1A.
• Successful English 1AX students tended to have been successful in their previous ESL classes and to have high retention rates as well.
• There is a substantial number of non-ESL students who take English 1AX. It is possible that these students, who tend to be more likely to drop the class, may be affecting retention rates in English 1AX.
• While a correlation between students’ success in English 1A and these students’ having taken 51 series courses is inconclusive, nevertheless, students who had taken 51B were 7% more successful than those who had not.
• In addition, Jenny Simon noted that the higher students placed into 51 series courses, the higher their success rate was in English 1AX. Specifically, students who placed into ESL 51C had an average 1AX success rate of 86%, students who placed into 51B had an
average 1AX success rate of 81.5%, and students who placed into 51A had an average 1AX success rate of 62.5%.
• Regarding IGETC courses, English 1AX ESL students overwhelmingly tended to take English 1C, rather than Phil. 5 or Psych. 3. The success rate for these 1C students was slightly higher than that of their non-ESL peers. On the other hand, the retention rate for ESL students in English 1C was notably lower than that of their non-ESL peers.
Implications and Future Directions What are the implications of the data? How should the institution as a whole or related programs act on the data?
• Make eligibility for ESL 51C “recommended preparation” for English 1AX
• Make sure that English 1AX instructors are using the rubric for English 1A to score students’ essays. The same goes for English AX instructors’ using the English A rubric for essay scoring
• Schedule essay-norming sessions for all ESL writing courses (including English !AX) for instructors teaching these courses. The purpose of the norming to ensure that grading standards are consistent department-wide
• Schedule norming sessions for ESL 52B and 52C where instructors can evaluate students’ written summaries of their reading assignments in order to ascertain the extent of these students’ reading comprehension.
Changes to the SLO statement: None
Changes to the assessment:
• Meet with Irene Graff (IR) to discuss measurement and assessment options before designing and implementing the assessment
• Include data from Compton Center’s non-credit ESL program to include its role in preparing students for the ECC ESL program, and omit ECLA data; in this report, only 4 student respondents to the survey (3.6%) indicated that they had taken ECLA courses prior to taking English 1AX
Changes to the rubric:
• A number of ESL students take English 1B after completing English 1AX, so instead of examining improvement and retention rates for IGETC critical thinking courses per se, examine improvement and retention rates for English 1C and English 1B. Omit improvement and retention rates for Philosophy 5 and Psychology 3. since so few students in this assessment’s survey sample took these courses
• Take into account the educational goals of non-university-bound students by adding a survey item such as “Educational goals: Transfer to a 4-year university? Associate’s degree? Certification? Other?”
• Include success and retention rates for students who go through the ESL program and who take English 1A (in addition to the success and retention rates for students who go through the ESL program and who take English 1AX)
• Include the results of this SLO assessment in the next SLO assessment so that the results of the next assessment can build on the results of this one. In this way, factors (e.g. progress/increases in success rates, etc.) can be tracked from this point (June, 2010) on into the future
Changes to the method:
If we choose to survey students again:
• Use a scan-able survey form to expedite data collection. IR has already developed a preliminary version of this type of survey.
• Be sure to clearly separate the non-ESL students from the ESL students before sending data to IR
• Elicit more specific information related to students’ first language backgrounds (vs. Survey Item 3. “Which high school did you attend, and where is it located?”). For example, this item can be reworded as: “Country?”; “Number of years speaking English?”; “Language spoken at home?”; etc.
• Develop a more “user-friendly,” possibly numerically-based, method for coding students’ home countries, if this data is germane to the next cycle’s assessment
• Omit Survey Item 5 “Please list your other classes this semester:”
If we choose NOT to survey students:
• To expedite the data-gathering process, get enrollment, success, retention, and improvement data from IR instead of gathering these data via a survey.
• Have IR track English 1AX students over a longer period (perhaps as long as 10 years) in order to reinforce the preliminary correlation between the successful completion of 51 series courses and/or the initial placement of students into 51 series courses, and success in English 1AX.
• Request that IR track success and improvement rates for English 1AX students who have successfully completed ESL 52C prior to taking 1AX
Additional Attached Files No Additional Files have been attached
top related