1 owlts online world language testing software pittsburgh public schools prismatic consulting llc
TRANSCRIPT
1
OWLTSOnline World Language Testing Software
Pittsburgh Public Schools
Prismatic Consulting LLC
2
Why create an online language assessment tool?
There can be no standards without assessments to measure student progress.
3
History
Used CAL’s, SOPI tape mediated test Created own tape mediated test Tape administration was difficult and
cumbersome Obtained FLAP grant to develop test software OWLeTS current version 2.5
– Working on version 3.0
4
OWLTS
“Authorable” Software Key Features
o User friendly student testo User friendly test administrationo Easy for teachers to rate / scoreo Improved rating reliabilityo Data aggregationo User friendly task / test development
5
User Friendly Student Test
Easy to understand student user interface Practice tests to increase student comfort
level Accommodations for special needs Students are more comfortable with computer
than tape recorder
6
User Friendly Test Administration - District
Flexibility in test size/grouping - Small or large groups from single student to large scale district wide testing
This year 1800 students tested – With audio tape only able to test 900 per year
Testing at widely distributed multiple sites – Tested at 36 sites this year
No distribution of testing equipment such as tape recorders, tapes etc.
7
User Friendly Test Administration - Teacher
Testing manual for teacher use Practice test increases teacher comfort Can test any language
– ESL– Non-traditional languages
Can test different languages at same time in one room
8
Easy for Teachers to Rate / Score
Teachers can rate from anywhere Teachers can do 1 rating in a sitting or 100 Can listening to same sample repeatedly
without fast forwarding or rewinding Easy to re-rate speech sample (second or
third rating) Teachers can easily provide rating / score
and feedback comments
9
Improved Rating Reliability
To follow ACTFL/OPI protocol OWLeTS can be configured to prevent teachers from rating own students
Has been validated with SOPI and OPI Easy to compare ratings between raters – Inter-rater
reliability checks Provides means to give feedback to teachers on
ratings Enhanced training – Can pull speech samples and
ratings for comparison in training sessions
10
Data Aggregation
Tied to student information system Easy to create new reports on collected data Data collection allows analysis of student
ratings by:– class, section, school, district, language, program,
race, socio-economic, etc.
Can compare data from ratings by raters – rater reliability
11
2003 – 2004 ComparisonY
ea
r
Te
ste
d
No
vic
e L
ow
No
vic
e M
id
No
vic
e H
igh
Inte
rme
ida
te L
ow
Inte
rme
dia
te M
id
Inte
rme
dia
te H
igh
No
-Ra
tin
g
%N
ov
ic L
ow
% N
ov
ice
Mid
% N
ov
ice
Hig
h
% In
term
eid
ate
Lo
w
% In
term
ed
iate
Mid
% In
term
ed
iate
Hig
h
% N
o-R
ati
ng
2004 1798 512 390 307 262 57 32 238 28% 22% 17% 15% 3% 2% 13%
2003 1335 310 276 226 228 0 3 292 23% 21% 17% 17% 0% 0% 22%
Difference 463 202 114 81 34 57 29 -54 5% 1% 0% -3% 3% 2% -9%
12
2003 – 2004 Comparison
28%
22%
17%15%
3%2%
13%
23%21%
17% 17%
0% 0%
22%
0%
5%
10%
15%
20%
25%
30%
%Novic Low % Novice Mid % Novice High % IntermeidateLow
% IntermediateMid
% IntermediateHigh
% No-Rating
2004
2003
13
User Friendly Task Creation - Authorable
Easy to create and edit tasks Records and plays multiple audio formats
– WAV– MP3
Accepts any image file– Clip art – Photos (jpg, bmp)– Movies (mpeg, wma)
14
User Friendly Test Creation - Authorable
Easy to create and edit tests including length Tests can be created for any
– Level -Novice, Intermediate, Advanced and Superior– Age - 5th grade, university admission, teacher candidates– Language – traditional, non-traditional, ESL
Can change test from day to day or hour to hour Tests can be created in any structure
– This test is structured like CAL (Center for Applied Linguistics) / SOPI– Can be structured for ESL - TESOL (Teaching of English to Speakers of
other Languages)– Other
15
Future – Version 3.0
Each teacher will have own task / test bank Students will have online practice from home Addition of writing and reading assessment Additional test item types; cloze, multiple
choice, short answer, etc. Improved student user interface .NET
16
Thank You