Slide 1

0 Automated Formative Assessment: Providing Linguistic Support through Online Modules Presented by: Ken Petersen 11/18/2011 ASEES 42nd Annual Convention Washington, DC

Slide 2

1 What is Formative Assessment? Formative assessment allows a teacher to capture learning as it occurs, and to make appropriate instructional adjustments. It is distinct from summative assessment in several aspects: o Frequency o Goals Stakes Stakeholders o Outcomes What effect does assessment have on the learners linguistic development and/or language career path? o Shelf-life of the data

Slide 3

2 Online Formative Assessment The state of the art for the critical languages is less than optimal. o First generation Web-based CALL applications provide little data on learner performance and live in strict isolation from one another. Russnet, Langnet, GLOSS, Arabic Without Walls, CenAsiaNet, etc. o Development of good materials is time and labor intensive. Russnet has been providing learners and teachers the opportunity to use online pedagogical materials for formative assessment for over a decade. o Reading, listening and writing activities that capture: Scores Student mistakes Student essays Teacher feedback o Data collected to date: Activities completed: 321,368 Mistakes: 566,077 Student Essays: 26,302 Writing Comments: 2,735

Slide 4

3 Moving Forward Opportunities for online formative assessment are growing at a staggering rate. o Access to authentic language content (text, audio and video) o Online communication provides immediacy in feedback o Natural Language Processing (NLP) tools and resources are moving online NLP Tools Part of speech taggers Morphological analyzers Syntactic parsers NLP resources Electronic dictionaries Bi-text corpora o Web-services are allowing data and functionality to be shared between applications.

Slide 5

4 Generating Dynamic Activities Together with the Center for Research in Computational Linguistics, American Councils is developing a suite of online tools for generating on the fly support for Russian lexis and morphology. The technology can embed instructional support into Russian texts from a broad array of contexts. The learner performance data derived from these activities could be used to provide pedagogically meaningful feedback to learners and teachers. An initial prototype is the Russian News Reader.

Slide 6

5 Russian News Reader (RNR): Search

Slide 7

6 RNR: Search Result

Slide 8

7 RNR: Lexical Support

Slide 9

8 RNR: Passive Morphological Support Highlight: Verbs

Slide 10

9 RNR: Passive Morphological Support Fill-in the Blanks: Adjectives

Slide 11

10 RNR: Passive Morphological Support Multiple-choice: Adjectives

Slide 12

11 Dynamic Activities & Formative Assessment The challenge at hand is to effectively model the data that we might capture from learner-text interactions. We have robust knowledge of lexis and morphology: o Part of speech o Gender o Number o Case We need to create a model that will provide informative and actionable evidence for formative assessment. There need to be intuitive graphical representations of the data. Khan Academy provides a nice framework for this.

Slide 13

12 Contact Ken Petersen American Councils for International Education 1828 L Street NW, Suite 1200 Washington, DC kpeter@actr.org You can download this presentation at: http://lab.americancouncils.org/pres/AFA.pptx