1 using ala-score software to score essays and concept maps wednesday june 22, 2005 dr. roy clariana...
TRANSCRIPT
![Page 1: 1 Using ALA-Score software to score essays and concept maps Wednesday June 22, 2005 Dr. Roy Clariana Penn State University email: RClariana@psu.eduRClariana@psu.edu](https://reader036.vdocuments.us/reader036/viewer/2022062411/56649ef35503460f94c068c0/html5/thumbnails/1.jpg)
1
Using ALA-Score software to score essays and concept maps
Wednesday June 22, 2005
Dr. Roy Clariana
Penn State University
email: [email protected]
Web: www.personal.psu.edu/rbc4
"First we build the tools, then they build us!" -- Marshall McLuhan
![Page 2: 1 Using ALA-Score software to score essays and concept maps Wednesday June 22, 2005 Dr. Roy Clariana Penn State University email: RClariana@psu.eduRClariana@psu.edu](https://reader036.vdocuments.us/reader036/viewer/2022062411/56649ef35503460f94c068c0/html5/thumbnails/2.jpg)
2
goals Your take away:
• Some understanding of how/why it works
• Some ideas that you could implement on Monday morning in your classroom (or in you research)
• Hands-on using ALA-Score software to score student essays
• Hands-on using CMAP tools concept map software
Quick interest survey (teacher, researcher, grade level, subject, writing?, concept maps?)
![Page 3: 1 Using ALA-Score software to score essays and concept maps Wednesday June 22, 2005 Dr. Roy Clariana Penn State University email: RClariana@psu.eduRClariana@psu.edu](https://reader036.vdocuments.us/reader036/viewer/2022062411/56649ef35503460f94c068c0/html5/thumbnails/3.jpg)
3
Roy
H.S. math and science teacher (4 years) E.S. technology teacher (5 years) Ed. Consultant in CO, WY, UT (5 years) College professor at Penn State (8
years) www.personal.psu.edu/rbc4 The grand kids live in Denver and we
visit our cabin in summers here
![Page 4: 1 Using ALA-Score software to score essays and concept maps Wednesday June 22, 2005 Dr. Roy Clariana Penn State University email: RClariana@psu.eduRClariana@psu.edu](https://reader036.vdocuments.us/reader036/viewer/2022062411/56649ef35503460f94c068c0/html5/thumbnails/4.jpg)
4
Important note
Concept maps and essays are proven powerful generative instructional approaches!
But the main focus of this presentation is assessment rather than instruction, both ongoing in class (formative) and at the end of units of instruction (summative)
![Page 5: 1 Using ALA-Score software to score essays and concept maps Wednesday June 22, 2005 Dr. Roy Clariana Penn State University email: RClariana@psu.eduRClariana@psu.edu](https://reader036.vdocuments.us/reader036/viewer/2022062411/56649ef35503460f94c068c0/html5/thumbnails/5.jpg)
5
The “structure” of knowledge
Most tests measure declarative knowledge (e.g., knowing who, what when, where, why and how, etc.)
But essays and concept maps can measure the organization or structure of knowledge
essaysinterviews
tests
lungsoxygenate
blood
CO2artery
pulmonary
atriumventricle
veinlungs
oxygenate
blood
CO2artery
pulmonary
atriumventricle
vein
lungs
oxygenateblood
removeCO2
pulmonaryvein
leftatrium
lungs
oxygenateblood
removeCO2
pulmonaryvein
leftatrium
observations
![Page 6: 1 Using ALA-Score software to score essays and concept maps Wednesday June 22, 2005 Dr. Roy Clariana Penn State University email: RClariana@psu.eduRClariana@psu.edu](https://reader036.vdocuments.us/reader036/viewer/2022062411/56649ef35503460f94c068c0/html5/thumbnails/6.jpg)
6
Eliciting structural knowledge
Vygotsky (in Luria, 1979); Miller (1969) card-sorting approaches
Deese’s (1965) ideas on the structure of association in language and thought
Kintsch and Landauer’s ideas on representing text structure, and latent semantic analysis
Recent neural network representations (e.g., Elman, 1995)
![Page 7: 1 Using ALA-Score software to score essays and concept maps Wednesday June 22, 2005 Dr. Roy Clariana Penn State University email: RClariana@psu.eduRClariana@psu.edu](https://reader036.vdocuments.us/reader036/viewer/2022062411/56649ef35503460f94c068c0/html5/thumbnails/7.jpg)
7
Deese, free recall data
moth? – ___________
yellow? – ___________
bug? – ___________
Deese, J. (1965). The structure of associations in language and thought. Baltimore, MD: John Hopkins Press, page 56
![Page 8: 1 Using ALA-Score software to score essays and concept maps Wednesday June 22, 2005 Dr. Roy Clariana Penn State University email: RClariana@psu.eduRClariana@psu.edu](https://reader036.vdocuments.us/reader036/viewer/2022062411/56649ef35503460f94c068c0/html5/thumbnails/8.jpg)
8
Deese, free recall data (p.56)m
oth
inse
ct
win
g
bird
fly yello
w
flow
er
bug
coco
on
colo
r
blue
bees
sum
mer
suns
hine
gard
en
sky
natu
re
sprin
g
butt
erfly
moth 100 12 12 12 11 1 0 4 11 0 0 2 2 5 1 1 1 1 15insect 12 100 9 9 17 1 1 33 10 1 1 3 0 0 0 0 1 0 12wing 12 9 100 44 19 0 0 3 2 0 0 10 0 0 0 0 3 0 13bird 12 9 44 100 21 1 0 3 2 1 1 10 0 1 0 1 5 0 12fly 11 17 19 21 100 1 1 8 6 1 2 6 0 3 0 2 4 0 11yellow 1 1 0 1 1 100 7 0 0 17 23 2 2 7 5 2 4 3 5flower 0 1 0 0 1 7 100 2 0 3 7 2 1 6 18 2 6 2 6bug 4 33 3 3 8 0 2 100 7 0 0 5 0 0 0 0 2 0 4cocoon 11 10 2 2 6 0 0 7 100 0 0 4 1 1 1 0 2 0 22color 0 1 0 1 1 17 3 0 0 100 32 0 0 2 0 8 0 0 0blue 0 1 0 1 2 23 7 0 0 32 100 1 2 4 4 46 3 2 2bees 2 3 10 10 6 2 2 5 4 0 1 100 1 2 3 0 4 2 7summer 2 0 0 0 0 2 1 0 1 0 2 1 100 5 2 0 1 10 0sunshine 5 0 0 1 3 7 6 0 1 2 4 2 5 100 2 3 2 15 4garden 1 0 0 0 0 5 18 0 1 0 4 3 2 2 100 0 4 4 2sky 1 0 0 1 2 2 2 0 0 8 46 0 0 3 0 100 0 1 0nature 1 1 3 5 4 4 6 2 2 0 3 4 1 2 4 0 100 2 3spring 1 0 0 0 0 3 2 0 0 0 2 2 10 15 4 1 2 100 2butterfly 15 12 13 12 11 5 6 4 22 0 2 7 0 4 2 0 3 2 100
Deese, J. (1965). The structure of associations in language and thought. Baltimore, MD: John Hopkins Press, page 56
![Page 9: 1 Using ALA-Score software to score essays and concept maps Wednesday June 22, 2005 Dr. Roy Clariana Penn State University email: RClariana@psu.eduRClariana@psu.edu](https://reader036.vdocuments.us/reader036/viewer/2022062411/56649ef35503460f94c068c0/html5/thumbnails/9.jpg)
9-2 -1 0 1
Dimension 1
-1,5
-1,0
-0,5
0,0
0,5
1,0
1,5
Dim
ensi
on
2
moth
insect
wing
birdflyyellow
flower
bug
cocoon
colorblue
bees
summer
sunshinegarden
sky
nature
spring
butterfl
Euclidean distance model
Derived Stimulus ConfigurationSimple visual representation from Multi-Dimensional Scaling of this Deese data
Novices will not have a developed knowledge structure
![Page 10: 1 Using ALA-Score software to score essays and concept maps Wednesday June 22, 2005 Dr. Roy Clariana Penn State University email: RClariana@psu.eduRClariana@psu.edu](https://reader036.vdocuments.us/reader036/viewer/2022062411/56649ef35503460f94c068c0/html5/thumbnails/10.jpg)
10
concept map example
Bahr (2004) using concept maps to teach English to German students
![Page 11: 1 Using ALA-Score software to score essays and concept maps Wednesday June 22, 2005 Dr. Roy Clariana Penn State University email: RClariana@psu.eduRClariana@psu.edu](https://reader036.vdocuments.us/reader036/viewer/2022062411/56649ef35503460f94c068c0/html5/thumbnails/11.jpg)
11
Side trip Wordnet: http://wordnet.princeton.edu/
http://wordnet.princeton.edu/cgi-bin/webwn
What is the Visual Thesaurus? – The Visual Thesaurus offers a unique visual display of the English language. Looking up a word creates an interactive visual map with your word in the center of the display, connected to related words and meanings.
For example, type “bird” in at: http://www.visualthesaurus.com/trialover.jsp
![Page 12: 1 Using ALA-Score software to score essays and concept maps Wednesday June 22, 2005 Dr. Roy Clariana Penn State University email: RClariana@psu.eduRClariana@psu.edu](https://reader036.vdocuments.us/reader036/viewer/2022062411/56649ef35503460f94c068c0/html5/thumbnails/12.jpg)
12
Tools to support concept mapping
Yellow stickies!! Pencil and paper may be best for your classroom
Software – PowerPoint is pretty good Inspiration is good but expensive CMAP tool is free, but your tech person
will have to agree to support it At least 22 other tools are available,
some free some not (see next slide)
![Page 13: 1 Using ALA-Score software to score essays and concept maps Wednesday June 22, 2005 Dr. Roy Clariana Penn State University email: RClariana@psu.eduRClariana@psu.edu](https://reader036.vdocuments.us/reader036/viewer/2022062411/56649ef35503460f94c068c0/html5/thumbnails/13.jpg)
13
Additional concept map automatic scoring approaches
CMap tools (IHMC) that we will use today C-TOOLS – Luckie (PI), University of Michigan NSF
grant available: http://ctools.msu.edu/ctools/index.html TPL-KATS – University of Central Florida (e.g., Hoeft,
Jentsch, Harper, Evans, Bowers, & Salas, 1990). TPL-KATS: concept map: a computerized knowledge assessment tool. Computers in Human Behavior, 19 (6), 653-657.
SEMNET – http://www.semanticresearch.com/about/ CMAT – Arneson & Lagowski, University of Texas,
http://chemed.cm.utexas.edu Plus 22 other non-scoring map tools
![Page 14: 1 Using ALA-Score software to score essays and concept maps Wednesday June 22, 2005 Dr. Roy Clariana Penn State University email: RClariana@psu.eduRClariana@psu.edu](https://reader036.vdocuments.us/reader036/viewer/2022062411/56649ef35503460f94c068c0/html5/thumbnails/14.jpg)
14
Some classroom uses of concept mapping
Usually involve individuals working alone, and involve in some way reading or writing text
Some collaborative strategies have been used
Lets look at a few to give you classroom ideas…
![Page 15: 1 Using ALA-Score software to score essays and concept maps Wednesday June 22, 2005 Dr. Roy Clariana Penn State University email: RClariana@psu.eduRClariana@psu.edu](https://reader036.vdocuments.us/reader036/viewer/2022062411/56649ef35503460f94c068c0/html5/thumbnails/15.jpg)
15
Using a student concept map to “capture” a text (i.e., text summary, note taking)
Textbook
Text text text text text text text text text text text
text
texttext
concept map notes
student
text
memo
As Homework?
Textbook
Text text text text text text text text text text text
Textbook
Text text text text text text text text text text text
Summary
Text text text text text text text text text text text
![Page 16: 1 Using ALA-Score software to score essays and concept maps Wednesday June 22, 2005 Dr. Roy Clariana Penn State University email: RClariana@psu.eduRClariana@psu.edu](https://reader036.vdocuments.us/reader036/viewer/2022062411/56649ef35503460f94c068c0/html5/thumbnails/16.jpg)
16
Using a concept map in place of an outline in the writing process
essay
Text text text text text text text text text text text
text
texttext
concept map notes
student
text
memo
Examples?
![Page 17: 1 Using ALA-Score software to score essays and concept maps Wednesday June 22, 2005 Dr. Roy Clariana Penn State University email: RClariana@psu.eduRClariana@psu.edu](https://reader036.vdocuments.us/reader036/viewer/2022062411/56649ef35503460f94c068c0/html5/thumbnails/17.jpg)
17
Using a student concept map to capture and organize research on a topic
textText text
text text tex Text text text text
textttext
texttext
concept map notes
student
text
memo
textText text
text text tex Text text text text
textt
www
video
Examples?
video
Report
Text text text text text text text text text text text
![Page 18: 1 Using ALA-Score software to score essays and concept maps Wednesday June 22, 2005 Dr. Roy Clariana Penn State University email: RClariana@psu.eduRClariana@psu.edu](https://reader036.vdocuments.us/reader036/viewer/2022062411/56649ef35503460f94c068c0/html5/thumbnails/18.jpg)
18
Using a team concept map to capture and organize research on a topic
textText text
text text tex Text text text text
textttext
texttext
concept map notes
team
text
memo
textText text
text text tex Text text text text
textt
www
video
Examples?
video
Report
Text text text text text text text text text text text
![Page 19: 1 Using ALA-Score software to score essays and concept maps Wednesday June 22, 2005 Dr. Roy Clariana Penn State University email: RClariana@psu.eduRClariana@psu.edu](https://reader036.vdocuments.us/reader036/viewer/2022062411/56649ef35503460f94c068c0/html5/thumbnails/19.jpg)
19
Example of dyad collaboration
text
texttext
text
text
texttext
text
Yergin
concept map artefact
Verbal discussion (taped)
Analyze the discussion
Blah blah blah blah Blah blah
Hannah
Blah blah blah blah Blah blah
Observations:On taskAbstract talk3-propositions/minQuestionAnswerCriticizeConflictElaborationCo-construction
van Boxtel, van der Linden, Roelofs, & Erkens (2002)
Problem: Sometimes unscientific notions are ingrained
Inferred:Active use of prior knowledgeAcknowledged problemsLook for meaningful relationsNegotiation
Shared objects play an important role in negotiation and co-construction
The incredible value of talk!
Note the attentional effects of the artifact
![Page 20: 1 Using ALA-Score software to score essays and concept maps Wednesday June 22, 2005 Dr. Roy Clariana Penn State University email: RClariana@psu.eduRClariana@psu.edu](https://reader036.vdocuments.us/reader036/viewer/2022062411/56649ef35503460f94c068c0/html5/thumbnails/20.jpg)
20
summary
Maps into text Text into maps There is an intimate relationship between
the two and both can provide measures of structural knowledge
Have you considered concept maps
![Page 21: 1 Using ALA-Score software to score essays and concept maps Wednesday June 22, 2005 Dr. Roy Clariana Penn State University email: RClariana@psu.eduRClariana@psu.edu](https://reader036.vdocuments.us/reader036/viewer/2022062411/56649ef35503460f94c068c0/html5/thumbnails/21.jpg)
21
ALA-Reader
… an electrical signal starts the heartbeat, by causing the atrium to contract. The blood then flows through the pulmonary valve into the pulmonary artery and then into the lungs. Once inside the lungs, the blood gives up the carbon dioxide (cleansed) and receives oxygen. This oxygenated blood …
atrium
contract
lungs
cleansed oxygenated
P artery
P valve
Text CMAP
Link array
![Page 22: 1 Using ALA-Score software to score essays and concept maps Wednesday June 22, 2005 Dr. Roy Clariana Penn State University email: RClariana@psu.eduRClariana@psu.edu](https://reader036.vdocuments.us/reader036/viewer/2022062411/56649ef35503460f94c068c0/html5/thumbnails/22.jpg)
22
Clariana & Koul
Participants – A group of 24 practicing teachers enrolled in CI 400
Lesson – while researching the topic “the structure and function of the heart” online, students completed concept maps using Inspiration software and later wrote an essay on this topic from their maps.
Clariana, R.B., & Koul, R. (2004). A computer-based approach for translating text into concept map-like representations. In A.J.Canas, J.D.Novak, and F.M.Gonzales, Eds., Concept maps: theory, methodology, technology, vol. 2, in the Proceedings of the First International Conference on Concept Mapping, Pamplona, Spain, Sep 14-17, pp.131-134. http://cmc.ihmc.us/papers/cmc2004-045.pdf
![Page 23: 1 Using ALA-Score software to score essays and concept maps Wednesday June 22, 2005 Dr. Roy Clariana Penn State University email: RClariana@psu.eduRClariana@psu.edu](https://reader036.vdocuments.us/reader036/viewer/2022062411/56649ef35503460f94c068c0/html5/thumbnails/23.jpg)
23
Posttests
Essays Multiple-raters using holistic rubric Computer-derived LSA Essay scores (
http://www.personal.psu.edu/rbc4/frame.htm)
Concept Maps Multiple-raters using Lomask’s rubric ALA-Mapper PFNet link and distance agreement
with an expert ALA-Reader PFNet link scores (from 1 to 5)
(so far, only looked at essay scores)
![Page 24: 1 Using ALA-Score software to score essays and concept maps Wednesday June 22, 2005 Dr. Roy Clariana Penn State University email: RClariana@psu.eduRClariana@psu.edu](https://reader036.vdocuments.us/reader036/viewer/2022062411/56649ef35503460f94c068c0/html5/thumbnails/24.jpg)
24
ALA-Rater PFNet scores
The scores for each text and rater-pair are shown ordered from best to worst.
ALA-Reader scores were moderately related to the combined text score, Pearson r = 0.69, and ranked 5th overall.
ALA
-Rea
der
![Page 25: 1 Using ALA-Score software to score essays and concept maps Wednesday June 22, 2005 Dr. Roy Clariana Penn State University email: RClariana@psu.eduRClariana@psu.edu](https://reader036.vdocuments.us/reader036/viewer/2022062411/56649ef35503460f94c068c0/html5/thumbnails/25.jpg)
25
other tools to score Essays
ETS – PEG (Project Essay Grade), e-rater, Criterion and other products… http://www.ets.org/research/erater.html
Walter Kintsch (and Landau) at CU-Boulder – Latent semantic analysis, many uses, i.e., score online training for the Army - http://lsa.colorado.edu/
Vantage Learning essay scoring products - http://www.vantagelearning.com/
ALA-Reader: http://www.personal.psu.edu/rbc4/score.htm
![Page 26: 1 Using ALA-Score software to score essays and concept maps Wednesday June 22, 2005 Dr. Roy Clariana Penn State University email: RClariana@psu.eduRClariana@psu.edu](https://reader036.vdocuments.us/reader036/viewer/2022062411/56649ef35503460f94c068c0/html5/thumbnails/26.jpg)
26
other tools to score Essays
http://Knowledge-Technologies.com http://www.ets.org/research/erater.html http://ericae.net/betsy/ http://torrseal.mit.edu/effedtech/ (the MIT physics
homework tutor) http://knowledge-technologies.com/hrw.html (Preparation
for Standardized Writing Tests) http://knowledge-technologies.com/KeysDemo/
Keys2.1D.html (Prentice Hall – Using the IEA to improve textbook learning)
http://knowledge-technologies.com/tradoc.html (U.S. Army TRADOC – Automated Essay Scoring for Officer Leadership training)
![Page 27: 1 Using ALA-Score software to score essays and concept maps Wednesday June 22, 2005 Dr. Roy Clariana Penn State University email: RClariana@psu.eduRClariana@psu.edu](https://reader036.vdocuments.us/reader036/viewer/2022062411/56649ef35503460f94c068c0/html5/thumbnails/27.jpg)
27
Comments and Questions
About foundation of the idea and its use Any ideas on how you have or will use
maps?
Next: demo ALA-Reader
![Page 28: 1 Using ALA-Score software to score essays and concept maps Wednesday June 22, 2005 Dr. Roy Clariana Penn State University email: RClariana@psu.eduRClariana@psu.edu](https://reader036.vdocuments.us/reader036/viewer/2022062411/56649ef35503460f94c068c0/html5/thumbnails/28.jpg)
28
Demo ALA-Reader Download ALA-Reader.exe from
www.personal.psu.edu/rbc4 Create terms file (can include 2 synonyms) Create 2 expert baseline reference texts
called expert1.txt and expert2.txt (i.e., Instructor, best student)
Use it Files created
• Summary file called report.txt• Multiple *.prx files (PRX folder)• CMAP files
![Page 29: 1 Using ALA-Score software to score essays and concept maps Wednesday June 22, 2005 Dr. Roy Clariana Penn State University email: RClariana@psu.eduRClariana@psu.edu](https://reader036.vdocuments.us/reader036/viewer/2022062411/56649ef35503460f94c068c0/html5/thumbnails/29.jpg)
29
DEMO of CMAP Tools
Lets walk through it together, follow me step by step
![Page 30: 1 Using ALA-Score software to score essays and concept maps Wednesday June 22, 2005 Dr. Roy Clariana Penn State University email: RClariana@psu.eduRClariana@psu.edu](https://reader036.vdocuments.us/reader036/viewer/2022062411/56649ef35503460f94c068c0/html5/thumbnails/30.jpg)
30
Example of an online collaboration
p.22, Chiu, Huang, & Chang (2000)
text
texttext
text
text
texttext
text
Hannah(lead)
Jari
Yergin
H: WE should …J: Did you see…Y: Yeah, but …Etc.Etc.
concept map artefact
Online chat
Analyzed the chat textAnd the concept map
creates
concept map session lasted 80 minutes. 3 x 12 online groups, communicate by chat, 745 messages were exchanged (avg. of 62 per group).
Only the lead could alter the concept map
The ‘other 2 members used chat to “advise”
Researchers
![Page 31: 1 Using ALA-Score software to score essays and concept maps Wednesday June 22, 2005 Dr. Roy Clariana Penn State University email: RClariana@psu.eduRClariana@psu.edu](https://reader036.vdocuments.us/reader036/viewer/2022062411/56649ef35503460f94c068c0/html5/thumbnails/31.jpg)
31
Demonstration – Cmap tools synschronous collaboration
(see the Project handout)
![Page 32: 1 Using ALA-Score software to score essays and concept maps Wednesday June 22, 2005 Dr. Roy Clariana Penn State University email: RClariana@psu.eduRClariana@psu.edu](https://reader036.vdocuments.us/reader036/viewer/2022062411/56649ef35503460f94c068c0/html5/thumbnails/32.jpg)
32
Brainstorm, then make the map
Open IHMC Cmap tools Fill in personal information on first use (I’ll tell
you what to type in here) Click Other Places Open brainstorm file Click collaborate icon
if necessary Type in your first name Collaborate
![Page 33: 1 Using ALA-Score software to score essays and concept maps Wednesday June 22, 2005 Dr. Roy Clariana Penn State University email: RClariana@psu.eduRClariana@psu.edu](https://reader036.vdocuments.us/reader036/viewer/2022062411/56649ef35503460f94c068c0/html5/thumbnails/33.jpg)
33
Demo 1
IHMC Public Cmaps conv v2 on Jan 22 2004
![Page 34: 1 Using ALA-Score software to score essays and concept maps Wednesday June 22, 2005 Dr. Roy Clariana Penn State University email: RClariana@psu.eduRClariana@psu.edu](https://reader036.vdocuments.us/reader036/viewer/2022062411/56649ef35503460f94c068c0/html5/thumbnails/34.jpg)
34
Oulu EDTECH Public
Demo 1
![Page 35: 1 Using ALA-Score software to score essays and concept maps Wednesday June 22, 2005 Dr. Roy Clariana Penn State University email: RClariana@psu.eduRClariana@psu.edu](https://reader036.vdocuments.us/reader036/viewer/2022062411/56649ef35503460f94c068c0/html5/thumbnails/35.jpg)
35
Final Questions
Time to play with the software
![Page 36: 1 Using ALA-Score software to score essays and concept maps Wednesday June 22, 2005 Dr. Roy Clariana Penn State University email: RClariana@psu.eduRClariana@psu.edu](https://reader036.vdocuments.us/reader036/viewer/2022062411/56649ef35503460f94c068c0/html5/thumbnails/36.jpg)
36
My research interests
Mind map assessment – automatic scoring software tool called ALA-Mapper http://www.personal.psu.edu/rbc4/ala.htm
Essay assessment – automatic scoring software tool called ALA-Reader http://www.personal.psu.edu/rbc4/score.htm
for Latent Semantic Analysis (LSA) see: http://www.personal.psu.edu/rbc4/frame.htm
prototypes
![Page 37: 1 Using ALA-Score software to score essays and concept maps Wednesday June 22, 2005 Dr. Roy Clariana Penn State University email: RClariana@psu.eduRClariana@psu.edu](https://reader036.vdocuments.us/reader036/viewer/2022062411/56649ef35503460f94c068c0/html5/thumbnails/37.jpg)
37
Poindexter and Clariana
Participants – 23 undergraduate students in intro EdPsyc course (Penn State Erie)
Food rewards for participation Setup – complete a demographic survey
and how to make a concept map lesson Text based lesson interventions –
instructional text on the “heart” with either proposition specific or relational lesson approach
Poindexter, M. T., & Clariana, R. B. (in press). The influence of relational and proposition-specific processing on structural knowledge and traditional learning outcomes. International Journal of Instructional Media, 33 (2), in press. link to doc file
#1st
![Page 38: 1 Using ALA-Score software to score essays and concept maps Wednesday June 22, 2005 Dr. Roy Clariana Penn State University email: RClariana@psu.eduRClariana@psu.edu](https://reader036.vdocuments.us/reader036/viewer/2022062411/56649ef35503460f94c068c0/html5/thumbnails/38.jpg)
38
Treatments Relational condition, participants were required to
“unscramble” sentences (following Einstein, McDaniel, Bowers, & Stevens, 1984) in one paragraph in each of the five sections or about 20% of the total text content
Proposition-specific condition (following Hamilton, 1985), participants answered three or four adjunct constructed response questions (taken nearly verbatim from the text) provided at the end of each of the five sections, for a total of 17 questions covering about 20% of the total text content (no feedback was provided).
![Page 39: 1 Using ALA-Score software to score essays and concept maps Wednesday June 22, 2005 Dr. Roy Clariana Penn State University email: RClariana@psu.eduRClariana@psu.edu](https://reader036.vdocuments.us/reader036/viewer/2022062411/56649ef35503460f94c068c0/html5/thumbnails/39.jpg)
39
Posttests
Concept map (use 26 terms provided)• Link-based common scores
• Distance-based common scores
Multiple-choice tests (Dwyer, 1976)• Identification (20)
• Terminology (20)
• Comprehension (20)
![Page 40: 1 Using ALA-Score software to score essays and concept maps Wednesday June 22, 2005 Dr. Roy Clariana Penn State University email: RClariana@psu.eduRClariana@psu.edu](https://reader036.vdocuments.us/reader036/viewer/2022062411/56649ef35503460f94c068c0/html5/thumbnails/40.jpg)
40
Means and sd
Treatments Posttests ID TERM COMP Map-prop Map-assoc control 15.1 12.3 7.3 14.1 9.0
(4.4) (4.6) (5.4) (4.6) (3.6)
proposition- 16.3 14.6 13.8 16.5 11.5 specific
(5.6) (5.7) (3.7) (8.3) (3.4)
relational 17.0 12.7 12.4 13.9 10.7 (2.6) (3.5) (3.0) (9.4) (4.6)
Map-link Map-dist
![Page 41: 1 Using ALA-Score software to score essays and concept maps Wednesday June 22, 2005 Dr. Roy Clariana Penn State University email: RClariana@psu.eduRClariana@psu.edu](https://reader036.vdocuments.us/reader036/viewer/2022062411/56649ef35503460f94c068c0/html5/thumbnails/41.jpg)
41
Analysis
MANOVA (relational, proposition-specific, and control) and five dependent variables including ID, TERM, COMP, Map-prop, and Map-assoc.
COMP was significance, F = 5.25, MSe = 17.836, p = 0.015, none of the other dependent variables were significance.
Follow-up Scheffé tests revealed that the proposition-specific group’s COMP mean was significantly greater than the control group’s COMP mean (see previous Table).
![Page 42: 1 Using ALA-Score software to score essays and concept maps Wednesday June 22, 2005 Dr. Roy Clariana Penn State University email: RClariana@psu.eduRClariana@psu.edu](https://reader036.vdocuments.us/reader036/viewer/2022062411/56649ef35503460f94c068c0/html5/thumbnails/42.jpg)
42
Correlations
ID TERM COMP Prop ID -- TERM 0.71 -- COMP 0.50 0.74 -- Map-prop 0.56 0.77 0.53 -- Map-assoc 0.45 0.69 0.71 0.73 All sig. at p<.05
Compare to Taricani & Clariana
next
Map-link
Map-linkMap-distance
![Page 43: 1 Using ALA-Score software to score essays and concept maps Wednesday June 22, 2005 Dr. Roy Clariana Penn State University email: RClariana@psu.eduRClariana@psu.edu](https://reader036.vdocuments.us/reader036/viewer/2022062411/56649ef35503460f94c068c0/html5/thumbnails/43.jpg)
43
Taricani and Clariana – Replication of Poindexter and Clariana
Taricani, E. M. & Clariana, R. B. (in press). A technique for automatically scoring open-ended concept maps. Educational Technology Research and Development, 53 (4), in press.
TermComp
Link data 0.78 0.54
Distance data 0.48 0.61
![Page 44: 1 Using ALA-Score software to score essays and concept maps Wednesday June 22, 2005 Dr. Roy Clariana Penn State University email: RClariana@psu.eduRClariana@psu.edu](https://reader036.vdocuments.us/reader036/viewer/2022062411/56649ef35503460f94c068c0/html5/thumbnails/44.jpg)
44
Compare these two . . .
Poindexter & Clariana TermComp
Link data 0.77 0.53
Distance data 0.69 0.71
Taricani & Clariana TermComp
Link data 0.78 0.54
Distance data 0.48 0.61
![Page 45: 1 Using ALA-Score software to score essays and concept maps Wednesday June 22, 2005 Dr. Roy Clariana Penn State University email: RClariana@psu.eduRClariana@psu.edu](https://reader036.vdocuments.us/reader036/viewer/2022062411/56649ef35503460f94c068c0/html5/thumbnails/45.jpg)
45
Clariana, Koul, & Salehi
Participants – A group of 24 practicing teachers enrolled in CI 400
Lesson intervention – while researching online, completed concept maps in pairs (newsprint & yellow stickies) to describe the structure and function of the heart and then individually wrote essays on this topic from their maps.
Clariana, R. B., Koul, R., & Salehi, R. (in press). The criterion related validity of a computer-based approach for scoring concept maps. International Journal of Instructional Media, 33 (3), in press.
# 2nd
![Page 46: 1 Using ALA-Score software to score essays and concept maps Wednesday June 22, 2005 Dr. Roy Clariana Penn State University email: RClariana@psu.eduRClariana@psu.edu](https://reader036.vdocuments.us/reader036/viewer/2022062411/56649ef35503460f94c068c0/html5/thumbnails/46.jpg)
46
Posttests
Essays Multiple-raters using holistic rubric Computer-derived LSA Essay scores (
http://www.personal.psu.edu/rbc4/frame.htm)
Concept Maps Multiple-raters using Lomask’s rubric ALA-Mapper PFNet link and distance
agreement with an expert
![Page 47: 1 Using ALA-Score software to score essays and concept maps Wednesday June 22, 2005 Dr. Roy Clariana Penn State University email: RClariana@psu.eduRClariana@psu.edu](https://reader036.vdocuments.us/reader036/viewer/2022062411/56649ef35503460f94c068c0/html5/thumbnails/47.jpg)
47
Correlation matrix
Map Essay LSA LinkMap 1 Essay 0.49 1 LSA 0.31 0.73 1 Link data 0.36 0.76 0.83 1 Distance data 0.60 0.77 0.71 0.82 1
p < .05 shown in boldface type.
Human Computer
Many investigators have noted the close relationship between maps and essays.