HyLighter Interactive Annotation

System

HyLighter Interactive Annotation

System

Shine a New Light on Teaching and Learning

FETC 2005 - Orlando, FL

Shine a New Light on Teaching and Learning

FETC 2005 - Orlando, FL

HyLighter PresentersDale W. Lick, Ph.D.University Professor, Florida State University

David G. Lebow, Ph.D.President, HyLighter, Inc.

Hope Hartman, Ph.D.Professor of Educational Psychology, City College of

New York & the City University of New York Graduate Center

Edie LeBas, M.A.T.English Instructional Team Leader, University of Miami

Online High School

on-line annotation programs spreading across the digital

landscape

Many types of annotation systems largely reflect the many purposes for reading itself.

Thousands of websites offer some form of annotation capability for public/social functions, collaboration, and education.

HyLighter supports learning, assessment, and knowledge-

building1. Makes thinking that is ordinarily

hidden, become transparent and available for self-reflection and feedback.

2. Allows users to continuously compare their developing understanding to others, monitor progress, and assess performance.

3. Provides new, technology-enhanced ways to organize and synthesize ideas from multiple sources and create meaning from multiple perspectives.

The HyLighter Difference:

draws broadly on the learning sciences (e.g., expert performance, cognitive load theory, self-regulated learning)

merges electronic markings on digital text and other "information objects" made by any number of readers

generates composite displays showing convergence and divergence of highlighting and comments

highlight a section of text

add a comment

view a comment

click the Compare button

view “map” of group highlighting

view merged highlighting and comments: the "collective annotation model"

The Interactive Annotation Spreadsheet

Task Perspective Elements

Interactive Annotation for Argument & Persuasion

1. Introductory training sequence.2. Participants read and annotate material.3. Participants compare views and generate

group views.4. Instructor performs assessment

activities, leads whole class discussion, models active reading strategies, provides feedback to students, and modifies instruction as necessary.

HyLighter and Metacognitive Teaching

Includes awareness of the sources and characteristics of students' misconceptions, selection of instructional strategies to overcome students' misconceptions, and monitoring/ evaluating progress.

HyLighter Interactive Annotation System

Manager

Malleable to meet needs of different user groups

Integration with other technologies

Assessment instrumentation for complex cognitive skills

Research across the curriculum and training contexts

HyLighter Use by CCNY Grad Students in Ed Psych Courses Fall 2004

Overview• Two different types of applications of HyLighter: Case

Studies & Explanations of Answers on Multiple Choice Tests.

• Students were working on their masters degrees in teaching.

• Approximately half of the students were born abroad.

Assignments in Preparation for Use of HyLighter• Text-based annotations preceded online annotations.• Online case annotations preceded multiple choice

explanations.

Data Sources1. Case Studies of Beginning Teachers2. Class Discussions3. Multiple Choice Test Answer Explanations 4. End of Multiple Choice Test Comments5. Essays on HyLighter

Case Study Results

1. Many students commented on how using HyLighter made them more active readers & that the acts of annotating & comparing annotations increased meaningfulness & memorability of what they read.

2. A few students didn’t like the highlighting but liked writing comments.

3. Students thought about what they were highlighting for the first time.

4. Students had thought that any and all highlighting was of equal value and there were no right or wrong ways to highlight - it was just a matter of style or personal preferences.

5. One student didn’t highlight anything in the case because she viewed everything in it as equally important. She and others didn’t realize it was important to be selective.

Case Study Results cont.

6. Teachers’ and selected students’ annotations can be used as models to scaffold students’ development of effective highlighting skills.

7. Some students were relieved that they could highlight a section of text and didn’t have to make a comment despite the appearance of the dialogue box.

8. Students liked comparing their highlights & comments with other students & the teacher so they could see if they missed any important information and if they included irrelevant details.

9. One student failed to use HyLighter when she heard there was a bug in the program because she thought that meant you would get a virus by going there (from Sweden).

Case Study Results cont.

10. Several students complained about the small size of the comment box making it hard to monitor your own comment writing.

11. Students persisted and enjoyed the HyLighter experience despite initial problems using it because of browser or other problems.

12. I identified and corrected students’ misconceptions through analyzing & responding to comments in the case studies.

13. Students were fascinated by the differences between their highlights and comments

14. Small group discussions about using HyLighter and students’ background influences were extremely rich and enlightening, especially because about half of the class were born abroad.

•Item/Credit None Partial Full#8 29 57 14#12 75 25#14 37.5 12.5 50#15 10 80 10#17 29 71#19 25 75

Mean 30.08 45.08 24.83

% Credit for Explanations for Selected Wrong Answers

Summary of Comments Re: HyLighter at the End of Multiple Choice

Test

Positive

I thought this worked very well. I liked the fact that it made one think in depth about the answers & try to justify them.

It allowed me to see gaps in my thinking, and knowledge, and I think it would have been nice to have had this opportunity on the first test.

I like this a lot! I would definitely be excited to do something like this with my students.

Negative

I wasn't exactly sure to what extent we were to explain ourselves and that might have been helpful to know.

One criticism is that the comment box is too small and very fiddly. If you lose your train of thought, you then have to mess around until you regain it

The test took much longer than I had expected. I wonder if an essay section is even necessary after all this writing.

Explanations of Multiple Choice Selections

FREQUENT CATEGORIES 1. Explanation/Description2. Elaboration3. Examples4. Problem with

Terminology5. Part/Whole6. Misconception7. Thought Report8. Personal Opinion9. Misapplication10. Problematic Item

FINDINGS

1. Students had more knowledge about content targeted in test items than their wrong answers implied, showing flaws in test items or their interpretation.

2. 90% of students in course benefited from analyzing their response choice explanations by receiving at least partial credit for some wrong answers.

3. Almost 50% of these students got at least partial credit for ALL wrong answers in the set of 6 items analyzed.

Essays on HyLighter Across Three Graduate Courses

Impact of Using HyLighter on Attitudes about Technology

Impact of Using HyLighter on Thoughts about Technology • Cognitive and Metacognitive Benefits

for Students• Highlighting as a Reading Strategy• Teaching• Assessment• Technology

Past, Present and Future of asynchronous discussions

in an online school AP Lit class

Students live in different cities in different time zones so asynchronous discussions are their only ways to discuss literature with each other.

Focused and interactive discussions are an integral part of the Advanced Placement Literature classroom to develop students’

understanding of texts and literary elements.

Pedagogical Challenges

Students have productive discussions about characters and plot development in novels, modern plays, and short stories.

However, students lack focus in responding to discussion questions about a specific passage in a difficult text or in analyzing stylistic features of a text.

In addition, students voice difficulty with answering multiple-choice questions about poetry.

Past: Apex Stranded Discussion Forum

Sample “Cry for Help” in a “Stranded” Discussion Post

about the Romantic Poets

To be honest, I did not really enjoy this module. I like reading poems because it is easy to come up with your own interpretation of it. However, when it comes to answering the questions on the quiz, I get most of them wrong because I view things differently. It's only after I see the answer that I understand it completely. I just want to know your opinion on reading these poems and how well you do when taking the quizzes.

Before students begin study of Twelfth Night by Shakespeare

I decide to ask the HyLighter people for help because of AP Lit students’ frustrations and difficulties with multiple-choice quizzes on Romantic poets,

I receive approval from Apex Learning to use their multiple-choice quizzes on the HyLighter site.

How students use the four multiple-choice quizzes in HyLighter

• Students highlight their first choices and submit a “thought report” about how they decided on their answers

• Students email teacher their first choices for feedback as to whether right or wrong. Teacher responds quickly.

• Students are allowed to select a second choice for each incorrect answer as long as they provide a thought report.

•Students compare their thought reports with others and then respond to questions about their choices and their reports. Students make suggestions for improvement.

Sample Student Thought Reports for First Quiz

• I think that the answer is 7A. Viola is referring to Olivia's physical complexion. Viola is comparing apples to apples. The only other choice that made sense to me was Choice D, but it compares two unlike things, apples and oranges, so i went with the choice A.

• For this question I struggled to pick between A and D. I eliminated D as my choice though because I think the statement was talking more about her physical beauty and not how she acted. For that reason it was logical to pick A because Viola is complimenting Olivia, saying that even if she was as ugly as the devil, she would still be beautiful.

Thought Report for Third Quiz: Identifying a speaker of a passage

Malvolia is thrilled to find out that his mistress has declared her love for him. Olivia supposedly sends him a letter (in reality, the letter is written by Maria) and in the letter she tells him that she has fallen in love and that she wishes him to wear the yellow stockings. To get to this conclusion, I disregarded answers B, D, and E. It is not B because Malvolio is simply Olivia's steward. It is not D because Maria is always finding ways to amuse herself by making fun of Malvolio, so there is no reason why he would want to get Maria's attention. The answer is not D because Malvolio puts on the yellow stockings before they put him in jail. Finally the answer is not A because Toby and Andrew are always making fun of Malvolio and not the other way around. The answer is C.

Discussion Post after Third Quiz

Note below the change in thinking pattern of an “apple and oranges” student

I didn't miss any questions the first time, but by re-reading the questions that I narrowed down to two choices I was able to better understand why I selected the correct answer. For example, question ten was pretty confusing for me. I narrowed it down to Feste and Toby. Feste was the fool, and I didn't know why Toby would consider himself a fool, and at first thought, I just guessed Toby because it didn't sound like Feste. After re-reading the quote and looking at the two choices, I understood not only the quote but also remembered the scene that it was from, and this helped me to understand the correct answer.

What I as a teacher observed from this first trial

• On the first quiz, some students struggled with having to write a thought report and wanted to summarize the text in order to give an answer.

•By the second quiz, most students had begun to model their thought reports after viewing other students’ thought reports, especially the ones who had correct answers.

•Students who started out using test-prep “apples and oranges” thinking began to reread the text and come up with their answers based on their reading a second time.

What students said about HyLighter•More interactive

One of my favorite options of this new system is that we get to compare and see how our classmates might have had a different view. This is definitely more interactive and enhances the learning process. Instead of just having an overall discussion, the compare option gives it greater depth by being more specific. It also gives me the opportunity to see how different people might have seen it in a different way.

What students said about HyLighter• More focused

I think it helps without a doubt. I enjoy seeing how other people approach the questions as well as having the opportunity to actual learn how to approach the question so I don't just continue to make the same mistakes. Having to discuss each question helps too because it keeps the discussion focused so it ends up helping instead of talking about pointless things.

What students said about HyLighter• Shows errors in thinking

I didn't get any questions wrong on my first attempt at the quiz, but looking at how some of the other kids answered their questions wrong was interesting. It was helpful to see how they could interpret some of the quotes incorrectly, but be able to defend their reasoning well.

What students said about HyLighter

• Showed more “efficient” way to reach the right answer

I found it really interesting to read the response to question ten. This was the question I was having the most trouble with, and it helped me to learn that other students also had the same problem. It also helped me to read the way they reached their answer. By reading these answers, I was able to find a shorter and more efficient way to reach the right answer.

What students said about HyLighter

• As part of their discussion questions, students were asked to make suggestions about how HyLighter could better serve their needs.

• Based on their suggestions, HyLighter is now designing a new version to meet the needs expressed by University of Miami Online High School students on the following slides.

What students said to add to HyLighter Provide direct quotes from text and have students explain them

The format overall is very helpful. I do think it could be beneficial to be able to post discussions on certain elements of the play that may be harder to understand. It's important for me to be able to see how my other classmates are thinking, and this becomes hard because we are not in a typical classroom-format. Any way that make taking AP English Lit online more like a classroom would be helpful to anyone.

What students said to add Provide the freedom to write an

overall comment about a passage

I like this format. Yes, I do think having difficult passages would be helpful not only to learn different approaches but also to see how different people interpret writing, especially Shakespeare. It's very personal and to see others perceptions would be nice. I think the best part of writing is that there is so much freedom in it. I miss that one part about a regular classroom so that feedback would be very helpful.

What students said to add

Provide a way to respond to other students’ ideas

I think it would be a good idea to create more questions that cover the style of writing and important soliloquies. It would also be a good idea to be able to respond to other students. Similar to the Apex system, HyLighter should include the option of being able to have discussions with our classmates on the play as a whole.

What students said to add

Ask questions about the author’s style in important passages

I think it would be a good idea to create more questions that cover the style of writing and important soliloquies. It would also be a good idea to be able to respond to other students. Similar to the Apex system, HyLighter should include the option of being able to have discussions with our classmates on the play as a whole.

What the teacher said to add

Based on the success of UMOHS students, HyLighter will provide the following

features in their new version

• Computer-graded multiple choice tests

• Computer-controlled feature that allows students to compare after completion of test, rather than wait on teacher to open up compare feature. This feature will be available on both multiple-choice tests and student discussions about the text.

Dale W. Lick Dale@hylighter.com

David G. Lebow David@hylighter.com

Hope Hartman gravity1@mindspring.com

Edie LeBas elebas@umohs.org PowerPoint Presentationwww.hylighter.com/FETC2005.ppt