Sheryl Nussbaum-Beach, M.Ed. (snbeach@cox.net)Christopher R. Gareis, Ed.D. (crgare@wm.edu)

The College of William & MaryWilliamsburg, Virginia

What Are They Talking About?What Are They Talking About?

Using Content Analysis Methodology to Evaluate Conversations Among Novice & Mentor

Teachers in an Online Community

1-to-1

Group

http://endapt.wm.edu/modules/emissary-admin/info.php?template=home_page.html

Info on demand

ENDAPT GoalsENDAPT Goals

1. Improve pre-service preparation2. Improve novice induction3. Improve teacher effectiveness4. Improve teacher retention5. Foster teacher leadership6. Improve K-12 student learning

Overcomes some recognized limitations of site-based mentoring:

• Adequate mentors• Scheduling & accessibility

• School politics

(Abbott, 2003; Schlager, Fusco, Koch, Crawford, & Phillips, 2003)

Why Online Mentoring for Novice Teachers?Why Online Mentoring for Novice Teachers?

Telementoring e-mentoring

Virtual learning community

Computer-mediated communication

The ENDAPT The ENDAPT Group Mentoring ForumGroup Mentoring Forum

Venue• Tapped-In• Virtual rooms (“chat rooms”)• Closed community• Asynchronous

Facilitation• Moderator at William & Mary• Graduate student

Mentors• Recognized excellence (TLN)• Trained online as mentors• Paid an honorarium

http://www.tappedin.org

Novice

Mentor

Mentor

Novice

Mentor

Threaded Discussion in a Virtual RoomThreaded Discussion in a Virtual Room

Purpose of StudyPurpose of Study

With whom, why, and about what

do novice and experienced teachers “talk”

in an online mentoring community?

Research QuestionsResearch Questions

• Flow: What are the directions and frequency of the posts among mentor and novice teachers?

• Function: What are the purposes and frequency of the posts among mentor and novice teachers?

• Content: What are the topics and frequency of the posts among mentor and novice teachers?

Content AnalysisContent Analysis

Unit of Data Collection

Discussion thread within an online mentoring forum

Unit of Data Analysis

Individual posting to discussion thread within an online mentoring forum

Type of Evidence: Manifest vs. Latent

Manifest or explicit meaning that can be objectively derived from the words used and the thoughts expressed in the postings.

Content Analysis MethodologyContent Analysis Methodology

1. Created function categories*

2. Identified content categories**

3. Developed codebook

4. Pilot coded

5. Trained coders (6)

6. Calculated inter-rater reliability

* Adapted from Bonk & Kim's (1998) 12 forms of mentoring communication. ** Adapted from William & Mary (2002) student teacher competencies

Inter-rater ReliabilityInter-rater Reliability

YEAR 2YEAR 2

Range: .80 - .94

Overall: .86

YEAR 1YEAR 1

Range: .82 - .93

Overall: .88

Pilot Participants

17 Novices- Year 2• 10 elementary• 2 middle• 4 high• 1 guidance counselor • 17 in VA

11 Mentors- Year 2• 5 elementary• 2 middle• 4 high• AL, CA, MO, SC, VA• 10-38 years experience• 19 years experience on average• TOY—school, district, region, state,

national• NBPTS certified • Danforth fellow• Officers/president of state

professional associations• State/national commissions• Department/team/grade-level chairs• Awards from professional assn.

13 Novices- Year 1• 5 elementary• 2 middle• 4 high• 2 reading specialists• 11 in VA• 1 in FL• 1 in AZ

11 Mentors- Year 1• 6 elementary• 3 middle• 2 high• AL, CA, FL, MI, MO, NC, VA• 5 - 31 years experience• 20 years experience on average• TOY—school, district, region, state,

national• NBPTS certified • Danforth fellow• Officers/president of state

professional associations• State/national commissions• Department/team/grade-level chairs• Awards from professional assn. 8

Mean Median Mode SDMentors 33.81 35 25, 35 23.37Novices 11.77 5 20 13.71

Year 2Mean Median Mode SD

Mentors 28.00 22 28 22.44Novices 11.65 2 0 17.39

Year 1

Number of Posts by Individual Novices and Mentors

0

10

20

30

40

50

60

70

80

90

Individual Mentors & Novices

Number of Posts

Novice

Mentor

Number of Postings by Individual Novices and Mentors

0

10

20

30

40

50

60

70

80

Nu

mb

er o

f P

ost

s

Mentors

Novices

Year 2

Year 1

Number of Posts per Month

0

20

40

60

80

100

120

140

Septe

mbe

r

Octobe

r

Novem

ber

Decem

ber

Janu

ary

Febru

ary

Mar

chApr

ilM

ayJu

ne

Number of Posts per Month - YEAR 2

0

20

40

60

80

100

120

140

160

Year 2Year 1

Frequency & Direction of PostsFrequency & Direction of Posts

SpecificMentor

SpecificNovice

Broadcast

Mentor

SpecificNovice

SpecificMentor

Broadcast

Novice

Direction of Mentor Posts

mentor to mentor, 11%

mentor to novice, 48%

mentor to broadcast,

41%

mentor to mentor

mentor to novice

mentor to broadcast

Year 2

Direction of Mentor Posts

mentor to novice41%

mentor to broadcast46%

mentor to mentor13%

Year 1

Direction of Novice Posts

novice to mentor33%

novice to broadcast56%

novice to novice11%

Direction of Novice Posts

novice to novice21%

novice to mentor21%

novice to broadcast

58%

novice to novice

novice to mentor

novice to broadcast

Year 2

Year 1

Mentor and Novice FunctionsMentor and Novice Functions

Mentor Novice(1) Support / Confirmation Acknowledgement / Thanks

(2) Guided Advice Guided Advice

(3) Modeling Sharing Experience,

No Problem

(4) Seeking Clarification / Direct Questioning

Seeking Information / Specific Question

(5) Prompting Reflection Explicit Issue / Problem

(6) Professional Growth Reflection /

Professional Growth

Mentor Function Categories(1) Support/Confirmation -- The mentor makes a supportive statement and/or confirms the actions of a novice teacher or another mentor: “You’re doing great…”, “Don’t doubt yourself…”, "You're really growing as a professional...", "That was a good decision..."

(2) Guided Advice -- The mentor provides specific direction, instruction, or advice to the novice teacher or another mentor: “You may want to try…”, “I suggest using…”

(3) Modeling -- The mentor describes his or her own experience or thinking but does not give direct advice, answers, or interpretations of a given situation: “The way I’ve handled that situation is to…”, "Once I...", "I remember...", "In my classroom..."

(4) Seeking Clarification/Direct Questioning -- The mentor poses a direct question or makes a statement inviting explanation or clarification: “How are you currently grading homework?”, “Have you discussed this with your principal?”, "Tell me more about..."

(5) Prompting Reflection -- The mentor prompts or otherwise creates an opportunity for a novice to think about, share, and evaluate his or her own thinking: "What do you think your actions resulted in?", "How might your students have taken what you said to them?", "What do you think the parent was thinking during the conference?"

(6) Professional Growth -- The mentor makes an explicit statement about his or her own professional growth, new understanding, or change in practice resulting from online forum: “I’m starting to see…”, “I feel more confident…”, “I used to think…but now...”

Novice Function Categories(1) Acknowledgement/Thanks -- The novice acknowledges his or her understanding someone else's ideas, or indicates acceptance of new ideas, or expresses thanks for someone else's thought: “I see what you’re saying…”, "That's good advice...”

(2) Guided Advice -- The novice provides specific direction, instruction, or advice to another novice teacher or mentor: “You may want to try…”, “I suggest using…”

(3) Sharing Experience, No Problem -- The novice describes an experience, but there is no indication that the experience is problematic: "Today was great! I had my class doing...", "I had my first parent conference yesterday and it went really well." , "In our school use..."

(4) Specific Question/Seeking Specific Information -- The novice teacher poses a question or invites others to provide him or her specific information: “How do you use manipulatives in middle school math?”, "What reading series do you use?", "What should I expect during my first observation?", "I need to know more about...", "Anything you can tell me is appreciated..."

(5) Explicit Issue/Problem -- The novice specifically describes a situation that he or she is confronting or has confronted in his or her practice, but does not pose a question or specifically seek a response: “I’m having trouble with…”, “In my school…”, "Yesterday was awful. I had a student..."

(6) Reflection/Professional Growth -- The novice teacher makes an explicit statement about his or her own professional growth, new understanding, or change in practice, either resulting from the online forum or not: “I’m starting to see…”, “I feel more confident…”, “I used to think…but now...”

Percentage of Novice Posts Including Each Novice Function

0%

10%

20%

30%

40%

50%

Percentage * 36% 7% 43% 27% 41% 11%

Acknowledge Thanks

Guided Advice

Sharing Experience

QuestionsIssue

ProblemsReflection

Growth

Percentage of Novice Posts Including Each Novice Function

0%

10%

20%

30%

40%

50%

Percentage 30% 11% 42% 37% 39% 12%

Acknowledge/ Thanks

Guided Advice

Sharing Experience

Questions Issue/

Problem Reflection/

Growth

Year 2

Year 1

Percentages exceed 100 because a single post may have multiple functions.

Percentages exceed 100 because a single post may have multiple functions.

Percentage of Mentor PostsIncluding Each Mentor Function

0%

20%

40%

60%

80%

Percentage 41% 38% 63% 12% 11% 3%

Support/ Confirmation

Guided Advice

Modeling QuestionsPrompt

ReflectionProfessional

Growth

Percentage of Mentor Posts Including Each Mentor Function

0%

20%

40%

60%

80%

Percentage* 37% 41% 62% 12% 7% 2%

Support Confirmation

Guided Advice Modeling QuestionsPrompt

ReflectionProfessional

Grow th

Year 2

Year 1

Content CategoriesContent Categories(1) Planning for Instruction• Plans lessons that align with local, state, and national standards.• Selects appropriate instructional strategies/activities.• Selects appropriate materials/resources.

(2) Instructional Delivery• Teaches based on planned lessons.• Provides for individual differences.• Uses motivational strategies to promote learning.• Engages students actively in learning.• Uses a variety of effective teaching strategies.• Helps students develop thinking skills that promote learning.• Monitors student learning.

(3) Assessment for Learning• Creates and selects appropriate assessments for learning.• Implements assessments for learning.• Interprets/uses assessment results to make instructional decisions.

(4) Classroom Management• Builds positive rapport with and among students.• Organizes for effective teaching.• Demonstrates use of effective routines and procedures.• Demonstrates efficient and effective use of time.• Maintains a safe learning environment.• Responds appropriately to inappropriate behavior.

(5) Professionalism• Demonstrates professional demeanor and ethical behavior.• Participates in and applies professional development.• Demonstrates effective oral and written communication.• Cooperates, collaborates, and fosters relationships with members of the school

community.

Comparison of Percentage of PostsContaining Each Content Category

by Mentors and Novices

0%

10%

20%

30%

40%

50%

Mentors 26% 35% 15% 36% 30%

Novices 41% 36% 17% 36% 26%

Planning for Instruction

Instructional Delivery

AssessmentClassroom

ManagementProfessionalism

Comparison of Percentage of Posts Containing Each Content Category by Mentors and Novices

0%

20%

40%

60%

Mentors 26% 36% 10% 39% 47%

Novice 25% 38% 10% 40% 37%

Planning Instruction

Instructional Delivery

AssessmentClassroom

ManagementProfessionalism

Year 1

Year 2

FINDINGSFINDINGS

Evidence of a Professional Learning CommunityEvidence of a Professional Learning Community

• Communication occurred between and among novices and mentors alike.

Evidence of Classic “Mentoring” BehaviorsEvidence of Classic “Mentoring” Behaviors

• Mentors supported, guided, modeled, asked questions, prompted reflection rather than only directing or giving advice

• Novices shared experiences as often as they asked direct questions

Evidence of Professional CompetenciesEvidence of Professional Competencies

• Conversations focused on professional teacher competencies

Lessons Learned about Lessons Learned about ImplementationImplementation

• Market & recruit early

• Familiarize mentors & novices with the platform

• Train the mentors

• Choose your moderator carefully

• Prompt participation behind the scenes

• Provide modest incentives

• Expect periods of activity & inactivity