presented at the mcri workshop may 30, 2008 redesigning general psychology @ frostburg state...

Presented at the MCRI Workshop May 30, 2008

Redesigning General Psychology @ Frostburg State University

MCRI Workshop

May 30, 2008

Primary FSU NCAT Team:

•Megan E. Bradley, Ph.D.

•Bill Southerly, Ph.D.

•Cindy D. Hay, MDE & MS

•Joseph Hoffman, Ph.D.

•John Bowman, Ph.D.


Successes

• Assessment #1: Comparison of comprehensive final exam

• All instructors blind to exam content


Successes

• Redesign sections significantly better than traditional sections*

• Redesign sections the difference between a student earning a C vs. D

*A one-way ANOVA of section (6 total: 2 traditional and 4 redesign) on final exam percentage grades was significant, F = 4.710, p = .000, η2 = .090.

Comparison of Common Comprehensive Final Exam:

75%

68%

62%

64%

66%

68%

70%

72%

74%

76%

78%

Section Type

Redesign

Traditional


Successes

• Redesign sections again significantly better than traditional sections by nearly 10%*

*A one-way ANOVA of section (2 total: traditional and redesign) on final exam percentage for

conceptual questions grades was significant, F = 27.508, p = .000, η2 = .102.

Comparison of Factually-Based Questions from

Common Comprehensive Final Exam:

79%

70%

63%

67%

71%

75%

79%

83%

Section Type

Redesign

Traditional


Successes

• Redesign sections yet again significantly better than traditional sections*

*A one-way ANOVA of section (2 total: traditional and redesign) on final exam percentage for

conceptual questions grades was significant, F = 7.452, p = .007, η2 = .030.

Comparison of Conceptually-Based Questions from

Common Comprehensive Final Exam:

71%

66%

55%

60%

65%

70%

75%

Section Type

Redesign

Traditional


Successes

• Reason for success on multiple choice part of final exam by redesign students?

• Final exam scores positively correlated with average scores on Mastery Quizzes

• r =.523, p = .000


Successes

• Assessment #2: Comparison of extra credit essay question on final exam

• All instructors blind to exam content

• Question: Discuss the psychological phenomenon of prejudice. In your answer, feel free to reflect on causes/explanations, consequences of being a victim of prejudice, and/or ideas to counter prejudice.


Successes

• Assessment #2: Comparison of extra credit essay question on final exam

• Scoring rubric emphasized use of psychological concepts

• No upper limit for scores• 1 point per correct use of psychological concepts• 1 point per accurate definition of terms• 1 point per additional psychological concept


Successes

• Redesign sections significantly better than traditional sections*

• Mode • Traditional = 0 or 1• Redesign = 1, 2, 2, 3

*A one-way ANOVA of section (6 total: 2 traditional and 4 redesign) on final exam percentage grades was significant, F = 6.787, p = .000, η2 = .153.

Comparison of Extra Credit Essay Question:

2.845

1.092

0.00

0.50

1.00

1.50

2.00

2.50

3.00

3.50

Section Type

Redesign

Traditional


Successes

• Reason for success on essay question of final exam by redesign students?

• Essay score positively correlated with

• Grade on semester long prejudice project• r =.328, p = .000

• Grade across all online discussions• r = .244, p = .005


Successes

RedesignTraditional

1) Identify struggling students? Yes No

2) % students meet for help? 10 - 30% 6%

3) Email communication? Average 60 0

Instructor Questionnaire Weeks 1-2


Successes

RedesignTraditional

4a) In class behavior• Lecture notes 7% 40%• Asking questions 10% 14%• Large group discussions 13% 4%• Small group discussions 35% 6%• Self-assessments 4% 7%• Psychology-related activities 28% 13%• Videos 5% 8%



Successes

RedesignTraditional

4b) In class behavior• Taking quizzes 0% 0%• Critical thinking skills 0% 9%• Writing about Psychology 0% 0%• Oral presentations 0% 0%• Taking tests 0% 0%



Successes

RedesignTraditional

5a) Out of class/online behavior• Group discussions 15% 0%• Self-assessments 5% 0%• Psychology-related activities 15% 40%• Videos 0% 0%



Successes

RedesignTraditional

5b) Out of class/online behavior• Visiting web pages 5% 0%• Taking quizzes 15% 0%• Retaking quizzes (master) 17% 0%• Writing about Psychology 20% 0%• Critical thinking skills 10% 18%



Successes

RedesignTraditional

4a) In class behavior• Lecture notes 18% 30%• Asking questions 13% 8%• Large group discussions 10% 8%• Small group discussions 23% 8%• Self-assessments 8% 5%• Psychology-related activities 13% 5%• Videos 8% 7%

Instructor Questionnaire Full Semester


Successes

RedesignTraditional

4b) In class behavior• Taking quizzes 0% 3%• Critical thinking skills 10% 13%• Writing about Psychology 0% 4%• Oral presentations 0% 3%• Taking tests 0% 10%



Successes

RedesignTraditional

5a) Out of class/online behavior• Group discussions 19% 3%• Self-assessments 8% 10%• Psychology-related activities 8% 3%• Videos 8% 0%



Successes

RedesignTraditional

5b) Out of class/online behavior• Visiting web pages 8% 0%• Taking quizzes 15% 0%• Retaking quizzes (master) 23% 0%• Writing about Psychology 9% 15%• Critical thinking skills 10% 35%



Successes

• ULAs (Undergraduate Learning Assistants)

• New course and Leadership Certificate Program• Accepted by MHEC May 19, 2008

• Nominated for campus-wide leadership award• 2 students earned prestigious summer

opportunities• 1 student earned graduate assistantship related to

teaching


Successes

• Source• Adjuncts

• 5 positions

• Full-time faculty• Need 1 less position

• TOTAL SAVINGS

• Savings• $11,000

• $62,699

• $73,699

Financial savings


Implementation Issues

• DWF rate / retention issue

• Previous average: 15%• Fall 2007 rate: 33%• Trend: increasing since 2002

• Spring 2008• Traditional sections: 4%• Redesign sections: 22%



• DWF rate / retention issue• Possible reasons for lower than normal DWF rate

for traditional sections• Final exam, believed by students to be required and

count in grade while taking test, was treated as extra credit upon completion

• Only traditional sections had students earning extra credit for participating in research studies



• DWF rate / retention issue• Possible reasons for higher than normal DWF

rate for redesign sections• Difficult beginning for 2 reasons

• Access issues to online (graded) content related to financial aid reimbursement

• Technological issues

• Students not ready for self-motivation needed for blended part

• Too many required activities in first half of semester



• DWF rate / retention issue• Adjustments

• Access to online portion course/e-text• Publisher• Future: Lab fee

• Computer labs with ULAs• Supplemental Instruction Meetings

• Student Support Group

• Extension of assignments from 1 week to 2 weeks• “Deadline Disorder”



• Financial reimbursement to Department

• Work study money to cover labs• Limited lab space requires 2 ULAs per lab time

• Recouping gains to make redesign effort worthwhile• University or system-wide policy



• Student Motivation/Preparation

• Readiness for active learning• Attitude towards online/blended learning• Willingness to check email

• Need for university change to student email system (too many emails)

• Preventing/treating Deadline Disorder• Other ideas?



• Reactions from colleagues

• Department: Very supportive

• Pre-approval before submitting proposal• Fall 2007 retreat



• Reactions from colleagues

• University-wide

• Hesitant or resistant to

• Online/blended courses• Large sections• “Inexperienced” undergraduates helping courses• Changing the perceived FSU identity



• Solutions for colleagues• Department

• Continue yearly retreat• Seek financial reimbursement for departmental initiatives

• University-wide• Held spring workshop for university

• Carolyn Jarmon guest speaker • Sponsored by Houghton-Mifflin

• Large positive response

• Plan for more in fall



• Technology issues

• Met with Worth representatives• Issue for labs

• Divide each section into 2 “shifts” in 2 classrooms• N = 75 per section for fall (replacement model)

• Largest lab size = 30

• Ways to motivate students to bypass lab??• Reward system that is easy to implement



• Retention and student support

• Removed “blended” label• Working with student support services to implement

university-wide policies• Tie-in

• Learning communities• Orientations (redesign)• General Psychology



• What would we have done differently?

• Reduce number of activities/deadlines

• Get financial policy finalized before implementation• System-wide policy• Deadlines needed for both the redesign team and the

administration for financial reimbursement

presented at the mcri workshop may 30, 2008 redesigning general psychology @ frostburg state...

Documents