using idea for assessment, program review, and sacs university of alabama birmingham september 11,...
TRANSCRIPT
![Page 1: Using IDEA for Assessment, Program Review, and SACS University of Alabama Birmingham September 11, 2012 Shelley A. Chapman, PhD](https://reader035.vdocuments.us/reader035/viewer/2022062716/56649de85503460f94ae2ea9/html5/thumbnails/1.jpg)
Using IDEA for Assessment, Program Review, and SACSUniversity of Alabama BirminghamSeptember 11, 2012
Shelley A. Chapman, PhD
![Page 2: Using IDEA for Assessment, Program Review, and SACS University of Alabama Birmingham September 11, 2012 Shelley A. Chapman, PhD](https://reader035.vdocuments.us/reader035/viewer/2022062716/56649de85503460f94ae2ea9/html5/thumbnails/2.jpg)
Plan for this Session
• Program Evaluation & Assessment of Student Learning• Group Summary Reports• Aggregate Data File• Benchmarking Reports• Accreditation Guides
![Page 3: Using IDEA for Assessment, Program Review, and SACS University of Alabama Birmingham September 11, 2012 Shelley A. Chapman, PhD](https://reader035.vdocuments.us/reader035/viewer/2022062716/56649de85503460f94ae2ea9/html5/thumbnails/3.jpg)
What makes IDEA unique?
1. Focus on Student Learning
2. Focus on Instructor’s Purpose
3. Adjustments for Extraneous Influences
4. Validity and Reliability
5. Comparison Data
6. Flexibility
![Page 4: Using IDEA for Assessment, Program Review, and SACS University of Alabama Birmingham September 11, 2012 Shelley A. Chapman, PhD](https://reader035.vdocuments.us/reader035/viewer/2022062716/56649de85503460f94ae2ea9/html5/thumbnails/4.jpg)
Student Learning Model: 2 Assumptions
Assumption 1:
Types of learning must reflect the instructor’s purpose.
![Page 5: Using IDEA for Assessment, Program Review, and SACS University of Alabama Birmingham September 11, 2012 Shelley A. Chapman, PhD](https://reader035.vdocuments.us/reader035/viewer/2022062716/56649de85503460f94ae2ea9/html5/thumbnails/5.jpg)
Student Diagnostic Form
Assumption 2:
Effectiveness determined by students’ progress on objectives stressed by instructor
![Page 6: Using IDEA for Assessment, Program Review, and SACS University of Alabama Birmingham September 11, 2012 Shelley A. Chapman, PhD](https://reader035.vdocuments.us/reader035/viewer/2022062716/56649de85503460f94ae2ea9/html5/thumbnails/6.jpg)
Diagnostic Report Overview Page 1 – Big Picture
How did I do?
Page 3 – Diagnostic What can I do differently?
Page 2 – Learning Details What did students learn?
Page 4 – Statistical Detail Any additional
insights?
![Page 7: Using IDEA for Assessment, Program Review, and SACS University of Alabama Birmingham September 11, 2012 Shelley A. Chapman, PhD](https://reader035.vdocuments.us/reader035/viewer/2022062716/56649de85503460f94ae2ea9/html5/thumbnails/7.jpg)
Your Average (5-point Scale)
Raw Adj.
A. Progress on Relevant Objectives1
Four objectives were selected as relevant (Important or Essential—see page 2)
4.1 4.3
1If you are comparing Progress on Relevant Objectives from one instructor to another, use the converted average.
The Big Picture
![Page 8: Using IDEA for Assessment, Program Review, and SACS University of Alabama Birmingham September 11, 2012 Shelley A. Chapman, PhD](https://reader035.vdocuments.us/reader035/viewer/2022062716/56649de85503460f94ae2ea9/html5/thumbnails/8.jpg)
ProgressOnRelevantObjectives
4
4.3 + 4.34.14.23.6
5
![Page 9: Using IDEA for Assessment, Program Review, and SACS University of Alabama Birmingham September 11, 2012 Shelley A. Chapman, PhD](https://reader035.vdocuments.us/reader035/viewer/2022062716/56649de85503460f94ae2ea9/html5/thumbnails/9.jpg)
Summary Evaluation: Five-Point Scale
Report Page 1
Your Average Score
(5-point scale)
Raw Adj.
A. Progress on Relevant ObjectivesFour objectives were selected as relevant (Important or Essential—see page 2)
4.1 4.3
Overall Ratings B. Excellent Teacher 4.7 4.9
C. Excellent Course 4.1 4.4
D. Average of B & C 4.4 4.7
Summary Evaluation(Average of A & D) 4.3 4.5
50%
25%
25%
![Page 10: Using IDEA for Assessment, Program Review, and SACS University of Alabama Birmingham September 11, 2012 Shelley A. Chapman, PhD](https://reader035.vdocuments.us/reader035/viewer/2022062716/56649de85503460f94ae2ea9/html5/thumbnails/10.jpg)
Individual Reports to Group Reports
![Page 11: Using IDEA for Assessment, Program Review, and SACS University of Alabama Birmingham September 11, 2012 Shelley A. Chapman, PhD](https://reader035.vdocuments.us/reader035/viewer/2022062716/56649de85503460f94ae2ea9/html5/thumbnails/11.jpg)
The Group Summary Report
How did we do?
How might we improve?
![Page 12: Using IDEA for Assessment, Program Review, and SACS University of Alabama Birmingham September 11, 2012 Shelley A. Chapman, PhD](https://reader035.vdocuments.us/reader035/viewer/2022062716/56649de85503460f94ae2ea9/html5/thumbnails/12.jpg)
Defining Group Summary Reports (GSRs)
• Institutional
• Departmental
• Service/Introductory Courses
• Major Field Courses
• General Education Program
![Page 13: Using IDEA for Assessment, Program Review, and SACS University of Alabama Birmingham September 11, 2012 Shelley A. Chapman, PhD](https://reader035.vdocuments.us/reader035/viewer/2022062716/56649de85503460f94ae2ea9/html5/thumbnails/13.jpg)
GSRs Help Address Questions
• Longitudinal• Contextual• Curricular• Pedagogical• Student Learning-
focused
![Page 14: Using IDEA for Assessment, Program Review, and SACS University of Alabama Birmingham September 11, 2012 Shelley A. Chapman, PhD](https://reader035.vdocuments.us/reader035/viewer/2022062716/56649de85503460f94ae2ea9/html5/thumbnails/14.jpg)
Adding Questions
Up to 20 Questions can be added
• Institutional
• Departmental
• Course-based
• All of the above
![Page 15: Using IDEA for Assessment, Program Review, and SACS University of Alabama Birmingham September 11, 2012 Shelley A. Chapman, PhD](https://reader035.vdocuments.us/reader035/viewer/2022062716/56649de85503460f94ae2ea9/html5/thumbnails/15.jpg)
Local CodeUse this section
of the FIF to code types of data.
![Page 16: Using IDEA for Assessment, Program Review, and SACS University of Alabama Birmingham September 11, 2012 Shelley A. Chapman, PhD](https://reader035.vdocuments.us/reader035/viewer/2022062716/56649de85503460f94ae2ea9/html5/thumbnails/16.jpg)
Defining Group Summary Reports• Local Code
• 8 possible fields• Example: Column one – Delivery Format
• 1=Self-paced• 2=Lecture• 3=Studio• 4=Lab• 5=Seminar• 6=Online
Example from Benedictine University
![Page 17: Using IDEA for Assessment, Program Review, and SACS University of Alabama Birmingham September 11, 2012 Shelley A. Chapman, PhD](https://reader035.vdocuments.us/reader035/viewer/2022062716/56649de85503460f94ae2ea9/html5/thumbnails/17.jpg)
Example Using Local code
Assign Local Code• 1=Day, Tenured• 2=Evening, Tenured• 3=Day, Tenure Track• 4=Evening, Tenure
Track• 5=Day, Adjunct• 6=Evening, Adjunct
Request Reports• All Day Classes
• Local Code=1, 3, & 5
• All Evening Classes• Local Code=2, 4, & 6
• Courses Taught by Adjuncts• Local Code=5 & 6
![Page 18: Using IDEA for Assessment, Program Review, and SACS University of Alabama Birmingham September 11, 2012 Shelley A. Chapman, PhD](https://reader035.vdocuments.us/reader035/viewer/2022062716/56649de85503460f94ae2ea9/html5/thumbnails/18.jpg)
Description of Courses Included in this Report
Number of Classes IncludedDiagnostic From 42
Short Form 27
Total 69
Number of Excluded Classes 0
Response RateClasses below 65% Response Rate 2
Average Response Rate 85%
Class SizeAverage Class Size 20
Page 1 of GSR
![Page 19: Using IDEA for Assessment, Program Review, and SACS University of Alabama Birmingham September 11, 2012 Shelley A. Chapman, PhD](https://reader035.vdocuments.us/reader035/viewer/2022062716/56649de85503460f94ae2ea9/html5/thumbnails/19.jpg)
UAB Spring 2012
Page 1 of GSR
![Page 20: Using IDEA for Assessment, Program Review, and SACS University of Alabama Birmingham September 11, 2012 Shelley A. Chapman, PhD](https://reader035.vdocuments.us/reader035/viewer/2022062716/56649de85503460f94ae2ea9/html5/thumbnails/20.jpg)
Assessment of Learning
What are our faculty emphasizing?
How do students rate their learning?
How do our courses compare with others?
How do our students compare with others (self-rated characteristics)?
What efforts can we make for improvement?
(How can we “close the loop”?)
![Page 21: Using IDEA for Assessment, Program Review, and SACS University of Alabama Birmingham September 11, 2012 Shelley A. Chapman, PhD](https://reader035.vdocuments.us/reader035/viewer/2022062716/56649de85503460f94ae2ea9/html5/thumbnails/21.jpg)
UAB Core Competencies
Shared Vision for a UAB Graduate
Possible IDEA Learning Objectives
Communication 8, 11
Knowledge 1, 2, 4, 7
Problem Solving 3, 9, 11
Citizenship 10, 12, extra questions
![Page 22: Using IDEA for Assessment, Program Review, and SACS University of Alabama Birmingham September 11, 2012 Shelley A. Chapman, PhD](https://reader035.vdocuments.us/reader035/viewer/2022062716/56649de85503460f94ae2ea9/html5/thumbnails/22.jpg)
Are we targeting “Core Competencies” in the Core Curriculum?
UAB Core Curriculum Courses
1 2 3 4 5 6 7 8 9 10 11 12
EH 101 English Composition X X X X
BY 101 Topics in Contemporary Biology
X X X X
MA 105 Pre-Calculus Algebra X X X
ARH 101 The Art Experience X X X
HY 101 Western Civilization I X X X X
PY 101 Introduction to Psychology
X X X X
IDEA Learning Objectives
![Page 23: Using IDEA for Assessment, Program Review, and SACS University of Alabama Birmingham September 11, 2012 Shelley A. Chapman, PhD](https://reader035.vdocuments.us/reader035/viewer/2022062716/56649de85503460f94ae2ea9/html5/thumbnails/23.jpg)
Are we targeting “Core Competencies” in the Core Curriculum?
UAB Core Curriculum Courses
1 2 3 4 5 6 7 8 9 10 11 12
EH 101 English Composition X X X X
BY 101 Topics in Contemporary Biology
X X X X
MA 105 Pre-Calculus Algebra X X X
ARH 101 The Art Experience X X
HY 101 Western Civilization I X X X X
PY 101 Introduction to Psychology
X X X
IDEA Learning Objectives
![Page 24: Using IDEA for Assessment, Program Review, and SACS University of Alabama Birmingham September 11, 2012 Shelley A. Chapman, PhD](https://reader035.vdocuments.us/reader035/viewer/2022062716/56649de85503460f94ae2ea9/html5/thumbnails/24.jpg)
What are We Emphasizing?Percent of Classes Selecting Obj. as
Important or Essential
This Group Institution IDEASystem
Objective 1 16% 70% 78%
Objective 2 13% 59% 75%
Objective 3 41% 58% 75%
Objective 4 32% 35% 55%
Objective 5 23% 19% 32%
Objective 6 32% 14% 25%
Objective 7 22% 27% 27%
Objective 8 78% 43% 47%
Objective 9 19% 23% 41%
Objective 10 7% 11% 23%
Objective 11 68% 42% 49%
Objective 12 20% 23% 41%
Average # of Obj. Selected 3.7 4.2 5.7
Page 2
![Page 25: Using IDEA for Assessment, Program Review, and SACS University of Alabama Birmingham September 11, 2012 Shelley A. Chapman, PhD](https://reader035.vdocuments.us/reader035/viewer/2022062716/56649de85503460f94ae2ea9/html5/thumbnails/25.jpg)
UAB Spring 2012
![Page 26: Using IDEA for Assessment, Program Review, and SACS University of Alabama Birmingham September 11, 2012 Shelley A. Chapman, PhD](https://reader035.vdocuments.us/reader035/viewer/2022062716/56649de85503460f94ae2ea9/html5/thumbnails/26.jpg)
What are We Emphasizing?
Page 9 Section B
NumberRating
Percent indicating amount required
None or Little
Some Much
Writing 66 2% 17% 82%
Oral Communication 66 6% 42% 52%
Computer Application 66 50% 44% 6%
Group Work 66 27% 59% 14%
Mathematics/Quantitative Work
65 97% 3% 0%
Critical Thinking 66 0% 30% 70%
Creative/Artistic/Design 66 61% 33% 6%
![Page 27: Using IDEA for Assessment, Program Review, and SACS University of Alabama Birmingham September 11, 2012 Shelley A. Chapman, PhD](https://reader035.vdocuments.us/reader035/viewer/2022062716/56649de85503460f94ae2ea9/html5/thumbnails/27.jpg)
How Did Students Rate their Learning on Core Competencies?
![Page 28: Using IDEA for Assessment, Program Review, and SACS University of Alabama Birmingham September 11, 2012 Shelley A. Chapman, PhD](https://reader035.vdocuments.us/reader035/viewer/2022062716/56649de85503460f94ae2ea9/html5/thumbnails/28.jpg)
Do Students’ report of learning meet our expectations?
Pages 5 and 6
Raw Average
Adj.Average
# ofClasses
ThisReport
3.9 3.9 11
Institution 4.2 4.2 3,963
IDEA System
4.0 4.0 31,991
Objective 1: Gaining factual knowledge (terminology, classifications, methods, trends)
![Page 29: Using IDEA for Assessment, Program Review, and SACS University of Alabama Birmingham September 11, 2012 Shelley A. Chapman, PhD](https://reader035.vdocuments.us/reader035/viewer/2022062716/56649de85503460f94ae2ea9/html5/thumbnails/29.jpg)
How do students rate their learning?
Page 3 Part 1: Distribution of Converted Scores Compared to the IDEA Database
![Page 30: Using IDEA for Assessment, Program Review, and SACS University of Alabama Birmingham September 11, 2012 Shelley A. Chapman, PhD](https://reader035.vdocuments.us/reader035/viewer/2022062716/56649de85503460f94ae2ea9/html5/thumbnails/30.jpg)
Overall Progress Ratings (Courses)
Page 3 Percent of Classes at or Above the IDEA database Average
![Page 31: Using IDEA for Assessment, Program Review, and SACS University of Alabama Birmingham September 11, 2012 Shelley A. Chapman, PhD](https://reader035.vdocuments.us/reader035/viewer/2022062716/56649de85503460f94ae2ea9/html5/thumbnails/31.jpg)
Overall Progress Ratings (Courses)
Part 3: Percent of Classes at or Above This Institution’s Average
Page 4
![Page 32: Using IDEA for Assessment, Program Review, and SACS University of Alabama Birmingham September 11, 2012 Shelley A. Chapman, PhD](https://reader035.vdocuments.us/reader035/viewer/2022062716/56649de85503460f94ae2ea9/html5/thumbnails/32.jpg)
Which teaching methods might we use to improve learning?
Page 7
Teaching Methods and Styles
Stimulating Student Interest # Classes Av. s.d.
15. Inspired students to set and achieve goals which really challenged them
42 3.8 0.5
![Page 33: Using IDEA for Assessment, Program Review, and SACS University of Alabama Birmingham September 11, 2012 Shelley A. Chapman, PhD](https://reader035.vdocuments.us/reader035/viewer/2022062716/56649de85503460f94ae2ea9/html5/thumbnails/33.jpg)
Assessing the QEP for UABUAB’s Core Competencies
IDEA Learning Goals IDEA Teaching Methods Associated with Progress on Learning Goal
Communication8 Developing skill in expressing myself orally or in writing11 Learning to analyze and critically evaluate ideas, arguments, and points of view
2, 7, 8, 15, 16, 18, 19
Knowledge1 Gaining factual knowledge (terminology, classifications, methods, trends)2 Learning fundamental principles, generalizations, or theories4 Developing specific skills, competencies, and points of view needed by professionals in the field most closely related to this field7 Gaining a broader understanding and appreciation of intellectual and cultural activity (music, science, literature, etc.)
1, 2, 4, 6, 7, 8, 10, 12, 13, 15, 19
Problem solving3 Learning to apply course material (to improve thinking, problem solving, and decisions) 9 Learning how to find and use resources for answering questions or solving problems11 Learning to analyze and critically evaluate ideas, arguments, and points of view
2, 7, 8, 9, 13, 14, 15, 18, 19
Citizenship10 Developing a clearer understanding of, and commitment to, personal values12 Acquiring an interest in learning more by asking my own questions and seeking answers
1, 2, 4, 6, 7, 8, 11,13, 15, 16, 18,
![Page 34: Using IDEA for Assessment, Program Review, and SACS University of Alabama Birmingham September 11, 2012 Shelley A. Chapman, PhD](https://reader035.vdocuments.us/reader035/viewer/2022062716/56649de85503460f94ae2ea9/html5/thumbnails/34.jpg)
Relationship
of Learning Objectives to
Teaching
Methods
![Page 35: Using IDEA for Assessment, Program Review, and SACS University of Alabama Birmingham September 11, 2012 Shelley A. Chapman, PhD](https://reader035.vdocuments.us/reader035/viewer/2022062716/56649de85503460f94ae2ea9/html5/thumbnails/35.jpg)
How do students view course work demands?
Page 8B Student Ratings of Course Characteristics
Diagnostic Form Item # & Item Average % Classes Below 3.0
% Classes 4.0 or Above
33. Amount of Reading
This Report 3.4 21% 24%
Institution 3.3 31% 19%
IDEA System 3.2 33% 15%
34. Amount of work in other (non-reading) assignments
This Report 3.3 24% 10%
Institution 3.4 23% 20%
IDEA System 3.4 21% 18%
35. Difficulty of subject matter
This Report 3.2 19% 0%
Institution 3.5 13% 19%
IDEA System 3.4 20% 18%
![Page 36: Using IDEA for Assessment, Program Review, and SACS University of Alabama Birmingham September 11, 2012 Shelley A. Chapman, PhD](https://reader035.vdocuments.us/reader035/viewer/2022062716/56649de85503460f94ae2ea9/html5/thumbnails/36.jpg)
Aggregate Data File
Allows you to •Use Excel Spreadsheet•Use with SAS or SPSS•Ask other types of questions•Display data in different ways
![Page 37: Using IDEA for Assessment, Program Review, and SACS University of Alabama Birmingham September 11, 2012 Shelley A. Chapman, PhD](https://reader035.vdocuments.us/reader035/viewer/2022062716/56649de85503460f94ae2ea9/html5/thumbnails/37.jpg)
A. Progress of Relevant Objectives-Converted Score Category
Expected Distribution
Fall 2010 Raw
Spring 2011 Raw
Fall 2011 Raw
Much Higher (63 or higher) 10% 6% 7% 7%Higher (56-62) 20% 32% 34% 31%Similar (45-55) 40% 46% 46% 44%Lower (38-44) 20% 11% 9% 10%Much Lower (37 or lower) 10% 5% 5% 8%
![Page 38: Using IDEA for Assessment, Program Review, and SACS University of Alabama Birmingham September 11, 2012 Shelley A. Chapman, PhD](https://reader035.vdocuments.us/reader035/viewer/2022062716/56649de85503460f94ae2ea9/html5/thumbnails/38.jpg)
B. Excellence of Teacher-Converted Score CategoryExpected Distribution
Fall 2010 Raw
Spring 2011 Raw
Fall 2011 Raw
Much Higher (63 or higher) 10% 0% 0% 0%Higher (56-62) 20% 35% 38% 34%Similar (45-55) 40% 42% 43% 40%Lower (38-44) 20% 12% 11% 12%Much Lower (37 or lower) 10% 11% 9% 14%
![Page 39: Using IDEA for Assessment, Program Review, and SACS University of Alabama Birmingham September 11, 2012 Shelley A. Chapman, PhD](https://reader035.vdocuments.us/reader035/viewer/2022062716/56649de85503460f94ae2ea9/html5/thumbnails/39.jpg)
C. Excellence of Course-Converted Score CategoryExpected
Distribution
Fall 2010 Raw
Spring 2011 Raw
Fall 2011 Raw
Much Higher (63 or higher) 10% 9% 11% 8%Higher (56-62) 20% 26% 26% 29%Similar (45-55) 40% 41% 40% 40%Lower (38-44) 20% 14% 13% 13%Much Lower (37 or lower) 10% 10% 10% 10%
![Page 40: Using IDEA for Assessment, Program Review, and SACS University of Alabama Birmingham September 11, 2012 Shelley A. Chapman, PhD](https://reader035.vdocuments.us/reader035/viewer/2022062716/56649de85503460f94ae2ea9/html5/thumbnails/40.jpg)
Summary Evaluation (Average of A, B, C)-Converted Score Category
Expected Distribution
Fall 2010
Spring 2011
Fall 2011
Much Higher (63 or higher) 10% 5% 4% 4%Higher (56-62) 20% 33% 35% 33%Similar (45-55) 40% 45% 45% 44%Lower (38-44) 20% 10% 10% 10%Much Lower (37 or lower) 10% 7% 6% 9%
![Page 41: Using IDEA for Assessment, Program Review, and SACS University of Alabama Birmingham September 11, 2012 Shelley A. Chapman, PhD](https://reader035.vdocuments.us/reader035/viewer/2022062716/56649de85503460f94ae2ea9/html5/thumbnails/41.jpg)
Primary Instructional Approaches
Fall 2010
Primary
Spring 2011
Primary
Fall 2011
PrimaryLecture 61% 60% 61%Discussion/Recitation 7% 8% 10%Seminar 7% 9% 9%Skill/Activity 7% 6% 7%Laboratory 8% 9% 9%Field Experience 1% 1% 1%Studio 2% 3% 2%Multi−Media 0% 0% 0%Practicum/Clinic 0% 0% 0%Other/Not Indicated 6% 4% 2%
Number Classes Rating 787 677 859
Secondary Instructional Approaches
Fall 2010
Secon-dary
Spring 2011
Secon-dary
Fall 2011
Secon-dary
Lecture 16% 13% 16%Discussion/Recitation 30% 27% 30%Seminar 2% 3% 4%Skill/Activity 11% 13% 13%Laboratory 3% 6% 4%Field Experience 2% 3% 2%Studio 0% 0% 1%Multi−Media 2% 3% 3%Practicum/Clinic 1% 1% 1%Other/Not Indicated 34% 31% 28%
Number Classes Rating 787 677 859
![Page 42: Using IDEA for Assessment, Program Review, and SACS University of Alabama Birmingham September 11, 2012 Shelley A. Chapman, PhD](https://reader035.vdocuments.us/reader035/viewer/2022062716/56649de85503460f94ae2ea9/html5/thumbnails/42.jpg)
Course Emphases: Writing
Fall 2010
Spring 2011
Fall 2011
None or Little 25% 26% 22%Some 46% 45% 44%Much 29% 30% 34%
0%
20%
40%
60%
80%
100%
Fall 2010 Spring 2011 Fall 2011
None or Little
Some
Much
Course Emphases: Oral communication
Fall 2010
Spring 2011
Fall 2011
None or Little 36% 35% 32%Some 47% 47% 44%Much 17% 18% 24%
0%
20%
40%
60%
80%
100%
Fall 2010 Spring 2011 Fall 2011
None or Little
Some
Much
Instructors’ Reports on Course Emphases: Selected Pairings-Writing and Oral Communication
![Page 43: Using IDEA for Assessment, Program Review, and SACS University of Alabama Birmingham September 11, 2012 Shelley A. Chapman, PhD](https://reader035.vdocuments.us/reader035/viewer/2022062716/56649de85503460f94ae2ea9/html5/thumbnails/43.jpg)
Course Emphases: Critical thinking
Fall 2010
Spring 2011
Fall 2011
None or Little 19% 19% 18%Some 49% 44% 41%Much 32% 37% 41%
0%
20%
40%
60%
80%
100%
Fall 2010 Spring 2011 Fall 2011
None or Little
Some
Much
Course Emphases: Writing
Fall 2010
Spring 2011
Fall 2011
None or Little 25% 26% 22%Some 46% 45% 44%Much 29% 30% 34%
0%
20%
40%
60%
80%
100%
Fall 2010 Spring 2011 Fall 2011
None or Little
Some
Much
Instructors’ Reports on Course Emphases: Selected Pairings-Critical Thinking & Writing
![Page 44: Using IDEA for Assessment, Program Review, and SACS University of Alabama Birmingham September 11, 2012 Shelley A. Chapman, PhD](https://reader035.vdocuments.us/reader035/viewer/2022062716/56649de85503460f94ae2ea9/html5/thumbnails/44.jpg)
Highlights for Sample U• Remarkably similar profiles
across terms
• Overall response rates ranged from 66% to 80%
• 1st term in which administration was primarily online achieved a 75% response rate
• Transition from paper to online (fall 2009 to fall 2010) does not show major differences in profiles
• Sample U faculty focus on 4-5 outcomes as essential/important
• Over the last 3 terms, a significant increase on several objectives has been observed: application of course material, oral and written communication skills, & analysis and critical thinking skills (objectives 3, 8, & 11, respectively)
![Page 45: Using IDEA for Assessment, Program Review, and SACS University of Alabama Birmingham September 11, 2012 Shelley A. Chapman, PhD](https://reader035.vdocuments.us/reader035/viewer/2022062716/56649de85503460f94ae2ea9/html5/thumbnails/45.jpg)
BenchmarkingInstitutional and Discipline Reports
![Page 46: Using IDEA for Assessment, Program Review, and SACS University of Alabama Birmingham September 11, 2012 Shelley A. Chapman, PhD](https://reader035.vdocuments.us/reader035/viewer/2022062716/56649de85503460f94ae2ea9/html5/thumbnails/46.jpg)
Benchmarking Reports
Comparison to•6-10 Peers•Same Carnegie Classification•IDEA database
![Page 47: Using IDEA for Assessment, Program Review, and SACS University of Alabama Birmingham September 11, 2012 Shelley A. Chapman, PhD](https://reader035.vdocuments.us/reader035/viewer/2022062716/56649de85503460f94ae2ea9/html5/thumbnails/47.jpg)
Comparison Groups
Your University ---------- ----------
Peer* ----------------------------
Carnegie ---------- ----------
National ---------- ----------
* Peer group is based on 6-10 institutions identified by your institution
![Page 48: Using IDEA for Assessment, Program Review, and SACS University of Alabama Birmingham September 11, 2012 Shelley A. Chapman, PhD](https://reader035.vdocuments.us/reader035/viewer/2022062716/56649de85503460f94ae2ea9/html5/thumbnails/48.jpg)
Benchmarking Reports
The student, rather than the class, is the unit of analysis
Percentage of positive ratings is given rather than averages
![Page 49: Using IDEA for Assessment, Program Review, and SACS University of Alabama Birmingham September 11, 2012 Shelley A. Chapman, PhD](https://reader035.vdocuments.us/reader035/viewer/2022062716/56649de85503460f94ae2ea9/html5/thumbnails/49.jpg)
Response Rates
Your University: Student participation is similar to that of each comparison group
Your University=79%
Peer=77%
Carnegie=79%
National=75%
![Page 50: Using IDEA for Assessment, Program Review, and SACS University of Alabama Birmingham September 11, 2012 Shelley A. Chapman, PhD](https://reader035.vdocuments.us/reader035/viewer/2022062716/56649de85503460f94ae2ea9/html5/thumbnails/50.jpg)
Students’ Perceptions
![Page 51: Using IDEA for Assessment, Program Review, and SACS University of Alabama Birmingham September 11, 2012 Shelley A. Chapman, PhD](https://reader035.vdocuments.us/reader035/viewer/2022062716/56649de85503460f94ae2ea9/html5/thumbnails/51.jpg)
Graduate Students
![Page 52: Using IDEA for Assessment, Program Review, and SACS University of Alabama Birmingham September 11, 2012 Shelley A. Chapman, PhD](https://reader035.vdocuments.us/reader035/viewer/2022062716/56649de85503460f94ae2ea9/html5/thumbnails/52.jpg)
Instructional Objectives Selected by Instructors
Instructors’ Intentions/ focus Students’ Self-Reported Progress on Learning
![Page 53: Using IDEA for Assessment, Program Review, and SACS University of Alabama Birmingham September 11, 2012 Shelley A. Chapman, PhD](https://reader035.vdocuments.us/reader035/viewer/2022062716/56649de85503460f94ae2ea9/html5/thumbnails/53.jpg)
Objective 3: Learning to apply course material (to improve thinking, problem solving, and decisions)
% of total classes where instructor selected objective as “Essential” or “Important”
IDEA Objective 3
Learning to apply course material (to improve thinking, problem solving, and decisions)
![Page 54: Using IDEA for Assessment, Program Review, and SACS University of Alabama Birmingham September 11, 2012 Shelley A. Chapman, PhD](https://reader035.vdocuments.us/reader035/viewer/2022062716/56649de85503460f94ae2ea9/html5/thumbnails/54.jpg)
IDEA Objective 8
Objective 8: Developing skill in expressing oneself orally or in writing
% of total classes where instructor selected objective as “Essential” or “Important”
Developing skill in expressing oneself orally or in writing
![Page 55: Using IDEA for Assessment, Program Review, and SACS University of Alabama Birmingham September 11, 2012 Shelley A. Chapman, PhD](https://reader035.vdocuments.us/reader035/viewer/2022062716/56649de85503460f94ae2ea9/html5/thumbnails/55.jpg)
Teaching Methods and Styles Reported by Students
(Diagnostic Forms Only)
• Fostering Student Collaboration
• Encouraging Student Involvement
![Page 56: Using IDEA for Assessment, Program Review, and SACS University of Alabama Birmingham September 11, 2012 Shelley A. Chapman, PhD](https://reader035.vdocuments.us/reader035/viewer/2022062716/56649de85503460f94ae2ea9/html5/thumbnails/56.jpg)
Benchmarking Look at “General Education”
![Page 57: Using IDEA for Assessment, Program Review, and SACS University of Alabama Birmingham September 11, 2012 Shelley A. Chapman, PhD](https://reader035.vdocuments.us/reader035/viewer/2022062716/56649de85503460f94ae2ea9/html5/thumbnails/57.jpg)
Benchmarking Look at “General Education”
![Page 58: Using IDEA for Assessment, Program Review, and SACS University of Alabama Birmingham September 11, 2012 Shelley A. Chapman, PhD](https://reader035.vdocuments.us/reader035/viewer/2022062716/56649de85503460f94ae2ea9/html5/thumbnails/58.jpg)
Using Aggregate Data for Assessment
UAB
Core
Competencies
Program Learning
Outcomes
Program Learning
Outcomes
Program Learning
OutcomesGroup Summary
Report
Group SummaryReport,
Include Extra Questions
Benchmarking: One Year or 3-5 Year Trend Report
Benchmarking: Discipline Report
Course Learning Outcomes
Course Learning Outcomes
Course Learning
Outcomes
Course Learning Outcomes
Course Learning
Outcomes
![Page 59: Using IDEA for Assessment, Program Review, and SACS University of Alabama Birmingham September 11, 2012 Shelley A. Chapman, PhD](https://reader035.vdocuments.us/reader035/viewer/2022062716/56649de85503460f94ae2ea9/html5/thumbnails/59.jpg)
Accreditation
Guides• SACS
• NCATE
• CACREP
![Page 60: Using IDEA for Assessment, Program Review, and SACS University of Alabama Birmingham September 11, 2012 Shelley A. Chapman, PhD](https://reader035.vdocuments.us/reader035/viewer/2022062716/56649de85503460f94ae2ea9/html5/thumbnails/60.jpg)
Questions?