developing an outcomes assessment program:
DESCRIPTION
Developing an Outcomes Assessment Program:. The CCBC Approach. Dr. Irving Pressley McPhail, Chancellor The Community College of Baltimore County Delgado Community College – February 18, 2005. LearningFIRST 2.0. The Strategic Plan For CCBC FY 2004 to FY 2008. - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Developing an Outcomes Assessment Program:](https://reader035.vdocuments.us/reader035/viewer/2022070502/5681302b550346895d95bbdc/html5/thumbnails/1.jpg)
Developing an Outcomes Assessment Program:
The CCBC Approach Dr. Irving Pressley McPhail, Chancellor
The Community College of Baltimore County
Delgado Community College – February 18, 2005
![Page 2: Developing an Outcomes Assessment Program:](https://reader035.vdocuments.us/reader035/viewer/2022070502/5681302b550346895d95bbdc/html5/thumbnails/2.jpg)
LearningFIRST 2.0
The Strategic Plan For CCBC
FY 2004 to FY 2008
![Page 3: Developing an Outcomes Assessment Program:](https://reader035.vdocuments.us/reader035/viewer/2022070502/5681302b550346895d95bbdc/html5/thumbnails/3.jpg)
StudentLearning
Learning College
InfusingTechnology
LearningSupport
EffectiveCommunication
Organizational Excellence
ValuingDiversity
Community& InstitutionalAdvancement
EnrollmentManagement
Learning First 2.0
![Page 4: Developing an Outcomes Assessment Program:](https://reader035.vdocuments.us/reader035/viewer/2022070502/5681302b550346895d95bbdc/html5/thumbnails/4.jpg)
CCBC: A Vanguard Learning College One of twelve community colleges from across
the U.S. and Canada chosen to participate in the League’s Vanguard Learning College Project
Five areas of concentration: Organizational Culture Staff Recruitment and Development Learning Outcomes Assessment Technology Underprepared Students
![Page 5: Developing an Outcomes Assessment Program:](https://reader035.vdocuments.us/reader035/viewer/2022070502/5681302b550346895d95bbdc/html5/thumbnails/5.jpg)
Middle States Standard 14: Assessment of Student Learning
Assessment of student learningdemonstrates that the institution’sstudents have knowledge, skills, and competencies consistent withinstitutional goals and that students atgraduation have achieved appropriate higher education goals.
![Page 6: Developing an Outcomes Assessment Program:](https://reader035.vdocuments.us/reader035/viewer/2022070502/5681302b550346895d95bbdc/html5/thumbnails/6.jpg)
Middle States Standard 14: Assessment of Student Learning
In order to carry out meaningful assessment activities, institutions must articulate statements of expected student learning at the institutional, program, and individual course levels, although the level of specificity will be greater at the course level. Course syllabi or guidelines should include expected learning outcomes.
![Page 7: Developing an Outcomes Assessment Program:](https://reader035.vdocuments.us/reader035/viewer/2022070502/5681302b550346895d95bbdc/html5/thumbnails/7.jpg)
SACS – Principles of Accreditation
3.3 Institutional Effectiveness 3.3.1 The institution identifies expected
outcomes for its educational programs and its administrative and educational support services; assesses whether it achieves these outcomes; and provides evidence of improvement based on analysis of those results.
![Page 8: Developing an Outcomes Assessment Program:](https://reader035.vdocuments.us/reader035/viewer/2022070502/5681302b550346895d95bbdc/html5/thumbnails/8.jpg)
Council on Innovation and Student Learning
(CISL)
How it all began!!
![Page 9: Developing an Outcomes Assessment Program:](https://reader035.vdocuments.us/reader035/viewer/2022070502/5681302b550346895d95bbdc/html5/thumbnails/9.jpg)
Chancellor’s Charge to CISL
Serve as a college-wide think tank Lead the transformation of the CCBC into
a premier, learning centered college Serve as change agents Help to frame policies, procedures and
infrastructure needed to become a learning college
![Page 10: Developing an Outcomes Assessment Program:](https://reader035.vdocuments.us/reader035/viewer/2022070502/5681302b550346895d95bbdc/html5/thumbnails/10.jpg)
Getting Started - Fall 1998 Council on Innovation and Student
Learning Membership included trustees, chancellor, faculty,
professional staff and classified staff (35-40 members)
Assessment subcommittee created the Learning Outcomes Assessment Plan for the College
(8 members) Guide to Learning Outcomes Assessment and
Classroom Assessment (updated Spring 2003) www.ccbcmd.edu/loa/index.html
![Page 11: Developing an Outcomes Assessment Program:](https://reader035.vdocuments.us/reader035/viewer/2022070502/5681302b550346895d95bbdc/html5/thumbnails/11.jpg)
Measuring Student Learning
![Page 12: Developing an Outcomes Assessment Program:](https://reader035.vdocuments.us/reader035/viewer/2022070502/5681302b550346895d95bbdc/html5/thumbnails/12.jpg)
Measuring Student Learning
Non-measures Student satisfaction
surveys Program evaluation MHEC reporting Student grades Retention rates Graduation rates Transfer rates
Indirect measures Exit interviews of
graduates Employer surveys Transfer studies
![Page 13: Developing an Outcomes Assessment Program:](https://reader035.vdocuments.us/reader035/viewer/2022070502/5681302b550346895d95bbdc/html5/thumbnails/13.jpg)
Measuring Student Learning
Direct measures include: Standardized tests Portfolio assessment Capstone experience Locally developed tests Externally reviewed exhibitions and
performances
![Page 14: Developing an Outcomes Assessment Program:](https://reader035.vdocuments.us/reader035/viewer/2022070502/5681302b550346895d95bbdc/html5/thumbnails/14.jpg)
CCBC’s Learning Outcomes Assessment Program
![Page 15: Developing an Outcomes Assessment Program:](https://reader035.vdocuments.us/reader035/viewer/2022070502/5681302b550346895d95bbdc/html5/thumbnails/15.jpg)
CCBC’s Assessment Plan Principles
Primary reason for assessment is to improve and expand student learning
Development of an effective program is a long-term, dynamic process
Must involve a multi-method approach Must include training and support for faculty
and staff Results are not used punitively for students or
faculty
![Page 16: Developing an Outcomes Assessment Program:](https://reader035.vdocuments.us/reader035/viewer/2022070502/5681302b550346895d95bbdc/html5/thumbnails/16.jpg)
CCBC’s Assessment Plan Principles
Seek to use the most reliable, valid methods and instruments available
Never an end in itself, only a means to an end – the improvement of student learning
Continuous Quality Improvement
DesignImpleme
ntAssess
![Page 17: Developing an Outcomes Assessment Program:](https://reader035.vdocuments.us/reader035/viewer/2022070502/5681302b550346895d95bbdc/html5/thumbnails/17.jpg)
CCBC’s Outcomes AssessmentPhilosophy
Assessment is: A natural and on-going part of instruction Consistent with traditional instructional
practices Designed to meet specific objectives Conducted in a risk-free environment
![Page 18: Developing an Outcomes Assessment Program:](https://reader035.vdocuments.us/reader035/viewer/2022070502/5681302b550346895d95bbdc/html5/thumbnails/18.jpg)
LOA Project Collaboration
Learning Outcomes Associate Planning, Research and Evaluation Liaison Learning Outcomes Assessment Advisory Board Deans’ Council General Education Review Board Developmental Education Advisory Committee
![Page 19: Developing an Outcomes Assessment Program:](https://reader035.vdocuments.us/reader035/viewer/2022070502/5681302b550346895d95bbdc/html5/thumbnails/19.jpg)
LOA Project Collaboration Learning Outcomes Associate
Development of design Survey/tool development Staff development Logistical plan
Institutional Research Data entry Data analysis Data interpretation
![Page 20: Developing an Outcomes Assessment Program:](https://reader035.vdocuments.us/reader035/viewer/2022070502/5681302b550346895d95bbdc/html5/thumbnails/20.jpg)
Types of Designs
Portfolio assessment Standardized tests External graders/experts Pre- and post-tests Cooperation with other schools Creativity Abounds!
![Page 21: Developing an Outcomes Assessment Program:](https://reader035.vdocuments.us/reader035/viewer/2022070502/5681302b550346895d95bbdc/html5/thumbnails/21.jpg)
Project Elements
Stage 1: Design Course Project
Determine Measurable Objectives Select Assessment Instrument Include External Validation Control Important Variables
Develop Full Proposal
![Page 22: Developing an Outcomes Assessment Program:](https://reader035.vdocuments.us/reader035/viewer/2022070502/5681302b550346895d95bbdc/html5/thumbnails/22.jpg)
Stage 2: Implement Design Administer Assessment Instrument(s) Collect Data Analyze Data
Project Elements
![Page 23: Developing an Outcomes Assessment Program:](https://reader035.vdocuments.us/reader035/viewer/2022070502/5681302b550346895d95bbdc/html5/thumbnails/23.jpg)
Stage 3: Test Objectives
Met? > Write Final ReportNot Met? > Design Course Improvements
Write Interim Report
Project Elements
![Page 24: Developing an Outcomes Assessment Program:](https://reader035.vdocuments.us/reader035/viewer/2022070502/5681302b550346895d95bbdc/html5/thumbnails/24.jpg)
Stage 4: Implement Course Improvements Collect Data Analyze Data
Stage 5: Final Report
Discipline/Campus College
Project Elements
![Page 25: Developing an Outcomes Assessment Program:](https://reader035.vdocuments.us/reader035/viewer/2022070502/5681302b550346895d95bbdc/html5/thumbnails/25.jpg)
In Summary
Establish Climate of Continuous Improvement
Empower Faculty Provide Substantial Administrative and
Fiscal Support Share Results - Celebrate Successes
![Page 26: Developing an Outcomes Assessment Program:](https://reader035.vdocuments.us/reader035/viewer/2022070502/5681302b550346895d95bbdc/html5/thumbnails/26.jpg)
Course Level Assessment
![Page 27: Developing an Outcomes Assessment Program:](https://reader035.vdocuments.us/reader035/viewer/2022070502/5681302b550346895d95bbdc/html5/thumbnails/27.jpg)
High Impact LOAs
CINS 101 HLTH 101 PEFT 101 SDEV 101 ENGL 101
MGMT 101 SOCL 101 PHIL 101 BIOL 110 SPCM 101
ARTS 104 ENVS 101 PSYC 105 MATH 082
FY ‘04FY ‘03 FY ‘05
![Page 28: Developing an Outcomes Assessment Program:](https://reader035.vdocuments.us/reader035/viewer/2022070502/5681302b550346895d95bbdc/html5/thumbnails/28.jpg)
HLTH 101: Health and Wellness High impact project Pre-test/post-test design 100 item, four option multiple choice format exam Topics included subjects commonly covered in
introductory health textbooks Draft instrument was reviewed by a health/curriculum
expert at a four-year university Two and four-year colleges and universities have been
invited to share our instrument so that we might develop external comparisons
![Page 29: Developing an Outcomes Assessment Program:](https://reader035.vdocuments.us/reader035/viewer/2022070502/5681302b550346895d95bbdc/html5/thumbnails/29.jpg)
Health 101 LOAResults Matched pair t-test analysis Students’ scores improved significantly from pre-
to post-test At least 75% of students scored a 75% or higher
on the post-test Scores varied by campus; small difference in
scores based on race Reassessment will be conducted after
recommendations have been implemented
![Page 30: Developing an Outcomes Assessment Program:](https://reader035.vdocuments.us/reader035/viewer/2022070502/5681302b550346895d95bbdc/html5/thumbnails/30.jpg)
Health 101 LOA Recommendations
Three topic areas were identified as needing improvement: nutrition, heart disease, and human sexuality
The campus with students who consistently scored higher on the pre-test will offer more honors’ sections
A faculty guide, “Strategies for Teaching Health Education” has been developed and shared with full-and part-time faculty
![Page 31: Developing an Outcomes Assessment Program:](https://reader035.vdocuments.us/reader035/viewer/2022070502/5681302b550346895d95bbdc/html5/thumbnails/31.jpg)
RDNG 052: College Reading
Instruments Used the Nelson-Denny Reading Test (Forms G
and H) pre and post for the 2001 and the 2003 assessments
Used the Learning Attitudes Study Strategies Inventory (LASSI) for the two semesters of pilot projects and the 2001 assessment
Design Pre-post project design Matched pair data analysis
![Page 32: Developing an Outcomes Assessment Program:](https://reader035.vdocuments.us/reader035/viewer/2022070502/5681302b550346895d95bbdc/html5/thumbnails/32.jpg)
RDNG 052: College Reading - Fall 2001 Results
Mean pre-post difference of 3.5 (raw points) was statistically significant at the .001 level
Post-test score was greater than pre-test score in 74% of cases
Resulting grade level increase of .7 grade level (in a 15 week semester)
![Page 33: Developing an Outcomes Assessment Program:](https://reader035.vdocuments.us/reader035/viewer/2022070502/5681302b550346895d95bbdc/html5/thumbnails/33.jpg)
RDNG 052: College Reading - Fall 2001 Results
Exiting students read at 9.6 mean grade level
Differences among campuses—learned from best practices on each campus
LASSI results showed significant improvement in 10 of 12 scales from pre to post
![Page 34: Developing an Outcomes Assessment Program:](https://reader035.vdocuments.us/reader035/viewer/2022070502/5681302b550346895d95bbdc/html5/thumbnails/34.jpg)
RDNG 052: College Reading – Fall 2001 Interventions
Interventions included: Using more Internet-based learning
opportunities Enhancing instructional activities that focused
on literal and inferential comprehension Greater focus on working with adjunct faculty
![Page 35: Developing an Outcomes Assessment Program:](https://reader035.vdocuments.us/reader035/viewer/2022070502/5681302b550346895d95bbdc/html5/thumbnails/35.jpg)
RDNG 052: College Reading – Fall 2003 Results Mean pre-post difference of 3.7 was statistically
significant at the .001 level Post-test score was greater than pre-test score
in 75% of cases (larger sample) Resulting grade level increase of 2.0! (in a 15
week semester) Exiting students read at 10.1 mean grade level Differences among campuses; every campus
improved
![Page 36: Developing an Outcomes Assessment Program:](https://reader035.vdocuments.us/reader035/viewer/2022070502/5681302b550346895d95bbdc/html5/thumbnails/36.jpg)
RDNG 052 - 2001 to 2003: Improvements in Student Learning
Significant improvements in students’ reading levels from 2001 to 2003 indicate increased learning in RDNG 052.
![Page 37: Developing an Outcomes Assessment Program:](https://reader035.vdocuments.us/reader035/viewer/2022070502/5681302b550346895d95bbdc/html5/thumbnails/37.jpg)
RDNG 052 College Reading - Final Recommendations The Cycle of Continuous Improvement
Research Accuplacer cut scores between RDNG 051 and RDNG 052
Continue to close the performance gap Continue to share best practices Continue to study post-Reading success rates
(success in subsequent courses)
![Page 38: Developing an Outcomes Assessment Program:](https://reader035.vdocuments.us/reader035/viewer/2022070502/5681302b550346895d95bbdc/html5/thumbnails/38.jpg)
Making the Transition from Course to Program Level Assessment
Program Review General Education Developmental Education
![Page 39: Developing an Outcomes Assessment Program:](https://reader035.vdocuments.us/reader035/viewer/2022070502/5681302b550346895d95bbdc/html5/thumbnails/39.jpg)
General Education
The Academic ProfileAnd
The “GREATs”
![Page 40: Developing an Outcomes Assessment Program:](https://reader035.vdocuments.us/reader035/viewer/2022070502/5681302b550346895d95bbdc/html5/thumbnails/40.jpg)
The Academic Profile
Norm-referenced, externally developed test of general knowledge
Widely used Sample size: 1,017 Students Purpose - to establish baseline data upon
implementation of new General Education Program in Fall 2001
![Page 41: Developing an Outcomes Assessment Program:](https://reader035.vdocuments.us/reader035/viewer/2022070502/5681302b550346895d95bbdc/html5/thumbnails/41.jpg)
Academic Profile - Findings The Academic Profile provides CCBC with a
baseline measure of how our students are acquiring academic skills developed through General Education courses compared to national norms. CCBC students scored at or close to the national
norms on the total mean score and all sub-scores the first time the test was administered.
The Academic Profile will be administered again in Fall 2004 (sample size 2,000+).
![Page 42: Developing an Outcomes Assessment Program:](https://reader035.vdocuments.us/reader035/viewer/2022070502/5681302b550346895d95bbdc/html5/thumbnails/42.jpg)
Academic Profile –Data Implications
Work on Critical Thinking skills Reinforce skills learned in one class in
other classes Provide Culturally Mediated Instruction Possible statewide initiatives
![Page 43: Developing an Outcomes Assessment Program:](https://reader035.vdocuments.us/reader035/viewer/2022070502/5681302b550346895d95bbdc/html5/thumbnails/43.jpg)
GREAT Projects General Education Assessment Teams (GREAT) Projects for all
Maryland Higher Education Commission general education categories
Common Graded Assignments (CGA) and accompanying Scoring Rubrics Created by faculty teams
External Consultant Faculty Training Three semesters of Pilot Projects Full implementation began in Fall 2003 Every course assessed once every three years
![Page 44: Developing an Outcomes Assessment Program:](https://reader035.vdocuments.us/reader035/viewer/2022070502/5681302b550346895d95bbdc/html5/thumbnails/44.jpg)
GREAT Data (Fall, 2003) Implications Increased awareness of faculty regarding how
General Education courses are defined by the six criteria
Midrange to higher scores in Content may indicate a traditional approach in how “Content” is conveyed with little expectation for higher level use of content in application, critical thinking, analysis, and synthesis activities
Workshops on how to use the GREAT data to improve student learning
![Page 45: Developing an Outcomes Assessment Program:](https://reader035.vdocuments.us/reader035/viewer/2022070502/5681302b550346895d95bbdc/html5/thumbnails/45.jpg)
LOA Successes Creating a “culture of assessment” with
increased faculty participation and buy-in Using outcomes assessment for Program-level
assessment Forging new partnerships between faculty teams,
institutional research staff, the Vice Chancellor for Learning and Student Development’s office, and the Outcomes Associate in analyzing data and making curricular and pedagogical recommendations for change
![Page 46: Developing an Outcomes Assessment Program:](https://reader035.vdocuments.us/reader035/viewer/2022070502/5681302b550346895d95bbdc/html5/thumbnails/46.jpg)
LOA Successes Creating a newly updated Guide for Learning
Outcomes Assessment and Classroom Learning Assessment, a model guide for assessment
www.ccbcmd.edu/loa/index.html Establishing the Learning Outcomes
Assessment Advisory Board Recognition by the League for Innovation in the
Community College and Middle States Commission on Higher Education as a national leader in Learning Outcomes Assessment
![Page 47: Developing an Outcomes Assessment Program:](https://reader035.vdocuments.us/reader035/viewer/2022070502/5681302b550346895d95bbdc/html5/thumbnails/47.jpg)
![Page 48: Developing an Outcomes Assessment Program:](https://reader035.vdocuments.us/reader035/viewer/2022070502/5681302b550346895d95bbdc/html5/thumbnails/48.jpg)
For Further Information:
Dr. Irving Pressley McPhailChancellorThe Community College of Baltimore County800 South Rolling RoadBaltimore, Maryland 21228-5317Telephone: 410-869-1220Fax: 410-869-1224Email: [email protected]: www.ccbcmd.edu
![Page 49: Developing an Outcomes Assessment Program:](https://reader035.vdocuments.us/reader035/viewer/2022070502/5681302b550346895d95bbdc/html5/thumbnails/49.jpg)
For Further Information:
Dr. Alvin Starr, Acting Vice Chancellor for Learning and Student Development
Email: [email protected]
Dr. Rose Mince, Assistant to the VCLSDEmail: [email protected]
Professor Tara Ebersole, Outcomes AssociateEmail: [email protected]
![Page 50: Developing an Outcomes Assessment Program:](https://reader035.vdocuments.us/reader035/viewer/2022070502/5681302b550346895d95bbdc/html5/thumbnails/50.jpg)
Questions /Comments