![Page 1: ASSESSMENT LEARNING ASSESSMENT COUNCIL. SAC ASSESSMENT REPORTS Submission rate holding steady at approximately 89%](https://reader035.vdocuments.us/reader035/viewer/2022070413/5697bfce1a28abf838ca9925/html5/thumbnails/1.jpg)
ASSESSMENT
L EA
RN
I NG
AS
SE
SS
ME
NT
CO
UN
CI L
![Page 2: ASSESSMENT LEARNING ASSESSMENT COUNCIL. SAC ASSESSMENT REPORTS Submission rate holding steady at approximately 89%](https://reader035.vdocuments.us/reader035/viewer/2022070413/5697bfce1a28abf838ca9925/html5/thumbnails/2.jpg)
SAC ASSESSMENT REPORTS
2010
2011
2012
2013
2014
0%
20%
40%60%
80%100%
% Submissions
% Submissions
Submission rate holding steady at approximately 89%
![Page 3: ASSESSMENT LEARNING ASSESSMENT COUNCIL. SAC ASSESSMENT REPORTS Submission rate holding steady at approximately 89%](https://reader035.vdocuments.us/reader035/viewer/2022070413/5697bfce1a28abf838ca9925/html5/thumbnails/3.jpg)
LAST YEAR’S ANNUAL REPORTING
Option 1 (new format) Option 2 (old format)
Checkboxes, limited narrative
Narrative
1 outcome per report More than 1 outcome per report
More structure/guidance Minimal structure/guidance
• 2 Reports (2 deadlines): Annual Plan (AP) and End-of-Year (EOY)
• Multi-Year Plan• Two options for report format
• Rationale:• Long reports & critical info still missing• Overall Picture
• Mid-Year feedback (31 respondents; 30 used new format) 77% like 10% unsure 13% did not like
![Page 4: ASSESSMENT LEARNING ASSESSMENT COUNCIL. SAC ASSESSMENT REPORTS Submission rate holding steady at approximately 89%](https://reader035.vdocuments.us/reader035/viewer/2022070413/5697bfce1a28abf838ca9925/html5/thumbnails/4.jpg)
SUMMER PEER REVIEW
Participants
• 12 CTE reviewers (1 floater)
• 10 LDC reviewers (3 floaters)
• 9 LAC coaches
![Page 5: ASSESSMENT LEARNING ASSESSMENT COUNCIL. SAC ASSESSMENT REPORTS Submission rate holding steady at approximately 89%](https://reader035.vdocuments.us/reader035/viewer/2022070413/5697bfce1a28abf838ca9925/html5/thumbnails/5.jpg)
SUMMER PEER REVIEW
Changes
• Reviewers used a checklist rather than a rubric
• Subsection and Global scores given to all reports
• Assessment data were quantitatively analyzed this year
• Awards now given in a single category: Exemplary Assessment•Eligibility determined by the global scores
![Page 6: ASSESSMENT LEARNING ASSESSMENT COUNCIL. SAC ASSESSMENT REPORTS Submission rate holding steady at approximately 89%](https://reader035.vdocuments.us/reader035/viewer/2022070413/5697bfce1a28abf838ca9925/html5/thumbnails/6.jpg)
LAC A
SSESSMENT
PROJE
CTS
20
13
- 20
14
FI N
DI N
GS
![Page 7: ASSESSMENT LEARNING ASSESSMENT COUNCIL. SAC ASSESSMENT REPORTS Submission rate holding steady at approximately 89%](https://reader035.vdocuments.us/reader035/viewer/2022070413/5697bfce1a28abf838ca9925/html5/thumbnails/7.jpg)
2013-2014 FINDINGS
2013-2014 is the first year that LAC has conducted comprehensive quantitative analyses:
1.Meta-analysis of outcome attainment dataStatistical technique that integrates data from multiple studies
2.Analysis of peer review data
This year’s analyses were proof-of-concept/pilot studies.
![Page 8: ASSESSMENT LEARNING ASSESSMENT COUNCIL. SAC ASSESSMENT REPORTS Submission rate holding steady at approximately 89%](https://reader035.vdocuments.us/reader035/viewer/2022070413/5697bfce1a28abf838ca9925/html5/thumbnails/8.jpg)
OUTCOME ATTAINMENT DATA
The fundamental measure in educational assessment is the outcome attainment rate: how many of them ‘got it’ at the level we want them to?
![Page 9: ASSESSMENT LEARNING ASSESSMENT COUNCIL. SAC ASSESSMENT REPORTS Submission rate holding steady at approximately 89%](https://reader035.vdocuments.us/reader035/viewer/2022070413/5697bfce1a28abf838ca9925/html5/thumbnails/9.jpg)
OUTCOME ATTAINMENT DATA
LDC/DE Outcome Attainment Data
Reported Not Reported
CTE Outcome Attainment Data
Reported Not Reported
32/80 38/100
39% of SAC reports include attainment rates for identified benchmark levels of performance.
![Page 10: ASSESSMENT LEARNING ASSESSMENT COUNCIL. SAC ASSESSMENT REPORTS Submission rate holding steady at approximately 89%](https://reader035.vdocuments.us/reader035/viewer/2022070413/5697bfce1a28abf838ca9925/html5/thumbnails/10.jpg)
OUTCOME ATTAINMENT DATA
LDC/DE Outcome Attainment Data
Reported Not Reported
CTE Outcome Attainment Data
Reported Not Reported
32/80 38/100
2014-2015 Opportunity for Improvement: SACs identify benchmark/desired performance levels.
![Page 11: ASSESSMENT LEARNING ASSESSMENT COUNCIL. SAC ASSESSMENT REPORTS Submission rate holding steady at approximately 89%](https://reader035.vdocuments.us/reader035/viewer/2022070413/5697bfce1a28abf838ca9925/html5/thumbnails/11.jpg)
OUTCOME ATTAINMENT DATA
Cautions about interpreting the attainment rate data:
• These rates cannot be definitively interpreted as ‘official’ attainment rates, because, e.g.:• Some SACs chose difficult-to-attain outcomes; others
did not;• We cannot be sure how representative the report data
are; etc.
View attainment rates as very rough, tentative estimates only.
![Page 12: ASSESSMENT LEARNING ASSESSMENT COUNCIL. SAC ASSESSMENT REPORTS Submission rate holding steady at approximately 89%](https://reader035.vdocuments.us/reader035/viewer/2022070413/5697bfce1a28abf838ca9925/html5/thumbnails/12.jpg)
OUTCOME ATTAINMENT DATA
Estimated overall outcome attainment rates:
• CTE – 82-90%
• LDC/DE – 70-80%
These rates are typical for similar institutions/programs.
![Page 13: ASSESSMENT LEARNING ASSESSMENT COUNCIL. SAC ASSESSMENT REPORTS Submission rate holding steady at approximately 89%](https://reader035.vdocuments.us/reader035/viewer/2022070413/5697bfce1a28abf838ca9925/html5/thumbnails/13.jpg)
OUTCOME ATTAINMENT DATA
Some moderator variables (variables that influence the strength/direction of effects) were significant in the LDC/DE, but not in the CTE data.
![Page 14: ASSESSMENT LEARNING ASSESSMENT COUNCIL. SAC ASSESSMENT REPORTS Submission rate holding steady at approximately 89%](https://reader035.vdocuments.us/reader035/viewer/2022070413/5697bfce1a28abf838ca9925/html5/thumbnails/14.jpg)
OUTCOME ATTAINMENT DATA – LDC/DE
Assessments requiring associate’s degree level work had lower student attainment rates:
• Associate’s Degree Level: 66% (between 58-73%)
• Other levels: 79% (between 73-85%)
(using the Degree Qualifications Profile standards)
![Page 15: ASSESSMENT LEARNING ASSESSMENT COUNCIL. SAC ASSESSMENT REPORTS Submission rate holding steady at approximately 89%](https://reader035.vdocuments.us/reader035/viewer/2022070413/5697bfce1a28abf838ca9925/html5/thumbnails/15.jpg)
Associate's Level
Other55.00%
60.00%
65.00%
70.00%
75.00%
80.00%
85.00%
Benchmark Attainment
Benchmark At-tainment
Opportunity: Increase student attainment rates for ‘more difficult’ outcomes.
![Page 16: ASSESSMENT LEARNING ASSESSMENT COUNCIL. SAC ASSESSMENT REPORTS Submission rate holding steady at approximately 89%](https://reader035.vdocuments.us/reader035/viewer/2022070413/5697bfce1a28abf838ca9925/html5/thumbnails/16.jpg)
CRITICAL THINKING ATTAINMENT (LDC/DE)Attainment rates for Critical Thinking and Problem Solving were lower (71%) than those for the other core outcomes (79%).
Not an uncommon finding in higher education
![Page 17: ASSESSMENT LEARNING ASSESSMENT COUNCIL. SAC ASSESSMENT REPORTS Submission rate holding steady at approximately 89%](https://reader035.vdocuments.us/reader035/viewer/2022070413/5697bfce1a28abf838ca9925/html5/thumbnails/17.jpg)
RECOMMENDATIONS FROM THE META-ANALYSIS
1. CTE & LDC/DE: establish SAC agreed-upon benchmarks/expectations for acceptable performance.
2. LDC/DE: SACs consider/explore associate’s level outcomes and assessment activities.
3. LDC/DE: consider ways to improve students’ success with critical thinking.
![Page 18: ASSESSMENT LEARNING ASSESSMENT COUNCIL. SAC ASSESSMENT REPORTS Submission rate holding steady at approximately 89%](https://reader035.vdocuments.us/reader035/viewer/2022070413/5697bfce1a28abf838ca9925/html5/thumbnails/18.jpg)
PEER REVIEW DATA – OVER-ALL SCORES
C T E – 2 1 P O S S I B L E
• 16-21 typical
• 21 higher
• 15 lower
L D C / D E – 1 8 P O S S I B L E
• 14-17 typical
• 18 higher
• 13 lower
Interquartile Range
![Page 19: ASSESSMENT LEARNING ASSESSMENT COUNCIL. SAC ASSESSMENT REPORTS Submission rate holding steady at approximately 89%](https://reader035.vdocuments.us/reader035/viewer/2022070413/5697bfce1a28abf838ca9925/html5/thumbnails/19.jpg)
PEER REVIEW DATA – SUBSECTION SCORES• About ½ of all reports attained the
highest marks on any particular subsection
• Two subsections were less strong institution-wide:
![Page 20: ASSESSMENT LEARNING ASSESSMENT COUNCIL. SAC ASSESSMENT REPORTS Submission rate holding steady at approximately 89%](https://reader035.vdocuments.us/reader035/viewer/2022070413/5697bfce1a28abf838ca9925/html5/thumbnails/20.jpg)
PEER REVIEW DATA – SUBSECTION SCORES• About ½ of all reports attained the
highest marks on any particular subsection
• Two subsections were less strong institution-wide:•SAC Participation/Coordination –these scores indicate the degree to which SAC coordination was demonstrated.
![Page 21: ASSESSMENT LEARNING ASSESSMENT COUNCIL. SAC ASSESSMENT REPORTS Submission rate holding steady at approximately 89%](https://reader035.vdocuments.us/reader035/viewer/2022070413/5697bfce1a28abf838ca9925/html5/thumbnails/21.jpg)
PEER REVIEW DATA – SUBSECTION SCORES• About ½ of all reports attained the
highest marks on any particular subsection
• Two subsections were less strong institution-wide:•Assessment Process – these scores indicate the degree to which assessments demonstrated technical competencies that help to ensure valid, reliable results.
![Page 22: ASSESSMENT LEARNING ASSESSMENT COUNCIL. SAC ASSESSMENT REPORTS Submission rate holding steady at approximately 89%](https://reader035.vdocuments.us/reader035/viewer/2022070413/5697bfce1a28abf838ca9925/html5/thumbnails/22.jpg)
IMPROVING ASSESSMENT PROJECTS - SACS• Peer Review Feedback has been posted in
each SAC’s Spaces page
• A cover letter has been sent to all identified SAC chairs and assessment project coordinators that provides more detailed information that will help you interpret and put your specific results in context
• Contact Michele to arrange coaching help for problem areas ([email protected]; x4786)
![Page 23: ASSESSMENT LEARNING ASSESSMENT COUNCIL. SAC ASSESSMENT REPORTS Submission rate holding steady at approximately 89%](https://reader035.vdocuments.us/reader035/viewer/2022070413/5697bfce1a28abf838ca9925/html5/thumbnails/23.jpg)
IMPROVING ASSESSMENT PROJECTS AT PCC
S A C C O O R D I N A T I O N
• When assessing a course, have all instructors submit randomly selected artifacts (or, with 5 or more instructors, at least 80% participate)
• Involve FT and PT instructors in rating artifacts
A S S E S S M E N T P R O C E S S E S
• When using a rubric, include a norming session and attend to inter-rater agreement
![Page 24: ASSESSMENT LEARNING ASSESSMENT COUNCIL. SAC ASSESSMENT REPORTS Submission rate holding steady at approximately 89%](https://reader035.vdocuments.us/reader035/viewer/2022070413/5697bfce1a28abf838ca9925/html5/thumbnails/24.jpg)
SAC COORDINATION - OUR MODEL
There is significant variation betweenfaculty members in how core andcourse outcomes are intentionally included in their courses.
INDIVIDUAL INSTRUCTORS
![Page 25: ASSESSMENT LEARNING ASSESSMENT COUNCIL. SAC ASSESSMENT REPORTS Submission rate holding steady at approximately 89%](https://reader035.vdocuments.us/reader035/viewer/2022070413/5697bfce1a28abf838ca9925/html5/thumbnails/25.jpg)
SAC COORDINATION - OUR MODEL
At PCC, SACs are charged withdeveloping course outcomes andwith determining the expectations forhow the college core outcomes areaddressed and assessed in their courses.
![Page 26: ASSESSMENT LEARNING ASSESSMENT COUNCIL. SAC ASSESSMENT REPORTS Submission rate holding steady at approximately 89%](https://reader035.vdocuments.us/reader035/viewer/2022070413/5697bfce1a28abf838ca9925/html5/thumbnails/26.jpg)
Shared/coordinated components vary in weight and emphasisin different SACs, courses, programs, etc.
![Page 27: ASSESSMENT LEARNING ASSESSMENT COUNCIL. SAC ASSESSMENT REPORTS Submission rate holding steady at approximately 89%](https://reader035.vdocuments.us/reader035/viewer/2022070413/5697bfce1a28abf838ca9925/html5/thumbnails/27.jpg)
Shared/coordinated components vary in weight and emphasisin different SACs, courses, programs, etc.
![Page 28: ASSESSMENT LEARNING ASSESSMENT COUNCIL. SAC ASSESSMENT REPORTS Submission rate holding steady at approximately 89%](https://reader035.vdocuments.us/reader035/viewer/2022070413/5697bfce1a28abf838ca9925/html5/thumbnails/28.jpg)
SAC COORDINATION - OUR MODEL
Testimonials
• Levi Query & Delia McQueen – PE
• Kendra Cawley – Dark Side
![Page 29: ASSESSMENT LEARNING ASSESSMENT COUNCIL. SAC ASSESSMENT REPORTS Submission rate holding steady at approximately 89%](https://reader035.vdocuments.us/reader035/viewer/2022070413/5697bfce1a28abf838ca9925/html5/thumbnails/29.jpg)
IMPROVING ASSESSMENT PROJECTS AT PCCAveraging-Away Meaning
Rubric (scale 1-4; 4 best; benchmark “3”)
Rubric AVG: 2.5
Row/Criteria #1 AVG: 2.52 Row/Criteria #2 AVG: 2.48
50 samples of student work
4 3Benchmark
2 1
Criterion #1
6 27 4 13
Criterion #2
10 7 30 3
![Page 30: ASSESSMENT LEARNING ASSESSMENT COUNCIL. SAC ASSESSMENT REPORTS Submission rate holding steady at approximately 89%](https://reader035.vdocuments.us/reader035/viewer/2022070413/5697bfce1a28abf838ca9925/html5/thumbnails/30.jpg)
ASSESSMENT SUPP
ORTS
20
13
- 20
14
![Page 31: ASSESSMENT LEARNING ASSESSMENT COUNCIL. SAC ASSESSMENT REPORTS Submission rate holding steady at approximately 89%](https://reader035.vdocuments.us/reader035/viewer/2022070413/5697bfce1a28abf838ca9925/html5/thumbnails/31.jpg)
LAC COACHESSAC support
• Peer Review Feedback•General (templates, sample size, norming)•Review annual reports •Advocate for Peer Review•Available Resources•Communication Channel (concerns/issues/successes)• (Possible facilitation)
Improvements noticed by admin!•Assessment Section of Program Review
![Page 32: ASSESSMENT LEARNING ASSESSMENT COUNCIL. SAC ASSESSMENT REPORTS Submission rate holding steady at approximately 89%](https://reader035.vdocuments.us/reader035/viewer/2022070413/5697bfce1a28abf838ca9925/html5/thumbnails/32.jpg)
LAC COACHING - CHANGES 2014-2015• More coach training
• Coach-led TLC assessment sessions
• 5 returning, 3 new coaches
• Coach Assignments: Contact Michele Marden [email protected]; x4786
Jamee Kristen (SOC) Sally Earll (Curriculum Coordinator)
Laura Sanders (ENG/WR) Shirlee Geiger (PHL, prior LAC Chair)
Michael Hunter-Bernstein (ABE, ED)
Susan Wilson (Acad. Support Coordinator)
Nora Stevens (BI, HIM) Torie Scott (LIB)
![Page 33: ASSESSMENT LEARNING ASSESSMENT COUNCIL. SAC ASSESSMENT REPORTS Submission rate holding steady at approximately 89%](https://reader035.vdocuments.us/reader035/viewer/2022070413/5697bfce1a28abf838ca9925/html5/thumbnails/33.jpg)
FACULTY ASSESSMENT CLASS – ASSESSING CORE OUTCOMES (CEU CREDIT) • For CTE & LDC/DE , PT/FT
• Fridays from 10-12
• Cascade Campus
• Cost covered as a PCC benefit
• Still some seats available
• Contact Wayne Hooke with questions [email protected]
![Page 34: ASSESSMENT LEARNING ASSESSMENT COUNCIL. SAC ASSESSMENT REPORTS Submission rate holding steady at approximately 89%](https://reader035.vdocuments.us/reader035/viewer/2022070413/5697bfce1a28abf838ca9925/html5/thumbnails/34.jpg)
DOI ASSESSMENT SUPPORT
• 10 hours per year per SAC for assessment work done by PT faculty
• Contact your SAC Administrative Liaison to arrange this
![Page 35: ASSESSMENT LEARNING ASSESSMENT COUNCIL. SAC ASSESSMENT REPORTS Submission rate holding steady at approximately 89%](https://reader035.vdocuments.us/reader035/viewer/2022070413/5697bfce1a28abf838ca9925/html5/thumbnails/35.jpg)
PCC’S CENTER FOR CAREERS IN EDUCATION• Winter term: Embedding and Assessing
Cultural Awareness
• Spring term: Assessing Critical Thinking and Problem Solving
![Page 36: ASSESSMENT LEARNING ASSESSMENT COUNCIL. SAC ASSESSMENT REPORTS Submission rate holding steady at approximately 89%](https://reader035.vdocuments.us/reader035/viewer/2022070413/5697bfce1a28abf838ca9925/html5/thumbnails/36.jpg)
CALENDAR/DUE DATES
• 2 annual plans (first sections of report) due Nov 7, 2014
• Changes to multi-year plan due Nov 7, 2014 (no changes – no reporting)
• 2 end-of-year reports (completed form) due June 19, 2015
![Page 37: ASSESSMENT LEARNING ASSESSMENT COUNCIL. SAC ASSESSMENT REPORTS Submission rate holding steady at approximately 89%](https://reader035.vdocuments.us/reader035/viewer/2022070413/5697bfce1a28abf838ca9925/html5/thumbnails/37.jpg)
EXPECT MORE CHANGES
• We are still seeing dynamic changes in assessment nationally, regionally, and locally
• Assessment related areas are being reported-on this year to NWCCU
• Expect more changes….
![Page 38: ASSESSMENT LEARNING ASSESSMENT COUNCIL. SAC ASSESSMENT REPORTS Submission rate holding steady at approximately 89%](https://reader035.vdocuments.us/reader035/viewer/2022070413/5697bfce1a28abf838ca9925/html5/thumbnails/38.jpg)
CONTACT INFORMATION
Website: www.pcc.edu/assessment
LAC email: [email protected]
Wayne Hooke, Chair ([email protected])
Michele Marden, Vice-Chair ([email protected])