accountability reporting for california community colleges
DESCRIPTION
Accountability Reporting for California Community Colleges. Patrick Perry Vice Chancellor of Technology, Research, & Info. Systems CCC Chancellors Office. Data Preamble. “Information is the currency of democracy.” -Thomas Jefferson - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Accountability Reporting for California Community Colleges](https://reader035.vdocuments.us/reader035/viewer/2022062519/5681513e550346895dbf5811/html5/thumbnails/1.jpg)
1
Accountability Reporting for California Community
CollegesPatrick Perry
Vice Chancellor of Technology, Research, &
Info. SystemsCCC Chancellors Office
![Page 2: Accountability Reporting for California Community Colleges](https://reader035.vdocuments.us/reader035/viewer/2022062519/5681513e550346895dbf5811/html5/thumbnails/2.jpg)
2
Data Preamble
“Information is the currency of democracy.” -Thomas Jefferson
“Get your facts first, then you can distort them as you please.” -Mark Twain
“In the twenty-first century, whoever controls the screen controls consciousness, information and thought.” -Timothy Leary
![Page 3: Accountability Reporting for California Community Colleges](https://reader035.vdocuments.us/reader035/viewer/2022062519/5681513e550346895dbf5811/html5/thumbnails/3.jpg)
3
The CCC System
109 campuses, 72 districts, all locally governed
2.6 million students (annual unduplicated) 1.1 million FTES (annual) 35% white; half over age 25; 70% part-time No admissions requirements $20/unit; 40% get fees waived Highest participation rate of any CC system
in US; 25% of all CC students are CCC
![Page 4: Accountability Reporting for California Community Colleges](https://reader035.vdocuments.us/reader035/viewer/2022062519/5681513e550346895dbf5811/html5/thumbnails/4.jpg)
4
Participation (and Fees)
![Page 5: Accountability Reporting for California Community Colleges](https://reader035.vdocuments.us/reader035/viewer/2022062519/5681513e550346895dbf5811/html5/thumbnails/5.jpg)
5
CCC Chancellor’s Office
Weak authority; powers vested locally
Unitary MIS data collection (1992) Student, faculty, course, section,
session, grade level detail Data collected end of term, 3x/yr Used for IPEDS, apportionment,
accountability, research, online data mart
![Page 6: Accountability Reporting for California Community Colleges](https://reader035.vdocuments.us/reader035/viewer/2022062519/5681513e550346895dbf5811/html5/thumbnails/6.jpg)
6
History of CCC Accountability
Simple reporting, fact books until 1998 1998: State provides $300m ongoing in
exchange for accountability reporting “Partnership for Excellence” was born
CCC developed report in isolation CCC allowed to determine “adequate progress” “Contingent funding” never triggered
Used 5 metrics to measure system and college-level performance
![Page 7: Accountability Reporting for California Community Colleges](https://reader035.vdocuments.us/reader035/viewer/2022062519/5681513e550346895dbf5811/html5/thumbnails/7.jpg)
7
PFE Metrics
Annual volume of transfers to CSU/UC Annual volume of awards/certificates Rate of successful course completions Annual volume of Voc. Ed. Course
completions Annual volume of basic skills
improvements (lower to higher level) 4 of 5 are volume metrics, only 1 rate
![Page 8: Accountability Reporting for California Community Colleges](https://reader035.vdocuments.us/reader035/viewer/2022062519/5681513e550346895dbf5811/html5/thumbnails/8.jpg)
8
The State Said:
Your metrics allow for no adequate college comparisons
Your method of determining “adequate progress” is suspicious
You only look good because you are growing
Partnership over (2001), but keep reporting, (until 2004) we have to spend your money buying energy
from Enron
![Page 9: Accountability Reporting for California Community Colleges](https://reader035.vdocuments.us/reader035/viewer/2022062519/5681513e550346895dbf5811/html5/thumbnails/9.jpg)
9
What Happened Next
Gov. Gray Davis: recalled for spending money buying energy from Enron
Replaced by “The Governator”
![Page 10: Accountability Reporting for California Community Colleges](https://reader035.vdocuments.us/reader035/viewer/2022062519/5681513e550346895dbf5811/html5/thumbnails/10.jpg)
10
The Governator
Likes Community Colleges Comes from a country that has
European “academic bifurcation” (Austria)-university vs trade paths
Attended Santa Monica Community College
Took ESL, PE, bookkeeping, micro/macroeconomics
Transferred to U. Wisconsin-Superior
![Page 11: Accountability Reporting for California Community Colleges](https://reader035.vdocuments.us/reader035/viewer/2022062519/5681513e550346895dbf5811/html5/thumbnails/11.jpg)
11
And Arnold Said:
We shall haves deez accountabeelity seeztem for de community collegez.
A bill was passed to create the framework, and eventually the framework was enacted.
Named: Accountability Reporting for Community Colleges (ARCC).
![Page 12: Accountability Reporting for California Community Colleges](https://reader035.vdocuments.us/reader035/viewer/2022062519/5681513e550346895dbf5811/html5/thumbnails/12.jpg)
12
Arnold Said:
There shall be no pay for performance, but there will be the ability to compare performance.
![Page 13: Accountability Reporting for California Community Colleges](https://reader035.vdocuments.us/reader035/viewer/2022062519/5681513e550346895dbf5811/html5/thumbnails/13.jpg)
13
We Said:
Some metrics will be system only; others will be at college-level
College metrics will be rates (to mitigate size for comparison)
No rankings—we will compare colleges against their “peers”
No $$$=ARCC is a “dashboard” accountability report.
![Page 14: Accountability Reporting for California Community Colleges](https://reader035.vdocuments.us/reader035/viewer/2022062519/5681513e550346895dbf5811/html5/thumbnails/14.jpg)
14
Arnold Said:
Colleges need to address their performance annually to the State.
![Page 15: Accountability Reporting for California Community Colleges](https://reader035.vdocuments.us/reader035/viewer/2022062519/5681513e550346895dbf5811/html5/thumbnails/15.jpg)
15
We Said:
Colleges are more responsive to their local district Board; annual requirement to take local ARCC results to local Board and submit minutes to State
Colleges must submit 500 word response, which becomes a part of the final report.
![Page 16: Accountability Reporting for California Community Colleges](https://reader035.vdocuments.us/reader035/viewer/2022062519/5681513e550346895dbf5811/html5/thumbnails/16.jpg)
16
Arnold Said:
The report shall be done in collaboration with the State, not in isolation.
![Page 17: Accountability Reporting for California Community Colleges](https://reader035.vdocuments.us/reader035/viewer/2022062519/5681513e550346895dbf5811/html5/thumbnails/17.jpg)
17
We Said:
The Dep’t of Finance, Leg Analyst, and Secretary of Education shall be a part of the technical advisory committee (along with CCC researchers and stakeholders).
We will either succeed or fail together. This was a really smart move.
![Page 18: Accountability Reporting for California Community Colleges](https://reader035.vdocuments.us/reader035/viewer/2022062519/5681513e550346895dbf5811/html5/thumbnails/18.jpg)
18
ARCC
The Model: Measures 4 areas with 13 metrics:
Student Progress & Achievement-Degree/Certificate/Transfer
Student Progress & Achievement-Vocational/Occupational/Workforce Dev.
Pre-collegiate improvement/basic skills/ESL Participation
“Process” is not measured
![Page 19: Accountability Reporting for California Community Colleges](https://reader035.vdocuments.us/reader035/viewer/2022062519/5681513e550346895dbf5811/html5/thumbnails/19.jpg)
19
Student Prog. & Achievement: Degree/Cert/Xfer
College: Student Progress & Achievement Rate(s)
(SPAR) “30 units” Rate for SPAR cohort 1st year to 2nd year persistence rate
System: Annual volume of transfers Transfer Rate for 6-year cohort of FTF’s Annual % of BA/BS grads at CSU/UC who
attended a CCC
![Page 20: Accountability Reporting for California Community Colleges](https://reader035.vdocuments.us/reader035/viewer/2022062519/5681513e550346895dbf5811/html5/thumbnails/20.jpg)
20
Student Prog. & Achievement: Voc/Occ/Wkforce Dev
College: Successful Course Completion rate:
vocational courses System:
Annual volume of degrees/certificates by program
Increase in total personal income as a result of receiving degree/certificate
![Page 21: Accountability Reporting for California Community Colleges](https://reader035.vdocuments.us/reader035/viewer/2022062519/5681513e550346895dbf5811/html5/thumbnails/21.jpg)
21
Precollegiate Improvement/Basic Skills/ESL College:
Successful Course Completion rate: basic skills courses
ESL Improvement Rate Basic Skills Improvement Rate
System: Annual volume of basic skills
improvements
![Page 22: Accountability Reporting for California Community Colleges](https://reader035.vdocuments.us/reader035/viewer/2022062519/5681513e550346895dbf5811/html5/thumbnails/22.jpg)
22
Participation
College: None yet…but coming.
System: Statewide Participation Rate (by
demographic)
![Page 23: Accountability Reporting for California Community Colleges](https://reader035.vdocuments.us/reader035/viewer/2022062519/5681513e550346895dbf5811/html5/thumbnails/23.jpg)
23
Major Advancements of ARCC
Creating a viable alternative to the GRS Rate for grad/transfer rate.
Finding transfers to private/out of state institutions.
Doing a wage study. Geo-mapping district boundaries. Creating peer groups.
![Page 24: Accountability Reporting for California Community Colleges](https://reader035.vdocuments.us/reader035/viewer/2022062519/5681513e550346895dbf5811/html5/thumbnails/24.jpg)
24
Defining Grad/Transfer Rate
Student Progress & Achievement Rate (SPAR Rate) IPEDS-GRS for 2-yr colleges stinks:
No part-timers How do you define degree-seeking? Tracking period too short Outcomes counting methodology terrible
AA/AS/Cert counted before transfer Transfer to 2-yr college is counted
![Page 25: Accountability Reporting for California Community Colleges](https://reader035.vdocuments.us/reader035/viewer/2022062519/5681513e550346895dbf5811/html5/thumbnails/25.jpg)
25
SPAR Rate
Defining the cohort: Scrub “first-time” by checking against
past records (CCC, UC, CSU, NSC)
![Page 26: Accountability Reporting for California Community Colleges](https://reader035.vdocuments.us/reader035/viewer/2022062519/5681513e550346895dbf5811/html5/thumbnails/26.jpg)
26
SPAR Rate
Define “degree-seeking” behaviorally for CC populations Not by self-stated intent; this is a poor
indicator Behavior: did student ever attempt
transfer/deg-applicable level math OR English (at any point in academic history) Students don’t take this for “fun”
![Page 27: Accountability Reporting for California Community Colleges](https://reader035.vdocuments.us/reader035/viewer/2022062519/5681513e550346895dbf5811/html5/thumbnails/27.jpg)
27
Defining Degree-Seeking Behaviorally
Separates out remedial students not yet at collegiate aptitude Measure remedial progression to this
threshold elsewhere Creates common measurement
“bar” of student aptitude between colleges Same students measured=viable
comparison
![Page 28: Accountability Reporting for California Community Colleges](https://reader035.vdocuments.us/reader035/viewer/2022062519/5681513e550346895dbf5811/html5/thumbnails/28.jpg)
28
SPAR Rate-Unit Threshold
CCC provides a lot of CSU/UC remediation Lots of students take transfer math/Eng
and leave/take in summer Should not count these as success or “our”
student Set minimum unit completed threshold
(12) for cohort entrance Any 12 units in 6 years anywhere in system
![Page 29: Accountability Reporting for California Community Colleges](https://reader035.vdocuments.us/reader035/viewer/2022062519/5681513e550346895dbf5811/html5/thumbnails/29.jpg)
29
SPAR Denominator:
First-Time (scrubbed) Degree-seeking (at any point in 6
years, attempt transfer/degree applicable math or English)
12 units (in 6 years)
This represents about 40% of students in our system
![Page 30: Accountability Reporting for California Community Colleges](https://reader035.vdocuments.us/reader035/viewer/2022062519/5681513e550346895dbf5811/html5/thumbnails/30.jpg)
30
SPAR Numerator
Outcomes the State wants: Earned an AA/AS/certificate; OR Transfer: to a 4-yr institution; OR Become “transfer-prepared”;OR
Completed 60 xferable units Became “transfer-directed”:
Completed both xfer level math AND English
No double-counting, but any outcome counts SPAR Rate=51%
![Page 31: Accountability Reporting for California Community Colleges](https://reader035.vdocuments.us/reader035/viewer/2022062519/5681513e550346895dbf5811/html5/thumbnails/31.jpg)
31
Tracking Transfers
SSN-level matches with CSU, UC Nat’l Student Clearinghouse for
private, proprietary, for-profit, out of state Match 2x/yr, send all records since 1992 Update internal “xfer bucket”
Works great for cohort tracking Needed method for “annual volume”
![Page 32: Accountability Reporting for California Community Colleges](https://reader035.vdocuments.us/reader035/viewer/2022062519/5681513e550346895dbf5811/html5/thumbnails/32.jpg)
32
Tracking Transfers
Annual Volume of Transfers CSU/UC: they provide these figures
based on their criteria We didn’t want to redefine this
Private/Out of State: NSC “cross-section” cut method
Validated against CSU/UC xfers from NSC source
Added another 30% to annual volumes
![Page 33: Accountability Reporting for California Community Colleges](https://reader035.vdocuments.us/reader035/viewer/2022062519/5681513e550346895dbf5811/html5/thumbnails/33.jpg)
33
97-98 98-99 99-00 00-01 01-02 02-03 03-04 04-05 05-06 06-07
FTF → → → → → → → → T
FTF → → → → → → → R
FTF → → → → → → A
FTF → → → → → N
FTF → → → → S
FTF → → → F
MIN FTF → → E
12 FTF → R
UNITS FTF 06-07
![Page 34: Accountability Reporting for California Community Colleges](https://reader035.vdocuments.us/reader035/viewer/2022062519/5681513e550346895dbf5811/html5/thumbnails/34.jpg)
34
Sector 01-02 02-03 03-04 04-05 05-06 06-07
CSU 50,473 50,746 48,321 53,695 52,642 54,391
UC 12,291 12,780 12,580 13,211 13,462 13,874
ISP 17,070 15,541 18,100 18,365 17,840 18,752
OOS 10,762 10,540 11,150 11,709 11,726 11,825
Total 90,596 89,607 90,151 96,980 95,670 98,842
![Page 35: Accountability Reporting for California Community Colleges](https://reader035.vdocuments.us/reader035/viewer/2022062519/5681513e550346895dbf5811/html5/thumbnails/35.jpg)
35
Transfer: Sector of Choice
% to UC% to CSU
% to Instate Private
% to Out of State
White 17.9% 60.7% 11.0% 10.4%
AfrAm 11.5% 51.2% 18.1% 19.2%
Hisp/Lat 15.1% 67.7% 12.1% 5.1%
Asian 37.0% 49.9% 9.2% 3.9%
![Page 36: Accountability Reporting for California Community Colleges](https://reader035.vdocuments.us/reader035/viewer/2022062519/5681513e550346895dbf5811/html5/thumbnails/36.jpg)
36
Demography of Transfer
Demog (06-07)
FTF Stdents
All Stdents
XFER-CSU
XFER-UC
XFER-ISP
XFER-OOS
AfrAm 9% 8% 5% 3% 11% 13%
Asian 11% 12% 12% 26% 8% 7%
Hisp/Latino 35% 29% 23% 16% 23% 13%
White 29% 35% 37% 40% 44% 55%
![Page 37: Accountability Reporting for California Community Colleges](https://reader035.vdocuments.us/reader035/viewer/2022062519/5681513e550346895dbf5811/html5/thumbnails/37.jpg)
37
The Rise of The Phoenix96-97 2,166
97-98 2,829
98-99 3,374
99-00 4,194
00-01 5,055
01-02 5,586
02-03 6,515
03-04 8,222
04-05 8,585
05-06 8,134
06-07 9,216
![Page 38: Accountability Reporting for California Community Colleges](https://reader035.vdocuments.us/reader035/viewer/2022062519/5681513e550346895dbf5811/html5/thumbnails/38.jpg)
38
Who Transfers to Phoenix?
Ethnicity UC CSU Phoenix
Asian 29.3% 14.2% 4.6%
African American 2.4% 5.2% 16.8%
Hispanic/Latino 13.6% 23.8% 28.6%
White 39.1% 43.6% 37.5%
![Page 39: Accountability Reporting for California Community Colleges](https://reader035.vdocuments.us/reader035/viewer/2022062519/5681513e550346895dbf5811/html5/thumbnails/39.jpg)
39
Wage Study
What was the economic value of the degrees (AA/AS/certificate) we were conferring?
Required data match with EDD Had to pass a bill changing EDD code
to allow match
![Page 40: Accountability Reporting for California Community Colleges](https://reader035.vdocuments.us/reader035/viewer/2022062519/5681513e550346895dbf5811/html5/thumbnails/40.jpg)
40
Wage Study
Take all degree recipients in a given year Subtract out those still enrolled in a
CCC Subtract out those who transferred to
a 4-yr institution Match wage data 5 years
before/after degree
![Page 41: Accountability Reporting for California Community Colleges](https://reader035.vdocuments.us/reader035/viewer/2022062519/5681513e550346895dbf5811/html5/thumbnails/41.jpg)
41
Wage Study
Separate out two groups: Those with wages of basically zero
before degree Those with >$0 pre wage
The result: The Smoking Gun of Success
![Page 42: Accountability Reporting for California Community Colleges](https://reader035.vdocuments.us/reader035/viewer/2022062519/5681513e550346895dbf5811/html5/thumbnails/42.jpg)
42
![Page 43: Accountability Reporting for California Community Colleges](https://reader035.vdocuments.us/reader035/viewer/2022062519/5681513e550346895dbf5811/html5/thumbnails/43.jpg)
43
Mapping Districts
CC Districts in CA are legally defined, have own elections, pass own bonds
We did not have a district mapping for all 72 districts So we couldn’t do district
participation rates
![Page 44: Accountability Reporting for California Community Colleges](https://reader035.vdocuments.us/reader035/viewer/2022062519/5681513e550346895dbf5811/html5/thumbnails/44.jpg)
44
Mapping Project
Get a cheap copy of ESRI Suite Collect all legal district boundary
documents Find cheap labor—no budget for
this
![Page 45: Accountability Reporting for California Community Colleges](https://reader035.vdocuments.us/reader035/viewer/2022062519/5681513e550346895dbf5811/html5/thumbnails/45.jpg)
45
![Page 46: Accountability Reporting for California Community Colleges](https://reader035.vdocuments.us/reader035/viewer/2022062519/5681513e550346895dbf5811/html5/thumbnails/46.jpg)
46
Peer Grouping
“Peers” historically have been locally defined: My neighbor college Other colleges with similar demography Other colleges with similar size
![Page 47: Accountability Reporting for California Community Colleges](https://reader035.vdocuments.us/reader035/viewer/2022062519/5681513e550346895dbf5811/html5/thumbnails/47.jpg)
47
Peer Grouping
Taking peering to another level: Peer on exogenous factors that predict the
accountability metric’s outcome Thus leaving the “endogenous” activity as the
remaining variance Cluster to create groups
We picked 6 clusters, with a min of 3 in a cluster
Each metric produces different factors, peers, clusters
![Page 48: Accountability Reporting for California Community Colleges](https://reader035.vdocuments.us/reader035/viewer/2022062519/5681513e550346895dbf5811/html5/thumbnails/48.jpg)
48
Peer Grouping: Example
Peering the SPAR Rate: 109 rates as outcomes Find data for all 109 that might
predict outcomes/explain variance Perform regression and other magical
SPSS things See how high you can get your R2
![Page 49: Accountability Reporting for California Community Colleges](https://reader035.vdocuments.us/reader035/viewer/2022062519/5681513e550346895dbf5811/html5/thumbnails/49.jpg)
49
Finding Data
What might affect a grad/transfer rate on an institutional level? Student academic preparedness levels Socioeconomic status of students First-gen status of students Distance to nearest transfer institution Student age/avg unit load
![Page 50: Accountability Reporting for California Community Colleges](https://reader035.vdocuments.us/reader035/viewer/2022062519/5681513e550346895dbf5811/html5/thumbnails/50.jpg)
50
Finding Data
We had to create proxy indices for much of these (142 tried) GIS system: geocode student
zipcode/ZCTA Census: lots of data to be crossed by
zip/ZCTA Create college “service areas” based
on weighted zip/ZCTA values Different than district legal boundaries
![Page 51: Accountability Reporting for California Community Colleges](https://reader035.vdocuments.us/reader035/viewer/2022062519/5681513e550346895dbf5811/html5/thumbnails/51.jpg)
51
![Page 52: Accountability Reporting for California Community Colleges](https://reader035.vdocuments.us/reader035/viewer/2022062519/5681513e550346895dbf5811/html5/thumbnails/52.jpg)
52
Finding Data
The Killer Predictor “Bachelor Plus Index”, or what % of
service area population of college has a bachelor’s degree or higher
“Bachelor Plus Index” a proxy for: First gen Academic preparedness Socioeconomic status Distance to nearest transfer institution
![Page 53: Accountability Reporting for California Community Colleges](https://reader035.vdocuments.us/reader035/viewer/2022062519/5681513e550346895dbf5811/html5/thumbnails/53.jpg)
53
Peering SPAR Rate
Exogenous factors that predict SPAR Rate: Bachelor Plus Index % older students % students in basic skills
R2 = .67 What’s left is implied institutional variance
Demo
![Page 54: Accountability Reporting for California Community Colleges](https://reader035.vdocuments.us/reader035/viewer/2022062519/5681513e550346895dbf5811/html5/thumbnails/54.jpg)
54
Peering: What’s Bad
Its complex and somewhat confusing and labor intensive.
Colleges traditional notion of “peer” is shaken
Multiple peers for multiple metrics; can change every year
You could do well vs. State average, increasing over time, but last in your peer group
![Page 55: Accountability Reporting for California Community Colleges](https://reader035.vdocuments.us/reader035/viewer/2022062519/5681513e550346895dbf5811/html5/thumbnails/55.jpg)
55
Peering: What’s Good
Its complex and somewhat confusing
You will likely look good in some areas, OK in others, and low in others
Its not very likely anyone will be high or low in all 6 metrics
It eliminated rankings.
![Page 56: Accountability Reporting for California Community Colleges](https://reader035.vdocuments.us/reader035/viewer/2022062519/5681513e550346895dbf5811/html5/thumbnails/56.jpg)
56
The ARCC Report
Is almost 800 pages. Comes out every March. Takes 4 PY’s to complete (about 6
months/yr) Is generally regarded highly in CA
academic and Legislative circles. DOF and LAO and Sec. of Ed love it. Local Trustees/Boards love it.
![Page 57: Accountability Reporting for California Community Colleges](https://reader035.vdocuments.us/reader035/viewer/2022062519/5681513e550346895dbf5811/html5/thumbnails/57.jpg)
57
The ARCC Collaboration
Has brought the system more money: $33 mil in basic skills Increased noncredit reimbursement
rates by $300/FTE Has brought about trust between
system and State stakeholders. Has educated both sides
tremendously.
![Page 58: Accountability Reporting for California Community Colleges](https://reader035.vdocuments.us/reader035/viewer/2022062519/5681513e550346895dbf5811/html5/thumbnails/58.jpg)
58
No More “Girlie-Man” Accountability!