Download - Academic Senate for California Community Colleges Accreditation Institute, March 19-20, 2010
Academic Senate for California Community Academic Senate for California Community CollegesColleges
Accreditation Institute, March 19-20, 2010Accreditation Institute, March 19-20, 2010
Purpose of the session SLOs for session Presenter intro/background
-Operational Definition-Building a Culture of Inquiry-Data Integration & Inquiry Strategies
Good evidence used in evaluations has the following characteristics:1. It is intentional, and a dialogue about its meaning and relevance
has taken place.2. It is purposeful, designed to answer questions the institution has
raised.3. It has been interpreted and reflected upon, not just offered up in
its raw or unanalyzed form.4. It is integrated and presented in a context of other information
about the institution that creates a holistic view of the institution or program.
5. It is cumulative and is corroborated by multiple sources of data.6. It is coherent and sound enough to provide guidance for
improvement.
Source: ACCJC/WASC Guide to Evaluating Institutions, 2009
Culture of EvidenceCulture of Evidence Culture of InquiryCulture of Inquiry
Data are provided for multiple purposes.
Data demonstrate performance & outcomes.
Data are accessible to all constituency groups.
Data are continuously distributed.
Dialogue centers around data/evidence.
Dialogue focuses on taking action.
Dialogue is open and collaborative.
Dialogue is continuous and widespread.
Dialogue is reflective and dynamic.
1. Make evidence and inquiry the paradigm.2. Communicate often, widely and clearly.3. Embed the cycle of collectively assessing,
planning, implementing, re-assessing and re-planning…..
4. Create opportunities for continuous collaboration with multiple and mixed constituency groups.
Data Integration & Inquiry MatrixData Integration & Inquiry Matrix Website Postings District Research Cmt. Facilitations
Action Research Teams College Research Cmt. Facilitations Sharing of Actions Taken on Data
Informational Sessions Presentations Technical Assistance Meetings
Briefings Data Integration Workshops Research & Assessment Methods Workshops Research Agenda Development & Implementation
Impact
Scop
e
Low High
Low
Hig
h
• Chancellor’s Datamarthttp://www.cccco.edu/ChancellorsOffice/Divisions/TechResearchInfo/MIS/DataMartandReports/tabid/282/Default.aspx• NCES National Center for Education
Statisticshttp://nces.ed.gov/• CPEC California Postsecondary
Commissionhttp://www.cpec.ca.gov/ • CalPASS - http://www.cal-pass.org/
Elevated Level of
Organizational Awareness
Improved Processes & Performance
Knowledge Gap Performance Gap
Strategic Planning Function
Institutional Effectiveness & Student
Success Function
(Poor Planning) (Good Planning) (Good Performance)
Classroom and Service Area Assessment
Standard Level of
Organizational Awareness
Institutional Outcomes and Benchmarks
Data collection does not equate to action or improvement.
Even the most valid and reliable data are not a substitute for action and will not by themselves motivate action.
Actionable data provide information that leads to improved practice.
Actionable data result from “action-oriented” research.
ARCC DATA Statewide Rates Indicator 2008 2009 2010
Student Progress & Achievement 51.2% 51.8% 52.3% Completed 30 or more units 70.4% 71.2% 72.4% Fall to Fall Persistence 68.3% 69.2% 68.7% Voc Ed Course Completion 78.2% 77.7% 77.6% Basic Skills Course Completion 60.5% 60.5% 61.5% Basic Skills Course Improvement 50.0% 51.2% 53.8% ESL Course Improvement 44.7% 50.1% 50.2%
Is this data actionable?
Is this data actionable?
Basic Skills Success Rates African-American 31.3%
Asian 75.0% Decline to state
Filipino 68.4% Hispanic 62.5%
Native American Pacific Islander
Unknown White 56.1% Total 59.7%
Is this data actionable?
Is this data actionable?
Basic Skills Success at Levels prior to TransferEnglish English English ESL ESL ESL Math Math Math
3 levels prior Pre-Intermediate 68% Fundamental 65%2 levels prior 67% Intermendiate 35% 69% Algebra 32% 67%1 levels prior 40% 74% High Intermediate 22% 37% 72% Intermediate 11% 27% 72%transfer 25% 46% 79% Transfer 11% 13% 31% 5% 13% 31%
16% 29% 42%
ESL Level 1 Outcomes Assessments % passed Listening/SpeakingSLO: Demonstrate understanding of frequently used words, phrases and
questions in familiar contexts. Engage in limited social conversations to communicate basic survival needs.
Assessment = Interview Outcomes = 84% passed successfully
Reading SLO: Construct meaning from simplified print materials on familiar topics. Assessment = CASAS Level A (A comprehensive standardized test) Outcomes = CASAS Success rate 73% Writing SLO: Produce simple sentences in paragraph format and complete simple
forms. Assessment = Written Paragraph and Rubric for scoring Outcomes = 74% pass rate
Principle 1 – Use longitudinal data when possible
Principle 2 – Use data in context
Principle 3 – Look for both direct and indirect data
Principle 4 – Do not oversimplify cause and effect of data
Principle 5 – Use appropriate levels of data for appropriate levels of decisions
Principle 6 – Perception is the reality within which people operate
Principle 7 – Use of data should be transparent
Principle 8 – Consider carefully when to aggregate or disaggregate data
Principle 9 – Focus on data that is actionable
Principle 10 – Consider implications and the “What if?
-Information Capacity Challenges-Research Processes and Procedures-Research Agendas-Action Research Guided Questions
Building an Evidence-based Infrastructure Managing and responding to myriad requests Maintaining quality and integrity of data process Making data and information widely accessible
Keeping Up with the Demand Responding to heightened accountability
mandates Linking research to (resource) planning Supporting data-driven decision-making
Turning Data into Action Making data available and applicable at all levels Making sense of and taking action on the data Building a Culture of Inquiry
Guidelines for use of data and information Protection of human subjects policy Review panels and committees Request and fulfillment procedures Criteria for prioritizing ad hoc requests Linking requests to broader goals &
initiatives Creating and using Research Agendas
College-wide Research Agenda Supports major college-wide initiatives &
activities Tied to college-wide plan (goals & priorities) Includes recurring requests
Topical Research Agenda Focused on a single topic or group of interest Tied to a specific initiative or activity Fewer research activities than college-wide
Developing the Research Agenda1. What and who will be researched?2. How is research tied to college plans, goals, initiatives and/or activities?3. How will the information be used, by whom and how often?4. Which methodology or approach will be used?
Turning Data into Information 1. What do the data tell us?2. Which questions were fully answered by the research and which need
more exploration?3. What are reasonable benchmarks based on the research?
Taking Action on the Information 1. What interventions or strategies do we need to deploy in order to move
the needle?2. How should this information be shared and applied across the college?
“The New Guys in Town”