c r e s s t / u c l a 1 ucla graduate school of education information studies center for the study...
DESCRIPTION
C R E S S T / U C L A 3 QSP—Making Use of Testing and Other Information Data Manager Resource KitTRANSCRIPT
C R E S S T / U C L A 1
UCLA Graduate School of Education & Information StudiesCenter for the Study of Evaluation (CSE)
National Center for Research on Evaluation, Standards, and Student Testing (CRESST)
John J. LeeSenior Researcher
Quality School PortfolioDecision Support System:Transforming Data into Information
C R E S S T / U C L A 2
The Quality School Portfolio (QSP)
Without tools to investigate and understand results, more testing will not impact educational practice
Software for use by educators to organize, collect, interpret, and respond to complex data
UCLA, OERI, Stuart, and Joyce Foundation support
C R E S S T / U C L A 3
QSP—Making Use of Testing and Other Information
Data ManagerResource Kit
C R E S S T / U C L A 4
QSP Goals Transform test and other results into usable,
actionable form Create coherent information structure for all
members of the educational community (policymakers, administrators, teachers, and parents)
Encourage using information as the basis for making decisions about educational options and services
Raise capacity and efficacy of educators Communicate findings in intuitive displays
C R E S S T / U C L A 5
Learning to Use Results: QSP Features
Relies on external and internal sources of information
Easy access and display Custom queries Disaggregation Report preparation
C R E S S T / U C L A 6
Applying QSP Use for school monitoring and improvement For distance learning assessment reporting For distributed program evaluation (LA’s BEST;
Artful Learning) Disaggregation Individual longitudinal records
C R E S S T / U C L A 7
How can the school focus on particular populations?
System Groups
Custom Groups
Combo Groups
Disaggregation
C R E S S T / U C L A 8
Conditional Disaggregation: Eight Filters
Hispanic + Boys + 9th Grade + Spanish L1 + Redesignated English Magnet schools + GPA below 2.0 + Less than 2 years in the school
C R E S S T / U C L A 9
Gauge Display
C R E S S T / U C L A 10
Scatterplot
C R E S S T / U C L A 11
Quick School Comparisons
C R E S S T / U C L A 12
School Performance Reports
C R E S S T / U C L A 13
Resource Kit MenuResource Kit Menu
C R E S S T / U C L A 14
Resource Kit Functions Clustered topics of resources Selection or modification of instruments in a
cluster given a description Supports Acrobat Reader and Rich Text
Formats Customized entry of standard surveys, etc., to
permit local use of evaluation data and simultaneous export of information to centralized sources
C R E S S T / U C L A 15
QSP Resource Kit
Local data to answer local questions Staff and student surveys, interview and
observation protocols Data entry mechanisms, e.g., student work—
in process (digital portfolios) Online assessments or surveys Transfer into Data Manager
C R E S S T / U C L A 16
Resource Kit Summary PageResource Kit Summary Page
C R E S S T / U C L A 17
Extensions
QSP Web (currently in development) Data analysis from descriptive to
growth projection Flexible report wizard Improved Web training component
C R E S S T / U C L A 18
User Challenges
Learning to ask productive questions Looking at measures over time Level of technical sophistication, e.g.,
hierarchical models, trajectories Reports appropriate to various audiences
and stakeholders
C R E S S T / U C L A 19
Continuing Research: A Sample of Impact Questions
Who is helped? What kinds of schools, teachers, and students? How do experience, beliefs, knowledge condition impact?
Do administrative practices change at school, district and board levels?
What happens in classrooms? Is student growth affected? For how long? What kind of combination of information is needed (CRT/NRT, process)?