texas education agency february 15,2012 nces winter forum and 25 th annual 2012 mis conference brian...

Data-Driven Decisionmaking in the Classroom, From Concept to Reality

Texas Education Agency February 15,2012

NCES Winter Forum and 25th Annual 2012 MIS ConferenceBrian Rawson & David Butler

slide 1Fictitious district, school, staff and student names. Stock photos. Release for web use of these photos on file.

Data Collection and Information Sharing Today – Room for Improvement

Districts spend significant time providing data to TEA for PEIMS

$Data that are shared back with the district are not timely and are not in a particularly useful format

Cost to districts is extremely high, estimated to be $323M annually statewide

Data rarely makes its way to the educators best positioned to improve student achievement

2

TSDS encompasses four key projects

State-sponsored SIS

Education Data Warehouse (EDW) PEIMS TPEIR

Opt-in, voluntary SIS TEA has selected two

options on the model for offering state-sponsored SIS

TSDS will integrate with other SIS’s – no requirement or mandate to switch

Powers student, campus, district data dashboards

System supported by the state but the data only available to educators

Will become conduit to submit PEIMS data

Loading of non-PEIMS data is strictly optional and at the districts’ discretion

XML data standard will make it easier to submit and certify data

Realign statewide data collection standards and protocol for districts

Expanded to link pre-K, college readiness, and workforce data

Load college readiness test score collections (SAT, ACT, AP Test data)

© 2011 Michael & Susan Dell Foundation 33Fictitious district, school, staff and student names.Stock photos. Release for web use of these photos on file.

http://www.texaseducationinfo.org/tpeir/default.aspx




Fictitious district, school, staff and student names.

District Environment Only

TEA Environment Only

How Will TSDS Work in the Future?

SSIS

DistrictSIS

EDW PEIMS

TPEIRPEIMS data

PEIMS data

Districts submit data

4×/year

Voluntary data

Voluntary data

Dashboards filled out

with as much data

as you upload

TEA will connect K-12

data with pre-K, college

readiness, workforce

data

Cert

ify/V

alid

ate

slide 4Fictitious district, school, staff and student names. Stock photos. Release for web use of these photos on file.

Texas Student Data System (TSDS) Vision

A data system for teachers, designed by teachers.

Delivers relevant, timely and actionable student data back to educators to continually improve performance

A comprehensive, easy to use resource for student data – brings together student information from multiple sources

Reduces reporting and collection burden to districts

Requires no additional data input

User friendly and intuitive and accessible from any location

Available free of charge to all Texas districts

Enable 100% of educators to have access to timely, relevant, actionable data to drive classroom and student success

How did we get from problem statement (2009) to solution (2012)?

6Fictitious district, school, staff and student names.Stock photos. Release for web use of these photos on file.

Process used for State-Sponsored Student Information System (SSIS) selection


SSIS Stakeholder Engagement Process

Plenary Q&A

SIS Features Feedback

SPOT Analysis

Final Q&A

The goal was to gain an understanding of district / campus SIS needs and obtain feedback on key features required. Questions were targeted to identify the needs of district and campus administrators, educators, PEIMS coordinators, and technical staff.

• Gauged initial reaction to TSDS vision from stakeholders in breakout groups, including: overall impressions, areas that are confusing/not clear, aspects that are most and/or least appealing

• Document basic demographic information, management systems currently in use, and comments on common SIS features

• We conducted an exercise whereby the participants identify the quality of their SIS features by placing green, yellow, and red stickers

• SPOT (Strengths, Problems, Opportunities, and Threats) analysis of current system versus changes to the new solution

• A final summary of group findings captured the key take-aways of each participant

• Any additional questions/concerns were captured as well

Fictitious district, school, staff and student names. Stock photos. Release for web use of these photos on file.

Conclusion of Features FeedbackStakeholder feedback allowed the ability to identify the strong and weak features of their current SIS systems.

Strong Weak• Attendance• Disciplinary• EYP Tracking• Free and Reduced Lunch• Gifted and Talented• Grade Reporting• Health Records and Reporting• Historical Records• Special Ed Services• Student Scheduling• Master Schedule Building• Address Verification• Emergency Notification• Student Demographics• Classroom Management• Parental Portal

• Extra-Curricular• Graduation Plan• Guidance and Counseling• Test Scores• Lesson Planner• Cafeteria Automation• Curriculum Maps• Textbook Tracking• Fees• Attachments• Data Mining and Summary• e-Signatures• Student Management Mobile

Portal

Fictitious district, school, staff and student names.

SSIS Contracts Conclusions

After an extensive RFI/RFO selection process: Two (2) SSIS vendors were chosen SSIS contracts are umbrella agreements that allow the

LEA to purchase without the need for an RFP or RFO Each LEA will have a separate contract with the vendor Contract Management teams will oversee the Service

Level Agreements to include:▪ SSIS availability▪ Performance▪ Business continuity

Not-to-exceed pricing allows LEA to negotiate with vendor

Texas Education Data Standard (TEDS)


EDW Based Upon Texas Education Data Standard

12

TEDS• XML-based standard• Common Data Model

across all 1200+ LEAs• 18 interchanges in total• For both PEIMS and

dashboards • 2013-14 standard stable,

released to public in March 2012

Texas Extensions• PEIMS specific

elements that can’t be fulfilled through Ed-Fi

• 7 interchanges

Ed-Fi • National CEDS compliant

standard • 11 interchanges• Some elements dashboard

specific, other used for PEIMS and dashboards

Process used for development of Student Performance Improving Dashboards


Key Processes

Dashboards are Grounded in Best Practices and Vetted by Educators

Nationwide Best Practices Audit

Academic Research

Focus Groups with 2,600 Educators

• Initial dashboards based on national education research and review of best practices across the country

• Received and incorporated feedback on dashboard from 2,600 educators in Texas

• Enhancements to dashboards based on stakeholder feedback

16

El Paso

Amarillo

Richardson

Edinburg

HoustonSan Antonio

Mount Pleasant

Midland

~2,000 people attended the regional forums – 204 Classroom

Teachers– 268 Campus

Administrators and Principals

– 693 LEA Administrators and Superintendents

– 637 PEIMS/IT Coordinators

– 160 Webinar participants; ongoing feedback collected via TSDS website

9 Regional Forums; 12 Feedback Sessions

85 Breakout Sessions by 4 stakeholders groups (Teachers, Principals, Superintendents/ Administrators, and PEIMS/IT Coordinators)

Stakeholder engagement meetings were conducted over a two-month period (Mar-Apr 2010), and consisted of 12 three-hour sessions at 9 different regional sites

Lubbock

Stakeholder Engagement Process Overview

Stakeholder Engagement Process

Plenary Presentat

ion

Plenary Q&A

“Clean Slate” Metrics Needs

Review Existing

Snapshots

Final Q&A

The engagement process enabled stakeholders to understand the TSDS vision, review progress to date, express feedback, and provide input into the design of reports & tools Full Group

Breakout Sessions

• Gauged initial reaction to TSDS vision from stakeholders in breakout groups, including: overall impressions, areas that are confusing/not clear, aspects that are most and/or least appealing

• TEA – Presented an overview of the TSDS vision

• MSDF – Provided an overview of performance management, including action video clips

• Prior to viewing any TSDS snapshots, stakeholders were asked to provide the “Top 10” critical questions/pieces of data they would want to include in a dashboard, including timing/frequency of use and importance

• Following a discussion of the “clean slate” metrics, participants were asked to provide feedback on both the student and campus snapshots that have been created based on best practice ideals• A final summary of group findings captured the key take-aways of each participant

• Any additional questions/concerns were captured as well

Teachers’ Focus: Campus Leaders’ Focus:

District Leaders’ Focus:– Complete historical

information (What is the status of my incoming class?)

– Attendance and Academic Progress during year (How are my students doing now?)

– Updated student demographics/contact information (What is going on with my student? Who do I contact regarding issues?)

– Peer comparisons (How is my class/my student comparing to like classes/students?)

– Student/Teacher performance trends (What is our progress? Where is help needed?)

– Ability to analyze/ correlate data (What does the data tell us?)

– Data security, quality, and timeliness (Is the data relevant, accurate, safe?)

– Implementation realities (Who will enter this data? Who sets targets?)

– Secure access to/oversight of data (Who will see this info?)

– Implementation realities (Who will pay for this? Who will input data?)

– Existing vendor overlap / integration (How will vendors react?)

– Report generation, customization capabilities (How can the data be used?)

– Tie to/fit with Campus Improvement Plans, Strategic Plans (How does this fit with existing processes?)

Stakeholder Feedback – By RoleOpen-ended stakeholder feedback, lines of inquiry, and areas of concern were largely tied to the responsibilities each group faces in their current roles

Use Information to Improve Student

Outcomes

Use System to Improve Reporting and Reduce Costs

Dashboards Consist of Four Levels Driven by Individual Student Data

District/LEA

Campus

Classroom

Student

Drill down to the individual Student level

from any aggregate

view

Data originates at the Student Level and rolls up to aggregate

views

Dashboards are Targeted Toward Improving Student Outcomes

Positive Interactive Involvement with

Students

More Teaching Time

Informed Decision Making

User Friendly Data System

“After the information has been gathered, hopefully I would have a user-friendly service that would allow me to look at the information, serve my students and meet their needs more effectively and efficiently.”

21

Metrics by “Useful” Rating – All StakeholdersIn general, the Academic Progress and Engagement metrics were considered more useful than the Academic Challenge and College/Career Readiness metrics

22

Stakeholder Feedback – Clean Slate “Top 10”Participants provided unsolicited feedback regarding the “Top 10” critical questions or pieces of data ideally seen on Student, Teacher, and Campus dashboards

Students – Top 3• Special

Programs/Interventions • Attendance• TAKS Results

17%17%13%

Teachers – Top 3• Student performance by*

• Teacher attendance• Certifications

32%15%12%

Campus – Top 3

• Expenditures• Dropouts• Graduation Rate

21%18%18%

Feedback generated during the Clean Slate exercise mapped very closely with metrics identified through best practices research, allowing for state-

specific nuances (e.g., ELL, career-ready, etc.)

*”Student performance by" varied, but generally includes students’ Grades, Assessment performance, Longitudinal testing data, and Teacher Fail Rate

Dashboard Content & Features

Attendance & Discipline

• Daily attendance

• Class period attendance

• Tardy rates• Discipline

Assessments & Grades

• State assessments

• Language assessment

• Early reading• District

benchmarks• Course /

subject area grades

• Credit accumulation and 4x4

Academic Potential

• Advanced course potential

• College entrance exams (SAT, ACT, PSAT)

• College readiness indicators

Student Information

• Program participation

• Enrollment dates

• Contact information

• Key transcript data:• Current and

historical courses and grades

• TAKS history

Highlights trends over time and flags negative trends Easily drill down for more detail (e.g., finer grain, historical data, student

exception lists) Highlights where students are not meeting performance goals

Demonstration of the DashboardsStudent and Classroom

https://lpr.dlpdemo.com//Docs/LubbockISD/TrainingMaterials/Trainer%20Demo%20Video.wmvhttps://lpr.dlpdemo.com//Docs/LubbockISD/TrainingMaterials/Trainer Demo Video.wmv


https://lpr.dlpdemo.com/Docs/LubbockISD/TrainingMaterials/Trainer%20Demo%20Video.wmv




Functionality: Classroom and Student List Views

Red text = not meeting goal

Trend arrows compare to prior period

Customize your view

• Summary view for class/teacher

• Same format for campus, student, and teacher list drill down (exception lists)

• Columns are customizable and can be sorted

• All lists can be exported to csv file

• Clicking any student name takes you to student dashboard


Functionality: Metric Presentation

Indicates if meeting campus goal

Trend direction, flag if negative

Tabs describe content and are clickable

Additional detail

• Same format for student, campus and district dashboards

• Metric name, status, value trend

• Date of last data capture

• ‘More’ detail option

• Navigation tools at top


Functionality: Campus Drill Down Capabilities

• Available for most metrics on campus dashboard• Drill down by grade (campus)• Generate exception lists of students not meeting goal (campus)• Historical trend detail

Grade Level DetailStudent Exception List

27Fictitious district, school, staff and student names.

Functionality: Student Drill Down Capabilities

• Available for most metrics on student dashboards• Historical trend detail • Metric details (student: attendance, TAKS, credits)

Historical Detail Metric Details


Functionality: Search

• Quickly get to information you need

• Available on all dashboard pages

• Search for student, teacher, or campus name

• Drop down list appears with available options

• Results limited to user access role

Metrics definition


Metric Definitions - Algebra I

Why measure Algebra I vs. other courses? Research indicates that students who enroll and complete Algebra I by the 9th

grade (a key prerequisite for higher math) will graduate in higher numbers and are more likely to be college ready.

Algebra I Metrics-High School: Taking or have Taken: % of students who have taken Algebra I by the 9th

grade. In the case of students who have never enrolled, teachers and counselors can quickly identify students who need to be enrolled and, based on each student’s academic history, provide the necessary support to register and prepare them for Algebra I.

Passing or have Passed: % who are passing/have passed by the 9th grade. For students who are currently enrolled in Algebra I, teachers can quickly identify students who need additional academic support to successfully complete the course. In the case of former Algebra I students, teachers of advanced math courses can view Algebra I performance to evaluate the level of support students may require as they engage in more rigorous coursework.

Algebra I Metrics - Middle School: This measure, defined as enrolling and successfully completing Algebra I by

the 8th grade, helps campuses identify the level of students who show potential for more advanced coursework and their ability to master higher level math skills to be successful at the post-secondary level.

Metric Definitions - Attendance (Campus Level)

Common measure used by districts to track attendance and for state reporting

Use to pinpoint specific students negatively impacting ADA. Value often lower than ADA.

Catch students with attendance problems that aren’t reflected in daily attendance measures above


Metric Definitions – Discipline (Campus Level)

All Discipline Incidents: percentage of students with repeat occurrences (5 or more) of discipline incidents representing minor infractions (school code of conduct incidents) or one or more discipline incidents representing the most serious incidents, excluding school code of conduct incidents.

School Code of Conduct Incidents: percentage of students who have one or more minor infractions (school code of conduct incidents) in a given grading period.

Identifies students with chronic discipline issues

Identifies students with early signs of discipline problems


TSDS Project Milestones and Status


EDW Projected Timeline

Initial EDW (Dashboards) prototype

Draft data standard defined

FY10

Draft data dashboards defined

SLDS 2009 ARRA Grant Awarded

Dashboards finalized based on feedback

Stakeholder engagement

Continued Deployment to all Districts

FY14FY13FY12FY11

RFI for DW Solution Issued

Begin Limited Prod Releases 1-2

Select Vendors

Limited Prod Releases 3-5

Dashboard Enhancements

Limited Prod Release 6

Integration with SSIS

Integration with other SISs

Interface with other Source Systems

PEIMS Data Standards Published

Begin Production Deployment

Hosted EDW

Dashboard Enhancements

PEIMS Collection through EDW

Draft Data Standards for EDWEvaluate RFI Responses

Re-engineer Data Collection Business Processes

Requirements (Solution Components)

Issue 3 RFOs

•The importance of prototyping and piloting activities was reinforced as significant learning and discovery continued during subsequent phases of the project.

•Sufficient time needs to be allocated for iterative change and refinement.

•The high level of stakeholder feedback is worth the additional efforts associated with the refinements.

Lessons learned

www.TexasStudentDataSystem.org

Contact: [email protected] about any additional TSDS questions and/or assistance


mailto:[email protected]

texas education agency february 15,2012 nces winter forum and 25 th annual 2012 mis conference brian...

Documents

actionable data

data system

actionable student data

ap test data

workforce data certifyvalidate

student names

stock photos

student success slide