It’s a Journey Not a Destination: The Development of Virginia’s Postsecondary Education Reports

Virginia Department of Education MIS Conference, San Diego, CAFebruary 2012

Images in this presentation are used with permission from Microsoft.

Longitudinal data bridged between the Virginia Department of Education (VDOE) and the State Council of Higher Education for Virginia (SCHEV).

SCHEV is Virginia’s coordinating body for higher education. As part of their responsibilities, SCHEV collects credits earned data from colleges in Virginia while also maintaining the confidentiality of student records in compliance with the Family Educational Rights and Privacy Act (FERPA) and state privacy laws.

Postsecondary achievement reports that meet the conditions of State Fiscal Stabilization Fund (SFSF) indicator (c)(12).

Postsecondary enrollment reports that meet the conditions of State Fiscal Stabilization Fund (SFSF) indicator (c)(11).

Virginia has collected postsecondary enrollment data from the National Student Clearinghouse for several years. To meet SFSF (c)(11),Virginia modified an existing report it provides to Local Education Agencies.

Points of Interest We’ll See During Virginia’s Journey

HhHhHhHh

This Lane For State Agencies With

Different Unique Student IDs

Virginia’s journey to create postsecondary achievement reports required that VDOE enter into a data-sharing agreement with the State

Council of Higher Education for Virginia (SCHEV).

Preparing The De-identified DataStep One:

IHEs in Virginia send

student enrollment & credits earned data to the State

Council of Higher Education for Virginia (SCHEV). Using

an algorithm SCHEV acquires as many VDOE student IDs as possible.

SCHEV de-identifies their data set so no meaningful

student identifier is present.

Step Two:

High schools in Virginia send cohort graduation data to VDOE. Using an algorithm and human intervention, VDOE acquires as many

SCHEV student IDs as possible. VDOE de-

identifies their data set so no meaningful

student identifier is present.

Step Three:

A 3rd party puts all the data sets through a 2nd

de-identification process. VDOE is provided with the final data sets. The data sets use random

numbers for VDOE and SCHEV student IDs that

permit the data sets to be merged without

identifying specific individuals.

SCHEVStudent

IDs

VDOEStudent

IDs

IHEs in Virginia

3rd Party

High Schools

in Virginia

VDOE was ready to begin the merge process after it acquired the twice de-identified secondary and postsecondary data sets.

YIELDTo SCHEV

Student IDsWhen

Merging Data

Linking Data

Students in the 2007-2008

FGI cohort

Enrollment and credits earned data

for First-time in college

(FTIC) students

The only cases included in the analytic dataset were cases

where the SCHEV Student ID, VDOE Student ID, or SCHEV &

VDOE Student ID matched.

Data Source Data Source

Virginia’s journey to develop postsecondary education reports couldn’t continue until VDOE confirmed that the data merge process with

SCHEV was successful.

Data ValidationNext Exit

CAUTIONPossible Mismatches On

Gender, Race/Ethnicity, or Birth Month/Birth Year

Birth Month/Birth Year

Gender 99%

Race/Ethnicity 87%

99%

Common VariableVDOE/SCHEVAlignment Rate Outcome

There were only a few instances where gender did not align between the two data sources. Cases where misalignment was present were further analyzed to see if other common variables aligned.

SCHEV and VDOE have different ways of collecting race/ethnicity. In addition, students may change their race/ethnicity classification upon entering postsecondary institutions. VDOE used these two facts as justifications for keeping cases where race/ethnicity didn’t align provided the other validation variables aligned.

In cases where birth month/birth year didn’t align, a random sample was analyzed to determine reasons for the mismatch. The analysis revealed most mismatches were a result of data entry error. Based on this result, VDOE kept cases where birth month/birth year didn’t align provided the other validation variables aligned.

Validation Decisions

After the de-identification process was completed and the data sets were merged, only 35 cases were removed due to

misalignment on multiple common variables.

What was Virginia’s match rate? 44,830 First-time first-year students with in-state

residency status in the year 2008-2009 (source: SCHEV).

35,381 Virginia public high school graduates from the 2007-2008 graduating cohort matched to SCHEV data (79%).

DOE and SCHEV independently estimated that ~6% of FTIC students would not come from Virginia public high schools or the graduating class of 2007-2008.

Approximately 15% unmatched = 85% match rate (estimated).

More on the data-linking process

The use of SCHEV student IDs allowed VDOE to add an additional 4,443 students into the longitudinal data set.

VDOE and SCHEV independently concluded that any estimates about postsecondary enrollment or postsecondary achievement that relied on the longitudinal data set would be underestimates.

Efforts to improve the probabilistic matching process are currently underway as part of Virginia’s grant-funded work to expand the Virginia Longitudinal Data System (VLDS).

Virginia’s journey continued with the creation of postsecondary achievement reports that met the conditions of SFSF indicator (c)(12).

COLLEGECREDIT MINIMUM

LIMIT

30FGI GRADS

VA Public IHE Zone

Conditions of SFSF (c)(12) Virginia created postsecondary education reports for

SFSF indicator (c)(12) consistent with the federally prescribed measure of calculating cohort graduation rates.

Only students who earned Virginia’s standard or advanced studies diploma count as graduates in Virginia’s Federal Graduation Indicator (FGI).

Students reporting as graduating within four, five and six years of first entering the ninth grade were included in the postsecondary achievement reports.

More Conditions of SFSF (c)(12)

Only students who enrolled in a Virginia public IHE within sixteen months of graduation were included.

Thirty units of college credit (applicable to a degree) was the benchmark for earning one year of college credit within two years of IHE enrollment.

Credits earned for passing developmental education courses were not included in the calculation.

SCHEV and VDOE did not have access

to transcript level credits earned data

With the conditions of SFSF indicator (c)(12) applied, VDOE was ready to calculate how many students earned one year of college credit

within two years of IHE enrollment.

The calculation for credits earned required VDOE to identify the college courses (applicable to a degree) a student enrolled in within a two-year time frame and sum the amount of credits earned for passing those courses.

The impact of unofficial transcript-level data resulted in

VDOE and SCHEV making decisions about the following critical topics:Credits awarded for passing AP courses or earning

qualifying scores on AP tests. Not included in this year’s calculation.

Dual Enrollment CreditMissing Grades

Impacts Of Unofficial Transcript-Level Data

This Lane For Including Earned Dual Enrollment Credits

VDOE created postsecondary achievement reports for SFSF indicator (c)(12) that show results with and without earned dual enrollment credits included in the calculation of credits earned.

Virginia Department Of EducationState Fiscal Stabilization Fund Indicator (c)(12) Report

2007-2008 FGI cohort year (students entering high school in 2004-2005)Graduation Rate Type: Four-Year Rate

Division: ALL, School: ALL  Students that earned a federally recognized diploma and enrolled in a public Institution of Higher Education (IHE) in Virginia.

 

Total number of students in cohort who graduated from high school with a federally

recognized diploma and enrolled in a public IHE in Virginia within 16 months

of graduation

Dual enrollment credits included in total count of credits earned

Dual enrollment credits excluded in total count of credits earned

Earned one year of college credit within two

years of enrollment

Amount of credits earned could not be determined due to the presence of

missing grades

Earned one year of college credit within

two years of enrollment

Amount of credits earned could not be determined due to the presence of

missing gradesSUBGROUP Number Number Percent Number Percent Number Percent Number PercentAll Students 31579 21102 67 395 1 20852 66 385 1Female 17443 12047 69 188 1 11906 68 178 1Male 14136 9055 64 207 1 8946 63 207 1American Indian 74 51 69 2 3 51 69 2 3Asian 2160 1737 80 36 2 1733 80 36 2Black 5600 2946 53 82 1 2911 52 81 1Hispanic 1181 673 57 31 3 668 57 31 3Native Hawaiian 34 23 68 1 3 23 68 1 3White 22140 15396 70 237 1 15191 69 228 1Economically Disadvantaged 3812 1879 49 52 1 1837 48 50 1Limited English Proficient Students 1245 831 67 27 2 829 67 27 2

Students with Disabilities 1467 580 40 27 2 570 39 25 2

Notes: < refers to a group below state definition for personally identifiable results.

- = no data available for that group

According to the State Council of Higher Education for Virginia (SCHEV), credits from remedial courses do not count towards a college degree and are to be excluded from the total number of credits earned.

According to the State Council of Higher Education for Virginia (SCHEV), 30 units is the equivalent of one-year of college credit.

Federally recognized diplomas include Standard, Advanced Studies, and International Baccalaureate (IB) diplomas.

The number of students enrolled and the number of students earning credit (including and excluding dual enrollment credits) are based on the available data the Virginia Department of Education (VDOE) was able to link with the State Council of Higher Education for Virginia (SCHEV).

Missing Grade

Crossing

In some records, public IHEs did not submit final course grades to SCHEV. Some of these missing grades impact the determination of whether or not a student earned 30 or more credits within two years of enrolling in college. The postsecondary achievement reports for SFSF (c)(12) document the percentage of students for whom insufficient data were available to make a final determination about whether they met the criteria of SFSF (c)(12).

Virginia Department Of EducationState Fiscal Stabilization Fund Indicator (c)(12) Report

2007-2008 FGI cohort year (students entering high school in 2004-2005)Graduation Rate Type: Four-Year Rate

Division: ALL, School: ALL  Students that earned a federally recognized diploma and enrolled in a public Institution of Higher Education (IHE) in Virginia.

 

Total number of students in cohort who graduated from high school with a federally

recognized diploma and enrolled in a public IHE in Virginia within 16 months

of graduation

Dual enrollment credits included in total count of credits earned

Dual enrollment credits excluded in total count of credits earned

Earned one year of college credit within two

years of enrollment

Amount of credits earned could not be determined due to the presence of

missing grades

Earned one year of college credit within

two years of enrollment

Amount of credits earned could not be determined due to the presence of

missing gradesSUBGROUP Number Number Percent Number Percent Number Percent Number PercentAll Students 31579 21102 67 395 1 20852 66 385 1Female 17443 12047 69 188 1 11906 68 178 1Male 14136 9055 64 207 1 8946 63 207 1American Indian 74 51 69 2 3 51 69 2 3Asian 2160 1737 80 36 2 1733 80 36 2Black 5600 2946 53 82 1 2911 52 81 1Hispanic 1181 673 57 31 3 668 57 31 3Native Hawaiian 34 23 68 1 3 23 68 1 3White 22140 15396 70 237 1 15191 69 228 1Economically Disadvantaged 3812 1879 49 52 1 1837 48 50 1Limited English Proficient Students 1245 831 67 27 2 829 67 27 2

Students with Disabilities 1467 580 40 27 2 570 39 25 2

Notes: < refers to a group below state definition for personally identifiable results.

- = no data available for that group

According to the State Council of Higher Education for Virginia (SCHEV), credits from remedial courses do not count towards a college degree and are to be excluded from the total number of credits earned.

According to the State Council of Higher Education for Virginia (SCHEV), 30 units is the equivalent of one-year of college credit.

Federally recognized diplomas include Standard, Advanced Studies, and International Baccalaureate (IB) diplomas.

The number of students enrolled and the number of students earning credit (including and excluding dual enrollment credits) are based on the available data the Virginia Department of Education (VDOE) was able to link with the State Council of Higher Education for Virginia (SCHEV)..

VDOE’s journey also included postsecondary enrollment reports that met the conditions of SFSF indicator (c)(11)

The data linking process for State Fiscal Stabilization Fund Indicator (c)(11)

Institution of Higher Education

Institution of Higher Education

Institution of Higher Education

Step One:

Colleges and universities across the nation send

student-level enrollment data to the National

Student Clearinghouse (NSC).

Step Two:

The Virginia Department of

Education (VDOE) sends student-level

cohort graduation data to NSC.

Step Three: NSC uses a proprietary algorithm (plus internal staff review) to compare

college enrollment records to VDOE data.

NSC supplies VDOE with a data set that shows the results.

VDOE creates SFSF (c)(11) reports.

Virginia Department Of Education State Fiscal Stabilization Fund Indicator (c)(11) Report

2010-2011 FGI cohort year (students entering high school in 2007-2008)Graduation Rate Type: Four Year Rate

Division: ALL, School: ALL

   

Total number of students in

the cohort earning a federally

recognized high school

diploma

Students who enrolled in any Institution of Higher Education

(IHE) within 16 months of earning a federally

recognized high school diploma

Students who enrolled in a 4-year public Institution of

Higher Education (IHE) within 16 months of earning a

federally recognized high school diploma

Students who enrolled in a 4-year private Institution of

Higher Education (IHE) within 16 months of earning a

federally recognized high school diploma

Students who enrolled in a 2-year Institution of Higher Education (IHE) within 16

months of earning a federally recognized high

school diploma.

Subgroup Number Number Percent Number Percent Number Percent Number PercentAll Students 80023 49739 62 24043 30 7551 9 18145 23Female 40945 27030 66 13170 32 4253 10 9607 23Male 39078 22709 58 10873 28 3298 8 8538 22American Indian 285 163 57 74 26 22 8 67 24Asian 4718 3574 76 2225 47 354 8 995 21Black 18628 10089 54 4011 22 1934 10 4144 22Hispanic 6257 3141 50 1184 19 367 6 1590 25Native Hawaiian 105 60 57 29 28 10 10 21 20White 47402 31013 65 15667 33 4621 10 10725 232 or More 2593 1681 65 851 33 242 9 588 23Economically Disadvantaged 20545 9453 46 2757 13 1373 7 5323 26Limited English Proficient Students 4228 2086 49 712 17 160 4 1214 29Students with Disabilities 5498 2328 42 476 9 311 6 1541 28

NOTES:< = A group below state definition for personally identifiable results.

- = no data available for that group

This report uses the best available data according to the National Student Clearinghouse.

Students who attended schools that do not participate in NSC are not included in the number or percent of students enrolled in an IHE.

Federally recognized high school diplomas include Standard, Advanced Studies, or International Baccalaureate (IB) diplomas.

Most subgroups are based on students most recent status.

Feb 10, 2012 12:39 PM Page 1 of 1

Resources Postsecondary achievement reports for SFSF (c)(12):

https://p1pe.doe.virginia.gov/postsec_public/postsec.do?dowhat=LOADREPORT_C12

Frequently Asked Questions about postsecondary achievement reports: http://www.doe.virginia.gov/school_finance/arra/stabilization/reported_data/assurance_c/faq_c12.pdf

Postsecondary enrollment reports for SFSF (c)(11) : https://p1pe.doe.virginia.gov/postsec_public/postsec.do?dowhat=LOADREPORT_C11

Frequently Asked Questions about postsecondary enrollment reports: http://www.doe.virginia.gov/school_finance/arra/stabilization/reported_data/assurance_c/faq_c11.pdf

Information about Virginia’s College and Career Readiness Initiative: http://www.doe.virginia.gov/instruction/college_career_readiness/index.shtml

The detours and decisions shown in this presentation were based on real people and actual events that occurred in the Virginia Department of Education. In some cases, the names of the people taking those actions were omitted to protect their identities.

For More Information ContactNathan Carter, Ph.D.

Nathan.Carter@doe.Virginia.gov

Deborah Jonas, Ph.D.Deborah.Jonas@doe.Virginia.gov

Virginia Department of Education