understanding performance measures · – ssn’s are the source to match the data between ged and...

51
Understanding Performance Measures: Processes, Methodologies, & Calculations Bill Schneider, AVP, Research and Performance Management, NCCCS Dr. Kristen Corbell, Coordinator of Research Projects, NCCCS Joann Ingoglia, Data Compliance Coordinator, NCCCS

Upload: others

Post on 12-Aug-2020

1 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Understanding Performance Measures · – SSN’s are the source to match the data between GED and LEIS data for the 2002 series of the test. – For the 2014 series of the GED, the

Understanding Performance Measures:Processes, Methodologies, & Calculations

Bill Schneider, AVP, Research and Performance Management, NCCCS

Dr. Kristen Corbell, Coordinator of Research Projects, NCCCS

Joann Ingoglia, Data Compliance Coordinator, NCCCS

Page 2: Understanding Performance Measures · – SSN’s are the source to match the data between GED and LEIS data for the 2002 series of the test. – For the 2014 series of the GED, the

Goals

• Go over how the Baseline and Goal were calculated

• Go over the data sources, methodologies, calculations, and reporting of the performance measures

• Discuss potential changes to the measures

2

Page 3: Understanding Performance Measures · – SSN’s are the source to match the data between GED and LEIS data for the 2002 series of the test. – For the 2014 series of the GED, the

3

Page 4: Understanding Performance Measures · – SSN’s are the source to match the data between GED and LEIS data for the 2002 series of the test. – For the 2014 series of the GED, the

Performance Measures

A. Basic Skills Student Progress

B. GED Diploma Passing Rate

C. Developmental Student Success Rate in College‐Level English Courses

D. Developmental Student Success Rate in College‐Level Math Courses

E. First Year Progression

F. Curriculum Student Completion

G. Licensure and Certification Passing Rate

H. College Transfer Performance

4

Page 5: Understanding Performance Measures · – SSN’s are the source to match the data between GED and LEIS data for the 2002 series of the test. – For the 2014 series of the GED, the

5

Page 6: Understanding Performance Measures · – SSN’s are the source to match the data between GED and LEIS data for the 2002 series of the test. – For the 2014 series of the GED, the

Baseline and Goal Calculation

• The Baseline and Goal are calculated and remain in effect for three years

• The Baseline and Goal were calculated based upon 1 – 3 years of data, depending on the measure and what was valid

• The Baseline is two standard deviations below the mean

• The Goal is one standard deviation above the mean

6

Page 7: Understanding Performance Measures · – SSN’s are the source to match the data between GED and LEIS data for the 2002 series of the test. – For the 2014 series of the GED, the

A. Basic Skills Student Progress

PurposeTo ensure adults with low literacy skills are progressing academically toward basic skill attainment necessary for employment and self-sufficiency

DescriptionPercentage of students who progress as defined by an educational functioning level.

MethodologyDenominator: Basic skills students attempting 60 or more contact hours during program year. Excludes ASEH initial placements.Numerator: Basic skills students attempting 60 or more contact hours during program year, who complete the program year at a higher educational functioning level. Excludes high adult secondary education initial placements.Note: Pre-test is considered if it occurs up to 90 days prior to program year

7

Page 8: Understanding Performance Measures · – SSN’s are the source to match the data between GED and LEIS data for the 2002 series of the test. – For the 2014 series of the GED, the

41.5% 42.0%

1

25

26

6

10%

20%

30%

40%

50%

60%

70%

2011-12 2012-13 Benchmarks

Goal 51.2%

Avg 41.3%

Base 20.6%

A. BASIC SKILLS STUDENT PROGRESSPercentage of students who progress as defined by an educational functioning level

8

Page 9: Understanding Performance Measures · – SSN’s are the source to match the data between GED and LEIS data for the 2002 series of the test. – For the 2014 series of the GED, the

Commonly Asked Questions

• What is an Educational Functioning Level?

– Education gain measures student literacy gains as a result of instruction. Education gain is defined by students advancing one or more educational functioning levels (EFL), where each EFL describes a set of skills or competencies in reading, writing, numeracy, speaking, listening, and functional and workplace areas.

• Basic Skills Programs have 12 literacy levels (4 Adult Basic Education, 2 Adult Secondary Education, and 6 English as a Second Language).

• All Basic Skills students must test with an NRS approved test.

• Students post-test in the same test and component as the initial placement for the program year. Test publisher guidelines set how many contact hours must pass between the pre and post tests.

9

Page 10: Understanding Performance Measures · – SSN’s are the source to match the data between GED and LEIS data for the 2002 series of the test. – For the 2014 series of the GED, the

Example of Cohort for 2014 Report

Step 1Use the NRS guidelines methodology to compute and assign Educational Functioning Levels (EFL) and initial placement for all LEIS students within the program year.-Program Year range is July 1, 2012 to June 30, 2013 for the 2014 report.-Compute EFL gains for all LEIS students within the reporting year.

Step 2Select all Basic Skills students in the

reporting year where:-Total Contact hours >= 60.

Remove from cohort all students where Initial NRS placement level is equal to High Adult Secondary Education (ASEH).Success is defined as all students from the cohort who achieve a level gain.

Students with initial NRS placement of High Adult Secondary Education cannot attain a level gain as this is the highest EFL. Some of these students would be included in the GED Diploma Passing Rate.

LEIS placement and level gains computed in Colleague is not being used due to errors. All placements are currently programmed in SAS at the System Office and provided to the colleges on a quarterly basis. Colleague is currently being corrected to automatically place students.

10

Page 11: Understanding Performance Measures · – SSN’s are the source to match the data between GED and LEIS data for the 2002 series of the test. – For the 2014 series of the GED, the

Educational Functioning Levels

ABE/ASE ESL

Beginning ABE Literacy Beginning ESL Literacy

Beginning Basic Education Low Beginning ESL

Low Intermediate Basic Education High Beginning ESL

High Intermediate Basic Education Low Intermediate ESL

Low Adult Secondary Education High Intermediate ESL

High Adult Secondary Education Advanced ESL

Note: Credit is given for Advanced ESL when exit criteria are met as specified by test publisher.

11

Page 12: Understanding Performance Measures · – SSN’s are the source to match the data between GED and LEIS data for the 2002 series of the test. – For the 2014 series of the GED, the

Example Student in Cohort

EFL_Level_Desc STUDENT_ID TOT_CONTACT_HOURS

TEST_DT TEST_CD TEST_COMP_CD TEST_SCORE

LOW ADULT SECONDARY EDUCATION 600618 192.5 18FEB2014 TABES READ 585

HIGH INTERMEDIATE BASIC EDUCATION

600618 192.5 19FEB2014 TABES TOMA 529

LOW INTERMEDIATE BASIC EDUCATION

600618 192.5 19FEB2014 TABES LANG 506

LOW ADULT SECONDARY EDUCATION 600618 192.5 21MAR2014

TABES LANG 581

LOW ADULT SECONDARY EDUCATION 600618 192.5 18APR2014 TABES TOMA 591

• Student is included in the cohort since they have more than 60 contact hours

• Initial placement is Low Intermediate Basic Education• Post Test must be in TABE Survey Language• Student received a Level Gain since the student increased from Low

Intermediate Basic Education to Low Adult Secondary Education12

Page 13: Understanding Performance Measures · – SSN’s are the source to match the data between GED and LEIS data for the 2002 series of the test. – For the 2014 series of the GED, the

Student Not in Cohort

EFL_Level_Desc STUDENT_INFO_SKEY

TOT_CONTACT_HOURS TEST_DT TEST_CD TEST_COMP

TEST_SCORE

BEGINNING BASIC

EDUCATION

648139 55 05SEP2013 CASAS READ 203

BEGINNING ABE LITERACY

648139 55 08MAY2014 CASAS READ 184

• Student has less than 60 hours, so they are not included in cohort.

13

Page 14: Understanding Performance Measures · – SSN’s are the source to match the data between GED and LEIS data for the 2002 series of the test. – For the 2014 series of the GED, the

B. GED Diploma Passing Rate

PurposeTo ensure quality GED preparation and high levels of GED attainment

DescriptionPercentage of students taking at least one GED test during a program year who receive a GED diploma during the program year

MethodologyDenominator: GED students with an Initial Placement of ASEL or ASEH who take at least one GED test during the program year (July 1 – June 30) and have 12 or more total contact hours.Numerator: GED students with an Initial Placement of ASEL or ASEH who take at least one GED test during the program year (July 1 – June 30), and have 12 or more total contact hours, and receive a GED diploma

14

Page 15: Understanding Performance Measures · – SSN’s are the source to match the data between GED and LEIS data for the 2002 series of the test. – For the 2014 series of the GED, the

69.6% 71.7%

2

26

20

10

30%

40%

50%

60%

70%

80%

90%

100%

2011-12 2012-13 Benchmarks

Goal 82.0%

Avg 73.6%

Base 49.3%

B. GED DIPLOMA PASSING RATEPercentage of students taking at least one GED test during a program year who receive a GED diploma during the program year

15

Page 16: Understanding Performance Measures · – SSN’s are the source to match the data between GED and LEIS data for the 2002 series of the test. – For the 2014 series of the GED, the

Example of Cohort for 2014 Report

Step 1Obtain official GED test records from Oklahoma Scoring where student attempted at least one test during the program year - Reporting Year range is July 1, 2012 to June 30, 2013- Merge test records with LEIS students for the program year

Step 2Select students in the reporting year where:

- Total Contact hours >= 12- Initial NRS placement in (“ASEL” “ASEH”)

Success is defined as all students from the cohort who Passed a GED during the program year.

16

Page 17: Understanding Performance Measures · – SSN’s are the source to match the data between GED and LEIS data for the 2002 series of the test. – For the 2014 series of the GED, the

Example for 2014 Report

ID Passed Pass Date Writing DateSoc Stu

Date Science Date Reading Date Math DateINIT_NRS

PLACEMENT

123654N

12/7/2012 12/14/2012 11/9/2012 12/7/2012 ASEL

937830 Y 5/9/2013 3/6/2011 2/8/2013 3/6/2011 5/9/2013 12/14/2012 ASEH

693038N

7/26/2012 8/3/2012 8/3/2013 ASEH

392304N

11/23/2012 10/26/2011 10/19/2012 11/16/2012 9/21/2012 ASEL

292840 Y 9/14/2012 9/14/2012 9/13/2012 9/13/2012 7/19/2012 7/19/2012 ASEL

354835 N 5/25/2013 ASEL

17

Page 18: Understanding Performance Measures · – SSN’s are the source to match the data between GED and LEIS data for the 2002 series of the test. – For the 2014 series of the GED, the

Example of Cohort for 2015 Report

Step 1Obtain Official GED test records where student attempted at least one test during the program year (each test series will be analyzed separately to determine the cohort and then merged for the final calculation)- Get Official GED test records from Oklahoma Scoring for the 2002 series of the tests (including those testing centers with waivers to administer the paper based test in 2014)- Obtain Official GED test records from GEDTS for the 2014 series of the test (January 1- June 30, 2014)- Reporting Year range is July 1, 2013 to June 30, 2014- Merge test records with LEIS students for the program year

Step 2Select students in the reporting year where:

- Total Contact hours >= 12- Initial NRS placement in (“ASEL” “ASEH”)

Success is defined as all students from the cohort who Passed a GED during the program year. 18

Page 19: Understanding Performance Measures · – SSN’s are the source to match the data between GED and LEIS data for the 2002 series of the test. – For the 2014 series of the GED, the

Commonly Asked Questions

• Are all students taking a GED included? Are colleges held accountable for people who come in and take the GED, but are not students at our college?– No. Only students who are enrolled in a Basic Skills class with 12 or more contact

hours and have an initial placement of ASE Low or ASE High are included. 12 or more contact hours ensure students received instruction from the college.

• Why are only ASE Low and ASE High students included?– ASE Low and ASE High students are functioning at a level equivalent to 9th grade or

higher, and thus the ones to most likely successfully pass a high school equivalency exam. Analyses were done on other initial placements to aid in this decision.

• Where does the data come from?– Oklahoma Scoring, the official source for GED data for the 2002 series of the tests.

They provide us a file with all students taking a GED 2002 series during the program year.

– SSN’s are the source to match the data between GED and LEIS data for the 2002 series of the test.

– For the 2014 series of the GED, the source of the data will be GEDTS and it is matched on first name, last name, and date of birth since SSN is not required by GEDTS in 2014.

19

Page 20: Understanding Performance Measures · – SSN’s are the source to match the data between GED and LEIS data for the 2002 series of the test. – For the 2014 series of the GED, the

C. Developmental Student Success Rate in College Level English Courses

PurposeTo ensure remedial English and reading coursework prepares students to succeed in credit-bearing English courses

DescriptionPercentage of previous developmental English and/or reading students who successfully complete a credit English course with a grade of “C” or better upon their first attempt.

MethodologyDenominator: All students enrolling in their first credit English course during an academic year who also enrolled in a developmental English and/or reading course during the same or previous academic year. Does not include students who do not attend the class i.e. transfer credits, credit by exam or reported grades of ‘NA’ and ‘NS’.Numerator: All students earning a grade of “C” or better in their first credit English course during an academic year who enrolled in a developmental English and/or reading course in the same or previous academic year. Does not include students who do not attend the class (ietransfer credit, credit by exam, etc).

20

Page 21: Understanding Performance Measures · – SSN’s are the source to match the data between GED and LEIS data for the 2002 series of the test. – For the 2014 series of the GED, the

66.3% 66.2% 64.7% 64.5% 64.3%

0

1

28

25

4

30%

40%

50%

60%

70%

80%

90%

2008-2009 2009-2010 2010-2011 2011-2012 2012-2013 Benchmarks

Goal 74.9%

Avg 64.4%

Base 45.2%

C. DEVELOPMENTAL STUDENT SUCCESS RATE IN COLLEGE-LEVEL ENGLISH COURSESPercentage of previous developmental English or reading students who successfully complete a credit English course

21

Page 22: Understanding Performance Measures · – SSN’s are the source to match the data between GED and LEIS data for the 2002 series of the test. – For the 2014 series of the GED, the

How is the cohort built?

• Includes all students in a developmental English or reading course in 2011-12 or 2012-13 academic year (ENG, RED, or DRE less than 100).

• Subset only the students who enroll in a credit English course in 2012-13.

• Remove anyone who was also enrolled in a credit English course in 2011-12.

• Sort credit English courses ascending by semester and course number. The first English course taken is selected. Thus if Eng111 is taken in the fall and the spring, the fall is selected. If ENG111 is taken in the fall and ENG112 is taken in the spring, ENG111 is selected.

• Remove all students from the denominator who never attended a class, as indicated by a local college grade of NS or NA.

22

Page 23: Understanding Performance Measures · – SSN’s are the source to match the data between GED and LEIS data for the 2002 series of the test. – For the 2014 series of the GED, the

Example of Cohort for 2013 Report

Step 1Where CURRICULUM_COLLEGE_YEAR

equal 2012 and 2013 Select all students enrolled in Courses

with prefix ENG0 RED0 DRE0

Step 2-Remove all students from Step 1 who enrolled in

Credit English in 2012-Delete from cohort if REPORTING _TERM is (Fall

2011, Spring or Summer 2012) and COURSE_PREFIX = “ENG” and COURSE_NUMBER

greater than 099

Step 3 Final CohortAll students left attempted a

Developmental English or Reading in 2012 or 2013 and their first credit level

English in 2013 (2012FA 2013SP 2013SU)

Also select all credit courses taken during this time frame.

This is the subset used for analysis. Students who earned a grade of “NA” or “NS” in their only Credit English Course are removed from the cohort.

These students would have been in the previous year’s report.Also, if the first Credit English was attempted before their Dev Course, student is removed from Cohort.

23

Page 24: Understanding Performance Measures · – SSN’s are the source to match the data between GED and LEIS data for the 2002 series of the test. – For the 2014 series of the GED, the

D. Developmental Student Success Rate in College Level Math Courses

PurposeTo ensure remedial math coursework prepares students to succeed in credit-bearing math courses

DescriptionPercentage of previous developmental math students who successfully complete a credit math course with a “C” or better upon the first attempt.

MethodologyDenominator: All students enrolling in their first credit math course during an academic year who also enrolled in a developmental math course during the same or previous academic year. Does not include students who do not attend the class i.e. transfer credits, credit by exam or reported grades of ‘NA’ and ‘NS’.Numerator: All students earning a “P”, “C”, or better in their first credit math course during an academic year who enrolled in a developmental math course in the same or previous academic year. Does not include students who do not attend the class (i.e. transfer credit, credit by exam, etc).

24

Page 25: Understanding Performance Measures · – SSN’s are the source to match the data between GED and LEIS data for the 2002 series of the test. – For the 2014 series of the GED, the

65.3% 65.7% 65.6% 64.1% 63.6%

Column1

4

24

22

8

30%

40%

50%

60%

70%

80%

90%

2008-2009 2009-2010 2010-2011 2011-2012 2012-13 Benchmarks

Goal 75.4%

Avg 64.4%

Base 47.5%

D. DEVELOPMENTAL STUDENT SUCCESS RATE IN COLLEGE-LEVEL MATH COURSESPercentage of previous developmental Math studentswho successfully complete a credit Math course

25

Page 26: Understanding Performance Measures · – SSN’s are the source to match the data between GED and LEIS data for the 2002 series of the test. – For the 2014 series of the GED, the

How is the cohort built?

• Take all students in a developmental Math course in 2011-12 or 2012-13 academic year (DMA or MAT less than 100).

• Subset to only the students who enroll in a credit Math course in 2012-13.

• Remove anyone who was also enrolled in a credit Math course in 2011-12.

• The first Math course attempted in the reporting year (2012-13) is selected.

• Thus if MAT141 is taken in the fall and then repeated again in spring, the fall record is selected. If MAT141 is taken in the fall and MAT151 is taken in the spring, MAT141 is selected.

• Remove any students who have never attended a class, as selected by a local college grade of NS or NA.

26

Page 27: Understanding Performance Measures · – SSN’s are the source to match the data between GED and LEIS data for the 2002 series of the test. – For the 2014 series of the GED, the

Example of Cohort for 2014 Report

Step 1Where CURRICULUM_COLLEGE_YEAR equal

2012 and 2013 Select all students enrolled in courses with

prefix MAT0 DMA0

Step 2-Remove all students from Step 1 who

enrolled in Credit Math in 2011-Delete from cohort if REPORTING _TERM in (fall 2011, spring or summer 2012) and

COURSE_PREFIX = “MAT” and COURSE_NUMBER greater than 099

Step 3 Final CohortAll students left attempted a

Developmental Math in 2012 or 2013 and their first credit level Math in 2013 (2012FA 2013SP 2013SU)

Also select all credit courses taken during this time frame.

This is the subset used for analyses. Students who earned a grade of “NA” or “NS” in their only credit Math Course are removed from the cohort.

These students would have been in the previous year’s report.Also, if their first credit math was attempted before their Dev Math, student is removed from cohort

27

Page 28: Understanding Performance Measures · – SSN’s are the source to match the data between GED and LEIS data for the 2002 series of the test. – For the 2014 series of the GED, the

E. First Year ProgressionPurposeTo ensure first-year students reach an academic momentum point that helps predict future credential completion

DescriptionPercentage of first-time fall credential-seeking students attempting at least twelve hours within their first academic year who successfully complete (“P”, “C” or better) at least twelve of those hours

MethodologyDenominator: A fall cohort of credential-seeking students (program code A, D, C) enrolled in curriculum courses at a college for the first time after high school graduation. Must attempt at least twelve hours (including developmental and withdraw) within the first year fall, spring, and summer semesters. Includes those dually enrolled previously at the same institution and excludes students previously enrolled at another college. Does not include students who do not attend the class i.e. transfer credits, credit by exam or reported grades of ‘NA’ and ‘NS’.Numerator: Those within the cohort above who complete at least twelve hours (including developmental) with a “P”, “C” or better within the first year.

29

Page 29: Understanding Performance Measures · – SSN’s are the source to match the data between GED and LEIS data for the 2002 series of the test. – For the 2014 series of the GED, the

67.6% 66.8% 67.2% 67.7% 68.3%

47.3%

3

22

23

10

40%

50%

60%

70%

80%

90%

2008-2009 2009-2010 2010-2011 2011-2012 2012-2013 Benchmarks

Goal 74.6%

Avg 68.3%

Base 53.2%

E. FIRST YEAR PROGRESSIONPercentage of first-time fall credential-seeking students attempting at least twelve hours within their first academic year who successfully complete at least twelve of those hours.

30

Page 30: Understanding Performance Measures · – SSN’s are the source to match the data between GED and LEIS data for the 2002 series of the test. – For the 2014 series of the GED, the

First Time Fall Cohort Schematic

Previous High School Students

Any students who was previously dual enrolled in High school and CC (T code or CCP Pathway) are only included in the cohort if these conditions are met:

They graduate from High school on or before the fall Year.They enrolled in a credential program for the first time in the current Fall at the same college which they were dual enrolled.

This group and the students from step 2 constitute the First Time Fall cohort

National Student Clearinghouse.

The clearinghouse database is used for a secondary analyses to verify and remove students from Step 1 with a postsecondary registration before August 1 of Fall year.

Student name and date of Birth are used as matching variable between databases.

Additionally, students with no postsecondary enrollment record but whose first registration term from data warehouse is before 2001 Fall are removed from the cohort.

Final cohort is unduplicated student across 58 colleges.

From CRPFA (Data warehouse) Select all credential seeking students enrolled in Fall term i.e. where CURRICULUM_CD in ( A D C).

Filter in only those who graduated from High school on or before the calendar year of the Fall term

where HIGH_SCHOOL_GRADUATE_IND = “Y” and year(HIGH_SCHOOL_LAST_ATTD_DT) < = Fall year.

Search Data warehouse historical records from 2001 Fall and remove all students with registration record from any college. Students who were previously enrolled in any of the Early College High School programs are handled separately.

Step 3Step 2Step 1

31

Page 31: Understanding Performance Measures · – SSN’s are the source to match the data between GED and LEIS data for the 2002 series of the test. – For the 2014 series of the GED, the

Performance Metric

• Only students from the first-time fall cohort who attempted 12 or more credit hours during the first year (Fall, Spring & Summer) are evaluated for this measure.

• Success is defined as the percentage of students who complete with a grade of “C” or better for at least 12 of the attempted credit hours.

• Notes:– Grade of “W” is counted as attempted– Grades of “NS” and “NA” are not counted as attempted

credit hours

33

Page 32: Understanding Performance Measures · – SSN’s are the source to match the data between GED and LEIS data for the 2002 series of the test. – For the 2014 series of the GED, the

Commonly Asked Questions

• Does it matter which semester the student enrolls for the first time?– Only students enrolling for the first time during a fall semester are

included in any cohort.• How are dual enrolled students treated?

– A student is only included after they have graduated from high school and re-enroll for the first time in the Fall at the same college in which they were previously dually-enrolled.

– If a student is dually-enrolled in one college, graduates from high school and enrolls in a second college, the student is not included in any cohort.

• If a student enrolls in two or more colleges, which college is held accountable?– The first college the student enrolls in after high school graduation

is counted.

34

Page 33: Understanding Performance Measures · – SSN’s are the source to match the data between GED and LEIS data for the 2002 series of the test. – For the 2014 series of the GED, the

More Commonly Asked Questions

• How are students matched in the National Student Clearinghouse?– The National Student Clearinghouse is used as a secondary

check to make sure it is the student’s first enrollment in post-secondary education. Match is by Student Name and Date of Birth.

• How are credit hours for split courses and ‘W’ grades handled since they are not recorded in Data warehouse?– Our work around until these issues are addressed in Data Warehouse

has been to pull the credit hours directly from the CRPFA flat file report.

• How confident are you that this methodology captures all first time fall students?– We are very confident with the checks put in place to search for first

time students using our historical records and a secondary check from the Clearinghouse.

35

Page 34: Understanding Performance Measures · – SSN’s are the source to match the data between GED and LEIS data for the 2002 series of the test. – For the 2014 series of the GED, the

F. Curriculum Student Completion Rate

PurposeTo ensure student completion and persistence toward a post-secondary credential

DescriptionPercentage of first-time fall credential-seeking students who graduate, transfer, or are still enrolled with 36 hours after six years

MethodologyDenominator: A fall cohort of credential-seeking students (program code A, D, C) enrolled in curriculum courses at a college for the first time after high school graduation. Includes those dually enrolled previously at the same institution and excludes students previously enrolled at another college.Numerator: Those within the cohort above who by the fall that occurs six years after original cohort designation either graduate (A, D, or C), transfer to a four year institution, or are still enrolled during that sixth fall semester previously completing 36 non-developmental hours.

36

Page 35: Understanding Performance Measures · – SSN’s are the source to match the data between GED and LEIS data for the 2002 series of the test. – For the 2014 series of the GED, the

39.3% 40.0% 38.6%41.1%

43.0%

Column1

27

10

21

20%

30%

40%

50%

60%

2003 2004 2005 2006 2007 Benchmarks

Goal 45.6%Avg 43.6%

Base 28.6%

F. CURRICULUM STUDENT COMPLETION RATEPercentage of first-time fall credential-seeking students who graduate, transfer, or are still enrolled with 36 hours after six years

37

Page 36: Understanding Performance Measures · – SSN’s are the source to match the data between GED and LEIS data for the 2002 series of the test. – For the 2014 series of the GED, the

Performance Metrics

Completers Students from the cohort who earned an Associate, Diploma or Certificate.

These awards must occur on or before their 6th Fall term after their initial Fall term. For 2006 cohort, students have until Fall 2012 to complete.

All completers are verified from Data Warehouse as reported in the CRPFA.

Transfer All students from the cohort who did not earn a credential but transferred to a

4 year university as archived in the National Student Clearinghouse records. Students from the 2006 Fall cohort will need to have transferred by October

of 2012

Persisted and Completed 36 non-developmental hours Students who are neither completers nor transfers, but are still enrolled

during that 6th Fall previously completing 36 non-development hours. For the 2006 cohort these students should have earned their 36 hours by the

end of the 2012 Summer. 38

Page 37: Understanding Performance Measures · – SSN’s are the source to match the data between GED and LEIS data for the 2002 series of the test. – For the 2014 series of the GED, the

Commonly Asked Questions

• How are the cohorts built?

– The same methodology used for First Year Progression is used. All students in the cohort are included in the denominator.

• Are completers credited to the graduating college?

– The college in which the students started during the first fall term of the cohort gets the credit for completers and transfers.

– For 36 non-developmental hours the student has to be enrolled in the same college that they started in their first fall.

• Are No-Show students included?

– No. Evaluation is not based on grades since this measure focuses on completers after 6 years.

39

Page 38: Understanding Performance Measures · – SSN’s are the source to match the data between GED and LEIS data for the 2002 series of the test. – For the 2014 series of the GED, the

How can colleges monitor progress?

• For the First Year progression measure, we can send you a file containing all the students in the Fall cohort in February.– Colleges can then use this to track students

performance at the end of each semester.

• For the Curriculum Completion measure we can also provide the cohort in advance and colleges can work with the NSC to get transfers and generate their own report at any time.– We are willing to share our SAS program and offer any

SAS assistance in using the code. 40

Page 39: Understanding Performance Measures · – SSN’s are the source to match the data between GED and LEIS data for the 2002 series of the test. – For the 2014 series of the GED, the

G. Licensure and Certification Passing Rates

PurposeTo ensure programmatic coursework prepares students to competently practice in their chosen profession

DescriptionAggregate institutional passing rate of first time test-takers on licensure and certification exams. Exams included in this measure are state mandated exams which candidates must pass before becoming active practitioners.

MethodologyDenominator: All licensure and certification exams taken for the first time during the licensure agency’s most recent reporting year. Only includes state mandated exams which candidates must pass before becoming active practitioners.Numerator: Licensure and certification exams passed on first attempt during the licensure agency’s most recent reporting year.

41

Page 40: Understanding Performance Measures · – SSN’s are the source to match the data between GED and LEIS data for the 2002 series of the test. – For the 2014 series of the GED, the

2012-13 Eligible Licensure and Certification Exams

American Registry of Radiologic TechnologistsNuclear Medicine Technology ExamRadiation Therapy ExamRadiography Exam

Council of Interstate TestingDental Hygiene Exam

Federal Aviation AdministrationAirframe ExamGeneral ExamPower Plant Exam

NC Board of Cosmetic Art ExaminersApprentice ExamCosmetology ExamCosmetology Instructor ExamEsthetician ExamManicurist Exam

NC Board of Massage & Bodywork TherapyMassage & Body Work Therapist

NC Board of NursingPractical Nursing ExamRegistered Nursing Exam

NC Board of OpticiansOpticianry Exam

NC Board of Physical Therapy ExaminersPhysical Therapist Assistant Exam

NC Department of Justice, Criminal JusticeStandards DivisionBasic Law Enforcement Training (BLET) Exam

NC Department of Justice Sheriff’s Standards DivisionDetention Officer

NC Office of Emergency Medical ServicesEMD ExamEMT ExamEMTI‐I ExamEMT‐P Exam

NC Real Estate CommissionProvisional Real Estate Broker

NC Veterinary Medical BoardVeterinary Medicine Technology Exam

42

Page 41: Understanding Performance Measures · – SSN’s are the source to match the data between GED and LEIS data for the 2002 series of the test. – For the 2014 series of the GED, the

G. LICENSURE AND CERTIFICATION PASSING RATEAggregate institutional passing rate of first time state-mandated test-takers on licensure and certification exams

85.9% 86.4% 87.0% 86.5%85.0%

Column1

4

20

31

3

60%

70%

80%

90%

100%

2008-2009 2009-2010 2010-2011 2011-2012 2012-2013 Benchmarks

Goal 91.7%

Avg 83.2%

Base 71.0%

43

Page 42: Understanding Performance Measures · – SSN’s are the source to match the data between GED and LEIS data for the 2002 series of the test. – For the 2014 series of the GED, the

Commonly Asked Questions

• How do you determine which exams are included?– Two years ago, a focus group reviewed current list and

selected exams based on a set criteria: 1. exam is required for employment in North Carolina, and 2. data was available for community college students based on first-time results of test-takers.

• Why are some state mandated exams not included in the list used to calculate the performance measure? – Results associated with first-time test takers is not

identified in data sets

– Data is not centrally collected by agency/board

– Agency/board cited security concerns44

Page 43: Understanding Performance Measures · – SSN’s are the source to match the data between GED and LEIS data for the 2002 series of the test. – For the 2014 series of the GED, the

H. College Transfer Performance

PurposeTo ensure the academic success of community college students at a four-year university or college

DescriptionAmong community college associate degree completers and those who have completed 30 or more credit hours who transfer to a four-year university or college, the percentage who earn a GPA of 2.00 or better after two consecutive semesters within the academic year at the transfer institution.

MethodologyDenominator: Students with an associate degree or at least 30 articulated transfer credits enrolled during the fall and spring semesters at a four-year institution who were enrolled at a community college during the previous academic year. Only includes North Carolina based four-year institutions and four-year institutions which the individual community college has an articulated transfer agreement.Numerator: Students included in the denominator who have earned a GPA of 2.00 or better aggregated over the fall and spring semesters at the transfer institution.

45

Page 44: Understanding Performance Measures · – SSN’s are the source to match the data between GED and LEIS data for the 2002 series of the test. – For the 2014 series of the GED, the

87.5% 87.4% 88.0% 87.8%

60.0%

1

26

26

5

60%

70%

80%

90%

100%

2008-2009 2009-2010 2010-2011 2011-2012 Benchmarks

Goal 93.8%

Avg 87.8%

Base 71.2%

H. COLLEGE TRANSFER PERFORMANCEAmong students with at least 30 credits prior to transferring to a 4-year institution, the percentage who earn a GPA of 2.00 or better after two consecutive semesters at the transfer institution

46

Page 45: Understanding Performance Measures · – SSN’s are the source to match the data between GED and LEIS data for the 2002 series of the test. – For the 2014 series of the GED, the

How is the Cohort created?

• We identify all students by college who have an associate degree or have completed 30 or more articulated credits to UNC General Administration or with private colleges (with whom we have a memorandum of understanding).

• UNCGA and private colleges provide us with a summary report of the number of students per college who have a 2.0 GPA or greater in their first two semesters.

• The college the student is last enrolled at is the college the student is associated with.

47

Page 46: Understanding Performance Measures · – SSN’s are the source to match the data between GED and LEIS data for the 2002 series of the test. – For the 2014 series of the GED, the

What colleges do we receive transfer performance data on?

Appalachian State University

East Carolina University

Elizabeth City State University

Fayetteville State University

North Carolina A&T State University

North Carolina Central University

North Carolina State University

UNC Asheville

UNC-Chapel Hill

UNC Charlotte

UNC Greensboro

UNC Pembroke

UNC Wilmington

UNC School of the Arts

Western Carolina University

Winston-Salem State University

NC School of Science and Mathematics

Barton College

Belmont Abbey College

Brevard College

Campbell University

Catawba College

Chowan University

Davidson College

Elon University

Gardner‐Webb University

Guilford College

Mars Hill College

Meredith College

Methodist University

Mount Olive College

Montreat College, N.C.

Wesleyan College

Pfeiffer University

Queens University of Charlotte

Salem College

Shaw University 48

Page 47: Understanding Performance Measures · – SSN’s are the source to match the data between GED and LEIS data for the 2002 series of the test. – For the 2014 series of the GED, the

Commonly Asked Questions

• Why aren’t all of the North Carolina Independents included?– They do not participate in the National Student Clearinghouse

– They do not sign a data sharing agreement with the System Office• Note: Some community colleges have encouraged local independents to participate, thus having

them included in the cohort.

• Can we have access to student level results?– Not at this time. We only receive aggregate results from UNC-GA and the

Independent Colleges and Universities.

• Can data from colleges outside the state be included?– Yes. Community college must:

• Provide the System Office with the articulation agreement with partnering college • The partnering college must be part of the National Student Clearinghouse• The partnering college must sign a data sharing agreement with the System Office

49

Page 48: Understanding Performance Measures · – SSN’s are the source to match the data between GED and LEIS data for the 2002 series of the test. – For the 2014 series of the GED, the

50

Page 49: Understanding Performance Measures · – SSN’s are the source to match the data between GED and LEIS data for the 2002 series of the test. – For the 2014 series of the GED, the

Three Year Review

• All performance measures are reevaluated every three years

• The 2015 Performance Measures Report will be the last based on the eight current measures

• Modifications will be made to measures associated with the 2013-14 academic year and reported in 2015

• Dr. Brenda Kays, Stanley Community College President, is chairing a committee tasked with the review of the measures

51

Page 50: Understanding Performance Measures · – SSN’s are the source to match the data between GED and LEIS data for the 2002 series of the test. – For the 2014 series of the GED, the

Questions and Answers

52

Page 51: Understanding Performance Measures · – SSN’s are the source to match the data between GED and LEIS data for the 2002 series of the test. – For the 2014 series of the GED, the

Who to Contact

• Contact Dr. Kristen Corbell for questions about Basic Skills Progress, GED Diploma Passing Rate, and Developmental Success: [email protected]

• Contact Bill Schneider for questions about First year progression and Curriculum Completion: [email protected]

• Contact Joann Ingoglia for questions Licensure Pass Rates and College Transfer: [email protected]

53