national reporting system for the 21 st century congress of adult education state directors

NATIONAL REPORTING SYSTEM FOR THE 21ST CENTURY

CONGRESS OF ADULT EDUCATION STATE DIRECTORS

Pre-Meeting Webinar

July 28-29, 2010

1

Introductions

Purpose of meeting

Review of issues to be discussed

Meeting structure

Questions

2

WEBINAR AGENDAWEBINAR AGENDA

2

To get your input on 6 changes planned or under consideration for the NRS

Changes that can be implemented by July 2011 and do not require reauthorization

Other changes being considered that will require more time, authority through reauthorization, or more research

3

PURPOSE OF MEETINGPURPOSE OF MEETING

3

Goal setting for employment, GED, and postsecondary entry— problematic

Difficulty identifying appropriate goals for students

Number of goals set appears too low

High fluctuation of performance rates within states and across years

44

GOAL SETTING CHANGES—GOAL SETTING CHANGES—ISSUES #1, 2, AND 3ISSUES #1, 2, AND 3

Discontinue goal setting; move to “automatic cohort definition”

Automatically select students for outcome based on student characteristics

Options considered:Include all students seeking work or only unemployed seeking work

Set a federal standard for number of students to include

55

ISSUE #1: EMPLOYMENT ISSUE #1: EMPLOYMENT OUTCOMESOUTCOMES

Automatically designate all students in the labor force who are unemployed as the cohort for which “entered employment” must be tracked.

Automatically designate all students who enter the program employed as the cohort for which “retained employment” must be tracked.

66

ISSUE #1: RECOMMENDATIONS ISSUE #1: RECOMMENDATIONS

How will this policy affect the population that local programs serve?

Should we continue to use program “exit” as a criterion for cohort identification and reporting?

What changes, if any, will states need to make to state-level MIS to implement the new policy, and at what cost?

77

ISSUE #1: DISCUSSION ISSUE #1: DISCUSSION QUESTIONS QUESTIONS

What are implications for the follow-up methods (survey v. data match)?

How can states overcome barriers to collecting valid Social Security Numbers?

What are the implications for state training to local providers?

How will this change affect local program funding, especially in states with performance-based funding?

Is this policy change feasible to implement in PY 2011-2012? 88

ISSUE #1: DISCUSSION QUESTIONS ISSUE #1: DISCUSSION QUESTIONS (CONT.) (CONT.)

Eliminate goal setting; options considered:

Include all students, track for multiple years

Set a standard for number of students to include for follow-up

Include only students who have a GED, high school diploma or enrolled in transition class.

99

ISSUE #2: POSTSECONDARY ISSUE #2: POSTSECONDARY FOLLOW-UPFOLLOW-UP

Automatically designate all students who - have earned a GED,

- have a secondary credential, or

- are enrolled in a class specifically designed for transitioning to community college (e.g., bridge program, college readiness)

as the cohort for which “entry into postsecondary education” must be tracked.

1010

Issue #2: Recommendation Issue #2: Recommendation

What effect will this policy have on local programs—both desirable and unintended effects? Are there ways to ameliorate unintended effects on service delivery?

How will states identify and report on those enrolled in a class specifically designed for transitioning to community college?

What changes will states need to make to state-level MIS, and at what cost?

11

Issue #2: Discussion Questions Issue #2: Discussion Questions

What are implications for the follow-up methods?

How can states that do not have adequate postsecondary data systems implement this change?

How can states that cannot or do not collect Social Security Numbers overcome barriers?

What are the implications for state training to local providers?

What effect, if any, will this change have on states using performance-based funding?

1212

Issue #2: Discussion Questions (Cont.) Issue #2: Discussion Questions (Cont.)

Options considered:Eliminate goal setting; include all students who take the GED Tests

Eliminate goal setting; include all students at ASE levels who do not have a credential

Maintain goal setting—no change.1313

ISSUE #3: SECONDARY CREDENTIAL ISSUE #3: SECONDARY CREDENTIAL FOLLOW-UPFOLLOW-UP

Match GED test records for all students who take tests during the year to calculate a pass rate.

For states with adult high school, report the number of students in high adult secondary education (ASE) who obtain a high school diploma.

For states with EDP, report the number of students enrolled in the assessment phase who obtain a high school diploma.

 1414

ISSUE #3: RECOMMENDATION ISSUE #3: RECOMMENDATION

What effects will this policy have on the populations served by local programs?

What changes will states need to make to state-level MIS, and at what cost?

What are the implications for the follow-up method (survey v. data match)?

How can states that cannot or do not collect valid SSNs overcome barriers?

What are the implications for state training to local providers? 1515

ISSUE #3: DISCUSSION ISSUE #3: DISCUSSION QUESTIONS QUESTIONS

Should OVAE require states to report test scores in addition to educational levels gains?

Long-term issue, requiring further study

Psychometric expert and OVAE will discuss methods and implications

16

ISSUE #4: OTHER EDUCATIONAL ISSUE #4: OTHER EDUCATIONAL GAIN MEASURES—REPORTING TEST GAIN MEASURES—REPORTING TEST

SCORESSCORES

Collect outcome data on students after transition to postsecondary education

Options:Require programs to follow all students after transition

Require students in integrated education and training programs to be tracked

17

ISSUE #6: MEASURING PROGRESS ISSUE #6: MEASURING PROGRESS IN POSTSECONDARY EDUCATIONIN POSTSECONDARY EDUCATION

Require programs with integrated education and training (IET) models to track progress towards and completion of a credential in the program of study in which the student is enrolled.

18

ISSUE #6: RECOMMENDATION ISSUE #6: RECOMMENDATION

How should postsecondary retention and completion be defined at the federal level?

How can states collect postsecondary retention and completion data at the postsecondary level?

What are the implications for State MIS?

1919

ISSUE #6: DISCUSSION ISSUE #6: DISCUSSION QUESTIONS QUESTIONS

Do states have postsecondary-level databases that provide this information? Can this information be collected within local postsecondary institutions?

Are there unintended consequences of this policy that may impede the use of the IET model? How can the effect of such consequences be ameliorated?

20

ISSUE #6: DISCUSSION QUESTIONS ISSUE #6: DISCUSSION QUESTIONS (CONT.) (CONT.)

GED students exit before posttesting–negative effect on educational gain

Educational gain not always appropriate measure for GED prep students

Options considered:Count GED as educational advancement

Allow posttesting at fewer hours

Create separate track for GED students, exempt from educational gain 21

ISSUE #5: REFINING OUTCOMES ISSUE #5: REFINING OUTCOMES MEASURES FOR GED STUDENTSMEASURES FOR GED STUDENTS

Create a separate level for GED prep students to include all students who score at the secondary level on any NRS-approved test battery.

Educational gain not reported for these students; they are not counted in calculation of ed gain for any NRS level.

The only reportable NRS outcome for these students is attainment of a secondary credential.

2222

ISSUE #5: UNDER ISSUE #5: UNDER CONSIDERATION CONSIDERATION

Would adding this track be a positive change in your state? Are there negative implications? Would the net result be positive or negative?

How would dropping the requirement for educational gain posttesting affect you?

Is this approach better aligned with GED delivery models in your state?

How much time would it take to implement?

Would your state support this recommendation?

2323

ISSUE #5: DISCUSSION ISSUE #5: DISCUSSION QUESTIONS QUESTIONS

MEETING STRUCTURE

DUPONT HOTELAUGUST 4-5, 2010

2424

SIX ISSUES FOR DISCUSSION

Issues 1, 2, 3, and 6—Day 1

Issues 4 (panel) and 5—Day 2

Think Sheets for states to prepare responses to questions Feedback Forms for states to offer comments—to ensure that OVAE hears from and considers input from every state

2525

Process10 Minutes for States to Reflect on Issue and Questions (Using Think Sheets)

Round-Robin Responses to Questions to Allow Input from All States

General Discussion of Issue

States Complete and Submit Feedback Forms

FOUR BREAK-OUT FOUR BREAK-OUT ROOMSROOMS

26

GROUND RULES FOR GROUND RULES FOR ROUND-ROBIN PROCESSROUND-ROBIN PROCESS

One state speaks at a time; One State—One Voice

No one individual dominates discussion

When identifying challenges, also identify potential solutions or ways to overcome challenges (Don’t say only “We can’t/ my state can’t” without offering some solutions.)

If nothing to add when it’s your state’s turn, say “pass.”

No interrupting round-robin approach; opportunity for clarification after every state has responded to the questions

Use Parking Lot for long-term issues 2727

QUESTIONS?

COMMENTS?

2828

Thank Thank You!You!

See you August 4-5 in Washington, DC!

Safe Travels!