2015-16 TitleIIA(3) Improving Teacher Quality
Competitive Grants-Proposal Needs/Evaluation Expectations
Technical Assistance Webinar—Round IIMarch 16, 2015
1
Needs Data: Category 1 Projects
• Proposals with all new participants must include:• Evidence at least 1 LEA partner meets federal criteria• Teacher Data
• Pre-proposal needs survey among potential teacher participants in high need and other targeted LEAs
• Student Data• Indication of student needs based on available test scores or
other pertinent student data
• Other Relevant Needs Data• Curriculum, instruction, assessment needs can also be included if they
relate to the proposal, but is not required
3
Needs Data: Category 2 Projects
• Proposals with new and returning participants should include:• Evidence at least 1 LEA partner meets federal criteria• Teacher Data
• Pre-proposal needs survey of potential new teachers; most recent survey data from returning teachers
• Student Data• Indication of student needs new and returning teachers based on available
test scores or other pertinent student data
• Research/evaluation data from current project• Briefly describe current project intervention (in existing project) and
value based on evaluation findings• Provide summary of findings to date, from evaluation of current project,
particularly data representing effects on teachers (including comparison group data) and their students
4
• Teacher Participant Needs Survey Data• MDE ITQ needs survey with option to add project-specific questions (use
only the subject matter section that is pertinent to your proposal)• Current needs survey reflects State Board of Education approved standards
at the time of this grant cycle• Available for five subject areas eligible for an ITQ grant:
• English Language Arts (Common Core State Standards)• Mathematics (Common Core State Standards)• Science (Michigan Science Grade Level Content Expectations)• Social Studies (Michigan Social Studies Grade Level Content Expectations)• World Languages (Michigan Standards and Benchmarks)
• May be administered in paper/pencil or electronic form
5
Needs Survey Data: All Projects
• SAMPI can assist in preparing customized paper/pencil or on-line survey
• SAMPI can compile and return raw data for use in proposal/ program development (need 2-3 work-day turnaround time; data provided in Excel or Access)
• Projects must submit a summary of needs survey data (tables, graphs, summary statements, etc.) and any additional data (student test scores, etc.) in the narrative portion of the proposal
• Submit raw data with code numbers in database (databases available for basic needs survey) in appendix
6
Needs Survey Assistance: All Projects
Example From Needs Survey
INSTRUCTION AND ASSESSMENT PROFESSIONAL DEVELOPMENT NEEDS
Need Already
Met
Low Need
(1)
High Need
(5)
Priority?(Check one for each.)
(Check one box for each.) Yes No
A. Using inquiry-based instructional strategies 0 1 2 3 4 5 Y N
B. Incorporating real-world applications into lessons 0 1 2 3 4 5 Y N
C. Learning how to integrate other curricular topics into lessons 0 1 2 3 4 5 Y N
7
PART A. INSTRUCTION AND ASSESSMENT PROFESSIONAL DEVELOPMENT NEEDS Rate each of the following items based on your personal professional development instructional and assessment needs, with 1 = low need; 5 = high need; 0 = need has already been met. Indicate whether the item is a high priority for you (yes or no).
Adding Project-Specific Items
All projects should have all respondents complete all existing items in all sections.
Projects may add specific instruction and assessment needs to Part B if desired. Extra rows are provided for additional items.
Part C is specific to proposed project content; each project should select their appropriate grade level content/subject matter section and delete all other content tables from the survey. Extra rows are supplied at the end of each content table are provided for additional items.
Part D of the needs survey is a section for projects to ask any open-ended questions desired.
• You may delete Part D if you do not have any other questions you would like to ask.
DO NOT REMOVE ANY ITEMS from the tables; if an item is not relevant to your project, simply ignore that data when analyzing needs survey results and planning your proposal.
8
Student Test Score Data
• Obtain Michigan standardized test score results for the LEA partners relevant to the subject area of your project
Directly from the LEAs
From MDE database• Include a table of student test data in the appropriate
section of your proposal narrative
9
Other Sources of Needs Data
10
• Other Data• Provide other data pertinent to partner needs if
available, such as LEA-generated student or teacher data, curriculum/instruction/assessment needs, etc.
• Previous Research/Evaluation Findings• Describe previous actions taken by IHE partner in
comparable projects to address identified needs• Describe value of proposed intervention based on IHE
partner research/evaluation findings
Analyzing and Submitting Needs Data
• Summarize and present all pertinent needs data
• Do not rely on only ONE source of needs data: identify commonalities in needs data across all sources; the same needs identified from different sources strengthens your case for which needs to address
• Review your needs data to determine links and relationships among the various types of available data
• Prioritize identified needs based on their importance and project-level resources/expertise to address them
11
Teacher/Building Commitment Forms
• Informal “Memo of Understanding” for teachers
• Informal “Memo of Understanding” showing partner school principal support
• Reminders:• Participation not duplication of effort in another project• Avoid conflict of interest for teachers or building
12
Evaluation Goals
1) Improve programming through use of evaluation data
2) Determine the impact of project and statewide effort on new and returning participants
3) Identify strengths and limitations of the projects
14
Proposal Section D (Narrative and Scoring Rubric): Evaluation
All sections of the scoring rubric must be clearly and adequately addressed in the narrative of your proposal.
Label each section of your proposal as per the sections of the rubric.
Project-Level Evaluation
• Evaluation strategies:• Common cross-site pre/post surveys• Pre/post lesson observations• Interview sample of participants• Collect teacher artifacts• Gather evidence of impact on students
• Use additional instruments/procedures specific to project as desired to determine impact or gather implementation data
• Gather and report participation/activity data• Support a person with adequate time dedicated to evaluation
• Internal staff or contractual, with appropriate expertise and without major program coordination duties
15
Cross-Site State-Level Evaluation
• Coordinated by SAMPI-Western Michigan University• Advise on local project use of data collection instruments
and procedures• Advise on evaluation as requested• Assist in coordinating cross-site meetings • Conduct observation of sample of project PD activities• Compile data from across projects• Prepare periodic statewide reports• Assist with pre-proposal needs surveys
16
Determine Impact on Teachers/Other Participants
• Pre/post surveys• Plus longitudinal analysis for continuing
projects
• Pre/post lesson observations• Plus longitudinal analysis for continuing
projects
• Sample of interviews of participants• Collect and analyze a sample of teacher
artifacts18
Determine Impact on Teachers/Other Participants
• Projects with both new and returning teachers (Category 2) must also report on differences observed• Comparison group analyses of data from
new and returning participants• Analyses may be quantitative, qualitative or
both
19
Determine Impact on Students of Participating Teachers
• Procedures to identify changes in student learning might include:• Pre/post tests• Pre/post surveys• Focus group interviews• Assessment of student work
• Collect sample of student artifacts to show changes
20
Required Participation in Cross-Site Activities
• Project directors and evaluators are expected to participate in cross-site activities throughout the duration of the grant• First six months: 1 face-to-face meeting in
Lansing, 1 webinar• Remainder of project funding period: 2 face-
to-face meetings in Lansing, 2 webinars
21
Required Common Reporting
• Required Common Reporting• Participation levels and demographics, PD types,
overall assessment of progress toward goals• Final evaluation report• End of project participant-specific data consistent
with cross-site statewide data collection• Sharing findings at cross-site sessions
22
End-of-Project Reporting
• Director Report•Use standard report format available in electronic version
•SAMPI can provide Access (or Excel) database to projects to maintain participation records and to facilitate report preparation
23
Part #1 Sample Reporting Tables
Category Number Major Target
Audience (Yes or No)
1. Number of different teachers served by the project
2. Number of different administrators served by the project
3. Number of different paraprofessionals served by the project
4. Number of different parents served by the project
5. Number of “others” served by the project
24
Part #2: Director Perceptionsof Project Accomplishments
• Rate progress toward outcomes• Provide evaluation/other evidence to
support rating
25
Part #2: Director Perceptions of Project Accomplishments-Sample Form
Section A:#1: Intended Outcome (Type in Outcome Statement):
Rating of Progress Toward Outcome: Rate the degree of progress you believe was made in accomplishing this outcome on a 5-point scale, with 1 = no progress and 5 = fully accomplished.
1 2 3 4 5
Evidence to Support Rating (Type in rational for your rating or, if pertinent, note that there is direct evidence in evaluation report section-see below):
Discussion of Progress in Evaluation Section
Yes No Page No.
26
Complete the following tables, one for each of your intended outcomes as per your proposal. Outcomes are statements of intended impacts or results that will occur as a result of your professional development programming or other project interventions. Prepare this section in collaboration with project evaluator.
Part #3: Nature of Professional Development/Interventions
• Hours, participants, schedule by type of intervention
• Role of college faculty/content experts• Problems planning, implementing project• Problems recruiting teachers
27
Part #3 :Nature of Professional Development/Interventions
Sample Form
PD Formats
Number of Hours of this PD
Provided Over Entire Project
PD Participants(Key: T=Teachers, A=Administrators,
PP=Para-Professionals, P=Parents, O=Others)
When was PD conducted?
(Key: RSD=Regular School Day, AS=After School, EVE=Evening, SAT=Saturday, SUM=Summer)
Workshops (usually one- or half-day sessions)
Institutes (5 or more days usually in summer)
College Course Work (for credit)
E-Learning Courses (self paced web-based course)
28
Part 3: Nature of PD/InterventionsSample Form
NamePosition/
InstitutionPrimary Role
Hours of Involvement
29
3. Use the chart below to describe higher education faculty (both content and education faculty) or other external expert.
PARTICIPATION IN YOUR PROJECT.
Part #4: Evaluation Report Required Components
Prepared by evaluator in collaboration with project director• All projects:
• Data collection• Progress toward project outcomes• Lesson observation data• Teacher and student artifacts• Impacts on participating students• Effectiveness of project partnership• Other
• Category #2 Projects• Results of comparison studies
30
Questions?
• SAMPI contact information:• Crystal Stein or Tessa Souers
• Mark [email protected]
• Tel: (269) 387-3791
SAMPI is located at Western Michigan University in Kalamazoo, MI
31