nce program assessment report template (2014-2015) · 2015-2016 nce program assessment report for...

2015-2016 NCE Program Assessment Report for TLA Program 1

NCE Program Assessment Report Template (2014-2015) Program: Teaching, Learning and Assessment (TLA) Assessment Report Writer(s): Karen F. Tardrew Section I: Program Overview A. 2015-2016 (June 30, 2015 – July 1, 2016)

Undergraduate Graduate

Total Number of Candidates in Program

72

Number of Candidates who Completed Program

19

B. Overview of Program Teaching Learning and Assessment (TLA) Program Overview and Conceptual Framework Launched in the 1980’s (previously called the IDS Program), The Teaching, Learning and Assessment program (TLA) foreshadowed what would become accepted practice in American teacher education. Its vision was innovative and courageous. The cohort structure, which is now commonplace in higher education, had its origin here, where we have a strong belief in the power of a reflective professional community as a catalyst for learning. In the TLA program, candidates assume high degrees of educational empowerment and professionalism within a model that is strong in proven adult education elements: interaction, autonomy, reflection, dialogue, productivity, and group support. IDS/TLA was also early in offering extensive coursework in action research well before teacher action research became a prominent practice in teacher education. Over the last 30 years, the IDS program has served an estimated 20,000 teachers in Chicago, Wisconsin, Florida, St. Louis, Washington, DC, and Heidelberg, Germany, equipping teachers of all levels and disciplines to become researchers of their own practice. In the TLA program, weekly cohort meetings and dialogue journals invite candidates to reflect on and inquire into their teaching practices, contexts, and the larger systemic and cultural factors that impact their students’ learning. The face-to-face field-based cohorts serve as laboratories in which teachers may practice a continuous cycle of action and reflection, which they in turn take back to their classrooms. We maximize our candidates’ contact with the resources around them and reinforce and build their skills in learning from a variety of living experiences as well as from texts and technology. The TLA program fosters candidate progress toward our nine outcomes: 1. Critical Reflection: Striving to be critically reflective about one’s teaching and profession 2. Issues: Identifying social, economic, and political issues in teaching, learning, and schooling 3. Action Research: Engaging in action research on personally and professionally relevant topics that will contribute to teaching, learning, and schooling 4. Community: Helping build a community of learners that engage in meaningful conversation about critical issues of education and teaching practice 5. History: Reflecting critically on teaching and learning within the context of one’s own life history and teaching career 6. Diversity: Engaging with the complexities of diversity and multiculturalism in our culture and our classrooms 7. Technology: Exploring ways to integrate technology with one’s professional life 8. Change Agents: Acting as change agents advocating for all learners 9. Student Growth: Integrating assessment into their professional practice to impact and evidence student growth and learning


Our mission is to help each teacher gain a renewed sense of what it means to be a teacher. To this end, the program promotes democracy, diversity, inclusiveness, and personal creativity in as many aspects as possible. Through readings, discussions, and experiences, the program encourages teachers to develop expanded visions, not only for their own teaching, but for their students, schools, and communities as well. Graduates of this program will be:

• Leaders in differentiated learning – able to develop curriculum and instruction approaches that address social-emotional learning factors, learning styles, cultural and language backgrounds, and cognitive, emotional, behavioral, and developmental differences in multiple-needs general education classrooms. As teacher-leaders, graduates will also be prepared to help others in their schools to improve their own practices in differentiating learning.

• Experts in assessment – adroit in designing assessment approaches to drive effective teaching and learning in real-time (“formative” evaluation) as well as in making appropriate use of standardized testing.

• Accomplished action researchers – agile in applying techniques for observing, documenting, reflecting on, and refining the learning process to create continuous improvement in outcomes for students.

• Exceptional teacher-leaders in their current environments. Since the TLA program is built around enabling candidates to actively address the pressing realities of their classrooms, candidates are able to apply new skills and techniques – and share them with others –to bring immediate value to their classrooms and schools. They also learn how to place themselves in their school systems as facilitators, evolving change agents, and leaders advocating for improving education for all learners.

Distinctive Qualities of the Program The TLA Program has been organized around four conceptual strands that are interwoven throughout the seven quarters of the program. These strands include the implementation of action research models of classroom inquiry, assessment and reflection on teaching and learning, inquiry into historical and current philosophical approaches to teaching and learning, and the art of reaching all learners through engagement and differentiation (see chart below). While the sequence of courses has been organized quarter by quarter, the work of the courses is grounded in the experiences of the teachers in each group. New coursework each quarter brings about new challenges and opportunities for active engagement, but the effect is cumulative. There is both a collective memory and a collective experience in the group, which can revive issues discussed previously with new information and challenge the ongoing inquiry. When possible with one core instructor, the integration of the courses during each quarter strengthens the integrity of the classroom inquiry process as teachers experience a variety of possible paths toward student success. While the classes are organized by quarters, the actual work of the classes may overlap the quarter boundaries. C. Sequence and Courses within Program

Qua

rter

Cou

rse

# Course Prefix

Course Name

SH

1 1 TLA 578 Developing School Cultures 2 2 TLA 541 Assessing Teaching and Learning: Introduction to

Assessment 2

2 3 FND 509 Building on Educational Foundations 3 4 ESR 531 Exploring Action Research: Action Research I 2

3 5 TLA 588 Constructing Curriculum for Engaging the Whole Learner

3

6 ESR532 Engaging in Action Research: Action Research II 2


D. Delivery Models within Program __ Alternative Licensure (ATL) __ Face to Face _X_ Online _X_ Blended/Hybrid _X_ Offered in FL _X_ Offered in WI __ Residency Program (i.e. AUSL, UStep, SStep) Section II: Program Signature/Key Assessments The following 6 assessment tools are used to assess the performance of our candidates and the TLA program. This is a brief overview of the tools. A full description is included in section IV: 1. Field Experience Documentation and Reflection - Reflection completed by candidate each quarter focused on 10 hours of fieldwork per course 2. TLA Outcomes Survey - Administered three times throughout program: Quarter 2, 5, and 7 through LiveText Survey 3. Action Research Demographic Survey - Completed by candidates during last action research course (ESR 534) through LiveText Survey 4. Research Project Analysis Checklist (RPA) - Instructors complete the RPA for each candidate in conjunction with last action research course ESR 534 through LiveText Survey 5. Assessment Project Analysis Rubrics (APA) (I, II, III) - Completed by instructor and candidate in conjunction with assessment strand courses, TLA 541, TLA 542, TLA 543 6. NCE Candidate Dispositions Assessment - Administered by NCE during the following courses/transition points – TLA 578 (Q1), TLA 588 (Q3) and TLA 579 (Q7)

4 7 TLA 591 Meeting Complex Challenges of Contemporary Classrooms: Field Study

≤ 2

8 TLA 542 Linking Assessment of Teaching and Learning: Field Study

≤ 2

5 9 TLA 589 Differentiating for Diverse Learners 3 10 ESR 533 Continuing Action Research: Action Research III 2

6 11 TLA 594 Examining Curriculum Perspectives: Independent Study

≤ 2

12 TLA 543 Reflecting on Teaching, Learning, and Assessment 2

7 13 TLA 579 Engaging With Critical Multicultural Perspectives 3

14 ESR 534 Completing Action Research: Action Research IV 2

TOTAL SH/QH 32


Assessment Domain Name of Signature/Key Assessment

(Note if assessments are different for Undergraduate,

Graduate, Alternative Programs or Other Delivery

Models)

When the Assessment Is Administered

(indicate course or other marker)

Transition Point in Program

(as applicable)

Assessment of Content Knowledge I (Note: Non SPA programs do not have required state licensure tests and may not have an assessment for this domain.)

N/A – non SPA program N/A N/A

Assessment of Content Knowledge II (Local Key Assessment)

1. TLA Outcomes Survey (Candidate completes through LiveText) 2. Research Project Analysis Checklist (RPA) (Instructor completes through Livetext)

Quarter 2, 5, 7 Quarter 7 (with last Action research course ESR 534)

Beginning, Middle, End of Program End of Program

Assessment of Professional/Pedagogical Knowledge for Developing and/or Supporting Meaningful Learning Experiences

Research Project Analysis Checklist (RPA) (Instructor completes through LiveText)

Completed during last Action Research course, Quarter 7 (ESR 534)

End of Program

Assessment of Field or Clinical Experiences/ Application to Professional Work

Field Experience Documentation and Reflection

End of all quarters in program (1-7)

Beginning, Middle and End of Program

Assessment of Candidate Impact on Student Learning

1. Assessment Project Analysis Rubrics I, II, III (APA) (Candidate and Instructor complete rubric) 2. Action Research Demographic Survey (Candidate completes through LiveText)

End of each of the three Assessment courses (TLA 541, TLA 542, TLA 543) Completed during last Action Research Course (ESR 534)

Beginning, Middle and End of Program End of Program

Assessment of Candidate Dispositions

NCE Candidate Dispositions Assessment

Completed three times in program during TLA 578 (Q1), TLA 588 (Q3) and TLA 579 (Q7)


Assessment of Candidate Diversity Proficiencies

Research Project Analysis Checklist (RPA) (Instructor completes through LiveText

Quarter 2, 5, 7


Assessment of Candidate Technology Proficiencies

TLA Outcomes Survey (Candidate completes through LiveText)

Quarters 2, 5, 7 Beginning, Middle and End of Program


Section III: Relationship of Assessments to Program Outcomes and Standards A. Alignment of Program Assessments to NCE Conceptual Framework/Outcomes

NCE Conceptual Framework/Outcomes

NCE Candidates:

Program Assessments

Envision, articulate and model democratic and progressive education

• TLAOutcomesSurvey

Design powerful learning environments that integrate appropriate technologies

• ResearchProjectAnalysisChecklist(RPA)• AssessmentProjectAnalysisI,II,III

Design powerful learning environments that utilize multiple meaningful assessments

• AssessmentProjectAnalysisI,II,III• ResearchProjectAnalysisChecklist

Design powerful learning environments that enable self-directed learning

• ResearchProjectAnalysisChecklist

Work collaboratively in diverse communities and with diverse learners to achieve learning goals

• TLAOutcomesSurvey• DemographicSurvey

Advocate for democratic values, equity, access and resources to assure educational success for all

• TLAOutcomesSurvey• DemographicSurvey

Cultivate curiosity and excitement for learning in themselves and others

• FieldExperienceDocumentationandReflection

Respect and learn from other peoples, cultures, and points of view • FieldExperienceDocumentationandReflection

Demonstrate a caring attitude in recognizing the needs of others and acting to promote their growth

• ResearchProjectAnalysisChecklist

Act with confidence and self-knowledge to assume professional leadership roles and responsibilities

• FieldExperienceDocumentationandReflection

Use information from self and others to continuously improve • AssessmentProjectAnalysisI,II,III• ResearchProjectAnalysisChecklist

B. Alignment of Program Assessments to Professional Standards and Program Outcomes

TLA Program Outcomes NCATE Standards/Domains

Program Assessments

Critical Reflection: Striving to be critically reflective about one's teaching and profession.

Content Knowledge

* Assessment Project Analysis I, II, III

Issues: Identifying social, economic and political issues in teaching, learning and schooling.

Professional Knowledge

* RPA Checklist


Technology: Exploring ways to integrate technology with one's professional life.

Technology Proficiencies

* TLA Outcomes Survey

Action Research: Engaging in action research on personally and professionally relevant topics that will contribute to teaching, learning and schooling.

Field Experience Content Knowledge

* RPA Checklist * Field Experience Documentation and Reflection

Life History: Reflecting critically on teaching and learning within the context of one's own life history and teaching career.

Diversity


Community: Helping build a community of adult learners who engage in meaningful conversation about critical issues of education and teaching practices.

Diversity: Engaging with the complexities of diversity and multiculturalism in our culture and our classrooms.

Dispositions

Diversity Proficiencies

* NCE Candidate Dispositions Assessment

* RPA Checklist

Change Agent: Acting as change agents advocating for all learners.

Dispositions


Student Growth:

Impact on Student Learning

* Demographic Survey * Assessment Project Analysis I, II, III

Section IV: Assessment Tools and Data Analysis A. Assessment of Content Knowledge I (For Programs with State Content Tests) Not applicable, as the TLA program is a non-SPA program and does not include state content tests. B. Assessment of Content Knowledge II (Local Assessment) The TLA program uses two assessment tools to assess candidate content knowledge, the TLA Outcomes Survey and the Research Project Analysis Checklist (RPA). Assessment #1 TLA Outcomes Survey 1. Name of and Description of the Assessment and What it Assesses The TLA Outcomes Survey is an online survey that is housed in Livetext and candidates access it there. https://c1.livetext.com/misk5/formz/public/79394/o3d3Ioq9YY This survey was developed around the 9 outcomes of the TLA program and each outcome includes 3-6 indicator statements in which candidates self-assess their development in relation to each of the outcomes. As is evidenced in the TLA Program Key Assessments chart in Section II, the data from specifically chosen TLA


Outcomes Survey indicators are used in assessing candidate acquisition of content knowledge, and candidate technology proficiencies. 2. How and when the Assessment is Implemented The TLA Outcomes Survey is administered three times throughout the program: (1) at the end of quarter 2, (2) at the end of quarter 5 and, (3) at the end of quarter 7 or the end of program. Candidates are directed to complete the survey in Livetext and the URL is provided to them. 3. When Data is Analyzed and by Whom The data is collected through LiveText throughout the year. Individual instructors are encouraged to review their candidates’ data upon completion in each of the quarters in which the assessment is administered. In late spring, the data is analyzed by TLA faculty during a data retreat designed to allow faculty to review all data from candidate and program assessments in the aggregate. 4. Remediation or Support Provided to Candidates Not Meeting Criteria This assessment is administered in quarter 2, 5 and 7, which are the quarters in which many students are enrolled in the required sequence of action research courses. Since the TLA program is taught mainly in a cohort style format and these are required courses, full time faculty are assigned to work with the candidates within a cohort to provide consistency and close support while completing these courses. Due to this, instructors work with candidates who are working toward meeting criteria and provide necessary support as they grow in their ability to demonstrate TLA program outcomes, specifically through their seven quarter long action research project work. 5. Blank sample of the assessment tool: Name of Tool: TLA Outcomes Survey, Appendix A 6. Blank sample of the scoring guide/rubric The TLA Outcomes survey is a survey completed in LiveText by the candidate. Each of the questions on the survey requires the candidate to respond using a six-point scale and add narrative response if desired:

1 - I’m not sure what this is 2 - I have a long way to go on this one 3 - I’ve begun to make progress 4 - I’ve made substantial progress 5 - I feel competent in this regard 6 - I feel highly competent in this regard

7. Aggregated Assessment Data Content Knowledge Data from Outcomes Survey at Beginning of Program (First Administration Point of Assessment) BELOW is the data represented in 2014-2015. The data sets were incomplete and reported errors from 2015-2016 We are in the process of revising the assessments for TLA to address these issues.

OUTCOME 2 ISSUES: Identifying social, economic and political issues in teaching, learning and schooling

Beginning of Program

Average Score N =26

Middle of Program Average

Score N =24

End of Program Average

Score N =20

Question 1 – I am aware of social, economic, and political issues that affect my teaching, my students, and American schools.

4.33 4.04 4.67

Question 2 – I am questioning my assumptions about how schooling best occurs.

4.33 4.00 4.80

Question 3 – Based on new information and/or classroom discussions, I am rethinking some of my past educational practices.

4.52 4.16 4.81

Question 4 – I am able to articulate a rationale for the kinds of teaching and schooling I believe that learners need.

4.63 4.24 4.71

Overall Average of Scores for Issues

4.45 4.11 4.75


OUTCOME 4

COMMUNITY: Helping build a community of learners who engage in meaningful conversation about critical issues of teaching, learning and schooling


Average Score N =26

Middle of Program Average

Score N =24

End of Program Average

Score N =20

Question 1 – I bring issues, problems, and insights to the attention of the group.

4.59 4.20 4.71

Question 2 – I contribute to learning experiences for my TLA group. 5.26 4.68 5.14 Question 3 – My cohort/group shares information about resources and opportunities.

5.22

4.48 5.05

Question 4 – I have gained from hearing about the experiences of my colleagues.

5.19 4.44 5.19

Overall Average of Scores for Community

5.07 4.45 5.02

8. Narrative Interpretation of What Data Means in Relation to Candidates’ Content Knowledge This is feedback from 2015-2016. TLA faculty believe candidates should be achieving at the level of "3" when responding to the questions at the beginning and middle points of administration within the TLA program, thus indicating that they have begun to make progress. By the end point of administration (last quarter/end of program) candidates should be achieving at least at the level of “5” or above, demonstrating that they have made become competent in this area (content knowledge) and program outcomes throughout their development in the program. The self-reported scores from the candidates fall in the range that candidates indicate they are “making substantial progress” within all indicators of this assessment. The mean scores for the second administration of this assessment shows the scores a bit lower than when first completed (beginning of program). This may be due to the fact that candidates have progressed through the program and have completed 5 quarters and have a better understanding for what the TLA program outcomes entail and include. In quarter 2 with the first administration, candidates do not have a full understanding of how the outcomes manifest within the program and in their demonstration of the outcomes and therefore may score themselves higher than the second administration where they have spent considerable time in the program. By the end of the TLA program, the mean score for candidates in these two outcomes related to content knowledge are 4.75 (issues) and 5.03 (community), thus demonstrating that they assess themselves as competent and meet program expectations for demonstrating TLA content knowledge. Assessing content knowledge of TLA candidates reflects the data from two assessments (outcomes survey and the RPA checklist) and narrative interpretation in the aggregate will be provided below following the and from the RPA checklist. Assessment #2 Research Project Analysis Checklist (RPA) 1. Name of and Description of the Assessment and What it Assesses The Research Project Analysis Checklist (RPA) is a tool that was developed in the 2008-2009 academic year by the IDS assessment committee and has been in use successfully ever since. The RPA was shared with the IDS faculty and they were trained on how to use the tool to assess candidates’ action research projects. The primary purpose of the RPA is to have faculty assess each individual candidate on the candidate’s action research project. In incorporating this assessment to the program’s assessment system, faculty felt the need to more fully evaluate candidate performance of action research projects, the project work that runs through the entire program. Prior to its inception, narrative “reader reports” (done at three transition points throughout a candidate’s program) were used as the primary assessment and feedback tool for the action research projects. This RPA assessment tool was originally designed around the 8 IDS Program Outcomes and refined to reflect the 9th program outcome, which was added when the program was redesigned into the TLA program. Many


of the 44 indicators on the RPA checklist originate directly from statements within the TLA Student Action Research Guide, and indicators directly from the TLA Outcomes. The RPA is seen as a significant instructor view of candidate performance as it relates to their program-long work on their action research projects that are implemented within their teaching/field placements. The RPA checklist not only assesses the quality of the work performed in the Action Research project (content knowledge), but also extends into other realms of candidate performance. Indicators on the RPA are aligned with specific areas of candidate performance as well as for specific program outcomes. As can be seen on the Program Key Assessments chart in section III, the results from specifically chosen RPA indicators are used in assessing candidate acquisition of content knowledge, candidate professional/pedagogical content knowledge and candidate diversity proficiencies. See the aggregated data below that delineates the indicators from the RPA assessment that assess candidate content knowledge. 2. How and when the Assessment is Implemented The RPA assessment of each candidate is administered near the end of the candidate’s 7 quarter program at the completion of the candidate’s program-long action research project. The instructor completes the RPA online through Livetext. 3. When Data is Analyzed and by Whom In late spring, the data is analyzed by TLA faculty during a data retreat designed to allow faculty to review all data from candidate and program assessments in the aggregate. 4. Remediation Provided to Candidates Not Meeting Criteria Since the TLA program is mainly taught in a cohort style format and the action research courses (4) are required, full time faculty are assigned to work with the candidates in these courses in order to provide consistency and close support while completing these key courses. Due to this, instructors work with candidates throughout the program on all aspects of their action research projects (including the indicators reflected in the RPA checklist that reflect Content Knowledge) and provide support and remediation to candidates where needed so they will successfully complete their action research project and thus meet all criteria on the RPA checklist. 5. Blank sample of the assessment tool: Name of Tool: Research Project Analysis Checklist (RPA), Appendix B 6. Blank sample of the scoring guide/rubric There is a four point scale for each indicator in which instructors assign a score when completing the checklist. 1 – Not met 2 – Partially met 3 – Met 4 – Beyond na – Not Applicable 7. Aggregated Assessment Data

Indicators from the RPA tool that represent TLA candidates’ content knowledge include: Content Knowledge Indicators from RPA Tool Average

Score N=20

Reflects on how teaching and learning function in his/her classroom 3.15 Speculates what changes might be beneficial to his/her classroom and/or students 3.00 Purposefully looks to gain insight into a classroom phenomenon 3.00 Builds understanding from how the students and the teacher are affected by specific incidents and classroom experiences

3.15

Notices and selects specific classroom moments or incidents that relate to his/her research questions

3.00

Thoughtfully contemplates what he/she and his/her students are experiencing 3.00 Meaningfully uses instructional strategies described in the literature to inform her/his research, teaching and role as a professional

3.00


Meaningfully alters his/her curriculum and research along the way in response to what he/she observes and what the students do and say

3.15

Uses his/her students as the major measure of how well the project is going 3.00 Interacts with a wide range of sources of information (i.e. conversations with colleagues, interviews of students and peers, books, articles, films, internet sources, professional conferences, the arts, etc.)

3.15

Demonstrates how one’s own classroom can be a community of learners 3.15 Keeps alive particular student stories along with those stories unique to the teacher 3.00 Brings in students’ life histories within the project and the analysis of data 2.86 Recognizes the diverse nature of students’ engagements with learning that relate to the teaching strategies that she/he is using

3.00

Includes evidence that a variety of student voices are heard and responded to 3.00 Documents an appreciation of individual learning within a class context 3.00 Develops new educational insights about teaching and learning, curriculum, self, students, or others

3.15

Stretches her own and her students’ capacities, through imagining richer outcomes for her teaching and/or student learning Average Scores for Content Knowledge 2015-2016

3.15

3.05 Average Scores for Content Knowledge 2014-15 3.58

Average of Scores for Candidate Content Knowledge 2013-14 2.74 Average of Scores for Candidate Content Knowledge 2012-13 3.67

8. Narrative Interpretation of What Data Means in Relation to Candidates’ Content Knowledge TLA faculty believe that candidates should be achieving at a “3” (Met) or “4” (beyond) for advanced achievements on the indicator and in demonstrating their acquisition of TLA content knowledge. The data shows that candidates fully meet program expectations related to the following indicator correlated with content knowledge: Reflects on how teaching and learning function in his/her classroom (3.15), builds understanding from how the students and the teacher are affected by specific incidents and classroom experiences (3.15), and thoughtfully contemplates what he/she and his/her students are experiencing (3.15). Other indicators show that candidates are well on their way to meeting expectations; Demonstrates how one’s own classroom can be a community of learners (3.15), Develops new educational insights about teaching and learning, curriculum, self, students, or others (3.15), interacts with a wide range of sources of information (i.e. conversations with colleagues, interviews of students and peers, books, articles, films, internet sources, professional conferences, the arts, etc.) (3.15), documents an appreciation of individual learning within a class context (3.00) and stretches her own and her students’ capacities, through imagining richer outcomes for her teaching and/or student learning (3.15). In comparing the 2014-15 data to the 2015-16 data: There was a small N for the data from the RPA in the 2013-14 academic year. Data from only 6 candidates was collected and aggregated from one cohort with one instructor. This may be the main reason the candidates’ scores are much lower in 2013-2014. Having a small N may skew the data, thus not providing an accurate view of the sum of the TLA candidates’ content knowledge as assessed with this tool. Furthermore the scores for 2014-15 have dropped slightly (insignificant) which reflects the need to look at other data sources due to the small sample thus representing data that is more valid. In general the data remained stable having met the proficient “3” level. Based on the data from the outcomes survey and the RPA tool, there is evidence that TLA candidates are making substantial progress and demonstrating attainment of the content knowledge required of the TLA program. C. Assessment of Professional/Pedagogical Knowledge for Developing and/or Supporting Meaningful Learning Experiences


The TLA program uses components of the Research Project Analysis Checklist (RPA) to assess professional/pedagogical knowledge for supporting meaningful learning experiences. 1. Name of and Description of the Assessment and What it Assesses See the description from B above of the Research Project Analysis Checklist (Content Knowledge assessment). See the aggregated data below that delineates the indicators from the RPA that assess candidate professional/pedagogical knowledge. 2. How and when the Assessment is Implemented The RPA assessment of each candidate is administered near the end of the candidate’s 7 quarter program at the completion of the candidate’s program-long action research project. The instructor completes the RPA online through Livetext. 3. When data is Analyzed and by Whom In late spring, the data is analyzed by TLA faculty during a data retreat designed to allow faculty to review all data from candidate and program assessments in the aggregate. 4. Remediation Provided to Candidates Not Meeting Criteria Since the TLA program is mainly taught in a cohort style format and the action research courses (4) are required, full time faculty are assigned to work with the candidates in these courses in order to provide consistency and close support while completing these key courses. Due to this, instructors work with candidates throughout the program on all aspects of their action research projects (including the indicators reflected in the RPA checklist that reflect Professional/Pedagogical Knowledge) and provide support and remediation to candidates where needed so they will successfully complete their action research project and thus meet all criteria on the RPA checklist. 5. Blank sample of the assessment tool: Name of Tool: Research Project Analysis Checklist (RPA), Appendix B 6. Blank sample of the scoring guide/rubric: There is a four point scale for each indicator in which instructors assign a score when completing the checklist. 1 – Not met 2 – Partially met 3 – Met 4 – Beyond na – Not Applicable 7. Aggregated Assessment Data

The indicators from the RPA tool that represent our candidates’ professional/pedagogical knowledge include: Professional/Pedagogical Content Knowledge Indicators from RPA Tool Average

Score N=20

Honors unanswered questions and points of confusion throughout the research process 3.00 Thoughtfully reflects on his/her journey through the research 3.00 Makes explicit the understandings upon which her/his judgments and decisions are made 3.00 Formulates new realizations that lead to further his/her journey into knowing and understanding 3.00 Makes explicit the social, political, and economic context in which she/he is working 3.00 Uses inquiry to help understand life as it is happening in his/her classroom and speculate what changes might be beneficial

3.00

Questions and/or analyzes social, cultural, economic, and/or gender issues within the action research story and its analysis 3.15 Meaningfully uses instructional strategies described in the literature to inform her/his research, teaching and role as a professional 3.00 Uses his/her personal experience with the topic to fuel the inquiry and the research project 3.00 Uses the research project as an authentic inquiry into him/herself as a teacher 3.00


Meaningfully alters his/her curriculum and research along the way in response to what he/she observes and what the students do and say 3.00 Looks for insights and meanings about learning and teaching emerging from the inquiry process 3.15 Continues her/his story of the inquiry through an active engagement with resources 3.00 Interacts with a wide range of sources of information (i.e. conversations with colleagues, interviews of students and peers, books, articles, films, internet sources, professional conferences, the arts, etc.) 3.15 Records ideas he/she has while interacting with a wide range of resources 3.15 Accepts him/herself as a researcher and exercises her/his own voice as a member of the community of resource producers 3.00 Interacts with colleagues within the work of action research 3.00 Values the interrelatedness of self as the researcher to what he/she is studying 3.00 Embraces his/her questions and feelings of ambiguity as part of the learning process 3.00 Shows an appreciation of differing perspectives in describing the data and in interpreting the meanings of the data 3.00 Records data through writing or other media (video, photography, online) that capture stories or incidents of classroom life and meaning

Red 3.15 Demonstrates that technology skills have been applied 3.00 Identifies potential new directions to engage in because of one’s study 3.00 Takes on new leadership roles or engages in new actions with others due to what is learned through the action research 2.86 Develops new educational insights about teaching and learning, curriculum, self, students, or others 3.15 Stretches her own and her students’ capacities, through imagining richer outcomes for her teaching and/or student learning Average Scores for Candidate Professional Knowledge 2015-2016

3.15

3.04 Average Scores for Candidate Professional Knowledge 2014-15

3.54

Average of Scores for Candidate Professional Knowledge 2013-14 2.65

Average of Scores for Candidate Professional Knowledge 2012-13 3.58 8. Narrative Interpretation of What Data Means in Relation to Candidates’ Professional/Pedagogical Knowledge TLA faculty believe that candidates should be achieving at a “3” (Met) or “4” (beyond) for advanced achievements on the indicators in demonstrating their acquisition of TLA pedagogical content knowledge. The data on all indicators on the RPA assessment show that TLA candidates in the aggregate are meeting the criteria (at the met level) for the action research project and demonstrating expectations of professional/pedagogical knowledge. No mean for any of the indicators on this assessment fall below the met level. As is our practice to compare RPA data from year to year, this year it is not valid to compare the scores from year to year due to our small and inconsistent N numbers. D. Assessment of Field or Clinical Experiences/Application to Professional Work The TLA program uses the Field Experience Documentation and Reflection assessment to assess candidate application to professional work. 1. Name of and Description of the Assessment and What it Assesses


In the TLA program, our master’s level candidates engage in field experience within their own classroom or educational settings and are required to document 10 hours per quarter of field experience or classroom application of course and program content for each course in which the candidate is enrolled. If a candidate is enrolled in all courses in the TLA program (14), he/she completes 140 total hours of required fieldwork. These activities document candidates’ application of the theories and practices encountered in the coursework, as well as their growth and application of the nine outcomes of the TLA program. The purpose of field experience is to nurture candidates to make connections to their thinking, deepen their insights into these new connections, and ultimately guide them in discovering how these insights can, have, or will transform their professional practice. Candidates are required to reflect on how they see their work within their professional settings demonstrating their application of the TLA program content and outcomes. In the spring of 2015, a standardized format and data collection form was created for this assessment, however, consistent data was not collected and therefore, no data is available this year. The new assessment form and rubric, however, are included in Appendix C. 2. How and when the Assessment is Implemented At the end of each quarter, candidates submit their completed field experience documentation form to their course instructor. 3. When data is Analyzed and by Whom Each TLA course instructor reviews the field experience documentation forms and provides them to the TLA assessment repository (monitored by TLA’s assessment coordinator) for analysis at the end of the academic year. All TLA candidate data is analyzed in late spring by TLA faculty during a data retreat designed to allow faculty to review all data from candidate and program assessments in the aggregate. 4. Remediation Provided to Candidates Not Meeting Criteria There is no remediation needed as all candidates must complete 10 hours of field experience and submit their personal reflections on what they accomplished and how it impacted them as teachers/educators. 5. Blank sample of the assessment tool: Name of Tool: Field Experience and Documentation Form, Appendix C 6. Blank sample of the scoring guide/rubric: Appendix C 7. Aggregated Assessment Data See note above. 8. Narrative Interpretation of What Data Means in Relation to Candidates’ Field or Clinical Experience/Application to Professional Work We are looking forward to collecting data and analyzing it using the revised tool with rubric developed from the NCE template for the 2016- 2017 academic year. We spent the summer revising the template to target our outcomes and audience. E. Assessment of Candidate Impact/Effect on Student Learning The TLA program uses two key assessments to assess candidate application to professional work: The Assessment Project Analysis Rubrics (APAI, APAII and APAIII) and the Action Research Demographic Survey. Assessment #1 Assessment Project Analysis Rubrics (APAI, APAII, APAIII) 1. Name of and Description of the Assessment and What it Assesses The Assessment Project Analysis Rubrics (APA) are a set of three different rubrics that coincide with the three required assessment courses in the TLA program. APAI rubric is used in TLA 541, Assessing Teaching and Learning: Introduction to Assessment (quarter 1), APAII rubric is used in TLA542, Linking Assessment to Teaching


and Learning Field Study (quarter 4) and APAIII rubric is used in TLA 543, Reflecting on Teaching, Learning and Assessment (quarter 6). The APA I is administered at the end of TLA 541, an introductory course on assessment where candidates learn about types of assessment used in K-12 schools and their strengths and weaknesses, reflect on the social, political and historical contexts of school based assessment related to race, class, equity and social justice, investigate contemporary issues related to assessment, critically analyze assessment practices within candidates’ classrooms, schools and communities, and reflect on how assessment has impacted him/herself as a student and teacher. The major project of this course has candidates reviewing and analyzing their own classroom assessment system and planning changes based on content from the course. The APA II is administered at the end of TLA 542, a course that requires candidates to apply their knowledge from TLA 541 into their own educational contexts and their work with PK-12 and beyond students. Candidates are guided in connecting this work to the candidates’ action research projects which are being completed simultaneously, and in using assessment data from their own students to ascertain their impact on student learning. The APA III focuses on candidates reflecting on their work with assessment within their action research and within their own classrooms, relating their findings to that of research on assessment and identifying broader audiences to share their work and the insights on assessment they have gained. A strong focus of the course and this accompanying assessment is encouraging candidates to become change agents as they engage in ongoing inquiry into assessment and continue to use assessment as a means to continually improve their practice. There are no common assignments or assessments for the three assessment courses at this time, however, suggested projects and assignments have been shared among faculty to use within the courses. This is an area the TLA faculty is working on currently to ensure consistent content and experiences for all TLA candidates. 2. How and when the Assessment is Implemented Candidates’ work in each of these courses is correlated with the indicators on the APA checklists, which are aligned with TLA program outcomes. Candidates engage in various assessment projects in each of these courses. At the end of each course, both candidates and instructors complete the APA checklist for that course as means to assess the candidate’s attainment of the course outcomes, thus demonstrating their impact on student learning. 3. When data is Analyzed and by Whom In late spring, the data is analyzed by TLA faculty during a data retreat designed to allow faculty to review all data from candidate and program assessments in the aggregate. 4. Remediation Provided to Candidates Not Meeting Criteria Instructors in the courses where these assessments are administered work with candidates individually who are not meeting the criteria for the course. 5. Blank sample of the assessment tool: Name of Tool: Assessment Project Analysis Tool (I, II, III) Appendix D (APAI), Appendix E (APAII), and Appendix F (APAIII) 6. Blank sample of the scoring guide/rubric: There is a three point scale for each indicator on each of the three APA rubrics in which instructors and candidates assign a score when completing the rubric. 0 – Not met 1 – Partially met 2 – Met 3 – Beyond 7. Aggregated Assessment Data Candidate data is represented in the data below for this year’s report from both the student self-assessment and instructor assessment from the APA I and APA III, related to student growth (TLA program outcome 9 – which correlates with the NCE domain of Impact on Student Learning. No data was submitted from the APA II this year.


Impact on PreK-12 Student Learning Data from APA Checklists Note: There was not data in 2015-16 for the APA II Assessment.

TLA Program Outcome

APA I (n=13) Student Self-Assessment

APA I (n= 13) Instructor Assessment

APA III (n= 18) Student Self-Assessment

APA III (n=18) Instructor Assessment

Critical Reflection 1.92 1.69 2.94 2.83 Issues 1.93 1.69 2.90 2.83 Action Research 2.31 2.15 2.90 2.90 Life History 2.62 2.08 n/a for assmt. n/a for assmt. Diversity 2.13 1.90 2.95 2.95

Student Growth 1.85 1.38 3.00 3.00 Community of Learners n/a for assmt. n/a for assmt. 3.00 3.00 Technology n/a for assmt. n/a for assmt. 3.00 3.00 Change Agents n/a for assmt. n/a for assmt. 2.79 2.78

8. Narrative Interpretation of What Data Means in Relation to Candidates’ Impact on PreK-12 Student Learning TLA faculty believe that candidates should be achieving at a “2” (Met) or “3” (beyond) for advanced achievements on the Student Growth indicator and in demonstrating their impact on student learning. The data from the APA I, which is completed in the first or second quarter of the TLA program demonstrate that both candidates and their instructor indicate they are not yet meeting the criteria. This is concerning to TLA program faculty and will be reviewed during the reworking of the assessment system in 2016-2017. This is critical due to the difference of the TLA and TL program. Data from the APA III, which occurs in the third assessment course near the end of the candidates’ program indicates that both candidates and their instructor rate the candidates as rating “beyond” the expectations related to student growth (impact on p12 learners). This data demonstrates that candidates apply the content from the assessment courses in assessing and impacting their own students. As assessing TLA candidates’ impact on student learning reflects the data from two assessments, the narrative interpretation in the aggregate will be provided below, after information and data from the Action Research Demographic Survey is shared. Assessment #2 for Impact on Student Learning - Action Research Demographic Survey 1. Name of and Description of the Assessment and What it Assesses The action research demographic survey is completed by candidates in Livetext forms and provides faculty with information regarding candidates’ research topics and foci, thus demonstrating the context in which candidates are impacting their PK-12 students. In addition, collecting data related to where candidate research is conducted allows


faculty to keep up to date with current trends in schools, districts and states (WI, IL, FL), and in various areas of these states (urban, suburban) to allow them to review, refine and revise TLA curriculum as needed. 2. How and when the Assessment is Implemented This survey is completed by candidates during the last action research course in the sequence, ESR 534, through Livetext, when candidates are near completion of their program-long action research projects. 3. When data is Analyzed and by Whom The data from the TLA Action Research Demographic Survey is aggregated and analyzed by faculty during a data retreat designed to allow faculty to review all data from candidate and program assessments in the aggregate. 4. Remediation Provided to Candidates Not Meeting Criteria There is no need for remediation related to this assessment, as it is primarily a record of the focus and topics of candidates’ program-long action research project. 5. Blank sample of the assessment tool: Name of Tool: Action Research Demographic Survey, Appendix G 6. Blank sample of the scoring guide/rubric: The questions on the survey that provide information on candidates’ impact on student learning include the following: #14 – What is the title of your action research project? #15 – What is the primary area of your research focus? 7. Aggregated Assessment Data General trends and content areas are noted from the data from both IDS and TLA students who completed their programs within the 2015-16 academic year: Candidate Data N=29 (18 candidates from WI, 4 from IL and 7 online)

28 candidates teach in public schools 1 candidate teaches in a private school Area they teach: 8 Urban 6 Rural 15 Suburban

Area they teach:

1 Early Childhood 12 elementary 6 High School 7 Middle/Jr High 1 K-12 1 Other 1 Adult

Of the 29 candidates, these are the primary areas of their research focus they indicated. Please note some candidates listed more than one area of focus. 5 Mathematics 4 Special Education 3 Social Studies 3 English, Literacy or Language 2 Foreign Language 2 Technology


1 Health 1 Science Action Research Project Titles

• The Sustainability of special Education teachers within the current structure • Anecdotes on the Power of Professional Development • How the Professional Transition from instructor to administrator can be Accomplished Successfully • Technology and Dental Hygiene: Is it beneficial to students and/or part-time faculty? • The Effects of Anxiety of Students in a competency based program, • Closing the Achievement Gap on College Campuses • The Effect of Active Learning Strategies on Student Participation and Learning • Students Self-Monitoring Personal FM Use • Building Inner Wealth in Early Elementary Students • Engaging for Learning: Action Research • Using Standards Based and Data Driven Decisions to Guide Math Instruction • Goal-Setting Strategies and their impact on academic achievement, student motivation, and class climate. • Increasing the Participation of Students with Special Needs in Turkish Language Class • Instruction Methods for Increasing Achievement and Engagement in Math Class. • My Journey with Standards-Based Grading • Conceptual Understanding versus Procedural Knowledge and their effect upon Long and Short Term

Retention: An Action Research Study • Differentiation in the Classroom

8. Narrative Interpretation of What Data Means in Relation to Candidates’ Impact on PreK-12 Student Learning By reviewing the titles of candidates’ action research projects, it is evident that the topics candidates engage in are personally and professionally meaningful to them and their focus is on improving their own instruction and overall teaching and learning, achievement and motivation of their students. The data from the APA rubric and Demographic survey overall indicate that TLA candidates are impacting their own students through the use of assessment and through their action research projects in a variety of topics that are important to them in their professional lives and educational settings. We were not able to make desired changes to assessments last year that would have resulted in more valid data beyond identifying trends in topics through the use of the demographic survey. We are working on our system and individual signature assessments in the 2016-17 academic year and plan to have validly designed assessments and more reliable data resulting from them, especially to demonstrate our candidates’ impact on their students’ learning. More information on this is located in section V. F. Assessment of Candidate Dispositions 1. Name of and Description of the Assessment and What it Assesses The TLA program uses the NCE Candidate Dispositions Assessment to assess candidate dispositions. The purpose of the NCE Candidate Dispositions Assessment tool is to help nurture NCE candidates’ growth in professional dispositions as they relate to the NCE Conceptual Framework. 2. How and when the Assessment is Implemented The tool is currently being administered as a survey monkey survey to TLA candidates in quarter 1 (TLA 578) at a beginning point in the program, quarter 3 (TLA 588) at a midpoint in the program and quarter 7 (TLA579) at the end of the program. 3. When data is Analyzed and by Whom


For the NCE pilot of this assessment, the data was aggregated and documented by the NCE Assessment Council and the data provided to the TLA program for further analysis. The data was analyzed by TLA faculty when the data was provided in fall 2015. There was only course that fell in this category between Fall 2015-Summer 2016. The data is represented below. 4. Remediation Provided to Candidates Not Meeting Criteria Since the TLA program is cohort based, instructors work closely with candidates throughout their program, thus assessing their professional dispositions on a consistent basis. If there is an issue with a candidate’s dispositions, instructors conference with the candidate to assist him or her in setting goals to work toward demonstrating growth in the area. 5. Blank sample of the assessment tool: Name of Tool: NCE Candidate Dispositions Assessment, Appendix H 6. Blank sample of the scoring guide/rubric: Same as above, Appendix H 7. Aggregated Assessment Data


NCE Candidate Dispositions Assessment Data by Program (2014-2015) Program: TLA Courses Included: TLA 588 Overall Ratings:

8. Narrative Interpretation of What Data Means in Relation to Candidates’ Dispositions The data indicate that candidates rate their dispositions highly for most dispositional statements related to their demonstration of those dispositions within the university classroom. By the middle of the program candidates feel they have fully demonstrated the dispositions appropriate for NCE candidates within the NLU classroom across all dispositions. This is a satisfactory position for candidates in the middle of their program. They are showing proficient in all categories. G. Assessment of Candidate Diversity Proficiencies The TLA program uses components of the Research Project Analysis Checklist (RPA) to assess candidate diversity proficiencies. 1. Description of the Assessment and What it Assesses The Research Project Analysis Checklist (RPA) is a tool that was developed in the 2008-2009 academic year by the IDS assessment committee and has been in use ever since. The RPA was shared with the IDS faculty and were trained on how to use the tool to assess candidates’ action research projects. The primary purpose of the RPA was to have faculty assess each individual candidate on the candidate’s action research project using a more standardized format in addition to the narrative reader reports that faculty provided to candidates related to their projects. We felt the need to more fully evaluate candidate performance of action research projects, the project work that runs through the entire program. This RPA assessment tool was originally designed around the 8 IDS Program Outcomes and refined to reflect the 9th program outcome, which was added when the program was developed into the TLA program. Many of the 44 indicators on the checklist originate directly from statements within the TLA Student Action Research Guide, and indicators directly from the IDS/TLA Outcomes. The RPA is seen as a significant instructor view of candidate performance as it relates to their program-long work on their action research projects. The RPA checklist not only assesses the quality of the work performed in the Action Research project, but also extended into other realms of candidate performance. Indicators were identified for specific areas of candidate performance as well as for specific program outcomes. As can be seen on the Program Key Assessments chart in section III, the results from specifically chosen RPA indicators are used in assessing candidate acquisition of content knowledge, candidate professional/pedagogical content knowledge and diversity proficiencies. See the aggregated data below that delineates the indicators from the RPA that assess candidate diversity proficiencies. 2. How and when the Assessment is Implemented


The RPA assessment of each candidate is administered near the end of the candidate’s 7 quarter program at the completion of the candidate’s seven month long action research project. The instructor completes the RPA online through Livetext https://c1.livetext.com/misk5/formz/public/79395/hLdeJLLvZI 3. When Data is Analyzed and by Whom In late spring, the data is analyzed by TLA faculty during a data retreat designed to allow faculty to review all data from candidate and program assessments in the aggregate. 4. Remediation Provided to Candidates Not Meeting Criteria Since the TLA program is mainly taught in a cohort style format and the action research courses (4) are required, full time faculty are assigned to work with the candidates in these courses in order to provide consistency and close support while completing these key courses. Due to this, instructors work with candidates throughout the program on all aspects of their action research projects (including the indicators reflected in the RPA checklist that reflect diversity) and provide support and remediation to candidates where needed so they will successfully complete their action research project and thus meet criteria on the RPA checklist. 5. Blank sample of the assessment tool: Name of Tool: Research Project Analysis Checklist (RPA), Appendix B 6. Blank sample of the scoring guide/rubric There is a four point scale for each indicator in which instructors assign a score when completing the checklist. 1 – Not met 2 – Partially met 3 – Met 4 – Beyond na – Not Applicable 7. Aggregated Assessment Data Indicators from the RPA tool that represent TLA candidates’ attention to diversity:

Candidates’ Attention to Diversity Indicators from RPA Tool TLA Average

Score N = 20

Recognizes the diverse nature of students’ engagements with learning that relate to the teaching strategies that she/he is using

3.50

Includes evidence that a variety of student voices are heard and responded to 3.50

Presents and analyzes a range of student responses to the instruction he/she uses as part of his/her action research

3.45

Documents an appreciation of individual learning within a class context 3.60

Shows an appreciation of differing perspectives in describing the data and in interpreting the meanings of the data

3.55

Average of Scores for Diversity 2014-15 3.52



8. Narrative Interpretation of What Data Means in Relation to Candidates’ Diversity Proficiencies TLA faculty believe that candidates should be achieving at a “3” (Met) or “4” (beyond) for advanced achievements on the indicator and in demonstrating their acquisition of TLA content knowledge. One of the TLA program outcomes is diversity: Engaging with the complexities of diversity and multiculturalism in our culture and our classrooms. The above indicators from the RPA tool reflect that this program outcome is demonstrated through candidates’ work within the action research project.


Looking at the longitudinal data, candidates have increased their competency related to diversity from the past year, however, the N in 2013-14 was only 6 candidates and this year, 20. Please see the narrative interpretation related to content knowledge for possible reasons for discussion of this from 2013-14. This year’s data, however, shows a comparable mean score for candidates related to diversity to that of 2012-13, when more candidates completed this assessment. Therefore, the data show that TLA candidates are meeting criteria that demonstrates their proficiency related to diversity. H. Assessment of Candidate Technology Proficiencies 1. Name of and Description of the Assessment and What it Assesses The TLA program uses components of the TLA Outcomes Survey to assess candidate technology proficiencies. The TLA Outcomes Survey is an online survey that is housed in Livetext. It was developed around the 9 outcomes of the TLA program and each outcome includes 3-6 indicator statements in which candidates self-assess their development in relation to each of the outcomes. As is evidenced in the TLA Program Key Assessments chart in Section II, the data from specifically chosen TLA Outcomes Survey indicators are used in assessing candidate acquisition of content knowledge, candidate diversity proficiencies, and candidate technology proficiencies. 2. How and when the Assessment is Implemented The TLA Outcomes Survey is administered three times throughout the program: (1) at the end of quarter 2, (2) at the end of quarter 5 and, (3) at the end of quarter 7 or end of program as candidates are directed to complete the survey in Livetext and the URL is provided. 3. When data is Analyzed and by Whom The data is collected through LiveText throughout the year. Individual instructors are encouraged to review their candidates’ data upon completion in each of the quarters in which the assessment is administered. In late spring, the data is analyzed by TLA faculty during a data retreat designed to allow faculty to review all data from candidate and program assessments in the aggregate. 4. Remediation Provided to Candidates Not Meeting Criteria This assessment is administered in quarter 2, 5 and 7, which are the quarters in which many students are taking the required sequence of action research courses. Since the TLA program is mainly taught in a cohort style format and these are required courses, full time faculty are assigned to work with the candidates within a cohort to provide consistency and close support while completing these courses. Due to this, instructors work with candidates who are working toward meeting criteria and provide necessary support as they grow in their ability to demonstrate TLA program outcomes, specifically through their seven quarter long action research project work. 5. Blank sample of the assessment tool: Name of Tool: TLA Outcomes Survey, Appendix A 6. Blank sample of the scoring guide/rubric: The TLA Outcomes survey is a survey completed in LiveText by the candidate. Each of the questions on the survey requires the candidate to respond using a six-point scale and add narrative response if desired:

1 - I’m not sure what this is 2 - I have a long way to go on this one 3 - I’ve begun to make progress 4 - I’ve made substantial progress 5 - I feel competent in this regard 6 - I feel highly competent in this regard

7. Aggregated Assessment Data

Data from Outcomes Survey from Candidates at the End of Program Related to Proficiency with Technology BELOW is the data represented in 2014-2015.


The data sets were incomplete and reported errors from 2015-2016 We are in the process of revising the assessments for TLA to address these issues.

OUTCOME 7

TECHNOLOGY: Exploring ways to integrate technology within one’s professional life


Average Score N=26

Middle of Program

Average Score N=24

End of Program

Average Score N=20

Question 1 – I model appropriate uses of technology in my classroom. 4.81 4.72 4.95 Question 2 – I encourage students to use appropriate technologies in their learning.

4.78 4.76 4.85

Question 3 – I offer learning experiences supported by the technology. 4.63 4.68 5.00 Question 4 – I use technology as an aid in assessing student learning. 4.44 4.60 5.14 Question 5 – I participate in professional development activities aimed at enhancing my abilities in the area of technology.

4.54 4.64 4.90

Overall Average of Scores for Technology 4.64 4.68 4.97

8. Narrative Interpretation of What Data Means in Relation to Candidates’ Technology Proficiencies TLA faculty believe candidates should be achieving at the level of "3" when responding to the questions at the beginning and middle points of administration within the TLA program, thus indicating that they have begun to make progress. By the end point of administration (last quarter/end of program) candidates should be achieving at the level of “5” or above, demonstrating that they have made become competent in this area and program outcome throughout their development in the program. The self-reported scores from the candidates show a progression of growth from the beginning of the program where candidates report their beliefs about their attainment of their proficiency related to technology at a mean of 4.64, midpoint at 4.68 and an end at 4.97. This indicates candidates feeling competent related to their technology proficiencies. This progression is to be expected as candidates’ progress through the TLA program, their self-reported scores tend to increase overall with their comfort and proficiency in using technology to teach and help their students learn. In all areas related to technology, candidates indicate that they feel they have made substantial progress in reaching the outcomes. All TLA courses are either blended or fully online, so candidates are highly engaged in using technology for their own learning within the TLA program which then is translated into use of technology within their own classrooms and educational environments. Section V: Use of Assessment Results to Improve Candidate and Program Performance A. Detail how your program has used or will use the aggregated data from the following key assessments related to: Overall the aggregated data from the key assessments this year suggest that our candidates are meeting the criteria for TLA program outcomes. However, it is challenging to analyze the data from a more drilled down perspective as we feel our assessments do not accurately or fully depict our candidates’ growth and attainment of the TLA program outcomes for reasons that are specified below. The assessments do not fully assess the recent updates and improvements of the program. In addition, the assessments have not been standardized with Livetext, which will happen in 2015-2016. Our strongest aspect of the current program is the disposition of our candidates. The data indicate that candidates rate their dispositions highly for most dispositional statements related to their demonstration of those dispositions within the university classroom. By the middle of the program candidates feel they have fully demonstrated the dispositions appropriate for NCE candidates within the NLU classroom across all dispositions. This is a satisfactory position for candidates in the middle of their program. They are showing proficient in all categories.


There is a historical context to the issues. When the TLA program was developed a several of years ago, most of the key assessments were directly transferred to the TLA program without change or adjustment (due to the short timeline for implementation and the challenges of beginning implementation of a newly developed program). Because of this, the program implemented the same key assessments from the IDS program with the TLA candidates. In addition, in 2014-15, the first cohort of the Teacher Leader program (TL) began. This is a program grounded in the core courses of the TLA program and candidates take other courses directly related to the teacher leader endorsement, which include courses from the EDL program and the Reading program. However, most courses are TLA courses, some of which currently contain the TLA signature assessments. Because of this, discussion began last year on how to integrate the two assessment systems – that of TLA and TL to make assessing candidates more standardized across programs, and because it made sense. Because of this planning of correlating assessments between TLA and TL, the rework of TLA assessments was not completed last year as was indicated in the last assessment report would be the TLA program faculty work for 2014-15. Work on this did begin, however, in 2015 and continues with a completion plan expected by summer of 2017. The new target is due to a grand revision of the TL program due to the current needs in this area. Assessments were going to be combined but now we anticipate having a separate programs and assessment reports. This is due to the need for a SPA for Teacher Leader and non-certification track of TLA One main issue that continues and needs to be addressed (and will be a part of this year’s work plan on the assessment system redesign) is the inconsistency in data collection and sharing of the data from the signature assessments. This inconsistency should be rectified with the use of LiveText for collecting all data from all TLA signature assessments beginning in the fall 2016 (or before as assessments are completed and submitted to LiveText) so that data analysis and accurate and focused reflection of the program and candidate growth can take place. The data reflected in this year’s report is a meager representation of our candidates meeting TLA program outcomes and NCE requirement. We have formulated a plan for key assessments and changes moving forward. These assessments will be in Livetext and consistent across the program. See the new key assessments that were developed this summer (2015)- below: TLA Signature Assessments beginning in LT in Fall 2016

Assessment Domain Name of Signature/Key Assessment

(Note if assessments are different for Undergraduate,

Graduate, Alternative Programs or Other Delivery

Models)

When the Assessment Is Administered

(indicate course or other marker)

Transition Point in Program

(as applicable)

Assessment of Content Knowledge I

N/A – non SPA program N/A N/A

Assessment of Content Knowledge II (Local Key Assessment)

1. TLA Outcomes Assessment

Candidates complete in FND 509 in TLA 589 in TLA 579

• Instructor assesses only at end of program in TLA 579

Beg. of Program Middle of Program End of Program


B. What does your program’s candidate achievement data say about candidate’s achievement toward the NCE Conceptual Framework overall? As evidenced in the chart on pp. 4, all assessments in TLA are correlated to the NCE conceptual framework. As well, TLA uses the NCE Candidate Dispositions Assessment, the unit-wide assessment that is grounded in the dispositional components of the NCE Conceptual Framework. All of our courses and assessments are mapped to the framework. As we rework and redesign signature assessments, the TLA program is keeping the framework in mind.

2. TLA 543 Assessment Project

TLA 543 End of Program

Assessment of Professional/Pedagogical Knowledge for Developing and/or Supporting Meaningful Learning Experiences

Research Project Analysis (RPA)

Completed during last Action Research course ESR 534 (Student uploads AR project in LT, Instructor completes rubric)

End of Program

Assessment of Field or Clinical Experiences/ Application to Professional Work

Field Experience Documentation and Reflection

End of each course in the program (all courses have 10 hours associated with them for fieldwork)


Assessment of Candidate Impact on Student Learning

1. TLA 543 Assessment Project

TLA 543 End of Program

Assessment of Candidate Dispositions

NCE Candidate Dispositions Assessment

Completed three times in program TLA 578 (Q1), TLA 588 (Q3) TLA 579 (Q7)

Beginning of Program Middle of Program End of Program

Assessment of Candidate Diversity Proficiencies

Research Project Analysis (RPA)

Completed during last Action Research Course, ESR 534 (Student uploads AR project in LT, Instructor completes rubric)

End of Program

Assessment of Candidate Technology Proficiencies

TLA Outcomes Assessment

Candidates complete in FND 509 in TLA 589 in TLA 579

• Instructor assesses only at end of program in TLA 579

Beginning of Program Middle of Program End of Program


C. Reflect on the previous year’s assessment and recommendation from the NCE Assessment Committee review (2014-2015). How did/can your program use the feedback? Were changes to the program’s assessment system implemented? What progress has been made? One key comment from the reviewers in the 2014-2015 review document commented that there was not enough information being shared related to candidate strengths and weaknesses in the domain of pedagogical content knowledge. While care was taken to make sure a generalization of candidate strengths and areas for development was included within the narrative interpretation of that domain, there is little variability in the data this year and not much can be said related to the candidates. As previously mentioned, we anticipate our new assessments will be a much better fit for our TLA candidates and assist us as program faculty in analyzing the data in a way that will assist us in helping our candidates grow in areas where deficiencies are noted. This is still in development as we work toward separate TLA and TL assessments. The candidates have different outcomes and professional goals therefore we need to update our assessments to meet these needs.

APPENDIX A TLA Outcomes Survey

TLA Outcomes Survey

A critically reflective practitioner is continually examining aspects of their teaching, as well as their practice as a whole, to insure consistent improvement. The following self-assessment will be anonymous. It will be used as a reflective tool for professional and program improvements. We value your time and honest input. This will take about 10 minutes to complete. We apologize that there is no way to stop and save your entries for finishing up later. Please be sure to click on the SUBMIT button at the end. In the first two questions we need to create a code to be able to track your progress over time anonymously. 1. Please enter the first two letters of your mother's first name. 2 . Please enter the two-digit number of your mother's birth month (ie. enter a 09 for September)

3 . Please enter your NLU group or cohort number as best as you can recall it.

4 . I take classes at this NLU location: Illinois-Wheeling Illinois-Chicago Illinois-Elgin Illinois-North Shore Illinois-Lisle Illinois- Other Florida Wisconsin

5 . What Term are you currently in?

At the beginning of Term 1 Term 1 Term 2 Term 3

6 . What year is it?

2008 2009


2010 2011

7. What grade level do you teach? Early childhood Elementary Middle / Jr high High school Adult K-12 Other I am not a teacher

8 . How many years of teaching experience do you have? 0 1-3 4-9 10-15 16+

9 . What is your age range?

16-20 20-29 30-39 40-49 50-59 60+

10 . What best describes the area you teach/work in?

Urban Suburban Rural

11 . Would you like to add any additional information about the above demographic information?

Outcome 1: Critical Reflection: Striving to be critically reflective about one's teaching and profession

12 . I examine and attempt to solve the dilemmas of classroom practice. Outcome 1: Critical Reflection: Striving to be critically reflective about one's teaching and profession

I'm not sure what this is I have a long way to go on this one I've begun to make progress I've made substantial progress I feel competent in this regard I feel highly competent in this regard

13 . I am attentive to the institutional and cultural context in which I teach.


14 . I am aware of and question the assumptions and values I bring to the classroom.

I'm not sure what this is I have a long way to go on this one I've begun to make progress


I've made substantial progress I feel competent in this regard I feel highly competent in this regard

15 . I think more critically about the professional reading I do.

Choose one... I'm not sure what this is I have a long way to go on this one I've begun to make progress I've made substantial progress I feel competent in this regard I feel highly competent in this regard

16 . If you have any narrative comments that would help us understand your selections above for this outcome, please enter your

thoughts here. Outcome 2: Issues: Identifying social, economic, and political issues in teaching, learning and schooling.

17 . I am aware of social, economic, and political issues that affect my teaching, my students, and American schools. I'm not sure what this is I have a long way to go on this one I've begun to make progress I've made substantial progress I feel competent in this regard I feel highly competent in this regard

18 . I am questioning assumptions about how schooling best occurs.


19 . Based on new information and/or classroom discussions, I am rethinking some of my past educational practices.


20 . I am able to articulate a rationale for the kinds of teaching and schooling I believe learners need.



thoughts here. Outcome 3: Teacher Action Research -- Engaging in action research on personally and


professionally relevant topics that will contribute to teaching, learning, and schooling. 22 . I identify issues and problems that can be effectively investigated through action research.


23 . I can articulate the personal and professional relevance of my action research.


24 . I use a variety of data gathering and analysis strategies.


25 . I analyze the data that I collect.


26 . I discuss the data and its implications with other teachers


27 . My teaching has improved through my action research.



thoughts here. Outcome 4: Community -- Helping build a community of learners who engage in meaningful conversation about critical issues of education and teaching practice.

29 . I bring issues, problems, and insights to the attention of the group.



30 . I contribute to IDS planning.


31 . My group/cohort shares information about resources and opportunities. I'm not sure what this is I have a long way to go on this one I've begun to make progress I've made substantial progress I feel competent in this regard I feel highly competent in this regard

32 . I have gained from hearing about the experiences of my colleagues.



thoughts here. Outcome 5: Life History -- Reflecting critically on teaching and learning within the context of one’s own life history and teaching career

34 . I am identifying in my personal history, themes, issues, and values that have shaped my educational perspectives I'm not sure what this is I have a long way to go on this one I've begun to make progress I've made substantial progress I feel competent in this regard I feel highly competent in this regard

35 . I am examining the evolution of my thinking about the nature of teaching and learning.


36 . I am examining my practice in relation to my beliefs and values.

I'm not sure what this is I have a long way to go on this one


I've begun to make progress I've made substantial progress I feel competent in this regard I feel highly competent in this regard

37 . In my action research, my personal and professional narratives converge.



thoughts here. Outcome 6: Diversity -- Engaging with the complexities of diversity and multiculturalism in our culture and our classrooms

39 . I encourage open dialogue in my classroom related to issues of diversity, equity, and social justice. I'm not sure what this is I have a long way to go on this one I've begun to make progress I've made substantial progress I feel competent in this regard I feel highly competent in this regard

40 . I examine my practice in light of new understandings related to diversity, equity, and social justice.


41 . I implement learning experiences to enhance my students’ understandings of issues related to diversity, equity, social justice.


42 . I study and evaluate my personal beliefs, values and knowledge related to issues of diversity, equity, and social justice.



thoughts here. Outcome 7: Technology -- Exploring ways to integrate technology with one’s professional life.

44 . I model appropriate uses of technology in my classroom.



45 . I encourage students to use appropriate technologies in their learning.


46 . I offer learning experiences supported by technology. I'm not sure what this is I have a long way to go on this one I've begun to make progress I've made substantial progress I feel competent in this regard I feel highly competent in this regard

47 . I use technology as an aid in assessing student learning.


48 . I participate in professional development activities aimed at enhancing my abilities in the area of technology.



thoughts here. Outcome 8: Change Agents -- Acting as change agents advocating for all learners.

50 . I am a change agent in my classroom. I'm not sure what this is I have a long way to go on this one I've begun to make progress I've made substantial progress I feel competent in this regard I feel highly competent in this regard

51 . I assume an advocacy role in my classroom and community, helping others to perceive and address inequities.

I'm not sure what this is I have a long way to go on this one I've begun to make progress


I've made substantial progress I feel competent in this regard I feel highly competent in this regard

52 . I take part in curriculum development and am involved in school change efforts. I'm not sure what this is I have a long way to go on this one I've begun to make progress I've made substantial progress I feel competent in this regard I feel highly competent in this regard

53

I am aware of my responsibility for my students' growth and learning. Outcome 9: Integrating assessment into one's professional practice to impact and evidence student growth and learning.


54 I am able to identify areas of critical growth for my students. Outcome 9: Integrating assessment into one's professional practice to impact and evidence student growth and learning.


55 I critique and adapt curriculum based on the evidence of my students' needs Outcome 9: Integrating assessment into one's professional practice to impact and evidence student growth and learning.


56 I differentiate instruction for my students based on specific instructional goals for individual and whole class instruction. Outcome 9: Integrating assessment into one's professional practice to impact and evidence student growth and learning.


57 I have an understanding of students' academic challenges and I plan instructional modifications accordingly.


Outcome 9: Integrating assessment into one's professional practice to impact and evidence student growth and learning. I'm not sure what this is I have a long way to go on this one I've begun to make progress I've made substantial progress I feel competent in this regard I feel highly competent in this regard

58 I can show evidence of using both formative and summative assessment throughout the course of instruction. Outcome 9: Integrating assessment into one's professional practice to impact and evidence student growth and learning.


59 I use alternative forms of authentic assessment to meet specific instructional goals for my students. Outcome 9: Integrating assessment into one's professional practice to impact and evidence student growth and learning.


60 If you have any narrative comments that would help us understand your selections above for this outcome, please enter your thoughts here.


APPENDIX B TLA Program – Research Project Analysis Checklist (RPA)

Name: Date:

Reader/Advisor: Group Number: Critical Reflection The teacher: Not Met Partially Met Met Beyond Not

Applicable Reflects on how teaching and learning function in his/her classroom

Speculates what changes might be beneficial to his/her classroom and/or students

Purposefully looks to gain insight into a classroom phenomenon

Honors unanswered questions and points of confusion throughout the research process

Thoughtfully reflects on his/her journey through the research

Builds understanding from how the students and the teacher are affected by specific incidents and classroom experiences

Makes explicit the understandings upon which her/his judgments and decisions are made

Notices and selects specific classroom moments or incidents that relate to his/her research questions

Thoughtfully contemplates what he/she and his/her students are experiencing

Formulates new realizations that lead to further his/her journey into knowing and understanding

Issues The teacher: Not Met Partially Met Met Beyond Not

Applicable Makes explicit the social, political, and economic context in which she/he is working

Uses inquiry to help understand life as it is happening in his/her classroom and speculate what changes might be beneficial

Questions and/or analyzes social, cultural, economic, and/or gender issues within the action research story and its analysis

Meaningfully uses instructional strategies described in the literature to inform her/his research, teaching and role as a professional


Action Research The teacher: Not Met Partially Met Met Beyond Not

Applicable Uses his/her personal experience with the topic to fuel the inquiry and the research project.

Uses the research project as an authentic inquiry into him/herself as a teacher

Meaningfully alters his/her curriculum and research along the way in response to what he/she observes and what the students do and say

Uses his/her students as the major measure of how well the project is going

Looks for insights and meanings about learning and teaching emerging from the inquiry process

Community of Learners The teacher: Not Met Partially Met Met Beyond Not

Applicable Continues her/his story of the inquiry through an active engagement with resources

Interacts with a wide range of sources of information (i.e. conversations with colleagues, interviews of students and peers, books, articles, films, internet sources, professional conferences, the arts, etc.)

Records ideas he/she has while interacting with a wide range of resources

Demonstrates an accurate knowledge of the rules regarding citations and the avoidance of plagiarism

Accepts him/herself as a researcher and exercises her/his own voice as a member of the community of resource producers

Demonstrates how one’s own classroom can be a community of learners

Interacts with colleagues within the work of action research

Life History The teacher: Not Met Partially Met Met Beyond Not

Applicable Clearly remains the center of the work, the teller and receiver of the story

Values the interrelatedness of self as the researcher to what he/she is studying

Embraces his/her questions and feelings of ambiguity as part of the learning process

Keeps alive particular student stories along with those stories unique to the teacher


Brings in students’ life histories within the project and the analysis of data

Diversity The teacher: Not Met Partially Met Met Beyond Not

Applicable Recognizes the diverse nature of students’ engagements with learning that relate to the teaching strategies that she/he is using

Includes evidence that a variety of student voices are heard and responded to

Presents and analyzes a range of student responses to the instruction he/she uses as part of his/her action research

Documents an appreciation of individual learning within a class context

Shows an appreciation of differing perspectives in describing the data and in interpreting the meanings of the data

Technology The teacher: Not Met Partially Met Met Beyond Not

Applicable Records data through writing or other media (video, photography, online) that capture stories or incidents of classroom life and meaning

When using alternative representations, reflects the authentic nature of the story and the inquiry

Demonstrates that technology skills have been applied Formatted and presented the project using technology skills

Change Agent The teacher: Not Met Partially Met Met Beyond Not

Applicable Identifies potential new directions to engage in because of one’s study

Takes on new leadership roles or engages in new actions with others due to what is learned through the action research

Develops new educational insights about teaching and learning, curriculum, self, students, or others

Stretches her own and her students’ capacities, through imagining richer outcomes for her teaching and/or student learning


Student Growth The teacher: Not Met Partially Met Met Beyond Not

Applicable Demonstratesknowledgeofhis/herresponsibilitiesforstudentgrowthandlearning

Identifiesareasofcriticalgrowthforhisorherstudents

Critiquesandadaptscurriculumbasedupontheevidenceofstudentneeds

Differentiatesinstructionbasedonspecificinstructionalgoalsforindividualandwholeclassinstruction

Demonstratesunderstandingofstudents’academicchallengesandplansinstructionalmodificationsaccordingly

Showsevidenceofusingbothformativeandsummativeassessmentthroughoutthecourseofinstruction

Usesalternativeformsofauthenticassessmenttomeetspecificinstructionalgoalsforthestudents


APPENDIX C

TLA Program: Field Experience Documentation and Reflection (AlsousedforUniversityWrittenCommunicationULOAssessment)

ContextandOverview:OneofthegoalsoftheTLAprogramistohelpeachteachergainarenewedsenseofwhatitmeanstobeateacher.Tothisend,theprogrampromotesdemocracy,diversity,inclusiveness,andpersonalcreativityinasmanyaspectsaspossible.Thisself-reflection/assessmentencouragesteacherstoreflectonhowtheyaredevelopingtowardsmeetingthenineTLAprogramoutcomes.EverystudentintheTLAprogramisrequiredtodocument10hourspercourseoffieldexperienceand/orclassroomapplications.Theseactivitiesdocumentyourapplicationofthetheoriesandpracticesencounteredinyourcoursework,aswellasyourgrowthandapplicationoftheNineOutcomesoftheTLAprogram.Thepurposeoffieldexperienceistohelpyoutomakeconnectionstoyourthinking,deepenyourinsightsintothesenewconnections,andultimatelyguideyouindiscoveringhowtheseinsightscan,have,orwilltransformyourprofessionalpractice.

TheTLAprogramfosterscandidateprogresstowardourNineOutcomes:1.CriticalReflectionStrivingtobecriticallyreflectiveaboutone’steachingandprofession

2.IssuesIdentifyingsocial,economic,andpoliticalissuesinteaching,learning,andschooling

3.ActionResearch(N/Aforquarterone.OnlyforESR534finalquarter.)Engaging in action research on personally and professionally relevant topics that will contribute toteaching,learning,andschooling

4.CommunityHelping build a community of learners that engage inmeaningful conversation about critical issues ofeducationandteachingpractice5.History

Reflecting critically on teaching and learningwithin the context of one’s own life history and teachingcareer

6.Diversity Engagingwiththecomplexitiesofdiversityandmulticulturalisminourcultureandourclassrooms 7.Technology Exploringwaystointegratetechnologywithone’sprofessionallife 8.ChangeAgents Actingaschangeagentsadvocatingforalllearners

9.StudentGrowthIntegrating assessment into their professional practice to impact and evidence student growth andlearning

Directions


1.Foreachactivity,completethechartbelow,listingthecoursetheactivitycorrelateswith,thedateyoucompletedtheactivityandthedurationoftheactivity.2.Intheactivitycolumn,brieflydescribetheactivity,whereitwasdoneandwithwhom.3.IntheTLAProgramOutcomecolumn,listtheprogramoutcome/stheactivitymostcloselycorrelateswith.Notalloutcomesarerequiredbuttrytoaddressatleast2-3.4.Inthereflectioncolumn,usethefollowingquestionsasastructureforreflectingoneachactivity.Yourreflectionshouldincluderesponsestoasmanyofthesequestionsaspossible,butneednotbelimitedtoonlythesequestions.Inthiscolumn,alsoincludereflectionthatarticulatesyourgrowthintheidentifiedprogramoutcome.

ReflectionQuestions:• Howorwhydidyouchoosetodocumentthisactivity?• Whatwastheoutcome?Whatwastheresultofthisactivity?• WhatTLAOutcomeorOutcomesdidthisactivityprovideyoumoreinsightinto?• Whatdidyoulearnaboutyourself,yourpeers,and/oryourstudents,asaresultofthisactivity?)

5.Aftercompletingthechart,writeasummary/conclusionnarrativedescribingyouroverallgrowththisquarter.6.Makesureyoudocumentatleast10hoursoffieldexperienceactivityforEACHcourseyoucompletedthisquarter.7.Whencomplete,uploadtothedropboxinD2L. CourseNumber

Date Duration(timespent)

Activity TLAProgramOutcome/sassociatedwithactivity

Reflection

Summary/ConclusionCriteriaandRubricFollowingistherubricthatwillbeusedtoprovidefeedbacktoyouonthisassessment.Reviewitcarefullytoensurethatyoumeetallcriteriaattheproficientlevel.


TLAFieldExperienceDocumentationRubric

PERFORMANCE LEVEL

Unsatisfactory (1)

Basic (2) Proficient (3) Distinguished (4)

Performance Description

Refers to candidate performance that does

not yield sufficient evidence to make a determination or is consistently below

standards.

Refers to candidate performance

demonstrating necessary knowledge

and skills but its application is inconsistent.

Refers to candidate demonstrating mastery

of performance at a consistently

professional level.

Refers to exemplary candidate

performance that stands as a model

for other candidates.

Not Applicable for this

Assessment Criteria*

Outcomes

Outcomes are not specified or are not reflected upon.

Most of the specified outcomes are reflected upon, but may not be detailed.

All specified outcomes are reflected upon clearly and are fully detailed.

Growth toward Outcomes

There is little or no articulation of growth for each specified outcomes over the course.

There is some articulation of candidate’s growth for each specified outcomes over the course.

There is clear articulation of candidate’s growth for each specified outcomes over the course.

Summary Statement Summary statement includes little or no reflection on how candidate has grown overall related to the course/ program or no summary statement is included.

Summary statement partially details candidate’s growth overall related to the course.

Summary statement clearly and fully details candidate’s growth overall related to the course.

The chart is not completed and/or 10 hours are not documented.

The chart is partially completed and reflects 10 hours.

The chart is completed and reflects 10 hours of activity.

Totals Total Points


APPENDIX D Assessment Project Analysis Tool I (APA I)

Course I – TLA 541: Assessing Teaching and Learning: Introduction to Assessment

Student:___________________________________ Check if Self Assessment ____ Instructor ________________ Cohort_______________ Academic Year __________

TLA Outcome

Course Outcome

NCATE Category

APA Indicators Not Met Partially Met

Met Beyond Not Applicable

Crit

ical

Ref

lect

ion 1 CK Identifies various types of assessments used in K-12 schools and

their strengths and weaknesses

1 PK Demonstrates an understanding of the social, political, and

historical contexts of student and school/ curricular assessment

4 PK Critically analyzes practices of their classroom/team/school/district

in terms of various types of assessments used

TLA Outcome

Course Outcome

NCATE Domain



Issu

es

4 CK Demonstrates understanding of contemporary issues relating to assessment in the national context (including, but not limited to, accountability, differentiation, standards-based assessment, data driven instruction and RTI)

1 PK Understands the social, political, and historical contexts of school-

based assessment in terms of race, class, equity, and social justice

TLA Outcome

Course Outcome

NCATE Domain



Act

ion

Res

earc

h 3 CK Can apply his/her understanding of various assessment approaches in planning to gather data about his/her students

.

TLA Outcome

Course Outcome

NCATE Domain



Life

H

isto

ry DIS Reflects on how assessment has impacted him/her as a student

and as a teacher

TLA Outcome

Course Outcome

NCATE Domain




Div

ersi

ty

CK Demonstrates an understanding of the challenges in eliminating bias and strengthening reliability and validity in all assessments to ensure fairness for all students

542-6 CK Considers how to adapt his/her assessment practice to promote

classroom and school equity

CK Understands the role of assessment as a possible restriction of

student performance

TLA Outcome

Course Outcome

NCATE Domain



Stud

ent

Gro

wth

542-5 PK, ISL Demonstrates an understanding of how effective assessment practices can be applied to his/her own classroom to improve student learning

CK Demonstrates an understanding of basic psychometric terms and

principles to look critically at individual and class assessments


APPENDIX E

Assessment Project Analysis Tool II (APA II) Course II – TLA 542: Linking Assessment to Teaching and Learning: Field Study

Student_____________________________ Check if Self Assessment____ Instructor _____________________ Cohort________________ Academic Year__________

TLA Outcome

Course Outcome

NCATE Domain



Crit

ical

Ref

lect

ion

1

FE

Chooses an assessment topic for project relevant to classroom, school, or school district

6 CK Explores and identifies appropriate assessment procedures for the assessment project

TLA Outcome

Course Outcome

NCATE Domain



Issu

es 1 PK Identifies the issues addressed in choosing a topic or issue in

assessment relevant to district, school or classroom

TLA Outcome

Course Outcome

NCATE Domain



Act

ion

Res

earc

h

3, 2 PK Applies knowledge of psychometric terms and principles to interpret diverse data sources

PK Applies a variety of appropriate assessment procedures to identify

student progress and processes of development within the action research

3 ISL Analyzes relevant data to inform instructional decisions

TLA

Outcome Course

Outcome NCATE Domain



Com

mun

ity

of L

earn

ers

3 PK Responds substantively with insight to classmates’ issues, rationale, processes and challenges of their assessment projects

7 DIS Shares issues, rationale, insights and challenges of his/her

assessment project

TLA

Outcome Course




Div

ersi

ty

1

ISL

Chooses a topic that is relevant to the diversity challenges within the candidates school, community, district, or classroom situation

5 FE Explores ways of using assessment to improve classroom instruction for all learners

3

DIV Analyzes relevant data to inform appropriate instructional decisions for diverse learners

6 ISL Implements assessment approaches to increase awareness of the

diversity of learning and that promote equity


TLA Outcome

Course Outcome

NCATE Domain



Tech

nolo

gy

4 PK Accesses digital and other resources to obtain school, district, or other relevant data on student performance and engagement in learning

4 PK Analyzes and displays assessment data using digital technology

4 PK Uses a variety of technology media for presenting processes and

results of the assessment project

TLA Outcome

Course Outcome

NCATE Domain



Cha

nge

Age

nts 7 PK Envisions ways to share findings of his/her inquiry especially in

terms of how to include more authentic and valuable assessments of student learning

TLA Outcome

Course Outcome

NCATE Domain



Stud

ent G

row

th

1 PK Selects a topic, situation, or issue in assessment that is suitable for in-depth study of student growth

3 ISL Considers ways in which their instruction can be redesigned to

enhance student learning, motivation, and achievement based on his/her assessment data

7 ISL Articulates his/her understanding of the power and limitations of

his/her assessments in demonstrating what student growth can mean


APPENDIX F

Assessment Project Analysis Tool III (APA III) Course III – TLA 543: Reflecting on Teaching, Learning and Assessment

Student_____________________________ Check if Self Assessment____ Instructor _____________________ Cohort________________ Academic Year__________

TLA Outcome

Course Outcome

NCATE Domain

APA Indicators Not Met Partially Met Met Beyond Not Applicable

Crit

ical

Ref

lect

ion

1 ISL Performs a synthesis of the research on the roles of assessment in student growth

2 PK Relates the analysis of their use of assessment in their teaching

to literature and research on assessment

6 PK Explores possibilities for ongoing inquiry into assessment

TLA

Outcome Course



Issu

es

2 PK Relates the performance of the assessments applied to their classroom, school, and/or district to the research on the history and contexts of assessment

1 DIV Synthesis includes the role of local cultural contexts that may

explain the issues raised by the results of their assessment work

5 PK Identifies broader audiences for their work and insights on the

assessment issues that they have gained

TLA Outcome

Course Outcome

NCATE Domain


Act

ion

R

esea

rch

3 CK

Produces a comprehensive final project from their assessment and action research studies

6

PK

Explores possibilities for ongoing inquiry and practical ways to continue their assessment/action research project after completing the program

TLA

Outcome Course



Com

mun

ity

of L

earn

ers

5 PK

Identifies broader audiences for their work on the insights they have gained

PK

Have shared lessons learned and insights gained with their classmates, and provided valuable responses to their classmates’ lessons learned and insights

TLA Outcome

Course Outcome

NCATE Domain




Div

ersi

ty

1

DV

Local cultural contexts have been identified and analyzed that may explain the issues raised by the results of candidate’s work on assessment

2 ISL

Results of the assessment applied compare and contrast with what can be found within the published models of assessment

5

PK

Demonstrates the ability to create and use various types of assessments with students

5

DV

Results of the assessment applied promote a better understanding of the diversity within the classroom, school and/or district

TLA Outcome

Course Outcome

NCATE Domain


Tech

nolo

gy PK Uses technology in innovative ways to communicate findings and

insights to a wider audience

TLA Outcome

Course Outcome

NCATE Domain



Cha

nge

Age

nt

1

ISL

Explores possibilities for ongoing inquiry and practical ways to continue their development of assessment after completing the program

6 FE

Engages in ongoing inquiry and continues to use assessment as a means to continuously improve practice

4

PK

Collaborates with other teachers in their own action research projects

5

PK

Mentors fellow teachers in engaging in action research

TLA Outcome

Course Outcome

NCATE Domain


Stud

ent

Gro

wth

542 - 5 ISL

Applies effective assessment practices to their own classrooms to improve student learning

542 - 6

ISL

Considers how to adapt their assessment practice to promote classroom and school equity

2014-2015 NCE Program Assessment Report Package 3/15 47

APPENDIX G

IDSActionResearchDemographicSurvey1 . Thank you for filling out this survey about your research project. We are interested in your research interests as they reflect

your community, school setting, and job description as a teacher and/or other community professional. Please enter your name in the field below. last name, first name, middle initial

2 . How many years have you taught or been a community professional? Choose one... 0-2 3-5 6-10 11-16 17-24 25+

3 . What is your current job description?

Choose one... Teacher Teacher Aide Other

4 . If you answered other in the previous question, please indicate what "other" is in the box below.

5 . Who is your core instructor?

last name, first name

6 . Who is your cohort's research reader/advisor? last name, first name

7 . Where are you located? Choose one... Florida Wisconsin Illinois Other


9 . What type of delivery program best describes your cohort?

Choose one... Face-to-Face Blended Fully Online

10 . Which best describes your school setting?

Choose one... Public School Private School Parochial School Alternative School Home School Other



12 . Which best describes your community setting? Choose one... Urban Suburban

Rural

13 . Is your school a Title I School? Choose one... Yes No Don't know

14 . What is the title of your action research project?

15 . What is the primary area of your research focus? Choose as many as apply.

Areas of focus Mathematics Science Social Studies English Literacy Writing Literature Special Ed. ESL/Bilingual Physical Ed. Health Foreign Language Fine Arts Industrial Arts Consumer and Family Science Technology Criminal Justice Other


17 . At what level was your study conducted? Choose as many as apply.

Study level(s) Early Childhood Education Elementary Education Middle-level Education Secondary Education Adult Education General Education Other



APPENDIX H

2013 Winter Pilot – NCE Candidate Dispositions Assessment

NCE Candidate Dispositions Rubric 1. Candidate cultivates an intellectual curiosity and excitement for learning in oneself. Examples of evidence cultivating an intellectual curiosity and excitement for learning may include: • Referencingcoursematerialsandreadings• Askingquestionsofothers• Makingconnectionstopractice• Activelyparticipatingingroup

activities/discussions• Sharingideasasameansforengaging

others• Attendingclasspreparedandreadyto

engage

___ Evident ___ Partially Evident ___ Not Evident Explain your rating:

2. Candidate demonstrates respect for people from other cultures and points of view. Examples of evidence of demonstrating respect for people from other cultures and points of view may include: • Activelylistening/invitingothers’ideasand

perspectives• Askingquestionsaboutculture,and/or

pointsofviewthatdifferfromone’sown


3. Candidate demonstrates learning from other peoples and cultures. Examples of evidence of demonstrating learning from other peoples and cultures may include: • Welcomingdifferentopinionsasawayto

understandunderlyingbeliefsandassumptions

• Beingopentolearningfromallpeoples/perspectives

• Welcomingdebatewithalternativeideas


4. Candidate demonstrates a caring attitude in recognizing the needs of others. Examples of evidence of demonstrating a caring attitude in recognizing the needs of others may include:

• Givingofhim/herselftoassistothers(inclassassignments,discussions,etc.)

• Beinganattentiveandactivelistener• Providinghelpfulfeedbackandsupport

toothers



5. Candidate promotes growth in others. Examples of evidence of promoting growth in others may include:

• Sharingknowledgeandresourcestoenhancetheeducationofothercandidates

• Promotinggrowthinotherclassmatesandfacultybyaskingcompellingquestionsandsharingrelevantexperiences

• Sharingideasasameansforengagingothers

• Askingquestionsofothers• Activelylistening/invitingothers’ideas,

perspectives


6. Candidate acts with confidence and self-knowledge to assume leadership roles and responsibilities. Examples of evidence of acting with confidence and self-knowledge to assume leadership roles and responsibilities may include:

• Sharingideas,learnings,lessonsandactivitieswithclassmatesandcolleaguestohelpimproveorassistothers

• Assumingleadershiprolesinone’sprofessionalcontextasevidencesincourseworkandreflections

• Seekingoutprofessionaldevelopmentexperiences

• Participatinginprofessionalorganizations

Additionalexampleforcandidatesinadvancedprograms:

• Assumingleadershiprolesinprofessionalorganizations


7. Candidate uses information from others meaningfully for self-reflection and continuous improvement. Examples of evidence of using information from others meaningfully for self reflection and continuous improvement may include:

• Seekingoutfeedback,critiquesandsuggestionsforimprovement

• Usingselfassessmentasakeylearningtool

• Applyingnewknowledgetopractice,and/orpolicyand/ortheory
