teacher leaders struggling with data use and finding their

Teacher leaders struggling with data use and finding their role in school improvement

Livia Roessler & Michael Schratz

ICSEI 2019Stavanger, 10th January

Introduction

ICSEI Stavanger I [email protected]; [email protected] I 10th January 2019

Setting the Scene

From Data to Evidence

Methodology

Findings

Discussion

Setting the Scene –National Quality Assurance in Primary and Secondary Schools in Austria

Facts about SQGE (School Quality in General Education )1

� a systematic and periodic examination of school quality and development taking into account empirical data

� Tools: • Development Plans (Regional Plan: School Authority, Local Plan:

School Leaders)• Target and Objectives Agreement Dialogues

� Bottom up (individual school formulates its needs and expectations) and top down (framework target value)

� Research results show mainly positively feedback: SQA is seen as helpful and useful (Kemethofer & Altrichter, 2015)


1) Wiesner et al. 2015

Learning Experience and

Learning Results

Learning and Teaching

Living Space of the Classroom

and School

Leadership and School

Management

professionalism and staff

development

School Partnership and

External Partnerships

� nationwide evaluation (Svecnik & Skliris, 2015) shows that the goals and the philosophy of SQA seem to be used at the schools and that principals see important moderator variables

� Qualified experts are placed at the schools’ disposal to assist with the development processes

à School quality coordinators

Setting the Scene –Teacher Leaders’ roles in Austria

� flat hierarchy within the whole system

� No assessments for positions (missing qualification regularities)

� Few initiatives particularly for professionalization of teacher leaders

� Missing career guidance initiatives


„In Austria, career guidance is provided by

counselling services managed by the local

education authorities (provinces). It is also part

of the teacher training colleges’ mission to

organise programmes for career guidance. The

Career Counselling for Teachers website,

supported by the Ministry of Education, also

provides information on the additional

responsibilities teachers may get to widen their

experience (2).“ (Eurydice 2018, p. 77)

Roles like student consultant 1, 2

‘administrative‘ support 2

Engagement in ICT and CPD 1, 2

Leading positions in unions 1, 2

1) Not a function,

2) no certain job qualifications beforehand +

missing CPD programs

http://studierende.cct-austria.at/karrieren-im-

bildungsbereich


SQCs 1

… inform educational and non-educational staff as well as school partnersabout SQGE (School Quality in General Education)

… are key persons for requests or concerns according SQGE… are contacts points for district SQGE coordinators… regularly attend in-service trainings for SQGE… support SQGE initiatives of the school… are members of the middle leader team (as it exists)… support the school leader in preparing the development meetings with the school authority… do editorial work at the development plan… support evaluation and feedback processes… collect, analyse and re-contextualize relevant data for SQGE

Setting the Scene –School Quality Coordinators (SQC)

1) informal job profile for SQC: http://www.sqa.at/course/view.php?id=184


Data INFORMATION KNOWLEDGEBase operation Coded perception

(through senses)

Systemic relevant

data

Embedding

information in

experience

Limitation Digital:

numbers/ words

Analog: visuals

Information relevant

for system

Community of

practice

Challenge innovative, hybrid

forms

Transfer of

information

Knowledge transfer

The quality of evidence depends on data, information, knowledge and the useful relevance of taking action and making decisions

Architecture of knowledge (Willke, 1998; Schratz, 2000)

Data1…………………………..Information…………………..……….Knowledgeunstructured……………………….……………………..……………….……………….…..structured

isolated………………….……………………………………………………………..………….embedded

independent of context……………………………………………..……dependent of context

few possibilities for action…………………………….……….high possibilities for action

little dependent on experience…………….… ……highly dependent on experience

1) Continuum of data, information and knowledge based on Romhardt, 1998

Quality of evidence

From Data to Evidence –a Journey for Evidence-oriented School Development

From Data to Evidence –a Journey for Evidence-informed School Development

as visible inData

Information

Action/ Knowledge

contexts, significance or experience turn data into

information

Relevance

Relevance

knowledge is the refinement of

information by relating it to practice

knowledge as action relevantunderstanding

information processed bysystemically relevant data

Data is the raw fabric for knowledge


cf. Daft & Weick 1995


data

User

organization

Organizational Perspective1, 2, 3

• No shared vision• no common practice praxis (assessment and

same professional language)• Use of data unclear (no school culture)• Using data unpopular• School leaders see data as irrelevant• No support for data use

User Perspective 1, 2, 3

• Missing information HOW to use data• Quality and source of data is unclear• Missing professionalization use of pedagogical

diagnostics• Honouring data or using it as every-day tool

isn't appropriated

Data Perspective1, 2, 3

• Difficult access to relevant data and information about students‘ data

• Data do not effect the own students • No current data available

• Untraceable data

1) Schildkamp et al. 20172) Bremm et al. 20173) Demski & Racherbäumer 2017

From Data to Evidence –hindering factors for evidence-informed school development

Methodology

1) Document analysis of school developmental plans (n46) from primary and lower secondary schools: whole Province of Tyrol.

2) Interviews with School Quality Coordinators (n8) from lower secondary schools: Modellregion Bildung Zillertal(Tyrol – turquoise region)

deductive-inductive analysis basedon categories (Mayring, 2010)

Findings –Quality assurance: school developmental plans

� 2 themes for three years• 1st theme has to be related to given development topics (assessment, differentiation etc.) and

are selected from a mandatory list provided by the school authority • 2nd topic is free of choice

� Data sources used for strategy of reasoning• theme 1

• Standardized test results • Certified quantitative tools• Questionnaires• Observations 1)

• theme 2: • diverse argumentation “important topic” (VS 9, VS 22, VS 27; NMS 7 ); “we have worked on

this for years”(VS 1; VS 6; VS 12; VS 15); “we attended a lecture according to this topic” (VS 17; VS 21, VS 25); “we want that seal of approval” (VS23, VS 25); “parents’ choice” (VS 18) à less data-related argumentation

� Data sources used as outcome indicator• Standardized test results • Certified quantitative tools

1) Observation is linked to the quote: „we have observed, that..“ - it is not observation in an scientific understanding

à High number of naming concrete data-gaining tools for evaluation;

àmissing connection between chosen topic and data or evidence!

• Questionnaires• Observations 1)

Findings –a Teacher Leader’s perspective of evidence-informed school development

Career &CPD

Cooperationswithin the

development process

Transparency and

Responsibility

Spreading SQGE

themes

• Career• CPD• Job profile

• Result and Target Agreement Dialogues• Goal setting and provide support for

development

• School Leader and SQC• SQC and faculty • SQC and students

• SQGE within the faculty• SQGE within students• SQGE within educational partnerships (parents) and external partnerships



CPD

Cooperation within the development process

„As an SQA leader, you have a close relation with the subject coordinators – as I do have

this position myself I multiply the ideas of our goals and pass them on to the team, but of

course we also have parts in the meetings with all faculty members - where the whole SQA

procedure is presented.“ (SQC_2; Z. 52-55)

.. then it is that, of course, with this newly created

position new courses are implemented and they are

of course obligatory, because they want to ensure

their new reform – but there isn‘t any attention

payed at the level of expertise some of us already

have.““ (SQC_3, Z.27-29 )

1 2 3 4 5 6 7 8

Working years

10 28 5 7 5 30 14 20

SQGE since

4 5 4 5 5 1 2 2

SQC since

1 4 3 1 4 1 2 2

years


„Well within our whole faculty meeting at the

beginning of the year […] we decided what to

chose by hand signs […] so we got three topics

to chose from, that were implemented top down

from the school authority.”

(SQC_1, Z. 67-69)

Goal setting and support provided for development

„[...] the sample in one subject isn’t big

enough. If for instance I would ask students

from the 8th grade, that would only be 50

students, you can’t get good data from that

to be honest”.“. (SQC_2_Z.170-173)



Competences SQC

Fostering a collaboration culture mostly SL and SQC

Accessing and Using Research No

Promoting Professional Learning Communities

No

Facilitating Improvements in Instruction and Learning

No

Promoting the use of Assessments and Data

No

Improving Outreach and Collaboration

In some cases

Advocating for Student Learning and the Profession

In some cases

“… support evaluation and feedback processes… collect, analyze and recontextualize relevant data for SQGE”

(SQC Job profile)

Career &CPD

Cooperationswithin the

development process

Transparency and

Responsibility

- Heterogeneous careers- Varying definitions and

experience of CPD- Learning by doing

less shared culture of -evidence-informed practice0

informal exchange with -subject-leaders

- Responsibility lies with the SL- top-down process for theme 1

(missing sense making) - theme 2 often related to a

project or seal of approval1) Tomal et al. 2014


TEACHER LEADERS

• Adequate CPD programs

• Knowledge how to use and analyse data

• Trustworthiness and expertise are key elements for teacher leaders to be successful in their environment.

• Leadership dynamics are crucial,especially shared responsibilities

• School culture, which provides evidence-orientation

SCHOOL DEVELOPMENTAL PLANS

• tool/template limits deeper focus on evidence-orientation

• formal document vs. dynamic tool for development

• duality of structure: top-down and buttom-up

• CPD programs and theme 2 (needs orientated)

Discussion –Quality Assurance Plans as catalysts for School Development


Bremm, N., Eiden, S., Neumann, C., Webs, T., van Ackeren, I., & Holtappels, H. G. (2017). Evidenzorientierter Schulentwicklungsansatz für Schulen in herausforderndenLagen. Schulentwicklungsarbeit in herausfordernden Lagen, 140.

Demski, D., & Racherbäumer, K. (2017). What data do practitioners use and why? Evidence from Germany comparing schools in different contexts. Nordic Journal of Studies in Educational Policy, 3(1), 82-94.

European Commission/EACEA/Eurydice, 2018. Teaching Careers in Europe: Access, Progression and Support. Eurydice Report. Luxembourg: Publications Office of the European Union.

Schildkamp, K., Poortman, C., Luyten, H., & Ebbeler, J. (2017). Factors promoting and hindering data-based decision making in schools. School effectiveness and school improvement, 28(2), 242-258.

Tomal, D., Schilling, C. & Wilhite, R. (2014). The Teacher Leader. Core competencies and strategies for effective leadership. Lanham: Rowman & Littlefield.

Westfall-Greiter, T., & Hofbauer, C. (2015). Fostering teacher leaders for sustainable school reform: system-wide strategies in Austria’s lower secondary school reform. RICERCAZIONE, 125-144

References

http://studierende.cct-austria.at/karrieren-im-bildungsbereich

THANK YOU

Livia Roessler: [email protected]

Michael Schratz: Michael. [email protected]

ICSEI 2019Stavanger, 10th January

teacher leaders struggling with data use and finding their

Documents