UNDERSTANDING ACADEMIC ENGAGEMENT AS A MULTIDIMENSIONAL AND DYNAMIC CONCEPT

Data analysis: methods and procedures

Aleksandra LazarevaLET Master's Program

University of Oulu

Student engagement

● Defining student engagement: linked to interest, emotions, motivation, and the theory of self-regulated learning in general

● Multiple factors affecting student engagement● Educational context: individual needs, school-level

factors, classroom context (autonomy support, task characteristics, classroom structure, teacher support, and peers)

Original direction of the study

How does use of technology in classroom affects students' engagement?

Data collection● Spring 2013 (second semester in the LET studies)● 5 researchers● Teacher training school in Northern Finland● English language classroom● Eleven 4th graders (in three groups) and their English

teacher● Intervention: 1 month, 6 lessons (45 minutes)● Video observations, learning diaries, students'

products

Methods of data analysis: on-task/off-task and phase-shift analysis

● Context-sensitive and process-oriented methods to study students' learning (Boekaerts & Corno, 2005; Järvelä et al., 2001; Perry & VandeKamp, 2000).

● Dynamic assessment: the main point is to capture the changes of an individual’s motivation in connection to the contextual changes (Järvelä et al., 2001).

● On-task/off-task analysis method is the most basic one to describe motivational characteristics of a learning activity (Järvelä et al., 2001; Järvelä et al., 2008).

On-task/off-task analysis● “On-task episode” is the phenomenon that implies

student approaching a task by “attending to the task or presenting task-focused nonverbal signs” (Järvelä et al., 2008, p. 306).

● “Off-task episode” means avoidance of a learning activity, such as “turning away, approaching other students for telling jokes or other substitute activities instead of learning” (Järvelä et al., 2008, p. 306).

● My coding was based on the previous research and own specific guidelines

On-task/off-task analysisA student was considered to be on-task if he/she:

● was asking task-related questions from the teacher, indicating involvement in the activity and readiness to begin;

● was regulating group’s behavior according to task instructions;● was indicating involvement and willingness to work on the task by non-

verbal means (e.g., leaning towards the iPad and revising instructions from the blackboard oor textbook);

● was providing task-related explanation or technological help to a peer.

A student student was considered to be off-task if he/she:● was indicating loss of focus on the task by non-verbal behaviors (e.g.,

staring into the air and looking bored);● was demonstrating disruptive behavior.

On-task/off-task and phase-shift analysis

● I chose specific learning situations to be analyzed (same for the three groups): activities with iPads (fun, choice, meaningfulness)

● I translated and transcribed them● I watched the selected video episodes and marked

each student's activity as on-task or off-task (10-seconds interval)

● I counted the time spent on-task and off-task● I counted the amount of shifts (coherence of the

activity)

On-task/off-task and phase-shift analysis: Example

Summing up the results

Research questions: Inductive approach● The video data were good for describing engagement● At the same time while I was carrying out the analysis I was

reading more theoretical and research literature● I realized that use of technology has to be taken into

account with multiple classroom factors● While carrying out the on-task/off-task analysis I kept my

mind open about other aspects that I could explore● I changed the focus from technological aspects only to

classroom factors in general. Technology became only one sub-point

Research questions: inductive approachThe main aim of the study is to better understand the dynamics of individual students' engagement in a real classroom context.

1. Did students perceive iPads as increasing the attractiveness of the task, and how did it influence engagement?

2. How did students perceive the degree to which they were able to make choice within the task, and how did it influence engagement?

3. How did meaningfulness of the task for an individual student influence engagement?

4. How did different types of interactions with the teacher influence engagement?

5. What strategies did students use to regulate each other's task engagement, and how successful were they?

Classroom environment factors: Descriptive analysis

Question 1: iPads and fun part of the task

● Students thought iPad made tasks more fun● iPad initiated engagement by causing positive

emotions and situational interest

– Antti: I will take this, I will take this...

– Aleksi: Hey, hey, hey, look who is drawing first, yeah, I

will draw first!

– Jussi: Then me, then me, Aleksi, give it to me after that.● iPads were distracting for some of the students

Question 2: Autonomy support● Tasks differed in the level of autonomy support● Was difficult to perceive autonomy support for

students● From video observations it was seen that students

were eager to make choices within the task – Terhi (asking the teacher): With what program?

– Teacher: You can choose yourself with what program.

– Terhi (asking the teacher): Can I write to the fun part, for example,

sports?

– Teacher: Yes.

Question 3: Meaningfulness

● Tasks differed in the level of meaningfulness– Anna: Stockholm is my favorite place in Sweden.

Question 4: Teacher support

● Three types of interactions with the teacher:– provision of additional instructions (e.g., answering

students' questions about the task)– regulation of students' behavior in the group (e.g., telling

individual students off)– intensive assistance of an individual student or a pair (i.e.,

fulfilling a task with students step by step) (see Rogat & Linnenbrink-Garcia, 2011)

● Different effect on students' engagement

Question 5: Peers● Most part of students' attempts to regulate each other's

behavior were related to structuring the activity, mainly by coordinating turn-taking and using support materials in a group/pair:

– Petri: Close your books! Elina, close your book!

– Laura: You should let Elina draw. Next – to Elina...

● In case when own efforts to structure the activity in the group were not sufficient, a student would draw the teacher's attention in order for her to help:

– Ville: Aleksi is playing a fool...

– Anna: Antti doesn't begin to do...

Discussion

● Findings of this study go in line with previous research findings

● The study was carried out in an authentic classroom context

● Source of engagement and disengagement were established

● Practical implications

Practical implications

References● Boekaerts, M. & Corno, L. (2005). Self-regulation in the classroom: A perspective on assessment

and intervention. Applied Psychology: An International Review, 54(2), 199–231.● Järvelä, S., Salonen, P., & Lepola, J. (2001). Dynamic assessment as a key to understanding student

motivation in a classroom context. In P. R. Pintrich & M. L. Maehr (Eds.), New directions in measures and methods: Advances in motivation and achievement, Volume 12 (pp. 207–240). Bingley: Emerald Group Publishing Limited.

● Järvelä, S., Veermans, M., & Leinonen, P. (2008). Investigating student engagement in computer-supported inquiry: a process-oriented analysis. Social Psychology of Education, 11, 299–322.

● Perry, N. E. & VandeKamp, K. O. (2000). Creating classroom contexts that support young children's development of self-regulated learning. International Journal of Educational Research, 7(33), 821–843.

● Rogat, T. K. & Linnenbrink-Garcia, L. (2011). Socially shared regulation in collaborative groups: An analysis of the interplay between quality of social regulation and group processes. Cognition and Instruction, 29, 375–415.