cooperative open learning in commercial education ......helm, c. cooperative open learning in...

Vol 9, No. 2, December 2014, pp. 63-84 http://reflectingeducation.net

© UCL Institute of Education, University College London ISSN 1746-9082

COoperative Open Learning in Commercial Education: Multilevel Analysis of Grade 9 Students’ Learning Outcomes in Accountancy

Christoph Helm Department of Education and Psychology, Johannes Kepler University, Linz, Austria

ABSTRACT

COoperative Open Learning (COOL) represents a student-centred way of teaching and learning which is frequently practiced in Austrian secondary vocational schools (BMHS). In 1996 teachers introduced COOL in order to cope with challenges resulting from high student heterogeneity in BMHS classes. According to the COOL concept and to literature on self-regulated learning, design principles of the successful implementation of COOL (such as individualisation and differentiation, empathy, and support of cooperative learning) have been identified. Due to the lack of longitudinal evaluation studies on COOL, the present study is the first to investigate to which extent COOL students develop differently from traditionally instructed students in the subject Accountancy. Data was collected from 602 students (13 COOL classes, 14 traditional classes) using an online questionnaire and competence tests. Descriptive statistics, t-test and Multilevel Regression Analysis revealed the impact power of COOL and COOL-related design principles, respectively. The results show that besides students’ characteristics (e.g. prior attainment in Maths), COOL-related principles have different effects on students’ academic achievement in Accountancy. Whereas students’ perception of individualisation (prior knowledge consideration) and support of learning strategies (doing task analysis) positively predict their competence, metacognition (reflecting on strengths and weaknesses) and class-level perception of differentiation (additional learning tasks for fast working students) negatively predict students’ learning outcomes in Accountancy. Results are discussed in the light of the consequences for teaching practice and the limitations of the present study.

INTRODUCTION

The Austrian education system requires students to have nine compulsory years of schooling. Grade 9 students – usually aged 14 and 15 – can choose between doing a pre-vocational year or switching to a variety of secondary academic, technical and vocational schools, where they can obtain a “Matura”, the university admissions certificate. In case of the latter option, students further need to choose between three (without Matura, referred to as BMS = berufsbildende mittlere Schule) and five year (with Matura, referred to as BHS = berufsbildende höhere Schule) tracks. Due to lower academic demands, the three year track seems to be an attractive choice especially for young people who have poor opportunities in the labour market and are not always easy to motivate for learning (Helm & Altrichter 2012), thus BMS is often labelled as a “problematic school type”. However, BMS is also attractive to engaged and performance-oriented students, as it provides them with essential vocational skills within just a short time period. These circumstances lead to student heterogeneity in BMS classrooms. When students of various performance levels, educational aspirations, and ethnic backgrounds need to be taught at the same time, traditional modes of teaching often do not seem appropriate and effective. Following the challenge of coping with student heterogeneity, two teachers, Georg Neuhauser and Helga

Helm, C. COoperative Open Learning in Commercial Education

Reflecting Education 64

Wittwer, from BMHS in Steyr, Upper Austria, implemented new ways of teaching and learning which they called COoperative Open Learning (COOL). COOL is based on the principles of Helen Parkhurst’s (1922) Education on the Dalton Plan. These principles are freedom (an individual’s choice and responsibility for his or her own learning), co-operation (working in teams) and budgeting time (self-determined planning and organising learning). Parkhurst’s idea of a modern school describes a way of learning that was strongly influenced by the works of Maria Montessori and John Dewey. COOL adopts these educational views and additionally includes the following main elements: 1. Students work on interdisciplinary assignments. For about one third of their class time

(i.e. the so-called COOL lessons), students independently decide when, where and how they work on their assignments.

2. During COOL lessons, teachers work as coaches and facilitate students’ individual learning processes. Instead of whole-class teaching, the focus is on the special needs of each individual student and on purposefully promoting their learning processes.

3. Teachers of a class collaborate in teams (team teaching). 4. Teachers and students are supported by an eLearning-platform (e.g. for giving feedback

or up-loading portfolios). 5. Class councils are regularly held in which students discuss their concerns and problems

and practice conversation roles and moderation techniques. 6. The school is open for parent participation. Parent-teacher conferences involve parents

in discussing school issues and the learning processes of their children. Thus far, there has been little attempt to investigate the impact of this model on students’ academic achievement. Hence, the aim of this study is to answer the following research question: Do COOL students perform better or worse on a standardised academic achievement test in the subject Accountancy than students from the traditional track? In the remainder of the paper, we will first elaborate on theoretical assumptions (mostly derived from empirical research on self-regulated learning) that explain successful self-regulated learning, which – together with findings from previous COOL-related research – provide the theoretical foundation of our hypotheses. Then, the method and results will be addressed, followed by a discussion of findings and theoretical and empirical implications of this research.

IMPACT THEORY

COOL lacks a theoretical foundation, as it represents a bottom-up innovation for Austrian teachers, introduced exclusively for practical reasons. Therefore, in this section, theoretical assumptions on good teaching practice are drawn from a more general educational perspective, which states that learning outcomes may be understood as a function of offer and use. Production models of academic performance were first introduced by Haertel, Walberg and Weinstein (1983) and elaborated by several researchers. In German-speaking countries, the ‘supply-use model’ (in German ‘Angebots-Nutzungsmodell’), developed by Fend (1998) and Helmke (2003), has gained the most attention. Based on international empirical research, it describes several factors that, separately and in combination, are related to and influence students’ learning outcomes. Teacher behaviour is one essential factor which is of particular interest to the present study. Since COOL stresses students’



self-regulated learning (SRL) above all, the aspects of teacher behaviour that support students’ SRL competence are the focus of this section, as they provide theoretical explanations for good teaching practice in cooperative open learning environments (COOL lessons). However, before characterising this way of teaching, it is necessary to clarify what is meant by self-regulated learning (SRL). SRL is – compared to other educational terms – rather well-defined and consistently used. Boekaerts (1999, 446), Reeve, Ryan, Deci and Jang (2008, 223), Schiefele and Pekrun (1996, 257) and Schunk and Zimmerman (2008, vii) define SRL quite similarly, and refer to the process by which learners personally activate, sustain and regulate resources to achieve (mostly cognitive) learning goals or other desired states. Resources in this context are understood as cognitive (e.g. activation of prior knowledge and existing cognitive schemata as well as available learning strategies and capacities), motivational, emotional (e.g. interests, self-efficacy beliefs, anxiety), volitional and behavioural (e.g. willingness to make an effort, causal attribution), and environmental aspects (e.g. choice of learning place). The efficient use of these resources can be learned. That is why SRL competence is considered not to be a stable trait but a teachable state. Amongst others, Vrieling, Bastiaens and Stijnen (2010) tried to answer the question of how teachers can increase their students’ SRL opportunities in order to foster this state. Based on a comprehensive literature review, they formulated six process-oriented design principles of a successful SRL implementation: (1) knowledge building, (2) metacognition and content matter, (3) modelling skills, (4) scaffolding, (5) collaboration, and (6) learning tasks. Knowledge building basically means that SRL will fail if students are not equipped with sufficient (prior) content knowledge, which means that students should not be expected to regulate their learning all by themselves. It is the teachers who are the experts and thus it is their task to make the domain accessible to their students. Metacognition and content matter mean that content knowledge and (meta-)cognitive learning strategies, such as monitoring the learning process, should be built simultaneously. Research (e.g. Hattie, 2009, 192) shows that generic competencies like SRL can hardly be taught and learned without considering their specific relation to the content. Modelling skills points out that SRL skills need to be explicitly demonstrated to students, e.g. by thinking aloud or using social cognitive models (e.g. Schunk & Zimmerman, 2007). In the latter, students observe the teacher’s behaviour and emulate it, first under supervision, then without, and in the end they try to apply it under different conditions. Scaffolding refers to the reduction of teacher support during students’ learning. Teachers should reduce their support to the extent to which students possess and gain knowledge. Collaboration facilitates SRL and points out that teachers “have to guide peer interactions […] by insuring positive interdependence in the group, giving clear instructions on how to cooperate and providing adequate feedback on the co-operating process” (Vrieling, Bastiaens & Stijnen, 2010, 145).



Learning tasks should provide students with (1) academic goals to reach, (2) references to prior knowledge and (3) metacognitive strategies (such as task analysis) needed for task completion, (4) indications of how to monitor their own learning process, (5) information on how they can make judgments “about the way their work relates to the criteria” (ibid.), (6) attributional feedback which stresses factors students can control, (7) the (present and future) value and relevance of the task and (8) prompts to plan learning time devoted to different learning tasks. These eight task elements aim to support students’ monitoring and regulating of their learning processes by themselves. To summarise the findings from Vrieling, Bastiaens and Stijnen (2010), teachers who strive to increase their students’ SRL opportunities and competencies should be first of all aware of modelling sufficient content knowledge and SRL skills. Scaffolding is considered the best way to do this. Furthermore, teachers should support collaboration among their students by giving appropriate instructions and feedback. It is also important that teachers foster students’ SRL with learning tasks that include the above mentioned prompts. However, these findings represent only half of the story COOL is about. According to the COOL concept (Neuhauser & Wittwer, 2014), at least individualisation and differentiation as well as autonomy support are core characteristics of COOL-related teacher behaviour and represent the second half of the story. Whereas the first two terms are rather broad ideas of student-centred teaching (including above all the consideration of students’ prior knowledge and interests as well as forms of internal differentiation such as additional tasks for fast learners or the choice of tasks of a different level of difficulty), autonomy support refers to an empathic teacher-student-relation (e.g. listening to students, seeing the students’ point of view, etc.) that fosters students’ self-determination (Reeve et al., 2008, 231). To conclude, this is what learning environments that foster SRL should look like, at least according to literature and from a COOL point of view. However, it must be added that educational scientists and teachers assume that this way of teaching and learning not only leads to higher SRL competencies but also to better academic performances. The theoretical argument can be found in self-determination theory (Deci & Ryan, 1993, 235), which assumes that optimal learning is directly connected to the development of the individual self and at the same time is dependent on the self’s commitment. Students can only learn in a qualitative way when it is based on their own initiative. The above mentioned principles are designed to support this theory of self-determination.

RESEARCH ON COOL

The theoretical section leads to the following central questions: Are teachers aware of these principles? And to what extent do they realise these rather general ideas? First of all, it has to be mentioned that in 2004 the COOL concept was included into the nationwide curriculum for both BMS and BHS, suggesting that COOL is familiar to most BMHS teachers. Secondly, teachers who decide to adopt COOL in their lessons are trained in specific COOL teacher courses. However, it is unclear to which extent these courses focus on the above-mentioned principles. Today more than 1,000 teachers teach according to the



COOL concept and more than 20,000 students learn in COOL environments (Neuhauser & Wittwer, 2014). Some research has been done on the implementation of COOL. Qualitative research, mostly based on semi-structured interviews with COOL teachers and students (see Doppler, 2008; Kallinger, 2011; Roither, 2006; Wandl & Weissmann, 2007; Wolfthaler, 2011; Zach, 2009), support the assumption that COOL lessons are often characterised by the principles mentioned above. However, so far there have been no representative quantitative studies that test this assumption. That is why we have been conducting the longitudinal study “Lernen in Offenen und Traditionellen UnterrichtsSettings” (LOTUS, Helm, 2014b) (English: Learning in Open and Traditional Instructional Settings), from which the data for the study at hand are also derived. Initial analyses on this data set (Helm, 2014a) demonstrated from a sample of 648 BMHS students in 9th grade that those attending COOL classes, compared to those attending traditional classes, do indeed perceive significantly higher degrees of freedom, individualisation and differentiation, scaffolding, autonomy support, collaboration and support of (meta-)cognitive learning strategies. But, they do not differ in their perception of learning task elements, above all regarding those task elements that inform students about the academic goals and the practical relevance of a learning task. This empirical evidence indicates that COOL lessons do significantly differ from traditional lessons and thus should have an impact on students’ learning. This study aims to analyse the effect of COOL on academic achievement and is significant given the lack of work in this area. So far, only Fortmüller, Redlinger and Seitlinger (2012) have compared Accountancy competence of COOL students and non-COOL students in a cross-sectional study. Three hundred and eighty eight students in grades 10 and 11 (aged 15 to 17) from six commercial colleges across Austria were tested on their Accountancy knowledge on the subject matter of the current school year, the previous school year and substantive ‘comprehension questions’. Controlling for school grades and mother tongue, COOL students did not significantly differ in any of the three dimensions from non-COOL students. Altrichter, Helm and Kallinger (2013) did a second cross-sectional study on a rather small sample of 154 students (aged 15-18) showing that COOL students did not differ from non-COOL students on a standardised school exam in English and German, as well as on a standardised mathematics test (percentage calculation). In 1999, Eder evaluated a grammar school with an open instructional design similar to COOL. He used items from Trends in International Mathematics and Science Study (TIMSS) to measure school achievement in mathematics. Again, there was no difference between students from open learning and students from traditional learning classes in cross section. So far, there have been no other studies that investigate the academic performance of COOL students compared to non-COOL students. At this point it must be discussed critically if using standardised testing is appropriate in the context of the principles of COOL. It should be pointed out that the primary goals of COOL are to foster students’ personality and their generic competences rather than their academic achievement. However, due to several reasons (above all challenges in operationalizing constructs like social competencies) and the fact that the content related gain in knowledge is still the primary goal of the schools under investigation (http://www.bildungsstandards.berufsbildendeschulen.at/), the author decided to focus on the analysis of students’ performance in standardised academic achievement tests. This is



also due to the fact that standardised testing is the only way that allows making inferences about group comparisons. However, further analysis on generic competencies are currently examined (Helm, 2015).

AIM AND DESIGN OF THE PRESENT STUDY

Aim. The small amount of empirical evidence on the impact of COOL on academic achievement suggests that there is none. However, because of missing pre- and post-measurements, it is not possible to draw casual inferences from these results. In addition, all of these studies did mean comparisons between two groups of students. Thus, they were not able to distinguish between the effects of different COOL-related principles such as those stated above. It could be that some of these do have positive effects whereas others do not. Therefore, this study represents the first longitudinal one that aims to examine COOL students’ competence development in grade 9 (students aged 14 and 15) and that investigates the different effects of the following different COOL-related teaching principles/features, derived in the section “Impact Theory”: individualisation, differentiation, empathy, support of (meta-)cognitive learning strategies and collaboration, scaffolding and learning tasks. Hypothesis 1. Bearing in mind the findings from cross sectional studies, it is assumed that there is no statistically significant difference in average Accountancy competence between COOL students and non-COOL students after one year of BMHS schooling at the end of grade 9. Hypothesis 2. Although there are no significant differences when comparing average competence, it is assumed that students’ perception of some of the COOL-related teaching principles do have a positive impact on students’ Accountancy competence development in grade 9, even if prior attainment in Maths is controlled. With regard to literature and existing findings, it is not clear which of the principles will have an impact. Sample. The data come from two measurement occasions from the LOTUS study (Helm, 2014b), allowing the examination of competence development in the domain of Accountancy. The participants of the LOTUS study are students from 13 COOL classes and 14 traditional classes of eight BMHS in Austria. The students participated in an online questionnaire and took a competence test twice, once at the beginning of their ninth year (Math test) and once at the end of their ninth year (Accoutnancy test), each of which lasted one teaching unit. Students were required to fill in a systematic participant code which guaranteed anonymity and the linking of the two measurement occasions. After single imputation, data is available for 602 students (age: mean = 14.5, sd = 0.76; gender: 71.5 % female; school profile: 285 COOL students, 317 traditional students) from both measurement occasions. The schools involved in the LOTUS study were selected according to the following two criteria: First, all participating schools had to be certified as an official COOL school by the COOL impulse centre (www.cooltrainer.at). This certification guarantees that the COOL concept is implemented at a minimum standard defined by the impulse centre. Second, due to practical reasons, we selected schools we already had connections with from former teacher trainings. In almost every school two COOL classes



and two traditional classes took part in the study. Since COOL is mainly implemented at secondary stage II the LOTUS study started to collect data at grade 9. Ethics. The LOTUS study strictly follows the ethical guidelines for research established by the University of Linz (JKU, 2007). All schools and all students took part voluntarily. All of them were provided with detailed information on the nature and purpose of the study and on how anonymity is guaranteed. In those schools where the school leader asked for parental information, letters with details on the study were handed out to the students. During the test situation students were allowed to withdraw at any time – however, this was never the case. Though the design of the study suggests that this research project represents an evaluation study commissioned by the COOL representatives Neuhauser and Wittwer, this is not the case. The initiative grew out of former projects done by the author. Neuhauser and Wittwer deserve special thanks for their organisational support (i.e. establishing contact to school leaders). However, they were not involved in the research process. Measurements. Accountancy competence (WBB): The students’ competence in Accountancy at the end of grade 9 (WBB) in vocational schools was measured with the standardized instrument Knowledge Testing of Basic Knowledge of Bookkeeping (“Wissensüberprüfung von Basiskenntnissen der Buchhaltung”). WBB was developed by Helm and Wimmer (2012) within the framework of the LOTUS project. The students had to solve 53 items in WBB, which essentially required the booking of current business transactions of a business (with and without receipts). These tasks represent the main curriculum content of grade 9 in upper secondary business schools (BMHS) in Austria. In order to get the competency values in light of the probabilistic test theory (Item Response Theory, IRT) the statistical software “R” (Rizopoulos, 2011; Mair, Hatzinger & Maier, 2011) was used. IRT-based test scores usually range from -3 to +3 with a theoretical mean of 0 and standard deviation of 1, thus the average students competence in the study at hand is expected to be around 0. The reliability of the test is satisfactory (α = .80). COOL-related teaching principles: In order to record students’ perception of teacher’s behaviour in Accountancy, scales from the German Institute for International Educational Research (DaQS, n.d.) were used. However, appropriate scales were also developed since there were no published scales available for some of the principles. All items had five point Likert scale response patterns (1 = “I totally disagree”/ “never”, 5 = “I totally agree”/ “always”). Factor Sample Item # N M SD α Individualisation (Interest)

In Accountancy we have opportunities to deal with tasks which meet our interests. 2 632 2.80 1.02 .79

Individualisation (Prior Knowledge)

In Accountancy we work on assignments which match my proficiency. 8 629 3.23 0.80 .85

Differentiation (Task Difficulties)

In Accountancy we can choose from easy, medium and difficult assignments. 2 636 1.78 0.89 .58

Differentiation (Additional Tasks)

In Accountancy we work on assignments which include additional tasks for good/fast students. 2 638 2.87 1.27 .75

Empathy Our Accountancy teacher takes care to not leave students 7 632 3.40 1.00 .91



behind.

Cooperative Learning In Accountancy we work on team assignments which require us to help each other. 5 641 3.05 0.77 .78

Scaffolding (General Prompts)

Our Accountancy teacher gives us advice on how to study this subject. 4 625 2.94 0.96 .88

Scaffolding (Assignment Prompts)

Our Accountancy teacher gives us advice and information which help us to solve the task more quickly. 5 626 3.41 0.90 .88

Support of Cognitive Learning Strategies (CLS, general)

In Accountancy we learn how to structure study material in a way that helps us to get a good overview. 5 630 3.21 0.93 .86

Support of CLS (Task Analysis)

When solving assignments, I try to understand what is asked first. 3 627 3.83 0.80 .73

Support of CLS (Constructivist Tasks)

In Accountancy I get assignments which are so manifold that different skills are necessary to solve them. 3 629 3.05 0.84 .62

Support of MCLS (Metacognition) In Accountancy we reflect on the way we learn. 5 619 2.73 0.84 .80

Learning Tasks (Goals & Criteria) An assignment states which steps lead to the solution. 4 623 3.23 0.89 .80

Learning Tasks (Value & Relevance)

In Accountancy we get assignments which are relevant for my further life. 4 634 3.46 0.85 .74

Table 1: Properties of Measurement Instruments. Notes. # = Number of items of the factor. N = sample size from second measurement occasion. M = mean. SD = standard deviation. α = Cronbach’s alpha.

The properties of measurement instruments in Table 1 already reveal some important information. First of all, Cronbach’s alpha (internal consistency) of most of the scales meets the required cut-off of .70. This means that all the measures are reliable in the sense that they are relatively free of measurement errors. In other words, in repeated measurements they would lead to the same result. Only two factors are below .70, which is probably due to the low number of items used to measure these constructs. Secondly, it is interesting to note that Accountancy teachers hardly offer any differentiation by offering tasks at different levels of difficulty, as demonstrated by the low mean on this scale. Prior attainment in mathematics and students’ characteristics. Teachers’ impact on students’ competence development can only be revealed if students’ prior attainment and students’ characteristics are controlled for. Since students did not attend Accountancy classes before grade 9, it can be assumed that they do not possess any Accountancy competencies at the beginning of grade 9. However, to control at least for some aspects of their cognitive abilities, Maths proficiency was measured using MATKOMP (Eder, Gaisbauer & Eder, 2002) – which consists essentially of TIMSS items – at the beginning of grade 9. As a second measure of prior attainment students’ Maths grade at the end of grade 8 was collected. Additionally, student characteristics such as gender and economic social and cultural status (ESCS) were measured using items from the Programme for International Student Assessment (PISA) 2006. Statistical Analyses. In order to test the two hypotheses stated above, the collected data is analysed by means of a t-test (hypothesis 1) and Multilevel Regression Analysis (hypothesis 2) by using SPSS and Mplus (Version 7.2, Muthen & Muthen, 1998-2012). Multilevel analysis takes the hierarchical structure of the data (i.e. students are nested within classes) into account. Nested data usually does not meet the independence



assumption that is assumed in multiple regression: The perceptions of teacher behaviour from students in the same class are not independent from each other as they are taught by the same teacher. Thus, they tend to be more similar than perceptions from students of different classes. Multilevel techniques address this violation. Furthermore, they allow for testing contextual effects, e.g. it is possible to investigate to what extent the measures of this study (as well as their effects) vary at student-level and class-level. In order to investigate context effects the measurements are aggregated at class-level to obtain a “shared perception” of the Accountancy teacher’s behaviour.

RESULTS

Descriptive Statistics. Table 3 and Table 4 in the appendix show the bivariate correlations among the measurements at student-level as well as at class-level. These correlations already give some notable insights: (1) In most cases, class-level correlations are higher than student-level correlations. This indicates that the “shared perception” of teacher behaviour is more reliable (in terms of showing a clear picture on what is really going on in the classroom) than the individual perception, since individual deviations from the average perception are ignored. (2) The correlations among Accountancy competence at the end of grade 9 and competence as well as grades in the domain of Mathematics at the beginning of grade 9 attest validity to the measurements since they are of similar magnitude to those found in other studies (e.g. Seeber, 2013, 86; Winther, Sangmeister & Schade, 2013). Furthermore, this indicates that prior cognitive abilities in Maths do matter and might be an important predictor for students’ competence development in the domain of Accountancy. (3) Some SRL design principles are correlated with students’ Accountancy competence at both levels. This is true for differentiation, empathy, scaffolding, task analysis, and learning task relevance. Whereas empathy, relevance and task analysis are positively related, both factors of differentiation are negatively correlated. Scaffolding in the sense of teacher support during assignment work is correlated positively at student-level and negatively at class-level. A plausible explanation for the latter finding is that, on the one hand, well-performing students perceive scaffolding more often, however, on the other hand, teachers in poor-performing classes more often provide students with individual support during work on assignments. (4) Some SRL design principles are only correlated with students’ Accountancy competence at one level: The consideration of prior knowledge in Accountancy is only positively correlated with Accountancy competence development at student-level, whereas constructivist tasks and metacognition are only negatively related to Accountancy competence at class-level. The reason for the latter finding can be seen again in the idea that in poor-performing classes teachers more often try to foster students’ learning strategies by applying tasks that can be solved in different ways and by reflecting with students about how they learn. Interpreting findings (3) and (4) in that way indicates that instructional measurements that should help to foster self-regulation are more often applied



in lower performing classes. A different interpretation would be to argue that these measurements indeed lead to lower average performance (see below). (5) There is evidence that students’ characteristics are related to teachers’ behaviour: The average students’ economic, social and cultural status of a class correlates negatively with all SRL design principles. This indicates that classes with an unfavourable class composition in terms of students’ social background perceive a less favourable learning environment in terms of the SRL principles than students from classes with a more favourable social composition. Furthermore, gender ratio of a class is positively related to the principles, meaning that if there are more boys in a class, the perception of the learning environment is evaluated more positively. Additional analyses (not reported here) show that this finding is not influenced by school form (HAK vs. HLW)1 or school type (BMS vs. BHS). Students’ prior attainment in Mathematics is both at student-level and at class-level negatively correlated with most of the principles, whereby the relations are stronger at class-level. This is another indication that COOL and its principles are more often applied in low-performing classes. (6) Finally, correlations show that the factors differentiation, cooperative learning and task analysis do not substantially correlate with the other factors. In contrast, individualisation, scaffolding and learning strategies, as well as learning tasks, are strongly related to each other, indicating that they more often occur at the same time and that they are considered more similar than the factors which do not correlate that strongly with each other. Hypothesis 1. The assumption that COOL students do not significantly differ from traditionally instructed students regarding their Accountancy competence at the end of grade 9 is not confirmed by the results of a t-test at student-level (MCOOL = -.69, Mnon-COOL = -.35; SDCOOL = 1.50, SDnon-COOL = 1.39; t[601]= -2.86, p = .004, d = .24). This indicates that on average COOL students seem to perform lower on WBB than traditional students. However, before interpreting this result, it is also important to note that students’ characteristics, such as their prior cognitive abilities, are not taken into account. Hypothesis 2. Hypothesis 2 assumes that COOL design principles have an impact on students’ competence development, even if students’ characteristics are controlled. Table 2 shows the results of four Multilevel Regression Analysis Models. These models are built up in the traditional way Multilevel Models are reported. First, model 1 investigates the class-level variance of students’ performance. This gives an idea of how strongly belonging to a specific school class influences students’ performance. Second, model 2 investigates the effect of the COOL concept when controlled for student-level predictors such as students’ characteristics. Third, model 3 investigates the effect of the individual perception of the teacher’s behaviour on their learning outcomes, controlling for students’ characteristics. Finally, model 4 includes class-level predictors as well; these are context effects, such as the shared perception of teacher behaviour.

1 HAK (Handelsakademie) and HLW (Höhere Lehranstalt für wirtschaftliche Berufe) are higher

commercial colleges (BHS), whereby HLW represents a female-dominated school form.



Model 1 represents a so-called “empty model”, which shows how much variance in students’ Accountancy competence is at student or class-level, respectively. The class-level variance is indicated by the Intraclass Correlation (ICC), which equals 46 %2, indicating that almost half of the WBB score difference between the students is at class-level and thus can be explained by class and school level variables such as COOL or the teachers’ behaviour or school type and location. Furthermore, model 1 shows that the average WBB score is -0.486. Model 2 includes student characteristics to explain WBB score differences at student level. Results show that Mathematics proficiency at the beginning of grade 9 (T1) measured via MATKOMP and Maths grades significantly predict Accountancy competence (WBB at T2). Furthermore, male students do significantly worse on the WBB test. However, due to the small sample of male students, this should not be overvalued. In contrast to the student characteristics mentioned, students’ economic, social and cultural statuses (ESCS) do not influence their competence development in the subject Accountancy. Model 2 additionally includes COOL/non-COOL as a class-level predictor of students’ WBB test scores. However, the low regression coefficient (magnitude of the effect of the design principle) and high standard error (in italics; indicator of the accuracy of the estimated effect) resulting in a p value of .655 (= error probability of 65.5 % when assuming that COOL has an effect on students’ outcome) clearly show that the COOL concept does not affect students’ competence development. Model 3 includes the COOL-related design principles as student-level predictors to test whether the individual perception of these factors of teacher behaviour makes a difference. Model 3 results show that from a students’ perception, at least considering their prior knowledge, doing task analysis and support of metacognition have an impact on their Accountancy competence development. However, contrary to the theoretical assumption that metacognition should have a positive impact on students’ learning, metacognition does have a negative effect in the study at hand. Due to the small number of classes, model 4 only includes those student-level predictors that were revealed to be significant in model 3. Furthermore, each class-level predictor (= aggregated design principles) was included separately. Otherwise, estimation problems occurred. This way of data analysis ended up with two significant class-level predictors: average class-level of Maths grades and average class-level of differentiation via additional learning tasks. Whereas a level of good Math grades in a class leads to higher Accountancy competence,3 differentiation – again against the assumption – is associated with poor Accountancy performance. One potential reason for the latter finding might be that differentiation is more often applied as a remedial instrument in low-performing classes.

2 The main reason for the outstandingly high ICC reported here is the fact that the sample includes classes

from different school types (BMS and BHS) which significantly differ in the average performance of their students, thus class-level variance is higher.

3 Since in the Austrian education system grades are represented as numbers and high numbers mean lower students’ performances, negative regression coefficients/correlations indicate positive effects or relations, respectively.



Another potential reason could be that differentiation via additional tasks might demotivate students, as argued in the discussion section. In the bottom part of Table 2, criteria for model evaluation are presented. Statistical models such as those in Table 2 are evaluated by the degree to which they explain the differences in the students’ individual test scores. The R2 values (Snijders & Bosker 2000) show how much percent of the variance in the WBB scores can be explained by the predictors (such as gender or perceived teacher behaviour) included in the models. In addition, the so-called Sample-Size Adjusted Bayesian Information Criterion (BICa) is used to compare two different models, whereby the model with the lowest value is the best fitting model and thus best represents the structure and information of the data. The figures clearly show that model 4 is the best model since it explains more variance in students’ Accountancy competence development than the others. This is not surprising since it includes only significant predictors. Model 4 explains almost half (45 %) of the WBB score variance at class-level. This is quite high, given the low number of class-level predictors. However, only one third (32 %) of the difference in the students’ Accountancy competence development at individual level is explained. This indicates that there are other important student-level predictors which are not included in the model yet. Overall, the findings from Multilevel Regression Analysis only partially support hypothesis 2. “Partially” means that after controlling for students’ characteristics, only four design principles out of 14 revealed to be statistically significant, which in turn means that their effect is actually different from 0. Moreover, two of these principles turned out to have a negative effect, contrary to what was suggested by literature.

M1 M2 M3 M4

Response: WBB β S.E. Β S.E. β S.E. β S.E.

Fixed Part

WBB-Intercept (Constant) -0.486 0.207 -0.701 0.609 -0.699 0.581 1.930 0.817 Mathematics Competence T1

0.026 0.007 0.025 0.007 0.025 0.007

Mathematics Grade T1

-0.240 0.050 -0.198 0.054 -0.187 0.049 Gender (male)T1

-0.234 0.122 -0.195 0.122 -0.221 0.113

ESCS T1

-0.044 0.051 -0.040 0.052 Individ. (Interests) T2

0.065 0.053

Individ. (Prior Knowledge) T2

0.359 0.122 0.411 0.089 Diff. (Additional Tasks) T2

-0.053 0.050

Diff. (Task Difficulties) T2

-0.014 0.080 Empathy T2

-0.043 0.097

Cooperative Learning T2

-0.022 0.063 Scaff. (General Prompts) T2

0.101 0.060

Scaff. (Assignment Prompts) T2

0.012 0.075 Support CLS (General) T2

-0.019 0.092

Support CLS (T. Analysis) T2

0.215 0.064 0.243 0.062 Support CLS (Const. Task) T2

-0.039 0.068

Support MCLS (Metacog.) T2

-0.214 0.070 -0.232 0.050 Learn. T. (Goals & Criteria) T2

-0.076 0.053

Learn. T. (Value & Rel.) T2

0.083 0.081



Random Part

COOL

0.165 0.368 0.159 0.349

Agg. Maths Grade -0.403 0.178 Agg. Diff. (Add. Tasks) -0.380 0.160 Class-level Variance 1.087 0.348 0.838 0.285 0.739 0.257 0.587 0.219 Student-level Variance 1.271 0.126 1.113 0.101 1.006 0.102 1.018 0.103 Total Variance 2.358 1.951 1.745 1.605 ICC % 46 39 35 37 Adjusted Bayesian IC (BICa): 1944.6

1877.5 1861.4

1826.5

R2 Class-level % 23 26 32 R2 Student-level % 17 32 45 Units: Class 27

27

27

27

Units: Student 602

602

602

602 Table 2: Multilevel Regression Analysis Explaining Accountancy Competence

Development. Notes. M1-M4 = Model 1 - Model 4. WBB = Student’s Score on Accountancy Test. β = unstandardized Regression Coefficient. S.E. = Standard Error. T1/T2 = Measurement Occasion at the Beginning of Grade 9/at the End of Grade 9. Agg. = Data-Aggregation at Class-level. ICC = Intraclass Correlation. IC = Information Criterion. All statistically significant results are marked bold.

DISCUSSION

Research aim. Given the widespread use of COOL-related teaching practices in Austrian secondary education and the lack of research providing an answer to the effectiveness of this education type, our study aimed to examine COOL students’ competence development in grade 9 and to investigate the different effects of COOL-related teaching principles. Following earlier research and theoretical work, we hypothesised that the COOL design principles “individualisation and differentiation, empathy, support of cooperative learning and (meta-)cognitive learning strategies, scaffolding, and the application of specific learning task characteristics” support students’ self-regulated learning and thus lead to higher student outcomes. Descriptive statistics, t-test and Multilevel Regression Analysis revealed the impact power of COOL or COOL-related design principles, respectively. Findings. The major findings from descriptive analyses showed that differentiation, scaffolding, the use of “constructivist” tasks and metacognition were negatively related to Accountancy scores that were aggregated on class-level. One potential interpretation is that these principles were used above all within poor-performing classes. The disproportional correlations (especially at class-level) between students’ prior attainment in Mathematics (MATKOMP, grades) and most of the design principles pointed in the same direction. This fits the assumption that COOL, or more precisely, its design principles, were more strongly applied in “difficult classes” as a kind of remediation measure. Since COOL was originally introduced as a remediation measure, this interpretation seems plausible.



Furthermore, the fact that students’ perception of an empathic teacher-student-relationship was positively correlated with academic performance in Accountancy at both levels needs to be interpreted with caution. On the one hand, one could argue that supportive and understanding teachers stimulate increased student performance. On the other hand, it could be the other way around as well: teachers were more supportive of good students. A third explanation could be that good students in general evaluated their learning environment more positively, as argued in other studies (Clausen, 2002; Ditton, 2002). In addition, teachers’ support of task analysis and task relevance was positively correlated with students’ performance at both levels. These findings might not be surprising. However, so far there has been very little research on what characterises well-performing students within the domain of Accountancy possess (e.g. see Tenberg, 2008 for studies on SRL competences in vocational domains). This study showed that, like in other domains, well-performing Accountancy students obviously solved tasks in a more systematic and strategic way. Regarding the factor “task relevance”, it seemed obvious that those Accountancy students who recognised the intrinsic value of the subject performed better (e.g. Helm, accepted, for reciprocal effects of intrinsic motivation and academic performance in Accountancy). Interestingly, consideration of students’ prior knowledge only mattered when individually perceived, whereas class-level perception did not correlate with students’ competence at class-level. This is in line with the assumption that linking new information to prior knowledge is a constructivist process that has to happen in a way appropriate to one’s needs. Thus, individual perception is a more valid evaluation of the degree to which this appropriateness occurs while learning. Finally, descriptive statistics showed negative correlations between students’ economic social and cultural status (ESCS) and perceived teacher behaviour. This suggested that students within classes with a higher number of socially “disadvantaged” students perceived a more unfavourable learning environment in terms of the implementation of SRL design principles. However, the relations between COOL-related design principles and students’ Accountancy competence did not allow a judgment on whether COOL students in fact performed better or worse on the Accountancy test WBB. Looking at the mean difference in WBB results, a t-test revealed significantly higher scores in favour of traditional students. This showed that student performances were lower in COOL classes; but did it say anything about the direction of the causal effect? Did COOL indeed lead to worse performance or was COOL simply applied more often in classes with unfavourable compositions in terms of students’ cognitive abilities? The latter assumption is not far-fetched since COOL originally was introduced as a remedial measure as mentioned in the introduction. Therefore, it was necessary to control for students’ prior knowledge in order to reveal the value added by COOL. Furthermore, it should be mentioned that according to Cohen’s effect size the mean difference in test scores was of low practical significance (d = .24). In addition, statistical assumptions of the t-test were violated due to the hierarchical structure of the data which made Multilevel Analysis necessary for reliable results anyway.



In hypothesis 2, it was assumed that the COOL-related principles positively affected students’ competence development. Since one could argue based on literature that all of these principles should positively affect students’ learning, it was a bit surprising that in fact only four of them were significant predictors. Above and beyond students’ characteristics, the consideration of students’ prior knowledge and the support of task analysis positively influenced students’ learning in the subject Accountancy (at student-level), as we also saw from the descriptive statistics. Since students’ Mathematics competence was also a significant predictor, Accountancy teachers are well advised to focus on instructional measures that help students (1) find out more about their prior knowledge and (2) link new concepts to existing ones. Doing task analysis is of course strongly connected with knowledge integration, since in task analysis students (with support from teachers) have to think about what the tasks actually require from them and which of the required knowledge they already possess. The negative impact of metacognition was not expected. Metacognition in the study at hand represented how often students (together with their teacher) reflect on their learning process and strengths and weaknesses during class. It might be that these activities tie up time resources that are actually needed for learning content-related concepts. This finding could be an indicator that teachers’ support of students’ metacognition is not done simultaneously with knowledge building, as suggested by Vrieling, Bastiaens and Stijnen (2010). The findings at class-level indicated what is well known from other disciplines: a low level of prior attainment in class hampers the individual learning process (e.g. Scharenberg, 2012 for Reading and Mathematics). One reason for this might be that low levels of prior attainment in class could be an indicator of classes which are not easy to manage, since low academic achievement is often accompanied with low levels of interest and commitment. The above mentioned results regarding classes with low ESCS students supported this assumption. On the other hand, findings showed that the use of additional tasks for good and fast working students negatively influenced students’ learning outcomes. Although students’ competencies in Maths were controlled, this finding might be interpreted in the same way as with descriptive statistics. Differentiation in the form of obligatory and additive tasks was applied as remedial measures, above all in poor-performing classes. The alternative interpretation, namely the idea that this design principle indeed affects students learning negatively, is hard to believe since additional tasks are a supporting measure. However, the key educational issue is in the nature of the tasks and how these are perceived by the students. Poorly designed and inappropriate tasks may undermine students’ progress in learning and cause demotivation. If hard working students are rewarded with doing more tasks that the student has already mastered, they might be discouraged and learn not to complete tasks so quickly. Thus, additional tasks need to be interesting and different from those already completed (e.g. they should focus on a specific issue). They should not just be more of the same. Further investigations are necessary here. The contribution of the present study. A hundred years after Helen Parkhurst published her ideas about what teaching and learning should look like, one could ask if these suggestions are still of relevance today. Not only Austrian teachers, but also teachers from the Netherlands, where the “Education on the Dalton Plan” is also prominent, would clearly answer with affirmative arguments: (1) the social society is becoming more and more heterogeneous. This is reflected in classrooms and teachers need to react appropriately. (2)



Furthermore, real world problems such as ecological and democratic ones call for human beings to be able to think critically, to think independently, and to act responsibly. Given that the social challenges facing schools have changed considerably since the beginning of the last century, alternative ways of teaching such as the Dalton Plan or COOL are needed now more than ever. Not only from a practical point of view, but also from a scientific point of view, constructivist approaches that foster self-regulated and self-determined learning are considered to meet those characteristics of good teaching practice that are known from educational psychology most closely (Deci & Ryan 1993). Against this background, teachers should be encouraged to think about alternative ways of teaching, such as the one presented here. Thus, the major research aim of the present study was to provide more insights about the effects of COoperative Open Learning (COOL) on students’ competence development in Accountancy. The above results showed that this aim was achieved. In summary, it can be said that the study at hand provides substantive further knowledge on the impact of cooperative open learning on students’ competence development. Whereas former research has tried to get along with cross-sectional studies, this study is the first longitudinal one which controls students’ competence development for a comprehensive set of students’ characteristics, such as their prior attainment. Furthermore, the present study used more appropriate statistics that allow to model COOL design effects as class-level effects. Earlier studies simply used a mean comparison between COOL students’ achievement and non-COOL students’ achievement. Furthermore, this study was the first to test the impact of design principles for SRL and open learning environments established by Vrieling, Bastiaens and Stijnen (2010) and “alternative” educational models such as the Education on the Dalton Plan (Parkhurst 1922). From the results of the study at hand, we can conclude that these design principles might be plausible and important for students’ learning from a theoretical point of view. However, when it comes to real world teaching, students’ characteristics such as their prior attainment seem more relevant for successful (self-regulated) learning processes than most of the instructional advice recommended by literature. However, this does not mean that teachers do not make a difference; quite the contrary is true: Implications for the teaching practise. One important conclusion of the presented study is that teachers’ should not fear that using alternative, modern ways of teaching might result in lower students’ competencies. Critics often fuel these unjustified fears using literature and research that open learning and “unguided” instruction are less effective than teacher-led instruction (Kirschner, Sweller & Clark 2006). Of course, these fears are justified if teachers fail to implement appropriate design principles. However, using the example of the Austrian “COOL movement”, we could show that students’ learning does not suffer from open learning environments. Although the present study did not fully reveal the desired and expected effects of COOL, the COOL concept does not lead to negative effects, even if the findings point out two principles teachers should be particularly aware of: support of metacognition and differentiation using additional tasks. The negative effects of the first principle could be interpreted as an indicator of inefficient use of metacognitive strategies. Items such as “In our Accountancy class we often think about our strengths and weaknesses” might indicate learning environments that support the development of individuals’ self, but may waste time necessary for learning content-based concepts. Thus, teachers should be aware that time is used efficiently. Supporting students’ metacognitive skills is a



fundamental goal of teaching, however with regard to students’ cognitive learning outcomes, teachers have to make sure that they are taught simultaneously (see principle “Metacognition and content matter” by Vrieling, Bastiaens & Stijnen, 2010). The negative effect of the differentiation-principle could be an artefact (see discussion above). However, teachers should again be aware that differentiation could have negative effects. When learning and working fast within lessons is “rewarded” with additional tasks (more of the same) this might lead to competition and performance avoidance which might not be encouraging for students, thus leading to lower efforts and competence gains. Apart from these negative effects, the positive effects found mainly point out that students’ prior attainment and knowledge needs to be considered when teaching. Although teachers can hardly influence students’ prior attainments, they can try to teach in a way that more often highlights how new concepts or content matter that needs to be learned are related and linked to what students already know. Using concept maps and advanced organisers, referring to daily examples, etc. might be appropriate ways to do so. Finally, the importance of students’ characteristics for their learning outcomes in Accountancy showed additionally that the teachers’ contribution is limited and that if things go wrong it is not always their fault. Knowing this might help teachers to more easily cope with the challenges of classroom practice. Limitations of the present study and directions for further research. In the present study, students’ perception was used to gain a picture of their learning environment. Sometimes these measures are appropriate as they are more closely related to students’ academic achievement (Krammer, 2009). However, more objective measures such as classroom observations or analyses of classroom videos might reveal different effects. Combining students’ and teachers’ perceptions might lead to more objective measures as well. Further COOL studies should do so, since students’ perception might be influenced by different factors (students’ abilities, teacher-student relationships, gender, age, etc.). Certainly, the effects of COOL are not limited to cognitive ones. The study at hand fails to shed light on non-cognitive students’ “outcomes” such as wellbeing, learning motivation and volition, social skills, study skills, etc. Previous cross-sectional studies (Altrichter, Helm & Kallinger, 2013; Eder, 1999; Neubauer, 2010) have provided evidence that COOL affects learning regarding these educational goals. Further research should test whether this is true using more representative samples and longitudinal designs. Moreover, the present study did not investigate the effects of further COOL elements such as COOL teachers’ work with parents or students’ work in their class councils. It might be that COOL mainly impacts educational practice in these areas. From a statistical point of view, one could argue that the present study did not include school level predictors. In LOTUS (Helm, 2014b), there is evidence that schools (BMHS locations) differ quite substantially with regard to students’ average performance. The magnitude of school level effects might exceed class-level effects, indicating that educational policies might work better at this level (e.g. improvement in school leadership). However, since no school level predictors have been measured in the present study, it was not possible to include them. Further research faces great challenges to do so since huge samples are required to investigate these effects on students’ achievement. In light of these limitations, the reader should use the findings from the study at hand as a starting point.



REFERENCES

Altrichter, H., Helm, C., & Kallinger, B. (2013). Perspectives of developing lower business schools. Findings and experiences from an implementation study on a reform-based school model. Teaching and Education, 163(1-2), 125-141. Boekaerts, M. (1999). Self-regulated learning: Where we are today. International Journal of Educational Research, 31, 445-475. Clausen, M. (2002). Teaching quality: A question of perspective? Münster: Waxmann. DaQS (n.d.). Database on the quality of schools. http://daqs.fachportal-paedagogik.de/constructlist (Retrieved date: February 20, 2014). Deci, E. L., & Ryan, R. M. (1993). Self-determination theory of motivation and its importance to education. Journal for education, 39(2), 223-238. Ditton, H. (2002). Teachers and teaching from students’ point of view. Study results in Maths. Journal for education, 48(2), 262-286. Doppler, B. (2008). Cooperative Open Learning in upper secondary school and its contribution to individualisation and differentiation in teaching. (Unpublished Master’s thesis). Department of Education and Educational Psychology, Johannes Kepler University, Linz. Eder, F. (1999). Open learning at Dornbirn Bundesgymnasium (research report). Department of Education and Educational Psychology, Johannes Kepler University, Linz. Eder, F., Gaisbauer, H., & Eder, C. (2002). MATKOMP. A procedure for the acquisition of mathematical competences at the end of lower secondary school (research report/test manual). Department of Education and Educational Psychology, Johannes Kepler University, Linz. Fend, H. (1998). Quality in the education system. School research on system documents, school profiles and teacher performance. Weinheim: Juventa. Fortmüller, R., Redlinger, B., & Seitlinger, C. (2012). A national comparison study of Accountancy proficiency of students in COOL classes and in conventional classes in Austria. wissensplus, 5-09/10, 15-18. Haertel, G. D., Walberg, H. J., & Weinstein, T. (1983). Psychological models of educational performance: A theoretical synthesis of constructs. Review of Educational Research, 53, 75-91. Hattie, J. (2009). Visible learning. New York: Routledge. Helm, C. (2014a). Measuring degrees of freedom in Accountancy classes using Latent Class Analysis. In I. Benischek, A. Forstner-Ebhart, H. Schaupp, H. Schwetz, & B. Swoboda (Eds.), Empirical research in educational areas (pp. 331-343). Vienna: Lit. Helm, C. (2014b). Learning in open and traditional instructional settings (LOTUS). (Unpublished doctoral thesis). Department of Education and Educational Psychology, Johannes Kepler University, Linz. Helm, C. (accepted). Reciprocal effects of teachers’ volitional functioning support and students’ intrinsic motivation as well as students’ academic achievement. Submitted to the Conference Proceedings of the Austrian Conference for Research in Vocational Education. Helm, C. (2015). Social Network Analysis of Cooperative Learning in Traditional and Progressive Education. Submitted for Paper Presentation at the 16th Biennial Conference of the European Association for Research on Learning and Instruction, Limassol, 25-29 August.



Helm, C., & Altrichter, H. (2012). How to motivate young people for school? Evaluation of a new school model in light of its retention power. In Niedermair, G. (Ed.), Evaluation as a challenge in vocational education and human resource development (pp. 285-305). Linz: Trauner. Helm, C., & Wimmer, B. (2012). How to measure students’ learning success in Accountancy? wissenplus, 5-11/12, 24-29. Helmke, A. (2003). Understanding, evaluating and improving teaching quality. Seelze-Velber: Kallmeyer. JKU (2007). Guidelines. To ensure good scientific practice at Johannes Kepler University of Linz. http://www.jku.at/wissen/content/e16607/e16332/e16280/e16274/e16273/files 140045/RichtliniezurSicherungguterwissenschaftlicherPraxis_2011_ger.pdf (Retrieved date: May 31, 2014). Kallinger, B. (2011). The evaluation of the concept HASNEU (new Bregenz business school) – a school with full-day classes and open forms of learning. (Unpublished Master’s thesis). Department of Education and Educational Psychology, Johannes Kepler University, Linz. Kirschner, P. A., Sweller, J., & Clark, R. E. (2006). Why minimal guidance during instruction does not work: An analysis of the failure of constructivist, discovery, problem-based, experiential, and inquiry-based teaching. Educational Psychologist, 41(2), 75-86. Krammer, K. (2009). Teacher support during individual work. Münster: Waxmann. Mair, P., Hatzinger, R., & Maier, M. (2011). eRm. Extended Rasch Modeling. 0.14-0. (R-package). http://www.r-project.org. (Retrieved date: February 20, 2014). Muthén, B. O., & Muthén, L. K. (1998-2012). Mplus (Version 7.2) [Computer software]. Los Angles, CA. Neubauer, M. (2010). Cooperative open learning at lower business schools. A study on effects on students’ self-efficacy, learning strategies and attitudes towards teamwork. (Unpublished doctoral dissertation). Institute for Business Education, University of Economics and Business, Vienna. Neuhauser, G., & Wittwer, H. (2014). Cooperative open learning – with e-learning. Official website. www.cooltrainers.at (Retrieved date: February 3, 2014). Parkhurst, H. (1922). Education on the Dalton Plan. New York: Dutton & Company. Reeve, J., Ryan, R., Deci, E. L., &. Jang, H. (2008). Understanding and promoting autonomous self-regulation. In D. H. Schunk, & B. J. Zimmerman (Eds.), Motivation and Self-Regulated Learning (pp. 223-244). London: Lawrence Erlbaum Associates. Rizopoulos, D. (2011). ltm. Latent Trait Models under IRT. 09-7. (R-package). http://www.r-project.org (Retrieved date: February 20, 2014). Roither, N. (2006). Cooperative open learning (COOL) in Austrian business schools. (Unpublished Master’s thesis). Institute for Business Education, University of Economics and Business, Vienna. Scharenberg, K. (2012). Performance diversity and competence development. Münster: Waxmann. Schiefele, U., & Pekrun, R. (1996). Psychological models of non-autonomous and self-controlled learning. In F. E. Weinert (Ed.), Psychology of learning and instruction (pp. 249-278). Göttingen: Hogrefe. Schunk, D. H., & Zimmerman, B. J. (2007). Influencing children’s self-efficacy and self-regulation of reading and writing through modeling. Reading and Writing Quarterly, 23, 7-25.



Schunk, D. H., & Zimmerman, B. J. (2008). Motivation and self-regulated learning: theory, research, and applications. New York: Lawrence Erlbaum Associates. Seeber, S. (2013). Mathematics competencies at the end of commercial education. Journal for vocational and commercial education, (special issue 26), 67-94. Snijders, T. A. B., & Bosker, R. J. (2000). Multilevel analysis. An introduction to basic and advanced multilevel modeling. London: SAGE Publications. Tenberg, R. (2008). What are study skills and how can they be measured? Journal for vocational and commercial education, 104(4), 539-555. Vrieling, E. M., Bastiaens, T. J., & Stijnen, S. (2010). Process-oriented design principles for promoting self-regulated learning in primary teacher education. International Journal of Educational Research, 49, 141-150. Wandl, R., &. Weissmann S. (2007). Cooperative open learning in upper secondary vocational schools. (Unpublished Master’s thesis). Institute for Business Education, University of Economics and Business, Vienna. Winther, E., Sangmeister, J., & Schade, A. K. (2013). Correlations among generic and vocational competencies in early commercial education. Journal for vocational and commercial education, (special issue 26), 139-160. Wolfthaler, B. (2011). COOL as an innovation in the school system – What were supportive and hindering factors of COOL at Steyr Business School? (Unpublished Master’s thesis). Department of Education and Educational Psychology, Johannes Kepler University, Linz. Zach, D. (2009). COOL – a teaching method with a future? (Unpublished Master’s thesis). Institute for Business Education, University of Economics and Business, Vienna.

Correspondence

Christoph Helm Johannes Kepler University Altenberger Straße 69, 4010 Linz, Austria, E-mail: [email protected]



Factors 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 1 Accountancy Competence T2 1 2 Mathematics Competence T1 .303** 1 3 Mathematics Grade T1 -.347** -.416** 1 4 Gender T1 -.018** .071** -.087** 1 5 ESCS T1 .026** .204** -.180** .187** 1 6 Individ. (Interests) T2 .060** -.109** .012** .106** -.067** 1 7 Individ. (Prior Knowledge) T2 .142** -.104* -.001** .082** -.076** .551** 1 8 Diff. (Additional Tasks) T2 -.153** -.119** .111** .037** .016** .194** .202** 1 9 Diff. (Task Difficulties) T2 -.165** -.259** .187** .111** -.049** .277** .256** .256** 1 10 Empathy T2 .132** -.113** -.010** .066** -.127** .520** .795** .139** .188** 1 11 Cooperative Learning T2 .008** -.032** .022** -.017** -.043** .219** .270** .226** .156** .214** 1 12 Scaff. (General Prompts) T2 .016** -.157** .139** .094** -.089** .505** .707** .258** .360** .667** .299** 1 13 Scaff. (Assign. Prompts) T2 .143** -.062** .020** .080** -.091** .472** .777** .194** .191** .769** .226** .689** 1 14 Support CLS (General) T2 .053** -.153** .050** .041** -.055** .493** .735** .211** .290** .690** .326** .741** .685** 1 15 Support CLS (T. Analysis) T2 .216** .066** -.098** -.033** -.021** .199** .270** .046** -.157** .261** .200** .208** .317** .278** 1 16 Support CLS (Const. Task) T2 .006** -.142** .061** .101** -.058** .418** .596** .242** .259** .493** .294** .541** .549** .562** .253** 1 17 Support MCLS (Metacog.) T2 -.060** -.159** .126** .124** -.046** .467** .597** .293** .426** .493** .346** .661** .529** .660** .219** .534** 1 18 Learn. T. (Goals & Criteria) T2 .022** -.109** .074** .100** -.105** .499** .630** .119** .227** .532** .166** .552** .570** .562** .213** .481** .499** 1 19 Learn. T. (Value & Rel.) T2 .153** -.024** .034** .007** -.087** .406** .690** .161** .124** .606** .208** .573** .661** .621** .395** .515** .478** .515**

Table 1: Bivariate Student-level Correlations Among the Measures.



Aggregated Factors 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18

1 Accountancy Competence T2 1 2 Mathematics Competence T1 .403** 1 3 Mathematics Grade T1 -.487** -.774** 1 4 Gender Ratio T1 .053** -.108** -.042** 1 5 ESCS T1 .104** .505** -.539** .287** 1 6 Individ. (Interests) T2 -.055** -.345** .207** .248** -.388** 1 7 Individ. (Prior Knowledge) T2 .027** -.300** .134** .186** -.498** .893** 1 8 Diff. (Additional Tasks) T2 -.350** -.634** .468** .124** -.286** .627** .516** 1 9 Diff. (Task Difficulties) T2 -.315** -.272** .213** -.020** -.081** .390** .221** .132** 1 10 Empathy T2 .097** -.266** .101** .166** -.448** .854** .965** .440** .218** 1 11 Cooperative Learning T2 .018** .012** -.030** -.251** -.176** .533** .433** .236** .456** .423** 1 12 Scaff. (General Prompts) T2 .062** -.294** .196** .209** -.499** .821** .966** .416** .185** .976** .362** 1 13 Scaff. (Assignment Prompts) T2 -.101** -.463** .293** .302** -.448** .899** .933** .643** .242** .908** .408** .914** 1 14 Support CLS (General) T2 -.065** -.428** .239** .236** -.431** .903** .960** .615** .283** .931** .469** .930** .973** 1 15 Support CLS (T. Analysis) T2 .269** .220** -.145** .129** -.051** .169** .242** -.485** .085** .311** .136** .340** .126** .189** 1 16 Support CLS (Const. Task) T2 -.144** -.386** .284** .214** -.547** .837** .886** .562** .155** .845** .390** .872** .880** .887** .284** 1 17 Support MCLS (Metacog.) T2 -.166** -.469** .310** .359** -.380** .880** .833** .693** .317** .761** .442** .788** .917** .908** .099** .872** 1 18 Learn. T. (Goals & Criteria) T2 -.105** -.344** .285** .267** -.478** .780** .891** .528** .060** .806** .319** .863** .876** .900** .203** .878** .835** 1 19 Learn. T. (Value & Rel.) T2 .083** -.168** .171** .174** -.596** .709** .840** .230** .139** .832** .246** .867** .725** .728** .438** .804** .616** .790**

Table 2: Bivariate Class-level Correlations Among the Aggregated Measures.

cooperative open learning in commercial education ......helm, c. cooperative open learning in...

Documents