the design of authentic tasks that promote higher-order ... · the design of authentic tasks that...

23
Page 1 of 23 The design of authentic tasks that promote higher-order learning Hanne ten Berge, Stephan Ramaekers, Albert Pilot IVLOS Institute of Education, Utrecht University, the Netherlands Abstract This study focuses on the use of authentic cases to provoke higher-order learning. It is based on a review of recent research literature and focuses in particular on the design features of these cases. First, we looked into the characteristics of real life problems academics are confronted with in their professional practices and the way they solve those problems. In the transformation of these to educational tasks, elements of authenticity get lost. Awareness of the features that are crucial to authenticity of the case seems important if higher-order learning is to be achieved. The literature not only provided indications for the design of authentic cases, we also came across examples of negative effects of authentic cases and improper design. The results of our search show that the impact of authenticity on learning outcomes is promising but it does not present conclusive results. Authenticity appears to be a much too wide ranging formulation to be helpful in the design of case attributes. In the future we will focus more closely on the coherence of the factors that play a part in higher-order learning. Introduction Over the last decade, the use of authentic cases, problems and projects as a starting point for learning has won ground in higher education. Usually, these cases refer to situations that are complex, contain some hidden, ill-defined or open-ended problems and often require a multidisciplinary approach. Part of the learning task might be to work in collaboration with others. Confronted with such tasks, students start off with an analysis of the case to establish which (theoretical) questions should be answered in order to address the underlying problems and issues adequately. In the process of analysis, gathering information, constructing and testing possible solutions, they supposedly develop the competencies that are needed to deal with the problems and issues that arise in a professional academic practice. This approach to learning contrasts with the common practice which has students working individually, grasping knowledge from their teachers and textbooks using artificial problems. The use of complex, real life cases is, nevertheless, in line with current concepts on learning, competence development and the ‘whole task’ approach as well as emerging views on the preparation of students for the complexity of issues and questions of modern society and work. From experiences with case-based learning it seems clear that the type of case and its actual design (situation, problems, context, etc) determine to a large extent the activities of students (Jacobs et al., 2003,) and, hence, their learning. For many teachers designing realistic cases that provoke higher-order learning seems a daunting task; simply putting hands-on, case- based activities in place does not ensure that students study relevant theories in depth. Active participation can easily become an end in itself, regardless of the intellectual quality of the students’ work (Newmann et al, 1996; Wiggins, 1996).

Upload: others

Post on 19-Jun-2020

11 views

Category:

Documents


1 download

TRANSCRIPT

Page 1: The design of authentic tasks that promote higher-order ... · The design of authentic tasks that promote higher-order learning Hanne ten Berge, Stephan Ramaekers, Albert Pilot IVLOS

Page 1 of 23

The design of authentic tasks that promote higher-order learning

Hanne ten Berge, Stephan Ramaekers, Albert Pilot IVLOS Institute of Education,

Utrecht University, the Netherlands

Abstract This study focuses on the use of authentic cases to provoke higher-order learning. It is based on a review of recent

research literature and focuses in particular on the design features of these cases. First, we looked into the

characteristics of real life problems academics are confronted with in their professional practices and the way they

solve those problems. In the transformation of these to educational tasks, elements of authenticity get lost. Awareness

of the features that are crucial to authenticity of the case seems important if higher-order learning is to be achieved.

The literature not only provided indications for the design of authentic cases, we also came across examples of

negative effects of authentic cases and improper design. The results of our search show that the impact of authenticity

on learning outcomes is promising but it does not present conclusive results. Authenticity appears to be a much too

wide ranging formulation to be helpful in the design of case attributes. In the future we will focus more closely on the

coherence of the factors that play a part in higher-order learning.

Introduction Over the last decade, the use of authentic cases, problems and projects as a starting point for learning has won ground in higher education. Usually, these cases refer to situations that are complex, contain some hidden, ill-defined or open-ended problems and often require a multidisciplinary approach. Part of the learning task might be to work in collaboration with others. Confronted with such tasks, students start off with an analysis of the case to establish which (theoretical) questions should be answered in order to address the underlying problems and issues adequately. In the process of analysis, gathering information, constructing and testing possible solutions, they supposedly develop the competencies that are needed to deal with the problems and issues that arise in a professional academic practice. This approach to learning contrasts with the common practice which has students working individually, grasping knowledge from their teachers and textbooks using artificial problems. The use of complex, real life cases is, nevertheless, in line with current concepts on learning, competence development and the ‘whole task’ approach as well as emerging views on the preparation of students for the complexity of issues and questions of modern society and work. From experiences with case-based learning it seems clear that the type of case and its actual design (situation, problems, context, etc) determine to a large extent the activities of students (Jacobs et al., 2003,) and, hence, their learning. For many teachers designing realistic cases that provoke higher-order learning seems a daunting task; simply putting hands-on, case-based activities in place does not ensure that students study relevant theories in depth. Active participation can easily become an end in itself, regardless of the intellectual quality of the students’ work (Newmann et al, 1996; Wiggins, 1996).

Page 2: The design of authentic tasks that promote higher-order ... · The design of authentic tasks that promote higher-order learning Hanne ten Berge, Stephan Ramaekers, Albert Pilot IVLOS

Berge, H. ten, Ramaekers, S., & Pilot, A. (2004). The design of authentic tasks that promote higher order learning. Paper presented at the EARLI-SIG Higher Education/IKIT-conference, June 18-21, 2004.

Page 2 of 23

In our study we focus on the design of tasks and cases, related to various levels and domains in higher education. The purpose of the study is to review recent research literature concerning the design features of cases that evoke higher-order learning. Some of the leading questions in our search are: which case attributes and design features of the task are crucial for higher-order learning? How do they affect processes of disciplined inquiry, knowledge construction and problem solving? How do they relate to development of competences? Currently, the emphasis in our search is on the case attributes and design features; in the end, we intend to establish practical guidelines and design principles that will support teachers in their efforts to choose and design cases which represent professional practices of academics adequately and tasks that are effective for the students’ learning. Scope of this study Being able to deal with the problems and issues in the professional practices of academics requires, besides a thorough understanding of relevant topics, competences related to critical thinking, reasoning, decision making and problem solving. This often involves cooperation with others. Furthermore, most of the problems that people face in their professional and everyday lives are ill-structured. They do not necessarily conform to the content domains of studies and their solutions are neither easily predictable nor convergent (Jonassen, 2004). If the aim of education is preparation of students for their future work and functioning as academics, then it should include learning to deal with such complex, ill-structured and uncertain situations, representing the authentic problems and activities that occur in professional practices. Educational methods such as problem-based learning, project-based education and authentic learning share their explicit aims at fostering skills in professional reasoning and decision making, and cases are considered the driving force behind student learning. Various studies have demonstrated that students repeatedly involved in solving problems in a particular domain perform better in the transfer of concepts to similar situations (Norman & Schmidt, 1992). On the other hand, poorly designed tasks easily result in students performing a scavenger hunt for information and finding ‘right’ answers (e.g. Houlden, et al., 2001; Moust & Schmidt, 1998). In our own observations this problem becomes particularly clear in the reform of curricula when case-based methods are introduced. Some design features of tasks appear to be particularly influential on the activities and learning processes of students (Jacobs et al., 2003): - how authentic is the case? To which extent does the case resemble cases from

students’ future professional academic practices? To which extent is the case in touch with students’ experiences and daily lives?

- How complex and open is the task? Are there many issues and uncertainties involved in the problem? Are there extensive relationships among these issues? Is there a know, unique solution to the problem or are there various alternative solutions to be considered?

- How well-structured is the case? Are there useful concepts, theories and methods available to support finding solutions? Are there any hidden aspects of the problem?

- Is the case challenging? Do the students consider the problem genuine and worth finding a solution? Do they find the content of the problem interesting? Does the task demand any effort? Can the problem be solved within the available time?

Page 3: The design of authentic tasks that promote higher-order ... · The design of authentic tasks that promote higher-order learning Hanne ten Berge, Stephan Ramaekers, Albert Pilot IVLOS

Berge, H. ten, Ramaekers, S., & Pilot, A. (2004). The design of authentic tasks that promote higher order learning. Paper presented at the EARLI-SIG Higher Education/IKIT-conference, June 18-21, 2004.

Page 3 of 23

As some of these ‘critical’ case features coincide with the aims of case-based learning, we felt a need to get a better insight in them and conducted a search in recent research literature with the following two questions: Which design features determine the extent to which tasks and their accompanying cases promote higher-order learning? How do the features relate to beginners and advanced levels of competence? We defined higher-order learning on the views of Newmann et al. (1992, 1996) in terms of academic achievements that meet the following, complementary criteria: - the authentic construction of meaning and knowledge, rather than reproducing the

knowledge that others produced or meaning they have given to it; - relying on disciplined inquiry: a. using the prior knowledge that has been accumulated in

a field, b. striving to achieve an in-depth understanding of the problem and its’ context rather than the superficial awareness or familiarity with fragments of knowledge, and c. involving elaborated communication with peers and experts to exchange views, express conclusions and critically question their rationale and the methods used;

- having value, professionally and personally, that is beyond passing exams and being successful in their studies.

The educational settings in which cases and problems are used as learning tasks, are numerous. We narrowed the scope of our search to tasks: - derived directly from academic practices in type of activities, methods used, results (or

products) and the standards of work; - that have a substantial size, exceeding the workload of a single classroom meeting.

They take at least a day or as much as a bachelor thesis (between 8 and 400 hrs of study load);

- allowing students freedom of choice in the way they handle the case, although their choices, the methods used and the results should be clarified / justified afterwards;

- in which interaction with others is functional in the sense that the nature and extent of the task do not allow students to complete the task on their own;

- with tangible results that express the knowledge and skills gained in written or oral discourse, in the design or construction of objects and in performances for audiences.

Theoretical background and method The use of authentic tasks and cases has a longstanding tradition in domains of education such as Medicine (Norman & Schmidt, 1992), Engineering (Powel & Weenk, 2003), Law (Garner, 2000), Management (Roth, 1985; Wagner, 1991), Aviation (Hays & Singer 1989, Jensen, 1989), etc. In these domains, skills and cognitive processes related to problem solving, reasoning and decision making are considered critical in their professional practices and cases are used to support students in the development of their (domain specific) problem solving and reasoning skills. As they are considered to be fundamental human skills (judgement, choice, etc) it is not surprising that decision making, problem solving and reasoning have received considerable study and they are subject of many different theories on learning and education. They are closely related to many other aspects of cognition such as pattern recognition (recognise familiar problem elements), mental modelling and schema formation (remember similar problems), concept development and creativity / lateral thinking (developing solutions).

Page 4: The design of authentic tasks that promote higher-order ... · The design of authentic tasks that promote higher-order learning Hanne ten Berge, Stephan Ramaekers, Albert Pilot IVLOS

Berge, H. ten, Ramaekers, S., & Pilot, A. (2004). The design of authentic tasks that promote higher order learning. Paper presented at the EARLI-SIG Higher Education/IKIT-conference, June 18-21, 2004.

Page 4 of 23

Furthermore, the issues of metacognition and transfer (applying concepts and methods to other, related problems) are highly relevant to them. The literature used in this study was found by introducing a list of over 30 descriptors -in various combinations- in on-line retrieval systems like ERIC (Educational Resources Information Centre) and search systems of editors of relevant magazines (e.g. Kluwer, Taylor & Francis). We also solicited the proceedings of two conferences on education, in particular the sections on higher education, to find authors that have published on the issue of open-ended, ill-defined problems for higher-order learning in higher education (Onderwijs Research Dagen i.e. Educational Research Days 2001 and 2002, and the European Association for Research on Learning and Instruction 2003). The articles we found can be divided in two main categories. The first consists of articles that focus on the design and effects of educational methods like problem-based, case-based, task-based and discovery learning. The second focuses on the problem solving process, complex cognitive activity and multidisciplinary decision making. These articles concern both professional practices as well as educational practices. Although in most of these articles case attributes such as complexity, structuredness and authenticity where not the primary aim of the research, often they were referred to as one of the key design factors in these educational methods. We used the references in these articles to lead us to literature on these factors. On the basis of a first analysis of the literature, our conceptual model covering case attributes and their possible relationships to higher-order learning was modified. The modified model (figure 1) was used to establish on which relationships (shown as arrows) we needed to conduct a second search, focussing on empirical literature.

Figure 1: Conceptual model of the literature search

learning task

complexity

structure

authenticity

challenge

Level of performance

Instructiveness for higher-order learning

authentic case

Page 5: The design of authentic tasks that promote higher-order ... · The design of authentic tasks that promote higher-order learning Hanne ten Berge, Stephan Ramaekers, Albert Pilot IVLOS

Berge, H. ten, Ramaekers, S., & Pilot, A. (2004). The design of authentic tasks that promote higher order learning. Paper presented at the EARLI-SIG Higher Education/IKIT-conference, June 18-21, 2004.

Page 5 of 23

Our study draws to a large extent on the literature concerning authentic achievement (Newmann et al., 1992; 1996), complexity of expertise (Daley, 1999; 2001), problem-based learning (Norman & Schmidt, 1992) situated learning (Brown et al., 1989; Schoenfeld, 1998), authentic model-based tasks (Gobert et al., 2004), problem solving and design theory (Jonassen, 2000; 2004; Shin et al., 2003). Results - The professional practices of academics Before we go into aspects of the use of authentic cases in an educational setting, let us first take a closer look at the work and activities of academics in their professional practices. Obviously, these practices cover many different domains and types of work and range from clinical work of, for example, a dentist or the work of an industrial designer to the research conducted in a specialised institute, and so on. The figures 2 to 4 visualize on which element of the professional practice each section focuses. The nature of real life cases in professional academic practices “Now imagine the thousands of discrete challenges that a typical business must handle. All of these challenges are generated by the multitude of decisions managers make every day about the company’s activities. It is often difficult for (large) companies to make complexity cost/benefit trade-offs because their (decentralized) decision-making is split between (many) managers in different divisions or groups, business units, and departments. . . . . In the end, decisions are made which add complexity without creating offsetting customer or competitive benefits. It is the exponentially expanding interaction of all these strategic and operational challenges and decisions that constitutes complexity and leads to severe reductions in competitiveness.” (P.K. Jagersma in ‘Managing Business Complexity’, 2004)

The complexity of today’s society is characterized by an infinite, dynamic and changing mass of information, massive use of various sources of information and communication technology, a rapid changing labour market demanding a more flexible labour force that is directed towards a growing proportion of knowledge-intensive work in teams and lifelong learning. Products become more advanced and processes are subject to a growing amount of regulations e.g. on safety and quality. As a consequence, today’s society expects graduates not only to have a specific knowledge base but also to be able to apply this knowledge to solve complex problems in an efficient way (Dochy et al, 2003). According to FORMAT, a European system used to classify functions and

establish the level of education needed to fulfil such function, the work of academics is generally characterised by: a high degree of self-reliance (usually in specialised fields), a high level of responsibility (in the performance of tasks which are ‘critical’, that is, contain risks or bearing responsibility for the work of others) and regular transfer (little routine, many unique complex tasks and tailor-made solutions/adaptations for a specific context). Presumably, in many domains, people become expert not to function as technicians solving new variants of the classic problems but to open the field further (Perkins & Salomon, 1989). Such tasks demand a high level of problem solving, decision making, critical thinking and

Professional practice

problem solving

process

problem solver

case

Fig. 2: Cases in professional academic practices

Page 6: The design of authentic tasks that promote higher-order ... · The design of authentic tasks that promote higher-order learning Hanne ten Berge, Stephan Ramaekers, Albert Pilot IVLOS

Berge, H. ten, Ramaekers, S., & Pilot, A. (2004). The design of authentic tasks that promote higher order learning. Paper presented at the EARLI-SIG Higher Education/IKIT-conference, June 18-21, 2004.

Page 6 of 23

reasoning, no matter whether it concerns solving a management problem, designing a product or developing a new theory. It should be taken into account however, that the problem solving process can differ considerably within this group of tasks. Greeno et al. (1990) claim that design tasks differ in two ways from problem solving tasks: first, the problem space is open and it is impossible to predict which solutions a designer might come up with since design is an inherently creative activity. Second, the final state is a matter of judgement; the designer decides when the task is completed. Most authors agree that the kind of problems academics encounter in their professional practice are often ill-structured and complex (e.g. Jacobs et al., 2003). Their solutions are not predictable or convergent and their solution may require the integration of several content domains. Verstegen (2004) calls design tasks ill-structured and inherently iterative. Problems that are ill-structured possess elements that are unknown or not known with any degree of confidence. Furthermore, these problems possess multiple solutions, solution paths, or no solution at all. There is uncertainty about which concepts, rules, and principles are necessary to solve the problem and how they are organized. Another characterization of these problems is that the problem solver often has to make judgements and express personal opinions or beliefs about the problem which makes the problem solving activities of these kind of problems "a uniquely interpersonal activity" (Jonassen, 2000). Another aspect of structure concerns the amount of information (Taconis et al. (2001). In practice the problem solver is most often confronted with a large amount of information, while it is most often insufficient for the solution. In real life the problem statement hardly ever contains all the information needed for the solution. It contains more information than necessary so that the solver has to decide which information he needs, while at the same time not all necessary information is available in the problem statement. Ill-structured problems can become well-structured problems with practice (Jonassen, 2000). Complexity is concerned with how many, how clearly, and how reliably components are represented implicitly or explicitly in the problem. Furthermore, complexity is a function of the number of subproblems that have to be solved to reach the final answer, and the number of formulas, laws and principles from which the solver has to make a choice when planning the solution. Complex problems, such as most design problems, are open problems. This significates that the problem can be solved with a variety of solutions and does not have one, unique answer (Taconis et al., 2001). Real world complex tasks require constant redefinition of goals and priorities because, frequently, new information is presented to the decision maker. Conditions of naturalistic decision making are therefore characterized as dynamic and continually changing (Randel et al., 1996). Decision making in complex, dynamic situations is often bound to a time constraint. Especially in stressful situations in which the decision has to be made right then and there the time constraint is an important element. Problem solving in professional academic practices Problem solving requires different types of knowledge: situational knowledge needed to recognize and classify the problem and to select declarative knowledge needed for the solution, declarative knowledge of facts, principles, and laws of the discipline for drawing conclusions about the situation and planning and execution of the solution, strategic knowledge of approach and methods, and procedural knowledge for applying declarative

Page 7: The design of authentic tasks that promote higher-order ... · The design of authentic tasks that promote higher-order learning Hanne ten Berge, Stephan Ramaekers, Albert Pilot IVLOS

Berge, H. ten, Ramaekers, S., & Pilot, A. (2004). The design of authentic tasks that promote higher order learning. Paper presented at the EARLI-SIG Higher Education/IKIT-conference, June 18-21, 2004.

Page 7 of 23

Professional practice

problem solving

process

problem solver

case

Fig. 3: The problem solving process in professional academic practice

knowledge in executing the plan. In practice problem solvers use combinations of these –frequently all- types of knowledge. Situational knowledge functions as a recognition device in this matter. “Only when these are linked to elements of declarative and procedural knowledge relevant for the solution can recognition lead to classifying the problem into some type already encountered and to planning and executing the solution. This makes it clear that not only the content of a problem schema but also its structure is important for the success of the problem solver. The details of the structure and the way the schemata are related to each other depend on the intrinsic properties of domain knowledge. Physics for instance, is a highly structured domain with a hierarchical structure” (Taconis et al., 2001).

The cognitive skills needed for problem solving can be denoted as general, some as domain specific and some fall between general and domain specific. Among the general skills are analyzing, planning and the constructing and revising mental representations of the problem. López de Mántaras and Plaza (1997) argue that the problem solving process consists of two aspects. The first is arguing whether or not a new situation should be treated like previous ones based on similarities and differences among them. The second aspect is building a solution to a new case based on the adaptation of solutions to past cases. Since in practice many problems have components of both types case-based reasoning is a cyclic process comprising retrieve, reuse, revise and retain.

Research that examines the nature of everyday reasoning, including judgement and decision making, has been an asset to the design of critical thinking programs for the past three decades (Osana et al., 2003). Central to the research in social cognition has been the demonstration of human shortcomings in reasoning within ill-structured social contexts. The research of Tversky & Kahneman (1986), for example, has repeatedly shown that individuals routinely ignore probabilistic data in favour of information that is representative or particularly salient. Others have described similar erroneous heuristics used when thinking about complex problems. Novice versus skilled performance One of the approaches to clarify the development of skill in problem solving is based on comparing the differences between novices and experts. Skilled performers show a smooth

and fast performance of the task, usually without hesitation or errors, and with a continual check on any mistakes that might occur. The speed, fluency and perfection of the cognitive activities of experts contrast with the performance of novices which tend to be slow, hesitating, with errors and mistakes (Taconis et al., 2001). From novice to expert, there are two dimensions along which the competencies of the problem solver develop: 1) from unstructured to structured and 2) from step by step to automated performance. The skills of the expert are highly structured, whereas the skills of beginners are unstructured. Novices try often different paths to

Professional practice

problem solving

process

problem solver

case

Fig. 4: The problem solver in a professional academic practice

Page 8: The design of authentic tasks that promote higher-order ... · The design of authentic tasks that promote higher-order learning Hanne ten Berge, Stephan Ramaekers, Albert Pilot IVLOS

Berge, H. ten, Ramaekers, S., & Pilot, A. (2004). The design of authentic tasks that promote higher order learning. Paper presented at the EARLI-SIG Higher Education/IKIT-conference, June 18-21, 2004.

Page 8 of 23

find pieces of a solution and usually, they don’t use all the information given in the problem statement. “An important measure of personal mastery of cognitive activities is the measure of automation with which they are performed. An expert for instance checks his solution while carrying out the calculation. A beginner works step by step. He checks his solution after he has finished his calculations” (Taconis et al., 2001). In design problems, the approach is not systematic but rather iterative (Verstegen, 2004). The designer has to produce some order from disorder and creativity plays an important role in the process. The task handling contains thought as well as action, and conceptual and procedural knowledge (Hill, 1998). Furthermore, novice solvers usually attempt to use so-called weak (domain-independent) strategies, such as means-ends analysis (Reimann & Schult, 1996). Research on mathematics word problems showed that such strategies impede problem schema development because solvers concentrate merely on eliminating discrepancies between the current state and goal state (Jonassen, 2003). Expert problem solvers, on the other hand, use strong, domain-specific strategies, which are specific to problem types, making them more efficient (Shin et al., 2003). Furthermore, novices often solve problems by focussing first on the unknown and then seeking rules that bridge back from the unknown towards the givens (backward reasoning), whereas experts reason in opposite direction (Perkins & Salomon, 1989). Comparing expert instructional designers with novices, Rowland (1991, 1992) found that experts spend more time in the first phase (problem understanding) than novices do, looking for additional information of trying to infer it. Although experts also consider possible solutions early on, they delay to commit to one solution early on. Experts consider a wide range of factors at the same time, beginners fewer and one after another. Rowland suggests that differences in the final solutions might be related to their individual interpretation of the problem rather than just generating different solutions. Most of the differences between novices and experts are by various authors attributed to differences in the way experts perceive the problem and their notices of meaningful patterns of information that are not noticed by novices, differences in content knowledge, the way their knowledge is organised and retrieved in the context of applicability. The earliest studies of expertise demonstrated that the same information is perceived and understood differently, depending on the knowledge someone brings to the situation. Experts recognize features and patterns that are not noticed by novices and their recall ability has been explaned in terms of how they 'chunk' various elements that are related by an underlying function or strategy. An example in the field of history is provided by a studie among history experts and a group of gifted, high-achieving students (Wineburg, 1991). On a factual test about the American Revolution the historians, of whom many had specialties in other areas, were outscored by several of the students. Next, the study compared how the historians and students made sense of historical documents and here, the results revealed distinct differences; the historians excelled in the elaborateness of understandings they developed in their ability to pose alternative explanations for events. The students were largely unacquainted with modes of inquiry with real historical thinking. They had no systematic way of making sense of contradictory claims.

Page 9: The design of authentic tasks that promote higher-order ... · The design of authentic tasks that promote higher-order learning Hanne ten Berge, Stephan Ramaekers, Albert Pilot IVLOS

Berge, H. ten, Ramaekers, S., & Pilot, A. (2004). The design of authentic tasks that promote higher order learning. Paper presented at the EARLI-SIG Higher Education/IKIT-conference, June 18-21, 2004.

Page 9 of 23

Nevertheless, this does not mean that experts always perform error-free; they also apply in case of unusual problems many general strategies, resort to analogies they understand better, refer to intuitive mental models, etc (Perkins & Salomon, 1989). As Greer (1997) observed, for example, mathematicians and theoretical scientists ignore aspects of reality all the time. The crucial difference, of course, is that they do it knowingly (and with regard to the context and goals, the likely level of inaccuracy, and the technical knowledge and power available). Results – translating professional practices and cases into learning tasks Authenticity in the context of learning Although the first references to ‘authentic learning’ date back as early as the beginning of the 20th century, Newmann et al. (1992) were among the first to use the term ‘authentic’ formally in the context of learning. Since then several concepts of authenticity in education have evolved. The first, dominant concept links authenticity to competence, academic achievement and performance. Performance concerns the actual execution of a task or process and can only be observed and assessed through actual demonstration, being a productive activity (Wiggins, 1993). It places an emphasis on the integration of all elements of knowledge and skill, and requires attention to the whole task and not just separate pieces of it (van Merriënboer, 1997). As far as it captures the notion of contextualisation, it means making the task apparently ‘real’ rather than artificial (Cumming & Maxwell, 1999). The rationale for the ‘whole task’ approach is that performance on the whole task may differ from performance on component skills and a transfer from component skills to the whole task should not be assumed (Carlson et al., 1990). The second concept of authenticity emerges from ‘situated’ theories of learning which claim that learning occurs best or perhaps only, within context (Brown et al., 1989). According to these theories learning and performance are so closely related to context, that the actual schema of the domain knowledge and processes are different for different contexts. Task performance demonstrated in one context may not be indicative of competence in another context. In this concept, authenticity is relative (Cronin, 1993). The use of human patient simulators, for example, to provide practical training in performing a physical examination is considered to be more authentic than practicing on their fellow students, but less authentic than examining a real patient during a clinical internship. According to Anderson et al. (1996) there is a continuum of theories of ’situatedness'. They range from theories that posit possibilities of transfer of learning across various contexts to those that argue complete contextual dependence. Complete contextual dependence would mean that a task could only be authentic if it occurred within the specific context and generalisation would not be possible. Although authenticity is still regarded as a central criterion for the use of cases and design of tasks, its concepts’ original frame of reference has gradually extended to other aspects of authenticity (MacDonald et al., 2000), including the specific role students and teachers fulfil, the sources of materials used and the physical environment. Despite all efforts to create circumstances which resemble professional practices as close as possible, cases and tasks in an educational setting remain to some extent being artificial and

Page 10: The design of authentic tasks that promote higher-order ... · The design of authentic tasks that promote higher-order learning Hanne ten Berge, Stephan Ramaekers, Albert Pilot IVLOS

Berge, H. ten, Ramaekers, S., & Pilot, A. (2004). The design of authentic tasks that promote higher order learning. Paper presented at the EARLI-SIG Higher Education/IKIT-conference, June 18-21, 2004.

Page 10 of 23

not ‘real' (Cumming & Maxwell, 1999). Even if they are conducted in work contexts, performing the task usually does not have real consequences and students carry limited responsibility for the outcomes. In a emergency medicine simulation for example, inadequate performance does not lead to death, or in a simulated capital investment task, inadequate performance does not lead to bankruptcy. Moreover, these activities are always undertaken under constraints which make the task artificial. Usually, for educational purposes or because of organisational constraints the task is framed by the deliberate construction of limits to the task. Consequently, the task is not as open-ended or as unbounded as ‘real world' tasks where extra complications and unforeseen contingencies may arise. Aside from internships, the kind of simulations used in aviation are probably the nearest we can get in education to resemble the experiences, performance and context of professional practices. ‘Fidelity’ refers to the extent to which a simulation matches the characteristic of the real system and/or equipment it is designed to model. With regards to fidelity a distinction is made between physical, psychological and functional fidelity. Functional fidelity is defined as the degree to which a simulation imitates the information and stimulus-response options that are present in the real world. From an educational point of view, the minimum level of fidelity should be determined by the training goals that are covered by the simulation. At least functional fidelity should be guaranteed: the stimulus-response relations should be the same as in the real situation (Verstegen, 2004). Impact of authentic context on student performance Across various domains, empirical studies have been conducted to establish the impact of authentic tasks and cases on student performance, some of which reveal which design factors influence the effectiveness in particular. Many of them report positive effects ranging from an increased motivation of students to enhanced learning of basic (science) concepts, improvement of problem solving skills or ‘transfer’ to new situations (e.g. Schnitzer, 1993; Newmann et al., 1996; Ferretti et al., 1996; van Berkel & Schmidt, 2000; McRobbie et al., 2001, Norman, 2002; Dori, 2003; Yoshioka et al., 2003). Others reveal negative effects for which explanations were sought. Most researchers recognise some dependence of their findings on the way authenticity was operationalised and on the presence or lack of additional support or guidance. Here we focus on the design of the task and the case itself.

Some of the explanations attribute negative effects of authenticity on the students’ performance to the way students perceive the task. De Bock et al (2003) studied the influence of authentic contexts and of self-made graphical representations on students’ tendency to improperly apply the linear model to represent and solve non-proportional word problems about area and volume. None of the experimental manipulations yielded the expected results. On the contrary, both factors yielded a negative effect on students’ performance. One of the possible explanations refers to Salomon’s (1981) account of the mediating effects of people’s perception of media characteristics on their willingness to invest mental effort during learning and problem solving. Video is

perceived as a less difficult medium than written materials and therefore students are inclined to invest less mental effort in working with information transmitted by this medium as

Educational practice

problem solving

process

student

task case

Fig. 5: The case attributes within the learning task

Page 11: The design of authentic tasks that promote higher-order ... · The design of authentic tasks that promote higher-order learning Hanne ten Berge, Stephan Ramaekers, Albert Pilot IVLOS

Berge, H. ten, Ramaekers, S., & Pilot, A. (2004). The design of authentic tasks that promote higher order learning. Paper presented at the EARLI-SIG Higher Education/IKIT-conference, June 18-21, 2004.

Page 11 of 23

compared to media that are perceived difficult. Other studies (e.g. Boaler, 1994; Black, 1991; Cooper, 1994) show similarly that students can be distracted by an ‘everyday context’ and underachieve as compared to problems presented in an abstract way (mathematics). And also the lack of an ‘experimental contract’ (Greer, 1997) with norms, rules and expectations about completion of a task, might affect the students’ perception: the less time they are given for completing a task, the more likely they perceive the task as easy. A second source for disappointing outcomes is related to the mixture of aims of the task. At least two types of aims and expectations are embedded within authentic cases used for educational purposes. The first type relates to understanding the underlying core concepts, development of skills and construction of new insights; the second to the specific task and context in which these are to be enacted or displayed. Cumming & Maxwell (1999) refer to them as first and second-order expectations. The first are linked to content and level of performance, the latter to form of performance or the product (Messick, 1994). Without carefully chosen assessment criteria, students easily focus on the expected outcome (report, prototype, performance for an audience, etc) rather than on an in-depth understanding of content (e.g. Wolf, 1995), and second-order expectations overwhelm the first-order. Cumming & Maxwell (1999) point at what they call ‘camouflage’, that is, trying to ‘dress up’ traditional form of tasks to appear authentic, just by introducing some real world elements. As their inclusion does not change the context of the problem or provide a degree of situatedness that facilitates a solution, such camouflage may only lead to confusion. Camouflage is most likely to occur if the task design starts with content which students are supposed to learn instead of using an authentic case from professional academic practice as a starting point. To many teachers, the concept of authentic tasks and cases appeared to be appealing (Wallace, 2001). The power of the concept has probably been in the simplicity of its central idea: students’ experiences in their studies should closely resemble the experiences they encounter in real life. Research on the impact of authenticity on learning outcomes is promising but, nonetheless, does not present conclusive results. Some seek further enhancements in extending the frame of reference to other aspects of authenticity; others focus on fostering and scaffolding the kind of thinking and problem solving skills that are important in out-of-school settings, whether or not the activities themselves mirror what practitioners do (e.g. Messick, 1994). The concept of authenticity might be still too general and not far enough developed to serve as a guide in the design of tasks. To understand how the design features of authentic tasks relate to higher-order learning, we need an approach which is more analytic than the current wide ranging formulation of authenticity and a more detailed insight in the effects of various case attributes. Task design features and case attributes Across the studies we found various criteria have been suggested for authentic tasks to enhance higher-order learning (e.g. Gijselaers, 1996; van Berkel & Schmidt, 2000; Abel, 2003; Jonassen, 2004). Here, we focus on those which were repeatedly mentioned and for which we found empirical support.

Page 12: The design of authentic tasks that promote higher-order ... · The design of authentic tasks that promote higher-order learning Hanne ten Berge, Stephan Ramaekers, Albert Pilot IVLOS

Berge, H. ten, Ramaekers, S., & Pilot, A. (2004). The design of authentic tasks that promote higher order learning. Paper presented at the EARLI-SIG Higher Education/IKIT-conference, June 18-21, 2004.

Page 12 of 23

Complexity Reducing the complexity of real life situations and problems to a level students can handle is probably one of the key issues of education. Even in authentic tasks the question how to match complexity to the level students can cope with, seems important; for example, experiences with training simulations show that a simplified, lower fidelity sometimes better serves beginning trainees (Verstegen, 2004). Jonassen (2004) uses the term ‘complexity’ specifically for the number of issues, functions or variables involved in a problem, the degree of connectivity among those variables and the stability among those properties over time. He distinguishes complexity from structuredness and dynamicity (i.e. changing variables over time), but acknowledges some overlap. In general, ill-structured problems tend to be more complex and more dynamic but there are examples of complex problems which are well-structured, etc. The suggestion of problem complexity and structuredness as separate entities seems to a certain extent intuitively recognizable by students. In their study, aiming at validation of a short questionnaire to be used to establish the degree of complexity and structuredness of tasks in problem-based learning, Jacobs et al. (2003) found that students were able to distinguish problems that were too simple and those that were too well-structured. Once problems were too complex or too ill-structured, students were not able to differentiate between the two.

Making sense of a complex system requires that we construct a network of concepts and principles, which represents the key phenomena and their (often dynamic) interrelationships (Hmelo-Silver & Pfeffer, 2004). On top of this, problem solving involves predicting the changes (effects) that take place as a result of certain interventions (solutions). A widely accepted explanation about why solving complex problems is more difficult than simple problems is that making sense of large numbers of variables and their (dynamic) relationships involves more cognitive operations than with small numbers and stable circumstances. In accordance with the cognitive load theory, working memory requirements increase at least proportionally (Jonassen, 2000). In other words: accommodating multiple factors during problem finding, analysis or structuring, and the generation of possible solutions places a heavy burden on working memory. The more complex a problem, the more difficult it will be to actively process all the components of the problem.

A high level of complexity can affect the students’ performance in various ways; if problems are too complex students experience difficulties in taking into account all the relevant components of the problem, in activating their prior knowledge, in actively engaging in discussions about the problem and in formulating useful learning issues (Jacobs, 2003). Furthermore, they have difficulties in generating alternative solutions or options to choose from (Osana et al, 2003), display a tendency to seek for information which confirms their initial understandings and hypothesis (conformation bias), design inconclusive ‘experiments’ to test their concepts and possible solutions, and they have difficulties in systematically planning and monitoring the process of discovery and construction (De Jong & van Joolingen, 1998).

Structuredness For long, it has been assumed that learning to solve well-structured problems transfers positively to learning to solve ill-structured problems (Jonassen, 2000). Research has shown this to be untrue, as solving ill-structured problems requires additional cognitive skills to solving well-structured problems (e.g. Gick & Holyoak, 1987; Shin et al, 2003). Domain

Page 13: The design of authentic tasks that promote higher-order ... · The design of authentic tasks that promote higher-order learning Hanne ten Berge, Stephan Ramaekers, Albert Pilot IVLOS

Berge, H. ten, Ramaekers, S., & Pilot, A. (2004). The design of authentic tasks that promote higher order learning. Paper presented at the EARLI-SIG Higher Education/IKIT-conference, June 18-21, 2004.

Page 13 of 23

knowledge and justification skills turned out to be significant predictors of well-structured problem solving scores, whereas ill-structured problem solving scores were significantly predicted by domain knowledge, justification skills, science attitudes, and regulation of cognition. One of the most important differences might be that solving ill-structured problems requires a high level of skill in justifying solutions, as many ill-structured problems are too complex to actually try out (Shin et al, 2003). The process of justification requires the solvers to identify the various perspectives that affect the problem situation, provide supporting arguments and evidence about opposing perspectives, evaluate information, and develop and argue for a reasonable solution. Various studies confirm that performance in solving ill-defined tasks demanded engaging a different set of epistemic beliefs, illustrate how preconceptions change in the course of learning and support the assumption that mental models are constructed in dependence on the demands of those learning situations (Seel, 2001; Gobert & Pallant, 2004). The results of their tests show that students with more sophisticated epistemologies of models were better able to further their content understanding as compared to students with less sophisticated epistemologies of models. Kyoo-Lak et al. (2003) showed that communication patterns in teams differed when solving well-structured and ill-structured problems, for example groups that solved ill-structured problems produced more extensive arguments in support of their solutions. Similarly Willoughby et al. (2003) found in their study that not only the familiarity of the materials to the students, but also how the information was organised affected the extent and effectiveness of elaboration (answering the ‘why’-questions). Students who were presented with the information in a random order achieved the largest memory scores. Their explanation for this phenomenon is that when students have to reconstruct as well as encode the information, these added task demands provide an added benefit for learning. Challenging Education should give students the opportunity to do develop their potential to the full and challenge them to go beyond the limits of what they already capable of. For various reasons, authentic tasks seem more challenging than conventional tasks: Realistic: Being involved in a realistic problem instead of just having to learn ‘dry’ theory has a strong motivating effect on many students. For them to perceive a task to be realistic or authentic, the problem should be grounded in their experiences. If a problem is too theoretical and out of touch with students’ daily lives and experiences, they will not easily be engaged by the problem. Even if a problem is not based in students’ current experiences, it may be perceived authentic if it relates to students’ future plans and expected careers (Weiss, 2003). De Bock et al (2003) conclude in their study: as long as they are meaningful, familiar and appealing to the students, realistic contexts do not necessarily have to refer to aspects of the ‘real’ social or physical world. It is not the amount of realism in the literal sense that is crucial for considering a context as realistic or authentic, but rather the extent to which it succeeds in getting students involved in the problem and engage them in situational meaningful thinking and interaction. Meaningful: From motivational research in education it has become clear that learners of all ages are more motivated when they can see the usefulness of what they are learning and when they can use that information to do something that has an impact on

Page 14: The design of authentic tasks that promote higher-order ... · The design of authentic tasks that promote higher-order learning Hanne ten Berge, Stephan Ramaekers, Albert Pilot IVLOS

Berge, H. ten, Ramaekers, S., & Pilot, A. (2004). The design of authentic tasks that promote higher order learning. Paper presented at the EARLI-SIG Higher Education/IKIT-conference, June 18-21, 2004.

Page 14 of 23

others, especially their local community. Tasks that represent activities students are likely to perform in their later academic practices often have a clear personal and practical usefulness. Moreover, they might not only be motivated because the problem they work on is real but also because they are addressed as professionals and their work (solutions, products) will actually be used in practice (e.g. McRobbie et al., 2001). If the task is not of a kind that students would expect to undertake in their life outside school, either now or later, it is likely to be seen by students as being ‘contrived' (Cumming & Maxwell, 1999). Open: Some students consider authentic problems particularly challenging because they often do not have just one best solution or even have no known solution at all. Instead of looking for the correct solution that was previously established by others, students may perceive some autonomy, as well as a need to use all their knowledge, curiosity and creativity to overcome the uncertainties due to variable levels, hidden information or randomness. Autonomy, challenge, curiosity and creativity can promote intrinsic motivation. Dilemma-focussed cases naturally pull participants into an inquiry mode by revealing some inherent pitfalls and ambiguities in learning science concepts (Barnett-Clarke, 2001). On the other side, a limitation to dilemma-focussed cases is that in case discussion participants might get frustrated that there is not one ‘right’ way of answer. They may loose confidence or feel lost because every alternative has its own drawbacks (Barnett-Clarke, 2001). Demanding: In the well-developed area of research on achievement motivation, the ‘motivation to achieve’ is said to be a function of the individual's desire for success, the expectancy of success, and the incentives provided (Atkinson & Raynor, 1974). Well-designed tasks provide opportunities to succeed. However, tasks with a proper level of difficulty are neither too easy (become boring), nor too difficult (become frustrating); student consider a problem challenging if they expect to be up to the task and on the other hand the task is not too simple (compare: Vygotski's zone of near development). In other words, the problem must match the students' prior knowledge (Cartier & Steward, 2000; Weiss, 2003; Taconis et al., 2001; Norman & Schmidt, 1992; Willoughby et al., 2003, Jonassen, 2000). Such careful match with students’ prior knowledge is also necessary to stimulate students to spent sufficient time on their task. Time-on-task positively influences the extent of what is learned from a task and seems a strong indicator for the level of performance to be achieved (Norman & Schmidt, 2000). On the other hand, it is important to be realistic about the amount of time it takes to learn complex subject matter. Time constraints as well as the pressure to perform for a real customer can arouse anxiety. Anxiety can be an inhibiting factor in learning and has received considerable attention. It is closely related to arousal, attention and motivation as well as the topic of emotions (Mandler, 1984). Anxiety has been shown to impair performance in a wide range of cognitive functions including attention, memory, concept formation and problem solving. As the stress in a situation mounts, performance deteriorates immediately (Child, 2004). Anxiety and task difficulty interact; anxiety results in poorer performance in complex tasks but may improve performance on very simple tasks. This could be explained by Hull's drive reduction theory in so far as arousal increases the strength of responding but competing responses are activated in complex tasks.

Page 15: The design of authentic tasks that promote higher-order ... · The design of authentic tasks that promote higher-order learning Hanne ten Berge, Stephan Ramaekers, Albert Pilot IVLOS

Berge, H. ten, Ramaekers, S., & Pilot, A. (2004). The design of authentic tasks that promote higher order learning. Paper presented at the EARLI-SIG Higher Education/IKIT-conference, June 18-21, 2004.

Page 15 of 23

Familiarity: The problem solvers’ familiarity with the problem type is perhaps the strongest predictor of problem-solving ability. Experienced problem solvers have better developed problem schemes, which can be employed more automatically (Sweller, 1990). Although familiarity with a type of problem facilitates problem solving, that skill seldom transfers to other kinds of problems or even to the same kind of problem represented in another way (Gick & Holyoak, 1980, 1983). Routine problems are obviously familiar to learners and consequently more transferable (at least within a task environment). Therefore routine problems appear more well-structured to the experienced solver. Transfer of non-routine problems (those not familiar to the problem solver) require far transfer, which is more effortful and conscious whereas routine problems rely more on near transfer, which involves less conscious attention (Perkins & Salomon, 1989). The amount of routine skills available to the problem solver as well as the amount of interpretation needed of new elements of the situation presented and the process involved may make the same problem a challenge of interpretation and the practice of new skills to the one and a routine task to the other. Problem solving strategies of students To which extent do tasks, based on authentic cases, foster higher-order learning, i.e. engage students in the construction of meaning and knowledge, in disciplined inquiry, and having value beyond their studies? In the previous section we approached the question from the perspective of task design features; in this section we will focus on the (mental) activities of students when they are confronted with authentic tasks. Over the past 40-50 years, a considerable part of the psychological and educational research has concentrated on the factors that underlie the cognitive skills and processes of problem solving in various domains. One of the main controversies concerns the issue whether effective problem solving and sound decision making depend primarily on a deep understanding of a specific domain or on general strategies and reflective awareness (Perkins & Salomon, 1989). Originally, it was widely assumed that effective problem solving and other intellectual skills reflected general strategies which could be applied irrespective of the domain, as long as the necessary specific information was available. Based on this premise, the educational approach to problem solving assumed that learning to solve well-structured problems would result in the ability to solve ill-structured problems, and methods such as problem-based or case-based learning would be more successful in teaching problem inquiry and solving skills. In the light of subsequent research, both are considered nowadays to be unlikely. In most domains, the research on problem solving supports the notion that problem solving is to a large extent domain-specific and there is little evidence of generalisability or improvement with education focussing on general heuristics (Norman & Schmidt, 1992). The skills and strategies used to solve problems not only vary on complexity and structuredness, but also on the type of problem and its specific content (Jonassen, 2004). Perkins & Salomon (1989) argue that problem solving skills are neither exclusively general nor contextualized; there are some general cognitive skills but they always function in contextualized ways. They refer to research showing that when general principles of reasoning are taught together with self-monitoring practices and applications in varied contexts, transfer often is obtained.

Page 16: The design of authentic tasks that promote higher-order ... · The design of authentic tasks that promote higher-order learning Hanne ten Berge, Stephan Ramaekers, Albert Pilot IVLOS

Berge, H. ten, Ramaekers, S., & Pilot, A. (2004). The design of authentic tasks that promote higher order learning. Paper presented at the EARLI-SIG Higher Education/IKIT-conference, June 18-21, 2004.

Page 16 of 23

Problem finding, analysis and conceptualisation Depending on the type of problem and the way it is precisely formulated, students start off with a first estimation whether there is a problem, what the (main) problem is and whether the problem, issue or situation is appealing to them. Particularly in ill-structured, complex situations the problem may not appear directly or be hidden. There might be not one distinctive problem, but a complex of several interrelated problems. Furthermore, a distinction should be made between the problem to be solved (first-order expectations) and the task (second-order expectations). To avoid confusion between the description of the situation and how the situation should be interpreted, the first step in the systematic approach used in problem-based learning is to clarify all the terminology which is unknown (Moust & Schmidt, 1998). Theoretically, after determining the existence of a problem the students are supposed to construct a (mental) representation of the problem that contains all of the possible states of the problem based on the possible causes and constraints. When constructing this representation, they attempt to locate and select critical information from memory that fits the context in the course of constructing the problem space (Shin, 2003; Taconis et al., 2001). Instead of constructing such a conceptual model of the problem, students confronted with an unfamiliar problem often resort to analogies with problems they previously dealt with, refer to intuitive, relatively simple models on how the system probably would behave or take a procedural approach similar to known procedures how to solve the problem (e.g. Jonassen, 2003). In mathematics, for example, a pervasive problem solving strategy students demonstrate in so-called story problems contains three steps: search for key word, select the algorithm (formula) based on the key words and apply the algorithm. Research shows that this direct translation strategy results in a lack of conceptual understanding and the inability to transfer any problem-solving skills that are developed (Jonassen 2003). Reimann & Schult (1996) consider the use of analogical reasoning by students when solving problems hardly surprising and experts confronted with a problem they are unfamiliar with, seem to do the same (Perkins & Salomon, 1989; Reimann & Schult, 1996; Deng, 1996). Various methods have evolved to compensate for the complexity of tasks and to support students in the process of conceptualisation:

Case-based reasoning: according to Deng (1996) case-based reasoning is the most preferred method for problem solving and decision making in complex and dynamically changing situations. Decision makers usually utilize previous cases to help evaluate and justify decisions. Case-based reasoning solves problems by relating previously solved problems or experiences to a current, unsolved problem in a way that facilitates the search for an acceptable solution. The problem solving process is based on four steps Burkhard (2001): Retrieve cases from earlier experiences (‘case base’) which might be useful for the problem at hand, Reuse the adapted solution(s) of the useful case(s) to the new problem, Revise the suggestions for reuse according to the evaluation of the new experience, and finally Retain the revised new experience (as a new case) to the case base. Aleven (2003) reveals one of the major constraints of the method: “more often than not, when comparing a problem and a past case, one sees similarities as well as differences. The significance of similarities and differences depends on context and should be interpreted in accordance with the specific domain. Nevertheless, there is no generally-accepted, detailed method for weighing the similarities and differences”.

Page 17: The design of authentic tasks that promote higher-order ... · The design of authentic tasks that promote higher-order learning Hanne ten Berge, Stephan Ramaekers, Albert Pilot IVLOS

Berge, H. ten, Ramaekers, S., & Pilot, A. (2004). The design of authentic tasks that promote higher order learning. Paper presented at the EARLI-SIG Higher Education/IKIT-conference, June 18-21, 2004.

Page 17 of 23

Explicit use of concept maps: Parkes et al (2000) studied the effects of the use of concept maps on variance in task performance. They concluded from the scores of the groups that those who used concept mapping demonstrated a higher generalisability than those who did not use a concept map. Similarly Gobert & Pallant (2004) engaged students in tasks which required them to explicitly construct their own models for deep content learning and provided some guidance to construct and revise models. Their result show that those who held more sophisticated epistemologies of science were better able to transfer what they had learned in order to answer more difficult conceptual items involving in the causal mechanisms. Even simple memory aids can be beneficial to support students conceptualising explicitly. In a study by Cary & Carlson (1999) students, using paper and pencil as a memory aid, often developed routines that corresponded to the conceptual structure of the task.

Model progression. The basic idea behind model progression is that presenting the learner with the full complexity of the simulation at once may be too overwhelming. In model progression the model is introduced gradually. De Jong & Van Joolingen (1998) found in their review of the literature several studies on the effects of structuring progression: providing students with a sequence of experimentation steps and with more detailed directions in each of these steps, with forms that had separate cells for writing down findings in separate steps, or with task assignments for each step were effective in helping students to distinguish between steps and made them perform better on a knowledge test. Furthermore, in complex simulations increasing the level of complexity of the interface was beneficial for initial learning and for transfer and students learning with a simulation that presented an increasing control over variables, scored significantly higher at a test measuring application of rules, than those who could exercise control in its full complexity from the start. Timing of information: Complex cognitive tasks involve several constituent skills. For the training of multi-dimensional, complex cognitive skills, Janssen-Noordman & van Merriënboer (2002) consider learning tasks as the backbone of a training program. Sunch program is based on the premise that complex cognitive skills can only be learned by practicing performance of whole-tasks. In working on the learning tasks students should be coached during practice and should be supplied with information they need at a particular moment. Students work on a carefully designed sequence of case studies, that develops from simple to complex, with deminishing guidance by the teacher. The cases consist of whole-tasks and in working on these tasks the student is supported by part-task practice. Both part-tasks practice and supportive information are supplied 'just-in-time'. Finding and testing solutions There seems little disagreement about the educational approach to the process of choosing a solution to a problem or decision making once students are aware of the problem and its nature is identified (Osana et al, 2003). The normative model which underlies this ‘optimal’ process entails linear progression through: a. listing all possible solutions (alternatives), b. evaluating each of the solutions (advantages/disadvantages; promises/dangers, etc), c. choosing the solution with the highest possible ‘value’, d. devising and implementing an action plan based on the aforementioned evaluation, and finally e. evaluating the consequences of the chosen action.

Page 18: The design of authentic tasks that promote higher-order ... · The design of authentic tasks that promote higher-order learning Hanne ten Berge, Stephan Ramaekers, Albert Pilot IVLOS

Berge, H. ten, Ramaekers, S., & Pilot, A. (2004). The design of authentic tasks that promote higher order learning. Paper presented at the EARLI-SIG Higher Education/IKIT-conference, June 18-21, 2004.

Page 18 of 23

In a study about the nature of decision making in ill-structured problems in social sciences, Osana et al. (2003) however found that the process students actually engage in was to a large extent influenced by their efforts to generate hypothesis (construct meaning) about why and how the dilemma arose. To make up for the lack of information and the uncertainties of the problem the students used strategies such as narrative interpretation, giving values, using assumptions, attributions and heuristic use of rules and information, concurrently. In most instances these were largely influenced by the values, beliefs and knowledge the students held about the given situation. Once the problem ‘made sense’ to them, they did not generate various alternative solutions or options to choose from but were satisfied to justify their solution based on the narratives they had constructed. Taken together, the data in the study provide evidence that normative models of decision making are incomplete and do not accurately reflect the types of processes actually used by adolescents. Regulating the solving process Some of the afore-mentioned alternatives to support students in the process of conceptualisation intervene in the regulation of the problem solving process (model progression, timing of information). Other ways to support process regulation have been suggested: Davies (2003) reported on two experiments, which consider planning behaviour in the context of a well-structured, relatively simple problem. The question is to what extent planning a solution to a problem takes place before attempting that problem and whether this takes precedence over planning while solving a problem ('concurrent planning'). An additional question is whether the adoption of one mode of planning confers a performance advantage and under what circumstances one strategy is adopted in preference to others. The results of these studies suggest that initial planning can enhance problem-solving performance, but only when problems remain relatively simple. As problem complexity increases the effects of initial planning appear to have little or no effect upon performance. In conclusion it is suggested that strategy use depends upon the interactions between individual preference for a given strategy, problem complexity, and the stage that one has reached in the development of a solution to a problem. On the basis of their study Shin et al. (2003) conclude that requiring students to justify (i.e. argue for) their positions while solving problems (especially in case of ill-structured problems) should be an essential part of problem-solving instruction. Students should be provided with regulating methods of their cognitive activities during ill-structured problem solving. Conclusions and discussion To prepare students for academic practice it appeared functional to use authentic cases. We presented the results of a review of the available literature, showing a number of factors and principles that were found to be important in the transformation of professional practices and cases into learning tasks. In short: Both structure and complexity need to be carefully matched to the level students can cope with. Too much structure leads to a superficial approach to the problem since students perceive the problem to be less difficult than it really is. This also leads them to think that the problem is simple and therefore not challenging. Too little structure, on the other hand, frustrates students and generates difficulties in getting started. Too much complexity leads to cognitive overload. To cope with this students use

Page 19: The design of authentic tasks that promote higher-order ... · The design of authentic tasks that promote higher-order learning Hanne ten Berge, Stephan Ramaekers, Albert Pilot IVLOS

Berge, H. ten, Ramaekers, S., & Pilot, A. (2004). The design of authentic tasks that promote higher order learning. Paper presented at the EARLI-SIG Higher Education/IKIT-conference, June 18-21, 2004.

Page 19 of 23

routine approaches they are familiar with which are often too simple for the purpose. Furthermore, they perceive the task to be too difficult to handle and therefore not challenging. Authentic tasks are challenging to most students as the underlying problems are often open-ended, realistic, demanding and meaningful. Students have to develop new capabilities but expect a success experience since the process is perceived to lead to a concrete, feasible outcome. From a motivational point of view, authenticity is important but it should not be achieved by just introducing some real world elements, but by using an authentic case from professional practice as a starting point. In this transformation of authentic professional practices into the educational practice, chances are that essential parts of the authenticity gets lost. The designer should be aware of the features that are crucial to guard the authenticity of the case and that provoke the higher- order learning. Based on the literature, we presented several factors and principles in more detail for this aspect of the design of authentic tasks. Furthermore, we presented examples and effects of cases that are not designed correctly, that is, did not enhance the learning results that the designers were aiming at. These elucidate that authenticity appears to be an important issue and that design principles, connected to this dimension, might be indeed helpful in the design of case attributes that simulate academic practice and promote high-order learning. Authenticity and the use of authentic cases in higher-order learning needs further investigation in the perspective of the developments in Higher Education. In the future we will therefore focus more in detail on the coherence of the factors that play a part in higher order learning based on cases. References Abel, C. F. (2003). Heuristics and problem solving. New Directions for Teaching and Learning 95(Fall 2003): 53-58.

Aleven, V. (2003). Using background knowledge in case-based legal reasoning: A computational model and an

intelligent learning environment. Artificial Intelligence 150(1-2): 183-237.

Anderson, J. R., Reder, L. M. & Simon, H.A. (1996). Situated learning and education. Educational Researcher

25(4): 5-11.

Atkinson, J. & Raynor, O. (1974). Motivation and Achievement. Washington: Winston.

Barnett-Clarke, C. (2001). Case Design and Use: Opportunities and Limitations. Research in science education

31(2): 325-329.

Berkel, H. J. M. van, & Schmidt, H.G. (2000). Motivation to commit oneself as a determinant of achievement in

problem-based learning. Higher Education 40(231 - 242).

Black, P. J. (1991). Social and educational imperatives for changing examinations. IAEA Conference: Issues in

Public Examinations, Uitgeverij Lemma.

Boaler, J. (1994). When do girls prefer foodball to fashion? An analysis of female underachievement in relation to

'realistic' mathematics contexts. British Educational Research Journal 20(5): 551-564.

Bock, D. de, Verschaffel, L., Janssens, D., Dooren, W. van, & Claes, K. (2003). Do realistic contexts and graphical

representations always have a beneficial impact on students' performance? Learning and Instruction 13:

441-463.

Brown, J. S., Collins, A., & Duguid, P. (1989). Situated cognition and the culture of learning. Educational Researcher

18(1): 32-41.

Burkhard, H.-D. (2001). Similarity and distance in case based reasoning. Fundamenta Informaticae 47: 201-215.

Carlson, R. A., Khoo, H., & Elliot, R.G. (1990). Component practice and exposure to a problem solving context.

Human Factors(32): 267-286.

Page 20: The design of authentic tasks that promote higher-order ... · The design of authentic tasks that promote higher-order learning Hanne ten Berge, Stephan Ramaekers, Albert Pilot IVLOS

Berge, H. ten, Ramaekers, S., & Pilot, A. (2004). The design of authentic tasks that promote higher order learning. Paper presented at the EARLI-SIG Higher Education/IKIT-conference, June 18-21, 2004.

Page 20 of 23

Cartier, J. L. & Stewart, J. (2000). Teaching the Nature of Inquiry: Further Developments in a High School Genetics

Curriculum. Science and Education(9): 247-267.

Cary, M. & Carlson, R.A. (1999). External Support and the Development of Problem-Solving Routines. Journal of

Experimental Psychology: Learning, Memory, and Cognition 25(4): 1053-1070.

Child, D. (2004). Psychology and the teacher. London/New York: Continuum,

Cooper, B. (1994). Authentic testing in mathematics? The boundary between everyday and mathematical

knowledge in national curriculum testing in English schools. Assessment in Education 1(2): 143-167.

Cronin, J. F. (1993). Four misconceptions about authentic learning. Educational leadership 50(7): 78-80.

Cumming, J. J. & Maxwell, G.S. (1999). Contextualising Authentic Assessment. Assessment in Education:

Principles, Policy & Practice.

Daley, B. J. (1999). Novice to expert: An exploration of how professionals learn. Adult education quarterly 49(4):

133-149.

Daley, B. J. (2001). Learning and professional practice: a study of four professions. Adult Education Quarterly 52(1):

39-54.

Davies, S. P. (2003). Initial and concurrent planning in solutions to well-structured problems. The quarterly journal of

experimental psychology 56A(7): 1147-1164.

Deng, P.-S. (1996). Using case-based reasoning approach to the support of ill-structured decisions. European

Journal of Operational Research 93: 511-521.

Dochy, F., M. Segers, et al. (2003). Effects of problem-based learning: a meta-analysis. Learning and Instruction

(13): 533-568.

Dori, Y. J., Tal, R.T. & Tsaushu, M. (2003). Teaching Biotechnology Through Case Studies - Can We Improve

Higher Order Thinking Skills of Nonscience Majors? Science Education(87): 767– 793.

Ferretti, R. P. & Okolo, C. M. (1996). Authenticity in Learning: Multimedia Design Projects in the Social Studies for

Students with Disabilities. Journal of Learning Disabilities 29(5): 450-460.

Garner, D. D. (2000). The continuing vitality of the case method in the twenty-first century. Brigham Young

University Education and Law Journal 0(2): 307-346.

Gick, M. L. & Holyoak, K.J. (1980). Analogical problem solving, Cognitive psychology, 12, 306-355.

Gick, M. L. & Holyoak, K.J. (1983). Schema induction and analogical transfer, Cognitive psychology, 15, 1-38.

Gick, M. L. & Holyoak, K.J. (1987). The cognitive basis of knowledge transfer. Transfer of Learning: Contemporary

Research and Applications. S. M. Cormier & J. D. Hagman. San Diego, Academic Press.

Gijselaers, W. H. (1996). Connecting problem-based practices with educational theory. Bringing Problem-Based

Learning to Higher Education: Theory and Practice. New Directions for Teaching and Learning. L. Wilkerson

& W. H. Gijselaers. San Francisco, Jossey-Bass Inc. Publishers. 68: 3-13.

Gobert, J. D. & Pallant, A. (2004). Fostering Students’ Epistemologies of Models via Authentic Model-Based Tasks.

Journal of Science Education and Technology 13(1): 7-22.

Greeno, J.G., Korpi, M.K., Jackson III, D.N. & Michalchi, V.S. (1990). Processes and knowledge in designing

instruction (Report No. N00014-88-K-0152). Stanford, CA: Stanford University.

Greer, B. (1997). Modelling reality in mathematics classroom; the case of word problems. Learning and instruction

7(4): 293-307.

Hays, R. T. & Singer, M.J. (1989). Simulation Fidelity in Training System Design. New York: Springer-Verlag.

Hill, A. M. (1998). Problem solving in real-life contexts: an alternative for design in technology education.

International Journal of Technology and Design Education 8: 203/220.

Hmelo-Silver, C. E. & Pfeffer, M.G. (2004). Comparing expert and novice understanding of a complex system from

the perspective of structures, behaviors, and functions. Cognitive science 28(1): 127-138.

Houlden, R. L., Collier, C.P., Frid, P.J., John, S.L. & Pross, H. (2001). Problems identified by tutors in a hybrid

problem-based Learning curriculum. Academic Medicine 76(1): 81.

Page 21: The design of authentic tasks that promote higher-order ... · The design of authentic tasks that promote higher-order learning Hanne ten Berge, Stephan Ramaekers, Albert Pilot IVLOS

Berge, H. ten, Ramaekers, S., & Pilot, A. (2004). The design of authentic tasks that promote higher order learning. Paper presented at the EARLI-SIG Higher Education/IKIT-conference, June 18-21, 2004.

Page 21 of 23

Jacobs, A.E.J.P., Dolmans, D.H.J.M., Wolfhagen, I.H.A.P. & Scherpbier, A.J.J.A. (2003). Validation of a short

questionnaire to assess the degree of complexity and structuredness of PBL problems. Medical Education

37: 1001-1007.

Jagersma, P.K. (2004). Managing Business Complexity. Source: http://www.managementsite.net/content/articles/

461/461.asp

Janssen-Noordman, A.M.B. & Merriënboer, J.J.G. van (2002). Innovatief onderwijs ontwerpen. Via leertaken naar

complexe vaardigheden. [Designing innovative education. From learning tasks to complex skills.] Hoger

Onderwijs Reeks, Groningen/Houten: Wolters-Noordhoff.

Jensen, R. S. (1989). Aviation Psychology. Aldershot, UK, Gower Technical Publishers.

Jonassen, D.H. (2000). Toward a design theory of problem solving. Educational Technology Research &

Development 48(4): 63-85.

Jonassen, D.H. (2003). Designing Research-Based Instruction for Story Problems. Educational Psychology Review

15(3).

Jonassen, D.H. (2004). Learning to Solve Problems: An Instructional Design Guide. San Francisco, CA, Pfeiffer.

Jong, T. de, & Joolingen, W.R. van (1998). Scientific discovery learning with computer simulations of conceptual

domains. Review of educational research (68): 179-202.

Kyoo-Lak, C. & Jonassen, D.H. (2002). The Effects of Argumentation Scaffolds on Argumentation and Problem

Solving. Educational Technology Research & Development 50(3): 5-22.

López de Mántaras, R. & Plaza, E. (1997). Case-based reasoning: an overview. AI Communications 10: 21/29.

MacDonald, M., Badger, R. & White, G. (2000). The real thing? authenticity and academic listening. English for

Specific Purposes(19): 253-267.

Mandler, G. (1984). Mind and Body. New York: Norton.

McRobbie, C.J., Stein, S.J., & Ginns, I. (2001). Exploring Designerly Thinking of Students as Novice Designers.

Research in Science Education(31): 91-116.

Merriënboer, J.J.G. van (1997). Training complex cognitive skills: a four component instructional design model for

technical training. New Jersey, Englewood Cliffs.

Messick, S. (1994). The matter of style: Manifestations of personality in cognition, learning, and teaching.

Educational Psychologist 29(3): 121-137.

Moerkerke, G. (1996). Assessment for flexible learning. Performance assessment, prior knowledge state

assessment and progress assessment as new tools. Onderwijstechnologisch Expertise Centrum (OTEC).

Heerlen, Open Universiteit: 254.

Moust, & H.G. Schmidt (1998). Probleemgestuurd onderwijs [Problem-based education]. Groningen, Wolters-

Noordhoff.

Newmann, F.M. & Archbald, D.A. (1992). The nature of authentic academic achievement, in: Toward a New

Science of Educational Testing and Assessment. H. Berlak, F. M. Newmann, E. Adamset al. Albany, NY,

State University of New York Press.

Newmann, F.M., Marks, H.M., & Garnoran, A. (1996). Authentic pedagogy and student performance. American

Journal of Education 104(4): 280-312.

Norman, G.R. (2002). Research in medical education: three decades of progress. British Medical Journal

324(7353): 1560-1562.

Norman, G.R. & Schmidt, H.G. (1992). The psychological basis of problem-based learning: a review of the

evidence. Academic Medicine 67(9): 557-565.

Osana, H.P., Tucker, B.J., & Bennett, T. (2003). Exploring adolescent decision making about equity: Ill-structured

problem solving in social studies. Contemporary Educational Psychology 28: 357-383.

Parkes, J., Zimmaro, D.M., Zappe, S.M., & Suen, H.K. (2000). Reducing Task-related Variance in Performance

Assessment Using Concept Maps. Educational Research & Evaluation 6(4): 357-378.

Page 22: The design of authentic tasks that promote higher-order ... · The design of authentic tasks that promote higher-order learning Hanne ten Berge, Stephan Ramaekers, Albert Pilot IVLOS

Berge, H. ten, Ramaekers, S., & Pilot, A. (2004). The design of authentic tasks that promote higher order learning. Paper presented at the EARLI-SIG Higher Education/IKIT-conference, June 18-21, 2004.

Page 22 of 23

Perkins, D.N. & Salomon, G. (1989). Are cognitive skills context-bound? Educational Researcher 18(1): 16-25.

Powel, P. & Weenk, W. (2003). Project-led Engineering Education. Utrecht: Lemma.

Randel, J.M., Pugh, H.L., & Reed, S.K. (1996). Differences in expert and novice situation awareness in naturalistic

decision making. International journal of human computer studies 45(5): 579-597.

Reimann, P. & Schult, T. J. (1996). Turning examples into cases: Acquiring knowledge structures for analogical

problem solving." Educational Psychologist 31(2): 123-132.

Roth, W.F. (1985). Problem-Solving for Managers. New York, Praeger.

Rowland, G. (1991). Problem solving in instructional design. Doctoral dissertation, Indiana University.

Rowland, G. (1992). What do instructional designers actually do? An initial investigation of expert practice.

Performance Improvement Quarterly, 5(2), 65-86.

Salomon, G. (1981). Communication and education; social and psychological interactions. Beverly Hills, CA, Sage

publications.

Schnitzer, S. (1993). Designing an authentic assessment. Educational Leadership 50(7): 32-35.

Schoenfeld, A.H. (1998). Toward a theory of teaching-in-context. Issues in education 4(1): 1-95.

Seel, N.M. (2001). Epistemology, situated cognition, and mental models: 'Like a bridge over troubled water'."

Instructional science 29(4/5): 403-427.

Shin, N., Jonassen, D.H., & McGee, S. (2003). Predictors of Well-Structured and Ill-Structured Problem Solving in

an Astronomy Simulation. Journal of research in science teaching 40(1): 6-33.

Sweller, J., Chandler, P., Tierney, P. & Cooper, M. (1990). Cognitive Load as a Factor in the Structuring of

Technical Material. Journal of Experimental Psychology: General 119(2): 176-192.

Taconis, R., Ferguson-Hessler & Broekkamp, H. (2001). Teaching science problem solving: an overview of

experimental work. Journal of research in science teaching 38 (4): 442-468.

Tversky, A. & Kahneman, D. (1986). Rational Choice and the Framing of Decisions. Journal of Business 59(4):

S251-S278.

Verstegen, D.M.L. (2004). Iteration in instructional design: an empirical study of the specification of training

simulators. IVLOS. Utrecht, Utrecht University: 260 pp.

Wagner, R.K. (1991). Managerial problem solving. Complex problem solving; principles and mechanisms. R. J.

Sternberg & P. A. Frensch. Mahwah, N.J., Erlbaum.

Wallace, J. (2001). Introduction: Science Teaching Cases as Learning Opportunities. Research in Science

Education(31): 185-190.

Weiss, R.E. (2003). Designing Problems to Promote Higher-Order Thinking. New Directions for Teaching and

Learning 95(Fall 2003): 25-31.

Wiggins, G.P. (1993). Assessing Student Performance: exploring the purpose and limits of testing. San Francisco,

CA, Jossey-Bass.

Wiggins, G.P. (1996). Practicing what we preach in designing authentic assessments. Educational Leadership

54(4): 18-25.

Willoughby, T., Wood, E. & Kraftcheck, E.R. (2003). When can a lack of structure facilitate strategic processing of

information? British Journal of Educational Psychology 73: 59-69.

Wineburg, S. S. (1991). Historical Problem Solving: A Study of the Cognitive Processes Used in the Evaluation of

Documentary and Pictorial Evidence. Journal of Educational Psychology 83(1): 73-87.

Wolf, A. (1995). Authentic assessments in a competitive sector: institutional prerequisites and cautionary tales.

Evaluating Authentic Assessment. H. Torrance. Buckingham, Open University Press.

Yoshioka, T., Uchida, Y., & Kozu, T. (2003). Format of Cases Affects Learning Outcomes in First Year Medical

Students. Education for Health 16(1): 59-67.

Page 23: The design of authentic tasks that promote higher-order ... · The design of authentic tasks that promote higher-order learning Hanne ten Berge, Stephan Ramaekers, Albert Pilot IVLOS

Berge, H. ten, Ramaekers, S., & Pilot, A. (2004). The design of authentic tasks that promote higher order learning. Paper presented at the EARLI-SIG Higher Education/IKIT-conference, June 18-21, 2004.

Page 23 of 23