authentic assessment reference

Authentic Assessment

What is Authentic Assessment?

Good question! Authentic assessment is a method of evaluation in which students perform real-life tasks to demonstrate their ability to apply relevant knowledge and skills. An authentic assessment typically includes a task for students to complete and a rubric which indicates how the task will be graded. Criterion-reference, a term typically associated with authentic assessment, stresses the ability of authentic assessment to evaluate a specific test or specific area of content material. In other words, authentic assessment directly assesses a student's mastery of certain knowledge and skills. Authentic assessment is unique to the individual experience of each student.

Authentic assessment is also known by other names:

performance-based assessment-this is this is a popular term when referring to authentic

assessments. However, some feel that this is not an appropriate term as there is not reference to the

authentic nature of the assessment, as it is possible to have the student perform a task that has no

authentic connection to the real world.

direct assessment-this refers to the direct nature of the assessment and the student shows directly

how to apply the knowledge. In contract, a student would indirectly show knowledge in a multiple-

choice type test.

alternative assessment- as it is an alternative to traditional assessment

Why do We Need Authentic Assessment? Preparing Students for the Real World

While multiple-choice tests can be valid indicators or predictors of academic performance, too often our tests mislead students and teachers about the kinds of work that should be mastered. Norms are not standards; items are not real problems; right answers are not rationales. Multiple-choice tests also encourage memorization of facts, rather than acquiring specific skills standards are designed to enforce.

What most defenders of traditional tests fail to see is that it is the form, not the content of the test that is harmful to learning; demonstrations of the technical validity of standardized tests should not be the issue in the assessment reform debate. Students come to believe that learning is cramming; teachers

Rain Gwenneth Waters 1

come to believe that tests are after-the-fact, imposed nuisances composed of contrived questions--irrelevant to their intent and success. Both parties are led to believe that right answers matter more than habits of mind and the justification of one's approach and results. This type of assessment also sends a message to students that information is learned for a test, and as soon as the test is completed, students often do not see the importance of retaining this information.

A move toward more authentic tasks and outcomes thus improves teaching and learning: students have greater clarity about their obligations (and are asked to master more engaging tasks), and teachers can come to believe that assessment results are both meaningful and useful for improving instruction.

When students leave high school or even college they are expected to be able to function in our world based on a certain standard set of skills. Those life skills often do not include knowledge on ancient civilizations or chemical composition. It is great for students to have a well rounded education and a plethora of content knowledge, but if they cannot apply their skills to any content or task, teachers or schools have not prepared them for the real world.

Click on the video below to learn more about authentic assessment in "Assessment Overview: Beyond Standardized Testing"

Basic Elements of Authentic Assessment:

Students are asked to develop responses rather than choose from a list of possibly correct

answers

Fosters higher order thinking

Takes a direct approach to evaluate projects and the process of creating the final product

Aligns with classroom instruction

Uses student work which has been collected over time

Based on clear criteria given to students

Allows for multiple interpretations

Students learn to evaluate own work

Relates more to classroom learning

Ten Features of Authentic Assessments

1. Authentic activities have real-world relevance: Activities match as nearly as possible the real-

world tasks of professionals in practice rather than decontextualized or classroom-based tasks

2. Authentic activities are ill-defined, requiring students to define the tasks and sub-tasks needed

to complete the activity: Problems inherent in the activities are ill-defined and open to multiple

interpretations rather than easily solved by the application of existing algorithms. Learners must

identify their own unique tasks and sub-tasks in order to complete the major task.


3. Authentic activities comprise complex tasks to be investigated by students over a sustained

period of time: Activities are completed in days, weeks and months rather than minutes or hours.

They require significant investment of time and intellectual resources.

4. Authentic activities provide the opportunity for students to examine the task from different

perspectives, using a variety of resources: The task affords learners the opportunity to examine

the problem from a variety of theoretical and practical perspectives, rather than allowing a single

perspective that learners must imitate to be successful. The use of a variety of resources rather than a

limited number of preselected references requires students to detect relevant from irrelevant

information.

5. Authentic activities provide the opportunity to collaborate: Collaboration is integral to the task,

both within the course and the real world, rather than achievable by the individual learner.

6. Authentic activities provide the opportunity to reflect: Activities need to enable learners to make

choices and reflect on their learning both individuall and socially.

7. Authentic activities can be integrated and applied across different subject areas and lead

beyond domain-specific outcomes: Activities encourage interdisciplinary perspectives and enable

students to play diverse roles thus building robust expertise rather than knowledge limited to a single

well-defined field or domain.

8. Authentic activities are seamlessly integrated with assessment: Assessment of activities is

seamlessly integrated with the major task in a manner that reflects real-world assessment, rather than

separate artificial assessment removed from the nature of the task.

9. Authentic activities create polished products valuable in their own right rather than as

preparation for something else: Activities culminate in the creation of a whole product rather than

an exercise or sub-step in preparation for something else.

10. Authentic activities allow competing solutions and diversity of outcomes: Activities allow a

range and diversity of outcomes open to multiple solutions of an original nature, rather than a single

correct response obtained by the application of rules and procedures.

How Does Authentic Assessment Compare to Traditional Assessment?

With traditional assessment students are asked to demonstrate their knowledge of subject matter based on multiple choice or true/false questions and matching. Unlike authentic assessment, traditional assessment does not show the thought process which led students to arrive at the answer they selected. In contrast to traditional assessment, authentic assessment is much less structured and provides a more in-depth method of evaluating understanding in a subject area. The process is valued just as much as the product when the assessment is complete. The chart below compares and contrasts some characteristic of authentic and traditional assessment.

Authentic Assessment ..................................................Traditional Assessment

perform a task................................................................select a responsereal-life task...................................................................simulated and contrivedapplication and original construction.................................recall or recognitionstudent-based................................................................teacher-baseddirect evidence...............................................................indirect evidenceongoing over a long period of time....................................completed once for a specific amount of timeintegrated seamlessly within learning...............................completed once learning is "finished"

Though there are differences in the two, it does not mean they cannot be used together. Sometimes, these two types of assessments make great partners. An widely used example is how to choose a


chauffeur if there was a choice between one that has only passed the writing portion of the test and one that has only passed the driving portion. Most would choose the chauffeur that has passed the driving portion (the authentic assessment), however most would prefer their chauffeur has past both parts, the authentic assessment and the traditional assessment. This would insure that the chauffeur had basic knowledge of driving and road laws, as well as the skill to drive.

Types of Authentic Assessment:

Scoring Guides/Rubric: A scoring scale is used to assess student performance along a task-specific

set of criteria. A list of required elements are grouped together to make the scoring guide with point

specific designations.

Portfolio/E-Portfolio: A collection of a student's work specifically selected to highlight achievements

or demonstrate improvement over time (e-portfolio is electronic and usually accessible on the

Internet).

Authentic Task: An assignment given to students designed to assess their ability to apply standard-

driven knowledge and skills to real-world challenges.

Self-Assessment: Evaluating one's own performance to determine strengths and weaknesses, as

well as reflecting on what improvements can be made to enhance product

Oral Interviews: The teacher asks the student questions about the subject matter

Story or Text Retelling: Student retells main ideas or selected details of text experienced through

listening or reading.

Writing Samples: Student generates narrative, expository, persuasive, or reference paper.

Projects/Exhibitions: Student works with other students as a team to create a project that often

involves multimedia production, oral and written presentations, and a display.

Experiments/Demonstrations: Student documents a series of experiments, illustrates a procedure,

performs the necessary steps to complete a task, and documents the results of the actions.

Constructed-Response Items: Student responds in writing to open-ended questions.

Teacher Observations: Teacher observes and documents the students attention and interaction in

class, response to instructional materials, and cooperative work with other students.

Why Use Authentic Assessment?

1. Highlights constructive nature of learning and education

2. Allows students to choose own path for demonstrating skill set

3. Evaluates how effectively students can directly apply knowledge to a variety of task

4. Legitimizes learning by completing it in a real-world context

5. Allows for collaboration among students and across curriculum


Authentic Assessment: Advantages and Disadvantages

Advantages Disadvantages

Focuses on analytical skills and the integration of

knowledge

Time-intensive to manage, monitor, and coordinate

Promotes creativity Difficult to coordinate with mandatory educational

standards

Reflection of real-world skills and knowledge Challenging to provide consistent grading scheme

Encourages collaborative work Subjective nature of grading may lead to bias

Enhances written and oral presentation skills Unique nature may be unfamiliar to students

Direct match of assessment, instructional

activities, and learning objectives

May not be practical for large enrollment courses

Emphasizes integration of learning over time Challenging to develop for various types of courses

and ranges of objectives

How to Use Authentic AssessmentFollow these helpful steps to create your own authentic assessment:

1. Identify which standards you want your students to meet through this assessment.

2. Choose a relevant task for this standard, or set of standards, so that students can demonstrate how

they have or have not met the standards.

3. Define the characteristics of good performance on this task. This will provide useful information

regarding how well students have met the standards.

4. Create a rubric, or set of guidelines, for students to follow so that they are able to assess their work as

they perform the assigned task.


Creating Rubrics for Authentic AssessmentBefore making a rubric teachers need to identify what they want to assess. Rubrics should be created before the unit to ensure the students are taught the main components. In addition, it can assess criteria from previous units. Assessments should usually evaluate no more than five elements for each task. If too much is being assessed it is difficult to truly identify the strengths and weaknesses of a student

Once the criteria for the assessment is identified, a rubric can be created. Making a rubric is simplified with the aid of online rubric-makers. Before teachers create a rubric it is best to do a search for the specific rubric to save time. For example, input letter writing rubrics into a search address box and numerous letter writing sample rubrics will be displayed.Making rubrics are time consuming in the initial stages but are worth the investment. Rubrics are a wonderful tool to ensure a more authentic assessment of student work. The assessment tool gives students a framework on expectations and teachers a framework on what is being graded.

A rubric provides a teacher with a scale of where the student's current knowledge and performance

are currently at as well as what they may need to improve upon.

A rubric provides a student with their own guidelines while they are working on an assessment. They

are able to guide themselves, as well as assess their own work or the work of their classmates using

the rubric provided to them.

A teacher can work with his or her students to develop assessment criteria for a rubric. This way,

students are taking part in the evaluation process and feel more of an attachment to what they are

working on. They need to live up to their own standards (criteria) as well as that of the teacher.

Examples of Authentic Assessment Rubrics:Web Project RubricClassroom Web Page RubricWebQuest RubricMiddle School Research Project Rubric*Rubric Template for creating your own rubric**

Challenges of Authentic Assessment:

Usually takes longer to plan, complete, and evaluate than other methods of assessment

Difficult to ensure assessment accurately aligns with curriculum and standards

Allows for greater margin of evaluator bias/judgments of assessment


Examples of Authentic Assessmenthttp://www.eduplace.com/rdg/res/litass/class.htmlhttp://www.funderstanding.com/coasterhttp://boe.ming.k12.wv.us/teachers/di/di_rubrics/authentic%20assessment.htmhttp://www.ndtwt.org/hotlists/hotlists_LPsites.htm#AA

Sources/References:Funderstanding-Authentic AssessmentAuthentic Assessment ToolboxNorth Central Regional Educational LaboratoryWik ED--Authentic AssessmentPark University-Incorporating Authentic Assessment

Assessment methods based as closely as possible to real world experiences are called authentic assessment. Originally these were restricted to internship and apprenticeship experiences but have been expanded to the arts and other performances. The student is observed in action and the instructor provides feedback (direction). Now authentic assessment is being applied to all areas of the curriculum.

Real world experiences or simulations are normally complex and multi-faceted. A system is needed to analyze the complexities and to create clear criteria for student performance or their creation of a product. When provided with the assignment, a rubric establishes expectations. It is an authentic assessment tool which is growing in popularity due to its useful in assessing complex and subjective criteria.

Advantages of using rubrics in assessment include:

allowing assessment to be objective and consistent

allowing the instructor to clarify his/her criteria in specific terms

clearly showing the student how their work will be evaluated and what is expected

providing useful feedback regarding the effectiveness of the instruction

provide benchmarks against which to measure and document progress

The common features rubrics share include:

delineation of primary traits of performances and products

descriptions of various levels of performance or of product quality


a range for rating performance

Why Use Authentic Assessment?

The question "Why use authentic assessment?" is not meant to suggest that you have to choose between traditional assessments such as tests and more authentic or performance assessments. Often, teachers use a mix of traditional and authentic assessments to serve different purposes. This section, then, attempts to explain why teachers might choose authentic assessments for certain types of judgments and why authentic assessments have become more popular in recent years.

Authentic Assessments are Direct Measures

We do not just want students to know the content of the disciplines when they graduate. We, of course, want them to be able to use the acquired knowledge and skills in the real world. So, our assessments have to also tell us if students can apply what they have learned in authentic situations. If a student does well on a test of knowledge we might infer that the student could also apply that knowledge. But that is rather indirect evidence. I could more directly check for the ability to apply by asking the student to use what they have learned in some meaningful way. To return to an example I have used elsewhere, if I taught someone to play golf I would not check what they have learned with just a written test. I would want to see more direct, authentic evidence. I would put my student out on a golf course to play. Similarly, if we want to know if our students can interpret literature, calculate potential savings on sale items, test a hypothesis, develop a fitness plan, converse in a foreign language, or apply other knowledge and skills they have learned, then authentic assessments will provide the most direct evidence.top

Can you think of professions which require some direct demonstration of relevant skills before someone can be employed in that field? Doctors, electricians, teachers, actors and others must all provide direct evidence of competence to be hired. Completing a written or oral test or interview is usually not sufficient. Shouldn't we ask the same of our students before we say they are ready to graduate? Or pass a course? Or move on to the next grade?

Authentic Assessments Capture Constructive Nature of Learning

A considerable body of research on learning has found that we cannot simply be fed knowledge. We need to construct our own meaning of the world, using information we have gathered and were taught and our own experiences with the world (e.g., Bransford & Vye, 1989; Forman & Kuschner, 1977; Neisser, 1967; Steffe & Gale, 1995; Wittrock, 1991). Thus, assessments cannot just ask students to repeat back information they have received. Students must also be asked to demonstrate that they have accurately constructed meaning about what they have been taught.


Furthermore, students must be given the opportunity to engage in the construction of meaning. Authentic tasks not only serve as assessments but also as vehicles for such learning.

Authentic Assessments Integrate Teaching, Learning and Assessment

Authentic assessment, in contrast to more traditional assessment, encourages the integration of teaching, learning and assessing. In the "traditional assessment" model, teaching and learning are often separated from assessment, i.e., a test is administered after knowledge or skills have (hopefully) been acquired. In the authentic assessment model, the same authentic task used to measure the students' ability to apply the knowledge or skills is used as a vehicle for student learning. For example, when presented with a real-world problem to solve, students are learning in the process of developing a solution, teachers are facilitating the process, and the students' solutions to the problem becomes an assessment of how well the students can meaningfully apply the concepts.

Authentic Assessments Provide Multiple Paths to Demonstrationtop

We all have different strengths and weaknesses in how we learn. Similarly, we are different in how we can best demonstrate what we have learned. Regarding the traditional assessment model, answering multiple-choice questions does not allow for much variability in how students demonstrate the knowledge and skills they have acquired. On the one hand, that is a strength of tests because it makes sure everyone is being compared on the same domains in the same manner which increases the consistency and comparability of the measure. On the other hand, testing favors those who are better test-takers and does not give students any choice in how they believe they can best demonstrate what they have learned.

Thus, it is recommended (e.g., Wiggins, 1998) that multiple and varied assessments be used so that 1) a sufficient number of samples are obtained (multiple), and 2) a sufficient variety of measures are used (varied). Variety of measurement can be accomplished by assessing the students through different measures that allows you to see them apply what they have learned in different ways and from different perspectives. Typically, you will be more confident in the students' grasp of the material if they can do so. But some variety of assessment can also be accomplished within a single measure. Authentic tasks tend to give the students more freedom in how they will demonstrate what they have learned. By carefully identifying the criteria of good performance on the authentic task ahead of time, the teacher can still make comparable judgments of student performance even though student performance might be expressed quite differently from student to student. For example, the products students create to demonstrate authentic learning on the same task might take different forms (e.g., posters, oral presentations, videos, websites). Or, even though students might be required to produce the same authentic product, there can be room within the product for different modes of expression. For example, writing a good persuasive essay requires a common set of skills from students, but there is still room for variation in how that essay is constructed.


In Step 1 of creating an authentic assessment, you identified what you wanted your students to know and be able to do -- your standards.

In Step 2, you asked how students could demonstrate that they had met your standards. As a result, you developed authentic tasks they could perform.

In Step 3, you identified the characteristics of good performance on the authentic task -- the criteria.

Now, in Step 4, you will finish creating the authentic assessment by constructing a rubric to measure student performance on the task. To build the rubric, you will begin with the set of criteria you identified in Step 3. As mentioned before, keep the number of criteria manageable. You do not have to look for everything on every assessment.

Once you have identified the criteria you want to look for as indicators of good performance, you next decide whether to consider the criteria analytically or holistically. (See Rubrics for a description of these two types of rubrics.)

Creating an Analytic Rubric

In an analytic rubric performance is judged separately for each criterion. Teachers assess how well students meet a criterion on a task, distinguishing between work that effectively meets the criterion and work that does not meet it. The next step in creating a rubric, then, is deciding how fine such a distinction should be made for each criterion. For example, if you are judging the amount of eye contact a presenter made with his/her audience that judgment could be as simple as did or did not make eye contact (two levels of performance), never, sometimes or always made eye contact (three levels), or never, rarely, sometimes, usually, or always made eye contact (five levels).

Generally, it is better to start small with fewer levels because it is usually harder to make more fine distinctions. For eye contact, I might begin with three levels such as never, sometimes and usually. Then if, in applying the rubric, I found that some students seemed to fall in between never and sometimes, and never or sometimes did not adequately describe the students' performance, I could add a fourth (e.g., rarely) and, possibly, a fifth level to the rubric.

In other words, there is some trial and error that must go on to arrive at the most appropriate number of levels for a criterion. (See the Rubric Workshop below to see more detailed decision-making involved in selecting levels of performance for a sample rubric.)

Do I need to have the same number of levels of performance for each criterion within a rubric?

No. You could have five levels of performance for three criteria in a rubric, three levels for two other criteria, and four levels for another criterion, all within the same rubric. Rubrics are very flexible Alaskan Moose. There is no need to force an unnatural judgment of performance just to maintain standardization within the rubric. If one criterion is a simple either/or judgment and another criterion requires finer distinctions, then the rubric can reflect that variation.

Here are some examples of rubrics with varying levels of performance......

Do I need to add descriptors to each level of performance?

No. Descriptors are recommended but not required in a rubric. As described in Rubrics, descriptors are the characteristics of behavior associated with specific levels of performance for specific criteria. For example, in the following portion of an elementary science rubric, the criteria are 1) observations are thorough, 2) predictions are reasonable, and 3) conclusions are based on observations. Labels (limited, acceptable, proficient) for the different levels of performance are also


http://jfmueller.faculty.noctrl.edu/toolbox/howstep4.htm#top

included. Under each label, for each criterion, a descriptor (in brown) is included to further explain what performance at that level looks like.

Criteria Limited Acceptable Proficient

made good observations

observations are absent or vague

most observations are clear and detailed

all observations are clear and detailed

made good predictions

predictions are absent or irrelevant

most predictions are reasonable

all predictions are reasonable

appropriate conclusion

conclusion is absent or inconsistent with observations

conclusion is consistent with most observations

conclusion is consistent with observations

As you can imagine, students will be more certain what is expected to reach each level of performance on the rubric if descriptors are provided. Furthermore, the more detail a teacher provides about what good performance looks like on a task the better a student can approach the task. Teachers benefit as well when descriptors are included. A teacher is likely to be more objective and consistent when applying a descriptor such as "most observations are clear and detailed" than when applying a simple label such as "acceptable." Similarly, if more than one teacher is using the same rubric, the specificity of the descriptors increases the chances that multiple teachers will apply the rubric in a similar manner. When a rubric is applied more consistently and objectively it will lead to greater reliability and validity in the results.

Assigning point values to performance on each criterion

As mentioned above, rubrics are very flexible tools. Just as the number of levels of performance can vary from criterion to criterion in an analytic rubric, points or value can be assigned to the rubric in a myriad of ways. For example, a teacher who creates a rubric might decide that certain criteria are more important to the overall performance on the task than other criteria. So, one or more criteria can be weighted more heavily when scoring the performance. For example, in a rubric for solo auditions, a teacher might consider five criteria: (how well students demonstrate) vocal tone, vocal technique, rhythm, diction and musicality. For this teacher, musicality might be the most important quality that she has stressed and is looking for in the audition. She might consider vocal technique to be less important than musicality but more important than the other criteria.So, she might give musicality and vocal technique more weight in her rubric. She can assign weights in different ways. Here is one common format:

Rubric 1: Solo Audition

0 1 2 3 4 5 weightvocal tone vocal technique x2rhythm diction musicality x3

In this case, placement in the 4-point level for vocal tone would earn the student four points for that criterion. But placement in the 4-point box for vocal technique would earn the student 8 points, and placement in the 4-point box for musicality would earn the student 12 points. The same weighting could also be displayed as follows:



Rubric 2: Solo Audition

NA Poor Fair Good Very Good Excellentvocal tone 0 1 2 3 4 5vocal technique

0 2 4 6 8 10

rhythm 0 1 2 3 4 5diction 0 1 2 3 4 5musicality 0 3 6 9 12 15

In both examples, musicality is worth three times as many points as vocal tone, rhythm and diction, and vocal technique is worth twice as much as each of those criteria. Pick a format that works for you and/or your students. There is no "correct" format in the layout of rubrics. So, choose one or design one that meets your needs.

Yes, but do I need equal intervals between the point values in a rubric?

No. Say it with me one more time -- rubrics are flexible tools. Shape them to fit your needs, not the other way around. In other words, points should be distributed across the levels of a rubric to best capture the value you assign to each level of performance. For example, points might be awarded on an oral presentation as follows:

Rubric 3: Oral Presentation

Criteria never sometimes alwaysmakes eye contact 0 3 4volume is appropriate 0 2 4enthusiasm is evident 0 2 4summary is accurate 0 4 8

In other words, you might decide that at this point in the year you would be pleased if a presenter makes eye contact "sometimes," so you award that level of performance most of the points available. However, "sometimes" would not be as acceptable for level of volume or enthusiasm.

Here are some more examples of rubrics illustrating the flexibility of number of levels and value you assign each level.


Criteria never sometimes usuallymakes eye contact 0 2 4volume is appropriate 0 4enthusiasm is evident 0 4summary is accurate 0 4 8

In the above rubric, you have decided to measure volume and enthusiasm at two levels -- never or usually -- whereas, you are considering eye contact and accuracy of summary across three levels. That is acceptable if that fits the type of judgments you want to make. Even though there are only two levels for volume and three levels for eye contact, you are awarding the same number of points for a judgment of "usually" for both criteria. However, you could vary that as well:




Criteria never sometimes usuallymakes eye contact 0 2 4volume is appropriate 0 2enthusiasm is evident 0 2summary is accurate 0 4 8

In this case, you have decided to give less weight to volume and enthusiasm as well as to judge those criteria across fewer levels.

So, do not feel bound by any format constraints when constructing a rubric. The rubric should best capture what you value in performance on the authentic task. The more accurately your rubric captures what you want your students to know and be able to do the more valid the scores will be.

Creating a Holistic Rubric

In a holistic rubric, a judgment of how well someone has performed on a task considers all the criteria together, or holistically, instead of separately as in an analytic rubric. Thus, each level of performance in a holistic rubric reflects behavior across all the criteria. For example, here is a holistic version of the oral presentation rubric above.

Rubric 6: Oral Presentation (Holistic)

Oral Presentation RubricMastery

usually makes eye contact volume is always appropriate enthusiasm present throughout presentation summary is completely accurate

Proficiency

usually makes eye contact volume is usually appropriate enthusiasm is present in most of presentation only one or two errors in summary

Developing

sometimes makes eye contact volume is sometimes appropriate occasional enthusiasm in presentation some errors in summary

Inadequate



never or rarely makes eye contact volume is inappropriate rarely shows enthusiasm in presentation many errors in summary

An obvious, potential problem with applying the above rubric is that performance often does not fall neatly into categories such as mastery or proficiency. A student might always make eye contact, use appropriate volume regularly, occasionally show enthusiasm and include many errors in the summary. Where you put that student in the holistic rubric? Thus, it is recommended that the use of holistic rubrics be limited to situations when the teacher wants to:

make a quick, holistic judgment that carries little weight in evaluation, or evaluate performance in which the criteria cannot be easily separated.

Quick, holistic judgments are often made for homework problems or journal assignments. To allow the judgment to be quick and to reduce the problem illustrated in the above rubric of fitting the best category to the performance, the number of criteria should be limited. For example, here is a possible holistic rubric for grading homework problems.

Rubric 7: Homework Problems

Homework Problem Rubric++ (3 pts.)

most or all answers correct, AND most or all work shown

+ (1 pt.)

at least some answers correct, AND at least some but not most work shown

- (0 pts.)

few answers correct, OR little or no work shown

Although this homework problem rubric only has two criteria and three levels of performance, it is not easy to write such a holistic rubric to accurately capture what an evaluator values and to cover all the possible combinations of student performance. For example, what if a student got all the answers correct on a problem assignment but did not show any work? The rubric covers that: the student would receive a (-) because "little or no work was shown." What if a student showed all the work but only got some of the answers correct? That student would receive a (+) according to the rubric. All such combinations are covered. But does giving a (+) for such work reflect what the teacher values? The above rubric is designed to give equal weight to correct answers and work shown. If that is not the teacher's intent then the rubric needs to be changed to fit the goals of the teacher.

All of this complexity with just two criteria -- imagine if a third criterion were added to the rubric. So, with holistic rubrics, limit the number of criteria considered, or consider using an analytic rubric.



Final Step: Checking Your Rubric

As a final check on your rubric, you can do any or all of the following before applying it.

Let a colleague review it. Let your students review it -- is it clear to them? Check if it aligns or matches up with your standards. Check if it is manageable. Consider imaginary student performance on the rubric.

By the last suggestion I mean to imagine that a student had met specific levels of performance on each criterion (for an analytic rubric). Then ask yourself if that performance translates into the score that you think is appropriate. For example, on Rubric 3 above, imagine a student scores

"sometimes" for eye contact (3 pts.) "always" for volume (4 pts.) "always" for enthusiasm (4 pts.) "sometimes" for summary is accurate (4 pts.)

That student would receive a score of 15 points out of a possible 20 points. Does 75% (15 out of 20) capture that performance for you? Perhaps you think a student should not receive that high of a score with only "sometimes" for the summary. You can adjust for that by increasing the weight you assign that criterion. Or, imagine a student apparently put a lot of work into the homework problems but got few of them correct. Do you think that student should receive some credit? Then you would need to adjust the holistic homework problem rubric above. In other words, it can be very helpful to play out a variety of performance combinations before you actually administer the rubric. It helps you see the forest through the trees.

Of course, you will never know if you really have a good rubric until you apply it. So, do not work to perfect the rubric before you administer it. Get it in good shape and then try it. Find out what needs to be modified and make the appropriate changes.

Okay, does that make sense? Are you ready to create a rubric of your own? Well, then come into my workshop and we will build one together. I just need you to wear these safety goggles. Regulations. Thanks.

(For those who might be "tabularly challenged" (i.e., you have trouble making tables in your word processor) or would just like someone else to make the rubric into a tabular format for you, there are websites where you enter the criteria and levels of performance and the site will produce the rubric for you.)

Workshop: Writing a Good Rubric

find Workshop here

Step 1: Identify the Standards

Step 2: Select an Authentic Task

Step 3: Identify the Criteria for the Task



http://jfmueller.faculty.noctrl.edu/toolbox/howstep1.htm



Step 4: Create the Rubric

TECHNIQUES in assessment

Learning is . . . a dynamic process in which learners actively construct knowledge . . . the acquisition and organization of information into a series of increasingly complex understandings . . . influenced by context (Holt 1992). Educators who view learning in this way realize that quantitative methods of evaluating learners do not "measure up." Authentic forms of assessment present a more qualitative and valid alternative. Authentic assessments (AAs) incorporate a wide variety of techniques "designed to correspond as closely as possible to `real world' student experiences" (Custer 1994, p. 66). They are compatible with adult, career, and vocational education. After all, apprenticeship is a time-honored form of authentic learning: skills taught in context. "High-performance workplaces" demand critical thinking, self-directed learning, and individual responsibility for career development (Borthwick 1995; Jones 1994)-which the process of AA can develop. This Practice Application Brief describes types of authentic assessment, explains some of the advantages and challenges they present, and highlights some best practices in design and implementation, with specific examples from adult, career, and vocational education.

What Are AAs?

Assessments are authentic when they have meaning in themselves-when the learning they measure has value beyond the classroom and is meaningful to the learner. AAs address the skills and abilities needed to perform actual tasks. The following are some tools used in authentic assessment (Custer 1994; Lazar and Bean 1991; Reif 1995; Rudner and Boston 1994): checklists (of learner goals, writing/reading progress, writing/reading fluency, learning contracts, etc.); simulations; essays and other writing samples; demonstrations or performances; intake and progress interviews; oral presentations; informal and formal observations by instructors, peers, and others; self-assessments; and constructed-response questions. Students might be asked to evaluate case studies, write definitions and defend them orally, perform role plays, or have oral readings recorded on tape. They might collect writing folders that include drafts and revisions showing changes in spelling and mechanics, revision strategies, and their history as a writer.

Perhaps the most widely used technique is portfolio assessment. Portfolios are a collection of learner work over time. They may include research papers, book reports, journals, logs, photographs, drawings, video and audiotapes, abstracts of readings, group projects, software, slides, test results; in fact, many of the assessment tools listed earlier could have a place in a portfolio. However, the hallmark of a portfolio used for assessment is that the contents are selected by the learner (Hayes et al. 1994). The items are chosen according to a set of standards or objectives connected to the curriculum or learning event. They should represent a documented history of learning and an organized demonstration of accomplishment. Portfolios can serve as a catalyst for reflection on one's growth as a learner and a means of identifying areas for improvement (ibid.). They can serve as a tool for presenting oneself to potential employers (Borthwick 1995; MacIsaac and Jackson 1994).



What's Good about AAs?

Many of these methods are worlds away from traditional tests and grading. What advantages do authentic techniques provide? Well-designed AAs demonstrate a rich array of what learners know and can do; they display both the products and the processes of learning, making learners aware of the processes and encouraging ownership. Authentic assessments are adaptable, flexible, ongoing, and cumulative, depicting learner growth over time (Custer 1994; Holt 1992). Because they should be closely aligned with the curriculum, they connect thinking and doing, theory and practice, in authentic contexts. Assessment should become an integral part of teaching and learning; other learning opportunities may arise during assessment. "The process of assessment is itself a constructivist learning experience, requiring students to apply thinking skills, to understand the nature of high quality performance, and to provide feedback to themselves and others" (Rudner and Boston 1994, p. 7). The feedback and results enable teachers and learners to consider the next steps for improving both teaching and learning.

Although they raise concerns about subjectivity, AAs allow multiple human judgments of learning. Teachers, peer reviewers, and community members may all be involved in various performance ratings, and-a critical element-learners evaluate and monitor themselves. Alternative assessments can accommodate varied learning styles and serve the purposes of instruction, not other reasons for evaluating students (comparing individuals, comparing programs, demonstrating accountability, etc.).

Authentic assessments do pose certain challenges. They require abandoning traditional notions about testing and evaluation and they change teacher and student roles. They are time consuming for teachers to prepare and implement, because they require clarity in goals, outcomes, criteria, and expectations and assurance that all stakeholders understand (Hayes et al. 1994). To ensure that evaluation standards are applied consistently, teachers and other raters need careful training (Borthwick 1995). Students need to be prepared for self-monitoring and reflection (Jones 1994). Some may be more comfortable with the traditional boundaries of grades and testing at set times.

AAs are potentially more equitable in accommodating learning styles and acknowledging multiple ways of demonstrating competence. However, not all schools and districts may have access to some of the resources needed to develop them, and they impose demands that may challenge some students (Rudner and Boston 1994). Authentic assessments do not necessarily have to replace other forms of evaluation but can be used to augment and broaden the picture of learner progress. Jones (1994) cautions, however, that it is a mistake to use authentic techniques such as portfolios while still teaching primarily through traditional methods such as lectures and assigned textbook readings.

Adult, Career, and Vocational Education Applications

Adult educators, especially adult literacy teachers, find authentic assessments especially appealing as an alternative to the problematic use of standardized tests with adults. Workplace literacy programs are particularly rooted in the context of the job site; Bousquet et al. (1994) describe a workplace assessment in which participants are given a scenario depicting a work-related situation-choosing among two job offers-and must make a choice, explain their strategy for choosing it, give supporting facts, and state why the alternative was not chosen. The scoring rubric has five major


categories: understands scenario, demonstrates strategy, performs calculations, arrives at solution, writes response. Each category has subcriteria that are scored on three levels; e.g., under "understands scenario" are distinguishes relevant/irrelevant facts, identifies relationships among facts, draws inferences, mentions external factors.

A career education example is the Employability Skills Portfolio (Stemmer, Brown, and Smith 1992) used in Michigan schools. The portfolios contain evidence of students' attainment of academic skills, personal management behaviors such as meeting deadlines and working without supervision, and teamwork skills such as listening and compromise. Students update their portfolios throughout high school in consultation with parents and counselors, and local business representatives review them and provide feedback that helps students identify and improve weaknesses in their employment potential.

Vocational education has a long tradition of activity-based learning and product assessment (Custer 1994). In a business communication course (Fitch 1993), high school students define and create a business and are evaluated on innovation, creativity, following directions, writing, and format. Each student prepares a scenario describing the business; a spreadsheet showing products and profit; a job description and resume for a prospective employee; a database of positions and salaries; a letter of complaint and response letter (on student-designed letterheads with logos); and a biweekly company newsletter. Fitch shows how this ambitious project enables assessment of integrated skills, allows both high- and low-ability students to succeed, and draws upon the resources of the business community.

Some Advice for Implementation

It should be clear that authentic assessments must be carefully designed and evaluation criteria rigorously selected. Among the characteristics of good AAs are the following: (Custer 1994; Rudner and Boston 1994):

Engaging, meaningful, worthy problems or tasks that match the content and outcomes of instruction

Real-life applicability

Multistaged-demonstrations of knowing, knowing why, and knowing how

Emphasis on product and process, conveying that both development and achievement matter

Rich, multidimensional, varied formats, both on-demand (in-class essays) and cumulative (portfolios)

Opportunities for learner self-evaluation

Cognitive complexity-requiring higher order thinking skills

Clear, concise, and openly communicated standards

Fairness in scoring procedures and their application

To ensure that assessment and instruction are linked, they should be planned at the same time. The following questions can guide planning (Reif 1995; Rudner and Boston 1994): What should learners know and be able to do? What cognitive, affective, and metacognitive skills should they demonstrate? What types of problems or tasks involve those skills? What concepts or principles


should be applied in performing those tasks? What are the reasons for the assessment? What use will be made of the results? By whom? What criteria should be used?

One type of performance evaluation criteria are rubrics. Rubrics are scoring devices or tools that specify performance expectations and the various levels to which learners should perform (Custer 1994). Rubrics provide a framework that helps raters to be consistent, focuses the attention of assessor and assessee on important outcomes, and establish benchmarks for documenting progress. Rubrics feature (1) a stated standard, objective, behavior, or quality; (2) a rating scale; and (3) specific performance characteristics arranged in levels indicating the degree to which the standard has been met. Custer gives an example of a performance scenario or "design brief" used in technology education. The learner is asked to design an environmental control system for a room. The rubric lists eight criteria: number and quality of sources of information, number and quality of sources of supplies, ingenuity and creativity, use of design criteria, quality of documentation, workability of the system, quality of futures thinking, quality of the systems model, and remaining within budget. Each criterion is rated as exemplary, acceptable, or not yet acceptable.

Because alternative assessments take time to prepare, and because at best they should be learner centered and individualized, teachers should collaborate whenever possible in their development. Collaboration with employers and community members helps ensure the real-world authenticity of the tasks. Collaboration with students prepares them to be peer assessors and helps them develop responsibility for their learning. Before assessment, students can suggest creative alternatives and possible criteria; during assessment, teacher-learner interaction can bring out deeper understanding; and afterward, teachers and learners can reflect on the results to identify individual patterns of progress and new directions.


authentic assessment reference

Documents