cafas online user guide

CAFAS Online - User Guide

Computer Aided Feedback & Assessment System – Online Version

Martin Freney and Hew Ellwood

2009

CAFAS Online - User Guide – page 1

Contents

Contents .................................................................................................. 1 Acknowledgements & Licence Information ..................................................... 2 Introduction .............................................................................................. 3

Background ........................................................................................... 3 Features & Objectives ............................................................................. 3

Navigation ................................................................................................ 5 Key Features ............................................................................................. 7 How to Use CAFAS ..................................................................................... 7

Dealing with Errors in CAFAS ................................................................... 8 How to Configure CAFAS ............................................................................. 9

Accessibility Settings ............................................................................ 11 How to Design a Feedback Form ................................................................ 12

Visibility Settings Section (Assignment Setup Tab) .................................... 14 Graduate Qualities/Attributes Section (Assignment Setup Tab) ................... 18 Grade Scheme Section (Assignment Setup Tab) ........................................ 19 Assessment Criteria Section (Assignment Setup Tab) ................................ 20 Creating a Slider .................................................................................. 20 Creating a Rubric ................................................................................. 22 Assessment Penalties Section (Assignment Setup Tab) .............................. 24

How to Export a Feedback Form................................................................. 25 How to Import a Student List .................................................................... 27

To Add a Student Manually .................................................................... 35 To Remove a Student............................................................................ 35

How to Setup Standard Comments ............................................................. 36 How to Assess a Student .......................................................................... 38

Data & Graphs section .......................................................................... 47 How to Publish an Assessment ................................................................... 49

Manual Send instructions ....................................................................... 52 How to Use the Mark Book ........................................................................ 53

Export Student Results from Mark Book ................................................... 53 About CAFAS .......................................................................................... 55 Acknowledgements .................................................................................. 56

CAFAS Project Team ............................................................................. 56 Key Staff Involved in Trials .................................................................... 56

FAQs ...................................................................................................... 57 Glossary ................................................................................................. 59


Acknowledgements & Licence Information

Support for this project has been provided by the Australian Learning and Teaching Council, an initiative of the Australian Government Department of Education, Employment and Workplace Relations. The views expressed in this report do not necessarily reflect the views of the Australian Learning and Teaching Council Ltd. This work is published under the terms of the Creative Commons Attribution-Noncommercial-ShareAlike 2.5 Australia Licence. Under this Licence you are free to copy, distribute, display and perform the work and to make derivative works. Attribution: You must attribute the work to the original authors and include the following statement: Support for the original work was provided by the Australian Learning and Teaching Council Ltd, an initiative of the Australian Government Department of Education, Employment and Workplace Relations. Noncommercial: You may not use this work for commercial purposes. Share Alike. If you alter, transform, or build on this work, you may distribute the resulting work only under a licence identical to this one. For any reuse or distribution, you must make clear to others the licence terms of this work. Any of these conditions can be waived if you get permission from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by-nc-sa/2.5/au/ or send a letter to Creative Commons, 171 Second St, Suite 300, San Francisco, CA 94105, USA. Requests and inquiries concerning these rights should be addressed to the Australian Learning and Teaching Council, PO Box 2375, Strawberry Hills NSW 2012 or through the website: http://www.altc.edu.au 2009


Introduction

The Computer Assisted Feedback and Assessment System (CAFAS) User Guide is designed to be read as a printed document that will assist new users in learning how to use the system effectively. This guide is based on the on-screen help resources available via the Help menu in CAFAS Online. In this guide you will find detailed descriptions, and examples of how to use CAFAS Online. There is also a Help Box in the lower left hand corner of the CAFAS window which you might find more convenient, however the information there is less detailed. Training videos can also be accessed via the project website, Training Videos page: http://cafas.pbworks.com/Training-Videos

Background

The CAFAS project was funded by the Australian Learning and Teaching Council and has been developed in collaboration with staff in the School of Architecture and Design, the School of Communication, the Learning and Teaching Unit, and the Information Strategy Technology Services unit at the University of South Australia.

CAFAS was conceived of in 2005 and has been continuously under research and development since then. Ongoing research into the efficacy of CAFAS is essential to the system’s future. Research reports will be posted on the website and findings will be presented at conferences and published in journals.

The system is published under the open-source, Creative Commons license so that it is free to use, and can be further extended by the higher education community. It has been developed as a web-based application so that users of Macs and PCs can use CAFAS via their web browsing software. An Excel based version is also available as a desktop solution for those unable to access the online option. The Excel version is also published under Creative Common license, and can be downloaded from the project website.

Features & Objectives

CAFAS is an online version of the traditional paper-based Feedback Form (a.k.a. Assessment Sheet, Feedback Proforma) commonly used by academics to communicate feedback and assessment information to students. Typically this information includes marks, grades and written comments which contain advice on how students can improve their work and to justify why a particular grade/mark was given.

Integral to CAFAS is the concept of weighted assessment criteria. The important aspects of the assessment task (a.k.a. assignment, project, assessment) are called assessment criteria. It is important that the assessment criteria are clearly explained and given a weighting to convey their relative importance as this helps students understand where they should concentrate their efforts. Typically there are 3-5 assessment criteria used to assess an assignment.

CAFAS provides three methods for communicating a student’s performance in a particular assessment criterion: A Slider (performance continuum); a Rubric (matrix/table) and Comments Boxes.

The performance continuum is a horizontal Slider similar to a scroll bar. Staff can position the pointer anywhere along the sliding scale to indicate the student’s performance for a particular assessment criterion. Comments can then be typed into a Comments Box underneath the Slider to help explain to students why they received a particular grade and how they can improve. There is an example of a Slider on page 20.

The Rubric comprises a series of descriptors, which define the indicators of various levels of performance outcomes. Typically one or two sentences would be used to explain each performance level. There are typically 4-7 performance levels.


Staff can tick a box to indicate to the student which descriptor is relevant to their assignment for a particular assessment criterion. There is an example of a Rubric on page 22.

There is a Summary Comments Box for both Rubric and Slider Feedback Forms. Staff can type information into this box to advise students of how they have performed overall. A Class Feedback Comments Box provides an opportunity to give advice to all students on how to improve with the subsequent assignment.

Sliders have a Comments Box associated with them as the Slider only conveys the general performance level. The comments box associated with the Slider can be used to give more detailed feedback to the student relating to the result for that assessment criterion.

Comments Boxes have a drop down menu, which enables staff to quickly access standard comments from a database. The database can be prepared, preferably by the assessment team (coordinator and tutors) before assessment starts, but it is also possible to develop the standard comments “on the fly” during the assessment process. It is anticipated that this functionality will help staff to quickly enter detailed comments that have been designed to communicate clearly. It is also possible to append additional customized comments to the standard comments so that the feedback can be tailored to the individual student.

CAFAS gives the staff member control over the visibility of marks (score out of 100) and grades so that it is possible to use CAFAS to provide formative feedback. In this case only the Comments Boxes for each assessment criterion are shown plus the Summary Comment. At a later time, after students have reworked their assignment based on the formative feedback, staff can re-evaluate the assignment, making the Sliders visible for a summative assessment scheme.

A section of the Feedback Form is dedicated to Grade Descriptors and which enables the staff member to describe the necessary attributes and indicators to qualify for a particular grade.

Likewise a section for Graduate Attributes enables the staff member to describe how a particular assignment develops the various graduate attributes specific to a particular institution.

Class Feedback can be provided to students as a way to communicate general comments about the performance of the class as a whole: what was generally done well and what aspects were misunderstood or overlooked. It is possible to publish a graph which displays the distribution of grades anonymously. This enables students to gauge their success in the context of their class mates’ performance.

CAFAS has been designed to enable it to be easily customized so that it conforms with the unique requirements of tertiary education institutions. A drop down menu enables staff to select their institution from a list and this populates CAFAS with the relevant terminology and grade/mark schemes that are specific to that institution. For example the term used to describe the typical 12-14 week “subject” can also be referred to as a “course” or a “unit”. Grade/mark schemes also vary widely and this information is recorded in the CAFAS database so that staff do not have to enter this information manually.

CAFAS is able to export tables of grades and marks so that it can be manipulated in Microsoft Excel and other spreadsheet software. The Feedback Forms can be exported as HTML files so that they can be included in course/unit/subject information prior to use in the assessment process.

Time intensive tasks such as recording of assignment grades/marks and totalling of the overall grade/mark have been automated by the system. Feedback Forms can be efficiently dispatched to students via email. It is anticipated that these automated functions will enable staff to staff focused on providing high quality feedback rather than tedious administrative tasks.


Consideration to accessibility issues has been integral to the development of CAFAS to ensure that staff and students with disabilities are not disadvantaged.

In summary, CAFAS replicates a commonly used system, the paper based Feedback Form through a user-friendly online system. Its online nature enables new possibilities and opportunities to improve feedback and assessment. It is hoped that the teaching and learning community will continue to actively participate in the continuous evolution of this highly flexible, open-source learning tool.

Initially CAFAS will be offered as a web-based system and will be hosted by the University of South Australia. Your data will only be available to you, the colleagues that you nominate, and your students. The licensing scheme (Creative Commons Non-commercial ShareAlike 2.5 Australia Licence) enables all universities to use CAFAS free of charge and to install and develop CAFAS for their local IT requirements in the event that the institution decides that the system hosted by UniSA is not suitable.

There is a website at http://cafas.pbworks.com that provides a forum for discussion about CAFAS and more information about the system including all publications relating to the project.

Navigation

CAFAS combines a WYSIWYG (What You See Is What You Get) Graphical User Interface (GUI) with an intuitive & logical tabbed navigation layout.

The main navigation of the program is through the 3 Tabs located along the top of the CAFAS program. These tabs from left to right are named: Admin Setup; Assignment Setup; and the Assessment tab.

Within each of these 3 main tabs, there is a navigation area called the Control Panel which is the area on the left hand side of CAFAS.

From the Control Panel you can navigate to the various Sections within each Tab. The logical progression for the user throughout the program is a left to right and top to bottom navigation setup. That is, start with the Admin Setup Tab before moving to the Assignment Setup Tab and then finally on to the Assessment Tab.

Each Tab is made up of various sections. Start with the top most Section of the Control Panel before moving down the screen systematically to the lower Sections of the Control Panel.

The Main Window is the large area of the screen to the right of the Control Panel. This area updates according to the selection in the Control Panel. The Main Window displays most of the information for CAFAS including the Sliders, Rubrics, and the Feedback Form.

The following screenshot shows the Assess Student Section of the Assessment Tab. The main areas of the interface are indicated;


1. The main drop down menus: Controls, Help, About 2. The 3 Tabs 3. The Sections. Each Tab has its own set of Sections. 4. The Control Panel. Each Section has its own Control Panel. 5. The on-screen Help Box (with scroll bar to the right). Handy hints for each Section. 6. The Selection Bar: displays the current selection of Course; Delivery; Assignment; Student’s Name; Mark; Grade 7. Main Window area: this area displays the bulk of the information and updates with each Section. 8. A pop-up window (in this example it is the Comments List). 9. The Logout button: use this when you have finished with CAFAS. 10. Main Window area scroll bar: to scroll the main window up/down.


Key Features

Create standardised Feedback Forms for student assessment Weighted assessment criteria using either Sliders (sliding scale) or Rubrics (feedback comment matrix) or both. Penalty criteria (subtracts marks from total mark e.g. for late submission) Email student results directly from the CAFAS program in HTML and Zip format. Automatic calculation of marks and grades based on evaluations made with the Sliders and Rubrics. Integrated Mark Book holds all results in a secure database. Add users and modify permission levels. View Course, Delivery & Assessment details via folder structure. Import list of students including email addresses, course codes etc. Export student marks & grades for uploading into your institutions information management system. Graphs automatically generated and included in Feedback Forms to display overall class results (distribution of grades).

How to Use CAFAS

CAFAS Online is a web based application. Thus, it can be used from any computer with an internet browser and broadband internet connection. Adobe Flash Player is also required. All testing of CAFAS has been done using Internet Explorer Version 7 (IE7). There are some issues with FireFox so IE7 is recommended.

Please visit http://www.cafas.pbworks.com for the most up to date URL to login to CAFAS. This URL might change from time to time as updated versions are implemented.

1. Login to CAFAS using your university username and password. You will need to register your details through Shibboleth the first time you login - see CAFAS website for registration details http://www.cafas.pbworks.com

2. Configure CAFAS to your institution's specific requirements and terminology and specify colleagues in your assessment team i.e. the people who will be helping you with assessment/feedback.

3. Design a Feedback Form. Define the assessment criteria and their weightings. Decide between the Slider and Rubric feedback mechanisms or use a combination. Penalty criterion can also be included in either Slider or Rubric format.

4. Initial Publication of the blank Feedback Form to students. Ideally you should publish the blank Feedback Form at the beginning of the assignment so students can understand how they will be assessed. CAFAS will produce an HTML file of the Feedback Form which can be copied/pasted into course/unit/subject material and formatted in a word processor, or it can simply be forwarded to students via email.

5. Use the Feedback Form. The assessment stage is conducted by the whole assessment team - collaborating online – concurrently or sequentially.

6. Review and moderate Feedback Forms. This stage is conducted by the course/subject/unit coordinator prior to emailing the completed forms to students.


7. Final Publication of the completed Feedback Forms to students is achieved via an automatically generated email message which is copied to whomever is logged in.

The Getting Started Section of Admin Setup provides video tutorials on the use of CAFAS.

Dealing with Errors in CAFAS

If an error window pops up while you are using CAFAS it is important to Refresh your web browser. In Internet Explorer the keyboard shortcut for Refresh is F5. You will have to log in again. WARNING: if this step (Refresh) is not taken, errors may be compounded. Always Refresh your internet browser after an error!

An error pop-up dialogue box allows users to register the error with the developers who will endeavour to fix any serious bugs/errors.

To view an online Tutorial of the registration and login process click on Tutorial 1

Tutorial 1- Registration & Login

From the Getting Started Section of Admin Setup you can view the 3 Tutorial videos that demonstrate how to use the CAFAS program. For other Tutorial videos please go to the CAFAS project website where others are available http://www.cafas.pbworks.com


How to Configure CAFAS

To Configure CAFAS begin in the Admin Setup Tab.

1. Click the Admin Setup Tab 2. Click the Institution & Terminology Section 3. Click on the drop down menu arrow to see a list of institutions (if your institution is

not listed email [email protected] 4. Click on your institution from the drop down list – it will now be displayed in the drop

down box. 5. Click the Save as default button so that CAFAS remembers this setting the next

time you login (you won’t need to use this Section of CAFAS again unless you work in multiple institutions).

The next step is to set Users & Permissions in the Admin Setup tab. From this section add new users that will have access to the assignments for this Course.

1. Click the Admin Setup Tab. 2. Click the Users & Permissions Section. 3. Click Add User to add a new user by inputting their Institution Email address in the

text area and then press the Enter/Return key on the keyboard. Select either Coordinator (C) or Assessor (A) permissions from the radio buttons to set the permission level for all new users.

4. A new text box will appear which says “enter email”. Type the email of your colleague who will be assisting with assessment. Press Enter/Return key on the keyboard to finalise this command. The person’s name will appear in the text box.

5. Click one of the radio buttons (to the right of the person’s name) to indicate whether they have “coordinator” or “assessor” permission. Coordinators can design and modify feedback forms whereas Assessors can only perform assessment tasks: awarding grades/marks, making comments, and sending feedback forms.

6. To delete a user, select the tick box (to the left of the person’s name) and click the Delete User button.


Accessibility Settings

1. From the Controls area of the Main Drop-Down Menu, select the Preferences option.

2. Select Font options using the drop down menu. 3. Select Colour options using the drop down menu. 4. Select Contrast options using the drop down menu. 5. Click the Save button. 6. Click the close dialogue box button (X).


How to Design a Feedback Form

The first step is to setup a folder structure to keep all your assignments organised in a logical structure. There are three levels of organisation: The Course (aka Unit or Subject) the Delivery (i.e. a time/place) and the Assignment (aka Assessment). To setup the folder structure and to start the process of creating a Feedback Form for an Assignment;

1. Click the Assignment Setup Tab. 2. Click the Assignment Management Section. 3. In the Control Panel type the name for a new Course/Subject/Unit in the text box. 4. Click the New Course Button. A new icon will appear at the bottom of the folder structure to the right of the Control Panel. 5. Select the new Course by clicking it (it will be highlighted to indicate that it is selected). 6. In the text box in the Control Panel first delete anything remaining there and type the name of the “Delivery” which is a period of time (e.g.

semester 1) or a place (e.g. Hong Kong) or both (e.g. semester 1, Hong Kong). 7. Click the New button. 8. You will notice that a triangle icon appears to the left of the new Course. Click this triangle to expand the folder so that you can see your

new Delivery. 9. Select the new Delivery by clicking it (it will be highlighted to indicate that it is selected). 10. In the text box in the Control Panel first delete anything remaining there and type the name of the “Assignment” you want to create. 11. Click the New button. 12. You will notice that a triangle icon appears to the left of the new Delivery. Click this triangle to expand the folder so that you can see your

new Assignment. 13. Select the new Assignment by clicking it (it will be highlighted to indicate that it is selected). 14. In the Main Window you will see a table with 8 rows. Review the data in the right column and add information where necessary (e.g.

“School”) and edit where necessary (e.g. Weighting and Due Date). 15. Click the Save button in the Main Window.

You will see the name of the newly created Course, Delivery and Assignment along the top of Main Window of CAFAS in the “Selection Bar”.


Visibility Settings Section (Assignment Setup Tab)

The Visibility Settings Section gives some control over what students will see on their Feedback Form. Regardless of the settings you make here you (the assessor) will see this information during the assessment process – so don’t be deterred by this. A preview will be displayed in the Main Window to demonstrate the effects of your settings: this is what the student’s Feedback Form will look like. You need to make these settings for each assignment.

Note that these settings are specific to the selected assignment only. By default they are all “shown” so if you wish to “hide” elements routinely, for all assignments, you will need to make these settings for each assignment.

1. Click the Assignment Setup Tab. 2. Click the Visibility Settings Section. 3. Use the radio buttons – Show or Hide – to control what elements of the Feedback Form the students will see.

The Overall Mark and Overall Grade control whether or not this information is included on the Feedback Form. For example you might want to hide the Overall Mark and only give an Overall Grade so that students don’t pester for an extra 1% so they can go up a grade. Or perhaps there are institutional policies regarding what you can and can’t display.

There are Criteria Visibility Settings for Mark and Grade. These control the visibility of the Mark and Grade for each Assessment Criterion and each Assessment Penalty.


The Class Feedback Visibility Settings relate to some items towards the end of the feedback form – you might need to scroll down in the Main Window to see the effects of changing these settings. Class Feedback is the last text box on the Feedback Form. Anything you type in this box will appear on every student’s Feedback Form – they will all get the same message.

Assignment Table, controls whether the list of grades will be displayed. Assignment Grade and Assignment Mark controls what columns are displayed in the table. For example you might want to hide the Mark column and only display the Grades – don’t hide both though, just hide the whole table (select Hide for Assignment Table) if you don’t want to display a table at all.

The Grade Graph settings control visibility of a graph which shows the distribution of grades for the whole class. An abbreviation for each grade is displayed on the X axis and the number of grades on the Y axis.


Graduate Qualities/Attributes Section (Assignment Setup Tab)

The Graduate Qualities/Attributes section provides you with the opportunity to customise the Description for each attribute so that you can describe how each particular assignment develops each attribute. Not all assignments develop all attributes therefore it would be appropriate to write e.g. “Not applicable to this assignment” for any attributes that are not applicable to the currently selected assignment.

1. Click the Assignment Setup Tab 2. Click the Graduate Qualities Section 3. If you want to edit the default descriptors of your institution, click the text you want to change (to highlight it) and start typing. Press

Enter/Return (on your keyboard) to end the editing and save your changes. Note that these changes are applicable to the selected assignment only.

4. If you make any edits that you are not satisfied with and want to return to the default descriptions for your institution click the Restore Institution Attributes button.


Grade Scheme Section (Assignment Setup Tab)

The section will allow you to define your grade scheme from a pre configured list of options that are available in your institution: you cannot create your own grade scheme.

The name of your institution should be displayed above the drop down menu in the Control Panel. If this is incorrect you need to go back to the Admin Setup Tab, Institution & Terminology section to select the correct institution.

1. Click the Assignment Setup Tab. 2. Click the Grade Scheme Section. 3. Use the drop down menu to select from the list of options. “Standard” is the most commonly used option, however there might also be

other schemes such as “Non Graded” (i.e. pass/fail) or “Honours”. After making a selection e.g. “Standard” the Main Window will update with a table that lists the Grade Name, Threshold Mark (i.e. the lowest mark needed to attain a particular grade), Grade (abbreviation), and Grade Descriptor. The drop down menu will now read “Currently Selected Scheme”.

4. Click the Save button which is under the drop down menu in the Control Panel. 5. You can edit the Grade Descriptor to make it more meaningful in the context of the currently selected assignment. Click on the Grade

Descriptor that you want to edit. 6. Click the Save button again after you have completed your editing of the Grade Descriptors. (To reinstate the original Grade Descriptors

follow steps 3-4)


Assessment Criteria Section (Assignment Setup Tab)

This is the main section of the Assignment Setup tab - where you actually define your assessment criteria, their weightings and what mechanism you will use to assess them. The two mechanisms you can choose from are Slider or Rubric. The Slider is a sliding scale from 0 to 100% whereas the Rubric is a matrix of feedback comments each with a corresponding mark.

1. Click the Assignment Setup Tab. 2. Click the Assessment Criteria Section. 3. Click the Add Criteria button in the Control Panel. 4. Enter a weighting in the % box (left box). 5. Click in the “Enter Name” box and enter the name of the assessment criterion. 6. Select one of the radio buttons to designate Slider or Rubric for this assessment criterion. 7. Click the Save button in the Control Panel. 8. Click the Radio button at the far left (next to the % box) to display this criterion in the Main Window.

Note that typically you will want the weightings of the Assessment Criteria to total 100% (the total is displayed in the Control Panel), however this is not essential. For example you could create a Bonus Marks criterion to award extra marks up to a maximum of 100%.

If you make a mistake you can delete a criterion. Use the radio button at the far left (next to the % box) to select the criterion you want to delete and then click the Remove Criteria button.

Creating a Slider

9. After completing step 6 above, you will be able to see the Slider in the Main Window. First, enter a description for this assessment criterion to clearly explain the performance criteria to the student (and as a reminder to the assessor).

10. Click the Save button at the bottom right corner of the Slider Window in the Main Window (not to be confused with the Save button in the Control Panel).

11. Optionally (preferably) add some Slider Descriptors in the text box above the Slider e.g. at the left end you could write “poor”, in the middle “pass” and at the far right end “excellent”. Use the space bar on your keyboard to space out your descriptors so that they align with the grades appropriately.

12. Click the Save button at the bottom right corner of the Slider window in the Main Window (not to be confused with the Save button in the Control Panel).


Creating a Rubric

1. After completing step 6 above, you will be able to see the Rubric in the Main Window. First, enter a description for this assessment criterion to clearly explain the performance criteria to the student (and as a reminder to the assessor).

2. Click the Save button at the bottom right corner of the Rubric Window in the Main Window (not to be confused with the Save button in the Control Panel).

3. Type a weighting for the first Rubric Descriptor in the long text box where it says “weighting for new descriptor”. You can choose a number from 0 to 100.

4. Click the Add button in the Rubric Window. The first row of the Rubric will appear. 5. In the first column highlight “Descriptor” and type a descriptor e.g. “Excellent”. 6. In the last column highlight “Enter Description” and type a description e.g. “You have demonstrated an extremely high level of

understanding in this assessment criteria…” 7. Click the Save button at the bottom right corner of the Rubric Window in the Main Window (not to be confused with the Save button in the

Control Panel). 8. Repeat steps 3 to 7 for each new row of the Rubric. 9. If you make a mistake you can delete a row of the Rubric by selecting the row(s) with the tick box(es) to the left of the rows and then

clicking the Remove button.


Assessment Penalties Section (Assignment Setup Tab)

This section is identical to the previous section (Assessment Criteria) with the exception that the Sliders and Rubrics generated will be used to deduct marks from the total e.g. if the student scored 70% based on the Assessment Criteria, a Penalty of 10% in the Assessment Penalties would result in an Overall Mark of 60% (not 70 minus 10% of 70 which would equal 63).

To setup Sliders and or Rubrics for Assessment Penalties follow the instructions for Assessment Criteria.

To view online tutorials on the Assignment Management Section of the Assignment Setup Tab click on Tutorials 2 and 3

Tutorial 2 - Assignment Management 1

Tutorial 3 - Assignment Management 2 .

To view an online Tutorial about setting Graduate Attributes click on Tutorial 4

Tutorial 4 - Graduate Attributes

To view an online Tutorial about setting the Grade Scheme click on Tutorial 5

Tutorial 5 – Grade Scheme

To view an online Tutorial about Assessment Criteria click on Tutorial 6

Tutorial 6 - Assessment Criteria

To view an online Tutorial about Assessment Penalties click on Tutorial 7

Tutorial 7 - Assessment Penalties

To view an online Tutorial about setting up Standard Comments click on Tutorial 8

Tutorial 8 - Standard Comments

To view an online Tutorial about setting up Students click on Tutorial 9

Tutorial 9 - Student and Assessors


How to Export a Feedback Form

You might need a copy of the blank (unused) Feedback Form to include in course/subject/unit documentation, or to circulate to students prior to assessment.

CAFAS can export the Feedback Form that you created in the Assignment Setup tab to your email inbox in HTML format. This can then be copied and pasted into a word processor for further formatting, or it can be forwarded to student email addresses using your email software (e.g. Outlook).

The design of a Feedback Form is described in the previous section of the Help in How to Design a Feedback Form .

1. Click the Assignment Setup Tab. 2. Click the Review Form Section. 3. Use the scroll bar for the Main Window to move up/down the form. Note that settings made in the Visibility Settings may not be displayed

correctly (known bug in the software) however it is useful to be able to review all the assessment criteria details and the Graduate Attributes and Grade Descriptors that have been setup previously. Any editing that is required must be done in the appropriate Sections – editing cannot be performed in the Review Form section – it is strictly for reviewing – not editing.

4. Click the Email form as HTML button to email yourself a copy of the Feedback Form. 5. Check your email for a message from “CAFAS Email Service”. 6. Open the email and Copy the contents of the message and Paste into a word processor for formatting and printing. Or Forward the email

to your students.


To view online tutorials on the Review Form section of the Assignment Setup tab click on Tutorial 10.

Tutorial 10 – Export a blank Feedback Form


How to Import a Student List

Following is a description of how to Import a Student List into CAFAS. By clicking the 'Import Students' button, a pop-up box will be displayed allowing you to Import a list of students as a CSV file. A CSV file is a standard format for database lists and can be exported easily from programs such as Excel.

There are 7 key pieces of data that CAFAS expects although not all of them are essential (* indicates essential items). Make sure they are included in your CSV file.

1. Remove any blank rows that may be at the top of your spreadsheet as this may prevent CAFAS from importing the data. (Note to UniSA staff: for a typical student list download from MyUniSA there will be two rows above the main column headings that need to be deleted.)

2. Student Number * 3. Student Email * 4. Student Family Name * 5. Student First Name * 6. Student Middle Name 7. Student Class Number 8. Student Program


Having prepared the CSV file as described above you are ready to import the student details into CAFAS.

1. Click the Assignment Setup Tab. 2. Select an assignment from the Assignment Management Section of the Assignment Setup Tab by clicking on the name of the Assignment

in the list of folders/assignments. 3. In the Students Section click the Import Students button. 4. A pop-up box will appear with four tabs along the top. Start with the Source Data tab. 5. Click the Load From File button, browse for the CSV file, and click Open in the browse dialogue box.


6. An “Uploading file” message will appear at the bottom of the pop-up box. 7. When the uploading file message disappears, click the OK button.

8. After clicking OK, a table displaying the contents of the CSV file should appear automatically in the Table tab of the Source Data tab. Review this data to ensure that it is correct.

9. Click the Mapping Fields tab of the pop-up box.


10. Click a row of the Source/Destination table. 11. Change the Source Column name from “not set” by selecting the appropriate option from the Source Column drop down list. 12. Click on the Update button to confirm the change. The objective of this process is to match the column heading names of your CSV file

(displayed in the Source Column drop-down) with the data fields in CAFAS (displayed in the “Destination” column of the table). 13. Repeat step 10-12 for each row in the table. 14. Once you have set all the desired fields click the OK button.” 15. Now click on the Preview Data tab of the pop-up box.


16. Review the information and if you are satisfied, click the Finish button. 17. A message saying “Saving a new student” will appear in the centre of your screen and you will see the students being added to the table in

the Students section of CAFAS.

This list of students will now appear for all assignments in the currently selected Delivery area.


To Add a Student Manually

1. In the Students section, type the student’s email address in the “Enter Email Address” text box at the top of the Main Window. 2. Click the Add button to the right of the text box. 3. Scroll to the bottom of the list and look in the Email column – the new student’s email address should be in the bottom row. 4. Click “TBA” in each column to type the student’s First and Last Name, Program and Class Number.

To Remove a Student

1. In the Students section, click the box in the Check column (far left column) to select the student you want to remove. 2. Click the Remove button at the top right of the Main Window.


How to Setup Standard Comments

A database of standard comments can be created prior to or during the assessment process. Thus the coordinator could pre-populate the comments database with comments that they think will be useful to the assessment team. Subsequently, assessors can add their own comments, or modify the pre-defined comments during the assessment process. By default all new comments are “public” i.e. all assessors can see and use them, however by unchecking the Public Comments box, users can create their own private list of comments that other assessors cannot see or use.

The setting up of comments is done in the Assignment Setup tab/Comments Section or the Assessment tab/Comment section.

At the top of the Main Window is a text box where you can type and edit comments. There are some basic formatting tools beneath the text box.

At the bottom of the Main Window is a list of comments that are in the database for this particular assessment criterion.

1. Click the Assignment Setup Tab (or Assessment Tab) 2. Click the Comments Section. 3. From the list in the Control Panel, select the assessment criteria category in which you want to add a new comment (or edit an existing

comment). 4. Leave the Public Comments box checked (or uncheck it if you don’t want to share the comment you are about to create with your

colleagues).


5. Type your comment in the large text box at the top of the Main Window. 6. You can use the formatting tools below the window. 7. You can also add a hyperlink by selecting a word(s) and then entering the web address in the hyperlink box adjacent the formatting tools.

Then click elsewhere in the text box to end the hyperlink insertion command. 8. When you are satisfied with the comment click the Add button. The new comment will appear in the list below. 9. To delete a comment, select the comment using the Check box in the far left column and then click the Remove button. 10. To edit an existing comment, select the comment using the Check box in the far left column. 11. Then make your changes in the text box at the top of the Main Window – when you’ve finished, click the Save button.


How to Assess a Student

The Assessment of a Student is conducted from the Assessment Tab. This is the main objective of CAFAS: to provide assessment and feedback on a student’s assignment. Using the sections of the Assessment tab you can select and assignment, select a student, add new comments to the database, and use the Feedback Form to specify marks/grades and feedback comments for each Assessment Criterion/Penalty.

From the Getting Started section you can view 3 detailed Tutorial videos which clearly explain how to use this section of CAFAS.

To view online tutorials on the Getting Started Section of the Assessment tab click on Tutorial 11

Tutorial 11 - Getting Started

1. Ensure that you have selected an Assignment in Select Assignment Section of the Assessment Tab. 2. Click the Comments section and review the comments briefly and add more comments if you want to (you can also do this directly from

the Feedback Form - while you are assessing the student). 3. Click the Select Student section, 4. Click on the name of the student you want to assess. 5. The selected student’s name will appear in the Selection Bar which will also display the Course/Subject/Unit, the Delivery and the

Assignment name. 6. Use the scroll bar to view more students’ names.


7. Click the Assess Student section to display the Feedback Form for the selected student – their name will still be displayed in the Selection Bar at the top of the Main Window to remind you who you are assessing.

8. Click on the Assess button if it is not already active (no harm in clicking it if it is already active). 9. Then start using the sliders and/or rubrics to assign grades/marks for each assessment criterion. 10. Sliders: drag the pointer (upward pointing triangle) to the desired position. The mark (out of 100) will be displayed next to the pointer and

in the mark box to the right of the slider. You can also click on the horizontal slider bar to position the pointer quickly. 11. Use the scroll bar to the right of the Main Window to review all parts of the Feedback Form.


12. Rubrics: select one of the radio buttons in the left column to designate a particular grade/mark and the associated comment (“description”).

13. After using a Slider or Rubric, ensure that you click the Save & Update Grade/Mark button in the Control Panel – this is very important. Failure to do this will mean that your settings on this student’s Feedback Form will not be saved). It is good practice to use the Save & Update Grade/Mark button after using each Slider/Rubric.

14. For both Sliders and Rubrics you can add a comment in the associated Text Box (that appears below the Slider/Rubric)


1. Click the Add Comments button which is associated with each Comments Box (except Class Feedback). A Pop-up box will appear with a list of comments for that particular assessment criterion.

2. You can drag it to a new location by clicking and dragging near the top of the box e.g. adjacent to the X symbol. 3. Click the check boxes on the left side of the list to specify which comments are to be entered into the Text Box on the Feedback Form. 4. Click the X symbol to close the pop-up box. The comments will be added to the Text Box on the Feedback Form. 5. You can then reorder the comments by highlighting them and Cutting and Pasting. 6. There are also formatting tools. 7. You can also type new comments directly into the Text Box. If you want to reuse the comment on another student’s Feedback Form you can

highlight the comment and then click the Disc Icon (Save) above the Text Box. 8. After adding a comment, ensure that you click the Save & Update Grade/Mark button in the Control Panel – this is very important.

Failure to do this will mean that your settings on this student’s Feedback Form will not be saved).


9. Review the Overall Grade and Mark in the Selection Bar at the top of the Main Window. (You must click Save & Update Grade/Mark button after moving a Slider for these values to update!)

10. Then make an overall comment in the Summary Comment Text Box (refer instructions in steps 10 – 15 above). 11. If there is a Class Feedback Text Box you can use this to type some comments to the whole class – this message appears on each

student’s Feedback Form. 12. When you are satisfied that the feedback form is complete, Click the Assessment Complete for this Student check box in the Control

Panel (this is not essential but will facilitate bulk emailing in the Publish Section). 13. Click the Save & Update Grade/Mark button in the Control Panel – this is very important. Failure to do this will mean that your settings

on this student’s Feedback Form will not be saved). A “Saving Mark” pop-up box will indicate that the Save process has been successful. If you don’t see this, refresh your internet browser and return to this Section.

14. Click Select Student section to select the next student and repeat this process for each student.


1. After using the sliders/rubrics, a summary of the selected student’s results can be viewed by clicking the Summary button in the Assess Student Section.

2. A table of grades/marks and some graphs will be displayed in the Main Window. 3. Click on the Assess button to return to the Feedback Form where the sliders/rubrics are displayed.


To view online tutorials on the Select Student section of the Assessment tab click on Tutorial 12

Tutorial 12 - Select Student

To view online tutorials on the Comments section of the Assessment tab click on Tutorial 13

Tutorial 13 - Comments

To view online Tutorials on the Assess Student section of the Assessment tab click on Tutorials 14 and 15

Tutorial 14 - Assess Student 1

Tutorial 15 - Assess Student 2

Data & Graphs section

1. Click the Assessment Tab. 2. Click the Data & Graphs Section. 3. In the Main Window you can review the overall results for all students for the selected assessment. The bar chart can be displayed on the

Feedback Form via the “Grade Graph” options in Assignment Setup/Visibility Settings.


To view an online Tutorial on the Data & Graphs section of the Assessment tab click on Tutorial 16

Tutorial 16 - Data & Graphs


How to Publish an Assessment

To Publish a completed Feedback Form, you click on the Publish section of the Assessment tab of CAFAS. The data in the table shows the student results for the selected assignment. By selecting a student from the table and clicking the 'Email Assessment Results to Students' button, you will get a pop-up box where you can confirm the sending of the selected student's results to the selected student's email address. See Publish an Assessment for further instruction on Publishing an assessment in CAFAS.

1. Click on the Assessment Tab. 2. Click on the Publish Section. 3. Review the Status column for an overview of which student’s assessments have been completed (this is driven by the Assessment

Complete for this Student check box in the Control Panel of the Assess Student section). 4. To filter the list of students to show only the students that you have personally assessed, click the Show only where I am assessor

tickbox. 5. To filter the list of students to show only the students that have incomplete assessments, click the Show only where status is

incomplete tickbox. 6. Click the Check All Completed button at the top left of the Main Window. 7. Or manually check the students that you want to send Feedback Forms to. 8. Click the Email Results to Students button in the Control Panel.

Note that the columns can be reordered by dragging (the title of the column) and dropping. The horizontal scroll bar will be needed to view all columns (unless you have a high resolution/wide screen).


9. A pop-up box will appear to manage the Emailing process. 10. Untick the Only Send to me tickbox (unless you want to run a test first in which emails will be sent to you only - instead of the students.

Not a bad idea for first time users of CAFAS). 11. If you are confident that all the Feedback Forms are ready to send (i.e. you have already reviewed them in Assess Student section), leave

Automatically send tickbox ticked. This will send the emails in bulk – this is the most efficient. Alternatively, you can send the emails one at a time, checking them as you go (see steps below).

12. Click the Start button. A “Sending Email” message will appear in the centre of the screen and the “Displaying Result X of Y” message a the bottom right of the pop-up window will index as each email is sent. When the preview area is blank and it says Displaying Result Y of Y you can close the window by clicking the X symbol at the top right of the pop-up window.


Manual Send instructions

13. If you are not confident that the information in the Feedback Forms are correct you should send them one at a time using the preview window (and scroll bar) to review each form prior to sending. Uncheck the Automatically send tickbox.

14. Click the Start button and review the preview of the form. 15. If is ok to send click Send 16. Or, if you don’t want to send it, click Skip. 17. During the emailing process the “Displaying Result X of Y” message will index as each Feedback Form is sent. 18. Check your email to ensure that you have received copies of the Feedback Forms, then archive the emails appropriately.


The Assessment Form is sent via email to the students. The email message contains all the critical information and is readily accessible to the students. However it does not contain the graphs and other graphical elements. To view the graphs etc the student needs to open the Zip file attached to email. Instructions for “extracting” the Zip file are included in the email.

To view an online Tutorial on the Publish section of the Assessment tab click on Tutorial 17.

Tutorial 17 - Publish

How to Use the Mark Book

The Mark Book Section collates all the grades/marks for each assignment and calculates a “Final Mark” based on the assignment weightings which were allocated in the Assignment Setup/Assignment Management Section.

As is typical for most of the spreadsheet tables in CAFAS the columns can be resized (drag right edge of column) and reordered (drag and drop).

The spreadsheet displayed in the Mark Book Section will be the same regardless of which assignment is currently selected.

The spreadsheet in the Mark Book can be exported in CSV format for further manipulation in spreadsheet software such as Microsoft Excel.

Export Student Results from Mark Book

1. Click the Assessment Tab. 2. Click the Mark Book Section. 3. Click the Email CSV file of Mark Book button in the Control Panel. 4. A message will pop-up in the centre of the screen: “Sending Email”. 5. After the message has disappeared, check your email. There will be an email from “CAFAS Email Service” with the Subject “CAFAS CSV

Results”. The email has a CSV file attached in a Zip file. 6. Email your students to advise them that they should have received an email from “CAFAS Email Service”.


To view an online Tutorial on the Mark Book section of the Assessment tab click on Tutorial 18

Tutorial 18 - Mark Book


About CAFAS

CAFAS is an Open Source program that is free to use in all Australian Universities. The University of South Australia is hosting CAFAS until the end of 2011 at which point participating Universities may need to host CAFAS themselves should UniSA decided to stop offering this service.

Support for this project has been provided by The Australian Learning and Teaching Council http://www.altc.edu.au an initiative of the Australian Government Department of Education, Science and Training. The views expressed in the project do not necessarily reflect the views of The Australian Learning and Teaching Council.

CAFAS stands for Computer Aided Feedback and Assessment System. Its main aims are to assist academics in providing timely, high quality feedback and assessment results to students in tertiary education institutions and to thereby improve learning outcomes. It has been designed to mimic commonly used paper-based Feedback Forms and to cater to a wide range of disciplinary assessment requirements.

There is also a CAFAS community wiki where you can find additional resources for CAFAS, contact other users, & learn more about online Feedback & Assessment. The CAFAS wiki is located at http://cafas.pbworks.com This project is led by Mr. Martin Freney from the School of Architecture and Design and Dr. Denise Wood from the School of Communication at the University of South Australia with funding from the Australian Learning & Teaching Council.

Programming, Database Design & Functionality was completed by staff in UniSA’s Flexible Learning Centre and Hew Ellwood, multi-media programmer. In particular, Michael Lewis, lead technical programmer, was instrumental in developing the interface and with programming and testing the final version of CAFAS. The FLC also provided additional project design & management support throughout the project. The FLC has been restructured as is now known as the Learning and Teaching Unit (LTU). UniSA’s Information Strategy and Technology Services (ISTS) are managing the provision of CAFAS via hosting it on servers and making it available via Shibboleth for the Australian higher education community.

Initially CAFAS will be offered as a web-based system and will be hosted by the University of South Australia. Your data will only be available to you, the colleagues that you nominate, and your students.

The licensing scheme (Creative Commons Non-commercial ShareAlike 2.5 Australia Licence) enables all universities to use CAFAS free of charge and to install and develop CAFAS for their local IT requirements.

Unless otherwise noted, content on this site is licensed under the Creative Commons Attribution-Noncommercial-ShareAlike 2.5 Australia Licence.


Acknowledgements

CAFAS Project Team

Martin Freney – Project Leader, Project Manager, Excel CAFAS Prototype, Publications, GUI Design and Testing.

Denise Wood – Teaching and learning support, Accessibility advice, Publications

Hew Ellwood - Graphic Design, Flex CAFAS Prototype, Help resources, GUI Design and Testing.

Michael Lewis - Technical Lead, CAFAS Programming, GUI Design and Testing.

Roman Muller - Database Design and Construction.

Ian Reid – Advisor.

Mark Wittervan - Management of programming staff and GUI Design.

Wayne Pedder – Management of programming staff.

Marty Friedel - Multimedia services.

Brook Schofield – Shibboleth consultant.

David Gameau – Shibboleth consultant.

Bradley Mack – Server support.

Damien Ellis – Server support.

Richard Lamb – IT Management.

Key Staff Involved in Trials

Special thanks to Sandy Walker, Peter Schumacher, Daniel McLean, David Gordon and Tim Williams and all the academics who trialled CAFAS.


FAQs

This is the Frequently Asked Questions FAQ section of CAFAS.

Q: How do I learn to use CAFAS?

A: Try the video tutorials which you can access via the Getting Started Sections in CAFAS. Also, use the Help drop down menu to access written Help resources - this includes links to online training video tutorials on the CAFAS wiki website. If you are still confused, email [email protected]

Q: What is CAFAS?

A: CAFAS is an online Feedback & Assessment system for student assessment in tertiary education. It mimics the typical paper-based Feedback Forms that are typically used in many disciplines of higher education.

Q: What does the name CAFAS stand for?

A: CAFAS is an acronym for Computer Aided Feedback & Assessment System.

Q: Is CAFAS free to use?

A: Yes, CAFAS is an open source project meaning that it is free for use. See the License in the About menu in CAFAS for more details.

Q: Is CAFAS easy to learn?

A: Yes CAFAS is easy to learn. You can quickly be creating Feedback Forms and assessing student work with this system. If you already have Feedback Forms (that are based on weighted assessment criteria) in Word doc format then you can copy and paste much of this information into CAFAS to quickly generate your CAFAS Feedback Form.

Q: Will CAFAS save me time with my assessments?

A: This is one of the aims of CAFAS. By automating many administration tasks (e.g. collating grades, adding up marks etc) and by providing features such as quick insertion of feedback comments, repetition and tedium are overcome. More importantly, there is the opportunity to provide high quality written feedback straight to the student’s inbox - where they are less likely to ignore it.

Q: Does CAFAS save student assessment results for future use?

A: Yes, CAFAS saves student assessment results to a secure database at UniSA for future reference (up until the end of 2011 when the system will be reviewed).


Q: When will CAFAS be available?

A: CAFAS is available as of July 2009. Please email [email protected] for details about how to access CAFAS. CAFAS will be re-evaluated at the end of 2011 by UniSA’s ISTS to determine if it will be developed or retired, however the CAFAS source code can be downloaded from the project website and installed locally within your institution.

Q: What do student’s think of CAFAS?

A: Student’s like CAFAS because it is a convenient way of receiving feedback and assessment results. In response to the question "Digital Feedback and Assessment Sheets were emailed to you (PDF file) for each assessment. What are the benefits/disadvantages of this new system for providing feedback and calculating assessment?" students commented:

"Convenience - I can receive them at home instead of going to Uni. Detailed comments were great, so were the graphs".

"This was a really good method of feedback. They provided in depth explanation of all facets of the assignments which enabled you to see exactly where you went wrong, or what could be improved".

"It was fantastic to receive such comprehensive feedback. Since I spent a lot of time on ensuring my assignments were at a high standard, it was nice to know that course staff made the effort to undertake a detailed review of my assignments and provide valuable feedback".

"This was very beneficial and excellent feedback! Just having a single comment and a score isn't very helpful, but having this digital feedback explains every assessment criteria, as well as percentage weightings, the grade and comments. This feedback should be kept this way".

"This was genuinely useful in seeing where criteria was and wasn't met and what to improve or look out for in future assessments".

"Yes, this was a new way of providing feedback to students. The assessment sheets themselves were very comprehensive and allowed for thorough, detailed feedback, which is greatly appreciated. I think overall, the system was beneficial to students, it was just different to see graphs and charts on a marking sheet. Different, but comprehensive = good".


Glossary

Term Description

Admin Setup (tab) The first tab in CAFAS is Admin Setup. From here the initial users, permissions, institution and relevant details for a course, delivery and assessment are set. Admin Setup is only available to users with Coordinator level permissions.

Assessment (tab) The Assessment tab is the third main tab in the CAFAS interface. This is where the actual assessment of students grades occurs.

Assessment Assessment is also a term used throughout CAFAS to describe the process of evaluating a student's assignment. It is also used to mean “assignment”.

Assessor An Assessor is a user with access permissions to only the Assessment tab in CAFAS. Generally an Assessor would be a Tutor or Lecturer at a university for example.

Assignment An Assignment is another name for a project, report, test or other assessment task. Assignment Setup (tab)

The Assignment Setup tab is the second main tab in the CAFAS interface. From Assignment Setup a Coordinator can design Feedback Forms.

Control Panel The Control Panel is the vertical panel on the left hand side of the CAFAS GUI. From the Control Panel most of the initial actions take place in the assessment process.

Coordinator A Coordinator is a user with access to all 3 tabs (Admin Setup, Assignment Setup and Assessment tabs) in CAFAS. Course A Course might also be referred to as a “subject” or “unit” in other institutions.

Delivery A Delivery is a time or a place when/where the course is offered. Feedback Form A Feedback Form is a proforma used to record feedback for students during the assessment process.

Grade A Grade represents a range of marks.

Grade Graph The Grade Graph is a simple bar graph that displays the overall performance of the class in terms of the total number of each grade.

Grade Scheme The Grade Scheme is a term which refers to the name given to the marking scheme for assessments in any particular institution.

Graduate Attribute A Graduate Attribute is the term given to describe general areas of skill and knowledge that will be developed by a particular assignment.

Institution An Institution is defined as a university or government agency involved in the education process of students.

Main Window The Main Window is the large main white area on the centre to the right hand side of the CAFAS GUI. This area automatically updates according to the selection in the Control Panel.

Mark A Mark is a score out of 100 used to assess the quality of a students performance in an assignment.

Mark Book The Mark Book section is a table which collates the marks for all aspects of an assignment and computes a final mark based on the weighting specified for each assignment.

Rubric The Rubric is a list of comments categorised by performance level for a specific assessment criterion.


Shibboleth A middleware system that enables users to login to CAFAS from any computer using their university username and password.

Slider A Slider is a sliding scale from 0 to 100 which is used to allocated a mark for an assessment criterion.

Selection Bar The Selection Bar is the thin horizontal section near the top of the CAFAS interface. The Selection Bar displays the selected Course, Delivery, Assignment and Student. The aim of the Selection Bar is to provide users with an reminder of what assignment and which student is currently selected. This avoids any confusion for the user, keeping the critical information for the assessment in full view of the user at all times.

User A User as defined in CAFAS is simply someone, generally a Tutor, Lecturer or Course Coordinator that has logged in to CAFAS via the secure Shibboleth login.

cafas online user guide

Documents