MANAGE AN ASSESSMENT SYSTEM FOR TRAINING OUTCOMES

D1.HRD.CL9.08D1.HHR.CL8.06

Slide 1

Element 1: Communicate the assessment system

Performance criteria are:

Define the purpose of the assessment system

Document and circulate assessment system features and procedures to relevant stakeholders

Establish procedures for keeping relevant stakeholders informed about key features of the assessment system

Slide 2

Define the purpose of the assessment systemSome purposes for developing an assessment system may include:

Recognising or determining the current competencies or skills of staff

Identifying training needs of staff

Checking that workplace skill levels are being maintained

Issuing internal certificates of achievement or awards for excellence

Recognising progress in learning or achievement of desired learning outcomes

Identifying areas for improvement or change

Maintaining current skill levels of staff Slide 3

Document and circulate assessment system features and procedures to relevant stakeholders

Slide 4

Does

Shows How

Knows How

Knows

Performance/Action

Competence

Document and circulate assessment system features and procedures to relevant stakeholders (cont’d)

There are many different parts to an assessment system and they may include:

An ability to determine the required competency levels and qualifications of assessors

Policies and procedures for obtaining, updating and storing records and information of candidates and assessors

How to conduct assessment including the timing and duration specifics

Flexibility in order to respond to individual needs of participants who may have particular challenges such as physical or language and literacy needs

Slide 5

Document and circulate assessment system features and procedures to relevant stakeholders (cont’d)

There are many different parts to an assessment system and they may include:

A review and evaluation process to determine if the assessment system is effective or needs improvement

Policies and procedures that allow for feedback including, grievances, moderation, re-assessment

A statement relating to competency levels and remuneration levels appropriate to those skills attained

Slide 6

Document and circulate assessment system features and procedures to relevant stakeholders (cont’d)

There are many different parts to an assessment system and they may include:

Outline country specific occupational safety and health regulations, employment requirements and equal opportunity legislation

Procedures for recognising previous educational qualifications or experience gained at other training organisations or employment contexts

Information about where assessment materials are located and where assessment can take place

Slide 7

Document and circulate assessment system features and procedures to relevant stakeholders (cont’d)

Some of the relevant stakeholders may include:

Department heads, team leaders, managers and the business owners

Participants and candidates from both current learners and past learners

Trainers and assessors. External assessors and trainers may also need to be informed of the assessment system especially if there is an external audit process or certification requirements

Slide 8

Document and circulate assessment system features and procedures to relevant stakeholders (cont’d)

Some of the relevant stakeholders may include:

Technical experts who may be required to provide feedback about the relevance and currency of skills and assessment

Training and assessment coordinators

Government and industry bodies related to the assessment components

Slide 9

Establish procedures for keeping relevant stakeholders informed about key features of the assessment system

There are several ways to keep stakeholders informed about assessment system features including

Scheduling regular meetings that relate directly to assessment issues

Special meetings which may be necessary at various times due to urgent issues or feedback

Regular staff meetings having a standing agenda item included that allows for feedback and comments regarding assessment issues

Slide 10

Establish procedures for keeping relevant stakeholders informed about key features of the assessment system

There are several ways to keep stakeholders informed about assessment system features including

The organisation of a dedicated newsletter or fact sheet that provides updates on assessment system changes or feedback

Developing an internal network of assessors who can communicate directly with each other and discuss particular issues that may be relevant to the assessment system delivery and effectiveness

Slide 11

Element 2: Provide support for enterprise assessors

Performance criteria for this unit include:

Verify assessors meet required competency standards

Identify required training for assessors

Provide for moderation of assessments

Provide feedback to assessors on their performance

Facilitate networking amongst assessors

Slide 12

Verify assessors meet required competency standards

In order for assessors to be able to assess effectively, it is critical that they have the necessary skills and experience required to assess participants.

If the assessors have limited experience in the assessment item, then the assessment may not be a true reflection of the participant’s ability in that skill.

It is important then to check all assessors experience and qualifications to ensure that they have the necessary skills to be able to conduct the assessment properly

Slide 13

Verify assessors meet required competency standards cont’d

The rules of evidence:

Valid

Sufficient

Currency

Authentic

Slide 14

Identify required training for assessorsAfter checking qualifications and experience of assessors, it may be necessary to also observe and interview them. If the assessors do not meet the required competency standards, then it may necessary to train them to fill any gaps in their skills.

Slide 15

Identify required training for assessors

Checking competency:

Skill

Knowledge

Attitude

Slide 16

Provide for moderation of assessments

Moderation is a process where you can compare your own assessment activities with that of other people and evaluate your methods and procedures.

It is concerned in particular with bringing assessment judgements and standards into alignment, thus ensuring the same standards are applied to all assessment results within the same unit of competency.

Slide 17

Provide feedback to assessors on their performance

Slide 18

The competency standard being assessed

Critical evidence required

Evidence collected

Does it relate to the

standards being

assessed? (Valid)

Is there enough of

it? (Sufficient)

Is it up to date? (Current)

Is it the candidates own

work? (Authentic)

Provide feedback to assessors on their performance cont’d

Analysing their assessments, including results and materials/tools as part of the assessment process

Developing feedback instruments for candidates to complete following assessments

Providing observation sessions while assessors conduct assessments

Slide 19

Provide feedback to assessors on their performance cont’d

Identifying specific foci for evaluation and feedback

Encouraging self-evaluation as part of the feedback process

Recognising effort and positive outcomes

Supporting assessors where feedback identifies negative outcomes

Slide 20

Facilitate networking amongst assessors

Networks allow assessors to:

Learn, discuss and share information about assessment issues

Promote the best standards for assessment

Explore ways to improve the assessment system

Learn from external assessment agencies

Keep informed about new changes or standards

Keep engaged with their assessment responsibilities

Slide 21

Facilitate networking amongst assessors (cont’d)

Networks can be developed in many different ways and may include:

Formal associations

Informal groups

Internal organisations within a company

External agencies linking groups of assessors

Meetings both regular and as required

Slide 22

Facilitate networking amongst assessors (cont’d)

Networks can be developed in many different ways and may include:

Seminars and workshops

Conferences

Notice boards, handouts, newsletters

Electronically through websites, emails, blogs

Slide 23

Element 3: Manage the assessment record keeping system

Performance criteria for this element include:

Identify and develop the internal records required to support the assessment system

Describe the requirements for completing the assessment records

File the assessment records

Review and up-date the assessment record keeping system

Slide 24

Identify and develop the internal records required to support the assessment system

What information needs to be recorded?

Slide 25

Identify and develop the internal records required to support the assessment system

Internal records could include:

Personal information relating to candidates

Evidence that required training has been delivered prior to assessment being attempted

Assessment tools and assessment items

Copies of relevant training curricula/training plans

Training rolls

Evidence captured as part of the assessment process

Documentation used to record the resources used and costs involved as part of the assessment process

Slide 26

Describe the requirements for completing the assessment records

Describing requirements for completing records may include:

Demonstration where an assessor trainer or supervisor completes a form with the assessor

Written guidelines that are distributed to all assessors

Providing examples or templates that show how to complete the records

Online tutorials with examples

FAQ sheets

Monitoring the compilation of records completed by assessors on an ongoing basis where feedback can be provided if required

Slide 27

File the assessment records

Who needs access to the assessment records

Assessors

Managers

External agencies

Training providers and trainers

Candidates or staff

Administrators

Assessment coordinators

Slide 28

File the assessment records (cont’d)

Why do they need access to the records?:

To update assessment records

To evaluate assessment procedures and policies

To provide feedback to candidates, staff, managers, external agencies, training providers and trainers

To check learning and qualification progress

To check work performance or promotion potential

To identify gaps in training or skills

Slide 29

Review and up-date the assessment record keeping systemA review mechanism should include:

Dates and time of review

Items for review

Data/information/feedback gathered

Issues or matters for attention

Who conducted the review

Recommendations for improvement

Date for next review

A mechanism for communicating updates or changes to the assessment record keeping system to relevant stakeholders

Slide 30

Review and up-date the assessment record keeping system

Slide 31

Improved Assessment

System

Conduct Consultations

with Stakeholders

Gather Evidence

Develop Implementation

Strategy

Implement Improvements

Ongoing Evaluation

Review Cycle

Element 4: Maintain quality assurance proceduresPerformance criteria for the element include:

Define quality assurance procedures for the assessment system

Undertake internal audits of the assessment system

Use the results of quality assurance audits to improve the assessment system

Review the applicability of the existing quality assurance procedures to the dynamic nature of the assessment system

Slide 32

Define quality assurance procedures for the assessment system

Validation is a quality review process.

It involves checking that the assessment tool produces valid, reliable, sufficient, current and authentic evidence to enable reasonable judgments to be made as to whether the requirements of the relevant aspects of the training package or accredited course have been met.

It includes reviewing and making recommendations for future improvements to the assessment tool, process and/or outcomes.

Slide 33

Define quality assurance procedures for the assessment system (cont’d)There are many reasons why organisations need to review including:

Confirming the credibility and recognition of certification and awards

Supporting the industry and community recognition of training participants

Providing the best service for stakeholders - learners and industry

Ensuring assessments reflect changes in current industry requirements

Improving the validity, reliability, flexibility and fairness of assessments

Slide 34

Define quality assurance procedures for the assessment system (cont’d)There are many reasons why organisations need to review including:

Providing greater justice for participants for assessment with improved relevance, transparency and support in assessment processes

Ensuring more meaningful feedback and guidance for students after they have been assessed

Supporting ongoing organisational quality assurance

Enhancing the reputation and recognition of the organisation and its services

Slide 35

Define quality assurance procedures for the assessment system (cont’d)There are many reasons why organisations need to review including:

Minimising risks associated with the assessment process

Improving the management of assessment systems

Informing assessor selection and guiding professional development

Slide 36

Define quality assurance procedures for the assessment system (cont’d)

There are certain areas that need to be outlined in implementing quality assurance procedures.

Systematically collecting and analysing data

Ensuring that data is relevant and sufficient

Data sources relevant to improving training and assessment

Demonstrating improvements

Slide 37

Undertake internal audits of the assessment systemIn developing quality assurance procedures for the assessment system there are five targets for validation that can be identified which should be included in an internal audit. They include:

Assessment systems

Assessment processes

Assessment methods and tools

Assessment evidence

Assessment judgments

Slide 38

Undertake internal audits of the assessment system cont’d

Slide 39

Section for audit Specific area for audit StandardsResult and

recommendationsRisk

Assessment systems  

Administrative proceduresRecording assessment

results

Participant personal details completed

Compliant: All samples checked were completed correctly. Maintain regular

monthly checks.

Low  

Assessment records for each participant

completed and up to date.

Non-compliant: All but one sample checked was up

to date and fully completed. Participant

1452 sample was missing assessor name. This

should be completed and regular checks continued.

Medium  

Use the results of quality assurance audits to improve the assessment system

There are three fundamental processes to identify opportunities for continuous improvement. These are:

Reviewing

Moderating

Evaluating

Slide 40

Review the applicability of the existing quality assurance procedures to the dynamic nature of the assessment system

It is important to ensure that quality assurance procedures are responsive to the dynamic nature of the assessment system. It is necessary to check:

That all aspects of the assessment system are being checked and evaluated regularly

The quality assurance procedures areflexible enough to still be able to reviewthe system even if change occurs

Slide 41

Element 5: Report on the contribution made by the assessment system to enterprise training and performance

Performance criteria for this element include:

Identify the outcomes that the assessment system has produced

Review the assessment system

Make recommendations for change to the assessment system

Produce and circulate a report

Revise the existing assessment system on the basis of responses to the report

Slide 42

Identify the outcomes that the assessment system has producedIt is important to analyse the results or outcomes and identify:

Whether the assessment system has been effective in achieving the desired outcomes of the system as a whole

The benefits that the assessment system has brought to the organisation by noting the new skills attained, the amount of certificates issued, the increased opportunities and performance that the system has provided to the enterprise

Slide 43

Identify the outcomes that the assessment system has produced

It is important to analyse the results or outcomes and identify:

Increased professional development opportunities that have occurred due to the implementation of the assessment system

Whether there were some areas of strength within the organisation that had not previously been acknowledged

Increased staff engagement, productivity or workplace satisfaction

Slide 44

Review the assessment system

The review should analyse all areas of the system including:

Assessment systems

Assessment processes

Assessment methods and tools

Assessment evidence

Assessment judgments

Slide 45

Make recommendations for change to the assessment systemRecommendations for change could include:

Whether to continue with the existing assessment system with minor alterations or to cease with the current system in favour of an external provider administering an assessment system

Making recommendations relating to staff resources, training, payment, costs, assessment content or administration procedures

Expanding the existing system or reducing the scope of the existing assessment system

Recommending changes to location and timing of the assessment system

Slide 46

Produce and circulate a report

A report should include:

Introduction

Main body

Appendices, references and glossary

Slide 47

Produce and circulate a report (cont’d)

It is important to identify all of the stakeholders they may need to receive the report and these could be:

Assessors

Managers

External agencies

Training providers and trainers

Candidates or staff

Administrators

Assessment coordinators

Slide 48

Revise the existing assessment system on the basis of responses to the report

Revisions to the assessment system could include:

Revising policies and procedures relating to the assessment system

Altering the assessment tools to make them more relevant and appropriate

Revising the assessors that have been used which may mean increasing or decreasing numbers, choosing assessors with different skill sets and experience, or providing more training for current assessors

Slide 49

Revise the existing assessment system on the basis of responses to the report (cont’d)

Revisions to the assessment system could include:

Revising the assessment times and locations to suit organisational needs

Communicating any changes that have been made to the assessment system and making sure that all relevant stakeholders are informed of these changes or updates

Slide 50