act enhancing the quality of training & assessment - v1.2 - 11.06.2010

The Shift in Learning & Development www.paradigm.edu.au

Enhancing Assessment Practice

ACT Assessors’

Professional Development Day

Presented by Greg Weis

Friday, June 11, 2010


Assessment

How would you define the term “assessment”?

Definition: “Assessment in training is about measuring an individual’s ability to see if they have reached the stated outcomes of a training program…Assessment is based upon ‘the gathering and judging of evidence about the performance of individuals’.” [italics and bold added]

Tovey, M., & Lawlor, D., (2008). Training in Australia. (3rd Edition).


Assessment

What implications for assessors?

“…high levels of professional expertise are needed to decide how to develop competence and how to make holistic, context-sensitive judgements about whether it has been achieved.” [italics and bold added]

Schofield and McDonald (2004, p. 4)


Assessment

Figure 1

Yes No

Indirect Life’s experiences

Reports

Records

Purposes Certification

RPL Extent of learning

Types Diagnostic Formative

Summative Methods

Knowledge Skills

Attitudes

Assessment

Qualification or

Statement of Attainment

issued

Learning

Direct Teacher facilitated Learner instigated

Judgement

Competent?

Interpretation

Evidence

N.B. For the sake of illustration, The ‘Evidence Cycle’ part of the above diagram, has been shown as a separate process to assessment, whereas in reality it is actually a part of the assessment process.


Assessment Validation & Moderation


Quality of Assessment Practices

http://www.nqc.tvetaustralia.com.au/nqc_publications


Assessment Moderation

What do we mean by the term “moderation”?

NQC definition: “Moderation is the process of bringing assessment judgements and standards into alignment. It is a process that ensures the same standards are applied to all assessment results within the same Unit(s) of Competency. It is an active process in the sense that adjustments to assessor judgements are made to overcome differences in the difficulty of the tool and/or the severity of judgements.” National Quality Council, (2009). A Code of Professional Practice for Assessment Validation and Moderation. Page 6 & 7.



What does “moderation” mean in plain English? Purpose / Definition:

• Allows for adjustments to be made before the final determination of a result.

• To determine, based on the evidence before another qualified assessor, that the assessment judgement is sound. In other words, another qualified assessor would have reached the same conclusion or judgement had he/she be presented with the same evidence.



Why?

1. Best Practice (We work for a Quality RTO)

2. Promoted by NQC (National Quality Council)

3. Benefits students

4. Benefits assessors



Reliability

Inter-rater reliability: Consistency of judgement by different assessors, using the same assessment tool and procedure.

Intra-rater reliability: Consistency of judgement across time and location, by the same assessor using the same assessment tool.

Definitions courteous of Gillis, S., & Griffin, P., (2008). Competency Assessment. In J. Athanasou (Ed.), Adult Education and Training.



That the original assessor and the moderating assessor need to reach consensus on the final determination of the assessment result.

Implications? • Competent / Not Yet Competent • Graded assessment – TAFE. Pass, Credit, Distinction.

What does “Inter-rater” reliability mean in plain English?


Assessment Validation

What do we mean by the term “validation”?

NQC definition: “Validation…involves checking that the assessment tool produced valid, reliable, sufficient, current and authentic evidence to enable reasonable judgements to be made as to whether the requirements of the relevant aspects of the Training Package or accredited course had been met. It includes reviewing and making recommendations for future improvements to the assessment tool, process and/or outcomes.” National Quality Council, (2009). A Code of Professional Practice for Assessment Validation and Moderation. Page 6. [bold added]



What does “validation” mean in plain English? Purpose / Definition:

To check that the assessment tool ‘mirrors’ the unit of competency.

This includes, as a minimum:

• Elements & Performance Criteria

• Required Skills & Knowledge; Attitude of the student

• The Critical Aspects of Assessment / Competency

In short…It assesses what it claims to assess



Construct Validity

Criterion related Validity

Face Validity

Consequential Validity

Content Validity

Five ‘facets’ of Validity

Face Validity: The extent to which the assessment task resembles real workplace activities.

Content Validity: How well the content of the assessment task reflects the required knowledge and skills specified in the Unit of Competency. Definitions courteous of Gillis, S., & Griffin, P., (2008). Competency Assessment. In J. Athanasou (Ed.), Adult Education and Training.

Validity



“Our project was based on…[a] broader approach to validity which sees it as the extent to which the interpretation and use of an assessment outcome can be supported by evidence.” [Italics in original] Thomson, Saunders and Foyster (2001, p. 10)

Validity

“Validity is the main assessment game––some would say the only game.” Thomson, Saunders and Foyster (2001, p. 10)



https://vetinfonet-staging.det.wa.edu.au/progdev/resources.aspx#Assessing%20competence



How can you assist your RTO with Content & Face validity? Feedback on assessment tasks

• Know the Units of Competency inside-out. Evidence Guide.

• Trial different assessment methods

• Realistic scenarios for assessment

• Role plays

• Know the workplace task/s – make assessments realistic.

• Realistic scenarios for assessment

• Role plays


Assessors’ Competencies

Implementation of AQTF 2010 circa 04/2011

(a) have the necessary training and assessment competencies as determined by the National Quality Council or its successors, and

1.4 Training and assessment is delivered by trainers and assessors who:

(b) have the relevant vocational competencies at least to the level being delivered or assessed, and (c) can demonstrate current industry skills directly relevant to the training/assessment being undertaken and (d) continue to develop their VET knowledge and skills as well as their industry currency and trainer/assessor competence.


Assessor’s Competencies

Implications for assessors?

• May need to work “voluntarily”

• May have to attend DET workshops on assessment practice

Demonstrate currency

• Professional reading

VET Knowledge & Skills

• Get back into the workplace

• Industry workshops / shows

• Professional reading • May have to attend DET workshops on assessment practice • Professional reading


In Conclusion…

Assessment is the ‘pointy end’ learning

It’s all about outcomes for the students!

Assessment is on the ‘radar’ of the regulators

Remember…

• Learning & Development professionals hone and enhance their assessment practice

Questions?


References Gillis, S., & Griffin, P., (2008). Competency Assessment. In J. Athanasou (Ed.), Adult Education and Training. Terrigal, NSW: David Barlow Publishing.

National Quality Council, (2009a). A Code of Professional Practice for Assessment Validation and Moderation. Melbourne, VIC: TVET Australia.

National Quality Council, (2009b). Implementation Guide: Validation and Moderation. Melbourne, VIC: TVET Australia.

National Quality Council, (2009c). Guide for Developing Assessment Tools. Melbourne, VIC: TVET Australia.

Schofield, K., & McDonald, R., (2004). High Level Review of Training Packages: Working Paper 7: Supporting quality teaching, learning and assessment. Brisbane: ANTA.

Thomson, P., Saunders, J., & Foyster, J., (2001). Improving the validity of competency-based assessment. Adelaide: NCVER.

Tovey, M., & Lawlor, D., (2008). Training in Australia. (3rd Edition). Frenchs Forest, NSW: Pearson Education Australia.

act enhancing the quality of training & assessment - v1.2 - 11.06.2010

Documents