specific and generic performance indicators for measuring
TRANSCRIPT
WAJID HUSSAIN DIRECTOR OFFICE OF QUALITY & ACCREDITATION FACULTY OF ENGINEERING ISLAMIC UNIVERSITY https://in4obe.org/
Specific and Generic Performance Indicators for
Measuring Learning Outcomes PIS & HYBRID RUBRICS DEVELOPMENT GUIDE AND WORK BOOK
© 2017 WAJID HUSSAIN
WAJID HUSSAIN: SPECIFIC AND GENERIC PIs TO MEASURE COURSE OUTCOMES WORKSHOP GUIDE & PRACTICE BOOK
1 | P a g e © 2020 WAJID HUSSAIN
Table of Contents A. TITLE ................................................................................................................................................................................ 2
B. DESCRIPTION .................................................................................................................................................................. 2
C. OUTCOMES ..................................................................................................................................................................... 2
D. REQUIRED RESOURCES AND MATERIALS ...................................................................................................................... 2
E. SCHEDULE OF ACTIVITY ................................................................................................... Error! Bookmark not defined.
F. PRESENTER BIOGRAPHY ................................................................................................................................................. 3
G. TABLE OF INPUTS ............................................................................................................................................................ 4
H. COURSE OUTCOMES & PERFORMANCE INDICATORS DEVELOPMENT TABLE .............................................................. 5
I. HIGH EXPECTATIONS: REQUIRED STEPS FOR GIVEN LEARNING ACTIVITY PI_1 ........................................................... 7
J. HIGH EXPECTATIONS: REQUIRED STEPS FOR GIVEN LEARNING ACTIVITY PI_2 ........................................................... 8
K. HIGH EXPECTATIONS: REQUIRED STEPS FOR GIVEN LEARNING ACTIVITY PI_3 ........................................................... 9
L. HYBRID RUBRIC DEVELOPMENT PI_1 .......................................................................................................................... 10
M. HYBRID RUBRIC DEVELOPMENT PI_2 .......................................................................................................................... 11
N. HYBRID RUBRIC DEVELOPMENT PI_3 .......................................................................................................................... 12
APPENDIX ............................................................................................................................................................................. 13
WAJID HUSSAIN: SPECIFIC AND GENERIC PIs TO MEASURE COURSE OUTCOMES WORKSHOP GUIDE & PRACTICE BOOK
2 | P a g e © 2020 WAJID HUSSAIN
A. TITLE
Specific and Generic Performance Indicators for Measuring Learning Outcomes
B. DESCRIPTION
Culminating student exit outcomes map to Bloom's 3 learning domains in various proportions. The measurement of affective, cognitive and psychomotor domains student learning activities require specific and/or generic performance indicators and their rubrics for appropriate assessment. Program outcomes covering all 3 learning domains in high proportions need specific/generic performance indicators for proper assessment. This workshop presents specific examples related to developing specific, generic performance indicators for the comprehensive measurement of program student outcomes. Participants will have hands on experience in the development of course outcomes, specific or generic performance indicators relating to program outcomes that implement scientific constructive alignment targeting required teaching/learning strategies and assessments. In summary, this workshop presents the benefits of using a combination of specific, generic performance indicators to accurately assess student learning activities related to the program student outcomes while supporting principles of authentic OBE, scientific constructive alignment, efficient performance failure analysis and continuous quality improvement. The learning model implemented in this workshop facilitates automation of assessment data using state of the art digital technology.
C. OUTCOMES Participants of various learning activities offered in this workshop will be learn how to:
1. Write Course Outcomes (COs) according to required progression of learning covering all major topics of the course
2. Write generic or specific Performance Indicators (PIs) aligned to Program Outcomes (POs) and targeting student performances corresponding to Bloom’s 3 domains and appropriate learning levels
3. Achieve proper coverage of Bloom’s Taxonomic learning domains and their levels in courses 4. Tightly align target student learning activity with assessments and teaching/learning strategy 5. Identify course outcomes that have gross semantic and syntactic errors
D. REQUIRED RESOURCES AND MATERIALS 1. Degree plan 2. International QA Agency Graduate Attributes 3. QA Agency Program Outcomes (POs) 4. Course Syllabus 5. Course lessons and lecture notes 6. Textbook and/or reference 7. All major assessments soft or hard copies 8. Working computers 9. Writing paper and necessary stationery 10. At least 10 copies of sections G and H of this workbook
WAJID HUSSAIN: SPECIFIC AND GENERIC PIs TO MEASURE COURSE OUTCOMES WORKSHOP GUIDE & PRACTICE BOOK
3 | P a g e © 2020 WAJID HUSSAIN
E. PRESENTER BIOGRAPHY
Wajid Hussain is a US scientist and world expert on authentic OBE, QA processes, outcomes assessment and
program evaluation for accreditation using digital technology and software [ https://in4obe.org/obe-expert/ ]. He
joined the academic field coming from an intensive Silicon Valley engineering background and more than 20
years industry experience of mass production expertise of a Billion Dollar Microprocessor Manufacture Life
Cycle. Over the many years, Wajid has managed several projects related to streamlining operations with
utilization of state of the art technology and digital systems giving him significant experience working with ISO
standard quality systems. He has received specialized Quality Leadership Training at LSI Corporation and also
received an award LSI Corporation Worldwide Operations Review 1999 for his significant contributions to the
Quality Improvement Systems. He was the lead product engineer supporting the Portal Player processor for
Apple’s IPOD plus many other world famous products at LSI Corporation.
In academics, Wajid has extensive experience supporting and managing outcomes assessment and CQI processes
to fulfill regional and ABET accreditation requirements for several EAC, CAC and ETAC programs. Wajid has
developed several revolutionary outcomes assessment instruments and models based on John K. Etsell’s FCAR
and the first on the planet to classify specific PIs as per all 3 domains and learning levels of Bloom’s taxonomy
and implement world class best assessment practices to support digital technology, automation, streamlining and
CQI. He led the first ‘tuning’ efforts in the Middle East at the Faculty of Engineering, Islamic University by
developing a complex database of thousands of outcomes and performance indicators with hundreds of rubrics
for the Electrical, Mechanical and Civil Engineering disciplines. A comprehensive automated ISO standards
Quality Systems CQI model for program evaluation with precision assessment instruments for capstone design,
team work, lifelong learning, ethics etc. was developed and implemented. The Faculty of Engineering, Capstone
Committee worked several hundreds of hours with Wajid to produce a revolutionary Capstone Design Course
Assessment Instrument. Wajid recently presented this Capstone Design Activity Assessment model at the ASEE
International conference 2018 at Salt Lake City, Utah. Wajid also worked closely with Dr. Mak Fong, Director
Makteam Inc. to implement ground breaking academic advising systems based on outcomes using digital
technology.
Wajid is currently reviewer for several international conferences on topics related to education & research
methods, outcomes assessment, quality and accreditation. Wajid has presented multiple world class research
papers on outcomes assessment and automation at the ASEE, FIE and other international conferences. The
National Institute of Learning Outcomes Assessment (NILOA) has published Wajid’s work at the Faculty of
Engineering at the Islamic University as an international example of best outcomes assessment practices. Wajid
Hussain and William Spady, internationally regarded as the ‘Father of OBE’, have recently co-authored a book
titled ‘Beyond Outcomes Accreditation’.
Wajid has been invited keynote or presenter in more than 40 international OBE and education conferences. Some
notable events where he presented are the ICA 2015, MTN 2016, OBE ICON 2016, FIE 2016, ASEE 2016, ASEE
2017, ICTIEE 2017, ABET Symposium 2017, IICEDubai 2018, QS ASIA 2018, ASEE 2018, EDUTECH 2018,
APAC STEM 2018, ICEE 2018, QS ASIA 2019, EDUTECH 2019, QS ASIA 2020, EDUTECH 2020. Wajid is
also part of the organizing committee for the OBE GLOBAL SUMMIT 2021. William Spady and Wajid have
also been recognized as international OBE experts in 2017 by Dr. Faisal Al Mishary, CEO National Assessment
Center (NAC) QIYAS, Riyadh. He received top award in December 2018 for best scientific poster for
“Management and Assessment of capstone Design Courses” at the International Conference on Education
Evaluation from Dr. Faisal Al Mishary, CEO National Assessment Center (NAC) QIYAS, Riyadh. Wajid has
conducted multiple massively participated workshops attended by several hundreds of faculty members in Saudi
Arabia such as the ones in 2018 at the Islamic University, Prince Muqrin, KFUPM (previously Hafr al Batin
branch) and King Faisal Universities. Wajid is a senior member of the IEEE, IEEE Education Society, Member
of the Assessment of Learning in Higher Education (AALHE), leading member of the International Network of
OBE Experts worldwide (IN4OBE) and member of the American Society of Engineers for Education (ASEE).
WAJID HUSSAIN: SPECIFIC AND GENERIC PIs TO MEASURE COURSE OUTCOMES WORKSHOP GUIDE & PRACTICE BOOK
4 | P a g e © 2020 WAJID HUSSAIN
F. TABLE OF INPUTS
PROGRAM QA COMMISSION:
COURSE TITLE CREDIT HOURS :
GRADUATE ATTRIBUTES
1. 2. 3. 4. 5. 6.
PROGRAM SOs
1. 2. 3. 4. 5. 6. 7. 8. 9. 10.
MAJOR LEARNING ACTIVITIES
1. 2. 3. 4. 5. 6. 7. 8. 9.
PROGRAM SOs COVERED
SO ‘1’ SO ‘2’ SO ‘3’ SO ‘4’ SO ‘5’ SO ‘6’ SO ‘7’ SO ‘8’ SO ‘9’ SO ‘10’
TEXT & REFERENCE
CHAPTERS 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13. 14.
WAJID HUSSAIN: SPECIFIC AND GENERIC PIs TO MEASURE COURSE OUTCOMES WORKSHOP GUIDE & PRACTICE BOOK
5 | P a g e © 2020 WAJID HUSSAIN
G. COURSE OUTCOMES & PERFORMANCE INDICATORS DEVELOPMENT TABLE
1 CHAPTER (Select relevant chapters for one CO)
1. 2. 3.
LECTURE HOURS:
2 TOPICS (Select relevant topics for one CO)
1. 2. 3. 4. 5. 6.
3 TEACHING STRATEGIES (for various learning activities related to this CO)
Learning Activity Teaching Strategy/Assessment
1.
2.
3.
4.
5.
6.
4 PROGRAM SO COVERAGE (Select all that apply) SO ‘1’ SO ‘2’ SO ‘3’ SO ‘4’ SO ‘5’ SO ‘6’ SO ‘7’ SO ‘8’ SO ‘9’ SO ‘10’
Cognitive Domain Remembering Understanding Applying Analyzing Evaluating Creating
Affective Domain Receiving Responding Valuing Organizing Values into Priorities Internalization
Psychomotor Domain Perception Set Guided Response Mechanism Complex Overt Response Adaptation Origination
5 COURSE OUTCOME (original) Operational Verbs Nominal Content/Topics Lab Project Team Work
6 COURSE OUTCOME (modified)
Operational Verbs Nominal Content/Topics Lab Project Team Work
7 PERFORMANCE INDICATOR 1
Operational Verb Nominal Content/Topics Techniques/Methods Major Steps
WAJID HUSSAIN: SPECIFIC AND GENERIC PIs TO MEASURE COURSE OUTCOMES WORKSHOP GUIDE & PRACTICE BOOK
6 | P a g e © 2020 WAJID HUSSAIN
PROGRAM SO COVERAGE (Select one only) SO ‘1’ SO ‘2’ SO ‘3’ SO ‘4’ SO ‘5’ SO ‘6’ SO ‘7’ SO ‘8’ SO ‘9’ SO ‘10’
Cognitive Domain Remembering Understanding Applying Analyzing Evaluating Creating
Affective Domain Receiving Responding Valuing Organizing Values into Priorities Internalization
Psychomotor Domain Perception Set Guided Response Mechanism Complex Overt Response Adaptation Origination
8 PERFORMANCE INDICATOR 2
Operational Verb Nominal Content/Topics Techniques/Methods Major Steps
PROGRAM SO COVERAGE (Select one only) SO ‘1’ SO ‘2’ SO ‘3’ SO ‘4’ SO ‘5’ SO ‘6’ SO ‘7’ SO ‘8’ SO ‘9’ SO ‘10’
Cognitive Domain Remembering Understanding Applying Analyzing Evaluating Creating
Affective Domain Receiving Responding Valuing Organizing Values into Priorities Internalization
Psychomotor Domain Perception Set Guided Response Mechanism Complex Overt Response Adaptation Origination
9 PERFORMANCE INDICATOR 3
Operational Verb Nominal Content/Topics Techniques/Methods Major Steps
PROGRAM SO COVERAGE (Select only one) SO ‘1’ SO ‘2’ SO ‘3’ SO ‘4’ SO ‘5’ SO ‘6’ SO ‘7’ SO ‘8’ SO ‘9’ SO ‘10’
Cognitive Domain Remembering Understanding Applying Analyzing Evaluating Creating
Affective Domain Receiving Responding Valuing Organizing Values into Priorities Internalization
Psychomotor Domain Perception Set Guided Response Mechanism Complex Overt Response Adaptation Origination
WAJID HUSSAIN: SPECIFIC AND GENERIC PIs TO MEASURE COURSE OUTCOMES WORKSHOP GUIDE & PRACTICE BOOK
7 | P a g e © 2020 WAJID HUSSAIN
H. HIGH EXPECTATIONS: REQUIRED STEPS FOR GIVEN LEARNING ACTIVITY PI_1
PI _1 HIGH EXPECTATIONS STUDENT ACTIVITY (SEQUENTIAL WITH ALL GRADABLE MAJOR STEPS INDICATED) :
STEP1:
STEP2:
STEP3:
STEP4:
STEP5:
WAJID HUSSAIN: SPECIFIC AND GENERIC PIs TO MEASURE COURSE OUTCOMES WORKSHOP GUIDE & PRACTICE BOOK
8 | P a g e © 2020 WAJID HUSSAIN
I. HIGH EXPECTATIONS: REQUIRED STEPS FOR GIVEN LEARNING ACTIVITY PI_2
PI _2 HIGH EXPECTATIONS STUDENT ACTIVITY (SEQUENTIAL WITH ALL GRADABLE MAJOR STEPS INDICATED) :
STEP1:
STEP2:
STEP3:
STEP4:
STEP5:
WAJID HUSSAIN: SPECIFIC AND GENERIC PIs TO MEASURE COURSE OUTCOMES WORKSHOP GUIDE & PRACTICE BOOK
9 | P a g e © 2020 WAJID HUSSAIN
J. HIGH EXPECTATIONS: REQUIRED STEPS FOR GIVEN LEARNING ACTIVITY PI_3
PI _3 HIGH EXPECTATIONS STUDENT ACTIVITY (SEQUENTIAL WITH ALL GRADABLE MAJOR STEPS INDICATED) :
STEP1:
STEP2:
STEP3:
STEP4:
STEP5:
WAJID HUSSAIN: SPECIFIC AND GENERIC PIs TO MEASURE COURSE OUTCOMES WORKSHOP GUIDE & PRACTICE BOOK
10 | P a g e © 2020 WAJID HUSSAIN
K. HYBRID RUBRIC DEVELOPMENT PI_1
Score Excellent (90-100%) Adequate (75-89%) Minimal (60-75%) Unsatisfactory (0-60%)
1. 1. 1. 1.
2. 2. 2. 2.
3. 3. 3. 3.
4. 4. 4. 4.
5. 5. 5. 5.
WAJID HUSSAIN: SPECIFIC AND GENERIC PIs TO MEASURE COURSE OUTCOMES WORKSHOP GUIDE & PRACTICE BOOK
11 | P a g e © 2020 WAJID HUSSAIN
L. HYBRID RUBRIC DEVELOPMENT PI_2
Score Excellent (90-100%) Adequate (75-89%) Minimal (60-75%) Unsatisfactory (0-60%)
1. 1. 1. 1.
2. 2. 2. 2.
3. 3. 3. 3.
4. 4. 4. 4.
5. 5. 5. 5.
WAJID HUSSAIN: SPECIFIC AND GENERIC PIs TO MEASURE COURSE OUTCOMES WORKSHOP GUIDE & PRACTICE BOOK
12 | P a g e © 2020 WAJID HUSSAIN
M. HYBRID RUBRIC DEVELOPMENT PI_3
Score Excellent (90-100%) Adequate (75-89%) Minimal (60-75%) Unsatisfactory (0-60%)
1. 1. 1. 1.
2. 2. 2. 2.
3. 3. 3. 3.
4. 4. 4. 4.
5. 5. 5. 5.
WAJID HUSSAIN: SPECIFIC AND GENERIC PIs TO MEASURE COURSE OUTCOMES WORKSHOP GUIDE & PRACTICE BOOK
13 | P a g e © 2020 WAJID HUSSAIN
APPENDIX
WAJID HUSSAIN: SPECIFIC AND GENERIC PIs TO MEASURE COURSE OUTCOMES WORKSHOP GUIDE & PRACTICE BOOK
14 | P a g e © 2020 WAJID HUSSAIN
WAJID HUSSAIN: SPECIFIC AND GENERIC PIs TO MEASURE COURSE OUTCOMES WORKSHOP GUIDE & PRACTICE BOOK
15 | P a g e © 2020 WAJID HUSSAIN
WAJID HUSSAIN: SPECIFIC AND GENERIC PIs TO MEASURE COURSE OUTCOMES WORKSHOP GUIDE & PRACTICE BOOK
16 | P a g e © 2020 WAJID HUSSAIN
Verbs and Language to Avoid or be Cautious about:
1. Verbs to Avoid or be careful and precise while writing Los statements:
Understand,
recognize,
develop,
relate,
consider,
prepare,
comply,
reflect,
realize,
anticipate,
foresee,
observe,
review,
extend,
work
note: the verbs in the offset brown color have to be used with appropriate nominal content
2. “Ability” or “Capacity” or “Awareness” or “Become aware of” or “Develop
understanding of” should not be used either as starter or nominal content of a learning
outcomes statement
3. Avoid language that indicates future tense or progressive learning such as “Students will
learn to…” or “Students will develop the capacity to …..” etc.
WAJID HUSSAIN: SPECIFIC AND GENERIC PIs TO MEASURE COURSE OUTCOMES WORKSHOP GUIDE & PRACTICE BOOK
17 | P a g e © 2020 WAJID HUSSAIN
Program Learning Outcomes Guidelines:
1. Start with verbs and use generic nominal content applicable to the program
2. Adopt key elements and aspects from graduate attributes for your specialization
3. Progress in a sequential manner as per Bloom’s Mastery Learning Model
4. Include the Cognitive, Psychomotor and Affective domain components
5. Include 21st century interpersonal skills such as leader ship in teamwork,
communication, professional ethics etc.
6. Include key elements of National educational plans such as entrepreneurship
7. Conclude with life performance roles of Transformational OBE
WAJID HUSSAIN: SPECIFIC AND GENERIC PIs TO MEASURE COURSE OUTCOMES WORKSHOP GUIDE & PRACTICE BOOK
18 | P a g e © 2020 WAJID HUSSAIN
WAJID HUSSAIN: SPECIFIC AND GENERIC PIs TO MEASURE COURSE OUTCOMES WORKSHOP GUIDE & PRACTICE BOOK
19 | P a g e © 2020 WAJID HUSSAIN
Contact:
Email: [email protected]
Telephone: +966562662492
URL: https://in4obe.org/