Comparing Generic Student Learning Outcomes Comparing Generic Student Learning Outcomes

among among

Fresh Graduates in the WorkplaceFresh Graduates in the Workplace

Comparing Generic Student Learning Outcomes Comparing Generic Student Learning Outcomes

among among

Fresh Graduates in the WorkplaceFresh Graduates in the Workplace

Focusing on Faculty of Medicine’s Graduates

This report includes relevant sections of a research This report includes relevant sections of a research report submitted to the Senate Committee on report submitted to the Senate Committee on Teaching & Learning for all of CUHK byTeaching & Learning for all of CUHK by

Hazlett, Clarke

Cheng Chun Yiu, Jack

Cheung Wai Hung, Gordon

Kwong Kai Sun, Sunny

Plus further data from a 2Plus further data from a 2ndnd similar survey completed only similar survey completed only

by Medicine, i.e., medical alumni from its new curriculum by Medicine, i.e., medical alumni from its new curriculum

as the surveys were done within one yr of graduation, the medical alumni as the surveys were done within one yr of graduation, the medical alumni

were internswere interns

Hazlett, Clarke

Cheng Chun Yiu, Jack

Cheung Wai Hung, Gordon

Kwong Kai Sun, Sunny

Plus further data from a 2Plus further data from a 2ndnd similar survey completed only similar survey completed only

by Medicine, i.e., medical alumni from its new curriculum by Medicine, i.e., medical alumni from its new curriculum

as the surveys were done within one yr of graduation, the medical alumni as the surveys were done within one yr of graduation, the medical alumni

were internswere interns

CUHK’s ObjectivesCUHK’s Objectives

Determine if recent (within 1 yr of graduation) CUHK alumni concluded that their major undergraduate programme had developed their generic skills so that they could effectively meet job requirements in their place of employment

Determine if these opinions of the alumni were consistent with those of

students enrolled all CUHK programmes during 1st & last yrs of the program (as assessed by the Student Engagement Questionnaire)

Hong Kong employers (as assessed by the Hong Kong Education and Manpower Bureau)

Faculty of Medicine’s Added ObjectiveFaculty of Medicine’s Added Objective

Determine if alumni from old and new curriculum were different in reporting how well their programme had developed these generic skills

Methods: SummaryMethods: Summary

Summary

Surveyed all alumni graduating in 2004/5 (N= 2646) and then repeated the survey on only medical graduates in 2005/06 (N=142)

Focus of survey instrument:

During my employment since graduation, the academic programme that I took in CUHK has enabled me to . . .

e.g., . . speak more effectively in Cantonese when communicating with clients and colleagues.”

Instrument and survey method pilot tested for content & face validity

Instrument’s construct validity established (construct validity = 0.91)

Generalizability of results estimated in initial survey (found no evidencethat non respondents had any different opinions)

Methodology details and relevant analyses available

ResultsResults

Response RatesResponse Rates

Overall Response Rate (CUHK) in 1st Survey: 1356 (51%)

Fac. of Medicine Response Rate in 1st Survey: 78 (54%)

Fac. of Medicine Response Rate in 2nd Survey: 73 (51%)

Faculty of Medicine’s Within Program ComparisonsFaculty of Medicine’s Within Program Comparisons

With reference to CUHK’s average for all respondents(i.e., includes those who did and did not require the skill in their job)

CUHK & Nursing Alumni Compared Level their Programmes Developed Generic Competencies

CUHK & Nursing Alumni Compared Level their Programmes Developed Generic Competencies

0

1

2

3

4

5

NumericalCompetency

PutonghuaProficiency

EnglishProficiency

Communicating InCantonese

InterpersonalSkills

Work Attitude AnalyticalProblem Solving

ComputerLiteracy

Mgmt Skills

CUHK Nursing

0

1

2

3

4

5

NumericalCompetency

PutonghuaProficiency

EnglishProficiency

Communicating InCantonese

InterpersonalSkills

Work Attitude AnalyticalProblem Solving

ComputerLiteracy

Mgmt Skills

CUHK Pharmacy

CUHK & Pharmacy Alumni Compared Level their Programmes Developed Generic Competencies

CUHK & Pharmacy Alumni Compared Level their Programmes Developed Generic Competencies

0

1

2

3

4

5

NumericalCompetency

PutonghuaProficiency

EnglishProficiency

Communicating InCantonese

InterpersonalSkills

Work Attitude AnalyticalProblem Solving

ComputerLiteracy

Mgmt Skills

CUHK Old Curric New Curric

*

* statistically significant at 0.05 (2-tailed) between old & new curriculum cohorts

CUHK & Medical Alumni Compared Level their Programmes Developed Generic Competencies

CUHK & Medical Alumni Compared Level their Programmes Developed Generic Competencies

SummarySummary

Expected differences among CUHK Faculties & programmes were often

found, adding evidence to the validity level of the measured outcomes

These results imply that caution is needed when attempting to make any

direct comparisons among Faculties or among major programmes

To evaluate the relative differences, note that on a scale from 1 to 5, the

absolute differences among Faculties & progammes were not large and

thus few differences are educationally meaningful

Comparative Findings for Faculties & ProgrammesComparative Findings for Faculties & Programmes

Variations between programmes were more pronounced than among Faculties

caution should be exercised as smaller programmes with fewer

respondents can more easily exhibit wider variations than large programmes

with larger number of respondents

ConclusionsConclusions

• Recent alumni from 3 programs in Faculty of Medicine are quite similar to how other CUHK alumni report that their respective programmes’ have developed the graduates’ generic competencies

• Medical programme’s alumni who graduated under the new curriculum reported one statistically significant improved competency (use of computers) & that the other 8 generic competencies were directionally better than parallel measures reported by alumni from the old curriculum

SummarySummary

Although reported results are based on a survey instrument which met acceptable standards for construct validity, there were only reasonable consistencies between CUHK’s SEQ and this Fresh Graduates Survey’s measures for the following constructs

a. Language proficiency

b. Computer literacy

c. Problem Solving Ability

Thus, comparative alumni ratings are likely more accurately informative for these 3 above noted competencies

most constructs measured in this survey are generic to all programmes; thus, most should be included in the student learning outcomes of any major programme or Faculty

these generic skills are those that both employers and the Educational Manpower Bureau regard as relevant skills for any university graduate entering the Hong Kong workforce

these skills are those which CUHK has operationally defined as appropriate for its students to acquire while they pursue a chosen discipline-based or professional-based education

the investigators hope that this feedback from new alumni will prove to be useful for the Faculty in its continuing efforts to enhance its respective curricula, instructional designs, learning activities and student assessments in order to achieve desired student learning outcomes

Notwithstanding the Afore Noted CautionsNotwithstanding the Afore Noted Cautions