the study of core competency learning and outcome ... · 1. introduction 2/2 considerable research...

Professor Yu-Hui Chen

Director, Center of General Education, National

Cheng-chi University, Taiwan

The Study of Core Competency Learning and Outcome Assessment in

General Education

1. Introduction 1/2

In the wake of globalization, the knowledge-based economy is ushering in

a new model of globally competitive work environments indicating the

key competencies that undergraduate students must acquire and develop.

This new model drives the current move in higher education toward

higher level learning outcomes such as critical thinking, communication,

problem solving, and the like.

To provide a clearer picture of these changes, this study first illustrates

how institutions of higher education around the world define the core

competencies that undergraduate students should be equipped with,

especially as regards their acquisition through general education.

Professor Yu-hui Chen, Director, Center of General Education. National Cheng-chi University, Taiwan

1. Introduction 2/2

Considerable research on general education assessment methods has

been conducted, but no methods provide a large-scale assessment of core

competencies.

The present study, then, reviews literature about multiple assessment

approaches, with a specific focus on the application of outcome-and-

performance-based rubrics to assess core competencies acquired from

general education.

approaches are critical to assessing the learning of core competencies,

and any institution of higher education wishing to undertake a large-scale

assessment of general education competencies may wish to consider using

standardized rubrics that serve the purposes of learning as well as of

evaluation and accountability.


2. The current worldwide trend in higher education:

Strengthening competency-oriented learning and learning outcomes


(1). The European Credit Transfer and Accumulation System (ECTS) and core competency acquisition as

requirement for graduation

1st Year 2nd Year 3rd Year Comments

Major Major 132 major

credits

Professional

advanced field Professional advanced field 18 credits

Core

competencies Core competencies 18 credits

Bachelor thesis Bachelor thesis 12 credits

180 total credits

Source: Tabulated by reference to Georg-August-Universität Göttingen (n.d.). Bachelor

of Arts, Bachelor of Science. Retrieved January 23, 2011,

from http://www.uni-goettingen.de/de/46536.html. Professor Yu-hui Chen, Director, Center of General

Education. National Cheng-chi University, Taiwan

Table 1: A single major/bachelor degree curriculum under the new

system at the University of Göttingen

http://www.uni-goettingen.de/de/46536.html



(2). How universities worldwide define core competencies and general occupational

competencies

Table 2: A summary table of core competencies for universities in Taiwan and the European Union

Academic objectives Personal objectives Social objectives Employment objectives

Core

competencies

Profession

al

knowledge

Critical

thinking

Moral

thinking

Self-

development

Civic

literacy Cultural

quality

International

mobility

Communicatio

n

Teamwork

Innovatio

n

Ethical

reasoning

Health

management

Social

care

Artistic

appreciati

on

Foreign

language Problem

solving

NCCU ○ ○ ○ ○ ○ ○ ○ ○

NTU ○ ○ ○ ○ ○ ○ ○ ○

NTHU ○ ○ ○ ○ ○ ○ ○ ○

THU ○ ○ ○ ○ ○ ○ ○

FCU ○ ○ ○

EU ○ ○ ○ ○

Subtotal 4 6 4 5 5 5 5 4

Source: Tabulated by reference to relevant data. Professor Yu-hui Chen, Director, Center of General


Table 3: A summary table of institutional surveys analyzing occupational core competencies of universities around the world

Surveying

Institution Report or project

Foreign

language

Computer

knowledge

Professional

knowledge

Communica

tion and

cooperation

Autono

mous

learning

Problem

solving

Innovatio

n

Work

attitude

/discipline

The Ministry

of Education

(Taiwan)

UCAN Common

Competencies for the

Workplace

○ ○ ○ ○ ○ ○

Department

of Labor

(USA)

The Survey of

Employability ○ ○ ○ ○ ○ ○

National

Taiwan

Normal

University

(Taiwan)

Taiwan Higher

Education

Integrated

Database

○ ○ ○ ○ ○ ○

National

Youth

Commission

(Taiwan)

Report on Survey of

Employability in

College Graduates

○ ○ ○ ○ ○ ○

Common

Wealth

Magazine

(Taiwan)

Eight Indicators of

Business Owners’

Evaluation of College

Graduates

○ ○ ○ ○ ○ ○

NTHU

(Taiwan)

Is the school helpful

to your development

of the following

competencies?

○ ○ ○ ○ ○

Australia Framework of

Employability Skills ○ ○ ○ ○ ○

Subtotal 5 5 5 7 6 6 4 1



(3). The use of curriculum mapping in Taiwan Table 4: NCCU's university-level core competencies

School motto Caring, Excellence, Integrity

Talent

characteristi

cs

Humanism Innovation Globalization

Education

objectives Academic objectives Personal objectives Social objectives Employment objectives

Core

competencies

Professional

competency

Liberal

arts

attitude

Critical

thinking and

innovation

Self-

development Civic literacy Social concern

Communicatio

n and

cooperation

International

mobility

Content

description

and the

development

of rubrics

Professional

knowledge

Macro

knowledge

Critical

thinking

Autonomous

learning

Ethical literacy Concern for the

natural

environment

Communication

and expression

International

perspectives

Lifelong

learning

Democratic

and legal

literacy

Concern for

humanity and

the social

environment

Writing

competency

Interdiscipli

nary

knowledge

Innovation Self-fulfillment Information-

media literacy

Respect for

diversity and

strong values

Teamwork Foreign

language

ability

Lifestyle of

health and

sustainability

Aesthetic

literacy

Public

participation

Problem solving

Science

literacy

Outcome

examination

unit

Professional

departments

and graduate

institutes

Center of

General

Education

Center of

General

Education

Center of

General

Education,

Chengchi

Residential

College

Office of

Student Affairs

Center of

General

Education

Center of

General

Education,

Chengchi

Residential

College

Center of

General

Education,

Chengchi

Residential

College

Office of

Student Affairs

Office of

International

Cooperation,

Foreign

Language

Center


3. Core competency assessment tools and the proposition of multiple

assessments


Assessment Methods 1/2

Written tests that only adopt a single criterion have been questioned

because they not only fail to completely evaluate students' learning

achievements in various dimensions but also no longer meet the current

demands of higher education, resulting in the neglect of performance,

portfolio, and observation assessments.

To improve teaching quality in university education, it is insufficient to

consider teaching objectives, contents, or methods only. A simultaneous

emphasis on research to improve assessment methods is necessary.


Assessment Methods 2/2

(1). The definition of multiple assessments

(2). The significance of multiple assessments

(A).The diversification of test objectives

(B). The diversification of assessment contents

(C). The diversification of assessment formats

(D). The diversification of assessment perspectives


Types of multiple assessments

(A). Portfolio assessment

(B). Performance assessment

(C). Affective assessment

(4). E-portfolio and Campus Pack e-learning

platform


Figure1: An example student e-portfolio on Campus Pack

Source for Figures 1 and 2: NCCU Campus Pack platform. Retrieved April 10, 2012, from

http://moodle.nccu.edu.tw/blocks/cpfusion/modlib/frameset.php?course_id=1408&url=http%3A%2F%2Fmoodle.nccu.edu.tw%2Fblocks%2Fcpfusion%2Flaunch.php%3Ftype%3Dcourse%26id%3D1408


http://moodle.nccu.edu.tw/blocks/cpfusion/modlib/frameset.php?course_id=1408&url=http://moodle.nccu.edu.tw/blocks/cpfusion/launch.php?type=course&id=1408



Relevant research on learning outcomes and core competencies from worldwide testing centers


Selecting appropriate assessment methods for different teaching

objectives is the concept to be emphasized when promoting the use of

multiple assessments in learning outcome assessment.

Knowledge and skill outcomes can be readily measured by graduation

credits, general examinations, capstone courses, certification exams,

graduate surveys, and employer satisfaction surveys

The assessment of soft skills, such as behaviors, attitudes, values, and

higher-dimensional thinking is relatively difficult to accomplish.

The soft skills emphasized in general education need to be learned and

honed in real life.

An individual high-level competencies test: The Collegiate Learning Assessment (CLA)

Professor Yu-hui Chen, Director, Center of General


CLA

- Critical thinking

- Analytic reasoning

- Problem solving

- Communication

Performance Tasks Analytic Writing Tasks

Make an argument Critique an argument

Figure 3. CLA test method for high-level competencies.

Table 5: International assessment tools for testing higher order skills

Name Testing Purpose and Application Participating institutions

Collegiate

Learning

Assessment

(CLA)

*To measure an institution’s contribution to the development of key higher order

competencies, including the effects of changes to curriculum and pedagogy

*To provide information that will help colleges and universities determine how much

their students are improving and whether that

improvement is in line with the gains of comparable students at other institutions

*Over 500 institutions and

250,000 students have

participated in the CLA to

date

Critical Thinking

Assessment Test

(CAT)

*To assess and promote the improvement of critical thinking and real-world problem

solving skills

*To redesign an existing course or develop a new course that implements these new

learning and teaching practices

More than 80 universities

have participated in CAT

and 7 universities are still

working on research and

development.

Collegiate

Assessment of

Academic

Proficiency

(CAAP)

*To measure the outcomes of general education programs

*To measure students' achievement levels on a group and individual basis

*To compare students' achievement levels with national user norms

*To evaluate the strengths and weaknesses of general education programs

*To document the performance gain in students' achievement levels over time

*To analyze what interventions may be necessary to enhance results

*To determine student eligibility for upper-division studies

*To advise individual students on how to achieve academic success

ETS Proficiency

Profile (also

known as MAPP)

*To measure the academic skills developed through general education courses.

*To assess student proficiency in core academic skill areas to identify strengths,

weaknesses, and opportunities to improve curricula and instruction

*To measure and document program effectiveness to meet requirements for

accreditation and accountability

*To compare performance against programs at more than 400 institutions nationwide

*To conduct trend analyses to evaluate program improvement efforts and overall

learning outcomes

More than 400 institutions

and over 500,000 students

nationwide have

participated in MAPP.

Wabash National

Study of Liberal

Arts Education

*To measure outcomes of liberal arts education programs

*To learn what teaching practices, programs, and institutional structures support

liberal arts education

*To develop methods of assessing liberal arts education

Forty-nine institutions

have participated in the

Wabash National Study of

Liberal Arts Education.

Table 76 Surveys administered by international testing centers to provide

large-scale assessments of students’ holistic learning outcomes Name Testing purpose Participating institutions

College Student

Experiences

Questionnaire (CSEQ)

*To determine the quality of effort undergraduate students

invest in using educational resources and opportunities

provided for their learning and development

*To assess the students’ perceptions of how much the campus

environment emphasizes a diverse set of educational

priorities

*To determine how the students’ efforts and perceptions

relate to personal estimates of progress made toward a

holistic set of learning outcomes

More than 140 colleges and universities have

participated, including Stanford University, the

University of Arizona, the University of

California-San Diego, the University of

California-Santa Barbara, the University of

Illinois at Chicago, the University of Virginia,

and the University of Georgia

College Student

Expectations

Questionnaire (CSXQ)

*To evaluate new student expectations for college including

student goals, motivations, and future plans.

*To provide clues about students expectations regarding (1)

the nature and frequency of interactions with faculty

members, (2) involvement with peers from diverse

backgrounds, (3) the use of campus learning resources and

opportunities, (4) satisfaction with college, and (5) the nature

of college learning environments

Adopted by more than 40 universities and

colleges

National Survey of

Student Engagement

(NSSE)

*To collect information about student participation in

programs and activities that institutions provide for their

learning and personal development

*To provide an estimate of how undergraduates spend their

time and what they gain from attending college

*761 colleges and universities participated in

NSSE in 2011; 1,493 have participated since 2000

*393,630 students completed NSSE in 2010; more

than 2.7 million students have completed the

survey since 2000



5. Conclusions 1/2



In summary, the core theme of this study is the assessment of core

competencies.

Through a literature review, we explored in detail the status of worldwide

assessment tools for core competence to clarify the contents and concrete

meaning of soft skills in general education.

We explored the contents of assessment rubrics for core competencies

developed by the Association of American Colleges and Universities.

The assessment of core competencies involves clear definitions of the

actual contents of core competencies and their operational definitions and

descriptions.

5. Conclusions 2/2



Core competencies cannot be assessed by traditional written tests.

It is necessary to combine the insights of multidimensional assessments.

It is necessary to to implement these assessments with the spirit and

methods of portfolio, performance, and affective assessments.

Data collection efforts are important steps in core competency assessment.

These data collection efforts can involve obtaining various forms of

relevant information, including information on student backgrounds,

learning environments, the utilization of various teaching resources,

learning consultations, and student club activities.

the study of core competency learning and outcome ... · 1. introduction 2/2 considerable research...

Documents