the impact study on effectiveness of akept’s t&l training programmes

8/2/2019 THE IMPACT STUDY ON EFFECTIVENESS OF AKEPTS T&L TRAINING PROGRAMMES

1/86

1

AKEPT LEADERSHIP RESEARCH GRANT YEAR 2009

END OF PROJECT REPORT

A. Pr oject Nu mb er File at AKEPT-680-3/3/2 (16)

Pr o je ct l e ad er A ssct . Pr o f. Dr . HAN IPA H HU SSIN

Pro ject T i t le THE IM PACT STUDY ON EFFECTIVENESS OF AKEPTS T& L TRAINING

PROGRAM M ES:

Topics : KNOW LEDGE APPLICATION AN D INFLUENCING CHA NGES AM ONG

PEERS AND COLLEGES

B. Su m m a ry f o r A KEPT Re p or t

(For pub l icat ion in AKEPT Annual Report , p lease summ aries the pr oject object ives, signi f icant

resul ts achieved, research approach and t eam str ucture)

Abstract

Purpose Th is st udy base d on sho r t t er m resear ch , gr an t by Ak ad em i Kep im p in an Pen ga j ian

Tinggi M alaysia (AKEPT). The purp ose of th is stu dy is to reveal th e im pact of AKEPT Train ingProgram on lectur ers tow ard th eir contr ibu t ion on t eaching and learning act iv i t ies as peers and

col leagues contr ibut ion on kn ow ledge dissem ination pr ocesses amo ng peers and col leagues at

facul ty level .

Design/ m et hodo lo gy / ap proach De sign / m et hodo lo gy / ap p roach Da t a w ere co ll ect ed f rom

519 academ ics from all IPTA, IPTS, in M alaysia dur ing AKEPT Teaching and Learning t rainin g

w i th in 2008 to 2009

Findings Th e st udy found on ly t h ree out o f t en el em en t s o f know le dge d issem in at io n

processes show a signi f icant level ( m ore f o l low -up after tr a in ing p= 0.037, supervisor in volve

m e a f te r t ra in ing p= 0 .27 and m ore conf ident a f te r t ra in ing p= 0 .037). W here another f ind ingshow t hat anot her seven element s of know ledge dissem ination pro cesses somew hat have less

change (asking qu estio ns, suggest id es in T& L, involve pee rs, makin g a decisions, hold grou p

m eet ing , tak ing m ore t im e to t ransform p lanned into im p lementa t ion and takes t im e to

re f lect ) . A l l e lement a re modera te ly p ract iced by lecturer a t M oHE


2/86

2

Pract ical implicat ions As co nclusio n , t h is pap er rai ses aw ar en ess an d p rovid es in it ia l

guidel ines to the AKEPTs as tra in ing centre for a l l lecturers to improve knowledge

dissem ination str ategies in t heir local universi ty.

Knowledge- In suggest ion , in t ensive organ iza t ions in fo rm ula t ing st ra teg ies on how to proper l

implement and manage their pedagogical knowledge dissemination processes are open to beexplore.

Origina l i ty / va lue Th is st udy has ext ended in ped agogica l know le dge d issem in at io n fo r it i

prob ably the f i rst to provide a com parative analysis betw een AKEPTs Train ing Centr e and local

teaching and learning tra in ing centre. I t fur ther opens up new l ines of future research

possibil i t ies.

Keywords : Know ledge Dissem inat ion Processes, Pedagogical Know ledge, Perceive

Com pet ence, Train ing

Project Object ives:

Pertain ing t o t he tr ansform ation p lan 2007 and AKEPT Centr e for Teaching and Learning,

evaluation and account abi l i ty, there are thr ee question s th at a lways need an answer fo r next

developm ent p lan in M OHE. The research object ives are:

To investigate t he impor tant of AKEPTs teaching and learning tr a in ing program s

To examine the imp act of AKEPT Train ing Program contr ibut ed to know ledge

appl icat ion in decision making in teaching and learning amo ng peers and col leagues

To examine the imp act of AKEPT Train ing Program contr ibut ed to know ledge

appl icat ion and dissem ination amo ng peers and col leagues

Research Questi on s

The most com m on reason fo r T&L evaluation is to deter m ine the effect iveness of a program

and w ays in w hich i t can be im proved. The research questions are:

1 . How impor tan t AKEPTs teach ing and learn ing program? (Leve l 2 -Re-act ion )

2 . Have the par t ic ipants ever app ly teach ing knowledge based they ga ined f rom AKEPTs

tr a in ing? (Level 2- Learning )


3/86

3

3 . How a re the pa r t i ci pan ts pe rfo rm ing d i ffe rent l y i n o rde r to i n f luence changes among

th eir p eers and colleagues? (Level 3-influ encing chan ges change)(self-appraisal)

4 . W hat is the observat ion s resu l t f rom super io r s feedback on par t icipants qua l ity

teach ing per fo rmance? (Leve l 4- the resu l t on super io r s feedback on t he i r

subordinat es perfo rm ance) know ledge based. (evaluator)


4/86

4

3.0 M ETHODOLOGY

3.1.Research App roach:

Quantit at ive based on quest io nnaires

1. Sect ion A: Demo graphic Prof i le and React ion t o AKEPT Tra in ing

2. Sectio n B: HOW DO YOU PERCIEVE YOUR OW NS TEACHING PERFORM ANCE

Part 1 DECISION M AKING IN TEACHING AN D

LEARNING

Part 2 KNOW LEDGE APPLICATION AN D

DISSEM INA TION

3. Sect ion C : HOW DO YOU OBSERVE YOUR SUBORDINATES TEACHING

PERFORM AN CE ON:

CONTENT AND CURRICULUM : Cont ent k now ledge of subject m at ter

KNOW LEDGE OF STUDENTS AND THEIR LEARNING: The int ellect ua l, social , phy sical, an d

personal developm ent of a l l student s.

LEARNING ENVIRONM ENTS: Learnin g envir onm ent s creat ion t hat encourag es positi ve

social interact ion, act ive engagement in learning, and self-mot ivat ion.

ASSESSM ENT: Com prehen sion and usage of a rang e of fo rm al and info rm al assessm ent

str at egies to evaluat e and ensure the cont inu ous developm ent of a l l learners.

PLANN ING AND IN STRUCTION: Inst ructio nal experiences design and creatio n ba sed on

ow n know ledge of cont ent and curriculum, stud ents, learning environm ents, andassessment.

PROFESSIONALISM : Recognit ion, par t icipat ion in, an d cont ribu ti on in tea ching as a

profession


5/86

5

THEORITICAL FRAM EW ORKS AN D M ODEL FOR M ETHODO LOGY

3. M ETHODO LOGY FOR EVA LUATING TRAIN ING IN HIGHER EDUCATION

INTRODUCTION

This par t o f s tudy d isclose the m ethodo logy used in order to expend data f r om the des ign o f

Donald Kirkpatr ick 1959 and James Kirkpatr ick 1980 abou t m easur ing and evaluate t ra in ing.

Thus th is study focused on know ledge o f how to use the four leve ls o f eva luat ion , how to

create eva luat ion t oo ls and how to use the eva luat ion pr ocess to imp rove t ra in ing

effectiveness.

WHY EVALUATETRAINING FOR AKEPT?

How m any t im es have you heard peer s and col leagues say, I t w as a great t ra in ing program

sponsored by AKEPT but .. . ? Unfo rtu nately, th at stat emen t (or var iat ion ) is spoken far to o

of ten and re f lects a grow ing concern by both , Dean or Head o f the Depar tm ent and sen ior

m anagement t ha t t ra in ing is cost ly and not a lways w or th t he investm ent o f t ime and m oney.

This att i tud e is a lso ref lected in th e act ions of senior m anagers w ho cut tra in ing budget s f i rst

w hen t im es are tough in facu l ty .

PURPOSES OF EVALUATION

This stud y also disclose som e of t he m ain reasons to evaluate AKEPT tr aining.

- To de t er m i ne w h e t he r A KEPT t r ain in g ach ie ve s it s o b je ct iv es

- To assess t h e v alu e o f AKEPT t r ain in g p ro gr am s

- To id en t if y ar eas o f t he AKEPT p ro gr am t h at n eed s im p ro vem e nt

- To id en t if y th e ap pro pr iat e au dien ce fo r fu t ur e AKEPT p ro gr am s

- To re vie w an d re in fo r ce ke y A KEPT p r ogr am p ar t s f o r p ar t icip an t s

- To se ll A KEPT p r o gr am t o m a n ag em e n t an d pa rt i ci pan t s a m o n g M a la ysi an o f Hi gh e r

Education

LINKIN G EVALUA TION TO THE NEEDS ASSESSM ENT

Clearly, need s assessm ent is crit ical to t he success of an y tr aining init iative in AKEPT. It pro vides

th e basis for p rogram d evelopm ent and establ ishes the cr i ter ia for m easur ing th e success of

the pr ogram af te r i ts com ple t ion . For eva luat ion t o have any mean ing, i t m ust be t ied t o th e

need s assessm ent p rocess. AKEPT w ith goo d strat egies alw ays em phasis th at t he evaluatio n


6/86

6

process shou ld ref lect specif ic perfor m ance-related out com es.

W HEN TO EVALUATE

Evaluation is an ongoing process, not just som eth ing that h appens at th e end of a session o r a

program . Consider evaluation d ur ing the session, at t he end, and after t he part ic ipants retu rn

to t he j ob .

Dur ing th e Session . As people p art icipate is skil l practice, case stud ies, exercises, sim ulat ions,

and o t her act iv it ies, observe the degree to w h ich they have m astered the content . (Leve l 1 )

At th e End o f t he Session. Part ic ipants evaluation qu estion naires w i l l indicate their personal

reactio ns to t he t raining session. (Level 1, Level 2 and Level 3)

Aft er the Training. A few weeks to several mont hs aft er th e session, observe the part ic ipants

jo b per fo rm an ce t o det erm in e w het her t hey ar e ap p ly in g w hat t hey le ar ned t o t hei r w o rk

situat ion s. (Level 4)

W HOM TO INVOLVE

Any category or num ber of peo ple can be involved in the evaluation pro cess, depending on

w hat you w ant t o know and w hat leve l eva lua t ion you conduct . The part ic ipants are d i rect ly

involved because they are the custo m ers w hose level of sat isfact ion you are try t o det erm ine.

The par t ic ipants are the m ajor source o f in fo rm at ion o f how m uch they have actua lly learned in

th e tra in ing. In addi t ion t o t he part ic ipants, involve others in t he evaluation pro cess. Survey or

interview th e part ic ipants managers, th eir co-wor kers, peers, subor dinates, or even custo m ers

or vendors. At some po in t , you need t o invo lve sen ior management .

HOW TO EVALUATE

Before developing an evaluation process, be clear of w hat yo u w ant t o evaluate. This is not as

easy as i t seem. Do you w ant t o evaluate how m uch th e part ic ipants have learned? Or do you

w ant to know w hat the par t ic ipants thought about the program and you? Do you want t o f ind

out w hether t he par t ic ipants are app ly ing w hat t hey learned on t he job? If t hey are using the

learned ski l ls and inform ation on t he job, is th eir enhanced perform ance m aking a di f feren ce to

th e organizat ion ?

FOUR-LEVEL M ODEL OF EVA LUATION

The m ost w ide ly known m ode l fo r eva lua t ing t ra in ing program s was in t roduced by Dona ld

Kirkpatr ick in 1959. It is regarded as a classic by train ing practit ione rs. Altho ugh all four levels of

m odel ( react ion, learning, behaviour, resul ts) are im port ant, you m ay choose not to evaluate at

all four levels. Stu dies sho w AKEPT , (2010) that a vast m ajorit y of o rganization s evaluat e

reaction. A signi f icantly h igh p ercentage m easur ing learning as wel l . The evaluation or


7/86

7

m easurem ent of behaviour lags behind th e f i rst tw o levels; evaluation of resul ts f in ishes last.

Todays organizat ions are much m ore cost-conscious, and th e need t o m easure th e

effect iveness of t ra in ing w i l l continu e to grow . Should you need to un dert ake a comp rehensive

approach t o eva luat ion , you w i l l ab le to m ake appropr ia te recomm endat ions or respond

confident ly wh en someone asks you t o im prove th at tr a in ing gets resul ts.

Table 1.1 Kirkpat rick in 1959:

M easur ing Train ing Resul ts -Provides an overview o f th e fou r levels of evaluation.

W hat W ho W hen How W hy

Level

1

Reaction:

Did they l ike

it ?

Part ic ipants End o f

Program

Smile

Sheet

Determine level o f

customer

satisfact ion; m ay

ind ica te need fo rrevision.

Level

2

Learning:

Wha t

knowledge

or skil ls did

they re ta in?

Participants;

t ra iner

During,

be fo re /a f te r

program

Pre-

tes t /pos t -

te st; skil ls

appl icat ion

throu gh ro le

play s, case

studies,

exercises

Ident i fy w hether

tra iner have been

successful in d elivery

o f course content

and achieving

program ob ject ives

Level

3

Behaviour:

how a re

they

pe r fo rm ing

d i f fe rent ly?

Participants;

bosses;

subordinates;

peers

3 to 6

mon t hs af te r

program

comple t ion

Surveys;

in terv iews;

observation;

per fo rmance

appraisal

Determine extent to

w hich par t icipants

have transferred hat

they learned in the

session t o t he actual

w ork si tua t ion

Level

4

Results:

W ha t i s the

impact ont h e b o t t o m

l ine?

Participants;

contro l g roup

Af ter

comple t ion

of Level 3fo l low-up

Cost/benefi t

analysis;

tracking;opera t iona l

data

Dete rm ine w he the r

benef i ts ou tw e igh

costs; ascert aindegree of

contr ibu t ion o f

p rogram t o

organizational goals.


8/86

8

EVALUATING TRAININ G

Level 1: Reactio n

Level 1 deals wi th part ic ipants react ion s, th at is, custo m er sat isfact ion. Level 1 evaluations

are oft en referred t o as smi le sheets, im plying that part ic ipant s react ions are based on how

m uch fun th ey had in th e tr a in ing session. For that reason, tra iners frequen tly d ismiss Level 1

evaluations as a waste of t im e.

On th e contrary, Level 1 is an imp ort ant f i rst step in determ ining the success of a

tra in ing pro gram. Part ic ipants react ion s can help you det erm ine th e effect iveness of a pro gram

and how i t can be im proved. Kirkpat r ick bel ieves th at you canno t b ypass th is f i rst level because,

as he put s it , I f they [par t ic ipants ] do not react favourab ly , they w i l l no t be m ot iva ted t o learn

(Kirkpatrick, 1994)

W hat Level 1 Cannot M easure. One o f t he prob lems w i th and t he main cause o f cr i t i c ism o f

Level 1 evaluation is th at i t is to o subject ive and oft en becom es not hing mo re than a popular i tycontest. Before constr uct ing a part ic ipants end-of-session evaluation for m , understand w hat i t

cannot and is no t in t ended to do ;

(1) it d oes not m easure learning or the abi l i ty to apply learning on t he job;

(2) i t a lso cannot m easure changes in att i tud es or b el iefs:

(3) because i t deals only w i th part ic ipant s percept ion and r eactions, a Level 1 instr um ent can

in no w ay measure organizat ional imp act;

(4) also, a l thou gh frequent ly asked, part ic ipants cannot m easure t he tra iners know ledge. Thinkabout i t . How cou ld the par t ic ipants have any way o f knowing w hat the t ra iner does and does

not know about t he sub ject? You ab i l ity t o communicateo r demons t ra te your kn ow ledge is an

ent i re ly d i f fe rent s to ry .

Decid ing What to M easure. Before you d esign a Level 1 instru m ent, you n eed to be clear about

wha t you w an t to know, w hat you w an t to know i t , and w ha t you are go ing to do w i th the

in form at ion . Do not ask fo r in fo rm at ion about somet h ing you cannot change or have no

inten tion of analyzing or report in g.

Designin g an End-of -Session Evaluat ion Form . Categories. Fi rst dec ide what you w ant to

m easure and create qu estion s or response i tem s that fa l l in to certa in categor ies, includingm any, i f no t a l l, o f the fo l low ing:

- Con t ent

- M at er ials

- In st ru ct io nal m e th od s


9/86

9

- Trainer

- Environm en t

- Logist ics

I t i s also a good idea to pr ov ide an oppor tun i t y fo r respondents to m ake recomm endat ions

as to h ow th e program can be imp roved and also to express th eir overal l react ions to t hesession.

Format . To counteract peop le s tendency to respond t he same w ay to every i tem on a

questionn aire or survey, use a var iety of response form ats. Choose at least f our from th e

fo l lowing opt ions:

- Tw o-choice quest ions wi t h roo m for explanat ions or comm ents . These w ould include

responses such as yes o r no and agree or d isagree. Example:Did the cour se

m eet the sta ted ob ject ives? Yes No W hy or why not?

- Short answers. The item s are wr i t ten as open-ended questions and requ ire the

respondent to w r i te dow n a br ie f response instead o f just check ing a box. Example:

W hat par ts o f the w orkshop w ere most va luab le /benef icia l to you? Wh y?

- Com plete t he sent ence. W ith t h is i tem, th e respondent is asked to com ple te a

sentence. Example:What i wan t / need to know m ore abou t i s. ..

- Ratings. Part ic ipants respond to a question or statem ent using som e type of scale or

rat ing such as a Likert scale. The Likert scale m easures bot h t he dir ection (positiv e and

negative) and inten si ty (strongly posi t ive to strongly negative) of an individual s opin ion

o r a t t i tude . Example:Todays session w as enjoyab le and satisfying learning experien ce

fo r me .

1 2 3 4 5 6 7 8

- Rankings. This i tem s asks respond ents to indicate pr io r i t ies or pr eferences. Example:

Please rank each to pic in order or i t s impo rt ance or re levance to you r job: 1 = mo st

impor tan t t o 5 = least im por tan t .

- Checklist. A checklist pr ovides a laundry l ist f rom w hich part ic ipants can choose

w ords that expr ess th eir react ion s. Example:Check the w ords that descr ibe your

reaction t o t odays session:

___ __ Excee ded m y exp ect at io ns

___ __ M et m y exp ec t at io ns

___ __ Fel l sh or t o f m y ex pect at io ns

A question could also be added fo cusing on th e imp act o f th e session on t he

part ic ipant , designed to o bt ain a deeper and mo re personal response, for example: Imagine

Str ongly Disagree Strongly AgreeNeutra l


10/86

10

th at a co-wor ker (a fr iend) of yours is th inking about atten ding th is program. He or she asks

you: Wh at w as th is p rogram l ike fo r you? How w ou ld you respond?

Evaluat ion s from Guidel ines. Evaluat ions fo rms are m ore d i f f i cu l t to construct t ha t you m ight

imagine. Use the fo l lowing guidel ines:

- Keep fo rm b r i ef . Pa r t ic ipan ts shou ld be ab le to comp lete i t qu i ck l y.

- Create a ba lance among the va r ious types o f i n fo rma t ion you are co l lect i ng. Fo r

exam ple , do not ask f ive quest ions about the inst ructor and on ly t w o about content .

- Ob ta in pa r t i ci pan ts imm ed ia te reac t ions. Have par t i ci pan ts comp le te the evaluat i on

befor e they leave the roo m. This w i l l ensure th at you r eceive a 100 percent response

rate. I t w i l l also prevent m ob m ental i ty response, the possibi l i ty of several people

gett in g together t o d iscuss th e class ei ther bef ore or w hi le they are com plet ing the

evaluation.

In terv iews. In addi t ion t o the end -of-session qu estion naires, you can use inter views to increase

th e rel iabi l i ty of th e data col lected fro m t he questionnaires. This m eth od of d ata col lect ion is

qui t e f lexib le, a l lowing the inter viewer t o prob e for m ore speci f ic answers and t o clar i fy

questions as needed. The m etho d also al lows the interview er to record spont aneous answ ers

and, therefo re, get m ore com plete picture of t he part ic ipants react ions. The int erview er can

explore in mo re detai l the reactions gleaned fro m t he questionnaires.

Plan on spend ing about th i r ty m inutes per in te rv iew. You w i l l no t be ab le to in te rv iew

every par t icipant , so select a r andom sample . I t i s im por tan t to ho ld t he in terv iews wi th in one

w eeks of t he session so th at th e exper ience is fresh in their m inds. Throu gh one-on-one

interview s, you can fur t her explore t he reasons for p art ic ipants react ion s and sol ici t

suggestion s for im provem ent . Ei ther tape t he interview s and have them t ranscr ibed, a llow ingyou t o ana lyze or in te rpre t t he responses more thorough ly , o r s imply take notes dur ing the

in terv iews.

W hen deve lop ing the in t erv iew quest ions, do not dup l ica te the qu est ions on t he

w ri t t en form . Instead, ask speci f ic questions about t he m eth ods used or the conten t covered.

For exam ple, below are several questions about th e met hods used in a tra in ing session on

leadership:

- W h a t f e el in gs d i d y o u h av e ab o u t t h e m e t h o d s u se d in t h e p r o gr am ?

- W h a t d id yo u li ke a bo u t t h e j igsa w d esi gn ?

- W h a t d id y ou l ik e ab ou t t h e le ar n in g t o u rn am e n t ?

- W h a t d id n t yo u lik e a bo u t t h e j igsa w d esi gn ?

- W h a t d id n t y o u l ik e a bo u t t h e l ea rn i ng t o u r n am e n t ?


11/86

11

Level 2: Learnin g

Level 2 evaluation d eals w i th w hat t he part ic ipants actual ly learned d ur ing th e tra in ing session.

Kirkpatr ick defines learning as th e extent to w hich part ic ipants changes att i tudes, im prove

know ledge, and/ or increase ski l l as a resul t o f att ending the p rogram (Kirkpatr ick 1994). It is

fa r easier to d e termine w hat new know ledge or sk i ll s the p ar t icipants acqu ired th a t to f ind th e

w ays in w hich th e tr a in ing changed th eir opin io ns, values, and bel iefs.

The three m ost appropr iate m ethod s used to evaluate learning are tests, observations,

and interview s, w i th tests being th e mo st frequent . See Table 13.2 for summ ary of the

advantages and disadvant ages of each m etho d.

M et hod Advan t ages Disadvant ages

Tests:

Object ives tests

M ul t ip le cho ice

M a tch ing

True-false

Fill-in

Subject ive tests

Essay

Short answers

Easy to score

Inexpensive to use

Easy to w r i te

Inexpensive to create

Di f f i cu l t to w r i te

T im e-consuming to w r i te

Grad ing is t im e-consuming

Expensive to gr ade

Observation:

Observe beh aviour in class

Skil ls demonstrated in skil lpract ices and learning

activi ty

Imm ediate app l icat ion

Lends i tsel f to on-th e-spotcoaching and feedback

Subject ive, open to

in terpre ta t ion

Unab le to spend adequa te t ime

observing behaviour of a l l

par t ic ipants

In terv iews:


12/86

12

Individual interviews

conducted shor t ly a f te r t he

t ra in ing

Random sam pl ing of

par t ic ipants

Can gath er m ore detai led

in fo rma t ion

Instant feedback

Time con sum ing; expensive

M ust be t igh t ly constructed t o

obt ain quanti f iable responses

Tab le 1 .2. Level 2 Evalu at io n M e t ho ds

Tests. Testing should be kept fa i r ly simple. M any tra iners give bot h a pre-test and a post- test

to get an even mo re accurate picture of wh at th e part ic ipant s have learned.

Types of test s. Fi rst de term ine w hether you w ant t o construct sub ject ive (shor t -answ er

or essay) or ob ject ive (m ult ip le-choice or true-fa lse) i tems or even a comb ination of t he tw otyp es. When constr uct ing test i tem s, consider th e t im e needed to grade the t est as w el l as the

val id i ty and th e re l iabi l ity of each i tem . M ake sure the test assesses the learning as speci f ied in

th e learning object ives. Wh en an i tem m easures wh at i t is supposed to m easure, i t has val id i ty.

Each test i tem m ust a lso be rel iable, that is, give consistent resul ts from one appl icat ion t o

anot her. For a b roader d iscussion of test constr uct ion, refer to Testing For Learning

Out com es by Deborah Grafinger Hacker, one of th e INFO-LINE booklet s publ ished by the

Am erican Society fo r Train ing and Developm ent (revised 1998). Hacker descr ibes var ious types

of t ests, provides i tem s in wr i t ing t est i tem s, d iscusses measur ing val id i ty and rel iabi l i ty of

tests, and includes a helpful and ext ensive resource l ist .

M ake sure t he test is m eaningful . Instead of asking for simp le inform ation or f actu al

recal l, ask questions that requ ire the part ic ipant s to apply or int erpret w hat t hey learned in the

session.

Quest ion fo rma ts . All m ult ip le-choice questions consist of a stem and a r esponse. The

stem pr esent s a problem , asks a question , or takes the for m o f an incomplet e stat emen t.

Respon ses include possible answers, al l of w hich m ust be plausib le. The greater t he nu m ber of

i tem s, th e bett er the t est s re l iabi l i ty. Fol low ing are som e form ats to consider:

- Correct an swer. The correct-answer fo rm at asks a sim ple question to w hich there is

only on e correct answer. I t is used pr imari ly to test t he recal l of facts. This typ e ofquest ion is appropr ia te to test p roduct knowledge, fo r exam ple .

- Best an swer. W ith t h is typ e of question, there is m ore t han one correct choice. Som e or

al l of t he choices and dr aw conclusions. This typ e of question can create m any

prob lems. Because the answ er is open to in te rpre ta t ion , the test i tem can be

chal lenged qui t e easi ly, and you m ight f ind yo urself e i ther arguing w ith t he individual


13/86

13

or group and m ost p robab ly having to g ive cred i t fo r o ther answ ers.

- Com bined respon se. This question is th e mo st com pl icated and t im e consum ing for

both the t est w r i te r and the t est t aker . The cho ices, one or m ore tha t m ay be cor rect ,

are num bered. A second set of choices l ists com binations of possib le correct r esponses.

This t ype of que stion assesses com plex cognit ive skil ls and t he abil i t y to an alyze and

evaluate. Exercise a great deal of th ought w hen w ri t ing the i t em . Because of i ts

complex i ty , the respondent w i l l p robab ly have to spend more t ime t h ink ing about th e

i t e m .

Test-W ri t ing Guidel ines. In m ost cases, you w i l l probably choose to develop m ult ip le-choice

questions. They are easy t o grade, but not necessar i ly easy t o w r i te. To help you construct a

m ul t ip le -cho ice test th a t w i l l p rov ide va luab le in fo rm at ion about the par t ic ipants content

m astery, consider the fo l low ing guidelines:

- A vo id a ll o f t he ab ov e a n d n o ne o f t he ab ov e o p t io n s.

- M a ke su r e t h e st e m ( t h e m a in p ar t o f t h e q u est i on ) co n t ai ns m o st o f t h e i n f o r m at i o nand define t he pro blem; p lace blanks for f i l ls- ins near the en d.

- M a in ta in g ramm at i ca l consi stency o r para ll e l st ruc tu re fo r bo th the stem and the

answer choices.

- Tr y t o cr e at e ch o ic es o f eq u al l en gt h .

- Avo id amb igu i t y and read ing d if f i cu l t y by sta t i ng quest i ons i n the posi t i ve rathe r than i n

the negative.

- Ke ep t h e se n t en ce st e m s im p l e an d l im i t i t t o o n e id ea .

- Use conve rsa t i onal l anguage when ph rasing the i tem and i t s cho ices.

- A r ra nge t h e q u est i o n s i n lo gi ca l o r d er .

- D o n o t gi ve cl ue s t o t h e co r r ect a n sw e r s i n t h e qu e st i on .

To gauge re ten t ion o f the in form at ion learned in t he program, admin ister anot her test

several m ont hs fo l low ing the tr a in ing.

Observat ion. Trainers can w atch p art ic ipants pract ic ing and applying ski l ls, too ls, and

techniqu es dur ing t he session. As t he t ra iners observe part ic ipant behaviour in ski lls pract ices,

ro le plays, sim ulat ions, case studies, and ot her act iv i t ies, they can get a good idea of w hat th e

tra inees have r eal ly learned.

In terv iews. Shor t ly a f te r the t ra in ing, in te rv iew the par t ic ipants and ask them w hat t hey

learned in the session. Conduct th e interview s w ith in t he w eek fo l low ing session.


14/86

14

Level 3: Behavio ur ( how has the t ra ining af fected the w ay par t ic ipants per fo rm on t he job? )

The cr i t ical question answ ered by Level 3 is, h ow has th e tra in ing affected th e way part ic ipant s

per form on the job?

Al tho ugh bot h m anagers and t ra in ing professionals agree that th e success of a t ra in ing

program is de termined by w hat t he par t icipants do w i th t he in format ion or sk il l s back on t he

jo b , t hese resu lt s are o f t en igno red . Le vel 3 ev aluat io n is bo t h t im e- co nsum in g a nd co st ly . I t

also requires good organizational and follow-up skil ls and processes.

Purpo se of Level 3 Evaluat ion. Use fo l low-up evalua t ion to do t he fo l lowing:

- M e asu r e l ast i n g r esu lt s o f t h e t r ai n in g

- Iden t i f y area i n wh ich t ra inees shows greatest and least imp rovemen t

- Co m p a r e f o ll ow - u p an d en d -o f -p r o gr am r e sp o n se s

Follow -up Guidel ines. Use these guidel ines for f o l low -up evaluation:

- Prepare pa r t ic ipan ts. At the end o f the t ra in ing session , te l l par t i ci pan ts that you w i l l be

conduct ing a fo l low -up evaluat ion and w hat t ype o f eva luat ion i t w i l l be .

- I f the t ra in ing wasn t e f fect i ve , f i nd ou t w hy . Encou rage pa r t ic ipan ts to i den t i f y reasons

w hy th ey haven t impro ved and wh at factors obstruct the i r p rogress. Som et imes there

are factors that inhib i t o r prevent t he appl icat ion of t he new know ledge and skil ls on

th e job. These barr iers m ight include poor environ m ental condi t ion s, lack of proper

equipm ent , the supervisor, exist ing pol ic ies and pro cedure, or even th e organizat ional

c l imate .

- Share fo l low-up evalua t ions w i th par t ic ipants managers or superv isors. These

ind iv idua ls shou ld know about p rogram r esu l ts and fo l low-up in form at ion and shou ld

be involved w ith t he part ic ipants pract ice and appl icat ion of t ra in ing.

Observat ions. Trainer o r anot her designated observer can actu al ly observe em ployees back on

th e job. Careful ly w atch emp loyees as th ey perfor m t heir rou t ine job tasks and responsibi li t ies.

To faci li tate th e process and ensure consisten cy in t he data gath er ing, create a checkl ist o f

desi red behaviours and t hen observe wheth er o r no t the em ployee is demonstra t ing these

behaviours

For exam ple, i f you are observing som eone w ho has recent ly attend ed a custom er

service program , your l ist o f custom er service behaviours might look l ike th is:

----- Sm ile

----- Gr ee t cu st o m e r w i t h Go o d m o r n in g o r G o od af t er no o n

----- U se s t h e cu st o m er s n am e .


15/86

15

----- Asks H ow m a y I help yo u?

----- O ff er s ad d it i on al assi st a nce

----- Gi ve s t h e cu st o m e r ch o ice s.

----- St a t es w h at w e c an d o, rat h er t h an w h at w e c an t .

In terv iews. Not o n ly shou ld you in terv iew t hose wh o w ent t h rough the t ra in ing , bu t you shou ld

interview th ose w ho are affected by or closely associated w ith t he program p art ic ipants.

Possible int erviewees include th e part ic ipant s managers, co-worker s, custo m ers, or

subordinat es. The int erview qu estion s w ould have t o be careful ly constructed and designed to

focus on speci f ic appl icat ion s and behaviour changes.

Surveys. Surveys are mo re eff ic ient and less expensive than int erviews to f ind o ut w het her the

part ic ipant s are actual ly applying w hat t hey learned. Once again, do not l imi t your sources of

in fo rm at ion . Others w ho in teract w i th t hose who par t ic ipated in t he t ra in ing are o f ten are

m ore rel iable source of feedback. You w i l l w ant t o know not only i f t ra inees are using the

tra in ing on th e job but a lso how they are using i t to per fo rm bet t e r .

A cont rol grou p is a lso h elpful f or a Level 3 evaluation. To val idate you resul ts, choose a

contro l g roup o f employees f rom the sam e funct ion as the par t ic ipants who received the

tra in ing. Ask the m em ber of th e cont rol group to com plete th e same surveys, question naires,

and tests as those comple ted by the p ar t icipants . Mo n i to r the i r job per fo rm ance and compare

i t w i th t ha t o f em ployees who rece ived t ra in ing .

Regardless of w hich evaluation m eth od(s) you use, m ake sure you al low enough t im e

for t he behaviour change to t ake p lace. The length o f t ime d epends on t he program, but th ree

to six mont hs shou ld g ive the par t ic ipants am ple oppor tun i ty to app ly what t hey learned and

develop new beh aviours.


16/86

16

Level 4: Resul t s (the im pact o f t he t ra in ing on the organ iza t ion)

Level 4 evaluation det erm ines th e imp act of th e tra in ing on t he organizat ion. Ideal ly, i t show s

how th e tra in ing has cont r ibut ed t o accompl ishing organizat ion al goals and object ives

business resul ts. I f the or ganizat ion choo ses to cond uct a Level 4 evaluation , the area of

m easurem ent m ust be the same as tho se identi f ied in the needs assessment .

To measure t ra in ings im pact on the bot tom l ine , re turn t o t he data gathered dur ing

th e needs assessment . Determ ine you cr i t ical success facto rs up-fro nt . Resul ts m easured could

inc lude any o f the fo l low ing or any measurab le i tem:

- Product ion ou t put

- Sales

- Operat ing cost s

- Cu st o m er sat isf act io n

- Qu alit y st an dard s

- Safet y records

- Turnover rat e

- Absen t eeism

- Em p lo yee gr iev an ces

- Em p l oy ee s sat i sf act io n

- Bu dget var ian ces

- Prom ot ions

Level 4 evaluation is bot h di f f icult and t im e-consum ing. I t can also be qu i te cost ly. It is

di f f icul t t o m easure because of th e many var iables th at can com e into play once the

part ic ipant s leave th e session. For t hese reasons, a Level 4 evaluation is not approp r iate f or a l lt ra in ing. From a pract ical stand point , consider Level 4 for t hese programs that are near and

dear to sen ior m anagem ent and have been ident i f ied as a top pr io r i ty .

Because of the com plexi ty of Level 4 evaluation and i ts infreq uency of u se, it is

addressed here only in ver y basic term s-just enough t o give you an idea of w hat i t involves and

th e purp oses i t serves. To explore Level 4 m ore det ai l , refer t o t he l ist ings in Append ix B,

includ ing Jack Phil l ips, Jane Ho lcom b, Don ald Kirkpatr ick and Dana Gaines Robin son and Jam es

Robinson.

As a star t ing po in t , test you cur rent kno w ledge o f m easurement te rm s by match ing the

term s in th e le f t -hand co lumn be low w i th the cor rect de f in i t ions in th e r igh t -hand co lumn (see

Appendix A for t he correct answ ers):


17/86

17

Term Def in it ion

1 ROI a. Su rvey dat a o f em p loyee at t it udes

2 Direct Cost s b . Salar ies and benefi t s paid to p art ic ipants for th e t im e they are

at tend ing th e program

3 In dir ect Co st s c. Exp en ses f or cr eat in g t he t rain ing p ro gr am

4 Overhead d . Involves quant i ty, qual i ty, cost, t im e; easi ly convert ed to

m onetary va lue

5 Deve lopment

Costs

e. Costs of shared r esources such as heating and b uilding

main tenance

6 Hard Dat a f . Expenses of oper at ing the tra in ing departm ent t hat are notdirect ly re lated to any t ra in ing program

7 So f t Dat a g. M athemat ica l fo rmula fo r ca lcu lat ing the d i f fe rence betw een

cost and value

8 Compensation

for Part ic ipant s

h . Costs re lated t o t he del ivery of th e for t ra in ing program, such as

m ater ia ls and trainer s salary

PARTICIPANT EVALUATION AN D ACCOUNTA BILITY FOR E-LEARNIN G

Unl ike tr adi t ional evaluations, e- learning evaluations are somew hat d i f f icul t because of d el ivery

opt ions a ind iv idua l learn ing so lu t ions tha t o f ten prevent consistency and un i fo rm i ty in t he

evaluation pr ocess. For exam ple, not a l l learners w i l l com plete sel f-stu dy courses from start t o

f in ish. Som e w i l l choose only the m odu les th ey th ink t hey need. Others may have to go back

and repeat a m odule to gain a greater understand ing of i ts cont ent.

For t he m ost part , however, you can apply the same pr incip les and strategies of

tradi t ional evaluation m etho ds and levels d iscussed ear l ier in t h is chapter t o t he e- learning

process. The basic d i f ference, of course, is th at t he var ious evaluation m etho ds wi l l be do ne

electronical ly. The fo l low ing are types of evaluations that can be adapted to an e- learning

env i ronmen t .

Level 1 evaluation m easures part ic ipants react ions met hods th at can be used include

questionn aires comp leted o n-screen w ith in t he course or as e-mai l feedback. Part ic ipant

reactions can also be captur ed via onl ine focus groups or in chat roo m s.


18/86

18

Level 2, w hich m easures w hat p art ic ipants actual ly learned, uses var ious tests. Alm ost

al l types of t ests can be adapt ed to an electron ic form at: t rue-fa lse, mult ip le-choice, essay, f i l l -

in-blanks, mat ching. Visi t W i l l iam Horto ns websi te (www.ho r ton .com ) for great examp les of

var ious test ing form ats. In addi t ion t o test ing, you can mon itor learners by observing th eir

behaviour in learning act iv i t ies such as sim ulat ions and learning games as w el l as ro le plays

conducted in chat room s.

To evaluate how w el l part ic ipants apply w hat t hey learned (Level 3), you w ould t urn t o

tradi t ional met hods such as observations of the em ployees on-t he- job perfor m ance, surveys

comple ted by t he par t ic ipants and o thers who in teract w i t h h im or her , and job per fo rm ance

records. i t m ight a lso be approp r iate to set up contr ol groups: som e emp loyees w ould

exper ience tradi t ion al classroom-based tra in ing w hi le oth ers w ould en gage in e- learning

activi t ies. Em ployees individual act ion p lans could also be m onit ored.

Level 4 evaluation fo r d istance learning is m uch th e same as i t is for tradi t ional learning

and w ould involve determ ining return on investm ent (ROI) as w el l as det erm ining benefi ts suchas a decrease in n um ber o f accidents, safety vio lat ions, tardiness, absent eeism , tur nover,

custom er com plaint s, or grievances. Business m et rics such as pro fit abil i ty (sales, revenues,

prof i t) and f inancial health (sto ck pr ice, m arket share) might a lso be targeted f or Level 4

evaluation.

As w ith any evaluation process, i t is im port ant t o col laborate wi t h th e organizat ions

leaders to d eterm ine the success cr i ter ia. W hat specif ical ly do t he key people w ant t o m easure

and how w i l l th ey use the info rm ation? In addi t ion to t he basic question s addressed in the

section o n t he fou r levels of evaluation, you m ay also be interested in evaluating th e e- learning

process by establ ishing ways to m easure th e fo l low ing:

- Ho w of ten user s lo g in?

- Ho w lo ng t hey st ay?

- H o w l o n g b e fo r e t h ey r ece iv e a r esp o nse t o a q u est i o n?

- H ow e f f ici en t t h e o n li ne sy st em i s?

ACCOUNTA BILITY FOR TRAININ G

Throughout th e business wor ld, account abi l ity for a l l funct ions is increasing. Staff fun ct ions

such as tr a in ing are expected to p rove th eir contr ibut ion and value to t he organizat ion. Topexecut ives are dem anding tha t t ra in ing depar tm ents o f fe r p roof o f the i r w or th or t ake budgets

cuts. Tw o appro aches to m easur ing the value of t ra in ing are covered below : (1) cost/ benefi t

analysis and (2) retur n on investm ent.


19/86

19

Cost/ Benefit An alysis

The cost/ benefi t analysis looks at t he tot a l cost to pro duce a tra in ing program and at tem pt s to

quant i fy t he ben efi ts. Costs includes everything fro m th e needs assessm ent, t hrough d esign,

developm ent, del ivery, and f inal ly to fo l low -up. Bot h direct and indirect costs are used to

deter m ine the to ta l cost of t he program . As table 13.3 show s, det erm ining tra in ing costs iscomplex.

Peop le Facil i t ies M at er ials

Design and

Deve lopment

Salaries, benefit s, tr avel

for : course

developm ent , cler ical

suppor t o r consu l tan t

fee and expenses or

costs of cert i fy ing in-

house t ra iner fo r

purchased programs.

M arket ing brochures

Part ic ipants mat er ia ls

Inst ructor m anua l

Purchased resource

mater ia ls

Purchased pr ograms

Salaries, benefit s, tr avel

for : t ra iner(s) ,

part ic ipant s (average

salary), clerical suppo rt ,

consul tant f ees and

expenses

Room renta l

Equ ipm ent ren t a l

Refreshments

No tebooks

Folders

Tent cards

Paper

Penci l /pens

Fl ip chart s

Handouts

Fi lm rental /purchase

Transparencies/sl ides

Sta t ionary i t ems

Cert i f icates

Books

Art ic les repr int s


20/86

20

Total benefi t s of the p rograms m ay be reduced costs or increased revenues direct ly

att r ibut able to th e tra in ing. In many cases, how ever, benefi t s can only be est imated .

Subtract t he to t a l costs f rom the to ta l benef i ts o f ind t he net benef i t o r the t ra in ing

program . The program is considered a f inancial success i f the costs are low er t han t he ben efi ts.

Return on Investm ent

The re turn on t he investm ent s imply shows what the payback is fo r the t ra in ing program . To

determ ine re turn on investm ent , you must w a it t h ree to s ix mont hs and even longer fo r

operat ional resul ts. The form ula for determ ining ret urn on investm ent: ROI = (net pr ogram

benefi t s/pr ogram costs) x 100. Stat ed simply, i f t ra in ing program s fa il to show a reasonable

retu rn on t he com panys investment , futur e (or even current ) tra in ing in i t iat ives are at r isk. For

m ore info rm ation on det erm ining ROI, refer to Jack Phi l l ips boo ks listed in App endix B.

SIGNIFICANCE OF THE EVA LUATION PROCESS

Evaluation is a comp lex issue. For o ne reason, m any var iables enter into the eq uation . No

m atter ho w hard you t ry to f ine- tune the eva luat ion p rocess, the rea l ity is tha t e f fect can on ly

be est im ated, and econom ic benefi ts cannot be calculated precisely. As th e ro le of t ra in ing

continu es to change and tra iners reposi t ion th emselves as perfo rm ance consul tants, there w i l l

be m ore pressure to m easure th e effect iveness of tra in ing. The good new s how ever, is th at th e

f ie ld of tra in ing and developm ent cont inues to grow at rapid pace. This trend w i l l cont inue to

provide m any oppor tu ni t ies for tra in ing professionals, both int ernal and extern al , to m ake an

impact on t he growt h and deve lopment o f ind iv iduals and organ iza t ions th roughout th e w or ld .

KEY POINTS

- Evaluat i on mus t be t i ed to the needs assessmen t p rocess.

- Eva luat ion shou ld re f lect specif ic business- re la ted or per fo rm ance- re lated outcom es.

- Ev al ua t io n in an o n go in g p r o ce ss

- Level 1 evaluat i on measu res pa r t ic ipan t react i on

- Leve l 2 evalua t ion uses tests, observat ions, and in terv iew s to m easure par t icipant

learning

- Level 3 evaluat i on measu res behaviou r , tha t i s, how the pa r t i ci pan ts app ly wha t they

learned back on t he job

- Level 4 eva lua t ion m easu res resu l t s and t ra in ing s impact on t he bo t tom l ine

- A cost / bene f it ana lysi s l ooks a t the benef i t s o f t r a in ing p rogram compared to the costto produce a t ra in ing program.

- Re tu rn on i nvestmen t add resses the degree to w h ich the cost o f a p rog ram y ie lds a

reasonab le re turn on investm ent .

The tra in ing pr ocess is a never ending cycle: analysis, design, developm ent , imp lemen tat ion,


21/86

21

and evaluation . As m entio ned several t imes thro ughout t h is book, th e ro le of th e tra in ing

prof essional contin ues to evolve. Often th e tra iner f i l ls m any roles, including that of a

consu l tan t . W hether in te rna l o r externa l , the t ra in ing consu l tan t m ust a lw ays be m indfu l o f t he

pro cess and t he business of con sultin g.

Data Gatherin gs Str uctur es

Demo graphic Prof i le

List of Organizat ions (133 IPTA/ IPTS), 30% respond ent s from t ot al AKEPT nam e list

Figure 1 and t able 1 show s lecturer s gender of t he respondent s.

Figure 1 Lecturer s Gender

Table 1 Lecturers Gender

N %

M a le 375 42.62Female 505 57.38

43%57%

male

M ethodo logy o f Su rvey

Ex-AKEPT Candidat e(Self Evaluation)

Evaluator(Peer Evaluations)

Non-AKEPT Candidates(Self Evaluation)

Suggest Suggest

Evaluate

Evaluate


22/86

22

Figure 2 and Table 2 show s lecturers academic age of t he respondent s.

Figure 2 Lecturers Academic Age

Table 2 Lectu rer s Academ ic Age

N %

Under 25 9 1.09

2 5 2 9 106 12.023 0 3 9 240 27.32

4 0 4 9 279 31.69

5 0 5 9 216 24.59

60 + 30 3.28

under 251.09%

2 5 - 2 9

12.02%

3 0 - 3 9

27.32%

4 0 - 4 9

31.69%

5 0 - 5 9

24.59%

60 +3.28%


23/86

23

Figure 3 and Table 3 show s Teaching Colleagues of t he respondent s.

Figure 3 Teaching Peers and Colleagues

Table 3 Teaching Peers and Colleagues

N %

Professor 82 9.29

PM 135 15.30

Senior 375 42.62

Lecturer /Tutor 288 32.79

Prof

9.29%

PM

15.30%

Senior

42.62%

Lectu rer /

Tutor

32.79%


24/86

24

Populat ion ( l ist of Respondent )

IPTS/IPTA

AGENSI KAUNSELING

& PENGURUSAN KREDIT

BERJAYA UNIVERSITY COLLEGE

COSMOPOINT

GLOBAL INSTITUTE OF STUDIES

HELP UNIVERSITY COLLEGE

IIUM

IMPERIA INSTITUTE OF TECHNOLOGY

IMU

INCEIF

INNOVATIVE INTERNATIONAL

COLLEGE

INSTEDTINSTITUT ADVERTISING

COMMUNICATION TRAINING SDN. BHD

INSTITUT MAKANAN

MALAYSIA

INSTITUT MUTIARA MOPAR

INSTITUT PENGURUSAN

DINAMIK (KUCHING)

INSTITUT PROFESSIONAL BAITULMAL

INSTITUT SINARAN

INSTITUT TEKNOLOGI PERAK

INSTITUTE GLOBAL

MANAGEMENT SDN. BHDINTI UNIVERSITY COLLEGE

K.K. ARAU

K.K. BANDAR DARULAMAN

K.K. BANDAR PENAWAR

K.K. BUKIT BERUANG

K.K. CHENDEROH

K.K. KEPALA BATAS

K.K. KUALA LANGAT

K.K. KUALA TERENGGANU

K.K. KUANTAN

K.K. KUCHING

K.K. KULIM

K.K. LEDANG

K.K. MENTAKAB

K.K. PAYA BESAR

K.K. ROMPIN

K.K. SEGAMAT 2

K.K. SG. PETANI


25/86

25

K.K. TAWAU

K.K. TELUK INTAN

KBU INTERNATIONAL COLLEGE

KK

BUKIT BERUANG

KK

HULU LANGAT

KK

HULU SELANGOR

KK

SABAK BERNAM

KK BANDAR DARULAMAN

KK BAYAN BARU

KK CHENDEROH

KK HULU LANGAT

KK HULU SELANGOR

KK JELEBU

KK JEMPOLKK KUALA LANGAT

KK KUANTAN

KK LANGKAWI

KK MASJID TANAH

KK MENTAKAB

KK PASIR GUDANG

KK PAYA BESAR

KK SABAK BERNAM

KK SEGAMAT

KK SEGAMAT 2

KK SELAYANG

KK SUNGAI PETANIKK TANJUNG KARANG

KK TELUK INTAN

KK

BANDAR DARULAMAN

KK

LANGKAWI

KK

MASJID TANAH

KK

PAYA BESAR

KLIUC

KLMU

KOLEJ ANTARABANGSA

TEKNOLOGI LANJUTAN SARAWAK

KOLEJ ISLAM ANTARABANGSA

KOLEJ METROPOLITAN

KOLEJ PENGURUSAN

WARISAN


26/86

26

KOLEJ POLY-TECH MARA

KOLEJ TEKNOLOGI TIMUR

KOLEJ TEKNOLOGI YAYASAN ALOR GAJAH

KOLEJ UNITI

KOLEJ UNIVERSITI INSANIAH

KPTM KOTA BHARU

KPTM-BANGI

KPTM-KOTA BHARU

KPTM-KUANTAN

KUIS

LIM KOK WING UNIVERSITY

MAHSA UNIVERSITY COLLEGE

MAPCU

MASTERSKILL UNIVERSITY COLLEGE OF HEALTH SCIENCES

MEDIU

NILAI UNIVERSITY COLLEGE

OPEN UNIVERSITY

POLI BALIK PULAUPOLI JB

POLI KOTA K.TRGANU

POLI MERLIMAU

POLI MUADZAM SHAH

POLI MUKAH

POLI PORT DICKSON

POLI SEBERANG PERAI

POLI SULTAN AZLAN SHAH

POLI SULTAN HJ AHMAD SHAH

POLI SULTAN MIZAN ZAINAL ABIDIN

POLI SULTAN SALAHUDDIN ABD AZIZ SHAH

POLI SULTANAH BAHIYAHPOLI TUANKU SYED SIRAJUDDIN

POLI

MUKAH

POLI

SULTAN ABDUL HALIM MUADZAM SHAH

POLI

SULTAN AZLAN SHAH

POLI

SULTAN MIZAN ZAINAL ABIDIN

POLI

SULTAN SALAHUDDIN

ABDUL AZIZ SHAH

POLI

TUANKU SYED SIRAJUDDIN

POLI. KOTA

KUALA TERENGGANU

POLI. MERLIMAU

POLI. MUADZAM SHAH


27/86

27

POLI. PORT DICKSON

POLI. SANDAKAN SABAH

POLI. SULTAN ABDUL HALIM MU'ADZAM SHAH (POLIMAS)

POLI. SULTAN IDRIS SHAH

POLI. SULTAN SALAHUDDIN ABD AZIZ SHAH

POLI. UNGKU OMAR

POLISAS

POLITEKNIK

SULTAN SALAHUDDIN ABDUL AZIZ SHAH

RIAM INSTITUTE

OF TECHNOLOGY

SATT COLLEGE

SEGI COLLEGE SARAWAK

SEGI COLLEGE SUBANG JAYA

SELANGOR INTERNATIONAL ISLAMIC UNIVERSITY COLLEGE

SIDMA COLLEGE

SIRIM BERHAD

SUNWAY UNIVERSITY COLLEGETAFE COLLEGE

TATI UNIVERSITY COLLEGE

TWINTECH

UCSI

UDM

UIAM

UiTM

SEGAMAT

UiTM JOHOR

UITM KELANTAN

UITM MELAKA

UiTM PAHANGUITM PAHANG

UITM PERAK

UiTM PERLIS

UITM SHAH ALAM

UITM

JOHOR

UiTM

KELANTAN

UiTM

N.SEMBILAN

UiTM

P.PINANG

UiTM

PERLIS

UiTM

SABAH

UiTM

SEGAMAT


28/86

28

UiTM

TERENGGANU

UKM

UM

UMK

UMP

UMS

UMT

UNIKL

UNIMAP

UNIMAS

UNISEL

UNITAR SARAWAK

UNITEN

UPM

UPNM

UPSI

USIMUSM

UTAR

UTeM

UTHM

UTM

UUM


29/86

29

4.0 RESULTS

INTRODUCTION

M alaysia of Higher Education has embark seven tru sts of the Nation al Higher Education Action Plan,

(M OHE (2007), in term s of : broaden access and enhancing equi ty, Im proving t he qual i ty of teaching

and learning, enhancing research and innovatio n, str engthen ing the inst i tu t ion , intensi fying

inter nation al izat ion, encul turat ion of l i fe long learning and im proving del ivery system. From al l seven

tr usts, im proving t he qual i ty o f t eaching and learning is very crucia l in any u niversi ty in M alaysia.

Cruc ial in t e rms o f the o utcom es are necessary to be w or ld renow ned in sustainab le deve lopment

and in concentra t ion of t a len t , the abundance o f resources and the accu l tu ra t ion o f suppor t ive

governance (Salmi, 2006). According to Robin M iddlehurst (2008) she argued that 21 centu ry

context is d if fe rent f rom the past , we need create the ba lance betw een the ro le o f un iversi t ies in

th e wo r ld and integrate i t w i th the real issues happened in local . Universi t ies have pictur ed of th e

global izat ion and for t h is reason higher education w i l l need to ju st i fy i ts ro le & p art icular

contr ibu t ions.

In order t o m ake som e transform ation in t eaching and learning, M alaysia of Higher Education ,

th rough AKEPT Cent re for Teaching and Learning has been off er part ic ipants a prem ier exper ience

th rough pr ogramm es speci f ical ly designed to develop and enhance th e prof essional know ledge and

skil ls essent ial to success and excellence in higher ed ucatio n t eaching and learnin g. Objectives set by

AKEPT Cent re fo r Teaching and Learning, is first t o ensure t he staff o f local higher edu catio n

inst i tut ions are professional ly com pet ence to teach.

Second t o ensure higher education leaders are com peten t in developing inst i tut ional teaching and

learning strat egies by l inking research w ith pract ice and th ird object ives is t ensure th e academ ic

staff of local h igher education in st i tut ions are know ledgeable about and capable of st imu lat ing and

pract icing teaching and learning innovat ion t hrough bo th act ion and conceptual research.

From January 2008 up t o 2009, mo re than 8 pro fessional deve lopm ent p rogram m ers in i t ia ted by

AKEPT Centr e for Teaching and Learning and the im pact of a l l the tr a in ing programm ers cond ucted

and analyzed.


30/86

30

4.1 SIGNIFICANT RESULTS ACHIEVED:

There are som e signif icant f ind ing on:

4.1.1-Sectio n A: Dem ograph ic Pro file and React ion t o AKEPT Training (Level 1 and 2)

4.2. 1-Secti on B: HOW DO YOU PERCIEVE YOU R OW N S TEACHING PERFORM AN CE BEFORE

AN D AFTER TRAININ G (Level 2 )

Part 1 DECISION M AKING IN TEACHING A ND LEARNING

Part 2 KNOW LEDGE APPLICATION AN D DI SSEM INA TION

4.3. 1-Secti on C : HOW DO YOU OBSERVE YOUR SUBORDIN ATES TEACHIN G

PERFORM AN CE ON (Level 3 and 4)

4.1.1 Please refer to Zuraidah (AKEPT) w orks

4.2 .1 HOW DO YOU PERCIEVE YOUR OW NS TEACHING PERFORM AN CE BEFORE AND AFTER

TRAININ G (Level 2)

Pair Sam pl e t -t est fo r it em s BEFORE and A FTER AKEPT t rai nin g for ex-AKEPT

Pair Sig. (2

ta i led)

Pai r 2 be foreAkept_33_fo l low_up_to_change - a f te rAkept_55

After AKEPTs tr a in ing course, do you do m ore fo l low-up t o th e

teaching and learning changes process in your instruct ion t o m ake

sure i t is going in t he r ight d irect ion?

0.037

Pair 4 befor eAkept_35_super ior_involve - afterAkept _57

After AKEPTs tr a in ing course, how oft en do es your super ior involve

you in t he depart men tal teaching and learning process?

0.027

Pair 8 befor eAkept_39_confiden ce_in_TL - afterAkept _61

After AKEPTs tr a in ing course, do you have confiden ce in th e t eaching

and learning decisions you m ade?

0.037

Pair 17 beforeAkept_48 - a f te rAkept_70

To wh at extent have you had access to th e necessary teaching

resources (e.g. m ater ia ls from AKEPT) to apply th e t eaching

kno w ledge aft er AKEPTs trainin g cour se?

0.008

Pair 1 8beforeAkept_49 received-af te rAkept_71

To wh at extent have you received help from AKEPT, th rough m ater ia ls

0.001


31/86

31

from AKEPT, w i th regards to app lying the teaching know ledge aft er

AKEPTs training course?

Pair 1 9

beforeAkept_50 super io r encourage app ly a f t e rAkept_72

To what ext ent d id your super ior encourages you t o apply the ideas

and t echniques you have learned in th is AKEPTs tra in ing course afterAKEPTs training course?

0.003

Pair 2 0

befor eAkept_51 Peers and Col leagues encourage apply

a f te rAkept_73

To what ext ent d id another peer s / col leagues encourage you to apply

th e ideas and t echniq ues you h ave learned in th is AKEPTs tr aining

pro gram af te r AKEPTs training cour se?

0.034

Pair 2 1

beforeAkept_52 App ly the ideas and techn iques among peers

af te rAkept_74

To what extent d id you app ly the ideas and techn iques w i th your

peer s / colleagues after AKEPTs tr aining cour se?

0.031

4.2 .2- HOW DO YOU OBSERVE YOUR SUBORDINA TES TEACHING PERFORM AN CE (Level 3 an d 4)

Pair Sam ple t -test fo r it em s ex-AKEPT and N ON-AKEPT that has been evaluated by super ior

CONTENT AND CURRICULUM : Cont ent Know ledge of subject m att er

Pair Sig. (2

ta i led)

Pair 4 x_q13 - q13

relates cont ent t o oth er subject areas, and m akes content re levant to

student s everyday l ives.

0.024

Pair 5 x_ q1 4 - q 14

uses m ult ip le r esources including techno logies to enhance know ledge

of cont ent area(s).

0.003

Pair 6 x_q15 - q15

uses M OHE standards to create learning object ives, wr i t e lesson

plans, select appro pr iate m ater ia ls, d i rect h is/ her t eaching, and can

show evidence of how th ese standards are re lated to student

assessment.

0.025


32/86

32

KNOW LEDGE OF STUDENTS AND THEIR LEARNING: t he i nt ellect ual, social, ph ysical, and p erso nal

deve lopment o f a l l student s.

Pair Sig. (2ta i led)

Pair 1 x_q16 - q16

clear ly and consisten tly com m unicates approp r iate and chal lenging

expectat ions that resul t in h igher achievem ent f or a l l student s

0.000

Pair 2 x_q17 - q17

consisten tly dem onstrat es det ai led know ledge of learning theor ies

and incorpora tes knowledge o f s tudent learn ing in to t he content

areas by planning instruct ional strategies th at resul t in m easurable

increase in studen t learning

0.030

Pair 3 x_q18 - q18

consisten tly provides evidence of being sensi t ive, a ler t , and

responsive t o th e int e l lectu al , socia l , physical , and personal

deve lopm ent needs o f a l l s tudents

0.023

Pair 4 x_q19 - q19

consisten tly pr ovides evidence of adaptin g instr uct ion and

assignm ent s for each individual learner based on student s

environm ents inside and out side of school

0.001

Pair 5 x_q20 - q20

consisten tly provides appropr iate evidence of effect ively adapt ing

inst ruct ion by m aking mo di f ica t ions appropr ia te t o each s tudent s

stage of developmen t

0.013

LEARNING ENVIRONM ENTS: learnin g enviro nm ent s creatio ns th at encou rage posit ive socialinter act ion, act ive engagem ent in learning, and sel f-mo tivat io n.

Pair Sig. (2

ta i led)

Pair 2 x_q22 - q22 0.005


33/86

33

provides evidence of m anaging t im e, space, and learning resources to

prom ote act ive and equ i tab le engagement o f d iverse s tudents

Pair 3 x_q23 - q23

clear ly com m unicate c lassroom m anagem ent p lan to student s

0.008

Pair 4 x_q24 - q24

provides evidence of effect ively and consistently u sing intr insic and

extr insic mot ivat ional str ategies to engage student s in learning

0.008

Pair 5 x_q25 - q25

provides evidence of cond ucting classroom activi t ies th at incorporat e

in form at ion f rom mul t ip le cu l tu res

0.003

Pair 7 x_q27 - q27

lecturer s spoken and w ri t t en language incorporat es w el l -chosen

vocabulary that enr iches th e lesson and is approp r iate to student s

ages and int erests

0.003

ASSESSM ENT: com pr ehension and usage of a ran ge of for m al and info rm al assessm ent st rat egies to

evaluate and ensure the cont inuou s developm ent of a l l learners.


Pair 1 x_q28 - q28

consisten tly and successful ly u ses di f ferent typ es of assessment to

infor m instru ct ion. The lecturer has a thorou gh understanding of

m easurem ent t heory and correct ly interp rets assessm ent r esul ts.

0.000

Pair 2 x_q29 - q29

provides evidence of col lect ing and using pre-assessm ent data t o

select or d esign clear,signi f icant , and var ied stud ent learning goals

0.000

Pair 3 x_q30 - q30

provides evidence of consistent ly choosing, developing, or u sing a

var iety o f aut hent ic and tr adi t ional classroom-based assessm ent

0.002


34/86

34

m etho ds th at are approp r iate for m aking instruct ion al decisions.

Pair 4 x_q31 - q31

provides evidence of consistent ly involving stu dents in a var iety of

self-assessmen t act ivit ies.

0.020

Pair 5 x_q32 - q32

consisten tly develops and uses valid and equi t able grading

procedures and m akes appropr ia te adapta t ions tha t m eet t he needs

of a l l student s

0.001

Pair 6 x_q33 - q33

provides evidence of consistent ly using assessm ent data t o ef fect ively

com m unicate stu dent pr ogress to student s, parent s, and other school

personnel

0.003

Pair 7 x_q34 - q34

provides evidence o f consistent ly using resources, including

techno logy, to m aintain accurate and up-to-dat e records of stu dent

w ork, behaviors, and accom pl ishm ents to m eet individual student s

needs

0.000

Pair 8 x_q35 - q35

consisten tly u ses a var iety of assessm ents t o iden ti fy stu dent sstrengths and needs

0.003

PLANNIN G AND INSTRUCTION: instr uctio nal experien ces design and creat ion based on o w n

know ledge of content and curr iculum, stu dent s, learning environm ent s, and assessm ent .


Pair 1 x_q36 - q36

clear ly explains rat ionales for instru ct ional choices

0.001

Pair 2 x_q37 - q37

lecturer s lesson plans consisten tly show evidence of int egrat ing

know ledge of subject m att er, curr iculum, stu dent s, learning th eory,

0.000


35/86

35

and assessm ent

Pair 3 x_q38 - q38

consisten tly uses var ious instruct ional strategies to en gage and

support d iverse stu dent learners and provides mu lt ip le perspectives

on key concept s, problem s, and areas of kn ow ledge

0.000

Pair 4 x_q39 - q39

able to recognize m isconception s and m akes m odif icat ions before

and dur ing the lesson t o address student needs

0.000

Pair 5 x_q40 - q40

uses m ult ip le ro les that are appropr iate fo r instruct ion and

successfully

changes roles dur ing t he instru ct ional pro cess

0.000

Pair 6 x_q41 - q41

provides evidence of consistent ly using and mo nitor ing the

effect iveness of a var iety of

approp r iate m ater ia ls and resources to enh ance instruct ion f or

diverse learners

0.000

PROFESSIONALISM : recognit ion , part icipation in, and cont ribu tio n t o t eaching as a prof ession

Pair Sig. (2

ta i led)

Pair 2 x_q44 - q44

provides evidence of understanding obl igat ion s and respon sibi l i t ies

required by t he Higher Education Strat egic Plan 2006-2010,

prof essional or ganizat ion s eth ics docum ent s, and r elated p ol ic ies.

0.004

Pair 3 x_q45 - q45

examines personal teaching pract ices, and provides evidence of

increasing stu dent achievement as a resul t o f m odifying teaching

pract ices

0.002

Pair 5 x_q47 - q47 0.031


36/86

36

provides evidence of part ic ipat ion in p rofessional developm ent

oppor t un i t ies based upon input f rom o thers and career goa ls

Pair 6 x_q48 - q48

provides evidence of part ic ipat ing, beyond w hat is required, in school

and system project s and m akes substant ia l cont r ibut ions in schoo l

and system project s.

0.002


37/86

37

4.2.3 How im port ant AKEPTs teaching and learning program ? (Level 2-Learning )

Figure 4 The Im por tan ce of AKEPTs Teaching & Learnin g Pro gram

Table 4 The Im por tan ce of AKEPTs Teaching & Learning Program

Ex-AKEPT

%

Non-AKEPT

%

a) Plann ing in T & L 98.08 99.00

b) Innovat ion in M OHE 94.07 74.00

c) Exem p lary Lect urer 91.67 68.00

d) Pro fessional Train ing 96.08 93.00

e) Innovat ive in Teach ing & Train ing 96.15 99.00

f ) Case M et hod in ETC 90.39 80.00

g) SKTIP 78.02 68.00h) Teach ing & Learn ing W orkshop 94.87 87.00

i ) Six Th inking Hat 78.21 55.00

j ) Linking Research in Teach ing 89.71 80.00

0%20 %

40 %

60 %

80 %

100%a) Plannin g in T & L

b) Innovat ion in

M O H E

c) Examp lary

Lecturer

d) Profe ssional

Training

e) Innovat ive in

Teaching & Train ing

f ) Case M etho d in

ETC

g) SKTIP

h) Teaching &

Learning W orkshop

i) Six Think ing Hat

j) Lin k in g Re sear h

in Teaching

HOW IM PORTAN T TEACHING & LEARNIN G PROGRAM

Ex-AKEPT

Non-AKEPT


38/86

38

4.2.4 How about chances and reality to att end AKEPT tr aining? (Level 2-Re-action )

Figure 5 The Impor tance of Teaching & Learning Program An d Chances to At tend

Table 5 The Im port ance of Teaching & Learning Program And Chances to At tend

Perceive Reali t y

a) Plann ing in T & L 98.08% 9.20%

b) Innovat ion in M OHE 94.07% 7.20%

c) Exem p lary Lect u rer 91.67% 4.00%

d) Pro fessional Train ing 96.08% 6.40%

e) Innovat ive in Teach ing & Train ing 96.15% 22.80%

f ) Case M et hod in ETC 90.39% 11.60%g) SKTIP 78.02% 4.00%

h) Teach ing & Learn ing W orkshop 94.87% 14.00%

i) Six Th inking Hat 78.21% 14.80%

j) Li nkin g Re sear ch in Teachin g 89.7 1% 6.0 0%

0%

20%

40%

60%

80%

100%a) Plannin g in T & L

b) Innova t ion in M OHE

c) Examp lary Lecturer

d) Professional Trainin g

e) Innovat ive in

Teaching & Train ing

f) Case M eth od in ETC

g) SKTIP

h) Teaching & Learning

Workshop

i) Six Think ing Hat

j ) Li n ki n g Re sear h in

Teaching

HOW IM PORTAN T TEACHING & LEARNIN G PROGRAM

AN D CHANCES TO ATTEND

Perceiveness

Reali ty


39/86

39

4.2.6 How did you know AKEPT tra in ing program?

Figure 6 How AKEPTs Training Pro gram Is Know n

Table 6 How AKEPTs Training Program Is Kno w n

N Ex-AKEPT

%

N Non-AKEPT

%

Peers & Co lleagues 299 33.99 518 58.82

W W W 17 1.97 0 0.00

AKEPT Train ing Program 286 32.51 207 23.53

AKEPT W ebsit e 130 14.78 103 11.76

Ot hers 148 16.75 52 5.88

0%

10%

20%

30%

40%

50%

60%

Peers &

Collegues

W W W

AKEPT Trainin g

ProgramAKEPT W ebsit e

Others

How Did You Know AKEPT Train ing Pro gram

Ex-AKEPT

Non-AKEPT


40/86

40

4.2.7 Do yo u kno w th e role of AKEPT (Non-AKEPT)

Figure 7 Do You Know t he Role of AKEPT?

Table 7 Do You Know th e Role of AKEPT?

N %

Yes 330 37.50%

No 550 62.50%

Yes

38%

No

63%

Do You Know t he Role of A KEPT

(Non-AKEPT)


41/86

41

4.2.8 Have the part ic ipants ever app ly teaching know ledge based t hey gained fro m AKEPTs

tra in ing? (Level 3- Learning )

Figure 8 Teaching Technique Application

Table 8 Teaching Technique Application

Non-AKEPT

%

Ex-AKEPT

%

a) PBL 43.00 33.82

b ) Innovat ion T& L 31.00 27.94

c) e - Learn ing Techn ique 37.00 38.24

d ) M ult im ed ia In t eract ive 62.00 45.59

e) Dem onst rat ion Techn ique 42.00 45.58

f ) Case St udy 62.00 32.36

g) Cr it ic & Creat ive 36.00 25.00

h ) Exem p lary Techn ique 37.00 27.94

i) Know ledge & Dissem inat ion 55.00 33.82

0%

10 %

20 %

30 %

40 %

50 %

60 %

70 %a) PBL

b) Innovat ion T& L

c) e - Learning

Technique

d) M u l t imed ia

Interact ive

e) Demo nst rat ion

Techniquef) Case Stud y

g) Crit ic &

Creat ive

h) Examp lary

Technique

i) Know ledge &

Diss iminat ion

Have You Ever App lied Teaching Techni qu e

Non-AKEPT

Ex-AKEPT


42/86

4 . 2 .9 How a re t he pa r t i ci pan t s p

pe ers and col leagu es? (Level 3- in

Decision M aking in Teaching an

Figure 9 Know ledge App l i cat i on

Table 9 Kn o

1) Ask ing peer s for suggest ion r

2) Fo l low -Up th e T & L changes

3) Invo lves peers for ideas

4 ) Invo l ve pee rs in depa r t m en t

5) Invo lve peers in T & L

6 ) Tendency t o pu t o f f m ak ing

7 ) Gr o u p m e e t i n g w i t h p e e r s

8 ) Con f idence w i t h ow n dec i sio

9) Us ing p lanned T & L in dec is i

10 ) D id you t ake t ime t o eva lua

4) Involve peers

depar tmenta l T

process

3) Involves peers for

ideas

2) Fol low -Up the T & L

changes to r ight

d irect ion

1) Asking peers for

suggest ion regarding

L changes

K

42

r f o rm ing d i f f e ren t l y in o rde r t o i n f l uence changes

f luencing changes change)(sel f -appraisal)

d Learni ng

and Dec ision M ak ing I n T & L

w ledge App l i ca t i on and Dec ision M ak ing In T & L

After

%

egard in g T & L ch an ges 3 3

t o r igh t d ir ect io n 4 4 .68

44.60

l T & L p ro cess 5 1

4 6

& L d ecisio n 3 6

5 3

n m ak in g 4 8

n m ak in g p ro cess 4 8

e resu lt s in T & L 4 6

0%

10 %

20 %

30 %

40 %

50 %

60 %

10) Did you take t im e to

evaluate resul ts in T & L

9) Using plan ned T & L in

decision m aking process

8) Conf idence

decision m

7) Group m eet i

peers

6) Tendency to put of f

m aking T & L decision

5) Involve peers in T & L

in

L

&

now ledge Appl icatio n and DecisionM aking In T & L

m o n g t h e ir

Before

%

40

31

45

38

33

42

29

45

45

43

i t h o w n

king

ng wi th

Af ter

Before


43/86

43

DISCUSSION AND SUGESSION

How are the p ar t icipants per fo rm ing d i f fe rent ly in o rder t o in f luence changes am ong the i r

peer s and co lleagues? (Level 3-in fluen cing changes)(self-appraisal)

Decision M aking in Teaching and Learnin g

32/54. The resul ts show that most of the part ic ipants of the AKEPT programme responded higher

percentage of no changefor asking suggestion f rom th eir peers aft er th e program m e (33%; 40%) as

compared to before. This impl ies that academics may not feel comfortable asking or shar ing

information. The cul tural problem is in fact considered a major chal lenge in change management

in i t iat ives among the HEIs because many facul ty members consider knowledge as propr ietary and

som eth ing tha t is no t shared f ree ly (W ind & M ain , 1999). On the o th er hand, it m ay also be due to

the super io r i ty in knowledge tha t the par t ic ipants fe l t as compared to the i r peers af te r a t t end ing the

programm e w hich inhib i ts them f rom consu l t ing the i r peers.

33 / 55. M ost o f th e par t ic ipants sta ted t ha t a fo l low up w as done somew hat m ore a f te r the AKEPT

programme compared to before i t (31%; 44.68%). The resul ts imply that tra in ing provides better

direct ion to part ic ipants to check and balance their teaching and learning process.

34/ 56. The resu lts show tha t a f te r t he AKEPT program m e, most o f the par t ic ipants som ewhat d id

not consul t their peers in teaching and learning as compared to before the tra in ing (45%/ 44.6%),but th is f igure is just minimal. This could be due to the confidence the part ic ipants fe l t after the

programm e w hich may m ake them fee l super io r in know ledge as com pared to the i r peers, thus, less

consu l ta t ion . On the o ther hand, i t cou ld a lso be due to the knowledge hoard ing cu l tu re o f

academics (Wi ig , 1997) wh ich inh ib i ts them f rom know ledge shar ing .

35/ 57. Surpr isingly, m ost of the super iors remained th e same t ow ards th e part ic ipants in re lat ion t o

involving them in teaching and learning process, in fact the percentage increased for no change

(38%; 51%). This may be due to fear that super iors face in re lat ion to change in the management

th at th eir dow n l ine may br ing aft er the t ra in ing conduct ed (Tr ice and Beyer 1993).

36 / 58. M ost o f the par t icipants simi la r ly p ro jected no change, in fact m ore (33%;46%) w hen asked

how oft en t hey included col leagues and p eers in teaching and learning process. . This again could b e

due to the conf idence the par t ic ipants fe lt a f t e r the programm e which m ay make them f ee l super io r


44/86

44

in know ledge as com pared to th eir peers, th us, less consul tat ion. On the o th er hand, i t could also be

due to the knowledge hoard ing cu l tu re o f academics (Wi ig , 1997) wh ich inh ib i ts them f rom

know ledge shar ing.

37/ 59. M ost o f t he par t ic ipants pro jected h igher no changebefor e th e AKEPT tra in ing com pared t o

after (42%;36%) for th is question. This could be due to the benefi ts that are der ived from thetr ain ing that guides th em not to procrast inate teaching or research act iv i t ies.

38 / 60. M ost o f the par t ic ipants pro jected h igher no changebefor e th e AKEPT tra in ing com pared t o

a f te r (29%;53%) fo r th is quest ion . Th is cou ld be due to the f act tha t the t ra in ing make them v isionary

leaders on t he i r own and consu l ta t ion w i th peers may be seen as w aste o f t im e w hen th ey cou ld

f igure out p rob lems by t hemselves. However , there is a negat ive connota t ion t o t h is, in t ha t i t cou ld

be viewed that some facul ty members view knowledge as a possible source of d i f ferentiat ion, and

th us defer shar ing certa in aspects of their know ledge. Unfo rtu nately, however, when kno w ledge is

viewed as a source of power i t acts as a separator between the haves and the have-nots (Wi ig,

1999) and in som e cases, kno w ledge loss occurs.

39/ 61. M ost of t he part ic ipants respond ed that t heir confidence increased after t he AKEPT

prog ramm e somew ha t more compared to be fo re i t ( 45%; 48%). Al though th i s f i gu re may be

m in imal , bu t i t i s a step t ow ards m ould ing the par t ic ipants to be a conf ident educator .

40 / 62. M ost o f t he par t ic ipants responded t ha t t hey used p lanned approach in dec ision m aking

somewhat more after the AKEPT tra in ing compared to before i t (45%; 48%). Again al l though th is

f igure may be m in imal , bu t i t i s a step t ow ards m ould ing the par t ic ipants to be m ore system at ic in

th eir teaching and learning effor ts.

41 / 63. M ost o f the par t ic ipants responded th a t they too k t im e to eva luate the i r teach ing dec isions

somew hat m ore (43%; 46%) after t he AKEPT program m e compared t o befor e i t . This im pl ies th at th e

AKEPT tra in ing tra ins them to ref lect on their teaching decisions in order to be better knowledge

disseminators.


45/86

45

How are the part ic ipants perform ing di f ferently in order to inf luence changes amo ng their peers and

colleagues? (Level 3-inf luen cing changes change)(self-app raisal)

4.2.10 Know ledge Applicat ion and Dissemin at ion

Figure 10 Know ledge Appl icat ion and Dissem ination

Table 10 Knowledge Appl icat ion and Dissem ination

Befo re Af t er

Pe er / Co lleagu es en co ur age t o ap ply T & L 4 2% 3 9%

Superviso r encou rage t o app ly T & L gained f rom AKEPT 26% 32%

Received help f rom AKEPT on m at er ials 36% 27%

Access t o t he necessary T & L resources 36% 46%

Con t en t o f t his course accu rat ely ref lect w hat happens on your job 41% 34%

Act ually Dissem inate Teach ing Know ledge 51% 46%Act ually App ly Teach ing Know ledge 45% 45%

0%

10 %20 %

30 %

40 %

50 %

60 %

Peer / Colleagues

encourage to apply T & L

Superv isor encou rage to

apply T & L gained f rom

AKEPT

Received h elp fr om AKEPT

on m ater ials

Access to t he n ecessary T

& L resources

Cont ent o f th is course

accurate ly ref lect w hat

happens on your job

Actually Disseminate

Teaching Know ledge

Actual ly Apply Teaching

Knowledge

Know ledge Appl icat ion and Dissemin ation

Before

A f te r


46/86

46

How are the par t ic ipants per fo rming d i f fe rent ly in o rder t o in f luence changes am ong the i r peers and

colleagues? (Level 3-inf luen cing changes change)(self-app raisal)

Know ledge Applicat ion and Dissemin at ion

42/64. Surpr isingly, most of the part ic ipants appl ied teaching knowledge before and after the

programme s imi la r ly (45%; 45%) to modera te extent . knowledge app l ica t ion happens when

knowledge is appl ied to new si tuat ions where users can learn and generate new knowledge. In the

learning process, ther e should be analysis and cr i t ical evaluation t o generat e new patt erns for fut ure

use. Decision m aking at t he organisat ion al level , innovation, and custom er re lat ionship m anagem ent

are examples of d irect knowledge appl icat ion. The appl icat ion of knowledge may generate new

knowledge or update current knowledge that wi l l have to be stored (Ear l and Scott, 1999;

M orse,2000; Par ikh, 2001). Going by th is defin i t ion used in th is study, the resul t impl ies th at th e

part ic ipant s m ay st i l l not b e adept to t ransfer their know ledge in teaching and learning process; thu s

AKEPT has to faci l itate t h is by provid ing w orkshops on ho w th is could be do ne.

43/ 65. M ost o f t he par t ic ipants d isseminated t each ing knowledge to a m odera te extent bu t be fore

the AKEPT tra in ing programme, the percentage was more as compared to after i t (51%; 46%).

Knowledge dissemination involves personal is ing knowledge and distr ibut ing i t in a useful format to

meet the speci f ic needs of users. The knowledge is art iculated in a common language using tools

that are understood by al l users. The transfer/d issemination of knowledge is vi ta l to organisat ional

success, qual i ty, and competi t iveness. In universi t ies, publ icat ions, presentat ions, websi tes, whi te

papers, teaching and learning act iv i t ies, pol ic ies, and repo rt s are examp les of m echanism used to

dissem inate/ transfer know ledge (Ear l and Scott , 1999; M orse,2000; Par ikh, 2001). In re lat ion t o th e

defin i t ion used in th is study, the part ic ipants may not real ise the importance of d isseminating their

know ledge fo r t he bet t e rment o f t he un iversi ty . Th is cou ld be due to the cu l tu re o f in f rast ructure in

the inst i tut ion which inhib i ts them from shar ing their knowledge (Shar iml lah, Chong and Hanipah,

2010). Hence, AKEPT has to conduct courses w hich guides to w ards the cul tur e of know ledge shar ing

and the inst i tu t ions wh ich the par t ic ipants be long to may need to prov ide the in f rast ructure

conducive for knowledge shar ing for example f i le shar ing soft wares. On the other hand, a smal l

num ber o f th e par t ic ipants d isseminated t each ing know ledge to a great extent a f te r th e program m e

as com pared to befor e (8%; 16%). How ever, in order f or a large percentage of part ic ipants to fo l low

th is tr ens, th e suggestion above has to be account ed for .

44/ 66. M ost of the part icipants appl ied teaching know ledge of th e AKEPT tra in ing

the impact study on effectiveness of akept’s t&l training programmes

Documents