Online Measurement of Academic Programme Preferences (MAPP) for Dis

tance Learners in Hong Kong

Online Measurement of Academic Programme Preferences (MAPP) for Dis

tance Learners in Hong Kong

Wei-yuan ZHANG & Lettice AU YEUNG

The Open University of Hong Kong

Wei-yuan ZHANG & Lettice AU YEUNG

The Open University of Hong Kong

Project Team Members

Dr. Wei-yuan ZHANG Dr. David MURPHY

Ms. Sylvia HUI Ms. Annie CHAN

Ms. Lettice AU YEUNGMs. Elaine KWOK

Mr. Albert TSE (IT Support)Mr. Henry LIN (IT Support)

Funded by the PACRD The Open University of Hong Kong

Advisory Groups

The National Career Development Association (USA)

The National Career Guidance Association (Mainland China)

Department of Psychology and SociologyNapier University (UK)

BackgroundBackground

• Needs for life-long education

• Rapid development of ODL

• Increase in ODL programmes

• OUHK: 114 programmes

ProblemsProblems

How to help potential distance learners understand:

• their psychological characteristics

• programme requirements

• make wise academic choices

A New IssueA New Issue

Providing distance guidan

ce on programme prefere

nces for distance learners

Literature ReviewLiterature Review

Students’ needs

• Understand programme preferences

• Programme selection guidance

Employers’ requirements

• Understand interests and job requirements

• Good psychological preparation

Literature ReviewLiterature ReviewMethod: Psychological testing

UK: JIIG-CAL (Job Ideas and Information

Generator - Computer Assisted

Learning); CENTIGRADE

USA & Canada: SDS (Self-directed Search)

Literature ReviewLiterature Review

Existing academic preferences

tests

• Limited to Western countries

• Charge high fees

Literature ReviewLiterature ReviewDifferences in:

• Socio-economic structure

• Cultural background

• Educational system

• Family dynamics

• Linguistic considerations

PurposePurpose To develop the first online

self-directed standardized test on the measurement of academic preferences (MAPP) for potential distance learners at the OUHK

ValueValue Expected to:

• Enhance students’ satisfaction with

programme selection

• Possibly reduce the rate of student a

ttrition

Trait-factor theoryTrait-factor theory

Three steps:• Trait: individual

• Factor: distance education study pr

ogrammes

• Relationship between the two

DesignDesignStep 1: Identify personal attributes

Step 2: Establish minimum

requirements for DE study

programmes

Step 3: Design an online matching

computer programme

MethodMethod

Psychological

testing

InstrumentInstrument• Initial pool: 330 items

• Related school, social, family

and leisure activities, etc.

• Three parts: interests, abilities

and temperaments

Sampling TechniqueSampling Technique Multi-stage stratified cluster

sampling technique:

• Geographical distribution

• School academic levels

• Gender

SampleSampleAge of sample: 17 –19• Age 15: preferences stabilized

• Age 17: preferences established

Aiken (1997) Psychological Testing and A

ssessment, 9th Edition, p. 234.

• OUHK programmes: 17 years old and a

bove

Step 1:Step 1:

Designing draft instrument

• 330 items (school, social, family and leisure

activities) in the Hong Kong context

• Reviewed by team members

Step 2:Step 2:

Validating draft instrument

• Item validation by

49 psychologists/ career counsellors

• Results: 66 items were deleted

Step 3:Step 3:

Pilot study

• 264 remaining items were randomized

• Sample: 13 students (Forms 4 and 5)

• Minor changes in language made

Step 4: Step 4: 1st survey: Test-retest

• Sample size: 54 students

• Time interval: 4 weeks

• Reliability coefficient: 0.67

• 33 items were further discarded with item-total correlation < 0.4

• Remaining 231 items

• Reliability coefficient: 0.72

Step 5:Step 5:2nd survey: Identifying personal attributes

• Sample: 1,217 students

• Factor analysis

• 8 interpretable factors (attributes) were

identified

Step 6:Step 6:3rd survey: Verification of 8 attributes

• Sample size: 764 students

• Factor analysis

• Same 8 attributes identified

• 132 items retained

• Alpha reliability: 0.962

Step 7:Step 7:Determining minimum desirability ratings

• 42 Course Coordinators

• 115 OUHK programmes

• Minimum entry requirements

• Based on 8 personal attributes

• Method: Job analysis

Step 8:Step 8:

Developing the Online MAPP

• Java programming language

• UNIX (Solaris) Platform

• Oracle 8i database

• Data exported into Excel format

The MAPP InventoryThe MAPP Inventory

132 items, 8 personal attributes•Influential •Social•Helping •Explorative•Technological •Logical•Reflective •Enterprising

(Accounting for 41.6% of the total variances, P<.05)

Personal AttributesPersonal AttributesAttribute Definition

Influential An individual who possesses leadership qualities is well rounded, confident, organized and perceived as a role model among his/her colleagues. This individual is also articulate and persuasive and works well with people. This individual is creative and analytical and thus adaptable to changing conditions.

Social This individual is interested in how people communicate and function as a society. He/she is well read and critical of the information he/she gathers. Staying current and being well informed in political and social issues also characterizes this individual. This individual engages with others in social and political activities.

Helping This individual’s main interest is in helping people. He/she possesses good oral, communication and conflict resolution skills and is confident in his/her abilities. This individual is open-minded and understanding and works well with others.

Explorative This individual is interested in science and engineering. He/she possesses skills to manipulate objects and perform experiments. This individual possesses strong analytical skills. Such an individual has a preference to work in a laboratory.

Technological This individual is interested in machinery and electronics. He/she has strong hands on skills to fix and manipulate machinery, specifically computers. This individual’s computer skills are analytical, creative and intuitive, being able to develop programs and solve technical problems. This individual works independently.

Logical This individual is interested in numbers and problem solving. He/she examines problems in abstract ways and performs tasks systematically. He/she enjoys solving mathematical games with others. This individual is rational and would be a valuable team member in group situation.

Reflective This individual has interests in literature and linguistics. He/she is reflective and analytical about what he/she reads and has a broad vocabulary. This individual enjoys the challenges of learning new skills in reading, writing and speaking. This individual is patient and focused in his/her work.

Enterprising This individual is attracted to a career in business. He/she is keen to study and acquire knowledge in business related disciplines such as administration and management. This individual also has a good business sense and stays current with news in the business world. He/she has a goal to succeed and is well read on business people who have achieved such goals.

ReliabilityReliability

Table 3: Subscale internal consistencies

Factor Name No. of Items Cronbach’s Alpha

Split-Half Reliability

Influential 22 .92 .92

Social 13 .90 .86Helping 21 .91 .89Explorative 20 .93 .92

Technological 15 .93 .87

Logical 14 .93 .87Reflective 18 .90 .84

Enterprising 9 .90 .82

Reliability (summary)Reliability (summary)From Cronbach’s Alpha

• The internal consistency of the total score of MAPP was 0.962;

• Subscale internal consistency coefficients ranged from 0.90 to 0.93.

Split-half reliability • Split-half reliability coefficient of the MAPP sc

ale was 0.92• Subscale split-half correlations ranged from 0.8

2 to 0.92

ValidityValidity• Content validity - items

evaluated by 49 experts

• Construct validity -

established through a series

of factor analysis

Sub-scale inter-correlationsSub-scale inter-correlations

  Influential Social Helping Explorative Technological Logical Reflective

Social .46            

Helping .63 .36          

Explorative .22 .15 .15        

Technological .23 .16 .11 .46      

Logical .23 .11 .13 .57 .44    

Reflective .40 .51 .35 .08 .09 .03  

Enterprising .34 .66 .24 .07 .17 .17 .29

Table 4: Sub-scale inter-correlations

Sub-scale inter-correlationsSub-scale inter-correlations

Summary

•Correlation coefficients ranged from 0.03 to 0.66

•There was no indication of multicollinearity

StandardizationStandardizationTable 5: Gender differences in subscale mean scores

Type Male (N=958) Female (N=966) P-value

Mean SD Mean SD

Influential 3.24 .57 3.19 .52 0.24

Social 2.74 .72 2.71 .66 0.31

Helping 3.42 .56 3.57 .51 0.00**

Explorative 3.06 .75 2.66 .72 0.00**

Technological 3.05 .79 2.52 .66 0.00**

Logical 2.98 .75 2.64 .72 0.00**

Reflective 2.91 .65 3.06 .63 0.00**

Enterprising 2.64 .83 2.60 .75 0.30

****Significant at the (P<0.01) level

Standardization (Summary)Standardization (Summary)

Attributes without Gender differences

• Influential , Social and Enterprising

Attributes with Gender differences

• Males: Explorative, Technological, and

Logical

• Females: Helping and Reflective

To MAPP

Auto-data CollectionAuto-data Collection

Excel format

• For further validation & to

automatically update gender-

based norms

Student EvaluationStudent Evaluation

Sample: 227 students

Degree of usefulness

• Very useful/ useful: 50.2%

• Neutral: 45.4%

• Not useful/not useful at all: 4.4%

Student EvaluationStudent Evaluation

Degree of satisfaction

• Very satisfied/ satisfied: 56.7%

• Neutral: 37.9%

• Not satisfied /not satisfied at all:

5.4%

Student EvaluationStudent EvaluationThe MAPP could

help me in

Strongly agree/ agree

Neutral  

Disagree / strongly disagree

Mean SD N

Understanding my psychological characteristics

141 (62.1%) 80 (35.2%) 6 (2.6%) 3.64 .638 227

Understanding my study programme preferences

138 (60.8%) 76 (33.5%) 13 (5.7%) 3.62 .715 227

Widening my considerations of programme choices

119 (52.4%) 88 (38.8%) 29 (8.8%) 3.52 .806 226

Planning my future educational direction

115 (50.7%) 98 (43.2%) 14 (6.2%) 3.47 .712 227

Making wise educational decisions

95 (41.9%) 110 (48.5%) 22 (9.7%) 3.35 .752 227

CCs’ EvaluationCCs’ EvaluationPositive responses (10 CCs)

“It is a very useful tool, which provides an indication of which OUHK programmes could be considered”.

“You have taken the proper steps to validate the constructs. I think the instrument will be useful to individuals considering which programme to go into”.

CCs’ EvaluationCCs’ EvaluationNegative Responses (2 CCs)

“Students would have some idea on what to study before they use this system”.

“Most prospective students would not have the patience to go through the MAPP carefully. How should students choose when the system recommends more than one programme?”

• Gender differences in

personal attributes

• May be influenced by

different experiences and

Chinese culture

Improving distance guidance to

support ODL

• Study satisfaction

• Potential reduction of attrition

LimitationLimitationLiterature: age 17, preferences

established

Results from: Age 17 –19

Further data from users will be

collected and automatically exported

into excel format for further study

and validation.

Recent InformationRecent Information

Some open and conventional univ

ersities have started to develop in

ventories on academic programm

e preferences:

UKOU, CCRTVU, UNISA;

CUHK, BNU.