Presented by Dr Clare Hourigan University Planning & Statistics, Office of Planning & Quality

The use of the general achievement test in student

selection at Monash University:

A preliminary evaluation

The Current Higher Ed Environment

• 2008 Review of Higher Education– Increase bachelor qualification attainment (40%

of 25-34 year olds by 2025)

– Increase participation of low SES students (20% of undergrad population by 2020).

• TEQSA– New round of quality audits are likely to focus on

student performance

Role of Admission Policies

• Admission policies will play important role in meeting these challenges.

• Aim is to grow and diversify without compromising on quality.– Important to continue to select students that

are likely to succeed

– Must aim for a student population diverse in social background but not academic ability.

Use of Year 12 results in selection

• Year 12 results (ENTER in Vic) current primary tool used for undergrad student selection.

• Best predictor of university performance that we have but there is room for improvement.- Monash internal analysis shows it can account for

35.4% of variance in first year marks but varies by course/field of education (Hyndman & Akram, 2006)

Use of Year 12 results in selection

- Reliance on Year 12 results may be part of the reason why low SES students are under-represented at university.- Year 12 completion rates lower (James, Anderson, Bexley,

Devlin, Garnett, Marginson & Maxwell, 2008)

- and ENTER scores are lower among low SES students (Birrell, Rapson, Dobson, Edwards & Smith, 2002)

Use of Year 12 results in selection

• Why do Low SES students obtain lower year 12 results?– Aspirations/encouragement from

peers/family/teachers (James, 2002)

– School resources (Edwards, Birrell & Smith, 2005)

Use of Year 12 results in selection

• Less prepared for university or do their marks under-reflect their ability/potential?– Study of Monash students found, when

ENTER was controlled for, students from non-selective Govt schools performed best (Dobson & Skuja, 2002)

• Challenge for universities is to compensate for disadvantage to the extent that it has resulted in Year 12 results that under-reflect likelihood of success at uni.

Current policies to compensate for disadvantage• Special Entry Access Scheme in Vic

– Bonus points given to applicants for different equity categories including socio-economic disadvantage.

– Systematic approach but qualitative in judgement of extent to which Year 12 results have been affected.

Current policies to compensate for disadvantage

• Under-represented schools– Assumes all students from particular

schools were disadvantaged in the same way.

- 2004 Admissions to Higher Education Review UK- Recommended aptitude tests be considered

by universities in student selection to ‘help uncover hidden talent’.

Aptitude Tests

• UK recommendation based on pilot of Scholastic Assessment Test (SAT).• A levels and SAT both influenced by SES (high SES

students do better) but measure different aspects of academic ability because some students do well on one but not the other.

• Students from schools with lower A levels least likely to score well on both tests (21% compared to 44%).

• If uni selection used both A levels and SAT results, uni offer rates for students from low attaining schools would double.

Aptitude Tests

• Monash started considering using admission tests to enhance selection.– In 2007 trialed the UniTest at Berwick

Campus but can be administratively costly and places extra burden on students.

– GAT examined because it’s undertaken by all VCE students

Aptitude Tests

• Monash undertook analysis into predictive validity of the GAT on 1st year marks.– ENTER is best predictor (35.4% of the

variance)

– GAT also strong predictor (32.3% of the variance)

– Using both GAT and ENTER can improve predictability by 3% and predictive nature of GAT is stronger at lower ENTER ranges.

General Achievement Test

- In 2008, Monash commenced 3 year pilot to use the GAT in middle band selection of VCE applicants with the aim to:- Differentiate between middle band applicants to

select those most likely to succeed

- Identify and select students likely to succeed at university but had underperformed in year 12 due to personal circumstances or educational disadvantage

Use of GAT in selection at Monash

- In 2009, GAT predicted ENTERS (GPENTERS) were used in middle band selection.- Middle Band: Approx 20% of places set aside for

assessment on range of extra criteria- Performance in selected VCE subjects- SEAS bonuses

- Bonus aggregate points (2 to 9 points) where applicants GPENTER > ENTER

Use of GAT in selection at Monash

GAT Preliminary Evaluation

• Based on students who commenced in 2009 as selection method changed.

• Did use of the GAT assist in:– selecting applicants likely to succeed who

otherwise would not have been selected?

– diversifying Monash’s commencing student cohort with regards to socio-economic status?

Performance of GAT offer students

Type of Offer Av. Mark

N Sig. Progress Rate

N Sig

GAT based offer (A) 60.4 118 82.9 121

Standard – app above clearly in (B) 67.1 2733 A C 91.6 2754 A C

Middle band – app below clearly in (C) 61.9 424 87.2 428

Range of Critiera/no clearly-in (D) 69.8 548 A B C

95.8 551 A B C

Total 66.7 3823 91.4 3854

First semester average marks and progress rates among domestic VCE school leaver students commencing a bachelor’s pass degree at Monash University by offer type, 2009

Can GAT mediate for effect of SES on ENTER?• Examined relationship between SES and

ENTER and SES and GPENTER

Can GAT mediate for effect of SES on ENTER?• Perhaps differences in ENTER/GPENTER due

to true differences in potential to succeed.– If true then uni performance should not differ by

SES status when ENTER/GPENTER is controlled for.

• Used linear regression model to examine relationship between ENTER/GPENTER and SES status and uni performance (first semester marks).

Can GAT mediate for effect of SES on ENTER?

• R Square = 0.21, Standard Error = 11.32• Extent to which year 12 results under-reflect

uni performance similar for low and medium SES students

• Perhaps Year 12 results of high SES students are inflated

ENTER MODEL: Average Mark = 0.96 + 0.74xENTER + 2.28xLow SES + 1.95xMedium SES + e1 (2.1) (0.02) (0.65) (0.41)

Can GAT mediate for effect of SES on ENTER?

• When GPENTER used: variation across SES status was not significant.

• GPENTER was not particularly strong predictor of uni peformance but less likely to under-reflect likelihood of success of low and medium SES students

• R Square = 0.08, Standard Error = 12.18

GPENTER MODEL: Average Mark = 41.1+ 0.31xGPENTER + 0.82xLow SES + 0.41xMedium SES + e1 (1.41) (0.02) (0.7) (0.44)

Did the GAT help Monash select more low SES students?

• 20,830 VCE school leavers applied to Monash in 2009

• Examined eligibility for GAT bonus• Middle band offer rates by SES status

Did the GAT help Monash select more low SES students?

• Overall, low (and medium) SES students more likely to be qualify for a GAT bonus

• However, many of the low SES applicants did not meet the eligibility score ruling.

• Among applicants above eligibility score, qualification for GAT bonus almost same for low and high SES applicants

Did the GAT help Monash select more low SES students?

• GAT offer rates among middle band applicants were:– Low: 2%

– Medium: 2.3%

– High: 1.6%

• GAT did not assist Monash in diversifying the student population with regards to low SES status predominately due to eligibility score ruling.

Conclusion• Findings are preliminary and we will be

monitoring this as the pilot progresses.• Results are promising

– 163 students who would not otherwise have been selected were given an offer

– These students performed as well as other middle band students.

Conclusion• The use of the GAT did not help diversify

Monash’s student population with regards to SES.

• This was predominantly due to the eligibility score ruling. Many of the low SES applicants did not meet the eligibility score.– Results may differ for universities with wider

range of eligibility scores