Audio Conferencing Dial In Line: 1-866-711-1482Participant Passcode: 7418049

Please, be considerate of other participants…Use your MUTE button (*6 if you don’t have a Mute

button)

22 Nov 10 – 1100 Central Time

Aviation Selection Test Battery (ASTB)

204/11/23

Overview

Background on ASTB Current ASTB administration rules APEX web-based test delivery

platform Release of ASTB-E

•New content•New administration rules

3

What is the purpose of ASTB?

A Testing All Officers Eligibility. B Developed by statistical methods to

predict the success of students in aviation officer training programs.

C Testing for applicants to join the JAG Program.

D. Identifying the trends of all Officer candidates.

Question # 1Question # 1

Answer: B

404/11/23

Aviation Selection Test Battery (ASTB)

Primary tool used to select student pilots and student flight officers for the USN, USMC, and USCG

Sections of the battery are also used to select candidates for OCS

ASTB was first implemented in 1942, with revisions in 1953, 1971, and 1992

In July 2004, three new forms (Forms 3, 4, and 5) were released to address a test compromise

504/11/23

ASTB Administration

Approximately 10,000 ASTBs administered every year

243 permanent custody locations: • OSOs, NRDs, NROTCs, Military Institutes, selected NAS

and MCAS in CONUS and overseas

Available in paper/pencil and secure web-based format via APEX platform

On-line delivery system for the ASTB (APEX) is in use at over 100 sites CONUS and OCONUS

6

Question # 2Question # 2

The ASTB tests which of the following categories?

A Spatial Apperception Test

B Navigational SkillsC VisionD Essay Writing

Answer: A

704/11/23

Current Structure & Content

The current battery (Forms 3, 4, & 5) consists of six multiple choice subtests

• Math Skills Test• Reading Comprehension Test• Mechanical Comprehension Test• Spatial Apperception Test• Aviation & Nautical Information Test• Aviation Supplemental Test

804/11/23

ASTB Score Components

Officer Aptitude Rating (OAR)• MST, RCT, MCT• Predicts academic performance in OCS• Range 20-80 (higher is better)

Academic Qualifications Rating (AQR)• Predictor of performance in academic portions of API and

ground school and attrition • Range 1-9 (higher is better)

Pilot & Flight Officer Flight Aptitude Ratings (PFAR & FOFAR)• Predictors of flight grades in primary flight training and

attrition • Range 1-9 (higher is better)

904/11/23

ASTB Minimums

USN Pilot: 4/5 (AQR/PFAR) USN FO: 4/5 (AQR/FOFAR)

USMC Pilot: 4/6 (AQR/PFAR) USMC ECMO/RIO: 4/6 (AQR/FOFAR)

USCG Pilot: 4/5 (AQR/PFAR)

Minimums specified in PAs 106, 107, 150D, & 150E

1004/11/23

ASTB Score and Flight School Grades

SNA AQR and Ground School Grades

20

30

40

50

60

70

80

USN USMC

Service

AP

I N

SS

3

4

5

6

7

8

9

SNFO AQR and Ground School Grades

20

30

40

50

60

70

80

USN USMC

Service

AP

I N

SS

3

4

5

6

7

8

9

SNA PFAR and Flight Grades

20

30

40

50

60

70

80

USN USMC

Service

Pri

mar

y F

lig

ht

NS

S 4

5

6

7

8

9

SNFO FOFAR and Flight Grades

20

30

40

50

60

70

80

USN USMC

Service

Pri

mar

y F

lig

ht

NS

S 4

5

6

7

8

9

NOTE: Error bars represent one standard deviation.

1104/11/23

ASTB & Attrition

- Current ASTB does an excellent job of predicting whether USN USNAs & USNFOs will attrite from training

AQR and SNA Attrition

0%

5%

10%

15%

20%

25%

30%

35%

40%

45%

3 4 5 6 7 8 9

AQR

% A

ttri

tio

n

USN

USMC

PFAR and SNA Attrition

0%

5%

10%

15%

20%

25%

30%

35%

4 5 6 7 8 9

PFAR

% A

ttri

tio

n

USN

USMC

FOFAR and Attrition for SNFOs

0%

5%

10%

15%

20%

25%

30%

35%

4 5 6 7 8 9

FOFAR

% A

ttri

tio

n

USN

USMC

1204/11/23

ASTB: The Bottom Line

The current ASTB is calculated to save the U.S. Government $38 Million in training costs annually

1304/11/23

ASTB Administration

Continued success of the ASTB depends on:1. Security of test materials2. Adherence to test procedures3. Timely processing of completed tests

1404/11/23

ASTB Security

Per OPNAVINST 1532.1, custodians of ASTB testing material shall:• Prohibit access to test materials by personnel who

may later wish to take the test• Prevent disclosure of any information about the test

and test materials• Perform strict accounting of all test materials before

and after each test administration, including page checks of all booklets

• NEVER give information to applicants about specific items on any subtests, or right/wrong answers for any test questions

1504/11/23

ASTB Security

Loss or Compromise of the ASTB Materials  (1)NAVCRUITDIST personnel shall immediately

contact NOMI (Code 0341), commercial 850-452-2257x1060, or DSN 922-2257x1060, and NAVCRUITCOM (N311).   

(2)Commanding Officers (a.) Direct an investigation of the circumstances

surrounding the loss or compromise of testing material

(b.) Provide a report to NAVCRUITCOM (N311) and (00IG) via the appropriate chain of command.

1604/11/23

Current ASTB Retest Rules

Limit of 3 administrations• One shot at forms 3, 4, &

5 Only the most recent score

counts Mandatory delay between

retests• 30 days between first and

second test• 90 days between second

and third test No waivers on retest

intervals, so older examinees should plan accordingly

Student AQR Means by Retakes

21.6 26.9 33.2 30.4 57.1 60

5.915.32

4.97 4.96 4.93

3.80

0102030405060708090

100

1 N=5954 2 N=2494 3 N=413 4 N=92 5 N=14 6 N=5

Test Count & Frequency

% A

ttri

tio

n

1

23

4

5

67

8

9

1704/11/23

ASTB Retest Rules

Examinees cannot replace parts of their scores• Retests are All or None

An illegal test (wrong form/too early) counts against an examinee’s limit even if not the examinee’s fault

Score waiver requests should go to PERS-435

1804/11/23

Test Merges

If an examinee takes the OAR (first three subtests of the ASTB) only, then decides s/he wants to be an aviator:

Within 90 days, examinee can take the 3 remaining subtests• does not count as an extra test against examinee’s

limit• If OAR given on paper, 2nd half web-based admin

requires assistance from NOMI 30/90 day retest clock starts on day of the 2nd half

administration

1904/11/23

ASTB Scoring

All ASTB scoring conducted by NOMI Mail completed tests to NOMI

• Include your e-mail address on the transmittal form so that the scores can be returned to you as quickly as possible.

Web-based ASTBs are scored automatically and immediately• Scores available for display and printing as soon as

the test is complete

2004/11/23

APEX.NET – Web-Based ASTB

APEX is a web-based platform for secure online test administration

Deployable to all permanent ASTB test sites System administrator issues user ID, user

password, and workstation ID APEX also available on a temporary basis to

non-permanent sites • Successfully used OCONUS & Shipboard

2104/11/23

How does APEX work? System requirements are minimal Requires a high-speed internet connection Installation requires admin user privileges Installs a small piece of client software on

the local machine

Examiner account granted upon verification of requestor ID

The application runs on the server Test content is downloaded

one item or test segment at a time, and erased upon completion of that item or segment

Official score letters can be printedimmediately upon test completion

2204/11/23

Release of ASTB-E

2304/11/23

Future Score Components NOMI delivered a major ASTB revision

• ASTB-E Current ASTB yields the following scores: Range:

• Academic Qualification Rating (AQR) 1-9• Pilot Flight Aptitude Rating (PFAR) 1-9• Flight Officer Flight Aptitude Rating (FOFAR) 1-9• Officer Aptitude Rating (OAR) 20-80

ASTB-E adds these new components:• Pilot Aviation Fit (PAF) 1-9• Flight Officer Aviation Fit (FOAF) 1-9

2404/11/23

Future Structure & Content

ASTB-E will consist of seven sections:• (1) Math Skills Test (MST)

• (2) Reading Comprehension Test (RCT)

• 3) Mechanical Comprehension Test (MCT)

• (4) Aviation & Nautical Information Test (ANI)

• (5) Naval Aviation Trait Facet Inventory (NATFI)

• (6) Performance Based Measures (PBM)

• (7) Biographical Inventory with Response Verification (BI-RV)

New Content

2504/11/23

Naval Aviation Trait Facet Inventory (NATFI)

NOMI has identified the trait scores that lead to success in aviation training, and is currently validating profiles of aviators mid-career, and upon selection for command for use in PAF/FOAF scoring.

The NATFI is a forced-choice personality assessment, consisting of 88 statement pairs:

• Some consist of two flattering statements

• Some consist of two unflattering statements

I am confident in my ability to learn new skills. I firmly believe that I am capable of excelling in a wide variety of situations.

I am more of a follower than a leader. I am almost always running behind schedule.

2604/11/23

What do these answers mean? Examinees establish a pattern of responses regarding

each trait.• No single answer on this test tells us who the

examinee really is. • The set of answers tells us where s/he stands on all

the traits we target. Training success data tells us which traits predict

success in training, and which are consistent with success in the fleet

Screen out candidates who have dangerously low scores.

Select candidates with those traits and you’ll get lower attrition and more career-level success

2704/11/23

Performance-Based Measure (PBM)

Measure of dexterity and multi-tasking skills

Tracking Tasks

Dichotic Listening

Tracking Tasks

• Presents a unique string of numbers & letters simultaneously in each ear

• Examinee focuses on the numbers heard in the designated ear

Left Ear

Right Ear

“R 8 N S M A 2 G B 7 F L 6”

“Y L 3 S R 4 F Z 9 X F 8 G”

2804/11/23

Biographical Inventory

A questionnaire of background experiences relevant to aviation

Self-proctored online BI will include Response Verification

• Will reduce faking• Will eliminate typo errors as an explanation for incorrect

info about experiences

If you selected a - d, indicate:

What year(s)? _________ What school(s)? _________

Sample Item: 1. What was your

involvement in high school sports?

a. Captain of a team b. Member of a team c. Participant in an

individual sport d. Worked in a supportive

role (e.g., equipment mgr) e. No involvement in high

school sports

2904/11/23

ASTB-E Administration Rules

3004/11/23

ASTB-E Administration

Parts of the ASTB-E can be administered in booklet format (for when computers are unavailable)

The final two sections must be administered online

(1) Math Skills Test (MST)(2) Reading Comprehension Test (RCT)(3) Mechanical Comprehension Test (MCT)(4) Aviation & Nautical Information Test (ANI)(5) Naval Aviation Trait Facet Inventory (NATFI)(6) Performance Based Measures (PBM)(7) Biographical Inventory with Response

Verification (BI-RV)

Pap

er

or

APEX

APEX

O

nly

An

y w

eb

-en

ab

led

PC

/lap

top

OA

R

3104/11/23

ASTB-E Administration Rules

Lifetime two-attempt limit at ASTB-E• Previous attempts at Forms 3, 4, & 5 do not

count toward this limit• Forms 3, 4, & 5 will be deactivated on the

day that ASTB-E goes operational• Form 3, 4, & 5 scores generated before

ASTB-E release will remain valid for a lifetime

• Form 3, 4, & 5 scores generated on or after ASTB-E release date will not be valid

3204/11/23

ASTB-E Administration Rules 180 day delay required before ASTB-E retest OAR Merges are still permissible within 90 days of

valid OAR administration Paper-to-Web Test Resumes are also legal within 90

days of valid paper exam date

Paper booklets will be issued in two forms: Forms 6 & 7• Web-based ASTB-E content has no form numbers• Subtests are administered in adaptive format.• Each examinee gets questions tailored to his/her

ability level.

3304/11/23

BI-RV Administration

Administration time for MST, MCT, RCT, ANI, NATFI, & PBM is approximately 2:45

BI-RV may take examinees an additional :45 to 2:00 BI-RV will be self-proctored

• During registration, system will generate a letter for the examinee with a username & password for accessing the BI-RV

• Examinee has 90 days from pw issue to complete the BI-RV on any computer

Complete ASTB-E scores will not be generated until the BI-RV is completed

3404/11/23

ASTB-E Merge Scenario 1

• (1) Math Skills Test (MST)• (2) Reading Comprehension

Test (RCT)• (3) Mechanical Comprehension

Test (MCT)• (4) Aviation & Nautical

Information Test (ANI)• (5) Naval Aviation Trait Facet

Inventory (NATFI) • (6) Performance Based

Measures (PBM)• (7) Biographical Inventory with

Response Verification (BI-RV)

OA

R

John Smith takes OAR on day 0

Day 90, decides he wants to be an aviator- He must complete ANI, NATFI, & PBM that day- Issued a BI-RV pw, and has 90 more days (i.e., until day 180) to complete it

3504/11/23

ASTB-E Merge Scenario 2

(1) Math Skills Test (MST)(2) Reading Comprehension Test

(RCT)(3) Mechanical Comprehension

Test (MCT)(4) Aviation & Nautical

Information Test (ANI)(5) Naval Aviation Trait Facet

Inventory (NATFI)(6) Performance Based Measures

(PBM)(7) Biographical Inventory with

Response Verification (BI-RV)

OA

R

If on Day 22, he decides he wants to be an aviator, how many days does he have to complete the ANI, NATFI, & PBM?

If he takes ANI, NATFI, & PBM on Day 45, how many days does he have to complete the BI-RV?

Joe Sixpack takes OAR on day 0

3604/11/23

ASTB-E Retest/Merge Scenario 3

(1) Math Skills Test (MST)(2) Reading Comprehension Test

(RCT)(3) Mechanical Comprehension

Test (MCT)(4) Aviation & Nautical Information

Test (ANI)(5) Naval Aviation Trait Facet

Inventory (NATFI)(6) Performance Based Measures

(PBM)(7) Biographical Inventory with

Response Verification (BI-RV)

OA

R

Jake Scully takes Full ASTB-E on Day 0He retakes OAR on Day 181. Is this legal?Day 268, decides he wants to be an aviator after all- By what day must he complete ANI, NATFI, & PBM ?If he takes ANI, NATFI, & PBM on Day 269- By what day must he complete the BI-RV?

3704/11/23

ASTB-E Resume Scenario 4

(1) Math Skills Test (MST)(2) Reading Comprehension

Test (RCT)(3) Mechanical Comprehension

Test (MCT)(4) Aviation & Nautical

Information Test (ANI)(5) Naval Aviation Trait Facet

Inventory (NATFI)(6) Performance Based

Measures (PBM)(7) Biographical Inventory with

Response Verification (BI-RV)

OA

R

Jane Goodall wants to be an aviator, but has no access to APEX.Her examiner gives her the ASTB-E booklet Form 6 (MST, RCT, MCT, ANI, NATFI) on Day 0.

Her scantron is received by NOMI & scored on Day 17.By what day does she have to complete the PBM? If she completes the PBM on Day 78, how long does she have to complete the BI-RV?

3804/11/23

ASTB-E Retest Scenario 5

(1) Math Skills Test (MST)(2) Reading Comprehension

Test (RCT)(3) Mechanical

Comprehension Test (MCT)(4) Aviation & Nautical

Information Test (ANI)(5) Naval Aviation Trait Facet

Inventory (NATFI)(6) Performance Based

Measures (PBM)(7) Biographical Inventory

with Response Verification (BI-RV)

OA

R

Bob Dooley takes the first six ASTB-E subtests online on Day 0, and is issued a BI-RV pw.He completes the BI-RV on Day 88. This becomes his completion date of record.When is he eligible to reattempt the ASTB-E?

He attempts ASTB-E Booklet Form 7 on Day 193.Upon receipt by NOMI, this is recorded as an illegal retest, and Bob is eligible for no more attempts. His Day 88 score remains his ASTB-E score of record.

3904/11/23

The Gouge

If this seems confusing, just use APEX for all ASTB-E administrations.

APEX will not allow you to give an illegal exam. APEX will allow you to look up an examinee’s

complete test history, including paper tests attempted.

4004/11/23

Computer Adaptive Tests

Computer adaptive tests (CATs) tailor items given to an examinee to that examinee’s ability level• CAT provides more accurate measurement

using fewer questions Four existing subtests will be

administered in CAT format in the revised ASTB

CAT also provides improved test security – no two test forms are alike

Com

pu

ter-

Ad

ap

tive

Test

s

ICC Plots for 3-PL Form 3

0.0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1.0

-3 -2.5 -2 -1.5 -1 -0.5 0 0.5 1 1.5 2 2.5 3

THETA

P(U

=1|T

HE

TA

)

4104/11/23

Will the ASTB-E Improve Diversity?

Existing ASTB subtests: similar to the SAT, ACT, MCAT, etc.• Cognitive ability tests have

historically been the best predictors of job/task performance

• Have been used because they are easy to measure

Title VII protected groups score lower on these cognitive ability tests

Title VII protected groups DO NOT score as low on personality & psychomotor tests • These are also good predictors of

job/task performance• These have historically been

difficult to measure Adding these measures to the ASTB

(not previously possible) is expected to improve minority pass rates without reducing predictive power

Compensatory Scoring

4204/11/23

Fake-Resistant Engine

The engine is software that delivers adaptive forced-choice personality tests• The statements you see later depend on what

you said earlier• You see two statements at a time, and must

choose the one that describes you better• The statements are always equally “desirable”

Faking answers on this engine is as hard as counting into a seven-deck shoe

The next 2 slides illustrate what’s so new about this

Pers

onalit

y T

rait

s

4304/11/23

Historically, the biggest problem with personality measurements is that they can be faked:

These Likert-scale types of items frequently have obviously “correct” answers.

Historically, the biggest problem with personality measurements is that they can be faked:

These Likert-scale types of items frequently have obviously “correct” answers.

Examinees will lie if they can.

Aggressiveness

Adaptability

Pers

onalit

y T

rait

s

4404/11/23

The ASTB-E engine uses a new capability: forced-choice items where each statement is equally “desirable”.• Not previously possible due to math limitations of existing

models

Pers

onalit

y T

rait

sWork Motivation

Confidence

Unrestraint

Warmth

4504/11/23

How do new ASTB-E measures improve prediction?

Biographical

Experiences

Training Performanc

e

Cognitive Ability

Cognitive ability can predict a big chunk of training performance by itself, which is what the ASTB already gives us

Psychomotor ability also predicts a lot of training variability, but note the overlap with cog ability

• This area represents the extra predictive power we get from psychomotor ability – good, but not enough.

• An aviation trait test captures a great deal of training performance, with minimal overlap from other measures – which is what you want.

Psychomotor Ability

Aviation-Relevant

Traits

4604/11/23

ASTB Information Web Site

http://www.med.navy.mil/sites/navmedmpte/nomi/nami/Pages/ASTBOverview.aspx

Hosted on NOMI website for Internet access and linking by CNRC, ROTC, USNA etc.

Provides access to:• ASTB Study Guide w/ Sample Items• APEX Users Guide w/ Screen Shots• ASTB & APEX FAQ list

4704/11/23

Summary

Background on ASTBCurrent ASTB administration rulesAPEX web-based test delivery platformRelease of ASTB-E•New content•New administration rules

4804/11/23

Questions?Contact:

NCC Carlin NCC Carlin Jason.carlin@navy.mil

(901)874-9297

For copies of this Webinar Training send email to:

Jason.carlin@navy.mil