1

Pilot Candidate Selection Method (PCSM)

Program Status16 May 2006

Dr. Thomas R. Carretta

Air Force Research Laboratory

Wright-Patterson AFB, OH

Thomas.Carretta@wpafb.af.mil

2

Pilot Candidate Selection Method

Air Force Officer

Qualifying Test

Basic Attributes Test

Previous Flying Experience

• Cognitive ability• Aviation-related

knowledge

• Psychomotor• Cognitive• Attitudes

+

+

• PCSM is a regression-

weighted pilot

aptitude composite• Operationally

implemented by

USAF in 1993

3

T-37 Outcome by PCSM Quartile

0

100

200

300

400

500

600

700

1-25 26-50 51-75 76-99

PCSM Score

Fre

qu

ency

(N

)

Eliminees

Graduates

Grad %: 58.5% 69.7% 81.8% 92.1%

64.1% Grad 89.3% Grad

N = 1,268

4

PCSM Changes

• All three PSCM components are undergoing change

• Air Force Officer Qualifying Test

– July 2005: Form S implemented; dropped 5 of 16 subtests and created new composites

• Basic Attributes Test

– July 2006: Scheduled to be phased out and replaced with the Test of Basic Aviation Skills (TBAS)

• Flying Experience

– Flight screening program has undergone several changes since 1993

5

AFOQT (Forms O, P1/P2, &Q1/Q2)

• Paper-and-pencil test

• Approximately 380 items in 16 subtests are combined into 5 composites:

– Verbal

– Quantitative

– Academic Aptitude (Verbal + Quantitative)

– Pilot

– Navigator/Technical

• 1978 norms based on active duty population

• New form implemented approximately every 8 years

6

AFOQT (Forms O, P1/P2, & Q1/Q2)

Composite ________________________________________ Verbal Quantitative Academic Pilot Navigator- Aptitude Technical Test Abbr. (V) (Q) (AA) (P) (N-T)________________________________________________________________________

Verbal Analogies VA X X X Arithmetic Reasoning AR X X XReading Comprehension RC X X Data Interpretation DI X X X Word Knowledge WK X X Math Knowledge MK X X XMechanical Comprehension MC X XElectrical Maze EM X XScale Reading SR X XInstrument Comprehension IC XBlock Counting BC X XTable Reading TR X XAviation Information AI XRotated Blocks RB XGeneral Science GS XHidden Figures HF X________________________________________________________________________Note. Abbr. means abbreviation.

7

AFOQT: What is Measured?

g

Verbal Math Spatial AviationPercep.Speed

ARVA RC WK GS DI MK SR MC EM HF IC AI TRBC RB

8

AFOQT: What is Measured?

• Confirmatory factor analysis:

– Like other cognitive tests, AFOQT has a hierarchical structure

– Higher-order factor is g

– 5 lower-order factors:

• Verbal

• Math

• Spatial

• Aviation Knowledge

• Perceptual Speed

67%

12%

4%

4%

9%4%

g

Verbal

Math

Spatial

Aviation

Per. Spd.

67% of the common

(explained) variance

is due to g

9

AFOQT: What is Predictive?

• Olea & Ree (1994) examined the predictiveness of g, specific ability, and specific knowledge from the AFOQT for pilot trainees (N = 1,867 to 3,942)

– Predictors were unrotated principle components from AFOQT

– Training criteria included academic grades, check flight grades, an overall performance composite, and a passing/failing score

– g was the best predictor for all pilot training criteria. The average validity of g was .31

– Specific knowledge (IC & AI tests) showed incremental validity beyond g of .098

– Specific ability (verbal, math, spatial, perceptual speed) showed little increment to g, about .02

10

AFOQT Form S

• Goals:

– Reduce test administration time

– Maintain composite score reliability and validity

– Retain previous factor structure (verbal, quantitative, spatial, aviation knowledge, perceptual speed)

– No adverse impact due to changes in battery

• Results:

– Dropped 5 subtests reducing administration time from 4.5 to 3 hours (RC, DI, MC, EM, SR)

– Retained reliability and validity of composites as well as factor structure

– Added Self-Description Inventory (SDI+) to assess personality factors (resultant test is 3.5 hours)

11

AFOQT Form SReliability & Validity Effects

• Intercorrelations between old and new Composites

Form Q Form R /S

Verbal .97 .96

Quant .97 .98

Acad Apt .98 .98

Pilot .97 .92

Nav-Tech .98 .94

• Little reliability or correlation loss caused by removing RC, DI, MC, EM, and SR

• Validity of new Pilot composite versus T-37 performance was comparable to old Pilot composite

12

AFOQT Form S

Composite

______________________________________

Test Abbr V Q AA P N/T

_________________________________________________________________________

Verbal Analogies VA X X X

Arithmetic Reasoning AR X X X X

Word Knowledge WK X X

Math Knowledge MK X X X X

Instrument Comprehension IC X

Block Counting BC X

Table Reading TR X X

Aviation Information AI X

General Science GS X

Rotated Blocks RB

Hidden Figures HF

Self-Description Inventory SDI+

13

AFOQT Form SSelf-Description Inventory (SDI+)

• Personality inventory based on Goldberg (1990) adjective clusters

– 220 items consisting of trait descriptions (behavioral statements)

– Captured Big Five personality dimensions and 2 additional factors – Service Orientation (SO) & Team Orientation (TO)

• SO – organizational commitment

• TO – preferences for working as member of a group or working alone

– Subsequent factor analysis confirmed Big Five factors and Service Orientation factor

• Team-Orientation factor not separate from other factors

• Proposed to be used for career counseling

14

Basic Attributes Test (BAT)

• Basic Attributes Test (BAT) measures:

– Psychomotor

– Cognitive

– Personality/Attitudes

• In operational use since 1993

– 386-based computer

– Test hardware can no longer be maintained

– To be replaced by the Test of Basic Aviation Skills (TBAS)

15

Test of Basic Aviation Skills (TBAS)

• Test of Basic Aviation Skills (TBAS) measures:

– Psychomotor

– Cognitive (spatial)

– Multi-task performance

– Memory

• Unlike BAT, several TBAS subtests are related to one another

• Scheduled for operational implementation in July 2006

16

TBAS Status

• Preliminary TBAS-based PCSM scoring algorithm completed

– A regression-weighted composite was developed using AFOQT, flying experience, & TBAS scores

– TBAS scoring algorithm designed to counter test-taking strategies (e.g., concentrate on part of multiple-task test)

– Predictive validity is comparable to BAT-based PCSM equation

• Ongoing tasks

– Development of test materials (e.g., operator’s manual, examinee information pamphlet)

– Compliance with professional test standards (e.g., APA)

– Deployment of test equipment

– Development of long-term sustainment plan

17

Flying Experience

• PCSM Flying Hour Code is an unequal interval scale based on examination of relation of previous flying hours with UPT performance

• Code categories:

– 0: 0 flying hours

– 1: 1 to 5 flying hours

– 2: 6 to 10 flying hours

– 3: 11 to 20 flying hours

– 4: 21 to 40 flying hours

– 5: 41 to 60 flying hours

– 6: 61 to 80 flying hours

– 7: 81 to 100 flying hours

– 8: 101 to 200 flying hours

– 9: 201 and Up flying hours

18

Flying Experience

• Prior to 1994: Flight Screening Program

– 15 to 20 hours in T-41

– Conducted by USAF at USAFA and Hondo, TX

• 1994-1997: Enhanced Flight Screening Program

– 21 hours in T-3 (Slingsby Firefly) & included more aerobatic training

– Conducted by USAF at USAFA and Hondo, TX

– Discontinued following several uncommanded engine stoppages

• 1997-1998: No flight screening program

• 1998: Introductory Flight Training Program

– Syllabus expanded to 40, then 50 hours

– Taught at Civilian training schools

• Present: Introductory Flight Screening Program

– 25 hours conducted by contractor instructors

19

Future Directions

• TBAS-based PCSM post-implementation S&T issues

– Subgroup differences

– Test-retest performance

– Extension of PCSM to other career fields (e.g., UAV operators)

• Examine utility of AFOQT SDI+ for career counseling

• Collaboration with the US Navy

– TBAS and iTBAS (internet TBAS) comparability

– Examine relations between USAF and USN officer commissioning & aircrew selection tests

• Development of common test battery

– Computer-administered AFOQT

20

Questions?

21

AFOQT1. Verbal Analogies

DIRECTIONS: This part of the test measures your ability to reason and

see relationships between words. You are to decide the answer that best

completes the analogy developed at the beginning of each question.

1. FINGER is to HAND as TOOTH is to _____.

a. tongue b. lips c. nose d. mouth e. molar

2. RAQUET is to COURT as __________.

a. tractor is to field b. blossom is to bloom c. stalk is to prey

d. plan is to strategy e. moon is to planet

3. FLY is to AIRPLANE as __________.

a. drive is to stake b. skate is to slide c. push is to fall

d. swim is to float e. rod is to hook

22

AFOQT 2. Arithmetic Reasoning

DIRECTIONS: This part of the test measures mathematical reasoning. It

is concerned with your ability to arrive at solutions to problems. Each

problem is followed by 5 possible answers. Decide which of the 5 answers

is most nearly correct.

1. If an aircraft travels at 564 miles per hour, how many miles did the

aircraft fly in 900 seconds?

a. 141 b. 226 c. 300 d. 451 e. 846

2. A typist took 3 typing tests. The average typing speed on these tests

was 48 words per minute (WPM). If the typists speed on 2 of these tests

was 52 WPM, what was the speed on the third test in WPM?

a. 38 b. 40 c. 42 d. 44 e. 46

23

AFOQT3. Word Knowledge

DIRECTIONS: This part of the test measures verbal comprehension

involving your ability to understand written language. For each question,

choose the answer that means the same as the capitalized word.

1. CRIMSON

a. crisp b. neatly dressed c. reddish d. colorful e. lively

2. CEASE

a. start b. change c. continue d. stop e. fold

3. FORTNIGHT

a. 2 days b. 1 week c. 2 weeks d. 1 month e. 2 months

4. TERSE

a. pointed b. trivial c. oral d. lengthy e. raggedy

24

AFOQT4. Math Knowledge

DIRECTIONS: This part of the test measures your ability to use learned

mathematical relationships. Each problem is followed by 5 possible

answers. Decide which one of them is most nearly correct.

1. The first digit of the square root of 59463 is:

a. 1 b. 2 c. 3 d. 4 e. 5

2. The distance in miles around a circular course with a radius of 35

miles is (use = 22/ 7):

a. 110 b. 156 c. 220 d. 440 e. 880

3. Solve for x: 2x/ 7 = 2x2

a. 1/ 7 b. 2/ 7 c. 2 d. 7 e. 14

25

AFOQT 5. Instrument Comprehension

DIRECTIONS: This test measures your ability to determine the position of

an airplane in flight from reading instruments showing its compass

heading, amount of climb or dive, and degree of bank to right or left.

26

AFOQT6. Block Counting

DIRECTIONS: This is a test of your ability to “see into” a 3-dimensional

pile of blocks and determine how many pieces are touched by certain

numbered blocks. All of the blocks in a pile are the same size and shape.

27

AFOQT7. Table Reading

DIRECTIONS: This test measures your ability to read tables quickly and

accurately. In the table below, the X values are indicated at the top and the

Y values at the left. In this test, you are to find the table entry that

corresponds to a given set of row (X) and column (Y) values.

1. For X = 0 and Y = -1:

a. 29 b. 32 c. 33 d. 34 e. 35

2. For X = -3 and Y = -3:

a. 22 b. 23 c. 28 d. 29 e. 31

3. For X = -1 and Y = +2:

a. 25 b. 27 c. 29 d. 30 e. 31

4. For X = +3 and Y = 0:

a. 30 b. 34 c. 35 d. 37 e. 39

28

AFOQT8. Aviation Information

DIRECTIONS: This part of the test measures your aviation knowledge.

Decide which of the choices best answers the question or completes the

statement.

1. If the elevator tabs on a plane are lowered, the plane will tend to:

a. nose up b. nose down c. pitch fore and aft d. go into a slow roll

e. wing over

2. The pilot always advances the throttle during a:

a. nose dive b. landing c. turn d. spin e. climb

3. The slipstream of an airplane will have the least effect on the plane’s direction

of flight when the plane’s:

a. stick is moved forward b. stick is moved backward c. stick is moved

sideways d. rudder bar is pressed with the left foot e. rudder bar is pressed

with the right foot

29

AFOQT9. General Science

DIRECTIONS: This part of the test measures your knowledge in the area of

science. Decide which one of the choices best answers the question or

completes the statement.

1. An eclipse of the sun throws the shadow:

a. moon on the earth b. earth on the sun c. sun on the earth

d. earth on the moon e. moon on the sun

2. Lack of iodine is often related to which of the following diseases?

a. Beriberi b. Scurvy c. Rickets d. Goiter e. Asthma

3. After adding salt to water, the freezing point of the water is:

a. variable b. inverted c. unchanged d. raised e. lowered

30

AFOQT10. Rotated Blocks

DIRECTIONS: This test measures your ability to visualize and manipulate

objects in space. In each item, you will be shown a picture of a block.

The problem is to find a second block that is identical to the first.

31

AFOQT11. Hidden Figures

DIRECTIONS: This part of the test measures your ability to locate a simple

geometric figure embedded in a complex drawing. Your task is to

determine which lettered figure (A, B, C, D, or E) is contained in each

numbered drawing.

32

AFOQT12. Self Description Inventory

DIRECTIONS: This inventory measures personal traits and attitudes. The inventory consists of a list of statements. The task is to read each statement carefully and decide how well each one describes you.

Look at the sample statement below.

S1. I enjoy reading poetry.

Decide if statement S1 is characteristic of you and indicate your agreement using the scale below.

A – Strongly Disagree

B – Moderately Disagree

C – Neither Agree nor Disagree

D – Moderately Agree

E – Strongly Agree

33

BAT1. Two-Hand Coordination

• Rotary pursuit tracking task

• Target moves in a fixed, elliptical pattern at varying speeds

• Task is to keep gunsight on target using control sticks

• Test length ~10 minutes.

34

BAT2. Complex Coordination

• Compensatory tracking task

• Dual-axis control stick is used to keep the small cross centered on the large cross

• Single-axis control stick is used to keep the rudder bar centered at the bottom of the screen

• Test length ~10 minutes.

35

BAT4. Time Sharing

• Assesses control precision and reaction time

• Combines a compensatory tracking task with a target identification (digit canceling) task

• Test length ~30 minutes

36

BAT3. Item Recognition

• Assesses short-term memory

• A string of 1 to 6 digits is presented quickly, removed, and replaced by a single digit

• Examinee must determine if single digit presented was also present in the original string

• Test length ~20 minutes

37

BAT5. Activities Interest Inventory

• Assesses attitudes toward risk-taking

• Examinee is presented with activity pair and asked to choose between the activities

• Activity pairs differ in level of risk

• Test length ~10 minutes

38

TBAS1. UAV Directional Orientation

• Assesses spatial orientation

• Examinee must determine position of target relative to the UAV

– The "tracker map” shows the examinee the location and heading of the UAV

• Example “select the North parking lot”

39

TBAS2 & 3. 3-Digit and 5-Digit Listening

• A string of letters and numbers is presented over headphones

• Task is to squeeze the trigger when any of three (3) or five (5) pre-specified target numbers are presented

• Test length = 60 seconds

40

TBAS4. Horizontal Tracking

• Compensatory tracking task

• Use rudder pedals to keep the yellow box over a moving airplane target

• Speed of the moving target increases during test

• Test length = 60 seconds

41

TBAS 5.Airplane Tracking

• Compensatory tracking task

• Dual-axis joystick is used to keep the yellow sight over a moving airplane target

• Test length = 60 seconds

42

TBAS6. Airplane & Horizontal Tracking

• Compensatory tracking task

• Combines Airplane Tracking Task and Horizontal Tracking Task

• Time length = 120 seconds

43

TBAS 7 & 8. Airplane & Horizontal Tracking &

Listening

• Compensatory tracking task and listening task

• Combines Airplane Tracking Task and Horizontal Tracking Task with either 3-Digit Listening or 5-Digit Listening tasks

• Time length = 180 seconds

44

TBAS9. Emergency Scenario

• Compensatory tracking & memorization

• Target digit listening is replaced with emergency scenarios task

– Examinee must resolve “emergency situations” by entering code memorized earlier from test instructions