the development of a method of evaluating flying skill
TRANSCRIPT
The Development of a Method of Evaluating Flying Skill'
By THOMAS GORDON, Universitp of Chicago
INTRODUCTION
THE need for accurate measurement of flying skill existed during the last war and has since become increasingly critical with the rapid increase in civilian flying, both private and commercial. The serious need for, as well as the difficulties involved in, developing criterion measures of pilot proficiency, however, repeatedly have been affirmed by many investigators in this field (23, 6, 18, 11).
Previous research aimed at the development of more objec- tive methods of evaluating flying skill has been concentrated in the military services and in private flying. The problem of getting improved pilot evaluation procedures accepted by such agencies, however, has troubled most of the previous investiga-
Thomas Gordon i s a n assistant professor of psychology and a counselor on the staff of the Counseling Center of the University of Chicago. He was at Ohio State University from 10.40 until 1 9 4 , as research assistant on a project in aviation psychology and a n assistant in the psychology department. He was a pilot in the Army Air Forces during the war, serving as instructor, research o&er and JEying safety officer. He received hi6 P h B . from the University of Chicago, Committee on Human Development, in 1949. His research on pilot evaluation procedures was carried out during 10&10.@ in his position as Director of Aviation Research in the American Institute for Research. During part of this period he also lectured in the psychology department of the University of Pitts- burgh. He i s an associate member of the American Psvchological Association.
1 This study was carried out when the writer was Director of Aviation Research of the American Institute for Research. It was made possible by a grant from the Civil Aeronautics Administration under auspices of the Committee on Aviation Psychology of the National Research Council. The writer wishes to express appreci- ation to John C. Flanagan, President of the American Institute for Research, for his invaluable guidance, assistance and encouragement during all stages of the study. The writer is grateful to his staff of able research workers and clerical assistants. The full report has been published as CAA Division of Research Report No. 85, April 1949.
71
72 THOMAS GORDON
tors in this field. Although several studies have reported the development of improved pilot proficiency measures and al- though psychologists have known for a t least ten years that traditional methods are unreliable and non-discriminative, all of the major agencies employing pilot evaluation procedures are still using the same basic method-subjective ratings. Stud- ies of the subjective type of method consistently have dernon- strated that it does not result in a satisfactory amount of agree- ment between check-pilots independently evaluating the proficiency of the same group of pilots; it does not satisfac- torily discriminate between relative proficiency in different aspects of flying; it does not give adequate ranges of the abili- ties of different pilots; and it does not adequately predict success in later stages of training (2, 4, 12, 18, 21, 9, 10, 3). Such research has led to repeated attempts to develop im- proved pilot evaluation procedures of three general types: (1) graphic and photographic methods; (2) rating methods; and (3) objective observation methods.
From the studies in which graphic or photographic methods were developed or utilized (24, 19, 1, 32, 15, 16, 17, 29, 26, 27, 28) , the following conclusions seem warranted :
1. The methods of recording performance are highly objec- tive, thus useful for research purposes.
2. The methods require special equipment which is exces- sively costly.
3. The records of the flight are not immediately available because of the time needed to print the film or analyze the records.
4. The measures are not comprehensive, i.e. do not cover all of the critical aspects of flying.
5. The records themselves do not yield a measure of pro- ficiency, hence, it is necessary to employ methods of eval- uating the records. The observer-observer reliability of these evaluation methods has not yet been established for graphic records and has been established for photographic records on an extremely small sample of raters.
EVALUATING FLYING SKILL 73
6. Test-retest reliabilities have not been adequately deter- mined for either graphic or photographic methods in a situation where ratings of the two flights are made by different raters and without knowledge of the pilots, per- formance on the first flight.
7. None of the studies has established the relevance of the methods with respect to flying proficiency in general.
The findings from the studies in which rating methods were developed or employed (12,25,13,30,5,20,22) may be sum- marized as follows:
1. No rating scale has been shown to have adequate test- retest reliability.
2. The relevance of rating-scale types of procedures has not been adequately established nor have such methods been developed from careful job analyses.
3. Only one rating-type method has been developed which seems comprehensive, but studies have not demonstrated its reliability or relevance.
The studies in which objective-observation methods have been developed or evaluated (4, 18, 19, 28) indicate that this is the most promising type of procedure. The method requires that numerous objective observations be made by check-pilots during a standard flight, that the check-pilot record his obser- vations on standard forms immediately after the observation is made, and that scores be assigned to small segments of the pilot’s performance rather than to the performance as a whole. The research on such objective methods, however, has shown that reliable test items can be developed but more research is needed to develop a single comprehensive flight-check com- bining many objective items. Furthermore, these studies indi- cate that improvements in the procedures are needed in order to make them more acceptable to those whose responsibility it is to use the procedures.
The researches sponsored by the Committee on Aviation Psychology (4, 19, 28) and those carried out by the AAF Aviation Psychology Program (18) have already made progress
74 THOMAS GORDON
in the development of such procedures, despite the difficulties inherent in the problem. These difficulties are related to the problem of satisfying the criteria of reZiubiZity, relevance and acceptability.
1 . Reliability. Procedures with adequate reliability have been difficult to devise for a number of reasons, chief of which are: difficulties of com- munication between pilot and check-pilot in modern aircraft; atmospheric variables affecting plane performance from one flight to the next; difficulties of recording performance during flight; differences in the standards and judgment of check-pilots. 2. Relevance. Getting relevant tasks into an evaluation procedure has
been difficult largely because of the following: the difficulty of simulating in the test situation all of the actual conditions encountered in flying; the prohibitive cost of long evaluation flights in high-powered aircraft; dis- agreement among “experts” as to which tasks are the most relevant.
3. Acceptability. Achieving the acceptance of objective procedures has been difficult for many reasons, such as: the fact that researchers have not always dealt with resistance to change as skillfully as they have dealt with the measurement problem itself; most of the procedures have been too difficult to administer in flight; investigators have not developed objective procedures which can be used with a number of different types of aircraft; measures of critical skills have often been left out of flight-checks because of the difficulty in measuring them “objectively”; pilots have objected to complicated scoring procedures usually incorporated into objective flight- checks; in striving for objectivity investigators have frequently constructed items requiring the check-pilot to use points of reference which are not used by him in actual practise.
The objective of the present study was to develop a single comprehensive flight-check which would be more reliable than currently used subjective procedures yet would measure the skills which are most relevant to success on the job and would be acceptable to those who eventually would use it. It was decided to develop the flight-check for the Airline Transport Rating flight examination, the one required of pilots in order to become certified as airline pilots. PRINCIPAL METHODS AND RESULTS
Determining the Critical Requirements of the Job. The problem of developing a flight-check which would measure the most
EVALUATING FLYING SKILL 75
CpIncAL INCIDENTS
Airline accidents
Pilot incidents (Near-ac- cidents)
Flight-check incidents (Near-accidents and reasom for failure)
relevant skills was approached through employing the “critical incident technique” for determining the critical requirements of the airline pilot’s job. This approach is essentially one of utilizing reports of observed behaviors which have been shown to be critical from the standpoint of successful or unsuccessful performance on the job. The method was used in the AAF Aviation Psychology Program in job analysis studies (31, 14) and has been described by Flanagan (6). This particular job analysis approach yields data in the form of observed behavior rather than lists of activities and traits based on opinions and
TABLE 1 Sources of Critical Incidents
mupa
Civil Aeronautics Board Accident Files
Interviews with Airline pilots
Interviews with Airline and CAA check-pilots
-
270
58
328
_ _ _ ~ ~ 185 121
601 395
137 137
~ _ _ _ ~ 923 653
judgments as typically obtained from other job analysis methods. In this study there were three principal sources of data about the critical requirements of the job which were used as the basis for the development of the evaluation procedure. These are summarized in Table 1.
Accident reports were copied from the actual files of the CAB. The pilot incidents were obtained through interviews with airline pilots in 18 different cities and from 27 different airline companies. Interviewers used questions devised spe- cially to elicit actual accounts of situations encountered by the pilots being interviewed-situations which had been brought on by some kind of ineffective behavior on their part or situa- tions in which they felt they behaved ineffectively. Flight-
76 THOMAS GORDON
check incidents were those collected from check-pilots in which they reported critical incidents in which pilots taking regular
TABLE 2 Critical Components of the Job of Airline Pilot as Determined from Znefecfive Acts
Eztracted from Accident Reports, Pilot Incidents and Flight-Check Incident8
CRITICAL JOB COW?ONENTS
Planning and Preparing for Flight 1. Obtaining information about conditions to be en-
countered in f igh t . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2. Checking on the condition of the airplane and its
equipment prior to fight. . . . . . . . . . . . . . . . . . . . . . . .
Controlling the Flight of the Airplane Within Prescribed Limits in the Performance of Routinely Used Maneuvers
3. Taxiing.. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 . Taking off under normal conditions.. . . . . . . . . . . . . . 5. Taking off under conditions of reduced visibility., 6. Taking off under cross-wind conditions. . . . . . . . . . . 7. Making a contact approach and landing under nor-
mal conditions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8. Making a contact approach and landing under con-
ditions of reduced visibility. . . . . . . . . . . . . . . . . . . . . . 9. Making a contact approach and landing under
cross-wind conditions. . . . . . . . . . . . . . . . . . . . . . . . . . . . 10. Making instrument approaches by means of refer-
ence to different types of radio aids. . . . . . . . . . . . . . 11. Recovering from a missed instrument approach or
missed landing. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12. Other maneuvers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Controlling the Flight of the Airplane Within Prescribed Limits Under Unusual Emergency Conditions
13. Recovering from sudden engine failure and per- forming maneuvers with an engine out . . . . . . . . . . .
14. Operating the airplane when the air is turbulent, when runways are slippery, when icing conditions are present, etc. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
15. Controlling the airplane in unusual attitudes or a t minimum airspeeds.. . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Employing Procedures to Locate or Keep Track of Posi- tion in Flight or to Fly a Prescribed Course
16. Navigating and orienting., . . . . . . . . . . . . . . . . . . . . . . . 17. Communicating with traffic control personnel.. . . .
BlEQUENCY 01 INEFm- ACTS IN:
- A& dents
0
1
3 6 0 0
16
25
14
0
1 0
3
23
0
7 0 -
__ y p t UIQ- dents
7
7
1 13 3 2
85
40
54
10
3 4
3
11
3
13 7
Flight !he+ inci- dents
3
1
2 5 3 0
33
20
5
23
3 1
23
2
20
41 2
Total
10
9
6 24
6 2
134
85
73
33
7 5
29
36
23
61 9 -
EVALUATING FLYING SKILL 77
TABLE 2-Colotinucd
CRIRCAL JOB COMPONENTS
Operating Equipment of Plane and Carrying out Cockpit Procedures
18. Remembering to carry out certain prescribed or appropriate tasks in connection with the operation of the equipment of the airplane. . . . . . . . . . . . . . . . .
19. Operating the controls, dials and switches of the plane's equipment in a correct manner. . . . . . . . . . .
Adhering to Prescribed Policies or Regulations and Tak- ing Precautions Consistent with Safety
20. Conforming to regulations and policies.. . . . . . . . . . 21. Keeping a constant lookout for possible collision
objects and remaining attentive and alert. . . . . . . . 22. Taking special precautions or remaining on safe
side. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Remaining Emotionally Organieed and Working Effi- ciently with Others
23. Remaining emotionally organiaed in emergency situations. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
24. Working efficiently with other crew members. . . . . . .
Total. . . . . . .
mQWNCr OF INXFFECZIVX A M S IN:
Acci- dents
7
8
5
13
2
0 0
134 -
Pilot ItlCl-
dents
19
17
21
8
11
8 4
351
Flight, % i dents -
11
28
4
3
4
7 1
245 ~
- Total
__
37
53
30
24
17
15 5
733 -
-
flight examinations behaved in such a way as to warrant a failing grade or to require the check-pilot to take over the controls because of a critical situation caused by the examinee.2
The 653 critical incidents were subjected to a content an- alysis which involved first extracting from each incident the specific ineffective pilot behaviors, then sorting these into sep- arate job components, such as landings, takeoffs and taxiing, depending on where the incident occurred. Finally, the inef- fective pilot acts in each job component were sorted into cate- gories of similar acts. Table '2 presents the '24 different job components into which the ineffective acts were classified and the frequency of these acts. As an illustration of the kinds of
This phase of the study is described in more detail in: Gordon, T. The airline pilot's job. Journal of Applied Psychology, 1949, 33, 2, 122-131.
78 THOMAS GORDON
ineffective acts classified under the job components, the cate- gories of acts under “Making a Contact Approach and Landing under Low Visibility Conditions” are presented in Table 3.
The Development of the Evaluation Procedure. The job of constructing the evaluation procedure involved the following steps: (1) selecting job-related tasks identical or similar to the
TABLE 3 Frequency of Inefectivc Behavior Occurring in Critical Incidents Involving Approaches
and Landings Under Low Visibility Conditions
INEFPECTIYE BEHAVIOR
I PgEQWEEICY OF INEPRECIIVE ACTS OBTAINED FPOM:
Failing to align with runway or flying in. correct heading from station to field. . . .
Failing to locate field after becoming con.
Failing to hold constant altitude when cir. cling field ..............................
Failing to hold proper glide angle in descent Failing to hold proper airspeed in descent. Failing to go-around after overshooting. .. Turning too steeply when close to ground. Leveling of f too high or too low.. . . . . . . . . . Flying partially instruments and partially
contact instead of one or the other.. . . . . Failing to stay aligned with runway on roll. Landing in field adjacent to airport.. . . . . . Landing downwind. ....................... Failing to plan approach. . . . . . . . . . . . . . . . . . .
Failing to keep within sight of field.. . . . . .
tact; mistaking landmark for field.. . . . . .
0 0
0
2 17 0 1 0 1
11 2
3
2 10 4 3 1 2
9 0
0
~ 20 ‘ 2
6 33 6 4 2 3
Totals.. .......................... . . . . . . . I 25 1 40 1 20 I 85
24 job components found most critical in the job analysis; (2) arranging these efficiently into a standard flight in such a way that the flight-check could be administered in the shortest possible time; (3) breaking down each job-related task into the critical observable behavioral units derived from the job analysis and devising items for each behavioral unit; (4) de- vising the flight-check form on which the check-pilot records his observations.
EVALUATING FLYING SKILL 79
In order to make the flight-check form usable with dif- ferent types of airplanes, a special kind of item was developed
I N S T R U E N T T A R E 0 F F
TELL ExAyINEE: you want it, hold it with brake untll I g i ve the signal for takeoff."
"Line the plane up with the m a y yourself. ;?hen you have it t h w a ~
Smooth and positive Jerky or hesitant Excessivoly rapid PLYJER .
APPLICATION . - /I - r 7 U
: s 4 __-___--- -----------
A T R 4 D O F . ROLL . (b) . mg$pu? 0 Nornal 0 Held d m Droppd back
FLIGHT PATH .
L1/ Assistance necessary /7 Assistsrice not - necessary CHEM-PIrn . BSsISTAN& .
which allows the check-pilot to write in different limits of air- speed, altitude and heading which are appropriate to that particular type of airplane being used for the examination (see Item 5, Figure 1). Other features were incorporated into
80 THOMAS GORDON
the flight-check form in order to solve the problem of having an evaluation procedure which is comprehensive yet can be used in the air without diverting too much of the check-pilot’s attention from keeping a close watch for other traffic or taking too much of his time to record observations. A special format was used for the flight-check form which separates different items and clearly labels each. The most distinctive feature, however, was the frequent use of graphic and pictorial items (see Items 2, 3, 4 and 6, Figure 1). Not only did these items decrease the amount of recording time but it is felt that they contributed greatly toward making the flight-check more reli- able largely because they provided a much more objective definition of the limits of performance on an item than is pro- vided by words alone.
The final form of the flight-check consisted of 18 different tasks (or maneuvers), each of which contained from 2 to 9 items. A sample maneuver is reproduced in Figure 1.
The Experimental Try-out of the Evaluation Procedure. The first try-out of the flight-check involved administration to 27 Air Force pilots on two successive flights on different days. During each of the two flights two check-pilots made inde- pendent observations, the two check-pilots on the second flight being different from those on the first. Thirty different check- pilots were involved, all of whom were experienced instrument instructors. Approximately 15 different airplanes, all TB-25’s7 were employed for the try-out. Each of the 27 pilots was tested by four different check-pilots, referred to as observers A, B, C and D. Observers A and C were the right-seat observers on the first and second day respectively and observers B and D were the jump-seat (the seat behind theright seat) observers on the first and second day respectively. The obtained ob- server-observer reliabilities (AB and CD) and the test-retest (ride-ride) reliabilities (AC, AD, BC, BD) are presented in Table 4, top half.
The second try-out involved testing 26 CAA pilots using the same procedure as in the first try-out. A revised flight-
EVALUATING FLYING SKILL 81
NuMBEP 0s
'ILoTs ~
27
26
check form was used for this try-out. Revisions were made on the basis of (1) an item analysis of the first form, (2) maneuver reliabilities determined from first try-out, and (3) suggestions of the check-pilots in the first try-out. Revisions consisted of adding more pictorial items, changing the sequence of several maneuvers, improving the directions for check pilots, eliminat- ing several items objectionable to the check-pilots. In this try-out CAA examiners responsible for administering the Air- line Transport flight exam acted as both check-pilots and ex- aminees. Tests were conducted in the DC-3 and DC4 air-
IELUBUIITP COEBnUENTS
AB AC AD BC BD CD OI&*&&~ - - - - - - - - .87 .49 .49 .59 .50 .76 .82 .52
.84 .81 .75 .74 .74 .87 .86 .76
TABLE 4 Observer-Observer and Test-Retest Reliabilities of the Flight-Check on First Try-out
First try-out: Total Flight-check score.. . . . .
Second try-out : Total Flight-check score . . . . . . . . .
* OIOz represents the combined oberver-observer coefficient (AB and CD) and R I R , represents the combined test-retest coefficient (AC, AD, BC and BD). These combined coefficients were calculated by means of the z-transformation technique.
planes. Obtained reliabilities are shown in the bottom half of Table 4. It can be seen that the high observer-observer reli- ability obtained in the first tryout held up on the second and that the test-retest reliability improved, probably as a result of the revisions of the form.
Empirical proof of the extent to which the critical incident approach produced a relevant flight-check will be obtained in a third try-out planned with already qualified airline pilots, on whom there are more adequate criterion data against which to validate the flight-check. In this study there were no ade- quate measures of proficiency available on either the AAF or the CAA pilots. An analysis was made of the CAA check-pilots' written
82 THOMAS GORDON
responses to a questionnaire in order to obtain an indication of the degree to which the flight-check satisfied the criterion of acceptability. In general, although they saw both advantages and disadvantages, the majority were either in favor of adopt- ing it as the regular authorized flight exam or in favor of con- tinuing work on it to make it ready for adoption.
CONCLUSIONS 1. The flight-check developed in this study can be con-
sidered a reliable procedure for arriving at an overall evaluation of pilots with experience similar to those tested in this study. Both observer-observer and test-retest reliabilities were con- siderably higher than those reported for other methods of eval- uating pilots.
2. The flight-check can be considered relevant to success or failure on the job to the extent that it is made up of tasks which are identical or similar to the requirements of the job found to be critical by job analysis. More acceptable evidence of its relevance must await further try-out.
3. The majority of check-pilots of the kind who will use the procedure consider it an acceptable method, although further revisions were recommended.
4. Several distinctive features of this flight-check differen- tiate it from previous objective flight-checks and probably contributed greatly to its high reliability and/or its accept- ability :
a. A new type of graphic or pictorial item was used fre- quently.
b. A new type of item was devised which makes it pos- sible to use the same flight-check form for different types of planes, whereas previous objective flight- checks were limited to use with only a single type of plane.
c. The items adequately cover the most critical aspects of piloting, because the flight-check was built upon the findings of the critical requirement study.
EVALUATING FLYING SKILL 83
REFERENCE s 1. BACESTROM, O., JR., AND VITELES, M. S. An Analysis of Graphic Records of Pilot
Performance Obtained by means of the R-S Ride Recorder. Washington: CAA Division of Research, Report No. 55, 1946.
2. COWLES, J. T., AND DAILEY, J. T. The measurement and prediction of civilian flying instructor proficiency. American Psychologist, 1946, 1. 292.
3. CRAWFORD, M. P., AND DAILEY, J. T. An analysis of elementary pilot performance from instructors’ comments. American Psychologist, 1946,1, 292.
4. EDQERTON, H. A., AND WALKER, R. Y. History and Development of the Ohio State Flight Inventory. Part I: Early Versions and Basic Research. Washington: CAA Division of Research, Report No. 47, 1945.
5. FESTINGER, L., KOQAN, L. S., ODBERT, H. S., AND WAPNER, S. An Analysis of Inspectors’ Ratings of Check Flights as Recorded on F m A C A SMZ. Washington: CAA Division of Research, Report No. 58, 1946.
6. FLANAGAN, J. C. (ED.). The Aviation Psychology Program in the Army A i r Forces. Army Air Forces Aviation Psychology Program Research Report No. 1, Wash- ington: U. S. Government Printing Office, 1948.
7. GORDON, T. The Airline Pilot: A Survey of the Critical Requirements of H i s Job and of Pilot Evaluation and Selection Procedures. Washington: CAA Division of Research, Report No. 73, 1947.
8. GUILFORD, J. P. Fundamental Statistics in Psychology and Education. New York: McGraw-Hill, 1942.
9. HENNEMAN, R. H. Proficiency measures for fighter pilots a t the operational level of training in the Army Air Forces. American Psychologist, 1946, 1, 293.
10. HENNEMAN, R. H., HAUSMAN, H. J., MITCHELL, P. H. The Measurement of Instru- ment Flying Proficiency of A i r Force Pilots. Washington: U. S. Government Printing Office, 1947.
11. JENKINS, J. G. Validity for what? Journal of Consulting Psychology, 1946, 10,
12. JOHNSON, H. M., AND BOOTS, bl. L. Analysis of Ratings in the Preliminary Phase of the CA4 A Training Program. Washington : CAA Division of Research, Report No. 21, 1943.
13. KELLY, E. L. The Development of “A Scale for Rating Pilot Competency.” Wash- ington: CAA Division of Research, Report No. 18, 1943.
14. LEPLEY, W. M. Psychological Research in the Theatres of War. Army Air Forces Aviation Psychology Program Research Report No. 17. Washington: U. S. Government Printing Office, 1947.
15. MCFARLAND, R. A., AND HOLWAY, A. H. The Measurement of Flight Performance in Relation to Piloting. Progress Report, National Research Council Com- mittee on Aviation Psychology, 1942.
16. MCFARLAND, R. A,, AND HOLWAY, A. H. The Theory and Measurement of Flight Performance. Progress Report, National Research Council Committee on Avi- ation Psychology, 1941.
17. MCKAY, W. The Development of the C.A.A.-N.R.C. Flight Recorder. Washington: CAA Airman Development Division, Report NO. 35, 1944.
18. MILLER, N. E. (ED.) Psychological Research on Pilot Training. Army Air Forces Aviation Psychology Research Report No. 8. Washington: U. S. Government Printing Office, 1947.
19. National Research Council Committee on Selection and Training of Aircraft Pilots. History and Development of the Ohio State Flight Inventory. Part 11:
93-98.
84 THOMAS GORDON
Recent Versions and Current Applications. Washington : CAA Division of Re- search, Report No. 51, 1945.
20. Pilot Assessment During Elementary Flying Training. Empire Central Flying School, Royal Air Force, Report No. 18, 1946.
21. PREGTON, H. 0. Analysis of C.A.A. Records on Airline Transport Pilots. Wash- ington: CAA Division of Research, Report No. 72, 1947.
22. Transport Command Categorization of Flying Personnel. Royal Air Force, 1948. 23. VITELES, M. S. The Aircraft Pilot: 6 Years of Research. A Summary of Outcomes.
WashiDgton: CAA Division of Research, Report No. 46, 1945. 24. VXTELES, M. S., AND BACKSTROY, O., JR. An Analysis of Graphic Records of Pilot
Performance Obtained by Means of the R-S Ride Recorder. Part I . Washington: CAA Division of Research, Report No. 23, 1943.
25. VITELES, M. S., FRANIEN, R., AND ROGEES, R. C. The Association between Ratings 012 SpeciJic Maneuvers and Success or Failure in Flight Training of R A P Cadets. Washington: CAA Airmen Development Division, Report Na. 37, 1944.
26. VITELES, M. S., AND THOMPSON, A. S. An Analysis of Photographic Records of Aircraft Pilot Performance. Washington : CAA Division of Research, Report No. 31, 1944.
27. VITELES, M. S., AND THOMPSON, A. S. The Use of Standard Flights and Motion Photography in the Analysis of Aircraft Pilot Performance. Washington: CAA Division of Research, Report No. 15, 1943.
28. WALKER, R. Y., WAPNER, S., BAKAN, D., AND EWART, E. S. The Agreement Be- tween Inspectors’ Observations as Recorded on the Ohio State Flight Inventory and Instrument Readings Obtained f rom Photographic Records. Washington: CAA Division of Research, Report No. 67, 1946.
29. WAPNER, S., FESTINGEE, L., AND ODBEET, H. S. Comparison of Student Pilot Performance on Successive Check Flights as Measured by Photographic Records. Washington: CAA Division of Research, Report No. 59, 1946.
30. WHERRY, R. J., AND ROGERS, R. C. A Factor Analysis of the Purdue “Scale for Rating Pilot Competency.” Washington: CAA Division of Research, Report No. 18, 1943.
31. WICKERT, F. (ED.) Psychological Research on Problems of Redistribution. Army Air Forces Aviation Psychology Research Report No. 14. Washington: U. S. Government Printing Office, 1947.
32. WILLIAMS, A. C., JR., MACMILLAN, J. W., AND JENKINS, J. G. Preliminary Ex- perimental Investigations of L‘Tension’’ as a Determinant of Performance in Flight Training. Washington: CAA Division of Research, Report No. 54, 1946.