the development of thebrief test of literacy
TRANSCRIPT
DATA EVALUATION AND METHODS RESEARCH
the development of
theBrief Testof literacy
A description of the development of a new test of literacycapable of providing information concerning the prevalence ofilliteracy in large sample populations.
DHEW Publication No. (HRA) 75-1291
U.S, DEPARTMENT OF HEALTH, EDUCATION, AND WELFARE
Public Health Service
Series 2Number 27
Health Resources AdministrationNational Center for Health Statistics
Rockville, Md. September 1974
Vital and Health Statistics-Series 2-No. 27First issued in the public Health Service publication series No. 1000 March 196[3
NAT ONAL CENTER FOR HEALTH STATIST Cs
THEODORE D. WOOLSEY, DirectoT
PHILIP S. LAWR ENCE, SC.D., Associate Director
OSWALD K. SAGEN, PH. D.,, Assistant Director for Health Statistics Development
WALT R. SIMMONS, M.A., Assistant Director for Research and Scientific Development
ALICE M. WATERHOUSE, M. D., Medical Consultant
JAMES E. KELLY, D. D. S., Dental Advisor
LOUIS R. STOLCIS, M. A., Executive Officer
DONALD GREEN, Information O//icer
DIVISION OF HEALTH EXAM1NATION STATISTICS
.+I+’I’ll UR J. McDu,d I{LL, Director
JAMES T. BAIRD,JR.,~hic{,Analysis and Reports ?rwnch
IIENRY W.MILLER, Cbie/, Operations and Quality Control Pr,’ncb
PETER v. HAMILL, M.D.,Medical Advisor
LAWRENCE E.VAN KIRK,D.D.s.,Dental Advisor
LOIS R. C21ATHAM,PH.D.,psycbo~ogica] Advisor
Public Health Service Publication No. 1000-Series 2-?40. 27
Library of Congress Catalog Card Number 67-62374
FOREWORDThe Health Examination Survey, one of the major programs of the
National Center for Health Statistics, collects, analyzes, and publishesthe kinds of health-related data which can be obtained only throughdirect examinations, laboratory tests, and measurements. Much of thedata collected pertains to prevalence levels of specific, medicallydefined diseases. Other data provide, for the population studied,distributions of a variety of physical, physiological, and psychologicalmeasurements. Reports in Series 1 and Series 11, described in theoutline at the back of this publication, present the descriptions andsome of the findings of the various programs already carried out.
In planning the third program of the series of Health ExaminationSurveys, consideration was given to including some measure of the ex-tent of illiteracy in the population. That there is some relationshipbetween various states of ill health and illiteracy has been recognized.It seemed desirable, therefore, to be able to investigate the relation-ships between some of the health findings and this measure. In ad-dition, officials in other parts of the Department of Health, Education,and Welfare expressed interest in obtaining such data.
The usual procedure followed in plaming programs of the HealthExamination Survey is to utilize tests, procedures, and instrumentsalready well established and generally accepted. In some instances, how-e~,er, the special requirements of the survey along with the “state of
the art” of measurement of the particular variable make this im-possible. This is discussed in the present publication. In this instance,presented with such a problem, it was decided to enter into a contract\\,ith the Educational Testing Service to del~elop the required instru-
ment. The results are presented in this report.It is not surprising that the National Center for Health Statistics
should sponsor such research. The Public Health Service is authorizedunder the National Health Survey Act (P L 652: Wth Congress) “to pro-vide (1) for a continuingsurvey and specialstudies to secure . . . sta-
tistical information on the amount, distribution, and effects of illnessand disability in the United States . . . and (2) for studying methodsand survey techniques for securing such statistical information, witha view toward their continuing improvement. ”
The results of this study are being made available, not only toprovide necessary information for evaluating later reports of findingsin the Health Examination Survey programs, but also because of theirmore general interest. The report will call attention to the need fortechnically superior, yet brief, psychometric instruments, and it willinform interested persons and groups as to what has been done, in oneinstance, to meet this problem.
Arthur McDowell, DirectorDivision of Health Examination Statistics
...111
SYMBOLS
Data not available ------------------------
Category not applicable -------------------
Quantity zero ----------------------------
Quantity more than O but less than0.05----
Figure doesnot meet standards ofreliabilityor precision ------------------
---
. . .
0.0
*
CONTENTS
Foreword ------------------------------------------------------------
Introduction ----------------------------------------------------------
General Back~ound --------------------------------------------------
Establishing Test Specifications for Reading ------------------------------
Establishing Test Specifications for Writing ----------------- -------------
Pretests and Their ,Results ---------------------------------------------
Construction of the Final Form -----------------------------------------
Screening Tryouts -----------------------------------------------------
Summary ------------------------------------------------------------
References -----------------------------------------------------------
Acknowledgments -----------------------------------------------------
Appendix I.
Appendix H.
Appendix III.
Appendix IV.
Appendix V.
Appendix VI.
Discussion oftheUseof Phi Coefficients --------------------
Description of the Coefficient of Sentence Consistency -------
Answer Sheets for Reading and Writing Tests ---------------
Instructions for Reading ----------------------------------
Five Items Used in Writing Test ---------------------------
BasicSkillsSurvey, Reading and Writing Manual forExaminers ----------------------------------------------------------
Introduction --------------------------------------------------------Administering the Reading Test --------------------------------------Scoring Information -------------------------------------------------Administering the Writing Test --------------------------------------Scoring Information ------------------------------------------------
Rige
...111
1
1
2
5
6
10
12
12
12
13
14
15
16
18
22
232424262727
v
THIS REPORT outlines the procedures involved in the development ofa test of litevacy suitable fov use in scveening lavgw numbers of pevsons.
In it the authovs discuss the problems which weve facedfvom the initia-tion of the pvoject through the find assembly of the test materials, de-scribing the diffic-ulty of definition, the pvactical constraints on the ad-ministration, and the limitations of test design.
On the basis of its use thus far, the vesulting instrument, which will bevefewved to as the Bn”ef Test of Literacy, would appear to discriminatequickly and accurately between litevate and illiterate pevsons. This ve -povt should pvovide valuable information to any prospective usev of thetest or to those who seek to develop theiv own instruments in this field.
vi
DEVELOPMENT OF
THE BRIEF TEST OF LITERACY
Thomas F. Donlon and W. Miles McPeek, Educational Testing ServiceLois R. Chatham, Division of Health Rzamination Statistics
INTRODUCTION
The Brief Test of Literacy was developed toassess literacy in reading and in writing withinthe framework of a national health survey. As suchit provides an instrument of marked utility, forno prior test intended for the direct assessmentof literacy has been developed.
There are several reasons for the lack of anyearlier development of a comparable instrument.In general, psychological testing has concentratedon the development of instruments which areappropriate for the measurement of individualdifferences, with a concomitant interest in thelonger tests that are necessary to achieve highreliability. Only recently has there been any stronginterest in instruments that are specifically in-tended to provide information concerning the edu-cational attainment of groups. While instrumentscapable of such description will be developed withincreasing frequency in the near future, the BriefTest of Literacy is one of the first of its type.
A second reason for the absence of a test ofthis kind is the concept of literacy. It is virtuallyimpossible to achieve a satisfactory definition ofliteracy. It is even more difficult to attain anoperational definition, and yet an operational defi-nition is a virtual sine qua non for the develop-ment of a psychological test. The problem of defi-nition is confounded by the varying demands ofdifferent cultures and subcultures and by culturalchange through time. As a result, a person who is
virtually illiterate by the standards of an advancedculture may well be able to meet the demands ofhis own less-developed civilization.
A third reason for the absence of an earliertest of this nature is that a large number of read-ing tests already exist. Many of these tests areintended to measure reading skill at approximatelythe level required. However, such existing testscan make a limited contribution to a survey ofliteracy because they are primarily designedeither to evaluate children who are in the firstyears of school or to provide diagnostic informa-tion concerning the nature of reading problems,rather than to provide categorical assessment ofliteracy versus illiteracy.
For these reasons, the Brief Test of Literacyrepresents an initial development both in the gen-eral field of survey instruments and in the assess-ment of literacy.
GENERAL BACKGROUND
The Brief Test of Literacy was developed forthe purpose of assessing literacy in reading andin writing within the framework of the NationalHealth Survey whose mission is to study the inci-dence and prevalence of various health and health-related probIems. Because of the nature of thesurvey, many different measures are obtainedfor each sample person; therefore, the amount oftime allotted for the assessment of anyone aspectof health is extremely limited.
1
As a result one of the primary constraintsplaced on the test was that it be so designed thatliteracy could be determined in a brief period oftime. Toward this end a target time of 5 to 8 min-utes was established.
In addition to the time restraint, the test hadto be suitable for use with the general populationof adolescents throughout the continental UnitedStates and, hopefully, with adults as well. Sincethe survey population excluded institutionalizedpersons, the test did not need to be designed to
“permit the rapid assessment of literacy in caseswhere the individual could not function in normalsociety because of extreme emotional disturbanceor severe mental retardation.
A third constraint on the test was that it hadto be so designed that the results could be inter-preted in terms of the prevalence of literacy andof illiteracy. Accordingly, the fundamental meas-urement concept was that of a cutting score. Any-one above a designated score would be consideredliterate; those below it would be considered illit-erate. Degrees of literacy would not be assessed.
ESTABLISHING TEST
SPECIFICATIONS FOR READING
The initial step in the development of specifi-cations consisted of a survey of the literature.This survey was disappointing. In spite of exten-sive work on the importance of literacy and onprojects for its improvement in various nations,there were no reports on techniques for its directassessment. In fact, as stated in the introduction,there is a general vagueness as to what consti-tutes literacy, with sundry definitions put forthby various writers. The most surprising findingwas the absence of any general description of theassessment of literacy during World War II. Thereundoubtedly was extensive work in the area at thattime: the military differentiated among low-levelinductees, determining who should be given a basiceducation course, but there was nowhere a sum-ma~ y of the devices used. I?rum a private commu-nication with a government psychologist, it waslearned that at present the Armed Forces use ageneral aptitude test to make these distinctions.This practice could not be followed by the survey,however, because of the obvious confounding oflow mentality and of illiteracy.
While no specific techniques were uncoveredin the literature search, a variety of definitionswas found. In general, these fell into two classes,the functional and the normative. Functional defi-nitions stressed an individual’s adjustment to hisculture. One was literate if he possessed a levelof ability sufficient to permit him to function wellin his society. Normative definitions stressedsome typical educational attainment. Thus, onewas literate if one read as well as the averagechild at the middle of the fourth grade in the UnitedStates, or at the end of the fifih year in Pakistar,et cetera.
The functional definition is inherently attrac-tive, for illiteracy is a functional deficit. At thepresent time, however, there simply is no realis-tic basis on which to determine a functional levelfor a. society as diverse as that of the UnitedStates; to attempt to describe the criteria forusing such a definition would be a truly formida-ble task. The following quotation of a IJNESCOdefinition 1 is an example of the difficulty.
A pevson is literate when he has acquired theessential knowledge and skills which cznablehim to engage in all those activities in whichlitevacy is ~equived for effective functioningin his group and community, and whose at-tainments in veading, wwiting, and arithmeticmake it possible fov him to continue to usethese skills towayds his own and the commu-nityrs development and foy active participa-tion in the life of his cowntry.
One would be hard-pressed to translate these gen-eralities into measurement specifics.
Therefore, in conjunction with the adminis-tration of the survey for which the test was to bedeveloped, it was decided to estimate the incidenceof illiteracy using a definition which is commonlyheld in the fields of education and health in thiscountry, namely, “literacy is that level of achieve-ment which is attained by the average child in tt eUnjted States at the beginning of the fourth grade,” 2
With the establishment of a working defini-tion, the development of the statistical specifica-tions was begun. As stated earlier, the test wasto be designed so that test scores could be assignedto one of two categories. The requirement built
2
in another specification—the use of a cutting-scoretechnique. Given the working definition, the cuttingscore would ideally be such as to minimize theerror in differentiating the top 50 percent of thenational population of children entering fourthgrade from the kottom 50 percent. The item sta-tistics should be specified so as to achieve, then,this optimal cutting score.
The theory of the cutting score is quite com-plex. Major theoretical work in the area has beenundertaken by Lord,3 and there are fairly sophis-ticated techniques for developing such tests andlocating the “cut.” For various practical reasons,however, a more pragmatic approach was usedin developing the Brief Test of Literacy. T’hispragmatic approach did retain one obvious featureof virtually all cutting-score work: the difficultyof the items was centered on a narrow band, ratherthan allowed to vary widely. This is in contrastto tests designed for differentiating among sev-eral levels of ability.
A practical limitation also arose in comectionwith the timing of the developmental work relativeto the school year. The working definition of liter-acy was defined as achievement at the beginningof the fourth grade, but the developmental workhad to be performed during the late winter months.If the scores made during winter months were toserve as estimates of the comparable difficultieswhich would be obtained using an entering fourthgrade population, some adjustment in the observeditem difficulties was needed. There was, however,no adequate empirical basis for determining thisadjustment. After a review of available data on thegrowth of reading ability, it was decided that anaverage item difficulty level of 80-percent-passat the time of pretesting would be a useful esti-mate of a difficulty of 50 to 60 percent for enter-ing fourth graders. Accordingly, the specificationsfor item difficulty were set as follows: the itemswould show an average difficulty of 80-percent-pass and a range of difficulty from 65-percent-pass to 95-percent-pass.
The difficulty of the reading materials wasspecified to be approximately fourth-grade level.Deviations were permitted only in the direction ofgreater difficulty, because of the intended use ofthe materials with an older population and be-cause the normal conception of reading difficulty
is based partly on dimensions of reading beyondthe kind of literal comprehension which was en-visioned for this test. This limitation to literalcomprehension is discussed later in the descrip-tion of the type of questions asked. The conclu-sion was, however, that normal estimates of pas-sage difficulty were likely to be overestimates,given the simplicity of the questions.
In the absence of any external criterion, itemvalidity was limited to an index of internal con-sistency; phi coefficients a were specified as theindexes of item-test correlation. No specific meanvalue of these was established. Instead it wasspecified that the mean of the phi coefficients bemaximized and that all items should show a phicoefficient significantly greater than chance.
The number of items in the test was also leftunspecified. In a sense, there were incompatiblegoals for the proposed test in that test reliabilityhad to achieve an acceptable level, while the timerequired for administration had to be minimized.A reliability between .70 and .80 was considereddesirable for this survey work, and the ideal test-ing time was 5 minutes per person. At the begin-ning, the format of the test was uncertain. Clearly,there would be a presentation of material to beread, and there would be questions to determinecomprehension, but the severe time constraintsposed some difficulty in the development of testformat. In any reading test there is usually anaverage ratio of the number of words which mustbe read for each question. This ratio must belarge enough that a reasonable test of reading canbe attained, and it must be small enough that test-ing time can be efficiently used. The problem posedin the test development work was the estimationof a workable value for this ratio.
Careful study led to the conclusion that theoptimum format would consist of a brief passageof 40 to 50 words followed by two or three ques-tions. Thus, another specification was established:
aThe ~hi coefficient is a measure of the correlation he-
tncen tno variables when the variables arc di~ ided into qutn.titntively discrete groups and thus can he represented in a
four-fold table. It ii identical to the product moment correla-
tion between tffo binomial variates. The phi coefficient is
discussed in a number of statistical tcxrhiis: for example,see \\al~er and Lot Statistical lrifewnce. Xe\v York, Hen~
Holt wrd Co., Inc., 1953.
3
the length of the passage to be read. The decisionwas also made to use three questions with eachreading passage on the pretest. Ultimately, a de-cision would need to be made as to the use of twoor three questions in the final form. This deci-sion could be based both on the speed factor andon the patterns of losses of items due to defectsuncovered in the pretesting.
Timing was a central concern. Reading pro-ficiency has always consisted of a combination oftwo abilities: the ability to read rapidly and theability to read accurately. Some reading testsattempt to provide diagnostic information as tothe relative proficiency y along these two dimen-sions. Generally the close correlation between thetwo measures, speed and accuracy (or compre-hension), poses no real difficulty. However, at thelevel of skill required to make a judgment of liter-acy, less emphasis should be placed on speed asthe source of variation among scores. Certainly,in a functional sense, speed of reading is impor-tant in achieving literacy; nevertheless, manypoor readers must have time to allow the wordsto come into focus before they can establishmeaning. It was obvious that, given the need fora 5-minute test, no power measure could be pro-vided. Every effort was made, however, to reducespeed variance to a minimum.
One underlying consideration in establishingtime specifications was not essentially psycho-metric, but it was such a powerful considerationwith those working on the test that it deservesmention. “Illiteracy” is not a complimentary attri-bute, and although it is capable of specific redefi-nition in an operational sense— “an ‘illiterate’ isone who does poorly on our test’’—the popularconception of illiteracy camot be ignored. Thispopular conception undoubtedly stresses compre-hension in reading far more than speed. In otherwords, to the extent to which it was possible, thetest construction process limited speed varianceto a level which seemed reasonable. The readingrates demanded by the test are not stringent incomparison with the everyday demands of oursociety.
Since a random sample of noninstitutionalizedpersons aged 12 through 17 living in the continen-tal United States would be drawn in the survey, thetypical sample subject should encounter no diffi-culty with the test. The poorer leaders however,
for whom there would exist a question of literacy,might have problems simply because of unfamili-arity with any testing situation. The muMple-choice format was specified for the reading testbecause of the efficiency it offered in responsetime and in scoring time. The use of a separateanswer sheet, as opposed to a test booklet in whichanswers are recorded directly, posed certainproblems. For exarflple, a subject might fail tocorrectly align his answer sheet and test booklet,leading to invalid test scores. However, since theuse of an answer sheet made it easier for the ex-aminer to keep track of the subject’s progress andto stop the examination when the cQt- off score wasachieved, the answer sheet method was adopted.
One concern remained. In a test of 5 minutes’duration, a subject of lmrderline intellectual abil-it y who is not used to taking tests might, if left tohimself, fail to divide his time properly. Thus hemight spend too much time on one particularlydifficult question and thereby score poorly on thewhole test. Such personal characteristics are acause of concern even in much longer tests. Be-cause it was decided to avoid ‘‘speededness” inall of its forms, personal characteristics seemedeven more likely to cause difficulty. To controlfor such variables, the test was made to consistof a number of separately timed units, monitoredby the examiner to insure that the appropriatepace was maintained.
There were other reasons for developing atest of several parts. Foremost among these wasthe opportunity it would provide for shortening thetotal testing time for any subject who succeededin passing the cutting score. Such a subject couldcomplete the part on which he was working butwould not need to attempt later parts. Anotheradvantage would be derived in that an error intest administration during one of the parts neednot require a complete retesting; rather, one addi-tional section could be added to replace the defec-tive one. The parts were specified to be separatelytimed units, consisting of a passage and two orthree questions. At this point no decision was madeconcerning the amount of time which would be de-voted to each passage, however, this was antici-pated to be 60 or 90 seconds, depending on the out-come of the pretesting.
The scoring formula was specified as the totalnumber of right answers minus one-fourth of the
4
number of wrong answers. While this is standardpractice in multiple-choice testing, it was partic-ularly indicated in this test, where the relativelyfew questions asked would make it possible tosecure a substantial change in rank positionmerely by chance, if only the number of correctanswers were used in the scoring.
Specifications regarding the content of the”test were difficult to define: Perhaps the clearestspecification was that the content had to be accept-able to adults and to adolescents, yet had to lenditself to pretesting on fourth graders. That is,materials from a storybook written for 10-year-olds would be inappropriate for adults. On theother hand, materials which would be pretestedon a group of fourth graders could not containlanguage or topics inappropriate for children. Inaddition, materials had to be suitable for usewith highly diversified populations. For example,the test had to be equally acceptable to boys andto girls, to persons with a science interest andto those with an art interest, to those who livedin the country and to those who lived in the city,to Negro and to white, and to rich and poor alike.
The anticipated use of the test on older popu-lations led to the “pretesting of a number of pas-sages aimed at simulating the functional readingdemands of adult life. These were in the form ofwant ads and brief instructions for operatingequipment.
One important specification concerned thetype of question which could be asked. In a readingtest there is typicaIly a variety of questions dif-ferentiated by the degrees of inference and judg-ment required to answer them correctly. Bothinference and judgment play a role in readingability, and each may be argued to be essentialto literacy, in one of its meanings. These morecomplex aspects of reading would be excludedfrom the definition of literacy used in developingthis test. Instead questions would be limited tostraightforward comprehension. As a result allanswers would be essentially restatements ofinformation presented in the reading passage.While no defense of this decision may be neces-sary, it may be restated that any definition ofliteracy is an arbitrary dichotomization of whatis fundamentally a continuum of varying readingability from little or none to highly developed.Reading has dimensions, and it is possible to be
more literate in one of these dimensions than inanother. The most basic dimension in reading isstraightforward comprehension, and the BriefTest of Literacy focused on this.
When the foregoing work had been completed,the test specifications for the reading test werevirtuaHy complete and the development of pretestmaterials was begun.
ESTABLISHING TEST
SPECIFICATIONS FOR WRITING
Very early in the development of the writingtest the decision was made to use the techniqueof having the subject ‘write a few brief, simplesentences in response to dictation by an examiner.The writing test, because it called for a con-structed response, required the development ofa scoring technique which would be efficient forthe examiner, consistent when used by varyingscorers, and valid in its differentiation amongsubjects. A central problem in developing thisscoring technique was that of spelling accuracy.If a person writes “Kum kwik wid the dokter ,“ itis difficult to say he is illiterate. On the otherhand, not all variations in orthography are soreadily translated, and it is difficult to judgewhen a message has been conveyed and when ithas not. Similar remarks pertain to handwritinglegibility. It was specified that the subject’s re-sponse could be either in printing or in cursivewriting. Some highly literate persons produce acursive script of formidable difficulty. How couldsuch products be fairly evaluated?
It was decided that a two-dimensional ap-proach, incorporating both a summation of thecorrectness of particular words and a globaljudgment of the sentence by the examiner wouldbe used. As stated below, however, this specifi-.cation was subsequently abandoned on the basis ofpretest results.
While the specification of writing sentencesas dictated was a practical decision, its centralimportance should not be overlooked. Literacy inwriting is typically conceived as the ability toproduce, rather than reproduce, a satisfactorymessage. IdeaIly, one would call for any sort ofwritten message from the subject, allowing thesubject to determine its content. The messagethen would be evaluated in some manner. Such
5
evaluations would be susceptible to variation, how-ever. Furthermore this would lead to a variety ofvocabulary samples since all subjects would notuse the same words. Even worse, vocabulary
content might well be selected by the subject toinsure his success,
The total time required for the writing testwas left unspecified. The time allotted for writ-ing a given sentence was set at 1 minute, subjectto modification following the results of the pre-
testing. Sentence length was to be approximately10 words. Sentence topics were to stress practi-cal situations, such as instructions.
The statistical specifications for the sen-tences could be more general, since the scorevariance would be spread over more categoriesthan the simple right-wrong of the multiple-choiceitems used in reading. No precise difficulty meas-ure was specified; an index of consistency with atotal score and with the reading score was requiredbut left unspecified until the nature of the scoringprocess was better defined.
Consideration was given to a format in whichthe subject would complete a brief document suchas an application blank. This would be, in a sense,the analog of the “want ad” passages which were
introduced into the reading test. This format wasrejected because it would produce responses whichwere unique to the individual; what it offered inface validity for evaluating adult literacy, it would
lose in comparability of subject performance.
PRETESTS AND THEIR RESULTS
In all, 25 reading passages and 75 questionswere presented. The pretest population consistedof 180 fourth-grade students selected from publicschools considered by the administrative officersof the school system to be about average in terms
of national norms on ability tests. One minute wasallowed for each passage and for its three ques-
tions. Observation of the first group confirmed
the appropriateness of this timing. The responseswere indicated by circling the answer in the testbooklet directly, rather than by use of an answersheet, because the mastery of an answer sheet is
sometimes not complete among fourth-grade pu-pils and because the group administration proce-
dure used in the pretest precluded the individualattention which could correct this.
Table 1. Grade level and number of wordsused in each pretest passage
Passage number
------ --.---- ------;-------------------
------- ------- -----:-------------------
+;------------------------- ------ ------ ------- ------ -------
:----------------------------- ------ ---
+1:-------------------11-------------------12-------------------13-------------------
--------- ------ ----+:;-------------------
16-------------------17-------------------18------------------
+;::::::::::::::::::::------ ------ ------ -
#;;-------------------#23-------------------
z4-------------------+25-------------------
Average ---------------Range -----------------
Gradelevell
4.1
::;4.96.35.9
HI4.86.5
M4.1
M4.44.94.54.55.74.96.1
i:;7.3
4.1-M
Numberof words
48.233-64
Grade level frequency distribution
8.0-8 .9-------,--------17.0-7 .9---------------26.0-6 .9---------------35.0-5 .9---------------64.0-4.9--------------13
lDetermined by Lorge formula.
+“Adul~” content (i. e., material fromwant ads or instruction manuals) .
The success of the pretest demanded thatthe
judgments of difficulty be quite accurate. As a
check on these judgments, the index of readingdifficulty proposed by Lorge4 was computed foreach passage. This index takes into account such
factors as thelengthofthe sentences andthenum-ber of “uncommon” words (defined as any wordsnot included in the Dale-Chall listofbasicwords).
Data concerning this index andpassagelengthare presented in table 1. This table shows that
6
the average Lorge index was 5.4—that is, itcorresponded in difficulty to the level of materialwith which the average pupil can cope in aboutthe fourth month of the fifth grade. This figurewas quite a bit higher than either the grade-levelindex of the pretest population, which was 4.5, orthat of the theoretical reference population, whichwas 4.0. It was felt that this was justified becausethe group for which the materials were being de-veloped would be over 11 years of age and there-fore, theoretically, beyond fourth-grade place-ment. Furthermore, the questions in the literacytest would be limited to assessment of compre-hension whereas the Lorge assessment was basedon a complex of skills.
As stated in the discussions of the specifica-tions, there was an attempt to develop materialswith a higher “face-validity” for adults, as illus-trated by items from want ads or instructionmanuals which accompany appliances or equip-ment. In spite of efforts toward reducing the diffi-culty of this type of material, it constituted theseven most difficult passages in terms of the Lorgeindex, as indicated by the high values associatedwith the passages in table 1 which are markedwith a dagger (+). Their possible value in securingsubject acceptance was sufficiently great to war-rant pretesting.
In addition to the 25 reading passages, 10 sim-ple sentences were read aloud, with instructions towrite them in the space provided. In general, therewas more difficulty with the pretesting than hadbeen anticipated, for writing in response to dicta-tion is not a routine school activity at this level.Fortunately, the true simplicity of the task madeit possible to elicit adequate responses with aminimum amount of assistance from proctors.
There were three related statistics used inthe evaluation of the reading pretest results. First,for each question there was computed a phi coeffi-cient measuring its consistency with the totalformula score on the entire 75 questions for thewhole group. Second, for each question there wascomputed a phi coefficient measuring its consist-ency with the total formula score for the lmttom40 percent of the total group. Finally, for eachpassage the sum of the phi coefficients of its ques-tions, as determined on the bottom 40 percent,was computed. These statistical results are pre-: ented in table 2, together with information con-
cerning the level of difficulty of the material (interms of the percentage passing). As describedin the foomote to this table, the phi coefficient forthe total group is referred to as “phi 20-8C},“ andthat for the bottom 40 percent as “phi 50-50,”reflecting the point at which the groups were di-vided. This point is, of course, actually the samein both cases, for the 20th percentile in the totalgroup is the 50th percentile in the lowest 40 per-cent. Two different phi coefficients were requiredto insure effective differentiation of questions inthe region of greatest interest. Appendix I pre-sents a more extended discussion of this.
As indicated in table 2, the pretesting wasgenerally successful. Of the 25 passages, 12 se-cured a cumulative sum of phi 50-50 which ex-ceeded 100. Among the 36 questions which per-tained to these passages, only 4 had related phicoefficients which failed to attain statistical signif-icance at the .01 level of confidence (phi equal toor greater than .31), and 29 questions had coeffi-cients significant at the .001 level (phi equal to orgreater than .39).
The passages with “adult” content were un-successful, with the exception of passage number22, largely because these passages were too diffi-cult to provide differentiation among the bottom40 percent. All of the 10 most difficult questionswere associated with these materials. The gen-eral success of the difficulty estimation is indi-cated by the average difficulty of the questionswhich were not “adult” content. For these 18 pas-sages, the average question was passed by 77 per-cent of the total group, which was very near thespecified value of .80. One other point becameclear in the pretesting. The third question wastypically not much affected by “drop-out,” theusual indication of “speededness.” Accordingly,the use of three questions with each readingpassage could be continued in the final form.
The writing pretest generally sustained theappropriateness of the 1-minute time limit. Theassessment of the consistency between successon a given sentence and success on a total scorefor writing (or for reading) was not easy, asscoring procedures for the sentences had notbeen developed. Rough approximations weresecursd by scoring the sentences word by word,the test of a word being the judgment that it waslegible and that fis meaning was cmveycd in
7
Table 2. Difficultyand validityindexes
Passageand item
l----------------------------2----------------------------3----------------------------
4-------------.--------------5----------------------------6----------------------.-----
--------- --------- ----------:----------------------------9----------------------------
10 ----------------------------11 --------- . . . . . . . . . ---------12---------------------------
A---------------------------
::.........---------.........15---------------------------
6
16-----------=-----------------------------------------
K--------------------------
J-
19---------------------------20---------------------------21---------------------------
_&
22-------------------“-------23----------------------------24-------------------.--.-..--
25---------------------------26---------------------------27“--------------------------
-
28---------------------------29...........”---------------30---------------------------
J_&
31-.--.--.--.---”.--.-”------32---------.........---------33---------------------------
*16 ------------------ .--------
36---------------------------
37---------------------------38---------------------------39--------.----------“-------
Total group
?ercent?assing
836359
90
E
937582
U69
X34
80
;$
79
2;
848365
86
&
%31
848378
787559
898382
Phi 20-80~’
293862
454945
453424
466330
495061
364369
546767
Next-to- Lowestlowest fifth fifth
I
Percentpassing
813625
862522
44816
868672
928186
33’
11
581416
E33
::
581111
44
$:
39366
5864
5:
11
%11
443322
331711
:;22
IWO lowestfifths
Phi 50-502
494718
311815
284720
18
4;
426830
6;-9
445450
396135
414064
Cumulativesum ofphi
. . .
1?:
. . .
lx. . .4798
. . .5974
. . .4964
. .75!95
.,.!53!33
.(,.
1:1.Oll~o
.!,.
6758
.,!.
81145
8
Table 2. Difficultyand validityindexes
Passage and item
41--------------------------42------------.--.----------
7QI43---------.--m------s------&&------.-------------------
x--------------------------2;--------------------------
~49--------------------------50---------------------.----51..-----.”-----------------
M
52------------------------------------..-----W---------
;;--------------------------
x
55--------------------------56.......-------------------57--------------------------
a
58--------------------------59----..-=------------------
~---------..--”-- -------- --
8---------------------------
63---------------------------
@
64---.-----------.-----..--------.----------s----------
::--------------------------
%
----------------------------% ---------------------------
*70 --------- .----=--- ---=----
7:--------------------------
7!2273-----------.=---.------------------------------------
;2------”-------------------
Total group
Percentpassing
876472
U21
%’80
87
H
897576
868678
265122
::66
;:53
53
;;
88
;:
415928
Phi 20-801
565265
2;12
486072
:;42
X66
676863
171820
z!46
50
%
:!29
41
2;
253713
Next-to- Lowestlowest fifth fifth’
Percentpassing
89
%
61316
899783
976964
976778
;;
332517
83
%
19338
5C1414
17
1;
424422
44
;:
47
?:
503925
11336
?;22
4;
11
251916
612819
172217
Two lowestfifths
Phi 50-502
42
::
4529-9
49
2?
58
::
56
;;
46
%
969
303823
493435
9
;
:?16
3
-H
-lativesum ofphi
. . .67113
...
E
...107168
...108147
...
l%
...108158
...
E
...6891
...83118
...1617
● ..6682
...151
1A phi coefficientbased on splittingthe total group into the top 80 percent and the bottom 20percent.
2A phi coefficientbased on splittingthe bottom 40 percent into upper and lower halves.#!tAdul~llcont~t (i.e.,material from Want ads Or inStmCtiOn IMnUa~S).
9
Table 3. Mean score and range of scores,by fifths -
Fifth
Highest scoring fifth--Next-to-highest fifth--Middle fifth -----------Next-to-lowest fifth---Lowest scoring fifth---
Meanscore 1
20.4719.6617.4714.61
2.92
Range ofscores
18-2117-2112-21
8-18-4-11
lMean score ~omDuted as follows: Totalnumber of correct-answers minus one-fourththe number of wrong answers.
context. The distributions of these scores werethen compared for the two lowest fifths, usingarough “consistency measure” which counted thenumber of times that those in the next-to-lowestfifth on total score were better on theparticu-
lar sentence than those in the lowest fifth, andvice versa. The greater this “distancemeasure,”the more the agreement between the score foreach item and the total score. Because the sen-tences wereof unequallength, however,theycould
not be readily compared. The labor of develop-ing the complex statistical information whichwould provide acomparison wasnotjustified. Vir -tually every sentence demonstrated a marked
consistency with the total score; final selectionwas, in general, based on other factors. A brief
description of the consistency measure is pro-vided in Appendix II.
CONSTRUCTION OF THEFINAL FORM
On conclusion of the pretesting, the final phaseof test specification and construction was under-taken. Of the 12 most successful passages, onepassage (number 14) was eliminated because itscumulative phi depended greatly on the lastques -tion, raising the danger of “speededness. ” Thenthe pretest data were examined inordertodeter-mine an optimal number of passages for the finalform. This number was approximately seven, or
21 questions. Accordingly, 7 passages were se-lected from the 11 possibilities. In this selection
both item statistics and content were considered.Thus, pretest passage number22 was preferred
over passages with better statistics because ofits “adult” content. The seven passages selectee’are the first seven shown in Appendix IV.
The total score characteristics of theseven-passage, 21-item test were examined. Table 3
presents themean score using the formula, totalnumber of correct answers minus one-fourth thenumber of wrong answers (R - !4W), .on the 21items for the ability groups defined by pretestitems and the score range observed in each group.
As shown, the test provides the greatest dif-ferentiation between the two lowest fifths and vir-
tually none between the two top fifths. This is, ofcourse, the desired characteristic. An additionalinvestigation of the separation between the twolowest fifths is provided by table 4, which shows
the score distribution for both.The data in table 4 were the basis for the
final decision concerning the location of the cut-ting score, which was set at 10.75 or greaser.
That is, persons scoring 10.50 would be classed“illiterate, ” and those scoring 10.75 would be
Table 4. Formula score frequency distri-butions (R-1/4W) for the two lowest to-tal-score fifths
Score
-4------------------3------------------2------------------1-----------------
0------- ----------1------- ------- ---2 -------- -------- -
------- ------- ---:-----------------5 ------- ------- ---6 ------- ------- ---
------ ------ -----:-----------------
------- ------- ----1:-----------------11-----------------12 ------- ------- ---13-----------------14-----------------15-----------------16-----------------17-----------------18-----------------
Lowestfifth
Next-to-lowest fifth
2
43
10
Table 5. Mean frequencies for and diff-erences between the two lowest fifths,by response category
Response Next-to- ~owes~ Col= 1category lowest fifth minus
fifth column 2
I Mean frequency
Total-- 21.00 21.00 I .*.I
Right -------- 15.61 5.92 +9.69Wrong -------- 3.39 11.56 -8.17Omitted ------ 0.19 -0.19Not reached-- 2.00 3.33 -1033
MeanR-1/4W-- , 14.61 2.92 ...
classed “literate’’w ithinthe meaning ofthework-ing definition.
Table 5 presents a comparison of the twolowest fifths with respect to the average numberof responses which fall into four basic categories:right, wrong, omitted, and not reached. Both“omitted” and “not reached” are blanks, withno
response indicated on theanswer sheet.An’’omit”is ablankwhich isfollowed (notnecessarilyimme-diately) bya responseto asubsequentquestion .Anitem is “not reached” if it is left blank at the endof a series of responses. “Not reached” responsesare used to indicate “speededness” in a test;“omit” responses are generally considered toindicate ample time for a response but a failureto perceive the correct response. There isalwaysambiguity abut the twocategories: an’’omit’’maynot have been read, due to pressure of time; a“not reached” item may have been considered.Nevertheless, the distinction offers some assist-ance in the quantitative assessment of speed.
As shown* in table 5, there is a negligibleamount of “speededness” in the test. The differ-ence in score means between the two groups is11.69; of this, only 1.33 is attributable to the dif-ference in “not reached” items, and then only ifthe lowest fifth can be assumed to have perfectsuccess on these items. In general, then, “speed-edness” is a very small factor in the test. Further,the small number of “omits” indicates that the
items are not skipped as the test is workedthrough. Apparently the salient characteristicsof the items are such as to encourage responding.
The reliability of the 21-item test was esti-mated to be .91 by a technique suggested by Rajuand Guttman. s This estimate indicates an excel-lent reliability for the survey work for which thetest is intended. Additional features of the testwhich heightened its utility for the survey werethe use of the cutting score for securing brieferrecords and the availability of substitute passagesfor “repairing” a record damaged by the faultyadministration of one of the passages.
The final development of the writing test wasbroadly similar to that of the reading test. Afive-sentence test, totaling 47 words, with 1 min-ute per sentence was developed (see Appendix V).The five sentences were selected for appropriateconsistency with a total-score criterion, forvariety of content and vocabulary, and for sen-tence length. Once a scoring technique was de-veloped, a cutting score was determined. This isbetween 27 and 28 (fractional scores are not pos-sible): a person scoring 27 is classed “illiterate”;a person scoring 28 is “literate.” This cuttingscore is estimated to divide subjects at grade level4.0 into two equal groups on the basis of the dataon the sample of subjects at grade level 4.5.
The principal labor concerning the writingtest was the devising of a reliable scoring pro-cedure. Initial attempts to develop a scheme whichrelied on judgment for accepting or rejectinghomophone approximations to standard orthog-raphy (“dokter, I! I‘tumorow”) proved unworkable.Even a group of staff members accustomed toworking together on verbal items could not securea sufficiently high degree of consistency. Aftermuch experimentation, it was decided to maximizethe reliability of the scores by creating a scoringsystem which assigned a score based principallyon errors of misspelling, of word inversion, andof word redundancy. This technique is describedin the examiner’s manual (Appendix VI). It hassatisfactory correlc tion with the various subjec-tive and judgmental approaches; what it loses inoccasional instances by overpenalizing spellingerrors, it gains in other cases by permitting dif-ierent raters to score complex sentences in asimilar manner.
11
Table 6. Length of time per test unit,based on performance of 12 studentsidentified as poor readers
Passage andsentence
Passage
1------- ----------.’ - - - -- - ---- - - - - ---
2-----------------4- - - ---- . - --- - - .- -
- - - -- --- - - -- -- - - -2------------------
--- -- - - -- - . --- - - -:-------------------
------- -----.- -q-1:--------------------11-----------------
Sentence
1 -------- ------.- --- -- -- - . - - - - - --- - - -3---------------------;------------------------------------
Average timein seconds
47.553.343.150.442:549.350.347.446.346.04.7● 3
27.334.833.333.938.4
SCREENING TRYOUTS
The construction of the final form wasfol-lowed byscreening tryoutsinwhich thenewinstru-ment was administered in a person-to-personsituation to 24 students aged 14 through 17 whohad been identified by reading teachers as havingreading difficulty. The purposes of this tryoutwere to assure that workable administrationpro-cedures were developed and that passage contentwas equally acceptable at the olderagerange,andto checkon “speededness.”
These trials were very successful. Whileno formal validity estimates were providedbytheteachers, there was informal evidence in thatthe three persons who wouldbejudged ’’illiterate”by the test were in fact so judged by the school.Expectations regarding the time element wereconfirmed. Even in this population, there was aconsiderable shortening of the time req,uired assoon as any appreciable literacy was found. Nouse was made of the cutting score, since allpas-sages neededto be screened for content accepta-bility, but the general practicality of theproce-durewas demonstrated.
An answer sheet enabling all responses tobe recorded, both for reading and for writing,had been devised (see Appendix III).
Table 6 presents the average time requiredfor each passage and for each sentence as ob-served by one examiner in screeningtryouts. Thegiven averages are based on only 12cases, butthe consistency of the results across passagesand sentences lends credence to their reliability.
These average times demonstrate that whilethe total working time for all tasks can beasmuch as 12 minutes, this will not often be thecase. The Brief TestofLiteracyisindeed’’brief.”
SUMMARY
This detailed account of the developmentalprocedures has concentratedon descriptionratherthan on critical evaluation. Manyof the steps in-volved were basedon assumptionor professionaljudgment, the adequacy of these being crucialtothe success ofthe enterprise. Similarly, where-ever statistical data were the basis for decision,the size of the sample from which they weredrawn was a practical maximum rather than atheoretical optimum.
Nevertheless, the general consistency of theresults and their coherence suggests that thedevelopmental procedures were highly success-ful. It is expected that, following the establish-ment of norms and the completion of validationstudies, the Brief Test of Literacy will providea useful instrument for survey purposes.
REFERENCES
l“W’orld Campaign for Universal Literacy ~’ Document sub-
mitted by UNESCO in response to a request of the United Na-
tions General Assembly at its Sixteenth Session. May 1963.p. 39.
2English, H. B., and English, A. C.: A Comprehensive
Dictionary of Psychological and Psych oanalytr%al Term. New
York. David hlcKay Co., Inc., 1958.
3Lord, F. M.: Cutting scores and errors of measurement.
Psychometrika 27:19-30, 1962.
4Lorge, 1.: The .Lorge Formula fop Estimating Difficulty of
Reading hlaterial,s. New York. Bureau of Publications, Teach-ers College, Columbia University, 1959.
5Raju, N. s,, and Guttman> ‘-: A new working formula for
the split half reliability model. Educ. & Psycho?. Measw.
25(4): 963-967, 1965.
12
Acknowledgments
The development of the test materials and theconduct of the pretesting tryouts were facilitatedwithin Educational Testing Service by the coopera-tion of Miss Susan Humphrey, Mrs. Helen Spiro,and Mrs. Sara Hufham. Further, the project couldnot have been completed without the cooperationof the public schools of the city of Trenton, N. J.,through Dr. Sarah Christie, Assistant Supervisorof Schools, and the public schools of Princeton,N. J., through hlr. Thomas Seraydarian, Directorof Guidance.
000
13
DISCUSSION
The need for two phi coefficients, as ,
APPENDIX I
OF THE USE OF PHI
mesented in them. Antable 1, may be most quickly demonstrated by the bottom 40following contrived examples of contingency tables. Ineach case the entxies in cells and margins are per-centages of the total gvoup.
The following question would show a sizable phicoefficient of consistency between item and test:
COEFFICIENTS
item might show the following table for thepercent, which would yield a sizable phi:
TEST
TEST
ITEM
%=
1070 801010 202080 100
Suppose, however, that the performance of the top80 percent, in which seven-eighths or 87.5 percent weresuccessful, was examined more closely as a 2 x 5 tablein which each fifth of the total group is presentedseparately:
“EM-ONote that the item actually differentiates most
Information on the top 60 percent, however, mightlead to a completed 2 x 5 table of
TESTITEM 5 15 5 10 20 55
15 5 15 10 - 4520 20 20 20 20 100
indicating that item ambiguity or some other peculiaritywas distorting the normal pattern of increasing itemsuccess with increasing ability. For the 2 x 5 table, phi20-80 would be computed from
TEST
ITEM
R
5 50 5515 30 4520 80 100
markedly between the Imttom 40 ‘percent and the top 60 which would be lower than phi 50-50, signaling the dis -percent.’ In fact, phi 50-50 on the bottom 40 percent torted pattern.would be zero, correctly indicating that this item should The foregoing cases are necessarily preselectednot be chosen in spite of the value of phi 20-80. and dramatic. Nevertheless, the practice of using two
On the other hand, there are anomalies in items, coefficients of this type in the development of a cuttingand it is the function of item analysis to guard against score instrument has much to recommend it.
14
APPENDIX II
OF THE COEFFICIENT OF SENTENCE
An example of the consistency measure used inevaluating the sentences is presented below. In general,
a given sentence is consistent with the total score ifthose in the more able group score higher than those inthe poorer group. The consistency measure totals thenumber of times that an individual in the superior groupscores higher than an individual in the less able group;from this total is subtracted the number of times thatindividuals in the less able group surpass individualsamong the superior group.
Suppose that a given six-word sentence yielded thefollowing distributions:
6 ----------- -------- ------------ -----------------
;------------------------3------- --------- --------2------- ------- ----------
------- -----------------:------------------------
(1)
Score
6 ------- ------- ------------ ------- ---
L----------------------- -.------- -
:------------------1------------------
(2)
Number ofsuperior group
in category
CONSISTENCY
The consistency measure would be computed as follows:
:1010
55
Sentence score
k,
(3)
Number ofless able group
they surpass
10
(4) I (5)
Number of lessable group whc, (2)x [(3)-(4)]
surpass them
1:20
200200350250
-%
I
I Index 1,000
Thechance expectation of this index is zero, nega- It is similar to several slch indexes proposed in thetive values indicate an inverse relationship, et cetera. psychometric literature.
15
ANSWER SHEETS
APPENDIX Ill
FOR READING AND WRITING TESTS
SAMW.EPAGE
Question AnswerNumber Choice
OIABCDE
02 ABCDE
03 ABCDE
Question AnswerNumber Choice
1
2
3
4
5
6
7
8
9
A
A
A
A
A
A
A
A
A
B
B
B
B
B
B
B
B
B
c
c
c
c
c
c
c
c
c
DE
DE
DE
DE
DE
DE
DE
DE
DE
Name
QueBtlon AnswerNumber Choice
10
11
12
13
14
15
16
17
18
19
20
21
AB
AB
AB
AB
AB
AB
AB
AB
AB
AB
AB
AB
c
c
c
c
c
c
c
c
c
c
c
c
DE
DE
DE
DE
DE
DE
DE
DE
DE
DE
DE
DE
Question An6werNmber Choice
22
23
24
25
26
e
28
29
30
31
32
33
ABC
ABC
ABC
ABC
ABC
ABC
ABC
ABC
ABC
ABC
ABC
ABC
DE
DE
DE
DE
DE
DE
DE
DE
DE
DE
DE
DE
16
ANSWER SHEEI! FOR WRHYiliG TEST
Nmte
1.
2.
3.
4.
5.
000
17
APPENDIX IV
INSTRUCTIONS FOR READING
ON EACH PAGE IN THIS BOOKLET THERE IS ASHORT PARAGRAPH WHICH IS FOLLOWED BY THREEQUESTIONS. BELOW EACH QUESTION ARE FIVESTATEMENTS, ONLY ONE OF WHICH MAKES A GOODAND SENSIBLE ANSWER. YOU SHOULD FIND THISSTATEMENT, AND MARK YOUR ANSWER BY CIR-CLING THE LETTER ON THE ANSWER SHEET WHICHCORRESPONDS TO THE STATEMENT YOU SELECT.
YOU MUST WORK AS QUICKLY AS YOU CAN, FOR YOUWILL BE ALLOWED ONLY ONE MINUTE TO WORI<ON EACH PARAGRAPH. BECAUSE THE TIME IS SOSHORT, YOU MAY NOT FINISH ALL OF THE QUES-TIONS. IF YOU DO FINISH A PAGE BEFORE THETIME IS UP, TELL ME AND YOU WILL BE ALLOWEDTO GO ON TO THE NEXT PAGE.
Department of Health, Education, and WelfarePublic Health Service
National Center for Health Statistics
Reprinted with Permissionof
Educational Testing ServicePrinceton, N.J. Berkeley, Calif.
@ Copyright 1966All rights reserved
SAMPLE PAGE
It was a beautiful gift, wrapped with bright redpaper and tied with silver string. It was small, but veryheavy. No one knew who had brought it, but it had Mr.Jones’ name on top. Mr. Jones just smiled and said,“1’11open it when I get home.”
01. Whose name was on the top of the gift?
(A) Mr. Jones(B) Mr. Pike(C) Winy(D) The postman(E) No one knew
02. In what color paper was the gift wrapped?
(A) Red(B) Silver(C) Green(D) Orange(E) Yellow
03. Where was the gift going to be opened?
(A) Where it was found(B) At the police station(C) In the car(D) At the office(E) At home
JX) NOT TURN THE PAGEUNTIL YOU ARE TOLDTO DO SO.
-o-It was spring. The young boy breathed the warm
air, threw off his shoes, and began to run. His armsswung. His feet hit sharply and evenly against the ground.At last, he felt free.
1. What time of year was it?
(A) Summer(B) Fall(C) Spring(D) December(E) July
2. What was the young toy doing?
(A) Running(B) Jumping(C) Going to sleep(D) Driving a car(E) Fighting
3. How did he feel?
(A) Hot(B) Free(C) Angry(D) Cold(E) Unhappy
DO NOT TURN THE PAGEUNTIL YOU ARE TOLDTO DO SO.
-1-
There were footsteps and a knock at the door.Everyone inside stood up quickly. The only sound wasthat of the pot boiling on the stove. There was anotherknock. No one moved. The footsteps on the other side ofthe door could be heard moving away.
4. The people inside the room
(A) Hid behind the stove(B) Stood up quickly(C) Ran to the door(D) Laughed out loud(E) Began to cry
5. What was the only sound in the room?
(A) People talking(B) Birds singing(C) A pot Imiling(D) A dog barking(E) A man shouting
6. The person who knocked at the door finally
(A)(B)(c)(D)(E)
Walked into the roomSat down outside tine doorShouted for helpWalked awayBroke down the door
DO NOT TURN THE PAGEUNTIL YOU ARE TOLDTO IX) SO.
-2-
Helen liked going to the movies. Sometimes shewent four times a week. Everyone said she was crazy.Why did she always want to go out and spend money,they Stid, when she could stay home and watch tele-
vision?
7. What did Helen Iike to do?
(A) She liked to eat(B) She liked to swim(C) She liked to watch baseball(D) She liked to watch movies(E) She liked to watch wrestling matches
8. What did people think aimut her?
(A) They thought she was crazy(B) They thought she was very smart(C) They thought she was very nice(D) They thought she was ugly(E) They thought she was very old
9. What did people think she should do?
(A) Write a book(B) Watch television(C) Goon a diet(D) Dye her hair(E) Stop talking so much
DO NOT TURN THE PAGEUNTIL YOU ARE TOLDTO DO SO.
-3-19
You could smell the fish market long before youcould see it. As you came closer you could hear mer-chants calling out about fresh catches or housewivesarguing about prices. Soon you could see the marketitself, brightly lit and colorful. You could see fishingboats coming in, their decks covered with silver-greyfish.
10. What kind of a market is described above?
(A) A vegetable market(B) A meat market(C) A fish market(D) A flower market(E) A fruit market
11. What could you see coming in?
(A) Tug boats(B) Rowboats(C) Passenger boats(D) Fishing boats(E) Sailboats
12. What covered the decks of the boats?
(A) Rope(B) People(C) Cars(D) Boxes(E) Fish
DO NOT TURN THE PAGEUNTIL YOU ARE TOLDTO DO SO.
-4-
Bill settled down sleepily into the seat at the backof the bus. All he wanted to do was to sleep until it wastime to get off. But the noise of a nearby radio and thevoices of the passengers kept him awake. Without think-ing, Bill stood up and shouted, “Shut up, everybody!”
13. In what was Bill riding?
(A) A boat(B) A car(C) A plane(D) A taxi(E) A bus
14. What did Bill want to do as he rode?
(A) Sleep
(B) Eat-----
(C) Drink(D) Talk(E) Read
15. What did he shout?
(A) “Help:”(B) “This is my stop!”(C) “Shut up, everybody!”(D) “There’s a fire!”(E) “We’re going to crash!”
DO NOT TURN THE PAGEUNTIL YOU ARE TOLDTO DO SO.
-5-20 ‘
Tiger is a large, yellow cat. At night he prowlsoutside and is very fierce. When he hears a noise, helowers his head and walks with stiff legs. All the othercats are afraid to come into his yard.
16. When does Tiger prowl?
(A) At dawn(B) At dinnertime(C) In the afternoon(D) In the morning(E) At night
17. What does Tiger do when he hears a noise?
(A) He runs away(B) He walks with stiff legs(C) He hides under the bushes(D) He walks on tiptoe(E) He pretends he doesn’t hear it
18. Who is afraid to come into his yard?
(A) All the other cats(B) The dog next door(C) The people who live in the house(D) The mailman(E) Most of the birds
DO NOT TURN THE.PAGEUNTIL YOU ARE TOLDTO DO SO.
-6-
The model number of your radio is A-;707. Weaksound may indicate weak batteries. Replace with freshbatteries. Failure of the radio to operate may indicatea loose connection. All connections should be checked.If the radio still does not work properly, take it to ourservice department, 17-B West 17th Street.
19. What is the model number of the radio?
(A) A-707(B) 17-B(c) W-17(D) B-17(E) AB-707
20. What should be done if the sound is weak?
(A)(B)
(c)
(D)(E)
Use weak batteriesSend the model number to the service depart-mentReplace the present batteries with fresh bat-teriesCheck all the connectionsReplace the comections
21. What is the address of the service department?
(A) 17-A West 17th Street(B) 17-B West 17th Street(C) 17-A West 7th Street(D) A-707 Weat 57th Street
(E) 17-B West 57th StreetDO NOT TURN THE PAGEUNTIL YOU ARE TOLDTO DO SO.
-7-
Sara hated big dinners. There were so many dishesto wash afterwards, and no one ever thought to thank herfor doing them. And people always stayed so late aftera big dinner. Sometimes it was midnight before shecould begin to clean up.
22. Why did Sara hate big dinners?
(A) Because she aIways ate too much(B) Because people were so noisy(C) Because there were so many dishes to wash(D) Because she was never invited(E) Because they were so exTensive
22. How often did people remember to thank Sara?
(A) Sometimes(B) Always(C) Never(i)) Once(E) Twice
l!q. HOW late did it sometimes get before Sara couldclean up?
(A) Noon(B)Morning(C) Afternoon(D) Midnight(E) Evening
DO NOT TURN THE PAGEUNTIL YOU ARE TOLDTO DO SO.
-8-
The cat brushed against the old man. He did notmove. He only stood, staring up into the window of thehouse. The party inside looked warm and friendly, butno one noticed him. The old man walked sadly on,followed @ the cat.
25. JWat kind of animal was with tie old man?
(A) blouse(B) Dog(C) Horse(D) Cat(E) Bird
26. mat was inside the ho~se~
(A) A party(B) Some dogs(C) An old lady(D) A meeting(Ej A salesman
27. The man is described as being
(A) Old(B) Young(C) Thin(D) Fat(E) Small
DO NOT TURN THE PAGELNTIL YOU ARE TOLDTO DO SO.
-9-
“1 know you are in there,” said the sheriff. “Youhave five seconds to come out.”
I,, shouted the robber from inside“Come get me.the house.
The sheriff began to count. “One. Two. Three.”Suddenly, the robber walked out with his hands up.
28. Where was the robber?
(A) Inside the house(B) By the river(C) In the bushes(D) On his horse(E) In the barn
29. How long did the sheriff give him to come out?
(A) Five seconds(B) One minute(C) Five minutes(D) Ten minutes(E) An hour
30. What did the robber do?(A) He ran out shooting both guns(B) He tried to escape and was shot down(C) He walked out with his hands up(D) He sneaked out ad got away(E) He didn’t come out, so the sheriff had to
go in and get himDO NOT TUIW THE PAGEUNTIL YOU ARE TOLDTO DO SO.
-1o-
His cigarette went cwt. His pen dropped from hishand. His head began to nsd. He was, all at once, asleep.Everyone in the room laughed, for he had come to workonly five minutes ago.
31. What dropped from his hand?
(A) A pen(B) i} penciI(C) A piece of paper(D) A telephone(E) A imok
32. What was he doing after his head began to nod?
(A) Talking(B) Sleeping(C) Crying(D) Smoking(E) Leaving
33. When had he come to work?
(A) Half an ho~- ago(B) Three hours ago(C) Yesterday(D) Five minutes ago(E) Forty minutes ago
DO NOT TURN THE PAGEUNTILYOU ARE TOLDTO DO SO.
-11-
21
APPENDIX V
FIVE ITEMS USED IN WRITING TEST
1. Turn left at the next corner.
2. School will be closed tomorrow because of heavy snow.
3. Send today for your free copy of this book.
4. If you need a doctor, call this number right away.
5. Drop a dime in the slot and turn the handle to the left.
22
APPENDIX VI
BASIC SKILLS SURVEY
READING AND WRITING
– MANUAL FOR EXAMINERS –
@ Copyright 1966by
Educational Testing ServicePrinceton, N.J. Berkeley, Calif.
Ml rights reserved
23
BRIEF TEST OF LITERACY
MANUAL FOR EXAMINERS
INTRODUCTION
The Brief Test of Literacy was intended to providea sound basis for classifying subjects as “literate” or“illiterate” within a very short time limit. There aretwo tests—one of reading and one of writing. The read-ing test contains seven brief paragraphs, each accom-panied by three questions, for a total of twenty-onequestions; the writing test consists of five sentencestotaling forty- seven words.
The tests and testing procedures were designedto provide the maximum information for the simplecategorical dec’ision, “literate’~ or “illiterate.” Forboth reading and writing, literacy was defined as ap-proximately that level of function which is attained bythe average student a~ the beginning of the fourth grade.Since the nature of the decision is essentially “either -or, ” a cutting score technique is used: all persons abovea certain test score are classed as literate, all personsbelow the score are classed illiterate. The cuttingscore in turn provides the basis for the very brieftesting times which are possible with this instrument,for the testing need only be continued until this scoreis achieved. That is, it is sufficient to be able to knowthat the subject is above the cutting score (hence “lit-erate” by definition); how far above ia not important,Indeed, the instrument is not well. suited for differ-entiating among persons who are not near the cuttingscore. It tends to bunch such people into a single scorecategory, since it haa been specially built to provideits maximum of information at and near the cuttingscore. In achieving this maximum, information aboutdifferences at other levels is necessarily lost.
Reauired Materials
An administration requires:(1) stopwatch(2) pencils (with erasers)(3) reading test booklet(4) answer sheets(5) manual for examiners
ADMINISTERING THE READING TEST
Procedures
Seat the subject at a desk or table, provide himwith a pencil, answer sheet and booklet, and have himwrite his name in the space provided. Then say:
This is a brief test of reading and writing. Itwill last about ten minutes. Read the instructionson the cover silently to yourself while I readthem aloud to you.
Read as follows:
On each page in this booklet there is a shortparagraph which is followed by three questions.Below each question are five statements, only oneof which makes a good and sensible answer. Youshould find this statement, and mark your answerby circling the letter on the answer sheet whichcorresponds to the statement you select.
You must work as quickly as you can, for youwill be allowed only one minute to work on eachparagraph. Because the time is so short, you maynot finish all of the questions. If you do finish apage before time is up, tell me and you will beallowed to go on to the next page.
After reading the instructions, ask if there are anyquestions. The typical subject will NOT haye any ques-tions; those who do will frequently m&rely requirerepetition of the appropriate part of the instructions.The following replies are suggested for possible ques-tions in two areas:
Erasing
Questions: Can I change my answer?Ia it o.k. to erase?Is it o.k. to cross out my first answer?
Reply: Yes, but work as quickly as you can.
24
Guessing
Questions:
Reply: We
Is it o.k. to guess?Do you count off for guessing?Can I guess?are subtracting a penalty for each
wrong answer, so wild guessing is unlikelyto improve your score, and it may lower it.However, if you can eliminate one or moreof the wrong answers, it is probably to youradvantage to guess.
When the subject is ready, read the following, point-ing to the appropriate section of the answer sheet:
Read the paragraph and then answer the questionsby circling the appropriate letters (point to 01,02, 03 on the answer sheet). There is only onecorrect answer for each question. Tell me whenyou have finished with the paragraph.
Ready? Begin work.
Begin timing. At the end of one minute, say:
Stop working. The time is up. Do you have anyquestions?
If the subject completes the sample page in lessthan a minute, say:
Finished? Fine. Do you have any questions?
Few questions will be asked. Some may inquireabout guessing or erasing, as described above; a fewmay wonder if the paragraphs in the test are any longer’than the sample paragraph. A simple reply is:
The paragraphs differ in length from page topage, but they are all about as long as this sample.
When all is ready, say:
I Now we will begin the test. Remember, if youfinish a page before time is called, tell me thatyou are finished. Do not turn to the next pageuntil you are told to do so.
I Ready? Turn over to page one and beginworking.
For each page, begin timing when the pages lie flat.Some subjects will smooth the booklet; others arrangetheir answer sheet; they vary in the way they spend thefirst few seconds. Therefore, there is a need for afixed starting point, and this is when the pages lie flat.Do not worry if individual subjects seem to take toolong before beginning worlg the time allotted is reallyquite generous and any capable reader has sufficienttime to demonstrate his ability.
The remaining work of giving the testis repetitive.If the subject indicates that he is finished, say:
Finished? Turn cwer to page — and beginwork.
If the subject does not finish in one minute, say:
I Stop. Turn over to page–and begin work. /
Always state the page number which the subjectshould be working on, in order to avoid confusion.
Substitute Para~aDhs
The administration of a rapidly-paced examinationoften leads to errors in timing, etc. In this examination,the time is so brief that a sneeze, a broken pencil, orother inadvertent interruption may cast doubt on theperformance on a given paragraph. For this reason,alternate passages are provided on pages 8-11 of thetest booklet. If one of the initial seven passages mustbe replaced, it is suggested that it be done accordingto the following program:
For Passage on Page Use Passage on Page
1 89
1110
999
The same cutting score of 11 may be used in eachcase. This procedure assumes an equivalence amongpassages that is not rigorously true. However, it wouldseem to be superior to the use of examiner judgmentin effecting remedies for deviant records, for suchjudgments are characteristically unreliable.
Use of the Reading Test Cutting Score
This test is scored by giving 1 point for a rightanswer, O for an omit, and -?4 for a wrong answer.The complete reading test consists of seven passages,with a total of twenty-one questions. Because of thepenalty for wrong answers, the scores could rangefrom - 5X (all wrong) to 21 (all right). II-Ipractice,however, all that we are interested in knowing iswhether or not the subject gets a ‘tformula score”(R-MW) greater than 10.5. If he does, he passes and is
classed “literate”; if his score is 10.5 or less, he failsand is “illiterate” in terms of this test. The cuttingscore was selected on the bssis of the statistical in-formation concerning the t -SC.
25
Obviously, if the subject completes the first fourparagraphs and gets all questions correct, he has ascore of 12 and is “literate. ” There is no need to giveadditional questions. Similarly, if he gets 11 right and1 wrong, he will pass. Almost all capable readers willanswer the 12 questions correctly, and in much lesstime than the four minutes allotted. Hence, the use ofthe cutting score can reduce the average testing timefor -geading, including instructions, to under five min-utes.
To use the cutting score, the examiner must be inposition to observe the subject’s work unobtrusively.In effect, he scores the answer sheet as the subjectworks. This is typically a simple operation and can bedeferred until the fourth paragraph is begun. The scoringkey is provided on page 6 of this manual.
Because the cutting score is between 10 and 11, itis possible to accept the decision “literate” before allquestions on the fourth paragraph are completed. It isalso possible to accept the other decision, “illiterate,”before the fourth page is completed. (In fact, the de-cision “illiterate” may be reached at the conclusionof the first three passages, if all of the nitie answersto these passages are wrong, for even if the subjectanswered the remaining twelve questions correctly, hewould fail to achieve a score greater than the cuttingscore.) It is recommended, however, that full seven-passage records be obtained for all subjects exceptingonly those who have 11 or 12 right answers on the firstfour passages.
This recommendation means that even subjectswho pass the cutting score in the course of their workon the fifth or sixth passage, should be continued forthe full seven passages. It awards a premium, in asense, to the perfect or near-perfect performance onthe early paragraphs. Subjects who attain these ex-cellent records may be presumed to be so capable thatnear-perfect performance on the remaining questionsmay be granted.
To summarize: the cutting score is between aformula score of 10.5 and one of 10.75; at 10.5 or less,the subject “fails” and is “illiterate,” at 10.75 orgreater, he “passes” and is “literate.” Subjects willachieve the cutting score, or demonstrate an inabilityto achieve it at varying points in their work. It isrecommended, however, that all subjects complete allseven passages excepting only those subjects who get11 or 12 right answers on the first four passages. Thetime saving of the cutting score will be realized for avery large percentage of the prospective group, ages12-17. Approximately 95X of this group may be antici-pated to answer the twelve simple questions correctlyand in a few short minutes. For the remainder of thegroup, the need for a complete record is more crucialand the attempt to save time by shortening the recordis not worthwhile.
SCORING INFORMATION
Answer Keys
Sample Questions
01 A02 A03 E
Test Questions (Pages 1-7)
Page 1 Question 1 C2A3B
Page 2 Question 4 B5C6D
Page 3 Question 7 D
8A9B
Page 4 Question 10 C11 D12 E
Page 5 Question 13 E
14 A15 c
Page 6 Question 16 E17 B18 A
Page 7 Question 19 A20 c
21 B
Supplementary Questions (Pages 8-11)
Page 8 Question 22 C23 C24 D
Page 9 Question 25 D26 A27 A
Page 10 Question 28 A29 A30 c
Page 11 Question 31 A32 B33 D
Decision Chart
After four passages:
Any subject having 11 or 12 right answers isclassed “literate” and testing is discontinued.
After seven passages:
Any subject having 13 or more right answersis classed “literate. ”
26
Any subject having 12 or more right answersand 5 or fewer wrong answers is classed “liter-ate.”
Any subject having 11 right answers and only1 or O wrong answers is classed “literate.”
All other subjects are classed “illiterate.”
ADMINISTERING THE WRITING TEST
Procedures
After the reading test is completed, say:
That’s the end of the reading test. The nexttest is the writing test. Turn over your answersheet.
When the subject is ready, say the following, point-ing to the three lines of the first answer space at theappropriate time:
Listen carefully. I am going to read a sen-tence to you and I want you to write it in thespace provided after I have read it twice. Useas much space as you need, and tell me if youwant the sentence repeated. You have oneminute.
Do you have any questions?
Most questions seem to be quasi-questions whichrepeat the instructions in different wording and merelyrequire some simple confirmation.
Example: “1 write down what you say?”Reply: 11ye5.11
Some subjects may ask: “Do you count off for poor
spelling?”A suggested reply would be: “Yes, spelling does
count, but just do the best you can.”
The questions which were mentioned earlier inconnection with the reading test, concerning erasing,crossing out, etc., may also be asked at the beginningof the writing test. Refer to the earlier discussion forthe suggested replies. Still another question may con-cern the possibility of breaking the pencil. If this isasked, say: “If you break your pencil, I will give youanother.”
When all is ready, read the sentence twice at amoderate rate. As you finish, say:
~
As you say “Begin writing,” you should begin tim-ing. Allow one minute and then say:
Stop. Listen carefully and I will read sen-tenc~ (the next sentence).
If the subject finishes before time is up, say:
Finished? I will read sentenc~.
In each case, say “Begin writing” as the signal tobegin.
The sentences are:
1. Turn left at the next corner.2. School will be closed tomorrow because
of heavy snow.3. Send today for your free copy of this book.4. If you need a doctor, call this number right
away.5. Drop a dime in the slot and turn the handle
to the left.
Some subjects will ask to have the sentence re-peated. Others may have an obvious difficulty but hesi-tate to ask. The examiner should watch carefully andrepeat the sentence on his own initiative if the subjectappears to need it. This is not a memory test. No realharm can come from repetition. The average subject,of course, has no trouble retaining the sentence andwould find further repetition an interruption.
In general, the writing test can be completed re-gardless of interruptions or breaking of pencils, etc.,for if the examiner wishes he can always instruct thesubject to begin over again and time him from the newstart. That is, since the test is not one of memory,practice makes little difference, and a broken pencilor a fit of coughing or other interruption can be copedwith by starting over again. If needed, the margins ofthe answer sheet will provide the space for a secondattempt on interrupted questions.
SCORING INFORMATION
The scoring of constructed responses alwaysposes difficulties, largely because of the variety ofdeviations from the norm which occur. Even in the sim-ple task used in this test, the poorest writers will pro-duce quite complex responses, difficult to evaluate. Toreduce the problems and to secure reliability, the scorefor the writing test is based simply on the number ofwords correctly spelled and on the correctness of theirorder. For example, the sentence
~ yu need a dokter, call this nmbr rite away.
receives a score of Q 1 point for each correctlyspelled (underlined) word. No credit is given for mis-
27
spellings, even when the approximations areas phoneti-tally acceptable as the words “yu,” “dokter,” and“rite.”
An immediate problem concerns the legibility ofthe handwriting. Inevitably, examiners will differ as towhat the subject actually wrote. In general, the guidingprinciple should be to give the subject the benefit of thedoubt on any given letter. That is, if the response to aword is so poorly written as to be meaningless, nocredit for that word is given, but if a letter is unclear,do not penalize. For example, if it is uncertain whetherthe subject really wrote “e” for “o” in “doctor,” givetbe subject tbe benefit of the doubt and 1 credit for theword. To repeat: If a single letter is ambiguous, assumethat it is correct; if whole words are illegible, do notgive credit.
The order of the words is important. The subjectdoes not get credit even for a correctly spelled wordif this word is out of place. For instance, if the exampleabove had been written
~ yu @ to call a dokter, use this nmbr rite—— —away.
the score for this would be ~. The subject would not‘receive credit for Q, which is out of place. Thissecond example also provides an instance of the intro-duction of new words into the response, for “use” and“to” do not appear in the original sentence. No creditis lost for such introductions, which occur chiefly inthe records of borderline subjects. The problem ofdetermining the correctness of order is more difficultthan is apparent at first glance. For example, one sub-ject responded to sentence 1:
Turn next at the left c-r.—— ——.
To cope with the diversity of possible subject responses,the scorer writes above each correctly spelled word thenumber which indicates the order in which it appearsin the sentence as dictated. For example:
l@3 @26Turn next at the left corner.—— . . . _
This receives a score of ~, according to the followingprocedure: No word is scored if its number is greaterthan the number of the word immediately on its right.In the example above, “next” and “the” would not bescored, for “5” is greater than “3” and “4” is greaterthan “2.” Because this procedure is basically mathe-matical and mechanical, it will not exclude the samewords as would a judge. In the foregoing example ajudge would rule out “next” and “left,” rather than“next” and “the.” However, the same score is arrivedat botb by judging and by applying the rule: four wordsare given credit. In problems of more complex reorder-ing, the merit of the mechanical approach will beapparent, for it is quite simple and reliable. A devicefor tallying the eliminated words is simply to draw acircle around the number above them.
One difficulty arises from the tendency of lmrder-line subjects to repeat words. Thus, one candidatewrote
Send today for this free book today.
This would be scored as 5:
12 3@59mSend today for this free book today.—— .— .— —
There are two instances of the word -. When-ever this occurs, if you give credit for the first suchword, by the basic rule, draw a square around the num-ber of the second such word and omit it from furtherconsideration in the scoring. By the normal applicationof the basic rule, the word “book” should not be scored,for its number, 9, is greater than the number of the nextword. However, having credited the first “today,” thesecond is deleted and does not affect the value of “book.”
Some Examples of the Scoring
The following sentences were actually encounteredin the testing:
23 Q49Example 1 Sent today for fee cpy of your fee book.—. .——
The score is ~. Do not credit “of” because the num-ber of the word to the right is less. Note that whileactuaIly “your” is more properly the misplaced word,the rule accounts for the inversion by excluding “of.”The net effect is the same.
12 4 78~9Example 2 Send today fou your charpy of this of book.—— . .— .—
The score is ~. When the first “of” is scored, noteits position number, 7, and draw a square around thesecond 7. This permits the word “this” to be scoredwhen it is encountered later, for the next correctlyspelled word is now f!bookr T with a position number
greater than that of “this.”
123456 8Example 3 if you need a doctor call these number—— —— ——
10write away.
The score is ~. Notice that punctuation errors, suchas the failure to begin the sentence with a capital letter,are not penalized.
Note: In sentence 5 of the test, the word “the” appearsthree times. Therefore, the rule for dealing withredundancy and misplacements cannot be appliedto this word in this sentence. After assigning thenumber to eacfi word, apply the basic rule.
4789 11 12Example 4 Dron in and turn the hand to the life..— —— ——
28
The score isQ. The second appearance of “the” is ‘indexed as 12, and is not considered to be a redundant—expression of the earlier appearance in which the wordwas indexed 9.*
The followinge xarnple was contrived to clarify anddemonstrate the scoring.
Example 5
Step 1:
Send tuday for copy of yore free copy ofthis book.
Underline all correctly spelled words:
Send tuday for copy of yore free copy of—— . —— —this book.——
Step 2: For each underlined word, write the num-ber which indicates its position in the original sentence.
1 3 67 5 67Send tuday for copy ~ yore free copy of—— .79this book.——
Step 3: Begin scoring, counting any word which hasa number less than that of the next correctly spelledword on its right. If you credit a word which appearstwice, draw a square around its second appearance. Forexample, the following sequence would be followed inthe sample sentence above.
Sendforcopy
of
freecopyofthisbook
Number Action
give credit: give credit6 give credit; box redundant sec-
ond “6”7 no credit (next number. 5. is
less,,t~an ‘7) ; do ~t ~ox’sec-ond 7
give credit; no credit, because boxed
give credit: give credit9 give credit
Step4: Total the number of credits to get score.Score would be 7.
Note that the rule isnotharsh. The subject couldscore at most 9 credits. He is penalized only for the
*The ewmincrmustusehis jurlgment inassigning positional num.
hers h tbe word “the” in the fifth sentence of the test, if therenrc
fewer than three “the’s” in the response. Here it seems clear that thefirst “the” in the original sentence was omitted by the subject and that
the “tbc’s” in this response should reassigned thepositirmal num-
hcrsof9 and 12ratberthrm 5and9.
misspellings, the only effect of the rules about mis-placement being to avoid an overcredit for redundantcorrectly spelled words. Note also that a word is notboxed in its second appearance if it is not credited inthe first place.
Summary of Scoring Rules
(1)
(2)
(3)
(4)(5)
Score one point for each correctly spelled wordif the positional number isnotcircled or boxed.Circle any positional number which is greaterthan thepositional number which next appearson the right. Qnorea positional number whichis boxed.Box any positional number which has appearedearlier with a word which was credited. Donotbox a number if it was not credited in itsearlier appearance, or if it is a second or thirdappearance of the word “the” in the fifth sen-tence of the test.Do not penalize for punctuation.The score is the sum of all of the creditedwords.
The Cutting Score for Writing
The cutting score for writing is aet between 27 and28. Accordingly, a subject getting 27 is classed “illiter-ate”; a subject getting 28 is classed “literate.” Whileit is possible to attain the cutting score before the fivesentences are completed, no decision basedon shortenedrecords, analogous to the four-passage decision forreading, is suggested, for the savings in time would benegligible and the complexity of the scoring processwould place too great a burden on the examiner.
Relationships between Reading and Writing Reading theBest Single Index
The Brief Test of Literacy produces two scores,each yielding a judgment “literate.” Because these twoscores are not perfectly correlated, some subjects maybe judged “literate” by one test but not by the other. Ifit is necessary to determine the relationship betweenliteracy and some other variable, the conflict in thestatus of these cases must be resolved. On the basisof the available data and logical considerations as to thenature of the abilities, it is recommended that in anysuch cases the decision reached by means of the readingtest be considered final. Thus, in relating literacy to age,for example, a subject who was “illiterate” in the lightof the reading test, but “literate” in terms of the writingtest would be classed “illiterate” in assessing the re-lationship in question.
000
* u. S. GOVERNMENT PRINTING oFFIcE - !970 - =IS=43 P. Q. 31 29
VITAL AND HEALTH STATISTICS PUBLICATION SERIES
Originally Public Health Service Publication No. 1000
Series 1. Pvograms and collection procedures, — Reports which describe the general programs of the NationalCenter for Health Statistics and its offices and divisions, data colkction methods used, definitions,and other material necessary for understanding the data.
Series 2. Data evaluation and methods vesearch.— Studies of new statistical methodology including: experi-mental testa of new survey methods, studies of vital statistics collection methods, new analyticaltechniques, objective evaluations of reliahiliry of collected data, contributions to statistical theory.
Series 3. Analytical studies .-Reports presenting analytical or interpretive studies basedon vital and healthstatistics, carrying the analysis further than the expository types of reports in the other series.
‘Swies 4. Documents and committee vepovts.— Final reports of major committees concerned with vital andhealth statistics, and documents such as recommended model vital registration laws and revisedbirth and death certificates.
Serz”es 10. Data jrom the Health Interview Survev.— Statistics on illness, accidental injuries, disability, useof hospital, medical, dental, and other services, and”other health-related topics, based on datacollected in a continuing national household interview survey.
Sem”es 11. Dati from the Health Examination Survey. —Data horn direct examination, testing, and measure-ment of national samples of the civilian, noninstitutional populaticm provide the basis for two typesof reports: (1) estimates of ,the medically defined prevalence of specific diseases in the UnitedStates and the distributions of the population with respect to physical, physiological, and psycho-logical characteristics; and (2) analysis of relationships among the various measurements withoutreference to an explicit finite universe of persons.
Sm”es 12. Data j%om the Institutional Popuikation Surveys. —Statistics relating to the health characteristics ofpersons in institutions, and their medical, nursing, and personal csxe received, based on nationalsamples of establishments providing these services and samples of the residenta or patients.
Sm”es 13. Data J%om the Hospital Disciuzvge Suwey. —Statistics relating to discharged patients in short-stayhospitals, based on a sample of patient records in a national sample of hospitals.
Series 14. Data on health resources: manpower and facilities. —Statistics on the numbers, geographic distri-bution, and characteristics of health resources including physicians, dentists, n&ses, other healthoccupations, hospitals, nursing homes, and outpatient facilities.
Series 20. Data on mortality. -Various statistics on mortality other than as included in regular annual ormontmy reports—special analyses by cause of death, age, and ether demographic variables, alsogeographic and time series analyses.
Sw”es 21. Data on natality, mamvizge, and divorce. —Various statistics on natality, marriage, and divorceother than as included in regular annual or monthly reports-+pecial analyses by demographicvariables, also geographic and time series analyses, studies of fertility.
S--es 22. Data from the National Natality and Mortality Suvveys. — Statistics on characteristics of birthsand deaths not available from the vital records, based on sanqple surveys stemming from theserecords, including such topics as mortality by socimconomic class, hospital experience in thelast year of life, medical care during pregnancy, health insurance coverage, etc.
For a list of titles of reports published in these series, write to: Office of InformationNational Center for Health StatisticsPublic Health Service, HRARockvilie, Md. 20352
