G~.United Slatesl~~ Department of

Agriculture

NationalAgriculturalStatisticsService

Research Division

SRB Research ReportNumber SRB-96-03

November 1996

ReportingConsistency forWinter Wheat PlantedAcreage: AComparison of QASand FSA Data forKansas

Robert G. Hood

REPORTING CONSISTENCY FOR WINTER WHEAT PLANTED ACREAGE: ACOMPARISON OF QAS AND FSA DATA FOR KANSAS, by Robert G. Hood. ResearchDivision, National Agricultural Statistics Service, United States Department of Agriculture,Washington, D.C. 20250-2000, November 1996. Report No. SRB-96-03.

ABSTRACT

The National Agricultural Statistics Service (NASS) conducts Quarterly Agricultural Surveys(QAS) in order to gather information on a variety of commodities, including winter wheat. Datacollected for winter wheat seeded (planted) acreage during the December Agricultural Survey isan essential component in forecasting the total acres to be harvested the following summer.There has been concern expressed over the degree of reporting inconsistency and change insurvey indications for winter wheat planted acreage from December to March.

NASS has recently conducted research on the feasibility of using a Farm Service Agency (FSA)list of tracts as a sampling frame to produce state level estimates for crop acreage in Kansas.Data from this research were used in a subsequent reinterview study to try to determine possiblesources of reporting inconsistency. The levels of quarter-to-quarter reporting consistency andchange in total acres planted were also compared between QAS reports and reporting based onFSA tracts. The results indicate that while the percentage of inconsistent reports is notappreciably lower, the magnitude of the survey-to-survey changes for FSA reported data wasmuch lower than that for QAS data.

KEY WORDS

Farm Service Agency, Reinterview, Winter Wheat, Reporting Consistency

This paper was prepared for limited distribution to the research community outside the U.S.Department of Agriculture. The views expressed herein are not necessarily those ofNASS orUSDA.

ACKNOWLEDGMENTS

The author would like to thank the Kansas SSO staff and enumerators for their participation inthis project. Robert McEwen, Mark Pierzchala, and Roger Schou were invaluable in thedevelopment of the Blaise CAT! instrument used for data collection. Suzan Benz and JoeParsons were responsible for the initial work with the FSA sampling frame. Special thanks toSuzan Benz for providing the initial data necessary to conduct the reinterview, as well asproviding general information on FSA procedures.

TABLE OF CONTENTS

SUMMARy iii

INTRODUCTION 1

BACKGROUND 3

METHODOLOGY 4November FSA Survey Design 4April Reinterview Survey Design 4Data Collection 5Estimation 6

RESUL TS 8April FSA Reinterview Response Rates 8Total Winter Wheat Planted Acreage and Harvested-Planted Ratio 8Change in Level 9Comparison of Reporting Consistency Between QAS and FSA Data 10Reasons for Differences Between November and April Reported Acreage 12Net Effect of Change in Responses Between Surveys 13Reporting Unit and Respondent Type Effects 15

CONCLUSIONS ' 15

RECOMMENDATIONS , 18

REFERENCES 19

APPENDIX A: Formulas for Estimating Acreage Totals and Variance 20

APPENDIX B: Screens From CATI Reinterview Instrument 21

11

SUMMARY

The lack of reporting consistency from quarter-to-quarter is a concern with the winter wheatplanted acreage indications from the Quarterly Agricultural Surveys (QAS). For crop years 1994through 1996, reported winter wheat planted acreage in December and the following March weredifferent for more than half of the Kansas operations that were sampled both quarters. Thisseems to be a rather high percentage given that at the time the December Agricultural Survey(DAS) is conducted essentially all of the winter wheat crop has been seeded.

The high number of individual quarter-to-quarter reporting changes suggests response error.Response error can occur if there is concept or reporting unit confusion; if a respondent does notknow the answer, misunderstands the question, or deliberately gives false information; if exactquestionnaire wording is not followed; if data are recorded incorrectly; or if the questionnairedesign is flawed.

A follow-on survey to the November 1995 Farm Service Agency (FSA) study was conducted inApril 1996 in Kansas to evaluate the degree of reporting consistency for winter wheat plantedacreage that could be expected from the FSA sampling approach. This reinterview study wasconducted to determine whether sampling from a frame composed of FSA tracts would reducethe severity of this problem, and to identify possible reasons for inconsistency that could becontributing to the problem in the QAS. Kansas was chosen for the study because of itsimportance to winter wheat production and because of the SSQ's participation in past FSAresearch projects, not because the problems are any more severe in that State than in others.

Using FSA data is not new to NASS. FSA (formerly, the Agricultural Stabilization andConservation Service) data have been used since 1983 to update and build the NASS list frame.Several research projects have also been conducted in recent years on the possible uses of FSAdata. Weaver (1994) showed that FSA data could provide an alternate sampling frame and thatFSA land in farm coverage exceeded the NASS list sampling frame land in fann coverage (in theStates studied). Research by Parsons (1996) in Kansas and Nebraska showed that the coveragelevel of certified cropland by FSA data exceeded 97 percent. Parsons also examined thefeasibility of estimating the proportion of crop acreage that is covered in FSA totals using fannerreported data and used the information to produce planted acreage indications. Most recently,Benz (1996 ) showed that a sampling frame based on an FSA list of certified tracts could be usedto produce viable State level estimates for crop acreage in Kansas. The April 1996 reinterviewstudy conducted in Kansas was based on the data collected during the November 1995 study byBenz.

In April 1996 a brief CATI survey was conducted in Kansas in which operators from asubsample of tracts selected from the November 1995 FSA study were reinterviewed. Questionspertaining to winter wheat planted and harvested acreage were asked and data from Novemberwere used to validate the reinterview responses. When reinterview responses differed from

III

November responses by a certain percentage, the difference was reconciled and a reason for thedifference was obtained.

One potential cause of response error that is of particular interest is the different wording used incollecting winter wheat planted acreage for the December and March Agricultural Surveys(MAS). In December, the number of acres of winter wheat that have been seeded or will beseeded is asked, while in March the number of acres planted is asked. The November and AprilFSA surveys used the same questionnaire wording currently used for the December and Marchsurveys, respectively. In no case was a conceptual difference between planted and seededacreage used to explain a difference in reported acreage.

State level estimates for winter wheat planted acreage, harvested acreage, and the ratio ofharvested to planted acreage were generated. The coefficient of variation for planted acres was2.8, which was essentially the same as was obtained from the MAS with a much larger samplesize. The ratio of harvested acreage to planted acreage was comparable to the most current ratiowhich was set after the June 1996 acreage survey, but (due primarily to continued adverseweather between March and April) much lower than that indicated by the MAS. The winterwheat planted acreage was 0.5% lower than the list portion from the November 1995 FSAsurvey, compared to a 6.4% decline from December to March in the QAS.

The frequency of differences between November and April FSA reported acreage was higherthan expected, but still lower than that demonstrated by the DAS and MAS for the last three cropyears. However, the differences observed for FSA tracts were generally smaller both absolutelyand on a percentage basis than those from the QAS. The November-April FSA data hadconsiderably fewer large changes (greater than 25%) and more small changes (less than 5%) thanDecember-March matched records over the last three crop years.

As a result of the 1996 Federal Agriculture Improvement and Reform Act, it is unclear whatadministrative data FSA will collect in future years. NASS should keep apprised of anyrevisions in the legislation and changes in FSA data collection. Future research can beconsidered once it has been fully decided what data FSA will collect to administer the new FarmBill.

IV

INTRODUCTIONThe level of winter wheat planted (seeded)acreage from the December AgriculturalSurvey (DAS) and the instability ofsubsequent indications from the MarchAgricultural Survey (MAS) has been asource of recent concern at NASS. Anassociated perplexity is the degree ofreporting inconsistency for winter wheatplanted (seeded) acreage from December toMarch. For crop years 1994-1996, reportedwinter wheat planted acreages fromDecember and the following March weredifferent for more than half of the operationsthat were sampled both quarters.

These quarter-to-quarter changes indicateresponse error, perhaps due to concept orreporting unit confusion. Response errorscan occur when a respondent does not knowthe answer and/or guesses, misunderstandsthe question, or deliberately gives falseinformation. Response errors can also occurwhen the enumerator fails to ask a questionas written, records the data incorrectly, ormisunderstands the answer. Questionnairedesign can also be a contributing factor toresponse error. Potential for this exists inthe way winter wheat acreage is asked in theDAS and the MAS. In December, thenumber of acres of winter wheat that havebeen seeded or will be seeded is asked,while in March the number of acres plantedis asked. Isolating the reason( s) whyreported winter wheat acreage data changeso often between quarters is one of the goalsof this study.

Several questions related to these issuesarise. What is the source of the concern overthe December DAS acreage indication? Isthe level too high or too low? Is the levelfrom March consistent with that of

December? Why does reported winterwheat planted acreage vary from Decemberto March for the same operation? Why isdifferent wording ("seeded acres" inDecember vs." planted acres" in March)used in trying to obtain the same data? Isthe reporting inconsistency a problem, and ifso, how does it affect the survey indications?

To study these problems, particularly theissue of reporting consistency, the editeddata files of the DAS and MAS from Kansaswere reviewed for the past three crop years(1994-1996). List and area records weresubset to identify records sampled in bothDecember and March in order to identifydata discrepancies. Namely, matchedrecords whose March and December reportsof winter wheat acreage differed by morethat a specified percentage were examined.Differences in reported planted acreage fromDecember to March were discovered inover 55% of the matched records for the lastthree crop years. The magnitude of thesediscrepancies and the net effect areexamined.

Graphs were produced to show thefrequency of change from December toMarch in reported winter wheat acreages formatched list and area records. No consistenttrends between quarters, such as Marchconsistently being less than December, wereevident. Figure 1 illustrates the changes inindividual firms' reported acreage fromDecember to March for crop years 1995 and1996.

To determine possible sources of reportinginconsistency for winter wheat plantedacreage, a reinterview study was conductedin Kansas in April 1996. One goal was toevaluate the level of reporting consistency

KANSAS WINTER WHEAT PLANTED ACRES

4 Records with Mar <> Dec106 Tolal Matched Record~ _

-----------------------

Dec1995 - Mar 1996 List R_:~~ I!

200

3200

6400

oDEC

: Quarter

R~e~co=rd=s)~ _532 T_otallllat~e_~R~e~co=rd=s)~ _

PIant

Ied

Acr

Ies

!-10000

- c,,~:::~ ~~~=-----

Quarter

-----------

1400 ~

1800 '

PIan

s

I Dec 1995 - Mar 1996 Area Records2000 '-----

----------- -----------------------Dec 1994 - Mar 1995 Area Records

2600 1

2400, -----

Dee 1994 - Mar 1995 List Records6400 - --

MAR

i

- -I' ,

--=1

0',DEC

i;s.; _200 4 __ ",,_~

400

-"":0- __

BOO-'~

~:.....-~-

PI

laI ~,ed

3200 -

I~Ie

~ II ,$

! I

~I'MARl

-Quarter

2000 -

1800 1-'~,1600 ,~ __

"1400,_~,i ---------:::-,

1200,-_

1000 ' ------------ _ ~ -"-

BOO~-

600

200 Ino jDEC

400

PIa

Quarter6. Recorda wtth Mar <> Dee 122 Records Differing by at L•• st 25%It24 Total Matchod Rocord.L ~! (674 Tota~oteh~_R~o~c~O~rd=.)~ _____._J

2200 -

Figure 1. December Winter Wheat Seeded Acres vs March Planted Acres

2

for winter wheat planted acreage that couldbe expected from sampling from a frame ofFarm Service Agency (FSA) tracts. Ifquarter-to-quarter reporting inconsistency isdue to reporting unit confusion, then usingFSA tracts as a sampling frame couldpossibly reduce the problem (assuming theFSA tract is a more stable reporting unitthan entire farm operation). A secondaryobjective was to determine whether thedifference in question wording betweenDecember and March in the QuarterlyAgricultural Surveys (QAS) could becontributing to response variability. Kansaswas selected for this study because of itsimportance to winter wheat production andbecause of the previous work by Parsons(1996) and Benz (1996) on FSA coverageand the evaluation of using FSA tracts as asampling frame for an acreage survey.

A subsample of FSA tracts was selectedfrom the November 1995 survey that wasconducted to evaluate the feasibility of usingFSA tracts as a sampling frame. Operatorsof these tracts were telephoned in April as afollow-up to verify the previously reporteddata and to collect current and up-to-datedata. The follow-up study was designed tohelp determine whether sampling FSA tractswould result in more consistent responsesfrom quarter-to-quarter and whether thedifference in question wording could becontributing to response variability. Thereinterview study also provided anotherindication of total winter wheat plantedacreage, as well as an indication of thepercentage of planted acres that wereexpected to be harvested.

BACKGROUNDThe Sampling and Estimation ResearchSection of the Research Division and the

3

Kansas SSO conducted a study in November1995 to evaluate the benefits of using analternative sampling scheme for the acreagesurveys. This approach utilized a samplingframe composed of Farm Service Agencytracts. One potential advantage of theapproach is the stability of the FSA tract,which is a fixed area of land, regardless ofwho operates it. Reporting for FSA tracts isperceived to be more stable than for entirefarm acreage operated by a particular firm,which is the unit for QAS reporting. FSAtracts are fixed pieces of land, smaller onaverage than entire farm acreages, and havea detailed "legal description" making theexact acreage in question easy to identify tothe respondent. If a substantial portion ofthe quarter-to-quarter reportinginconsistency observed in the QAS isattributable to reporting unit confusion orinstability, then this problem could besignificantly reduced by using the FSAframe.

The FSA has historically maintained a list offarms and tracts in order to implement andadminister farm programs. Farm operatorshave reported to FSA throughout the yearfor various purposes, including the farmprogram sign-up and the certification of cropacreage. During the farm program sign-up,farm operators would sign contracts with theCommodity Credit Corporation agreeing toabide by the rules of the current farmprogram. All operators that were in farmprograms would identify crops and acreageby field for an FSA farm, including non-program crops. This "certification" ofacreage was done at the county FSA office.Other farm operators who were not in farmprograms could also certify their acreage inorder to maintain their crop acreage base andfor crop insurance purposes (Parsons 1996).

FSA farms are composed of tracts. An FSAtract is a contiguous piece of land under oneownership and operated as a farm or part ofa farm. The sample for the Novembersurvey was selected from a list of FSAcertified tracts in Kansas. Preparing thesampling frame for use involved severalsteps. First, data for fields that werecertified with FSA for the 1994 crop yearwere obtained from local FSA offices.These field level data were then aggregatedto the tract level to form the sampling frame.Next, the sample was stratified on totalcropland and a wheat presence indicator(based on the sum of wheat, summer fallow,and fallow in the tract) using certificationdata for the 1994 crop year. Finally,additional strata were created for tracts withland only in the Conservation ReserveProgram (CRP) and for tracts that had over1,000 acres of cropland (Benz 1996).

Using such a list of certified FSA tracts as a"list frame" should not present too muchconcern in terms of coverage for Kansas.Weaver (1994) showed that FSA land infarm coverage exceeded the NASS listsampling frame land in farm coverage (inthe States that were evaluated) and that FSAcoverage of land in farms in Kansasexceeded 99%. Coverage levels exceeded98% for the farm program crops of com,wheat, and sorghum in Kansas. Parsons(1996) showed that for crop year 1995,almost 99% of all winter wheat acres foundin Kansas were FSA certified. While thisproject dealt with a list of certified tractsfrom 1994 FSA data, there is no reason tosuspect that 1994 coverage would be muchdifferent than 1995. However, to accountfor any incompleteness, a small sample wasselected for the November study from an

4

"area frame" of cropland areas believed tobe non-certified for the 1994 crop year.

METHODOLOGY

November FSA Survey DesignFor the November study a total of925 FSAtracts were sampled from a list framecomposed of 209,080 FSA tracts in Kansas.In addition, a sample of 43 land areas werechosen from an area frame of non-certifiedland. This "area frame" consisted of 77areas that were split into two strata based onexpanded cropland in the land areas. Sincethese area samples were merely areas of landdelineated on a map and had no descriptors,they were field enumerated. Enumeratorswere provided a state map and a larger scalecounty map to aid in locating the samples.The area frame survey contained a questionto determine if the land area was certifiedwith FSA for the 1994 crop year. This wasan additional attempt to verify that the landarea was truly not certified in 1994 (as hadbeen reported in the June area survey) andwas therefore non-overlap (NOL) with thelist sample frame of certified tracts (Benz1996), Data for several items of interest,including winter wheat planted acres, werecollected. See the section on Estimation fora complete description of the sample design.

April Reinterview Survey DesignA subsample of FSA tracts from theNovember list sample was selected forreinterview in April. NOL samples fromNovember were not selected for tworeasons. First, they contributed very little(less than 1%) to the total expanded winterwheat planted acreage. Second, reinter-viewing the NOL samples would haverequired personal enumeration, becausethere was no easy way to identify these

parcels of land to the respondent. Since theyhad not been certified, no FSA farm or tractmunbers existed for them.

The goal of the April project was torecontact as many of the tracts with usableNovember data as possible in order to testreporting consistency for FSA tracts. Statelevel estimates for total winter wheat plantedacres and a ratio of harvested to plantedacreage were also produced.

The April reinterview sample was takenfrom the 723 usable list reports from theNovember survey. In order to minimizerespondent burden, any operation amongthese 723 that could be identified as being inthe 1996 March Ag Survey sample wasexcluded from the reinterview sample. Thisreduced the possible sample by 66 (22 fromthe 196 records reporting zero planted acresin November, and 44 from the 527 recordsreporting positive planted acreage inNovember). All of the remaining tracts thatreported wheat plantings in November andabout half of those reporting zero wheatacreage were selected for the Aprilreinterview. Therefore, the final sample forApril consisted of 583 tracts, 100 thatreported zero planted acres in November and483 that reported positive planted acres inNovember.

Data CollectionThe sample for the November survey wasdrawn from a list of 1994 certified tracts andincluded names that were no longeroperating the tracts. The April sample,however, consisted of only operations thatwere able to report data in the November1995 survey. Operator name, address, andphone number updates were made to severalof these November records in order to make

5

it easier to contact the operator of thespecified FSA tract.

If the respondent no longer operated theFSA tract, but could report for it, the Aprilinterview was conducted. If the farmer nolonger operated the FSA tract and could notreport for it, or if the tract had been split, wedid not require the enumerator to follow upon the new operator(s). In this case, theinterview was concluded by entering 'NO',and the questionnaire was coded as beinginaccessible. This situation occurred rarely,and following up these cases would haveincreased the complexity and cost of thestudy while yielding little insight into thereasons for the reporting variability.

With respect to winter wheat plantedacreage, the wording of the November FSASurvey paralleled the DAS and the wordingof the April FSA Reinterview survey, theMAS. In the November FSA Survey thewinter wheat question was asked thefollowing way: Did you seed, or will youseed, any winter wheat for all purposes inFSA tract XXXX for the 1996 Crop Year?(If yes), How many acres of winter wheathave been seeded or will be seeded for allpurposes for the 1996 Crop Year? The DASwinter wheat acreage question was asked inthe following way: Please report winterwheat (and rye) seedings for the 1996 CropYear. Winter wheat acres seeded and to beseeded for all purposes.

The April FSA Reconciliation followed thewording of the MAS. The MAS winterwheat acreage was asked in the followingway: For winter wheat (and oats), pleasereport acres planted (and to be planted) forall purposes last fall (or this spring), andacres to be harvested for grain for the 1996

Crop Year. The April FSA Reconciliationasked winter wheat the following way: Howmany acres of winter wheat were planted forall purposes last fall for the 1996 Crop Yearin FSA Tract XXXX. Both surveyinstruments also contained a question onwinter wheat acreage to be harvested forgram.

To collect data for ApriL a short CATIinterview was conducted. Survey QualityResearch Section (SQRS) and TechnologyResearch Section (TRS) of ResearchDivision developed the reinterviewinstrument using Blaise software. (SeeAppendix B for screen captures of the BlaiseCATI instrument.) The CATI instrumentused data from the November FSA TractAcreage Project to validate the currentsurvey response for winter wheat plantedacreage. When the November and Aprilresponses differed by more than a certainpercentage, the April response was flagged,and the difference was reconciled,capturing the correct response and thereason for the difference.

The rules for flagging differences varied bythe number of reported acres of wheatplanted. This was done to avoid flaggingsmall changes for small acreages andreduce respondent burden. The consistencychecks (shown in Table 1) were based on

Table 1.FSA Reinterview Consistency Checks

April Acreage Response Flag Condition

Zero IfNov > 0

Positive IfNov = 0

> 500 If Apr/Nov > 1.10 or < 0.90

30 - 499 If Apr/Nov > 1.15 or < 0.85

<= 30 If Apr/Nov > 1.25 or < 0.75

6

the ratio of April acres to November acresand used ranges that were chosen afteranalyzing the distribution of the Novemberwinter wheat data and consulting with theKS SSO.

EstimationThe November FSA sample consisted of968 sample units, 925 list and 48 NOL.Nine list strata were created based on totalcropland and the presence of wheat. Table2 (Benz 1996) gives a breakdown of thestrata counts used in November.

Recall the sampling scheme for April, inwhich FSA tracts were selected from usablereports from the November FSA study.NOL tracts and any operations that could beidentified as being in the MAS wereexcluded from the April reinterview sample.All positive usables from November wereselected for reinterview (excluding 66observations that were also selected for theMAS). One hundred tracts were randomlyselected from the 196 tracts with zero winterwheat planted acres in November. Thisproduced a sample size of 583 for April,with 483 tracts that had reported positiveplanted acreage and 100 tracts that hadreported zero planted acres.

Tracts reporting zero planted acreage inNovember were sampled at a different ratethan those reporting positive acreage.Therefore, a tenth stratum was created torepresent the zero acreage tracts in the frameand the other nine strata were adjusted torepresent only positive acreage tracts. Sincethe population counts for the restratificationwere unknown, they had to be estimated.Using the original population counts for thenine strata and the proportion of zero andpositive reports from the November FSA

Table 2.Sample Design of November FSA Tract Acreage Study

Pop UsableStratum Description Count Sample Usable Positive

1 CRP onlywith< 1000 acrescroplandI 13,986 25 21 I

2 0.1 - 100 acres 'cropland'withnowheatindicator 33,419 100 81 233 100-400 acres 'cropland'withnowheatindicator 12,198 50 39 124 400-1000 acres'cropland'withnowheatindicator 304 25 21 35 0.1 - 100 acres 'cropland'withwheatindicator 56,503 100 80 626 100 - 200 acres'cropland'withwheatindicator 62,719 275 217 1837 200 - 400 acres'cropland'withwheatindicator 23,171 200 161 1478 400-1000 acres'cropland'withwheatindicator 6,327 100 71 679 1000+ acres'cropland' 453 50 32 29

NOL (Not sampled in April reinterview) 43 43 12TOTAL 209,080 968 766 539

1/ 'cropland'is thesumofthe certifiedacresinthetract2/ thewheatindicatorisbasedonthesumofwheat,summerfallow,fallowinthetract

survey, strata population counts wereestimated for the ten strata. The estimatedpopulation count for the new tenth stratumwas the sum of the estimated number of zeroacreage units in the original nine strata.

Specifically, the adjusted strata populationcounts for strata 1 through 9 were estimatedby multiplying the original populationcounts by the proportion of tracts withpositive planted acres in November. Theremaining tracts with no planted acres werethen grouped together in a new tenthstratum. Since there were usable zeroacreage reports in all of the original ninestrata, we can treat the 100 zero reports as asimple random sample from the newlycreated stratum 10 and use the same basicexpansion formula for all strata. Table 3shows the November strata counts, the

7

proportion of usable positive reports, and theApril adjusted stratum population counts.

The expansion factor, adjusting fornonresponse in the reinterview survey, wascalculated as the April adjusted stratum levelpopulation counts divided by the number ofusable reports in that stratum. For the ratioof harvested to planted acres, the expansionfactor was based on the number of reportsthat were usable for both items in April.That is, both harvested acreage and plantedacreage had to be usable for the report to beused in calculating the ratio. The varianceswere estimated using bootstrap estimation.See Appendix A for a discussion of theestimators used in calculating acreage totalsand variance.

Table 3.Adjusted Strata Counts for April

Proportionof Nov. April

November Usable AdjustedPopulation Positive Population

Stratum Count Reports Count

I 13,986 1/21 666

2 33,419 23/81 9,489

3 12,198 12/39 3,753

4 304 3/21 43

5 56,503 62/80 43,790

6 62,719 183/217 52,892

7 23,171 147/161 21, 156

8 6,327 67171 5,971

9 453 29/32 411

10 70,909

Total 209,080 209,080

RESULTS

April FSA Reinterview Response RatesThe 583 tracts from the November FSAstudy selected for the following Aprilreinterview study yielded 548 usable reports,with 452 positive usables. There were 23refusals and inaccessibles and 12 cases inwhich the respondent was unable to reportfor the tract. Ninety-two of the hundredtracts with zero planted acres in Novemberwere usable, with three reporting positivewinter wheat acreage in April. Note that theApril reinterview survey excludedinaccessible reports and refusals from theNovember survey, which no doubtcontributed to the high rate of usability inApril.

There were 497 usable reports for winterwheat harvested acres, with 344 usable

8

pOSItlves. There were 57 which reportedzero h,rrvested acres and 51 which reported"Don't Know", all of which had plantedacres. The average number of planted acresfor these 51 "Don't Know" reports was 122.By comparison, the average number ofplanted acres for the 344 tracts whichreported positive harvested acres was 144.The overall average harvested acreage for all497 usable reports was 114 acres. For the57 reports of zero harvested acres andpositive planted acres, the average numberof planted acres was 122.

Only two percent of the April samplerespondents were unable to report for FSAtracts. In November, less than two percentof sampled operators were unable torecognize the specific FSA tract requested(Benz 1996). This lends support to thebelief that FSA tracts may be a viablereporting unit for use with NASS surveys, atleast in Kansas.

Total Winter Wheat Planted Acreageand Harvested-Planted RatioThe State level indication for expandedwinter wheat planted acres for the Aprilstudy was 11,356,797 which wascomparable to the level from the MAS. Thebootstrap estimated coefficient of variationof 2.8 was the same as that obtained fromthe MAS, which had five times the samplesize. Thus, using a small sample of datafrom FSA tracts, we were able to produce anestimate that was comparable in both leveland precision to the MAS. (In all fairness, itmust be noted that the November FSAsample, and thus the April reinterviewsample, was stratified particularly for winterwheat, while the MAS is a multi-purposesurvey also targeting commodities otherthan winter wheat.) A comparable ratio of

harvested/planted acres was computed fortracts with usable planted and harvestedacreages, with the following results.

current forecasts) of harvested acreage thanthe MAS.

About 91% of the expanded planted acreagewas covered by tracts reporting harvestedacres. The ratio of harvested to plantedwinter wheat acreage from the April FSAsurvey (0.77) was much closer to the currentBoard estimate of the harvested to plantedacreage ratio of 0.82 than to the 0.95 ratioobtained from the MAS. The difference inlevel of the March and April ratios largelyreflected deteriorating expectations for thewinter wheat crop. The weather conditionsin Kansas as well as other Plains States weresevere during the winter and spring. Coldtemperatures and drought had a tremendousimpact on the wheat that was planted and theamount farmers expected to harvest. Kansasexperienced dry conditions from the time ofseeding, resulting in poor emergence androot development, leaving wheat cropssusceptible to wind damage and freezing(NASS 1996). There was still someoptimism for the crop during the time periodthat the March Ag Survey was beingconducted. However, an early April freezeand spring drought created further problemsand the wheat crop had deteriorated by thetime the reinterview survey was conducted.This is reflected in the number of acresfarmers expected to harvest. The forecastfor winter wheat harvested acreage wasrevised to 8.8 million acres after the Juneacreage survey, resulting in a harvested toplanted ratio of 0.82. The April FSA surveyprovided a better indication (based on

Planted Acres:Harvested Acres:HIP:Std. Err.

10,318,0997,928,877

0.770.025

9

The April study results for wheat plantedacreage compared favorably to theNovember FSA list results, with only aslight decrease (0.5%) in reported acreagefrom November to April. The change fromthe December to March Ag Surveys wasmuch higher (but within the bounds ofsampling variability) with a decrease of6.4%. The reason for the big decline seemsto be largely attributable to the samplerotation in the list. The ratio to previous forMarch list units that matched December listunits was 1.003.

Change in LevelQuestions about the current level of totalwinter wheat planted acreage and about theinstability of the indication from Decemberto March have been noted in varioussources. The April reinterview study wasnot designed specifically to address the issueof level, but relative information may begleaned from the results, at least for Kansas.

Generally, indications derived fromsumming up FSA data are thought toprovide a lower bound to NASS acreageindications, since FSA data are expected tobe somewhat incomplete. In this study, theNovember and April FSA indications werequite close to the DAS indication for winterwheat acreage, and actually higher (thoughnot significantly so) than the subsequentMarch indication. This reinterview studytherefore tended to support the DAS level ofplanted acres in Kansas. Until this cropyear, there had not been any exceptionallylarge changes from December to March inKansas in recent years. Table 4 shows thechanges from December to March for the

% Change = (Mar· Dee )/Dee • I00%

Table 4.Percent Change: A Comparison of QASIndications for Winter Wheat PlantedAcres by Crop Year

Crop Year

Frame 1996 1995 1994 1993LIST -6.2 -1.1 1.4 -3.0NOL -7.5 -7.2 7.8 -2.6TOTAL -6.4 -~.2 2.5 -2.9

last four crop years. Changes fromDecember to March have been less than 3%,except for the current crop year which had achange of 6.4%.

While there was almost no change in theindication for the total winter wheat plantedacreage from November to April (-0.5%)compared to a substantial change fromDecember to March (-6.4%), several factsmust be noted. The FSA surveys involvedthe same people for both surveys, unlike theQAS surveys in which roughly only 40% ofthe samples were in both surveys. Bailey(1994) noted that QAS indications decline inquarters following the June Survey and thatthe data adjustment factor (DAF) is a majorfactor in the decline. He also noted thatquarter-to-quarter changes in the DAF havehad a much larger effect on surveyexpansions than changes in the data.

Comparison of Reporting ConsistencyBetween QAS and FSA DataIn order to study the issue of consistencyand stability in QAS data for winter wheatplanted acres, the edited files of the DASand MAS from Kansas were reviewed forthe last three crop years (1994-1996). Listand area records that were sampled both

10

quarters were used to identify datadiscrepancies. Only records with usablereported data for both quarters were used inthese comparisons. Records whose datawere imputed or estimated (including knownzeros) were not used as that alone couldaccount for differences (or artificially thelack thereof). Valid zeros from reporteddata. including records that reported beingout of business in March, were included, butknO\\-TIzeros were excluded. Reportingconsistency in QAS winter wheat plantedacreage for these three crop years wascompared to reporting consistency betweenthe November and April surveys ofFSAtracts. Responses were compared foroperations that were in both the DAS andMAS (for crop years 1994-1996) and thathad usable reported winter wheat plantedacreage.

For crop years 1994-1996, the percentage ofmatched DAS and MAS records that haddifferent responses for winter wheat plantedacres was between 56-60%. The percentageof matched area records with differentresponses between December and March forthese three crop years ranged from 51-56%.A similar rate of differences was observedfor matched list records, where the rangewas 57-60%. Matched area recordsaccounted for 12-16% of all matched recordsfor these three years.

Table 6 compares the rates of consistencyfor the total QAS sample (combined list andNOL) for individual crop years versus thatof the FSA studies. From this table, theredoes not appear to be much difference. Onlyabout 40-46% of the matched recordsreported exactly the same for either FSA orQAS surveys. The percentage of recordsreporting exactly the same data from

Table 6.Comparison of Reporting Consistency for Winter Wheat Planted Acreage

Nov-Apr FSA Surveys vs. Dee-Mar Quarterly Ag Surveys

I I Nov95-Apr96 I Dec95-Mar96 I Dec94-Mar95 I Dec93-Mar94 IReporting Consistency No. % No. % No. % No. %

No Change 251 45.8 303 41.1 352 44.1 317 40.3

Zero-Zero 89 16.2 162 22.0 190 23.8 154 19.6

Pas-Pas 162 29.6 141 19.1 162 20.3 163 20.7............................................. .................. .................. .................. .................. ..................... .................. ................. -. ...................Change 297 54.2 435 58.9 446 55.9 470 59.7

Zero-Pas 3 0.5 16 2.2 12 1.5 17 2.2

Pas-Zero 7 1.3 23 3.1 20 2.5 17 2.2

Pas-Pas 287 52.4 396 53.7 414 52.0 436 55.4~~ _________~_u __~_--"" __ 'Y ••• _~ __ •••••• ,.. ---........... ...-- .......- _.....--_ •••••••••••• _____ ••••••• \11 ,.._--... __ ••••.••• Ir.& ___ ., •••••••.•

Matched Records 548 738 798 787

November to April for FSA tracts is onlyslightly higher than the percentage formatched records for the DAS and MAS forcrop years 1994-1996.

Table 7.

However, if we consider consistency interms of percent change from previousquarter, reporting for FSA tracts appears tobe substantially more consistent than thatfor QAS. Table 7 shows a finer breakdown

Comparison of Percent Change for Winter Wheat Planted AcreageFor Matched Records With a Change in Response

(Positive Data Both Quarters as a Percentage of All Matched Records)Nov95-Apr96 Dec95-Mar96 Dec94-Mar95 Dec93-Mar94

Change (C) No. %1 No. % No. % No. %

C~ 5% 156 28.5 139 18.9 134 16.8 129 16.45% -<C ~ 10% 53 9.7 79 10.7 76 9.5 99 12.610%-< C ~ 15% 23 4.2 38 5.1 43 5.4 64 8.115%-< C ~ 20% 16 2.9 40 5.4 36 4.5 41 5.220%-< C ~ 25% 7 1.3 13 1.8 23 2.9 17 2.2C >- 25% 2 32 5.8 87 11.8 102 12.8 86 11.0

1/ Percentages based Ontotal number of matched records2/ Change calculated as absolute value of {(Dec - Mar)/Decl • 100%

11

for matched positive records that haddifferent responses across surveys. Whenresponses for positive planted acreage didchange, fewer large changes (as a percent ofinitially reported data) and more smallerchanges occurred with FSA tract data thanwith QAS entire farm operation data. So,while the percentage of records changingresponse was only slightly lower with theFSA frame, the changes observed were, inrelative terms, much smaller than thoseobserved in the QAS.

Figure 2 is a graphical comparison of thefrequency and magnitude of changes inreported winter wheat planted acreagebetween the (November and April) FSAsurveys and matched records from the DASand MAS (for the last three crop years).

Reasons for Differences BetweenNovember and April Reported AcreageEdit checks were built into the datacollection instrument to flag cases in whichthe difference between the November andfinal April responses was outside a specifiedrange. When a response was flagged, awarning message for the enumerator wasdisplayed on the screen with instructions toresolve the difference with the help oftherespondent. The respondent was given boththe original November response and thecurrent April response to help reconcile thedifference between responses. Therespondent was then given the opportunityto verify the current response or change hisresponse.

FSA vs QAS: Winter Wheat Planted Acres

Change in Reported Acreage

Survey

FSA

1996

1995

1994

No Wheat

Both Surveys

No Difference Pos Both Surveys

Pos Both Surveys Change < = 5 %Pos Both SurveysChange > 25 %

o 10 20 30 0 10 20 30 0 10 20 30 0 10 20 30

Percentage of Matched Records Falling Into Category

Figure 2. Quarter-to-Quarter Change in Reported Planted Acreage: FSA vs QAS(December - March) for Last Three Crop Years

12

When a response was flagged, therespondent was asked to give a reason, ifpossible for the difference between theNovember and April responses. If therespondent was unable to provide a reason,the enumerator was then allowed to promptthe respondent with the following possiblereasons:

a) The wrong tract level data wasrecorded in November

b) A change was made to the landagreement of the tract involvingrented acres of winter wheat

c) One or more fields of winterwheat in the tract were excludedin November

d) Reported fann level datae) Acres seeded (as asked in

November and for DAS) has adifferent meaning than acresplanted (as asked in April and forMAS)

f) Included only acres to beharvested. Excluded acres forhay, grazing, etc ...

g) Planting intentions changedh) Other explanation

Although the April reported winter wheatplanted acreage differed from the Novemberresponse for seeded acres in over half of thematched usable reports, only 11% (62) ofthe differences were large enough to beflagged. In two of the flagged cases, therespondent changed the April response tomatch the November acreage. Seventeen ofthe remaining 60 could not or would notgive a reason. Reasons cited for the other 43records, the majority of which involved tractreporting problems (either reporting for the

13

wrong tract or excluding fields from thetract) are indicated in Table 8.

The fact that no differences were due to thewording of the question suggests that theremay be no conceptual difference between"seeded" and "planted" acres, at least inKansas. However, consistent wordingshould be used whenever possible. Anotherreason that was not reported was "includingonly acres to be harvested." It has beensuggested that total wheat planted acreage islow because people do not report wheat forgrazing. This appeared to be a problem foronly one sample.

Net Effect of Change in ResponsesBetween SurveysTo examine the net effect of changes inreported planted acreage of winter wheatbetween surveys, the data were grouped bythe "Change-No Change" categories andexpanded. The numbers in Table 9 werederived from the matched reports from theNovember and April FSA surveys. Thepercent change groups (left most column ofthe table) are based on reported data. Thegrouped November and April data wereexpanded based on the same expansionfactor (April's) to eliminate any differencesdue to weighting and survey design. Thedifference between expanded April andNovember data was then calculated todetermine each group's proportion of thetotal percent change from November. Thegoal of this analysis was to see if any groupcontributed more to the overall change in theexpansion. Note that the total expandedplanted acres for November is not theindication from the November FSA survey,since it only represents matched recordswhich were weighted by the April expansionfactors.

Table 8.Reasons for Differences between November and April Responses

IReason I Number IThe wrong tract level data was recorded in November 19

Planting intentions changed 9

One or more fields of wheat were excluded in November 9

Included only acres to be harvested, excluded acres for hay, grazing, etc 1

Estimated acreage in November 1

Forgot some acres in summer fallow I

Tore up some acreage 1

This analysis (based only on matchedrecords) indicates a 1.5 percent increasefrom November, contrary to the full directcomparison of November and Aprilexpanded data which indicated a 0.5 percentdecrease. This difference, which is withinthe range of sampling variability, wouldappear to be due to the change in expansionfactors from November to April. Thepositive-zero, zero-positive, and positive-positive categories all showed relatively thesame amount of change, with the positive tozero group showing a decrease fromNovember. The group contributing themost, in terms of absolute change, in thepositive-positive category was the "> 25%"group. However, this group onlycontributed about 6% of the (November)total planted acreage as compared to 40%for the "< 5%" group.

14

Since the change from November was smallto begin with, the above analysis was unableto provide much insight into the factors ortype of reports contributing to the difference.Similar analyses were performed formatched DAS and MAS records for cropyears 1994-1996. While matched responseswere possible for all of the FSA tracts, thiswas not the case for QAS samples. Due tothe rotation of replicates and other factors,less than 30 percent of the sample units inboth the DAS and MAS were matched andusable (reported, non-estimated) for winterwheat planted acres. These matched recordsaccounted for roughly 40 percent of the totalplanted acreage. Results on matched DASand MAS reports were inconclusive and noobvious trends were discovered.

Table 9.Proportion of Total Percent Change of Expanded Planted Acres

Based on Grouping Matched Records by Percent Change of Reported AcresChange as a

Matched Nov Exp April Exp Percentage ofRecords Acres Acres Nov Total

No Change 251 3,495,055 3,495,055 0

Zero-Zero 89 0 0 0

Pos-Pos 162 3,495,055 3,495,055 0

Change 297 7,692,029 7,861,742 1.52

Pos-Zero 3 159,383 0 -1.42

Zero- Pos 7 0 167,022 1.49

Pos- Pos 287 7,532,646 7,694,720 1.45-------- ------ ------ ------ ------%Cd 156 4,503,157 4,528,938 0.23

5~%C<10 53 1,406,608 1,396,034 -0.09

10~%C<15 23 459,289 451,298 -0.07

15~%C<20 16 330,173 327,312 -0.03

20~%C<25 7 158,305 161,518 0.03

%C > 25 32 675,113 829,621 1.38

ITotal 1548 I 11,187,084 I 11,356,797 I 1.52 IReporting Unit and RespondentType EffectsNext, the reporting unit and respondenttypes for matched DAS and MAS recordswere analyzed by the above categories andpercent change groups. Nothing unusualstood out. Table 10 shows that for crop year1996, 85% of the matched, positive acreagereports showing differences involved thesame reporting unit. Also, 80% of thereports with differences were for records inwhich the respondent type was the same.The positive to zero category contains 11reports that reported positive data inDecember prior to being coded as "out ofbusiness" in March. Roughly 90% of thezero to positive changes involved the same

15

reporting units and respondent types. Sincethey are relatively infrequent, changes inreporting unit and respondent type do notappear to be the driving force in differencesbetween quarters.

CONCLUSIONS

The goal of this project was to study theproblem of quarter-to-quarter reportinginconsistency for winter wheat plantedacreage. Additional goals includedproducing State level indications for totalwinter wheat acreage planted and for theharvested to planted ratio. We also hoped togain insight on whether questionnairewording ("seeded" vs. "planted") or

Table 10.Percent of Records with Same Reporting Unit and Respondent Type

Matched DAS and QAS Records: Crop Years 1995 and 1996Dec95 - Mar96 Dec94-Mar95

% With Matching % With Matching % With Matching % With MatchingReporting Unit Respondent Type Reporting Unit Respondent Type

No Change 76 76 78 75Zero-Zero 64 64 70 65

Pos-Pos 89 91 86 86

Change 82 80 80 81Zero-Pos 88 94 58 50

Pos-Zero 22 65 25 70

Pos-Pos 85 80 83 82......................... - .......... ................. - .................. .................................... ..... -- ............................. ...................................%C~5 80 78 89 865~%C<10 91 85 84 8810~%C<15 87 79 84 8415~%C<20 90 80 78 6920~%C<25 85 92 78 78%C > 25 85 78 78 77

Total 79 78 79 78

reporting unit (FSA tract vs entire farmoperation) plays a role in reportingconsistency. Kansas was selected for thisstudy because of its importance in winterwheat production and its participation in the1995 FSA Coverage Research Project andthe November 1995 study on the use of FSAtracts as a sampling frame for an acreagesurvey. The general implications of using aframe based on FSA tracts are not addressedhere. Parsons (1996) and Benz (1996)discuss the advantages and disadvantages ofusing FSA data in their recent researchreports.

16

There does appear to be considerablereporting inconsistency in QAS data forwinter wheat acreage from December toMarch. There is also some quarter-to-quarter instability in the indication. Thisstudy has shown that reporting based onFSA tracts from a frame of certified FSAtracts has potential to both reduce thenumber and size of inconsistencies, andproduce State level indications for totalplanted acreage comparable to QASindications with a fraction of the samplesize. However, it must be noted that not allstates have the high coverage of FSA land in

farms that made Kansas a prime choice forthe recent studies concerning FSA data.

Although the results were not over-whelming, they were encouraging. Withonly one study for comparison, it is difficultto say if more stable acreage estimates couldbe produced using a frame of FSA tracts, butthe FSA approach deserves additionalconsideration. Even though the pointestimate for acreage planted was notsignificantly different for QAS and FSAdata, reducing the reporting variability isadvantageous in reducing our estimated"sampling variability". Further researchshould be conducted before making anydefinite conclusions about using thisapproach and especially before trying toextend these results to States other thanKansas where coverage levels are not ashigh.

A new farm bill was recently implementedthat affects the type of data that will beavailable in the future, with FSA data likelynot being as complete. The current farm billwill be in effect for the next seven years, butcould be revised. Benz (1996)recommended that if, in the future, FSA databecome available again, another FSA tractwheat acreage survey be conducted, perhapsexpanding to other states such as Oklahomaand Nebraska. If this should transpire, thena reinterview study such as this one wouldbe a logical follow up.

Reporting was more consistent for winterwheat planted acreage between theNovember and April FSA surveys thanbetween December and March QASsurveys. Matched records for FSA tractsshowed almost the same rate of reportingchanges from November to April as did

17

matched records from December to Marchfor QAS for the last three crop years.However, the magnitude of these changes,measured as a percentage change fromprevious survey, was much smaller. Reportsfor FSA tracts exhibited fewer large survey-to-survey changes and a higher frequency ofsmall survey-to-survey changes thancomparable DAS and MAS reporting. Thiscould be due in part to a more stablereporting unit, FSA tract versus entire farm.

Unfortunately, not much insight on whydifferences occurred was gained through thereasons provided during the reconciliationprocess. Perhaps this was because too fewdifferences were actually flagged, but it isquestionable whether more flags would haveresulted in more meaningful reasons. Ingeneral, I have observed from severalreinterview studies that during thereconciliation process, when responses arecompared and a difference occurs, that thereasons recorded are generally vague. Thiscould be due to several reasons. Therespondent may not want to take the time todetermine the cause and just say "I don'tknow why", different people could haveresponded for the two surveys, it is easy tosay that the previous number was recordedor reported incorrectly, or proper probingmay not have been carried out by theenumerator.

Indications comparable to QAS resultswere produced for winter wheat plantedand harvested acreage. Both theNovember and April indications based onFSA tracts were comparable to DAS andMAS indications. The FSA surveys, withonly a fraction of the sample size of theDAS and MAS, produced estimates for totalplanted acres that were comparable in level

and precision to the quarterly surveys.There was essentially no change (0.5%decrease) in the level from November toApril based on FSA reporting, compared toa 6.4% decrease from December to Marchfor QAS data. Although this is only onestudy in one State, indications based on FSAtract reporting appeared to be quite stableand consistent across time.

One underlying issue that probablycontributes to the change from December toMarch is land operation changes that occurprior to March. This would include sellingor renting part of an operation, or anoperation going out of business betweenDecember and March. Since the FSA tractremains a viable unit in spite of ownershipchanges, using the land area as a samplingunit would avoid instability in reporting thatresults from these types of changes.

The FSA tract is a viable reporting unit.Several results indicate that using a framebased on FSA tracts may be beneficial.First, only two percent of the Aprilreinterview sample was considered notusable because respondents could not reportfor the sampled tract. Second, there wasalmost no change in the estimated total forwinter wheat planted acreage between theNovember survey and the April reinterviewstudy. Third, Benz (1996) found that farmoperators thought reporting data for FSAtracts or FSA farms was easier and moreaccurate than reporting for an entireoperation.

RECOMMENDATIONS

Continue to look at QAS data for sourcesof reporting inconsistency. This projectdealt primarily with examining the level of

18

reporting consistency that could be expectedfrom data collected from a frame of FSAtracts. While some analyses involving QASdata were performed, there are possibilitiesfor more research using the data that areavailable to us through the AgriculturalSurveys in order to investigate the high levelof reporting inconsistency and level changefrom December to March. Possible areasinclude effects from the mode of datacollection, a more in-depth analysis ofrespondent effects, and the effects ofimputation and estimation.

Use consistent wording in December andMarch when asking for winter wheatplanted (seeded) acreage. There does notappear to be any reason why the DAS andMAS use different wording for collectingacreage: data for winter wheat. Becausethere was no indication from the Aprilreinterview study that the difference inwording was contributing to inconsistency,it is recommended that consistent wordingbe used in the future to avoid any confusionand to head off any future questions aboutthe issue.

Monitor changes in the availability offarmer' reported FSA acreage data thatresult from modifications in the FarmBill. Due to provisions of the current FarmBill, FSA data will likely not be as completeand usable as a sampling frame in the future.The situation could change again later on, inwhich case NASS should take another lookat these data as a potential sampling frame.

REFERENCESBailey, J. (1994), "Results of the December1993 Acreage Reconciliation Study", UnitedStates Department of Agriculture, NationalAgricultural Statistics Service.

Benz, S. M. (1996), "Use of FSA Tracts as aSampling Frame for an Acreage Survey inKansas", United States Department ofAgriculture, National Agricultural StatisticsService.

Kott, P. (1990), "Mathematical Formulae forthe 1989 Survey Processing System (SPS)Summary", United States Department ofAgriculture, National Agricultural StatisticsService.

National Agricultural Statistics Service(1996), "1996 Crop Production Highlights

19

and Summary, May 10 1996", United StatesDepartment of Agriculture.

Parsons, Joseph (1996), "EstimatingCoverage of Farm Service Agency CropAcreage Totals", United States Departmentof Agriculture, National AgriculturalStatistics Service.

Sarndal, Carl-Erik, Swensson, Bengt, andJan Wretman (1992), Model AssistedSurvey Sampling, Springer-Verlag, NewYork.

Weaver, W. (1994), "Evaluating the Use ofthe ASCS List of Farm Operators as aSurvey Sampling Frame", United StatesDepartment of Agriculture, NationalAgricultural Statistics Service.

APPENDIX A: FORMULAS FOR ESTIMATING ACREAGE TOTALS AND VARIANCE

Acreage EstimatesThe winter wheat planted and harvested acreage estimates were calculated as described in Kort's(1990) discussion on estimating totals for stratified samples.

H

Y = L Yh where Yhh =1

gIvenH = nurnberofstrahunUh = set of selected units with usable values in strahun h in Aprilah = reweighted expansion factor (N* A/Uh) based on adjusted strahun level population

counts, where N* A is the adjusted stratum population count as shown in Table 3and Uh is the number of selected units with usable values for selected unit i

y, - value for selected unit i

Variance EstimatesThe bootstrap technique is one method that can be used for estimating the sampling distributionof a parameter estimator 6. In this case, it is the variance of total winter wheat planted acreage(or the harvested to planted ratio) that needs to be estimated.

A brief description of how to produce a bootstrap estimate is given by Sarndal, Swensson, andWretman (1992).

1) Using the sample data (n observations), construct an artificial population P' of size n thatmimics the true unknown population P.

2) Draw a series of independent samples from P* ,.vith replacement using the same samplingscheme that was used to draw the original sample. For each bootstrap sample, calculatean estimate 6a ~a = 1,2, ... , A) in the same way that 6 would be calculated.

3) The observed distribution of 6;, ... , e~ is considered an "estimate" of the samplingdistribution of the estimator 6 and v(6) is estimated

A 1 Aby VBS = _1_ L (6; - e')2 where 6' = I: 8;

A-I a=1 A <1=1

For this project, n=548 (the number of usable reports) with A=lOOObootstrap samples.

20

APPENDIX B: SCREENS FROM CATI REINTERVIEW INSTRUMENT

The QuestionnaireTo collect the data for the April FSA Reinterview Project, a short CATI interview was conductedin which the enumerator confirmed the reporting unit and asked for the winter wheat planted andharvested acreage. The CATI instrument used data from the November FSA Tract AcreageProject to validate the current survey response. The respo,ndent's April response was reviewedonly when the November and April responses differ by more than a certain percentage, inwhich case the differences were reconciled, capturing the correct response and the reason forthe difference.

The Dial Menu shown below is a standard one used in Blaise CATI applications. Commentsfrom the November survey can be viewed by scrolling down or by pressing <CTRL><ENTER>once the survey has started.

21

A standard introductory paragraph was provided to explain the purpose of the April reinterviewsurvey to the respondent.

The next screen was used to determine if the respondent could report for the selected FSA tract.If the respondent no longer operated the FSA tract, but could report for the it, the interview wasconducted. If the person no longer operated the FSA tract and could not report for it, or if thetract had been split, we did not require the enumerator to follow up on the new operator(s). Inthis case, 'NO' was entered and the interview was concluded. If the tract was split or was nowoperated by more than one person, the questionnaire was coded inaccessible. If the respondentcould report for the tract, the reinterview proceeded, otherwise it was concluded.

22

The next screen contains the question for the number of winter wheat acres that were planted.Note that this question's wording followed the March Agricultural Survey wheat question."Refusal" and "Don't Know" were allowed for this question. If the respondent could not reportfor the tract, the enumerator would go back and code the previous questions as either "No" or"Don't Know" and conclude the interview.

23

The current response was compared to the November response. lfthe responses were within agiven percentage of each other, the instrument was routed to the next question. However, if thetwo responses differed by more than the preset range, the enumerator was instructed to reconcilethe difference and obtain a reason, if possible, for the difference between the two responses.

Note: This question was asked only if April and November responses differed by more than agiven percentage. If the respondent replied 'YES', the enumerator continued and obtained areason for the difference. If the respondent replied 'NO', the instrument routed back to question2, in which the acres planted question was re-asked.

Seven possible reasons were provided on the screen so that the enumerator had only have to enterthe number corresponding to the reason, in the event that one of these reasons was supplied bythe respondent. If the respondent gave a reason other than what was supplied, "8" was enteredfor "Other Explanation" and the reason was then entered by the enumerator. If the respondentwas unable to provide a reason, the enumerator was instructed to prompt him/her with thefollowing reasons which were provided on the screen.

a. The wrong tract level data was recorded.b. A change was made to the land agreement of the tract involving rented acresc. Reported farm level data.d. One or more of the fields of winter wheat in the tract were excluded.e. Acres seeded (as asked in November) has a different meaning than acres planted.f. Included only acres to be harvested. Excluded acres for hay, grazing, etc.g. Planting intentions changed.h. Other explanation.

24

If planted acres was greater than zero, then the respondent was asked how many acres heintended to harvest. Harvested acres had to have been less than or equal to planted acres or anerror would occur. A warning would appear if harvested acres was less than 25% of plantedacres. The interview was then concluded after the response to the harvested acres question wasrecorded.

25