the pro-incumbent bias in the 1978 and 1980 national election studies

The Pro-Incumbent Bias in the 1978 and 1980 National Election StudiesAuthor(s): Robert B. Eubank and David John GowSource: American Journal of Political Science, Vol. 27, No. 1 (Feb., 1983), pp. 122-139Published by: Midwest Political Science AssociationStable URL: http://www.jstor.org/stable/2111056 .

Accessed: 29/11/2014 19:03

Your use of the JSTOR archive indicates your acceptance of the Terms & Conditions of Use, available at .http://www.jstor.org/page/info/about/policies/terms.jsp

.JSTOR is a not-for-profit service that helps scholars, researchers, and students discover, use, and build upon a wide range ofcontent in a trusted digital archive. We use information technology and tools to increase productivity and facilitate new formsof scholarship. For more information about JSTOR, please contact [email protected].

.

Midwest Political Science Association is collaborating with JSTOR to digitize, preserve and extend access toAmerican Journal of Political Science.

http://www.jstor.org

This content downloaded from 155.97.178.73 on Sat, 29 Nov 2014 19:03:56 PMAll use subject to JSTOR Terms and Conditions

http://www.jstor.org/action/showPublisher?publisherCode=mpsa

http://www.jstor.org/stable/2111056?origin=JSTOR-pdf

http://www.jstor.org/page/info/about/policies/terms.jsp


The Pro-Incumbent Bias in the 1978 and 1980 National Election Studies*

Robert B. Eubank, Rice University David John Gow, State University of New York at Binghamton

This study demonstrates that there is a pronounced proincumbent response bias to the reported House vote in the 1978 and 1980 National Election Studies. The magnitude of the bias ranges from approximately 7 percentage points in 1980 to 14 percentage points in 1978. Only a small portion of the bias, less than 3 points, is attributable to the particular sample of congressional districts. We show that the major source of bias is the survey instrument itself. In the survey a number of questions before the vote question focus the attention of the respondent on the name and activities of the incumbent. Consequently, the survey instrument influences respondents who have little information and who are susceptible to prompting effects to identify the incumbent as the person for whom they voted. This means that studies using the 1978 or 1980 surveys run the risk of incorporating the bias into their findings. The nonrandom nature of the bias suggests that the impact of incumbency in de- termining the outcome of congressional elections has been overestimated.

The 1978 American National Election Study (NES) signifies the first time in 20 years that a national survey has been explicitly designed and suited for research on congressional elections. Two dramatic changes dis- tinguish the 1978 study from its immediate predecessors. First, in 1978 congressional districts became the primary sampling units (PSU's), and consequently the multistage cluster sampling procedure yielded 2,304 post- election interviews clustered in 108 congressional districts. Prior to 1978 the sampling procedure yielded respondents who were scattered over 150 or more congressional districts. Of course, the 1978 sample was not in- tended to provide estimates of each district's constituency, but it was designed to yield accurate estimates of the relevant universe of 432 eligible congressional districts in the 48 coterminous states. The second change occurred in the survey content, which was expanded to include a wide array of items dealing with congressional elections, including candidate thermometers, incumbent job ratings, and candidate contacts. These and other additions emphasized issues relevant to studying the dynamics of congressional elections.

*We wish to thank David Brady, Robert Erikson, Elizabeth Sanders, and the Journal's referees for their comments on an earlier draft of this article.

American Journal of Political Science, Vol. 27, No. 1, February 1983 01983 by the University of Texas Press 0092-5853/83/010122-18$01.65



THE PROINCUMBENT BIAS 123

The changes in survey content and sampling procedure introduced in 1978 were subsequently incorporated in the 1980 NES survey. However, the 1980 study also included a component on the presidential race and was not limited in scope to congressional elections. Thus it is not as rich a source of information about House elections as its immediate forerun- ner. Indeed, a large portion of our current understanding of congressional elections is based on the 1978 NES survey. Since the release of the data there have been conferences, conference panels, books, symposia, and a number of articles published that predominantly (or entirely) focus on the 1978 NES data set. The resultant research (for example, Hinckley, 1980; Jacobson, 1981a; Mann & Wolfinger, 1980) reports the importance of both party and incumbency as cues for voters in choosing among competing candidates.

The purpose of this article is to demonstrate that in the 1978 and 1980 NES surveys there is a systematic bias in the respondents' first-person reports of the congressional candidate for whom they voted.' This bias inflates survey estimates of voting for incumbents and therefore af- fects the validity of any research that uses the respondents' reports of the candidates for whom they cast their votes. Our conjecture of pro-incumbent bias should not be viewed as only an interesting methodological footnote to the 1978 and 1980 surveys; more important, it questions the foundation of all research using these data bases. Inasmuch as theories and explanations depend upon data for their corroboration or falsifica- tion, the quality of that data is of direct concern to researchers because the data provide a standard of acceptance or rejection.

The 1978 NES Survey

Insofar as the 1978 NES represents a break with earlier studies and subsequently became the prototype for the 1980 NES, we begin with a detailed analysis of it and defer our examination of the 1980 survey until the second part of this analysis. The first hint that something is awry in the 1978 NES survey comes from time series evidence on the impact of incumbency on the outcome of congressional elections. If incumbency is an important factor, more voters will defect from voting for a candidate of their own party when the candidate of the opposite party is an incumbent than will defect when the incumbent is a member of their own party. (We use the label "incumbent partisan" to designate a supporter of

' Except where otherwise indicated, the data used in this article were made available by the Inter-university Consortium for Political and Social Research (ICPSR). The data for the 1978 and 1980 American National Election Studies were originally collected by the Cen- ter for Political Studies, Institute for Social Research, under a grant from the National Sci- ence Foundation. Neither the original collectors of the data nor the ICPSR bear responsi- bility for the analysis or interpretations presented here.



124 Robert B. Eubank and David John Gow

the incumbent's party and the label "challenger partisan" to signify a supporter of the challenger's party.) The incumbents' net advantage can be summarized in an index constructed by subtracting the percentage of incumbent partisans who defect to the challenger from the percentage of challenger partisans who defect to the incumbent.

Table 1 reports the survey estimates of the defection rates of incumbent and challenger partisans, and the resultant incumbency index for In- stitute for Social Research-based election surveys from 1956 to 19802.

TABLE 1

Partisan Defection Rates and the Incumbents' Advantage, 1956-1980

Incumbent Partisans Who Challenger Partisans Who Incumbents' Defected to Challenger Defected to Incumbent Advantage

Year (%) (N) (%) (N) (%)

1956 6.4 528 15.1 344 8.7 1958 7.0 401 16.1 299 9.1

1960 8.4 521 18.6 381 10.2

1964 10.8 435 24.8 315 14.0

1966 8.8 296 33.7 205 24.9 1968 13.0 361 32.3 297 19.3 1970 5.8 278 32.9 216 27.1 1972 10.0 542 37.2 347 27.2

1974 11.9 403 32.0 344 20.1

1976 8.1 521 40.7 432 32.6

1978 4.4 408 53.5 275 49.1 1980 9.5 367 48.3 269 38.8

SOURCE: Computations by the authors from the Survey Research Center-Center for Political Studies surveys.

NoTE: We have treated "independent-leaners" as partisans.

2Because our focus is on incumbency, we have excluded from our calculations (except where otherwise indicated) all persons who were in districts in which there was no incumbent or in which there was no challenger. We define an incumbent as one who was a member of Congress at the time of the November election under study. Consequently, incumbents were in a few instances elected in special elections in the same year as the general election.




Between 1956 and 1966 there is a consistent increase, then a slight decrease in the incumbency index occurs in 1968 before stabilizing near 27 percentage points in 1970 and 1972. The incumbents' advantage then decreases to 20 percentage points in 1974 (reflecting the defeat of 37 Republican incumbents in the aftermath of Watergate) before rising to a new high of almost 33 percentage points in 1976 and then jumping to 49 percentage points in 1978. This represents an increase of 16 points in the incumbents' advantage from the previous high in 1976. (In 1980 the advantage of incumbents dropped to below 39 percentage points, down 10 points from the record 1978 level of 49.1, but still significantly higher than any pre-1978 value.)

The second hint that something may be amiss is that aggregate returns indicate that nineteen incumbents were defeated in 1978, six more than in 1976. One would anticipate that an increase in pro-incumbent voting of the magnitude found in the 1978 NES survey would be associated with a decrease in the number of incumbents suffering electoral de- feats. (The disparity between the aggregate and survey data results is based on different levels of analysis. Thus the results are not necessarily contradictory because aggregate analysis may mask the shifts that are taking place at the individual level.) It is this dramatic increase in the 1978 survey estimate of pro-incumbent voting, the increase in victories by challengers notwithstanding, that has reinforced the view that incumbents are relatively insulated from defeat in House elections.

There are three commonly accepted explanations that purport to account for the incumbents' big leap forward in the 1978 survey. The first view is that the large increase in defection rates in 1978 is due, in part, to the adoption of a survey format that more faithfully reflects actual voting behavior. In 1978 a name recognition format was introduced to re- place name recall in the congressional voting questions. With this new format respondents are no longer asked to recall for whom they voted, but are provided with a list of candidates in order to better simulate the conditions in the voting booth. Of course it is easier for respondents to recognize than recall the name of their preferred candidate (Tedin and Murray, 1979). Mann (1978, p. 13) has suggested that the name recall format used in the pre-1978 surveys may have inflated estimates of party line voting and deflated defection rates. He reasons that respondents who are unable to recall the name of the candidate for whom they voted may fall back on their own party identification (or partisan forces of the moment) as the basis for their response.3

The second reason advanced for the incumbents' advantage in 1978 is that there were sampling difficulties, both in the sample of congressio-

3A rigorous test of this conjecture has been undertaken by Eubank (1982) who finds little supporting evidence.




nal districts and the sample of voters within the districts. The congressional races that took place in the respondents' districts were not representative of the races in the 432 congressional districts that define the relevant universe. The 108 sample districts had unusually weak challengers (Ja- cobson, 1981a), and only one district was sampled in which a House incumbent lost. Nationwide, nineteen congressmen were defeated in the 1978 election, so the sample was expected to include four or five districts in which the incumbents were unseated. Jacobson (1981a, p. 240) has suggested that in addition to including the unusual districts the survey oversampled voters who were "inordinately hostile" to the challengers. The ability of the survey respondents to recall the names of the candidates challenging incumbents reached an all-time low (in the history of Survey Research Center-Center for Political Studies [SRC-CPS] surveys) of 16 percent.

The third and most popular view is that the NES survey reflects a change occurring in electoral behavior, in which heavily financed incumbents use the perquisites of office, including franking privileges and case work, to distance themselves from their opposition. Consequently, incumbents are highly visible in comparison with their opponents and, coupled with an erosion of party loyalties, voters are supporting incumbents in increasing numbers.

These three views of the increased advantage of incumbents in the 1978 congressional election are not contradictory, and most researchers would endorse all three, albeit in differing proportions. A rigorous test of these assertions is difficult to devise ex post facto, although scholars can gather indirect evidence. Putting aside for the moment the possibility of a behavioral change, we can test the propositions the other two factors generate. Adopting a name recognition format suggests that the NES survey data should at least correspond to aggregate data for the districts that form the sample. Put another way, it suggests that the name recall format of the pre-1978 surveys may have induced an element of bias. To the extent that there were atypical races (such as weak challengers) in the survey districts, there will be a disparity btween returns in the 108 sample districts and the returns nationwide. Relevant data to test these propositions are reported in Table 2. The basis for comparison is the vote for incumbents.

The first column reports the NES survey estimates of pro-incumbent voting. The second column reports the comparable statistics for the congressional districts in which the sample was administered, and the third column reports the population parameters for the universe of 309 contested districts with incumbents. Comparisons between the first and second columns gauge the survey's accuracy within the respondents' con-




gressional districts; and comparisons between the second and third columns reflect the national representativeness of the sample districts.4

It is clear from an inspection of Table 2 that the NES survey estimate of pro-incumbent voting overestimates the report of the Congres- sional Directory (U.S. Joint Committee on Printing, 1981) of the returns for the sample districts by 11.5 percentage points, and it exceeds the national vote for incumbents by 14.0 points.5 This disparity between the survey and the aggregate vote totals demonstrates, ceteris paribus, that the adoption of the name recognition format has not itself improved the validity of the responses to the question of the respondents' votes. In addition, Table 2 indicates that the disparity between the survey estimates and the returns in the survey districts is distributed evenly among the incumbents of both major parties, as districts with Democratic incumbents (11.9 points) and Republican incumbents (10.9 points) manifest much the same bias.

Is the pro-incumbent bias in the NES sample a function of the unusual races (in particular, the weak challengers) in the sample districts? This rival hypothesis is not well supported, for it accounts only for the 2.5 percentage difference between the aggregate returns in the sample dis-

4In order to replicate our analysis it will be necessary to correct the contextual data distributed as parts of both the 1978 and 1980 NES data sets. The following corrections must be made to release 1 (Summer, 1981) of the 1980 NES data set. The missing data code (of 9999999) should be replaced by the following returns, in which the numbers in paren- theses refer to the Democratic and Republican vote, respectively: Indiana 9 (136,574, 75,601), Indiana 6 (99,089, 98,302), Indiana 3 (85,136, 103,972), Colorado 4 (76,849, 178,221), Colorado 1 (107,364, 67,804), and Michigan 1 (123,286, 6,244). Other major er- rors in the contextual data include transposed vote totals for the Democratic and Republi- can candidates in New Jersey 12 (36,577, 134,973); a Democratic candidate who is listed as having received no votes, Pennsylvania 24 (86,567, 86,687); and a vote total that indicates the Democratic candidate won Republican Gregory Carman's seat, New York 3 (83,389, 87,952). Finally, for our purposes West Virginia 3 must be treated as an open seat even though there was, by our definition, an incumbent at the time of the 1980 election. The NES survey staff asked the incumbent questions about former Representative John M. Slack who died eight months before the 1980 election.

In the Spring 1979 release of the 1978 NES data, the type of race is miscoded for three congressional districts: New York 19 is coded as having a Republican challenger, New York 38 as having a Democratic challenger, and Mississippi 3 (129,088, 8,321) as having no Re- publican challenger (Miller, 1978, p. 643-44). Moreover, Virginia 3 is given a Republican vote, yet there was no Republican challenger. Our corrected vote totals were obtained from the Congressional Directory (U.S. Joint Committee on Printing, 1981) and checked against Scammon and McGillivray (1981), and Barone and Ujifusa (1981). There were some minor inconsistencies among these sources.

5As a rough guide, at the 95 percent confidence level the estimated sampling error for a simple random sample is 3.8 percent with a sample size of 700 and a 50:50 binomial percentage (Babbie, 1973, p. 377). Of course, the 1978 and 1980 NES surveys used multistage cluster sampling rather than simple random sampling.




tricts and the national vote. There remains a disparity of 11.5 points between the 1978 NES survey estimate and the aggregate figures within the respondents' survey districts.

In addition, the use of a measure designed to gauge the strength of the challenge (namely, whether a candidate has held prior elective office) reveals the presence of the same pro-incumbent bias in districts with both putatively "strong" and "weak" challengers (11.0 and 11.5 points respectively). Table 3 indicates that the bias remains even when challengers did in fact run strongly, as indicated by their ability to capture at least 40 percent of the vote, and thereby fall within the range we call marginal. Indeed the pro-incumbent bias within marginal districts (10.8 points) is almost as high as that in nonmarginal districts (11.0 points). The bias remains high and at approximately the same level across many types of districts, and in both Democratic and Republican incumbent districts within each of these categories. It is improbable that a sample was drawn from each of these districts in which voters disproportionately support incumbents and are hostile to challengers. The magnitude of the bias is too

TABLE 2

Pro-Incumbent Voting in the 1978 Congressional Election

1978 NES Survey NES Sample Districts National Result District (No) (N) (%) (N) (n) (Gb) (N) (n)

All contested districts with incumbents 78.8 761 67.3 10,188,122 76 64.8 41,359,973 309

Districts with Democratic incumbents 80.1 438 68.2 5,825,596 47 64.3 25,533,204 201

Districts with Republican incumbents 77.1 323 66.2 4,362,526 29 65.6 15,826,769 108

Marginal districts 66.5 191 55.7 2,923,171 20 53.7 13,351,356 90

Nonmarginal districts 83.0 570 72.0 7,264,951 56 70.1 28,008,617 219

SOURCE: Computations by the authors from the 1978 NES survey, Barone et al. (1979), and the 1981 Congressional Directory (U.S. Joint Committee on Printing, 1981).

NOTE: N signifies the number of voters, and n signifies the number of congressional districts.




large to attribute to sampling error, and on the basis of the analysis reported in Tables 2 and 3, we conclude that the bias is systematic in its impact.

The analysis reported above provides the groundwork for our conjecture. Even the most cautious interpretation of these results suggests that the popular understanding of the 1978 NES survey, particularly the advantage attributed to incumbency, is not well founded. The 1978 survey estimates are not only out of line with previous SRC-CPS surveys, but also out of line with the corresponding parameters in the congressional districts in which the survey was administered, as well as with the population parameters.

The Location of the Bias

Our extensive analysis of the 1978 survey indicates that the pro-incumbent bias can be isolated when controls are introduced for the salience of the candidates, as measured by the extent to which respondents could recall the candidates' names. Table 4 reports the rates of defection

TABLE 3 Pro-Incumbent Voting in the 1978 Sample Districts

1978 NES Survey NES Sample Districts

Type of District (N) (N) (%) (n) Marginal districts 66.5 191 55.7 20

Democratic incumbent 63.9 133 55.7 13 Republican incumbent 72.4 58 55.7 7

Nonmarginal districts 83.0 570 72.0 56 Democratic incumbent 87.2 305 74.1 34 Republican incumbent 78.1 265 69.5 22

"Experienced" challengers 74.4 160 63.4 16 Democratic incumbent 88.6 35 62.0 4 Republican incumbent 70.4 125 63.9 12

"Inexperienced" challengers 80.0 600 68.5 60 Democratic incumbent 79.4 403 68.8 43 Republican incumbent 81.2 197 67.7 17

SOURCE: Computations by the authors from the 1978 NES survey, Barone et al. (1979), and the 1981 Congressional Directory.

NoTE: N signifies the number of voters, and n signifies the number of congressional districts.



TABLE 4 Partisan Defection Rates, 1966-1980

Recalls Own Party's Recalls Other Party's Recalls Both Recalls Neither Candidate Only Candidate Only Candidates Candidate

Year Type of Voter (?7) (N) (07) (N) (Wo) (N) (070) (N)

1966 Incumbent partisans 4.6 87 - 4 8.3 84 10.7 121 Challenger partisans - 5 66.7 36 34.3 79 21.2 85



1972 Incumbent partisans 6.7 90 - 2 13.0 69 17.4 115 Challenger partisans - 3 - 22 34.0 47 27.9 86


Pre-1978 Incumbent partisans 3.4 101 - 4 16.7 90 11.2 127 means Challenger partisans - 10 64.1 47 32.1 90 23.4 97



SOURCE: Computations by the authors from the SRC-CPS election studies. NOTE: The percentages are not reported when the associated N iS less than 30.




(from their own party) of incumbent and challenger partisans for the SRC-CPS surveys from 1966 to 1980, excluding 1976 in which year the name recall questions were not included in the survey. These time series together with the pre-1978 means provide a baseline against which the 1978 and, presently, the 1980 survey can be evaluated. The voters have been classified on the basis of four categories of candidate salience: recalls own party's candidate only; recalls other party's candidate only; recalls both candidates; and recalls neither candidate.

In 1978 voters supported incumbents at rates greater than the pre- 1978 means in five of the six categories that have adequate cell frequen- cies. Incumbent partisans in each of the three salience categories defect to the challenger at rates less than expected, while in two of the three categories challenger partisans defect at rates greater than the pre-1978 means.

In order to avoid resting our analysis on small sample fluctuations, we can focus on those cells in 1978 that depart dramatically from previous values, say by more than 10 percentage points from the corresponding pre-1978 mean. By this criterion the pro-incument bias evidently occurs in three of the four salience categories, but it is especially prevalent among those respondents who have low levels of information, as indicated by their inability to recall the name of either candidate.6 In elections before 1978 these respondents manifested relatively little variation with defections (to the incumbent) by challenger partisans, averaging 23 percent and never exceeding 28 percent. In 1978, 50 percent of the voters in this category defected to incumbents, while defections to challengers dropped to a new low of 5.8 percent. The resultant incumbency index of 44.2 percentage points is almost triple the size of the largest previous incumbents' advantage in this category. Moreover, the incumbents' advantage extends across both marginal and nonmarginal districts, indicating that the advantage is not a result of the challengers' failure to challenge.

While it is challenger partisans with low information who defect from their party, it is challenger partisans with high information who are exceedingly loyal when judged against the relevant mean. However, the large pro-incumbent bias is not due solely to the low name recall of challengers,7 but to a change in the apparent behavior of voters -particularly those unable to recall the name of House candidates.

6Respondents who could only recall the other party's candidate also have high defection rates from their own party's candidate.

7Jacobson, almost alone, has treated the problem of the bias. His focus on low challenger recall is certainly a valid one, although it cannot account for all the bias we have documented. Much of the low level of name recall of challengers is a function of the weak challengers. We have developed methods for partitioning the bias among competing factors and found that name recall would, at best, still leave 9 percent of the bias to be explained.




The Cause of the Bias

Why then is the bias associated with less knowledgeable respondents? On the basis of empirical evidence to be presented below we demonstrate that the survey format, particularly the sequence of questions, prompts the voters to give biased responses. The fact that the order of survey questions can bias responses is well established in survey research methodology (McFarland, 1981; Schuman, Presser, and Ludwig, 1981; Sigelman, 1981). Our conjecture is also in line with Clausen's finding (1968) that response bias from short-term forces is especially prevalent among the least interested respondents. Unlike the partisan forces in the 1964 election, however, there are no universal short-forces favoring incumbents in the 1978 election.

In 1978 there is a series of questions placed before the congressional voting question that may prompt pro-incumbent responses from respondents, especially the less interested ones, when they are subsequently asked for whom they voted. Respondents in districts with incumbents were questioned about contact with the incumbent and challenger, with the incumbent-challenger distinction stated in the questions. Consider, for example, the following questions located before the question asking respondents for whom they voted: "V186-There are many ways in which the U.S. Representatives can have contact with the people from their districts. On this page are some of the ways. Think of [name of House incumbent] who has been the U.S. Representative from this district. Have you come into contact or learned anything about (him/her) . .. V227 Have you (or anyone in your family living here) ever contacted Represen- tative [name of House incumbent] or anyone in (his/her) office?"8

There are several sets of questions dealing with the incumbent and the House of Representatives. These questions ask about the respondent's contact with the House incumbent (V227 to V239), the incumbent's roll call voting (V240 to V247); and the respondent's perception of the most important activities of the- House (V248 to V253). Whatever the responses to these questions, the questions themselves focus the attention of the respondent on the incumbent-both the concept ("the U.S. Repre- sentative from this district") and the actual name of the incumbent, which is supplied by the interviewer. During the course of a typical interview, the interviewer will mention the House incumbent's name about a

IThere were questions about the challenger, but they were not as numerous. For example: "V197-How about [name of Democratic or Republican House candidate who op- posed the District House Incumbent] who also ran for the U.S. House of Representatives from this district in the last election. Have you come into contact or learned anything about (him/her) .... ." These questions create a qualitative distinction between "the U.S. Representative in this district" and the challenger who "also ran."




dozen times, and will indirectly identify the incumbent as "he" or "she" many more times. (The exact number depends upon the respondent's previous answers to contingency questions). Subsequently, the interviewer will provide a list of candidates and ask the respondents to identify the person for whom they voted. The "incumbency" questions contribute to the incumbency effect, which they were designed to explore, because they are placed before the question asking respondents for whom they voted. These questions provide information about the incumbent and thereby prompt voters to subsequently report that they voted for their representative.

Of course a rigorous test of our conjecture would require a split sample with two different questionnaires-one in which the contaminat- ing questions are presented before, and the other in which the questions are presented after respondents reveal the candidate of their choice. Only in this way could the impact of the question sequence be accurately evaluated. In lieu of such an experiment we can demonstrate the superiority of our conjecture over the most plausible rival hypothesis. The alterna- tive explanation is that the NES survey reflects a tendency for respondents, all of whom were being interviewed after the election, to report that they voted for the winner. This bandwagon effect would then manifest itself as a pro-incumbent bias because only one incumbent was defeated in the 1978 sample districts.

There are three points that lend support to our conjecture of prompting effects rather than bandwagon effects. First, we can simulate the split sample experiment with a quasi-experimental design. The questions about incumbents will not contaminate the responses of respondents in districts with open seats. A bandwagon effect, if present, will occur in races with both open and occupied seats, whereas the prompting effect will occur only in races for seats with incumbents running for re- election. In open seats the- 1978 survey accurately matches the aggregate returns. Here the survey estimate is off by less than 1 percent, documen- ting that the bias occurs only when respondents are asked questions about the incumbent in their district.

Second, the argument for a bandwagon effect is not compatible with evidence from pre-1978 surveys. Table 5 provides information on the earlier CPS studies in a form similar to Table 2. The table reports the survey estimate of pro-incumbent voting and the aggregate vote extracted from the Congressional Directory. In each election the survey corres- ponds very closely to the aggregate results. The mean (absolute) devia- tion of the survey results from the aggregate returns in the survey districts is 2.5 percent, and the mean disparity between the survey estimate and the national vote is 3.1 percent. Both figures are within the bounds




of the estimated sample error.9 Evidently there was no bandwagon effect in the pre-1978 surveys, and its introduction as an explanation for the bias in 1978 has a decidedly ad hoc quality.

Third, if the questionnaire does contaminate responses its effect would be greatest on those who have low levels of information, namely those who cannot recall the name of either candidate. The respondents who, at the beginning of the interview, could not recall either candidates' name are prompted to identify the incumbent as the person for whom they voted because of the repetition of the name. Presumably a bandwagon effect would be distributed more evenly across the different categories of voters.

Whatever may be the merits of any one of the three points we have outlined above, it seems that collectively the weight of evidence is against

TABLE 5

Survey Estimates and Aggregate Pro-Incumbent Votes, 1970-1976

Survey Survey National Estimate Districts Vote

Year (070) (N) (0) (N) (07o) (N)

1970 67.7 535 65.8 16,498,259 64.4 44,308,381 1972 69.5 970 65.3 20,507,003 65.2 54,951,050 1974 63.0 826 64.4 14,335,736 62.8 41,223,613 1976 69.6 1,058 67.0 21,634,170 64.9 58,740,027

SOURCE: Computations by the authors from the CPS surveys, Barone et al. (1979), the 1977 Congressional Directory, and Scammon and McGillivray (1981).

9Until 1978 the sampling frame was built around about 40 standard metropolitan sta- tistical areas (SMSAs) and 30 non-SMSAs, which are either single counties or county groups. Since the PSU's in the pre-1978 studies were not congressional districts the vote totals in the sample districts should be interpreted with caution because some districts have only one respondent. We have also computed weighted estimates of the proincumbent vote. When the percentage of votes in each district is weighted by the number of sample voters the results are within 3 percent of the results obtained by the unweighted method. In addition, for the 1974 survey we weighted the number of votes in each district by the number of sample voters and found that the survey is off by 2.8 percentage points. Clearly, these results indicate the accuracy of the pre-1978 surveys.

Independent confirmation of the accuracy of the pre-1978 surveys is available. Jacob- son (1981a) has reported a similar analysis of the vote for House challengers in 1972, 1974, and 1978. He found that the CPS survey estimate of the vote for challengers in 1972 and 1974 is practically identical-within 1 percent-to the average district figures for these years. For 1978 he found a difference of nearly 13 percentage points.




the rival hypothesis of a bandwagon effect, and supports our interpretation that the survey is the source of the problem.

The 1980 NES Survey

To this point we have established that there is a pro-incumbent bias of 14 percentage points in the 1978 NES survey. The evidence supporting our explanation of the source of the bias is compelling although not conclusive. Cross-validation in an independent national sample provides the best test of our thesis. If our conjecture of prompting effects is correct then the bias present in the 1978 study should also be found in the 1980 survey. Most of the questions about incumbents that were originally asked in the 1978 survey were retained in much the same sequence in the 1980 survey-only the roll call questions were deleted. However, the 1980 election survey focuses on the presidential race with the congressional questions relegated to secondary status. In addition, the congressional vote question no longer occupies the top ballot position, although it still precedes the senatorial and gubernatorial questions. Thus there is reason to sus- pect that prompting effects may be somewhat less in 1980.

Table 6 reports relevant data to test for bias. This table is in the same format as Table 2 save that the data now represent 1980 rather than 1978. It is clear from an inspection of this table that the pro-incumbent bias is present once again, although in proportions diminished from the record 1978 levels. The disparity between the survey results and the nationwide returns is 7.0 percentage points, and the disparity between the survey and the aggregate returns in the sample districts is 5.5 percentage points.

In 1980 the magnitude of the bias is less; however, the pattern is very similar to that of 1978. Just as in 1978 the bias occurs evenly across districts with Democratic and Republican incumbents and in marginal as well as in nonmarginal districts. And, just as in 1978, the survey is within 3 points of the aggregate returns in open seats where the contaminat- ing questions about incumbents are not asked. In controlling for candidate saliency, we again find the bias centered almost exclusively in the low information voters. The challenger partisans who recall neither candidate have a 47 percent defection rate, close to the 50 percent rate in 1978 yet twice the pre-1978 mean of 23 percent. The smaller bias present in the other salience categories has now disappeared.

Even though the bias in 1980 is about one-half the 1978 level, the survey estimates are still outside the region of accuracy of the pre-1978 surveys. The emphasis on the presidential race, both in the number and sequencing of the questions, is the most likely reason for the decrease in




the bias.'0 There also is a more representative sample of races including six districts with losing incumbents (excluding West Virginia 3), whereas there was only one in 1978. The decline in the pro-incumbent survey vote in 1980 can also be attributed to a decrease in the percentage of incumbent partisans in the sample and to a change in the distribution of voters among the salience categories. However, it is important that any discus- sion of the decrease in the bias does not cloud our basic finding. The difference of 7 percentage points between the survey estimate and the population parameter still exceeds the sampling error and is indicative of a systematic bias. Thus the presence of the bias, in both direction and magnitude, corroborates our conjecture of proincumbent prompting effects in the 1978 and 1980 NES survey.

TABLE 6

Pro-Incumbent Voting in the 1980 Congressional Election

1978 NES Survey NES Sample Districts National Result District (Oo) (N) (07) (N) (n) (lo) (N) (n)

All contested districts with incumbents 72.4 693 66.9 16,021,549 84 65.4 62,712,374 335

Districts with Democratic incumbents 69.4 405 63.6 8,815,217 50 62.9 36,911,733 208

Districts with Republican incumbents 77.0 287 71.0 7,206,332 34 68.9 25,800,641 127

Marginal districts 58.3 192 52.5 4,335,970 22 53.3 20,771,006 105

Nonmarginal districts 77.8 501 72.3 11,685,579 62 71.4 41,941,368 230

SOURCE: Computations by the authors from the 1980 NES survey, Barone and Ujifusa (1981), the 1981 Congressional Directory, and Scammon and McGillivray (1981).

NaOE: N signifies the number of voters, and n signifies the number of congressional districts.

'?The apparent bias among independent voters also disappears in 1980. In the 1978 NES 82 percent of independents voted for incumbents, far greater than in any previous year. In 1980 independents were consistent with past levels of support by voting for incumbents at a 69 percent level.




Consequences of the Bias

The bias in the respondents' report of their vote in the 1978 and 1980 NES surveys distorts our understanding of voting behavior in congressional elections, especially the degree of credence given to the impact of incumbency. Most of our knowledge about congressional elections, particularly about the impact of incumbency, comes from the 1978 and 1980 studies. The general consequence of the location of the bias is to call into question all results utilizing this variable, particularly as a de- pendent variable.

Of course the pro-incumbent bias is visible only when the respondents' vote is treated in terms of the incumbent/challenger distinction. This conceptual scheme arises most often in studies of the incumbents' advantage and in studies of the economic determinants of voting. It is a simple matter to demonstrate that the extent to which the bias is evident is a function of the particular model specification. For example, when the respondents' votes are conceptualized in terms of traditional partisan divisions (Republican/Democrat) the bias is undetectable although still present. Both the 1978 and 1980 survey estimates are within 3 percentage points of the aggregate two-party vote totals in both the sample districts and the nationwide returns in contested districts (including open seats). The reason for the apparent accuracy is that the bias towards Democratic incumbents is largely offset by the bias towards Republican incumbents. The greater number of Democratic incumbents causes slightly inflated (2 percent) survey estimates of Democratic voting in the sample districts. However, it would be a mistake to assume that the pro-incumbent bias is absent. It is masked by the model specification incorporating partisan- ship, but not incumbency.

The bias that we have identified should be distinguished from the bias stemming from over-reports of voting. Katosh and Traugott (1981) found that "mis-reporters" make up about 12 percent of the 1978 sample, much the same as our estimate of the pro-incumbent bias. Their analysis indicates that there is no systematic difference in defection rates between validated and mis-reported voters in House races. Our analysis confirms that the pro-incumbent bias remains at the same level among validated voters as in the full sample.

Conclusions

Our primary concern is to provide an accurate view of the effects of incumbency on voting and to ensure future surveys are not flawed, yet continue to make use of the excellent set of questions devised for the 1978 and 1980 surveys. To support our case we have established two central points. The first is that there is a pro-incumbent bias of about 14 percentage




points in the 1978 NES survey, and about 7 percentage points in the 1980 survey. This bias cannot be attributed to the atypical races in the sample districts. Even in the sample districts the survey is off by more than 11 points in 1978 and 5 points in 1980. The pro-incumbent bias in the NES surveys cannot be attributed to a particular type of congressional race, or to particular attributes of the district.

Our second goal has been to explain why this bias occurs. The evidence indicates that the less knowledgeable respondents manifest the bias to an extent greater than any other group. Based on this observation and other evidence, we have shown that the probable cause of the bias is the order of the questions. The less knowledgeable respondents are prompted by prior questions to identify the incumbent as the person for whom they voted.

We do believe that enough evidence has now been accumulated to justify two recommendations about the NES surveys. Our first recom- mendation is that past research that utilized these NES data bases should be reevaluated in light of the impact of the pro-incumbent bias on their substantive findings. The magnitude of the pro-incumbent bias in the voting response makes it especially important not to estimate incumbency effects from these surveys. Our second and most important recommenda- tion is that in future NES surveys the vote questions should be located before, rather than after the questions delving into incumbency effects. It also seems prudent to place the congressional voting question in its usual ballot position sequence, after the senatorial and gubernatorial questions.

Manuscript submitted 18 January 1982 Final Manuscript received 21 June 1982.

REFERENCES

Abramowitz, Alan I. 1980. A comparison of voting for U.S. Senator and Representative in 1978. American Political Science Review, 74 (September):633-640.

Babbie, Earl R. 1973. Survey research methods. Belmont, Calif.: Wadsworth Publishing Co.

Barone, Michael, and Grant Ujifusa. 1981. The almanac of American politics, 1982. Wash- ington, D.C.: Barone and Company.

Barone, Michael, Grant Ujifusa, and Douglas Matthews. 1979. The almanac of American politics, 1980. New York: E. P. Dutton.

Clausen, Aage. 1968. Response validity: Vote report. Public Opinion Quarterly, 32 (Win- ter):56-64.

Cover, Albert D. 1977. One good term deserves another: The advantage of incumbency in congressional elections. American Journal of Political Science, 21 (August):523-542.

Eubank, Robert B. 1982. Those exaggerated incumbency figures. Unpublished manuscript, Rice University, Houston.

Hinckley, Barbara. 1980. The American voter in congressional elections. American Political Science Review, 74 (September):641-650.




Jacobson, Gary C. 1981a. Congressional elections, 1978: The case of the vanishing challengers. In Louis Sandy Maisel and Joseph Cooper, eds., Congressional elections. Be- verly Hills, Calif.: Sage Publications: 219-247.

. 1981b. Incumbents' advantages in the 1978 U.S. congressional elections. Legislative Studies Quarterly, 6 (May): 183-200.

Katosh, John P., and Michael W. Traugott. 1981. The consequences of validated and self- reported voting measures. Public Opinion Quarterly, 45 (Winter): 519-535.

Maisel, Louis Sandy, and Joseph Cooper, eds. 1981. Congressional elections. Beverly Hills, Calif.: Sage Publications.

Mann, Thomas E. 1978. Unsafe at any margin: Interpreting congressional elections. Wash- ington, D.C.: American Enterprise Institute.

Mann, Thomas E., and Raymond E. Wolfinger. 1980. Candidates and parties in congressional elections. American Political Science Review, 74 (September): 617-632.

McFarland, Sam G. 1981. Effects of question order on survey responses. Public Opinion Quarterly, 45 (Summer): 208-218.

Miller, Warren E. 1978. American national election study, 1978. Ann Arbor, Mich.: Inter- university Consortium for Political and Social Research.

Scammon, Richard M., and Alice V. McGillivray, eds. 1981. America votes 14. Washington, D.C.: Congressional Quarterly, Inc.

Schuman, Howard, Stanley Presser, and Jacob Ludwig. 1981. Context effects on survey responses to questions about abortions. Public Opinion Quarterly, 45 (Summer): 216-223.

Sigelman, Lee. 1981. Question-order effects on presidential popularity. Public Opinion Quarterly, 45 (Summer): 199-207.

Tedin, Kent L., and Richard W. Murray. 1979. Public awareness of congressional representatives: Recall versus recognition. American Politics Quarterly, 7 (October): 509-517.

U. S., Congress, Joint Committee on Printing. 1977. Congressional Directory, 95th Con- gress. Washington, D.C.: U.S. Government Printing Office.

. 1981. Congressional Directory, 97th Congress. Washington, D.C.: U.S. Govern- ment Printing Office.



the pro-incumbent bias in the 1978 and 1980 national election studies

Documents