jamm 444: public opinion survey methodology & question wording feb. 12, 2008
TRANSCRIPT
JAMM 444: Public Opinion
Survey Methodology& Question Wording
Feb. 12, 2008
Tuesday, Feb. 12
Quiz 2 Schedule update Research paper topic memo Democratic, Republican contests Survey methods & question wording End of class: Return news clipping
assignment
Quiz 2
1. What term is given to governors, senators, and other “party bigwigs” who have voting privileges at the Democratic national convention?
2. Which Republican candidate pulled out of the race after Super Tuesday?
3. What might explain why Barack Obama received more support in primaries in Alabama and Georgia than pre-election polls indicated?
Quiz 2
4. Why might a potential candidate for public office commission a poll before deciding whether to run? (Give one reason.)
5. In Travis Ridout’s opinion, which of the two remaining Democratic candidates has the best chance of winning the nomination? Why?
BONUS: What color tie did Travis wear during his guest lecture on Feb. 7?
Schedule update
TODAY: Chapter 3, Question Wording Turn in extra-credit papers on Idaho
caucus Thursday: Chapter 4: Sampling
Guest speaker: Chris Williams, Department of Statistics
Feb. 19: Interviewing & Data Collection Research topic memo due (new date)
Research Topic Memo
Due Tuesday, Feb. 19 See Web site:
http://www.class.uidaho.edu/jamm444/assignments.htm
Possible Research Topics
1. Practice (methodology)
2. Issue or controversy
3. Person (historical figure)
4. Presidential candidate
5. Primary or caucus (one state)
Elements of memo
1. Topic, why you have chosen it, and its importance to public opinion.
2. At least three research questions (things you hope to discover).
3. At least five sources:1. One must be from a scholarly journal,
such as Public Opinion Quarterly
2. No more than one Web site
Superdelegates
Delegates drawn from the Democratic National Committee, members of Congress, governors and distinguished party leaders, such as former presidents, vice presidents, and congressional leaders. Former House Speaker Tom Foley is a
superdelegate from Washington state.
Why superdelegates matter
Current tally of superdelegates (CNN) Clinton – 224 Obama – 135 Neutral/undecided/no public preference – 437*
TOTAL: 796* *This number may change over the next several
months as people die, leave office or leave the Democratic Party.
The race for the nomination
Democrats Clinton 1,148 Obama 1,121 Edwards 26 Needed for nomination: 2,025
The race for the nomination
Republicans McCain 783 Romney 286 Huckabee 217 Paul 16 Needed for nomination 1,191
Brokered convention
A situation in U.S. politics in which no candidate obtains a majority of delegates during the primary and caucus process. Because no candidate will receive enough votes on the first ballot to win the nomination, the convention is brokered through political horse-trading and/or multiple ballots.
Last brokered conventions: 1948: Republicans (Thomas Dewey) 1952: Democrats (Adlai Stevenson)
Survey methodology
Why do a survey? Measure opinion of a population too large
to contact individually Save time, save money
“Surveys can be used in a scientific way to realize the great benefits of interviewing a representative sample instead of the whole population.” Salant & Dillman, 1994
Surveys work, if...
Sample is large enough to yield desired precision.
Everyone in population has equal chance of being selected.
Questions enable respondents to give accurate answers.
Sampled respondents have similar characteristics to non-respondents.
5 key elements of a survey
1. Identify population
2. Choose type of sample
3. Choose method(s) for gathering data
4. Write (and test) questionnaire
5. Analyze the results
Choosing a sample
Identify population of interest: University of Idaho students Moscow residents Idaho voters ‘The American public’
Choosing a sample
Non-probability sampling 1936, Literary Digest
Probability sampling Simple random sampling (rarely used) Systematic random samples (i.e., every 30th
name in student directory) Stratified sample: divided into groups (strata) Cluster sampling (spread over a wide area)
Cluster sampling
Used for many national surveys: National Election Survey (NES) General Social Survey (GSS) Gallup Organization
350 geographical segments Sample to reflect U.S. as a whole
(geographic area, size of community) Random telephone numbers in each area
How big is a typical sample?
UI or Moscow: 200 to 500 Idaho: 500 to 800 National: 1,000 to 1,500
For Thursday
Read: P&P 4: Sampling Techniques Non-probability vs. probability samples Sample size Sampling error
Chapter 3: Question wording
1. How can a sponsor with an ax to grind manipulate wording of questions? What types of problems with question
wording are more subtle but just as harmful to the accuracy of responses?
Chapter 3: Question wording
2. Describe the potential problems with the following types of questions:
a) Compound (multi-topic) questions
b) Factual questions
c) Double-negative questions
d) Leading (argumentative) questions
e) Branching (follow-up questions)
f) Use of multiple items or indexes
Chapter 3: Question wording
3. The book gives several questions of how alternative wording of questions influenced responses. Discuss the examples presented in these cases:a) Impeachment of Bill Clinton
b) Use of government wiretaps to fight terrorism
c) The National Security Agency’s collection of telephone records of U.S. citizens
Impeachment of Bill Clinton
Q: If the full House votes to send impeachment articles to the Senate for a trial, then do you think it would be better for the country if Bill Clinton resigned from office, or not?
Q: If the full House votes to impeach Bill Clinton, then do you think it would be better for the country if Bill Clinton resigned, or not?
Use of government wiretaps
Q: After 9/11, President Bush authorized government wiretaps on some phone calls in the U.S. without getting court warrants, saying this was necessary to reduce the threat of terrorism. Do you approve or disapprove?
Q: After 9/11, President Bush authorized government wiretaps on some phone calls in the U.S. without getting court warrants. Do you approve or disapprove of George W. Bush doing this?
NSA collection of phone records
Q: It’s been reported that the NSA has been collecting phone records of millions of Americans. It then analyzes calling patterns in an effort to identify possible terrorist suspects… Would you consider this an acceptable way for the federal government to investigate terrorism?
Q: Based on what you have heard about the program to collect phone records, would you say you approve or disapprove of this government program?
Chapter 3: Question wording
4. How can the order in which questions are asked affect responses? Give an example. Context effect: When the preceding question
influences a response Q: Would you say that traffic contributes more or
less to air pollution than industry? Q: Would you say that industry contributes more
or less to air pollution than traffic?
Tips for better question wording
1. Include filter questions to exclude some respondents based on interest or knowledge of issue
2. Use a neutral tone Unbiased phrasing of controversial topics Avoid words that lead respondent to
preferred answer (pseudo-surveys) Examples: Bill Sali constituent surveys
Question wording
3. Provide middle alternatives Offer a position between extremes Likert scale: 5 options, from ‘strongly
agree’ to ‘neutral’ to ‘strongly disagree’ 4. Be aware of context effect:
preceding questions affect responses hard knowledge vs. attitude general vs. specific
Question wording
5. Note the order of alternatives (especially in phone surveys) Primacy (first asked) Recency (last asked) Solution: rotate the alternatives.
Reading for Thursday
P&P 4: Sampling Techniques Non-probability vs. probability samples Sample size Sampling error
Pick up news clipping assignments