the research process, surveys, questionnaires & interviews communication research week 3
TRANSCRIPT
Communication Research Spring 2005 2
Steps in the research process 1) Establish the need for research 2) Define the problem 3) Establish research objectives 4) Determine research design 5) Identify information types and sources 6) Determine methods of accessing data
Communication Research Spring 2005 3
Steps in the research process 7) Design data collection forms 8) Determine sample plan and size 9) Collect data 10) Analyse data 11) Prepare final research report
Communication Research Spring 2005 4
The research process
PR
OB
LEM
S
TA
TE
ME
NT
Purpose of the study
Exploration
Description
Hypothesis
Types of investigation
Establishing:
Causal relationships
Correlations
Group differences, ranks etc
Extent of researcher
interference
Minimal: Studying events as they normally occur
Manipulation and/or control
and/or simulation
Study setting
Contrived
Non contrived
Measurement & measures
Operational defnItems (measure)
ScalingCategorising
Coding
Unit of analysis (pop to be studied)
Individuals
Dyads
Groups
Organisations
Machines
Sampling design
Probability/non
probability
Sample size (n)
Time Horizon
One-shot (cross-
sectional)
Longitudinal
Data-collection method
Observation
Interview
Questionnaire
Physical measurement
Unobtrusive
Data-analysis
1.Feel
for data
2.Goodness
of data
3. Hypothesis
testing
DETAILS OF STUDY MEASUREMENT
Communication Research Spring 2005 5
The Classical Ideal of Science
Generalization
Theories
Hypotheses
Observation
Deduction
OperationalizationMethod & measurement
Induction
Communication Research Spring 2005 6
Deductive vs Inductive Reasoning
DEDUCTIVE Argues from the general
to the particular Eg you observe that all
deciduous trees lose their leaves …
You therefore reason that your bare tree is deciduous
INDUCTIVE Argues from the
particular to the general Eg if you burn one finger
on a hot stove … You therefore reason
that you could burn all of them
Communication Research Spring 2005 7
Preparing an hypothesis or research question
An hypothesis is a ‘reasonable scientific proposal’ or a ‘statement of expected results’
It is not a statement of fact but a declarative statement which tells the reader what you are going to do NOT how you plan to do it
An hypothesis is common to scientific research methodologies while qualitative research more commonly uses research questions to focus
Communication Research Spring 2005 8
Writing a hypothesis A well written hypothesis
Is stated in declarative form Posits a relationship between variables Reflects a theory or body of literature upon
which it is based Is brief and to the point Is testable
Communication Research Spring 2005 9
Developing an hypothesis An hypothesis should be developed from a well-
researched body of knowledge which is both logical and feasible eg Inland waterways are becoming polluted through fuel
discharge from petrol-driven powerboats Powerboats are noisy thus creating auditory pollution Alternatives to petrol engines exist
Electric (battery-powered) boats will decrease pollution on
inland waterways
Communication Research Spring 2005 10
Ideas, questions & hypothesesResearch Interest or
IdeasResearch Problem or
QuestionsHypothesis
Open classroom and academic success
What is the effect of open vs traditional classrooms on reading level?
Children taught reading in open classroom settings will read at a higher grade level than children taught reading in a traditional setting.
Test-taking skills and grades
Will students who how to “take” a test improve their scores?
Students who receive training in the “Here Today – Gone Tomorrow” method will score higher on the SATs than students who do not receive the training.
Television and consumer behaviour
How does watching television affect buying behaviour of adolescents?
Adolescent boys buy more of the products advertised on television than do adolescent girls.
Drug abuse and child abuse
Is drug abuse related to child abuse?
There is a positive relationship between drug abuse among adults and their physical and psychological abuse as children
Adult care How have many adults adjusted to the responsibility of caring for their aged parents?
The number of children who are caring for their parents in the child’s own home has increased over the past ten years.
Communication Research Spring 2005 12
Problems with asking questions as evidence (from Vance Packard (1956) The Hidden Persuaders)
1. You can’t assume that people know what they want In a survey of male drinkers the men expressed a
strong preference for a ‘nice dry beer’. When they were then asked how a beer could be dry, they were stumped.Those able to offer any answers at all revealed widely different notions.
Communication Research Spring 2005 13
Problems with asking questions as evidence (from Vance Packard (1956) The Hidden Persuaders)
2. You can’t assume people will tell you the truth about their wants and dislikes even they know them. Psychologists at the McCann-Erikson ad agency
asked a sampling of people why they didn’t buy one client’s product – kippered herring.The main reason the people gave under direct questioning was that they just didn’t like the taste of kippers. More persistent probing however uncovered the fact that 40% of the people who said they didn’t like kippers had never in their entire lives tasted kippers.
Communication Research Spring 2005 14
Problems with asking questions as evidence (from Vance Packard (1956) The Hidden Persuaders)
3. It is dangerous to assume that people can be trusted to behave in a rational way. A test was designed to establish the influence of the
package on the product. It gave housewives three different boxes filled with detergent and requested that they try them all out for a few weeks and then report which was the best for delicate clothing (Note: actually only the boxes were different, the detergents were identical – one box was predominantly yellow; the second was blue and the third was blue with splashes of yellow).
Result: the detergent in the yellow box was too strong, in the blue box it left the clothes dirty and in the blue and yellow it was ‘fine’ and ‘wonderful’.
Communication Research Spring 2005 15
Survey research
Purpose is to determine the current status of a population with respect to one or more variables
Can be qualitative or quantitative, depending on data sought
Communication Research Spring 2005 16
Advantages of surveys Inexpensive Can obtain current information Enable the researcher to obtain a great deal of
information at one time Provide quantitative or numerical data Very common and so some of the info you seek
may have already been gathered eg ABS
Communication Research Spring 2005 17
Problems with surveys People often don’t tell the truth, especially
about personal matters People make mistakes about what they’ve
done Obtaining representative samples is frequently
difficult People often refuse to participate Relatively small percentages of people answer
and return questionnaires Writing good survey questions is difficult
Communication Research Spring 2005 18
Self-administered questionnaires
ADVANTAGES Inexpensive No interviewer bias to worry
about You can ask about very
personal matters You can ask complex, detailed
questions
DISADVANTAGES People may misinterpret
questions Low response rates the norm You don’t know who actually
filled out the questionnaire Sampling errors frequent
Communication Research Spring 2005 19
Stages of a survey 1. Define your information needs
what information? purpose of information? who knows the answers? practical issues eg time
2. Carry out background research – what work has been done before in this area?
3. Choose a survey technique questionnaires diaries individual interviews group discussions
Communication Research Spring 2005 20
Stages of a survey 4. Define and test your methodology
Open or closed questions? Pre-test questions Ordering of questions Trial questioning techniques
5. Administer your survey – consider any problems By mail/email Is target group representative? Were all questions answered satisfactorily?
6. Analyse your results Did enough people reply? Is target group representative? Were all questions answered satisfactorily?
Communication Research Spring 2005 21
Stages of a survey 7. Present your findings
Usually in report format Outline parameters of survey critically Do your results prove or imply results? Acknowledge limitations eg time, sample size,
demographics of sample etc
Communication Research Spring 2005 22
Problems with Survey Research
Inadequate response
Including unrelated items on questionnaire
Poorly worded items
Complex items
Leading questions
Assuming facts not necessarily in evidence
Analysing open-ended questions
Communication Research Spring 2005 23
Problems with Survey Research People often don’t tell the truth about
themselves People make mistakes even if they are
trying to tell the truth eg ratings books Obtaining representative samples is
difficult Relatively small percentage of people
answer and return questionnaires
Communication Research Spring 2005 24
Survey & Questionnaire design
When designing a survey or questionnaire you need to consider the type of question which will give you the most accurate data
There are five main types of questions 1. Close-ended questions which use yes/no
responses Q. I have good communications with my supervisor
A. Yes No
Communication Research Spring 2005 25
Survey & Questionnaire design
2. Open-ended questions which allow the respondents to give an unlimited answer
Q. What problems are you having with your supervisor?
Communication Research Spring 2005 26
Survey & Questionnaire design 3. Checklist which present a list of items where
participants are asked to check those items that apply to their particular situation.
Q. Please check the following types of communications that you have with your supervisor.
Informal meetings After hours discussions
Formal meetings Telephone
Written reports Social gatherings
Emails Committee meetings
Communication Research Spring 2005 27
Survey & Questionnaire design 4. Multiple-choice questions which offer several
choices and the respondent is asked to select the most correct one. Ensure the choices presented cover all the possible options.
Q. How often do you purchase items from the company vending machines?
a) Once a dayb) 2-3 times a dayc) 3-5 times a dayd) 6 or more times a day
Communication Research Spring 2005 28
Survey & Questionnaire design 5. Ranking scales which require the participants
to rank order a list of items. Q. Of the following list of five types of communication that you
might have with your supervisor, rank from 1 (most important) to 5 (least important).
— Formal meetings— Informal conversations— Written reports— Letters or emails— Telephone discussions
Communication Research Spring 2005 29
Survey & Questionnaire design 6. Likert scales which usually measure attitude toward a
concept or idea. It allows the respondent to indicate the degree of agreement usually on a 5 or 7 point scale.
Q. Please indicate the degree to which you agree or disagree with this statement. “The 1995 Ford Falcon is a substantial improvement on the 1994 model.”
Strongly agree Agree Neither agree Disagree Stronglynor disagree
Disagree
5 4 3 21
Communication Research Spring 2005 30
Survey & Questionnaire design 7. Semantic differential scales measures attitudes by displaying
pairs of opposite terms and asking respondents to check which term best describes their feeling toward the concept or topic.
Q. Place an X in the space between the two terms that best describes how you see the XYZ Corporation Office.
XYZ CORPORATION DISTRICT OFFICE[the topic or entity being evaluated]
Pleasant - - - - - - - - - - - - -- - - - - UnpleasantEfficient - - - - - - - - - - - - -- - - - - InefficientNot helpful - - - - - - - - - - - - -- - - HelpfulProfessional - - - - - - - - - - - - -- - UnprofessionalInsensitive - - - - - - - - - - - - -- - - SensitiveFriendly - - - - - - - - - - - - -- - - - - UnfriendlySlow - - - - - - - - - - - - -- - - - - FastRigid - - - - - - - - - - - - -- - - - - Flexible
Communication Research Spring 2005 31
Other types of qualitative research – depth interviews
one-on-one interview which is relatively unstructured with a subject by a trained interviewer.
The direction of the interview is guided by the responses to the questions
Communication Research Spring 2005 32
Other types of qualitative research – personal interviews
ADVANTAGES Interviewer can explain Qs in
detail Interviewer can use a variety of
data collection methods Interviewer can spend a lot of
time with respondents You know who is answering the
questions A higher likelihood of achieving
the desired response rate Not intimidating
DISADVANTAGES Can be intrusive (too personal) Time-consuming and expensive Hard to find people in sample at
times People are reluctant to answer
some questions Needs well-trained interviewers Possible language barriers
Communication Research Spring 2005 33
Other types of qualitative research – focus groups
Often used in market research A group is asked a series of structured
questions and guided through structured sessions
Designed to probe their attitudes and feelings about a range of issues
Communication Research Spring 2005 34
Other types of qualitative research – projection techniques
Such as word association tests, sentence and story completion, cartoon tests, consumer drawings, photo sorts
These techniques belong to the field of clinical psychology
They are designed to probe and penetrate a person’s defense mechanisms and allow true feelings to emerge