Research Methods

Surveys

Survey

The survey is a non-experimental, descriptive research method.

Surveys can be useful when a researcher wants to collect data on phenomena that cannot be directly observed.

In a survey, researchers sample a population. Basha and Harter (1980) state that "a population is

any set of persons or objects that possesses at least one common characteristic."

Terminologies

Survey Instrumentation. The survey instrument is name given to the schedule of questions or response items to be posed to respondents. It is equivalent to the terms questionnaire or interview schedule, depending on the type of survey research being conducted.

Items are the individual survey questions or statements for which a response is solicited from the respondent.

Terminologies

Response structure refers to the format of items, which may be open-ended or structured. Structured items may be multiple choice, multiple response, dichotomies, rank lists, or a variety of other formats.

Terminologies

Survey error: Deviation from real result. survey error includes such factors as faults in sampling, coding, tabulating, data processing, interviewer bias, researcher bias, and data misinterpretation.

Terminologies

Analysis of non-response is almost always a required aspect of survey research. Planning ahead for analysis of non-response may cause the researcher to add items to the survey instrument.

Terminologies

Types of Surveys

Cross-Sectional Surveys Longitudinal Surveys

Trend Studies Cohort Studies Panel Studies

Cross-Sectional Surveys

Cross-sectional surveys are used to gather information on a population at a single point in time.

An example of a cross sectional survey would be a questionnaire that collects data on how parents feel about Internet filtering, as of March of 1999.

A different cross-sectional survey questionnaire might try to determine the relationship between two factors, like religiousness of parents and views on Internet filtering.

Longitudinal Surveys

Longitudinal surveys gather data over a period of time.

The researcher may then analyze changes in the population and attempt to describe and/or explain them.

The three main types of longitudinal surveys are trend studies, cohort studies, and panel studies.

Trend Studies

Trend studies focus on a particular population, which is sampled and scrutinized repeatedly.

While samples are of the same population, they are typically not composed of the same people.

Trend studies, since they may be conducted over a long period of time, do not have to be conducted by just one researcher or research project.

A researcher may combine data from several studies of the same population in order to show a trend.

An example of a trend study would be a yearly survey of librarians asking about the percentage of reference questions answered using the Internet.

Cohort Studies

cohort studies have a different focus. A cohort study would sample the same class, every time.

For example, a sample of 2004 graduates of GU could be questioned regarding their attitudes toward professionals in libraries.

Four years later, the researcher could question another sample of 2004 graduates, and study any changes in attitude.

If the researcher studied the class of 2008 four years later, it would be a trend study, not a cohort study.

Panel Studies

Panel studies allow the researcher to find out why changes in the population are occurring, since they use the same sample of people every time.

That sample is called a panel. A researcher could, for example, select a sample of GU graduate

students, and ask them questions on their library usage. Every year thereafter, the researcher would contact the same people,

and ask them similar questions, and ask them the reasons for any changes in their habits.

Panel studies, while they can yield extremely specific and useful explanations, can be difficult to conduct.

They tend to be expensive, they take a lot of time, and they suffer from high attrition rates. Attrition is what occurs when people drop out of the study.

Instrument Design

Surveys should be just as rigorously designed and administered as any other research method.

Meyer (1998) has identified five preliminary steps that should be taken when embarking upon any research project:

1. choose a topic,2. review the literature,3. determine the research question, 4. develop a hypothesis, and 5. operationalization (i.e., figure out how to accurately measure the factors you

wish to measure). For research using surveys, two additional considerations are of

prime importance: 1. representative sampling and 2. question design.

Representative Sampling

A sample is representative when it is an accurate proportional representation of the population under study.

If you want to study the attitudes of GU students regarding library services, it would not be enough to interview every 100th person who walked into the library.

That technique would only measure the attitudes of GU students who use the library, not those who do not.

In addition, it would only measure the attitudes of GU students who happened to use the library during the time you were collecting data.

Therefore, the sample would not be very representative of GU students in general.

In order to be a truly representative sample, every student at GU would have to have had an equal chance of being chosen to participate in the survey.

This is called randomization.

Representative Sampling

If you took a list of GU students, uploaded it onto a computer, then instructed the computer to randomly generate a list of 2 percent of all GU students, then your sample still might not be representative.

What if, purely by chance, the computer did not include the correct proportion of seniors, or honors students, or graduate students?

In order to further ensure that the sample is truly representative of the population, you might want to use a sampling technique called stratification.

Representative Sampling

In order to stratify a population, you need to decide what sub-categories of the population might be statistically significant.

For instance, graduate students as a group probably have different opinions than undergraduates regarding library usage, so they should be recognized as separate strata of the population.

Once you have a list of the different strata, along with their respective percentages, you could instruct the computer to again randomly select students, this time taking care that a certain percentage are graduate students, a certain percentage are honors students, and a certain percentage are seniors.

You would then come up with a more truly representative sample.

Question Design

It is important to design questions very carefully. A poorly designed questionnaire renders results meaningless. There are many factors to consider. Babbie gives the following pointers:

Data collection modes

Face-to-face interviewing is usually the most expensive but it can obtain the highest response rate because it maximizes engagement with the respondent and imposes the lowest burden on the respondent.

For the same reasons, face-to-face interviews can support longer instruments.

It is also the best data collection type for open-ended responses. In situations where the respondent is initially unknown, this may be the

only feasible data collection type. Liabilities of this method include possibility of interviewer bias,

respondent bias on sensitive items, slow data collection time, and cost. Face-to-face interviewing is often used when the research purpose

requires in-depth exploration of opinions.

Data collection modes

Mail surveys are usually among the least expensive modes, are often best for sensitive items, and there is no interviewer bias.

However, mail surveys are a poor choice for open-ended items or complex survey designs as they place a high burden on the respondent.

Other liabilities include lack of control over respondent interaction with unknown others while taking the survey and while being faster than face-to-face administration, mail surveys are slower than phone or web surveys.

Mail surveys are often used for very short instruments.

Data collection modes

Telephone interviewing has the advantage of speed of data collection while supporting longer instruments than mail surveys and supporting open-ended responses, though not as well as face-to-face interviewing.

There is also moderate control over interviewer bias since interviewers follow predetermined protocols, and there is high control over respondent interaction with others during the interview.

Telephone interviewing can be a good moderate-cost compromise between mail and face-to-face survey research.

Data collection modes

Web surveys are often the least expensive to administer yet can be fast in terms of data collection, particularly since implementation time may be low.

Web surveys can be administered to very large populations, even internationally.

Also, they support complex survey designs. The central drawback to web surveys is the near-impossibility of drawing

random samples since web access is known to be biased by income, race, gender, and age.

Also, web surveys impose almost as high a burden on the respondent as do mail surveys, and thus are not optimal for long instruments or open-ended questions.

Web surveys are frequently a good choice when an unscientific but quick “straw poll” is needed on some subject.

Survey design considerations

Survey layout Survey order Survey length Item length

Survey Layout

Survey layout. For mail and web surveys, the survey layout itself needs to be attractive, easy-to-use, and non-intimidating. Some rules of thumb are:

– Include plenty of white space (as opposed to dense single-spaced print) – Have response areas be consistent and easy to check off (checkboxes

are clearer than blank lines) – Have response areas on the right since most respondents are right-

handed – Use one or at most two colors to set of sections and items, avoiding

distracting color schemes. – Avoid multiple fonts, italics (harder to read), , animations, blinking font, – Have clear separation of the question part of an item and the response

part, by line spacing and/or by putting response categories in bold – In general, follow good print or web design principles

Survey Order

Survey order. Typically, after an introduction which discloses the sponsorship of a survey, a survey begins with non-threatening items which arouse interest.

The first question should be clearly related to the announced purposes of the survey (not a background item, for instance).

Some recommend the second question be open-ended, to allow the respondent to "get into" the subject.

If the survey is mixed open-ended and structured items, some researchers prefer to put the open-ended questions first so as to avoid priming the respondent with the structured items.

Survey Order

Non-threatening background information questions (ex., demographic information) should be posed early so these controls will be available if the respondent fatigues and does not answer all the later items.

The survey then proceeds to attitude questions, often sequencing from general and less threatening items toward more specific and more sensitive items.

Sensitive background items, particularly the income item, are usually put at the end.

Survey Length

Survey length. There is no correct length to a survey: they should be as long as needed within the constraint of respondent attention span.

Surveys need to have a minimum of one item per variable referenced in the set of hypotheses the researcher is investigating and at least three items per variable are recommended, along with demographic items.

Item Length

Item length. The longer the item, the more the chance respondents will misunderstand.

As a rule of thumb, items should target a length of 25 words or less.

Items with such terms as and, or, except, with, and but may be a sign of over complexity and excessive length.

Item bias

Bias may creep into survey items in a variety of ways.

Performing the Survey

Define Your Research Aims Identify the Population and Sample Decide How to Collect Replies Questionnaire Design Run a Pilot Survey Carry Out the Main Survey Analyze the Data

References

Babbie, Earl R. Survey Research Methods. Belmont, CA: Wadsworth Pub. Co., 1973.

– Exhaustive description of the purposes, methods, and techniques of surveying. If you are going to conduct a survey, you need to read this book first.

Busha, Charles H., and Stephen P. Harter. Research Methods in Librarianship: Techniques and Interpretation. Orlando, FL: Academic Press, Inc., 1980.

– The bible of library research.