quantitative research. experimental research laboratory experiments artificial – low realism few...

Quantitative Research

Experimental Research

Laboratory Experiments Artificial – low realism Few extraneous variables High control Low cost Short Duration Subjects aware of

participation

Field Experiments Natural – high realism Many extraneous variables Low control High cost Long Duration Subjects unaware of

participation

Quantitative Research

Emotional Intelligence Survey

http://quiz.ivillage.co.uk/uk_work/tests/eqtest.htm

Validity

Extent to which a measurement instrument actually measures the attribute it was intended to measure.

Validity can be examined from a number of different perspectives, including:Face, content, criterion-related, and construct

validity

Assessing Validity

Face validity Researchers judge the degree to which a measurementinstrument seems to measure what it is supposed to.

Content validity The degree to which the instrument items represent the universe of the concept under study.

Criterion-related The degree to which a measurement instrument canvalidity predict a variable that is designated a criterion.

(a) Predictive ability; (b) Concurrent validity

Construct validity The degree to which a measure confirm a hypothesiscreated from a theory based upon the concepts understudy.(a) Convergent validity; (b) Discriminate validity

Reliability

ReliabilityDegree to which measures are free from

random error and, therefore, provide consistent data.

There are three ways to assess reliabilityTest-retest, equivalent forms, and internal

consistency

Assessing the Reliability of a Measurement Instrument

Test-retest reliability: Use the same instrument a second time under

as nearly the same conditions as possible.

Equivalent form reliability: Use two instruments that are as similar as

possible to measure the same object during the

same time period.

Internal consistency reliability: Compare different samples of items being

used to measure a phenomenon during the

same time period.

Reliability and Validity Situations in Measurement

.... .....

.

...

. .

.

..

.

.

.

.. ..

..

.

Situation 1 Situation 3Situation 2

Neither reliable nor Valid

Highly reliable but not valid

Highly reliable and valid

.

..

Error

Types of Error in Survey Research Random Sampling Error (Random error)

Error that results from chance variation Impact can be decreased by increasing

sample size and through statistical estimation (confidence interval) or “rule of thumb”

Systematic Error (non sampling error)Error that results for the research design or

execution.

R an d om E rro r

N on resp on s e e rro r

D e lib e ra te F a ls ific a tion U n c on sc iou s m is rep resen ta tion

A cq u iesc en ce E xtrem ity b ias In te rview er b ias A u sp ices b ias soc ia l d es irab ility

R esp on s e b ias

R esp on d en t e rro r

D a ta P roc es s in g S am p le se lec tion

In te rview er ch ea tin g In te rview er e rro r

A d m in is tra t ive e rro r

S ys tem atic E rro r

Tota l E rror

Types of Systematic Error

1. Administrative ErrorError that results from improper execution.

Data Processing ErrorQuality of data depends on quality of data

entry.Use of verification procedures can

minimize

Sample Selection ErrorSystematic error resulting from improper

sampling techniques either in design or execution.

Interviewer ErrorData recorded incorrectly (error or selective

perception). Interviewer Cheating

Mitigate by random checks

2. Respondent ErrorHumans interviewing humans...

Non-response errorStatistical difference between a survey that

includes only those who responded and a survey that also includes those who failed to respond.

Non-respondent: person not contacted or who refuses to participate

Self selection bias: extreme positions represented

Response biasErrors that result from tendency to answer in

“a certain direction”.Conscious or unconscious misrepresentation

Types:1. Deliberate falsification (why?)

Why would people deliberately falsify dataAppear to be what they are notDon’t trust confidentialityProtectTo end the interviewer quicker“Average man effects”

Types of response bias continued:1. Deliberate falsification2. Unconscious misrepresentation

Reasons for unconscious misrepresentation:Question formatQuestion contentMisunderstanding of question leading to

biased answerLack of time to consider answer fullyCommunication or semantic confusionother

Types of response bias Acquiescence bias: individuals have a

tendency to agree or disagree with all questions or to indicate a positive/negative connotation

Extremity bias: results for response styles varying from person to person; some people tend to use extremes when responding to questions

Types of response bias continued... Interviewer bias: Bias in the responses of the

subject due to the influence of the interviewer Auspices bias: respondents being influenced

by the organization conducting the study Social desirability bias: caused by

respondents’ desire, either consciously or unconsciously to gain prestige or to appear in a different social role

Measurement

The Concept of Measurement and Measurement Scales

• Measurement• Process of assigning numbers or labels to

things in accordance with specific rules to represent quantities or qualities of attributes.

• Rule: A guide, method, or command that tells a researcher what to do.

• Scale: A set of symbols or numbers constructed to be assigned by a rule to the individuals (or their behaviors or attitudes) to whom the scale is applied.

Types of Measurement Scales

• Nominal Scales• Scales that partition data into mutually

exclusive and collectively exhaustive categories.

• Ordinal Scales• Nominal scales that can order data.

• Interval Scales• Ordinal scales with equal intervals between

points to show relative amounts; may include an arbitrary zero point.

• Ratio Scales• Interval scales with a meaningful zero point so

that magnitudes can be compared arithmetically.

Nominal

Ordinal

Interval

Ratio

Win Place Show

1 length 2 lengths

40 to 1 long-shot pays $40

Type of Scale Numerical Operation

Descriptive Statistics

Nominal Counting Frequency;

Percentage; mode

Ordinal Rank ordering (plus…)Median

Range; Percentile

Interval Arithmetic operations on intervals bet numbers

(plus…) Mean;

Standard deviation;

variance

Ratio Arithmetic operations on actual quantities

(plus…) Geometric mean; Co-efficent of variation

Selecting appropriate univariate statistical method

Scale Business Problem

Statistical question to be asked

Possible test of statistical significance

Nominal Scale

Identify sex of key executives

Is the number of female executives equal to the number of males executives?

Chi-square test

Scale Business Problem

Statistical question to be asked

Possible test of statistical significance

Nominal Scale

Indicate percentage of key executives who are male

Is the proportion of male executives the same as the hypothesized proportion?

T-test

Scale Business Problem

Statistical question to be asked

Possible test of statistical significance

Ordinal scale

Compare actual and expected evaluations

Does the distribution of scores for a scale with categories of poor,good, excellent differ from an expected distribution?

Chi-square test

Scale Business Problem

Statistical question to be asked

Possible test of statistical significance

Interval or Ratio scale

Compare actual and hypothetical values of average salary

Is the sample mean significantly different from the hypothesized population mean?

Z-test (sample is large)

T-test (sample is small)

Surveys

Survey data

Survey data are obtained when individuals respond to questions asked by interviewers or when the individual responds to questions that he has read

Quantitative and qualitative information Census or sample?

Survey Sampling

Sample SurveySample of the population

(e.g., sample of Alberta CEOs) Census Survey

Complete population(e.g., all CEOs in Lethbridge)

Functions

Descriptive Causal (limited function) Exploratory (limited function)

Information Provided by Surveys

respondent’s knowledge of facts how many times a month do you buy cookies?

respondent’s attitudes “chocolate chip is my favorite variety of cookie.”

May describe processes undertaken by respondent Stages in the purchase decision, for example

usually self-report data

The type of information gathered depends on a survey’s objectives.

Surveys typically have multiple objectives

Classifications Method of Communication

TelephoneMailPersonal interview

Degree of StructureUnstructuredstructured

Degree of Disguise

Limitations:

Structure and disguise

are not clear categories;

most surveys are hybrids

Temporal classificationsCross sectional studiesLongitudinal studies

Trend Studies Samples general population at each point; complete turnover in

who is actually sampled

Cohort studies (tracking studies) Samples from one group over time; e.g., sample of a graduating

class

Panel studies sample cohort, same specific respondents each sample

Method of Communication

Media Personal Interviews Telephone Interviews Self administered Questionnaires

Questionnaire design

A survey is only as good as the questions it asks

What should you ask?

The questions asked are a function of previous decisions

The questions asked are a function of future decisions (such as statistical analysis)

Key criteria

Questionnaire relevancyNo unnecessary information is collected and

only information needed to solve the problem is obtained. Be specific about your data needs; tie each question to an objective

Questionnaire accuracy Information is both reliable and valid

Phrasing Questions

Open ended response versus fixed alternative questions

“?” Decision criteria: type of research; time;

method of delivery; budget; concerns regarding researcher bias

Avoid

Leading questions Overly complex questions Use of jargon Loaded questions (can use a counterbiasing

statement) Ambiguity Double barreled questions Making assumptions

Order?

Order bias results from an alternative answer’s position in a set of answers or from the sequencing of questionsFunneling technique: general to specific helps

understand the frame of reference first Anchoring effect: the first concept

measured tends to become a comparison point from which subsequent evaluations are made

Types of questions

• Single dichotomy or dichotomous-alternative questions“Are you currently registered in a course at the

University of Lethbridge?Yes____ No____”

• Respondent chooses one of two alternatives (yes/no; male/female)

• What scale would this data create?

Types of fixed alternative questions…

Multi-choice alternativeRespondent chooses from several

alternativesMany types…

Types of fixed alternative questions…

Determinant choice Choose only one from several possible responses

“Which faculty are you currently registered in at the University of Lethbridge?

Management ___Education ____Arts/Science____Health sciences____Combined degree____

What type of scale would these data create?

Multi-choice alternative questions…

Frequency determinationAsks for an answer about frequency

of occurrence

In a typical week, how often do you purchase chocolate chip cookies?

__never

__ once

__ 2 or more timesWhat type of scale would these data create?

Check listProvide multiple answers to a single questionShould be mutually exclusive and exhaustive“What brands of chocolate chip cookies have

you, to the best of your memory, purchased in the past month (check all that apply?)”

__ Dare__ Chips A’hoy__ Presidents Choice Decadent etc. etc.

What type of scale would these data create?

Unidimensional Scaling

Multidimensional Scaling

Procedures designed to

measure only one attribute of a respondent or

object

Procedures designed to measure several dimensions of a respondent or

object

Attitude rating scalesAttitude:

An enduring disposition to consistently respond to various aspect of the world, including persons, events and objects

Typically seen as having three components:

CognitiveAffectiveBehavioural

Affective

The feelings or emotions toward an object

Cognitive

Knowledge and beliefs

Behavioral

Predisposition to action Intentions Behavioral expectations

Attitude measuring process

Ranking Rating Sorting Choice

Types of attitude scales

Simple attitude scales Most basic form – respondent responds to a

single question Do not allow for fine distinctions or placement on

continua You are at a company party and are feeling nervous,

but you are obligated to be there. Do you:__ find someone you know to buddy up with__ take it as an opportunity to meet new people

What type of scale would these data create?

Category scales More sensitive; provides more information Overall, how satisfied are you with the high speed

performance of your Mercedes:

__ very satisfied

__ somewhat satisfied

__ neither satisfied nor dissatisfied

__ somewhat dissatisfied

__ very dissatisfied

If you could choose, how long would each term be?

___26 weeks __ 13 weeks __ 6 weeks ___4 weeks

What type of scale would these data create?

Summated rating scales – the Likert scaleRespondents indicate their attitudes by

checking how strongly they agree or disagree with statements

Chocolate chip cookies are my preferred variety of cookie

Strongly disagree Disagree Uncertain Agree Strongly Agree

(1) (2) (3) (4) (5)

What type of scale would these data create?

Semantic Differential Rating scaleAn attitude measure consisting of a series

of seven-point bipolar rating scales allowing response to a “concept”

Think of your favorite type of cookie. Rate it on each of the following continua:

Hard------------------------------------------------------Soft

Lots of chips---------------------------------------Fewer chips

Crispy---------------------------------------------------chewy

What type of scale would these data create?

Numerical Rating scale Similar to a semantic differential except that it

uses numbers as response options to identify response positions instead of verbal descriptions

Think of your favorite type of cookie. Rate it on each of the following continua:

Hard------------------------------------------------------------------------Soft

8 7 6 5 4 3 21

This scale is called an 8 point numerical scale, why?

What type of scale would these data create?

Constant Sum Scales Attributes based on their importance to the person.

Respondents are asked to divide a constant sum to indicate the relative importance of attributes

Example: Suppose the photocopy budget per professor was $100 per month. How much should be allocated to the following. Divide the $100 according to your preference:

____ photocopying for student needs; ____ photocopying for research needs; ____ photocopying for committee needs.====$100 TOTAL

Stapel Scales An attitude measure that places a single adjective

in the center of an even-number range of numerical values

Example:Research Methodology

+3+2+1

Exciting-1-2-3

Graphic Rating ScalesAn attitude measure consisting of a graphic

continuum that allows respondents to rate an object by choosing any point on the continuum

Rank-Order ScalesScales in which the respondent compares

one item with another or a group of items against each other and ranks them.

Example: handout

Most important skills Adaptability to change Problem identification Listening skills Written communication Leadership Informal Oral communication Analytical thinking/problem solving Time management Coping with stress/job pressures Interpersonal relations Formal oral presentations

Most important skillsAdaptability to changeProblem identificationListening skillsWritten communicationLeadershipInformal Oral communicationAnalytical thinking/problem solvingTime managementCoping with stress/job pressuresInterpersonal relationsFormal oral presentations

8 96 61 12 44 23 35 57 1011 79 810 11

Managers

Business

grads

Paired Comparison ScalesRespondent is presented with two objects

and is asked to pick the preferred.

Example: Which type of cookie do you prefer

__ chocolate chip

__ oatmeal

__ I do not have a preference between these two

SortingRespondent indicates their attitudes or

beliefs by arranging items.

Example: Please sort the following cards with pictures of cookies into the following categories

Like

Dislike

Neither like nor dislike