Agenda for today

Research

Statistics

Communication

Logic & Observation

SelectQuestion

ReviewResearch

DevelopHypothesis Research

Design

Measurement

Data Analyses CommunicateLogic & Results

Ethics

Another view of where we are

Research Designs

Measurement

Data Analyses

Field Studies

Sample Survey (Meta-analysis)

Scaling

CommunicateLogic & Results

Written Assignments

What we covered so far…what’s next

• Via Methodology Assignment– Submit Nov 2/06

• Via “Rationale of your research” assignment– Submit Dec 14/06

Communicating Logic

What’s Required in the Rationale Assignment?

1. What is the phenomenon?

2. How is it different/similar to another phenomenon?

3. When/where/how is it exhibited? Why?

4. When is it not? Why?

5. What common process explains its occurrence vs. non occurrence?

6. Why might your hypothesis not be supported given you will use valid measures and a large enough sample size?

• Because answering these questions will help you improve your writing skills– Evidence from 2 Meta-analyses: Hillocks, 1985;

Atkinson, 1993

• Because it is similar to what you will do at work when you make an effective proposal or effectively evaluate the outcome of a completed project…– E.g., loan officer proposal

Why do you need to answer these questions?

• Hillocks, 84; cf Atkinson, 1993

Type of instruction G meta-analytic effect

Answering these questions

.56

Grammar -.29

Previous Research shows that answering these questions improves writing quality

• Evaluate loan application from a multi-national that • Owned farms, food processing & manufacturing plants etc. • Defaulted on payment of earlier loan for millions of dollars but submitted a request for

extension on loan & an additional loan

• Bank would consider additional loan to enhance probability of recovering initial loan from multi-national

• While writing, loan officer had to

– Assess performance w/initial loan – Evaluate current viability

• Assess productivity, markets, current financial structure, management, etc.

– Predict potential for recovery & continued viability – Convince superiors recommendation was valid

• Success of recommendation determined officer’s future in bank

Example of One Loan Officer’s Writing Task

(Hillocks, 95)

• Should be clear and narrow in scope– Examples of a hypothesis

• X is related to Y

• Can convert your Focused Research Qs to a hypothesis: – How is x related to y? Why is x related to Y?

– Examples of non-hypotheses• A topic: what is X? what is Y? \

• A research qs: What affects X?

The Phenomenon is Your Hypothesis

• Consider only two variables, not special conditions– What makes it too broad?

• E.g., your research question “what factors affect X” is too broad

• E.g., some focused research questions: “X affects Y, Z, A, B etc” can be too broad

– If it is too narrow you won’t be able to do the rationale assignment

• E.g., x affects y in the presence of z (i.e., finding counter evidence will be hard!)

What makes a hypothesis Good & Narrow?

• Define both variables in hypothesis

1. Give Examples of items of variables2. List features variables3. Explain how both variables in hypothesis are

different from & similar to other, closely-related variables

• Can have either theoretical/empirical sources for definitions of variables in hypothesis– How are definitions similar/different from

methods of measurement?

Differences & Similarities to your Phenomenon: An Overview

1. Give Examples of items

2. Define variables by features– Focus on what the variable is rather than what

are its effects/causes? • E.g., Identify features/dimensions/of job satisfaction

rather than its causes & effects – see lecture notes on Development Index Article for examples

• E.g., Extraversion is – Being socially out-going

– Having a lot of positive affect

2 things to do when defining variables…

3. Compare & contrast variables effectively– E.g., How is job satisfaction different from AND

similar to organizational commitment? NOT how is job satisfaction different & similar to marital satisfaction – poor counter example

– E.g., What is extraversion? How is it different from agreeableness? NOT how is extraversion different from introversion? Introversion is low extraversion!

1 more thing to do when defining variables…

1. List dimensions/features of the variable

2. Give example items/measures that illustrate the variable

3. Identify the contrasting variable that can be confused with examples of the concept

Summary:To define a variable well, you need to…

• Can help you evaluate whether the evidence you found is relevant– More later on relevance

• Can help you understand why there is counter evidence– More later on how you can accomplish this

Why do definitions this way?

What we covered so far

• What is the phenomenon?• Description of 2-variable hypothesis

• How is it different/similar to another phenomenon?• Define BOTH variables• Give effective counter-examples of variables

What’s next

• When/where/how is it exhibited? Why?

• When is it not? Why?

• Exhibition means providing ‘evidence’ which relies heavily on good sources

• What are good empirical sources?– Have reliable & valid facts

• Implications of not having reliable & valid facts to support your counter hypothesis (covered later)

– Logical explanations

Good sources for getting evidence for your hypothesis & counter hypothesis

• At least – 2 good empirical sources to support hypothesis– 1 good empirical source that does not support

your hypothesis • More on what kind of evidence is regarded as NOT

supporting your hypothesis later..

Number of sources needed for your hypothesis & counter hypothesis

• Give specific, relevant & sufficient detail about sources supporting hypothesis & counter hypothesis

• Describe results/methods of source, Focus on relevant findings/methods

• Avoid describing conclusions, describe evidence

– E.g., N? Measures used? Size/direction of correlation? Difference in mean? Significance level? Establish Relevance when describing

Descriptions of sources

• Surveyed 369 managers in 83 business units by using Spreitzer’s 12-item questionnaire for empowerment & Rusbult’s 6-item questionnaire for job satisfaction [and found]....a significant positive correlation [.72] between psychological empowerment and job satisfaction

Example good description Student draft, Spring 2006

• Explaining is different from describing

• Explain why & how empirical findings support pro-hypothesis– Establish relevance of empirical finding to your

hypothesis– Explain why one thing is related to another

• Convert hypothesis into focused research question and answer it

– E.g., extraversion is related to non verbal skills (hypothesis) to WHY is extraversion related to non verbal skills (focused research question)

Why is your hypothesis supported

Extraversion Amount ofSocial experiences

+ NV Skills+

• Extraverts are more skilled at non verbal communication because they have different amounts of social experiences (Student draft Fall 05)

Example explanation of evidence for hypothesis

• Use additional sources to explain findings (e.g., theoretical review) if sources w/good reliable & valid data don’t explain findings well– E.g., Medical Journals

A tip on using sources to explain

What we covered so far..what’s next

• What is the phenomenon?

• How is it different/similar to another phenomenon?

• When/where/how is it exhibited? Why?– Getting good evidence for your hypothesis,

establishing relevance of evidence for your hypothesis, giving a good explanation

• When is it not exhibited?

• Why is not exhibited?

• What is non-support for your hypothesis (i.e., what is your counter hypothesis?)– Directly counters the relation specified in your

hypothesis• E.g., “extraversion is not related to non verbal skills” is

the STRONG counter hypothesis

• “General Ability is related to non verbal skills” is the WEAK counter hypothesis

When is your phenomenon not exhibitedA.K.A. when is your hypothesis not supported

• Don’t use a straw-man type source to support your counter-hypothesis

• Meta-analyses are good sources for getting evidence for your counter-hypothesis– How to find meta-analyses in psycinfo

Good sources for getting evidence for your counter hypothesis

• Describe relevant parts of the evidence you found that – Supports your hypothesis – Counters your hypothesis

• If you didn’t find either yet (as specified in course timetable), set goals on how you will go about finding it– e.g., What terms to use in electronic database? (i.e.,

what are your definitions of your variables?)

Discuss with peer & write down goals to improve

What we covered so far..what’s next

• What is the phenomenon?

• How is it different/similar to another phenomenon?

• When/where/how is it exhibited? Why?

• When is it not exhibited?

• Why is not exhibited?

After describing the reliable counter evidence …e.g., Participation does not lead to setting more difficult goals

1. Discuss explanation for pro-hypothesis finding

2. Demonstrate its limitations• Assumptions, definitions/measurement, nature of

sample etc.

3. State the more valid explanation and illustrate its effectiveness

Explaining the counter evidence

• E.g., Participation does not lead to setting more difficult goals

1. Discuss explanation behind pro-hypothesis• E.g., Participation results in more difficult goals being

set because subordinates want their supervisors to believe that they are highly capable and therefore choose more difficult goals than those that may be assigned to them by the supervisor

C24 Fall05 Student Paper

1st step to explaining the counter evidence

2. Demonstrate limitations of explanations for the pro hypothesis with counter-evidence

• With assumptions made• E.g., the explanation for participation leads to more

difficult goals assumes that supervisors do not know the abilities of the subordinates and so assign easy goals

C24 Fall05 Student Paper

2nd step to explaining the counter evidence

3. State the more empirically valid explanation and illustrate its effectiveness

– E.g., When supervisors know the abilities of subordinates, participation does not result in more difficult goals

C24 Fall05 Student Paper

3rd step to explaining the counter evidence

After describing the reliable counter evidence …e.g., Participation does not lead to setting more difficult goals

1. Discuss explanation for pro-hypothesis finding

2. Demonstrate its limitations• Assumptions, definitions/measurement, nature of

sample, etc.

3. State the more valid explanation and illustrate its effectiveness

Another way to explain the counter evidence

ExtraversionIn Total

Social experiences+ NV Skills+

ExtraversionAs Positive Affect

Social experiences- NV Skills-

Using definitional differences to demonstrate the limitations of pro-hypothesis explanation

AgreeableExtraverts

Social experiences+ NV Skills+

Using nature of sample differences to explain the Counter Evidence

(aka using Moderator Variables)

Disagreeable Extraverts

Social experiences- NV Skills-

• State the more empirically valid explanation and illustrate its effectiveness

– e.g., agreeable extraverts have more social experiences whereas disagreeable extraverts have fewer social experiences

– e.g. when extraversion is measured as frequency of positive affect, it is not related to Non Verbal Skills

C24 Fall05 Student Paper

After using definitional differences or nature of sample differences to

explain the counter evidence

• Explain why & how counter evidence…– Supports the counter hypothesis

• Does it test assumptions made by pro-hypothesis explanations ?

– E.g., Participation does not increase goal difficulty when supervisors know abilities of subordinates

• Does it differ in definitions? Measures used? Size/Nature of sample? Context of study? Etc.?

– Relevance of detail given

Summary of what to do toexplain the counter evidence

• When, where & how is it not exhibited? Why? • Describing evidence against the hypothesis &

explaining why

• What common process explains its occurrence AND non occurrence?

• Why might your hypothesis not be supported given you will use valid measures and a large enough sample size?

What we covered so far…what’s next

• Differences must be explained by common explanation– Can identify differences in studies & relate to

meaningful variable within phenomenon• e.g., agreeable extraverts have more social experiences

whereas disagreeable extraverts have fewer social experiences

• e.g. when extraversion is measured as frequency of positive affect, it is not related to Non Verbal Skills

What explains the differences in evidence for & against hypothesis?

• Explaining valid counter evidence should lead you to answer how the two pieces of evidence can be reconciled with current/future research=creative insight & good analytical skills!

Why explain the differences for & against hypothesis using valid evidence?

• How & why does the evidence you found so far– Supports your hypothesis – Counters your hypothesis

• NOTE: describing is different from explaining

• If you don’t know the answer set goals on how you will go about answering this question

• Why might your hypothesis not be supported given you have reliable & valid measures and a large enough sample size

Discuss with peer & write down goals to improve

1. What is the phenomenon? • Description of 2-variable hypothesis

2. How is it different & similar to another phenomenon?

• Definition of variables

3. When, where & how is it exhibited? Why?• Describing evidence for the hypothesis &

explaining why

Summary of what is involved in “rationale assignment”

4. When, where & how is it not exhibited? Why?

• Describing evidence against the hypothesis & explaining why

5. What common process explains its occurrence vs. non occurrence?

• Why is there a difference in the findings?

6. Why might your hypothesis not be supported given you will use valid measures and a large enough sample size?

Summary of what is involved in ” rationale assignment” (cont’d)

Research Designs

Measurement

Data Analyses

Field Studies

Sample Survey (Meta-analysis)

Scaling

CommunicateLogic & Results

Written Assignments

What we covered so far…what’s next