empirical methods – g. grote ethz, fall08 obectives of the course basic knowledge of social...

Empirical Methods – G. Grote ETHZ, Fall08

Obectives of the course

• Basic knowledge of social science methods (design, data collection and analysis)

• Ability to assess quality of empirical research• Ability to design empirical studies in an

organizational context• Ability to evaluate/hire external experts for

carrying out empirical studies in organi-zations


Examples of social science methods used in organizational contexts• Interview

– Expert interview

– Performance appraisal

• Questionnaire– Customer survey

– Personality test

• Observation– Communication analysis

– Shadowing

• Archival data– Meeting protocols

– Personnel data


Special issues in social research

• Control of complexity– Constraints on manipulation of study conditions– Studying "hypothetical constructs"– Limited possibilities for reducing complexity

• Studies with humans– Effects through researcher / researched individual and social

embeddedness– Ethical principles: voluntary participation, informed consent, no

physical/psychological harm, confidentiality

It is all about grey zones, there is no black and white Building competence for navigating in these grey zones.


Special issues for social research in organizational contexts

• Objectives for study by different stakeholders in the organization

• Role of people carrying out the study

• Role of people being studied

• Distribution of control over use of results


Empirical methods: Overview

Top ic Tutorial 1 Tutorial 2 Tutorial 3 Lecturer

17.9. Introduction Grote

24.9. Study design Grote

1.10. Definition of variables Grote

8.10. Methods for data collection and analysis Grote

15.10. Questionnaire: Design of questions and response formats Belting

22.10. Data analysis: Types of data and statistical tests Belting

29.10. Interviews: Overview Belting

5.11. Example: Expert interview Belting

12.11. Guest speaker Belting

19.11. Observation: Overview Kolbe

26.11. Observation: Data collection and analysis Kolbe

3.12. Data collection for tutorial 3: Non-technical skills for team coordination Kolbe

10.12. Example: Interaction in high-risk teams Kolbe

17.12. Preparation for exam All


Textbook

• Darren Langdridge, Introduction to Research Methods and Data Analysis in Psychology. Pearson Prentice-Hall, 2004. ISBN 0-13-097832-9


Organization of course

• 3 ETCS points (approx. 75-90 work hours).• Besides the lecture, the prerequisite for credits

points and exam participation is the completion of three tutorials.

• Tutorials are worked on in groups of 2-4 students, there is one tutorial per course part (study design; survey&quantitative analysis; observation)

• The exam is written and open book; provisional date: Jan. 7, 2009, 13:15-14:45. Overall grade: 50% tutorials & 50% exam.

• Material for the lecture at www.oat.ethz.ch.


Basic research design

Independent variable/sConditions/Causes

Dependent variable/sEffects

Intervening variable/sConfounding variable/s


Example

• How are innovation, technology use and firm performance related?– What is cause and effect?– What are important intervening variables?


Steps in empirical research

• Define general question of interest• Draw on existing knowledge to develop

specific questions/hypotheses to be studied

• Determine design of study• Data collection• Data analysis• Interpretation/conclusions regarding initial

questions/hypotheses


Developing a study design (1)

• Deciding on overall objective:– Testing hypotheses (incl. evaluation)– Generating hypotheses– Describing a population– Testing methods

• Defining variables of interest– Analytical definition– Operational definition

determining methods for data collection


Criteria for good data collection methods

• Reliability: stability of measurement– Internal consistency– Retest– Split half

• Validity: measuring what is intended to be measured– Face validity– Criterion validity: concurrent, predictive– Construct validity


Developing a study design (2)

• Determine sample – Type of sample: Random, stratified,

cluster, convenience, etc.– Size of sample– Case study

• Determine methods for data analysis

• Planning resources

• Internal and external validity check


Criteria for good study design• Internal validity = unambiguous interpretation of

results/few alternative explanations for results– Possible problems: Uncontrolled influences by e.g.

external events, selection of participants, repeated measuring.

• External validity = generalizability of results regarding other samples of people, situations, time– Possible problems: e.g. special character-istics of

participants, reactive measurement


Reporting social science research• Overall objectives

– Consistent, transparent documentation of research process– Building trust in interpretation/conclusions

• Structure of report– Management summary– Introduction/rationale for study– Motivation for specific questions studied based on existing

knowledge– Methods: sample, data collection and analysis– Results– Interpretation/Conclusions– Appendix: Instruments used, raw data etc.


Tutorial 1

• Please read the article and write a short critique (ca. 2-3 pages) of the article using the steps of study design, validity criteria and reporting guidelines presented in the lecture.

• You may work on this tutorial alone or together with another student.

• The article:Brown, S.L. & Eisenhardt, K.M. (1997). The art of continuous change: linking complexity theory and time-paced evolution in relentlessly shifting organizations. Administrative Science Quarterly, 42, 1-34.

• Deadline: Oct. 6, noon, hand in to [email protected]


Types of variables

• Role in study design– independent– dependent– intervening

• Level of measurement: What do the numbers mean that we assign to a variable?– discrete variables: nominal– continuous variables: rank, interval, ratio


Identify different types of variables:

• "Larger firms are more innovative."• "Larger firms are more innovative in growth

markets."• "In open plane offices compared to cell offices

frequency of communication increases and duration decreases."

• "Higher levels of task uncertainty demand more personal forms of coordination."

• "Successful strategy implementation can only be top-down in stable markets."


Analytic and operational definition of variables

• Analytic definition:

Content/meaning of variable

• Operational definition:

Measurement of variable


Exercise on defining variables

• Are part-time employees more discriminated against in smaller companies?

• Does communication technology improve performance of software development teams?

• Does liberalization of energy markets decrease energy prices?

Which are the independent, dependent, and inter-vening variables? Provide analytic and operational definitions for all variables. What level of measure-ment do you suggest for each variable?


Quantitative versus qualitative approaches to social science research

• Quantitative approaches: measuring and counting phenomena+ precise, controlled, claims on causation/prediction

- oversimplification, no recognition of subjectivity/ individuality

• Qualitative approaches: capturing qualities of phenomena+ recognition of subjectivity/individuality, openness

- difficult to establish internal and external validity, difficult to separate data analysis and interpretation


The seminar ... (from ASQ, 1979)

• Qualitative researcher: "Many people these days are bored with their work and are ..."

• Quantitative researcher (interrupting): "What people, how many, when do they feel this way, where do they work, what do they do, why are they bored, how long have they felt this way, what are their needs, when do they feel excited, where did they come from, what parts of their work bother them most, which ..."

• Qualitative researcher: "Never mind."


Overview of methods for data collection

• Archival data+ non-reactive, "condensed organizational knowledge"

- not aligned with purpose of the investigation

• Questionnaire + objective, applicable for large samples

- no control over the actual data collection, response biases

• Interview + control over data collection, treatment of complex issues possible

- resource-intensive, interviewer influence

• Observation + access to implicit knowledge, natural situation

- subjective meaning of the observed unknown, no control over the occurrence of the events under study


Examples: How the questions shape the answers (1)

• "Does it seem possible or does it seem impossible to you that the Nazi extermination of the Jews never happened?" 33% responded with "possible"

versus

"Do you believe the Holocaust happened?" 89% yes, 3.5% no, 7% don´t know

• Taboo behaviors: don´t ask if but when the behavior has been shown in the past



Own behavior yes/no

Percent yes

Own behavior frequency scale 1-5

Percent 4/5

Correct behavior yes/no

Percent yes

Correct behavior scale 1-5

Percent 4/5 Avoid use of spray cans 84 11 72 83 Bio-shopping during the last fortnight

16 9 96 41

Separate glass collection 96 93 100 89 Separate paper collection 94 95 98 85 Separate aluminium collection

78 62 90 69

Use compost 64 38 100 83 Collect batteries 92 98 100 91 Return unused drugs to pharmacy

32 33 91 69

Reduce water when showering/taking a bath

40 31 66 63



Zur Anzeige wird der QuickTime™ Dekompressor „TIFF (Unkomprimiert)“

benötigt.

Zur Anzeige wird der QuickTime™ Dekompressor „TIFF (Unkomprimiert)“

benötigt.


Overview of methods for data analysis

• Overall objective of data analysis– Description– Reduction– Preparation for interpretation and

generalization

• Quantitative analysis– Descriptive statistics– Inferential statistics

• Qualitative analysis


Basics on inferential statistics (1)

• Overall objective:– Estimating the likelihood of results found in

a particular sample occurring in the relevant population

– Accuracy of this estimation depends on sample size and variance of variables studied

– Error probability is controlled for both adopting a false hypothesis and rejecting a correct hypothesis



• Steps:– Determining null hypothesis and alternative

hypothesis – Calculate estimate for relevant parameter

(e.g. mean)– Calculate and compare test statistic and

critical value– Decide for/against null hypothesis



• Select appropriate tests– Based on measurement level of variables

(nominal, rank, interval, ratio)– Based on distribution of variable

(e.g. normal distribution) – Based on research question

• Comparing means (two groups, e.g. t-test; more groups, e.g. analysis of variance)

• Testing associations (e.g. correlation, regression)

empirical methods – g. grote ethz, fall08 obectives of the course basic knowledge of social...

Documents

fall08 empirical methods

grote ethz

research methods

empirical studies

fall08 organization

overview slide

fall08 steps

fall08 obectives