SCIENTIFIC KNOWLEDGE & HOW TO BUST MYTHSDesign Factory13.5.2014

Me

• Doctoral candidate at Aalto ENG• M.Soc.Sci in administration and

organization research(University of Helsinki)

• Working title of dissertation:”Inter-organizational interactions in innovation processes in the real estate and construction sector”

lauri.pulkka(a)aalto.fi

After today’s session you should

• understand the difference between scientific and non-scientific knowledge;

• recognize different methods of data collection and analysis; and

• be able to apply the basic principles of scientific knowledge creation to busting myths

This is how we’re going about it

1. We’ll start off with a 1-slide introduction to scientific knowledge and how it differs from it’s non-scientific counterpart, and

2. draft a research plan based on one of last year’s myths: “new is better than the old”

– in pairs– in sequence

PART I: 1-SLIDE INTRO TO SCIENTIFIC & NON-SCIENTIFIC KNOWLEDGE

Scientific and non-scientific knowledge

Scientific Non-scientificGeneral approach Empirical IntuitiveObservation Controlled UncontrolledReporting Unbiased BiasedConcepts Clear definitions Ambiguous definitionsInstruments Accurate InaccurateMeasurement Reliable & repeatable Non-reliableHypotheses Testable UntestableAttitude Critical Uncritical

Adapted from: Research Methods In Psychology (Shaughnessy & Zechmeister 1990, 6)

Example: how the Mythbusters do it

http://www.youtube.com/watch?v=YjzgsLqaWNU

• Empirical approach?• Controlled observation?• Unbiased reporting?• Clear definitions?• Accurate instruments?• Reliable & repeatable measurements?• Testable hypotheses?• Critical attitude?

PART II: HOW TO DRAFT A RESEARCH PLAN

One way to draft a research plan around ”new is better than the old”

Define concepts

Delineate and focus

Working hypotheses

Empirical approach

Results

Limitations

Relevance

Does our delineation introduce new concepts?

Does prior knowledge change our research question?

What kind of methodological considerations exist?

What kind of limitations do we recognize and how could we minimize them?

Unfortunately it’s not that straightforward

But we’ll still do it this way;Outline of today’s exercise

Define concepts

Delineate and focus

Working hypotheses

Empirical approach

Results

Limitations

Relevance

2. Produce a research question to test the myth. Explain it to another pair

4. What do you assume and how do you plan on testing your assumptions? Explain to a different pair

6. What kind of results do you expect and how do you know you did it right? Explain to a different pair

7. Discuss in a group why your research is relevant and to whom

1. Minilecture: about concepts and delineation

3. Minilecture: hypothesising; data collection and analysis methods

5. Minilecture: reliability, validity, ethics

Define concepts: what are we dealing with here

• Ideally a concept is a shared meaning for a specific thing– Concise, precise, clearly articulated, unambiguous– Operational vs. descriptive definition

• Concepts should be revisited multiple times during the research process– You may likely find or come up with better definitions

Delineate and focus: because you can’t research the entire universe

• Object of study– What are you interested in? New vs. old…

…things?…people?…technology?…social systems?

– In which context?– Better in which way?

resiliencedurabilityaesthetics …

Outline of today’s exercise

Define concepts

Delineate and focus

Working hypotheses

Empirical approach

Results

Limitations

Relevance

2. Produce a research question to test the myth. Explain it to another pair

4. What do you assume and how do you plan on testing your assumptions? Explain to a different pair

6. What kind of results do you expect and how do you know you did it right? Explain to a different group

7. Discuss in a group why your research is relevant and to whom

1.Minilecture: about concepts and delineation

3. Minilecture: hypothesising; data collection and analysis methods

5. Minilecture: reliability, validity, ethics

Define concepts, delineate, and draft a research question to test the myth

• You can discuss for example– What do you mean by old and new?– What part of the myth do you want to focus on?– What new concepts does your perspective introduce?– What question needs to be answered to test your

perspective on the myth?

• Make notes for the discussion• Write your research question on a post-it

Outline of today’s exercise

Define concepts

Delineate and focus

Working hypotheses

Empirical approach

Results

Limitations

Relevance

2. Produce a research question to test the myth. Explain it to another pair

4. What do you assume and how do you plan on testing your assumptions? Explain to a different pair

6. What kind of results do you expect and how do you know you did it right? Explain to a different group

7. Discuss in a group why your research is relevant and to whom

1.Minilecture: about concepts and delineation

3. Minilecture: hypothesising; data collection and analysis methods

5. Minilecture: reliability, validity, ethics

Working Hypotheses

• A hypothesis is a proposed explanation for a phenomenon

• A hypothesis has to be testable to be scientific• Non-testable hypothesis: “All men are mortal”• Testable hypothesis: “All men who were born

before 2014 are mortal”• Theoretical and practical constraints• Preferably based on theory, but can be less rigid

Empirical approach

Data collection method Used in (for example) / some considerationsExperiment Natural sciences; psychology; behavioral science

+ codified data, unambigious, objective- can’t fix it afterwards

Observation Anthropology; biology; organization research+ easy to set-up; real-world data- can be difficult to interpret

Action research Business; pedagogy+ has a practical impact- biased

Survey All social sciences+ enables large N-studies- bad for why questions

Interview All social sciences+ good for why questions- lot’s of data, low generalizability

H: “Asphalt is a slow-moving liquid”

• Cool experiment: http://www.nature.com/news/world-s-slowest-moving-drop-caught-on-camera-at-last-1.13418

Outline of today’s exercise

Define concepts

Delineate and focus

Working hypotheses

Empirical approach

Results

Limitations

Relevance

2. Produce a research question to test the myth. Explain it to another pair

4. What do you assume and how do you plan on testing your assumptions? Explain to a different pair

6. What kind of results do you expect and how do you know you did it right? Explain to a different group

7. Discuss in a group why your research is relevant and to whom

1.Minilecture: about concepts and delineation

3. Minilecture: hypothesising; data collection and analysis methods

5. Minilecture: reliability, validity, ethics

Form hypotheses; think about ways to test them?

• Write your hypotheses and research setting on separate post-its

• What do we assume is the answer? Why?• Can we test it? How?

Outline of today’s exercise

Define concepts

Delineate and focus

Working hypotheses

Empirical approach

Results

Limitations

Relevance

2. Produce a research question to test the myth. Explain it to another pair

4. What do you assume and how do you plan on testing your assumptions? Explain to a different pair

6. What kind of results do you expect and how do you know you did it right? Explain to a different group

7. Discuss in a group why your research is relevant and to whom

1.Minilecture: about concepts and delineation

3. Minilecture: hypothesising; data collection and analysis methods

5. Minilecture: reliability, validity, ethics

Reliability and validity

Source: William M.K. Trochim (http://www.socialresearchmethods.net/kb/relandval.php)

Research ethics (Aalto ethics committee)Ethical principles (NABoRE)

1. Respecting the autonomy of research subjects– Voluntary participation– Information for subjects

2. Avoiding harm– Mental, physical, social, financial harm

3. Privacy and data protection– Reporting– Data handling and storage

Outline of today’s exercise

Define concepts

Delineate and focus

Working hypotheses

Empirical approach

Results

Limitations

Relevance

2. Produce a research question to test the myth. Explain it to another pair

4. What do you assume and how do you plan on testing your assumptions? Explain to a different pair

6. What kind of results do you expect and how do you know you did it right? Explain to a different pair

7. Discuss in a group why your research is relevant and to whom

1.Minilecture: about concepts and delineation

3. Minilecture: hypothesising; data collection and analysis methods

5. Minilecture: reliability, validity, ethics

After today’s session you should

• understand the difference between scientific and non-scientific knowledge;

• recognize different methods of data collection and analysis; and

• be able to apply the principles of scientific knowledge creation to busting myths

Online resources

• How to conduct scientific experiments:– http://

www.wikihow.com/Conduct-a-Science-Experiment– http://

www.sciencebuddies.org/science-fair-projects/project_scientific_method.shtml#overviewofthescientificmethod

• Qualitative methods for social sciences– http://

www.socialresearchmethods.net/kb/contents.php

Challenge breakers course topics, critical review

• Golf technology improves game play? Or is it a money machine?• Are gasoline explosions in the movies real?• Lightning storms• Are doors as fragile as in the movies?• Do electronics have an expiry date? Are they designed to break?• Can parts of a car be used as shields for bullets?• Can satellites be used to spy on people like on movies? / Can the ginger

bread house on the tale of Hansel and Gretel exist?• Twin tower conspiracy, can a plane crash do all that damage?• Hollywood fiction• Magical diets fact or fiction / sale hoax? Flying with balloons or

umbrellas possible?• Can you Fly with many Vappu Balloons?• Does higher tire pressure make a bike go faster?