G54REM

The Scientific Method

REM Lecture 2 Dr. Julie Greensmith

G54REM

Lecture OverviewThe development of the scientific method

Inductive versus deductive practices

Theory, empiricism and certainty

Hypotheses

Designing experiments

Results can be positive or negative

Challenges for science

G54REM

Learning OutcomesTo understand what motivated the scientific method, and the accepted processes

To understand what is a hypothesis

An appreciation of experimental design

An awareness of the challenges facing science

G54REM

In the beginningThe study of the universe is really ancient

Astronomy, weather, seasons, medicine, anatomy, botany, alchemy

‘Natural philosophers’ doubted that natural phenomena were attributed to the actions of the gods

Facilitated by the written word vs the oral tradition

Evidence across many civilisations across the world

Egyptian, Indian, Babylonian, Chinese

G54REM

Grecian and Roman Science Thales and Aristotle

Greek natural philosopher Thales one of the first recorded ‘scientific enquirers’

Socrates and Plato laid foundations for reasoning, logic and enquiry - and wrote it down.

Aristotle arguably the most influential and his mode of thought dominated dominated for more than 1000 years

Classification of animals, movement of planets, dreams, psychology, geology, deduction, ethics, poetry

G54REM

The Dark AgesWestern Europe regressed

Only religious scholars

Much of the learning of the ancients lost

Plato and Aristotle believed as truth and fact

Superstition and the oral tradition

G54REM

The Enlightenment The Age of Reason (1685-1815)

The Scientific Revolution

Went against the teaching of the establishment and religion, questioned the status quo

Rediscovery of classical knowledge

As a result of persecution, secret societies were formed

G54REM

Francis Bacon 🥓 Credited with the ‘Baconian Method’ (1620)

Introduced inductive methodology

Testing and refining hypotheses by observing, measuring, and experimenting

Described in his opus magnum the Novum Organum - new instrument

Argues that there are progressive stages of certainty which are achieved through cycles of empirical observation and induction

G54REM

“Man, being the servant and interpreter of Nature, can do and understand so much and so

much only as he has observed in fact or in thought about the order of Nature: beyond this he neither

knows anything nor can do anything”

–Francis Bacon “Aphorism I” Novum organum

G54REM

Sir Issac Newton 🍎Controversial, argumentative, mathematician, philosopher, alchemist, politician, economist, physicist and (perhaps) personality disordered

“Making pies on Sunday night... punching my sister... threatening my Father and Mother Smith to burn them and the house over them”

“Plato is my friend, Aristotle is my friend, but my best friend is truth.”

Greatest contribution in the form in his own theory of calculus the “Philosophiae Naturalis Principia Mathematica”

The three laws of motion and the first rigorous account of his theory of universal gravitation

G54REM

Karl Popper Influential modern science philosopher

The problem of demarcation - distinguishing scientific from non-scientific theories

Development of verifyability criterion

“a statement is cognitively meaningful if and only if it is, in principle, possible to verify”

Scientific theories have potential falsifiers, that they make claims about the world that might be discovered to be false

Non-scientific theories do not have falsifiers, where there is literally nothing which could potentially invalidate the theory

G54REM

The Scientific MethodMake an observation {from the universe or from a body of existing literature}

Form a hypothesis

Conduct and experiment to test the hypothesis

Analyse data to accept or reject the hypothesis

Draw conclusions

G54REM

Basic Scientific Methodology

Methods: tools, strategies, techniques used to conduct science

Methodology: the study of how research is performed

G54REM

Assumptions of ScienceThe Universe operates according to stable underlying laws

e.g. the nature of gravitational forces

Our senses give us accurate information about the world - empiricism

e.g. we are not living in some simulated reality

Simplicity is preferable - Occam’s Razor

G54REM

Reasoning 🐑 🐑 🐑You see a field of white sheep

Inductive reasoning states:

“All the sheep I see in this field are white, therefore all sheep are white”

Deductive reasoning states:

“The law of sheep states that all sheep are white and as the animals in the field are sheep, therefore they are all white”

G54REM

Inductive vs Deductive Induction and deduction can both be used to convince others of a truth or to falsify some proposed idea

Induction infers the general case from a set of specific observations, depending on the cases being typical of a general form - science

Deduction is the act of reaching a conclusion by showing that such a conclusion must follow from a set of premises - logic

G54REM

Scientific TheoriesWhen do scientific observations become theories?

The general usage for the word theory means potentially “I have an intuition”

In science the word theory has a more specific meaning

E.g. theory of special relativity; theory of evolution; the big bang theory; germ theory

When something has such overwhelming evidence that it is accepted as (almost) a fact

In maths a ‘theorem’ (related word) also has special meaning

G54REM

Continuum of CertaintyDefinitely False Definitely True

G54REM

Continuum of CertaintyDefinitely False Definitely True

1 + 1 = 3 Fermats last theorem sqrt(2) = a/b as integers

1+1=2 Pythagoras Logical deductions

G54REM

Derive Hypothesis and Assign Certainty

Science assigns probabilities to hypotheses which are updated with new knowledge

Limiting the amount of assumptions

Can never be totally certain, but satisfied above a threshold of evidence

A theory in science is a strongly supported hypothesis which can be used to make predictions and can also be falsifiable

G54REM

What is a hypothesisA specific way of framing a research question

We looked at this on Friday - some groups appeared to grasp this better than others…

Make a prediction about a phenomena which is verifiable or falsifiable through testing

To eliminate bias by avoiding a predetermined outcome

G54REM

Constructing a Hypothesis

A hypothesis should be:

Testable using a methodology

Results should be verifiable or falsifiable

The results to prove or disprove the hypothesis should be reproducible

G54REM

The Null HypothesisA test for the status quo

Changing a system/component/entity will have no observable effect on the results

e.g administration of propranolol will have no significant effect on the blood pressure of the patient

G54REM

The Alternative Hypothesis

Seeks to express what will happen if we predict an effect on a system

E.g. administration of propranolol will significantly reduce the blood pressure of the patient

One sided vs Two sided tests

(we will cover this more in quantitative methods)

G54REM

Controlled experimentsAn experiment in which everything is held constant except for one variable

Involves the use of a control group

Eliminate as much uncertainty as possible within a specific experiment

The more control you have in your experiments, the easier it is to convince others of the significance of your results

G54REM

Parameters and Assumptions

Sometimes its not possible to control all variables - especially true in psychological experiments

Have to make assumptions - these must be explicitly communicated when writing about an experiment

G54REM

Experimental DesignAppropriate design depends on your hypothesis

E.g. running a clinical study into the effects of a new drug

Randomised controlled trials preferable

Groups receiving the experimental treatment are compared with control groups receiving no treatment (placebo) or a previously tested treatment (a positive-control study)

How might this apply in Computer Science or HCI?

G54REM

Measurement issuesThe act of observing a system changes the behaviour of the system

e.g. if users know their web browsing behaviour is being monitored, they will not express their ‘natural’ behaviour

Measurement error: to be aware of the amount of error in measurement to eliminate bias from the results e.g. false positives

G54REM

Getting positive resultsWe all like to get positive results, to show that our experiment was right

Tests to show that we are in ‘less doubt than before’, towards understanding our chosen phenomena

Relies on having chosen the correct methods, performing the correct number of experiments, and choosing the most appropriate statistical tests

Rarely become a ‘fact’, just adds more evidence towards a case being accepted as a theory

G54REM

Don’t sweat negative results

Negative results is where you could not show any significant difference to reject the null hypothesis

Often just as useful as it prevents others from potentially pursuing a fruitless line of enquiry

Can be very disheartening as we all want that ‘eureka’ feeling

Publication bias towards positive results

G54REM

Replication studies are very useful too

Must perform and document studies in a way in which experiments can be reproduced

Replication studies aim to reproduce the results of another to check, verify and to add weight to an assertion - its how theories are made

Might seem ‘boring’ but are essential to verifying claims

Many instances of scientific misconduct where replication studies have disproved ‘faked papers’ *

* an MSc project is great for replication studies

G54REM

Drawing conclusionsConclusions should be specific to the evidence presented in the research

Generalise and highlight the relationship to other related results and research

Derive novel questions which have arisen from the results of the experiments

Often overstated ‘deduction’ style claims made in papers

view such claims with healthy skepticism

G54REM

Big QuestionIs ‘Computer Science’ a real

science?

G54REM

Challenges for scienceOnly 52% of AAAS scientists say that today generally a good time for science (down from 79% in 2009)

e.g. A majority of the US public (57%) says that genetically modified (GM) foods are generally unsafe to eat, while 37% says such foods are safe; by contrast, 88% of AAAS scientists say GM foods are generally safe.

Science biased by corporate funding and underfunding by government agencies

G54REM

Battling information overload

Both scientists and the public are bombarded with both information and misinformation

Exponential growth of scientific publications, fuelled by online publishing, pay-for-publication venues

Wikis, forums, whitepapers, videos, podcasts

G54REM

Knowing your sourcesGet to know the prominent journals and books in a field - the ‘old way’

Do you read and article and question the sources?

Readers only tend to question sources when they disagree with a fact or opinion

News feeds edited for articles similar to your own views - not to mention fake news and academic misconduct

Avoid blindly believing tertiary sources as ‘fact’, including questioning your own beliefs

G54REM

Lies, Damn Lies and Statistics

Beware of the presentation of ‘statistics’ without questioning

Shampoo adverts in which “87% of women agree” - but they only had 50 test participants

A DNA match for a murder being “One in a Million” chance - 64.1 million people in UK. $ = 64 people (not including visitors!)

Have a positive test for a rare disease (1 in 10000), but the test is 99% accurate - which is the more likely?

G54REM

It's okay to not know about something, or to say "I don't know" when put on the spot. That

was one of the things I found hardest to do

Not everything you read is correct or valid. Critically assess other peoples research to

inform yours.

Study simple cases before going general. But once you've done the general case don't forget that most people need to see the simple cases first to understand what is going on. I think

that some people are worried that their ideas will seem trivial if they reveal the simple

cases that motivated and explain them. Not true - simple is better!

G54REM

A lot of science is about using a structured approach (can be qual or quant) to answer a set question. It is very much the process, and if you get a negative result but have done the process right you've still learned something.

Research often starts with a curiosity and bunch of questions in hand. If you can

answer some of those and end up having even more questions than before, then you have really done well - sustainable research

The point of research is to find stuff out and tell people. Without doing the latter

*effectively* it's just playing.

G54REM

“Disappointment is when a beautiful hypothesis is spoiled by an ugly fact”

–Sir Issac Newton