Lecture 2: The Scientific Method

What is Science?

What is Science?

a) the state of knowing : knowledge as distinguished from ignorance or misunderstanding

b) a: a department of systematized knowledge as an object of study <the science of theology> b: something (as a sport or technique) that may be studied or learned like systematized knowledge <have it down to a science>

c) 3 a: knowledge or a system of knowledge covering general truths or the operation of general laws especially as obtained and tested through scientific method b: such knowledge or such a system of knowledge concerned with the physical world and its phenomena : natural science

d) 4: a system or method reconciling practical ends with scientific laws <cooking is both a science and an art>

http://www.merriam-webster.com/dictionary/science

The Scientific Methodthe process

• characterization of existing data

• formulation of a hypothesis

• deduction - formulation of a predictive test

• experimental testing

• error elimination and characterization

• validate or revise hypothesis

How to evaluate a report on a scientific work

• What are the assumptions?– Are they consistent with the conclusions?

• What is the data set?– How many trials?– Is the data public?

• Was the result peer reviewed?– By whom?– Has it been cited?

• Is the claim falsifiable?– Are there enough details to reproduce the result?

Values and Scienceis science good or bad?

• GOOD Science Things– Medicine and longer Life Spans– Better Transportation– Spam, Tang, and Artificial Food Products

• BAD Science Things– Atomic Bombs– Telephones and Computers– Spam, Tang, and Artificial Food Products

Is science good?

• Science is a process to investigate ideas• There are Ethical issues in the science

process– Human Subject and Animal Research– Bioethics and Medical Ethics– Scientific Integrity and Fraud

• Science is a tool that can be used ethically and unethically

• Science is NOT Technology

Random Dataimage one

Random Dataimage two

Combined

True random data has unusual clusters and patterns

What is Science?In-class problem 1

• Suppose each student in class flips a coin and they all end up heads. Can we conclude that the coins are fair? What would happen if we repeated this experiment 100 times and every time the same number of students got heads and the same number tails?

Pseudo-Sciences

• Astrology

• UFOs

• Cold Fusion

Astrologyvery old ideas

“THAT a certain power, derived from the ethereal nature, is diffused over and pervades the whole atmosphere of the earth, is clearly evident to all men. Fire and air, the first of the sublunary elements, are encompassed and altered by the motions of the ¾ther. These elements in their turn encompass all inferior matter, and vary it as they themselves are varied; acting on earth and water, on plants and animals.”

Ptolomy, Tetrabiblos 2nd Century AD (translated by J.M. Ashmand 1822)

Astrologya short history

• Celestial motions were used to measure time• It was clear that the things in the sky influenced

Earth– Planting and harvesting of crops were associated with

which stars were in the sky– Seasonal changes, including the flooding of the Nile,

was predicted from the stars– The position of the Moon clearly was related to tides

• Perhaps the positions of the planets affected the fates of Kingdoms and Kings

Modern Astrology

• Sun Sign Astrology– Related to the position of the Sun along the

Zodiac at your time of Birth– Twelve different constellations that have been

associated with personal characteristics

• Sun Sign Astrology can be tested

Typical Sun Sign Horoscopes

Sagittarius November 22 - December 21

For Monday, January 21 -Are you hungry for a new adventure? Of course you are! There is so much in life to discover, and today is just as good as any other day to start exploring! Travel is a great idea right now -- especially travel to a foreign country. Get online and start planning your next trip. If you don't have the time or the finances for such an elaborate expedition, then just grab some friends and check out a restaurant that serves food from a country you'd like to visit. Feed your wanderlust with food.

© Astrology.com 1996-2008

Typical Sun Sign Horoscopes

Capricorn December 22 - January 19

For Monday, January 21 -It won't be easy to get alone time today -- your presence will be much in demand from two different groups. So if you are craving some solo hours, you could be facing a tough challenge. If you can, just give in and get together with the people who are clamoring for your attention. There will be plenty of time in the coming days to go off on your own and think or relax. For now, you will need to try to find happiness in the energy generated when you're in the midst of people.

© Astrology.com 1996-2008

Typical Sun Sign Characteristics

Sagittarians

... have a positive outlook on life, are full of enterprise, energy, versatility, adventurousness and eagerness to extend experience beyond the physically familiar. They enjoy travelling and exploration, the more so because their minds are constantly open to new dimensions of thought. They are basically ambitious and optimistic, and continue to be so even when their hopes are dashed. Their strongly idealistic natures can also suffer many disappointments without being affected. They are honorable, honest, trustworthy, truthful, generous and sincere, with a passion for justice. They are usually on the side of the underdog in society they will fight for any cause they believe to be just, and are prepared to be rebellious. They balance loyalty with independence.

2008 - Astrology.comby Pamela McHenry, "StarGoddess"

Typical Sun Sign Characteristics

Sagittarians

... have a positive outlook on life, are full of enterprise, energy, versatility, adventurousness and eagerness to extend experience beyond the physically familiar. They enjoy travelling and exploration, the more so because their minds are constantly open to new dimensions of thought. They are basically ambitious and optimistic, and continue to be so even when their hopes are dashed. Their strongly idealistic natures can also suffer many disappointments without being affected. They are honorable, honest, trustworthy, truthful, generous and sincere, with a passion for justice. They are usually on the side of the underdog in society they will fight for any cause they believe to be just, and are prepared to be rebellious. They balance loyalty with independence.

2008 - Astrology.comby Pamela McHenry, "StarGoddess"

Typical Sun Sign Characteristics

Aquarians

...basically possess strong and attractive personalities. They fall into two principle types: one shy, sensitive, gentle and patient; the other exuberant, lively and exhibitionist, sometimes hiding the considerable depths of their character under a cloak of frivolity. Both types are strong willed and forceful in their different ways and have strong convictions, though as they seek truth above all things, they are usually honest enough to change their opinions, however firmly held, if evidence comes to light which persuades them that they have been mistaken. They have a breadth of vision that brings diverse factors into a whole, and can see both sides of an argument without shilly-shallying as to which side to take. Consequently they are unprejudiced and tolerant of other points of view. ...

2008 - Astrology.comby Pamela McHenry, "StarGoddess"

Testing Sun Sign Astrology

• Hypothesis: The zodiacal sign you are born under influences your career, health, and life

• Deduction: Certain objective elements, such as career, disease and personality should be correlated with when you are born

• Experiment: Study the careers, health, and personality types of people and see if they depend on birthday

Sun Sign Results

• There is no correlation between Birthday and career, health, and personality as measured in objective large studies

• Numerous studies show the same results

• There is no evidence that Sun Sign Astrology works

Astrologyhow to write a good horoscope

• Be Vague– ``Changes are happening in your life''

• Be Double-Headed– ``You have sound judgement, but sometimes make

terrible mistakes''

• Be Modal– ``At times, you find people very difficult''

• Be Favorable– ``Your intelligence can often be misunderstood by

others"

Perception and Science

• Humans are excellent at finding patterns, even when there aren't any

• It is very easy to let subjective perception overwhelm objective tests

• We need to make specific predictions before the experiment takes place

• We need to test the predictions using statistics

Other Sciencesfrom The USA Today,Jan 15, 2008

Existing Evidence

Hypothesis

Extended Hypothesis

Two Big Rules in Science

• The Burden of Proof is on the person MAKING the CLAIM.

• EXTRAORDINARY CLAIMS DEMAND EXTRAORDINARY PROOF

The Science Debate

• Every day, people are debating scientific issues

• Most scientific debates have two sides– Majority opinion – Minority opinion

Scientific Opinions

• Opinions held by Scientists in the Field

• Opinions held by the General Public

• Do opinions matter in science?

How do we resolve scientific disagreements?

• EVIDENCE and DATA– How was the experiment done?– Was it repeated by others?– Is the new theory TESTABLE?

Warning Signs in Public Debates

• Beware Citations to Authority– DATA is important, not words.

• Beware of Minority Opinion– Why don't most scientists believe it?

• Beware Magicians choice– Attacks where the other is weakest

Beware the Underdog Syndrome

• Everyone loves an underdog.– Everyone thought Galileo was wrong.– Everyone thinks I am wrong.– Galileo was right.– I might be right too.

http://www.alleenschule.teck.de/schueler/hotpot-h1-h5/cloze/under%20the%20table.jpg

Science and the media

• What role does the media play in science stories?

Science and the media

• The media reports new things - hence the name "news"

• New ideas in science are often released before peer review through press releases

• According to one study, about half of press releases in science end up being wrong!

• That's why we need peer review!

Science and the media

• The media cares about interesting stories - stories that are exciting, new and controversial

• All sources are ``equal'' in many science stories

• Not everyone on all sides of a debate have equal credibility

• Not all stories have equal credibility, and some with little credibility are very interesting

Cold Fusion

Cold Fusionall Physics Labs Try to Recreate the Experiment

• Measurements taken off Television

• Hundreds of Experiments begin with in days

Cold Fusioninitial results

• Some other labs detect Fusion

• Most other labs detect nothing?

• Utah creates ``Cold Fusion Institute''

• Multimillion dollar project

Cold Fusionwhat happened?

• Errors were found in poorly designed experiments

• There was no PEER REVIEW of the original results

• Original Results were Release in a Press Conference, not a Journal

Scientific Publications and Peer Reviewthe publication process

• A Scientist comes up with a new result• A Paper is written and submitted to a Journal• The Editor gives it to one or more anonymous reviewers

who provides feedback about the paper and the science• Basically, there are three possible results from the

reviewer– This is perfect! Publish it NOW!– This is crap! Don't publish it in this journal - EVER!– This is OK, but it needs some work. I want to see revisions.

• This gets more complicated when there are multiple reviewers

Scientific Publications and Peer Reviewthe publication process - revision

• The author gets the results, and usually makes revisions

• After the reviewer and author finish the revision cycle, the paper is put into proof form

• Final edits are made... and

• The paper is published

Scientific Publications and Peer Reviewthe publication process – published papers

• Once the paper has been published, other scientists read it

• If the paper is relevant to their work, the paper is cited in other papers written by other scientitst

• The impact of the paper is often judged by its number of Citations

• Different scientific fields have different standards

• In astronomy, a transformational paper could have thousands of citations

The broader impact of publicationsfuture directions in Science

• Publications and their impact are often used to judge the importance of scientific work

• The previous work helps determine the funding for projects – Science agencies have review panels, but previous work is an

important consideration in judging proposals– Science funding can be extremely competitive – Success rates are typically between 5\% and 25\% in most

programs

• Funding helps determine the future direction of research• So... peer reviewed publications drive science

The bottom line

• Peer review in Science is essential!

• Objective testing and unbiased review help keep scientific progress on track