science & scientific method. daimihenrik bærbak christensen2 literature [wikipedia, 2005]...

Science &Scientific Method

DAIMI Henrik Bærbak Christensen 2

Literature

[Wikipedia, 2005]– Scientific Method.

[Carter, 1996]– The Scientific Method.

[Zobel, 1997]– Writing for Computer Science, Chap 8. Springer-

Verlag 1997


Definition

Science is both a process of gaining knowledge, and the organized body of knowledge gained by this process.

The scientific process is the systematic acquisition of new knowledge about a system. This systematic acquisition is generally the scientific method, and the system is generally nature.

Science is also the scientific knowledge that has been systematically acquired by this scientific process.


Example

Science is the “opposite” of faith or belief. Often observations of the world can lead to false conclusions, or people generalize too quickly. People mistake truth for belief.

Classical trick: Which of the two horizontal lines is the longest?


Example

For centuries people believed in spontaneous generation. This was the idea that non-living objects can give rise to living organisms. It was common “knowledge” that simple organisms like worms, beetles, frogs, and salamanders could come from dust, mud, etc., and food left out, quickly “swarmed” with life.

For example: Observation: Every year in the spring, the Nile River flooded areas

of Egypt along the river, leaving behind nutrient-rich mud that enabled the people to grow that year’s crop of food. However, along with the muddy soil, large numbers of frogs appeared that weren’t around in drier times.

“Conclusion”: It was perfectly obvious to people back then that muddy soil gave rise to the frogs.


New ‘evidence’

Microscope in the 1600s: observation of bacteria growth in soup seemed to support spontaneous generation.

But – what exactly goes on?

“Life force” present in the molecules of all inorganic matter, including air and the oxygen in it, that could cause spontaneous generation to occur?


Scientific question

Question: – The scientist then raises a question about what (s)he

sees going on. The question raised must have a “simple,” concrete answer that can be obtained by performing an experiment.

– For example, “How many students came to school today?” could be answered by counting the students present on campus, but “Why did you come to school today?” couldn’t really be answered by doing an experiment.


Example question

Question: – Is there indeed a “life force” present in air (or oxygen)

that can cause bacteria to develop by spontaneous generation? Is there a means of allowing air to enter a container, thus any life force, if such does exist, but not the bacteria that are present in that air?


Hypothesis

Hypothesis

This is a tentative answer to the question: a testable explanation for what was observed. The scientist tries to explain what caused what was observed.


Example

Hypothesis: • There is no such life force in air, and a container of

sterilized broth will remain sterile, even if exposed to the air, as long as bacteria cannot enter the flask.


Aspects of Hypotheses

In a cause and effect relationship, what you observe is the effect, and hypotheses are possible causes. A generalization based on inductive reasoning is not a hypothesis. An hypothesis is not an observation, rather, a tentative explanation for the observation. – For example, I might observe the effect that my throat

is sore. Then I might form hypotheses as to the cause of that sore throat, including a bacterial infection, a viral infection, or screaming too much at a ball game.



Hypotheses reflect past experience with similar questions (“educated propositions” about cause) and the work of others. Hypotheses are based on previous knowledge, facts, and general principles. Your answer to the question of what caused the observed effect will be based on your previous knowledge of what causes similar effects in similar situations. – For example, I know that colds are contagious, I don’t

know anyone with a cold, I was at the ball game yesterday, and I was doing a lot of yelling while I was there, so I think that caused my sore throat.



Multiple hypotheses should be proposed whenever possible. One should think of alternative causes that could explain the observation (the correct one may not even be one that was thought of!) – For example, maybe somebody sitting near me at the

ball game had a sore throat and passed it on to me.



Hypotheses should be testable by experimentation and deductive reasoning. – For example, throat culture and other tests yielded no

signs of a bacterial or viral infection, I have no fever or other signs/symptoms, and the doctor says my vocal cords are “swollen” in a way that would indicate overuse.



Hypotheses can be proven wrong/incorrect, but can never be proven or confirmed with absolute certainty. It is impossible to test all given conditions, and someone with more knowledge, sometime in the future, may find a condition under which the hypothesis does not hold true.

– Example: Newtonian physics does not hold at relativist speeds…


Aspects of hypothesis

That a hypothesis cannot be proven to be correct only incorrect has some consequences.

First, you should actually try to falsify the hypothesis and if this fails repeatedly only then should your confidence in it grow.

It is generally difficult for people to be brutally critical about their own work!


Types of reasoning

– Inductive reasoning goes from a set of specific observations to general conclusions: I observed cells in x, y, and z organisms, therefore all animals have cells.

– Deductive reasoning flows from general to specific. From general premises, a scientist would extrapolate to specific results: if all organisms have cells and humans are organisms, then humans should have cells. This is a prediction about a specific case based on the general premises.


Prediction

Prediction:– Next, the experimenter uses deductive reasoning to test the

hypothesis.

Example: Prediction:

– If there is no life force, broth in swan-neck flasks should remain sterile, even if exposed to air, because any bacteria in the air will settle on the walls of the initial portion of the neck. Broth in flasks plugged with cotton should remain sterile because the cotton is able to filter bacteria out of the air.


Experiment/testing

Testing: – Then, the scientist performs the experiment to see if

the predicted results are obtained. If the expected results are obtained, they support (but does not prove) the hypothesis.

– In science when testing, when doing the experiment, it must be a controlled experiment. The scientist must contrast an “experimental group” with a “control group”. The two groups are treated EXACTLY alike except for the ONE variable being tested.


Example

Testing: – Wide-mouth jars each containing a piece of meat

were subjected to several variations of “openness” while all other variables were kept the same.control group — These jars of meat were set out without lids so the meat would be exposed to whatever it might be in the butcher shop.experimental group(s) — One group of jars were sealed with lids, and another group of jars had gauze placed over them.


Theory

A theory is a generalization based on many observations and experiments; a well-tested, verified hypothesis that fits existing data and explains how processes or events are thought to occur. It is a basis for predicting future events or discoveries. Theories may be modified as new information is gained. – This definition of a theory is in sharp contrast to

colloquial usage, where people say something is “just a theory,” thereby intending to imply a great deal of uncertainty.

Summary

from Wikipedia


Process

The essential elements of the scientific method are iterations and recursions of the following four steps:– Characterization – Hypothesis (a theoretical, hypothetical explanation) – Prediction (logical deduction from the hypothesis) – Experiment (test of all of the above)


Characterization

The scientific method depends upon the careful characterization of the subject of the investigation.

Observation demands careful measurement and the use of operational definitions of relevant concepts.

– When the terms used are formally defining, they acquire exact meanings which do not necessarily correspond with their use in natural language:

– for example, mass and weight are quite distinct concepts. On the surface of the Earth, the distinction may not be obvious, but in orbit around Earth, we are weightless and not massless.

– Isaac Newton: "I do not define time, space, place and motion, as being well known to all"


Hypothesis development

A hypothesis includes a suggested explanation of the subject. It will generally provide a causal explanation or propose some correlation.

Observations have the general form of existential statements, stating that some particular instance of the phenomena has some characteristic. Causal explanations have the general form of universal statements, stating that every instance of the phenomena has a particular characteristic. It is not deductively valid to infer a universal statement from any series of particular observations.


Prediction from the hypothesis

A useful hypothesis will enable predictions to be made, by deductive reasoning, that can be experimentally assessed. If results contradictory to the predictions are found, the hypothesis under test is incorrect or incomplete, requiring either revision or abandonment. If results confirming the hypothesis are found, the hypothesis might be correct, but is always subject to further test.


Experiment

Once a prediction is made, an experiment is designed to test it. The experiment may seek either confirmation or falsification of the hypothesis. Yet an experiment is not an absolute requirement. In observation-based fields of science, like astronomy, actual experiments are never performed, and control experiments are impossible.

Scientists assume an attitude of openness and accountability on the part of those conducting an experiment. Detailed recordkeeping is essential, to aid in recording and reporting on the experimental results, and providing evidence of the effectiveness and integrity of the procedure. They will also assist in reproducing the experimental results.


Evaluation and Iteration

The scientific process is iterative.– Verification

• Science is a social enterprise, and scientific work will become accepted by the community only if they can be verified.

• Scientific journals use a process of peer review, in which scientists' papers describing experimental results and their conclusions are submitted to a panel of fellow scientists for evaluation

– Reproducibility • Your experiments should be reproducible so other scientists

can validate your hypothesis and conclusions.

– Reevaluation• All scientific knowledge is in a state of flux, for at any time

new evidence could be presented that contradicts a long-held hypothesis.


Evidence and assumptions

Evidence comes in different forms and quality, mostly due to underlying assumptions.

An underlying assumption that 'objects heavier than air fall to the ground when dropped' is not likely to incite much disagreement. An underlying assumption like 'aliens abduct humans' however is an extraordinary claim which requires solid proof. Most extraordinary claims also do not survive Occam's razor.

Scientifc Process case

Debugging as science


Debugging…

The scientific method may sound heavy, but often you can use the

observe hypothesis prediction experiment– and detailed record keeping

...in fast iteration to analyze precisely

One good example is debugging...


Case: Haken-Blostein algoritm

Haken-Blostein is a complex algoritm for spacing sheet music.

Basically it treats the graphical objects like a system of connected springs with rods.


Observation

Observation: Each invokation of the algoritm spaced the sheet music differently. Something gets accumulated or changed between invokations?

Hypothesis: The observed behaviour is due to some kind of accumulating defect in the code. Some variable(s) change value during invocation

But - Too vague to be useful...


Iteration

Parameters:– external algorithm parameters: a

– spring constants ci, rod sizes Rj

Hypothesis 1:– The algorithm parameter, a, changes value

Experiment:– Inspect the value of a before and after invokation– Easy: there is a dialogue box where it can be

inspected.

Result: Hypothesis false.


Iteration

Hypothesis 2:– Spring constants changes

Experiment:– Inspect the values of a before and after invocation– Difficult:

• There are a lot of springs and no user interface. • Inserting debug output into code.• Manually comparing long lists of double values.

Result: Hypothesis false.


Iteration

Hypothesis 3:– Rod sizes changes

Experiment:– Inspect the values of a before and after invocation– Even more Difficult:

• There are a hundreds of rods and no user interface. • Inserting debug output into code.• Using diff-tool to compare very long lists of double values.

Result: Hypothesis accepted.– a few rod sizes somehow accumulate width…

… and computer science ?Software Engineering ?


Some observations…

Computer science / software engineering has some twists…

Computer science deals more with artifacts than nature itself.

Thus the material we make hypotheses about and experiment with are constructed objects.– i.e. there exists a relation between hypothesis and the

material in the experiment that we often do not see in physics or biology…

– gives problems with reproducibility• who else builds systems like ours?

– scope of hypothesis (our system versus all systems?)


Some observations…

Computer science/software engineering is a creative discipline just as much as a descriptive discipline.– You do not invent nature in physics or biology– but software engineers do!

Part of computer science is inventing stuff– meaning there is room for presenting hypotheses

without experimentation, only convincing argumentation.

Example: Inheritance in Simula– started object-orientation


Some observation...

Another research tradition: Mathematics– No experiments but proofs.– [but proofs are subject to errors as well ]

Diploma Thesis

Science in the small…


Science in the small

From the studieordning:

– Ved udarbejdelsen af hovedopgaven skal den studerende demonstrere fortrolighed med almindelige principper for videnskabelig metode og færdighed i at anvende metoder og teorier til selvstændigt at afgrænse og behandle problemstillinger inden for området softwarekonstruktion.


… Thus

This is the yardstick that we will use to evaluate your thesis!– softwarekonstruktion: the focus is on constructed

software objects (not necessarily constructing them!)

– selvstændigt at afgrænse og behandle problemstillinger: you must define your focus yourself.

– videnskabelig metode, færdighed i at anvende metoder og teorier : adherence to the process is an important aspect.


Corollary

Note an important implication:– building application X or Y is not interesting in

itself!

The focus is on – A) the scientific process– B) the product is knowledge

This is usually in contrast to engineering degrees where the product is the focus…


Diploma considerations…

Now – you are not supposed to become world leading scientists in software engineering :

– we do no expect ground-breaking new hypothesis

– comprehensive, large sample, double-blind experiments

– large software artifacts being build



But – we do expect– Characterization

• that you master the terminology and precision and clarity in statements and writing

• that you provide clear definitions in new concepts or ideas you present or introduce

– Assumptions and scope• that you state assumptions• that you know what “variables” you control and what remain

fixed• that you clearly scope your work (what do you not consider

part of the work, and argue how this influence your conclusions).



But – we do expect– Scientific method

• that you either– try hard to formulate your problem as a hypothesis to be

evaluated (prediction and experiment)

– perform an experiment to falsify/confirm an existing hypothesis/claim in a ‘novel’ environment

• that software construction techniques are used as part of the process



But – we do expect– Scientific progress

• that you not replicate well known facts in well known situations

• that you consult the (scientific) literature for inspiration and relations to your work (including web, products, etc).

– Scientific experiment• “Detailed recordkeeping is essential, to aid in recording

and reporting on the experimental results”• that you document both experiments and process in a clear

and unambiguous way so your observations are easy to understand and reproduce

• record ‘open issues’ as you go along!



But – we do expect

– Scientific evaluation and iteration• that you present and discuss your work with your fellow

‘researchers’• Short papers and presentations at seminars

– include as part of your final report

• Peer reviewing groups• Consulting senior researchers (course teachers )


Record keeping

If you do an interview– you analyse and conclude on the interview– ... but the interview raw data must be present also

Not only the result but also the steps in achieving the result is important (you may make an error that a fellow scientist will correct!)


’we build a system type’

hypothesis: ”using our tool is better and easier”– better: ”will achieve result in half the used time”

how to handle this:– describe the current situation using a couple of

concrete scenarios• inductive reasoning

– describe same output using the tool • deductive reasoning confirms hypothesis


Rationale

Every time you make a decision you potentially invalidate your analysis (because you may make the wrong choice!)

Thus every decision you make must be supplemented with a rationale. Why did you choose as you did?

Designing a project

… or finding a problem…


At least two ways to go…

Question– observer some interesting aspect

• from the work place; open source system; old project…

– turn the scientific machine• formulate hypothesis; make predictions; experiment

Evaluation– that is reproduce someone else’s results:– take existing hypothesis/technique– make predictions and experiment


Process of making hypothesis

Hypothesis making is difficult ! Example:

– General idea:• “I would like to write something about vegetation in Africa”

– Specific topic• “Trees in Africa”

– Problem • “Imbalance between felling and re-growth”

– Hypothesis• “Urban wood consumption is more harmful to tree cover than

rural consumption”.


Software example

– General idea:• “I would like to work with frameworks”

– Specific topic• “Frameworks for classic board games”

– Problem • “Hot spots required to cover large set of board games”

– Hypothesis• “It is possible to construct a framework for classic board

games with 90% code reuse that covers backgammon, chess, and checkers”.

It is only the latter that we can objectively measure…


Software example

– General idea:• “We want to build software to plot weather data from the

school’s weather station”

– Specific topic• “Plotting software for weather data (time series)”

– Problem • “Adding hot spots to existing plot framework for weather

data”

– Hypothesis• “It is possible to extend framework X with relevant hotspots

using patterns A, B and C to provide hot spots for weather data manipulation”.

Less “objectively measurable” but adds a lot of precision.


Advice

Even small questions take a lot of time! – Better to answer small, specific, questions and add a

few more questions than– … drown in work on a very broad hypothesis.

Problem X turns out to be– problem A– problem B– problem C– problem D– ...


Advice

Important to be explicit about assumptions and limitations:

Problem X consists of subproblems A..D. We will here delimit us to subproblem C.


Example

Health care scenario– [Nurse showing graph on TV using PDA]

Subproblems:– locating TV to use– combining ctrl and viewer for PDA and TV– steady-state interaction between PDA and TV– MFC combines ctrl and viewer in concrete code

... but most of these are independent problems and can be abstracted away...


Open issues

’Open issues’ is an important product!!!

Scientific process leads to– results (typically only on a single subproblem)– open issues

• that is: all the unsolved subproblems that we encountered during the process of trying to produce the result!

Scientists do not solve all the problems! The usually identify more problems than they

solve! (We don’t want to become unemployed)


Example: Open Issues

Health care scenario– [Nurse showing graph on TV using PDA]

Experiment– .NET on Laptop and PDA

• Laptop keeps model and viewer; PDA keeps controller

Open issue– MFC’s architecture merge ctrl and view making

experiment really troublesome– analyze and propose alternative implementation.


Evaluation

Another path is to evaluate existing hypotheses/techniques and report results– Using Quality Attribute workshops to assess qualities

in an architecture– Report experience on architectural documentation

using multiple view – Use architectural prototyping to assess buildability of

new project


Acknowledgements

Parts of the presentation is work by J. Stein Carter.

Parts are taken from Wiki pedia

science & scientific method. daimihenrik bærbak christensen2 literature [wikipedia, 2005]...

Documents

scientific knowledge

scientific process

science scientific method

life force present

spontaneous generation

computer science

centuries people

common knowledge