The Scientific Method and the Design of Experiments

Download The Scientific Method and the Design of Experiments

Post on 08-Jan-2018

217 views

Category:

Documents

0 download

DESCRIPTION

VIL 2010 All Rights Reserved Introduction This Lecture covers the following topics: What is The Scientific Method and where did it come from? Different types of experiments How to design experiments The Rule of Three The iterative nature of experiments How to interpret the results How to present the results VIL 2010 All Rights Reserved Xian 2010.

TRANSCRIPT

The Scientific Method and the Design of Experiments
Xian Jiaotong UniversityMarch 2010 The Scientific Method and the Design of Experiments Dr Leslie T Falkingham VIL 2010 All Rights Reserved VIL 2010 All Rights Reserved
Introduction This Lecture covers the following topics: What is The Scientific Method and where did it come from? Different types of experiments How to design experiments The Rule of Three The iterative nature of experiments How to interpret the results How to present the results VIL 2010 All Rights Reserved Xian VIL 2010 All Rights Reserved
What is The Scientific Method and where did it come from? Sir Francis Bacon and the Baconian Method Novum Organum 1620 Born 1561 Trinty College Cambridge University Lord Chancellor of England April 1626 Died - inventing the Frozen Chicken! Sources: Adapted from Wikipedia article on Sir Francis Bacon VIL 2010 All Rights Reserved Xian VIL 2010 All Rights Reserved
What is The Scientific Method and where did it come from? The Baconian Method Bacon is credited with inventing the modern Scientific Method.Originally called the Baconian Method. This was based onInductive Reasoning and a number of principles: The researcher should follow the Logic sequence Observation (Facts) Axiom (Hypothesis) Law. The Researcher should remove bias from the research toarrive at true Facts and a Theory based solely on the facts. If proven the Theory then becomes a Law. VIL 2010 All Rights Reserved Xian VIL 2010 All Rights Reserved
Different types of experiments The Logic Sequence Experiments are carried out as an application of logic. Thereare three main types of scientific experiment; To provide data on which a hypothesis may be based. To confirm or deny a hypothesis by testing its positivepredictions. To confirm or deny a hypothesis by testing its negativepredictions VIL 2010 All Rights Reserved Xian VIL 2010 All Rights Reserved
Different types of experiments To provide data on which a hypothesis maybe based. This where data on a subject is collected before inducing ahypothesis of the relationship between variables. By definition it predates the creation of a hypothesis and so itis important to collect all data which may be relevant. Theresearcher must be careful not to bias the experiment byforming a working hypothesis and selecting data accordingly. VIL 2010 All Rights Reserved Xian VIL 2010 All Rights Reserved
Different types of experiments To confirm or deny a hypothesis by testing itspositive predictions. This where the experiment is designed to test a hypothesisof the relationship between variables. Predictions are made using the hypothesis to be tested Whichstates that if a certain action is taken then it will cause acertain outcome. Care must be taken to ensure that a truerelationship is proven between the action and the effect, andthat this could not occur due to unrelated reasons. VIL 2010 All Rights Reserved Xian VIL 2010 All Rights Reserved
Different types of experiments To confirm or deny a hypothesis by testing itsnegative predictions. This where the experiment is designed to test a hypothesisof the relationship between variables. Predictions are made using the hypothesis to be tested Whichstates that if a certain action is taken then it will not cause acertain outcome. Care must be taken to ensure that a truerelationship is proven between the action and the effect, andthat this could not occur due to unrelated reasons. VIL 2010 All Rights Reserved Xian VIL 2010 All Rights Reserved
How to design experiments Control experiments testing the capability ofthe experiment. When setting up the experiment try to perform tests where theresult is already known. Both for positive and negative results. For example, if performing bil or HV ac testing. Operate theequipment with the device on test removed and the connectionsshort circuited and then repeat with an open connection. Thisshould give a result of no withstand in the first case, and totalwithstand in the second case. Sanity tests such as these aresimple, but are vital to remove doubt and errors in an experimentalsetup. VIL 2010 All Rights Reserved Xian VIL 2010 All Rights Reserved
How to design experiments The Rutherford Experiment In 1904 it was hypothesised by J J Thomson that the atomconsisted of electrons embedded in a sphere of positive matter,known as the Plum Pudding model. Which became theaccepted atomic theory at the time. In 1909 an experiment was carried out by Geiger and Marsdenon behalf of ernest Rutherford. This had unexpected results andresulted in the present Planetary model of the atom, proposedin Rutherfords paper of 1911. VIL 2010 All Rights Reserved Xian VIL 2010 All Rights Reserved
How to design experiments The Rutherford Experiment The Plum Pudding Model The Planetary Model Source Wikipedia, The Rutherford Experiment VIL 2010 All Rights Reserved Xian VIL 2010 All Rights Reserved
How to design experiments The Rutherford Experiment The experiment was intended to probe thestructure of the atom, it did this by firingalpha particles at a thin sheet of gold.ThePlum Pudding theory predicted that theparticles would suffer small deflections intheir path which could be used todetermine the distribution of charge withinthe atom. This was actually seen, but alsooccasionally a particle was deflected backtowards the source. This result wascompletely unexpected and was notpredicted by the theory. Source Wikipedia, The Rutherford Experiment VIL 2010 All Rights Reserved Xian VIL 2010 All Rights Reserved
How to design experiments The Rutherford Experiment Because of the strange results Rutherford hypothesised that theatom had a solid centre of positive charge which contained mostof the mass of the atom. This is now essentially thought to becorrect. The reason that his results were not published until 1911, twoyears after the experiments were carried out, was that he and hisco-workers spent over a year trying to understand what waswrong with the experiment! Only after he was convinced that theresults were correct did he come up with his new theory. He did not reject the results which did not fit the theory. Butbecause they did not fit, he very carefully checked to ensure thatthey were both correct and repeatable. Then he changed thetheory and the world! VIL 2010 All Rights Reserved Xian VIL 2010 All Rights Reserved
How to design experiments Important factors in designing experiments. The experiment must be designed to meet the objectives of theexperiment. Sounds obvious, but if after perfoming the experimentthere is still doubt about the meaning of the results then it was notwell designed. The experimental equipment must be able to unambiguously meet theobjectives of the experiment. Many experiments fail due to inadequateor inappropriate equipment or instrumentation. When designing any experiment it is important to try not toextrapolate results, and to make sure if possible that the experimentaldata set covers more than just the region of interest. It is important to understand that when performing any experimentthe results will be affected by uncontrolled variations. No matter howwell designed the experiment is it is impossible to remove all externalfactors. This will show itself as Rogue results. VIL 2010 All Rights Reserved Xian VIL 2010 All Rights Reserved
How to design experiments the Rule of Three The Rule of Three If something happens once - it is chance If something happens twice - it is coincidence If something happens three times - it is significant This is a crude but effective statistical approach to the probability of a resultbeing real. The essential feature of an experiment should be that when itis repeated you find the same result. By applying this rule you significantlyreduce the probability of experimental error, and you demonstraterepeatability. VIL 2010 All Rights Reserved Xian VIL 2010 All Rights Reserved
How to design experiments the Rule of Three The Rule of Three Depending on the experiment this may mean; Repeating the experiment three times (Three sets of data points) shows that the experiment for the sample under test gives consistentresults, and gives an indication of the level of experimental error Performing the experiment on three identical samples (Three sets ofdata points), - shows that the results are consistent for that type ofsample Repeating the experiment three times on each of three samples (Ninesets of data points). Shows that the experiment gives consistentresults with a level of experimental error, and that the results aregenerally true for the type of sample under test VIL 2010 All Rights Reserved Xian VIL 2010 All Rights Reserved
How to design experiments the Rule of Three Rogue Results When performing any experiment it is important to understandthat the results will be affected by uncontrolled variations. No matter how well designed the experiment is it is impossibleto remove all external factors. This will show itself as Rogueresults. These may be clearly observed as deviations in a sequence forexample. As shown in the following example; VIL 2010 All Rights Reserved Xian VIL 2010 All Rights Reserved
How to design experiments the Rule of Three Rogue Results VIL 2010 All Rights Reserved Xian VIL 2010 All Rights Reserved
How to design experiments the Rule of Three Rogue Results VIL 2010 All Rights Reserved Xian VIL 2010 All Rights Reserved
How to design experiments the Rule of Three Rogue Results VIL 2010 All Rights Reserved Xian VIL 2010 All Rights Reserved
How to design experiments the Rule of Three Rogue Results VIL 2010 All Rights Reserved Xian VIL 2010 All Rights Reserved
How to design experiments the Rule of Three Rogue Results VIL 2010 All Rights Reserved Xian VIL 2010 All Rights Reserved
The Iterative Nature of Experiments The Experimental Sequence Generally experiments are iterative to some extent. As thefirst results are known then areas of interest will appear, andweaknesses in the experimental set up and procedure willbe exposed. Once the experiment is designed, it is normal to perform theexperiment in three phases. Phase 1: Initial set up and experimental verification Phase 2: Improved experiment be careful, depending on howyou change the experiment the original results may no lnogerbe compatible with the later results. Phase 3: Focus of investigation on areas of interest VIL 2010 All Rights Reserved Xian VIL 2010 All Rights Reserved
The Iterative Nature of Experiments The Experimental Sequence Phase 1: Initial set up and experimental verification The experiment must be carefully designed to meet theobjectives of the experiment. This means taking time tounderstand the relationship between the vaiables beingmeasured and to identify any other factors which may affectthe realtionship. Verfiy that the experiment can give you thedesired results and establish the level of likely experimentalerror. If the experiment is not well thought out, then it is worthless.Take time to carefully consider the experiment, also considerhow the experimental results may be criticised once theexperiment is completed. If you do not do this before theexperiment, others will do so after you publish! VIL 2010 All Rights Reserved Xian VIL 2010 All Rights Reserved
The Iterative Nature of Experiments The Experimental Sequence Phase 2: Improved experiment be careful, depending onhow you change the experiment the original results may nolonger be compatible with the later results. In this phase you collect most of the data. This is the main partof the experiment and the collection and recording of datamust be rigorous. Record everything! VIL 2010 All Rights Reserved Xian VIL 2010 All Rights Reserved
The Iterative Nature of Experiments The Experimental Sequence Phase 3: Focus of investigation on areas of interest The data in phase 2 normally will have identified points of interest,rogue data points, extrapolations, areas of change. In this phaseyou look at the data and investigate these points of interest morethoroughly, normally by performing additional tests to investigatespecific identified points. VIL 2010 All Rights Reserved Xian VIL 2010 All Rights Reserved
The Iterative Nature of Experiments Points of interest VIL 2010 All Rights Reserved Xian VIL 2010 All Rights Reserved
How to Interpret the Results The Results Normally there is a body of knowledge already existing which canbe applied to the results. Similar work by others, Theoreticalbackground, etc. Interpret and explain the results in relation to thisprevious knowledge. Where appropriate use basic statistical tests to validate yourresults, such as Chi-squared ( 2 test), or Students T-test, etc. Inaddition if the problem is complex, then the data may besusceptible to analysis using statistical tyechniques such asOrthoganol Arrays The important point is not to ignore any data even if it does not fityour existing theory or understanding See: World Class Quality, K R Bote, and Statistics for Technology, C Chatfield VIL 2010 All Rights Reserved Xian VIL 2010 All Rights Reserved
How to Interpret the Results The Results VIL 2010 All Rights Reserved Xian VIL 2010 All Rights Reserved
How to Present the Results The Key Principles There are six main principles: Choose the method which presents the results in the mostclear and easy to understand way Explain what you think your results mean and how they relateto theory Explain why you did the experiment and what you were tryingto achieve Describe the experiment in such a way as to allow others toverify the results Explain the limitations of the experiment and the experimentalerror If you do not understand a data point, or a result, indicate itand say so. You do not have to explain everything! VIL 2010 All Rights Reserved Xian VIL 2010 All Rights Reserved
How to Present the Results Graphical or tabular presentations VIL 2010 All Rights Reserved Xian VIL 2010 All Rights Reserved
How to Present the Results A graph without explanation can be misleading ormeaningless VIL 2010 All Rights Reserved Xian VIL 2010 All Rights Reserved
How to Present the Results Rogue Results VIL 2010 All Rights Reserved Xian VIL 2010 All Rights Reserved
Conclusions Plan your experiment carefully to ensure that it can meet theobjectives of the experiment. Understand the limitations of the experiment, andexperimental error and ensure that you have allowed forthese. Realise that Rogue results are normal and treat themseriously these can actually be the most important part ofthe experiment. Interpret your results in the light of existing knowledge andprevious work of yourself of others Remember the six key principles of presenting the results. Ifyou follow them, publication and explaining your work toothers is quite simple. VIL 2010 All Rights Reserved Xian VIL 2010 All Rights Reserved
Questions? VIL 2010 All Rights Reserved Xian