Experimental DesignExperimental Design

Source: Christine Ambrosino @ http://www.hawaii.edu/fishlab/NearsideFrame.htm

DefinitionDefinition• In general usage: is the design of any information-

gathering exercises where variation is present, whether under the full control of the experimenter or not. – wikipedia

• In research: a study design used to test cause-and-effect relationships between variables. The classic experimental design specifies an experimental group and a control group. The independent variable is administered to the experimental group and not to the control group, and both groups are measured on the same dependent variable. Subsequent experimental designs have used more groups and more measurements over longer periods. True experiments must have control, randomization, and manipulation. – on-line medical dictionary

Originality in ResearchOriginality in Research

• Science depends on original thinking in two main areas:– The original ‘guess’ – called an hypothesis– The test or experiment to determine likelihood of

hypothesis being correct

A good scientist relies on ‘inspiration’ in the same way as a good artist. Some feel this point is largely ignored in present day science education.

Encourage your students to be creative, original or inspired by the everyday when designing their research!

Being Discerning in DesignBeing Discerning in Design

• Examples:– Teaching a dog French– Fortune Teller’s long term success

Elements of a Good ExperimentElements of a Good Experiment

• Features common to all good experiments which exist regardless of utilizing advanced equipment or basic technique:– Discrimination– Replication & Generality– Controls– ‘Blind’ Design– Measurement

DiscriminationDiscrimination

• Experiments should be capable of discriminating clearly between different hypotheses. It often turns out that two or more hypotheses give indistinguishable results when tested by poorly-designed experiments.

Replication & GeneralityReplication & Generality

• Living material is notoriously variable. • Usually experiments must be repeated enough times

for the results to be analyzed statistically. • Similarly, because of biological variability, we must be

cautious of generalizing our results either from individual creatures to others of the same species, or to other species. For instance, if our hypothesis is about mammals, it is inadequate simply to carry out our experiments on laboratory rats. Similarly, it is dangerous to extrapolate from healthy students to elite athletes.

ControlsControls

• The experiment must be well controlled. We must eliminate by proper checks the possibility that other factors in the overall test situation produce the effect we are observing, rather than the factor we are interested in.

• Example

“Blind” Design “Blind” Design

• Investigators can subconsciously 'fudge' their data if they know what result they want to find.

• The answer is to do the experiment 'blind', so the investigators (and the subjects, if humans are being studied) do not know which treatment's effect they are observing.

• This can make the logistics of doing the experiment more complex.

• Example

MeasurementMeasurement• Good experiments usually involve measuring something i.e. - length. • Important you know both the accuracy and the precision of your measuring system. • These two terms are not synonymous:

– 'accuracy' means the ability of the method to give an unbiased answer on average, – 'precision' is an index of the method's reproducibility.

• Ideally your method should be both– accurate (i.e., give the true mean)– and precise (i.e., have a low standard deviation). Sometimes one is more important than the other. For example, if you were looking for small changes with time in a quantity (such as an athlete's hemoglobin

concentration), you would need a precise measure of it rather more than an accurate one.Accuracy and precision together help you to judge the reliability of your data. They also help you to judge to how many significant figures you should quote your results. For example, if you use

a balance reading to the nearest gram, you should give the results to the nearest gram and not to the nearest tenth of a gram.

• Some experiments are very difficult to do because it is not obvious what can be measured.– This is a real problem in animal behavior: for example, there is no obvious unit or measure for 'emotional

state'. It is usually necessary to isolate measurable components of behavior. Thus the speed at which a tiger paces up and down a cage can give some indication of the internal state of the animal but can never give a full picture of it.

MeasurementMeasurement

• Plant Example• Detergent Example

Experimental Design & StatisticsExperimental Design & Statistics

• Good experimental design involves having a clear idea about how we will analyze the results when we get them.

• That's why statisticians often tell us to think

about the statistical tests we will use before we start an experiment.

ExamplesExamples

• Example 1. Experiments that yield no useful results because we did not collect enough data

• 9:3:3:1

Gametes AB Ab aB ab

AB AABB AABb AaBB AaBb

Ab AABb Aabb AABb Aabb

aB AaBB AaBb aaBB aaBb

ab AaBb Aabb aaBb aabb

Example 1 (Cont)Example 1 (Cont)

• Chi squared test• Poisson distribution• Student’s t-Test

• How many replicates to use?

ExamplesExamples

• Example 2. Experiments that seem to give useful results but our procedures let us down!

• Latin Square vs Randomization

Row 1 A B C D

Row 2 D C A B

Row 3 B A D C

Row 4 C D B A

Example 2 (Cont)Example 2 (Cont)

• Never put all the replicates of one treatment together!

• To block or not to block – real world dilemmas– Field Examples

More on Eliminating BiasMore on Eliminating Bias

• Yale Experimental Design Example

Steps to FollowSteps to Follow• 1. Define the objectives.

– Record (i.e. write down) precisely what you want to test in an experiment.

• 2. Devise a strategy. – Record precisely how you can achieve the objective. – This includes thinking about the size and structure of the experiment –

• how many treatments? • how many replicates? • how will the results be analysed?

• 3. Set down all the operational details. – How will the experiment be performed in practice? – In what order will things be done? – Should the treatments be randomised or follow a set structure? – Can the experiment be done in a day?– Will there be time for lunch? etc.

SOURCE USED FOR ABOVE INFO

The Really Easy Statistics Site by: Jim Deacon, Biology Teaching Organisation, University of Edinburgh

ADDITIONAL RESOURCES

Introduction to Experimental Design Lesson Plan with Worksheets By: Daniell Difrancesca; Critical Thinking in Science

Concepts of Experimental Design (pdf)By: SAS; A SAS White Paper

Questions?