experiments and observational studies thought questions 1.in studies to determine the relationship...

Experiments and Observational Studies

Thought Questions

1. In studies to determine the relationship between two conditions (activities, traits, etc.), one of them is often defined as the explanatory (independent) variable and the other as the outcome or response (dependent) variable.

In an experiment to determine whether the drug Memox improves cognition of patientswith moderate to severe Alzheimer’s disease, whether or not the patient received Memoxis one variable, and cognitive score is the other.

Which is the explanatory variable and which is the response variable?

2. Suppose you are interested in determining if drinking a glass of red wine each day helps prevent heartburn. You recruit 40 adults age 50 and older to participate in an experiment. You want half of them to drink a glass of red wine each day and the other half to not do so. You ask them which they would prefer, and 20 say they would like to drink the red wine and the other 20 say they would not. You ask each of them to record how many cases of heartburn they have in the next six months.

At the end of that time period, you compare the results reported from the two groups.

Give three reasons why this is not a good experiment.


3. When experimenters want to compare two treatments, such as an old and a new drug, they use randomization to assign the participants to the two conditions.

If you had 50 people participate in such a study, how would you go about randomizing them?

Why do you think randomization is necessary?

Why shouldn’t the experimenter decide which people should get which treatment?

4. In an experiment, researchers assign “treatments” to participants, whereas in an observational study, they simply observe what the participants do naturally.

Give an example of a situation where an experiment would not be feasible for ethical reasons.

Thought Questions


Confounding VariablesA confounding variable is related to the explanatory variable, and affects the response variable.

Example: Study of the relationship between smoking during pregnancy and child’s subsequent IQ a few years after birth.

Explanatory variable?

Response variable?

Possible confounding variables?

• The effect of confounding variables on the response variable cannot be separated from the effect of the explanatory variable

Randomized Experiment versus Observational Studies•In a randomized experiment we create differences in the explanatory variable and examine results (response variable).

•In an observational study we observe differences in the explanatory variable and then notice whether these are related to differences in the response variable.

Experiments and Observational StudiesDesigning a Good Experiment

• Randomization• Randomizing the Order of the Treatments

Example: Quitting Smoking with Nicotine Patches (JAMA, Feb. 23, 1994, pp. 595-600)

•Nicotine and control "look-alike" patches were randomly assigned to participants.

•Baseline values were compared for the two groups on numerous characteristics

•Because they examined comparability on 28 characteristics, a few showed substantial differences in the two groups, as would be expected just by chance.

•For instance, 85% of those receiving the nicotine patches were employed, while only 73.3% of those receiving the control patches were employed.


Control Groups, Placebos and the Placebo Effect

Control Groups: Handled identically to the treatment group(s) in all respects, except that they don’t receive the actual treatment. Handled identically to the treatment group(s) in all respects, except that they don’t receive the actual treatment.

Special type of control group: Placebos. Why do we use them?

What is the Placebo Effect?


Effect of colour of drugs: systematic review of perceived effect of drugs and of their effectiveness – BMJ 1996; 313 : 1624 (Published 21 December 1996)

Objectives: To assess the impact of the colour of a drug's formulation on its perceived effect and its effectiveness and to examine whether antidepressant drugs available in the Netherlands are different in colour from hypnotic, sedative, and anxiolytic drugs.

Design: Systematic review of 12 published studies. Six studies examined the perceived action of different coloured drugs and six the influence of the colour of a drug on its effectiveness.

•The colours of samples of 49 drugs affecting the central nervous system were assessed using a colour atlas.

Main outcome measures: Perceived stimulant action versus perceived depressant action of colour of drugs.

•The trials that assessed the effect of drugs in different colours were done in patients with different diseases and had different outcome measures.


Results: The studies on perceived action of coloured drugs showed that red, yellow, and orange are associated with a stimulant effect, while blue and green are related to a tranquillising effect.

•The trials that assessed the impact of the colour of drugs on their effectiveness showed inconsistent differences between colours.

•The quality of the methods of these trials was variable. Hypnotic, sedative, and anxiolytic drugs were more likely than antidepressants to be green, blue, or purple.

Conclusion: Colours affect the perceived action of a drug and seem to influence the effectiveness of a drug.

•Moreover, a relation exists between the colouring of drugs that affect the central nervous system and the indications for which they are used.

•Research contributing to a better understanding of the effect of the colour of drugs is warranted.

Effect of colour of drugs: systematic review of perceived effect of drugs and of their effectiveness – BMJ 1996; 313 : 1624 (Published 21 December 1996)

Experiments and Observational StudiesStudy: Sham device v inert pill: randomised controlled trial of two placebo treatments

Objective To investigate whether a sham device (a validated sham acupuncture needle) has a greater placebo effect than an inert pill in patients with persistent arm pain.

Design A single blind randomised controlled trial created from the two week placebo run-in periods for two nested trials that compared acupuncture and amitriptyline with their respectiveplacebo controls.

Comparison of participants who remained on placebo continued beyond the run-in period to the end of the study.

Setting: Academic medical centre. Participants 270 adults with arm pain due to repetitive use that had lasted at least three months despite treatment and who scored ≥ 3 on a 10 point pain scale.

Interventions Acupuncture with sham device twice a week for six weeks or placebo pill once a day for eight weeks.

Main outcome measures: Arm pain measured on a 10 point pain scale.

Secondary outcomes were symptoms measured by the Levine symptom severity scale, function measured by Pransky’s upper extremity function scale, and grip strength.


Results: Pain decreased during the two week placebo run-in period in both the sham device and placebo pill groups, but changes were not different between the groups ( − 0.14, 95%confidence interval − 0.52 to 0.25, P = 0.49)

Longitudinal regression analyses that followed participants throughout the treatment period showed significantly greater downward slopes per week on the 10 point arm pain scale in the sham device group than in the placebo pill and on the symptom severity scale.

Differences were not significant, however, on the function scale or for grip strength. Reported adverse effects were different in the two groups.

Conclusions: The sham device had greater effects than the placebo pill on self reported pain and severity of symptoms over the entire course of treatment but not during the two weekplacebo run in.

Study: Sham device v inert pill: randomised controlled trial of two placebo treatments


Blinding: Single and Double-Blind experiments.

Example: Case Study 5.1, measuring the impact of nicotine patches on smoking, could be double-blind experiment.

Why is this a good idea?

Matched Pairs, Blocks, and Repeated Measures

Experiments using pairing or blocking are those in which the same or similar experimental units are used for all treatments. These are sometimes called repeated measures experiments.

Mozart Example: used the same participants to listen to Mozart and to the other conditions (listening to relaxation tape, silence).

Why is this a good idea?

For other studies, like the nicotine study, it isn't possible to use the same people.


Example: Quitting Smoking with Nicotine Patches• 240 smokers recruited (all met entry criteria).• Randomly assigned to either nicotine patch or placebo patch for 8 weeks. • Double-blinded.• After 8 weeks: 46% of nicotine group quit, only 20% of placebo group quit.• After 1 year: 27.5% of nicotine group quit, only 14.2% of placebo group quit.

Difficulties and Disasters in Experiments

Confounding variablesExample: Quitting Smoking with Nicotine Patches• Nicotine patch more effective when no other smokers in home.

• If first 120 volunteers assigned to placebo and last 120 to nicotine patch, and if those with noother smokers in home more eager to volunteer => what would be the problem with the

results?

Another Example: In Case Study 1.1 Lee Salk did not randomize the infants to either hear theheartbeat sound or not. He used the same nursery on subsequent days, with different groups ofbabies.

What problems could occur?

Experiments and Observational StudiesInteracting VariablesEffect of explanatory variable on response variable may vary over levels of other variables.

The solution?

Example: Quitting Smoking with Nicotine Patches

•The effectiveness of nicotine patches varied depending on whether or not there were other smokers at home, but the effectiveness of the control patches did not.

•The presence of other smokers interacted with the treatment in producing the results.

Percent quitting Nicotine Placebo (Control Patches)

Smoker at home 31% 20%

No smoker at home 58% 20%

Experiments and Observational StudiesInteracting Variables

Example: Quitting Smoking with Nicotine Patches

Note: The researchers who conducted this study looked at a variety of potential interacting variables but found only “other smokers at home” to be significant.

Some of the additional possibilities they considered were age, weight and whether or not the participant suffered from depression.


Hawthorne EffectsIndicate that people respond differently when they know they are part of an experiment.

1920’s Experiment by Hawthorne Works of the Western Electric Company

What changes in working conditions improve productivity of workers?– More lighting?– Less lighting?– Other changes?

Ecological Validity and GeneralizabilityVariables measured in labs or artificial setting, results do not accurately reflect impact in real world.

The problem:lack of generalizability due to:

unnatural settingssample that is not representative of population


• Observational study: compared 665 men admitted to hospital with 1st heart attack to 772 men (same age group) admitted to same hospitals for other reasons.

• Percent with pattern baldness higher for heart attack group (42%) compared to no heart attack (34%).

• Included adjustments for age and other heart attack risk factors.

What are the advantages of a case-control study?What do you think of the controls used in this study?

Example: In determining whether owning a pet bird is related to incidence of lung cancer, researchers would identify lung cancer patients as the cases and then find people with similar backgrounds who do not have lung cancer as the controls. They would then compare theproportions of cases and controls who owned a pet bird.

Example: Baldness and Heart Attacks

Case-Control Studies attempts to include an appropriate control group.‘Cases’ who have particular attribute or condition are compared with ‘controls’ who do not.

Designing a Good Observational Study

In an observational study, researchers don’t assign choices; they simply observe them.Manipulation occurs naturally, not imposed. Can’t assume the explanatory variable is the onlyone responsible for any observed differences in the response variable.


Retrospective or Prospective Studies

Prospective: participants followed into future, and events recorded.

Retrospective: participants are asked to recall past events.

Which is better and why?

Designing a Good Observational Study


Confounding Variables and the Implications of CausationProblem: No way to establish causation with an observational study.

Example: Smoking During Pregnancy• IQs lower for children of women who smoked.

• Difference as high as 9 points before accounting for confounding variables (diet and education); reduced to 4 points after accounting for those factors.

• Can’t conclude smoking caused lower IQs in children.

Extending the Results InappropriatelyThe problem: Because many observational studies use convenience or volunteer samples, it may not be valid to extend the results to other groups.

Difficulties and Disasters in Observational Studies

Example : Baldness and Heart Attack

• Observational study only used men who were hospitalized.

• Should consider whether results should be extended to all men.


Using the Past as a Source of Data

The problem: In some observational studies participants are required to remember the past.

Partial solution: Use official records if they exist, rather than relying on memory. If possible, use prospective study.

Example: Pearson, Ross and Dawes (1992, pgs. 75-76) reported on a study in which chronic headache sufferers were asked to keep a diary of the intensity of their pain for a week.

• At the end of the week, they were asked to rate their current level of pain, then to recall the maximum and minimum pain they had felt during the week.

•Those who were currently suffering from a high level of pain overestimated their prior levels of pain, while those with current low levels underestimated their prior pain.

Go easy on the coffee, you could start seeing things – Daily Mail, January 14th, 2009Drinking cup after cup of coffee dramatically increases the risk of hallucinating, research shows. Healthy young men and women who had more than seven cups of instant coffee a day were three times more likely to hear or see things that were not there than those who limited their intake to less than a cup.

•Confirming the link could lead to new treatments for those who suffer severe hallucinations, including schizophrenics, some victims of child abuse and the recently bereaved.

•The Durham University researchers asked 219 students to document their caffeine intake, working on the principle that a cup of instant coffee contains 45mg of caffeine.

•The volunteers were also asked how often they suffered hallucinations.

•The high caffeine users were three times as likely to have had problems as those who rarely drank coffee.

•Large amounts of caffeine also made people more likely to think they could sense the presence of ghosts, the journal Personality and Individual Differences reports.



Years Later, No Magic Bullet Against Alzheimer's - NY Times, August 29th, 2010•The scene was a kind of science court. On trial was the question ''Can anything -- running on a treadmill, eating more spinach, learning Arabic -- prevent Alzheimer's disease or delay its progression?''

•To try to answer that question, the National Institutes of Health sponsored the court, appointing a jury of 15 medical scientists with no vested interests in Alzheimer's research.

•For a day and a half last spring, researchers presented their cases, describing studies and explaining what they had hoped to show.

•The studies included research on nearly everything proposed to prevent the disease: exercise, mental stimulation, healthy diet, social engagement, nutritional supplements, anti-inflammatory drugs or those that lower cholesterol or blood pressure, even the idea that people who marry or stay trim might be saved from dementia. And they included research on traits that might hasten Alzheimer's onset, like not having much of an education or being a loner. …..

•''I was surprised and, at the same time, very sad'' about the lack of evidence, said Dr. Martha L. Daviglus, the panel chairwoman and a professor of preventive medicine and medicine at the Feinberg School of Medicine at Northwestern University.

•''This is something that could happen to any of us, and yet we are at such a primitive state of research.'' ….

Years Later, No Magic Bullet Against Alzheimer's - NY Times, August 29th, 2010•The problem, the group wrote, was that ''the quality of the evidence was typically low.''

•Most studies observed people who happened to use or not use a possible preventive measure and then determined whether they got Alzheimer's or not.

•Such studies, known as observational ones, are not the gold standard, like those in which people are randomly assigned to take a pill or do something like exercise, or not. Observational studies are useful in generating hypotheses but are not proof. Still, if several well-done studies of this type come to the same conclusion, they can be valuable evidence.

•In the case of Alzheimer's prevention, though, the studies tended to have problems, Dr. Williams said.

•Often it was not clear precisely what subjects were doing. They might have been using a drug or a supplement at the start of the study but the dose was not specified, nor was it clear whether subjects were taking the same doses, or for how long.

•Some studies of drugs to lower blood pressure used self-reports as opposed to, for example, pharmacy data. A 12-year study asked participants about their use of cholesterol-lowering statins at the start of the study but never did again.

•A nine-year statin study used pharmacy records but included as users those who took the drugs at any time during the study period.



Text questions pg 102

experiments and observational studies thought questions 1.in studies to determine the relationship...

Documents