exhibit 3 – epidemiological background

EXHIBIT 3 – EPIDEMIOLOGICAL BACKGROUND

Page - 1 - of 21

What is Epidemiology?

1. Methods for conducting valid epidemiological research to identify risk factors and causes

for disease in human populations have been well-defined for nearly six decades. Over

sixty years ago, Sir Austin Bradford Hill (1953) delivered a lecture entitled “Observations

and Experiment” to the Royal College of Occupational Medicine. In his lecture, Sir Austin

stated:

The observer may well have to be more patient than the

experimenter – awaiting the occurrence of the natural

succession of events he desires to study; he may well have to

be more imaginative – sensing the correlations that lie below

the surface of his observations; and he may well have to be

more logical and less dogmatic – avoiding as the evil eye the

fallacy of post hoc ergo propter hoc, the mistaking of

correlation for causation. [page 1000]

2. Sir Austin presented this lecture in response to what he perceived to be the tendency of

his colleagues – primarily occupational physicians – to declare that they had identified

the “cause” of a particular disease based on their observations.

3. Twelve years later, Sir Austin published his seminal and still highly relevant essay, “The

Environment and Disease: Association or Causation?” in which he outlined “. . . nine

different viewpoints from all of which we should study association before we cry

causation.” (Hill 1965). These viewpoints provided the impetus for the development of

modern epidemiological methods and the evaluation of evidence in the determination of

causation.

4. Despite the appreciation for over 60 years that an observed correlation or association in

an epidemiological study does not constitute causation, and that standard

epidemiological research methods to identify causation continue to be refined, a major

challenge for epidemiologists continues to be the communication of these principles to

lay audiences. This is in part attributable to the tendency – perhaps simply reflecting

human nature – to seek and identify “causes” for the health problems we suffer,

whether or not there is any scientific basis supporting a causal relationship.

KM-3 of 21


Page - 2 - of 21

5. For example, many people today (particularly elderly people) believe that exposure to a

draft or cold temperature “causes” “colds” (ironically the name “cold” is still used);

however, we now know that colds are not caused by temperature or cold air, but by viral

infections of the upper respiratory tract. Another example is stomach ulcers, which, until

recently, were commonly believed to be caused by stress, or by drinking too much

coffee. However, we now understand that most gastric ulcers are due to Helicobacter

pylori infection. That these conditions were observed (even accurately) to correlate with

other phenomena may be perceptive, but did not lead to a valid scientific understanding

of causation.

6. Critical evaluation of a body of epidemiological research is the preferred approach to

evaluating observed statistical correlations to assess the possibility of underlying

causation. Each relevant study should be critically evaluated and its findings weighted

based on study quality and scientific validity, followed by a synthesis of the overall body

of evidence. It is the totality of evidence, with more value placed on stronger studies,

which provides a basis for drawing a causal interpretation. The validity and strength of

an individual epidemiological study depends on the research approach, study design,

data quality and completeness of the study.

7. The quality of an epidemiological study also depends on its ability to avoid various forms

of bias (or “leaning” away from the correct result). There are numerous types of bias,

many of which can be classified into three broad categories:

(a) Biases resulting from the selective participation of certain subsets of individuals that

are not representative of the study group (leading to selection bias);

(b) systematic errors in ascertaining disease outcomes or exposure estimation (i.e.,

information or misclassification bias), especially where self-reporting or other

subjective methods are relied upon; and

(c) the mixing of the exposure or risk factor of interest with the effects of other strong

risk factors for the same disease (i.e., confounding bias).

8. Epidemiologists are particularly concerned with the impact of bias due to study design

issues and systematic error. Systematic error in the design, conduct, statistical

evaluation and interpretation of an epidemiological study all can lead to biases and

render results invalid and possibly misleading.

KM-3 of 21


Page - 3 - of 21

9. A critical review and synthesis of epidemiological research results is the preferred source

of evidence on which disease causation determinations as well as many other health-

and policy-related decisions are based, including decisions relating to medical treatment,

risk management and regulatory policy. For these purposes, the comprehensive review,

critical assessment and weight-of-evidence syntheses are typically combined with

balanced and informed judgment, to determine whether the available body of evidence

is sufficient in terms of study quality (i.e., reasonably free from bias, confounding and

measurement error), strength, consistency and specificity to support a causal judgment

or other decision.

10. As mentioned above, and detailed below, my review of the epidemiological literature on

exposure to industrial wind turbines and human health determined that the available

evidence is of insufficient methodological quality and strength to validly base claims that

wind turbines cause serious harm to human health. Furthermore, the accumulating body

of epidemiological studies consistently indicates a lack of serious harm to human health.

Epidemiological Study Approaches versus Other Approaches

11. There are two general epidemiological study approaches preferred by epidemiologists in

evaluating causal associations between risk factors (i.e., exposures) and disease (i.e.,

human health effects):

(a) cohort studies in which disease rates are compared between exposed persons and

unexposed persons within a clearly defined study population; and

(b) case-control studies in which exposure history among individuals with disease

(cases) is compared with exposure history among individuals without the disease

(controls) within a defined study population.

12. There are many variations of both cohort and case-control studies with different

strengths and weaknesses. Selection of the best design depends on the specific

research question(s) to be addressed as well as the resources available.

13. A particular strength of cohort studies is that individual-level exposure is determined

prior to the occurrence of disease. Cohort studies are also less susceptible to significant

sources of bias such as reporting bias. No cohort studies have been published that

examine the question of whether exposure to wind turbine noise increases the risks of

any disease or objectively measured health effect.

KM-3 of 21


Page - 4 - of 21

14. A particular strength of case-control studies is their efficiency, especially for rare

diseases. However, because exposure usually is not ascertained until after the disease

has occurred, it is difficult to avoid information bias in case-control studies. Information

bias tends to manifest itself in cases over-reporting or over-recalling their exposures,

due to their particular interest in the disease and its risk factors, and/or controls under-

reporting or under recalling their exposures, due to their lesser personal interest. No

case-control studies have been published that examine the question of whether

exposure to wind turbine noise increases the risks of any disease or objectively

measured health effect.

15. Critical reviews and syntheses of the epidemiological literature conducted to evaluate

disease causation typically give good quality cohort studies the greatest weight. Case-

control studies may also be given considerable weight; however, due to inherent

weaknesses in many case-control studies (especially those relying on the recall of study

participants to estimate historical exposures), they may be discounted. For example, the

International Agency for Research on Cancer (IARC – a World Health Organization

agency) recently reviewed whether perineal application of talcum powders could cause

ovarian cancer. The expert panel of epidemiologists identified and evaluated one cohort

study (which showed no correlation between predetermined exposure to talcum powder

and cancer risk) and nineteen case-control studies, most of which showed some positive

correlation (but relying only on post hoc identification of exposure). The committee,

heavily weighting the one cohort study, found that the epidemiological evidence was

limited and therefore insufficient for concluding a causal relationship (IARC 2010).

16. A third epidemiological study approach is the cross-sectional study, in which exposure

and disease outcome are simultaneously ascertained and statistical correlations between

them are evaluated. This approach is simple and inexpensive, but subject to many

potential sources of bias and therefore the weakest epidemiological approach. Cross-

sectional studies, therefore, tend to be given little weight, if any, in a causal

determination. All of the epidemiological studies published to date have been cross-

sectional studies.

17. Ultimately, no single study, regardless of its merits, is likely capable of demonstrating

causation. In epidemiology, causal evaluation generally requires multiple studies

conducted among different groups under differing circumstances. The critical evaluation

KM-3 of 21


Page - 5 - of 21

of the individual studies and weight-of-evidence synthesis of the body of evidence

available on a given topic is the preferred approach for determining causation.

18. Other forms of inquiry are often mistaken for epidemiological studies. For example,

“case reports” and “case series” are not epidemiological studies. “The case report is the

most basic type of descriptive study of individuals, consisting of a careful, detailed report

by one or more clinicians of the profile of a single patient.” Furthermore: “The individual

case report can be expanded to a case series, which describes characteristics of a

number of patients with a given disease” (Hennekens 1987).

19. Case reports and case series often are published in medical journals and do serve a

number of purposes. “While case reports and case series are very useful for hypothesis

formulation, they cannot be used to test for the presence of a valid statistical

association. One fundamental limitation of the case report is that it is based on the

experience of only one person. The presence of any risk factor, however suggestive,

may simply be coincidental. Although case series are frequently sufficiently large to

permit quantification of frequency of an exposure, the interpretability of such

information is severely limited by the lack of an appropriate comparison group. This lack

can either obscure a relationship or suggest an association where none actually exists”

(Hennekens 1987).

20. Self-reported accounts of symptoms or disease experience do not even constitute case

reports, as they are unlikely to have been objectively reviewed and presented by

medical professionals, are not subject to medical peer-review, may not meet a valid

case definition and may be generated for purposes other than expanding scientific

knowledge. Self-reported health complaints are often solicited via “health surveys” that

may resemble epidemiology, but may not be valid indicators of symptoms or underlying

pathological process or disease.

21. Similarly, surveys often do not validly measure or represent the spectrum of exposures

and other risk factors for the disease(s) under investigation, including the primary

exposure of interest (in this case, objective measurements of wind turbine noise for

each individual study participant within an appropriate timeframe relevant to a health

effect).

KM-3 of 21


Page - 6 - of 21

22. Other serious problems with poorly designed surveys include selective participation,

which may result if individuals with health complaints or related beliefs preferentially

agree to participate rather than a representative sample of the group of interest.

Furthermore, some poorly designed surveys “telegraph” the purpose to the participants,

which not only can influence who participates (i.e., selection bias), but can influence

how participants answer survey questions (i.e., information bias). Surveys and other

data-gathering and statistical analysis activities may resemble epidemiological studies or

share some components of epidemiological studies but do not constitute bona fide

epidemiological studies. Therefore they do not contribute valid or reliable evidence to

the determination of disease causation, and may mislead the lay reader unable to

discern whether reported study findings and interpretations are accurate.

Determinants of the Quality of an Epidemiological Study

Incidence vs. Prevalence

23. In epidemiological studies, disease in a population is preferably characterized using

measures of disease incidence (vs. prevalence, described below). Incidence is the

number of newly diagnosed or identified cases occurring in an enumerated population at

risk over a specified amount of time and can be expressed as a disease incidence rate.

For example, the age-adjusted incidence rate of breast cancer among women in Canada

is 102/100,000/year. In contrast, prevalence is the proportion of diseased individuals

in a population at any given point in time, e.g., the prevalence of breast cancer would

include all newly diagnosed (i.e., incident) cases, as well as existing (i.e., prevalent)

cases at a point in time. Prevalence is less useful for epidemiological purposes, as it not

only reflects (imperfectly) disease risk or rate, but also is a function of disease duration

(i.e., case fatality rate, survivability, treatability), which may be the result of many other

factors. For example, the Canadian Breast Cancer Foundation estimated that last year,

about 160,000 women in Canada had had a breast cancer diagnosis in the prior 10

years.1

1 Source:

http://www.cbcf.org/central/AboutBreastCancerMain/AboutBreastCancer/Pages/BreastCancerinCa

nada.aspx. Accessed 9/10/2014.

KM-3 of 21

http://www.cbcf.org/central/AboutBreastCancerMain/AboutBreastCancer/Pages/BreastCancerinCanada.aspx





Page - 7 - of 21

24. An increased prevalence of a specific disease observed in a population does not

necessarily reflect an increase in the incidence rate of disease. For example, a new

treatment that prolongs survival, which in turn increases the number of existing disease

cases in the population, could increase prevalence of disease without affecting the

incidence rate of new cases. The apparently high prevalence of breast cancer today

substantially reflects tremendous progress in breast cancer treatment. Disease

prevalence also can increase due to earlier and more complete detection of disease,

even if the underlying incidence rate has not increased.

25. Epidemiological studies often determine the statistical correlation between an exposure

or other risk factor and a disease or other “health effect” or “outcome”. Relative risk

(RR) or rate ratios (also referred to as RR) are measures of association, or correlation,

commonly estimated from cohort studies. Quite simply, the RR is the ratio of disease

risk or rate among the exposed, compared to the risk or rate of disease among the

unexposed. If the disease risk is greater among those with the exposure than among

those without the exposure, the RR will be greater than 1.0. Conversely, if the rate is

lower among the exposed, the RR will be lower than 1.0. RRs that are close to or not

statistically different from 1.0 demonstrate no association. The odds ratio (OR), which

under certain circumstances is an unbiased approximation of the relative risk, can be

calculated from case-control studies as the ratio of the odds of exposure among cases

compared to the odds of exposure among the controls. The OR is typically interpreted

the same way as the RR.

Exposure Data

26. Quantified measures of individual exposure – although the most desirable – are rarely

available in most occupational and environmental epidemiological studies. Therefore,

epidemiologists often rely on surrogate measures of exposure. Surrogates may range

from very crude classification such as “ever vs. never” exposed to sophisticated

multidimensional estimates that consider exposure sources, intensity, frequency,

duration, age at first exposure, time since last exposure (if discontinued), and other

parameters. Studies that validly measure specific exposures for individual study

participants provide stronger evidence of an association, if it exists, between the

exposure and the disease of interest, than studies that either employ indirect indicators

or surrogates of exposure, or estimate exposures at the group level (“ecological”

evaluation). The more accurate the surrogate measure or direct measurement, the less

KM-3 of 21


Page - 8 - of 21

likely exposure misclassification occurs, which can lead to invalid or even misleading

results.

27. Accurate timing of exposure relative to the disease process, including consideration of a

reasonable period of disease induction and disease latency, is critical to the validity of

epidemiological study results. Maximum latency is usually described as the time elapsed

between the first known or possible exposure to the agent of interest and the diagnosis

of the disease of interest. It includes an induction period, at some (unknown) point

during which the disease is initiated, as well as a true latency period, which is the time

between disease initiation and clinical detection and diagnosis.

28. Exposure assessed or measured in the wrong time intervals (such as that occurring after

the disease progresses to an irreversible point) may be irrelevant to disease causation.

The timing of exposure unrelated to disease causation is irrelevant, and therefore the

latency of a non-causal exposure-disease relationship is etiologically meaningless.

Bias

29. In epidemiology, “bias” refers to systematic (or methodological) errors that lead to

inaccurate and potentially invalid or even misleading study results. As described above,

most forms of bias can be grouped into three broad categories: selection bias,

information bias and confounding bias. The degree to which sources of systematic error

leading to potential biases are identified and prevented in the study design, or

addressed statistically (as with confounding bias), determine the validity of study

results. The combined impact of multiple biases may be to invalidate results, and in

extreme cases, generate misleading findings.

30. Selection bias results from incomplete and/or selective participation of certain subsets

of individuals in a study, resulting in distorted or even invalid results. Selection bias is of

concern when an entire study population is not included, and selection of a subset of

volunteers is studied. Ideally, true random selection of an adequately large sample will

generate a study sample representative of the study population. However, self-selection

(i.e., selection of self-identified volunteers) may result in a biased sample, tending to

include certain groups of individuals (such as those who do not work outside of the

home on phone surveys), or individuals aware of and having a particular interest in the

study topics. For example, in surveys of symptoms among people living nearer wind

KM-3 of 21


Page - 9 - of 21

turbines, correlations between residential distance and health complaints will be seen if

a higher proportion of people with health problems participate among those living nearer

turbines than among those living further away. The surveys by Shepherd (2011) and

Nissenbaum (2012) are examples where reported findings might reflect selective

participation.

31. One type of selection bias can occur in studies of individuals anticipating or actively

involved in litigation matters. The inclusion of litigants in a non-blinded study population

can bias results towards positive findings, as each participant has a clear financial

interest in the study findings. A number of studies have shown that litigants are more

likely to report the disease outcome that is part of or perceived to be related to the

success of their litigation (e.g., Williams et al.(1999); Peterson (2007)). A more basic

difference between a litigant population and a general community-based referent group

is that the litigants are suing for reasons that might include health complaints. In other

words, those same individuals, absent their health problems, might not have become

study participants.

32. The degree of selection bias in any epidemiological study depends on the type and

severity of the various forces acting upon the individuals that ultimately become the

study sample. Comparing characteristics of study participants, (e.g., age, gender,

socioeconomic status, etc.), with non-participants is the standard approach to

determining the degree to which selection may have biased results, but cannot prevent

or undo such bias. Selection bias also will determine whether, and to which population,

the results of a specific study may be applicable (i.e., externally valid). For example,

effects of exercise on joint pain among the elderly would not be applicable to children.

33. Information bias results from systematic errors in questionnaire responses or

measured data, for example, leading to the misclassification of persons with respect to

exposure level or disease status. Information bias may be reduced or prevented if

participants are not relied upon for self-reporting of critical information on exposure or

disease, or are at least blinded to the purpose of the study. For example, workers

exposed to and aware of unpleasant chemical odors in their work environment may be

more likely to complain of unrelated respiratory symptoms than workers in areas

without such odors. Information bias may be identified by validating portions of self-

reported information against a more objective record. For example, self-reported

occupational exposures may be compared with employer personnel records and

KM-3 of 21


Page - 10 - of 21

industrial hygiene measurement data. Ultimately, self-reporting and other subjective

measures of exposures – as well as disease and other health endpoints – is inherently

susceptible to bias (i.e., over- or under-reporting, and this is often different by exposure

or disease group). Therefore, objective measures are preferable, and contribute to the

quality of epidemiological studies. In nearly all of the epidemiological studies of wind

turbine noise and human health published to date, however, annoyance – and where

included, indicators of health and well-being – have primarily been based on self-

reporting, and as demonstrated in recent laboratory studies, psychological expectations

contribute to the associations reported (Crichton 2014b).

34. Confounding bias occurs due to the failure to account for other risk factors for the

same disease outcome that are correlated with the exposure or risk factor of interest.

The effects of these other risk factors (i.e., confounders), if appropriately identified and

measured, can be controlled statistically, at least in part. Uncontrolled confounding and

residual confounding can result in inaccurate or invalid study results. For example,

environmental exposure to air pollution is associated with increased risks of

adenocarcinomas of the lung; however, if individuals living in the city are more likely to

smoke, an observed increased risk of adenocarcinomas of the lung in city dwellers might

reflect – at least in part – the confounding effects of smoking. There are many risk

factors for common symptoms and diseases, and therefore there are many possible

causes, some or which will be coincidental with wind turbine noise or other unrelated

phenomenon (e.g., an odor, the weather, etc.).

KM-3 of 21


Page - 11 - of 21

Statistical Significance

35. Chance – also called random or measurement error – can also lead to inaccurate or

invalid results. Statisticians and epidemiologists evaluate the probability that an

observed result might be due to chance by applying tests of statistical significance.

Chance cannot reasonably be ruled out as an explanation for a reported correlation if the

results are not statistically significant. Statistical tests are typically set to accept a 5%

rate of committing a “type one” error, i.e., incorrectly identifying as statistically

significant an incorrect result. Therefore, by definition, 5% of all statistically significant

results arise by chance: even in the absence of a true underlying association, any single

result, even if statistically significant, may have occurred by chance alone. Therefore,

statistical significance of a relative risk estimate does not necessarily indicate a valid or

causal association.

36. Confidence intervals (CI) describe a range of values for an estimated sample parameter

that are consistent with the study data. Confidence intervals with a wide range (e.g.,

more than 10-fold from the lower to the upper value) indicate low precision in the

estimated parameter, usually due to small sample size. Confidence intervals with a

narrow range indicate greater precision. Confidence intervals with an alpha error rate of

5% may be used to test statistical significance at the p<0.05 level. Confidence intervals

achieve statistical significance when the 95% CI excludes the null value (for relative

risks this is 1.0). However, the confidence interval provides no direct indication of where

the true population parameter might lie (i.e., the validity of the estimated parameter

and confidence interval), as is frequently erroneously described.

37. Ultimately, even if a statistical test suggests that an association is statistically

significant, the observed association might reflect an underlying study error, or bias. A

large study with narrow confidence intervals that excludes the value 1.0 (i.e., is

statistically significant) still may be invalid due to bias. For example, consider a

hypothetical study comparing the length of 1000 new born babies in Ontario with 1000

babies born in Quebec that finds Ontario babies are, on average, 5 mm longer, and this

difference was highly statistically significant. Later it is determined that due to a

calibrating error, rulers shipped to participating maternity wards in Montreal

underestimated the length of babies, on average, by 5 mm. Therefore, the careful

evaluation of epidemiological studies for potential bias is critical to their valid

interpretation.

KM-3 of 21


Page - 12 - of 21

Peer Review

38. Epidemiological studies are usually submitted to scientific journals where they undergo

peer review. The peer review process is intended to provide objective critical review of

the manuscript by individuals familiar with the topical area addressed in the paper.

Reviewers’ comments are provided to the editor to support an editorial decision whether

to accept the paper, to reconsider the manuscript after revisions or additional work is

completed, or to reject it. Authors are also provided reviewers’ comments and generally

are required to address them by either modifying the paper or explaining why the

comment may not apply or require revising the manuscript. Ultimately, the quality of the

peer-review process depends entirely on the background and skills of the reviewer, the

effort expended in reviewing the submission, the reviewer’s objectivity and the degree

to which the editor relies on the reviewer’s comments.

39. That a study is accepted for publication, peer reviewed and is published does not

necessarily indicate that the study is of high quality or that its results or findings are

valid or reliable for any, including decision-making, purposes. Rather, it indicates that

the journal editor, with the aid of the peer reviewers’ comments, determined that the

paper is of sufficient quality and topical interest to warrant publication in that journal.

What is immediately rejected by one journal might readily be accepted by another.

Editorial decisions are also subject to the familiarity of the topic to the editor, as well as

personal or institutional bias. Unfortunately, with the recent proliferation of on-line

journals, there is great demand for publishable articles. While most journals have not

allowed this to impact the quality of papers selected for publication, the possibility exists

that poor-quality manuscripts may be accepted by journals with insufficient numbers of

submissions. Most (but not all) journals allow scientific and editorial debate through

“Letters to the Editor” sections, where issues pertaining to the quality of methods and

validity of interpretations may be openly discussed.

40. The key elements of an epidemiological study are summarized below. Studies designed

to reasonably eliminate or reduce the probability that results are unlikely due to chance,

confounding or other bias contribute more to the weight-of-evidence synthesis in order

to reach sound scientific conclusions.

KM-3 of 21


Page - 13 - of 21

Elements of an Epidemiologic Study

41. I have identified below several standard epidemiological design elements, as defined in

leading epidemiological textbooks that form the core of a standard detailed study

protocol that should be developed in advance of conducting any epidemiological study. A

proper study protocol specifies the approach and methods to be followed and also

documents any changes or modifications to these procedures as the study is

undertaken. Deviations from the protocol need to be documented, to ensure the

scientific integrity of the research. Guidelines for developing study protocols have been

published (Aday 2006; Checkoway 2004; Chemical Manufacturers Association 1991;

Rothman 2008).

42. Study methods should be described in final study reports and publications in adequate

detail to allow evaluation of the strengths and weaknesses of the methodology, and

therefore the potential validity of the study findings. The methods should be adequately

detailed so as to allow another investigator to replicate the study.

Research question

43. Scientific inquiry generally requires that a focused research question be formulated such

that an experiment (or in epidemiology, an observational study) may be conducted that

validly addresses the research question. For example, the following is an actual research

question that has been (and continues to be) addressed using epidemiological study

methods: does the use of cell phones increase the risk (i.e., new occurrence) of brain

tumors?

Hypothesis

44. The standard epidemiological approach to a research question requires the testing of

one or more clearly and specifically stated hypotheses. Under what is called the

“Scientific Method,” hypotheses are generally stated as “null” hypotheses such that tests

of the hypothesis can, given adequate evidence, refute the null hypothesis. For example,

a null hypothesis for the research question above would state that there is no

association between cell phone use and brain cancer. An observation of a valid

statistically significant difference in brain cancer rates between cell phone users and

non-users would lead to the investigator rejecting this null hypothesis, so as to conclude

that the brain cancer risk differs by cell phone use, i.e., there is a statistical correlation

KM-3 of 21


Page - 14 - of 21

between cell phone use and brain cancer occurrence. Note, however, that this would not

prove that cell phone use caused brain cancer as observation of a statistical correlation

does not equate with causation.

Study design

45. The investigator should choose a study design that is appropriate to address the

particular research question and study hypothesis. Studies of greatest value for

identifying risk factors for disease include well-designed and conducted cohort or case-

control studies, which are considered “analytical” approaches. In contrast, “descriptive”

studies, as the name implies, describe characteristics of a population, and might

generate interesting hypotheses, but provide little evidence of disease etiology or causal

association. Examples of descriptive studies include case reports, case series, ecological

and cross-sectional studies or surveys. Series of self-reported symptoms – regardless of

any patterns or commonalities that may be present – do not constitute an

epidemiological study of any kind, and do not elucidate causation.

Definition of disease

46. Regardless of the choice of study design, the specific disease(s) or health condition(s) of

interest in the study hypothesis should be clearly defined. Different diseases and health

conditions typically have many different causes. Specificity of diagnosis therefore is

critical in epidemiological studies and can affect study validity. As has become

increasingly appreciated, different subtypes of diseases – including different subtypes of

cancers of the same site – represent discrete disease entities with different

constellations of causes. Studies of disease causation preferentially study disease risk,

which is the rate of occurrence of new cases of a validly diagnosed, specific disease

entity in a defined study population.

47. Some diseases or health conditions are more easily and validly diagnosed such as

cancers confirmed by pathological examination. Other conditions – especially self-

reported symptoms and health complaints – are more difficult to objectively diagnose,

and may be entirely subjective. Self-reported conditions are also highly susceptible to

reporting bias, which can affect the validity of study findings. The determination of any

disease or health complaint in an epidemiological study requires objective means of

detecting and diagnosing the adverse health outcome, usually according to a standard

KM-3 of 21


Page - 15 - of 21

case definition. A valid case definition is “a set of uniformly applied criteria for

determining whether a person should be identified as having a particular disease, injury,

or other health condition. In epidemiology, particularly for an outbreak investigation, a

case definition specifies clinical criteria and details of time, place, and person” (CDC

2012). Valid case definitions will have high sensitivity (the probability of classifying true

cases correctly as “cases”) and specificity (the probability of classifying true non-cases

correctly as “non-cases”), determined by evaluating cases meeting the case definition

against a “gold standard” (method known for accuracy and repeatability). Therefore,

epidemiological studies of symptoms and other self-reported subjective complaints are

particularly challenging and as a consequence, valid attribution of symptoms to one or

another factor or exposure is especially difficult.

Definition of exposure

48. Specificity in defining exposures under investigation as part of the study hypothesis is an

important part of a standard study design approach. Specific exposures of interest,

chemical form (if applicable), route of exposure, concentration, timing of exposure, co-

exposures, etc. all may be important aspects of exposure characterization. The specific

exposures and exposure characteristics of interest, and how each will be ascertained,

should be clearly defined prior to undertaking a study. Again, objective measurements of

each study participant’s exposure is preferred, and to the extent that exposure is

determined indirectly, estimated or subjectively reported the validity of study results

may be compromised. For example, distance from an emitter of air pollution (or

hazardous waste site) often does not correlate well with individual exposures, as wind

patterns (or groundwater movement) may result in individuals equidistant from the

source having dissimilar exposures. Therefore, for most exposures (including wind

turbine noise), valid direct measurements at the individual level are preferred over

indirect and often crude (and therefore inaccurate) surrogates such as “living within two

(or five or ten) kilometers” of an industrial wind turbine.

Definition of confounding factors

49. Defining, measuring and taking into consideration the potential impact of other known

risk factors – also known as confounding factors – for a disease under study is a critical

design consideration. Some confounding factors can have a sufficiently strong effect on

the association under investigation such that failure to control for these factors may

KM-3 of 21


Page - 16 - of 21

result in invalid study results and conclusions. For example, if smoking was more

common among coffee drinkers, then a comparison of coffee drinkers and non-coffee

drinkers (without considering smoking) would give the appearance that coffee drinking is

associated with lung cancer. Proper consideration of smoking (the confounding factor)

would show that there is no independent association between coffee drinking and lung

cancer risk. Certain characteristics, such as age, gender, and race/ethnicity, are often

important predictors of disease and should be taken into account in all epidemiological

studies.

50. For common symptoms or health effects – such as headache and sleeplessness – there

are a host of possible “causes” and associated factors that would need to be considered

as confounding factors in a properly designed epidemiological study.

Identification of the study population

51. As part of the study protocol, standard methods include clearly defining the target

population at risk to be studied. The way in which study participants are identified,

selected, recruited and participate influences the final study group, which in turn

determines whether the study group is a representative and valid sample of the target

study population. Comprehensive or at least representative inclusion and participation of

a target population eliminates the potential for selective participation, which can lead to

biased and invalid results. Self-selected individuals from diverse locations (e.g., those

living near vs. those living more distant from wind turbines) do not constitute a defined

study population. It is unlikely that such individuals are a valid or representative sample

of any group, and therefore conclusions derived from studies of such groups are of little

or no inferential value. More importantly, they may reflect different groups with different

characteristics and therefore different health experiences.

Inclusion criteria

52. As part of the definition of a study population, criteria for inclusion need to be stated a

priori. Such criteria specify the dimensions of the study population to be included, and

often consist of elements such as time period of interest, demographics (i.e., age,

gender, and race/ethnicity), residence, vital status, specific disease diagnoses, exposure

definitions, timing of potential exposure, etc. Inclusion criteria will vary depending on

the specific research question and hypotheses being tested. Self-selected symptomatic

KM-3 of 21


Page - 17 - of 21

individuals who elect to respond to surveys on their health, do not meet any inclusion

criteria (valid or not), as none have been described. They also do not represent any

specific study target population.

Exclusion criteria

53. The criteria by which potential participants are excluded from the study group, i.e.,

ineligible or precluded participants, should also be defined a priori. For example, not

having lived in the study communities at a relevant time period at risk is an exclusion

criterion that might be used in a community-based cohort study. Inappropriate

designation of inclusion and exclusion criteria introduces potential for uncontrolled

confounding and bias in the study results. Individuals unable to provide objective data

on exposure or disease typically would be excluded, as their inclusion could bias or

invalidate results. Again using the selection of individual questionnaire responses as an

example, the criteria for not including other individuals is not stated.

Comparison population

54. As part of the study protocol, standard methods include clearly defining the comparison

population to be studied, such as the control population in a case control study, or the

unexposed portion of a population in a cohort study. Similar to the study population,

inclusion and exclusion criteria need to be defined in the study protocol for each of the

groups being compared, as these decisions can determine the degree to which selection

or participation bias influence study results. Ideally, the study and comparison

populations should differ only on exposure or disease status: therefore, inclusion and

exclusion criteria other than those related to exposure or disease status should be the

same for study and comparison populations. Case series are of limited value specifically

(among other reasons such as very small numbers studied) for their lack of an

appropriate comparison group.

Sources of potential bias

55. The detailed study protocol or methods section in a report should anticipate and identify

potential sources of bias, and more importantly, ways in which the design will ensure

reasonable elimination or minimization of the most common sources of bias, including

confounding, selection bias, and recall / information bias. Participants or investigators

aware of the reasons for conducting a survey can impact study results by influencing

KM-3 of 21


Page - 18 - of 21

who participates or not (i.e., selection bias) and how respondents answer questions

(reporting or recall bias). Leading or transparent questions also generate bias. Questions

that retrospectively elicit objective and valid information are extremely difficult to

construct – even for trained epidemiologists – even if respondents are unaware of the

study hypotheses.

Survey methods

56. Surveys, interviews or other forms of eliciting study data from participants are often

necessary. However, these approaches are subject to a wide range of biases. Standard

methods and procedures for administering questionnaires and for minimizing bias have

been widely documented. The following are common survey methods that may be

employed to reduce bias:

(a) Questionnaire. A questionnaire should consist of carefully written questions and

appropriate response metrics that address the study objectives. Clear, unambiguous

questions with appropriate response choices and clear unbiased, non-leading

instructions to participants are critical aspects to constructing a valid questionnaire.

Survey questions should be evaluated for consistency (e.g., test-retest reliability)

and internal reliability (i.e., different questions with the same underlying concepts

should yield similar responses). Ideally, questionnaires should be validated before

use as a data collection instrument in a study. A validated questionnaire should

reasonably predict or agree with established criteria for assessing the exposure or

outcome of interest. A validated questionnaire should be used in its entirety without

editing: changes that may seem minor, such as those that affect question order or

formatting, can affect responses and compromise validity (Juniper 2009). A validated

questionnaire, however, cannot remedy other study flaws such as a poor response

rate or other factors leading to a biased selection of participants.

(b) Blinding. In order to minimize the potential for bias in survey responses,

participants, as well as those collecting and coding data, ideally should be “blind” to

the specific study hypotheses. If possible, interviewers and data coders should be

blind to the exposure and disease status of participants. It is well documented that

surveys in which the study hypothesis is known to the respondents and/or the

interviewers are subject to potentially serious forms of response and information

bias, affecting study findings.

KM-3 of 21


Page - 19 - of 21

(c) Incentive. Any monetary or other incentive offered to potential study participants

should be described in the study protocol and designed to minimize the degree to

which individuals may participate solely because of the incentives, which may be

more likely among some participating subgroups and lead to selection bias and

invalid results. Furthermore, in some settings, potential participants may be

influenced by perceptions that participation might impact personal gains or losses

(e.g., actively involved in litigation, concerned about losing a job, etc.).

(d) Administration of survey. Methods for administering, collecting, and reviewing

surveys should be defined in the study protocol. Written instructions for and training

of interviewers should be documented. Deviations from the standard method of

administering the survey, or deviations in administering the survey between study

groups, can introduce bias.

Statistical analyses

57. Statistical analytical methods appropriate for the study design used should be well-

defined and specified in the study methods. Approaches to control for possible

confounding, handling of missing data and “outliers,” as well as sensitivity analyses to

test for possible sources of bias should also be specified in advance of data collection

and analysis. Standard questionnaires or health surveys may be accompanied by

instructions for appropriate analytical methods, including proper weighting schemes.

Results from analyses should be summarized in text as well as in presented in tables or

figures, and results considered notable (e.g., statistically significant, differences from

expected values, inconsistent findings, etc.) should be highlighted.

Reporting

58. Study reports, especially peer-reviewed published journal articles, generally include

structured sections that provide background or rationale for conducting the study; the

hypothesis under investigation; a description of methods used to identify study and

comparison populations, collect data, and perform analyses; descriptive characteristics

of study and comparison populations; response rates; results; and a discussion of the

results, in light of study strengths and weaknesses, including the likelihood and possible

effects of various biases and uncontrolled confounding. Journal relevance and quality

vary with respect to topical areas covered, review rigor and other quality standards and

KM-3 of 21


Page - 20 - of 21

accordingly, the validity and reliability of published reports also varies from journal to

journal.

Reference List

Aday LA, Cornelius LJ. 2006. Designing and Conducting Health Surveys: A Comprehensive

Guide. Third Edition: John Wiley & Sons Inc.

Checkoway H, Pearce N, Kriebel D. 2004. Research Methods in Occupational Epidemiology.

Second Ed. New York: Oxford University Press.

Chemical Manufactureres Association. 1991. Guidelines for Good Epidemiology Practices for

Occupational and Environmental Epidemiologic Research. J Occup Med. 33:1221-1229.

Crichton F, Dodd G, Schmid G, Gamble G, Petrie KJ. 2014. Can expectations produce

symptoms from infrasound associated with wind turbines? Health Psychol. 33:360-364.

Hennekens C and Buring J. Epidemiology in Medicine. First. 1987. Boston, Little, Brown and

Company.

Hill AB. 1953. Observation and experiment. N Engl J Med. 248:995-1001.

Hill AB. 1965. The environment and disease: association or causation? Proceedings of the

Royal Society of Medicine. 295-300.

IARC. 2010. Carbon Black, Titanium Dioxide, and Talc. [93]. Lyon, France, WHO Press. IARC

Monographs on the Evaluation of Carcinogenic Risks to Humans.

Juniper EF. 2009. Validated questionnaires should not be modified. Eur Respir J. 34:1015-

1017.

Nissenbaum M, Aramini J, Hanning C. 2012. Effects of industrial wind turbine noise on sleep

and health. Noise & Health. 14:237-243.

Peterson DI. 2007. The effect of litigation on claims of personal injury: a statistical study of

249 cases. J Long Term Eff Med Implants. 17:289-296.

KM-3 of 21


Page - 21 - of 21

Rothman KJ, Greenland S, Lash TL. 2008. Modern Epidemiology. 3rd Edition ed.

Philadelphia: Lippincott Williams & Wilkins.

Shepherd D, McBride D, Welch D, Dirks KN, Hill EM. 2011. Evaluating the impact of wind

turbine noise on health-related quality of life. Noise & Health. 13:333-339.

Williams CW, Lees-Haley PR, Djanogly SE. 1999. Clinical scrutiny of litigants' self-reports.

Professional Psychology: Research and Practice. 30: 361-367.

KM-3 of 21

exhibit 3 – epidemiological background

Documents