polygraph faq ~20 questions - talepitalepi.com/downloads/nelson-4.pdfraymond nelson (2014). do not...

1

Raymond Nelson (2014). Do not reproduce without permission.

Polygraph FAQ ~20 Questions

Raymond Nelson

July 2015

2


Objectives General information about science and testing Account for polygraph accuracy with presently

available published evidence Goal of science and scientific testing is to quantify the

degree of uncertainty associated with a conclusion Statistic Statistical confidence interval

Sensitivity – ability to detect deception Specificity – ability to determine truth-telling FP error rate FN error rate Think probabilistically

3


Probabilistic or Deterministic Deterministic test – (perfect test)

One possible outcome Behavior and choice make no difference Randomness and error have no role

Probabilistic test - Randomness and error may have a role All data is a combination of signal and noise

Diagnostic variance Uncontrolled variance Categorical results are determined when the statistical

probability of error is less than a stated tolerance level– Goal of science is to quantify the degree of uncertainty

4


Pseudoscience Theoretical premise does not evolve Vocabulary and technology not consistent with other related

fields of science Often one central person who serves as the source of all

expertise Technology does not contribute synergistically to other forms

of science Technology tends not to advance with related fields of

knowledge Standard of practice tend not to evolve with evidence

5


Pseudoscience (cont.)

Does not adhere to a scientific method Not supported by evidence Unprovable claims Over-reliance on confirmation instead of

falsification Over-emphasis on wholism instead of

validation Over reliance on faith instead of skepticism

6


Pseudoscience (3)

Refusal to acknowledge and address problems with a theory

Public inaccessibility of knowledge Deep magic

Failure to progress in terms of explanatory scope Failure to acknowledge alternative explanations for

unknowns Untestable assertions Assertions not consistent with evidence or existing

theories

7


How science works

Hypothesis Explanation without evidence Research question

Theory Published replicated evidence Continuous learning – (all theories are incomplete)

Law Evidence not expected to change with any new information

8


Goals of scientific testing and research is to quantify (probabilistically) the likelihood of a correct or incorrect result

9


10


20 Questions about the Polygraph

11


Does it work?

Is it valid? How accurate is it? Is it infallible?

12


NAS (2003)

Page 214 ...can discriminate lying from truth telling at rates

well above chance, though well below perfection.

13


What is a polygraph test?

Polygraph is a form of credibility assessment, used to evaluate the veracity of a person´s statements regarding an issue of concern.

In practical terms, the polygraph is referred to as a “lie detector test.” The term “lie detector” is a term of convenience, not a term

of science

Polygraph instruments consist of multiple sensors that acquire and record data that is correlated at statistically significant levels with differences in physiological responses that occur when a person is deceptive or truthful.

14


How is the test conducted?

The test consists of 3 parts Pre-test interview phase Data acquisition phase Test data analysis phase

All phases of testing are audio/video recorded Each of these parts has an important effect on the

accuracy and meaning of the test result A 4th phase is also used to review the initial test results,

and to resolve any remaining inconsistencies

15


Who makes polygraph instruments?

There are 4 polygraph instrument manufacturers in north america (alphabetical order) Axciton Lafayette Limestone Stoelting

16


What professional associations and standards exist for polygraph?

AAPP - American Association of Police Polygraphists

ASTM International - American Society for Testing and Measures

APA - American Polygraph Association NCCA – National Center for Credibility

Assessment NPA - National Polygraph Association

17


Who trains polygraph examiners?What about continuing education?

Polygraph examiners are trained according to rigorous accreditation standards set by the professional associations 400 hours of academic and practical training Accreditation is independent of any instrument

manufacturers or technology companies Technology companies are not prohibited form providing training

Continuing education requirements are a matter of local laws and regulations in addition to requirements from professional associations

18


What is the polygraph used for?

Forensic testing Criminal investigation

Screening Operational security / Information security Police pre-employment Post-conviction sex offender treatment and

supervision

19


Does it measure lies?

Lies, per se, are not a form of physical substance. “lie detector test” is a term of convenience not science

Lies are an event - an action that takes place in the form of activity on the part of the actor.

Actions are observable, recordable, and quantifiable through statistical and normative methods Though not always subject to linear physical measurement

PDD testing is a measure of discrimination between deception and truth-telling

20


How does the polygraph work? Stimulus --> response

Responses are statistically correlated with the criterion (deception) There is no response that is perfectly correlated with deception No response that in uniquely correlated with deception Testing requires a combination of valid features (symptoms)

Present the stimulus and measure the response Measure several times Aggregate the data together Compare the result to normative data for deceptive and truthful

persons Calculate a statistical classifier to describe the confidence level or

probability of error Compare the test statistic to predetermined requirements

Alpha / cut-scores – allow us to understand the potential accuracy and inaccuracy of the test result

21


What does the polygraph measure?● PDD testing is a process of measuring or scoring differences in

responses to the test stimulus questions.

– Responses are based in emotion, cognition and behavior

● Responses are partitioned according to the stimulus source, whether relevant target stimuli or comparison stimuli.

● Differences in response magnitude are compared to normative data for confirmed deceptive and confirmed truthful persons to calculate a statistical classifier.

● Because there is no such thing as a perfect test, all test results are probability statements that describe the level of statistical significance of the test result.

● A categorical conclusion is made when the probability of error or confidence level are within the required margins for precision or tolerance for error – calculated using published normative data

22


Who are the polygraph researchers?

Industry pioneers

Early publications Academics

Scientific journals, textbooks Department of Defense

Government publications Technology companies

Industry publications, scientific journals Independent professional researchers

Industry publications, textbooks, scientific journals

23


How accurate is the polygraph?

● Published studies show some polygraph techniques capable of average accuracy over 90% with a 95% confidence range from 86% to 96%.

● Scientific reviews tend to converge at an average accuracy rate that averages 89% for diagnostic exams with a confidence range of 83 to 95%.

● Data suggests that multiple issue polygraphs have less accuracy with an average near 85% and a confidence range from 77 to 93%.

● The aggregated average accuracy has been reported as 87% with a confidence range from 80% to 94%

● Claims of accuracy outside these ranges are probably too good to be true

24


Are some polygraph techniques extremely accurate?

Studies showing extremely high accuracy have been found to suffer from numerous serious methodological flaws that introduce the possibility of alternative explanations for the reported extreme high accuracy rates

A conservative view would hold that conclusions of extreme high accuracy cannot are not supported by confounded studies

25


Why do earlier reports claim near perfect accuracy?

Early scientific studies and scientific reviews can be shown to suffer from methodological limitations that introduce the potential for alternative explanation for the reported extreme high accuracy rates

Extreme high accuracy statistics have not replicated

The trend of historical and recent studies tends to show greater convergence of accuracy results between academic scientists and industry scientists

26


27


2011 Meta-analysis

38 different published studies 295 different examiners 3,723 exams 11,737 scores

45 surveys and experiments

28


Sensitivity and specificity - ESS

Sensitivity .817 (.706 - .927)

Specificity .846 (.747 - .946 FN .077 (.004 - .151) FP .064 (.001 - .130

29


Is the test biased against innocent persons?

Published research has shown that modern scientific polygraph techniques are not biased against truthful persons

Test sensitivity and test specificity have been shown to be essentially equivalent (not significantly different) for validated polygraph techniques

Test specificity may exceed test sensitivity under some circumstances

Sensitivity and specificity requirements are ultimately a policy matter expressed by referring agencies or referring professionals

Expressed in the form of test technique, decision rules, and statistical alpha boundaries that determine decision cut-scores

30


Does it measure stress?

The polygraph does not measures stress per se – it is not a test of stress – however taking tests is stressful for some people.

It is assumed that all or most people may experience some stress when taking tests.

Responses to test stimuli are compared to a comparative or baseline responses for each individual.

Resulting test scores are then compared to published normative data for statistical analysis.

31


Does it measure nervousness?

Polygraph does does not measure nervousness per se, but does measure activity in the autonomic nervous system.

It is assumed that all or most people are nervous when taking tests.

Responses to test stimuli are compared to a comparative baseline for each individual, and the resulting scores are then compared to normative data for statistical analysis.

32


Is there any unique lie response? There is no known physiological phenomena that is uniquely

correlated with deception Lies are actions or physiological events, and not a form of

physical substance. Because virtually all physiological functions have multiple

purposes there is no unique lie response. Test accuracy is achieved by recording and combining several

physiological responses into a single test index. Each physiological response index is independently correlated at

a statistically significant level with the difference in response when lying or telling the truth

Numerical/quantitative response indices are used to calculate a statistical probability using normative data for deceptive and truthful persons.

33


How does scientific testing work?

Valid symptoms or features Published correlation with the criterion

Numerical transformations Published structural model

Normative data Published knowledge-base

Decision rules Statistical combinatorics

34


Scientific testing and normative data Identify the published normative model

Normative reference data Used to quantify the degree of uncertainty

Alpha boundaries and cut-scores Tolerance for error Confidence level

Calculate the test statistic Interpret the test result

Categorical p < a = statistically significant

35


Features = Proxy Data

Often impossible to directly measure what we want to measure Pregnancy

Proxy data are substitute signals for which correlation is known and published Effective combination of multiple valid proxy signals

can increase classification accuracy

36


Valid Features?

Published peer-reviewed evidence Statistically significant

Greater than chance

Independent Statistically optimal mathematical combination

Published structural function

Combined signals outperform any individual signal

37


Respiration activity data

Correlation with criterion state r = .472 Accounts for 22.3% of the difference in guilty and

innocent cases

69.4% score concordance with criterion state

38


Cardiovascular activity data

Correlation with criterion state R = .530 Accounts for 28.1% of differences between guilty

and innocent cases


39


Electrodermal activity

Correlation with criterion state

– r = .702 (auto EDA)• Manual EDA = .683

– Accounts for 49.2% of the difference in guilty and innocent cases

91.6% score concordance with criterion

40


Scientific testing – predictive model Published feature development Published feature transformation

Data reduction

Structural equation modeling Published normative data

What do we know about the scores of guilty and innocent persons

Generalization Evidence of effectiveness with validation data Independent analysis and critical review

41


Valid Polygraph Features

Respiration activity Cardiovascular activity Electrodermal activity Combined effectively

Accounts for 89.6% of difference between guilty and innocent cases


42


Can it predict the future?

Polygraph test questions should address past behaviors that the examinee may have engaged in.

Test questions should not address future behaviors that can not yet have occurred.

Polygraph testing results can be thought of as intended to quantify the probability that a person has engaged in a past behavior.

Polygraph testing may also be used to obtain or confirm information that is used in the formulation or calculation of actuarial or probabilistic models that are intended to predict the potential risk that an individual may engage in some future behavior or event. Risk modeling and predictive modeling are complex statistical

processes that often depend on multiple forms of information.

Polygraph test questions and test results alone should not be used to make direct judgement about past behavior or to formulate deterministic assumptions/predictions about future behavior or events.

43


Can a single physiological index discriminate truth and deception?

For a single physiological index to predict deception at 95% Requires a correlation coefficient in excess of .97

Accuracy at 99.6% Requires a correlation coefficient in excess of .998

44


How reliable are the results? Reliability, in scientific testing, refers to the level of

agreement between different professionals or different test results.

Reliability is different from validity which refers to whether the test does what it is intended to do and whether the test result is accurate.

In polygraph testing the primary concern involves the reliability of results when different professionals evaluate the data.

Evidence at this time suggests reliability concordance coefficients are about .90, meaning different professionals agree on the results about 90% of the time.

Automated statistical analysis provide automated reliability.

45


Does the blood pressure cuff hurt?

Normal blood pressure is 120/80 mmHg Cuff pressure during testing is approximately 65mmHg.

It is not possible (i.e., mechanically impossible) to occlude or cut off the circulation during testing, and the testing procedure is not harmful.

Some people report that the cardio cuff is uncomfortable.

46


How long does the test take?

The entire testing process includes an interview to review and explain the case details, testing procedure, and test questions, followed by the recording of physiological responses to the test questions, and then analysis of the test data.

The entire procedure can take 90 minutes or more, and all phases of the procedure are important.

Complex cases and complex topics can require more time.

There is nothing factually to gain from conducting examinations too quickly.

47


How quickly are test results be available?

Test results can sometimes be evaluated at the time of testing, after all test data has been recorded.

In some cases the result is clear and available immediately.

Under other circumstances it may be necessary to complete a quality assurance review before the test result can be made available.

48


Can the results be used as evidence?Is the test admissible in court?

Polygraph testing is generally used as an aide to investigation of known or alleged incidents, in addition to other uses supervision, and monitoring of compliance with safety or security rules.

Under some circumstances the test may be used in court, but results are generally not intended for that purpose.

49


What happens if a person is confused?

Confusion during testing would confound the meaning of the test results.

Examiners will complete a thorough interview and discussion of the test material as the first part of the polygraph test.

Data collection will not occur until the examiner has established the test questions together with the examinee, based on an agreed upon understanding of the topical material.

50


What about alcohol drugs and medications?

There is no published research or theoretical rational suggesting that prescription medication use will affect responses to some test questions and not others.

Medication use is not expected to contribute to test inaccuracy.

Some medications may contribute to an increased likelihood of obtaining inconclusive results, and this may be more likely when a person is taking multiple prescription medications.

If a person is functioning optimally while taking medications, it should be expected that the person will be capable of producing optimally interpretable test data while taking any required medications.

In some cases it may be worthwhile to delay testing until medication levels are titrated effectively and until after any debilitating side effects have subsided.

51


What about health problems?Will health problems affect the test?

Physical health problems are not likely to affect responses to some test questions and not others.

There may be an increased likelihood of obtaining inconclusive results.

Physical health problem that do not involve the autonomic nervous system, including stable injuries, are not thought to be a contributor to inconclusive results.

Acute health problems may be contribute to levels of discomfort or distraction that warrant delaying of polygraph testing until a later time.

52


What about pregnancy?Can pregnant persons be tested? Pregnancy is a normal condition that is neither a disease nor a

disorder.

There is no published research or theoretical rationale suggesting that pregnancy would interfere with polygraph test results or that polygraph testing would interact with a normal pregnancy.

Polygraph data obtained from pregnant persons are indistinguishable from data obtained from non-pregnant persons.

Some examiners and some agencies cautiously avoid testing pregnant persons, there is no published research or theoretical rationale suggesting that pregnancy would affect responses to some test questions in a manner different than other test questions.

Persons experiencing a high level of risk or unusual complications associated with pregnancy should not undergo polygrpah testing.

53


How is the test scored? Does the examiner or the computer

decide the result? Researchers in many fields have demonstrated that numerical and quantitative

analysis have tended to outperform clinical methods, though the optimal solution may be a combination of clinical and quantitative analysis.

Developments in computer technology now permit numerical analysis with automated reliability – accuracy outperforms most humans.

Because polygraph decisions affect individual persons, the traditional approach has been to require that examiners evaluate the data manually to complete the quantitative analysis.

Test methods that emphasize human judgement, without the quantitative analysis of data, are referred to as “clinical” methods.

Clinical methods are valuable in the absence of numerical and quantitative methods, they are known to be more subjective and therefor less reliable.

Clinical methods are more common in the formative stages of an evaluation method

54


Can an attorney be present during testing?

Accurate test results can be obtained when the examinee attends to the test stimuli without distraction or interference.

The presence of a third party observer whose presence is not related to the accuracy of the test introduces a serious confound to the test results because of the unknown ways that an examinee will think about the test stimulus while also attending to the observer.

Referring professionals may observe the exam through video at the time of the exam or through observation of the recorded video at a later time.

55


Why are some results inconclusive?

Inconclusive results will occur on a small percentage of occasions when the numerical test score is not statistically significant for deception or truth-telling.

Inconclusive results may be caused by confusion or distraction, or by some forms of physical or mental impairment.

If the exact cause of inconclusive results could be known or predicted then the cause could be corrected, eliminated or controlled.

Unfortunately, the exact cause of inconclusive results may never be known with certainty.

There is no such thing as a perfect test that works every time with every person Any impulse to force the test to a result will likely increase errors

56


What causes errors? All test data is a matter of variance, or changes in activity that occur in response to the

test stimuli.

All test data, and therefore all test results, are a combination of diagnostic variance (signal) and error variance (noise).

Test accuracy is achieved by developing methodologies that optimize the availability of diagnostic variance that can be controlled and explained as a response to test stimuli.

To the extent that error variance is uncontrolled and un-explained, the exact cause for testing errors may not be known, and for this reason error variance is also referred to as random variance.

Scientific tests achieve their effectiveness by developing a normative database of published peer reviewed evidence that describes our knowledge of what can be expected from the population of interest ( i.e. guilty or innocent persons).

Normative data are used to quantifying and calculate the probability of a correct or incorrect result within an expected range, and to compare a test result with predetermined requirements for precision and tolerance for error.

57


Can a person beat the test? The National Research Council (2003) wrote: “Claims that it is easy to train

examinees to 'beat' both the polygraph and trained examiners require scientific supporting evidence to be credible” p147.

Effective faking strategies would have to increase testing errors beyond the confidence interval surrounding know error rates.

Requires research into whether a faking strategy can cause an error rate outside the upper limit of the confidence range of the normally expected error rate.

In the event that a testing error does occur it would be difficult to know with whether the cause of error is related to intentional effort, or is simply within the known range of test accuracy and inaccuracy.

There is no such thing as a perfect test and vulnerability to faking is a concern pertinent to all types of testing.

Many tests, including the polygraph test, may include methods designed to identify and detect faking attempts.

58


Is there such a thing as a perfect test?

59


60


Good Science Peer review and publication Concurrence between industry scientists and

independent scientists Multiple signals

Published construct description Published correlation coefficients

Model accountability – Published normative data Published statistical structure - discriminate coefficients or

regressions coefficients for the symptoms/signals of interest

61


Probabilistic or Deterministic Assumptions

Perfect test - deterministic? No random error No potential for alternative outcome Never wrong

Imperfect test - probabilistic? Quantify the degree of uncertainty? Publish a description of the discriminate coefficients Publish a description of the structural model Publish a description of the normative model Validation

62


Professionalism In addition to practical skills, professional training should

include: Physiology Psychology Ethics Science and scientific testing Statistics and probability theory

Training for deception detection professionals should include science and test theory Reduce vulnerability to confounded research Reduce vulnerability to economic interests

63


Lie Detection and Community Safety No such thing as a perfect test

All tests are probabilistic not deterministic Scientific tests are accountable

Published constructs Published correlations Published structural model Published normative data Scientific ethics Scientifically competent professionals

Risk for false-negative error Potential harm to the community

Risk for false-positive error Potential harm to the individual

64


[email protected](917) 293-3208

polygraph faq ~20 questions - talepitalepi.com/downloads/nelson-4.pdfraymond nelson (2014). do not...

Documents