igor pacheco, clpe (miami-dade police department) brian cerchiai, ctpe (miami-dade police...

Igor Pacheco, CLPE (Miami-Dade Police Department)Brian Cerchiai, CTPE (Miami-Dade Police Department)

Stephanie Stoiloff, M.S. (Miami-Dade Police Department)Sneh Gulati, Ph.D. (Florida International University)

Presented by: Igor Pacheco, CLPE & Brian Cerchiai, CTPE

This project was supported by Award No. 2010-DN-BX-K268 awarded by the National Institute of Justice, Office of Justice Programs, U. S. Department of Justice.

The opinions, findings, and conclusions or recommendations expressed in this publication/program/exhibition are those of the authors and do not necessarily reflect those of the Department of Justice.

RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT EXAMINATIONSEXAMINATIONS

2

BackgroundThe 2009 National Academy of Sciences’ report titled, “Strengthening Forensic Science in the United States: A Path Forward” announced thirteen recommendations to address challenges facing the forensic science community.

◦ This study was conducted specifically to address concerns of their third recommendation, “Research is needed to address issues of accuracy, reliability, and validity in the forensic science disciplines.”

MDPD study was modeled after previous study by Glen Langengurg◦ Langenburg, G. A Performance Study of the ACE-V Process: A Pilot

Study to Measure the Accuracy, Precision, Reproducibility, Repeatability, and Biasability of Conclusions Resulting from the ACE-V process. J. For. Ident., 2009, 59 (2).

RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT EXAMINATIONSEXAMINATIONS

3

BackgroundEvaluated the reliability of Latent Fingerprint Examiners using the Analysis, Comparison, Evaluation, and Verification (ACE-V) methodology in latent fingerprint examinations by Measured the Accuracy, Precision, Reproducibility, Repeatability, and Bias of four categorical opinions:

◦ Identification◦ Exclusion◦ Inconclusive◦ No value

RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT EXAMINATIONSEXAMINATIONS

4

Participation

109 Experienced Latent Print Examiners Participated

◦ Requirements to participate: 1 of active latent casework experience uses the ACE-V methodology enforcement agency (crime laboratory), or like agency, in the

United States. This includes all active, retired or contracted latent examiners.

RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT EXAMINATIONSEXAMINATIONS

5

Announcement & QuestionnaireParticipants from 75 Different Law Enforcement Agencies

RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT EXAMINATIONSEXAMINATIONS

53Local

Agencies

Announcement & QuestionnaireParticipants from 75 Different Law Enforcement Agencies

RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT EXAMINATIONSEXAMINATIONS

53Local

Agencies

16State

Agencies

Announcement & QuestionnaireParticipants from 75 Different Law Enforcement Agencies

RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT EXAMINATIONSEXAMINATIONS

53Local

Agencies

16State

Agencies

Federal Agencies

3

Announcement & QuestionnaireParticipants from 75 Different Law Enforcement Agencies

RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT EXAMINATIONSEXAMINATIONS

53Local

Agencies

16State

Agencies

Federal Agencies

4N/A3

Announcement & QuestionnaireLatent Print Examination Experience

RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT EXAMINATIONSEXAMINATIONS

10

Announcement & QuestionnaireLatent Print Examination Experience

RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT EXAMINATIONSEXAMINATIONS

11

Announcement & QuestionnaireLatent Print Examination Experience

RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT EXAMINATIONSEXAMINATIONS

12

Announcement & QuestionnaireLatent Print Examination Experience

RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT EXAMINATIONSEXAMINATIONS

13

Announcement & QuestionnaireIAI Latent Print Certification

RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT EXAMINATIONSEXAMINATIONS

Certified

Not Certified

14

Research Questions

1)Will Latent Examiners be able to correctly identify or exclude unknown latent impressions from known standards using the ACE methodology?

2)Will Latent Examiners be able to correctly identify or exclude unknown latent impressions from known standards using the ACE-V methodology?

3)Will Latent Examiners be able to reproduce and repeat conclusions from unknown latent impressions to known standards using the ACE and ACE-V methodology?

4)Will Latent Examiners be able to reproduce and repeat conclusions from unknown latent impression to known standards using the ACE-V methodology under high bias conditions?

RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT EXAMINATIONSEXAMINATIONS

15

Research Design and MethodThirteen confidential volunteer sources were identified

8 Men & 5 Women chosen due to not having their fingerprints or palmprints in any known AFIS database outside of Miami-Dade County.

13 sets of Tenprint and Palm Print Standards collected 2,711 known latent impressions were created (fingers and palms)

Collected from non-porous flat and curved surfaces (plastic, tile, metal, and glass) Processed using black powder and clear tape on white backing cards (scanned and

printed as photos) The latent impressions included various sizes, clarity, and levels of distortion

RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT EXAMINATIONSEXAMINATIONS

16

Research Design and MethodThree (3) Certified Latent Print Examiners independently evaluated and compared 320 latent prints to the known standards using a rating scale that was created for this research.

◦ 80 Latent Prints were chosen for the final test setsRating Scales for each latent analysis and comparison:

= (Analysis)

Strength of Value of Latent Print (Comparison)

Latent in Agreement with Standard+2

Difficulty of Comparison

RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT EXAMINATIONSEXAMINATIONS

17

Strength of Value of Latent Print

0 7 14 21

Insufficient to Difficult

Minutiae: 4.33

MinutiaeFormations: 0.00

Clarity: 0.67

Total: 5.00

Latent in Agreement with

Standard

Difficulty of Comparison: 5.67

4.33

0.00

2.00

6.33

18

Strength of Value of Latent Print

0 7 14 21

Difficult to Moderate

Minutiae: 10.00

MinutiaeFormations: 1.00

Clarity: 1.33

Total: 12.33

Latent in Agreement with

Standard

Difficulty of Comparison:

10.5

6.00

0.66

2.00

8.66

19

Strength of Value of Latent Print

0 7 14 21

Moderate to Easy

Minutiae: 14.00

Minutiae Formations: 3.33

Clarity: 2.00

Total: 19.33

Latent in Agreement with

Standard

Difficulty of Comparison: 19.33

14.00

3.33

2.00

19.33

20

Research Design and MethodDistribution of Latent Prints for Testing

RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT EXAMINATIONSEXAMINATIONS

30% 24 Latent Prints

Source was not Present

70% 56 Latent Prints

Source was Present

21

Research Design and MethodSource Not Present: Strength of Value of Latent Print

(24 Latents)

RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT EXAMINATIONSEXAMINATIONS

22

25% 6 Latent Prints

Insufficient to Difficult

Research Design and MethodSource Not Present: Strength of Value of Latent Print

(24 Latents)

RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT EXAMINATIONSEXAMINATIONS

37.5% 9 Latent Prints

Moderate to Easy

23

25% 6 Latent Prints

Insufficient to Difficult

Research Design and MethodSource Not Present: Strength of Value of Latent Print

(24 Latents)

RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT EXAMINATIONSEXAMINATIONS

25% 6 Latent Prints

Insufficient to Difficult

37.5% 9 Latent Prints

Difficult to Moderate

37.5% 9 Latent Prints

Moderate to Easy

24

Research Design and Method•Study was conducted in 3 Phases over a one year period

RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT EXAMINATIONSEXAMINATIONS

25

Phase 1 (ACE)

109 Participants

40 Latent Trials

Phase 2 (ACE)

88 Participants

40 Latent Trials

(Group A)20 Latent Comparisons

(Group B)20 Latent Comparisons

Phase 3 (ACE-V)

86 Participants

ACE-V 25 Latent Trials

Repeatability27 Latent Trials

High Bias ACE-V37 Latent Trials

High Bias Repeatability24 Latent Trials

Research Design and Method

• Phase 1 and 2– 80 Latent ACE Trials• For each latent trial, participants were asked to conduct an

analysis, comparison and evaluation• Analysis: Rate clarity, identify anatomical source, certainty

of orientation, and whether the impression was of value or of no value.

• Comparison: Compare to 3 of the 10 sets of standards that were provided.

• Evaluation: For each evaluation, the participants were to indicate whether they had made an identification, an exclusion, or an inconclusive decision.

RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT EXAMINATIONSEXAMINATIONS

26

Research Design and Method

• Phase 1and 2– 80 Latent ACE Trials• Answer Sheets

RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT EXAMINATIONSEXAMINATIONS

27

Phase 1 and 2 ACE Trials – AnalysisValue and No Value Decisions

Phase 1109

Participants

Phase 288

Participants

Total Decisions:

Value 3210 1342 4452

No Value 1023 388 1411

Total Decisions:

4233 1730 5963

RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT EXAMINATIONSEXAMINATIONS

28

Phase 1 and 2 ACE Trials – ResultsValue and No Value Decisions (5,963)

• 2,457 Correct Identifications

• 953 Correct Exclusions

• 42 Erroneous Identifications

• 235 Erroneous Exclusions

• 446 Inconclusives when from the same source

• 403 Inconclusives when from different sources

RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT EXAMINATIONSEXAMINATIONS

29

Phase 1 and 2 ACE Trials Error Rates

• False Positive Rate•Percentage of the time the participant made an erroneous identification when given the possibility of making any of the three categorical opinions (Identification, Inconclusive, and Exclusion).

RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT EXAMINATIONSEXAMINATIONS

30

With Inconclusives Without Inconclusives

3.0% 4.2%

The accuracy of ACE and ACE-V examinations are reported as an overall participant error rate after participants made a sufficiency determination that a latent was of “value” for Identification.

Phase 1 and 2 ACE Trials – Results42 Erroneous Identifications

•35 Erroneous IDs – Appear to be Clerical Errors• 19 times correct standards but opposite finger or opposite palm was

chosen• 8 times correct standards and not opposite finger was chosen• 8 times incorrect standards but same finger or palm was chosen

•4 Erroneous IDs - Don’t appear to be Clerical Errors• 4 times incorrect standard and not opposite finger was chosen

•3 Erroneous IDs the source was not present• Based on 2 Latent Prints

RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT EXAMINATIONSEXAMINATIONS

31

Made by 4 different Examiners

Phase 1 and 2 ACE Trials – Results3 Erroneous IDs the source was not present

Latent Print #1

Strength of Value of Latent:

17.00

Correctly Excluded: 31 times

Erroneously Identified: 1 time

Inconclusive:8 times

RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT EXAMINATIONSEXAMINATIONS

32

Latent Print #2

Strength of Value of Latent:

16.00

Correctly Excluded: 33 times

Erroneously Identified: 2 times

Inconclusive:5 times

RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT EXAMINATIONSEXAMINATIONS

Phase 1 and 2 ACE Trials – Results3 Erroneous IDs the source was not present

33

Phase 1 and 2 ACE Trials Error Rates

• False Negative Rate•Percentage of the time the participant made an erroneous exclusion when given the possibility of making any of the three categorical opinions (Identification, Inconclusive, and Exclusion)

RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT EXAMINATIONSEXAMINATIONS

34

With Inconclusives Without Inconclusives

7.5% 8.7%

Phase 3 ACE-V Trials – Results(Verification, Bias & Repeatability)

RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT EXAMINATIONSEXAMINATIONS

35

Phase 3 ACE-V Trials – Results(Verification, Bias & Repeatability)

Phase 3 was comprised of four different groups

• Group 1 – ACE-V Trials• Group 2 – ACE-V Trials (High Bias)• Group 3 – ACE-V Trials (Repeatability)• Group 4 – ACE-V Trials (Repeatability, High Bias)

RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT EXAMINATIONSEXAMINATIONS

36

Phase 3 ACE-V Trials – Results

ACE-V + Repeatability, Answer Sheet

If your comparison agrees with the identification, please mark the box labeled "Agree".

RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT EXAMINATIONSEXAMINATIONS

37

Phase 3 ACE-V Trials – Results

• 487 Correct Identifications (agreed with the correct identification)

• 13 Correct Exclusions (disagreed with incorrect identification)

• 0 Erroneous Identifications (agreed with incorrect identification)

• 15 Erroneous Exclusions (disagreed with the correct identification)

• 15 Inconclusives when from the same source

• 2 Inconclusives when from different sources

RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT EXAMINATIONSEXAMINATIONS

38

Phase 3 ACE-V Trials Error Rates

• False Positive Rate

RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT EXAMINATIONSEXAMINATIONS

39

With Inconclusives Without Inconclusives

0.0% 0.0%

Phase 3 ACE-V Trials Error Rates

• False Negative Rate

RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT EXAMINATIONSEXAMINATIONS

40

With Inconclusives Without Inconclusives

2.9% 3.0%

Phase 3 ACE-V Trials – ResultsACE-V + Repeatability, Answer Sheet – High Bias

RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT EXAMINATIONSEXAMINATIONS

41

Phase 3 ACE-V Trials (Biased Conditions)– Results

• 178 Correct Identifications (agreed with the correct identification)

• 78 Correct Exclusions (disagreed with incorrect identification)

• 3 Erroneous Identifications (agreed with incorrect identification)

• 15 Erroneous Exclusions (disagreed with the correct identification)

• 51 Inconclusives when from the same source

• 4 Inconclusives when from different sources

RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT EXAMINATIONSEXAMINATIONS

42

Phase 3 ACE-V Trials (Biased Conditions)– Results

RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT EXAMINATIONSEXAMINATIONS

43

Phase 3 ACE-V Trials – Repeatability

• To determine if the participants would repeat their own conclusions

from comparisons of unknown latent prints to known standards the

results of identification decisions, erroneous exclusions, and

inconclusive results (where the source was present) from Phase 1

were sent to the same participants in Phase 3.

• 27 latent prints latent prints presented to participants for

repeatability

• 1,311 participant decisions.

RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT EXAMINATIONSEXAMINATIONS

44

Phase 3 ACE-V Trials – Repeatability

RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT EXAMINATIONSEXAMINATIONS

45

Phase 3 ACE-V Trials – Repeatability

RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT EXAMINATIONSEXAMINATIONS

46

Phase 3 ACE-V Trials – Repeatability

RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT EXAMINATIONSEXAMINATIONS

47

Phase 3 ACE-V Trials – Repeatability

RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT EXAMINATIONSEXAMINATIONS

48

Phase 3 ACE-V Trials – Repeatability (Biased Conditions)

• To determine if the participants would repeat their own conclusions

from comparisons of unknown latent prints to known standards under

biased conditions, the results of identification decisions, erroneous

exclusions, and inconclusive results (where the source was present)

from Phase 1 were sent to the same participants in Phase 3.

• 24 latent prints latent prints presented to participants for

repeatability under biased conditions

• 333 participant decisions

RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT EXAMINATIONSEXAMINATIONS

49

Phase 3 ACE-V Trials – Repeatability (Biased Conditions)

RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT EXAMINATIONSEXAMINATIONS

50

Phase 3 ACE-V Trials – Repeatability (Biased Conditions)

RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT EXAMINATIONSEXAMINATIONS

51

1 participant

Phase 3 ACE-V Trials – Repeatability (Biased Conditions)

RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT EXAMINATIONSEXAMINATIONS

52

Phase 3 ACE-V Trials – Repeatability (Biased Conditions)

RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT EXAMINATIONSEXAMINATIONS

53

Summary of ResultsSummary of Results◦5,963 sufficiency determinations, ◦4,536 ACE decisions◦532 ACE-V decisions ◦1,311 repeatability decisions, ◦326 ACE decisions under biased

conditions◦333 repeatability decisions under

biased conditions

54

Summary of ResultsSummary of Results◦For ACE decisions

False Positive Rate of 3.0% False Negative Rate of 7.5%.

◦For ACE-V decisions Positive Rate of 0% False Negative Rate of 2.9%.

55

Summary of ResultsSummary of Results◦Repeatability

94.6% of the time previous correct identifications repeated

93.1% of the time, participants did not repeat their previous erroneous exclusions

56

◦Repeatability -Bias 73.0% of the time

previous correct identifications repeated

96.5% of the time, participants did not repeat their previous erroneous exclusions

In conclusion…In conclusion…◦ With respect to this study

fingerprint examiners are highly accurate and reliable when conducting examinations using a process that is similar to actual casework, in which all fingerprint results are verified by a second fingerprint examiner

examiners are less likely to miss an identification the easier the latent print is to compare, and more likely to report an inconclusive decision when comparing the most difficult latent prints.

examiners are more likely to miss an identification than to report an incorrect identification.

This study indicated that bias may influence decision making. For example, participants who were asked to perform a second verification, in which they were given two previous conclusions, agreed less often with an initial correct identification and reported more inconclusive decisions.

57

THE END58