igor pacheco, clpe (miami-dade police department) brian cerchiai, ctpe (miami-dade police...

58
Igor Pacheco, CLPE (Miami-Dade Police Department) Brian Cerchiai, CTPE (Miami-Dade Police Department) Stephanie Stoiloff, M.S. (Miami-Dade Police Department) Sneh Gulati, Ph.D. (Florida International University) Presented by: Igor Pacheco, CLPE & Brian Cerchiai, CTPE

Upload: william-webster

Post on 18-Dec-2015

227 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Igor Pacheco, CLPE (Miami-Dade Police Department) Brian Cerchiai, CTPE (Miami-Dade Police Department) Stephanie Stoiloff, M.S. (Miami-Dade Police Department)

Igor Pacheco, CLPE (Miami-Dade Police Department)Brian Cerchiai, CTPE (Miami-Dade Police Department)

Stephanie Stoiloff, M.S. (Miami-Dade Police Department)Sneh Gulati, Ph.D. (Florida International University)

Presented by: Igor Pacheco, CLPE & Brian Cerchiai, CTPE

Page 2: Igor Pacheco, CLPE (Miami-Dade Police Department) Brian Cerchiai, CTPE (Miami-Dade Police Department) Stephanie Stoiloff, M.S. (Miami-Dade Police Department)

This project was supported by Award No. 2010-DN-BX-K268 awarded by the National Institute of Justice, Office of Justice Programs, U. S. Department of Justice.

The opinions, findings, and conclusions or recommendations expressed in this publication/program/exhibition are those of the authors and do not necessarily reflect those of the Department of Justice.

RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT EXAMINATIONSEXAMINATIONS

2

Page 3: Igor Pacheco, CLPE (Miami-Dade Police Department) Brian Cerchiai, CTPE (Miami-Dade Police Department) Stephanie Stoiloff, M.S. (Miami-Dade Police Department)

BackgroundThe 2009 National Academy of Sciences’ report titled, “Strengthening Forensic Science in the United States: A Path Forward” announced thirteen recommendations to address challenges facing the forensic science community.

◦ This study was conducted specifically to address concerns of their third recommendation, “Research is needed to address issues of accuracy, reliability, and validity in the forensic science disciplines.”

MDPD study was modeled after previous study by Glen Langengurg◦ Langenburg, G. A Performance Study of the ACE-V Process: A Pilot

Study to Measure the Accuracy, Precision, Reproducibility, Repeatability, and Biasability of Conclusions Resulting from the ACE-V process. J. For. Ident., 2009, 59 (2).

RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT EXAMINATIONSEXAMINATIONS

3

Page 4: Igor Pacheco, CLPE (Miami-Dade Police Department) Brian Cerchiai, CTPE (Miami-Dade Police Department) Stephanie Stoiloff, M.S. (Miami-Dade Police Department)

BackgroundEvaluated the reliability of Latent Fingerprint Examiners using the Analysis, Comparison, Evaluation, and Verification (ACE-V) methodology in latent fingerprint examinations by Measured the Accuracy, Precision, Reproducibility, Repeatability, and Bias of four categorical opinions:

◦ Identification◦ Exclusion◦ Inconclusive◦ No value

RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT EXAMINATIONSEXAMINATIONS

4

Page 5: Igor Pacheco, CLPE (Miami-Dade Police Department) Brian Cerchiai, CTPE (Miami-Dade Police Department) Stephanie Stoiloff, M.S. (Miami-Dade Police Department)

Participation

109 Experienced Latent Print Examiners Participated

◦ Requirements to participate: 1 of active latent casework experience uses the ACE-V methodology enforcement agency (crime laboratory), or like agency, in the

United States. This includes all active, retired or contracted latent examiners.

RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT EXAMINATIONSEXAMINATIONS

5

Page 6: Igor Pacheco, CLPE (Miami-Dade Police Department) Brian Cerchiai, CTPE (Miami-Dade Police Department) Stephanie Stoiloff, M.S. (Miami-Dade Police Department)

Announcement & QuestionnaireParticipants from 75 Different Law Enforcement Agencies

RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT EXAMINATIONSEXAMINATIONS

53Local

Agencies

Page 7: Igor Pacheco, CLPE (Miami-Dade Police Department) Brian Cerchiai, CTPE (Miami-Dade Police Department) Stephanie Stoiloff, M.S. (Miami-Dade Police Department)

Announcement & QuestionnaireParticipants from 75 Different Law Enforcement Agencies

RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT EXAMINATIONSEXAMINATIONS

53Local

Agencies

16State

Agencies

Page 8: Igor Pacheco, CLPE (Miami-Dade Police Department) Brian Cerchiai, CTPE (Miami-Dade Police Department) Stephanie Stoiloff, M.S. (Miami-Dade Police Department)

Announcement & QuestionnaireParticipants from 75 Different Law Enforcement Agencies

RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT EXAMINATIONSEXAMINATIONS

53Local

Agencies

16State

Agencies

Federal Agencies

3

Page 9: Igor Pacheco, CLPE (Miami-Dade Police Department) Brian Cerchiai, CTPE (Miami-Dade Police Department) Stephanie Stoiloff, M.S. (Miami-Dade Police Department)

Announcement & QuestionnaireParticipants from 75 Different Law Enforcement Agencies

RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT EXAMINATIONSEXAMINATIONS

53Local

Agencies

16State

Agencies

Federal Agencies

4N/A3

Page 10: Igor Pacheco, CLPE (Miami-Dade Police Department) Brian Cerchiai, CTPE (Miami-Dade Police Department) Stephanie Stoiloff, M.S. (Miami-Dade Police Department)

Announcement & QuestionnaireLatent Print Examination Experience

RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT EXAMINATIONSEXAMINATIONS

10

Page 11: Igor Pacheco, CLPE (Miami-Dade Police Department) Brian Cerchiai, CTPE (Miami-Dade Police Department) Stephanie Stoiloff, M.S. (Miami-Dade Police Department)

Announcement & QuestionnaireLatent Print Examination Experience

RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT EXAMINATIONSEXAMINATIONS

11

Page 12: Igor Pacheco, CLPE (Miami-Dade Police Department) Brian Cerchiai, CTPE (Miami-Dade Police Department) Stephanie Stoiloff, M.S. (Miami-Dade Police Department)

Announcement & QuestionnaireLatent Print Examination Experience

RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT EXAMINATIONSEXAMINATIONS

12

Page 13: Igor Pacheco, CLPE (Miami-Dade Police Department) Brian Cerchiai, CTPE (Miami-Dade Police Department) Stephanie Stoiloff, M.S. (Miami-Dade Police Department)

Announcement & QuestionnaireLatent Print Examination Experience

RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT EXAMINATIONSEXAMINATIONS

13

Page 14: Igor Pacheco, CLPE (Miami-Dade Police Department) Brian Cerchiai, CTPE (Miami-Dade Police Department) Stephanie Stoiloff, M.S. (Miami-Dade Police Department)

Announcement & QuestionnaireIAI Latent Print Certification

RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT EXAMINATIONSEXAMINATIONS

Certified

Not Certified

14

Page 15: Igor Pacheco, CLPE (Miami-Dade Police Department) Brian Cerchiai, CTPE (Miami-Dade Police Department) Stephanie Stoiloff, M.S. (Miami-Dade Police Department)

Research Questions

1)Will Latent Examiners be able to correctly identify or exclude unknown latent impressions from known standards using the ACE methodology?

2)Will Latent Examiners be able to correctly identify or exclude unknown latent impressions from known standards using the ACE-V methodology?

3)Will Latent Examiners be able to reproduce and repeat conclusions from unknown latent impressions to known standards using the ACE and ACE-V methodology?

4)Will Latent Examiners be able to reproduce and repeat conclusions from unknown latent impression to known standards using the ACE-V methodology under high bias conditions?

RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT EXAMINATIONSEXAMINATIONS

15

Page 16: Igor Pacheco, CLPE (Miami-Dade Police Department) Brian Cerchiai, CTPE (Miami-Dade Police Department) Stephanie Stoiloff, M.S. (Miami-Dade Police Department)

Research Design and MethodThirteen confidential volunteer sources were identified

8 Men & 5 Women chosen due to not having their fingerprints or palmprints in any known AFIS database outside of Miami-Dade County.

13 sets of Tenprint and Palm Print Standards collected 2,711 known latent impressions were created (fingers and palms)

Collected from non-porous flat and curved surfaces (plastic, tile, metal, and glass) Processed using black powder and clear tape on white backing cards (scanned and

printed as photos) The latent impressions included various sizes, clarity, and levels of distortion

RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT EXAMINATIONSEXAMINATIONS

16

Page 17: Igor Pacheco, CLPE (Miami-Dade Police Department) Brian Cerchiai, CTPE (Miami-Dade Police Department) Stephanie Stoiloff, M.S. (Miami-Dade Police Department)

Research Design and MethodThree (3) Certified Latent Print Examiners independently evaluated and compared 320 latent prints to the known standards using a rating scale that was created for this research.

◦ 80 Latent Prints were chosen for the final test setsRating Scales for each latent analysis and comparison:

= (Analysis)

Strength of Value of Latent Print (Comparison)

Latent in Agreement with Standard+2

Difficulty of Comparison

RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT EXAMINATIONSEXAMINATIONS

17

Page 18: Igor Pacheco, CLPE (Miami-Dade Police Department) Brian Cerchiai, CTPE (Miami-Dade Police Department) Stephanie Stoiloff, M.S. (Miami-Dade Police Department)

Strength of Value of Latent Print

0 7 14 21

Insufficient to Difficult

Minutiae: 4.33

MinutiaeFormations: 0.00

Clarity: 0.67

Total: 5.00

Latent in Agreement with

Standard

Difficulty of Comparison: 5.67

4.33

0.00

2.00

6.33

18

Page 19: Igor Pacheco, CLPE (Miami-Dade Police Department) Brian Cerchiai, CTPE (Miami-Dade Police Department) Stephanie Stoiloff, M.S. (Miami-Dade Police Department)

Strength of Value of Latent Print

0 7 14 21

Difficult to Moderate

Minutiae: 10.00

MinutiaeFormations: 1.00

Clarity: 1.33

Total: 12.33

Latent in Agreement with

Standard

Difficulty of Comparison:

10.5

6.00

0.66

2.00

8.66

19

Page 20: Igor Pacheco, CLPE (Miami-Dade Police Department) Brian Cerchiai, CTPE (Miami-Dade Police Department) Stephanie Stoiloff, M.S. (Miami-Dade Police Department)

Strength of Value of Latent Print

0 7 14 21

Moderate to Easy

Minutiae: 14.00

Minutiae Formations: 3.33

Clarity: 2.00

Total: 19.33

Latent in Agreement with

Standard

Difficulty of Comparison: 19.33

14.00

3.33

2.00

19.33

20

Page 21: Igor Pacheco, CLPE (Miami-Dade Police Department) Brian Cerchiai, CTPE (Miami-Dade Police Department) Stephanie Stoiloff, M.S. (Miami-Dade Police Department)

Research Design and MethodDistribution of Latent Prints for Testing

RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT EXAMINATIONSEXAMINATIONS

30% 24 Latent Prints

Source was not Present

70% 56 Latent Prints

Source was Present

21

Page 22: Igor Pacheco, CLPE (Miami-Dade Police Department) Brian Cerchiai, CTPE (Miami-Dade Police Department) Stephanie Stoiloff, M.S. (Miami-Dade Police Department)

Research Design and MethodSource Not Present: Strength of Value of Latent Print

(24 Latents)

RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT EXAMINATIONSEXAMINATIONS

22

25% 6 Latent Prints

Insufficient to Difficult

Page 23: Igor Pacheco, CLPE (Miami-Dade Police Department) Brian Cerchiai, CTPE (Miami-Dade Police Department) Stephanie Stoiloff, M.S. (Miami-Dade Police Department)

Research Design and MethodSource Not Present: Strength of Value of Latent Print

(24 Latents)

RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT EXAMINATIONSEXAMINATIONS

37.5% 9 Latent Prints

Moderate to Easy

23

25% 6 Latent Prints

Insufficient to Difficult

Page 24: Igor Pacheco, CLPE (Miami-Dade Police Department) Brian Cerchiai, CTPE (Miami-Dade Police Department) Stephanie Stoiloff, M.S. (Miami-Dade Police Department)

Research Design and MethodSource Not Present: Strength of Value of Latent Print

(24 Latents)

RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT EXAMINATIONSEXAMINATIONS

25% 6 Latent Prints

Insufficient to Difficult

37.5% 9 Latent Prints

Difficult to Moderate

37.5% 9 Latent Prints

Moderate to Easy

24

Page 25: Igor Pacheco, CLPE (Miami-Dade Police Department) Brian Cerchiai, CTPE (Miami-Dade Police Department) Stephanie Stoiloff, M.S. (Miami-Dade Police Department)

Research Design and Method•Study was conducted in 3 Phases over a one year period

RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT EXAMINATIONSEXAMINATIONS

25

Phase 1 (ACE)

109 Participants

40 Latent Trials

Phase 2 (ACE)

88 Participants

40 Latent Trials

(Group A)20 Latent Comparisons

(Group B)20 Latent Comparisons

Phase 3 (ACE-V)

86 Participants

ACE-V 25 Latent Trials

Repeatability27 Latent Trials

High Bias ACE-V37 Latent Trials

High Bias Repeatability24 Latent Trials

Page 26: Igor Pacheco, CLPE (Miami-Dade Police Department) Brian Cerchiai, CTPE (Miami-Dade Police Department) Stephanie Stoiloff, M.S. (Miami-Dade Police Department)

Research Design and Method

• Phase 1 and 2– 80 Latent ACE Trials• For each latent trial, participants were asked to conduct an

analysis, comparison and evaluation• Analysis: Rate clarity, identify anatomical source, certainty

of orientation, and whether the impression was of value or of no value.

• Comparison: Compare to 3 of the 10 sets of standards that were provided.

• Evaluation: For each evaluation, the participants were to indicate whether they had made an identification, an exclusion, or an inconclusive decision.

RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT EXAMINATIONSEXAMINATIONS

26

Page 27: Igor Pacheco, CLPE (Miami-Dade Police Department) Brian Cerchiai, CTPE (Miami-Dade Police Department) Stephanie Stoiloff, M.S. (Miami-Dade Police Department)

Research Design and Method

• Phase 1and 2– 80 Latent ACE Trials• Answer Sheets

RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT EXAMINATIONSEXAMINATIONS

27

Page 28: Igor Pacheco, CLPE (Miami-Dade Police Department) Brian Cerchiai, CTPE (Miami-Dade Police Department) Stephanie Stoiloff, M.S. (Miami-Dade Police Department)

Phase 1 and 2 ACE Trials – AnalysisValue and No Value Decisions

Phase 1109

Participants

Phase 288

Participants

Total Decisions:

Value 3210 1342 4452

No Value 1023 388 1411

Total Decisions:

4233 1730 5963

RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT EXAMINATIONSEXAMINATIONS

28

Page 29: Igor Pacheco, CLPE (Miami-Dade Police Department) Brian Cerchiai, CTPE (Miami-Dade Police Department) Stephanie Stoiloff, M.S. (Miami-Dade Police Department)

Phase 1 and 2 ACE Trials – ResultsValue and No Value Decisions (5,963)

• 2,457 Correct Identifications

• 953 Correct Exclusions

• 42 Erroneous Identifications

• 235 Erroneous Exclusions

• 446 Inconclusives when from the same source

• 403 Inconclusives when from different sources

RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT EXAMINATIONSEXAMINATIONS

29

Page 30: Igor Pacheco, CLPE (Miami-Dade Police Department) Brian Cerchiai, CTPE (Miami-Dade Police Department) Stephanie Stoiloff, M.S. (Miami-Dade Police Department)

Phase 1 and 2 ACE Trials Error Rates

• False Positive Rate•Percentage of the time the participant made an erroneous identification when given the possibility of making any of the three categorical opinions (Identification, Inconclusive, and Exclusion).

RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT EXAMINATIONSEXAMINATIONS

30

With Inconclusives Without Inconclusives

3.0% 4.2%

The accuracy of ACE and ACE-V examinations are reported as an overall participant error rate after participants made a sufficiency determination that a latent was of “value” for Identification.

Page 31: Igor Pacheco, CLPE (Miami-Dade Police Department) Brian Cerchiai, CTPE (Miami-Dade Police Department) Stephanie Stoiloff, M.S. (Miami-Dade Police Department)

Phase 1 and 2 ACE Trials – Results42 Erroneous Identifications

•35 Erroneous IDs – Appear to be Clerical Errors• 19 times correct standards but opposite finger or opposite palm was

chosen• 8 times correct standards and not opposite finger was chosen• 8 times incorrect standards but same finger or palm was chosen

•4 Erroneous IDs - Don’t appear to be Clerical Errors• 4 times incorrect standard and not opposite finger was chosen

•3 Erroneous IDs the source was not present• Based on 2 Latent Prints

RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT EXAMINATIONSEXAMINATIONS

31

Made by 4 different Examiners

Page 32: Igor Pacheco, CLPE (Miami-Dade Police Department) Brian Cerchiai, CTPE (Miami-Dade Police Department) Stephanie Stoiloff, M.S. (Miami-Dade Police Department)

Phase 1 and 2 ACE Trials – Results3 Erroneous IDs the source was not present

Latent Print #1

Strength of Value of Latent:

17.00

Correctly Excluded: 31 times

Erroneously Identified: 1 time

Inconclusive:8 times

RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT EXAMINATIONSEXAMINATIONS

32

Page 33: Igor Pacheco, CLPE (Miami-Dade Police Department) Brian Cerchiai, CTPE (Miami-Dade Police Department) Stephanie Stoiloff, M.S. (Miami-Dade Police Department)

Latent Print #2

Strength of Value of Latent:

16.00

Correctly Excluded: 33 times

Erroneously Identified: 2 times

Inconclusive:5 times

RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT EXAMINATIONSEXAMINATIONS

Phase 1 and 2 ACE Trials – Results3 Erroneous IDs the source was not present

33

Page 34: Igor Pacheco, CLPE (Miami-Dade Police Department) Brian Cerchiai, CTPE (Miami-Dade Police Department) Stephanie Stoiloff, M.S. (Miami-Dade Police Department)

Phase 1 and 2 ACE Trials Error Rates

• False Negative Rate•Percentage of the time the participant made an erroneous exclusion when given the possibility of making any of the three categorical opinions (Identification, Inconclusive, and Exclusion)

RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT EXAMINATIONSEXAMINATIONS

34

With Inconclusives Without Inconclusives

7.5% 8.7%

Page 35: Igor Pacheco, CLPE (Miami-Dade Police Department) Brian Cerchiai, CTPE (Miami-Dade Police Department) Stephanie Stoiloff, M.S. (Miami-Dade Police Department)

Phase 3 ACE-V Trials – Results(Verification, Bias & Repeatability)

RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT EXAMINATIONSEXAMINATIONS

35

Page 36: Igor Pacheco, CLPE (Miami-Dade Police Department) Brian Cerchiai, CTPE (Miami-Dade Police Department) Stephanie Stoiloff, M.S. (Miami-Dade Police Department)

Phase 3 ACE-V Trials – Results(Verification, Bias & Repeatability)

Phase 3 was comprised of four different groups

• Group 1 – ACE-V Trials• Group 2 – ACE-V Trials (High Bias)• Group 3 – ACE-V Trials (Repeatability)• Group 4 – ACE-V Trials (Repeatability, High Bias)

RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT EXAMINATIONSEXAMINATIONS

36

Page 37: Igor Pacheco, CLPE (Miami-Dade Police Department) Brian Cerchiai, CTPE (Miami-Dade Police Department) Stephanie Stoiloff, M.S. (Miami-Dade Police Department)

Phase 3 ACE-V Trials – Results

ACE-V + Repeatability, Answer Sheet

If your comparison agrees with the identification, please mark the box labeled "Agree".

RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT EXAMINATIONSEXAMINATIONS

37

Page 38: Igor Pacheco, CLPE (Miami-Dade Police Department) Brian Cerchiai, CTPE (Miami-Dade Police Department) Stephanie Stoiloff, M.S. (Miami-Dade Police Department)

Phase 3 ACE-V Trials – Results

• 487 Correct Identifications (agreed with the correct identification)

• 13 Correct Exclusions (disagreed with incorrect identification)

• 0 Erroneous Identifications (agreed with incorrect identification)

• 15 Erroneous Exclusions (disagreed with the correct identification)

• 15 Inconclusives when from the same source

• 2 Inconclusives when from different sources

RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT EXAMINATIONSEXAMINATIONS

38

Page 39: Igor Pacheco, CLPE (Miami-Dade Police Department) Brian Cerchiai, CTPE (Miami-Dade Police Department) Stephanie Stoiloff, M.S. (Miami-Dade Police Department)

Phase 3 ACE-V Trials Error Rates

• False Positive Rate

RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT EXAMINATIONSEXAMINATIONS

39

With Inconclusives Without Inconclusives

0.0% 0.0%

Page 40: Igor Pacheco, CLPE (Miami-Dade Police Department) Brian Cerchiai, CTPE (Miami-Dade Police Department) Stephanie Stoiloff, M.S. (Miami-Dade Police Department)

Phase 3 ACE-V Trials Error Rates

• False Negative Rate

RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT EXAMINATIONSEXAMINATIONS

40

With Inconclusives Without Inconclusives

2.9% 3.0%

Page 41: Igor Pacheco, CLPE (Miami-Dade Police Department) Brian Cerchiai, CTPE (Miami-Dade Police Department) Stephanie Stoiloff, M.S. (Miami-Dade Police Department)

Phase 3 ACE-V Trials – ResultsACE-V + Repeatability, Answer Sheet – High Bias

RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT EXAMINATIONSEXAMINATIONS

41

Page 42: Igor Pacheco, CLPE (Miami-Dade Police Department) Brian Cerchiai, CTPE (Miami-Dade Police Department) Stephanie Stoiloff, M.S. (Miami-Dade Police Department)

Phase 3 ACE-V Trials (Biased Conditions)– Results

• 178 Correct Identifications (agreed with the correct identification)

• 78 Correct Exclusions (disagreed with incorrect identification)

• 3 Erroneous Identifications (agreed with incorrect identification)

• 15 Erroneous Exclusions (disagreed with the correct identification)

• 51 Inconclusives when from the same source

• 4 Inconclusives when from different sources

RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT EXAMINATIONSEXAMINATIONS

42

Page 43: Igor Pacheco, CLPE (Miami-Dade Police Department) Brian Cerchiai, CTPE (Miami-Dade Police Department) Stephanie Stoiloff, M.S. (Miami-Dade Police Department)

Phase 3 ACE-V Trials (Biased Conditions)– Results

RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT EXAMINATIONSEXAMINATIONS

43

Page 44: Igor Pacheco, CLPE (Miami-Dade Police Department) Brian Cerchiai, CTPE (Miami-Dade Police Department) Stephanie Stoiloff, M.S. (Miami-Dade Police Department)

Phase 3 ACE-V Trials – Repeatability

• To determine if the participants would repeat their own conclusions

from comparisons of unknown latent prints to known standards the

results of identification decisions, erroneous exclusions, and

inconclusive results (where the source was present) from Phase 1

were sent to the same participants in Phase 3.

• 27 latent prints latent prints presented to participants for

repeatability

• 1,311 participant decisions.

RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT EXAMINATIONSEXAMINATIONS

44

Page 45: Igor Pacheco, CLPE (Miami-Dade Police Department) Brian Cerchiai, CTPE (Miami-Dade Police Department) Stephanie Stoiloff, M.S. (Miami-Dade Police Department)

Phase 3 ACE-V Trials – Repeatability

RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT EXAMINATIONSEXAMINATIONS

45

Page 46: Igor Pacheco, CLPE (Miami-Dade Police Department) Brian Cerchiai, CTPE (Miami-Dade Police Department) Stephanie Stoiloff, M.S. (Miami-Dade Police Department)

Phase 3 ACE-V Trials – Repeatability

RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT EXAMINATIONSEXAMINATIONS

46

Page 47: Igor Pacheco, CLPE (Miami-Dade Police Department) Brian Cerchiai, CTPE (Miami-Dade Police Department) Stephanie Stoiloff, M.S. (Miami-Dade Police Department)

Phase 3 ACE-V Trials – Repeatability

RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT EXAMINATIONSEXAMINATIONS

47

Page 48: Igor Pacheco, CLPE (Miami-Dade Police Department) Brian Cerchiai, CTPE (Miami-Dade Police Department) Stephanie Stoiloff, M.S. (Miami-Dade Police Department)

Phase 3 ACE-V Trials – Repeatability

RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT EXAMINATIONSEXAMINATIONS

48

Page 49: Igor Pacheco, CLPE (Miami-Dade Police Department) Brian Cerchiai, CTPE (Miami-Dade Police Department) Stephanie Stoiloff, M.S. (Miami-Dade Police Department)

Phase 3 ACE-V Trials – Repeatability (Biased Conditions)

• To determine if the participants would repeat their own conclusions

from comparisons of unknown latent prints to known standards under

biased conditions, the results of identification decisions, erroneous

exclusions, and inconclusive results (where the source was present)

from Phase 1 were sent to the same participants in Phase 3.

• 24 latent prints latent prints presented to participants for

repeatability under biased conditions

• 333 participant decisions

RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT EXAMINATIONSEXAMINATIONS

49

Page 50: Igor Pacheco, CLPE (Miami-Dade Police Department) Brian Cerchiai, CTPE (Miami-Dade Police Department) Stephanie Stoiloff, M.S. (Miami-Dade Police Department)

Phase 3 ACE-V Trials – Repeatability (Biased Conditions)

RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT EXAMINATIONSEXAMINATIONS

50

Page 51: Igor Pacheco, CLPE (Miami-Dade Police Department) Brian Cerchiai, CTPE (Miami-Dade Police Department) Stephanie Stoiloff, M.S. (Miami-Dade Police Department)

Phase 3 ACE-V Trials – Repeatability (Biased Conditions)

RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT EXAMINATIONSEXAMINATIONS

51

1 participant

Page 52: Igor Pacheco, CLPE (Miami-Dade Police Department) Brian Cerchiai, CTPE (Miami-Dade Police Department) Stephanie Stoiloff, M.S. (Miami-Dade Police Department)

Phase 3 ACE-V Trials – Repeatability (Biased Conditions)

RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT EXAMINATIONSEXAMINATIONS

52

Page 53: Igor Pacheco, CLPE (Miami-Dade Police Department) Brian Cerchiai, CTPE (Miami-Dade Police Department) Stephanie Stoiloff, M.S. (Miami-Dade Police Department)

Phase 3 ACE-V Trials – Repeatability (Biased Conditions)

RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT EXAMINATIONSEXAMINATIONS

53

Page 54: Igor Pacheco, CLPE (Miami-Dade Police Department) Brian Cerchiai, CTPE (Miami-Dade Police Department) Stephanie Stoiloff, M.S. (Miami-Dade Police Department)

Summary of ResultsSummary of Results◦5,963 sufficiency determinations, ◦4,536 ACE decisions◦532 ACE-V decisions ◦1,311 repeatability decisions, ◦326 ACE decisions under biased

conditions◦333 repeatability decisions under

biased conditions

54

Page 55: Igor Pacheco, CLPE (Miami-Dade Police Department) Brian Cerchiai, CTPE (Miami-Dade Police Department) Stephanie Stoiloff, M.S. (Miami-Dade Police Department)

Summary of ResultsSummary of Results◦For ACE decisions

False Positive Rate of 3.0% False Negative Rate of 7.5%.

◦For ACE-V decisions Positive Rate of 0% False Negative Rate of 2.9%.

55

Page 56: Igor Pacheco, CLPE (Miami-Dade Police Department) Brian Cerchiai, CTPE (Miami-Dade Police Department) Stephanie Stoiloff, M.S. (Miami-Dade Police Department)

Summary of ResultsSummary of Results◦Repeatability

94.6% of the time previous correct identifications repeated

93.1% of the time, participants did not repeat their previous erroneous exclusions

56

◦Repeatability -Bias 73.0% of the time

previous correct identifications repeated

96.5% of the time, participants did not repeat their previous erroneous exclusions

Page 57: Igor Pacheco, CLPE (Miami-Dade Police Department) Brian Cerchiai, CTPE (Miami-Dade Police Department) Stephanie Stoiloff, M.S. (Miami-Dade Police Department)

In conclusion…In conclusion…◦ With respect to this study

fingerprint examiners are highly accurate and reliable when conducting examinations using a process that is similar to actual casework, in which all fingerprint results are verified by a second fingerprint examiner

examiners are less likely to miss an identification the easier the latent print is to compare, and more likely to report an inconclusive decision when comparing the most difficult latent prints.

examiners are more likely to miss an identification than to report an incorrect identification.

This study indicated that bias may influence decision making. For example, participants who were asked to perform a second verification, in which they were given two previous conclusions, agreed less often with an initial correct identification and reported more inconclusive decisions.

57

Page 58: Igor Pacheco, CLPE (Miami-Dade Police Department) Brian Cerchiai, CTPE (Miami-Dade Police Department) Stephanie Stoiloff, M.S. (Miami-Dade Police Department)

THE END58