igor pacheco, clpe (miami-dade police department) brian cerchiai, ctpe (miami-dade police...
TRANSCRIPT
Igor Pacheco, CLPE (Miami-Dade Police Department)Brian Cerchiai, CTPE (Miami-Dade Police Department)
Stephanie Stoiloff, M.S. (Miami-Dade Police Department)Sneh Gulati, Ph.D. (Florida International University)
Presented by: Igor Pacheco, CLPE & Brian Cerchiai, CTPE
This project was supported by Award No. 2010-DN-BX-K268 awarded by the National Institute of Justice, Office of Justice Programs, U. S. Department of Justice.
The opinions, findings, and conclusions or recommendations expressed in this publication/program/exhibition are those of the authors and do not necessarily reflect those of the Department of Justice.
RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT EXAMINATIONSEXAMINATIONS
2
BackgroundThe 2009 National Academy of Sciences’ report titled, “Strengthening Forensic Science in the United States: A Path Forward” announced thirteen recommendations to address challenges facing the forensic science community.
◦ This study was conducted specifically to address concerns of their third recommendation, “Research is needed to address issues of accuracy, reliability, and validity in the forensic science disciplines.”
MDPD study was modeled after previous study by Glen Langengurg◦ Langenburg, G. A Performance Study of the ACE-V Process: A Pilot
Study to Measure the Accuracy, Precision, Reproducibility, Repeatability, and Biasability of Conclusions Resulting from the ACE-V process. J. For. Ident., 2009, 59 (2).
RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT EXAMINATIONSEXAMINATIONS
3
BackgroundEvaluated the reliability of Latent Fingerprint Examiners using the Analysis, Comparison, Evaluation, and Verification (ACE-V) methodology in latent fingerprint examinations by Measured the Accuracy, Precision, Reproducibility, Repeatability, and Bias of four categorical opinions:
◦ Identification◦ Exclusion◦ Inconclusive◦ No value
RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT EXAMINATIONSEXAMINATIONS
4
Participation
109 Experienced Latent Print Examiners Participated
◦ Requirements to participate: 1 of active latent casework experience uses the ACE-V methodology enforcement agency (crime laboratory), or like agency, in the
United States. This includes all active, retired or contracted latent examiners.
RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT EXAMINATIONSEXAMINATIONS
5
Announcement & QuestionnaireParticipants from 75 Different Law Enforcement Agencies
RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT EXAMINATIONSEXAMINATIONS
53Local
Agencies
Announcement & QuestionnaireParticipants from 75 Different Law Enforcement Agencies
RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT EXAMINATIONSEXAMINATIONS
53Local
Agencies
16State
Agencies
Announcement & QuestionnaireParticipants from 75 Different Law Enforcement Agencies
RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT EXAMINATIONSEXAMINATIONS
53Local
Agencies
16State
Agencies
Federal Agencies
3
Announcement & QuestionnaireParticipants from 75 Different Law Enforcement Agencies
RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT EXAMINATIONSEXAMINATIONS
53Local
Agencies
16State
Agencies
Federal Agencies
4N/A3
Announcement & QuestionnaireLatent Print Examination Experience
RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT EXAMINATIONSEXAMINATIONS
10
Announcement & QuestionnaireLatent Print Examination Experience
RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT EXAMINATIONSEXAMINATIONS
11
Announcement & QuestionnaireLatent Print Examination Experience
RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT EXAMINATIONSEXAMINATIONS
12
Announcement & QuestionnaireLatent Print Examination Experience
RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT EXAMINATIONSEXAMINATIONS
13
Announcement & QuestionnaireIAI Latent Print Certification
RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT EXAMINATIONSEXAMINATIONS
Certified
Not Certified
14
Research Questions
1)Will Latent Examiners be able to correctly identify or exclude unknown latent impressions from known standards using the ACE methodology?
2)Will Latent Examiners be able to correctly identify or exclude unknown latent impressions from known standards using the ACE-V methodology?
3)Will Latent Examiners be able to reproduce and repeat conclusions from unknown latent impressions to known standards using the ACE and ACE-V methodology?
4)Will Latent Examiners be able to reproduce and repeat conclusions from unknown latent impression to known standards using the ACE-V methodology under high bias conditions?
RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT EXAMINATIONSEXAMINATIONS
15
Research Design and MethodThirteen confidential volunteer sources were identified
8 Men & 5 Women chosen due to not having their fingerprints or palmprints in any known AFIS database outside of Miami-Dade County.
13 sets of Tenprint and Palm Print Standards collected 2,711 known latent impressions were created (fingers and palms)
Collected from non-porous flat and curved surfaces (plastic, tile, metal, and glass) Processed using black powder and clear tape on white backing cards (scanned and
printed as photos) The latent impressions included various sizes, clarity, and levels of distortion
RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT EXAMINATIONSEXAMINATIONS
16
Research Design and MethodThree (3) Certified Latent Print Examiners independently evaluated and compared 320 latent prints to the known standards using a rating scale that was created for this research.
◦ 80 Latent Prints were chosen for the final test setsRating Scales for each latent analysis and comparison:
= (Analysis)
Strength of Value of Latent Print (Comparison)
Latent in Agreement with Standard+2
Difficulty of Comparison
RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT EXAMINATIONSEXAMINATIONS
17
Strength of Value of Latent Print
0 7 14 21
Insufficient to Difficult
Minutiae: 4.33
MinutiaeFormations: 0.00
Clarity: 0.67
Total: 5.00
Latent in Agreement with
Standard
Difficulty of Comparison: 5.67
4.33
0.00
2.00
6.33
18
Strength of Value of Latent Print
0 7 14 21
Difficult to Moderate
Minutiae: 10.00
MinutiaeFormations: 1.00
Clarity: 1.33
Total: 12.33
Latent in Agreement with
Standard
Difficulty of Comparison:
10.5
6.00
0.66
2.00
8.66
19
Strength of Value of Latent Print
0 7 14 21
Moderate to Easy
Minutiae: 14.00
Minutiae Formations: 3.33
Clarity: 2.00
Total: 19.33
Latent in Agreement with
Standard
Difficulty of Comparison: 19.33
14.00
3.33
2.00
19.33
20
Research Design and MethodDistribution of Latent Prints for Testing
RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT EXAMINATIONSEXAMINATIONS
30% 24 Latent Prints
Source was not Present
70% 56 Latent Prints
Source was Present
21
Research Design and MethodSource Not Present: Strength of Value of Latent Print
(24 Latents)
RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT EXAMINATIONSEXAMINATIONS
22
25% 6 Latent Prints
Insufficient to Difficult
Research Design and MethodSource Not Present: Strength of Value of Latent Print
(24 Latents)
RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT EXAMINATIONSEXAMINATIONS
37.5% 9 Latent Prints
Moderate to Easy
23
25% 6 Latent Prints
Insufficient to Difficult
Research Design and MethodSource Not Present: Strength of Value of Latent Print
(24 Latents)
RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT EXAMINATIONSEXAMINATIONS
25% 6 Latent Prints
Insufficient to Difficult
37.5% 9 Latent Prints
Difficult to Moderate
37.5% 9 Latent Prints
Moderate to Easy
24
Research Design and Method•Study was conducted in 3 Phases over a one year period
RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT EXAMINATIONSEXAMINATIONS
25
Phase 1 (ACE)
109 Participants
40 Latent Trials
Phase 2 (ACE)
88 Participants
40 Latent Trials
(Group A)20 Latent Comparisons
(Group B)20 Latent Comparisons
Phase 3 (ACE-V)
86 Participants
ACE-V 25 Latent Trials
Repeatability27 Latent Trials
High Bias ACE-V37 Latent Trials
High Bias Repeatability24 Latent Trials
Research Design and Method
• Phase 1 and 2– 80 Latent ACE Trials• For each latent trial, participants were asked to conduct an
analysis, comparison and evaluation• Analysis: Rate clarity, identify anatomical source, certainty
of orientation, and whether the impression was of value or of no value.
• Comparison: Compare to 3 of the 10 sets of standards that were provided.
• Evaluation: For each evaluation, the participants were to indicate whether they had made an identification, an exclusion, or an inconclusive decision.
RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT EXAMINATIONSEXAMINATIONS
26
Research Design and Method
• Phase 1and 2– 80 Latent ACE Trials• Answer Sheets
RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT EXAMINATIONSEXAMINATIONS
27
Phase 1 and 2 ACE Trials – AnalysisValue and No Value Decisions
Phase 1109
Participants
Phase 288
Participants
Total Decisions:
Value 3210 1342 4452
No Value 1023 388 1411
Total Decisions:
4233 1730 5963
RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT EXAMINATIONSEXAMINATIONS
28
Phase 1 and 2 ACE Trials – ResultsValue and No Value Decisions (5,963)
• 2,457 Correct Identifications
• 953 Correct Exclusions
• 42 Erroneous Identifications
• 235 Erroneous Exclusions
• 446 Inconclusives when from the same source
• 403 Inconclusives when from different sources
RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT EXAMINATIONSEXAMINATIONS
29
Phase 1 and 2 ACE Trials Error Rates
• False Positive Rate•Percentage of the time the participant made an erroneous identification when given the possibility of making any of the three categorical opinions (Identification, Inconclusive, and Exclusion).
RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT EXAMINATIONSEXAMINATIONS
30
With Inconclusives Without Inconclusives
3.0% 4.2%
The accuracy of ACE and ACE-V examinations are reported as an overall participant error rate after participants made a sufficiency determination that a latent was of “value” for Identification.
Phase 1 and 2 ACE Trials – Results42 Erroneous Identifications
•35 Erroneous IDs – Appear to be Clerical Errors• 19 times correct standards but opposite finger or opposite palm was
chosen• 8 times correct standards and not opposite finger was chosen• 8 times incorrect standards but same finger or palm was chosen
•4 Erroneous IDs - Don’t appear to be Clerical Errors• 4 times incorrect standard and not opposite finger was chosen
•3 Erroneous IDs the source was not present• Based on 2 Latent Prints
RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT EXAMINATIONSEXAMINATIONS
31
Made by 4 different Examiners
Phase 1 and 2 ACE Trials – Results3 Erroneous IDs the source was not present
Latent Print #1
Strength of Value of Latent:
17.00
Correctly Excluded: 31 times
Erroneously Identified: 1 time
Inconclusive:8 times
RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT EXAMINATIONSEXAMINATIONS
32
Latent Print #2
Strength of Value of Latent:
16.00
Correctly Excluded: 33 times
Erroneously Identified: 2 times
Inconclusive:5 times
RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT EXAMINATIONSEXAMINATIONS
Phase 1 and 2 ACE Trials – Results3 Erroneous IDs the source was not present
33
Phase 1 and 2 ACE Trials Error Rates
• False Negative Rate•Percentage of the time the participant made an erroneous exclusion when given the possibility of making any of the three categorical opinions (Identification, Inconclusive, and Exclusion)
RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT EXAMINATIONSEXAMINATIONS
34
With Inconclusives Without Inconclusives
7.5% 8.7%
Phase 3 ACE-V Trials – Results(Verification, Bias & Repeatability)
RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT EXAMINATIONSEXAMINATIONS
35
Phase 3 ACE-V Trials – Results(Verification, Bias & Repeatability)
Phase 3 was comprised of four different groups
• Group 1 – ACE-V Trials• Group 2 – ACE-V Trials (High Bias)• Group 3 – ACE-V Trials (Repeatability)• Group 4 – ACE-V Trials (Repeatability, High Bias)
RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT EXAMINATIONSEXAMINATIONS
36
Phase 3 ACE-V Trials – Results
ACE-V + Repeatability, Answer Sheet
If your comparison agrees with the identification, please mark the box labeled "Agree".
RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT EXAMINATIONSEXAMINATIONS
37
Phase 3 ACE-V Trials – Results
• 487 Correct Identifications (agreed with the correct identification)
• 13 Correct Exclusions (disagreed with incorrect identification)
• 0 Erroneous Identifications (agreed with incorrect identification)
• 15 Erroneous Exclusions (disagreed with the correct identification)
• 15 Inconclusives when from the same source
• 2 Inconclusives when from different sources
RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT EXAMINATIONSEXAMINATIONS
38
Phase 3 ACE-V Trials Error Rates
• False Positive Rate
RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT EXAMINATIONSEXAMINATIONS
39
With Inconclusives Without Inconclusives
0.0% 0.0%
Phase 3 ACE-V Trials Error Rates
• False Negative Rate
RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT EXAMINATIONSEXAMINATIONS
40
With Inconclusives Without Inconclusives
2.9% 3.0%
Phase 3 ACE-V Trials – ResultsACE-V + Repeatability, Answer Sheet – High Bias
RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT EXAMINATIONSEXAMINATIONS
41
Phase 3 ACE-V Trials (Biased Conditions)– Results
• 178 Correct Identifications (agreed with the correct identification)
• 78 Correct Exclusions (disagreed with incorrect identification)
• 3 Erroneous Identifications (agreed with incorrect identification)
• 15 Erroneous Exclusions (disagreed with the correct identification)
• 51 Inconclusives when from the same source
• 4 Inconclusives when from different sources
RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT EXAMINATIONSEXAMINATIONS
42
Phase 3 ACE-V Trials (Biased Conditions)– Results
RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT EXAMINATIONSEXAMINATIONS
43
Phase 3 ACE-V Trials – Repeatability
• To determine if the participants would repeat their own conclusions
from comparisons of unknown latent prints to known standards the
results of identification decisions, erroneous exclusions, and
inconclusive results (where the source was present) from Phase 1
were sent to the same participants in Phase 3.
• 27 latent prints latent prints presented to participants for
repeatability
• 1,311 participant decisions.
RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT EXAMINATIONSEXAMINATIONS
44
Phase 3 ACE-V Trials – Repeatability
RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT EXAMINATIONSEXAMINATIONS
45
Phase 3 ACE-V Trials – Repeatability
RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT EXAMINATIONSEXAMINATIONS
46
Phase 3 ACE-V Trials – Repeatability
RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT EXAMINATIONSEXAMINATIONS
47
Phase 3 ACE-V Trials – Repeatability
RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT EXAMINATIONSEXAMINATIONS
48
Phase 3 ACE-V Trials – Repeatability (Biased Conditions)
• To determine if the participants would repeat their own conclusions
from comparisons of unknown latent prints to known standards under
biased conditions, the results of identification decisions, erroneous
exclusions, and inconclusive results (where the source was present)
from Phase 1 were sent to the same participants in Phase 3.
• 24 latent prints latent prints presented to participants for
repeatability under biased conditions
• 333 participant decisions
RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT EXAMINATIONSEXAMINATIONS
49
Phase 3 ACE-V Trials – Repeatability (Biased Conditions)
RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT EXAMINATIONSEXAMINATIONS
50
Phase 3 ACE-V Trials – Repeatability (Biased Conditions)
RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT EXAMINATIONSEXAMINATIONS
51
1 participant
Phase 3 ACE-V Trials – Repeatability (Biased Conditions)
RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT EXAMINATIONSEXAMINATIONS
52
Phase 3 ACE-V Trials – Repeatability (Biased Conditions)
RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:RESEARCH STUDY FOR THE RELIABILITY OF THE ACE-V PROCESS:ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT ACCURACY, PRECISION, REPRODUCIBILITY, AND REPEATABILITY IN LATENT FINGERPRINT EXAMINATIONSEXAMINATIONS
53
Summary of ResultsSummary of Results◦5,963 sufficiency determinations, ◦4,536 ACE decisions◦532 ACE-V decisions ◦1,311 repeatability decisions, ◦326 ACE decisions under biased
conditions◦333 repeatability decisions under
biased conditions
54
Summary of ResultsSummary of Results◦For ACE decisions
False Positive Rate of 3.0% False Negative Rate of 7.5%.
◦For ACE-V decisions Positive Rate of 0% False Negative Rate of 2.9%.
55
Summary of ResultsSummary of Results◦Repeatability
94.6% of the time previous correct identifications repeated
93.1% of the time, participants did not repeat their previous erroneous exclusions
56
◦Repeatability -Bias 73.0% of the time
previous correct identifications repeated
96.5% of the time, participants did not repeat their previous erroneous exclusions
In conclusion…In conclusion…◦ With respect to this study
fingerprint examiners are highly accurate and reliable when conducting examinations using a process that is similar to actual casework, in which all fingerprint results are verified by a second fingerprint examiner
examiners are less likely to miss an identification the easier the latent print is to compare, and more likely to report an inconclusive decision when comparing the most difficult latent prints.
examiners are more likely to miss an identification than to report an incorrect identification.
This study indicated that bias may influence decision making. For example, participants who were asked to perform a second verification, in which they were given two previous conclusions, agreed less often with an initial correct identification and reported more inconclusive decisions.
57
THE END58