FBI Laboratory Decision Analysis Studies in Pattern Evidence Examinations

FBI Laboratory Decision Analysis Studies in Pattern Evidence Examinations

← Back to Webinar Series Page

This webinar originally occurred on March 19, 2024
Duration: 1 hour


From 2007 through the present, the FBI Laboratory has led a series of decision analysis “black-box” studies aimed at evaluating the accuracy, repeatability, and reproducibility of pattern evidence examiner decisions in forensic casework.  The findings of this research have been presented at more than a dozen scientific conferences and published in more than 20 scientific papers, with several additional publications in development.  The results of this research can inform researchers, casework examiners, and the judicial system regarding the significance and effectiveness of forensic pattern evidence examination.   

This research has also sparked significant discussion at conferences, in the trade press, and in scientific literature, with commentary on and responses to various aspects of the study. This presentation will focus on addressing and responding to the most frequently encountered critical observations regarding the experimental design and interpretation of results.   

Critical factors in project management and experimental design include securing funding and logistics; definitions of objectives; method independence; design of experiments, such as open set design, randomization in samples, sample collection, digital versus physical samples, selection of close non-matches, and numbers of participants, samples, and decisions; sample difficulty and introduced biases to create bounding scenarios; participant recruitment, representativeness, and human subject research protections; and harmonization and imposition of potentially unfamiliar reporting scales.   

Factors affecting the interpretation of results include participant drop-out and non-response; volunteer bias; the Hawthorne effect; calculation of accuracy, repeatability, and reproducibility rates; analysis of covariates among metadata and participant demographics; absence of quality control measures and secondary reviews.    

One critical area for consideration is how inconclusive results are treated in research, especially compared to forensic casework where ground truth is often unknown and unknowable. Following the publication of our research results, a variety of approaches have been suggested, each with its own set of advantages and disadvantages. 

Detailed Learning Objectives

  1. Attendees will gain insight as to how this research was designed to assess reliability of pattern evidence examinations. 
  2. Attendees will develop a deeper understanding of the context behind the results and the inherent challenges in their interpretation. 
  3. Attendees will understand the ongoing dialogue about the admissibility of pattern evidence testimony. 


  • Gene Peters | Chief, Forensic Science Research, FBI Laboratory 

Funding for this Forensic Technology Center of Excellence webinar has been provided by the National Institute of Justice, Office of Justice Programs, U.S. Department of Justice.

The opinions, findings, and conclusions or recommendations expressed in this webinar are those of the presenter(s) and do not necessarily reflect those of the U.S. Department of Justice.

Contact us at with any questions and subscribe to our newsletter for notifications.

Related Content

DNA Recovery After Sequential Processing of Latent Fingerprints on Black Polyethylene Plastic

Publication Journal of Forensic Sciences, February 2024  Authors Abigail S. Bathrick | Bode Technology Sarah Norsworthy | RTI International Dane T. Plaza | Bode Technology Mallory N. McCormick | United States Secret Service Donia Slack | RTI International Robert S. Ramotowski | United States Secret Service …

What FSSP Leaders Should Know about Artificial Intelligence and its Application to Forensic Science In-Brief

Date December 2023 Overview Artificial intelligence (AI), which leverages computers to perform tasks that enhance decision-making, problem-solving capabilities, and technology-driven innovativeness,” is a rapidly growing area of interest for both industry and the federal research and development (R&D) community.1 For forensic science service…

FLN-TWG: A Roadmap to Improve Research and Technology Transition in Forensic Science

← Back to FLN-TWG Main Page  Forensic Laboratory Needs Technology Working Group (FLN-TWG) The National Institute of Justice (NIJ), in partnership with the Forensic Technology Center of Excellence (FTCOE) at RTI International, formed the Forensic Laboratory Needs Technology Working Group…