Reframing the debate: A question of probability, not of likelihood ratio

A practical treatment of sensitivity analyses in activity level evaluations

Evaluations of forensic observations considering activity level propositions are becoming more common place in forensic institutions. A measure that can be taken to interrogate the evaluation for robustness is called sensitivity analysis. A sensitivity analysis explores the sensitivity of the evaluation to the data used when assigning probabilities, or to the level of uncertainty surrounding a probability assignment, or to the choice of various assumptions within the model. There have been a number of publications that describe sensitivity analysis in technical terms, and demonstrate their use, but limited literature on how that theory can be applied in practice. In this work we provide some simplified examples of how sensitivity analyses can be carried out, when they are likely to show that the evaluation is sensitive to underlying data, knowledge or assumptions, how to interpret the results of sensitivity analysis, and how the outcome can be reported. We also provide access to an application to conduct sensitivity analysis.

A Review of Probabilistic Genotyping Systems: EuroForMix, DNAStatistX and STRmix™

Probabilistic genotyping has become widespread. EuroForMix and DNAStatistX are both based upon maximum likelihood estimation using a γ model, whereas STRmix™ is a Bayesian approach that specifies prior distributions on the unknown model parameters. A general overview is provided of the historical development of probabilistic genotyping. Some general principles of interpretation are described, including: the application to investigative vs. evaluative reporting; detection of contamination events; inter and intra laboratory studies; numbers of contributors; proposition setting and validation of software and its performance. This is followed by details of the evolution, utility, practice and adoption of the software discussed.

Bayesian multivariate models for case assessment in dynamic signature cases

Dynamic signatures are recordings of signatures made on digitizing devices such as tablet PCs. These handwritten signatures contain both dynamic and spatial information on every data point collected during the signature movement and can therefore be described in the form of multivariate data. The management of dynamic signatures represents a challenge for the forensic science community through its novelty and the volume of data available. Much as for static signatures, the authenticity of dynamic signatures may be doubted, which leads to a forensic examination of the unknown source signature. The Bayes' factor, as measure of evidential support, can be assigned with statistical models to discriminate between competing propositions. In this respect, the limitations of existing probabilistic solutions to deal with dynamic signature evidence is pointed out and explained in detail. In particular, the necessity to remove the independence assumption between questioned and reference material is emphasized.

How can courts take into account the uncertainty in a likelihood ratio?

As legal practitioners and courts become more aware of scientific methods and evidence evaluation, they are demanding measures of the reliability of expert opinion. In particular, there are calls for error rates to accompany opinion evidence in comparative forensic sciences. While error rates or confidence intervals can be useful for those disciplines that claim to identify the source of a trace, the call for these statistical tools has extended to sciences that present opinions in the form of a likelihood ratio. In this article we argue against presenting both a likelihood ratio and numerical measures of its uncertainty. We explain how the LR already encapsulates uncertainty. Instead we consider how sensitivity analyses can be used to guide the presentation of LRs that are informative to the court and not unfair to defendants.

Interpol review of glass and paint evidence 2016-2019

This review paper covers the forensic-relevant literature in paint and glass evidence from 2016 to 2019 as a part of the 19th Interpol International Forensic Science Managers Symposium. The review papers are also available at the Interpol website at: https://www.interpol.int/content/download/14458/file/Interpol%20Review%20Papers%202019.pdf.

A review of causal inference in forensic medicine

The primary aim of forensic medical analysis is to provide legal factfinders with evidence regarding the causal relationship between an alleged action and a harmful outcome. Despite existing guides and manuals, the approach to formulating opinions on medicolegal causal inference used by forensic medical practitioners, and how the strength of the opinion is quantified, is mostly lacking in an evidence-based or systematically reproducible framework. In the present review, we discuss the literature describing existing methods of causal inference in forensic medicine, especially in relation to the formulation of expert opinions in legal proceedings, and their strengths and limitations. Causal inference in forensic medicine is unique and different from the process of establishing a diagnosis in clinical medicine. Because of a lack of tangibility inherent in causal analysis, even the term “cause” can have inconsistent meaning when used by different practitioners examining the same evidence. Currently, there exists no universally applied systematic methodology for formulating and assessing causality in forensic medical expert opinions. Existing approaches to causation in forensic medicine generally fall into two categories: intuitive and probabilistic. The propriety of each approach depends on the individual facts of an investigated injury, disease, or death. We opine that in most forensic medical settings, probabilistic causation is the most suitable for use and readily applicable. Forensic medical practitioners need, however, be aware of the appropriate approach to causation for different types of cases with varying degrees of complexity.

The Limits of Bayesian Thinking in Court

We comment on the contributions of Dahlman and of Fenton et al., who both suggested a Bayesian approach to analyze the Simonshaven case. We argue that analyzing a full case with a Bayesian approach is not feasible, and that there are serious problems with assigning actual numbers to probabilities and priors. We also discuss the nature of Bayesian thinking in court, and the nature and interpretation of the likelihood ratio. In particular, we discuss what it could mean that a likelihood ratio is in some sense uncertain.

Juror comprehension of forensic expert testimony: A literature review and gap analysis

Forensic scientists and commentators including academics and statisticians have been embroiled in a debate over the best way to present evidence in the courtroom. Various forms of evidence presentation, both quantitative and qualitative, have been championed, yet amidst the furor over the most "correct" or "accurate" way to present evidence, the perspective of the fact-finder is often lost. Without comprehension, correctness is moot. Unbeknownst to many forensic practitioners, there is a large, though incomplete, body of literature from the cognitive psychology domain that explores the question of what jurors understand when forensic scientists testify. This body of work has begun to test different proposed methods of testimony in an effort to understand which are most effective at communicating the strength of evidence that is intended by the expert. This article is a review of that literature that is intended for the forensic scientist community. Its aim is to educate that community on the findings of completed studies and to identify suggestions for further research that will inform changes in testimony delivery and ensure that any modifications can be implemented with confidence in their effectiveness.

A response to "Likelihood ratio as weight of evidence: A closer look" by Lund and Iyer

Recently, Lund and Iyer (L&I) raised an argument regarding the use of likelihood ratios in court. In our view, their argument is based on a lack of understanding of the paradigm. L&I argue that the decision maker should not accept the expert's likelihood ratio without further consideration. This is agreed by all parties. In normal practice, there is often considerable and proper exploration in court of the basis for any probabilistic statement. We conclude that L&I argue against a practice that does not exist and which no one advocates. Further we conclude that the most informative summary of evidential weight is the likelihood ratio. We state that this is the summary that should be presented to a court in every scientific assessment of evidential weight with supporting information about how it was constructed and on what it was based.

Forensic likelihood ratio: Statistical problems and pitfalls

This article is a response to the position papers published in the Science & Justice virtual special issue on measuring and reporting the precision of forensic likelihood ratios. I point out a number of serious statistical errors in some of these papers. These issues need to be properly addressed before the philosophical debate can be conducted in earnest.