Evaluation of transfer evidence for three-level multivariate data with the use of graphical models

Toward Developing Techniques─Agnostic Machine Learning Classification Models for Forensically Relevant Glass Fragments

Glass fragments found in crime scenes may constitute important forensic evidence when properly analyzed, for example, to determine their origin. This analysis could be greatly helped by having a large and diverse database of glass fragments and by using it for constructing reliable machine learning (ML)-based glass classification models. Ideally, the samples that make up this database should be analyzed by a single accurate and standardized analytical technique. However, due to differences in equipment across laboratories, this is not feasible. With this in mind, in this work, we investigated if and how measurement performed at different laboratories on the same set of glass fragments could be combined in the context of ML. First, we demonstrated that elemental analysis methods such as particle-induced X-ray emission (PIXE), laser ablation inductively coupled plasma mass spectrometry (LA-ICP-MS), scanning electron microscopy with energy-dispersive X-ray spectrometry (SEM-EDS), particle-induced Gamma-ray emission (PIGE), instrumental neutron activation analysis (INAA), and prompt Gamma-ray neutron activation analysis (PGAA) could each produce lab-specific ML-based classification models. Next, we determined rules for the successful combinations of data from different laboratories and techniques and demonstrated that when followed, they give rise to improved models, and conversely, poor combinations will lead to poor-performing models. Thus, the combination of PIXE and LA-ICP-MS improves the performances by ∼10-15%, while combining PGAA with other techniques provides poorer performances in comparison with the lab-specific models. Finally, we demonstrated that the poor performances of the SEM-EDS technique, still in use by law enforcement agencies, could be greatly improved by replacing SEM-EDS measurements for Fe and Ca by PIXE measurements for these elements. These findings suggest a process whereby forensic laboratories using different elemental analysis techniques could upload their data into a unified database and get reliable classification based on lab-agnostic models. This in turn brings us closer to a more exhaustive extraction of information from glass fragment evidence and furthermore may form the basis for international-wide collaboration between law enforcement agencies.

PIXE based, Machine-Learning (PIXEL) supported workflow for glass fragments classification

This paper presents a structured workflow for glass fragment analysis based on a combination of Elemental Analysis using PIXE and Machine Learning tools, with the ultimate goal of standardizing and helping forensic efforts. The proposed workflow was implemented on glass fragments received from the Israeli DIFS (Israeli Police Force's Division of Identification and Forensic Sciences) that were collected from various vehicles, including glass fragments from different manufacturers and years of production. We demonstrate that this workflow can produce models with high (>80%) accuracy in identifying glass fragment's origins and provide a test-case demonstrating how the model can be applied in real-life forensic events. We provide a standard, reproducible methodology that can be used in many forensic domains beyond glass fragments, for example, Gun Shot Residue, flammable liquids, illegal substances, and more.

Bayesian Hierarchical Random Effects Models in Forensic Science

Statistical modeling of the evaluation of evidence with the use of the likelihood ratio has a long history. It dates from the Dreyfus case at the end of the nineteenth century through the work at Bletchley Park in the Second World War to the present day. The development received a significant boost in 1977 with a seminal work by Dennis Lindley which introduced a Bayesian hierarchical random effects model for the evaluation of evidence with an example of refractive index measurements on fragments of glass. Many models have been developed since then. The methods have now been sufficiently well-developed and have become so widespread that it is timely to try and provide a software package to assist in their implementation. With that in mind, a project (SAILR: Software for the Analysis and Implementation of Likelihood Ratios) was funded by the European Network of Forensic Science Institutes through their Monopoly programme to develop a software package for use by forensic scientists world-wide that would assist in the statistical analysis and implementation of the approach based on likelihood ratios. It is the purpose of this document to provide a short review of a small part of this history. The review also provides a background, or landscape, for the development of some of the models within the SAILR package and references to SAILR as made as appropriate.

Evidence evaluation for discrete data

Methods for the evaluation of evidence in the form of measurements by means of the likelihood ratio are becoming more widespread. There is a paucity of methods for the evaluation of evidence in the form of counts by means of the likelihood ratio. Two suggestions for such methods are described. Examples of their performance are illustrated in the context of a problem in forensic phonetics. There is discussion of the problems particular to the evaluation of evidence for discrete data, with suggestions for further work.

Mixed effects modelling for glass category estimation from glass refractive indices

520 Glass fragments were taken from 105 glass items. Each item was either a container, a window, or glass from an automobile. Each of these three classes of use are defined as glass categories. Refractive indexes were measured both before, and after a programme of re-annealing. Because the refractive index of each fragment could not in itself be observed before and after re-annealing, a model based approach was used to estimate the change in refractive index for each glass category. It was found that less complex estimation methods would be equivalent to the full model, and were subsequently used. The change in refractive index was then used to calculate a measure of the evidential value for each item belonging to each glass category. The distributions of refractive index change were considered for each glass category, and it was found that, possibly due to small samples, members of the normal family would not adequately model the refractive index changes within two of the use types considered here. Two alternative approaches to modelling the change in refractive index were used, one employed more established kernel density estimates, the other a newer approach called log-concave estimation. Either method when applied to the change in refractive index was found to give good estimates of glass category, however, on all performance metrics kernel density estimates were found to be slightly better than log-concave estimates, although the estimates from log-concave estimation prossessed properties which had some qualitative appeal not encapsulated in the selected measures of performance. These results and implications of these two methods of estimating probability densities for glass refractive indexes are discussed.

A two-level model for evidence evaluation in the presence of zeros

Likelihood ratios (LRs) provide a natural way of computing the value of evidence under competing propositions. We propose LR models for classification and comparison that extend the ideas of Aitken, Zadora, and Lucy and Aitken and Lucy to include consideration of zeros. Instead of substituting zeros by a small value, we view the presence of zeros as informative and model it using Bernoulli distributions. The proposed models are used for evaluation of forensic glass (comparison and classification problem) and paint data (comparison problem). Two hundred and sixty-four glass samples were analyzed by scanning electron microscopy, coupled with an energy dispersive X-ray spectrometer method and 36 acrylic topcoat paint samples by pyrolysis gas chromatography hyphened with mass spectrometer method. The proposed LR model gave very satisfactory results for the glass comparison problem and for most of the classification tasks for glass. Results of comparison of paints were also highly satisfactory, with only 3.0% false positive answers and 2.8% false negative answers.

Compositional data analysis for elemental data in forensic science

Discrimination of material based on elemental composition was achieved within a compositional data (CoDa) analysis framework in a form appropriate for use in forensic science. The methods were carried out on example data from New Zealand nephrite. We have achieved good separation of the in situ outcrops of nephrite from within a well-defined area. The most significant achievement of working within the CoDa analysis framework is that the implications of the constraints on the data are acknowledged and dealt with, not ignored. The full composition was reduced based on collinearity of elements, principal components analysis (PCA) and scalings from a backwards linear discriminant analysis (LDA). Thus, a descriptive subcomposition was used for the final discrimination, using LDA, and proved to be more successful than using the full composition. The classification based on the LDA model showed a mean error rate of 2.9% when validated using a 10 repeat, three-fold cross-validation. The methods presented lend objectivity to the process of interpretation, rather than relying on subjective pattern matching type approaches.

Differentiation and evaluation of evidence value of styrene acrylic urethane topcoat car paints analysed by pyrolysis-gas chromatography

Pyrolysis (Py)-GC/MS was applied to differentiate between automobile paint samples. The method was used for analysis of 36 samples of styrene acrylic urethane clearcoats that were indistinguishable on the basis of their infrared spectra and elemental composition. Differences observed in the obtained pyrograms of the compared paint samples were relatively small. Therefore, statistical analysis of the obtained results was performed. The likelihood ratio test suitable for multivariate data analysis supported by analysis of data structure by graphical model was used. This approach allowed not only distinguishing the samples compared, but also allowed the evaluation of the evidential value of such observations, which is very important from a forensic point of view.

A two-level model for evidence evaluation

A random effects model using two levels of hierarchical nesting has been applied to the calculation of a likelihood ratio as a solution to the problem of comparison between two sets of replicated multivariate continuous observations where it is unknown whether the sets of measurements shared a common origin. Replicate measurements from a population of such measurements allow the calculation of both within-group and between-group variances/covariances. The within-group distribution has been modelled assuming a Normal distribution, and the between-group distribution has been modelled using a kernel density estimation procedure. A graphical method of estimating the dependency structure among the variables has been used to reduce this highly multivariate problem to several problems of lower dimension. The approach was tested using a database comprising measurements of eight major elements from each of four fragments from each of 200 glass objects and found to perform well compared with previous approaches, achieving a 15.2% false-positive rate, and a 5.5% false-negative rate. The modelling was then applied to two examples of casework in which glass found at the scene of the criminal activity has been compared with that found in association with a suspect.