From data to a validated score-based LR system: A practitioner's guide

Likelihood ratios (LRs) are a useful measure of evidential strength. In forensic casework consisting of a flow of cases with essentially the same question and the same analysis method, it is feasible to construct an 'LR system', that is, an automated procedure that has the observations as input and an LR as output. This paper is aimed at practitioners interested in building their own LR systems. It gives an overview of the different steps needed to get to a validated LR system from data. The paper is accompanied by a notebook that illustrates each step with an example using glass data. The notebook introduces open-source software in Python constructed by NFI (Netherlands Forensic Institute) data scientists and statisticians.

Calculating LRs for presence of body fluids from mRNA assay data in mixtures

Messenger RNA (mRNA) profiling can identify body fluids present in a stain, yielding information on what activities could have taken place at a crime scene. To account for uncertainty in such identifications, recent work has focused on devising statistical models to allow for probabilistic statements on the presence of body fluids. A major hurdle for practical adoption is that evidentiary stains are likely to contain more than one body fluid and current models are ill-suited to analyse such mixtures. Here, we construct a likelihood ratio (LR) system that can handle mixtures, considering the hypotheses H₁: the sample contains at least one of the body fluids of interest (and possibly other body fluids); H₂: the sample contains none of the body fluids of interest (but possibly other body fluids). Thus, the LR-system outputs an LR-value for any combination of mRNA profile and set of body fluids of interest that are given as input. The calculation is based on an augmented dataset obtained by in silico mixing of real single body fluid mRNA profiles. These digital mixtures are used to construct a probabilistic classification method (a 'multi-label classifier'). The probabilities produced are subsequently used to calculate an LR, via calibration. We test a range of different classification methods from the field of machine learning, ways to preprocess the data and multi-label strategies for their performance on in silico mixed test data. Furthermore, we study their robustness to different assumptions on background levels of the body fluids. We find logistic regression works as well as more flexible classifiers, but shows higher robustness and better explainability. We test the system's performance on lab-generated mixture samples, and discuss practical usage in case work.