How can courts take into account the uncertainty in a likelihood ratio?

Extending the discussion on inconsistency in forensic decisions and results

The subject of inter- and intra-laboratory inconsistency was recently raised in a commentary by Itiel Dror. We re-visit an inter-laboratory trial, with which some of the authors of this current discussion were associated, to diagnose the causes of any differences in the likelihood ratios (LRs) assigned using probabilistic genotyping software. Some of the variation was due to different decisions that would be made on a case-by-case basis, some due to laboratory policy and would hence differ between laboratories, and the final and smallest part was the run-to-run difference caused by the Monte Carlo aspect of the software used. However, the net variation in LRs was considerable. We believe that most laboratories will self-diagnose the cause of their difference from the majority answer and in some, but not all instances will take corrective action. An inter-laboratory exercise consisting of raw data files for relatively straightforward mixtures, such as two mixtures of three or four persons, would allow laboratories to calibrate their procedures and findings.

Partial validation of multiplexed real-time quantitative PCR assays for forensic body fluid identification

Confirmatory body fluid identification using messenger RNA (mRNA) is a well-established technique to address issues encountered with conventional testing - such as poor sensitivity, specificity, and a lack of available tests for all body fluids of interest. For over a decade, endpoint reverse-transcription polymerase chain reaction (RT-PCR) assays have been used in forensic casework for such purposes. However, in comparison with real-time quantitative RT-PCR (RT-qPCR), endpoint RT-PCR has lower sensitivity, precision, and linear dynamic range. This research details the multiplexing and partial validation of confirmatory RT-qPCR assays. We have previously described novel assays for a range of body fluid targets and identified an optimal commercial kit for their amplification. Here, multiplexing was undertaken to form three assays: circulatory blood (SLC4A1) and menstrual fluid (STC1), saliva (HTN3) and vaginal material (CYP2B7P), and spermatozoa (PRM1) and seminal fluid (KLK2), all including a synthetic internal control RNA. Partial validation of the multiplexed assays incorporated the MIQE guidelines, ISO requirements, and SWGDAM guidelines. Using receiver operating characteristic (ROC) curves, each marker was significantly different from an uninformative assay and optimal cut-offs were all above 35 cycles. All assays showed a wide LDR (ranging from 3 to 5 logs with most R2 > 0.99), and high precision (most mean CV < 1 %). STC1 showed some instances of sporadic expression in blood, semen, and vaginal material at high CT values. CYP2B7P showed off-target expression in semen and blood. The sensitivities were approximated as; saliva: 1 in 1,000 dilution of a whole buccal swab, circulatory blood: 0.01-0.1 µL blood, menstrual fluid: 1 in 10,000 dilution of a whole menstrual swab, spermatozoa: 0.001 µL semen, seminal fluid: 0.01 µL semen, and vaginal material: 1 in 1,000 dilution of a whole vaginal swab. A total of 16 mock body fluid extract mixtures and 18 swab mixtures were tested and had 100% and 99% detection of target markers below each specific cut-off, respectively. Some mixtures containing high volumes of blood and semen showed off-target CYP2B7P expression. The successful application of a probabilistic model to the RT-qPCR data was also demonstrated. Further work will involve full developmental validation.

Uncertainty in probabilistic genotyping of low template DNA: A case study comparing STRMix™ and TrueAllele™

Two probabilistic genotyping (PG) programs, STRMix™ and TrueAllele™, were used to assess the strength of the same item of DNA evidence in a federal criminal case, with strikingly different results. For STRMix, the reported likelihood ratio in favor of the non-contributor hypothesis was 24; for TrueAllele it ranged from 1.2 million to 16.7 million, depending on the reference population. This case report seeks to explain why the two programs produced different results and to consider what the difference tells us about the reliability and trustworthiness of these programs. It uses a locus-by-locus breakdown to trace the differing results to subtle differences in modeling parameters and methods, analytic thresholds, and mixture ratios, as well as TrueAllele's use of an ad hoc procedure for assigning LRs at some loci. These findings illustrate the extent to which PG analysis rests on a lattice of contestable assumptions, highlighting the importance of rigorous validation of PG programs using known-source test samples that closely replicate the characteristics of evidentiary samples. The article also points out misleading aspects of the way STRMix and TrueAllele results are routinely presented in reports and testimony and calls for clarification of forensic reporting standards to address those problems.

Examining performance and likelihood ratios for two likelihood ratio systems using the PROVEDIt dataset

A likelihood ratio (LR) system is defined as the entire pipeline of the measurement and interpretation processes where probabilistic genotyping software (PGS) is a piece of the whole LR system. To gain understanding on how two LR systems perform, a total of 154 two-person, 147 three-person, and 127 four-person mixture profiles of varying DNA quality, DNA quantity, and mixture ratios were obtained from the filtered (.CSV) files of the GlobalFiler 29 cycles 15s PROVEDIt dataset and deconvolved in two independently developed fully continuous programs, STRmix v2.6 and EuroForMix v2.1.0. Various parameters were set in each software and LR computations obtained from the two software were based on same/fixed EPG features, same pair of propositions, number of contributors, theta, and population allele frequencies. The ability of each LR system to discriminate between contributor (H1-true) and non-contributor (H2-true) scenarios was evaluated qualitatively and quantitatively. Differences in the numeric LR values and their corresponding verbal classifications between the two LR systems were compared. The magnitude of the differences in the assigned LRs and the potential explanations for the observed differences greater than or equal to 3 on the log10 scale were described. Cases of LR < 1 for H1-true tests and LR > 1 for H2-true tests were also discussed. Our intent is to demonstrate the value of using a publicly available ground truth known mixture dataset to assess discrimination performance of any LR system and show the steps used to understand similarities and differences between different LR systems. We share our observations with the forensic community and describe how examining more than one PGS with similar discrimination power can be beneficial, help analysts compare interpretation especially with low-template profiles or minor contributor cases, and be a potential additional diagnostic check even if software in use does contain certain diagnostic statistics as part of the output.