Probative value of absolute and relative judgments in eyewitness identification

Absolute-judgment models better predict eyewitness decision-making than do relative-judgment models

When presented with a lineup, the witness is tasked with identifying the culprit or indicating that the culprit is not present. The witness then qualifies the decision with a confidence judgment. But how do witnesses go about making these decisions and judgments? According to absolute-judgment models, witnesses determine which lineup member provides the strongest match to memory and base their identification decision and confidence judgment on the absolute strength of this MAX lineup member. Conversely, relative-judgment models propose that witnesses determine which lineup member provides the strongest match to memory and then base their identification decision and confidence judgment on the relative strength of the MAX lineup member compared to the remaining lineup members. We took a critical test approach to test the predictions of both models. As predicted by the absolute-judgment model, but contrary to the predictions of the relative-judgment model, witnesses were more likely to correctly reject low-similarity lineups than high-similarity lineups (Experiment 1), and more likely to reject biased lineups than fair lineups (Experiment 2). Likewise, witnesses rejected low-similarity lineups with greater confidence than high-similarity lineups (Experiment 1) and rejected biased lineups with greater confidence than fair lineups (Experiment 2). Only a single pattern was consistent with the relative model and inconsistent with the absolute model: suspect identifications from biased lineups were made with greater confidence than suspect identifications from fair lineups (Experiment 2). The results suggest that absolute-judgment models better predict witness decision-making than do relative-judgment models and that pure relative-judgment models are unviable.

pyWitness 1.0: A python eyewitness identification analysis toolkit

pyWitness is a python toolkit for recognition memory experiments, with a focus on eyewitness identification (ID) data analysis and model fitting. The current practice is for researchers to use different statistical packages to analyze a single dataset. pyWitness streamlines the process. In addition to conducting key data analyses (e.g., receiver operating characteristic analysis, confidence accuracy characteristic analysis), statistical comparisons, signal-detection-based model fits, simulated data generation, and power analyses are also possible. We describe the package implementation and provide detailed instructions and tutorials with datasets so that users can follow. There is also an online manual that is regularly updated. We developed pyWitness to be user-friendly, reduce human interaction with pre-processing and processing of data and model fits, and produce publication-ready plots. All pyWitness features align with open science practices, such that the algorithms, fits, and methods are reproducible and documented. While pyWitness is a python toolkit, it can also be used from R for users more accustomed to this environment.

Models of lineup memory

Face recognition memory is often tested by the police using a photo lineup, which consists of one suspect, who is either innocent or guilty, and five or more physically similar fillers, all of whom are known to be innocent. For many years, lineups were investigated in lab studies without guidance from standard models of recognition memory. More recently, signal detection theory has been used to conceptualize lineup memory and to motivate receiver operating characteristic (ROC) analysis of lineup performance. Here, we describe three competing signal-detection models of lineup memory, derive their likelihood functions, and fit them to empirical ROC data. We also introduce the notion that memory signals generated by the faces in a lineup are likely to be correlated because, by design, those faces share features. The models we investigate differ in their predictions about the effect that correlated memory signals should have on the ability to discriminate innocent from guilty suspects. A popular compound signal detection model known as the Integration model predicts that correlated memory signals should impair discriminability. Empirically, this model performed so poorly that, going forward, it should probably be abandoned. The best-fitting model incorporates a principle known as "ensemble coding," which predicts that correlated memory signals should enhance discriminability. The ensemble model aligns with a previously proposed theory of eyewitness identification according to which the simultaneous presentation of faces in a lineup enhances discriminability compared to when faces are presented in isolation because it permits eyewitnesses to detect and discount non-diagnostic facial features.

Theoretical vs. empirical discriminability: the application of ROC methods to eyewitness identification

Receiver operating characteristic (ROC) analysis was introduced to the field of eyewitness identification 5 years ago. Since that time, it has been both influential and controversial, and the debate has raised an issue about measuring discriminability that is rarely considered. The issue concerns the distinction between empirical discriminability (measured by area under the ROC curve) vs. underlying/theoretical discriminability (measured by d’ or variants of it). Under most circumstances, the two measures will agree about a difference between two conditions in terms of discriminability. However, it is possible for them to disagree, and that fact can lead to confusion about which condition actually yields higher discriminability. For example, if the two conditions have implications for real-world practice (e.g., a comparison of competing lineup formats), should a policymaker rely on the area-under-the-curve measure or the theory-based measure? Here, we illustrate the fact that a given empirical ROC yields as many underlying discriminability measures as there are theories that one is willing to take seriously. No matter which theory is correct, for practical purposes, the singular area-under-the-curve measure best identifies the diagnostically superior procedure. For that reason, area under the ROC curve informs policy in a way that underlying theoretical discriminability never can. At the same time, theoretical measures of discriminability are equally important, but for a different reason. Without an adequate theoretical understanding of the relevant task, the field will be in no position to enhance empirical discriminability.

The impact of fillers on lineup performance

Filler siphoning theory posits that the presence of fillers (known innocents) in a lineup protects an innocent suspect from being chosen by siphoning choices away from that innocent suspect. This mechanism has been proposed as an explanation for why simultaneous lineups (viewing all lineup members at once) induces better performance than showups (one-person identification procedures). We implemented filler siphoning in a computational model (WITNESS, Clark, Applied Cognitive Psychology 17:629-654, 2003), and explored the impact of the number of fillers (lineup size) and filler quality on simultaneous and sequential lineups (viewing lineups members in sequence), and compared both to showups. In limited situations, we found that filler siphoning can produce a simultaneous lineup performance advantage, but one that is insufficient in magnitude to explain empirical data. However, the magnitude of the empirical simultaneous lineup advantage can be approximated once criterial variability is added to the model. But this modification works by negatively impacting showups rather than promoting more filler siphoning. In sequential lineups, fillers were found to harm performance. Filler siphoning fails to clarify the relationship between simultaneous lineups and sequential lineups or showups. By incorporating constructs like filler siphoning and criterial variability into a computational model, and trying to approximate empirical data, we can sort through explanations of eyewitness decision-making, a prerequisite for policy recommendations.

[The effect of suggestibility on eyewitness identifications: A comparison between showups and lineups]

There are two types of eyewitness-identification procedures: showups and lineups. Although the false-identification rate of showups was considered to be higher than that of lineups, experimental research has not always supported the superiority of lineups. Further, suggestibility of showups is believed to produce higher false-identification rates, but no experimental study has manipulated suggestibility. In this study, we manipulated suggestibility; 258 participants performed photo identification in a showup or lineup. The results revealed that the correct-identification rate was higher in the showups than the lineups, and the rate of dangerous false identification for the innocent suspect did not differ between showups and lineups. In lineups alone, the false-identification rate of the high-suggestibility.condition was marginally higher than that of the low-suggestibility condition. The results indicate that suggestibility, which results from the preconception that the perpetrator must exist in the photos, increases false identifications in relative judgments, such as in lineups.

Eyewitness Identification Reform: Data, Theory, and Due Process

Some commentators view my analyses (Clark, 2012, this issue) as an important step forward in assessing the costs and benefits of eyewitness identification reform. Others suggest that the trade-off between correct identifications lost and false identifications avoided is well-known; that the expected utility model is misspecified; and that the loss of correct identifications due to the use of reformed eyewitness identification procedures is irrelevant to policy decisions, as those correct identifications are the illegitimate product of suggestion and lucky guesses. Contrary to these criticisms, the loss of correct identifications has not been adequately considered in theoretical or policy matters, criticisms regarding the various utilities do not substantively change the nature of the trade-off, and the dismissal of lost correct identifications is based not on data but on an outdated theory of recognition memory.

Costs and Benefits of Eyewitness Identification Reform: Psychological Science and Public Policy

Psychological science has come to play an increasingly important role in the legal system by informing the court through expert testimony and by shaping public policy. In recent years, psychological research has driven a movement to reform the procedures that police use to obtain eyewitness identification evidence. This reform movement has been based in part on an argument suggesting that recommended procedures reduce the risk of false identifications with little or no reduction in the rate of correct identifications. A review of the empirical literature, however, challenges this no-cost view. With only one exception, changes in eyewitness identification procedures that reduce the risk of false identification of the innocent also reduce the likelihood of correct identification of the guilty. The implication that criminals may escape prosecution as a result of procedures implemented to protect the innocent makes policy decisions far more complicated than they would otherwise be under the no-cost view. These costs (correct identifications lost) and benefits (false identifications avoided) are discussed in terms of probative value and expected utility.

Revisiting absolute and relative judgments in the WITNESS model

The WITNESS model (Clark in Applied Cognitive Psychology 17:629-654, 2003) provides a theoretical framework with which to investigate the factors that contribute to eyewitness identification decisions. One key factor involves the contributions of absolute versus relative judgments. An absolute contribution is determined by the degree of match between an individual lineup member and memory for the perpetrator; a relative contribution involves the degree to which the best-matching lineup member is a better match to memory than the remaining lineup members. In WITNESS, the proportional contributions of relative versus absolute judgments are governed by the values of the decision weight parameters. We conducted an exploration of the WITNESS model's parameter space to determine the identifiability of these relative/absolute decision weight parameters, and compared the results to a restricted version of the model that does not vary the decision weight parameters. This exploration revealed that the decision weights in WITNESS are difficult to identify: Data often can be fit equally well by setting the decision weights to nearly any value and compensating with a criterion adjustment. Clark, Erickson, and Breneman (Law and Human Behavior 35:364-380, 2011) claimed to demonstrate a theoretical basis for the superiority of lineup decisions that are based on absolute contributions, but the relationship between the decision weights and the criterion weakens this claim. These findings necessitate reconsidering the role of the relative/absolute judgment distinction in eyewitness decision making.