Comparing the performance of a large language model and naive human interviewers in interviewing children about a witnessed mock-event

PURPOSE

The present study compared the performance of a Large Language Model (LLM; ChatGPT) and human interviewers in interviewing children about a mock-event they witnessed.

METHODS

Children aged 6-8 (N =  78) were randomly assigned to the LLM (n =  40) or the human interviewer condition (n =  38). In the experiment, the children were asked to watch a video filmed by the researchers that depicted behavior including elements that could be misinterpreted as abusive in other contexts, and then answer questions posed by either an LLM (presented by a human researcher) or a human interviewer.

RESULTS

Irrespective of condition, recommended (vs. not recommended) questions elicited more correct information. The LLM posed fewer questions overall, but no difference in the proportion of the questions recommended by the literature. There were no differences between the LLM and human interviewers in unique correct information elicited but questions posed by LLM (vs. humans) elicited more unique correct information per question. LLM (vs. humans) also elicited less false information overall, but there was no difference in false information elicited per question.

CONCLUSIONS

The findings show that the LLM was competent in formulating questions that adhere to best practice guidelines while human interviewers asked more questions following up on the child responses in trying to find out what the children had witnessed. The results indicate LLMs could possibly be used to support child investigative interviewers. However, substantial further investigation is warranted to ascertain the utility of LLMs in more realistic investigative interview settings.

A field assessment of child abuse investigators' engagement with a child-avatar to develop interviewing skills

BACKGROUND

Child investigative interviewing is a complex skill requiring specialised training. A critical training element is practice. Simulations with digital avatars are cost-effective options for delivering training. This study of real-world data provides novel insights evaluating a large number of trainees' engagement with LiveSimulation (LiveSim), an online child-avatar that involves a trainee selecting a question (i.e., an option-tree) and the avatar responding with the level of detail appropriate for the question type. While LiveSim has been shown to facilitate learning of open-ended questions, its utility (from a user engagement perspective) remains to be examined.

OBJECTIVE

We evaluated trainees' engagement with LiveSim, focusing on patterns of interaction (e.g., amount), appropriateness of the prompt structure, and the programme's technical compatibility.

PARTICIPANTS AND SETTING

Professionals (N = 606, mainly child protection workers and police) being offered the avatar as part of an intensive course on how to interview a child conducted between 2009 and 2018.

METHODS

For descriptive analysis, Visual Basic for Applications coding in Excel was applied to evaluate engagement and internal attributes of LiveSim. A compatibility study of the programme was run testing different hardware focusing on access and function.

RESULTS

The trainees demonstrated good engagement with the programme across a variety of measures, including number and timing of activity completions. Overall, knowing the utility of avatars, our results provide strong support for the notion that a technically simple avatar like LiveSim awake user engagement. This is important knowledge in further development of learning simulations using next-generation technology.

Enhancing questioning skills through child avatar chatbot training with feedback

Training child investigative interviewing skills is a specialized task. Those being trained need opportunities to practice their skills in realistic settings and receive immediate feedback. A key step in ensuring the availability of such opportunities is to develop a dynamic, conversational avatar, using artificial intelligence (AI) technology that can provide implicit and explicit feedback to trainees. In the iterative process, use of a chatbot avatar to test the language and conversation model is crucial. The model is fine-tuned with interview data and realistic scenarios. This study used a pre-post training design to assess the learning effects on questioning skills across four child interview sessions that involved training with a child avatar chatbot fine-tuned with interview data and realistic scenarios. Thirty university students from the areas of child welfare, social work, and psychology were divided into two groups; one group received direct feedback (n = 12), whereas the other received no feedback (n = 18). An automatic coding function in the language model identified the question types. Information on question types was provided as feedback in the direct feedback group only. The scenario included a 6-year-old girl being interviewed about alleged physical abuse. After the first interview session (baseline), all participants watched a video lecture on memory, witness psychology, and questioning before they conducted two additional interview sessions and completed a post-experience survey. One week later, they conducted a fourth interview and completed another post-experience survey. All chatbot transcripts were coded for interview quality. The language model's automatic feedback function was found to be highly reliable in classifying question types, reflecting the substantial agreement among the raters [Cohen's kappa (κ) = 0.80] in coding open-ended, cued recall, and closed questions. Participants who received direct feedback showed a significantly higher improvement in open-ended questioning than those in the non-feedback group, with a significant increase in the number of open-ended questions used between the baseline and each of the other three chat sessions. This study demonstrates that child avatar chatbot training improves interview quality with regard to recommended questioning, especially when combined with direct feedback on questioning.

An overview of mock interviews as a training tool for interviewers of children

Mock (simulated) interviews can be used as a safe context for trainee interviewers to learn and practice questioning skills. When mock interviews are designed to reflect the body of scientific evidence on how questioning skills are best learned, research has demonstrated that interviewers acquire relevant and enduring skills. Despite the importance of this exercise in learning interview skill and its prevalence as a learning tool in other fields such as medicine and allied health, there has been relatively little discussion about mock interviews from an educational perspective in investigative interview training. This paper addresses that gap by providing the first comprehensive overview of the way mock interviews have been used in training interviewers of children. We describe the research that supports their utility, and the various ways they can be implemented in training: providing insight to learners; allowing opportunities for practice, feedback, and discussion; and as a standardized way to assess skill change over time. The paper also includes an overview of the cutting-edge use of avatars in mock interviews to enhance efficiency, provide unique learning experiences, and ultimately reduce training costs. We explain why avatars may be particularly useful in basic training, freeing up human trainers to facilitate mock interviews around advanced topics and discussion.

The Kenyan Survivors of Sexual Violence Network: Preserving Memory Evidence with a Bespoke Mobile Application to Increase Access to Vital Services and Justice

Police interviews gather detailed information from witnesses about the perpetrator that is crucial for solving crimes. Research has established that interviewing witnesses immediately after the crime maintains memory accuracy over time. However, in some contexts, such as in conflict settings and low-income countries, witness interviews occur after long delays, which decreases survivors’ access to vital services and justice. We investigated whether an immediate interview via a mobile phone application (SV_CaseStudy Mobile Application, hereafter MobApp) developed by the Kenyan Survivors of Sexual Violence Network preserves people’s memory accuracy over time. Participants (N = 90) viewed a mock burglary and were then interviewed either immediately using MobApp or MobApp+ (which included additional questions about the offender’s behaviour) and again one week later (n = 60), or solely after a one-week delay (n = 30). We found that memory accuracy one week later was higher for participants immediately interviewed with MobApp or MobApp+ compared to those interviewed solely after a one-week delay. Additionally, memory accuracy was maintained for those interviewed with the mobile application across the one-week period. These findings indicate that the mobile phone application is promising for preserving memory accuracy in contexts where crimes are reported to the police after a delay.

Information Gathering in Investigative and Medical Interviewing: Drawing Parallels Across Contexts

Information gleaned from a patients' medical history is a core determinant of a medical diagnosis. Accurate and effective history-taking is, therefore, a foundational skill for medical practitioners and is introduced early in medical training. Recognizing and developing the skills of effective medical interviewing is an ongoing challenge for medical students and experienced clinicians alike. Important parallels exist between the information gathering skills required in medicine and health, and those required in investigative interviewing. Qualitative interviews were conducted with 19 experienced medical professionals from a range of specialty areas. They were asked about the role of the medical interview in their discipline, and about challenges they experience when gathering information from patients. Both theory-driven and grounded-theory approaches were used in combination to identify common themes. The interviews were rich with themes including approaches to introductory phases of the interview, eliciting a narrative account, and several topics that specifically paralleled issues in interviewing of vulnerable witnesses. We explore these themes through a lens of investigative interviewing by applying the knowledge of effective interviewing skills and structures to the data gained from the medical context. In general, themes indicated that there are numerous parallels to information gathering approaches in both contexts. As such, there may be scope for medical education to adopt some of the training techniques employed in the investigative interviewing field. Further, it is hoped that the present findings be used to spark an interdisciplinary conversation about communication from which both sides can learn.

Forensic Interviewers' Difficulty With Invitations: Faux Invitations and Negative Recasting

An ongoing challenge for forensic interviewers is to maximize their use of invitations, such as requests that the child "tell me more about" details mentioned by the child. Examining 434 interviews with 4- to 12-year-old children questioned about abuse, this study analyzed (1) faux invitations, in which interviewers prefaced questions with "tell me" but then asked a noninvitation, (2) negative recasts, in which interviewers started to ask an invitation but then recast the question as a wh- or option-posing question, and (3) other aspects of questions that may relate to productivity independent of their status as invitations. About one fourth of "tell me" questions were faux invitations, and over 80% of recasts were negative. The frequency of both faux invitations and negative recasts increased during the substantive phase of the interviews, and these were related to decreased productivity, increased nonresponsiveness, and increased uncertainty. In contrast, use of exhaustive terms (e.g., "tell me everything") and nonstatic questions (e.g., about actions) was related to increased productivity. The results suggest that training should teach interviewers when and how strategic use of invitations and other question types can elicit specific types of forensically relevant information.

Young Children's Activity Involvement and Responses to Yes/No Questions

In the present study, we investigated younger and older Persian preschoolers' response tendency and accuracy toward yes/no questions about a coloring activity. Overall, 107 three- to four-year-olds and five- to six-year-old children were asked positive and negative yes/no questions about a picture coloring activity. The questions focused on three question contents namely, actions, environment and person. As for children's response tendency, they showed a compliance tendency. That is, they provided yes and no responses to positively and negatively formed questions respectively. Children especially younger ones were more compliant toward positive questions and their tendency decreased by age. In addition, the results revealed children's highest rate of compliance tendency toward environment inquiries. Concerning response accuracy, the effects of age and question content were significant. Specifically, older children provided more accurate responses than their younger counterparts, especially to yes/no questions asked about the actions performed during the activity. The findings suggest that depending on the format and the content of yes/no questions younger and older children's response accuracy and tendency differ.

Police Interviewers' Perceptions of Child Credibility in Forensic Investigations

During investigative interviews, police practice can influence key aspects of child credibility, namely the accuracy, competency, reliability, and truthfulness of their testimony. To date, police interviewers' perceptions of how best to assess child credibility at interview, and how practice impacts upon credibility, have been overlooked. We conducted a qualitative study that examined data from focus groups with 16 English police officers who regularly interview children. The focus group transcripts were analysed using thematic analysis, and four main themes were identified - the 4Es: eliciting information, evaluating credibility, empowering the interviewee, and a high-quality end product. Within these themes, police officers acknowledged some responsibility for the perceived credibility of child victims. Poor interviewing practice could decrease the accuracy of the information elicited and cross-examined in court. Registered intermediaries could empower child interviewees and increase their competency. A lack of reliability contributed to evaluating credibility, but this relationship was not straightforward. Finally, obtaining the most truthful account from child victims was not always possible, because there are many barriers to overcome. Our findings suggest the need for a continued focus on interview protocols that facilitate disclosure from child victims and a review of the professional relationship between those who interview children and prosecutors.

Searching for Truth: The Forensic Interviewer's Use of an Assessment Approach While Conducting Child Sexual Abuse Interviews

This study examined the assessment approach interviewers use while conducting interviews to assess truth as narratives are gathered in children's disclosure statements by examining 100 forensic interviews completed at a Children's Advocacy Center. A descriptive review was used to examine the steps engaged by interviewers as they followed a protocol and content analysis was used to identify interviewers' questioning strategies as they assessed children's disclosure narratives during interviews. Findings indicate that interviewers apply a protocol in order to support advancing to a phase of eliciting details in children's narratives. Questioning strategies included using a variety of question types to progress from general to specific, incorporating interview aids sparingly as necessary, and integrating multidisciplinary team feedback. Findings suggest that an assessment approach is inherent to the process of actively conducting a forensic interview. Rather than assessment beginning strictly upon completion of children's narratives, this paper describes how interviewers incorporate an assessment framework throughout interviewing.

Organisational challenges to delivering child investigative interviewer training via e-learning

This article examined organisational challenges arising from the implementation of a new training course for investigative interviewers of vulnerable witnesses. The course was delivered via e-learning (computer exercises) and also involved mock interviews conducted over the telephone. Thematic analysis was conducted of: (a) trainees’ anonymous written feedback submitted to an online discussion forum on the training programme’s website, (b) trainees’ responses to face-to-face questions during semi-structured qualitative interviews, and (c) correspondence between trainees and trainers regarding the training programme. Despite unanimous support for the new training programme, three challenges were identified: limited allocated work time to complete the training, conflicting work practices arising from staggered course enrolment, and difficulties associated with computer and technical skills. These organisational challenges must be addressed to ensure that any future evaluation of the programme on skill performance provides a true indication of the programme’s impact on skill development. From a managerial perspective, organisational challenges need to be addressed in order to maximise the accessibility, completion and long-term success of an e-learning training model for interviewers.

Evaluation of Operation RESET: An initiative for addressing child sexual abuse in Aboriginal communities

This paper presents a qualitative evaluation of Operation RESET: a multi-agency child protection strategy for remote Western Australian Aboriginal communities. RESET is a proactive engagement strategy involving a mobile multi-disciplinary specialist child abuse team comprising detectives and social workers. It is underpinned by the principles that preventing and responding to child sexual abuse must be a shared responsibility, address the underlying causes and contextual issues and enhance children's safety and well-being by empowering families and communities. The evaluation procedure consisted of in-depth interviews with 64 stakeholders of various backgrounds and affiliations who lived or worked in regions where RESET had been deployed for the past 18 months. Collectively, the interviews revealed strong support for Operation RESET's implementation and outcomes. Stakeholders highlighted four elements as being valuable components of the reform: the programme's proactive outreach approach, dedication to capacity building, holistic focus and the establishment of relationships that facilitated trust. These findings are discussed within the broader literature and suggestions for further research are offered.

An examination of the association between interviewer question type and story-grammar detail in child witness interviews about abuse

OBJECTIVE

This study compared the effects of open-ended versus specific questions, and various types of open-ended questions, in eliciting story-grammar detail in child abuse interviews.

METHODS

The sample included 34 police interviews with child witnesses aged 5-15 years (M age=9 years, 9 months). The interviewers' questions and their relative sub-types were classified according to definitions reported in the child interview training literature. The children's responses were classified according to the proportion of story grammar and the prevalence of individual story grammar elements as defined by Stein and Glenn (1979).

RESULTS

Open-ended questions were more effective at eliciting story grammar than specific questions. This finding was revealed across three age groups, two interview phases and irrespective of how question effectiveness was measured. However, not all types of open-ended questions were equally effective. Open-ended questions that encouraged a broad response, or asked the child to elaborate on a part of their account, elicited more story-grammar detail compared to open-ended questions that requested clarification of concepts or descriptions of the next (or another) activity or detail within a sequence.

CONCLUSIONS

This study demonstrates that children's ability to provide story-grammar detail is maximised when there is minimal prompting from the interviewer.

PRACTICAL IMPLICATIONS

Given the association between story grammar production and victim credibility, greater guidance is warranted in interviewer training programs in relation to the effects and administration of different types of open-ended questions.

Investigative Interviewing of Suspected Sex Offenders: A Review of What Constitutes Best Practice

Research on the topic of investigative interviewing of suspected sex offenders is still in its infancy, with the majority of work to date focusing on developing theories underlying confessions, and reflecting on the value of specific interview techniques that have been observed in the field. This paper provides a synthesis of the literature in order to produce a preliminary guide to best practice for the interviewing of this particular interviewee group. Specifically, this review is structured around five elements that should be considered when planning for and administering the interview. These elements include: (a) establishing rapport, (b) introducing the topic of concern, (c) eliciting narrative detail, (d) clarification/specific questions and (e) closure. The unique contribution of this paper is its practical focus, and its synthesis of findings across a variety of streams, including the general eyewitness memory literature, legislation and case law, therapeutic literature, and research specifically related to the interviewing of offenders (including confessions). At the conclusion of the review, recommendations are offered for further research.

The effect of using trained versus untrained adult respondents in simulated practice interviews about child abuse

OBJECTIVE

A single study tested the hypothesis that simulated practice interviews for investigative interviewers of children are more effective when the role of the child respondent is played by trained actors (i.e., postgraduate psychology students) than untrained fellow participants (i.e., child protection workers).

METHOD

The interviewers included 50 child protection service workers. Each interviewer received instruction in the use of open-ended questions and then engaged in two simulated practice interviews. The role of the child respondent in the practice interviews was played by either a trained psychology student or an untrained fellow participant. The key outcome measure was the proportion of open-ended questions, which was assessed immediately prior to and after the practice sessions, as well as 12 weeks post-training.

RESULTS

Interviewers who had practiced with trained actors had higher post-training performance (M=.83, SD=.12) compared to those who had practiced with untrained fellow participants (M=.73, SD=.13, p<.05), even at the 12-week follow up (M actors=.66, SD=.25; M untrained actors=.49, SD=.23, p<.05).

CONCLUSIONS

Training programs that make better use of practice opportunities (e.g., by using trained respondents) will be more effective in improving the performance of investigative interviewers.

PRACTICE IMPLICATIONS

A single study investigated the relative effectiveness of two simulated practice exercises for professionals who interview children about abuse. This research is relevant to professionals who design investigative interviewer training programs because it indicates that practical exercises, which are currently chosen on an 'ad hoc' or convenience basis, can vary markedly in their effectiveness in encouraging adherence to open questions.

The effect of intellectual disability on the adherence of child witnesses to a "story grammar" framework

BACKGROUND

This study examined the ability of 78 children (aged 9-12 years) with an intellectual disability (ID) to provide a narrative account of a staged event they had participated in four days earlier.

METHOD

The children were interviewed using open-ended questions. The quality of their responses (using a story grammar framework) was compared with that of two control groups: mainstream children matched for mental and chronological age.

RESULTS

While the children with an ID and those matched for mental age provided narratives of similar length and used similar proportions of each story grammar element, the ID group was less likely than both control groups to provide a narrative account at all. Among those children with an ID who did provide a narrative account, their accounts included proportionately fewer story grammar elements than those of both control groups.

CONCLUSION

Children with an ID are disadvantaged as witnesses with respect to their ability to provide a detailed and coherent narrative account of events under optimal investigative interviewing conditions.

The effect of intra- versus post-interview feedback during simulated practice interviews about child abuse

OBJECTIVE

This study compared the effectiveness of two types of instructor feedback (relative to no feedback) on investigative interviewers' ability to adhere to open-ended questions in simulated practice interviews about child abuse.

METHOD

In one condition, feedback was provided at the end of each practice interview. In the other, the instructor stopped the interviewer at various stages during the practice interviews to provide feedback. The relative effect of these conditions was examined by measuring interviewers' performance in a standardized mock interview paradigm immediately prior to, immediately after, and 12 weeks after the practice and feedback sessions.

RESULTS

Prior to and 12 weeks after the practice sessions were administered, there was no significant difference in participants' adherence to open-ended questions irrespective of the nature of the feedback, or whether feedback was received. At the immediate post-practice assessment interval, however, the participants who received feedback during the practice interviews performed better (M proportion of open-ended questions=.85, SD=.13) than the other participants (post-interview feedback M=.67, SD=.18, p<.001; no feedback M=.56, SD=.16, p<.001). This heightened use of open-ended questions was associated with a greater tendency among the interviewees to provide abuse-related details in response to open-ended questions (M=.91, SD=.11) compared to the other participants (post-interview feedback M=.77, SD=.15, p<.05; no feedback M=.69, SD=.16, p=.001).

CONCLUSIONS

Different types of feedback can be differentially effective in training child abuse investigators to adhere to open-ended questions. The benefits of any training program, however, are likely to be short-lived without ongoing practice.

PRACTICE IMPLICATIONS

A single study compared the relative effectiveness of two types of instructor feedback (relative to no feedback) on investigative interviewers' ability to adhere to open-ended questions in simulated practice interviews about child abuse. This research is relevant to trainers of investigative interviewers because there is currently large variability in the type of feedback employed in training programs. This study is one of the first to empirically demonstrate that different types of feedback may be differentially effective in improving the performance of investigative interviewers.